Alirector: Alignment-Enhanced Chinese Grammatical Error Corrector (2024)

Abstract: Chinese grammatical error correction (CGEC) faces serious overcorrection challenges when employing autoregressive generative models such as sequence-to-sequence (Seq2Seq) models and decoder-only large language models (LLMs). While previous methods aim to address overcorrection in Seq2Seq models, they are difficult to adapt to decoder-only LLMs. In this paper, we propose an alignment-enhanced corrector for the overcorrection problem that applies to both Seq2Seq models and decoderonly LLMs. Our method first trains a correction model to generate an initial correction of the source sentence. Then, we combine the source sentence with the initial correction and feed it through an alignment model for another round of correction, aiming to enforce the alignment model to focus on potential overcorrection. Moreover, to enhance the model’s ability to identify nuances, we further explore the reverse alignment of the source sentence and the initial correction. Finally, we transfer the alignment knowledge from two alignment models to the correction model, instructing it on how to avoid overcorrection. Experimental results on three CGEC datasets demonstrate the effectiveness of our approach in alleviating overcorrection and improving overall performance. Our code has been made publicly available1.

heading: 1 Introduction
text: Chinese grammatical error correction (CGEC) (Zhao et al., 2018), which aims to identify and correct potential grammatical errors in given Chinese sentences while adhering to the principle of minimal editing, has broad applications in scenarios such as writing assistant and search engine (Wang et al., 2021). Chinese grammatical errors can be basically categorized into component missing, component redundancy, improper collocation, and improper word order (Ma et al., 2022), which are
* Corresponding authors 1https://github.com/yanghh2000/Alirector
similar to those in English but tend to be more intricate due to the complexities of Chinese grammar.
Existing CGEC methods can be mainly divided into three categories: sequence-to-edit (Seq2Edit), sequence-to-sequence (Seq2Seq), and decoderonly large language models (LLMs). Seq2Edit methods treat CGEC as a sequence tagging task by predicting token-level edit operations (Liang et al., 2020; Zhang et al., 2022a). While offering fast inference and robust error detection, these methods may compromise text fluency and exhibit weak migration ability due to the reliance on languagespecific vocabulary (Li et al., 2022). Seq2Seq methods tackle CGEC using neural machine translation techniques (Fu et al., 2018; Zhao and Wang, 2020) and excel in generating fluent sentences but often lack controllability. More recently, decoderonly LLMs have demonstrated breakthrough performance in various NLP tasks, showing significant potential in CGEC (Fang et al., 2023; Qu and Wu, 2023). However, research suggests that decoderonly LLMs still fall short of surpassing lightweight
2531
state-of-the-art models (Zhang et al., 2023). Besides, Seq2Seq models and decoder-only LLMs may suffer from severe overcorrection issues, resulting in the modification of error-free characters of the source sentence (Park et al., 2020), as illustrated in Figure 1. This can be attributed to the tendency of these generative models to generate target sequences with higher probabilities and replace low-frequency words with more frequent ones (Li et al., 2022). While increasing the number of training examples empirically alleviates this problem, obtaining high-quality annotated examples remains a challenge. Previous studies have explored mitigating overcorrection in Seq2Seq models. Among them, a copy module can be incorporated to enable the direct copying of correct tokens from source sentences to output sentences (Zhao et al., 2019). Another approach involves integrating error detection results from a Seq2Edit model into a Seq2Seq correction model (Li et al., 2023a). However, these methods prove challenging to migrate to decoderonly LLMs due to differences in their architectures. Given the emerging breakthroughs of LLMs in various NLP tasks, there is an urgent need to explore their potential in CGEC, where the overcorrection problem presents a significant obstacle.
To fill this gap, we first explore a two-stage predict-and-align method for mitigating overcorrection caused by Seq2Seq models and decoderonly LLMs. As illustrated in Figure 1, we first train a correction model to generate an initial correction of the source sentence. Then, we combine the source sentence with the initial correction and feed it through an alignment model for another round of correction. The alignment2 model is tasked not only with copying correct edits in the initial correction but also retaining error-free characters in the source sentence, thereby reducing overcorrections.
2For the explanation and discussion about the term "alignment" in this paper, please refer to Appendix G.
Preliminary results in Figure 2 show that the twostage method substantially enhances the overall performance of the original correction model.3
The above predict-and-align method requires deploying two models during inference, which is inefficient in terms of both time and storage. Therefore, we propose to enhance the correction model with knowledge acquired from the alignment model, resulting in an alignment-enhanced corrector (Alirector) better at alleviating the overcorrection problem. Moreover, previous studies (Lu et al., 2022; Qin et al., 2023) have shown that language models are sensitive to the ordering of the input sequence. Hence, we train another alignment model to explore the reverse combination of the source sentence and the initial correction. For knowledge transfer, we apply KL-divergence to constrain the output distributions of the correction model and the two alignment models, guiding the correction model on how to avoid overcorrection. Note that the proposed alignment method applies to both Seq2Seq models and decoder-only LLMs.
Extensive experiments were conducted on three CGEC datasets, and the experimental results demonstrate that our method achieves substantial improvements over baselines and effectively alleviates the overcorrection problem. Among various findings, our in-depth analysis reveals that the alignment information between the source sentence and the initial correction is crucial for mitigating overcorrection and improving the robustness of the correction model. Besides, we confirm that current decoder-only LLMs underperform Seq2Seq models, which warrants further investigation.

heading: 2 Related Work

heading: 2.1 Traditional CGEC Methods
text: Traditional CGEC methods typically follow the approaches used in English GEC, which are broadly categorized into Seq2Edit and Seq2Seq methods.
Seq2Edit methods (Awasthi et al., 2019; Omelianchuk et al., 2020; Liang et al., 2020; Zhang et al., 2022a) treat GEC as a sequence editing task, which predicts token-level edit operations for the input sentence. PIE (Awasthi et al., 2019) utilizes BERT to iteratively predict edit labels. GECToR (Omelianchuk et al., 2020) further extends the tag vocabulary with fine-grained edit tags. Liang et al. (2020) and Zhang et al. (2022a) explore adapting GECToR for CGEC tasks. The strengths
3More preliminary results are provided in Section 4.3.
of Seq2Edit methods lie in its high inference efficiency and strong error detection performance. However, they rely heavily on manually designed vocabularies and language-specific lexical rules, limiting their adaptability (Li et al., 2022).
On the other hand, Seq2Seq methods (Zhao et al., 2019; Zhao and Wang, 2020; Kaneko et al., 2020; Zhang et al., 2022b) employ encoder-decoder models inspired by neural machine translation to model the GEC task, where the encoder encodes the source sentence and the decoder sequentially generates the target tokens. While Kaneko et al. (2020) further adapts pre-trained knowledge into the encoder-decoder model, Zhang et al. (2022b) explore incorporating syntax information. Besides, efforts have been made to combine Seq2Edit and Seq2Seq to enhance the inference efficiency (Chen et al., 2020) or improve the correction results (Yuan et al., 2021; Li et al., 2022, 2023a).

heading: 2.2 LLMs for GEC
text: With the success of LLMs across various NLP tasks, researchers have explored their potential for CGEC. Recent studies (Fang et al., 2023; Li et al., 2023b; Qu and Wu, 2023; Fan et al., 2023) assess the performance of diverse LLMs, including both closed-source and open-source models, on the CGEC task. Fang et al. (2023) evaluate ChatGPT’s performance on CGEC through in-context learning, highlighting its ability to generate fluent sentences as well as susceptibility to overcorrection. Fan et al. (2023) explore open-source LLMs for CGEC via instruction tuning (Ouyang et al., 2022a). Zhang et al. (2023) suggest that fine-tuned LLMs still struggle to match the performance of existing stateof-the-art lightweight GEC models. Besides, some research endeavors (Kaneko and Okazaki, 2023; Song et al., 2023) aim at generating explanations for corrections utilizing LLMs’ powerful capability. While these studies often overlook the overcorrection issue, our work presents a novel approach capable of mitigating overcorrection in LLMs.

heading: 2.3 Overcorrection in GEC
text: Seq2Seq models tend to generate sentences with higher probabilities and replace infrequent words with more frequent ones, leading to overcorrection. Previous works (Zhao et al., 2019; Li et al., 2022, 2023a) expore various approaches to relieve this problem. Zhao et al. (2019) employ a copy module to directly copy the correct tokens from the source sentence to the output sentence. Li et al.
(2022) propose a sequence-to-action module based on the seq2seq model to generate a token-level action sequence. Li et al. (2023a) propose a two-stage approach by integrating detection results from a Seq2Edit model into a Seq2Seq correction model. While these methods are challenging when applied to decoder-only LLMs due to architectural differences, the approach proposed in this work applies to both Seq2Seq models and decoder-only LLMs.

heading: 3 Methodology
text: As depicted in Figure 3, our alignment-enhanced corrector (Alirector) for Chinese grammatical error correction (CGEC) comprises three main steps to build. First, we train a correction model to generate an initial correction of the source sentence. Second, we perform bidirectional alignment by combining the source sentence with the initial correction forward and backward respectively, and passing each combination through an alignment model for another round of correction. Third, we employ knowledge distillation to transfer the knowledge from the two alignment models to the correction model. In the following sections, we first formulate the CGEC task and introduce the correction model in Section 3.1. Then, we delve into the details of the alignment models in Section 3.2 and specify the knowledge distillation in Section 3.3.

heading: 3.1 Correction Model
text: Given a source sentence X = {x1, x2, .., xm} that may contain grammatical errors, the goal of CGEC is to identify and correct the potential grammatical errors within X and output the corresponding gold sentence Y = {y1, y2, .., yn}. The models we investigate to implement the correction model include Transformer-based (Vaswani et al., 2017) Seq2Seq models and decoder-only LLMs.
Seq2Seq The training objective of the Seq2Seq correction model is to minimize the negative loglikelihood (NLL) loss (Williams and Zipser, 1989):
Lgec = n∑
t=1
−logP (yt|y<t, X; θ1), (1)
where y<t represents the tokens preceding time step t, and θ1 denotes the trainable parameters.
Decoder-only LLMs The input to the decoderonly correction model is formulated by converting X and Y into a natural language sequence Z with
an instruction template Tgec(X,Y ):4
Z = Tgec(X,Y )
= { instruction︷ ︸︸ ︷ z1, ..., zi−1, X︷ ︸︸ ︷ zi, ..., zj , Y︷ ︸︸ ︷ zj+1, ..., zj+n}.
(2)
Then, we compute the NLL loss on the target tokens as the training objective:
Lgec = j+n∑
t=j+1
−logP (zt|z<t; θ2). (3)

heading: 3.2 Alignment Model
text: The purpose of our alignment model is to mitigate potential overcorrections in the initial correction generated by the aforementioned correction model. This is achieved by using a separate dataset and training the alignment model to predict the target sentence based on alignment information between the source sentence and the initial correction. Similar to the correction model, both Seq2Seq models and decoder-only LLMs can be employed to build the alignment model. However, to reduce the difficulty of transferring knowledge from the alignment model to the correction model, we require the two
4Instruction templates are provided in Appendix A.
stages to share the same architecture5.
Input Construction We use Ŷ to represent the initial output generated by the correction model for the source sentence X . Then, we construct the input to the alignment model based on X and Ŷ as follows. For Seq2Seq models, we simply concatenate X and Ŷ separated by “[SEP]” as the input, denoted as Xalign = X + [SEP] + Ŷ . As for decoder-only LLMs, we follow Eq. (2) and construct the input by transforming X , Ŷ and Y into a natural language sequence W using another instruction template Talign: W = Talign(X, Ŷ , Y )
= {..., X︷ ︸︸ ︷
wi, ..., Ŷ︷ ︸︸ ︷ wj , ..., wk, Y︷ ︸︸ ︷ wk+1, ..., wk+n}.
(4)
Training Objective The alignment model aims to predict Y based on the alignment of X and Ŷ . For Seq2Seq models, the training objective is:
Lalign = n∑
t=1
−logP (yt|y<t, Xalign; θ3), (5)
5Knowledge distillation between different architectures may require an additional tokenizer and distribution alignment (Wan et al., 2024).
where θ3 denotes the trainable parameters in the alignment model. As for decoder-only LLMs, the NLL loss is computed on the target tokens in W :
Lalign = k+n∑
t=k+1
−logP (wt|w<t; θ4). (6)
Bidirectional Alignment Previous studies (Lu et al., 2022; Qin et al., 2023) have shown that language models are sensitive to input ordering. Motivated by this, we further introduce bidirectional alignment by incorporating a reverse alignment model, which takes the combined source and initial correction in reverse order as input. For example, the input to the Seq2Seq-based reverse alignment model is Ŷ + [SEP] +X . Intuitively, the reverse alignment model may capture different information compared to the forward alignment model between the source sentence and the initial correction. Our empirical analysis in Section 5.2 also demonstrates that combining these two alignment models helps alleviate the impact of overcorrection and improves the overall robustness of the correction model.

heading: 3.3 Bidirectional Alignment Distillation
text: The alignment models described above can be employed alongside the correction model in a twostage predict-and-align paradigm to mitigate overcorrection. However, this approach presents two potential issues. Firstly, deploying both the correction model and the two alignment models during inference increases both time and storage requirements. Secondly, the correction model and the alignment models are trained separately, overlooking the possibility of mutual enhancement. To address these issues, we propose enhancing the correction model with knowledge distilled from the alignment models, guiding the correction model to avoid overcorrection, as well as eliminating the need for the alignment models during inference.
Knowledge Distillation We consider the two alignment models as the teachers and the correction model as the student for knowledge distillation (Hinton et al., 2015). During this process, only the parameters of the correction model are updated, while the parameters of the alignment models remain fixed. For training, we construct inputs for both the correction model and the alignment models following the methods introduced in Section 3.1 and Section 3.2, and obtain the final output logits over the target tokens. Let zc, zf , and zr denote the output logits from the correction model, the
forward alignment model, and the reverse alignment model, respectively. We use KL-divergence as the distillation objective. The forward and reverse alignment distillation losses are defined as:
Lfkd = DKL(σ( zf τ )||σ(z c τ ))
Lrkd = DKL(σ( zr τ )||σ(z c τ )),
(7)
where τ is the temperature, σ is the softmax function, and DKL(·) denotes the KL-divergence.
The overall distillation loss is the weighted sum of these two distillation losses:
Lkd = αLfkd + (1− α)Lrkd, (8) where α ∈ (0, 1) is a hyperparameter.
Overall Objective To train the correction model, we formulate the overall objective by combining the GEC loss with the alignment distillation loss:
L = Lgec + βLkd, (9) where β is a hyperparameter that controls the importance of the distillation loss. More training details are provided in Appendix C.1.

heading: 4 Experiments

heading: 4.1 Datasets and Metrics
text: Based on the data sources, the datasets utilized in our experiments fall into two categories: i) datasets sourced from Chinese-as-a-SecondLanguage (CSL) learner texts, and ii) datasets sourced from Chinese native speaker texts. For CSL learner data, following previous works (Zhang et al., 2022b; Li et al., 2023a), we employ a combination of the Chinese Lang8 dataset (Zhao et al., 2018) and the HSK dataset (Zhang, 2009) as our training set, MuCGEC-Dev (Zhang et al., 2022a) as the development set, and NLPCC18-Test (Zhao et al., 2018) as the test set. For native speaker data, we first randomly partition 1000 samples from the FCGEC (Xu et al., 2022) training set as the development set, with the remainder used for training. For testing, we utilize FCGEC-Dev and NaCGEC-Test (Ma et al., 2022) as our test sets6. Further details regarding the datasets can be found in Appendix B.
For evaluation metrics, we follow previous work and report word-level precision (P)/recall (R)/Fmeasure (F0.5) results on NLPCC18-Test using the official MaxMatch scorer (Dahlmeier and Ng,
6We use FCGEC-Dev here since we can not access the gold labels of FCGEC-Test.
2012) and PKUNLP word segmentation tool provided by Zhao et al. (2018). For FCGEC-Dev and NaCGEC-Test, we report the character-level P/R/F0.5 scores using the ChERRANT scorer7.

heading: 4.2 Base Models and Baselines
text: As previously mentioned, the proposed method applies to both Seq2Seq models and decoder-only LLMs. For Seq2Seq, we choose Transformer-large and Chinese BART-large (Shao et al., 2021) as the base models. For decoder-only LLMs, we choose Baichuan2-7B (Yang et al., 2023), a powerful Chinese LLM, and Chinese-LLaMA2-7B8, which is obtained by incremental training of LLaMA2 (Touvron et al., 2023b) with Chinese corpus.
For comparison, we first employ the following Seq2Seq models as baselines. Vanilla Finetuning (FT) means directly fine-tuning the base models on the entire training set. TemplateGEC (Li et al., 2023a) constructs a detection template to integrate the Seq2Edit and Seq2Seq methods. SynGEC (Zhang et al., 2022b) incorporates syntax information into Seq2Seq models. Copy (Zhao et al., 2019) employs a copy mechanism for Seq2Seq models to directly copy unchanged words from the source sentence to the target sentence. Besides, we also employ decoder-only baselines. Except for vanilla fine-tuning, we implement the copy method (Zhao et al., 2019) in decoder-only LLMs for comparison. The implementation details and hyperparameter settings are presented in Appendix C.2.

heading: 4.3 Preliminary Results
text: As mentioned earlier, we conducted preliminary experiments of the predict-and-align method on NaCGEC and FCGEC datasets. In addition to the results shown in Figure 2 using Baichuan2-7B, we also present the results with two Seq2Seq models,
7https://github.com/HillZhang1999/MuCGEC/tree/ main/scorers/ChERRANT
8https://huggingface.co/Linly-AI/ Chinese-LLaMA-2-7B-hf
namely Transformer-large and BART-large, in Figure 4. From these results, we observe that after alignment, both Baichuan2-7B and Transformer exhibit a substantial performance improvement, especially in precision, revealing the potential of alignment in enhancing overall performance and mitigating overcorrection. While BART’s performance improvement may not be as remarkable as Baichuan2, the alignment approach still demonstrates favorable enhancement. More analysis and discussion regarding the potential of the alignment method are presented in Appendix D.

heading: 4.4 Main Results
text: The main results are shown in Table 1.9 We note that our Alirector consistently outperforms all baselines in F0.5 across all the datasets, demonstrating the effectiveness of this method. In contrast, the Copy method even underperforms vanilla finetuning in some cases. Besides, Alirector achieves considerable improvements in precision across all the datasets, highlighting the efficacy of this approach in mitigating overcorrection. Further analysis regarding the effect of Alirector on reducing overcorrection is presented in Section 5.1.
Moreover, we make several interesting observations. First, despite the notable enhancement achieved by Alirector, the decoder-only LLMs of Baichuan2 and Chinese-LLaMA2 still struggle to outperform BART. This can be attributed to the fact that BART’s pre-training involves a series of denoising tasks utilizing strategies like token masking, token deletion and text infilling, which are naturally suitable for the CGEC/GEC task (Lewis et al., 2020; Wang et al., 2023). Second, different models exhibit varying degrees of improvement by employing Alirector, with decoderonly LLMs generally experiencing more notable improvements than Seq2Seq models. The performance of Chinese-LLaMA2 is much worse than
9More experimental results can be found in Appedix E.
Model Method NLPCC18-Test NaCGEC-Test FCGEC-DevP R F0.5 P R F0.5 P R F0.5
Vanilla FT 42.37 23.49 36.50 59.67 28.69 49.07 47.83 22.99 39.33 TemplateGEC 42.00 22.20 35.60 - - - - - -
Baichuan2, which may be attributed to their different capabilities achieved through pre-training. Third, Alirector yields more pronounced improvements on the FCGEC and NaCGEC datasets than on the NLPCC18 dataset. We attribute this discrepancy to the differing error distributions across datasets. The errors in NLPCC18, derived from Chinese-as-a-Second-Language learners, are less common, while the errors in FCGEC and NaCGEC, stemming from native speakers, exhibit more prevalent patterns that are easier for the model to learn.

heading: 5 Analysis

heading: 5.1 Overcorrection Mitigation
text: To further verify the effectiveness of Alirector in mitigating overcorrection, we use Baichuan2-7B as the backbone and present fine-grained precision results across the four categories of CGEC errors, including missing (M), redundant (R), substitution
(S), and word-order (W), in Figure 5. Moreover, we present in Table 2 the number of overcorrections and undercorrections that Alirector reduces on the four error types compared to Baichuan27B. The results depicted in Figure 5 and summarized in Table 2 demonstrate that Alirector significantly enhances precision for all error types while notably decreasing the number of overcorrections without deteriorating undercorrection, particularly for the redundant and substitution types. These findings support the effectiveness of Alirector in mitigating overcorrection induced by generative language models and in enhancing the robustness of our method across different error types. For a more intuitive illustration of Alirector’s effectiveness, we provide a case study in Appendix F.

heading: 5.2 Ablation Study
text: To investigate the contribution of key components of our approach, we conduct in-depth ablation experiments on NaCGEC-Test and FCGEC-Dev
datasets using BART and Baichuan2-7B.
Distillation from Alignment Models We first ablate different alignment distillation components in turn to analyze their contribution. As shown in Table 3, while removing either forward distillation Lfkd or reverse distillation Lrkd causes noticeable performance degradation, there is a significant performance drop after removing the overall distillation loss Lkd, particularly in recall and F0.5. This indicates that bidirectional alignment contributes more to performance improvement through knowledge distillation compared to unidirectional alignment. Moreover, a notable drop in precision is observed when removing the entire distillation loss (as shown in the line labeled w/o Lkd), suggesting that the alignment distillation is essential for our method to mitigate overcorrection.
Input of Alignment Models To further investigate the effect of the alignment between the source sentence and the initial correction, we conduct additional experiments by ablating the source sentence or initial correction from the input of the alignment models during training Alirector. To keep the format of the input, we ablate the source sentence by replacing it with the initial correction, e.g., Ŷ + [SEP] + Ŷ for Seq2Seq. Similarly, we construct the input as X + [SEP] +X when ablating the initial correction. As shown in Table 3, we observe that ablating the source sentence causes an obvious decline in precision while ablating the initial correction leads to a notable drop in recall. These findings highlight the role of alignment in reducing both overcorrection and undercorrection.

heading: 5.3 Impact of α and β
text: The training objective of Alirector involves α to control the weight of forward and reverse alignment losses, as well as β to balance between the original GEC loss and the distillation loss. To investigate their impact on model performance, we use BART as the backbone and show the results of different values of α and β on the FCGEC development set in Figure 6, where we change one while fixing the other. From the first subfigure, we observe that as α increases, the P/R/F0.5 scores consistently rise and achieve the best results around 0.9. The second subfigure shows that as β increases, the precision rises accordingly while recall gradually falls. This trend indicates that β plays a role in balancing between precision and recall. Similar trends can be observed when other models are employed, and the optimal values of α and β are provided in Table 6.

heading: 6 Conclusion
text: In this paper, we first investigate a predict-and-align method that effectively leverages alignment information between the source sentence and the initial correction to alleviate the overcorrection issue in CGEC. Then, we propose transferring knowledge from the alignment process to enhance the correction model, resulting in an improved model termed Alirector. Experimental results on three CGEC datasets showcase the efficacy of our approach in mitigating overcorrection for both Seq2Seq models and decoder-only LLMs. Detailed analysis further demonstrates the effectiveness of this method across various error types, as well as the pivotal role of alignment in enhancing performance.
Broadly speaking, the overcorrection challenge
falls within the realm of uncontrollability of generative language models. Besides straightforward efforts to acquire more high-quality training data or employ specific pre-training strategies such as BART, this study introduces an alignment-based method that has demonstrated effectiveness in addressing this issue. Despite the improvement of our approach for decoder-only LLMs, their performance in CGEC still lags behind that of the strongest Seq2Seq models, even though they are smaller in size, which contradicts their outstanding performance in other NLP tasks. In future research, we will further exploring enhancing the performance of decoder-only LLMs for CGEC.

heading: Acknowledgements
text: We appreciate the anonymous reviewers for their valuable comments. This work was supported by the National Natural Science Foundation of China (No. 62176270) and the Guangdong Basic and Applied Basic Research Foundation (No. 2023A1515012832).
Limitations
The potential limitations of our work are threefold. First, we have exclusively validated our approach on Chinese GEC datasets. However, our approach is language-independent, and it can be investigated in other languages. Second, our approach incurs additional training costs, as training alignment models and performing knowledge distillation are required. Third, our experiments are confined to 7B-scale LLMs using the QLoRA efficient finetuning technique. Due to computational resource constraints, we have not explored the impact of larger-scale LLMs and full-parameter fine-tuning, which may lead to improved performance.
Ethics Statement
This work aims to propose a technical method to mitigate overcorrection caused by Seq2Seq models and decoder-only LLMs in Chinese grammatical error correction, which will not cause ethical issues. All datasets and models used in this work are publicly available, and we adhere strictly to their usage policies. We are committed to conducting our research in an ethical and responsible manner.

heading: B Datasets
text: The statistics of the datasets used in our experiments are shown in Table 5. For CSL learner data, we adopted the same training set as Zhang et al. (2022a), which involves discarding all samples without grammatical errors in the Lang8 and HSK datasets and replicating the HSK dataset five times and combining with the Lang8 dataset, resulting in a total of 1,568,885 sentence pairs.

heading: C Experimental Details
text: C.1 Training Details
Training on Native Speaker Datasets Since FCGEC contains only 35,341 training samples, which is insufficient for model training, we performed continuous training on the FCGEC training set with the model trained on the CSL learner data.
Training of Alignment Models As described in Section 3.2, before training the alignment models, we need to obtain initial corrections using an initial correction model. For this purpose, we divided the training data into two parts, one for training the initial correction model and the other for training the alignment models. In the case of CSL learner data, 80%10 of the training samples are randomly selected to train the initial correction model. Then, this correction model is used to generate initial corrections for the remaining training samples. These initial corrections along with their corresponding source and target sentences are used to train the alignment models. For the native speaker datasets, we used the correction model trained on the CSL learner data to generate initial corrections on the FCGEC training set.
Training of Alirector As outlined in Section 3.3, we perform knowledge distillation using the correction model as the student and the alignment models as the teachers. For this training, we used the same training set as that used for training the teachers. The student was initialized with the weights of the well-trained initial correction model.
10We experimented with different ratios, including 4:6, 5:5, and 8:2, and found that 8:2 works the best.
C.2 Implementation Details
For Seq2Seq model training, following Zhang et al. (2022b), we utilized the Dropout-Src technique (Junczys-Dowmunt et al., 2018) that applies dropout on input embeddings for alleviating over-fitting. As for LLMs tuning, considering the time and computational resources, we applied QLoRA (Dettmers et al., 2023) for efficient fine-tuning instead of full-parameter fine-tuning. Our code implementation mainly follows the Alpaca LoRA project11, and is based on the Huggingface Transformers (Wolf et al., 2020) and bitsandbytes12 (Dettmers et al., 2022) toolkit in Pytorch. We searched for the optimal value of α in {0.1, 0.3, 0.5, 0.7, 0.9}, β in {0.5, 1.0, 1.5, 2.0} and the temperature τ in {1, 2, 3, 4, 5} on the development set. We used the Adam optimizer (Kingma and Ba, 2014) and polynomial learning rate decay. The hyperparameter settings are presented in Table 6. All experiments are carried out on 8 GeForce RTX 4090 24GB GPUs.
11https://github.com/tloen/alpaca-lora 12https://github.com/TimDettmers/bitsandbytes

heading: D Potential of Alignment
text: The alignment models are introduced to mitigate overcorrection by leveraging the alignment information between the source sentence and the initial correction. To demonstrate the potential of the alignment models, we employed a BART-based alignment model to align the predictions of a Baichuan2-based correction model, and present the comparison results between vanilla fine-tuning (i.e., without alignment) and predict-and-align method in Table 7. From the results, we note that predictand-align improves precision and F0.5 by a large margin compared to vanilla fine-tuning. Notably, predict-and-align even outperforms our Alirector, highlighting the effectiveness and the potential of the two-stage alignment method. Moreover, when
only source information is retained in the input (namely repl. src+src), we observe high precision but low recall, while retaining only prediction information (namely repl. pred+pred) exhibits the opposite trend. This observation once again emphasizes the role of alignment in reducing both overcorrection and undercorrection.

heading: E Additional Experimental Results
text: E.1 Results on FCGEC-Test
We conducted additional experiments on FCGECTest13, the test set of FCGEC (Xu et al., 2022), for more comprehensive evaluation. As shown in Table 8, our Alirector improves the P/F0.5 score by 5.59/1.77 over vanilla fine-tuning on BART, while the improvement is 4.37/2.25 on Baichuan2-7B. In contrast, the Copy method has only a minor improvement over vanilla fine-tuning.
E.2 Results on More LLMs
We also implemented our Alirector method on other Chinese LLMs, namely Yi-6B14, a Chinese LLM that employs the same architecture as LLaMA (Touvron et al., 2023a), and ChatGLM36B15, a chat model based on GLM (Du et al., 2022). The results on NLPCC18-Test are shown in Table 9. We observe that our Alirector method improves the precision/F0.5 score over vanilla fine-tuning by 3.62/1.75 on Yi-6B and 3.14/1.05 on ChatGLM36B, respectively. This highlights the generalizability of Alirector across other LLMs. However, the performance of Yi-6B and ChatGLM3-6B lags significantly behind that of Baichuan2-7B. This discrepancy can be attributed to the different capabilities achieved through pre-training.
13FCGEC-Test provides online evaluation at https:// codalab.lisn.upsaclay.fr/competitions/8020.
14https://huggingface.co/01-ai/Yi-6B 15https://huggingface.co/THUDM/
chatglm3-6b-base

heading: F Case Study
text: We provide four examples in Table 10 to illustrate the effectiveness of our Alirector in mitigating overcorrection. We can note that the vanilla fine-tuned model often tends to overcorrect by modifying the error-free characters. In contrast, Alirector is able to correct all the errors in the sentence while preserving the error-free characters. These cases intuitively show that Alirector learns to identify and correct the potential errors in the sentence while actively avoiding overcorrection as much as possible.

heading: G Explanation of Term "Alignment"
text: We adopt the term "alignment" in this paper as it works in a similar way to the alignment in RLHF (Ouyang et al., 2022b). RLHF aims to align the LLM’s response with human preferences, while our alignment model seeks to refine the GEC model’s over-corrected predictions by aligning them with the source text. From this perspective, our approach ensures that the corrections are more accurate and contextually appropriate, maintaining faithfulness to the original input while avoiding unnecessary changes. Additionally, Ji et al. (2024) use an Aligner to refine the initial output of LLMs, which shares a similar concept with our paper.

heading: Future Work
text: In future research, we will further exploring enhancing the performance of decoder-only LLMs for CGEC