C hinese Grammatical Error Correction via Large Language Model Guided Optimization Training (2024)

Abstract: “Pre-trained language model-based methods for Chinese Grammatical Error Correction (CGEC)are categorized into Seq2Seq and Seq2Edit types. However, both Seq2Seq and Seq2Edit mod-els depend on high-quality training data significantly. Considering the strong generation andinference ability of large language models (LLMs), we propose a large language model-guidedoptimization training method to exploit LLMs to extract error knowledge to optimize the tradi-tional CGEC model training process. On the one hand, we use error types and confusion sets asextra knowledge to guide LLMs to generate diverse pseudo data, thus extending the error distri-bution of our training data. On the other hand, LLMs are utilized to infer the predicted resultsfrom our CGEC models and obtain the re-training data, thus iteratively optimizing our pre-trainedCGEC models. Experiments on two benchmark datasets show that our LLMs-guided optimiza-tion method with small-scale training data can achieve comparable results with baseline modelswith large-scale training data. Detailed comparison experiments demonstrate that both the earlydeviser pseudo data and the later re-training data are extremely useful for traditional CGEC modeloptimization training, and can benefit from each other. We will release our code and prompts athttps://github.com/SakuraAcedia/llm-cgec-got to facilitate future work.”

## introduction grammatical error correction (gec) aims to detect and correct any possible grammatical errors in a given sentence @xcite , i.e., missing, redundant, substitution, and word-order four error types @xcite . as shown in figure 1 , the traditional gec model receives a sentence containing errors and generates its corrected one. the gec results have widely applied to various artificial intelligence tasks such as writing scenarios, speech recognition fields, and information retrieval scenes @xcite @xcite . current typical gec methods can be categorized into two lines, i.e., seq2seq models and seq2edit models. seq2seq models treat gec as a generation task, which takes an erroneous sentence as input and obtains its corrected one autoregressively @xcite . seq2edit models regard gec as a token classification task, which first predicts a sequence of text-editing operations and then generates target tokens based on the sequence operation labels @xcite . with the advance in pre-trained language models, both seq2seq and seq2edit models have achieved obvious improvements in chinese grammatical error correction. however, their performances are highly dependent on the quality and scale of the training corpus @xcite . in the past few years, different data augmentation techniques have been extensively studied @xcite @xcite . on the one hand, researchers attempt to decline the error distributions between synthetic and reality corpus by back-translation or error pattern constraint @xcite . on the other hand, some works exploit noise injection or mixedit approaches to increase the diversity of synthetic corpus @xcite @xcite . more recently, large language models (llms) have demonstrated breakthrough performance in various natural language processing tasks @xcite . the llms benefit from a vast training corpus, providing them with ample grammatical knowledge, exceptional semantic comprehension, diversity generation, and strong representation capabilities @xcite . several studies have investigated the possibility of llms for cgec @xcite . research findings indicate that despite receiving supervised fine-tuning using cgec data, the performance of llms on the cgec task continues to be unsatisfactory @xcite . the result may be due to the serious issue of overcorrection in llms. recent research trends indicate that more comprehensive grammar information can be derived from llms @xcite . this grammar information can guide the gec model to correct grammatical errors, enabling the model to correct complex errors that it was previously unable to correct. due to the serious hallucination problem in llms, the corpus generated by llms may have slight deviations from real corpus. therefore, simulating the distribution of grammatical errors in real-world scenarios and constructing high-quality training data remains a significant challenge. to address this issue, we propose a large language model-guided optimization training method to enhance traditional gec models with knowledge acquired from the llms. first, we utilize error types and the confusion set as extra knowledge to construct prompt instructions based on our given correct sentences, thus guiding llms to generate a diverse original synthetic corpus. second, we directly fuse a few synthetic and reality corpora for the traditional gec model training. third, we utilize llms to analyze predicted sentences from gec models, inferring whether there are any chinese grammatical errors in the sentence that have not been corrected. finally, we exploit llms to generate similar sentences based on the analysis results, which are used for re-training our gec models iteratively, thus further enhancing the performance of gec models. experiments on two benchmark datasets show that our proposed method achieves significant enhancements compared with the strong baselines, demonstrating that llms is extremely useful for optimizing the traditional gec model training. our thorough investigation highlights that error types and confusion sets are important factors in guiding llms to generate a diverse synthetic corpus, thus further expanding the error distribution of our training data. detail comparison experiments show that re-training data generated by llms based on the inference and analysis results are essential for optimizing gec model training since its error distributions are very similar to our testing data. the ablation study indicates c c l 2 0 2 4 that both the early pseudo-corpus generation and the later iterative optimization are equally important in enhancing the performance of traditional gec models, and they can benefit from each other. ## related work grammatical error correction has been a challenging task in both natural language processing and machine learning fields. recently, grammatical error correction has achieved significant improvement in english and chinese. now, we try to give a brief review of some representation approaches from two aspects, i.e., pre-trained language model-based methods and large language model-based methods. pre-trained language model-based methods. motivated by the successful application of pre-trained language models on various artificial intelligence tasks, most previous works focus on enhancing the grammatical error correction performance by fusing the representations from pre-trained language models, which are mainly categorized into seq2seq and seq2edit two paradigms. seq2seq models utilize encoder-decoder architectures to address the gec task, which is drawn inspiration from neural machine translation @xcite @xcite . later, @xcite extend the integration of pre-trained knowledge into the encoder-decoder models. @xcite utilizes syntactic information to enhance the model's performance. @xcite employ an external critic to appraise the suitability of the token to be produced gradually, impacting the selection of the succeeding token dynamically. for seq2edit models, they treat gec as a sequence tagging task and predict token-level edit operations to be implemented on the input sentence @xcite . gector first iteratively annotates and revises the modified sequence, and then proceeds to correct it once more, thereby improving the speed of inference @xcite . @xcite integrate the gector model into the cgec task and explore the impact of different pre-trained language models on this paradigm. compared with the seq2seq model, the seq2edit model has more efficient inference and robust error detection capabilities, but its performance has slightly declined @xcite . in this work, we will give a detailed analysis of the influence of different training corpus on both seq2seq and seq2edit models. large language model-based methods. recent researches on large language models indicate that increasing the size of language models can result in notable enhancements in performance across various downstream tasks @xcite . these llms have shown exceptional skill in adhering to and carrying out a variety of user instructions, leading to extensive acceptance among users who frequently engage with these models and utilize them for professional purposes @xcite @xcite . recently, there has been significant progress in corpus construction focused on reducing the manual annotation workload and automatically enhancing the scale, diversity, and creativity of corpus @xcite @xcite . @xcite provide an almost annotation-free method for aligning pre-trained language models with instructions. @xcite encourage llms to participate in self-reflective retrieval enhances the generation process, leading to improved quality and accuracy of language model output. @xcite design a small dataset containing 1000 samples that is used for training with standard supervised loss and demonstrate the potential of fine-tuning the llms on a high-quality small-scale dataset. recent work has explored the performance of various llms on the cgec task @xcite @xcite @xcite . @xcite evaluate the performance of chatgpt on the cgec task through in-context learning, highlighting its ability to produce coherent sentences while also exposing its tendency for excessive correction. @xcite employs chatgpt to create ungrammatical sentences by incorporating specific clues. in cases where ungrammatical sentences lack clues, the author manually gathers such sentences from publicly accessible websites and rectifies them. @xcite investigate the potential benefits of llms and enhance a writing-assistant scenario through multi-task instruction tuning. despite supervised fine-tuning, the performance of llms remains unsatisfactory, indicating that they have not yet fully adapted to the gec field. consequently, researchers have started exploring if llms can play different roles in the gec task, apart from merely acting as the corrector. @xcite propose a method to enhance the performance of the gec task by leveraging llms for predicting edit c c l 2 0 2 4 spans. @xcite introduce a novel concept of elucidating grammar mistakes and demonstrate the capability of llms to provide explanations for grammatical errors. in a word, although the notable advancements achieved by typical seq2seq and seq2edit methods in the task of cgec, their effectiveness is limited by the quality and scale of the training corpus. considering the strong capability of generation and inference demonstrated by llms, we will attempt to utilize llms to simulate the distribution of realistic errors and further optimize the typical seq2seq and seq2edit model training. ## our proposed approach large language model has shown its strong capability of generation and inference in various natural language processing tasks, such as question answer, summary generation, and knowledge inference @xcite @xcite . however, since the parameters for llms are huge, it is difficult to fine-tune llms to adapt to each specific natural language processing task. hence, the utilization of the ability of llms to enhance the performance of traditional gec models is still a key challenge for chinese grammatical error correction. to address this issue, we propose a large language model-guided optimization training method. on the one hand, we utilize the llms to generate a smallscale diverse pseudo corpus automatically based on error types and confusion sets, thus enlarging the error distributions of training data. on the other hand, we exploit the inference ability of llms to analyze the predicted results from the traditional gec models, thus obtaining a more high-quality re-training corpus to make up for the limitations in traditional model training. as depicted in figure 2 , our proposed method mainly contains three stages, i.e., pseudo corpus generation, traditional model training, and iterative optimization. first, we combine the error types, correct sentences, and an additional confusion set in one prompt to guide the llms in creating sentences with different grammatical errors. second, the generated pseudo corpus is used to enlarge the original training data for traditional gec model training. third, we iteratively utilize llms to analyze predicted sentences by gec models, inferring whether there are any chinese grammar errors in the sentence that have not been corrected. finally, we use llms to generate similar sentences based on the analysis results to retrain gec models, further enhancing the model performances. traditional grammatical error correction models always rely on a large amount of high-quality parallel corpus to ensure effective performances. therefore, how to automatically generate a high-quality corpus becomes an important factor in determining the model performance. considering the strong generation capability of llms, we incorporate four error types and confusion set information as extra knowledge to constrain llms to generate a diverse pseudo corpus that covers all error types. in the practical experiments, we opt for the top 5 candidate characters that match each position within unlabeled sentences to maximize the utilization of the confusion set information. the process involves iterating through these candidates based on their probability in descending order. this selected candidate is then deemed as the final answer. the input of llms is formulated by converting error type e, confusion set information c, and unlabeled sentence y into an instruction template t gen (e, c, y ): specifically, we first encourage llms to analyze grammatical details such as punctuation, vocabulary, and sentence structure in the correct sentences. then, we utilize llms to accurately analyze the part-ofspeech of each word in a given sentence. next, specific error operations are performed in this sentence based on the analyzed part-of-speech and confusion set information. for instance, when encountering a substitution error, we will substitute partial words in the sentence according to the part-of-speech and confusion set information. finally, llms are employed to identify chinese grammar errors related to punctuation, vocabulary, and sentence structure, thereby verifying the presence of grammatical errors in the generated sentences. through the above process, we restrict the generation of pseudo-corpus by llms using error types and confusion set information. this "constrain-then-generate" approach enables us to generate more diverse, informative, and realistic pseudo instances. ## conclusion in this paper, we propose a large language model-guided optimization training method to exploit the strong generation and inference capability of llms to enhance the performance of traditional gec c c l 2 0 2 4 models. on the one hand, we propose using error types and confusion sets as constraint information to guide llms in generating diverse pseudo corpus, thus enhancing the robustness of traditional gec models. on the other hand, we leverage llms to infer the predictions of our gec models to reduce the gap between the error distributions of re-training and realistic corpora. iteratively optimizing our cgec models. experimental results on two cgec datasets demonstrate the effectiveness of our approach on seq2edit and seq2seq models. further detailed analysis shows the effectiveness of our proposed method in reducing the distribution of error types, as well as the key role of iterative optimization in enhancing performance. in future research, we will further explore how to fully utilize the grammatical information of large models to enhance the performance of seq2seq and seq2edit models.