System Report for CCL 23-Eval Task 7: THU KEL ab (sz) - Exploring Data Augmentation and Denoising for C hinese Grammatical Error Correction (2023)

Abstract: “This paper explains our GEC system submitted by THU KELab (sz) in the CCL2023-Eval Task7 CLTC (Chinese Learner Text Correction) Track 1: Multidimensional Chinese Learner TextCorrection. Recent studies have demonstrate GEC performance can be improved by increasingthe amount of training data. However, high-quality public GEC data is much less abundant. To address this issue, we propose two data-driven techniques, data augmentation and data de-noising, to improve the GEC performance. Data augmentation creates pseudo data to enhancegeneralization, while data denoising removes noise from the realistic training data. The resultson the official evaluation dataset YACLC demonstrate the effectiveness of our approach. Finally,our GEC system ranked second in both close and open tasks. All of our datasets and codes areavailabel at https://github.com/THUKElab/CCL2023-CLTC-THU_KELab .”

## introduction the ccl2023-cltc track 1 (multidimensional chinese learner text correction) is a subtask of grammatical error correction (gec) @xcite , aiming to correct sentences written by chinese learners through a two-dimensional annotation scheme, namely grammar and fluency @xcite . adhering to the minimal edits principle, the former ensures that the structure of the original sentence is maintained as much as possible with the smallest number of revisions. conversely, the latter emphasizes fluency-based correction, where annotators strive to make the sentences more fluent and native-sounding. numerous studies @xcite @xcite have shown that the performance of gec can be improved by increasing the volume of training data. however, obtaining publicly-available and high-quality data for gec is a challenge @xcite . training gec models with limited data could lead to the fact that gec models are very likely to overfit and make predictions based on spurious patterns @xcite , owing to the huge gap between the number of model parameters and limited data available for gec. this paper attempts to alleviate the aforementioned problem using two techniques, namely data augmentation and data denoising. thanks for the ease of constructing pseudo grammatical errors, various gec data augmentation methods have been widely explored, including noise injection @xcite @xcite , pattern noise @xcite , backtranslation @xcite @xcite and round-trip translation @xcite . inspired by the success of gec data augmentation, we first generate synthetic parallel data from clean monolingual corpora, which is used for pre-training gec models 0 . then, we introduce cutoff in the fine-tuning stage to encourage gec models to make consistent predictions regardless of random noise applied to the sentences @xcite . furthermore, we observe that the provided official lang8 training set contains a significant amount of noise due to low-quality annotation, which could harm the performance of gec models. to address this issue, we reconstruct the denoised training set using a well-trained gec model or ensemble. specifically, we use a gec model/ensemble to further correct the target sentences from lang8, which effectively removes some of the noise present in the data. we then replace the original noisy target sentences with the new corrected target sentences based on the assumption that the outputs of the well-trained model/ensemble can denoise the original dataset caused by low-quality annotation. we evaluate our data-driven ideas on the official evaluation dataset yaclc using two backbone models: bart @xcite and gector @xcite . in the close task, our best single model achieves 71.88 f 0.5 for minimal correction and 42.02 f 0.5 for fluent correction (with an average of 56.95 f 0.5 ). our best bart + gector ensemble secured the 2nd position in the close task with 74.92 f 0.5 for minimal correction and 43.89 f 0.5 for fluent correction (with an average of 59.41 f 0.5 ), and also secured the 2nd position in the open task with 76.14 f 0.5 for minimal correction and 44.17 f 0.5 for fluent correction (with an average of 60.16 f 0.5 ). in words, the contributions of our paper are three folds: ## background generally, gec models learn the monolingual translation probability p (y | x; θ), where x denotes an ungrammatical source sentence and y represents a grammatically correct target sentence. given a parallel training dataset d, the standard training objective is to minimize the empirical risk: recent studies have attempted to improve the performance of gec models by incorporating various data augmentation techniques. to this end, we examine and compare the effectiveness of two data augmentation methods that aim to improve generalization through increased training data scale. pattern noise (pn). pn introduces in-distribution grammatical errors to sentences @xcite . specifically, it first identifies error patterns in gec datasets using an automated error annotation toolkit (e.g., errant @xcite ). then, it applies a noising function to sentences by randomly replacing text segments with pre-extracted grammatical errors. backtranslation (bt). bt generates more genuine grammatical errors by learning the distribution of human-written grammatical errors using noisy seq2seq models @xcite @xcite . the noisy model is trained with the inverse of the gec parallel dataset, where the ungrammatical sentence is treated as the target and the grammatical sentence as the source. several variants of bt were proposed by @xcite , and their study revealed that the variant bt (noisy) achieved the best performance. consequently, we focus on this variant in our work. during decoding of ungrammatical sentences, bt (noisy) adds rβ random to the score of each hypothesis in the beam for each time step, where r is drawn uniformly from the interval [0, 1], and β random is a hyper-parameter that controls the noise degree. ## conclusion in this ccl2023-cltc track 1 open&close task, we improve gec models by adopting two datadriven techniques, namely data augmentation and data denoising. our experiments on yaclc evaluation c c l 2 0 2 3 datasets annotated with two principles demonstrate the effectiveness of our proposed methods. our best ensemble, which is consisting of seq2seq and seq2edit models, achieves an average f 0.5 of 59.41 in the close task and 60.16 in the open task, ranking second in both tasks. in the future, we will explore the effectiveness of our approach in other languages and datasets.