Improving Grammatical Error Correction via Contextual Data Augmentation (2024)

Abstract: Nowadays, data augmentation through synthetic data has been widely used in the field of Grammatical Error Correction (GEC) to alleviate the problem of data scarcity. However, these synthetic data are mainly used in the pre-training phase rather than the data-limited fine tuning phase due to inconsistent error distribution and noisy labels. In this paper, we propose a synthetic data construction method based on contextual augmentation, which can ensure an efficient augmentation of the original data with a more consistent error distribution. Specifically, we combine rule-based substitution with model-based generation, using the generation model to generate a richer context for the extracted error patterns. Besides, we also propose a relabeling-based data cleaning method to mitigate the effects of noisy labels in synthetic data. Experiments on CoNLL14 and BEA19-Test show that our proposed augmentation method consistently and substantially outperforms strong baselines and achieves the state-of-the-art level with only a few synthetic data.

## introduction grammatical error correction (gec) aims to detect and correct grammatical errors in a text @xcite . it is a challenging task with a wide range of application scenarios, including search engines, writing assistants @xcite , and automatic speech recognition (asr) systems. due to the low frequency of grammatical errors in real corpus, obtaining and annotating a certain number of high-quality gec datasets is usually difficult and costly. therefore, the currently available high-quality annotated gec data is very limited @xcite , making synthetic data an important research direction for the data-starved task. nowadays, using synthetic data or data augmentation to improve the performance of gec models has become a mainstream approach @xcite @xcite . common construction methods can be categorized into rule-based substitution @xcite and model-based generation methods @xcite @xcite . however, the synthetic data constructed by the above methods are mainly used in the pre-training phase to initialize a better gec model. the data augmentation methods used for the data-limited fine-tuning phase are of great research value. there are two main reasons why previous synthetic data cannot apply to joint training in the fine-tuning phase. (1) inconsistent error distribution. the high randomness of synthetic data makes it difficult to perfectly match the distribution of a certain high-quality data, leading joint training 1 10898 wrong sentence public transport enables our body to move one place to another. ## method the main flow of our proposed synthetic data construction method based on context augmentation is illustrated in figure 2 . first, we generate the synthetic data with contextual augmentation according to section 2.1's method. after denoising by relabeling (section 2.2), we use the synthetic data to augment the original data in the joint training (section 2.3). ## conclusion in this paper, we propose a synthetic data construction method based on contextual augmentation. it stably augments the context of the source data and ensures a consistent error distribution. previous methods suffer from noisy labels of synthetic data. we significantly improve the performance of synthetic data in joint training through a re-labelingbased denoising method. we validate the effectiveness of our proposed method on several common datasets.