A Self-Refinement Strategy for Noise Reduction in Grammatical Error Correction (2020)

Abstract: Existing approaches for grammatical error correction (GEC) largely rely on supervised learning with manually created GEC datasets. However, there has been little focus on verifying and ensuring the quality of the datasets, and on how lower-quality data might affect GEC performance. We indeed found that there is a non-negligible amount of “noise” where errors were inappropriately edited or left uncorrected. To address this, we designed a self-refinement method where the key idea is to denoise these datasets by leveraging the prediction consistency of existing models, and outperformed strong denoising baseline methods. We further applied task-specific techniques and achieved state-of-the-art performance on the CoNLL-2014, JFLEG, and BEA-2019 benchmarks. We then analyzed the effect of the proposed denoising method, and found that our approach leads to improved coverage of corrections and facilitated fluency edits which are reflected in higher recall and overall performance.

## introduction grammatical error correction (gec) is often considered a variant of machine translation (mt) @xcite due to their structural similarity-"translating" from source ungrammatical text to target grammatical text. at present, several neural encoderdecoder (encdec) approaches have been introduced for this task and have achieved remarkable results @xcite @xcite . encdec models tend to further improve in performance with increasing data size @xcite , however, this is not necessarily true in gec. for example, @xcite reported that an encdec-based gec model trained on ef-1 : errors are inappropriately edited source: i want to discuss about the education. target: i want to discuss of the education. camdat @xcite foot_0 , the largest publicly available learner corpus as of today (two million sentence pairs), was outperformed by a model trained on a smaller dataset (e.g., 720k pairs). they hypothesized that this may be due to the noisiness of efcamdat, i.e., the presence of sentence pairs whose correction still contained grammatical errors due to inappropriate edits or being left uncorrected. for example, in we recognise data noise as a generally overlooked issue in gec, and consider the question of whether a better gec model can be built by reducing noise in gec corpora. to this end, we designed a self-refining approach-an effective denoising method where residual errors left by careless or unskilled annotators are corrected by an existing gec model. this approach relies on the consistence of the gec model's predictions (section 4). we evaluated the effectiveness of our method over several gec datasets, and found that it considerably outperformed baseline methods, includ-ing three strong denoising baselines based on a filtering approach, which is a common approach in mt @xcite @xcite . we further improved the performance by applying task-specific techniques and achieved state-of-the-art performance on the conll-2014, jfleg, and bea-2019 benchmarks. finally, through our analysis, we found unexpected benefits to our approach: (i) the approach benefits from the advantage of self-training in neural sequence generation due to its structural similarity (section 6.3), (ii) resulted in significant increase in recall while maintaining equal precision, indicating improved coverage of correction (section 6.4), and (iii) there seems to be a tendency for more fluent edits, possibly leading to more native-sounding corrections (section 6.5). the last is reflected in performance on the jfleg benchmark, which focuses on fluency edits. in summary, we present a data denoising method which improves gec performance, verify its effectiveness by comparing to both strong baselines and current best-performing models, and analyze how the method affects both gec performance and the data itself. ## related work in gec, previous studies have generally focused on typical errors, such as the use of articles @xcite , and noun numbers @xcite . more recently, many studies have addressed gec as a mt problem where ungrammatical text is expected to be translated into grammatical text. this approach allows the adoption of sophisticated sequenceto-sequence architectures @xcite @xcite that have achieved strong performance but require a large amount of data @xcite @xcite . in gec, the data are usually manually built by experts, which lead to an underlying assumption that the data is noise-free. therefore, to the best of our knowledge, noise in existing common datasets remains largely under-explored and no previous research has investigated the effectiveness of denoising gec datasets. recently, @xcite proposed a method for filtering large and noisy synthetic pretrained data in gec by deriving example-level scores on their pretrained data. however, what they regard as noise consists of instances in source sentences (i.e., not target sentences) of the synthetic data that are outside the genuine learner error distribution, where they perform data selection based on the small and higherquality genuine data (namely, the learner corpora we attempt to denoise in this study). therefore, our methods are not comparable, and it is expected to further improve the performance by combining both methods, which we plan to investigate in our future work. in contrast, data noise is becoming an increasingly important topic in mt, where it is common to use automatically acquired parallel data via web crawling in addition to high-quality curated data. as a result, the mt field faces various data quality issues such as misalignment and incorrect translations, which may significantly impact translation quality @xcite . a straightforward solution is to apply a filtering approach, where noisy data are filtered out and a smaller subset of high-quality sentence pairs is retained @xcite @xcite . nevertheless, it is unclear whether such a filtering approach can be successfully applied to gec, where commonly available datasets tend to be far smaller than those used in recent neural mt research. hence, in this study, we investigate its effectiveness by conducting a comparative experiment using the proposed denoising approach. ## noise in gec datasets in this study, we define noise as two types of residual grammatical errors in target sentences: inappropriate edits and those left uncorrected ( the presence of noise in gec data was uncovered by previous work such as @xcite , but the exact nature of it was unexplored. to confirm this, we manually assessed how much noise was contained in the following three commonly used training datasets: the bea official training dataset (henceforth, bea-train) provided in the (1) bea-train x : i will make a poet to kill this pain. y : i will make a poem to kill this pain. y : i will write a poem to get rid of this pain. y : the restaurant is located opposite the movie theater. bea-2019 workshop @xcite foot_1 , ef-camdat, and the non-public lang-8 corpus (henceforth, lang-8) 4 . for 300 target sentences y from each dataset, one expert reviewed them and we obtained denoised ones y ( here, the wer values are slightly higher than expected, but this is most likely caused by fluency edits by the editor, making the sentence more native-like. thus, we found that (i) there is a nonnegligible amount of "noise" in the most commonly used training data for gec, and (ii) efcamdat is much noisier than the other two training datasets. ## proposed denoising method the supervised learning problem for gec is formally defined as follows. let θ be all trainable parameters of a gec model, and d be training data consisting of pairs of an ungrammatical source sentence x and a grammatical target sentence then, the objective is to find the optimal parameters θ that minimize the following loss function l(d, θ) on training data d: conventionally, training data d is assumed to be "clean" parallel data. however, as argued in section 3, this assumption typically does not hold in gec. here, we assume that training data d is "noisy", and, for clarity, we use the notation d to represent "clean" parallel data, where "clean" means "denoised" in this context. the goal is, first, to obtain a new set d by denoising d, and then, to obtain a gec model θ on the new training data d. to deal with data noise, a straightforward solution is to apply a filtering approach, where noisy data are filtered out and a smaller subset of highquality sentence pairs is retained, as employed in mt. however, applying a filtering approach may not be the best choice in gec for two reasons: (i) gec is a low-resource task compared to mt, thus further reducing data size by filtering may be critically ineffective; (ii) even noisy instances may still be useful for training since they might contain as an alternative to filtering, we propose a selfrefinement (sr) approach for denoising gec training data (algorithm 1). the main idea is to train a gec model (henceforth, base model) on noisy parallel data d and to use it for refining target sentences in d. noisy annotations are potentially caused by carelessness or insufficient skills of annotators. this causes inconsistent corrections in similar context. in contrast, machine learning-based gec models, such as encdec, tend to be reliably consistent given similar contexts. given noisy parallel data @xmath0, y i )} n @xmath1, we generate new target sentences ŷi from the original target sentences y i and pair them with their original source sentences x i (line 4 in algorithm 1). the consistency of the base model predictions ensures that the resulting parallel data @xmath2, ŷi )} n @xmath3. it is worth noting that sr can be regarded as a variant of self-training due to its structural similarity, except that it takes the target sentences rather than the source sentences as input to the model. the algorithm itself is the key difference from existing methods based on selftraining @xcite @xcite . one challenge of this approach is that the base model may consistently make inaccurate corrections. we thus incorporate a fail-safe mechanism as a sub-component to restore the original target sentence if the gec model makes incorrect corrections (lines 5-9). for example, in cases such as in ## experiments we evaluate the proposed method in two ways. first, we exclusively focus on investigating the effectiveness of the proposed denoising method (section 5.3). then, we compare our strongest model trained with denoised data (henceforth, denoised model), with current best-performing ones to investigate whether the proposed method has a complementary effect on existing task-specific techniques (section 5.4). ## analysis 6.1 noise reduction to evaluate the quality of the dataset after denoising, a researcher with a high level of english proficiency (not involved with this work) manually evaluated 500 triples of source sentences x, original target sentences y , and generated target sentences y obtained by applying sr to efcamdat satisfying @xmath4. we can see that 73.6% of the replaced samples were determined to be appropriate corrections, including cases where both were correct. for reference, we provide examples of a confusion set before and after denoising in the appendix d. ## conclusion and future work in this study, we focused on the quality of gec datasets. the motivation behind our study was based on the hypothesis that the carelessness or insufficient skill of the annotators involved in data annotation could often lead to producing noisy datasets. to address this problem, we presented a self-refinement approach as a simple but effective denoising method which improved gec performance, and verified its effectiveness by comparing to both strong baselines based on filtering approach and current best-performing models. furthermore, we analyzed how sr affects both gec performance and the data itself. recently, several methods that incorporate pretrained masked language models such as bert, xlnet @xcite , and roberta @xcite into encdec based gec have been proposed and achieved remarkable results @xcite . these approaches modify the model architecture and do not directly compete with the data-driven approaches discussed in this study. thus, the combination of these methods can be expected to further improve the performance, which we plan to investigate in our future work.