Improving Grammatical Error Correction Models with Purpose-Built Adversarial Examples (2020)

Abstract: A sequence-to-sequence (seq2seq) learning with neural networks empirically shows to be an effective framework for grammatical error correction (GEC), which takes a sentence with errors as input and outputs the corrected one. However, the performance of GEC models with the seq2seq framework heavily relies on the size and quality of the corpus on hand. We propose a method inspired by adversarial training to generate more meaningful and valuable training examples by continually identifying the weak spots of a model, and to enhance the model by gradually adding the generated adversarial examples to the training set. Extensive experimental results show that such adversarial training can improve both the generalization and robustness of GEC models.

## introduction the goal of grammatical error correction (gec) is to identify and correct different kinds of errors in the text, such as spelling, punctuation, grammatical, and word choice errors, which has been widely used in speech-based dialogue, web information extraction, and text editing software. a popular solution tackles the grammatical error correction as a monolingual machine translation task where ungrammatical sentences are regarded as the source language and corrected sentences as the target language @xcite . therefore, the gec can be modeled using some relatively mature machine translation models, such as the sequence-to-sequence (seq2seq) paradigm @xcite . they are many types of grammatical errors and all of them can occur in a sentence, which makes it impossible to construct a corpus that covers all kinds of errors and their combinations. deep learning so far is data-hungry and it is hard to train a seq2seq model with good performance without suf-clean his reaction should give you an idea as to whether this matter is or is not your business . ## conclusion in this paper, we proposed a data augmentation method for training a gec model by continually adding to the training set the adversarial examples, particularly generated to compensate the weakness of the current model. to generate such adversarial examples, we first determine an important position to change and then modify it by introducing specific grammatical issues that maximize the gec model's prediction error. the samples generated by our adversarial attack algorithm are more meaningful and valuable than those produced by recently proposed methods, such as the direct noise and the back translation. experimental results demonstrate that the gec models trained with the data augmented by these adversarial examples can substantially improve both generalization and robustness.