A dversarial G rammatical E rror C orrection (2020)

Abstract: Recent works in Grammatical Error Correction (GEC) have leveraged the progress in Neural Machine Translation (NMT), to learn rewrites from parallel corpora of grammatically incorrect and corrected sentences, achieving state-of-the-art results. At the same time, Generative Adversarial Networks (GANs) have been successful in generating realistic texts across many different tasks by learning to directly minimize the difference between human-generated and synthetic text. In this work, we present an adversarial learning approach to GEC, using the generator-discriminator framework. The generator is a Transformer model, trained to produce grammatically correct sentences given grammatically incorrect ones. The discriminator is a sentence-pair classification model, trained to judge a given pair of grammatically incorrect-correct sentences on the quality of grammatical correction. We pre-train both the discriminator and the generator on parallel texts and then fine-tune them further using a policy gradient method that assigns high rewards to sentences which could be true corrections of the grammatically incorrect text. Experimental results on FCE, CoNLL-14, and BEA-19 datasets show that Adversarial-GEC can achieve competitive GEC quality compared to NMT-based baselines.

## introduction grammatical error correction (gec) has grown into a popular nlp task that deals with building systems for automatically correcting errors in written text @xcite . evolving from the approaches of building error-specific machine learning classifiers @xcite @xcite @xcite , it has gained popularity as a monolingual machine translation (mt) problem, where the system learns to "translate" a given erroneous text to its corrected form @xcite @xcite . initially, statistical phrase-based machine translation (smt) techniques were successfully applied to the task @xcite @xcite as as a way to handle all error types concurrently. more recently, several neural machine translation (nmt) systems have been developed with promising results @xcite @xcite , and their successful application to gec, either in combination with smt models @xcite @xcite , or strictly as neural models, has emerged as the new state-of-theart @xcite @xcite @xcite @xcite . despite the successes of nmt-based models for gec, a major challenge still lies in the definition of the evaluation metrics. ideally, the metric should be able to quantify the (a) lexical overlap, (b) semantic similarity, and (c) grammaticality of a generated sentence, given a grammatically incorrect input sentence. in a straightforward application of nmtbased models to the gec task, one would minimize a surrogate loss (e.g., cross-entropy), which is an upper bound on the true loss, and hence a loose approximation of these complex criteria. moreover, nmt-based gec models try to maximize n-gram or edit-based metrics, such as m 2 @xcite , i-measure @xcite , or gleu @xcite pushing the nmtbased models to generate sentences with n-gram precisions as high as possible, which may not necessarily lead to high-quality generation for the gec task. in order to avoid these issues, we take a different approach, inspired by generative adversarial networks (gans) @xcite , which provide a framework that can be leveraged to directly model the task based on the differences in the input-output distributions and the complex criteria mentioned above. moreover, gans have shown remarkable ability to generate coherent and semantically meaningful text in many natural language processing tasks such as machine translation @xcite , dialogue generation @xcite , and abstractive summarization @xcite among others. we propose a gan-based generatordiscriminator framework for grammatical error correction. the generator is a sequence-to-sequence (seq2seq) model, which is trained to "translate" a grammatically incorrect sentence to its grammatically correct rewrite. the discriminator, a deep neural sentence-pair classification model is trained to evaluate the probability of the generated sentence being a lexically-similar, meaningpreserving, and grammatically correct rewrite of the incorrect input sentence. adversarial training between the two models is set up as optimizing a min-max objective, where the discriminator learns to distinguish whether a given input is sampled from the ground-truth (human-generated) or generator (artificially-generated) distributions, maximizing the difference between them. the generator, on the other hand, learns to trick the discriminator by producing high-quality correction candidates, thus, minimizing the difference between its output and a ground-truth corrected sentence. further, the discriminator is used to fine-tune the generator using a policy gradient @xcite @xcite , rewarding high quality generated text when conditioned on the source, improving, thus, the generation results. by minimizing the difference between the human-and the artificially-generated distribution, we aim at directly optimizing the task based on the criteria mentioned above. we evaluate the effectiveness of our approach on three standard datasets on the task, observing that the discriminator can provide reasonably consistent guidance to the generator and further help improve its performance. experimental results indicate that our model can achieve significantly better performance than strong nmt-based baselines. in summary, we make the following contributions: ## adversarial gec fig. 1 outlines our approach which consists of two components the (a) generator (g) and (b) discriminator (d). ## results in contrast to related works on neural gec, we do not use a lot of the heuristics that most recent systems leverage in order to enhance their model performance pre-and post-training. these heuristics include using spellcheckers to correct spelling errors in the data, pre-trained language models trained on large quantities of external data, synthetic data generation, re-ranking systems to sort the outputs of the generator model, among others. we chose to keep our framework simple compared to most contemporary works in that we do not lever- the results of adversarial-gec compared to baseline models are presented in as mentioned in sections 2.2 and 3.2, we experiment with three discriminator formulations (ss, sp, gleu) in the adversarial-gec setting to provide the rewards to guide the generators. ## discussion in this section, we describe experimental results on adversarial training strategies, based on validation data splits. there are three parts to making the training work (a) formulating the discriminator task to compute the reward, (b) reducing the variance in rewards for better gradient estimation, and (c) combining the mle and adversarial objectives for more stable training. ## related work while the choice of a sentence-pair discriminator is close to @xcite and @xcite , our work differs from @xcite in that their learning objective is a combination of the discriminator reward (d) and a smoothed sentence-level bleu @xcite as the static reward (q). the use of a sentence-pair discriminator is related to our work, we do not combine rewards from d and q. incorporating q in the objective stems from the motivation to directly optimize for the evaluation metric, we choose to not force the evaluation metric-based reward into the objective, since most gec metrics are reference-based, and have shown to be limiting for the task @xcite . similarly, among existing works for gec, our work is the closest to @xcite , but they also directly maximize gleu in training their gec system, using a reinforce-based approach similar to ours. we instead let the model learn the latent nuances of the objective directly from the data, and provide the appropriate reward to the generator, preserving the learning objective as in @xcite , albeit with a different discriminator framework. our work is closest to @xcite , who built an rnnsearch-based generator @xcite and a cnn-based sentence-pair discriminator for nmt. ## conclusion we propose a task-appropriate training objective for gec, using an adversarial training framework consisting of a generator and a discriminator, based on the adversarial-nmt framework of @xcite . the generator is modeled as a seq2seq model, and the discriminator is modeled as a deep sentence-pair matching model, which provides rewards to the generator input-output. the framework supervises the generator to reflect the mapping within (source, target) sentence, and an efficient policy gradient algorithm to tackle the optimization difficulty brought by the discrete nature of generation. experiments on standard gec test datasets demonstrate the effectiveness of our framework for the task. additionally, we provide insights into how the discriminator setup, pre-training and integration into the framework can be optimized for stable training and better performance. we show that the proposed framework consistently achieves better results in a self-contained single model setting, without relying on any external resources. in the future, we plan to improve the task-specific framework and training techniques based on recent state-of-the-art methods @xcite , and improve issues with sparse rewards by exploring better credit assignment techniques. north american chapter of the association for computational linguistics: human language technologies, volume 1 (long papers), pages 1470-1480, new orleans, louisiana. association for computational linguistics. kyunghyun cho, bart van merrienboer, caglar gulcehre, dzmitry bahdanau, fethi bougares, holger schwenk, and yoshua bengio. 2014. learning phrase representations using rnn encoder-decoder for statistical machine translation. in proceedings of the 2014 conference on empirical methods in natural language processing (emnlp), pages 1724-1734, doha, qatar. association for computational linguistics. yo joong choe, jiyeon ham, kyubyong park, and yeoil yoon. 2019. a neural grammatical error correction system built on better pre-training and sequential transfer learning. in proceedings of the fourteenth workshop on innovative use of nlp for building educational applications, pages 213-227, florence, italy. association for computational linguistics. shamil chollampatt and hwee tou ng. 2018a. a multilayer convolutional encoder-decoder neural network for grammatical error correction. in proceedings of the thirty-second aaai conference on artificial intelligence. shamil chollampatt and hwee tou ng. 2018b. neural quality estimation of grammatical error correction. in proceedings of the 2018 conference on empirical methods in natural language processing, pages 2528-2539, brussels, belgium. association for computational linguistics. shamil chollampatt and hwee tou ng. 2018c. a reassessment of reference-based grammatical error correction metrics. in proceedings of the 27th international conference on computational linguistics, pages 2730-2741, santa fe, new mexico, usa. association for computational linguistics. shamil chollampatt, kaveh taghipour, and hwee tou ng. 2016. neural network translation models for grammatical error correction. in proceedings of the 25th international joint conference on artificial intelligence, new york, usa. leshem choshen and omri abend. 2018. inherent biases in reference-based evaluation for grammatical error correction. in proceedings of the 56th annual meeting of the association for computational linguistics (volume 1: long papers), pages 632-642, melbourne, australia. association for computational linguistics.