Improving S eq2 S eq Grammatical Error Correction via Decoding Interventions (2023)

Abstract: The sequence-to-sequence (Seq2Seq) approach has recently been widely used in grammatical error correction (GEC) and shows promising performance. However, the Seq2Seq GEC approach still suffers from two issues. First, a Seq2Seq GEC model can only be trained on parallel data, which, in GEC task, is often noisy and limited in quantity. Second, the decoder of a Seq2Seq GEC model lacks an explicit awareness of the correctness of the token being generated. In this paper, we propose a unified decoding intervention framework that employs an external critic to assess the appropriateness of the token to be generated incrementally, and then dynamically influence the choice of the next token. We discover and investigate two types of critics: a pre-trained left-to-right language model critic and an incremental target-side grammatical error detector critic. Through extensive experiments on English and Chinese datasets, our framework consistently outperforms strong baselines and achieves results competitive with state-of-the-art methods.

## introduction automatically correcting grammatical errors is an important task of practical value in the nlp field. the potential applications include document proofreading, writing assistant, language learning education, text post-processing for automatic speech recognition @xcite , etc. there are two mainstream approaches to grammatical error correction (gec), namely sequence-to-sequence (seq2seq) @xcite and sequence-to-edit (seq2edit) @xcite . the seq2seq approach treats gec as a monolingual text translation/transduction task, whereas the seq2edit approach casts gec into a sequence labeling task. input: but there had no buyers . reference: but there were no buyers . first, a seq2seq model can only utilize parallel sentence pairs as training data, in which the input sentence is potentially ungrammatical whereas the target one is considered as correct. usually, a major proportion of the training data is automatically collected from language learner websites such as lang-8. on the one hand, the lang-8 data contains a certain amount of noises, considering the voluntary contributors may make mistakes as well. on the other hand, the data scale is quite limited, compared with the non-parallel data used for training large language models. for instance, the cleaned version of english lang-8 corpus (clang8) contains only 2.4m sentence pairs @xcite . data augmentation is a popular approach for addressing the limited scale issue, i.e., synthesizing large amount of training data @xcite . however, it can be very difficult and tricky to control the error distribution in the generated data so that it resembles the realistic scenario. moreover, it brings a heavy computational cost to train a gec model with very large training data. the second issue of the seq2seq gec model is that the decoder lacks an explicit awareness or evaluation of whether the generated token is correct during decoding. there are indeed several works that perform grammatical error detection (ged) for the input sentence, and use the results as extra features for the encoder so that the decoder pays extra attention to the erroneous spans in an implicit manner @xcite @xcite . however, we are not aware of any previous works that explicitly checks the correctness of generated tokens during decoding (e.g., target-side ged). as pointed out by @xcite , a seq2seq gec model tends to generate wrong corrections when the model encounters errors unseen in the training data. in this work, we propose a decoding intervention framework to address both issues of the seq2seq gec approach. as illustrated in figure 1 , we employ an external critic to assess the appropriateness of the token to be generated incrementally, and then dynamically influence the choice of the next token. specifically, at each decoding step, the critic will evaluate the appropriateness of the candidate tokens, if the token is inappropriate, the critic will punish the generation of the token by reducing the log-probability score of the token. the key to our decoding interventions is to find a suitable critic. we discover and investigate two useful critics. the first critic is a pre-trained leftto-right language model (lm). using the language model as the critic can take advantage of its knowledge learned from the vast amount of text. if the language model gives a low probability to a token, then the token is probably wrong even if the gec model gives it a high probability. the second critic is a ged model which is an ideal critic to incorporate the explicit awareness of correctness into the seq2seq gec model during the decoding process. however, the conventional ged cannot be directly used as the critic because it does not match the incremental manner of the decoding process. to address this problem, we propose an incremental target-side ged, which acts in a seq2seq manner making judgments on the token to be generated y t based on both the input sentence x and the tokens generated so far y ăt . we conduct experiments on three english gec datasets, including two english-as-a-secondlanguage (esl) datasets, a multi-domain native-english dataset, and a chinese dataset. experimental results demonstrate that our decoding intervention brings consistent and substantial improvements on all datasets. the results also show that with the help of decoding intervention, our gec model can achieve comparable performance to the state-of-the-art models on all datasets under comparable settings without any re-training. our code is available at https://github.c om/jacob-zhou/gecdi 2 the basic seq2seq gec model this work aims to improve the seq2seq gec approach. in this section, we briefly describe it. given a potentially erroneous sentence, a seq2seq gec model tries to generate a correct one without changing its meaning, similar to machine translation, yet in a monolingual fashion. we adopt the widely-used transformer architecture @xcite as our model backbone, which comprises an encoder and a decoder. given an input sentence x " x 1 . . . x n , the encoder first encodes it into a sequence of hidden states h " h 1 . . . h n . at each timestamp, given the input sentence representation h and the previously generated tokens y ăt , the decoder calculates a probability distribution over the vocabulary for the next-token generation: the score of an output sentence y is the sum of the log-probabilities of all predicted tokens: during training, seq2seq models commonly employ the teacher forcing method, aiming to maximize the log-likelihood of the ground-truth next token g t , given the input sentence x and the previous ground-truth tokens g ăt : the main advantage of teacher forcing is that it allows for parallel training. the inference of seq2seq gec models is to find the best output sentence y ˚by solving the following optimization problem: ## decoding intervention in this work, we propose a decoding intervention framework to improve the seq2seq gec. concretely, we use an external critic model to dynamically evaluate the correctness of the next token predicted by the gec model during decoding. the evaluation produces a penalty score to the existing probability distribution from the gec model as follows: the first term is the original probability from the gec model. the logarithm transform stretches the probability into a wider range and thus makes it more influential. this also gives more flexibility to the design of the critic model 1 . the second term is the penalty score from the critic model to "y t " given the input sentence x and the generated prefix y ăt and λ is a coefficient that controls the trade-off between two model scores. please note that λ is not a global hyper-parameter but is instead decided by the scores in a token-wise manner. we detail this in section 3.3. from eq. ( 5 ), we can draw two characteristics of our framework. moreover, the critic model may or may not use the input sentence x. in this work, we discover and investigate two useful critic models, i.e., a pure leftto-right pre-trained language model which does not use x, and an incremental target-side ged model that uses x. ## ablation studies impact of critic sizes we perform experiments using four distinct sizes of language models and two different sizes of target-side geds. as shown in due to the uniform size of the chinese gpt-2 models we found, we only performed experiments on the target-side geds for the chinese dataset. the results show that a larger target-side ged is more effective when used with the chinese dataset. effectiveness of the dynamic coefficient as mentioned in section 4.1, the language model and the target-side ged exhibit specific tendencies when improving the gec model. however, we also observed that a critic tends to decrease one score when improving another. for instance, while the target-side geds improve the precision score, they also result in a decline in the recall score. it might be caused by the misjudgment of the critics when they are unconfident. as a result, the improvement of the f 0.5 score is potentially hindered. to address this issue, we propose a coefficient strategy that dynamically adjusts the coefficient of the critic at each decoding time step according to the confidence levels of the critic and the gec model. results in ## conclusions in this paper, we propose a unified decoding intervention framework for gec models. within this framework, we discover and investigate two useful critics: the language model critic and the targetside ged critic. among them, the target-side ged critic represents a novel contribution. while most existing research has employed ged on the input side, this work is the first to leverage ged on the target side to assist gec. although the concept of a language model critic may not be entirely new, we argue that it is still worth investigating its effectiveness on the gec task, especially in the era of pre-trained language models. experiments conducted on four english and chinese gec datasets lead to several promising findings. firstly, the decoding intervention framework can consistently and substantially improve the performance of gec models, regardless of whether a language model or error detector is used as the critic. secondly, the language model critic is better at improving the recall rate, while the target-side ged critic is better at improving the precision rate. thirdly, while the size of the critic has a minor impact on the esl dataset, it becomes substantial on the multi-domain english dataset from native speakers, as well as the chinese dataset. finally, aided by the decoding intervention framework, our baseline gec model shows competitive performance when compared to state-of-the-art models.