LM -Critic: Language Models for Unsupervised Grammatical Error Correction (2021)

Abstract: Grammatical error correction (GEC) requires a set of labeled ungrammatical / grammatical sentence pairs for training, but obtaining such annotation can be prohibitively expensive. Recently, the Break-It-Fix-It (BIFI) framework has demonstrated strong results on learning to repair a broken program without any labeled examples, but this relies on a perfect critic (e.g., a compiler) that returns whether an example is valid or not, which does not exist for the GEC task. In this work, we show how to leverage a pretrained language model (LM) in defining an LM-Critic, which judges a sentence to be grammatical if the LM assigns it a higher probability than its local perturbations. We apply this LM-Critic and BIFI along with a large set of unlabeled sentences to bootstrap realistic ungrammatical / grammatical pairs for training a corrector. We evaluate our approach on GEC datasets on multiple domains (CoNLL-2014, BEA-2019, GMEG-wiki and GMEG-yahoo) and show that it outperforms existing methods in both the unsupervised setting (+7.7 F0.5) and the supervised setting (+0.5 F0.5).

## introduction grammatical error correction (gec) is the task of fixing grammatical errors in text, such as typos, tense and article mistakes. recent works cast gec as a translation problem, using encoder-decoder models to map bad (ungrammatical) sentences into good (grammatical) sentences @xcite @xcite @xcite . these methods rely on a combination of human-labeled data (i.e., bad, good pairs) @xcite @xcite and synthetic data, which are generated by corrupting good sentences into synthetic bad, good pairs @xcite . human-labeled pairs are representative of real human errors but are expensive to obtain, while synthetic pairs are cheap but are unrealistic, deviating from the distribution of grammatical lm-critic deems a sentence to be grammatical if a pretrained language model (e.g., gpt2) assigns it a higher probability than candidates in its local neighborhood (e.g., edit distance 1). in this work, we propose lm-critic, a simple approximate critic for assessing grammaticality ( §3), and apply it with bifi to learn gec from unlabeled data ( §4). specifically, motivated by recent progress in large language models (lms) (e.g., gpt2, gpt3; @xcite ; @xcite ) and an intuition that a good lm assigns a higher probability to grammatical sentences than ungrammatical counterparts, we use an lm's probability to define a critic for grammaticality. a naive approach is to deem a sentence as grammatical if its probability exceeds an absolute threshold, but this does not work in practice, e.g., lms may assign a high probability just because the sentence has more common words. we hence compare probabilities in local neighborhood of sentences. concretely, lm-critic is defined by two components, an lm (e.g., gpt2) and a neighborhood function (e.g., edit distance 1), and deems a sentence to be grammatical if the lm assigns it the highest probability in its local neighborhood (figure 1 ; local optimum criterion). using this lm-critic, we apply bifi to the gec task. notably, our approach, both the lm-critic and gec learning, does not require labeled data. we evaluate our proposed approach on gec benchmarks across multiple domains, @xcite , bea-2019 @xcite , gmeg-yahoo, and gmeg-wiki @xcite . we achieve strong performance in the unsupervised setting (i.e., no labeled data), outperforming the baseline fixer trained on synthetic data by 7.7 f 0.5 on average. we also evaluate in the supervised setting, where we take the stateof-the-art model gector @xcite as the baseline fixer, and further fine-tune it by applying our approach using unlabeled data. we achieve 65.8 / 72.9 f 0.5 on conll-2014 / bea-2019, outperforming gector by 0.5 f 0.5 . our results also suggest that while existing bifi assumed access to an oracle critic (i.e., compiler), an approximate critic (i.e., lm-critic) can also help to improve model learning. ## problem setup the task of grammatical error correction (gec) is to map an ungrammatical sentence x bad into a grammatical version of it, x good (one that has the same intended meaning). a gec model (fixer) f aims to learn this mapping, typically using a paired dataset d @xmath0,x good (i) )}. in particular, we call it labeled if the pairs are human-annotated. in contrast, we call unlabeled data a set of raw sentences d @xmath1. for simplicity, we use "good"/"bad" to mean grammatical/ungrammatical interchangeably. unlike a fixer, which maps x bad to x good , a critic c merely assesses whether an input is good or bad: for a sentence x, given unlabeled data x's (some of which are good, some of which are bad), and a language model (lm), which returns a probability distribution p(x) over sentences x, we aim to define the critic ( §3; lm-critic) and use that to obtain the fixer ( §4; bifi). ## lm-critic the core of our approach to gec is a critic, which returns whether a sentence is good (grammatical) or bad (ungrammatical). motivated by recent progress in large-scale pre-trained lms (e.g., gpt2, gpt3; @xcite ; brown et al. ( 2020 )), we aim to use an lm's probability score to define a critic for grammaticality. specifically, we propose a criterion that deems a sentence to be good if it has the highest probability within its local neighborhood (local optimum criterion; §3.1). we implement this criterion using a pretrained lm and a sentence perturbation function (lm-critic; §3.2). we then do an intrinsic study on how well lm-critic works in practice ( §3.3). ## learning gec with lm-critic break-it-fix-it (bifi; yasunaga and liang ( 2021 )) is an existing method that uses a critic to obtain realistic paired data from unlabeled data. bifi was originally studied in the source code repair task where an oracle critic (e.g., compiler) exists, but there is no oracle critic in gec. here, we propose to apply bifi to the gec task by using lm-critic as the critic ( §4.1), and evaluate this approach on gec benchmarks ( §4.2). the difference from the original bifi is that our task is gec rather than code repair, and we use an approximate critic (i.e., lm-critic) instead of an oracle critic (i.e., compiler). ## related work and discussion grammatical error correction (gec). gec models are commonly trained from human-labeled data @xcite @xcite , or synthetic data generated by heuristically corrupting unlabeled sentences @xcite @xcite @xcite . several works aim to improve the methods for generating paired data, such as learning a breaker from existing labeled data @xcite , applying backtranslation @xcite to gec @xcite , and synthesizing extra paired data by comparing model predictions and references @xcite . different from the above works, our method (i) does not require labeled data (works for both unsupervised and supervised settings), and (ii) uses lm-critic to filter the "bad"-side and "good"-side of generated pairs. ## conclusion we presented lm-critic, a method that uses a pretrained language model (lm) as a critic for assessing sentence grammaticality. using lm-critic and the bifi algorithm, we learn grammatical error correction (gec) by generating realistic training data from unlabeled text. notably, our approach does not require labeled data, and can also be viewed as an unsupervised method to turn a (gpt2scale) pretrained lm into an actual gec system. using multiple gec datasets, we showed that our approach achieves strong performance on unsupervised gec, suggesting the promise of our method for domains and languages with no labeled gec data. we hope this work opens up research avenues in lm-based critics and unsupervised gec.