Beyond Grammatical Error Correction: Improving L 1-influenced research writing in E nglish using pre-trained encoder-decoder models (2021)

Abstract: In this paper, we present a new method for training a writing improvement model adapted to the writer’s first language (L1) that goes beyond grammatical error correction (GEC). Without using annotated training data, we rely solely on pre-trained language models fine-tuned with parallel corpora of reference translation aligned with machine translation. We evaluate our model with corpora of academic papers written in English by L1 Portuguese and L1 Spanish scholars and a reference corpus of expert academic English. We show that our model is able to address specific L1-influenced writing and more complex linguistic phenomena than existing methods, outperforming what a state-of-the-art GEC system can achieve in this regard. Our code and data are open to other researchers.

## introduction writing for international readerships can be challenging for scholars whose working language is not english @xcite , even when they are reasonably proficient in the language. however, existing research on tools for improving writing focuses mostly on grammatical error corrrection (gec), and often relies on college-level corpora as a benchmark for lower-english proficiency levels. while gec can address more straightforward grammar mistakes, it does not adequately address fluency and more complex linguistic issues @xcite . moreover, few studies address the effects of first-language (l1) transfer on writing @xcite . another limitation is that state-of-the-art methods in gec such as neural machine-translationbased (nmt) approaches @xcite and transformer-based sequence-to-sequence models @xcite typically require a large amount of pseudo-errors generated from monolingual data using rule-base corruption @xcite , back-translation @xcite , or round-trip translation @xcite . these methods tend to introduce errors with limited diversity, most of which involve spelling and grammar. there have been recent attempts to eliminate the time-consuming pre-training step by employing pre-trained transformer models. @xcite used pre-trained transformer models in a language-model setting, and katsumata and komachi (2020) fine-tuned bart @xcite with a small corpus of annotated sentences. both approaches achieved results comparable to models trained with millions of sentences. this study proposes a method for improving l1influenced english texts beyond gec through the use of pre-trained encoder-decoder models. our approach uses parallel corpora of english aligned with english that has been machine-translated from portuguese and spanish as training data that emulates l1-influenced writing. the models generate sentences that have a higher level of acceptability and are more linguistically diverse than what a state-of-the-art gec system can achieve. we also propose new metrics for evaluating improvement beyond correctness. more recently, @xcite proposed a novel data-synthesis method to generate errorcorrected sentence pairs based on a pair of machine-translation models of different qualities. however, good quality nmt translation may not be good enough as a reference. ## method our method consists of fine-tuning a pre-trained encoder-decoder transformer model with parallel corpora of english and machine-translated english. ## experiments in this section, we describe the two corpora compiled for evaluating our model and the experimental setup used in the evaluation. ## results and discussion the results for brace are presented in our results show that both small and large models generate sentences that have a greater degree of acceptability and are more lexically and syntactically diverse by a large margin in comparison with the gector baseline. on the other hand, gec-tor performed better on grammatically. this was not unexpected, as our model was not optimized for grammatically and was trained with fewer sentences. in addition, we analysed the ability of the models to capture l1-influence. we extracted the 1000 most frequent 2-,3-,4-grams of the copep corpus of journal articles written in portuguese @xcite , translated them into english and compared their frequency in brace and expace, to test whether the copep n-grams were overused in brace. we found that 74% of the n-grams were indeed overused in brace when compared with expace. our model revised 23% of these, while gector only tackled 14%. in doing so, overall ngram overuse was reduced by 34% with our model against 19% with gector. a preliminary error analysis of the output of our model was conducted using a sample of 100 sentences from brace that were submitted to our system. we annotated each sentence pair with er-rant @xcite and manually classified 284 changes in one or more categories of errors. although these issues could be mitigated by increasing τ filtering, it would also filter out relevant suggestions. instead, we suggest further research on incorporating local biases on specific tokens, giving either a boost or reduction on the likelihood of changing tokens. the gector approach of tagging sentences instead of rewriting them allows for the use of bias to keep tokens unchanged. however, this restricts the flexibility of more complex and linguistic diverse changes, as seen in our results. we argue that future research should incorporate the benefits of both approaches in a single method, achieving high-flexibility with controlled results. ## final remarks we introduced a new method for improving l1influenced academic english by using a pre-trained encoder-decoder transformer. we showed that our model generates sentences with a higher acceptability and that are more linguistically diverse than a state-of-the-art gec model. however, further research is needed to assess the extent to which our model over-corrects or introduces errors. the approach taken in this study can be extended to other l1s by modifying existing parallel corpora with machine translation. it can also be extended to other domains, beyond academic, as our method does not rely on annotated corpora. it is evident that more research is needed beyond grammatical correction, including the development of corpora and models addressing more linguistically diverse issues.