Efficient Grammatical Error Correction Via Multi-Task Training and Optimized Training Schedule (2023)

Abstract: Progress in neural grammatical error correction (GEC) is hindered by the lack of annotated training data. Sufficient amounts of high-quality manually annotated data are not available, so recent research has relied on generating synthetic data, pretraining on it, and then fine-tuning on real datasets; performance gains have been achieved either by ensembling or by using huge pretrained models such as XXL-T5 as the backbone. In this work, we explore an orthogonal direction: how to use available data more efficiently. First, we propose auxiliary tasks that exploit the alignment between the original and corrected sentences, such as predicting a sequence of corrections. We formulate each task as a sequence-to-sequence problem and perform multi-task training. Second, we discover that the order of datasets used for training and even individual instances within a dataset may have important effects on the final performance, so we set out to find the best training schedule. Together, these two ideas lead to significant improvements, producing results that improve state of the art with much smaller models; in particular, we outperform the best models based on T5-XXL (11B parameters) with a BART-based model (400M parameters).

## introduction grammatical error correction (gec) is an important problem setting with obvious applications that often require high-level understanding. recent research has developed a common view of grammatical error correction (gec) as monolingual text-to-text rewriting @xcite . gec is often tackled with encoder-decoder architectures: the encoder receives an errorful sentence, and the decoder pro-duces its corrected version. in line with mainstream nlp developments, transformer-based architectures have proven to be beneficial for gec, leading to recent breakthroughs on multiple benchmarks @xcite @xcite . gec is a challenging task that goes beyond simple spell-checking. the greatest difficulty comes from the complexity of grammar and punctuation rules, inconsistency in rules usage, contextual factors, and multiple correct answers. to solve gec, a model should have a thorough understanding of grammar, punctuation, and syntax, and be aware of the ambiguity introduced by errors to correctly understand intended meaning. most recent works approach these challenges by providing new complex architectures, using larger models, or ensembling multiple models, e.g. @xcite ; @xcite ; @xcite ; @xcite . however, practicality demands faster and smaller models, so in this work, we examine another direction to improve gec: modifying the training pipeline. every step is important: (i) data preparation includes generating synthetic data for pretraining and preprocessing available datasets; (ii) pretraining is done on synthetic data and/or large-scale datasets with supervised pretext tasks such as language modeling; (iii) fine-tuning on downstream datasets is also far from straightforward; at the very least, one has to choose the order of training on available datasets, which may significantly impact the results. due to the scarcity of annotated data for fine-tuning, we are concerned with using it as efficiently as possible and introduce two novel approaches for this. first, one can acquire additional information from parallel annotated corpora, e.g., an alignment between two sentences that can be used to derive operations that would transform one into the other. one can decompose the editing process into elementary edits: insertion, deletion, replacement, and change of word order; the latter can also be viewed as a sequence of deletions and insertions. this information is exploited by sequence tagging models that show good results on the gec task, see @xcite ; @xcite . a drawback of this approach lies in the need to specify a vocabulary of available operations, use several decoding steps to correct complex errors, and in the hardness/inability to make complex rephrasing. we utilize additional information by converting it into auxiliary tasks and formulating them as sequence-to-sequence problems. as such, the model learns separate "skills" required for the successful correction of grammatical errors. second, gec-specific datasets have different quality and sources, e.g., different language proficiency levels. some datasets are collected from online platforms where users correct each other (they are noisier); in other datasets, sentences are corrected by teachers or annotators (these are cleaner). some datasets consist of essays written by native speakers; others, of essays written by non-native students. thus, the distribution of errors may severely differ. it is tempting to remove "noisy" or out-of-distribution examples from training sets, but it seems that the model can learn even from such instances. we propose to use a training schedule for gec datasets and show that it does matter how we order them. also, we find that the order of sentences within the dataset matters as well; namely, we have found that placing sentences from the same block of text (e.g., the same essay) in the same batch is beneficial. our primary contributions are as follows. first, we introduce a multi-task pretraining approach and a fine-tuning schema that together yield an improvement of up to 3% in f 0.5 score compared to similarsized state of the art models 1 . second, we show that our approach is able to outperform state of the art large models (t5-xl with 3b parameters and t5-xxl with 11b parameters) using a model with 400m parameters, reducing the computational load and ecological footprint. third, our approach makes the model more robust, improving metrics on several datasets rather than trading them off 1 we plan to release the source code of our models upon acceptance. ## related work neural approaches to grammatical error correction follow two main lines of research: (i) sequence tagging models and (ii) sequence-to-sequence models. synthetic data. training gec models is difficult due to the natural lack of suitable training data and possible erroneous corrections, so synthetic data becomes a crucial part of any gec pipeline @xcite @xcite . it had been used for gec even before the deep learning era that required larger datasets @xcite . synthetic data generators can mimic common typos and grammatical errors but usually cannot capture the target error distribution found in real-life evaluation sets or standard benchmarks. methods for synthetic data generation include character perturbations, dictionary-or edit-distance-based replacements, shuffling word order, rule-based suffix transformations, and more @xcite @xcite . an empirical study of how to generate and use the synthetic data was done in @xcite . another line of research upsamples training data in existing datasets. @xcite train a gec model on a natural noisy dataset and then use its outputs for source sentences to construct a less noisy parallel dataset; @xcite use sentence rewriting approaches; @xcite apply delta-log-perplexity scoring to rank sentences according to the difference in perplexity between two base model checkpoints and use higher-scoring sentences for final fine-tuning. multi-stage fine-tuning. due to data scarcity, training gec models from scratch could be cumbersome. one of the options is to pre-train a model on some auxiliary task, e.g. @xcite proposed to initialize the gec model with the pre-trained denoising autoencoder. many gec pipelines utilize pre-trained language models as backbone models for gec; in particular, @xcite used t5 @xcite while katsumata and komachi (2020) and @xcite used bart @xcite . pretrained language models are also beneficial for reranking output hypotheses generated with beam search @xcite . choe et al. (2019), omelianchuk et al. (2020) and tarnavskyi et al. (2022) decompose the training process of a gec model into several stages: (i) pre-training on an errorful synthetic dataset; (ii) fine-tuning on natural high-quality datasets that combine both errorful and error-free sentences. each stage requires its own tuning of hyperparameters such as the number of training steps and the learning rate. multi-task learning. several works aim to utilize additional information along with the standard parallel mapping. first, the grammatical error detection (ged) task can be extracted from gec; @xcite perform multi-task training with ged and gec tasks and use ged features for reranking. a similar approach by @xcite trained a ged model separately and used it to filter edits. @xcite separately pretrain a ged model and use its outputs as auxiliary inputs to fine-tune the encoder-decoder gec model and rerank its outputs. @xcite incorporate syntactic dependency information into the encoder. non-autoregressive decoding. another line of research introduces non-autoregressive decoding to speed up models. @xcite predict language-specific edits to be applied to the output sequence. iterative refinement is also possible. @xcite non-autoregressively refine an output sequence using language-agnostic insertions and deletions. @xcite decompose the inference stage into permutation and decoding. first, a permutation network repositions the tokens of an input sequence with possible deletions and inser-tions. second, the intermediate sequence is passed to a decoder network that iteratively fills in inserted placeholders. ## approach in this section, we introduce our approach that uses multi-task learning and optimizes the training schedule for the sequence-to-sequence model architecture. in section 3.1, we outline the model and hyperparameters used for training. in section 3.2, we describe existing gec datasets for training and evaluation, highlighting the specific properties of each. in section 3.3, we specify the main and auxiliary tasks formulated as sequence-to-sequence mapping. section 3.4 describes the training steps. ## conclusion in this work, we have introduced a new approach to training and fine-tuning sequence-to-sequence models for the grammatical error correction task based on multi-task pretraining and optimized training schedules. the proposed approach fares better than previous state of the art models with a much smaller model size and outperforms models of comparable size by a wide margin on both conll-14 and bea-test datasets; in particular, we have achieved results exceeding state of the art models based on t5-xxl (11b parameters) with a bartbased model (400m parameters). our multi-task approach encodes auxiliary tasks as text rewriting problems and does not require any changes in the model architecture. we believe that the proposed approach is of significant value for practical gec pipelines and see several directions for possible improvements in further work, including modifications of text encodings for auxiliary tasks and adding new auxiliary tasks to the training process.