Efficient Grammatical Error Correction Via Multi-Task Training and Optimized Training Schedule
Authors: Andrey Bout
Alexander Podolskiy
Sergey Nikolenko
Irina Piontkovskaya
Venue: EMNLP 2023 Main (2023)
Keywords: Grammatical error correction
Multi-task training
Sequence-to-sequence
Fine-tuning
TL;DR: Multi-task training is formulated as a sequence-to-sequence problem and the specific training order improves grammatical correction models.

Abstract: Progress in neural grammatical error correction (GEC) is hindered by the lack of annotated training data. Sufficient amounts of high-quality manually annotated data are not available, so recent research has relied on generating synthetic data, pretraining on it, and then fine-tuning on real datasets; performance gains have been achieved either by ensembling or by using huge pretrained models such as XXL-T5 as the backbone. In this work, we explore an orthogonal direction: how to use available data more efficiently. First, we propose auxiliary tasks that exploit the alignment between the original and corrected sentences, such as predicting a sequence of corrections. We formulate each task as a sequence-to-sequence problem and perform multi-task training. Second, we discover that the order of datasets used for training and even individual instances within a dataset may have important effects on the final performance, so we set out to find the best training schedule. Together, these two ideas lead to significant improvements, producing results that improve state of the art with much smaller models; in particular, we outperform the best models based on T5-XXL (11B parameters) with a BART-based model (400M parameters).

Efficient Grammatical Error Correction Via Multi-Task Training and Optimized Training Schedule
Andrey Bout
Alexander Podolskiy Huawei Noah's Ark Lab podolskiy. podolskly.alexander alexander @huawei.com
Huawei Noah's 's Ark Lab
bout andrrey@awe........cccc com
Sergey Nikolenko
Irina Piontkovskaya
St. Petersburg Department of the
Huawei Noah's Ark Lab piontkovskaya @huawei.com com
Steklov Institute of Mathematics sergeyêlogic, pdmi. ras. ru
Abstract
duces its corrected version. In line with main- stream NLP developments, Transformer-base ar- chitectures have proven to be beneficial for GEC, leading to recent breakthroughs on multiple bench- marks (Rothe et al., 2021; Tarnavskyi et al., 2022; Sun and Wang, 2022).
Progress in neural grammatical error correc- (GEC) hindered by the lack anno- tated training data. Sufficient amounts of high- quality manually annotated data are not avail- able, so recent research has relied on generating synthetic data, pretraining it, and then fine- tuning on real datasets; performance gains have been achieved either by ensembling or by using huge pretrained models such as XXL-TS as the backbone. In this work, we explore an orthog- onal direction: how to available data more efficiently. First, we propose auxiliary tasks that exploit alignment between original and corrected sentences, such predicting sequence of corrections. We formulate each task requence-to-sequenne problem and perform multi-task training. Second, we dis- cover that the order of datasets used for training and individual instances within dataset may have important effects the final perfor- mance, so we set out find the best training schedule. Together, these two ideas lead to sig- nificant improvements, producing results that improve state of the art with much smaller mod- els; in particular, we outperform the best mod- els based on T5-XXL (1IB parameters) with BART-based model (400M parameters).
GEC is a challenging task that goes beyond sim- ple spell-checking. The greatest difficulty comes from the complexity of grammar and punctuation rules, inconsistency in rules usage, contextual fac- tors, and multiple correct answers, To solve GEC, model should have thorough understanding of grammar, punctuation, and syntax, and be aware of the ambiguity introduced by errors to correctly understand intended meaning.
Most recent works approach these challenges by providing new complex architectures, using larger models, ensembling multiple models, e.g. Yuan al. (2021); Rothe al. (2021); Tarnavskyi al. (2022); Zhang al. (2022). However, practical- ity demands faster and smaller models, so in this work, we examine another direction to improve GEC: modifying the training pipeline. Evvery step is important: (i) data preparation includes generat- ing synthetic data for pretraining and preprocess- ing available datasets; (ii) pretraining is done on synthetic data and/or large-scale datasets with pervised pretext tasks such language modeling; (iii) fine-tuming on downstream datasets is also far from straightforward; at the very least, one has to choose the order of training on available datasets, which may significantly impact the results. Due to the scarcity of annotated data for fine-tuning, we concerned with using it as efficiently possible and introduce two novel approaches for this.
1 Introduction
Grammatical error correction (GEC) is an impor- tant problem setting with obvious applications that often require high-level understanding. Recent re- scarch has developed common view of gram- matical error correction (GEC) monolingual text-to-text rewriting (Náplava and Straka, 2019; Grundkiewicz et al., 2019). GEC is often tackled with encoder-decoder architectures: the encoder receives an errorful sentence, and the decoder pro-
First, one can acquire additional information from parallel annotated corpora, e.g., an alignment

---

between lwo sentences that can be used to derive operations that would transform one into the other. One can decompose the editing process into el- ementary edits: insertion, deletion, replacement, and change of word order; the latter can also be viewed as a sequence of deletions and insertions. This information is exploited by sequence tagging models that show good results the GEC task, Omelianchuk et al. (2020, 2021); Tarnavskyi al. (2022). drawback of this approach lies in the need to specify a vocabulary of available opera- tions, use several decoding steps to correct complex errors, and the hardness/imability to make com- plex rephrasing. We utilize additional information by converting it into auxiliary tasks and formulat- ing them sequence-to-sequence problems. As such, the model learns separate "skills" required for the successful correction of grammatical errors. Second, GEC-specific datasets have different quality and sources, e.g., different language profi- ciency levels. Some datasets are collected from line platforms where correct each other (they are noisier); in other datasets, sentences are cor- rected by teachers or annotators (these are cleaner). Some datasets consist of essays written by native speakers; others, of essays written by non-native students. Thus, the distribution of errors may severely differ. It is tempting to remove "noisy" out-of -utto--distribution examples from training sets, but it seems that the model can learn such instances. We propose to a training sched- ule for GEC datasets and show that it does matter how we order them. Also, we find that the order sentences within the dataset matters as well; namely, we have found that placing sentences from the same block of text (e.g., the same essay) in the same batch is beneficial.
Dataset
Sentences % errorful
Stages
C4200M
180M
99.4
PIE-synthetic
~ 9M
100.0
Lang-8
947 344
52.5
NUCLE
56958
38.0
FCE
34
62.4
W&I+L
34.004
67.3
II, III
W&I+L dev
4.38
64.3
Dev
CoNLL test
1312
71.9
Test
W&I+L test
4477
N/A
Test
Table 1: Dataset statistics and training stages.
of each other (as usual). In what follows, Sec- tion surveys related work, Section 3 introduces our approach, Section 4 shows evaluation results, an ablation study, and an analysis of our auxiliary tasks, Section 5 concludes the paper, and Section 6 discusses the limitations of our approach.
Related work
Neural upproaches to grammatical error correction follow two main lines of research: (i) sequence tag- ging models and (ii) sequence-to-sequenee models. Synthetic data. Training GEC models is dif- ficult due to the natural lack of suitable training data and possible erroncous corrections, so syn- thetic data becomes crucial part of any GEC pipeline (Choe et al., 2019; Stahlberg and Ku- mar, 2021; Htut and Tetreault, 2019). It had been used for GEC even before the deep learning era that required larger datasets (Foster and Ander- sen, 2009; Brockett et al., 2006). Synthetic data generators can mimic common typos and gram- matical errors but usually cannot capture the tar- get error distribution found in real-life evaluation sets standard benchmarks. Methods for syn- thetic data generation include character perturba- tions, dictionary- edit-distance-based replace- ments, shufiling word order, rule-based suffix trans- formations, and more (Grundkiewicz al., 2019; Awasthi et al., 2019a; Náplava and Straka, 2019; Rothe et al., 2021). )). An empirical study how to generate and use the synthetic data was done in Kiyono et al. (2019).
from
Our primary contributions are as follows. First, we introduce multi-task pretraining approach and a fine-tuning schema that together yield an improve- ment of up to 3% in Fo.5 score compared to similar- show
sized state of the art m Second, that our upproach is able to outperform state of the art large models (T5-XL with 3B parameters and T5-XXL with IIB parameters) using model with 400M parameters, reducing the computational load and ecological footprint. Third, our approach makes the model more robust, improving metrics on several datasets rather than trading them off
Another line of research upsamples training data in existing datasets. Mita et al. (2020) train a GEC model on natural noisy dataset and then use its outputs for source sentences to construct a less noisy parallel dataset; Zhang et al. (2019)
'oo plan to release the source code of our models upon acceptance.

---

natu- ral high-quality datasets that combine both errorful and error-free sentences. Each stage requires its use sentence rewriting approaches; Lichtarge et al. (2020) apply delta-log-perplexing scoring rank sentences according to the difference in perplex- ity between two base model checkpoints and use higher-scoring sentences for final fine-tuning.
tions. Second, the intermediate sequence is passed to decoder network that iteratively fills in inserted placeholders.
GPT-3.5 and GPT-4. The recent success of GPT-based models for wide variety of tasks has led to several parallel works that compare how well these models fare in grammatical error correction. Fang et al. (2023) show that ChatGPT is still worse on GEC benchmarks than fine-tuned sequence-to- sequence models, both in the zero-shot and few- shot scenarios. Coyne et al. (2023) provide the corresponding analysis for GPT-4, with even lower results, which means that specialized models are still relevant for GEC scenarios and validate our research.
Multi-stage fine-tuning, Due to data scarcity, training GEC models from scratch could be cum- bersome. One of the options is to pre-train model on some auxiliary task, e.g. Choe et al. (2019) proposed to initialize the GEC model with the pre-trained denoising autoencoder. Many GEC pipelines utilize pre-trained language models as backbone models for GEC; in particular, Rothe et. (2021) used T5 (Raffel et al., 2020) while Kat- sumata and Komachi (2020) and Sun and Wang (2022) used BART (Lewis et al., 2020). Pre- trained language models are also beneficial for reranking output hypotheses generated with beam search (Kaneko et al., 2019). Choe et al. (2019), Omelianchuk et al. (2020) and Tarnavskyi et al. (2022) decompose the training process of a GEC model into several stages: (i) pre-training on an errorful synthetic dataset; (ii) fine-tuning
Approach
In this section, we introduce our approach that uses multi-task learning and optimizes the training schedule for the sequence-to-sequenee model archi- tecture. In Section 3.1, we outline the model and hyperparameters used for training. In Section 3.2, describe existing GEC datasets for training and evaluation, highlighting the specific properties of each, In Section 3.3, we specify the main and aux- iliary tasks formulated as seateenceto-sqquenee mapping. Section 3.4 describes the training steps.
own tuning of hyperparameters such as the number of training steps and the learning rate.
Multi-task learning. Several works aim to uti- lize additional information along with the standard parallel mapping. First, the grammatical error de- tection (GED) task can be extracted from GEC; Yuan al. (2019) perform multi-task training with GED and GEC tasks and use GED features for reranking. similar approach by Fang et al. (2020) trained a GED model separately and used it to fil- ter edits. Yuan et al. (2021) separately pretrain a GED model and use its outputs as auxiliary inputs to fine-tune the encoder-decoder GEC model and rerank its outputs. Zhang al. (2022) incorporate syntactic dependency information into the encoder.
3.1 Model
We use straightforward text-only approach while also keeping the model size limited. Namely, we formulate all training tasks as sequence-to- sequence text rewriting problems and do not in- troduce any additional heads for edit prediction or other tasks. As the backbone, we use the BART (Lewis al., 2020) model with a 12-layer encoder and decoder. We train the model fp16 mode with learning rate 1e-5 and warmup ratio 7% with a linear scheduler. We 12 instances in a batch and use gradient accumulation with 4 steps to achieve larger effective batch size. We did not perform hyperparameter search except for the learning rate, which was chosen from [1e-5, -5 1e le-6j, JJ. All training experiments were done 8 NVIDIA Tesla V100 GPUs with 32GB of memory.
Non-autoregressive decoding. Another line of research introduces non-autoregressive decoding to speed up models. Awasthi et al. (2019b) predict language-specific edits to be applied to the output sequence. Iterative refinement is also possible. Gu et al. (2019) non-autoreerressively refine output sequence using language-agnostic insertions and deletions. Yakovlev et al. (2023) decompose the in- ference stage into permutation and decoding. First, a permutation network repositions the tokens of an input sequence with possible deletions and inser-
3.2 Training Data
For pretraining (Stage 1), we use C4200M or PIE datasets. C4200 synthetic corpus based on clean sentences from the C4 dataset. This corpus was generated by a tagged corruption model to

---

<Gorrect>
<apply>
Target:
Parrple can wlso find our great deal of information from Thviter and such as Obama and BU Gates and get frrs-hare hand study materille these
Peoplee
information about the
People
mush celerity Teeter and
Moddal
People can also know much information about the oolote by Twitter and
Facebook sush Ciaama, dates soudy matorials
of
greas deal
find
Bill Gurkes and get the fius-
hand sludy matioiabs on
Nodel
from Twitter
find Raplecs colebrity with colebiitis
Reclace
Facebook such as Oberma Bill Gattes and gael fired- hind stlwty
(a) Basic "correct" task
Dalele the
Replaize with from
Replatae, wilh and
<explain>>
Deleto the
Replano
these.
Input:
Edits
can Information about mush
(c) "Apply" task
colebbiit tee and
Replace know with find but
cacebook such as Obeama and gel the hend sludy
Repiaazz much graal cieal
<odiie
Edita
celebrity with colcbrilics
Ingut:
Replico with great
Target:
Pecple can also find out
Replace with from Replace with and Delete the Replace it with these
much informetion about the aillbrity in "witter as
Model
Delete the
great deal of informaiin cesshifliss
celebrities
Facebook such Obema and Bill Gatcs and get trrs hand etJoy matiria these.
and Facehook Gales get the first hoah study materiale on it.
Replaos. witt and
Raplana in with from
Delete the Replace il with these
(b) "Explain" task
(d) "Edit" task
Figure 1: Tasks automatically generated from a source- tturee-target pair for multi-task pretraining.
meet the error distribution of BEA-dev (Bryant et al., 2019a); see Stahlberg and Kumar (2021) ) for details. PIE is synthetic dataset of 9M parallel sentences generated by using based grammat- ical errors such as deletion, insertion, and word replacement (Awasthi et al., 2019a).
opment set, i.e. W&I+L-dev, to choose the best model and report results on the CoNLL2014 test set (Ng et al., 2014) evaluated by the official M2 scorer (Dahlmeier and Ng, 2012) and the BEA- 2019 test set evaluated by ERRANT (Bryant al., 2017). Table 1 summarizes dataset statistics and shows which training stages they are used for.
For other training stages, we use the following datasets: (i) National University of Singapore Cor- pus of Learner English (NUCLE) (Dahlmeier et al., 2013) that consists of essays written by undergrad- uate students different topics and annotated by professional English instructors; (ii) Lang-8 Corpus of Learner English (Lang-8) (Tajiri al., 2012) collected from the online language learn- ing site Lang-8; this dataset is relatively "noisy" as the users corrected themselves, and it comes with several annotations; (iii) First Certificate in English (FCE) (Yannakoudakis al., 2011) with short texts written by English learners answers exam questions assessing the upper-intermediate level; this dataset is relatively clean but covers only a single group of English learners; (iv) Write & Improve + LOCNESS Corpus (W&I+L) (Bryant et al., 2019a); Write & Improve dataset consists of text blocks (essays, letters etc.) written by En- glish learners and submitted to the W&T system; LOCNESS is composed of essays written by native English students, used only for evaluation; these datasets are the "cleanest" and have a wide distri- bution over different levels of English. Here and below, errorful sentences are those that contain at least one error. We use the BEA-2019 devel-
3.3 Multi-task learning
The standard approach is to view GEC as a scquence-to-sequee
mapping from errorful sen- tences to corrected ones. Since the sentences are from the same language, one can align two sen- tences using an edit distance algorithm. The align- ment can be transformed into a sequence of insert, delete, and replace operations that transform the errorful sentence into correct one. We find the list of operations using ERRANT (Bryant et al., 2019b) and use them for auxiliary tasks.
Inspired by recent works on chain of thought prompting (Wei et al., 2023; Zhou et al., 2022), we construct several tasks and examine their influence the model's performance. Each task, as illus- trated in Fig. 1, is explicitly marked by a prefix written as (prefix) and followed by an input string:
(i) Correct: standard generation of a corrected sentence given the original, encoded as "(correct) source" and trained to produce the target sentence (Fig. la);
"W use version 2.3 of ERRANT for both evaluation and training set construction.

---

pretrain our model on large-scale synthetic data. We consider two datasets that differ both in size and generation approach: C4200MM and PIE. This step adapts the model for the downstream task. On Stage II, we fine-tune on four GEC datasets but modify the above procedure. First, we use all sen- tences, not only errorful ones. Second, we use the datasets in strict order: Lang-8, NUCLE, FCE, W&I+L,, with shuffling across datasets. Third, we do not shuffle samples inside the datasets ei- ther: each dataset is composed of coherent texts, so wc thus preserve their original structure and place examples from the same texts together.
Lang-8
FCE
C4
Optional
NUCLE
W&I+L
PIE
W&I+L
Stage
Stage II
Stage III
Pretraining on synthetic data
Fine-tuning on real data
Fine-tuning clean real data
Figure 2: Training order for GEC model.
For a more complete comparison with previous works, we add Stage III where we additionally fine-tune the model the W&I+L dataset. We that this step helps increase recall without substantial decrease in precision, yielding modest improvements in Fo.5- Note that in previous works, this step was mandatory as the target distribution is correlated with the W&I+L dataset. In contrast, we add this dataset as the last in our training order, which is more natural approach; the suggested scheme also looks more suitable for real world tasks where there is obvious way split the data into different stages.
(ii Explain: generation of a sequence of expla- nations (atomic edits) given both original and corrected sentences; encoded as "(explain) Input: src Target: tgr" with the result in the format "Delete smth Insert smtt..." (Fig. 1b); if no correction needed, the model should generate "No correction",
(iii) Apply: application of edits to the original sen- tence to get the target, encoded as "(apply) Input: src 'n Do: Delete smth 'n Insert smth VI Replace smth with smih"; the result should be the correct target sentence (Fig. Ic);
3.5 Re-weighted Sampling
(iv) Edit: generation of the sequence of edits given the original sentence but not the tar- get, encoded as "(edit) src" with the target the form "Delete smth Insert smth 'vt Replace smth with smthi (Fig. Id); if no cor- rection is needed, the model should generate "No correction".
The Fo.5 metric is commonly used for the evalu- ation of GEC models. It puts a higher weight on the model's precision than recall. In experi- ments, see that the proposed training pipeline makes the model less aggressive in editing which improves performance. Another approach mak- ing the model choose corrections that is confident about is the Align-Pred method proposed by Sun and Wang (2022). It increases the probability of the next original token (obtained by aligning the original and currently generated sequences) during decoding. In order to show that method is thogonal this, we apply Align-Pred Pred to our best model. We introduce modification this method that goes beyond the original, applying tempera- ture scaling before Align-Pred. This significantly improves Align-Pred (see Table 2).
We generate an auxiliary sequence-to-sequene pair for every task and for every pair from the orig- inal dataset.
3.4 Training Order
Tarnavskyi et al. (2022) and Omelianchuk et al. (2021) mention that training order is essential to obtain the best possible results. They separate the overall process into three stages: pretrain the model on synthetic data, fine-tune it on errorful sentences from the four GEC datasets: LANG-8, NUCLE, FCE, and W&I+L, and then fine-tune the clean GEC dataset W&I+L.
Evaluation and Analysis
4.1 Comparison with baselines
In this work, we suggest modify this proce- dure and claim that the training process benefits from choosing a correct ordering of data. Our pro- cess is illustrated in Fig. 2: on Stage I, we similarly
We compare the proposed model with several state of the art basclines. We used three variations of the tagging model GECToR: with the XLNet-L back-

---

CoNLL-14
BEA-test
Model
Synthetic
Prec
Rec
Fa.
Rec
F0.5
T5 (Rothe et al., 2021)
770M mCC4:300M)+CLANG
66.04
72.06
TS-XL (Rothe al., 2021)
3B mC44:300M)CCLANG
67.65*
73.92
TS-XXL (Rothe 2021)
11B MC40300M)HCLANG
68.75
75.88
GECToR-XLL Net (Omelianchuk al., 2020)
350M
PIB(9M)
77.5
40.1
72.4
HECToR-RoBERTa (Tarnavskyi et al., 2022)
350M
PIR(9M)
74.40
41.05
64.0
80.70
53.39
73.21
GECTOR-XLNct (Lai et al., 2022)
350M
PIE(YM)
78.18
42.67
67.02
81.33
51.55
72.91
Transformer (Stahlberg and Kumar, 2021)
340M
C42mma
72.8
49.5
66.6
72.1
64.4
70.4
BART-hased (Sun and Wang, 2022)
400M
300M
76.1
65.6
73.8
BART-based Align-and-Proeee (Sun Wang. 2022)
400M
300M
78.7
63.2
75.0
BART (ours)
400M
C42oM
75.43
51.20
68.91
78.19
65.54
75.28
BART (ours) Align-und- Prend-treeict-- Temp. Scaling
400M
PIE(9M)
75.45
47.03
67.31
78.49
58.66
73.52
BART (ours) Align-and- Hign-and-Preeicct Temp. Scaling
400M
C4
78.00
49.12
69.79
80.88
61.15
75.97
Table Evaluation results on CoNLL-I4 14 and BEA-test. Best results shown in bold, second best underlined; results shown by the arXiv version (Rothe et al., 2022) that differs from (Rothe et al., 2021) on the CONLL test set
Pre- training
Stage II
Stage III
Prec
Rec
0.5
4.2 Inftuence of the Pretraining Dataset
In this section, we analyze the choice of the pre- training dataset, comparing two publicly available synthetic datasets: C4200M and PIE. They differ not only in size (see Table 1) but also in the type of generated errors that are moddel-aased and rule- based respectively. Table 3 shows the performance of models pretrained in different setups. The model pretrained on C420MM has higher recall indicating that it covers more errors, while the model pre- trained on PIE reaches higher precision. Note that almost sentences from synthetic datasets contain errors, which means that the pretraining distribu- tion differs a lot from the development set. Hence, to make fair comparison we further fine-tune the models on GLC--pecifii datasets using the standard three-stage approach. Table 3 shows that the model pretrained on C4200M performs better in terms of precision, recall, and the Fo.5 score.
C4g0nM
36.67
24.83
33.88
C4200M
4 datasets
W&I+L
66.77
50.14
62.62
PIE PIE
44.55
20.04
35,79
4 datasets
W&I+L
65.00
46.29
60.14
Table 3: Influence of the pretraining dataset. Evaluation results on BEA-dev.
bone (Omelianchuk et al., 2020), with RoBERTa backbone and multi-stage training (Tarnavskyi et al., 2022), and with the XLNet-L backbone and multiturn corrections (Lai et al., 2022). For seq2seq baselines we consider the vanilla GEC Transformer trained on C4200M (Stahlberg and Kumar, 2021), T5-XL and T5-XXL models (Rothe et al., 2021) also trained on the C4 dataset, and a BART-based model trained on 300M synthetic data (Sun and Wang, 2022). All baselines have comparable or larger model sizes than ours and use the same PIE synthetic dataset larger synthetic collections.
4.3 Order of the Datasets
We also examine the influence of the ordering of training datasets and examples within a dataset. We are mainly concerned with multi-task training, but note that the training schedule also impacts the single-task pipeline (see Table 4). The model trained using specific schedule achieves good results after the first stage of fine-tuning, while the third stage improves the model's recall and makes its results surpass the standard three-stage training.
Table 2 shows evaluation results for the CoNLL- 14 (Ng et al., 2014) and BEA-test (Bryant al., 2019a) datasets. Our approach shows the best re- sults for comparable model sizes with a significant margin and outperforms all models, including T5- XL with 3B parameters and even T5-XXL which is 30x larger than our model (1 IB parameters) and trains a larger synthetic dataset. We also present evaluation results of our approach with pretrain- ing on the PIE dataset to compare with the models pretrained on it. Again, we see that our model out- performs more sophisticated methods such as (Lai et al., 2022).
For the multi-task approach. we use three tasks- Correct, Explain, and Apply-and different dataset schedules. There are 16 possible orderings, but Lang-8 a "noisy" dataset so is reasonable to place early in the fine-tuning, while W&I+L is

---

CoNLL-114
BEA-test
REA-dev
Training approach
Prec
Rec
Fa.5
Prec
Rec
Fo.5
Prec
Rec
Fo.5
Two stuges
66.47
47.65
61.60
$9.61
63.74
60.39
47.94
40.91
46.35
Three stages
73.64
53.16
68.37
75.65
68.94
74.20
66.77
50.14
62.62
Two stages, optimized schedule
77.18
48.52
69.02
78.71
63.05
74.90
68.60
41.56
60.70
Three stages, uptimized schedule Multi-tusk: three tasks, stages Multi-task: three tasks, ihree stages Multi-tusk: three tasks, optimized schedule
74,96
52.34
69.00
77.17
66.39
74.74
68. 18.15
46.31
62.28
66.51
48.47
61.79
59.55
63.53
60.31
48.22
41.50
46.71
73.96
53.30
75.85
68 63
74.29
52.07
62.38
75.43
51.20
68.91
78. 19
65.54
75.28
68.64
46.50
62.67
Table 4: Comparison of standard and multi-task fine-tuning; all models are pretrained on C4200M. Best results are shown in bold, second best are underlined.
Order
Prec
Rec
Fa.s
Tasks
Prec
Rec
Fa.a
Order preserved within each dataset
Correct + Explain
69.03
44,20
62.06
Correct + Apply
68.64
46.50
62.67
FCE Lung-8 NUCLE W&
68.72
44.98 62.15
Correct Edit
69.19
44.92
62.45
NUCLE Lang-8 + JCE W&1+L
68.86
45.02 62.26
Correct Explain + Apply
68.78
45.98
62.57
Lang-8 + FCE NUCLE - W&I+L
68.52
45.32 62.16
Correct + Apply Edit
68.98
44.95
62,32
Lung-8 + NUCLE - FCE - W&I+L 68.78
45.98 62.67
Correct + Explain + Apply Edit
68.58
45.81
62.38
PCE Lang-8 W&I+L NUCLE 77.41 13.70 40.11
Edit Apply
49.41
28.80
43.22
Instances shuffled within each dataset
FCE + Lang-8 + NUCLE W&I+L 66.41 48.47 61.84
Table 7: Ablation study on auxiliary tasks. Evaluation is done on the BEA-dev dataset.
Lang-8 + NUCLE - FCE - W&I+L 66.99 48.48 62.24
Instances shuffled across datasets
Lung-8, NUCLE, FCE, W&I+L
48.21 41.20 46.63
performance achieved when W&I+L the last dataset the schedule, as evidenced by the drop performance for the "FCE - Lang-8 8 - W&I+L
Table 5: Ablation study of the dataset fine-tuning order. The model is trained in 2-stages, evaluated on BEA-dev.
+ NUCLE" sctup and improvement after further fine-tuning on the W&I+L dataset.
Stage 11
Stage III
Prec
Rec
Fus
Order preserved within each dataset
4.4
Multi- Pretraining
FCE + Lang-8 + NUCLE W&l+
W&1+L.
67.88
47.02 62.34
NUCLE Lang-8 8 -a FCE Walel
W&141.
68.23 47.12 62.62
Table 4 compares standard fine-tuning with the proposed multi-task fine-tuning. We show the com- monly used metrics--pecisse score-on three GEC datasets: CoNLL-14, BEA- test, and BEA-dev. In all cases, multi-task training leads to improved performance for the final task, especially for the BEA-dev dataset.
Lang-8 FCE + NUCLE + W&I+L
W&1+L
68.14
47.61 62.73
Lang-8 NUCLE FCE W&I+L
W&1+L
67.35
recall, and Fo.5
FCE - Lang W&1+L NUCLE
W&I+L
67.66 47.32 62.30
Iustances shuflled within each dataset
FCE -+ Lang-8 NUCLE W&l+L W&1+L 66.05 50.03
62.08
Lang-8 + NUCLE PCE - W&l+L
W&1+L 66.05 50.34 62.17
Instances shuffled across datasets
Lang-8, NUCLE, FCE, W&I+L
W&1+L 65.62 52.07 62.38
Table 7 presents ablation study related to the choice of auxiliary tasks; we fix the training sched- ule and use different task compositions. We make the following observations. First, the best combi- nation of tasks is "Correct" and "Apply", yielding the highest recall and high enough precision com- pared to other settings. Second, adding the edit prediction task ("Edit") in combination with other tasks lowers the model's performance; below, we argue that this could be due to the complexity of the task that the model cannot effectively learn. On the other hand, while the "Edit" task does not im- prove performance, could help to make the model interpretable without using external tools.
Table 6: Ablation study the dataset training order. The model is trained in 3-stages. Evaluation is done the BEA-dev dataset.
"cleaner" than others so we use it late in the train- ing. Tables 5 and 6 shows the performance of models trained in different setups. It shows that, first, the order of datasets really matters: some cases outperform others by significant margin. Second, shuffling instances within a dataset also re- duces performance, which supports our hypothesis that sentences from the same block text should be placed together while training. Third, the best
Additionally, we study the case where we replace

---

the GEC task with consecutive "Edit" and "Apply" tasks, decomposing GEC into a chain-of-thought process: first the model says what wrong and what actions should be taken and then it applies these actions to the original sentence to produce the final output. Table 7 shows that the model trained with only these two tasks severely underperforms. so "classical" GEC is still necessary.
bomb' ' will explode. Here, the correction "delete will" targets the second will, but the model does not make changes to the original sentence, ignoring the correction. In this work, we restricted ourselves to simplistic prompts, and we leave an exploration of more detailed edit deseriptions for further work.
The last auxiliary task, "Edit", is the prediction of a list of edits. This task is hard for the model: the exact match score only 23.5%, and the Fo.5 score 30.69. This low performance might be the reason why adding this task to the training pipeline decreases GEC performance. There are many cases where the model correctly applies an edit using prediction prompt but omits it with the "Edit" prompt, which could indicate that the two tasks do not interact properly. It seems that the model struggles to predict combination of connected or related corrections. For example, for the sentence "The people with albinism have sensitive skin and it needs regular treatment .", annotators gave the following edits: replace The people with People; replace it with this. But the model predicts the following: delete The; replace it with they. Thus, after correctly removing the it does not capitalize people, and while replacing it with they it does not replace needs with need.
4.5 Auxiliary Tasks Analysis
Next, we examine the model's performance on aux- iliary tasks. For the analysis of "Explain" and "Ap- ply" tasks, we use the model that had them both in pretraining: to analyze the "Edit" task, we use a model pretrained on all four tasks. We use the CoNLL-14 dataset for evaluation.
The "Explain" task urges the model to align to sentences and produce a correct sequence of edits to transform the source into the target. The exact match score of generated edits is 90% compared to the gold standard edits, and the Fo.5 score is 91.55. Moreover, many "errors" here come from the am- biguity of how to perform replacements or from merging/splitting corrections compared to the gold standard. For example, for the original sentence "People can also know much information about the celebrity in Twitter and Facebook such as Obama, Bill Gates and get the first-hand study materials on it ." an annotator suggested the following correc- tions: replace know much with find out a great deal of: delete the; replace celebrity with celebrities; replace in with from; replace, with and; delete the; replace it with these. The model correctly predicts all of them except one. It splits the first correction into two: replace know with find out and replace much with great deal.
Since our model can be trained on both "Edit" and "Apply" tasks, it is tempting to apply chain- of ofothhught procedure where we first predict the list of operations and then apply them to the orig- inal sentence. Applying such procedure, we get the Fo.5 score of 52.18 on the CoNLL-14 dataset (precision 55.85, recall 41.33), i.e., the most prob- lematic metric is precision (see also Table 7). The main problem of this task is the ambiguity of the target sentence and edits needed to obtain it: ed- its can interact with each other, which is hard to account for given only the source sentence. This chain-of thought approach appears unnecessarily challenging. while adding the target sentence in the "Explain" task makes the problem much easier and more helpful for pretraining.
The model deals quite well with the "Apply" task, getting 93.5% accuracy. Most errors corre- spond to doing a different operation (c.g., inserting a new word instead of replacing one word with another), applying edits in the wrong place (e.g.. placing new tokens after a wrong word), and sim- ply ignoring some of the edits, especially if the list of edits is long. It appears that some errors arise from ambiguous descriptions of edits our ap- proach we specify edits as, c.g., "Insert the", and the model has to decide where to insert it by itself- so performance could be improved further. For an example, consider the following sentence: It is a concern that will be with us during our whole life because we will never know when the "potential
We have also studied how tasks interact with each other. In the first experiment, we correct a sentence with the model and then ask it to explain the corrections. We compare the explanations with edits produced by the ERRANT tool, obtaining an exact match score of 95.9%, higher by 5.9% than for human-corrected sentences. This indicates some interaction between tasks as the model's cor- rections are more explainable by itself. Second, we

---

chain three tasks: first the model corrects sen- tence, then it explains the corrections, and finally applies edits the original sentence. Interestingly, the exact match between the corrected sentence on the first step and the sentence obtained after chaining is 97.02%, so the model is not always consistent across the tasks. Many errors come from the ambiguity of edits. It may be possible to use the discrepancy between corrected and generated sentences for filtering, either leaving the initial sen- tence intact or taking the intersection of the edits related to the two sentences. We find, however, that these approaches do not improve the final perfor- mance or even decrease it.
System
Fa.s
Human level
72.58
Transformer
66,97
GECToR
80,49
T5-large
81.19
ChatGPT (0-shot)
69.74
ChatGPT (1-shot)
71.55
ChatGPT (3-shot)
71.73
ChatGPT (5-shot)
70.66
Our
81.36
Table 8: Comparison to automatic human evaluation performance on 10 references CoNLL-14; ChatGPT results are taken from Fang al. (2023).
than previous state of the art models with much smaller model size and outperforms models of com- parable size by a wide margin on both CoNLL- 14 and BEA-test datasets; particular, we have achieved results exceeding state of the art models based on T5-XXL (1IB parameters) with BART- based model (400M parameters). Our multi-task approach encodes auxiliary tasks text rewriting problems and does not require any changes in the model architecture. We believe that the proposed approach is of significant value for practical GEC pipelines and see several directions for possible improvements in further work, including modifi- cations of text encodings for auxiliary tasks and adding new auxiliary tasks to the training process.
4.6 Automatic Human Evaluation
In this part, compare the performance of our model with ChatGPT with chain-of-thoughh prompting, using the results by Fang al. (2023) who showed that ChatGPT performs poorly on common benchmarks, far below state of the art fine- tuned models in the oo. metric, with high recall but very low precision. They show that the model often proposes corrections that are reasonable but far from minimal with respect to the number of required edits. To punish such over-corrections less, Bryant and Ng (2015) propose use a test set where each erroneous sentence annotated by several experts and averages the scores obtained in comparison to each annotator; this approach in- creases the scores for edits proposed by at least one annotator. Following Fang et al. (2023), we use the evaluation set from this paper consisting of CoNLL- 14 sentences with 10 human annotations each. We also compare with human-level baseline which is measured as the average score of each human annotator with respect to others. Table shows that even in this setup our model outperforms ChatGPT by a large margin; we compare with Fang et al. (2023) who use tf-hout prompting and re- port results in the zero-shot and few-shot (1, 3, and 5) settings. Interestingly, ChatGPT performance is slightly below the human level, while model performs significantly better.
6 Limitations
Limitations of our study can serve as motivation further work. First, we do not test our approach in the increasingly important multilingual setup; we believe that our ideas may contribute even more to multilingual models that need greater robustness. Second, the highest possible results in GEC are currently obtained by ensembles rather than indi- vidual models, including the recent state of the art approach by Qorib al. (2022); while ensembling would make our reduced model size less important, would interesting to study how well our model can perform in such ensembles. Third, we consider only simple prompts and small models, and our results could be extended further.
5 Conclusion
A Acknowledgements
In this work, we have introduced a new approach training and fine-tuning costnncc-----queene models for the grammatical error correction task based on multi-task pretraining and optimized train- ing schedules. The proposed approach fares better
The work of Sergey Nikolenko was done under support of the grant no. 075-15-2022-289 for the creation and development of Euler International Mathematical Institute.

---

References
Daniel Dahlmeier and Hwee Tou Ng. 2012. Better evaluation for grammatical error correction. In Pro- ceedings of the 2012 Conference of the North Amer- ican Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 568-572 Montréal, Canada. Association for Compu- tational Linguistics.
Abhiject Awasthi, Sunita Sarawagi, Rasna Goyal, Sabyasachi Ghosh, and Vihari Piratla. 2019a. Par- allel iterative edit models for local sequence trans- duction. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natu- ral Language Processing GEMNLP-JCCNLP pages 4260-4270, Hong Kong, China. Association for Com- putational Linguistics.
Daniel Dahlmeier, Hwee Tou Ng. and Siew Mei Wu. 2013. Building a large annotated corpus of learner English:
The NUS corpus of learner English. In Pro- ceedings of the Eighth Workshop on Innovative Use of NLP for Building Educational Applications, pages 22-31, Atlanta, Georgia. Association for Computa- tional Linguistics.
Abhijeet Awasthi, Sunita Sarawagi, Rasna Goyal, Sabyasachi Ghosh, and Vihari Piratla. 2019b. Paral- lel iterative edit models for local sequence transduc- tion. ArXiv, abs/1910.0255)
Chris Brockett, William B. Dolan, and Michael Gamon. 2006. Correcting ESL errors using phrasal SMT techniques. In Proceedings of the 21st International Conference on Computational Linguistics and 44th Annual Meeting of the Association for Computational Linguistics, pages 249-256, Sydney, Australia, Asso- ciation for Computational Linguistics.
Meiyuan Fang, Kai Fu, Jiping Wang. Yang Liu, Jin Huang, and Yitao Duan. 2020. A hybrid system for NLPTEA-2020 CGED shared task. In Proceedings of the 6th Workshop on Natural Language Processing Techniques for Educational Applications. pages 67- 77, Suzhou, China. Association for Computational Linguistics.
Christopher Bryant, Mariano Felice, Øistein E. Ander- sen, and Ted Briscoe. 2019a. The BEA-2019 shared task on grammatical error correction. In Proceedings of the Fourteenth Workshop on Innovative Use of NLP for Building Educational Applications, pages 52-75, Florence, Italy. Association for Computational Lin- guistics.
Tao Fang. Shu Yang. Kaixin Lan, Derek F. Wong, Jin- peng Hu, Lidia S. Chao, and Yue Zhang. 2023. Is chatgpt highly fluent grammatical error correction system? comprehensive evaluation.
Jennifer Foster and Oistein Andersen. 2009. Gen- RRRaae: Generating errors for use in grammatical error detection. In Proceedings of the Fourth Work- shop on Innovative Use of NLP for Building Educa- tional Applications, pages 82-90, Boulder, Colorado. Association for Computational Linguisties.
Christopher Bryant, Mariano Felice, Distein E. Ander- sen, and Ted Briscoe. 2019b. The 2ea-2019 shared task on grammatical error correction. In BEA@ACL
Christopher Bryant, Mariano Felice, and Ted Briscoe. 2017. Automatic annotation and evaluation of error types for grammatical error correction. In Proceed- ings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume I: Long Papers), pages 793-805. Vancouver, Canada. Association for Computational Linguistics.
Roman Grundkiewicz. Marcin Junezys- lunezys-Dowmund and Kenneth Heafield. 2019. Neural grammatical error correction systems with unsupervised pre-training synthetic data. In Proceedings of the Fourteenth Workshop on Innovative Use of NLP for Building Educational Applications, pages 252-263, Florence, Italy. Association for Computational Linguistics.
Christopher Bryant and Hwee Tou Ng. 2015. How far are we from fully automatic high quality grammatical error correction? In Proceedings of the 53rd Annual Meeting of the Association for Computational Lin- guistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 697-707, Beijing, China. Association for Computational Linguistics.
Jiatao Gu, Changhan Wang. and Jake Zhao. 2019. Lev- enshtein transformer. In Neural Information Process- ing Systems.
Phu Mon Htut and Joel Tetreault. 2019. The unbearable weight of generating artificial errors for grammatical error correction. In Proceedings of the Fourteenth Workshop on Innovative Use of NLP for Building Educational Applications, pages 478-483, Florence, Italy. Association for Computational Linguisties.
Yo Joong Choe, Jiyeon Ham, Kyubyong Park, and Ycoil Yoon. 2019, A neural grammatical error correction system built on better pre-training and sequential transfer learning, In Proceedings of the Fourteenth Workshop on Innovative Use of NLP for Building Educationnal Applications, pages 213-227, Florence, Italy, Association for Computational Linguistics.
Masahiro Kaneko, Kengo Hotate, Satoru Katsumata, and Mamoru Komachi. 2019, TMU transformer sys- tem using BERT for re-ranging at BEA 2019 gram- matical eтror correction on restricted track. In Pro- ceedings of the Fourteenth Workshop on Innovative Use of NLP for Building Educational Applications, pages 207-212, Florence, Italy, Association for Com- putational Linguistics.
Steven Coyne Keisuke Sakaguchi, Diana Galvan-Sosa, Michael Zock, and Kentaro Inui. 2023. Analyzing the performance of gpt-3.5 and gpt-4 grammatical error correction.

---

Satoru Katsumata and Mamoru Komachi. 2020. Stronger baselines for grammatical error correction using pretrained encoder-decoder model. In Pro- ceedings of the 1st Conference of the Asia-Pacific Chapter of the Association for Computational Lin- guistics and the 10th International Joint Conference on Natural Language Processing, pages 827-832, Suzhou, China. Association for Computational Lin- guistics.
Kostiantyn Omelianchuk, Vitaliy Atrasevych, Artem Chernodub, and Oleksandr Skurzhanskyi. 2020. GFCToR grammatical enor correction: Tag, not rewrite. In Proceedings of the Fiffeenth Workshop on Innovative Use of NLP for Building Educational Applications, pages 163-170, Seattle, WA, USA + Online. Association for Computational Linguistics.
Kostiantyn Omelianchuk, Vipul Raheja, and Oleksandr Skurzhanskyi. 2021. Text Simplification by Tagging, In Proceedings the 16th Workshop on Innovative Use NLP for Building Educational Applications. pages 11-25, Online, Association for Computational Linguisties.
Shun Kiyono, Jun Suzuki, Masato Mita, Tomoya Mizu- moto, and Kentaro Inui. 2019. An empirical study of incorporating pseudo data into grammatical error cor- rection. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natu- ral Language Processing TEMNI.P-JCCI.P. pages 1236 1242. Hong Kong, China. Association for Com- putational Linguistics.
Muhammad Qorib, Seung- Na, and Hwee Tou Ng. 2022. Frustratingly easy system combination for grammatical error correction. In Proceedings of the 2022 Conference of the North American Chap- ter of the Association for Computational Linguistics: Human Language Technologies, pages 1974,1974, Seattle, United States. Association for Computational Linguistics.
Shaopeng Lai, Qingyu Zhou, Jiali Zeng, Zhongli Li, Chao LI, Yunbo Cao, and Jinsong Su. 2022. Type- driven multi-turn corrections for grammatical error correction. In Findings of the Association for Com- putational Linguistics: ACL 2022, pages 3225-3236. Dublin, Ireland. Association for Computational Lin- guistics.
Colin Raffel, Noam Shazeer, Adam Roberts. Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, Peter J Liu, et al. 2020. Exploring the limits of transfer learning with a unified text-to-text trans- former. J. Mach. Learn. Res., 21(140):1-67.
Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, and Luke Zettlemoyer. 2020. BART: Denoising sequence-tor requence-to-sequeeee sequence pre-training for natural language generation, translation, and com- prehension. In Proceedings of the 58th Annual Meet- ing of the Association for Computationnll Linguistics. pages 1871-7 Online. Association for Computa- tional Linguisties.
Sascha Rothe, Jonathan Mallinson, Eric Malmi, Sebas- tian Krause, and Aliaksei Severyn. 2021. A simple recipe for multilingual grammatical error correction. In Proceedings of the 59th Annual Meeting of the As- sociation for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing ( (Volume 2: Short Papers), pages 702-707. Online. Association for Computational Linguistics.
Jared Lichtarge, Chris Alberti, and Shankar Kumar. 2020. Data weighted training strategies for grammat- ical error correction. Transactions of the Association for Computational Linguistics, 8:634-644 646.
Sascha Rothe, Jonathan Mallinson, Eric Malmi, Sebas- tian Krause, and Aliaksei Severyn. 2022. A simple recipe for multilingual grammatical error correction.
Felix Stahlberg and Shankar Kumar. 2021. Synthetic data generation for grammatical error correction with tagged corruption models. In Proceedings of the 16th Workshop on Innovative Use of NLP for Build- ing Educational Applications, pages 37-47, Online. Association for Computational Linguisties.
Masato Mita, Shun Kiyono, Masahiro Kaneko, Jun Suzuki, and Kentaro Inui. 2020. A self-refinement strategy for noise reduction in grammatical eтror cor- rection. In Findings of the Association for Compu- tational Linguistics: EMNLP 2020, pages 267-280, Online. Association for Computational Linguistics.
Xin Sun and Houfeng Wang. 2022. Adjusting the precision-recall trade-off with aiig--and-predic de- coding for grammatical error correction. In Proceed- ings of the 60th Annual Meeting of the Association for Computational Linguistics ( (Volume 2: Short Pa- pers), pages 686-693 Dublin, Ireland. Association for Computational Linguistics.
Jakub Náplava and Milan Straka. 2019. Grammatical er- ror correction in low-resource scenarios. In Proceed- ings of the 5th Workshop on Noisy User-generated Text (W-NUT 2019), pages 346-356, Hong Kong, China. Association for Computational Linguistics.
Hwee Tou Ng. Siew Mei Wu, Ted Briscoe, Christian Hadiwinoto, Raymond Hendy Susanto, and Christo- pher Bryant. 2014. The CoNLL-2014 shared task on grammatical error correction. In Proceedings of the Fighteenth Conference on Computational Natu- ral Language Learning: Shared Task, pages 1-14, Baltimore, Maryland, Association for Computational Linguistics.
Toshikazu Tajiri, Mamoru Komachi, and Yuji Mat- sumoto. 2012. Tense and aspect error correction for ESL learners using global context. of the 50th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Popers), pages 198-202, Jeju Island, Korea, Association for Computational Linguistics.
Proceedings

---

Maksym Tarnvskky,, Artem Chernouu,, and Kostiantyn Omelianchuk. 2022. Ensembling and knowledge dis- tilling of large sequence taggers for grammatical error correction. In Proceedings of the 60th Anmual Meer- ing of the Association for Computational Linguistics (Volume I: Long Pupers), pages 3842-3852, Dublin, Ireland. Association for Computational Linguisties.
Jason Wei, Xuezhi Wang. Dale Schuurmans, Maarten Bosma, Brian Iehter, Fei Xia, Edd Chi, Quoe Le, and Denny Zhou. 2023. Chain-of-thought prompting elic- its reasoning in large language models.
Konstantin Yakovlev, Alexander Podolskiy, Andrey Bout, Sergey Nikolenko, and Trina Piontkovskaya. 2023. GEC-DePenD: von-autoregressive grammati- cal error correction with decoupled permutation and decoding. In Proceedings of the 6l Annual Meet- ing of the Association for Computational Linguistics (Volume 1: Long Papers). pages 1546-155 1558. Toronto, Canada. Association for Computational Linguistics.
Helen Yannakoudakis, Ted Briscoe, and Ben Medlock. 2011. A new dataset and method for automatically grading ESOL texts. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, pages 180-189, Portland, Oregon, USA. Association for Computational Linguistics.
Zheng Yuan, Felix Stahlberg, Marek Rei, Bill Byrne, and Helen Yannakoudakis. 2019. Neural and FST- based approaches to grammatical error correction. In Proceedings of the Fourteenth Workshop on Innova- live Use of NLP for Building Educational Applica- tions, pages 228-239, Florence, Italy. Association for Computational Linguistics.
Zheng Yuan, Shiva Taslimipoor, Christopher Davis, and Christopher Bryant. 2021. Multi-class grammatical error detection for correction: A tale of two systems. Proceedings of the 2021 Conference on Empiri- cal Methods in Natural Language Processing, pages 8722-8736. Online and Punta Cana, Dominican Re- public. Association for Computational Linguistics.
Yi Zhang, Tao Ge, Furu Wei, Ming Zhou, and Xu Sun. 2019. Sequence-to-see to-sqqueee pre-training with data augmentation for sentence rewriting. arXiv preprint arXiv:1909.06002.
Yue Zhang, Bo Zhang, Zhenghua Li, Zuyi Bao, Chen Li. and Min Zhang. 2022. SynGEC:
enhanced grammatical error correction with tailored GEC- oriented parser. Proceedings of the 2022 Con- ference on Empirical Methods in Natural Language Processing, pages 5518-2531, Abu Dhabi, United Arab Emirates. Association for Computational Lin- guistics.
Denny Zhou, Nathanael Schärli, Hou, Jason Wei, Nathan Scales, Xuezhi Wang, Dale Schuurmans, Olivier Bousquet, Quoe Le. and Fd Chi. 2022. Least-to-most prompting enables complex reason- ing in large language models. arXiv preprint arXiv:2205.106: 10625.

---

A Error Type Distribution in Datasets
62.62 versus 62.67. However, the models' behav- ior on W&I+L-dev differs. The 3-stage model has higher recall and the other has higher precision. As we have noted in the previous section, error dis- tributions on parts of W&I+L datasets differ from cach other since different people are prone to mak- ing different types of errors. Therefore, we expect that model with higher precision that makes more accurate corrections would generalize better. Look- ing the models' performance on W&I+L-test and CONLIL-14, we see that the gap between the models is indeed large.
order to analyze the difference between the GEC datasets, we compute the error type statistics using the ERRANT tool. Next, we compare each dataset with W&I+L-dev, see Figures 4-9. We see that W&I+L-train and W&I+L-dev have only slight discrepancy in the proportions of each error type, with major differences coming from punctuation and spelling errors. Hence, we should expect that the distribution on W&I+L-test is close but not exactly the same as the articles that comprise the datasets were written by different people. Thus, extensive hyperparameter search on W&I+L-dev may lead to lower performance. The FCE dataset is also close to these datasets except for SPELL, NOUN, and VERB errors. These differences can be explained by the fact that essays were written by authors who had only begun learning English, and these error types are more common for them. Comparing W&I+L-dev with NUCLE and Lang- 8 datasets, we note that for some error types the difference is striking. We highlight that the pro- portion of OTHER errors is high for both datasets. In the ERRANT toolkit, OTHER corresponds to errors that cannot be labeled by predefined of error types. After examining some instances with that error type, we have noticed that a large part of them is related to sentence rephrasing or rewriting, perhaps regrouping some parts of the sentence. Training on such examples might hurt the model's performance because it would becomes aggressive in performing corrections, leading much lower precision. Therefore, mark those datasets as "noisy" and others as "clean".
To draw the distinction further, we compute the statistics of error types corrected and introduced by them with the ERRANT toolkit. Figure 10 presents the absolute number of corrected, generated, and non-corrected errors for every type. Again, we that the 3-stage model is more aggressive. It cor- rects more errors of each type but also introduces more new errors. Figures 1 11-13, we show the dis- tribution of corrected, induced, and non-corrected errors by type. Again, we see that the models' behavior differs.
The combination of all datasets-FCE W&I+L- train, Lang-8, 8, and NUCLE does not improve the situation, as shown in Figure 9. Nevertheless, if we train only on FCE and W&I+L-train, we obtain poor performance. This indicates that not only the distribution of errors should be close but also the errors should be diverse in order generalize well. These two factors reveal why we needed multi-stage fine-tuning.
Our approach can be viewed as curriculum learn- ing where we gradually train a model on "less noisy" data.
B Model comparison
Performance on the W&I+L.-evv dataset of the stage model and of the multi-tast model with the improved training schedule seems to be marginal:

---

0.2
M_Jooness_ddd WL, Jocaess_train
0.15
0.1
0.05
VERSa VERBIFORM FORM
sezp
PuIcct
VERS
HOUN
Air
DES
OTveRt
PaRT
NOUN
VERta tEhoe
ORTH
MORDH
PROM
cona
NOUN NOUN
SPELL VENU SVA
CONTR
HOUR INP.
ROLYORR FORM
VERDI INPL
Figure 3: W&I+L-dev and W&I+L-train error type comparison.
0.2
nucle
0.15
0.05
VERB FORB
PREP
PUNCT
Noun
Der
OTvia
PART
NOuN NUM
VERt TENSE
DRTH
MORPH
FROM
col
venk SVA NouN pats
SPELL
AD3
CONTR
Nouk INF.
ADD ADDTTOR
VERU DNPL
Figure 4: W&I+L-dev and NUCLE error type comparison.
0.25
vi__ooness__ee dev
LANG-8
0.2
0.15
0.05
VERB VERB: FORM
PXU
PUNCT
VERR
NOUN NUM
No
11300
PROM
cONO
NOUV POSS
Vesga SWs
SpD
AD]
CONTR
NOUN: rNn.
ADI
wo
VERB: INFL
Figure 5: W&I+L-dev and LANG-8 emror type comparison.

---

wi Jocness_dee dev
FCE
0.15
0.1
0.05
VERB: FORM
PREP
PUNCT
VERS
NOUN
ADV
DET
OTHER
PART
NOLK:NUM
VERB: TENSE
ORTH
MD RPH
PRON
CONI
NOUN:POSS
VERS: SWS
CONTR
NOLN:
ADI:FORM
wo
VERB: INFL
Figure 6: W&I+L-dev and FCE error type comparison.
wi, Jloooooesss.c dev
contl 2014 test
0.15
0.05
VERSI RERBUFOR
PREP
PuNCi
Visee
houR
AD4
DET
orvith
PART
touth nuse
Vena TENGE
CaTTa
MORDH
PRON
ConS
Nous pois
veRu SuA
SPELL
AD3
CONTR
Nouh IWF..
AD FORM
VERE rer
Figure 7: W&I+L-dev and CoNLL-14 error type comparison.
nude comll_204 2_011_testtt
0.7
0.05
VeRB VERBUFORM
VER0-2VA
PREP
vena
Navn
VeRts tease
orrish
DET
RUNCT
PROM
CON1
SPELL
ORTH
ADV
Ab1
MCRmH
wo
NOUNsPORS
haat
houlktww.
CONTR
VERe ter
Figure 8: NUCLE and CoNLL-14 14 error type comparison.

---

WoJoonees dev bea_train_ful
0.15
0.05
VERB FORM
PREp
PUNCT
Vesee
KoW
Der
OTVER
NOUN
Yeas, TeNSE
ORTH
MORMH
RRON
comu
NOUN Patt
vens QUA
SPELL
AD3
contr
NOUN
AD3 FORM
VERO INL
Figure 9: W&I+L-dev and BEA-train enror type comparison.
Corrected 3stage
800
multi_ept
Generatad 3stage
Generated multi
Left 3stage
600
Left mull_ppp
500
300
200
100
CON
CONTR
DET
MORPH
Kaup
KOUN:INN
NOUN.NUD
PART
PREP
PRON
PUNCT
SpaLL
UNK
VERB. FORB
VERS: IN
VERB: SVA
VERB: TENSE
wo
200
Figure 10: 3-stage and multi-task improved schedule models comparison. Corrected - errors corrected by the model. Generated emors introduced by the model. Left emrors that were not corrected.

---

Corrected 3stage Corrected multi_opt
0.2
3.05
(ON)
DET
NoReH
Houn
HOUnA
PART
SPEEL
whs Ta
asvaLL'ae
ON
Figure 11: 3-stage and multi-task improved schedule models corrected errors distribution by type.
Generated 3stage
Generated multi_oot
0.1
0.05
L09
youn
voun
MOUnH
O
PART
PRE:
PRON
PUNCT
Saae TEVSE
oN
Figure 12: 3-stage and multi-task improved schedule models introduced errors distribution by type.
0.16
Left 3stage
multi_opt
0.14
0.12
0.1
0.04
0.02
400
AaV
CON)
ORrH
BEHLO
PUNCT
11365
ON
Figure 13: 3-stage and multi-task task improved schedule models non-corrected errors distribution by type.


Reviews:
[{"review_id": "Fza6weHB2u", "reviewer": "Reviewer_Yt3z", "summary": "", "questions": "1. Optimizing the training schedule appears to have close ties with curriculum learning. It might be valuable to draw comparisons with existing curriculum learning methods. Could the authors elaborate on why such comparisons were not made?\n2. In Table 6, what is the result of the correct-only setting? If the score is indeed 62.62 (as listed in Table 4), then the improvement of MTL amounts to only 0.05, which seems to be a rather insignificant gain.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your time and effort in this thorough review!\n\n1.  By \"noisy\", we mean datasets that were automatically collected from language learning sites and other web resources and were not carefully annotated by experts. As such, they may contain inaccuracies, overly aggressive rewrites, and so on. By \"clean\", we mean datasets that were annotated by experts, so we can expect the corrections to be more accurate and controlled. See also the discussion on dataset quality in (Lichtarge et al. 2020). For English, the Lang-8 dataset might be considered as \"noisy\" as it was automatically collected and qualitative analysis indeed reveals that it has many flaws. In contrast, W&I+LOCNESS was annotated by experts and is considered to be \"clean\". For other languages, we also have automatically collected datasets, e.g., WikiEdits or LANG-8 for some of the languages. According to the results of this paper, we propose to first fine-tune on \"noisy\" data and then fine-tune the model on GEC-specific datasets for the target language. In the present paper, we limited ourselves to GEC for the English language, thus, we didn't conduct such experiments.\nWe also observe that for LANG-8 and NUCLE there is a high proportion of the edits labeled as OTHER in the ERRANT notation. The OTHER label means that ERRANT could not classify an error into any predefined class. Many such errors are related to rephrasing parts of the sentence or adding an auxiliary structure. Paraphrasing persuades the model to be more aggressive in its corrections that negatively influence the target metric. As these corrections were applied in an uncontrolled manner, in contrast to expert annotation, we may consider them as \"noise\". We tried different approaches for dataset cleaning or making the error distribution on LANG-8 to be similar to the BEA-dev (by removing some sentences); however, we have only seen a slight degradation in the performance. This indicates that every example is valuable even if it is \"noisy\". \nWe are not allowed to attach a picture of the error distribution to illustrate it, but add it to the final version if accepted.\n\n2.\nIndeed, our approach might be considered as a variation of curriculum learning [Curriculum learning] (Bengio et al. ICML 2009) . We have experimented with a standard curriculum learning approach where we first give the model \"easy\" examples and then increase the complexity during training. We used the number of errors in a sentence as a measure of complexity. Also, we augmented the data with sentences from existing data with some of the errors already fixed; e.g., if the sentence contains 3 errors we produce sentences with one or two errors where the rest were corrected using annotations. We observed only a slight improvement over the standard 3-stage fine-tuning approach, i.e. described in [GECToR – Grammatical Error Correction: Tag, Not Rewrite] (Omelianchuk et al., BEA 2020) . Therefore and due to space constraints, we did not include those experiments in the paper. We can add these experiments and draw a connection to the curriculum learning in the appendix. \n\n3. \nThe small gap in Bea-dev results (62.67 vs 62.62) appears to arise from the difference in error distributions across train, dev, and test set; this is our hypothesis for BEA-test as far as this set is unavailable for analysis (we cannot attach the picture, but we add it to the final version of paper). Using the suggested combination of optimized schedule and multi-task learning we find better precision-recall tradeoffs for more difficult test sets. In this case, the precision with MTL and an optimized schedule is much better than with a three-stage approach. We did not explicitly mention it in the paper but we are going to add it to the final version if accepted.\n\n4. We agree that Table 5 looks too complex and we will use your suggestion in the final version if accepted.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The paper is well written, with a clear and compelling motivation that addresses an important challenge in GEC. The auxiliary tasks are interesting and may inspire further exploration. Moreover, the empirical recommendation regarding training order is a valuable contribution, offering practical guidance that could specifically benefit researchers and practitioners within the English GEC community.\n\nWhat I appreciate most about this paper is its pioneering experimental exploration of the effects of training order on GEC performance, a subject that has not been previously investigated, to the best of my knowledge. As a researcher in the GEC field myself, during my research, I have observed considerable variance in results across different random seeds (with gaps as large as 1.9 F0.5 on the CoNLL-2014 test set). Since the parameters initialized by BART remain consistent across different random seeds, this variation in performance must be attributed to the instance order during training. Consequently, I find the exploration of training order effects in this paper to be particularly exciting and valuable.", "reasons_to_reject": "The primary weakness of this paper, in my opinion, lies in the lack of an in-depth analysis of the training order and a definitive method to determine the training order. Although the authors mention that a \"noisy\" dataset should be placed earlier in the training sequence over a \"clean\" one, they fail to provide a concrete methodology to classify datasets into these categories. This omission raises questions regarding the placement of new datasets within the training sequence, and how to determine the training order for languages. Consequently, this limitation substantially restricts the practical applicability and overall impact of the paper.", "questions_to_authors": "1. Optimizing the training schedule appears to have close ties with curriculum learning. It might be valuable to draw comparisons with existing curriculum learning methods. Could the authors elaborate on why such comparisons were not made?\n2. In Table 6, what is the result of the correct-only setting? If the score is indeed 62.62 (as listed in Table 4), then the improvement of MTL amounts to only 0.05, which seems to be a rather insignificant gain.", "paper_topic_and_main_contributions": "This paper focuses on how to efficiently use limited data to improve the performance of grammatical error correction (GEC). The main contributions are: 1) proposing the addition of several alignment-related auxiliary tasks to enhance GEC performance; 2) exploring the effects of training order, and providing the community with an empirically recommended training sequence (Lang8 $\\rightarrow$ NUCLE $\\rightarrow$ FCE $\\rightarrow$ W\\&I+L)", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {"Typos_Grammar_Style_and_Presentation_Improvements": "Table 5 in the paper is somewhat challenging to interpret. To enhance clarity, it would be beneficial to separate the results with and without Stage III, especially since the table's primary focus is to compare the variations in training orders during Stage II. This separation could facilitate a more straightforward comparison and lead to a more reader-friendly presentation."}}, {"review_id": "OJLveJOQNp", "reviewer": "Reviewer_bMUs", "summary": "", "questions": "It would be interesting to see an ablation on the choice of the seq-to-seq model. How much impact does the choice of BART vs. T5 have?", "limitations": "", "rating": 4, "confidence": 5, "soundness": 5, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your time and effort in this thorough review!\n\n1. We have begun experimenting with the T5 model, but, unfortunately, did not complete the experiments before the paper submission deadline. In preliminary experiments, we did not see a significant difference depending on backbone type, but it needs more careful evaluation.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "  * Strong SotA results with a dramatically smaller model.\n  * Simple and easy to reproduce method.\n  * Extensive and well thought out ablation study.", "reasons_to_reject": "None.", "questions_to_authors": "It would be interesting to see an ablation on the choice of the seq-to-seq model. How much impact does the choice of BART vs. T5 have?", "paper_topic_and_main_contributions": "This paper reports SotA in two important GEC tasks with much fewer parameters than the current SotA (400M vs. 11B) and with simple modifications to the training strategy:\n  * reordering training stages,\n  * adding appropriate auxiliary tasks,\n  * including a method known as Align-Pred to increase model precision at inference time.\nThis is a surprising and useful result!\n\nThe authors make a number of interesting discoveries:\n  * it’s better to leave in out-of-distribution examples in training sets,\n  * it’s better to place sentences from the same essay in the same batch,\n  * it’s better to train on different dataset sequentially with no shuffling between them,\n  * it's better to use temperature scaling in Align-Pred.\nThese are all non-trivial findings that could help future research in GEC.\n\nFinally the paper contains careful ablation studies, which will also be useful.", "excitement": 4, "reproducibility": 5, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "OslsYnPzyu", "reviewer": "Reviewer_ZskQ", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 5, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your time and effort in this thorough review!\n\n1. We plan to add qualitative analysis and more comparisons of the models to the appendix. From the quantitative analysis, we see that the model trained with our approach makes more accurate corrections.\n\n2. We have also experimented with the Edit + Apply + Explain multi-task fine-tuning, and its performance is comparable to Edit + Apply. Thus, the Correct task is crucial to achieve a good performance. We added only the \"Edit+Apply\" task to Table 6 to show this and that the \"Edit+Apply\" chain has low performance, mainly due to the \"Edit\" step. We will add \"Edit + Apply + Explain\" to Table 6 to make the full coverage. Also, it is an interesting question why the \"Edit\" operation is so aggressive. The reason could be that the model should not only correct errors but also generate full text by copying the needed parts in the \"Correction\" task. Alternatively, the model size could be the reason, and it would be interesting to experiment with other T5 models. \n\n3. Due to the space constraints and the length of meaningful examples, we were not able to add examples to the paper, but we plan to add them to the appendix. The most common errors occurring during the chaining \"Correct + Explain + Apply\" are as follows: \n-- the model does not apply an edit (for example, if it should delete the word \"will\" and the sentence has multiple instances of \"will\"), \n-- does not properly replace (instead only inserts the word), \n-- deletes multiple words instead of deleting just one.\nIt seems that most errors come from the model's confusion as to which word to apply the change.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "* Proposes a novel multi-task strategy consisting of 4 auxiliary tasks to improve grammatical error correction. Some of these tasks could be valuable to other tasks besides GEC e.g. Question Answering or Machine translation\n* Using the proposed approaches, the paper reports impressive gains relative to prior models which are substantially larger e.g. 11B T5.\n", "reasons_to_reject": "* The paper does not provide adequate error analysis which makes it harder to appreciate some of the gains", "questions_to_authors": "", "paper_topic_and_main_contributions": "Summary:\nThis paper proposes two techniques to improve the quality of grammatical error correction. The first idea - which is fairly novel - is to use multiple auxiliary tasks such as Correct, Explain, Apply and Edit, each of which is modeled as a sequence-to-sequence task. The second idea, which has been investigated in prior work to some extent, is to optimize the ordering of datasets in fine-tuning and fix the ordering of instances within a given dataset. Using these two ideas with a pre-trained BART model (with 400M parameters), the paper reports an improvement of 3% in F0.5 on top of similar sized models in the literature. Furthermore, the approach outperforms results from substantially larger models (up to 11B parameters) reported in the literature.\n\nStrengths:\n* Proposed a novel multi-task strategy consisting of auxiliary tasks which can help improve the performance of a GEC model\n* Reports improvements from a systematic optimization of ordering of datasets in fine-tuning.  Though prior work (e.g. Lichtarge et al. 2020) has investigated ordering of datasets, this paper reports a thorough investigation.\n* The combination of approaches improves upon models which are substantially larger\n\nWeaknesses:\n* It would be good to see some error analysis (wins/losses) to understand why some combination of auxiliary tasks don't help e.g. why does Edit+Apply underperform the other combinations in Table 6?\n* Some additional experiments are missing e.g. Does Edit+Apply+Explain improve upon Edit+Apply?\n* The paper states that a) model's corrections are more explainable by itself, b) exact match between the corrected sentence after correct and correct+explain+edits (chaining) is 97%. It'd be good to see some examples for these cases, which would give the readers some insights.\n", "excitement": 4, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Missing_References": "* The optimization of ordering of fine-tuning datasets has been explored in prior work e.g. Lichtarge et al. (2020) to a limited extent.\n"}, "extra_text": {}}, {"review_id": "rQjcQjPqRp", "reviewer": "Reviewer_7gp4", "summary": "", "questions": "- The authors have used lang8 dataset for their experiment. However, there is a cleaner version of lang8 https://github.com/google-research-datasets/clang8 . Why do the authors use lang8 instead of clang8?\n- In Table 4, which dataset orderings did you use for \"Two stages\" and \"Three stages\"?", "limitations": "", "rating": 2, "confidence": 4, "soundness": 2, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your time and effort in this thorough review!\n\nAs for the typos, presentation improvement, and missing references, we fully agree, thank you for your detailed suggestions, and will modify the paper accordingly.\n\n1. We politely disagree with the novelty point, since our work does not simply accumulate previous works to obtain the final improved results. Pretraining followed by finetuning is indeed a common practice. As we described in the paper, the standard approach is to split the fine-tuning process into two stages. In the first stage, the model is fine-tuned on sentences from all available data that contain errors, and the other sentences are discarded. In the second stage of the finetuning, the model is trained on FCE and WI+L datasets as they are considered \"clean\". The purpose of such finetuning is to initially train the model to correct sentences (improving its recall) and then to adapt the model to the real error distribution. However, to the best of our knowledge, it has not been studied why this order is beneficial for training GEC models and whether other efficient fine-tuning methods exist. In the present paper, we examine how the order of datasets could influence training and how multi-task finetuning adds to it. We show that the optimized schedule boosts performance both on the CoNLL-14 and BEA-test compared to the standard approach, that a single stage is enough (thus simplifying finetuning), and that examples related to the same document should be placed together during training as it substantially boosts the performance (shown in Table 5).\n\n2. First, the \"multi-task with optimized schedule\" results should be compared with \"3 stages, optimized schedule\" results as they use a similar fine-tuning setup. The \"3 stages\" model should rather be compared with the \"multi-task 3 stages\" where we see a drop in the performance on BEA-dev but an improvement on CoNLL-14 and BEA-test. The \"3 stages with optimized schedule\" is also worse on BEA-dev but substantially better on the other two sets. Thus, multi-task fine-tuning along with the optimized schedule builds a synergy producing the SOTA model. \nThe small gap in Bea-dev results (62.67 vs 62.62) appears to arise from the difference in error distributions across train, dev, and test set; this is our hypothesis for BEA-test as far as this set is unavailable for analysis (we cannot attach the picture, but we add it to the final version of paper). Using the suggested combination of optimized schedule and multi-task learning we find better precision-recall tradeoffs for more difficult test sets. In this case, the precision with MTL and an optimized schedule is much better than with a three-stage approach. We did not explicitly mention it in the paper but we are going to add it to the final version if accepted.\n\n3. The controlled comparison of models presented in Table 1 is not possible as the mT5-based models and model from (Sun and Wang, 2022) used pre-training data that is not publicly available. As shown in Table 3, a larger pretraining dataset leads to a huge improvement. Thus, we took for our study the C4_200M dataset which is large and publicly available. The three-stage model shows performance comparable to the model from (Sun and Wang, 2022), thus we assume that the two pre-training datasets give a similar boost in performance. Even though we did not make a thorough hyperparameter search for the model pretrained on PIE dataset (we concentrated on the model pretrained on the larger dataset), it still achieves 67.31 F_0.5 score on the CoNLL-14 dataset slightly outperforming the more sophisticated method by (Lai et al., 2022) and substantially outperforms the original GECToR model and T5 (770M). For the Bea-test we obtain 73.52, that also slightly better the previous reported results with models pre-trained on PIE dataset. We can add these results to Table 2 if accepted.\n\n4. The values from (Rothe et al. 2021) were taken from the arXiv version of the paper, and we were not aware that they differ from the values given in the conference paper. We will fix the metrics in Table 1. Nevertheless, only metrics for CoNLL-14 were slightly changed and they do not influence respective positions of the models and our paper's conclusions. Moreover, the authors added these changes to version 2 of the paper, which appeared in 2022 after ACL 2021, so we report the latest metrics, and we believe that this should be explicitly remarked upon in the paper. Thanks a lot for this important point!\n\nQuestions to the authors.\n\n1. The CLANG-8 dataset is obtained through the generation of the target sentences by the gT5-XXL model (Rothe et al., 2021); thus, fine-tuning on that dataset should be considered as distillation. We have fine-tuned our model on CLANG-8, but have not observed any improvements compared to LANG-8. The only work that used CLANG-8 for the experiments was (Rothe et al., 2021) itself, so we just followed a more common setup to make the comparison more honest.\n\n2. In Table 4, we use the best ordering of the datasets in the second stage, i.e. Lang-8, NUCLE, FCE, W&I+L, and the third stage is just more fine-tuning on the WI+L.\n\nTypos and grammar improvements: \nThe W&I+L-dev and -test datasets referenced in Table 1 are used in the BEA-dev and BEA-test evaluations. We will point it out explicitly in the paper or remove the inconsistency by other means. We clarify in details the unit of batch size (512 tokens with padding).", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- The proposed method outperforms the current state-of-the-art performance", "reasons_to_reject": "#### Lack of novelty?\n\nMy biggest concern with this paper is the lack of novelty, especially on the dataset ordering for training.\nAs the authors discuss in L168-174, pretraining followed by the finetuning is a common practice employed by many previous studies.\nThe authors might claim the novelty on the extensive experiment reported in Table 5. \nHowever, I am not convinced with the effectiveness of the \"optimized\" dataset ordering.\nThis is because, in Table 5, \"instances shuffled across datasets\" seems to achieve sufficiently high performance of F0.5=62.38. Compared with this value, the improvement by the optimized score (62.94) is marginal.\n\nTo improve the manuscript, I strongly recommend the authors that they re-write the related work section and discuss how their work can be contextualized with regard to the existing works.\nCurrently, the related work section is simply a list of previous works, with no meaningful discussion.\n\n#### Multi-task training seems to be ineffective.\n\nAnother contribution of this work is the incorporation of multi-task training.\nHowever, again, I am not convinced with the effectiveness of this method.\nIn Table 4, the performance of \"Three stages\" (w/o multi-task training) and \"Multi-task: three tasks, optimized schedule\" are 62.62 and 62.67 on BEA-dev, respectively.\nThis demonstrates that the multi-task training can only achieve comparable performance (or tiny improvement).\nI believe that multi-task training is not worthy for an extra computational cost added by the multi-task training.\n\n#### Inappropriate Comparison on Table 2\n\nIf I understand the paper correctly, Table 2 compares the values reported by the previous studies with the values obtained by the authors.\nSuch comparison is inappropriate, because there is no guarantee that the experiment is carried out in a controlled manner (i.e., implementation, hyperparameter and dataset).\nIn fact, their training data is different.\n\nIn addition, the values of [Rothe et al 2021] is wrongly reported in Table 2.\nThe CoNLL-14 score of T5-XXL is 68.87, however, it is reported as 68.75.\nSimilar mistakes can be found for T5 and T5-XL.", "questions_to_authors": "- The authors have used lang8 dataset for their experiment. However, there is a cleaner version of lang8 https://github.com/google-research-datasets/clang8 . Why do the authors use lang8 instead of clang8?\n- In Table 4, which dataset orderings did you use for \"Two stages\" and \"Three stages\"?", "paper_topic_and_main_contributions": "This paper tackles the task of grammatical error correction.\nMore specifically, the paper investigates the following two concepts:\n\n1. means of making a full use of given training dataset, especially on the dataset order\n2. incorporation of auxiliary tasks with multi-task training.\n\nThe contribution of this paper is as follows:\n\n- The proposed method outperforms the current state-of-the-art performance\n- The authors conduct thorough experiments on the aforementioned concepts", "excitement": 2, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {"Missing_References": "- Yo Joong Choe, Jiyeon Ham, Kyubyong Park, and Yeoil Yoon. 2019. A Neural Grammatical Error Correction System Built On Better Pre-training and Sequential Transfer Learning. In Proceedings of the Fourteenth Workshop on Innovative Use of NLP for Building Educational Applications, pages 213–227, Florence, Italy. Association for Computational Linguistics.\n- Shun Kiyono, Jun Suzuki, Masato Mita, Tomoya Mizumoto, and Kentaro Inui. 2019. An Empirical Study of Incorporating Pseudo Data into Grammatical Error Correction. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 1236–1242, Hong Kong, China. Association for Computational Linguistics.", "Typos_Grammar_Style_and_Presentation_Improvements": "- L53-55: \"Most recent works approach these challenges by providing new complex architectures, using larger models, or ensembling multiple models\" This sentence should be followed by citation(s).\n- L78: \"This information is exploited by sequence tagging models that show good results on the GEC task.\" This sentence should be followed by citation(s).\n- L227: \"GeForce V100 GPU\" I think that such GPU card does not exist. The authors might be confused with Tesla V100?\n- L221: \"We use batch size 12\": What is the unit of batch size (e.g., tokens, sentences, or paragraphs)?\n- L257-L266: The notations used for each dataset do not correspond to those in Table 1. For example, BEA-2019 development set does not exist in Table 1. This makes it really difficult to understand the experimental results.\n- L376: \"We used three variations the of 376 tagging model GECToR...\" This sentence contains grammatical error.\n- Please make \"CoNLL\" and \"CONLL\" consistent."}}]

Author rebuttal:

Meta-review: The paper provides a clear and compelling motivation to address a key challenge in GEC. 
It presents interesting auxiliary tasks that could inspire further exploration. 
The authors acknowledged that the proposed method does not have much technical novelty.
However, the authors also indicate that the novelty of this paper comes from dataset ordering for training.
The empirical recommendation regarding training order is a valuable contribution, offering practical guidance to researchers and practitioners in the English GEC community on a topic that has not been previously investigated. 
The paper also achieves strong results with a smaller model and includes a simple and reproducible method. 
The experimental results show that the proposed method outperforms the current state-of-the-art performance.
The potential weakness of this paper is the lack of a thorough analysis and definitive methodology for determining the training order. 
For example, the authors mention noisy datasets should be placed earlier in the training sequence than clean ones. Still, no concrete way to classify datasets as loud or clean is out of focus in this paper. 

Overall, there seem to be some contributions to the community.
Decision: Accept-Main