Grammatical Error Correction via Mixed-Grained Weighted Training
Authors: Jiahao Li
Quan Wang
Chiwei Zhu
Zhendong Mao
Yongdong Zhang
Venue: EMNLP 2023 Findings (2023)
Keywords: Grammatical Error Correction
Weighted Training

Abstract: The task of Grammatical Error Correction (GEC) aims to automatically correct grammatical errors in natural texts.
Almost all previous works treat annotated training data equally, but inherent discrepancies in data are neglected. 
In this paper, the inherent discrepancies are manifested in two aspects, namely, accuracy of data annotation and diversity of potential annotations. 
To this end, we propose MainGEC, which designs token-level and sentence-level training weights based on inherent discrepancies therein, and then conducts mixed-grained weighted training to improve the training effect for GEC.
Empirical evaluation shows that whether in the Seq2Seq or Seq2Edit manner, MainGEC achieves consistent and significant performance improvements on two benchmark datasets, demonstrating the effectiveness and superiority of the mixed-grained weighted training. 
Further ablation experiments verify the effectiveness of designed weights for both granularities in MainGEC.

Grammatical Error Correction via Mixed-Grained Weighted Training
Jiahao Li Quan Wang?, Chiwei Zhu', Zhendong Mao Yongdong Zhang!
UUnivessity of Science and Technology of China, Hefei, China MOE Key Laboratory of Trustworthy Distributed Computing and Service, Beijing University of Posts and Telecommunications, Beijing, China jiahao66@mail. wut...ssstt..m.......o edu. cn, wangquan@bupt. edu. cn tanz@mail. ustc. www.uusttccuuuu..uo.o cn, zdmao@ustc. edw. cn zhyd73@ustc. edu. cn
Abstract
Accuracy Data Annotation
Sample 1:
Their new house near the heach is very nice. Annotation: Your (X) new house near the beach is very nice
The task of Grammatical Error Correction (GEC) aims automatically correct grammati- cal errors in natural texts. Almost all previous works treat annotated training data equally, but inherent discrepancies in data are neglected. In this paper, the inherent discrepancies are manifested in two aspects, namely, accuracy of data annotation and diversity of potential an- notations. To this end, we propose MainGEC, which designs token-level and sentence-level training weights based on inherent discrepan- cies in accuracy and potential diversity of data annotation, respectively, and then conducts mixed- grained weighted training to improve the training effect for GEC. Empirical evalu- ation shows that whether in the Seq2Seq or Seq2Edit manner, MainGECC achieves consis- tent and significant performance improvements on two benchmark datasets, demonstrating the effectiveness and superiority of the mixed- grained weighted training. Further ablation ex- periments verify the effectiveness of designed weights of both granularities in MainGEC.
Sample 2:
read your email yesterday but I had n't had the time to reply yet.
Annotation: read your email yesterday but I have (V) n't had the time to reply until now (X).
Sample 3: do you have hest friend in your life
Annotation: Do (V) you have best friend (V) in your life ?
Diversity of Potential Annotations
Sample 4:
Natural environment destroyed that is a people focus on frequently problem.
Annotation:
The natural environment is being destroyed That is a problem people focus on frequently
Alternative:
The natural environment is being destroyed which is problem people focus on frequently
Sample 5:
Secondly there are not much variety of dessert mainly fruits and puddings.
Annotation: Secondly there is not lot of variety the desserts. mainly fruits and puddings.
Alternative: Secondly. there are not many varieties of desserts mainly fruits and puddings.
Sample
One of my favourite books are Diary of Wimpy Kid..
Annotation: One of my favourite books is Diary a Wimpy Kid.
1 Introduction
Table 1: Instances from the BEA-19 19 (Bryant et al.. 2019) training set to show the discrepancies the annotated training data. Erroneous annotations are in red, correct annotations are in blue, and multiple potential annota- tions are in green.
The task of Grammatical Error Correction (GEC) aims to automatically correct grammatical errors in natural texts, which is extremely beneficial for language learners, such as children and non-native speakers (Bryant et al., 2022). The currently domi- nant neural GEC methods are categorized into two groups, i.e., Seq2Seq methods and Seq2Edit meth- ods. Seq2Seq methods treat GEC as monolingual translation task, regarding errorful sentences as the source language and error-free sentences as the tar- get language (Yuan and Briscoe, 2016; Sun et al., 2021; Zhang et al., 2022b). Seq2Edit methods treat GEC as a sequence tagging task, which predicts tagging sequence of edit operations to perform cor- rection (Awasthi al., 2019; Omelianchuk al., 2020; Tarnavskyi et al., 2022).
Whether in the Seq2Seq or Seq2Edit manner, almost all previous works treat annotated training data equally (Rothe et al., 2021: Tarnavskyi et al., 2022), that is, assigning the same training weight to each training sample and each token therein. How- ever, inherent discrepancies in data are completely neglected, causing degradation of the training ef- fect. Specifically, inherent discrepancies may be manifested in two aspects, i.e., accuracy of data annotation and diversity of potential annotations.
The discrepancy in accuracy of data annotation refers to the uneven annotation quality, which is
"Corresponding author: Quan Wang.

---

caused by differences in the annotation ability of annotators and the difficulty of samples (Zhang et al., 2022a). For example, in Table 1, Sample and Sample 2 contain annotation errors to varying degrees, while the annotation of Sample 3 is com- pietely correct. The discrepancy in diversity of po- tential annotations refers to the different amounts of potential reasonable alternatives to annotation. Usually, it differs due to different sentence struc- tures or synonymous phrases. For example, Sample 4 and 5 potentially have multiple reasonable anno- tations, while Sample 6 probably only has a single reasonable annotation. Due to the above data dis- crepancies, training data should be distinguished during the training process, by being assigned well- designed weights.
per. From another perspective, our method can be regarded as an "alternative" knowledge distillation method. Compared to Xia et (2022) applying general knowledge distillation on GEC, our method uses a teacher model to obtain mixed-grained train- ing weights based on inherent discrepancies in data to guide the training process, rather than forcing the output distribution of the student model to be consistent with that of the teacher model.
We apply our mixed-grained weighted training to the mainstream Seq2Seq and Seq2Edit methods, and both of them achieve consistent and signifi- cant performance improvements two benchmark datasets, verifying the superiority and generality of the method. In addition, we conduct ablation experiments, further verifying the effectiveness of the designed weights of both granularities. Besides, we conduct the comparative experiment with the general knowledge distillation method on GEC, verifying that our mixed -exxidgr training weight- ing strategy outperforms the general knowledge distillation strategy.
In this paper, we propose MainGEC (i.e., Mixed-grained weighted training for GEC), which designs mixed-grained weights for training data based on inherent discrepancies therein improve the training effect for GEC. First, we use well- trained GEC model (called a teacher model) to quantify accuracy and potential diversity of data annotation. On the one hand, the accuracy of anno- tations is estimated by the generation probability of the teacher model for each target token, which represents the acceptance degree of the teacher model for the current annotation. Then, the quanti- fied accuracy is converted into token-level training weights, as the accuracy of annotations may vary not only across samples but even across tokens in a single sample, e.g., sample 2 in Table 1. On the other hand, the diversity of potential annotations is estimated by the information entropy of output distribution of the teacher model for each training sample, which actually represents the uncertainty, i.e. diversity, of the target sentences that the teacher model is likely to generate. Then, the quantified potential diversity is converted into sentence-level training weights, considering that the potential an- notations may involve the semantics and structures of the entire sentence. Finally, the token-level and sentence-level weigths constitute our mixed- grained weights for the training process.
The main contributions of this paper are summa- rized as follows: (1) We investigate two kinds of inherent discrepancies in data annotation of GEC for the first time, and propose MainGEC, which designs mixed- training weights based on the discrepancies above to improve the training effect. (2) The extensive empirical results show that MainGEC achieves consistent and significant performance improvements over the mainstream Seq2Seq and Seq2Edit methods on two bench- marks, proving the effectiveness and generality of our method for GEC.
2 Preliminary
This section presents the formulation of GEC task and currently mainstream Seq2Seq and Seq2Edit methods for GEC.
2.1 Problem Formulation
Grammatical Error Correction (GEC) is to cor- rect grammatical errors in natural texts. Given an errorful sentence X = [m1, T2, , Fmm] with m tokens, GEC system takes X input, cor- rects grammatical errors therein, and outputs a corresponding error-free sentence Y {w1 82. , yn1 with n tokens. In general, the target sentence Y often has substantial overlap with the source sentence X.
Lichtarge et al. (2020) also considers to allo- cate training weights for samples. However, they only consider discrepancies in synthetic data and still treat numan-annotated data equally, while the discrepancies we consider are across all data. Addi- tionally, they only design sentence-lewe weighting, without token-level weighting considered in this pa-

---

Correct sentence
Edit sequence
SR. school
SKEER
SKEEP
1.1 10
SKEED
Correct sentence
Encoder
Decoder
Encoder
Post-process
school
Errorful sentence X
Correct sentence Y (right shifled)
Enroeful sentenee
Seq2Seq methods
Seq2Edit methods
Figure 1: Overview of Seq2Seq and Seq2Edit methods for GEC. Left: Seq2Seq methods encode the errorful sentence X by an encoder, and autoregressively generate the corresponding correct sentence via decoder. Right: Seq2Edit methods employ sequence tagging model predict tagging sequence 7' of edit operations corresponding to the errorful sentence X, and the correct sentence Y is obtained by applying editing operations to via post-prooe post-procsssin. Here, the tag SA_to denotes appending a new token "to" next to the current token "goes", and the tag SR school denotes replacing the current token "schol" with "school".
2.2 Seq2Seq Methods
and token aoken-deppndenn edits, (e.g. SAPPEND _e; for appending a new token e; next to the current token, SREPLACE e; for replacing the current token with another token et). Considering the linear growth of tag vocab's size taken by token-dependent ed- its, usually, a moderate tag vocab's 's size is set balance edit coverage and model size based the frequency of edits. Then, the original sentence pair (X, Y) converted into a sentence-tags pair (X, T) of equal length. Specifically, the target sentence Y is aligned to the source sentence X by minimiz- ing the modified Levenshtein distance, and then converted to tag sequence T = {11, 12. f'mm Refer to Omelianchuk et al. (2020) for more details.
The Seq2Seq methods employ the encoder-decoder framework, where the encoder encodes the en- tire errorful sentence X into corresponding hidden states, and the decoder autoregressively generates each token in based on the hidden states and the previously generated tokens, as shown on the left in Figure 1.
The general objective function of the Seq2Seq methods is to minimize the negative log-likelihood loss:
( = Σ log p(): = y/ Ye,,),
In training, the general objective function of the Seq2Edit methods to minimize the negative log- likelihood loss for the tag sequence:
where is learnable model parameters, ýu is the i-th token predicted by the model, and Yzi = {31, 3/+ 3 denotes a set of tokens before the i-th
token yi.
( -- Σ log p(f: = (x
2.3 Seq2Edit Methods
Due to the substantial overlap between X and Y. autoregressive generation for the entire target Y is inefficient, and Seq2Edit methods is good alter- native. The Seq2Edit methods usually employ sequence tagging model made up of BERT-like encoder stacked with simple classifier on the top, as shown on the right in Figure 1. At first, a pre- defined set tags is required to denote edit opera- tions. In general, this set of tags contains universal edits, (e.g. SKEEP for keeping the current token un- changed, SDELETE for deleting the current token, SVERB FORM for conversion of verb forms, etc)
where I, is the i-th tag predicted by the model. Dur- ing inference, Seq2Edit methods predict tagging sequence 7 first, and then apply the edit opera- tions in the source sentence X via post-processing to obtain the predicted result Y.
Our Approach
This section presents approach, MainGEC which designs mixed-grained weights for training data based on inherent discrepancies therein im- prove the training effect for GEC. Below, we first claborate on how to quantify accuracy and poten- tial diversity of data annotation at the token-level
'Here we take GRCToR's 's tags (Omelianchuk et al., 2020)
for illustration.

---

token-level training weights for Seq2Seq methods defined as:
Forward
Teacher
Wtoket (y:) = Acc(y:).
Protabtary Dimmmbutton
For Seq2Edit Methods of the data annotation is estimated by the genera- tion probability of a well-trained Seq2Edit teacher model for each target tag 4;
Similarly, the accuracy
Qmuntification
PCG to
(at(Foorop) Sennoms-leee Weighus
Tolum-dessl Weoghes
Mixed-grained
Acc(L) p((; 1,(x,0r),
Weighted Trnining
where E {1,2,-. m). Correspondingly, the token-level training weights for each target tag is defined as:
Figure 2: Illustration of MainGEC. MainGEC converts the target distribution generated by teacher model and original targets into mixed-grained weights, and conducts weighted training with them.
Wioken (4) = Acc(4).
3.2 Sentence-Level Weigths
and sentence-level respectively, and convert quanti- fied features to training weights of both granulari- ties, correspondingly. Then, based both-grained weights, the overall mixed-grained weighted train- ing strategy is introduced. Figure summarizes the overall architecture of MainGEC.
Due to different sentence structures synonymous phrases, there can be multiple potential reasonable alternatives the single target sentence Y of a training sample (X, Y). Further, the amounts potential reasonable alternatives may differ across all samples, which is referred to as the discrepancy in the diversity of potential annotations. Therefore, we quantify the diversity of potential annotations for each training sample by the same teacher model above, and convert them into sentence-level train- ing weights.
3.1 Token-Level Weights
Due to differences in the annotation ability of an- notators and the difficulty of samples, there is discrepancy in the accuracy of data annotation. Ac- tually, this discrepancy exists not only across ples but across tokens in single sample. To this end, well-trainee GEC model used quan- tify the accuracy of data annotation for each token all training samples, and then they are converted into token-level training weights.
For Seq2Seq Methods We feed the source sen- tence X into the teacher model to obtain the proba- bility distribution of its prediction result. For this sample (X, Y), the diversity of potential annota- tions is estimated by the information entropy of this distribution:
For Seq2Seq Methods
The source sentence X is fed into well-trained Seq2Seq GEC model (called the teacher model), and the accuracy of the data an- notation is estimated by the generation probability the teacher model for each target token y:
1 n
H Yo,, Vo
Div(X,Y)
where |V| is the vocab size and H() denotes the entropy of random variable, with log] V| for normalization. Here, lower information entropy means that the teacher model produces a sparser and sharper probability distribution. This leads to the fact that fewer candidate target sentences are likely to be generated, i.e., there is less diver- sity of potential annotations therein. Further, this means the teacher model has more confidence for the target annotation, and higher sentence-level training weight should be assigned during train- ing. Therefore, a monotonically decreasing func- tion and proper boundary processing are applied
Acc(y: = p( = Xx Yei, Or),
where i E (1,2..... m). fr parameters of the teacher model. Actually, this estimation implies the extend to which the teacher model agrees with the current annotation, which can be measure of the accuracy. Then, quantified accuracy of data annota- tion for each target token can be directly regarded the token-level training weight, as the higher accuracy of data annotation means the better anno- tation quality and thus a higher token-level train- ing weight should assigned for training. The

---

the quantified diversity of potential annotations obtain the sentence-level training weight for the sample (X;
Training
#Sent
#Tokens
#Errors
Troy-IBW
1.2M
30.9M
100%
CLang-8
2.4M
28.0M
58%
NUCLE
57K
1.16M
62%
log((iv( (iv((X, Y) +
E) c),
FCR
28K
455K
62%
Wsent (x, Y) Max
loge
W&I+LOCNESS
34K
628K
67%
where is a small positive quantity e)
Test Set
#Sent
#Tokens
#Errors
CONLL-14 BEA-19
1.3K
30.1K
72%
For Seq2Edit Methods Similarly, the diversity of potential annotations is estimated by the infor- mation entropy of output distribution of a Seq2Fdit teacher model for a sample (X;
4.5K
85.7K
Table 2: Statistics of the datasets, including the number of sentences, tokens, and the proportion errorful sen- tences.
1 m
m22 H (f. (ff Br)
Di)(X,T)
=
logiEn
4 Experiments and Results
where |E| is the size of the pre-defined tag set. Cor- respondingly, the sentence-level training weight for the sample (X, T) is defined as:
This section introduces our experiments and results on two benchmarks, i.e., CONLL-14 (Ng et al., 2014) and BEA-19 (Bryant et al., 2019). Then, conduct ablation experiments both-grained training weights and comparative experiments with the general knowledge distillation method. Finally, case study is presented to visualize the weights in MainGEC.
log (X, T) +
W seent( (X, T) Max
log
3.3 Mixed-Grained Weighted Training
The mixed- -exeed-rained weighted training is to simply integrate both grained weights into the training pro- cess. During training. the sentence-level weights determine the contribution of each sample to up- date the model parameters, while further token- level weights are used to adjust the importance of each token/tag therein.
4.1 Experimental Setups
Datasets and Evaluation Metrics As in Tar- navskyi et al. (2022), the training datasets we used consist of Troy-1BW (Tarnavskyi et al., 2022), CLang-82 (Rothe et al., 2021), NUCLE (Dahlmeier et al., 2013), FCE (Yannakoudakis et al., 2011). 1), W&I+LOCNESS (Bryant et al., 2019). The statis- tics of the used datasets are shown in Table 2.
For Seq2Seq Methods We the sentence-level and token-level weights as factors of the training loss for the samples and the tokens in them, re- spectively. The overall loss function of our mixed- grained weighted training is defined as:
For evaluation, we consider two benchmarks, i.e., CONLL-14 and BEA-19. CONLL-14 test set is evaluated by official M scorer (Ng et al., 2014), while BEA-19 dev and test sets are evaluated by ERRANT (Bryant et al., 2017), Both evaluation metrics are precision, recall and Fo.s-
(o)
Σ
Wsent (X, Y)*
(X.Y)D
Wtoken (y1) * p(h V, Y.r
Baseline Methods We compare MainGEC
where D is all training corpus.
against the following baseline methods. All these methods represent current state-of-the-art on GEC, in a Seq2Seq or Seq2Edit manner.
For Seq2Edit Methods Similarly, the loss func- tion of our MainGEC for Seq2Edit methods is de- fined as:
Seq2Seq Methods
(f)
Σ
Wsent (X, x
(X
Lichtarge et al. (2020) introduces sentence- level training weighting strategy by scoring cach sample based delta-log perplexity. Appl, which represents the model's log per-
Wioken (t,) log 0),
where Dd all training data after the tag transfor- mation.
Here, CI anaggg& is a clean version of Lang-8 used in Tar- navskyi et al. (2022).

---

CONLL-14
BEA-
Method
Model
Data Size
Architecture
P
R
Fo.c
P
Fu.s
Lichtarge et al. (2020)
340M
Transformer-big
69.4
43.9
62,1
67.6
62.5
66.5
Stahlberg and Kumar (2021)
540M
Transformer-be
72.8
49.5
66.6
72.1
64.4
70.4
T5GEC (Rothe et al.. 2021)
2.4M
T5-large
66.1
72.1
Seq2Seq
T5GEC (Rothe al., 2021)*
2.4M
T5-xxl
68.8
75.9
SAD (Sun al., 2021)
300M
BART (12+2)
66.4
72.9
BART (Zhang et al., 2022b)
2.4M
BART
48.6
66.7
72.0
SynGEC (Zhang al., 2022h)
2.4M
BART DepGCN
74.7
49.0
67.6
75.1
65.5
72.9
BART (reimp)'
74.3
47.7
66.8
78.1
73.3
2.4M
BART
MainGEC (BART)
77.3
45.4
67.8
78.9
59.5
74.1
PIE (Awasthi et al., 2019)
BERT-large
66.1
43.0
59.7
GECToR (Omelianchuk
2020)
10,2M
XLNET-base
77.5
40.1
65.3
79.2
72.4
TMTC (Lai al., 2022)
10.2M
XLNET-base base
77.9
41.8
66.4
81.3
51.6
72.9
Seq2Edit
GECToR-L (Tarnavskyi et al., 2022)
RoBERTa-large
74.4
41.1
64.0
80.7
53.4
73.2
Lichturge et (2020) (reimp)
3.6M
RoBERTa-large
76.4
40.5
64.9
80.4
54.4
GECToR-L (reimp)
75.9
40.2
64.4
80.9
53.3
73.3
3.6M
RoBERTa-large
MainGEC (GECToR-L)
78.9
39.4
65.7
82.7
53.8
74.5
Table 3: Performance on the test sets CONLL-14 4 and BEA-19, where precision (P), recall (R), Fo.s (Po.s) are reported (%). Baseline results are directly taken from their respective literatures. Results marked by "f" are obtained by applying a decoding approach (Sun and Wang. 2022) to adjust the precision-recall trade-off inference, while the result marked by "f" is not comparable here because it uses much larger model capacity (IIB parameters). Note: Better scores in MainGEC and the directly comparable baseline are bolded.
plexity difference between checkpoints for single sample.
GECToR (Omelianchuk al., 2020) defines a set of token-level transformations and con- ducts 3-stage training on tagging model.
Stahlberg and Kumar (2021) generates more training samples based on an error type tag in back-translation manner for GEC pre- training.
TMTC (Lai et al., 2022) customizes the order the training data based on error type, under GECToR's framework.
T5GEC (Rothe et al., 2021) pretrains large multi-lingun language models GEC, and trains Seq2Seq model distillation data generated by the former more efficiently.
GECToR-L (Tarnavskyi et al., 2022) applys Transfomer-ben based encoders of large configu- rations on GECToR.
Implementation Details For the Seq2Seq im- plementation, BART-large (Lewis et al., 2020) is choosed as the model backbone. At first, fine- tune BART with vanilla training as the teacher model with fairseq implementation. For fair comparison with SynGEC (Zhang al., 2022b), the training scheme here is to just fine-tune BART on the collection of all training sets excluding Troy- 1BW dataset, for just one stage. More training details are discussed in Appendix A.
SAD (Sun al., 2021) employs an asymmetric Seq2Seq structure with shallow decoder to accelerate training and inference efficiency of GEC.
BART (Zhang et al., 2022b) applies a multi- stage fine-tuning strategy pre-trained lan- guage model BART.
SynGEC (Zhang et al., 2022b) extracts depen- deney syntactic information and incorporates with output features of the origin encoder.
For the Seq2Edit implementation. we choose GECToR-L. based RoBERTa (Liu et al., 2019) as the model backbone. The checkpoint released by GECToR-L is used for the teacher model' gen- erate training weights of both granularities. We
Seq2Edit Methods
PIE (Awasthi et al., 2019) generates tag se- quence edit operations and upplys parallel decoding to accelerate inference.
hhups///iittub http:://gitub..cooyytttrttt//// com/pytorch/tairee
Pllass refer to Appendix for effect of different teachers on MainGEC.

---

CONLL-14
BEA-19
CONLL-14
BEA-19
Model
Method
P
R
Fo.s
R
Fo.s
P
R
F6.o
P
Fu.s
BART
74.3
47.7
66.8
78.1
58.9
73.3
GECTOR-1.
75.9
40.2
64.4
80.9
53.3
73.3
MainGEC
77.3
45.4
67.8
78.9
59.5
74.1
KD
76,9
40.7
65.3
81.0
54,4
73.8
w/o Token
74.3
48.0
67.0
79.0
57.6
73.6
MainGEC
78.9
39.4
65.7
82.7
53.8
74.5
w/o Sent
74.4
49.6
67.6
78.1
61.1
74.0
Table 5: Comparison between MainGEC and the gen- cral knowledge distillation method for GEC.
GECToR-I.
75.9
40.2
64.4
80.9
53.3
73.3
MainGECC
78.9
39.4
65.7
82.7
53.8
74.5
w/o Token
74.4
43.1
64.9
81.2
53.1
73.4
w/o Sent
74.3
43.8
65.2
80
57.2
74.1
4.3 Ablation Study
Table Ablation results on MainGEC. with the Seq2Seq group at the top and the Seq2Edit group the bottom. The following changes are applied to MainGEC: removing the token-level training weights (w/o Token), and removing the sentence- training weights (w/o Sent).
We also conduct ablation study on MainGEC to investigate the effects of both-grained training weights, in the Seq2Seq and Seq2Edit manners. Table 4 presents the ablation results. It is obviously observed that whether token-level sentence-level training weights included in MainGEC, certain degree of improvement over the baseline. Moreover, the mixed- -rainng weighted training can provide more improvements on the basis of a single grained weighted training.
bring
also conduct 3-stage training in GECTOR-L. In Stage I, the model is pretrained on the Troy- IBW dataset. Then, in Stage II, the model is fine-tuned on the collection of the CLang-8, NU- CLE, FCE, and W&I+LOCNESS datasets, filtered out edit-free sentences. In Stage III, the model is fine-tuned on the W&I+LCCNES dataset. All training hyperparameters used in MainGEC are set to their default values as in GECTOR-L. Besides. we re-implement the most closely-related work, Lichtarge et al. (2020), based on GECToR-L for more equitable comparison.
4.4 Exploration w.r.t Knowledge Distillation
As there is "teacher" model used obtain training weights in MainGEC, it is necessary to compare MainGEC with the general knowledge distillation method (Xia et al., 2022) for GEC, refered as KD, In KD, the probability distribution generated by the teacher model is regarded as soft objective, which supervises the entire training process with the original groundtruth together. Here. we reim- plement KD in the Seq2Edit manner, where the teacher model the same before and GECToR-L (RoBERTa-large) is choosed as the student model. The experimental result is presented Table 5, As we can see, KD brings a significant improvement over the baseline, due to extra knowledge from the teacher model. More importantly, with the same teacher model, MainGEC outperforms KD with a considcrable margin. This proves grained weighted training is superior to KD, fore- ing the output distribution of the student model be consistent with that of the teacher model.
All checkpoints selected by the loss on BEA- 19 (dev) and all experiments conducted on Tesla A800 with 80G memory.
4.2 Main Results
Table 3 presents the main results of Seq2Seq and Seq2Edit methods. We can see that whether in the Seq2Seq or Seq2Edit manner, MainGEC brings consistent performance improvements on both benchmarks, verifying the effectiveness of our method. Concretely, compared to vanilla train- ing, our mixed-graine grained weighted training leads to 1.0/0.8 improvements in the Seq2Seq manner, and 1.3/1.2 improvements in the Seq2Edit manner. In addition, MainGEC outperforms all baselines on BEA-19 benchmark, with 1.2/1.3 improvements over previous SOTAs, while it also has a compara- ble performance on CONLL- 14 benchmark. These results prove the superiority of our method.
mixed-
4.5 Case Study
Figure 3 shows the same cases as in Table and their token-level or sentence-level weights obtained in MainGEC. The weights here obtained in the Seq2Edit manner. As we can see, token-level and sentence-level weights MainGEC indeed reflect the accuracy and potential diversity of data anno- tation respectively, to some extend. Specifically,

---

Accuracy of Data Annotation (Token-level Weights)
Sumplel
X: Their new house near the beuch is very nice
Y: Your hause the heach very nice.
SATET
T:
Weights:
Sumple2
your email yesterday Y: read your email yesterday but I have (vr) n't had the time reply until now (x)
to axply yeu,,
SAXEUP
StruP
9255?
Weights:
0.30
Sample3
X: do you have best friend in your lnfe: (vr) you have (v) friend in your life
STone cale
Weights:
Diversity of Potential Annotations (Sentence-level Weights)
Samples
Weights
Sumple4
X: Natural environment destroyod that a people locus on frequently problem.
0.25
Sample5
Secondly there are not variety of dessert mainly fruits and puddings.
Sample6
X: One my favourite books are Diary of Wimpy Kid.
Figure 3: The samples Table and corresponding token-level or sentence-level weights obtained in MainGEC. For those token with problematic annotations those samples with multiple potential appropriate annotations, MainGEC will assign relatively low token-level or sentence-level training weights, respectively. The correct annotations are green, erroneous annotations are red, and the corresponding spans the source sentences are in blue.
for those problematic annotation, MainGEC will assign a relatively low token-level weight, and vice versa. When there are multiple potential appro- priate annotations for single sample, only one objective contained in the training set will be as- signed relatively low sentence-level weight. For example, the sentence-level weights of Sample and Sample Table 1 are relatively low due to multiple candidate sentence structures and synony- mous phrases, respectively. This demonstrates that MainGEC is consistent with our motivation at first.
tion. In addition, some works (Kaneko et al., 2020; Zhang et al., 2022b) feed additional features into the neural network to improve GEC, such as the BERT (Devlin et al., 2019) presentation syntac- tic structure of the input sentence.
Seq2Fdit methods treat GEC as sequence tag- ging task, which predicts a tagging sequence edit operations to perform correction (Malmi et al., 2019). Parallel Iterative Edit (PIE) (Awasthi et al.. 2019) and GECToR (Omelianchuk et al., 2020) de- fine set of tags representing the edit operations to be modelled by their system. Lai et al. (2022) investigates the characteristics of different types of errors multi-turn correction based on GECTOR. Tarnavskyi et al. (2022) applies multiple ensem- bling methods and knowledge distillation on the large version of the GECToR system.
5
Related Works
GEC is fundamental NLP task that has received wide attention the past decades. Besides of the early statistical methods, the eurrently main- stream neural GEC methods are categorized into two groups, i.e., Seq2Seq methods and Seq2Edit methods, in general.
Conclusion
Seq2Seq methods treat GEC monolingual translation task, regarding errorful sentences the source language and error-free sentences the target language (Yuan and Briscoe, 2016). Some works (Ge et al., 2018; Sun et al., 2022) generate considerable synthetic data based on the symme- try of the Seq2Seq's 's structure for data augmenta-
This paper proposes MainGEC, which assigns mixed-grained weights to training data based on inherent discrepancies in data improve the train- ing effect for GEC. Our method uses well-trained GEC model quantify the accuracy and poten- tial diversity data annotation, and convert them into the mixed- -gaiined weights for the training

---

process. Whether in the Seq2Seq or Seq2lidit manner, MainGEC achieves consistent and signifi- cant performance improvements on two benchmark datasets, verifying the superiority and generality of the method. In addition, further ablation experi- ments and comparative experiments with the gen- eral knowledge distillation method provide more insights on both-graines training weights and the perspective of knowledge distillation.
Canada, July 30 August 4, Volume 1: Long Pa- pers, pages 793-805 805. Association for Computational Linguisties.
Christopher Bryant, Zheng Yuan, Muhammad Reza Qorib, Hannan Cao, Hwee Tou Ng. and Ted Briscoe. 2022. Grammatical error correction: A survey of the state of the art. CoRR, abs/2211.05EB
Daniel Dahlmeier, Hwee Tou Ng. and Siew Mei Wu. 2013. Building a large annotated corpus of learner english: The NUS corpus of learner english. In Proceedings of the Eighth Workshop on Innovative Use of NLP for Building Educational Applications, BEA @ NAACL-HLT 2013, June 13, 2013, Atlanta. Georgia, USA, pages 22-31. The Association for Computer Linguistics.
Limitations
Our approach requires a well-trained model (called a teacher model) to obtain weights of two granulari- ties before training. Therefore, compared to vanilla training, MainGEC has the additional preparation step to first acquire a teacher model (publicly re- leased or trained by yourself) and then compute the weights by a forward propagation. In addition, the teacher model needs to be consistent with the weighted trained model in terms of type (Seq2Seq or Seq2Edit) and tokenizer.
Jacob Devlin, Ming-Wo Wei Chang. Kenton Lee, and Kristina Toutanova. 2019. BERT: pre-training of deep bidirectional transformers for language under- standing. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguisticx: Human Language Tech- nologies, NAACL-HLL 2019, Minneapolis, MN, USA, June 2-7, 2019, Volume (Long and Short Papers). pages 4171-4186. Association for Computational Linguistics.
Acknowledgements
Tao Ge, Furu Wei, and Ming Zhou. 2018. Reaching numan--evee performance automatic grammati- cal error correction: An empirical study. CoRR, abs/ 00011107010000
We would like to thank all the reviewers for their valuable advice to make this paper better. This re- search is supported by National Science Fund for Excellent Young Scholars under Grant 62222212 12 and the General Program of National Natural Sci- ence Foundation of China under Grant 62376033.
Masahiro Kaneko, Masato Mita, Shun Kiyono, Jun Suzuki, and Kentaro Inui. 2020. Encoder-devoder models can benefit from pre-trained masked language models in grammatical error correction. In Proceed- ings of the 58th Annual Meeting of the Association for Computational Linguistics, ACL 2020, Online, July 5-10, 2020, pages 4248 4254. Association for Computational Linguistics.
References
Abhijeet Awasthi, Sunita Sarawagi, Rasna Goyal, Sabyasachi Ghosh, and Vihari Piratla. 2019, Parallel iterative edit models for local sequence transduction. In Proceedings of the 2019 Conference on Empiri- cal Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, EMNLP-IJCNLP 2019, Hong Kong, China, November 3-7, 2019, pages 4259-4269. Association for Computational Linguisties.
Shaopeng Lai, Qingyu Zhou, Jiali Zeng, Zhongli Li, Chao Li. Yunbo Cao, and Jinsong Su. 2022. Type- driven multi-turn corrections for grammatical error correction. In Findings of the Association for Com- putational Linguistics: ACL 2022, Dublin, Ireland, May 22-27, 2022, pages 3225-32366 Association for Computational Linguistics.
Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, and Luke Zettlemoyer. 2020. BART: denoising sequence-to-sete to-seuuencee pre-training for natural language generation, translation, and com- prehension. In Proceedings of the 58th Annual Meer- ing of the Association for Computational Linguistics, ACI. 2020, Online, 5-10, 2020, pages 7871-7880 Association for Computational Linguistics.
Christopher Bryant, Mariano Felice. Øistein E. Ander- sen, and Ted Briscoe. 2019. The BEA-2019 shared task on grammatical error correction. In Proceedings of the Fourteenth Workshop on Innovatvee Use of NLP for Building Educational Applications, BEA @ ACL 2019, Florence, Italy, August 2, 2019, pages 52-75. Association for Computational Linguistics.
Christopher Bryant, Mariano Felice, and Ted Briscoe. 2017. Automatic annotation and evaluation of error types for grammatical error correction. In Proceed- ings of the 55th Annual Meeting of the Association for Computational Linguistics, ACL 2017, Vancouver,
Jared Lichtarge, Chris Alberti, and Shankar Kumar. 2020. Data weighted training strategies for gram- matical error correction. Trans, Assoc. Comput. Lin- guistics, 8:634-646.

---

Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Man- dar Joshi, Danqi Chen, Omer Levy, Mike Lewis. Luke Zettlemoyer and Veselin Stoyanov. 2019. Roberta: A robustly optimized BERT pretraining approach. CoRR, abs/1907.11692.
2021, (Volume 1: Long Papers), Virtual Event, Au- gust 1-6, 2021, pages 5937-5944 Association for Computational Linguistics.
Xin Sun and Houfeng Wang. 2022. Adjusting the cerisionn-reaal trade-off with align-and-predit de- coding for grammatical error correction. In Proceed- ings of the 60th Anmual Meeting of the Association for Computational Linguistics ( (Volume 2: Short Papers), ACL 2022, Dublin, Ireland, May 22-27, 2022, pages 686 693. Association for Computational Linguistics.
Eric Malmi, Sebastian Krause, Sascha Rothe, Daniil Mirylenka, and Aliaksei Severyn. 2019. Encode, tag, text editing. In Proceedings of the 2019 Conference on Empirical Methods in Nat- ural Language Processing and the 9th International Juint Conference on Natural Language Processing. EMNL.--JCNLPP 2019, Hong Kong, China, Novem- ber 3-7, 2019, pages 5053-5064. Association for Computational Linguistics.
realize:
Maksym Tarnavskyi, Artem N. Chernodub, and Kos- tiantyn Omelianchuk. 2022. Ensembling and knowl- edge distilling of large sequence taggers for gram- matical error correction. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics ( (Volume 1: Longg Papers), ACI. 2022, Dublin, Ireland, May 22-27, 2022, pages 3842-3852. Association for Computational Linguisties.
Hwee Tou Ng. Siew Mei Wu, Ted Briscoe, Christian Hadiwinoto, Raymond Hendy Susanto, and Christo- pher Bryant. 2014. The con1l-2014 shared task on grammatical error correction. In Proceedings of the Eighteenth Conference on Computational Natural Language Learning: Shared Task. CoNLL 2014, Bal- timore, Maryland, USA, June 26-27, 2014, pages 1-14AACL.
Peng Xia, Yuechi Zhou, Ziyan Zhang, Zecheng Tang, and Juntao Li. 2022. Chinese grammatical error cor- rection based on knowledge distillation.
Kostiantyn Omelianchuk, Vitaliy Atrasevych, Artem N. Cheroodu,, and Oleksandr Skurzhanskyi. 2020. Gee- tor - grammatical error correction: Tag, not rewrite. In Proceedings of the Fiffeenth Workshop Innova- tive Use of NLP for Building Educational Applica- lions, BEA @ ACL 2020. Online, July 10, 2020, pages 163-170. Association for Computational Linguistics.
Helen Yannakoudakis, Ted Briscoe, and Ben Medlock. 2011. A new dataset and method for automatically grading ESOL texts. In The 49th Annual Meeting of the Association for Computational Linguistics: Hu- man Language Technologies, Proceedings of the Con- ference, 19-24 June, 2011, Portland, Oregon, USA, pages 180-189. The Association for Computer Lin- guistics.
Sascha Rothe. Jonathan Mallinson. Eric Malmi, Sebas- tian Krause, and Aliaksei Severyn. 2021. A simple recipe for multilingual grammatical error correction. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the
Zheng Yuan and Ted Briscoe. 2016. Grammatical ror correction using neural machine translation. In NAACL HLT 2016, The 2016 Conference of the North American Chapter of the Association for Computa- tional Linguistics: Human Language Technologies, San Diego California, USA, June 12-17, 2016, pages 380-386. The Association for Computational Lin- guistics.
1 1th International Joint Conference on Natural Lan- guage Processing, ACL/IJCNLP 2021, (Volume 2: Short Papers), Virtual Event, August 1-6, 2021, pages 702-707. Association for Computational Linguistics.
Felix Stahlberg and Shankar Kumar. 2021. Synthetic data generation for grammatical error correction with tagged corruption models. In Proceedings of the 16th Workshop on Innovative Use of NLP for Building Edu- cational Applications, BEA @ EACL, Online, April 20. 2021, pages 37-47, Association for Computational Linguistics.
Yue Zhang, Zhenghua Li. Zuyi Bao, Jiacheng Li. Bo Zhang, Chen Li, Fei Huang, and Min Zhang. 2022a. Muegee; a multi-reference multi-source eval- uation dataset for chinese grammatical error correc- tion. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Com- putational Linguistics: Human Language Technolo- gies, NAACL 2022, Seattle, WA, United States, July 10-15, 2022, pages 3118-3130, Association for Com- putational Linguistics.
Xin Sun, Tao Ge, Shuming Ma, Jingjing Li, Furu Wei, and Houfeng Wang. 2022. A unified strategy for multilingual grammatical error correction with pre- trained cross-lingual language model. In Proceed- ings of the Thirty-Firs First International Joint Confer- ence on Artificial Intelligence, IJCAI 2022, Vienna, Austria, 23-29 July 2022, pages 4367-4374 ijcai.org.
Yue Zhang. Bo Zhang. Zhenghua Li, Zuyi Bao, Chen Li. and Min Zhang. 2022b. Syngee: Syntax grammatical error correction with tailored gee- oriented parser. In Proceedings of the 2022 Con- ference on Empirical Methods in Natural Language Processing, EMNLP 2022, Abu Dhabi, United Arab Emirates, December 7-11, 2022, pages 2518-2531. Association for Computational Linguistics.
Xin Sun, Tao Ge. Furu Wei, and Houfeng Wang. 2021. Instantaneous grammatical error correction with shal- low aggressive decoding. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguisties and the 11th International Joint Confer- ence on Natural Language Processing, ACLAJCCNLP

---

A Training Details
The hyper-parametee parameters for MainGEC (BART) are listed in Table 6.
Configurations
Values
Fine-tune
Model Architecture Number of epochs Devices
BART-large
30
Tesla A800 with 80G
Max tokens per GPU
20480
Update Frequency
2
Method
CONLL-14
BEA-19 19 (dev)
Learning rate
3c-05
Adam
GECToR (w/o teacher)
63.4
52.9
Optimizer
( =09,
0.98, c
MainGEC (w/ base teacher)
64.9
55.6
Learning rate scheduler
polynomial decay
MainGEC (w/ large teacher)
65.1
54.9
Weight decay
0.01
Loss Function
cross entropy 2000
Table 7: Performance of MainGEC based teachers of different model scales.
Warmup
Dropout
0.3
Table 6: Hyper-parameters values for MainGEC
(BART).
B Effect of Different Teachers
In MainGEC, teacher is used to quantify training weights of both granularities, which is the main contribution of this work. To investigate effect of different teacher on MainGEC, conduct com- parative experiments under settings: (1) Teach- ers of different model scales: we use GECTOR (RoBERTa-base) and GECTOR (RoBERTa-large) as the teacher respectively for weighted training of GECTOR (RoBERTa-base). (2) MainGEC with self- self-paced learning: we use MainGEC stronger teacher for new round of weighted training, r.e. iterative weighted training with MainGEC. The teacher used in the second round of training is the same model scale the teacher used in the first round but performs better in GEC.
Method
CONLL-14
BEA-19 (dev)
GECToR-L (original teacher) MainGEC (1st round)
64.4
56.4
Table and Table present the experiment re- sults respectively. Experiment results show that no matter what teacher model you use, mixed- grained weights generated by them can bring im- provement over the baseline, verifying effective- ness of MainGEC. Besides, this improvement is not sensitive to the choice of the teacher, either with different model sizes or with different perfor- mances in GEC.
65.7
57.6
MainGEC (2nd round)
66.1
57.4
Table learning.
Performance MainGEC with self-paced


Reviews:
[{"review_id": "qRGN9wZroV", "reviewer": "Reviewer_6raQ", "summary": "", "questions": "Question A: Why do you think this approach is particularly suited for GEC compared to other text generation tasks?", "limitations": "", "rating": 3, "confidence": 5, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We would like to thank the reviewer for the constructive comments and answer the questions as follows.\n\n## Weakness #1 (Question #1)\nOur method is originally designed for the GEC task. If it is considered to be applied to other text generation tasks, the performance improvement may be not as significant as that of GEC. The reason is explained as follows:\n\nFor data annotation in GEC, the distribution of each target token is sparse and sharp, only on one or a few tokens, due to the limitation of the principle of minimal editing. Then, it is relatively easy for the teacher model to accurately fit these distributions, and thus appropriate weights can be given to guide training. Conversely, in other generative tasks (such as machine translation), the probability of each target token is distributed over a wider range of tokens due to the freedom of text expression and word usage. Therefore, it is difficult to accurately fit the probability distribution of each token to obtain training weights, which finally affects the performance gains from weighted training. \n\nTherefore, if MainGEC is applied to other generation tasks, its performance improvement may not be as significant as that of GEC.\n\n\n## Weakness #2\nIt does exceed our computational overhead to conducting experiments with full data on the 11B T5-xxl. Therefore, our experiments are set on the model scale commonly used in existing advanced methods on GEC (BART: 400M, RoBERTa-large: 340M). For T5GEC, we choose T5GEC-large of the same magnitude but still about twice as large (770M) as a performance comparison. From the experimental results, it can be seen that MainGEC can still achieve relatively higher performance with a smaller number of model parameters.\n\nWe will modify the conclusion of SOTA to best performance under model sizes of the same order in subsequent revisions.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "* The data weighting methods proposed by the authors gives some improvements over the baselines they selected.\n* The paper presents useful ablations and an interesting comparison to knowledge distillation.\n* The concrete example in Figure 3 is also quite nice to see.\n", "reasons_to_reject": "* The method described by the authors could be equally applied to any seq-to-seq task. The paper would be stronger if it showed similar improvements in other language tasks, maybe machine translation, data-to-text, or other text generation tasks. If the weighting mechanism proposed only works for GEC, it could be a good research question to find out why.\n* The paper fails to report SotA T5GEC-XXL results in their main results table (Table 3), which are much better than theirs. Lines 345-348 also claim that the methods listed in the paper represent the current SotA on GEC, which is not true. The paper would be much stronger if it showed improvements at the scale of T5GEC-XXL, or at least the paper should note that this model was outside their computational budget.\n", "questions_to_authors": "Question A: Why do you think this approach is particularly suited for GEC compared to other text generation tasks?", "paper_topic_and_main_contributions": "The paper proposes MainGEC, a method for modeling (1) annotation accuracy and (2) potential diversity in GEC data, using token-level and sentence-level training weights. The training weights are computed by first pass trained GEC model: generation probability for each target token (token level weight) is used to estimate (1), entropy of the distribution of output sentences (sentence level weight) is used to model (2).\n\nThe paper reports positive results from this approach on two GEC benchmarks, with seq-to-seq and tagging models.\n", "excitement": 3, "reproducibility": 5, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {"Typos_Grammar_Style_and_Presentation_Improvements": "* It seems that “discrepancies” might not be the right word to use here. Line 48-50 claims that accuracy and diversity are the only discrepancies, but there are many discrepancies in GEC data, much more than just accuracy of annotations and multiple potential annotations. What about domain or topic differences? Or speaker first language and speaker proficiency? Or access to a GEC model while writing?\n* The examples on the first page are not very convincing. The alleged annotation error in Sample 1 could be a justified correction depending on context. The alleged annotation error Sample 2 could also be justified because it makes the sentence more clear. Sample 6 also does not seem to showcase multiple potential annotations, so it should not be in green according to the caption."}}, {"review_id": "PmBCTlcmVb", "reviewer": "Reviewer_45hm", "summary": "", "questions": "A. Do you think it is necessary to add further weight to balance the learning of two different granularities?\n\nB. In the later training stage, the student model could evolve into a well-trained GEC model. Could we then utilize a well-trained student model to derive the weights? (This may be connected to other interesting topics, such as self-paced learning and curriculum learning.)\n", "limitations": "", "rating": 3, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate the constructive comments, and would like to make some clarifications which we hope could address the reviewer’s concerns. The preliminary experiments given here will be refined later and added to the revised paper. \n\n## Weakness #1\nIn fact, the preparation of the teacher model is not difficult. In general scenarios, you definitely need to train a usable GEC model. In our scenario, you can directly use this GEC model as a teacher model, generate mixed-grained weights, and then perform weighted training on an identical model that will perform better than the general trained GEC model, similar to the idea of self-distillation.\n\n\n## Weakness #2\nThe original intention of designing MainGEC is to improve the performance of the existing advanced GEC system, so we choose to train the model from scratch in order to get the strongest possible GEC model. Alternatively, our method can also be used to directly fine-tune the teacher model to improve performance.\n\nExperimentally, we only fine-tune GECTOR-L (teacher) with weighted training to get MainGEC (finetune). The performance comparison of F_0.5 on conll14 and bea19-dev is as follows:\n\n| Methods | conll14 | bea19-dev |\n| :-----| :----: | :----: |\n| GECTOR-L (teacher) | 64.4 | 56.4 |\n| MainGEC (finetune) | 65.6 | 56.4 |\n| MainGEC (from scratch) | 65.7 | 57.6 |\n\nIt can be seen that MainGEC (finetune) also has a certain performance improvement compared to the teacher model. Besides, it sacrifices part of the performance improvement compared to the MainGEC trained from scratch, with saving some training cost.\n\nTherefore, if you are pursuing training efficiency, you can also choose to use the MainGEC method to fine-tune the teacher only. Even so, it can also bring about a certain performance improvement, while more improvement is brought if you train a student model with MainGEC from scratch. \n\n## Weakness #3\nWe conduct preliminary supplementary experiments to explore the effect of different teacher models, using GECTOR (RoBERTa -large) and GECTOR (RoBERTa -base) as the teacher model for weighted training of GECTOR (RoBERTa -base). The performance comparison of F_0.5 on conll14 and bea19-dev is as follows:\n\n| Methods | conll14 | bea19-dev |\n| :-----| :----: | :----: |\n| GECTOR (w/o teacher) |63.4 | 52.9 |\n| MainGEC (w base teacher) | 64.9 | 55.6 |\n| MainGEC (w large teacher) | 65.1 | 54.9 |\n\nIt can be seen that the mixed-grained weights generated by different teacher models are both improved for students, and this improvement is not sensitive to the size of the teacher model itself.\nWe leave additional experiments on other languages as future works.\n\n\n## Question #1\nWe think that there may be no need to design a further weight. In our method, the impacts of both-grained weights on the training objective appear to be complementary. Specifically, the importance of each sample to model optimization is measured by the sentence-level weight. And, each token in a sample shares the sentence-level weight while reflecting its own importance by the token-level weight. Experimental results also verify its effectiveness, without a further weight. \n\nIn addition, there are some difficulties in introducing a further weight. Formally, these two weights have a product relationship in the objective function. As far as we know, there is no general weighting method for this case. And, designing a further weight itself will introduce additional complexity, including specific selection of weighting methods and weighting value.\n\n## Question #2\nIn fact, we conducted an experiment in which the stronger MainGEC model was used as a new teacher model to guide a new round of weighted training process, that is, self-paced learning. The experimental results are as follows:\n\n| Methods | conll14 | bea19-dev |\n| :-----| :----: | :----: |\n| GECTOR-L (original teacher) |64.4 | 56.4 |\n| MainGEC (w/ origin teacher) | 65.7 | 67.6 |\n| MainGEC (w/ stronger teacher) | 66.1 | 57.4 |\n\nThe results show that the newly stronger teacher does not bring significant further performance improvement to MainGEC. This is consistent with the above experimental conclusion \"MainGEC is not sensitive to the selection of the teacher\".", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- **Novelty**: The proposed method introduces a novel approach by quantifying and addressing the discrepancy in training instances for the GEC task. This innovative perspective may inspire future research and provide valuable insights, especially considering the challenges posed by data scarcity in GEC system training.\n\n- **Generality**: The MainGEC method exhibits a high level of generality and effectiveness, making it suitable for both seq2seq and seq2edit paradigms. \n\n- **Clarity, Organization**: The paper's methodology is clearly presented, and the provided examples, such as Table 1 and Figure 3, aid in comprehending the problem and acknowledging the effectiveness of the proposed methods. \n", "reasons_to_reject": "- **Flexibility**: One of the limitations of this approach lies in the requirement for a homologous teacher model in terms of paradigm and vocabulary table with the student model. This can hinder the method's flexibility and may pose challenges during the preparation of the teacher model.\n\n- **Scope**: The pre-trained teacher model tends to be already a strong model; do we truly need to train a student model from scratch using weighted training? Is it possible to simply fine-tune the teacher model with weighted training? Weighted training would be meaningful if our goal is to obtain a small yet strong student model. However, the authors have not clearly indicated the specific targeted scenarios in this paper.\n\n- **Experiments**: \n  1) The authors did not explore how different teacher models affect the student's learning effectiveness, which could have provided valuable insights into the impact of varying teacher models on the proposed method's performance. The choice of teacher model may lack flexibility; however, it is worth noting that there are numerous robust GEC systems that share the same PLM architecture. \n  2) The effectiveness of the proposed approach for other language families remains unknown.\n", "questions_to_authors": "A. Do you think it is necessary to add further weight to balance the learning of two different granularities?\n\nB. In the later training stage, the student model could evolve into a well-trained GEC model. Could we then utilize a well-trained student model to derive the weights? (This may be connected to other interesting topics, such as self-paced learning and curriculum learning.)\n", "paper_topic_and_main_contributions": "This paper presents a teacher-based weighted training method for prevalent grammatical error correction (GEC) paradigms, including seq2seq and seq2edit. \n\nThe main contribution of this paper lies in its consideration of the intra-discrepancy of training data and the proposal of an effective solution to alleviate the potential negative effects caused by these discrepancies during the training stage. To distinguish different data during the training process, the authors suggest using a homologous teacher model to measure the accuracy (token level) and diversity (sentence level) of data annotation. The probabilistic scores generated by the teacher model are then used to assign weights to the training instances.\n\nThe results demonstrate that the proposed MainGEC method outperforms the baselines, and remains competitive when compared to the traditional teacher-student knowledge distillation method.\n", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Typos_Grammar_Style_and_Presentation_Improvements": "- Line 183: REPLACE_ei -> *REPLACE_ei* (to be consistent with other expressions)\n- Line 324: i.e. , -> i.e., (delete the redundant space)\n- Line 387: Seq2Seq2Seq -> Seq2Seq"}, "extra_text": {}}, {"review_id": "kfAjGuEPQM", "reviewer": "Reviewer_uf6d", "summary": "", "questions": "None", "limitations": "", "rating": 3, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thanks to the reviewer's valuable suggestion, we make some explanations here and will improve them in the latter version. \n\n## Weakness #1\nOur contribution is to design a novel mixed-grained training weight based on discrepancies of GEC data to guide the training process under the GEC task. This has not been explored before.\n\n## Weakness #2\nWe have re-implemented the work of Lichtarge et al. (2020), and compared it with the baseline and MainGEC. The F_0.5 performance on conll14 and bea19 dev are as follows:\n\n| Methods | conll14 | bea19-dev |\n| :-----| :----: | :----: |\n| GECTOR-L | 64.4 | 56.4 |\n| Lichtarge et al. | 62.1 | - |\n| Lichtarge et al. (reimp) | 65.1 | 57.1 |\n| MainGEC | 65.7 | 57.6 |\n\nIt can be seen that Lichtarget et al. (2020) also has a considerable performance improvement for the GEC model, while MainGEC improves more performance, which proves the superiority of our method.\n\n## Weakness #3\nWe leave additional experiments on other languages as future works.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. the performance improvement over strong baselines seems good.\n2. the proposed training method is easy to implement and apply.", "reasons_to_reject": "1. The concept of weighted training, even when mixed-grained, is not particularly novel. This paper can be more accurately classified as an empirical contribution.\n2. The comparison with Lichtarge et al. (2020) appears to be somewhat skewed. As this is the most closely-related work and serves as a significant baseline, it would be more appropriate to re-implement their method for a more equitable comparison. This would provide a stronger basis for asserting the superiority of your method.\n3. To further validate the effectiveness of your method, it would be beneficial to conduct additional experiments in other languages, such as Chinese, as well as in low-resource languages like Czech. This would provide a more comprehensive understanding of the method's applicability and robustness.", "questions_to_authors": "None", "paper_topic_and_main_contributions": "This paper introduces a mixed-grained weighted training approach for grammatical error correction (GEC). The authors identify inherent discrepancies in GEC training data, considering both annotation quality and diversity. They thereby propose weights at both the token and sample levels.\n\nThe empirical results demonstrate that this weighted training strategy enhances the performance of sequence-to-sequence (seq2seq) and sequence-to-edit (seq2edit) baselines, underscoring the efficacy of their proposed method.", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper addresses two issues with the training data used in Grammatical Error Correction (GEC) – namely, errors in annotation and the lack of annotation diversity. The paper proposes an approach that uses a weighting schema both on the token and sentence level to address these issues. Experiments on English benchmarks with the frameworks of seq2seq and seq2edit modeling paradigms demonstrate improvements over the baseline models. 

The proposed method is interesting and the experimental results demonstrate the utility of the approach. The paper can be significantly strengthened by presenting experimental results on other languages. The paper should also make it clear that the obtained results are not SOTA. If larger models (such as T5-xxl) cannot be used due to their computational cost, the paper should nonetheless include those results (along with the SOTA results on the benchmarks).
Decision: Accept-Findings