Improving Seq2Seq Grammatical Error Correction via Decoding Interventions
Authors: Houquan Zhou
Yumeng Liu
Zhenghua Li
Min Zhang
Bo Zhang
Chen Li
Ji Zhang
Fei Huang
Venue: EMNLP 2023 Findings (2023)
Keywords: grammatical error correction
decoding
sequence-to-sequence
seq2seq
TL;DR: We introduce a decoding intervention framework that uses critics to assess and guide token generation.

Abstract: The sequence-to-sequence (Seq2Seq) approach has recently been widely used in grammatical error correction (GEC) and shows promising performance.
However, the Seq2Seq GEC approach still suffers from two issues.
First, a Seq2Seq GEC model can only be trained on parallel data, which, in GEC task, is often noisy and limited in quantity.
Second, the decoder of a Seq2Seq GEC model lacks an explicit awareness of the correctness of the token being generated.
In this paper, we propose a unified decoding intervention framework that employs an external critic to assess the appropriateness of the token to be generated incrementally, and then dynamically influence the choice of the next token.
We discover and investigate two types of critics: a pre-trained left-to-right language model critic and an incremental target-side grammatical error detector critic.
Through extensive experiments on English and Chinese datasets, our framework consistently outperforms strong baselines and achieves results competitive with state-of-the-art methods.

First, a Seq2Seq model can only utilize parallel sentence pairs as training data, in which the input sentence is potentially ungrammatical whereas the target one is considered correct. Usually, ma- jor proportion of the training data is automatically collected from language learnrr websites such Lang-8. On the hand, the Lang-8 data contains certain amount of noises, considering the volun- tary contributors may make mistakes as well. On the other hand, the data scale quite limited, com- pared with the non-parallel data used for training large language models. For instance, the cleaned version of English Lang-8 corpus (CLang8) con- tains only 2.4M sentence pairs (Rothe et al., 2021). Improving Seq2Seq Grammatical Error Correction via Decoding Interventions
Houquan Zhou, Yumeng Liu Zhenghua Li, Min Zhang
Bo Zhang, Chen Li; Ji Zhang, Fei Huang
Institute of Artificial Intelligence, School of Computer Science and Technology, Soochow University, China;
(hqzhou, ymlllllllll @ttu.. suda edu. cn, { (zhlil3, minzhang @suda edu. cn DAMO Academy, Alibaba Group, China
klaayhang zb, puji 1c, i1122146, f huang @al www.bbbbaanccoo
Abstract
Input: : But there had no buyers.
Reference: But there no buyers
The sequence-tor teroce-ttt-equennee (Seq2Seq) approach has recently been widely used in grammatical error correction (GEC) and shows promising performance. However, the Seq2Seq GEC ap- proach still suffers from two issues. First, Seq2Seq GEC model can only be trained parallel data, which, in GEC task, is often noisy and limited quantity. Second, the de- coder of Seq2Seq GEC model lacks plicit awareness of the correctness of the token being generated. this paper, we propose unified decoding intervention framework that employs an external critic to assess the appro- priateness of the token be generated incre- mentally, and then dynamically influence the choice the next token. We discover and in- vestigate types of critics: pre-trained left- to-right language model critic and an incremen- target-side grammatical error detector critic. Through extensive experiments on English and Chinese datasets, our framework consistently outperforms strong baselines and achieves re- sults competitive with state-of-the-an methods.
Candidate:
would
Generated sentence But there
log yot
GEC:
-0.59
-3.00
Critie: Penalt
log À Ponalry
Final:
1.23
-1.15
4.26
eg.
0.8
Figure 1: Decoding intervention uses critic to score correctness the token attaching to the partially generated target sentence. The final score of candidate token is the log-probability from the GEC model sub- tracted by the critic penalty, which is scaled by
Recent studies show that the Seq2Seq approach consistently outperforms the Seq2Edit approach on a variety of languages and datasets, especially in handling more complex errors such as word- ordering ones (Qorib et al., 2022; Zhang et al., 2022b). However, the Seq2Seq approach still suf- fers from two issues.
1 Introduction
Automatically correcting grammatical errors is an important task of practical value in the NLP field. The potential applications include document proof- reading, writing assistant, language learning edu- cation, text post-processing for automatic speech recognition (Leng et al., 2021), etc. There are two mainstream approaches to grammatical error correction (GEC), namely ceuuennc-to-ssquenee (Seq2Seq) (Sun et al., 2021; Rothe et al., 2021) and sequence-to-edit (Seq2Edit) (Awasthi et al., 2019; Omelianchuk et al., 2020). The Seq2Seq approach treats GEC monolingual text trans- lation/transduction task, whereas the Seq2Edit ap- proach casts GEC into sequence labeling task.
Data augmentation popular approach for ad- dressing the limited scale issue, i.e., synthesizing
Zhenghua is the corresponding author.

---

large amount of training data (Grundkiewicz et al.. 2019; Stahlberg and Kumar, 2021). However, can be very difficult and tricky to control the error distribution in the generated data so that it resem- bles the realistic scenario. Moreover, brings a heavy computational cost to train a GEC model with very large training data.
making judgments the token be generated ye based on both the imput sentence æ and the tokens generated so far yet-
We conduct experiments on three English GEC datasets, including two English-as-a--econd language (ESL)) datasets, a multi-domain native- English dataset, and Chinese dataset. Experi- mental results demonstrate that decoding inter- vention brings consistent and substantial improve- ments on all datasets. The results also show that with the help of decoding intervention, our GEC model can achieve comparable performance to the state-of-the-ar models on all datasets under com- parable settings without any re-training.
The second issue of the Seq2Seq GEC model is that the decoder lacks an explicit awareness or evaluation of whether the generated token correct during decoding. There are indeed several works that perform grammatical еrror detection (GED) for the input sentence, and use the results as extra features for the encoder so that the decoder pays extra attention the erroneous spans in im- plicit manner (Chen et al., 2020; Yuan al., 2021; Zhang et al., 2022b). However, we are not aware of any previous works that explicitly checks the correctness of generated tokens during decoding (e.g., target-side GED). As pointed out by Mita and Yanaka (2021), a Seq2Seq GEC model tends to gen- erate wrong corrections when the model encounters errors unseen in the training data.
Our code is available at https: / // thub.c om Seccob--hooll hecchh/uggeedd
The Basic Seq2Seq GEC Model
This work aims to improve the Seq2Seq GEC ap- proach. In this section, we briefly describe it. Given a potentially erroneous sentence, a Seq2Seq GEC model tries to generate a correct one without changing its meaning, similar to machine transla- tion, yet in monolingual fashion.
In this work, we propose a decoding intervention framework to address both issues of the Seq2Seq GEC approach. As illustrated in Figure 1, we em- ploy an external critic to assess the appropriateness of the token to be generated incrementally, and then dynamically influence the choice of the next token. Specifically, at each decoding step, the critic will evaluate the appropriateness of the candidate tokens, if the token is inappropriate, the critic will punish the generation of the token by reducing the log- eee-oobabilltyy score of the token.
We adopt the widely-used Transformer archi- tecture (Vaswani al., 2017) as our model back- bone, which comprises an encoder and a decoder. Given an input sentence a = 21 21. Un. the en- coder first encodes it into a sequence of hidden states h. = h At each timestamp, given the input sentence representation h and the previ- ously generated tokens Y-u. the decoder calculates probability distribution over the vocabulary for the next-token generation:
The key to our decoding interventions is to find a suitable critic. We discover and investigate two useful critics. The first critic is a pre-trained left- to-right language model (LM). Using the language model the critic can take advantage of its knowl- edge learned from the vast amount of text. If the language model gives low probability to a token, then the token is probably wrong even if the GEC model gives it a high probability. The second critic is GED model which is an ideal critic to incorpo- rate the explicit awareness of correctness into the Seq2Sey GEC model during the decoding process. However, the conventional GED cannot be directly used as the critic because it does match the incremental manner of the decoding process. To address this problem, we propose an incremental tadget-side GED, which acts Seq2Seq manner
po(y y=ı, .)) Decoder h). (1)
The score of an output sentence is the sum of the nog-probabilities of all predicted tokens:
s(x, y)
log polu yet,
During training, Seq2Seq models commonly em- ploy the teacher forcing method, aiming to max- imize the log likeeiiolooo of the ground-trute next token gt. given the input sentence æ and the previ- ground-truth tokens g-ri
[(0, x. log po( 20). (3)

---

The main advantage of teacher forcing is that allows for parallel training.
Dynamic. The critic model dynamically in- fluences the choice of tokens during decoding, in contrast to re-ranking complete sentences.
The inference of Seq2Seq GEC models to find the best output sentence y" by solving the following optimization problem:
Moreover, the critic model may or may not use the input sentence æ. In this work, we discover and investigate two useful critic models, i.c., a pure left- (4)
y = arg max s(x, y) yEy
to-right pre-trained language model which does not use æ, and an incremental target-side GED model that uses
where Y is the set of all possible sentences. This optimization problem is typically tackled using the beam search algorithm, in which the model predicts token each decoding step, appends it to the partial sentence, and subsequently selects the top partial sentences based on their scores for the next decoding step.
3.1 Left-to-Right Pre-trained LM
conventional pre-trained left-to-right language model, unlike masked language models (e.g.. BERT (Devlin et al., 2019)) and Seq2Seq mod- els (c.g., BART (Lewis et al., 2020)), can be natu- rally used to evaluate the possibility of a sentence y, which is factored product of probabilities of tokens in an incremental manner.
3 Decoding Intervention
In this work, we propose decoding intervention framework improve the Seq2Seq GEC. Con- cretely, we use an external critic model dynam- ically evaluate the correctness of the next token predicted by the GEC model during decoding. The evaluation produces a penalty score to the existing probability distribution from the GEC model as follows:
Pr (y)
I Pr yt y t=1
(6)
where и denotes parameters of the language model. The possibility of token, i.e., Px y can also be understood as how likely the token appears after previous tokens y<t-
y log (yte yeu. x)
The GEC task aims to produce correct sentence that keeps the same meaning as the input sentence. We propose to language model evaluate the correctness of a sentence from a purely linguistic perspective, without referring its input sentence. In this work, we select the GPT-2 models, which are trained on a very large amount of sentences, much more than the parallel sentences that are used for training GEC models, as the pure left-to-right language models. The rationale is that if the lan- guage model gives low probability to token, then the token is probably wrong even if the GEC model gives a high probability. Specifically, we define the penalty from the language model critic as follows.
s(x, 3)
(5)
A x Penally(w, Uet, Ø)).
The first term is the original probability from the GEC model. The logarithm transform stretches the probability into a wider range and thus makes more influential. This also gives more flexibility to the design of the critic model!.
The second term is the penalty score from the critic model to "yr" " given the input sentence and the generated prefix y<r and À a coefficient that controls the trade-off between two model scores. Please note that À is not global hyper- parer--arrmeter but is instead decided by the scores in a token-wise manner. We detail this in Scction 3.3.
Penalty (yr, Uetts x) = 1 Pa(Ue Hee)
(7)
From Eq. (5), we can draw two characteristics our framework.
3.2 Incremental Target-side GED
Incremental. Similar to the Seq2Seq GEC model, the critic model incrementally evaluates a target sentence from left to right, token by token.
As discussed in Section 1, one potential weakness of Seq2Seq GEC models is that the decoder may be unaware of the correctness of its output tokens. Several recent works try to alleviate this issue by performing GED on the input sentence, and using the GED labels as extra inputs to the encoder (Chen et al., 2020; Yuan et al., 2021; Zhang et al., 2022b).
'Based on our carly-stage trials, we find it problematic to directly integrate the probabilities of the GEC model and the critic model via weighted interpolation, since the models usu- ally have diffcrent vocabulary spaces and smaller vocahulary leads to a relatively larger prohability for each token.

---

Type
Generated Tokens
Next
3.3 Coefficient for the Crities
COR
The quick brown fox jumps The quick brown fox already The quick brown fox runs The quick brown fox over
The coefficient Eq. (5) is important for leverag- ing the power of the critic model. Instead of using fixed value for all contexts, we find it is bene- ficial to dynamically set the value by comparing the confidences from the two participating models. Intuitively, model can be trusted if it has high confidence in its prediction, as a strong correlation holds between model's confidence and the accu- racy of its prediction (Guo et al., 2017; Kull et al., 2019). After several experimental trials, we find the following formula works quite well, especially when we use two critics at the same time. Here we use the language model critic for illustration.
RED
SUB
MISS
Table 1: Examples of the four GED labels, ie., cor- rect (COR), redundant errors (RED), substitution errors (SUB), and missing errors (MISS). In this example, the input sentence is "The quick brown fox jump over the lazy dog", and the reference is "The quick brown fox jumps over the lazy dog".
To some extent, this approach can make the model more explicitly aware of the correction process. In this work, we for the first time propose to apply an incremental target-side GED to the output sentence under our framework, which we believe is a more effective intervention strategy.
B x Entropyl po(-) +
== a x B Entropy p=(+) )
(9)
where po(-) and (+(()) refer to the probability distri- bution of the GEC model and the language model their own vocabulary space V; 0 is coefficient that controls the overall scale of penalty scores, and / > governs their variation, both of which aim to balance the influence of the critic models.
Given an input sentence æ, partial target sen- tence generated so far Y-uz. and candidate token Y/. to be generated, the GED model judges the cor- rection of yt into four labels, as shown in Table 1. Please notice that the GED model must look at æ, instead of only accessing y<t. The reason is that the GED model as critic provides a complemen- tary impact versus the language model critic that the target sentence should keep the same meaning as the input sentence. In the absence of æ, many tokens can be considered correct given yer. Formally, we design the penalty from the GED critic model as follows.
For the GED critic, we simply replace P+(-) with (+(-). Please notice that its vocabulary only con- tains the four GED tags?.
We separately select and / for the two crities based dev data. The search space of is {0.1, 0.2, ... 1.0) and that of is {0.01, 0.1, ..., 100). In Section 5, we study the impact of a and B. Results show that our decoding intervention is robust on a wide range of a and B.
Penalty (yt, yea. x) = 11 111-pp(Co (COR yeg x)
(8)
Using both crities When use two critics at the same time, we directly add penalties from the two critic models in Eq. (5).
where Φ is the model parameters, and "COR" is the correct label as shown in Table 1.
Training Our incremental target-side GED acts in a Seq2Seq manner, which is much like the GEC model, requiring parallel sentence pairs for train- ing. Yet we cannot directly use the GEC training data, because the target sentences are all correct. The other consideration is that it is obviously ben- eficial that errors in the target sentences are more consistent with those generated by GEC models. Basically, we use the bascline GEC model to gen- erate K output sentences which may be erroneous via beam search. Then we obtain the error labels for each token (subword, to be accurate) using the editing distance algorithm, which is the same as the evaluation metrics. Section 4 gives more details.
Xm x Penalty (ye, yett x) Xged x Penalty (vr, UUtyy-ea v=t,
(10)
For simplicity, we directly re-use the hyper- parameters separately selected above, and find the performance is satisfactory.
Experiments
Datasets
In this paper, we conduct experiments on two languages: English and Chinese.
"To avoid the entropy imbalance resulting from different vocabulary sizes between the two critics, we normalize the entropies to a range of [0, 1]. This normalization is achieved by dividing by the upper bound of the entropy (log)

---

English
Chinese
System
CoNLL.-14 test
BEA-19 test
GiEGwinffest
MuCGEC test
P
R
Fo.5
P
Fo.s
P
R
Fo.5
P
R
Fo.5
Omelianchuk et al. (2020)
77.5
40.1
65.3
79.2
53.9
72.4
Rothe et al. (2021)
66.1
72.1
Sun et al. (2021)
71.0
52.8
66.4
72.9
Yasunaga al. (2021)
78.0
40.6
65.8
79.4
55.0
72.9
57.9
33.6
50.61
Sun and Wang (2022)
78.7
63.2
75.0
Zhang et al. (2022b)
74.7
49.0
67.6
75.1
65.5
72.9
54.69
29.10
46.51
Vanilla Decoding
76.1
48.3
68.2
76.3
60.7
72.5
72.3
34.7
59.4
56.13
29.38
47.41
Decoding Intervention
Language Model
77.0
48.6
68.9
76.2
61.1
72.6
72.7
35.5
60.1
55.55
30.51
47.66
Target-side GED
78.6
46.3
69.0
77.5
59.5
73.0
75.0
33.5
60.1
56.94
30.06
48.24
Both
79.2
46.8
69.6
77.4
59.9
73.1
75.6
34.4
61.0
56.74
31.00
48.61
Table Results GEC test datasets. t: The model of Yasunaga et al. (2021) in GMEGwon dataset is only trained synthetic data, which makes direct comparisons less meaningful.
For English, we follow the convention of using BEA-19 dev set (Bryant et al., 2019) for method- ological development and the BEA-19 19 and CoNLL- 14 test sets for final evaluation. should be noted that both the BEA-19 and CoNLL-14 test sets are collected from ESL learners. To better understand the effectiveness of our method in real-world sce- narios, also conduct experiments on GMEG- wiki dev/test set (Napoles al., 2019), multi- domain dataset derived from native English speak- ers. For performance metrics, we use M2 Scorer on CoNLL-14 and ERRANT v2.0.0 on the others. For Chinese, we conduct experiments on the MuCGEC dataset (Zhang et al., 2022a), a multi- reference and multi-source ' and use the official ChERRANT scorer to measure the perfor- mance.
ESL datasets, we use the gpt2 model, while for the GMEG-wiki dataset, we opt for the larger gpt2-large model. For the Chinese dataset, MuCGEC, we employ the uer/gpt2-
Target-side GED critic We initialize the back- bone of our target-side GED critic models with pre-trained BART models. Specifically, we use the facebook httprbbaatrrbbeeeeeeeee bass for ESL datasets, the larger facebook brtboookkbrtttarrge for the GMEG- wiki dataset, and the fnlp/bart-large- chinese for the MuCGEC dataset.
We use the FCE, NUCLE, W&I+LOCNESS train set to generate the English training data. And, we use the HSK train set (Zhang, 2009) for Chinese critic models training*. Hyper-parameter details can be found in Appendix C.
Baseline & general settings The GEC model used this paper is a BART model (Lewis et al., 2020) fine-tuned on GEC datasets. Detailed infor- mation on this model can be found in Appendix B. We take "Vaiilla Decoding" as our baseline, which refers to decoding using the original proba- bility score as defined in Eq. (2).
4.1 Main Results
The main results are presented in Table 2. Re- sults show that compared to the baseline "Vanilla Decoding", our decoding intervention consistently improves Fo.5 scores across all datasets, regard- less of the critic used. The two crities improve the model's performance in different ways. The language model critic is better at improving the recall rate, while the target-side GED critic is bet- ter at improving the precision rate. Results also show that our decoding intervention can be further improved by combining the two critics ("Both")). Specifically, "Both" achieves 1.4, 0.6, 1.6, and 1.2 Fo.s improvement the CoNLL-2014, BEA-19,
During the decoding process, we employ the commonly used beam-scarch algorithm find the sequence with the highest score s(x, y). For all experiments, we use beam size 12.
We repeat all the experiments 4 times with dif- ferent random seeds and report the average results.
Language model critic We take off-the-shelf GPT-2 models
language model critics. For Plccase note that we omit the experiments on the NIPCCC- 18 (Zhao et al., 2018) since it is included in MuCGEC.
"We allocate 90% of the generated data for training and 10% for development.

---

Input
Scientists can not conclude whether this smell factic is used attract the Danman (evil partner
crime whom mad guitar skillz) or to ward of predators (PhD supervisors).
Reference 0
crime who has mad guitar skillz) or to ward of predators (PhD supervisons).
Reference I
crime who has mad guitar skills) or to ward off predators (PhD supervisors).
Vanilla Decoding
crime who has mad guitar skills) or to ward of predators (PhD supervisors).
Decoding Intervention
Language Model
crime who has mad guitar skills) to ward off predators (PhD supervisors).
Target-side GED
crime who has mad guitar skills) or to ward of predators (PhD supervisors).
Both
crime who has mad guitar skills) or to ward off predators (PhD supervisors).
Input
Girls were first admitted to Hurlstone Agricultural High School in 1978.
Reference
Girls first admitted to Hurlstone Agricultural High School in 1978
Vanilla Decoding
The girls were first admitted to Hurlstone Agricultural High School 1978.
Decoding Intervention
Language Model
The girls were first admitted to Hurlstone Agricultural High School in 1978
Target-side GED
Girls were first admitted to Hurlstone Agricultural High School in 1978
Both
Girls were first admitted Hurlstone Agricultural High School in 1978
Table 3: Qualitative examples of decoding intervention versus vanilla decoding. Corrections marked in "Blue" are correct or suggested by the reference, while those in "Red" are incorrect.
GMEG-wiki, and MuCGEC test sets, respectively. We also compare model with the recent state- of-the-art models. Note that our baseline model is already competitive with the sate-offtthe-ar mod- els. The tricks that we have used to improve the baseline model's performance are listed in Ap- pendix B.3. Results show that our decoding in- tervention method ("Both") achieves an absolute improvement of 2.0 Fo.5 on the CoNLL-2014 and 2.10 on the MuCGEC. It is worth noting that the best performance in the BEA-19 19 test is achieved by Sun and Wang (2022) with an Fo.s score of 75.0 However, can not be directly compared with our results since they use private synthesized dataset and the size of it is hundreds of times larger than our training data (300M vs. our 2.4M).
model critic fails to correct this by intervening in the decoding process, since the sentence with the definite article is still grammatically correct, albeit with a different meaning.
These two examples also show that the "Both", which uses the target-side GED and the language model at the same time, manages to integrate the advantages of both critics.
Ablation Studies
Impact of critic sizes We perform experiments using four distinct sizes of language models and two different sizes of target-side GEDs.
As shown in Table 4, we can observe that, on BFA--9,, the ESL learners dataset, a larger critic only results in slight improvement in the Fo.5 score (+0.07 for language models, and +0.08 for target-side GEDs). However, the GMEG-wiki, multi-domain dataset from native speakers, a larger critic can lead large improvement on Fo.5 score (+0.30 for language models, and +0.76 for target- side GEDs). This may be because the errors on the ESL dataset are relatively simple and can be captured by smaller critics. In contrast, errors on the multi-domain native dataset are more complex and may require domain knowledge to identify.
4.2 Qualitative Examples
We include two qualitative examples in Table 3. In the first example, the baseline "Vanilla De- coding" and the decoding intervention using the target-side GED the critic both fail to correct the error "ward of"" to "ward off". It is because the error pattern ("ward of"" to "ward off"") has not ap- peared in the training data of both the GEC model and the target-side GED. However, the language model critic is able to correct this error successfully, demonstrating that a language model, pre-trained on vast amounts of data, can help the GEC model identify and correct emoor that do not appear in the GEC training data.
Due the uniform size of the Chinese GPT-2 models we found, we only performed experiments on the target-side GEDs for the Chinese dataset. The results show that a larger target-side GED is more effective when used with the Chinese dataset.
In the second example, the input sentence is grammatically correct. Yet, the baseline "Vanilla Decoding" introduces a new error by inserting a definite article "The" before "Girls". The language
Effectiveness of the dynamic coefficient As mentioned in Section 4.1, 1, the language model and the target-side GED exhibit specific tendencies

---

Size
P
R
Fo.5
P
R
Fo.s
BEA-19 dev
BEA-19 der
Vanilla Decoding
64.85
39.44
57.40
Vanilla Decoding
64.85
39.44
57.40
Decoding Intervention
Decoding Intervention
Language Model
Language Model
65,30
39.75
57.81
gpt2
117M
65.12
39.83
57.74
w/o dynamic coefficient
64.33
40.30 57.43
9pt2-meda um
345M
63.10
39.65
57.64
Target-side GED
66.01
38.71
57.80
gpt2-1arge
774M
65.10
39.92
57.76
w/o dynamic coefficient
66.09
38.59 57.79
gpt2-x1
1.5B
65.30
39.75
57.81
GMEGwis der
Target-side GED
Vanilla Decoding
70.21
32.85
57.19
bart-base
110M
66.52
37.86
57.72
Decoding Intervention
bart 1oart-lame large
400M
66.01
38.71
57.80
Language Model
71.13
33.50
58.07
GMEGwIR dev
w/o dynamic coefficient
70.23
33.08
57.34
Vanilla Decoding
70.21
32.85
57.19
Target-side GED
73.01
31.84
57.98
Decoding Intervention
w/o dynamic coefficient
73.44
31.49
57.95
Language Model
MuCGEC dev
gpt2
117M
70.76
33.31
57.76
Vanilla Decoding
55.69
28.87 46.88
gpt.2 um
345M
70.91
33.36
57.88
Decoding Intervention
9pt2-large
774M
71.13
33.50
58.07
Language Model
55.88
30.02
47.59
gpt2-x1
1.5B
71.07
33.54
58.06
w/o dynamic coefficient
55.85
30.00
47.57
Target-side GED
Target-side GED
55.82
29.83
47.48
bart-base
110M
71.16
32.10
57.22
w/o dynamic coefficient
56.43
28.68 47.20
large
400M
73.01
31.84
57.98
MuCGEC dev
Table 5: Results on GEC dev datasets of w/ or w/o dynamic coefficient strategy. Underline means the result is inferior to the vanilla decoding baseline.
Vanilla Decoding
55.69
28.87
46.88
Decoding Intervention
Target-side GED
139M
56.44
28.60
47.15
bart-large
406M
55.82
29.83
47.48
tervention contains two hyper-parameters: an a for controlling the global scale of the coefficient and a 3 for the coefficient's 's variability. We use a heatmap to visualize the impact of these two hyper- parameters the Fo.5 score, shown in Figure 2. In general, our decoding intervention robust to the hyper-parameters, and surpasses the baseline in most cases. However, we can also observe some interesting phenomena. Compared to the target- side GEDs, the language models are more sensitive to hyper-parameters, particularly to the variability B. Specifically, the English dataset, when vari- ability / is small, meaning that the À is almost the same at different decoding time steps, the language model not only fails to gain improvement but even leads to decrease the Fo.5 score when a is large. Although the target-side GEDs are robust to the syper-parameters, they tend to perform better with a larger a and smaller / in English, and with moderate a and larger 3 in Chinese.
Table 4: Results on GEC dev datasets different sizes of GPT-2 and target-side GED models.
when improving the GEC model. However, we also observed that critic tends decrease one score when improving another. For instance, while the tadget-side GEDs improve the precision score, they also result in a decline in the recall score. might be caused by the misjudgment of the critics when they are unconfident. As a result, the im- provement of the Fo.s score is potentially hindered. To address this issue, we propose coefficient strat- egy that dynamically adjusts the coefficient of the critic at each decoding time step according to the confidence levels of the critic and the GEC model. Results Table 5 show that the dynamic coef- ficient strategy can alleviate the decrease in either the precision score or recall score. Furthermore, this strategy can
lead to an improvement of the P score on the BEA-19 dataset when using the language model as the critic and an improvement of the R score on the MuCGEC dataset when em- ploying the target-side GED as the critic.
Related Works
6.1 Grammatical Error Correction
There exist two main approaches: sequence- to-sequence (Seq2Seq) and sequence-to-edit (Seq2Edit). The Seq2Seq-based approach regard- ing GEC as a monolingual machine translation task,
Robustness of the decoding intervention The dynamic coefficient strategy of the decoding in-

---

Language Model
Target-side GED
BEA-19 der
GMEG dev
MuCGEC dev
BEAA-19 dev
GMEGwIke dev
MuCGEC dev
0.9
0.8
0.7
0.6
0.5
0.4
0.1
0.01
0.1
10
100
0.01
0.1
10
100
0.01 0.1
10
100
0.01 0.1
10
100
0.01
0.1
10
100
0.01
0.1
10
100
(a)
(b)
(c)
(d)
(c)
(f)
Figure 2: Model performance (Fo.s) of decoding intervention compared to vanilla decoding with different scale a and variability 3. The z-axis is 3 and y-axis a. The red cells denote superior performance decoding intervention compared to the vanilla decoding. The blue cells X represent inferior performance. A deeper color indicates a larger performance difference.
is the most widely used approach in the GEC com- munity recently. Though the Seq2Seq- based ap- proach has achieved the state-of-the-art (SOTA) performance on various benchmarks (Sun al.. 2021; Rothe et al., 2021; Zhang al., 2022b, inter alia), they typically have a slow inference speed due to their autoregressive decoding process.
(Gülçehre et al., 2015; Kannan et al., 201 8; Zhao et al., 2019, inter alia). To the best of our knowl- edge, these early-stage works mainly focus on tasks like machine translation and automatic speech recognition, with no known attempts to apply them to GEC. This kind of decoding intervention has be- come less popular recent years, as the advent powerful pre-trained models has largely mitigated the problem of lacking parallel data.
In order deal with the slow inference speed of the Seq2Seq- based approach, numerous recent works focus on the second approach, the Seq2Edit- based approach (Gu et al., 2019; Awasthi et al., 2019; Omelianchuk et al., 2020; Zhang et al., 2023, inter alia). This approach regards GEC as a se- quence labeling task. A Seq2Edit model is trained to predict the edit operations (e.g., keep, insert, delete, replace) for each token in the input sen- tence to transform into correct one. The most representative work, GECToR (Omelianchuk et al., 2020), achieves comparable performance to the state-of-the-aa Seq2Seq approach, with a 10x faster inference speed.
Recent works, the other hand, mostly focus on using decoding interventions to steer pre-trained language models towards generating desired out- puts, such as certain topics, sentiments, or the avoidance or inclusion of specific words (Dathathri et al., 2020; Krause et al., 2021; Liu et al., 2021; Chen et al., 2022, inter alia).
Our work shares similarities with early-stage works in that we also use a language model intervene the decoding process. However, we distinguish ourselves by focusing on the GEC task and proposing the use of a target-side GED model to incorporate explicit grammaticality awareness into the decoding process. It is worth noting that there is work conducts decoding intervention in GEC (Sun and Wang, 2022). However, their motivation is to adjust the precision- recalll trade- off.
6.2 Decoding Intervention
The idea of decoding interventions has been widely used many NLPP tasks. Existing works can be categorized into two temporal stages: early and contemporary.
Early-stage works are mainly used to improve the performance of Seq2Seq-base approaches by using a language model trained on large amount monolingual data to intervene the decod- ing process remedying the lack of parallel data
Conclusions
In this paper, we propose unified decoding inter- vention framework for GEC models. Within this framework, we discover and investigate two useful

---

critics: the language model critic and the target- side GED critie, Among them, the target-side GED critic represents a novel contribution. While most existing research has employed GED on the input side, this work is the first to leverage GED on the target side to assist GEC. Although the concept of a language model critic may not be entirely new, we argue that it is still worth investigating its effec- tiveness the GEC task, especially in the era of pre-trained language models.
ful to Yu Zhang and Yue Zhang for their early-stage exploration on the decoding intervention frame- work. The code of this paper largely based on the open-source framework SUPAR which is de- veloped and maintained by Yu Zhang.
This work was supported by National Nat- ural Science Foundation of China (Grant No. 62176173), Alibaba Group through Alibaba Inno- vative Research Program, and a Project Funded by the Priority Academic Program Development (PAPD) of Jiangsu Higher Education Institutions.
Experiments conducted on four English and Chi- nese GEC datasets lead to several promising find- ings. Firstly, the decoding intervention framework can consistently and substantially improve the per- formance of GEC models, regardless of whether language model error detector is used as the critic. Secondly, The language model critic is better improving the recall rate, while the target-side GED critic is better improving the precision rate. Thirdly, while the size of the critic has a minor impact on the ESL dataset, it becomes substantial on the multi-domain English dataset from native speakers, as well the Chinese dataset. Finally, aided by the decoding intervention framework, our baseline GEC model shows competitive perfor- mance when compared to state-of-the-art models.
References
Abhijeet Awasthi, Sunita Sarawagi, Rasna Goyal, Sabyasachi Ghosh, and Vihari Piratla. 2019. Parallel iterative edit models for local sequence transduction. In Proceedings of EMNLP, pages 4260 4270, Hong Kong. China.
Christopher Bryant, Mariano Felice, Øistein E. Ander- sen, and Ted Briscoe. 2019. The BEA-2019 shared task on grammatical error correction. In Proceedings of BEA, pages 52-75, Florence, Italy.
Howard Chen, Huihan Li, Danqi Chen, and Karthik Narasimhan. 2022. Controllable text genera- tion with language constraints. ArXiv preprint, abs/22 22 10466.
Limitations
Mengyun Chen, Tao Ge, Xingxing Zhang, Furu Wei, and Ming Zhou. 2020. Improving the efficiency of grammatical error correction with erroneous span detection and correction. In Proceedings of EMNLP. pages 7162-7169, Online.
The use of the critic introduces additional com- putational costs and GPU memory usage. Con- sequently, the decoding intervention has slower decoding speeds than the vanilla decoding, espe- cially in the native writing dataset where a larger critic is required for better performance. In the fu- ture, we will further explore methods to reduce the computational costs of the decoding intervention framework, for example, distilling larger critic into a small one, or using lightweight mechanism to decide when to use the critic.
Daniel Dahlmeier, Hwee Tou Ng, and Siew Mei Wu. 2013. Building large annotated corpus of learner English: The NUS corpus of learner English. In Proceedings of BEA, pages 22-31. Atlanta, Georgia.
Sumanth Dathathri, Andrea Madotto, Janice Lan, Jane Hung, Eric Frank, Piero Molino, Jason Yosinski, and Rosanne Liu. 2020. Plug and play language models: A simple approach to controlled text generation. In Proceedings of ICLR.
Besides, this work primarily focuses on the de- coding intervention framework for GEC models. It would be interesting to investigate whether the decoding intervention framework be applied to other Seq2Seq based approaches in different NLP tasks, such as machine translation and text summa- rization, or how to design a suitable critic for these tasks. We leave these questions for future work.
Jacob Devlin, Ming-Wel Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language under- standing. In Proceedings of NAACL-HLT, pages 4171 4186, Minneapolis, Minnesota.
Roman Grundkiewicz, Marcin Munzzys-Dowman and Kenneth Heafield. 2019. Neural grammatical error correction systems with unsupervised pre-training synthetic data. In Proceedings of BEA, pages 252-263, Florence, Italy.
Acknowledgments
We thank the anonymous reviewers for their valu- able comments and suggestions. We are very grate-

---

Jiatao Gu, Changhan Wang, and Junbo Zhao, 2019. Lev- enshtein transformer. In Advances in NeurlPS, pages (1179-11188)
Ilya Loshchilov and Frank Hutter. 2019, Decoupled weight decay regularization. In Proceedings of ICLR.
Masato Mita and Hitomi Yanaka. 2021. Do grammatical error correction models realize grammatical gener- alization? In Proceedings of ACL-IJCNLP, pages 4554-4561, Online.
Çaglar Gülçehre, Orhan Firat, Kelvin Xu, Kyunghyun Cho, Loic Barrault, Huei-Chi Lin, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. 2015. On using monolingual corpora in neural machine trans- lation. ArXiv preprint, abs/1503.035 503.03535.
Courtney Napoles, Maria Nädejde, and Joel Tetreault. 2019. Enabling robust grammatical error correction in new domains: Data sets, metrics, and analyses. TACL, 7:551-566.
Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q Wein- berger. 2017. On calibration of modern neural net- works. In Proceedings of ICML, pages 1321-1330.
Haochen Jiang. Yumeng Liu, Houquan Zhou, Ziheng Qiao, Bo Zhang, Chen Li, Zhenghua Li, and Min Zhang. 2023. CCL223-val task 7 track I system report: Suda &.libbbb team text error correction system. In Proceedings of CCL: Evaluations, pages 220-229, Harbin, China.
Kostiantyn Omelianchuk, Vitaliy Atrasevych, Artem Chernodub, and Oleksandr Skurzhanskyi. 2020. GECToR grammatical error correction: Tag, not rewrite. In Proceedings of BEA, pages 163-170, Seat- tle, WA, USA + Online.
Muhammad Qorib, Seung-Houd -oon Na, and Hwee Tou Ng. 2022. Frustratingly easy system combination for grammatical error correction. In Proceedings of NAACL-HLT, pages 1964-1974.
Anjuli Kannan, Yonghui Wu, Patrick Nguyen, Tara N. Saimath, Zhifeng Chen, and Rohit Prabhavalkar. 2018. An analysis of incorporating an external language model into seguuence-tosequenee model. In Pro- ceedings of ICASSP, pages 5824-5828.
Sascha Rothe, Jonathan Mallinson, Eric Malmi, Sebas- tian Krause, and Aliaksei Severyn. 2021. A simple recipe for multilingual grammatical error correction. In Proceedings of ACI,, pages 702-707, Online.
Satoru Katsumata and Mamoru Komachi. 2020. Stronger baselines for grammatical error correction using a pretrained encoder-decoder model. In Pro- ceedings of AACL-IJCNLP. pages 827-837-7 832, Suzhou, China.
Felix Stahlberg and Shankar Kumar. 2021. Synthetic data generation for grammatical error correction with tagged corruption models. In Proceedings of BEA, pages 37-47, Online.
Ben Krause, Akhilesh Deepak Gotmare, Bryan McCann, Nitish Shirish Keskar, Shafiq Joty, Richard Socher, and Nazneen Fatema Rajani. 2021. GeDi: Gener- ative discriminator guided sequence generation. In Proceedings of EMNLP, pages 4929-4952, Punta Cana, Dominican Republic.
Xin Sun, Tao Ge, Furu Wei, and Houfeng Wang. 2021. Instantaneous grammatical error correction with shal- low aggressive decoding, In Proceedings of ACL, pages 5937-5947, Online.
Meelis Kull, Miquel Perell6-Nieto, Markus Kängsepp. Telmo de Menezes e Silva Filho, Hao Song, and Peter A. Flach. 2019. Beyond temperature scaling: Obtaining well-calibrated multi-class probabilities with dirichlet calibration. In Advances in NeurIPS, pages -
Xin Sun and Houfeng Wang. 2022. Adjusting the precision-recall recall trade-off with align-and-predis de- coding for grammatical error correction. In Proceed- ings of ACL. pages 686-693, Dublin, Ireland.
Christian Szegedy, Vincent Vanhoucke, Sergey Ioffe. Jonathon Shlens, and Zbigniew Wojna. 2016. Re- thinking the inception architecture for computer vi- sion. In Proceedings of CVPR, pages 281 2818-2826.
Yichong Leng, Xu Tan, Linchen Zhu, Jin Xu, Renqian Luo. Linquan Liu, Tao Qin, Xiangyang Li, Edward Lin, and Tie-Yan Liu. 2021. Fastcorrect: Fast error correction with edit alignment for automatic speech recognition. In Advances in NewrIPS, pages 21708- 21719.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances NearIPS, pages 5998- 6008.
Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, AbdeIrahman Mohamed, Omer Levy, Veselin Stoyanov, and Luke Zettlemoyer. 2020. BART: Denoising sequence-to-sequee to-sequecce pre-training for natural language generation, translation, and com- prehension. In Proceedings of ACI. pages 787 7880, Online.
Helen Yannakoudakis, Ted Briscoe, and Ben Medlock. 201 I. A new dataset and method for automatically grading ESOL texts. In Proceedings of ACI. pages 180-189, Portland, Oregon, USA.
Alisa Liu, Maarten Sap, Ximing Lu, Swabha Swayamdipta, Chandra Bhagavatula, Noah A. Smith, and Yejin Choi. 2021. DExperts: Decoding-tine con- trolled text generation with experts and anti-experts. In Proceedings of ACL, pages 6690,, Online.
Michihiro Yasunaga, Jure Leskovec, and Percy Liang. 2021. LM-critic: Language models for unsupervised grammatical error correction. In Proceedings of EMNLP, pages 7752-7763, Online and Punta Cana. Dominican Republic.

---

Zheng Yuan, Shiva Taslimipoor, Christopher Davis, and Christopher Bryant. 2021. Multi-class grammatical error detection for correction: A tale of two systems. Proceedings of EMNLP, pages 8722-8736, Online and Punta Cana, Dominican Republic.
Baolin Zhang. 2009. Features and functions of the HSK dynamic composition corpus. International Chinese Language Education, 4:71-79.
Yu Zhang, Yue Zhang, Leyang Cui, and Guohong Fu. 2023. Non-autoregressive text editing with copy-aware latent alignments. ArXiv preprint, abs/2310.07821.
Zhang. Zhenghua Li, Zuyi Bao, Jiacheng Li, Bo 7hang, Chen Li, Fei Huang, and Min Zhang. 2022a. MuCGEC: multi-reference multi-source evaluation dataset for Chinese grammatical error cor- rection. In Proceedings of NAACL-HLT, pages 31 188 3130, Seattle, United States.
Yue Zhang, Bo Zhang, Zhenghua Li. Zuyi Bao, Chen Li, Min Zhang. 2022b. SynGEC: Syntax-enhanced grammatical eтror correction with tailored GEC- oriented parser. In Proceedings EMNLP, pages 2518-2531, Abu Dhabi, United Arab Emirates.
Ding Zhao, Tara N. Sainath, David Rybach, Pat Ron- don, Deepti Bhatia, Bo Li, and Ruoming Pang. 2019. Shallow-fusibe end-to-end contextual biasing. In Proceedings of INTERSPEECH, pages 1418-1422.
Yuanyuan Zhao, Nan Jiang, Weiwei Sun, and Xiaojun Wan. 2018. Overview of the NLPCC 2018 shared task: Grammatical error correction. In Proceddings of NLPCC, pages 439-445.

---

Dataset
#Sentences
Error
Usage
Configuration
Value
CLang8
2.372.119
Pre tra-iinin
Fine-tuning
FCE
34,490
62.6
Fine-tuning I
Pre-trained Model
BART-arge (Lewis et al., 2020) 60
NUCLE
57.151
38.2
Fine-tuning I
Number of epochs Devices
W&I+LOCNESS
34,308
66.3
Fine-tuning I&II
2 Tesla V100 GPU (32GB)
BEA-19 Dey
4,384
65.2
Validation
40960 tokens 16384 (C)
GMEGwa Dev
992
Validation
Batch size
Co Test
1,312
72.3
Testing
BEA-19 Test
4,477
Testing
Update frequency
GMEG Test
992
82.3
Testing
Optimizer
AdamW (1 oshchhloo and Hutter, 2019)
(3) - 0.9, 0.909.c - 10-*)
Table 6: Statistics of English GEC datasets. #Sentences denotes the number of sentences. % Error refers to the proportion of erroneous sentences.
Max input length
64 (E); 128 (C)
Label smoothed cross entropy 0 (Szegedy al., 2016)
Loss function
Dropout
Dataset
#Sentences
Frror
Usage
Weight decay
2 (E); (C)
Lang8
1,220,906
89.5
Training
Gradient clip
(E); (C) (C)
HSK
15,6870
60.8
Training
Learnnng rate (E Stage 1)
3 10
MuCGEC dev
1,125
95.1
Validation
Learning rate (E Stage 2) Learning rate (E Stage
x 10-6
3 10-6
MuCGEC test
5,938
92.2
Testing
Learning rate (C)
3 10
Table 7: Statistics of Chinese GEC datasets.
Warmup updates (E Stage
2000
Warmup updates (E Stage &
Warmup updates (C)
2000
Generation
A Dataset Statistics
Beam size
12 64 (E): 128 (C)
Max input length
The information of datasets used in our English and Chinese experiments is listed Table 6.
Table Hyper-parameter values used our GEC model. E and denote the Englishh and Chinese model, respectively.
B Details of GEC Model
B.1 Hyper-parameters
B.3 Tricks
The main hyper-parameters adopted by our GEC are presented in Table 8. We use the facebook/ba bbatoolakbbtttrrggee for English and for Chinese.
As shown in the Table 2, our baseline model out- performs most previous works on two datasets: CoNLL-14 and MaCGEC. The reasons are as fol-
fnlp/bart -ar---arrgee-h-- For fine-tuning BART on GEC data, we directly utilize the same hyper-parameters described in Kat- sumata and Komachi (2020). When confronting sentences longer than the max input length, we keep them unchanged during predicting.
lows:
CoNLL-14: We use the same hyperparameters as Katsumata and Komachi (2020), but different from them, we follow the suggestion of Rothe et al. (2021) to post-process the model's output on CoNLL-14 to make its tokenization more con- sistent with the evaluation data. For instance, we remove spaces in Hyphenated words like "face to face" to produce "face-to-face", We implemented this post-processing step after observing that GED critic performs better than vanilla decoding for this type of tokenization error, thereby isolating improvements not attributable to syntactic factors. MuCGEC: The improvement of our baseline model on this dataset comes from two aspects: While we employ the same as previous works, the model's parameters were updated earlier this year, resulting in substantial performance boost (from 46.51 47.04). We noticed that the Chinese correction model sometimes produces repeated tokens during decoding, e.g., Input: "
B.2 Train procedure
We adopt three-stage finetuning strategy for our English GEC model, following Omelianchuk et al. (2020):
STAGE 1: We train GEC model the cleaned version of the Lang8 dataset (CLang8) released by Rothe et al. (2021);
STAGE 2: We fine-tune the model on FCE dataset (Yannakoudakis et al., 2011), NUCLE dataset (Dahlmeier et al., 2013) and the W&T + LOCNESS train-sct (Bryant al., 2019);
STAGE 3: We further fine-tune the model on the W&I LOCNESS test set only.
For Chinese, we use Chinese Lang8 dataset (Zhao
et al., 2018) and HSK dataset as the finetuning data, following Zhang et al. (2022b).

---

3 A ?E? ", Output: " f+ 9
To counter this, we use a simple decoding rule that limits the model's output to no more than times the length of the input, leading to further improvement (from 47.04 to 47.41). A better solution, as suggested by Jiang et al. (2023), is to filter out the instances of which the output length is more than 1.5 times the input length during training.
C Details of Target-side GED Critic
C.1 Hyper-parameters
During training, (Loshchilov and Hutter, 2019) with weight de- cay of 0.01, model and the classifier have a learning rate of 5x10% and 2.5 10", respectively. We use an expo- nential decay scheduler with decay rate of 0.75 and decay step of 5000. The batch size is 65536 tokens, and the maximum sequence length is 1024. The dropout rate is set to 0.3, and the label smooth- ing rate is set to 0.01. The maximum number of epochs is set to 5.
the AdamW optimizer
0.9 and $2 0.98, The BART


Reviews:
[{"review_id": "jIsdsa3Y3q", "reviewer": "Reviewer_hcJu", "summary": "", "questions": "1. Does the subword tokenizer need to be the same for the GEC model and critic models? If the vocabulary is different, how can they be added together in the probability space?\n2. I am curious about the two-stage GEC experiment results, which means performing the target-side GEC directly on the vanilla decoding outputs.", "limitations": "", "rating": 2, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "1. **Not robust enough. The dynamic coefficient sometimes may lead to a decline in the Precise and Recall.**\n\n    We are unclear as to which experimental results you are referring to.\n\n    In fact, the dynamic coefficient serves as a balancing mechanism and, as demonstrated in Table 5, and does not lead to a simultaneous reduction in both precision and recall.\n    As mentioned in Section 4.1, the language model critic is better at improving the recall rate, whereas the target-side GED critic is better at improving the precision rate.\n    However, improving one score often leads to a decline in the other.\n    The role of the dynamic coefficient is to balance these opposing influences, enabling the model to achieve a better $F_{0.5}$ score.\n\n2. **The improvement from gpt2-large to gpt2-xl is not significant, which makes me doubt the potential improvement brought by stronger LM.**\n\n    The observation that the improvement from gpt2-large to gpt2-xl is not significant can be explained by our decoding intervention framework.\n\n    The main prediction is made by the GEC model, which contains only 400 million parameters.\n    For context, this is $0.52$x and $0.27$x the size of gpt2-large and gpt2-xl, respectively.\n    The language model is only an intervention critic, designed to further improve the performance  of an existing GEC model without requiring any **re-training** or **fine-tuning**.\n\n    Therefore, the extent to which the language model critic can improve performance is limited by the capabilities of the GEC model itself.\n\n3. **The critic needs to be finetuned on the specific GEC dataset.**\n\n    In this paper, we discover and investigate two useful critics.\n    \n    The *first* is the target-side GED critic, which does require training, albeit on a GED dataset rather than a GEC dataset.\n    We believe this is acceptable, particularly given the absence of an off-the-shelf target-side GED model.\n\n    The *second* critic is the language model critic, which is a left-to-right pre-trained language model like GPT2. \n    For this critic, there is **no** need for, **nor** have we performed, any fine-tuning on the GEC dataset.\n\n4. **Does the subword tokenizer need to be the same for the GEC model and critic models? If the vocabulary is different, how can they be added together in the probability space?**\n\n    In this paper, we use the same subword tokenizer for the GEC model and the critic models for the sake of convenience.\n\n    However, it's worth noting that using the same subword tokenizer is not a strict requirement.\n    Some simple **engineering solutions** can address the situation where the subword tokenizer is different.\n    For instance, we can change the penalty from a subword level to a word level.\n\n5. **I am curious about the two-stage GEC experiment results, which means performing the target-side GEC directly on the vanilla decoding outputs.**\n\n    What do you mean by target-side GEC? Our GEC model performs correction to the source-side input sentence at a single stage.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. The paper proposes to use an external pre-trained LM to further correct the generated sentence in the decoding process dynamically, which is a potential method to scale up to LLM and get further improvement.", "reasons_to_reject": "1. Not robust enough. The dynamic coefficient sometimes may lead to a decline in the Precise and Recall.\n2. The improvement from gpt2-large to gpt2-xl is not significant, which makes me doubt the potential improvement brought by stronger LM.\n3. The critic needs to be finetuned on the specific GEC dataset.", "questions_to_authors": "1. Does the subword tokenizer need to be the same for the GEC model and critic models? If the vocabulary is different, how can they be added together in the probability space?\n2. I am curious about the two-stage GEC experiment results, which means performing the target-side GEC directly on the vanilla decoding outputs.", "paper_topic_and_main_contributions": "The paper proposes to use an external pre-trained LM as a critic to intervene in the seq2seq decoding process. The critic dynamically influences the choice of the next token by producing a penalty score to the probability distribution of the GEC model. The authors investigate two kinds of pre-trained LM, GPT2 and BART, and conduct experiments on both English and Chinese datasets to verify the efficacy.", "excitement": 2, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "FXX2JfiNM6", "reviewer": "Reviewer_izec", "summary": "", "questions": "The paper very much brags about  'state-of-the-art (SOTA)'.\nHowever, looking at best results, it would be interesting to have a note (just a note) whether this level of SOTA can be useful for practical applications.  \n\nThe paper mentions 'they typically have a slow inference speed' (line 496).\nWhat about the presented model - is its speed slow? \nIs such a concern important for research systems? ", "limitations": "", "rating": 4, "confidence": 3, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "1. **The paper very much brags about 'state-of-the-art (SOTA)'. However, looking at best results, it would be interesting to have a note (just a note) whether this level of SOTA can be useful for practical applications.**\n\n    Thank you for your suggestion, we will add relevant content in the next version.\n\n2. **The paper mentions 'they typically have a slow inference speed' (line 496). What about the presented model - is its speed slow? Is such a concern important for research systems?**\n\n    In this paper, we still use the Seq2Seq model, which is auto-regressive during  the prediction phase, so the inference speed is relatively slow.\n    But for research experiments, this speed is acceptable.\n    However, some papers may focus on how to speed up inference for practical application, such as online editor.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The idea of this paper is to consider two interventions in the decoder stage of the model. One intervention uses a large left-to-right language model for predicting the next word. The other intervention is to integrate a Grammar Error Detection (GED) module, which just labels whether the next-word-to-be-produced by GEC would be correct relative to the whole sentence. \nThis proposal is interesting and worthy of investigation.\n\nThe paper describes the implementation of those ideas and their extensive testing with several published datasets, including in English and Chinese. Ablation studies and qualitative analysis are also included.\n\nOverall the results  indicate that the proposed modifications are useful, as they manage to slightly improve the results relative to previous work with the same datasets.  \n\nThe paper is well-written and the exposition is very clear.", "reasons_to_reject": "N/A", "questions_to_authors": "The paper very much brags about  'state-of-the-art (SOTA)'.\nHowever, looking at best results, it would be interesting to have a note (just a note) whether this level of SOTA can be useful for practical applications.  \n\nThe paper mentions 'they typically have a slow inference speed' (line 496).\nWhat about the presented model - is its speed slow? \nIs such a concern important for research systems? ", "paper_topic_and_main_contributions": "This paper is situated in the research are of Grammar Error Correction (GEC). Specifically it proposes modifications for a seq2seq neural model, in order to improve GEC results.", "excitement": 4, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Justification_for_Ethical_Concerns": "no ethical concerns are apparent"}, "extra_text": {}}, {"review_id": "1mYZ8zR1jt", "reviewer": "Reviewer_cq4q", "summary": "", "questions": "1. Why does the baseline performance in the paper appear to be significantly higher than that in most prior related works?", "limitations": "", "rating": 3, "confidence": 3, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "1. **The paper lacks clarity in explaining the motivation behind the design of the lambda formula, which should be further elaborated to provide a better understanding of its purpose and significance.**\n\n    The motivation behind the design of the lambda formula is to mitigate the impact of potential errors made by the critic. \n\n    Given that critics are not infallible, we require a mechanism to assess the reliability of a critic's predictions and to adjust its influence dynamically.\n    Previous research on model calibration has shown that high confidence in a model's prediction is often indicative of accuracy. This strong correlation between a model's confidence and the accuracy of its predictions led us to use the entropy of the critic's prediction as a proxy for estimating its confidence.\n    Rather than relying on the absolute value of entropy, we employ the entropy ratio between the critic and the GEC model. This approach allows the critic to exert greater influence when its confidence exceeds that of the GEC model and lesser influence otherwise.\n    Finally, to ensure that the critic's influence remains within an acceptable range, we introduce the $\\beta$ coefficient.\n\n2. **On line *294*, it is recommended that Beta should be treated as a static coefficient instead of a dynamic one to enhance the consistency and comprehensibility of the approach.**\n\n    Thank you for your suggestion, we will improve the paper for this issue.\n\n3. **While the paper presents a method, it fails to showcase notable innovation, as several similar works have been previously published. The paper should emphasize and highlight distinct features or improvements over existing approaches to strengthen its contribution.**\n\n    The main contribution of our work is that we propose a unified decoding intervention framework.\n    Within this framework, we discover and investigate two useful critics: the language model critic and the target-side GED critic.\n\n    Among them, the target-side GED critic represents a novel contribution.\n    While most existing research has employed GED on the input side, this work is the first to leverage GED on the target side to assist GEC.\n\n    Although the concept of a language model critic may not be entirely new, we argue that it is still worth investigating its effectiveness on the GEC task, especially in the era of pre-trained language models.\n\n    In addition, the language model critic and the target-side GED critic have complementary effects, and combining them can achieve greater improvement.\n\n    We will clarify these contributions in the next version of the paper.\n\n4. **Why does the baseline performance in the paper appear to be significantly higher than that in most prior related works?**\n\n    As shown in the Table 2, our baseline model outperforms most previous works across multiple datasets: CoNLL-14, GMEG-wiki, and MuCGEC.\n\n    The reasons are as follows:\n\n    - CoNLL-14: We use the same hyperparameters as Zhang et al. (2022), but different from them, we follow the suggestion of Rothe et al. (2021) to post-process the model’s output on CoNLL-14 to make its tokenization more consistent with the evaluation data. For instance, we remove spaces in Hyphenated words like \"face - to - face\" to produce \"face-to-face\".\n        We implemented this post-processing step after observing that our GED critic performs better than vanilla decoding for this type of tokenization error, thereby isolating improvements not attributable to syntactic factors.\n    - GMEG-wiki: The only previous result on this dataset is from Yasunaga et al. (2021), who trained their model on synthetic data only. \n        This makes direct comparisons less meaningful.\n        We will include a clear note regarding this in the next version of the paper.\n    - MuCGEC: The improvement on this dataset comes from two aspects:\n         1) While we employ the same ``fnlp/bart-large-chinese'' as previous works, the model's parameters were updated earlier this year, resulting in a substantial performance boost (from 46.51 to 47.04).\n         2) We noticed that the Chinese correction model sometimes produces repeated tokens during decoding (e.g., Input: \"为怎么呢？\", Output: \"为什么呢？为什么呢？\"). \n        To counter this, we use a simple decoding rule that limits the model's output to no more than 1.8 times the length of the input, leading to further improvement (from 47.04 to 47.41).\n\n\n***\n\n- [SynGEC: Syntax-Enhanced Grammatical Error Correction with a Tailored GEC-Oriented Parser](https://aclanthology.org/2022.emnlp-main.162) (Zhang et al., EMNLP 2022)\n- [A Simple Recipe for Multilingual Grammatical Error Correction](https://aclanthology.org/2021.acl-short.89) (Rothe et al., ACL-IJCNLP 2021)\n- [LM-Critic: Language Models for Unsupervised Grammatical Error Correction](https://aclanthology.org/2021.emnlp-main.611) (Yasunaga et al., EMNLP 2021)", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. The proposed GEC model exhibits commendable performance, achieving results on par with state-of-the-art models across all datasets.\n\n2. The paper demonstrates a rigorous approach by conducting numerous ablative experiments, effectively exploring the impacts of various model sizes and coefficients on the results. This thorough investigation adds depth to the study and enhances the credibility of the findings.", "reasons_to_reject": "1. The paper lacks clarity in explaining the motivation behind the design of the lambda formula, which should be further elaborated to provide a better understanding of its purpose and significance.\n\n2. On line 294, it is recommended that Beta should be treated as a static coefficient instead of a dynamic one to enhance the consistency and comprehensibility of the approach.\n\n3. While the paper presents a method, it fails to showcase notable innovation, as several similar works have been previously published. The paper should emphasize and highlight distinct features or improvements over existing approaches to strengthen its contribution. ", "questions_to_authors": "1. Why does the baseline performance in the paper appear to be significantly higher than that in most prior related works?", "paper_topic_and_main_contributions": "The paper introduces a decoding intervention framework that utilizes an external critic to assess the suitability of tokens generated during decoding. Two types of critics are investigated: a pre-training language model critic and a grammatical error detector critic.\n\nStrengths:\n\nThe proposed GEC model demonstrates comparable performance to state-of-the-art models across all datasets.\nThe paper conducts numerous ablative experiments to explore the effects of different model sizes and coefficients on the results.\nWeaknesses:\n\nThe motivation behind the design of the lambda formula remains unclear and needs further explanation.\nOn line 294, it is suggested that Beta should be a static coefficient rather than a dynamic one to enhance clarity and consistency.\nThe method proposed in the paper lacks notable innovation, as similar works have been previously published in the field. Further emphasis on the novel aspects of the approach could strengthen the paper's contribution.", "excitement": 3, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "bI0Iu8hQo0", "reviewer": "Reviewer_wsFU", "summary": "", "questions": "", "limitations": "", "rating": 2, "confidence": 5, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "1. **LM-critic is not new. It was proposed as early as 2016~2018 which adopts a n-gram LM or a neural LM to improve the GEC performance. See [1], [2].**\n\n    The main contribution of our work is that we propose a unified decoding intervention framework.\n    Within this framework, we discover and investigate two useful critics: the language model critic and the target-side GED critic.\n\n    Although the concept of a language model critic may not be entirely new, we argue that it is still worth investigating its effectiveness on the GEC task, especially in the era of pre-trained language models.\n    In addition, the language model critic and the target-side GED critic have complementary effects, and combining them can achieve greater improvement.\n\n    Notably, our utilization of the language model critic diverges from previous approaches. \n    Our early trials show that existing methods not only failed to improve performance but sometimes lead to inferior performance.\n    From Table 5 and Figure 2, we can see that the dynamic coefficient $\\beta$ is the key to the success of our method.\n\n2. **I don’t think using a GED model trained on the same data as a critic for GEC would introduce more information, because the information they have is the same or similar.**\n\n    We would like to clarify this. \n\n    Please kindly note that the training data used for the target-side GED model is different from that used for GEC model. \n    Specifically, our target-side GED model requires the presence of errors in the target sentences.\n\n    Consider the following input-output pair for the target-side GED model: Input: `{source=\"The quick brown fox jump over the lazy dog\", generated_tokens=\"The quick brown fox\", next_token=\"runs\"}`, Output label: `\"substitution error\"`. It is essential for an error to be present in the `\"next_token\"` field, as otherwise, the GED model would invariably predict a `\"correct\"` label.\n\n    Given that target sentences in GEC training data are grammatically correct, they are unsuitable for training our target-side GED model.\n\n    To address this issue, we use the baseline GEC model to generate 12 output sentences which may be erroneous via beam search.\n    Then, we label the errors in each token by employing an edit distance algorithm.\n\n3. **If we allow the same additional time overhead and make the GEC model larger, what will be the effect? I think it may not be worse than adding a GED - at least the author should add an experiment to justify the practical value of this method.**\n\n    First, it is worth noting that our preliminary experiments indicate that a larger GED model does not necessarily yield better results.\n    For example, in CoNLL-14 dataset, our model (**400M+400M**) outperforms the much larger T5-xxl model with **11B** parameters, as proposed by Rothe et al. (2021).\n\n    Secondly, the cost-effectiveness of scaling up the GED model's parameters appears to be limited. Rothe et al. (2021) reported a parameter increase from **770M** to at least **3B**, merely to achieve an $F_{0.5}$ score improvement from $66.10$ to $67.75$ on CoNLL-14 and from $72.06$ to $73.92$ on the BEA test. \n    In contrast, we argue that the addition of a smaller **400M** GED model during decoding incurs a much more reasonable cost.\n\n    Finally, our decoding intervention is not in conflict with deploying a larger GED model.\n    We can also add a GED-critic to a larger GEC model to further improve its performance.\n\n    However, it should be noted that due to computational constraints, we were unable to conduct experiments with a larger GEC model.\n\n***\n\n[A Simple Recipe for Multilingual Grammatical Error Correction](https://aclanthology.org/2021.acl-short.89) (Rothe et al., ACL-IJCNLP 2021)", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The GED critic sounds interesting and may potentially help improve the performance.", "reasons_to_reject": "1. LM-critic is not new. It was proposed as early as 2016~2018 which adopts a n-gram LM or a neural LM to improve the GEC performance. See [1], [2].\n\n2. I don’t think using a GED model trained on the same data as a critic for GEC would introduce more information, because the information they have is the same or similar. Of course, due to the size limitations of the GEC model, it has to remember more things, so it may overlook some details at the grammar level, and these details can be well grasped by the GED model. But if the model size is further increased, I believe that the GEC model itself can do this well enough. We can see that the most advanced generative models (GPT-3.5, GPT-4) are all decoder-only unidirectional attention layers, but they do not have grammar-level problems, which also fully illustrates this point. What I want to say is that I do not deny that using an additional GED to assist decoding may improve performance when the model is small, but the author must also be clear that this will bring additional time overhead. If we allow the same additional time overhead and make the GEC model larger, what will be the effect? I think it may not be worse than adding a GED - at least the author should add an experiment to justify the practical value of this method.\n\n[1] https://aclanthology.org/N18-2046/\n[2] https://aclanthology.org/P18-1097/", "questions_to_authors": "", "paper_topic_and_main_contributions": "This paper proposes a decoding intervention framework for improving the sequence-to-sequence (Seq2Seq) approach to grammatical error correction (GEC). The main idea is to use an external critic model to dynamically evaluate and influence the next token generated by the GEC model during decoding. The paper investigates two types of critics: a pre-trained left-to-right language model and an incremental target-side grammatical error detector. The paper conducts experiments on four English and Chinese GEC datasets and shows that the decoding intervention framework can consistently and substantially improve the performance of GEC models, achieving results competitive with state-of-the-art methods.", "excitement": 2, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper proposes a decoding intervention framework for Grammatical Error Correction (GEC), where they employ an external critic to improve the choice of the next token. They investigate 2 types of critics – pre-trained LMs and a target-side Grammatical Error Detection (GED) critic that requires training. Experimental results on English and Chinese benchmarks are presented that show improvements over the baseline approaches and demonstrate performance competitive with SOTA models.

The paper is thorough and the experiments demonstrate the effectiveness of the approach. 

While the study is interesting, and the experiments are thorough, the main issue pointed out in the reviews is the lack of novelty. In particular,  one of the two critic components -- using a language model to improve GEC performance -- has been studied before. 

The reviewers also express concern regarding the utility of the approach in practical applications, given that adding a critic increases the model complexity and thus is expected to increase decoding time. It would also be useful to see, given these complexity concerns,  whether increasing the GEC model size would have a similar effect, as adding the critic component.
Decision: Accept-Findings