Unsupervised Grammatical Error Correction Rivaling Supervised Methods
Authors: Hannan Cao
Liping Yuan
Yuchen Zhang
Hwee Tou Ng
Venue: EMNLP 2023 Main (2023)
Keywords: Unsupervised Grammatical Error Correction
TL;DR: We propose an unsupervised grammatical error correction (GEC) method using only monolingual corpora, and its performance rivals supervised GEC systems.

Abstract: State-of-the-art grammatical error correction (GEC) systems rely on parallel training data (ungrammatical sentences and their manually corrected counterparts), which are expensive to construct. In this paper, we employ the Break-It-Fix-It (BIFI) method to build an unsupervised GEC system. The BIFI framework generates parallel data from unlabeled text using a fixer to transform ungrammatical sentences into grammatical ones, and a critic to predict sentence grammaticality. We present an unsupervised approach to build the fixer and the critic, and an algorithm that allows them to iteratively improve each other. We evaluate our unsupervised GEC system on English and Chinese GEC. Empirical results show that our GEC system outperforms previous unsupervised GEC systems, and achieves performance comparable to supervised GEC systems without ensemble. Furthermore, when combined with labeled training data, our system achieves new state-of-the-art results on the CoNLL-2014 and NLPCC-2018 test sets.

Unsupervised Grammatical Error Correction Rivaling Supervised Methods
Hannan Cao
Liping Yuan² Yuchen Zhang
Hwee Tou Ng
'Department of Computer Science, National University of Singapore 2ByteDance
caohēu. nus. edu, yuingiping. @o0@bytedance.com zhangyuchen zyc@bytedance. com, nght@comp.. nus. edu.sg
Abstract
come this limitation. However, the current perfor- mance of unsupervised GEC systems (Alikaniotis and Raheja, 2019; Yasunaga et al., 2021) I) much lower than supervised systems. Moreover, they still require manually defined or extracted confusion sets to generate synthetic data and assess sentence grammaticality. As a result, this greatly hinders the applicability of unsupervised GEC systems.
State-of-the-ar grammatical error correction (GEC) systems rely on parallel training data (ungrammatical sentences and their manually corrected counterparts), which are expensive to construct. In this paper, we employ the Break-It-Fix-It (BIFT) method to build an un- supervised GEC system. The BIFI framework generates parallel data from unlabeled text us- ing fixer to transform ungrammatical sen- tences into grammatical ones, and critic to predict sentence grammaticality. We present an unsupervised approach to build the fixer and the critic, and an algorithm that allows them iteratively improve each other. We evalu- ate unsupervised GEC system English and Chinese GEC. Empirical results show that our GEC system outperforms previous unsuper- vised GEC systems, and achieves performance comparable to supervised GEC systems with- out ensemble. Furthermore, when combined with labeled training data, our system achieves new state-of-thh-aae results on the CoNLL-2014 and NLPCC-2018 test sets.!
The SOTA unsupervised GEC system, LM-critic (Yasunaga et al., 2021), uses the Break-It-Fix: (BIFI) framework (Yasunaga and Liang, 2021) extract realistic parallel data from unlabeled data. Specifically, the BIFI framework utilizes fixer and a critic. The fixer is designed to perform the GEC task, while the critic is designed for the grammat- ical error detection (GED) task, which classifies an input sentence as grammatical or ungrammati- cal. Given critic which classifies each unlabeled sentence grammatical or ungrammatical, BIFI generates parallel data train a better fixer by the following four steps. (1) Correct ungrammatical sentences with the existing fixer and collect out- puts that are classified as grammatical by the critic. (2) Train grammatical eтror generator (called a breaker) using the sentence pairs obtained in (1). (3) Corrupt the grammatical sentences with the breaker and collect the outputs that the critic clas- sifies as ungrammatical. (4) Obtain parallel data by combining outputs of (1) and (3). LM-Critic uses local neighborhood information and perplex- ity (PPL) to build the critic and uses synthetic data to initialize the fixer. However, the synthetic data relies the edit pairs provided by Awasthi et al. (2019), which are extracted from labeled sentences. Moreover, significant performance gap remains between LM-critic and supervised systems (See Section 4).
1 Introduction
Grammatical Error Correction (GEC) (Chollam- patt et al., 2016; Chollampatt and Ng, 2018; Qorib et al., 2022; Bryant et al., 2023) is the task cor- recting errors in a source sentence and generating a grammatically correct target sentence. Current tate-of-thhe-rt (SOTA) systems (Rothe et al., 2021) have reached good performance using sequence-to- sequence (seq2seq) models. However, a common drawback of these systems is their extensive re- liance significant quantity of labeled data. For instance, Rothe
(2021) utilized over mil- lion sentence pairs, which are time-consuming and costly to obtain as they require human manual cor- rection. Unsupervised GEC systems aim to over-
In this paper, propose a novel method for generating synthetic data and building critic, with the aim of building an unsupervised GEC system that can rival supervised systems. By examining
"ork done during Cao's internship at ByteDance. SSurce code available at https: / thub, com/nusnl p/ugec.

---

the grammatical errors in labeled data, we identi- fied several language-independent error patterns, Using these patterns, we propose a synthetic data generation method based on a masked language model (MLM) to build a fixer. Subsequently, we use this fixer as a basis for building our critic. The critic is trained using grammaticality labels ob- tained from high-confidence fixer predictions. To address the data scareity problem that arises from high-confidence filtering, we propose a masking- based approach and self-knowledge distillation method for data augmentation. The unsupervised GEC system is trained using the BIFI framework, with the fixer and the critic being refined repeatedly in iterations.
to validate these corrections. Yasunaga al. (2021) utilize the confusion sets and LM a different way. Instead of constructing a GEC model directly, Ya- sunaga et al. (2021) use them to create a GED model. This GED model is then combined with the BIFI method to build an unsupervised GEC system. In contrast to these works, our method does not rely any manually constructed confusion sets, making it easy extend to low-resource languages. Synthetic data generation. Synthetic data gen- eration for GEC commonly adopts two strategies: corruption methods using labeled data (Kiyono et al., 2019; Stahlberg and Kumar, 2021; Xie et al., 2018), and error injection corruption methods via edit pairs or confusion sets extracted from labeled data (Awasthi et al., 2019; Lichtarge et al., 2019; Yuan and Felice, 2013). Methods that do require labeled GEEC data have been explored by Grundkiewicz et (2019) and Sun et al. (2022). The former utilizes spellchecker- based confusion sets to generate erroneous sen- lences, while the latter applies machine transla- tion pairs and a pre-trained cross-lingual language model (XLM) for sentence corruption. Our method avoids external dependencies, such as confusion sets, spellcheckers, or translation pairs.
backtranslation-ber
We evaluate the performance of our system both English and Chinese GEC tasks. Specifically, we evaluate our system on the CoNLL-2014 (Ng et al., 2014) and BEA-2019 (Bryant al., 2019) test sets for English GEC, and on the NLPCC-2018
(Zhao et al., 2018) test set for Chinese GEC. Our unsupervised system outperforms the prior unsu- pervised SOTA by 12.5 Fo.5 and 13.8 F0.5 on the CoNLL-2014 and BEA-2019 test sets, respectively, Our unsupervised system also compares favorably with the best-performing supervised systems for both languages. Furthermore, when we further train our system with labeled data, we surpass the SOTA results on both CoNLL-2014 and NLPCC- 2018 test sets.
Text evaluation. Prior work in GEC (Bryant al., 2019; Dahlmeier and Ng. 2012: Niu and Penn, 2020) assesses sentence grammaticality through reference text or syntactic information, such as part- of-speech tags. Yasunaga et al. (2021) mitigate this reliance with an LM-based method, yet it still needs pre-defined confusion sets. Our method constructs critic using high-confidence predictions from the fixer model, thereby completely eliminating the need for external information.
The contributions of our paper are as follows:
We introduce novel method for unsupervised synthetic data generation, based on MLM and eanguage-independeee error patterns. Com- pared to existing approaches, our method gen- erates more realistic synthetic data, and pro- vides a better unsupervised fixer.
Masked
Language Model
Monolingual corpus
Monolingual
corpus
We propose new method to build an unsuper- vised critic with high-confidence predictions from the fixer model. This approach enables the critic model to continually enhance its per- formance over iterations, demonstrating better performance than prior methods.
Synthetic data
High predictions
New fixer
High-confidannce predictions
GED pseudo- labels
Initial Fixer
Critic
Paired Data
2 Related Work
Figure 1:
Our unsupervised GEC system involves the following four steps. (1) Create an initial fixer with the synthetic data generated through an MLM-based method. (2) Construct critic based on high-confidence predictions from the fixer. (3) Build new fixer using the parallel data extracted by BIFI. (4) Repeat steps 2 and until the fixer's performance converges.
Unsupervised grammatical error correction. Prior research (Alikaniotis and Raheja, 2019) builds an unsupervised GEC system by leveraging manually constructed confusion sets to provide pos- sible corrections, and uses language models (LMs)

---

3 Method
Figure illlustrates our method to build an unsu- pervised GEC system. contains two key com- ponents: initial fixer construction ($3.2) and the critic construction ($3.3).
Frequeeryy
Prqqeetty
2000
Dattncce
3.1 Problem Setup
The character-level edit distance between erroncous token and its corresponding target token for replacement errors. Left: For English, compute the character-leved edit distance directly. Right: For Chinese, we convert tokens into Pinyin before com- puting the character- edit distance. Instances where the edit distance equals 0 are due to homophonic tokens.
Figure 2:
Grammatical enror correction aims to correct an ungrammatical sentence ( into its grammatical version y0) while preserving the original semantics, In the supervised setting with annotated data avail- able, the GEC model leverages labeled sentence pairs Dj = ((x(0) ( to learn mapping from x to y. However, in the unsupervised setting, the GEC model must infer this mapping from mono- lingual corpus Dm {x (z()). The BIFI framework offers mechanism extract realistic parallel data from unlabeled sentences using fixer / and critic c. The fixer maps r y, and the critic evaluates the grammaticality of given sentence. Our goal to construct a good initial fixer fo ($3.2) and critic ($3.3) through unsupervised methods and utilize them to develop the final fixer fn ($3.4).
2000
Insert
Delete
PRL
(R540)
(00.))
Perceedle
Pencnnle
Figure The erroneous token distribution for insertion and deletion errors. The tokens of the vocabulary are ordered by decreasing frequency. Left: English; Right: Chinese.
3.2 Training an Initial Fixer
The BIFI framework relies on good initial fixer fo. Intuitively, fo could be obtained by training model with synthetic data generated via unsu- pervised approaches. However, how generate realistic synthetic data without reliance on super- vised information (e.g., edit pairs) remains an open problem. To tackle this problem, we analyze the parallel data in English and Chinese to identify some eanguagg-indepenneee Leveraging these patterns, we propose an unsuper- vised synthetic data generation method ($3.2.2).
target token is typically small for both English and Chinese replacement errors. In either language, the majority of the edit distances are confined by the typical length of "word". In Figure 3, we can see that the vast majority of incorrect tokens resulting from insertion and deletion errors are found within the top 5% of the vocabulary. This leads to the conclusion that these errors are commonly associ- ated with high-frequency tokens. Based on these observations, we define two ingguuge-iddeeedeentt error patterns:
error patterns ($3.2.1).
3.2.1 Exploiting Error Patterns
Replacement errors. The edit distance between an erroneous token and its corresponding target token is typically small.
We carry out analysis on the GEC validation and categorize the errors into three categories: inser- tion errors, deletion errors, and replacement errors. Inspired by context-free spell-checkers, we plot the edit distance distribution between erroneous source tokens and their corresponding target tokens for replacement errors. For both deletion and insertion errors, plot the frequency distribution of each erroneous token of the vocabulary.
Insertion and deletion errors. The erroneous to- ken usually has high frequency the vocabulary. Leveraging these two patterns, we outline our unsupervised synthetic data generation approach in $3.2.2.
3.2.2
Unsupervised Synthetic Data Generation
As depicted in Figure 2, evident that the edit distance between an erroneous token and its
We synthesize erroncous sentences from a clean corpus using the following steps: for each sentence (i) from the seed corpus Deeed we first sample the error count per sentence from a pre-defined dis-
"To differentiate hetween the fixer obtained via synthetic data and the fixer ohtained via paired data through BIFI, we name the former fixer as the initial fixer.

---

tribution (Awasthi et al., 2019). We introduce each error by performing one of these three operations:
(1) delete token Wy E (1) with probability Pacti (2) insert a token Wy at a random position with probability Pinsi (3) replace a token Wy C 2((0) with
Peedkton
Peecaton
wr by probability Prep."
Probabiltty
Probability
Specifically,, to generate the replacement token Wr, we replace randomly selected token Wj E 20() with the mask token [MASK] and utilize ML. to predict of candidate tokens at the masked position based on its surrounding context. In this work, we choose RoBERTa as the MLM in our im- plementation. As described in Section 3.2.1, only candidates with a low edit distance from wy are appropriate replacements. Therefore, we eliminate candidate tokens that have an edit distance exceed- ing certain threshold. Finally, sample wo from the remaining candidates using a pre-defined distri- bution solely based on the edit distance.
Figure 4: Correlation between the probability of pro- ducing g( and precision of z Left: English; Right: Chinese.
3.3 Training a Critic
LM-Critic integrates word-level perturbations with sentence perplexity to define the critic. However, the efficacy of word-level perturbations relies on pre-defined confusion sets. To circumvent this liance, an intuitive approach is to extract the GED pseudo-labels from existing fixer and then train a binary classifier from such pseudo-labels as the critic. Specifically, we begin by randomly choos- ing a subset D'm from Dm- For each sentence (1)) E D'm- we use the fixer to make corrections and obtain the output (( If g( is different from (f) then assign pseudo-label z00) 0, mean- ing that z00) is "ungrammatical" Otherwise, we assign 2(1 1, meaning that (x) "grammati- cal"..
To circumvent the problem of consistently sam- pling the same high-frequency tokens for insertion and deletion errors, we design a smoothing function to smooth the frequency of tokens in the vocabulary. This process is detailed in Algorithm 1. In Algo- rithm 1, LISTID represents a list of breakpoints (id)) which are positive integers in ascending or- der used for comparing against the rank of token. Note that the tokens of the vocabulary are orga- nized in descending order of frequency, where a token with a smaller rank occurs more frequently. This design ensures that high-frequency tokens in a collection possess equal chance of being sam- pled, while maintaining a higher frequency than the less frequent tokens. We diverge from sampling based on the raw frequency of tokens in the vocab- ulary, opting to sample according the smoothed frequency smooth.
Since the initial fixer is far from optimal, the pseudo-labels assigned by the initial fixer may have low precision. To address this problem, we analyze the relation between the confidence of (() and the precision of 200), In Figure 4, we observe that high- confidence predictions (i.e., g(() predicted with a high probability) are associated with more accurate grammaticality labels. Therefore, we propose to select highly confident subset D from D', such that for every (t() D.ub. the fixer predicts ((() with probability greater than 0.9.
Algorithm 1: Smoothing Function
It is worth noting that when the critic trained on fixer predictions, it may unintentionally cause over-fitting the fixer, which undermines the critic's 's ability to enhance the fixer further through iterations. Xie et al. (2020) has demonstrated the importance of introducing noise throughout the self-training process. Accordingly, we propose a masking- based data augmentation approach when building the critic. Specifically. for each sentence z(() E Drub. we generate an augmented sentence x. maskedd by randomly replacing p% tokens with the [MASK] token, and minimize the loss function Lymaseed with respect to the critic's model parame-
Input: LISTID [ūdg, id1, ... idn]
wi:: a token the vocabulary
Output: Smoothed probability fsmooth of wy
1: Find the rank for we in the vocabulary
2: Find the smallest i such that & < id,
3: if O then
4: Ssmooth 1/ido
else
fsmooth 1/(ids idj-1)
end if
"The sum of Pdets Dines and Prep equals to one

---

ters Oer:
3.4 Iteratively Refining the Fixer and Critic
Imaakodd
1
Σ
Σ
1x=00 cc-.
Algorithm 3 provides high-level overview of our unsupervised grammatical error correction (GEC) system. We start by applying the unsupervised technique outlined in $3.2.2 to corrupt Dered and yield synthetic data. This synthetic data is then employed to train an initial fixer, denoted by fo- In the next phase, we leverage So and Dm to derive pseudo labels and train a RoBERTa-basee critic, as described in $3.3. By utilizing this critic, we segre- gate Dm into grammatically correct (D%) and in- correct (D%) subsets. We then use the BIFI mech- anism generate realistic parallel data that is then employed train new fixer fi- We subsequently substitute fo with fi and repeat this procedure until the fixer achieves satisfactory performance.
Deubl
CDrt (0,1)
(log (logP(cla1)) fo)) + log log P(
masked fr))
Another issue of selecting high-confidence pseudo-labels is data scarcity. With the initial fixer, only 20% of the sentences from D'm are selected. To mitigate this issue, we utilize self-knowledge distillation (SKD) technique to gather additional training data and enhance the model's generaliz- ability. Specifically, for each (1 E D'm we follow the method used by (Xic et al., 2016; Meng ct al., 2020) to construct soft pseudo-labels 2! (1)4
z(-) =
(2)
Algorithm 3: Unsupervised GEC system
Input: Monolingual corpora D,sedd Dm
I: Generate synthetic data using the method described in $3.2.2 to corrupt Deeed
where fc = ELMED,,, ; is the sum over soft frequencies for class c, and 6rr is the critic's model parameters in the previous epoch. In the first epoch, &'rr represents the critic's model pa- rameters obtained by minimizing (1). Once the soft pseudo-labels are obtained, we train a new critic model by minimizing the following loss function:
2: Build fo with synthetic data
3: for t t 1, 2, 3.... do
4: Extract GED pseudo-labels with fi-1
5: Train critic (grammaticaliity classifier) by minimizing Eqn (3), then use it to split Dm into D%, and D
Leritie Lmaskee + Likd
where
6: Use BIFI (Algorithm 2) generate parallel data to train a new fixer ft-
Lskd
[Dmul]
Σ
Σ
c
=(0)
(3)
7: end for
2DDED mn 0C [0.1)
(IogP(c)x()) (or)) + log [C] (8)
maskod 0cr))
Experiments on English GEC
4.1 Data and Model Configuration
Algorithm 2: Break-It-Fix-Ht (BIFI)
Input: Fixer f. critic c, grammatical sentences Dih and ungrammatical sentences D Output: (crroneous, corrected) sentence pairs.
Following prior work (Awasthi et al., 2019; Grund- kiewicz et al., 2019), we use the combination of WMT NewsCrawl corpus (Bojar et al., 2018) and corpus (Chelba et al., 2014) the seed monolingual corpus Deeed We gener- ate 145 million synthetic sentence pairs with the method described in $3.2.2. These synthetic pairs are used to fine-tune the Flan-T5-xx T5-xxl model (Chung et al.. 2022) to create the initial fixer fo- Following Yasunaga et al. (2021), our monolin- gual dataset Dm contains both grammatical and ungrammatical sentences. Concretely, we ran- domly select 10 million unlabeled sentences from various sources: Yahoo!Answer corpus (Zhang et al., 2015), Wikipedia history (Grundkiewicz and 2014), Lang8 (Mizumoto et al.,
1: Correct D using the fixer f and retain output deemed grammatical by the critic c.
2: Train a breaker (error generator) on the resulting paired data.
Corrupt D% using the breaker and retain output deemed ungrammatical by the critic c.
4: Combine the parallel data obtained Step and 3.
'The intuition is to (1) strengthen predictions: (2) empha- size data with high confidence; and (3) normalize the loss contribution of cach centroid, Refer to (Xie et al., 2016) for details.

---

CONI.L-2014
BEA-2019
System
Architecture
Ens
Fo.5
Fo.5
Unsupervised System
Alikaniotis and Raheja (2019)
GPT2
58.5
24.9
46.
Grundkiewicz et al. (2019)*
BART-base
59.7
18.5
41.3
62.4
25.4
48.8
Yasunaga et al. (2021)*
Base-
64.4
35.6
55.5
67.3
46.1
61.6
ChatGPT with prompting?
ChatGT
50.3
59.7
51.9
42.6
69.3
46.1
GPT4 with promptingt
GPT4
60.8
57.0
59.9
56.8
70.2
59.1
Supervised System
Sorokin (2022)
RoBERTa-lane
79.4
36.1
64.0
86.2
54.2
77.1
Zhang et al. (2022h)
BART-large
74.7
49.0
67.6
75.1
65.5
72.9
Rothe al. (2021)
TS-xxl
68.9
75.9
Lai al. (2022)
78.2
42.7
67.0
77.9
Qorib al. (2022)
81.5
43.8
69.5
86.7
60.9
79.9
Our Unsupervised System
Initial fixer
BART-base
66.2
35.8
56.6
63.1 1
41.3
57.1
1st iteration
BART-base
67.2
40.2
59.3
68.3
48.8
63.2
2nd iteration
BART-base
69.3
40.5
60.6
67.2
51.7
63.4
3rd iteration
BART-base
66.8
44.5
60.7
65.6
$7.4
63.8
Initial lixer
Hla--T3-xxl
70.0
36.5
59.1
73.1
52.1
67.6
+ supervised data
Flan-TS-xxi
74.5
53.6
69.1
78.6
67.8
76.1
st iteration
FSax-T5-xxl
75.5
42.0
65.2
79.5
55.4
73.1
2nd iteration
Flan-TS-XA
75.6
45.6
66.8
80.5
57.8
74.6
3rd iteration
Flan-T5-x-
74.9
49.6
68.0
79.6
62.5
75.4
+ supervised data
Flan-TS-xxl
75.0
53.8
69.61
78.8
68.5
76.5
Table 1: Performance (in %) GEC systems on English GEC test sets. Ens: indicates if system uses the ensemble method. *; represents our reproduced result. Specifically, Grundkiewicz et al. (2019) proposed unsupervised synthetic data generation method. We use this synthetic data to train the BART-base model to make a fair comparison to AM-Critic and our unsupervised system. t: The zero-shot performance ChatGPT and GPT4 using the hest prompt from (Coyne et al., 2023). The best unsupervised and supervised results are shown in bold and bold, respectively. Statistically significant improvements (p < 0.01) over the initial fixer + supervised data is marked #
2011), NUCLE (Dahlmeier et al., 2013), and FCE (Yannakoudakis et al., 2011) 1) datasets. Notably, as Wikipedia history, Lang8, NUCLE, and FCE are labeled datasets, we only take sentences from the source side of these datasets When constructing the critic, choose RoBERTa-base as our classifier model.
supervised system achieves Fo.5 score of 68.0 and 75.4 on the CoNLL-2014 and BEA-2019 test set, respectively, surpassing the current leading unsu- pervised system (Yasunaga et al., 2021) by 12.5 points the CoNLL-2014 and 13.8 points the BEA--009 test set. Our system also exceeds the zero-shot performance the GPT4 model by 8.1 points and 16.3 points on the CoNLL-2014 and BEA-2019 test set, respectively. Notably, our sys- tem compares favorably with the state-of-the-art supervised single system (Rothe et al., 2021), lag- ging behind by just 0.9 points on the CoNLL-2014 test set and 0.5 points the BEA-2019 test set.
the Lang8 dataset D/m and
We evaluate the performance of the English GEC system on the CoNLL--001 and BEA-2019 test sets with the Maxhatch scorer (Dahlmeier and Ng. 2012) and the ERRANT scorer (Bryant et al., 2019), respectively. Following Cao al. (2021), we use one-tailed sign test with bootstrap resam- pling to carry out statistical significance tests. fer to Appendix A.3 for the detailed experimental settings.
To enable a fair comparison with Yasunaga et al. (2021), we replace the Flan-T5-xX model with the smaller BART-base (Lewis et al., 2020) model when building the fixer. With BART-base. our un- supervised system still outperforms Yasunaga al. (2021), with a 5.2 Fo.5 increase on CoNLL-2014 and a 2.2 Fo.5 increase on BEA-2019. This high- lights the superiority our unsupervised training algorithm.
4.2 Main Results
Table shows the performance of our system on both CoNLL-2014 and BEA-2019 test sets, includ- ing a comparison with existing supervised and un- supervised systems on the leaderboard. Our un-
When we further fine-tune our model using su- pervised data, the cLang8 (Rothe et al., 2021)
"The souree side sentences are not annotated sentences, and they could be grammatical or ungrammatical.

---

dataset, our system achieves an Fo.5 of 69.6 on CoNLL-2014 and 76.5 on BEA-2019. This sets a new SOTA result on the CoNLL-2014 test set.
Monolingual
Tim mentioned his goal discover the hidden spy among us. Tim mentioned his her goal discover the hidden spy among
Grundkiewizz al. (2019)
Tim mentioned his goal is to discover tind the hidden spy among
Sun er al. (2022)
4.3 Analysis
Awasthi al. (2019)
Tim mentioned his gool is to discover the hidden eniingg herween
meniioned gool discover hadden spy among us.
Synthetic data. We compare our synthetic data generation method with relevant methods proposed by (Grundkiewicz et al., 2019; Sun et al., 2022), and the method by Awasthi et al. (2019) which was used by (Yasunaga et al., 2021). To enable a fair comparison with the aforementioned data synthesis methods, we randomly select 8 million sentences from the UN Parallel Corpus v1.0 (Ziemski et al., 2016) and corrupt the same monolingual data using each method. We then train a Transformer- base model (Vaswani et al., 2017) on the resulting syn- thetic data.
Our method
During Second World War islands were uccupied by Germany, causing
Monelingual sentence
considerable sufforing to the locals.
Doring the Second World War the islands were occupied Geemany, causing considerable suffering to the locals
Grundkiewicz al. (2019)
During the Second World War the islands isles occupicd by Geemanyy causing considerable suffering to the locals.
Sun et (2022)
During the Second World War the islands were uvcupied by Germany, causing considerable sufforing time the locals. During the Second World War islands were eccspied occupy by Germany, causing considerable sufforing to lacals.
Awasthi al. (2019)
Our method
Table 3: Example erroneous sentences produced by different approaches.
Initial lixer
P
R
Fa.s
Spellchecker (Grundkiewicz et at.., 2019)
Translation pair (Sun et al., 2022)
31.5
8.1
19.8
GEC validation set from the BEAA-2019 dev set, the CoNLL-2013 dev (Ng et al., 2013), and the validation set (Napoles
Edit pair (Awasthi
2019)
39.9
11.0
25.9
Our method
38.1
12.3
2688
wro edit distance
14.2
4.5
9.9
w/o high-ffequency tokens
10.2
3.9
7.7
SMMBG-wik/Yahho/Fee et al., 2019).
Table 2: Performance of the fixer on the BEA-2019 dev set (Bryant al., 2019). Statistically significant improvements (p < 0.01) over Awasthi et al. (2019) is marked #.
We analyze the performance of our critic and compare it to LM-Critic in Table 4. We conduct an ablation study using the following configura- tions: (1) without employing the self-knowledge distillation method (SKD); (2) without applying the data augmentation approach (DA); and (3) without utilizing the tigh-confidence subset D Duub (CF). Re- sults indicate that all three methods are crucial in enhancing the critic's s performance. Notably, our critic outperforms LM-Critic by a significant mar- gin, exhibiting 13.4 Fo.s increase in grammati- cal and 14.1 Fo.5 increase in ungrammatical sen- tences. Our statistical significance test shows that our critic significantly improves over LM-Critic, and our critic without each individual component (SKD, DA and CF) still significantly improves over LM-Critic.
Table 2 shows that our method outperforms com- peting approaches. As demonstrated in Table 3, the erroncous sentences generated by the competing methods tend to either be grammatically correct change the intended meaning of the original sentences. This observation explains the better per- formance our method relative to these competing approaches. Notably, Sun et al. (2022) implements an approach similar to ours, which also generates replacement errors by inserting masks and then uses XLM to predict the mask. The difference is that they translation pairs to guide the creation of candidate tokens, while our method relies on edit distance and frequency information.
Critic
Girammatical
Engrammatical
In our ablation study (Table 2), we find that edit distance and frequency controls are crucial to gen- erate realistic synthetic data, confirming the effec- tiveness of the eтror patterns reported in 1..2.1.
R
Fn.5
P
R
Fn.
LM Critic
63,2
76.0
65.4
69.9
55.7
66.5
Our Critic
77.8
83.0
76.8
81.8
76.3
80.6
wio SKD
72.3
80.5
73.8
78.0
69.1
76.2
DA
81.7
72.9
78.5
75.7
wlo CF
68.9
81.0
71.1
76.9
64.0
73.9
Critic's training methods. Following (Yasunaga et al., 2021), randomly sample 600 grammat- ical sentences and 600 ungrammatical sentences from GEC validation sets and use the averaged F0.5 score over runs to measure the performance of the critic. Specifically, to measure the perfor- mance across various domains, we assemble our
Table 4: Performance of our critic (in %) after the 3rd iteration. The ablation study confirms the effectiveness of self-knowlede distillation (SKD), data augmentation (DA) and using high-confidence pseudo labels (CF).
Fixer's performance through iterations. In Fig- ure 5, the performance of the fixer across BIFT

---

iterations is shown. It observed that the fixer's improvement is stagmant in the absence of the high- confidence subset (CF). Additionally, the fixer's improvement is considerably smaller when data augmentation (DA) or self-knowledge distillation (SKD) is excluded. Moreover, similar to LM-critic, the fixer's improvement comes to halt after the first iteration without updating the critic. This demonstrates the significance of updating both the critic and the fixer throughout the process.
Keep information socret their spoace good.
Keep information as socret to from their spouse
GPT4
gond.
LM
Keep information secret to spouse good.
Our method
Keep the information sceret to from their spouse good. tell
push
Inpul
problem.
Laws carrier tell hisher relatives about his their problem..
GPT4
Laws push carrier tell hisher relatives about him their problem..
LM-Critc
Lawe pash carrier rell hisher relotives ahout his problem.
Our method
The knowledge of the genetic risk was be shared within family.
Input
The knowledge of the genetic risk was were to be shared within the family.
GPT4
The knowledge of the genetic risk was to be shured within the fomuly families.
LM-Critc
60
LM-critic
knowledge genetic the family.
shared within
Our critic
Our method
w/o critic update
w/o SKD
Table 5: Examples comparing system to GPT4 and LM-Critic. Both GPT4 and LM-Critic tend to make un- necessary edits by adding articles changing pronouns or noun number.
w/o DA
55
w/o CF
50
not. The advantage of our system over LM-Critic could be attributed to two components: a better initial fixer which corrects more enrors, and a better critic which assesses sentence grammaticality more precisely, illustrated in Table 2 and Table 4.
0
2
Training Iteration
Figure 5: The performance of the fixer across iterations on the BEA-2019 dev set.
Critic's performance through iterations. In Fig- ure 6, we observe a consistent improvement in the performance of the critic throughout the iterations. This indicates a mutually beneficial learning pro- cess between the critic and the fixer: the critic im- proves the fixer, which in turn refines the critic even further. The plot on the right shows a correlation between pscudo-label precision and fixer iteration. This suggests that the fixer enhances the critic by providing more accurate GED pseudo-labels.
Experiments on Chinese GEC
5.1 Data and Model Configuration
We generate 10 million synthetic sentence pairs using 10 million monolingual sentences crawled from the Toutiao website We train the Chinese BART-large model (Shao et al., 2021) I) on this data to create the initial fixer fo- To build the monolin- gual dataset Dm, we randomly select million sen- tences from the CCMatrix corpus (Schwenk et al.. 2021), Chinese Lang8 (Zhao et al., 2018), and HSK (Zhang, 2009). For both Lang8 and HSK datasets, only take the sentences from the source side. When creating the critic, we the HSK dataset D'n and use RoBERTa- -wm--et (Cui al., 2020) as our classifier model.
&Gammotiial
Grammatioal
Unggrammettcal
Ungaomantical
Pro5Som
72
We evaluate the performance our Chinese GEC system on the NLPCC-2018 test set with the MaxMatch scorer. Following Cao et al. (2021), we use the one-tailed sign test with bootstrap resam- pling carry statistical significance tests.
Figure 6: Left: The performance of the critic in different iterations on the BEA-2019 dev set. Right: The preci- sion of 2(6) using the fixer in different iterations the BEA-2019 dev set. Specifically, iteration 0 represents the initial fixer.
5.2 Results
Examples. In Table 5, we provide qualitative examples to compare the sentences generated by our system with those of GPT4 and LM-Critic. We find that both GPT4 and LM-Critia tend to make unnecessary edits, while our system does
Since unsupervised results are available for Chi- nese GEC, compare model with existing supervised models on the NLPCC-2018 test set. Ta-
"htts://www..outiaa.oom

---

System
Ens
NIPCC-20018
further into this area. Additionally, we trained our models using extensive GPU resources, up 32 37.0
P
Fo.s
Zhao Wang (2020)
X
44.4
22.2
Sun al. (2022)
x
46.0
27.8
40.7
A A100 GPUs, though similar results can be achieved with just 8 V100 GPUs.
Wu and Wa (2022)
X
50.6
25.2
42.1
Zhang et al. (2022b)
X
45.0
33.0
45.3
7hang et al. (2022a)
59.4
24.2
46.0
Acknowledgements
Our Unsupervised System
Initial fixer
X 46.5
25.6
39,9
+ supervised data
X
55.5
29.7
47.3
We thank the anonymous reviewers for their help- ful comments. This research is supported by research grant from TikTok (WBS No. A- 8000972-00-00), The computational work for this article was partially performed on resources of the National Supercomputing Centre, Singapore (https:/wwwwnscc..s))
1st iteration
X
51.2
25.3
42.5
2nd iteration
X 52.1
28.2
44.7
supervised data
X 57.1
28.9
47.8#
Table 6: Performance (in %) GEC systems on the NLPCC-2018 test set. Ens: represents if the system uses the ensemble method. The best unsupervised result and the supervised result are shown in bold and bold, re- spectively. Statistically significant improvements (p 0.01) over initial fixer + supervised data is marked #.
References
Dimitris Alikamiotis and Vipul Raheja. 2019, The unrea- sonable effectiveness of transformer language models in grammatical error correction. In Proceedings of the Fourteenth Workshop on Innovative Use of NLP for Building Educational Applications, pages 127- 133.
ble 6 shows that model achieves 44.7 Fo.5 score, surpassing Wu and Wu (2022) and Sun et al. (2022). It only 0.6 points below the best-performing supervised single system. When we further fine- tunc our unsupervised GEC system with labeled data, the combination of the Chinese Lang8 dataset, and the HSK dataset, our system achieves 47.8 Fo.s score, setting a new SOTA on NLPCC-201 8. It demonstrates that our unsupervised model can serve strong initial checkpoint for supervised training.
Abhijeet Awasthi, Sunita Sarawagi, Rasna Goyal, Sabyasachi Ghosh, and Vihari Piratla. 2019. Parallel iterative edit models for local sequence transduction. In Proceedings of the 2019 Conference Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-JJNLP), pages 4260-4270.
Ondfej Bojar, Christian Federmann, Mark Fishel, Yvette Graham, Barry Haddow, Mathias Huck, Philipp Koehn, and Christof Monz. 2018. Findings of the 2018 conference on machine translation (WMT18) In Proceedings of the Third Conference on Machine Translation: Shared Task Pupers, pages 272-303.
6 Conclusion
In this paper, we present innovative unsupervised techniques to produce synthetic parallel data and train critic evaluate the grammaticality of sen- tences. By combining our methods with BIFI, we develop an unsupervised GEC system that achieves results comparable to models utilizing substantial labeled data. The core idea is to employ language- independent erroncous models to construct realis- tic synthetic data, and then create unsupervised critic utilizing high-confidence predictions from the fixer model. Our system does not require any manually defined or extracted confusion sets, mak- ing it an ideal solution for developing GEC models for fow-resource languages.
Adriane Boyd, Jirka Hana, Lionel Nicolas, Detmar Meurers, Katrin Wisniewski, Andrea Abel, Karin Schöne, Barbora Štindlová, and Chiara Vettori. 2014. The MERLIN corpus: Learner language and the CEFR. In Proceedings of the Ninth International Conference on Language Resources and Evaluation (IREC' pages 1281 1281-1288, Reykjavik, Iceland.
Christopher Bryant, Mariano Felice, Øistein E. Ander- sen, and Ted Briscoe. 2019. The BEA-2019 shared task grammatical error correction. In Proceed- ings of the Fourteenth Workshop on Innovative Use of NLP for Building Educational Applications, pages 52-75.
7 Limitations
Christopher Bryant, Zheng Yuan, Muhammad Reza Qorib, Hannan Cao, Hwee Tou Ng, and Ted Briscoe. 2023. Grammatical Error Correction: A Survey of the State of the Art. Computational Linguistics.
We identified and utilized error patterns in both English and Chinese labeled corpora. While believe such patterns are language-agnostic, we have not explored their application to other low- resource languages. Future research may delve
Hannan Cao, Wenmian Yang, and Hwee Tou Ng. 2021. Grammatical error correction with contrastive learn- ing in low error density domains. In Findings of the

---

Association for Computational Linguisties: EMNLP 2021, pages 4867-4874.
Roman Grundkiewicz and Marcin Junczys- Dowmunt. 2014. The wiked error corpus: A corpus of corrective wikipedia edits and its application to grammatical error correction. In Advances in Natural Language Processing, pages 478-
Hannan Cao, Wenmian Yang, and Hwee Tou Ng. 2023. Mitigating exposure bias in grammatical error cor- rection with data augmentation and reweighting. In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Lin- guistics, pages 2123-22 2135.
Roman Grundkiewicz. Marcin Dosayyy, and Kenneth Heafield. 2019. Neural grammatical error correction systems with unsupervised pre pre-trannig on synthetic data. In Proceedings of the Fourteenth Workshop Innovative Use of NLP for Building Educational Applications, pages 252-263.
Ciprian Chelha, Tomas Mikolov, Mike Schuster, Qi Ge, Thorsten Brants. Phillipp Koehn, and Tony Robinson. 2014. One billion word benchmark for measuring progress in statistical language modeling.
Shun Kiyono, Jun Suzuki, Masato Mita. Tomoya Mizu- moto, and Kentaro Inui. 2019. An empirical study of incorporating pseudo data into grammatical error correction. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Process- ing and the 9th International Joint Conference on Natural Language Processing, pages 1236-124..
Shamil Chollampatt and Hwee Tou Ng. 2018. A multi- layer convolutional encoder-decoder neural network for grammatical error correction. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 5755-5762.
Shamil Chollampatt, Kaveh Taghipour, and Hwee Tou Ng. 2016. Neural network translation models for grammatical error correction. In Proceedings of IJ- CAL, pages 2768-22 2774..
Shaopeng Lai, Qingyu Zhou, Jiali Zeng. Zhongli Li, Chao Li, Yunbo Cao, and Jinsong Su. 2022. Type- driven multi-turn corrections for grammatical error correction. In Findings of the Association for Com- putational Linguistics: ACL 2022, pages 3225-3236.
Hyung Won Chung. Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus. Yunxuan Li, Xuezhi Wang, Mostafa Dehghami, Siddhartha Brahma, Al- bert Webson, Shixiang Shane Gu, Zhuyun Dai, Mirac Suzgun, Xinyun Chen, Aakanksha Chowdh- ery, Alex Castro-Ros, Marie Pellat, Kevin Robinson, Dasha Valter, Sharan Narang, Gaurav Mishra, Adams Yu, Vincent Zhao, Yanping Huang, Andrew Dai, Hongkun Yu. Slav Petrov, Ed H. Chi, Jeff Dean, Ja- cob Devlin, Adam Roberts, Denny Zhou, Quoe V. Le, and Jason Wei. 2022. Scaling instruction-finttueed language models.
Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, and Luke Zettlemoyer. 2020. BART: Denoising sequence-to-sequees for natural language generation, tramslation, and com- prehension. In Proceedings of the 58th Annual Meet- ing of the Association for Computational Linguistics, pages 7871-7880.
pre-training
Jared Lichtarge. Chris Alberti, Shankar Kumar, Noam Shazeer, Niki Parmar, and Simon Tong. 2019. Cor- pora generation for grammatical error correction. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computa- tional Linguistics: Human Language Technologies, pages 3291-3301.
Steven Coyne, Keisuke Sakaguchi, Diana Galvan-Sosa, Michael 7ock, and Kentaro Inui. 2023. Analyzing the performance of GPT-3.5 and GPT-4 in grammati- cal error correction.
Yiming Cui, Wanxiang Che, Ting Liu, Bing Qin, Shijin Wang. and Guoping Hu. 2020. Revisiting pre- trainre models for Chinese natural language processing. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: Findings, pages 657-668.
Yu Meng, Yunyi Zhang, Jiaxin Huang, Chenyan Xiong, Heng Ji. Chao Zhang. and Jiawei Han. 2020. Text classification using label names only: A language model self-training approach. In Proceedings of the 2020 Conference on Empirical Methods Natural Language Processing (EMNLP)), pages 9006-9017.
Daniel Dahlmeier and Hwee Tou Ng. 2012. Better evaluation for grammatical error correction. In Pro- ceedings of the 2012 Conference of the North Amer- ican Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 568-572.
Tomoya Mizumoto, Mamoru Komachi, Masaaki Nagata, and Yuji Matsumoto. 2011 I. Mining revision log of language learning SNS for automated Japanese error correction of second language learners. In Proceed- ings of 5th International Joint Conference Natural Language Processing, pages 147-155.
Daniel Dahlmeier, Hwee Tou Ng. and Siew Mei Wu. 2013. Building a large annotated corpus of learner English: The NUS corpus of learner English. In Proceedings of the Eighth Workshop on Innovative Use of NLP for Building Educational Applications, pages 22-31.
Courtney Napoles. Maria Nädejde, and Joel Tetreault. 2019. Enabling Robust Grammatical Error Correc- tion in New Domains: Data Sets, Metrics, and Anal- yses. Transactions of the Association for Computa- lional Linguistics, pages 551 551-566.

---

Hwee Tou Ng. Siew Mei Wu, Ted Briscoe, Christian Hadiwinoto, Raymond Hendy Susanto, and Christo- pher Bryant. 2014. The CoNI1--2014 shared task on grammatical error correction. In Proceedings of the Eighteenth Conference on Computational Natural Language Learning: Shared Task, pages 1-14.
Felix Stahlberg and Shankar Kumar. 2021. Synthetic data generation for grammatical error correction with tagged corruption models. In Proceedings of the 16th Workshop on Innovative Use of NLP for Building Educational Applications, pages 37-47.
Xin Sun, Tao Ge, Shuming Ma, Jingjing Li, Furu Wei, and Houfeng Wang. 2022. A unified strategy for multilingual grammatical error correction with pre- trained cross cross-lingual language model. In Proceed- ings of the Thirty-First International Joint Confer- ence on Artificial Intelligence, IJCAI-22, pages 4367- 4374.
Hwee Tou Ng. Siew Mei Wu, Yuanbin Wu, Christian Hadiwinoto, and Joel Tetreault. 2013. The CoNLL- 2013 shared task on grammatical error correction. In Proceedings of the Seventeenth Conference on Computational Natural Language Learning: Shared Task, pages 1-12.
Jingcheng Niu and Gerald Penn. 2020. Grammaticality and language modelling. In Proceedings of the First Workshop on Evaluation and Comparison of NLP Systems, pages 110-119.
Ashish Vaswani. Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, F. ukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in Neural Information Pro- cessing Systems, pages 5998 6008.
Muhammad Qorib, Seung-Hoon Na, and Hwee Tou Ng. 2022. Frustratingly casy system combination for grammatical error correction. In Proceedings of the 2022 Conference of the North American Chap- ter of the Association for Computational Linguistics: Human Language Technologies, pages 1964-199 1974.
Xiuyu Wu and Yunfang Wu. 2022. From spelling to grammar: A new framework for Chinese grammati- cal eπrоr correction. In Findings of the Association for Computational Linguistics: EMNLP 2022, pages 889-902.
Colin Raffel, Noam Shazeer, Adam Roberts, Kather- ine Lee, Sharan Narang, Michael Matena. Yanqi Zhou, Wei Li, and Peter J. Liu, 2020. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of Machine Learning Research, pages 1-67.
Junyuan Xie, Ross Girshick, and Ali Farhadi. 2016. Unsupervised deep embedding for clustering analysis. In Proceedings of the 33rd International Conference on International Conference on Machine Learning- Volume 48, ICML' 16. page 478-487
Qizhe Xie, Monh-Thang Luong, Eduard H. Hovy, and Quoe V. Le. 2020. Self-training with noisy student improves imagenet classification. 2020 IEFF/CVF Conference on Computer Vision and Pattern Recog- nition, CVPR 2020, Seatlle WA, USA, June 13-19, 2020, pages 10684-10695. Computer Vision Founda- tion IEEE.
Sascha Rothe, Jonathan Mallinson, Eric Malmi, Sebas- tian Krause, and Aliaksei Severyn. 2021. A simple recipe for multilingual grammatical error correction. In Proceedings of the 59th Annual Meeting of the As- sociation for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, pages 702-707.
Ziang Xie, Guillaume Genthial, Stanley Xie, Andrew Ng. and Dan Jurafsky. 2018. Noising and Denois- ing Natural Language: Diverse Backtranslation for Grammar Correction. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume (Long Papers). pages
Alla Rozovskaya and Dan Roth. 2019. Grammar Error Correction in Morphologically Rich Languages: The Case of Russian. Transactions of the Association for Compustational Linguistics, 7:1-17.
Holger Schwenk, Guillaume Wenzek, Sergey Edunov. Bdouard Grave, Armand Joulin, and Angela Fan. 2021. CCMatrix: Mining billions of high-quality parallel sentences on the web. In Proceedings of the 59th Annual Meeting of the Association for Compu- tational Linguistics and the 1 11th International Joint Conference on Natural Language Processing (Vol- ume 1: Long Papers), pages 6490-6500.
Helen Yannakoudakis, Ted Briscoe, and Ben Medlock. 2011. A new dataset and method for automatically grading ESSL texts. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, pages 180-189.
Yunfan Shao, Zhichao Geng, Yitao Liu, Junqi Dai, Fei Yang. Li Zhe, Hujun Bao, and Xipeng Qiu.
2021. CPT: A pre-trained unbalanced transformer for both chinese language understanding and generation. CoRR.
Michihiro Yasunaga, Jure Leskovee, and Percy Liang. 2021. LM-eritic: Language models for unsupervised grammatical error correction. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 7752-7763.
Alexey Sorokin. 2022. Improved grammatical error cor- rection by ranking elementary edits. In Proceedings of the 2022 Conference on Empirical Methods in Nat- ural Language Processing, pages 11416-11444 11429, Abu Dhabi, United Arab Emirates.
Michihiro Yasunaga and Percy Liang. 2021. Break-it- fix-it: Unsupervised learning for program repair. International Conference on Machine Learning.

---

Zheng Yuan and Mariano Felice. 2013. Constrained Grammatical Error Correction using Statistical Ma- chine Translation. In Proceedings of the Seventeenth Conference on Computational Natural Language Learning: Shared Task, pages 52-61.
UN Parallel Corpus v1.0 (Ziemski et al., 2016) for English and Chinese, respectively. Error type creation. We use the ERRANT toolkit' to extract edits. Specifically, we use the 'all-split' configuration. which merges nothing, when extracting edit pairs from the labeled data. In this way, both the target side and the source side of an edit pair contain most one token. If the source side of an edit pair is empty, the edit cat- egorized as an insertion error. If the target side of an edit pair is empty, the edit is categorized as a deletion error. For the rest of the cases, the edit is categorized as a replacement error.
Baolin Zhang. 2009. Features and functions of the HSK dynamic composition corpus (in Chinese). In International Chinese Language Education, pages 71 71-79.
Xiang Zhang, Junbo Zhao, and Yann LeCun. 2015. Character-level convolutional networks for text clas- sification. In Advances in Neural Information Pro- cessing Systems, volume 28.
Yue Zhang, Zhenghua Li, Zuyi Bao, Jiacheng Li, Bo Zhang, Chen Li, Fei Huang. and Min Zhang. 2022a. MuCGEC: a multi-reference multi-source evaluation dataset for Chinese grammatical error rection. Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Techh nologies, pages 31 18-3130.
Complete figures. We show the insertion and deletion error pattern for English Figure 7. The insertion and deletion error pattern for Chinese is shown in Figure 8. The replacement error pattern for English is shown in Figure 9. The replacement error pattern for Chinese is shown in Figure 10.
Yue Zhang, Bo Zhang, Zhenghua Li, Zuyi Bao, Chen Li, and Min Zhang. 2022b. SynGEC: syntax-enhanced grammatical error correction with tailored GEC- oriented parser. In Proceedings of the 2022 Con- ference Empiricul Methods in Natural Language Processing, pages 1518-...1.
A.2 Extracting GED Pseudo-Labels from the Fixer
The complete correlation between the probability of producing g(6) and precision of z(0) is shown in Figure 11.
Yuanyuan Zhao, Nan Jiang, Weiwei Sun, and Xiaojun Wan. 2018 Overview of the NLPCC 2018 shared task: Grammatical error correction. In CCF Interna- tional Conference on Natural Language Processing and Chinese Computing, pages 439-445.
A.3 Detailed Experimental Settings
Implementation details and training configura- tion.
We build our fixer using both the fairseq" and transformers toolkit. Specifically, since the Flan- T5-xxl model has around 1 11B parameters, we use the transformers toolkit with DeepSpeed ZeRO- Offload to build the fixer for English and use the fairseq toolkit to build the rest of the components. For English GEC, we use 32 NVIDIA A100 GPUs. For Chinese GEC,, we use 8 NVIDIA A100 GPUs. The experiments took 14 days for English and 2 days in total for Chinese. We use the default train- ing configuration under different toolkits unless otherwise stated. The detailed training configura- tions for English and Chinese are shown in Table and Table 9, respectively. The best checkpoint is selected based on the performance on the val- idation set. Specifically, when building the fixer, we follow Yasunaga and Liang (2021) to randomly sample 5,000 sentences from the obtained training
Zewei Zhao and Houfeng Wang. 2020. Maskgec: Im- proving neural grammatical error correction via dy- namic masking. Proceedings of the AAAI Conference on Artificial Intelligence, (4(01):1226-1 1233.
Michat Ziemski, Marcin Mannzys-Dowmiln and Bruno Pouliquen. 2016. The United Nations parallel cor- pus v1.0. In Proceedings of the Tenth International Conference Language Resources and Evaluation (I.REC pages 3530-3534.
A Appendix
A.1 Details on Exploiting Error Patterns
Validation set selection. We carry out error pat- tern analysis on the validation set. Specifically, we follow previous work (Cao et al.. 2023; Wu and Wu, 2022) to use BEA-2019 dev set (Bryant et al., 2019) and randomly sample 5,000 sentences from the NLPCC-2018 training set (Zhao et al., 2018) as the validation set for English and Chinese, respec-
tively.
8 hhtps://giiuub.coom//o com//faebookkrseack/iiiee wwwp://wwwww.uggggggscaannntttno http://hhw....mmmmeeoss/////pppp
Vocabulary creation. We derive the vocabulary from the C4 corpus (Raffel et al., 2020) and the

---

sentence pairs the validation data for both En- glish and Chinese. When building the critic, we follow the approach used by Yasunaga et (2021) to randomly select 600 grammatical sentences and 600 ungrammatical sentences from the BEA-2019 dev set and Chinese Lang8 dataset as the validation set for English and Chinese, respectively.
FERLO-MERLIN (P/R/Fo.s)
RULEC-GEC
System
(P/R/Fo.s)
Our Unsupervised System
Initial fixer
74.3/50.1/67 I/67.8
55.8/22.0042.6
Hyper-paramete settings. We tune two hyper- parameters in our system, the edit distance thresh- old, as mentioned in $3.2.2, and the masking per- centage, denoted as p%, which is outlined in $3.3. We select the edit distance threshold from {1, 2. 3, 4, 5) for English GEC and select the the edit distance threshold from {0, 1, 2 for Chinese. For both English and Chinese p is selected from 15, 10, 15). For English, the edit distance threshold 2 and p equals 5% give the best performance the vali- dation set. For Chinese, the edit distance threshold and p% equals 10% give the best performance on the validation set.
1st iteration
76.2/64.2/73.4 76.5/67.8 8/74.5
60.1/27.7/48.7
2nd iteration
60.4/30.1/50.3
Supervised SOTA System
Rothe et al. (2021)
-/-/76.0
-/-/51.6
Sorokin (2022)
-1-/-
73.7/27.3/5500
Table 7: Performance (in %) of GEC systems the MIKO-MMRRLN and RULEC- GEC test sets.
Parameters for synthetic data generation. Table 10 shows the parameter values used when generat- ing the synthetic data. Note that these values are set to mimic the error distribution in real erroneous corpora.
Configuration
Value
Fixer
Devices
32 A100 GPU
Batch Size per GPU
256
A.4 Experiments on German and Russian
Update Frequency
label smoothed cross entropy -0oothin)o)))
We use German (Falko-MERLINN dataset) and Rus- sian (RULEC-GEC dataset) demonstrate our method's performance in additional languages.
Loss function
Model architecture
Flan-T5-xx Tan-T5-xxl
Adamw
Optimizer
(8: =0,, 00.99,
For both languages, we use mT5-xxl instead of Flan-T5-xx xxl as the base model and generate 10 million synthetic sentence pairs by corrupting the sentences from UN-Corpus v1.0. Following the setup in Section 4.1 and Section 5.1, we randomly collect 10 million sentences from the CCMatrix (Schwenk et al., 2021) corpus, Falko-MERLIN (Boyd et al., 2014) dataset, and (Rothe et al., 2021) dataset for German. For both Falko- MERLIN dataset and cLang8 dataset, we take the sentences from the source side (not annotated sen- tences), which could be grammatical or ungram- matical. We randomly collect 10 million sentences from the CCMatrix (Schwenk et al., 2021) cor- pus, RULEC-GEC (Rozovskaya and Roth, 2019) dataset, and cLang8 (Rothe al., 2021) dataset for Russian. For both RULEC-GER GEC dataset and cLang8 dataset, we also take the sentences from the source side. The results are shown in the Ta- ble 7. Note that no unsupervised baselines exist in German and Russian GEC.
10
Learning rate
2.00 x 10
Learning rate scheduler Warmup
Linear
0
Number of epochs
10
Critic
Devices
NVVDDA A10O GPU
Batch Size per GPU
10000
Update Frequency
Loss function
cross entropy
Model architecture
RoBERTa-base
Adam
Optimizer
(3: 0.9, 9g 92 0.98, c 1 x 10 10-9)
Learning rate Learning rate scheduler
1.00 x 10
Polynomial decay
Warmup
400
Number of epochs
Table 8: Experimental configuration English.

---

3000
Insert
2500
Delete
Frequency
1500
1000
500
[300, 95)
[95, 90)
(90,85)
[8S 80)
[80,
[75,70]
[20,65
65)
[65,60)
[60,55)
[55,50]
[s0, (5)
[45 40)
[40,35] 35)
[35, 30)
[30,28]
[25,20)
[20,19 (..
[25,10)
[10,5)
(5.0)
Percentile
Figure 7: The erroneous token distribution for insertion and deletion errors for English. The tokens in the vocabulary are ordered by decreasing frequency.
Delete
2000
Frequency
1500
1000
500
[100.90 95)
195,9
[90,
(00,55)
(85
80)
(00, 75)
175,
(70,65)
165
60)
(60,55)
[55, 50)
[50
(45, (45,40)
[40
35)
[35,
(35,30)
[30,25)
[25
(20.
[15,10]
[30.9]
[5.0]
Percentile
Figure 8: The erroncous token distribution for insertion and deletion errors for Chinese, The tokens in the vocabulary ordered by decreasing frequency.
2000
1500
Frequency
1000
500
2
5
6
9
10
11
12
13
14
Edit Distance
Figure The character-level edit distance between an erroneous token and corresponding target token for replacement error for English.

---

3500
3000
2500
Frequency
2000
1500
1000
500
4
5
Edit Distance
Figure 10: The character-level edit distance between an erroncous token and its corresponding target token for replacement enrrr for Chinese.
Grammatial
Grammatical Ungrammatica cal
Precision
Precision
23
0.3
0.4
0.7
0.1
0.2
0.5
0.6
0.7
0.8
0.9
Probability
Probability
Figure 11: Correlation between the probability of producing g(0 and precision 20) Left: English; Right: Chinese.

---

Configuration
Value
Fixer
Devices
NVIIDA A10 GPU
Max tokens per GPU
7000
Update Frequeney
label smoothed cross entropy mabel-smmothing-O.1)
Loss function
Model architecture
Chinese BART-large
Adam
Optimizer
(81 = 0.9, =
< 10-9)
Learning rate
1.00 x 1
Learning rate
Polynomial decay
scheduler
Warmup
Number of epochs
15
Critic
Devices
NVIDIA A GPU
Max tokens per GPU
10000
Update Frequency
Loss function
cross entropy
Model architecture
RoBERTa-wwm-ext
Adam
Optimizer
(8i 0.9, 0.98,
<= c=1x100 1011x1055) 10-5)
Learning rate
1.00 x 10
Learning rate
Polynomial decay
scheduler
Warmup
400
Number epochs
40
Table 9: Experimental configuration on Chinese.

---

English
Chinese
Pdet
0.15
0.15
Pins
0.35
0.35
Prep
0.50
0.50
ertor count
multinoulli (0.05, 0.07, 0.25, 0.35, 0.28)
multinoulli (0.01, 0.32, 0.29, 0,20, 0.18)
distribution
LISTTD
[5, 10, 40, 80, 200, 500, 1000, 28001
[35, 95, 187, 274, 561, 787, 1176, 1995]
Table 10: Parameters for synthetic data generation.


Reviews:
[{"review_id": "nuG3Se6Fhz", "reviewer": "Reviewer_XoJu", "summary": "", "questions": "A) I doubt your method will work for low-resource languages since you have no FLAN-XXL for these languages. Even the analogue of BART-base would be of much lower quality or your would have to use mBART.\nB) is it true that your method of token replacement works only for the words that consist of single BPE token?\nC) I do not understand the intuition behind soft labeling in formula (2), in particular, the squaring of probabilities. Can you briefly explain it inside your work?", "limitations": "", "rating": 4, "confidence": 5, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you very much for the positive review. Here is the response to the questions: \n\nQuestion A:\nWe use German (Falko-MERLIN dataset) and Russian (RULEC-GEC dataset) to demonstrate our model's performance in low-resource languages. German and Russian have been widely used to evaluate low-resource languages’ performance in GEC.\n\nFor both languages, we use mT5-xxl instead of Flan-T5-xxl as the base model and generated 10 million synthetic sentence pairs by corrupting the sentences from UN-Corpus v1.0. Following the setup in Section 4.1 and Section 5.1, we randomly collect 10 million sentences from the CCMatrix corpus, Falko-MERLIN dataset, and clang8 dataset for German. For both Falko-MERLIN dataset, and clang8 dataset we take the sentence from the source side (not annotated sentences), which could be grammatical or ungrammatical. We randomly collect 10 million sentences from the CCMatrix corpus, RULEC-GEC dataset, and clang8 dataset for Russian. For both RULEC-GEC dataset and clang8 dataset, we also take the sentence from the source side. The results are shown in the table below (No unsupervised baselines exist in the German and Russian GEC).\n| | German (P/R/F0.5) | Russian (P/R/F0.5) |\n|-----------|-----------|-----------|\n|Our Unsupervised System|\n| Pretrain | 74.3/50.1/67.8 | 55.8/22.0/42.6 |\n| Stage1 | 76.2/64.2/73.4 | 60.1/27.7/48.7 |\n| Stage2 | 76.5/67.8/74.5 | 60.4/30.1/50.3 |\n|Supervised SOTA Systems|\n| gT5-xxl (Rothe et al., 2021) | -/-/76.0 | -/-/51.6 |\n| EditScorer (Sorokin, 2022) without editscorer | -/-/- | 65.7/27.4/51.3 |\n\nThis result demonstrates that our method works well on low-resource languages. In particular, our unsupervised F0.5 scores are in close proximity to strong supervised results (Rothe et al., 2021, Sorokin, 2022). The best model for Russian GEC is (Sorokin, 2022), which proposed an editscorer on top of the GEC model. In the table below, we demonstrate that when we combine our unsupervised model with the editscorer, we can get a comparable performance. \n| | Russian (P/R/F0.5) |\n|-----------|-----------|\n|Our Unsupervised System|\n| Stage2 | 60.4/30.1/50.3 |\n|   +editscorer | 70.3/28.9/54.6 |\n|Supervised SOTA Systems|\n| EditScorer (Sorokin, 2022) with editscorer | 73.7/27.3/55.0 |\nThese results demonstrate that our unsupervised system compares favorably with strong supervised SOTA systems on low-resource languages. \n\nQuestion B:\nNo, the token replacement operation works for both single-token words and multi-token words. Our token replacement operation masks a word and uses the RoBERTa model to generate corruption. Therefore, the token replacement operation also works for multi-token words.\n \nQuestion C:\nYes, we will include it in the revision. The reason for squaring the probability is that it has been normalized by the frequency per class f_c as shown in Equation 2. The intuition for Equation 2 is described in Xie et al. (2016)’s work in Section 3.1.2. The soft-frequency formula is designed to:\n1. Strengthen predictions.\n2. Place more emphasis on data points with high confidence.\n3. Normalize the loss contribution of each centroid to prevent large clusters from distorting the hidden feature space.\n\nMissing Reference: \nThank you very much, we will update Table 1 and its description paragraph in Section 4.2 accordingly.\n \nReferences:\n\nJunyuan Xie, Ross Girshick, and Ali Farhadi. Unsupervised deep embedding for clustering analysis. In ICML 2016.\n\nSascha Rothe, Jonathan Mallinson, Eric Malmi, Sebastian Krause, and Aliaksei Severyn. 2021. A simple recipe for multilingual grammatical error correction. In ACL 2021.\n\nAlexey Sorokin. Improved grammatical error correction by ranking elementary edits. In EMNLP 2022.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "* SOTA in unsupervised grammatical error correction for English and Chinese\n* a new method that does not utilize any manual effort for creating the unsupervised data is proposed\n* good analysis and ablations", "reasons_to_reject": "No reasons", "questions_to_authors": "A) I doubt your method will work for low-resource languages since you have no FLAN-XXL for these languages. Even the analogue of BART-base would be of much lower quality or your would have to use mBART.\nB) is it true that your method of token replacement works only for the words that consist of single BPE token?\nC) I do not understand the intuition behind soft labeling in formula (2), in particular, the squaring of probabilities. Can you briefly explain it inside your work?", "paper_topic_and_main_contributions": "The paper proposes a new method of unsupervised grammatical error correction. In comparison to the previous approach of (Yasunaga et al., 2021) it uses a more natural method of text corruption, thus significantly improving the model quality. When using a Flan-XXL fixer, the proposed model almost reaches the level of supervised models. Using the same fixer that (Yasunaga et al., 2021) it also provides a slight improvement over previous results.", "excitement": 4, "reproducibility": 3, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Missing_References": "The best single system on BEA-2019 is (Sorokin, 2022) https://aclanthology.org/2022.emnlp-main.785.pdf, its score is 77.1. Please refer to it in Table 1 and correct the last paragraph of Section 4.2"}, "extra_text": {}}, {"review_id": "IaoWYE6LuS", "reviewer": "Reviewer_fd3a", "summary": "", "questions": "- A. Table 3 and 5 present some intuitive cases produced by different methods. How were these examples selected? Are they representative enough of the behavior of each method?\n- B. Line 298: The [MASK] token is used to introduce noise in the self-training process. Is it different from other specific tokens regarding injecting noise?\n- C. Line 312: $f_c$ represents the sum over soft frequencies for class $c$, but in the previous section, $f$ is used to indicate the fixer. In my understanding, $f_c$ should be unrelated to the fixer since it is conditional on the critic model parameter. Did you misuse the $f$ symbol?", "limitations": "", "rating": 3, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you very much for the review suggestions. Here is our response to the “reasons to reject” and to the questions. \n \nNovelty: we want to clarify that our method has four innovative contributions:\n1. Although both LM-Critic and our method require a fixer and a critic, we define the critic in a completely different way. Different from LM-Critic which defines the critic from GPT2 and a confusion set, we train and refine the critic from the fixer. This design has two advantages. First, our method doesn’t rely on the confusion set, making it fully unsupervised. Second, the fixer and the critic can iteratively improve each other. As illustrated in Figure 5, this further improves the fixer’s eventual performance.\n2. Our synthetic data generation method includes some important new techniques. For example, in using masked language models (MLM) to generate synthetic data, Sun et al. (2022) present a similar idea, but we are the first to propose to use edit-distance information and high-frequency tokens to filter the generated errors. As shown in Table 2, this simple technique has been proven to be extremely important to the fixer’s performance. \n3. Our paper is the first to demonstrate that scaling up the backbone model size (BART-base -> T5-large (we provide this model’s performance below) -> Flan-T5-xxl) can greatly improve unsupervised performance. This is non-trivial, because in the supervised GEC setting, people have found that the model size is not that important. For example, comparing Zhang et al. (2022) (BART-large) and Rother et al. (2021) (T5-xxl), the performance only differs by 1.9 points on the CoNLL-2014 test set. On the other hand, we have shown in Table 1 that our method significantly outperforms ChatGPT and GPT4, showing that good results not only require a big backbone model, but also a carefully designed training method proposed by us.\n4. Our method is the first unsupervised method that rivals the performance of supervised SOTA systems, not only in high-resource languages (English, Chinese), but also in low-resource languages (German and Russian). German and Russian have been widely used to evaluate low-resource languages’ performance in GEC. Their performance is shown below.\n\nThe performance of T5-large using our unsupervised method is shown here: \n\n| | CoNLL-2014 (P/R/F0.5) | BEA-2019 (P/R/F0.5) |\n|-----------|-----------|-----------|\n| Pretrain | 67.3/38.5/58.5 | 71.3/45.2/63.9 |\n| Stage1 | 70.3/44.3/62.9 | 74.4/55.3/69.6 |\n| Stage2 | 72.3/46.0/64.8 | 76.6/56.3/71.4 |\n| Stage3 | 72.0/50.4/66.3 | 75.5/61.6/72.3 |\n\nThe unsupervised performance on German and Russian is shown here: \n\nWe use German (Falko-MERLIN dataset) and Russian (RULEC-GEC dataset) to demonstrate our model's performance in low-resource languages. For both languages, we use mT5-xxl instead of Flan-T5-xxl as the base model and generated 10 million synthetic sentence pairs by corrupting the sentences from UN-Corpus v1.0. Following the setup in Section 4.1 and Section 5.1, we randomly collect 10 million sentences from the CCMatrix corpus, Falko-MERLIN dataset, and clang8 dataset for German. For both Falko-MERLIN dataset and clang8 dataset, we take the sentence from the source side (not annotated sentences), which could be grammatical or ungrammatical. We randomly collect 10 million sentences from the CCMatrix corpus, RULEC-GEC dataset, and clang8 dataset for Russian. For both RULEC-GEC dataset, and clang8 dataset we also take the sentence from the source side. The results are shown in the table below (No unsupervised baselines exist in the German and Russian GEC).\n| | German (P/R/F0.5) | Russian (P/R/F0.5) |\n|-----------|-----------|-----------|\n|Our Unsupervised System|\n| Pretrain | 74.3/50.1/67.8 | 55.8/22.0/42.6 |\n| Stage1 | 76.2/64.2/73.4 | 60.1/27.7/48.7 |\n| Stage2 | 76.5/67.8/74.5 | 60.4/30.1/50.3 |\n|Supervised SOTA Systems|\n| gT5-xxl (Rothe et al., 2021) | -/-/76.0 | -/-/51.6 |\n| EditScorer (Sorokin, 2022) without editscorer | -/-/- | 65.7/27.4/51.3 |\n\nThis result demonstrates that our method works well on low-resource languages. In particular, our unsupervised F0.5 scores are in close proximity to strong supervised results (Rothe et al., 2021, Sorokin, 2022). The highest model for Russian GEC is (Sorokin, 2022), which proposed an editscorer on top of the GEC model. In the table below, we demonstrate that when we combine our unsupervised model with the editscorer, we can get a comparable performance. \n| | Russian (P/R/F0.5) |\n|-----------|-----------|\n|Our Unsupervised System|\n| Stage2 | 60.4/30.1/50.3 |\n|   +editscorer | 70.3/28.9/54.6 |\n|Supervised SOTA Systems|\n| EditScorer (Sorokin, 2022) with editscorer | 73.7/27.3/55.0 |\n\nThese results demonstrate that our unsupervised system compares favorably with strong supervised SOTA systems on low-resource languages. \n\n\nScalability & Robustness: the reviewer questions whether the methods and the error patterns proposed in the paper work across different domains and languages. Here we present some additional analysis to prove that it is true.\n\nFirst, we test the two error patterns proposed in the paper: \n1. Most deletion and insertion errors occur within 95th percentile of the vocabulary.\n2. Most replacement errors have small (<5) edit distance.\n\nWe analyze three additional commonly used English GEC evaluation datasets across different domains: JFLEG, CoNLL-2013, and the CWEB dataset. In the table below, we show how much of the erroneous token lies within the first 95 percentile of the vocabulary for deletion and insertion errors. We also show how much percentage of the edit pairs has an edit distance less than 5 for the replacement error. The result matches our proposed patterns.\n| | Deletion & Insertion | Replacement |\n|-----------|-----------|-----------|\n| JFLEG | 93% | 90% |\n| CoNLL-2013 | 90% | 88% |\n| CWEB | 96% | 94% |\n\nWe now analyze two representative low-resource languages in GEC: German (Falko-MERLIN dataset) and Russian (RULEC-GEC dataset). The table below shows how much of an erroneous token lies within the top 95 percentile of the vocabulary for deletion and insertion errors, as well as the percentage of edit pairs with an edit distance of less than 5 for replacement errors. The results reveal patterns similar to those observed in English and Chinese.\n| | Deletion & Insertion | Replacement |\n|-----------|-----------|-----------|\n| German | 89% | 88% |\n| Russian| 90% | 89% |\n\nOur synthetic data generation method also shows competitive performance against related works in German and Russian. Following the setup in Section 4.3, we compare our synthetic data generation method with Sun et al. (2022) and Grundkiewicz and Junczys-Dowmunt. (2019). The results are shown in the following table. It shows that our synthetic data generation method works well for low-resource languages and outperforms existing methods.\n| | Deletion & Insertion | Replacement |\n|-----------|-----------|-----------|\n| Spellchecker (Grundkiewicz and Junczys-Dowmunt. (2019)) | 24.8/13.7/21.3 | 15.4/11.1/14.2 |\n| Translation pair (Sun et al. (2022)) | 28.8/12.4/22.8 | 16.7/11.4/15.2 |\n| Ours| 32.0/14.0/25.5 | 19.8/13.8/18.2 |\n\nFinally, we test the F0.5 scores of the unsupervised GEC system on German (Falko-MERLIN dataset) and Russian (RULEC-GEC dataset) end-to-end. The results shown in the response for “Novelty” demonstrate that our unsupervised system compares favorably with strong supervised SOTA systems on low-resource languages. \n\nAll these analyses show that our method is scalable, robust, and works effectively across different languages. \n\n\nQuestions:\n1. The cases presented in Table 3 and Table 5 are randomly selected. Due to space limitation, we only show a few representative examples produced by different systems. \n2. In our experiment, we tried using both [MASK] and [UNK] tokens during the self-training process. There is not much difference between the critic’s and fixer’s performance when using either [MASK] or [UNK] token. For consistency consideration, we use the [MASK] token. \n3. Thanks for pointing out this problem. Yes, we misused the f symbol in line 312, f should be used to indicate the fixer. We will change this symbol in line 312 to avoid the confusion. \n\nTypos:\nThanks for spotting these problems, we will correct them. \n \nReferences:\n\nXin Sun, Tao Ge, Shuming Ma, Jingjing Li, Furu Wei, and Houfeng Wang. A unified strategy for multilingual grammatical error correction with pre-trained cross-lingual language model. In IJCAI 2022.\n\nRoman Grundkiewicz and Marcin Junczys-Dowmunt. Minimally-Augmented Grammatical Error Correction. In W-NUT 2019.\n\nYue Zhang, Bo Zhang, Zhenghua Li, Zuyi Bao, Chen Li, and Min Zhang. SynGEC: Syntax-enhanced grammatical error correction with a tailored GEC oriented parser. In EMNLP 2022.\n\nSascha Rothe, Jonathan Mallinson, Eric Malmi, Sebastian Krause, and Aliaksei Severyn. 2021. A simple recipe for multilingual grammatical error correction. In ACL 2021.\n\nAlexey Sorokin. Improved grammatical error correction by ranking elementary edits. In EMNLP 2022.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- **Performance**: The proposed method demonstrates superior performance compared to the fundamental LM-Critic approach. The experimental results are promising, showing comparable performance to supervised methods, which highlights the effectiveness of the new synthetic data generation and training techniques.\n- **Motivation**: The motivation behind adopting most components of the fixer-critic training loop from the mature LM-Critic framework is sound. Additionally, the new perturbation method based on empirical observations of error patterns adds credibility to the proposed approach.", "reasons_to_reject": "- **Novelty**: While the experimental results are encouraging, the high-level design of the LM-Critic framework remains largely unchanged. The primary contribution lies in providing improved synthetic data generation methods for training the fixer and critic, limiting the novelty of the paper.\n- **Scalability**: The empirical observation of error patterns may pose challenges when dealing with new languages and benchmarks. Analyzing error patterns could result in additional development costs and may not be easily scalable to other contexts.\n- **Robustness**: There is a concern about the vulnerability of the proposed generation method if the validation data does not highly represent the error patterns of other data splits. This could impact the robustness of the system in different scenarios.\n\n**Update:** I increased my soundness score after reading the reply. Please see my follow-up comment below.", "questions_to_authors": "- A. Table 3 and 5 present some intuitive cases produced by different methods. How were these examples selected? Are they representative enough of the behavior of each method?\n- B. Line 298: The [MASK] token is used to introduce noise in the self-training process. Is it different from other specific tokens regarding injecting noise?\n- C. Line 312: $f_c$ represents the sum over soft frequencies for class $c$, but in the previous section, $f$ is used to indicate the fixer. In my understanding, $f_c$ should be unrelated to the fixer since it is conditional on the critic model parameter. Did you misuse the $f$ symbol?", "paper_topic_and_main_contributions": "This paper presents an unsupervised pipeline for training a grammatical error correction (GEC) system using unlabeled data. In comparison to the LM-Critic framework, the primary contribution of this work lies in the integration of several techniques for synthetic data generation into the existing framework, thereby taking a step towards eliminating the reliance on external sources of labeled data. Specifically, the proposed method leverages the empirical distribution of error patterns extracted from the developmental data to corrupt the data through three token-level edit operations, including replacement, deletion, and insertion. A masked language model (MLM) is employed in this process to handle token replacement. Additionally, the paper introduces self-knowledge distillation in the LM-Critic training loop to enhance the model's generalization ability. Based on the experiments conducted, the new training pipeline outperforms the unsupervised baselines, as well as the vanilla LM-Critic pipeline.", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Typos_Grammar_Style_and_Presentation_Improvements": "- Line 280: In Figure $4^4$ -> In Figure 4\n- Line 394: clang8 -> cLang8\n- Line 433: Table 4.6. -> Table 4."}, "extra_text": {}}, {"review_id": "MgDbp0FMtq", "reviewer": "Reviewer_Y4qz", "summary": "", "questions": "1. In Table 1, can you provide the performance of using T5 with different model sizes (e.g., T5-base or T5-large)?\n2. In Figure 5, can you offer any insightful explanations about why the performance of your fixer continuously increases while other counterparts don't?", "limitations": "", "rating": 3, "confidence": 4, "soundness": 4, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you very much for the review suggestions. Here is our response to the “reasons to reject” and to the questions. \n \nFor the novelty of the paper, we want to clarify that we have made 4 substantial innovative contributions beyond a “straightforward extension of LM-Critic”: \n1. Although both LM-Critic and our method require a fixer and a critic, we define the critic in a completely different way. Different from LM-Critic which defines the critic from GPT2 and a confusion set, we train and refine the critic from the fixer. This design has two advantages. First, our method doesn’t rely on the confusion set, making it fully unsupervised. Second, the fixer and the critic can iteratively improve each other. As illustrated in Figure 5, this further improves the fixer’s eventual performance.\n2. Our synthetic data generation method includes some important new techniques. For example, in using masked language models (MLM) to generate synthetic data, Sun et al. (2022) present a similar idea, but we are the first to propose to use edit-distance information and high-frequency tokens to filter the generated errors. As shown in Table 2, this simple technique has been proven to be extremely important to the fixer’s performance. \n3. Our paper is the first to demonstrate that scaling up the backbone model size (BART-base -> T5-large (we provide this model’s performance below) -> Flan-T5-xxl) can greatly improve unsupervised performance. This is non-trivial, because in the supervised GEC setting, people have found that the model size is not that important. For example, comparing Zhang et al. (2022) (BART-large) and Rother et al. (2021) (T5-xxl), the performance only differs by 1.9 points on the CoNLL-2014 test set. On the other hand, we have shown in Table 1 that our method significantly outperforms ChatGPT and GPT4, showing that good results not only require a big backbone model, but also a carefully designed training method proposed by us.\n4. Our method is the first unsupervised method that rivals the performance of supervised SOTA systems, not only in high-resource languages (English, Chinese), but also in low-resource languages (German and Russian). German and Russian have been widely used to evaluate low-resource performance in GEC. Their performance is shown below.\n\n\nThe performance of T5-large using our unsupervised method is shown here: \n| | CoNLL-2014 (P/R/F0.5) | BEA-2019 (P/R/F0.5) |\n|-----------|-----------|-----------|\n| Pretrain | 67.3/38.5/58.5 | 71.3/45.2/63.9 |\n| Stage1 | 70.3/44.3/62.9 | 74.4/55.3/69.6 |\n| Stage2 | 72.3/46.0/64.8 | 76.6/56.3/71.4 |\n| Stage3 | 72.0/50.4/66.3 | 75.5/61.6/72.3 |\n\n\nThe unsupervised performance on German and Russian is shown here: \nFor both German (Falko-MERLIN dataset) and Russian (RULEC-GEC dataset) we use mT5-xxl instead of Flan-T5-xxl as the base model and generated 10 million synthetic sentence pairs by corrupting the sentences from UN-Corpus v1.0. Following the setup in Section 4.1 and Section 5.1, we randomly collect 10 million sentences from the CCMatrix corpus, Falko-MERLIN dataset, and clang8 dataset for German. For both Falko-MERLIN dataset and clang8 dataset ,we take the sentence from the source side (not annotated sentences), which could be grammatical or ungrammatical. We randomly collect 10 million sentences from the CCMatrix corpus, RULEC-GEC dataset, and clang8 dataset for Russian. For both RULEC-GEC dataset, and clang8 dataset we also take the sentence from the source side. The results are shown in the table below (No unsupervised baselines exist in the German and Russian GEC).\n| | German (P/R/F0.5) | Russian (P/R/F0.5) |\n|-----------|-----------|-----------|\n|Our Unsupervised System|\n| Pretrain | 74.3/50.1/67.8 | 55.8/22.0/42.6 |\n| Stage1 | 76.2/64.2/73.4 | 60.1/27.7/48.7 |\n| Stage2 | 76.5/67.8/74.5 | 60.4/30.1/50.3 |\n|Supervised SOTA Systems|\n| gT5-xxl (Rothe et al., 2021) | -/-/76.0 | -/-/51.6 |\n| EditScorer (Sorokin, 2022) without editscorer | -/-/- | 65.7/27.4/51.3 |\n\nThis result demonstrates that our method works well on low-resource languages. In particular, our unsupervised F0.5 scores are in close proximity to strong supervised results (Rothe et al., 2021, Sorokin, 2022). The highest model for Russian GEC is (Sorokin, 2022), which proposed an editscorer on top of the GEC model. In the table below, we demonstrate that when we combine our unsupervised model with the editscorer, we can get a comparable performance. \n| | Russian (P/R/F0.5) |\n|-----------|-----------|\n|Our Unsupervised System|\n| Stage2 | 60.4/30.1/50.3 |\n|   +editscorer | 70.3/28.9/54.6 |\n|Supervised SOTA Systems|\n| EditScorer (Sorokin, 2022) with editscorer | 73.7/27.3/55.0 |\n\nThese results demonstrate that our unsupervised system compares favorably with strong supervised SOTA systems on low-resource languages. \n\n\nThe reviewer questions whether our method works well for low-resource languages. Our results in the above tables demonstrate that our unsupervised system is competitive with strong supervised SOTA GEC systems in German and Russian (two representative low-resource languages for GEC).\n\n\nThe reviewer also questions whether our method works well for small backbone models. As shown in Table 1 (also line 385), we have used BART-base (220 million parameters) as one of the smaller backbone models to compare our method against LM-Critic. The result shows that our method is 5.3 points and 2.1 points higher than the LM-Critic on the CoNLL-2014 and BEA-2019 test set, respectively. When comparing against humans using the method proposed by Bryant and Ng, (2015), our BART-base model (73.0 F0.5 score) also surpassed human performance (72.5 F0.5 score). This makes ours the first unsupervised system that surpasses human performance. We will add this comparison to the revision.\n\nIn the following table, we show the unsupervised system’s performance based on T5-large (a smaller backbone).\n| | CoNLL-2014 (P/R/F0.5) | BEA-2019 (P/R/F0.5) |\n|-----------|-----------|-----------|\n| Pretrain | 67.3/38.5/58.5 | 71.3/45.2/63.9 |\n| Stage1 | 70.3/44.3/62.9 | 74.4/55.3/69.6 |\n| Stage2 | 72.3/46.0/64.8 | 76.6/56.3/71.4 |\n| Stage3 | 72.0/50.4/66.3 | 75.5/61.6/72.3 |\n\nThe competitive results from T5-large also shows that our method works well under the small backbone model setting. \n  \nQuestions:\n1.   Yes. Our response to the second reason to reject has provided the performance of the T5-large model. \n2.   Yes. Figure 5 shows that the fixer’s performance only marginally improves using either LM-Critic or our critic w/o critic update. This is because the critic is fixed throughout the iteration. Once the critic is not updated, the paired data extracted by the critic in the subsequent iterations may not further benefit the fixer’s training since no extra knowledge has been injected into the system. As a result, the fixer is learning from itself, unintentionally causing the over-fitting problem. This phenomenon can be broken by adding masking-based data augmentation methods and updating the critic throughout the iteration, as shown in Figure 5 and Figure 6.\n\nTypos:\n\nThanks for the suggestion regarding these typos, we will update accordingly. For the third typo, it should be our critic, as labeled in Figure 5. This is because Figure 5 shows the performance of the fixer under different types of critic. \n \nReference:\n\nChristopher Bryant, and Hwee Tou Ng. How Far are We from Fully Automatic High Quality Grammatical Error Correction? In ACL 2015. \n \nAlexey Sorokin. Improved grammatical error correction by ranking elementary edits. In EMNLP 2022.\n\nSascha Rothe, Jonathan Mallinson, Eric Malmi, Sebastian Krause, and Aliaksei Severyn. 2021. A simple recipe for multilingual grammatical error correction. In ACL 2021.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. SoTA performance on both English and Chinese benchmarks.\n2. Clear paper writing.\n3. Reasonable expanded analyses to support the motivation of each component.", "reasons_to_reject": "1. The method the authors proposed seems to be a straightforward extension of the LM-Critic work (Yasunaga et al., 2021). The heuristic rules for generating data for training the initial fixer, the self-knowledge distillation when training the critic, the masked data augmentation when training the critic, and selecting high-confidence pseudo labels when training the critic, collectively form a set of useful techniques. However, in my view, these strategies do not represent a significant novelty.\n2. Despite achieving impressive performance, this work relies heavily on a large backbone model and extensive multi-task pretraining, i.e., Flan-T5-11B. Moreover, the method only yields a modest improvement of approximately 0.5 F score under the supervised setting on both English and Chinese benchmarks. The authors should conduct additional experiments under low-resource settings to further validate the effectiveness of their methods. This could include testing on low-resource languages and smaller backbone models.", "questions_to_authors": "1. In Table 1, can you provide the performance of using T5 with different model sizes (e.g., T5-base or T5-large)?\n2. In Figure 5, can you offer any insightful explanations about why the performance of your fixer continuously increases while other counterparts don't?", "paper_topic_and_main_contributions": "This paper proposes an unsupervised method for grammatical error correction based on the Break-It-Fix-It method. \n\nThe primary contributions of this study lie in the enhancements made to both the 'fixer' and the 'critic' components within the BIFI framework. \n\nFor the fixer, this paper puts forward a simple data augmentation method based on MLM and heuristic rules. For the critic, this paper proposes to select high-confidence samples from the fixer's predictions and use them for training. During training, they also use two simple tricks to further improve performance, i.e., masked data augmentation and self-knowledge distillation.\n\nThe results show their unsupervised method significantly surpasses previous methods, and can further enhance model performance under the supervised setting.", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Typos_Grammar_Style_and_Presentation_Improvements": "1. line 126: confusion set -> confusion sets.\n2. line 139: backtranslation -> back translation\n3. In Figure 5: our critic or our fixer?\n4. All figures in this paper should be in vector graphic format."}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper proposes an unsupervised approach to Grammatical Error Correction (GEC) based on method proposed in Yasunaga et al., (2021). The approach relies on using a large pre-trained language model (Flan T5-xxl) and proposes several novel techniques for synthetic data generation and enhancements to the critic component. Experiments on English and Chinese benchmarks demonstrate superior performance compared to unsupervised baselines and the LM-Critic pipeline.

The paper is clear, well-written, and demonstrates strong results on two languages. Two concerns raised in the reviews are the limited novelty (the paper is an extension of the Yasunaga paper), and the use of large backbone models. It is not clear whether the method can be effective when used with low-resource languages and smaller language models.
Decision: Accept-Main