SlowGuess/NAACL_2021_OCR huggingface dataset:
ranking features and basic GEC score to conduct the ranking score. We assign the hypotheses with the highest  $\mathrm{F}_{0.5}$  score as positive instances and the others as negative ones. The Coordinate Ascent method is implemented by RankLib<sup>1</sup>.
We conduct experiments to study the performance of VERNet from three aspects: token-level quality estimation, sentence-level quality estimation, and the VERNet's effectiveness in GEC models. Then we present the case study to qualitatively analyze the effectiveness of the proposed two types of attention in VERNet.
5.1 Performance of Token Level Quality Estimation
We first evaluate VERNet's effectiveness on token-level quality estimation. BERT-GED (SRC) is the previous state-of-the-art GED model (Kaneko and Komachi, 2019). Additional two variants, HYP and JOINT, of BERT-GED are conducted as baselines by considering the first-ranked GEC hypothesis in beam search decoding.
As shown in Table 3, there are two scenarios, source and hypothesis, are conducted to evaluate model performance. The source scenario evaluates the ability of grammaticality quality estimation, which is the same as GED models (Rei and Søgaard, 2019). The hypothesis scenario tests the quality estimation ability on GEC accuracy.
For the source scenario, BERT-GED (JOINT) outperforms BERT-GED (SRC) and illustrates that the GEC result can help estimate the grammaticality quality of source sentences. For the hypothesis scenario, BERT-GED (JOINT) shows better performance than BERT-GED (HYP), which thrives from the supervisions from source sentences. For both scenarios, BERT-VERNet shows further improvement compared with BERT-GED (JOINT). Such improvements demonstrate that various GEC evidence from multiple hypotheses benefits the token-level quality estimation.
Moreover, the detection style pre-trained model ELECTRA (Clark et al., 2020) is also used as our sentence encoder. VERNet is boosted a lot on all scenarios and datasets, which illustrates the strong ability of ELECTRA in token-level quality estimation and the generalization ability of VERNet.
1https://sourceforge.net/p/lemur/wiki/RankLib/
Model	FCE test set	CoNLL-2014 ann. 1	CoNLL-2014 ann. 2
P	R	\( F_{0.5} \)	P	R	\( F_{0.5} \)	P	R	\( F_{0.5} \)
Source	BERT-GED (SRC)	74.22	43.34	64.97	59.84	27.11	48.20	77.94	25.02	54.77
BERT-GED (JOINT)	75.62	44.44	66.32	60.79	27.33	48.83	77.42	25.23	54.77
BERT-VERNet	81.53	45.71	70.48	62.64	30.62	51.80	82.25	28.49	59.71
ELECTRA-VERNet	80.94	50.51	72.24	62.50	35.61	54.30	81.69	32.97	63.06
Hypothesis	BERT-GED (HYP)	80.27	40.58	67.14	74.28	34.20	60.17	66.49	27.68	51.93
BERT-GED (JOINT)	76.71	46.94	68.07	71.15	38.30	60.73	64.79	31.52	53.50
BERT-VERNet	81.85	44.27	69.97	76.03	34.02	60.97	71.79	29.04	55.46
ELECTRA-VERNet	80.62	49.16	71.48	74.80	39.26	63.33	72.55	34.42	59.39
Table 3: Performance of Token Level Quality Estimation. Both source and hypothesis scenarios are conducted to evaluate grammatical quality estimation ability on source sentences and GEC quality estimation ability on hypotheses, respectively. BERT-GED (SRC) only encodes source sentences while others encode <source, hypothesis > pairs. BERT-GED (JOINT) is supervised with golden labels from source and hypothesis sentences.
Model	CoNLL-2014 (M2)	FCE	BEA19	JFLEG
P	R	F0.5	PCC (ann.1)	PCC (ann.2)	P	R	F0.5	PCC	P	R	F0.5	GLEU	PCC
NQE (RR)	61.38	33.03	52.39	23.43	6.62	51.43	30.36	45.16	28.74	57.22	46.33	54.65	55.90	1.29
NQE (RC)	60.09	33.11	51.67	24.12	5.52	53.97	31.35	47.17	31.20	57.87	47.24	55.37	56.91	1.66
NQE (CR)	62.52	35.24	54.14	24.80	9.12	51.77	31.46	45.85	30.69	57.92	47.43	55.47	56.92	6.48
NQE (CC)	60.62	35.77	53.23	22.94	8.39	50.21	32.09	45.11	29.23	56.83	49.47	55.19	57.22	7.68
BERT-LM	52.82	49.59	52.14	3.47	17.62	36.97	43.42	38.10	8.59	46.32	64.05	49.03	59.72	26.85
BERT-GQE	52.67	50.39	52.19	2.56	14.54	36.05	43.53	37.33	10.18	46.15	64.01	48.88	60.17	29.05
BERT-GED (SRC)	52.98	52.07	52.79	3.78	20.56	37.58	45.81	38.98	12.71	47.15	65.09	49.90	60.32	27.28
BERT-QE	62.24	38.27	55.31	22.85	12.17	52.01	36.89	48.07	33.84	58.63	54.19	57.69	59.73	26.16
BERT-GED (HYP)	68.90	34.35	57.36	30.06	16.79	57.21	36.03	51.19	43.48	68.18	53.85	64.73	60.00	29.90
BERT-GED (JOINT)	69.33	36.02	58.51	28.62	16.28	58.53	37.24	52.53	45.08	66.80	55.09	64.07	60.49	33.03
BERT-VERNet	68.75	40.26	60.22	31.02	22.75	58.32	39.99	53.42	47.19	66.86	58.60	65.02	61.36	36.98
ELECTRA-VERNet	69.97	42.12	61.80	37.18	28.77	58.77	41.86	54.37	48.12	69.09	60.91	67.28	61.61	38.63
Table 4: Performance of Sentence Level Quality Estimation. The ranked top-1 hypothesis is used to calculate GEC metrics. NQE (Chollampatt and Ng, 2018b) uses RNN or CNN models for GEC quality estimation. BERTLM (Chollampatt et al., 2019) measures perplexity without fine-tuning. BERT-GQE (Kaneko et al., 2019) and BERT-GED (SRC) are supervised with sentence-level and token-level labels from source sentences to estimate grammatical quality, respectively. NQE and BERT-QE encode <source, hypothesis> pairs and directly predict  $\mathbf{F}_{0.5}$  score. BERT-GED (HYP) and BERT-GED (JOINT) encode the <source, hypothesis> pairs to estimate the quality of generated tokens.
5.2 Performance of Sentence Level Quality Estimation
In this part, we evaluate VERNet's performance on sentence-level quality estimation by reranking hypotheses from beam search decoding.
Baselines can be divided into two groups: language model based and GEC accuracy based quality estimation models. The former focuses on grammaticality and fluency, including BERT-LM, BERT-GQE and BERT-GED (SRC). The others focus on estimating the GEC accuracy, including NQE, BERT-QE, BERT-GED (HYP)/(JOINT).
As shown in Table 4, we find that language model based quality estimation prefers higher recall but lower precision, which leads to more redundant corrections. Only considering grammaticality is insufficient since such unnecessary correction suggestions may mislead users. By contrast, GEC accuracy based quality estimation models get much better Precision and  $\mathrm{F}_{0.5}$ , and provide more precise feedback for users. Furthermore, BERT-GED
(HYP) outperforms BERT-QE, manifesting that token-level supervisions provide finer-granularity signals to help the model better distinguish subtle differences among hypotheses. VERNet outperforms all baselines, which supports our claim that multi-hypotheses from beam search provide valuable GEC evidence and help conduct more effective quality estimation for generated GEC hypotheses.
5.3 VERNet's Effectiveness in GEC Models
This part explores the effectiveness of VERNet on improving GEC models. We conduct  $\mathrm{VERNet}^{\dagger}$  by aggregating scores from the basic GEC model and VERNet for hypothesis reranking.
As shown in Table 5, two baseline models are compared in our experiments, Basic GEC (Kiyono et al., 2019) and BERT-fuse (GED) (Kaneko et al., 2020). Compared to BERT-fuse (GED), BERT-VERNet $^{\dagger}$  achieves comparable performance on CoNLL-2014 and more improvement on BEA19. It demonstrates that reranking hypotheses with VER
Model	CoNLL-2014 (M2)	CoNLL-2014	FCE	BEA19	JFLEG
P	R	F0.5	P	R	F0.5	P	R	F0.5	P	R	F0.5	GLEU
Basic GEC	68.59	44.87	62.03	64.26	43.59	58.69	55.11	41.61	51.75	66.20	61.48	65.20	61.00
Basic GEC w. R2L*	72.4	46.1	65.0	-	-	-	-	-	-	74.7	56.7	70.2	61.4
BERT-fuse (GED)	69.2	45.6	62.6	-	-	-	-	-	-	67.1	60.1	65.6	61.3
BERT-fuse (GED) w. R2L*	72.6	46.4	65.2	-	-	-	-	-	-	72.3	61.4	69.8	62.0
BERT-VERNet† (Top2)	69.98	43.69	62.47	65.62	41.98	58.98	58.57	41.53	54.13	68.42	60.32	66.63	61.17
BERT-VERNet† (Top3)	70.49	43.16	62.57	65.92	41.22	58.86	59.20	41.53	54.55	69.03	60.20	67.06	61.24
BERT-VERNet† (Top4)	70.79	42.72	62.56	66.65	40.94	59.21	59.55	41.55	54.80	69.43	60.17	67.36	61.16
BERT-VERNet† (Top5)	70.60	42.50	62.36	66.41	40.74	58.98	59.68	41.48	54.86	69.39	60.12	67.32	61.10
ELECTRA-VERNet† (Top2)	71.21	44.24	63.47	66.95	42.97	60.22	58.31	41.97	54.09	69.27	61.22	67.50	61.60
ELECTRA-VERNet† (Top3)	71.87	44.13	63.84	67.51	42.38	60.35	59.02	41.99	54.59	70.64	61.78	68.67	61.80
ELECTRA-VERNet† (Top4)	71.85	43.81	63.69	67.48	42.19	60.25	59.65	42.12	55.07	70.96	62.03	68.98	62.05
ELECTRA-VERNet† (Top5)	71.58	43.57	63.43	67.15	42.10	60.01	59.95	42.19	55.29	70.79	61.74	68.77	62.07
Table 5: Performance of Hypothesis Reranking. BERT/ELECTRA-VERNet<sup>†</sup> aggregates the scores of Basic GEC Model (Kiyono et al., 2019) and VERNet for hypothesis reranking with Coordinate Ascent. BERT-fuse (GED) (Kaneko et al., 2020) is the Transformer model that fuses BERT representations. *Note that R2L models incorporate four right-to-left Transformer models that are trained with unpublished data and these models are not supplied in their open source codes, thus these results are hard to reimplement.
Figure 4: Model Performance of Different Grammatical Error Types on BEA19. VERNet† reranks hypotheses with the aggregated score of basic GEC model and VERNet. All types are from ERRANT (Bryant et al., 2017).
Net provides an effective way to improve basic GEC model performance without changing the Transformer architecture. R2L models incorporate four right-to-left Transformer models to improve GEC performance. However, these R2L models are not available. ELECTRA-VERNet† incorporates only one model and achieves comparable performance on BEA19 and JFLEG.
Figure 4 presents  $\mathrm{VERNet}^{\dagger}$  's performance on different grammatical error types. We plot the  $\mathbf{F}_{0.5}$  scores of both basic GEC model and  $\mathrm{VERNet}^{\dagger}$  on BEA19.  $\mathrm{VERNet}^{\dagger}$  achieves improvement on most types and performs significantly better for word morphology and word usage errors, such as Noun Inflection (NOUN:INFL) and Pronoun (PRON). Such results illustrate that  $\mathrm{VERNet}^{\dagger}$  is able to leverage clues learned from multi-hypotheses to verify the GEC quality. However, we also find that  $\mathrm{VERNet}^{\dagger}$  discounts GEC performance on a few
error types, e.g., Contraction (CONTR). The annotation biases may cause such a decrease in CONTR errors. For example, for both "n't" and "not", they are both right according to grammaticality, but annotators usually come up with different corrections with different GEC standards.
We select one case from CoNLL-2014 and visualize node interaction and node selection attention weights to study what VERNet learns from multi-hypotheses of beam search, as shown in Figure 5.
Given a source sentence, "Do one who suffered from this disease keep it a secret of infrom their relatives ?", and its five hypotheses from the Basic GEC Model, we plot the node interaction attention weights towards the word "suffers" in the hypothesis of node 2, which is assigned more higher score by BERT-VERNet. The word usage "suffers" is
Node1 (17.41%) Do one who suffered from this disease keep it a secret of in ##f ##rom their relatives? [SEP] Does someone who suffered from this disease keep it a secret from their relatives? [SEP]  
Node2 (24.09%) Do one who suffered from this disease keep it a secret of in ##f #rom their relatives? [SEP] Does anyone who suffers from this disease keep it a secret from their relatives? [SEP]  
Node3 (20.07%) Do one who suffered from this disease keep it a secret of in ##f #rom their relatives? [SEP] Does anyone who suffered from this disease keep it a secret from their relatives? [SEP]  
Node4 (17.67%) Do one who suffered from this disease keep it a secret of in ##f #rom their relatives? [SEP] Does anyone who suffered from this disease keep it a secret from their relatives? [SEP]  
Node5 (20.77%) Do one who suffered from this disease keep it a secret of in ##f #rom their relatives? [SEP] Does someone who suffers from this disease keep it a secret from their relatives? [SEP]
Figure 5: Visualization of Attention Weight. Each node is the concatenation of the source sentence (with [SEP]) and a corresponding hypothesis sentence (with [SEP]). The selected node by BERT-VERNet is annotated (Node2). The node selection attention assigned to each hypothesis is annotated with dark orange. The node interaction attention towards the edited token "suffers" in the second node is also plotted. Darker red indicates higher attention weights.
more appropriate than "suffered" according to the context.
The node interaction attention accurately picks up the associated tokens "Does" from nodes 1, 3, and 4, and "suffers" from node 5. "Does" and "suffers" indicate the present tense and provide sufficient evidence to verify the quality of "suffers" in node 2. For node selection attention, the hypothesis (node 2) shares more attention than other nodes, which is more appropriate than other hypotheses. It demonstrates that the node attention is effective to select high-quality corrections with the source-hypothesis interactions.
The attention patterns are intuitive and effective, which further demonstrates VERNet's ability to well model the interactions of multi-hypotheses for better quality estimation.
This paper presents VERNet for GEC quality estimation with multi-hypotheses. VERNet models the interactions of multiple hypotheses by building a reasoning graph, and then extracts clues with two kinds of attention: node selection attention and node interaction attention. They summarize and aggregate GEC evidence from multi-hypotheses to verify the quality of tokens. Experiments on four datasets show that VERNet achieves the state-of-the-art GED and quality estimation performance, and improves one published state-of-the-art GEC system. In the future, we will explore the impact of different kinds of hypotheses used in VERNet.
We thank the reviewers and Shuo Wang for their valuable comments and advice. This research is mainly supported by Science & Tech Innovation 2030 Major Project "New Generation AI" (Grant no. 2020AAA0106500) as well as supported in part by a project from Shanghai-Tsinghua International Innovation Center and the funds of Beijing Advanced Innovation Center for Language Resources under Grant TYZ19005.
Abhijeet Awasthi, Sunita Sarawagi, Rasna Goyal, Sabyasachi Ghosh, and Vihari Piratla. 2019. Parallel iterative edit models for local sequence transduction. In Proceedings of EMNLP, pages 4260-4270.
Christopher Bryant, Mariano Felice, Øistein E. Andersen, and Ted Briscoe. 2019. The BEA-2019 shared task on grammatical error correction. In Proceedings of the Fourteenth Workshop on Innovative Use of NLP for Building Educational Applications, pages 52-75.
Christopher Bryant, Mariano Felice, and Ted Briscoe. 2017. Automatic annotation and evaluation of error types for grammatical error correction. In Proceedings of ACL, pages 793-805.
Shamil Chollampatt and Hwee Tou Ng. 2018a. A multilayer convolutional encoder-decoder neural network for grammatical error correction. In Proceedings of AAAI, pages 5755-5762.
Shamil Chollampatt and Hwee Tou Ng. 2018b. Neural quality estimation of grammatical error correction. In Proceedings of EMNLP, pages 2528-2539.
Shamil Chollampatt, Weiqi Wang, and Hwee Tou Ng. 2019. Cross-sentence grammatical error correction. In Proceedings of ACL, pages 435-445.
Kevin Clark, Minh-Thang Luong, Quoc V. Le, and Christopher D. Manning. 2020. ELECTRA: pretraining text encoders as discriminators rather than generators. In Proceedings of ICLR.
Yiming Cui, Zhipeng Chen, Si Wei, Shijin Wang, Ting Liu, and Guoping Hu. 2017. Attention-over-attention neural networks for reading comprehension. In Proceedings of ACL, pages 593-602.
Daniel Dahlmeier and Hwee Tou Ng. 2012. Better evaluation for grammatical error correction. In Proceedings of NAACL-HLT, pages 568-572.
Daniel Dahlmeier, Hwee Tou Ng, and Siew Mei Wu. 2013. Building a large annotated corpus of learner English: The NUS corpus of learner English. In Proceedings of the Eighth Workshop on Innovative Use of NLP for Building Educational Applications, pages 22-31.
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of NAACL-HLT, pages 4171-4186.
Mariano Felice, Christopher Bryant, and Ted Briscoe. 2016. Automatic extraction of learner errors in ESL sentences using linguistically enhanced alignments. In Proceedings of COLING, pages 825-835.
Marina Fomicheva, Lucia Specia, and Francisco Guzmán. 2020. Multi-hypothesis machine translation evaluation. In Proceedings of ACL, pages 1218-1232.
Tao Ge, Furu Wei, and Ming Zhou. 2018. Fluency boost learning and inference for neural grammatical error correction. In Proceedings of ACL, pages 1055-1065.
Roman Grundkiewicz, Marcin Junczys-Dowmunt, and Kenneth Heafield. 2019. Neural grammatical error correction systems with unsupervised pre-training on synthetic data. In Proceedings of the Fourteenth Workshop on Innovative Use of NLP for Building Educational Applications, pages 252-263.
Masato Hagiwara and Masato Mita. 2020. GitHub typo corpus: A large-scale multilingual dataset of misspellings and grammatical errors. In Proceedings of the 12th Language Resources and Evaluation Conference, pages 6761-6768.
Duc Tam Hoang, Shamil Chollampatt, and Hwee Tou Ng. 2016. Exploiting n-best hypotheses to improve an SMT approach to grammatical error correction. In Proceedings of IJCAI, pages 2803-2809.
Marcin Junczys-Dowmunt, Roman Grundkiewicz, Shubha Guha, and Kenneth Heafield. 2018. Approaching neural grammatical error correction as a low-resource machine translation task. In Proceedings of NAACL-HLT, pages 595-606.
Masahiro Kaneko, Kengo Hotate, Satoru Katsumata, and Mamoru Komachi. 2019. TMU transformer system using BERT for re-ranking at BEA 2019 grammatical error correction on restricted track. In Proceedings of the Fourteenth Workshop on Innovative Use of NLP for Building Educational Applications, pages 207-212.
Masahiro Kaneko and Mamoru Komachi. 2019. Multihead multi-layer attention to deep language representations for grammatical error detection. Computación y sistemas, (3).
Masahiro Kaneko, Masato Mita, Shun Kiyono, Jun Suzuki, and Kentaro Inui. 2020. Encoder-decoder models can benefit from pre-trained masked language models in grammatical error correction. In Proceedings of ACL, pages 4248-4254.
Diederik P. Kingma and Jimmy Ba. 2015. Adam: A method for stochastic optimization. In Proceedings of ICLR.
Shun Kiyono, Jun Suzuki, Masato Mita, Tomoya Mizumoto, and Kentaro Inui. 2019. An empirical study of incorporating pseudo data into grammatical error correction. In Proceedings of EMNLP, pages 1236-1242.
Jared Lichtarge, Chris Alberti, Shankar Kumar, Noam Shazeer, Niki Parmar, and Simon Tong. 2019. Corpora generation for grammatical error correction. In Proceedings of NAACL-HLT, pages 3291-3301.
Eric Malmi, Sebastian Krause, Sascha Rothe, Daniil Mirylenka, and Aliaksei Severyn. 2019. Encode, tag, realize: High-precision text editing. In Proceedings of EMNLP, pages 5054-5065.
Donald Metzler and W Bruce Croft. 2007. Linear feature-based models for information retrieval. Information Retrieval.
Tomoya Mizumoto, Mamoru Komachi, Masaaki Nagata, and Yuji Matsumoto. 2011. Mining revision log of language learning SNS for automated Japanese error correction of second language learners. In Proceedings of IJCNLP, pages 147-155.
Courtney Naples, Keisuke Sakaguchi, Matt Post, and Joel Tetreault. 2015. Ground truth for grammatical error correction metrics. In Proceedings of ACL, pages 588-593.
Courtney Naples, Keisuke Sakaguchi, and Joel Tetreault. 2017. *JFLEG: A fluency corpus and benchmark for grammatical error correction.* In *Proceedings of EACL*, pages 229-234.
Hwee Tou Ng, Siew Mei Wu, Ted Briscoe, Christian Hadiwinoto, Raymond Hendy Susanto, and Christopher Bryant. 2014. The CoNLL-2014 shared task on grammatical error correction. In Proceedings of the Eighteenth Conference on Computational Natural Language Learning: Shared Task, pages 1-14.
Kostiantyn Omelianchuk, Vitaliy Atrasevych, Artem Chernodub, and Oleksandr Skurzhanskyi. 2020. GECToR - grammatical error correction: Tag, not rewrite. In Proceedings of the Fifteenth Workshop on Innovative Use of NLP for Building Educational Applications, pages 163-170.
Myle Ott, Michael Auli, David Grangier, and Marc'Aurelio Ranzato. 2018. Analyzing uncertainty in neural machine translation. In Proceedings of ICML, pages 3953-3962.
Marek Rei. 2017. Semi-supervised multitask learning for sequence labeling. In Proceedings of ACL, pages 2121-2130.
Marek Rei and Anders Søgaard. 2019. Jointly learning to label sentences and tokens. In Proceedings of AAAI, pages 6916-6923.
Ilya Sutskever, Oriol Vinyals, and Quoc V. Le. 2014. Sequence to sequence learning with neural networks. In Proceedings of NIPS, pages 3104-3112.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Proceedings of NIPS, pages 5998-6008.
Liang Wang, Wei Zhao, Ruoyu Jia, Sujian Li, and Jingming Liu. 2019a. Denoising based sequence-to-sequence pre-training for text generation. In Proceedings of EMNLP, pages 4003-4015.
Shuo Wang, Yang Liu, Chao Wang, Huanbo Luan, and Maosong Sun. 2019b. Improving back-translation with uncertainty-based confidence estimation. In Proceedings of EMNLP, pages 791-802.
Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Remi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander Rush. 2020. Transformers: State-of-the-art natural language processing. In Proceedings of EMNLP, pages 38-45.
Ziang Xie, Guillaume Genthial, Stanley Xie, Andrew Ng, and Dan Jurafsky. 2018. Noising and denoising natural language: Diverse backtranslation for grammar correction. In Proceedings of NAACL-HLT, pages 619-628.
Helen Yannakoudakis, Ted Briscoe, and Ben Medlock. 2011. A new dataset and method for automatically grading ESOL texts. In Proceedings of ACL, pages 180-189.
Helen Yannakoudakis, Marek Rei, Øistein E. Andersen, and Zheng Yuan. 2017. Neural sequence-labelling models for grammatical error correction. In Proceedings of EMNLP, pages 2795-2806.
Zheng Yuan and Ted Briscoe. 2016. Grammatical error correction using neural machine translation. In Proceedings of NAACL-HLT, pages 380-386.
Wei Zhao, Liang Wang, Kewei Shen, Ruoyu Jia, and Jingming Liu. 2019. Improving grammatical error correction via pre-training a copy-augmented architecture with unlabeled data. In Proceedings of NAACL-HLT, pages 156-165.
A.1 Model Details of Sentence Quality Estimation Score Calculation
This part describes the details of sentence score calculation of BERT based quality estimation models.
Given a source sentence  $s$  with  $m$  tokens and  $k$ -th hypothesis  $c^k$  with  $n$  tokens, we can get the representation  $H^k$  of the  $k$ -th  $\langle$  source, hypothesis  $\rangle$  sentence pair through BERT:
$$
H ^ {k} = \operatorname {B E R T} ([ \mathrm {C L S} ] s [ \mathrm {S E P} ] c ^ {k} [ \mathrm {S E P} ]), \tag {14}
$$
or only the representation  $\mathcal{H}^k$  of the  $k$ -th hypothesis through BERT:
$$
\mathcal {H} ^ {k} = \operatorname {B E R T} ([ \mathrm {C L S} ] c ^ {k} [ \mathrm {S E P} ]). \tag {15}
$$
The " [CLS]" representations are  $H_0^k$  and  $\mathcal{H}_0^k$
BERT-LM. We mask tokens in the  $k$ -th hypothesis sentence  $c^k$  and calculate the Perplexity of the  $k$ -th hypothesis sentence:
$$
f _ {\mathrm {L M}} \left(c ^ {k}\right) = - \operatorname {P P L} \left(\mathcal {H} _ {1: n} ^ {k}\right). \tag {16}
$$
BERT-GQE. BERT-GQE uses the “[CLS]” representation  $\mathcal{H}_0^k$  of  $k$ -th hypothesis to estimate the sentence quality with the probability  $P(y_s|c^k)$ :
$$
P \left(y _ {s} \mid c ^ {k}\right) = \operatorname {s o f t m a x} _ {y _ {s}} \left(W \cdot \mathcal {H} _ {0} ^ {k}\right), \tag {17}
$$
where  $W$  is the parameter and the label  $y_{s}$  is categorized into two groups: correct  $(y_{s} = 1)$  and incorrect  $(y_{s} = 0)$ .
Then the sentence-level quality estimation score of hypothesis  $c^k$  is calculated:
$$
f _ {\mathrm {G Q E}} \left(c ^ {k}\right) = P \left(y _ {s} = 1 \mid c ^ {k}\right). \tag {18}
$$
BERT-QE. BERT-QE uses the " [CLS]" representation  $H_0^k$  of  $k$ -th  $\langle$  source, hypothesis  $\rangle$  sentence pair to estimate the quality of GEC hypothesis: