Improved Pseudo Data for Machine Translation Quality Estimation with Constrained Beam Search
Authors: Xiang Geng
Yu Zhang
Zhejian Lai
Shuaijie She
Wei Zou
shimin tao
Hao Yang
Jiajun Chen
Shujian Huang
Venue: EMNLP 2023 Main (2023)
Keywords: Quality Estimation
Machine Translation
Pseudo Data

Abstract: Machine translation (MT) quality estimation (QE) is a crucial task to estimate the quality of MT outputs when reference translations are unavailable. Many studies focus on generating pseudo data using large parallel corpus and achieve remarkable success in the supervised setting. However, pseudo data solutions are less satisfying in unsupervised scenarios because the pseudo labels are inaccurate or the pseudo translations differ from the real ones. To address these problems, we propose to generate pseudo data using the MT model with constrained beam search (CBSQE). CBSQE preserves the reference parts with high MT probabilities as correct translations, while the rest parts as the wrong ones for MT generation. Therefore, CBSQE can reduce the false negative labels caused by synonyms. Overall, beam search will prefer a more real hypothesis with a higher MT generation likelihood. Extensive experiments demonstrate that CBSQE outperforms strong baselines in both supervised and unsupervised settings. Analyses further show the superiority of CBSQE. The code is available at https://github.com/NJUNLP/njuqe.

Improved Pseudo Data for Machine Translation Quality Estimation with Constrained Beam Search
Xiang Gengl* Yu Zhang' Zhejian Lai'
Shuaijie She'
Wei Zou'
Shimin Tao? Hao Yang² Jiajun Chen' Shujian Huang' t
National Key Laboratory for Novel Software Technology, Nanjing University, Nanjing, China
2 Huawei Translation Services Center, Beijing, China
zhangy, laizj, shesj, zouw)@smaill nju. edu. cn
(taoshimin, yanghaao yehuaaae com, {chenjj,
jenjj. edu. cn
Abstract
the Symbol screener tool applies opacity to symbol instances
Source
Machine translation (MT) quality estima- tion (QE) is a crucial task to estimate the qual- ity MT outputs when reference translations unavailable. Many studies focus gener- ating pseudo data using large parallel corpus and achieve remarkable success in the super- vised setting. However, pseudo data solutions are less satisfying in unsupervised scenarios because the pseudo labels are inaccurate the pseudo translations differ from the real ones. To address these problems, we propose to gen- erate pseudo data using the MT model with constrained beam search (CBSQE). CBSQE preserves the reference parts with high MT probabilities correct translations, while the rest parts as the wrong ones for MT generation. Therefore, CBSQE can reduce the false nega- tive labels caused by synonyms. Overall, beam search will prefer a more real hypothesis with higher MT generation likelihood. Extensive experiments demonstrate that CBSQE outper- forms strong baselines in both supervised and unsupervised settings. Analyses further show the superiority of CBSQE. The code is avail- able https://githe thub. com/NJUNLP/njjn
mit dem cumbol-aufteichher Werkzeug können Sie die Deckkraft auf Symbol-
MT
instanzen onwenden || HTER 0.2727
mit dem Symbolautzeichnungs- Srarehhnunng-eekzzeegg können Sie die Deckkrafl für Symbol- instanzen festlegen.
PE
Table I: An example from the WMT19 English-German (EN-DE) QE dataset. For word-level tags, the token with translation error is labeled "BAD" (indicated in italic font with red color); otherwise, it is labeled "OK". Sentence-level HTER measures the whole effort of manually correcting the MT based on edit distance.
translation words (Specia, 2011); QE can guide decoding (Wang et al., 2020) and re-ranking pro- cess (Bhattacharyya et al., 2021) to improve the translation performance.
As shown in Table 1, sentence- and word-level quality labels can be derived by matching the hy- pothesis with post-edit references using TER tools. As noted by Snover et al. (2006), post-editing con- siders the semantic equivalence and preserves cor- rect parts of MT outputs. Therefore, TER tools will acknowledge the semantically correct translations simply by exact matching. Although post-editing improves quality labels, it remarkably increases annotation costs.
1 Introduction
With the rapid development of machine transla- tion (MT), evaluating the quality of MT outputs becomes increasingly essential. Common MT metrics, such as BLEU (Papineni et al., 2002) or TER (Snover et al., 2006), rely on reference trans- lations which are unavailable in most applications of MT systems. Quality estimation (QE) is the task of automatically assessing the quality of machine translations at run-time without relying on refer- ences (Specia et al., 2018). QE plays an important role real-world scenarios. For example, QE im- proves post-editing (PE) workflows by selecting high-quality MT outputs and indicating incorrect
Therefore, many studies turn generate pseudo QE data (pseudo MT outputs and pseudo QE la- bels) using the large parallel corpus. Furthermore, these pseudo data are used to pre-train a super- vised model or train an unsupervised model. Di- rectQE (Cui et al., 2021) uses a masked language model conditioned source sentence as the trans- lation language model (TLM) to generate pseudo translations. Given a parallel pair, Cui al. (2021) randomly mask reference tokens and replace them with ones sampled from the TLM generation distri- bution. The replaced tokens are annotated as errors,
"Equal Contribution
"Corresponding Author

---

REF: "What is the way to stroke cat ?"
P("How to")
P("strike"paast)
BS
?"|past)
P("What the way lo") -
P("stroke" past)
Output: 'How to strike cat ?"
Tags: B42 OK BAD OK OK OK
P( H""How to")= 0.4
P("strike = 0.8
CBS
P("a cat ?""ss))
P("What is the way to") 0.55 1
P("stroke(pas)
Output: "What is the way strile cat
Tags: OK OK OK OK OK BAD OK OK OK
Figure 1: A toy example of CBSQE in English with beam size 2. "BAD" tags are marked in italic font with red color and will further underlined their labels are wrong. CBSQE increases the generation probability P("What is the way to") of the synonym while keeping P("strike"|pasi past) of the translation enror.
and the sentence score is calculated as the ratio of replacement. BSQE (Tuan et al., 2021> directly generates pscudo translations using neural ma- chine translation (NMT) with the beam search al- gorithm (BS) and obtains labels by matching these translations with corresponding references using TER tools.
follows:
We propose novel and general pseudo data generation method that reconciles the pseudo label with pseudo translation quality.
The proposed method is viable in evaluations based on pscudo data and superior in both supervised and unsupervised settings.
Although pseudo data solutions achieve remark- able performance in the supervised setting (Cui et al., 2021), they are less satisfying in unsuper- vised scenarios (Zheng al., 2021) due gen- eration noise. E.g., DirectQE yields repetition incomplete translation which different from the real ones due to its non ann-arrrrrressivee genera- tion (Wang et al., 2019). Furthermore, its negative sampling strategy sacrifices the quality of pscudo translation for accurate pseudo labels. On the other hand, BSQE suffers from inaccurate pseudo labels. As shown the top of Figure 1, the default beam search yields the synonyms of the reference sen- tence. However, it difficult for existing reference- based evaluation methods to match these synonyms correctly, especially when the sentence structure changes too much.
The analysis reveals that the designed thresh- old strategy and adjustment function matter. Besides, CBSQE is efficient and open source.
Preliminary
2.1 Machine Translation
Machine translation (MT) models condi- tional generation of target sequence y {v), 122 .2. Un) given source sentence The generation can be described in an autoregessive style as P(y\x)
I P(ydy< x)0), where Yer denote the translation history and is the pa- rameter set. The translation is acquired by solving ý = arg max, ((y P x 0) via beam search. The beam search can be further guided by additional penalties for constrained generation (Hokamp and Liu, 2017: Hu et al., 2019). Although beam search yields fluent translations. its diversity of utterance lesser than the actual parallel corpus (Zhou al., 2019). Thus, the MT outputs are often evaluated by references distinct utterances (Chatterjee et al., 2019).
To improve the quality pseudo data, we pro- pose to generate pseudo translations using an MT model with constrained beam search (CBSQE). CB- SQE corrupts translations with reference parts by adjusting the generation probability. CBSQE im- proves the probability of references when the prob- ability of any reference token is above threshold that the less fluent hypothesis will not be mistak- enly rewarded. Besides, we design an adjustment function that assigns larger improvements to closer reference tokens. This allows us to preserve the main structure of references thereby obtaining accu- rate labels using TER tools. We show example at the bottom of Figure
In many applications, we have access to the tar- get MT model that needs to be evaluated. Follow- ing this trend, WMT QE shared tasks have adopted the white-box setting recent years (Specia al., 2021). In the black-box setting, we can obtain a competitive multilingual MT model (Tang et al., 2020) for low low-resource languages.
Our main contributions can summarized as

---

E2
current position
Reference
h
Decoding Path
beam
(a) Decoding Path.
(b) Distance dy.
(c) Distance
Figure 2: Illustration for adjusting generation probabilities.
2.2 Quality Estimation for Machine Translation
both structure and semantic equivalence are in- volved. As shown in Algorithm we propose a constrained beam search algorithm (CBS) to solve this problem at the decoding stage instead of the annotation stage. Inspired by targeted human an- notation (Snover et al., 2006), we constrain the decoding algorithm preserve parts of references that the MT model tends to generate. To this end, is essential decide when to preserve references and which reference tokens should be preserved.
QE assesses the quality of translation without ac- cess references. The evaluation is conducted in different grains: word-, sentence- document- level. Besides, there are several types of supervi- sion for QE as noted in (Belouadi and Eger, 2022): (1) the supervised setting with both QE and par- allel data accessible; (2) the unsupervised setting without QE data, but parallel data accessible; (3) the fully unsupervised setting without QE and par- allel data. In this paper, we focus on word- and sentence-level QE in supervised and unsupervised settings which cover plenty of language directions.
Threshold Strategy. We expect to generate flu- ent sentences with higher NMT probabilities as well as reflect real translation errors. Thus, CBSQE preserves reference tokens when the probability of any reference token is above threshold (line 9 in Algorithm
Given a source language x and a target language translation y = {y1 32,........ ........ }. the word-level labeling is sequence of tags g {91, ................................ ................................ J where 9i ="OK" represents the corresponding yi is correct while "BAD" be further post-edited. The sentence-level HTER score h indicates the overall edit effort of correcting arbitrary y into post- edited reference:
Adjustment Function. On the other hand, we want to preserve the main structure of references so that we can obtain accurate labels using TER tools. Therefore, we design an adjustment function that assigns larger improvements to closer refer- ence tokens when adjusting generation probabili- ties. Consider the example in Figure 2(a), we need to define which are closer tokens. There are two ex- tremes: (i) "x" and "y" are the insertion errors, and the rest dccoding parts are aligned with reference tokens r>j = "b c d e gg h", where j denotes the position of the last reference token being decoded. The distance from to the anchor rý11 ="b" is d kw j- (Figure 2(b)), where kw is the po- sition of the nearest r>j from left; (ii) "x" and "y" are the substitution errors, and the rest decoding parts are aligned with reference words r>; "d g h", where denotes the beam step. We have d, = A il (Figure 2(c)). We assume that these two extremes occur with equal probability, thus we define the adjustment function as follows:
number of edits post-editing sentence length"
These labels are produced via TER tools by taking the translation and PE sequences as inputs.
3 Method
We first introduce the constrained beam search al- gorithm, then the training and inference procedures of the QE model.
3.1 Constrained Beam Search
As aforementioned in Section 1, due to the dif- ference between translations and references, many correct translation words will be mistakenly labeled as negative by TER tools. Synonym annotation requires fine- fine-grained reference-based evaluation, which is yet another challenging task like QE since
(dy dd) if E Ery
r>j if & r>j
A(w)
(2)
INF

---

Algorithm 1 Constrained Beam Search.
Input: Parallel pair (x, r), beam size k, translation model with vocabulary w, temperature т, threshold
6 combined weight À < [0, !].
Output: Pseudo translation y.
1: Heur - { (BOS, 0, 0)) 1. # Initialization.
2: repeat
3: Hnextt f- 0.
4: for all {(y.P.)}} Heur do
5: if ylyl = EOS then
6:
Hnent f Hner U {(y.p.5)).
7: else
8:
p = P(wly, x): 0).
9:
if 3w E T>ji Pw e. then
q softmax A(W)), ). # Normalize A(w) with softmax function.
p =x
xq + Alp.
12:
end if Hacxt - Hnex U UwEEw(y w, x Pwi max(kwr))) denotes the concatenate operation.
13:
14:
end
15:
end for
16:
Heur < ((y,P.)) E Hnext: K(y 1''',,/',3 .', E Hmexi p pHl < k). # Select k-best candidates.
17:
()) = arg (.p.)HH)) P, i == i 1.
18:
until 8181 EOS
We calculate the adjustment over vocabulary and normalize the results using the softmax function with temperature 7 (line 10 in Algorithm 1). Fi- nally, we combine the increment q with original probabilities by weight A:
3.2 Training and Inference
With the development of pre-trained language models (PLMs), multilingual PLMs such as mBERT (Devlin et al., 2019) and XLMR (Conneau et al., 2020) have been widely used for initializing QE models (Kim al., 2019; Ranasinghe et al.. 2020). In this work, adopt the XLMR-large model the base model for all pseudo data solu- tions. Given a (pseudo) QE sample (x. y, g. h), we concatenate the source x and the translation ý as the input. We take the corresponding outputs from the last layer as representations of each sub-tokens. The word representations are the average across representations of all sub-tokens. Likewise, the sentence score representation averages all represen- tations of the target sub-token: These represen- tations are passed through linear layers predict word tags and sentence scores, respectively.
p by A)p.
Beam Search. Since the generation procedure is autoregressive, incremental adjustments may rel- atively decrease the future decoding probabilities. Therefore, we adopt beam search to find the best candidate to meet our constraints. That is, we keep a set of active candidates Heur- In each iteration, we expand all incomplete candidates (without EOS) aforementioned and place them into Hnext (line 4-15 in Algorithm 1). Then we collect the best k candidates in H according to the adjusted probabilities as the active set Heur in the next it-
We pre-train and fine-tune the QB model under the multi-task learning framework by summing up the objective of the sentence- and word-level tasks:
eration (line 16 in Algorithm 1). We iteratively refine the best candidate in Heur until the end of sequence (EOS).
Overall, CBSQE maintains the main structure of the reference, which ensures the labeling accuracy of TER tools. The mismatched parts compared to reference are hardly labeled felse-negative since their generation probabilities are distinct compared to those of the correct parts.
JQE = log P(h|x, y; 0) + log P(g 9; (4)
where sentence- and word-level tasks are regarded as regression and sequence labeling, respectively. Fomicheva et al. (2020) and Zheng et al. (2021) quantify NMT or TLM model uncertainty using the

---

Monte Carlo dropout (Gal and Ghahramani, 2016). The obtained uncertainty features have achieved re- markable performance for unsupervised QE. This strategy can also be regarded as a self-ensemble technique (Laine and Aila, 2016). In this paper, we also explore Monte Carlo dropout for CBSQE. Specifically, we use dropout to perturb the QE model and predict the QE labels with the perturbed model several times. Then, scores as the final result for the sentence-level task and obtain word-level results by voting.
EN-ZH tasks adopt the glass-box setting, we di- rectly use the target NMT model provided by WMT. We use the TER tool called TERCOM to annotate the pseudo translations generated by NMT or CB- SQF. Our experiments are conducted on NVIDIA 3090-Ti/V100 GPUs. We provide more implemen- tation details in Appendix A.
average all output
Evaluation Metrics. Following WMT QE shared tasks, the sentence-level task will be eval- uated using Pearson's 's correlation coefficient. For the word-level task, WMT19 used FI-MULT as the primary metric, while WMT20/21 conducted the Matthews correlation coefficient (MCC) as the primary metric. F1-MULT is denoted as the mul- tiplication of Fl-scores for the "OK" and "BAD" words. We provide the results of MAE and RMSE metrics in Appendix A for reference.
4 Experiments
4.1 Setup
Datasets. We conduct the experiments on the widely used benchmarks for QE from WMT QE shared tasks: WMT19 English-German (EN-DE), (EN-ZH). WMT19, 20, and 21 datasets contain 13K/IK/IK, 7K/KK/IK and 8K/IK/IK QE samples for training, validation, and testing, respectively. In the main experiments of each language direction, we only use the same 500K parallel pairs for all QE methods. These datasets are all officially released by WMT organiz- ers and have already been tokenized and true-cased.
WMT20/21 English-
4.2 Main Results
Table 2 shows the main results for baselines and CBSQE on different QE datasets. Monte Carlo dropout is essential for the SSQE and can slightly improve the performance of CBSQE We only use this strategy for fair comparison between CBSQE and SSQE because it significantly increases the inference times. For word-level tasks, CBSQE mainly improves the FI-BAD score, which implies that CBSQE could reduce false-negative labels.
Baselines. We compare the CBSQE with strong baselines, including the pscudo data method (Di- rectQE (Cui et al., 2021) and BSQE (Tuan al., 2021)), the unsupervised method based on uncer- tainty features (SSQE (Zheng et al., 2021)). We also build a standard supervised baseline XLM- RQE by dircctly fine-tuning the XLMR using real QE data.
For all unsupervised tasks, CBSQE significantly outperforms strong baselines (significance test can be found in Appendix C). Specifically, CBSQE (w/ dropout) increases the Pearson coefficient by +3.18 and the F1-MULT by +0.93 on the WMT19 EN- DE dataset. For WMT20/21 EN-ZH tasks, CB- SQE (w/ dropout) increases the Pearson coeffi- cient by +4.69/47.42 and the MCC by +4.25/42.71 on WMT20 and WMT21 tasks, respectively. We achieve more improvements on EN-ZH than EN- DE. We assume there are more synonyms in Chi- nese, while CBSQE successfully reduces the gap between pseudo translations and references. Fur- thermore, we improve the performance in unsuper- vised settings towards that of XLMRQE in super- vised settings.
Implementation Details. We reproduce the SSQE using the codes released by Zheng et al. (2021)2. SSQE also employs large pre-trained lan- guage model, mBERT-large. We follow the guide by Zheng et al. (2021) to set hyper-parameters and search for the best threshold for word-level tags using the validation set. We implement all pseudo data methods in NJUQE based on the open- source toolkit Fairseq (Ott et al., 2019). We fol- low the setting of (Cui et al., 2021) to train the TLM for all QE tasks. We train NMT model for WMT19 EN-DE, which is the Transformer-bas setting (Vaswani et al., 2017). Since WMT20/21
For all supervised scenarios, CBSQE slightly im- proves the performance across the board compared to alternative pseudo data methods. Fine-tuning on small QE datasets involves inductive bias so that it reduces the benefits of improved pseudo data.
'https:/ statmt. stat t/g com/ Sel SepeSvueve
wtw/THUNLTCCEEEE THETTTHUULLLPLL P-M/
hhttp://www... www. umd. amm.eddu-ssnne eduu/=nnove//eercom/

---

Sent-level Test
Word-level Test
Dataset
Settings
Method
Pearsonf
MCCT
FI-MULT
FI-OK1
FI-BAD
DireciQE BSQE
42.52
33.88
32.74
93.02
35.19
46.30
31.98
36.59
86.15
42.47
WMT19 EN-DE
SSQE
40.57
30.58
35.95
90.88
39.56
Unsupervised
SSQE (w/ dropout)
45.04
32.92
38.41
90.45
42.46
CBSQE CBSQE (w/ dropout)
48.47
33.90
38.71
87.56
44.20
49.48
34.45
39.34
88.11
44.65
XIMRQR
52.21
38.54
42,70
91.91
46.46
DirectQE
56.26
42.21
45.88
92.46
49.62
Supervised
BSQE
56.45
41.87
45.73
92.19
49.60
CBSQE
56.65
42.69
46,25
92.66
49.92
DirectQE
45.15
32.32
43.47
66.12
65.74
BSQE
$4.99
36.12
46.25
65,86
70.23
NANTE TITTOO EN-ZH
SSQE SSQE (w/ dropout)
40.15
30.70
42.65
63.08
67.61
Unsupervised
43.82
33.50
44.52
64.98
68.52
CBSQE CBSQE (w/ dropout)
59.09
39.72
48.71
67.61
72.05
59.68
40.37
49.23
68.55
71.82
XLMRQE
60.55
45.78
52.10
73,42
70.96
DirectQE
63.59
49.02
74.65
73.52
Supervised
BSQE
64.71
48.85
54.84
74.50
73.61
CBSQE
65.42
49.74
55.67
74.74
74.49
DirectQE
20.35
15.72
27.03
83.25
32.47
BSQE
21.22
17.49
28.74
77.38
37.14
WMTZI EN-ZH
SSQF
21.39
20.35
30,92
80.10
38.61
Unsupervised
SSQE (w/ dropout)
21.40
19.79
30.12
77.49
38.87
CBSQE CBSQE dropout)
27.41
22.53
78.83
40.67
28.82
23.06
32.92
82.14
XLMRQE
30.38
28.01
35.63
80.49
44.27
DirectQE
31.99
30.15
36,92
80.02
46.13
Supervised
BSQE CBSQE
31.88
29.45
36,38
79.69
45.65
32.15
30.97
37.63
80.58
46.70
Table Main results for baselines and CBSQE on different QE test sets.
Pseudo Data
P(y]=;0) t
Acc-All
Acc-BAD f
DirectQR
0.0613
94,99 %
88.11 %
BSQE
0.7018
63.00%
35.71% %
230
CBSQE
0.6905
88.37%
58.76%
255
CB502
19906
Table 3: Average probability of the target NMT model and word tags accuracy for different pseudo data meth- ods EN-ZH direction.
1M 1.
3M Pstudo Data SOMe
Poeudo Data Scale
(a) FI-MULT.
(b) Pearson.
Figure (a) FI-MULT or (b) Pearson score of differ- methods the WMT19 EN-DE QE test set using different amounts of parallel pairs.
4.3 Amount of Pseudo Data
We investigate how data size affects the perfor- mance of unsupervised QE methods. We conduct further experiments using different amounts of par- allel pairs for different QE methods on the WMT19 EN-DE dataset. As shown in Figure 3, CBSQF consistently outperforms basclines with different data sizes. For the word-level task, pseudo data methods achieve remarkable improvements with the growth data size (Figure 3(a)). However, us- ing over 500K data does not significantly improve the sentence-level results for methods (Figure
3()).. We assume that the sentence-level scores fol- low obvious preferences such as fluency (Sun et al., 2020), causing a limited contribution of increased data size.
Analysis
In unsupervised settings, we analyze the quality of CBSQE pseudo data, factors contributing to the improvement, and the efficiency CBSQE.

---

Source
This is important development that deserves to be widely adopted
Reference
i2 -
1'12 Kettt
th 1R HH ffl
#g
DirectQE
13 P
1 14
क de
& Its Back
This is important enterprise that deserves deserves be explained.
BSQE
İ -1f
+ nb R
333 F Rah
& Its Back
This is an important development that deserves to be widely adopted.
CBSQE
i2 12
fEf I" "2 Rin #6 the F "R.
Its Back
This is an important development of the situation that deserves to be widely adopted
Table Pseudo QE data EN-ZH QE task. The tokens with "BAD" tags are marked italic font with red color and will be further marked with the underline if their labels wrong.
5.1 Improved Pseudo Data
Method
Pearson
FI-MULTY
Quantitative analysis. We evaluate the quality of pseudo data from two aspects: (1) the quality of pseudo translations; (2) the accuracy of the pseudo labels. To answer the first issue, we check the get NMT's generation likelihoods of the pseudo translations by averaging the NMT probabilities of all 3.4M pseudo translations generated by different methods. To answer the second issue, we randomly sample 40 pseudo samples for each method and manually check the accuracy of word tags. The results are summarized in Table 3. The negative sampling strategy improves the pseudo label accu- racy of DirectQE. However, DirectQE achieves rel- atively low NMT probabilities because of the non- autoregressive TLM and negative sampling strat- egy. CBSQE obtains improved word tag accuracy than BSQE by reducing the false negative. Using threshold and beam search, CBSQE achieves slightly lower NMT probability than BSQE though CBSQE constrains the MT model to preserve the major reference parts.
CBSQE w/o threshold
46.80
34.03
CBSQE w/o adjustment function
42.62
35.48
CBSQE
48.47
38.71
Table 5: Ablation studies on the WMT19 EN-DE test set.
Method
Generation Time (ms) + Speed
DirectQE BSQE CBSQE
17.45
4.3x
75.06
1x
100.07
0.75x
Table 6: Average generation time per sample of different pseudo data methods single 3090Ti GPU.
Effect of adjustment function D. The adjust- ment function D assigns larger probabilities to ref- erence words that are closer to current position or beam step i. Without adjustment function D, CB- SQE assigns the same adjustment to each reference word and causes performance degradation (Table
5).
Qualitative analysis. Table shows the pseudo data generated by different methods for EN-ZH tasks. As aforementioned, DirectQE generates ir- relevant pseudo translation with accurate labels; BSQE generates translations with noisy labels be- cause of synonyms; CBSQE successfully generates proper translation errors with accurate labels.
5.3 Efficiency
Generation timelapse. We record the genera- tion timelapse of different methods for 125K sam- ples. As shown in Table DirectQE generates pseudo data 4.3x faster than BSQE due the non- autoregressive decoding. Although CBSQE intro- duces complex constraints, the computational com- plexity does not increase too much (only 0.75x of the original timelapse).
5.2
Ablation Studies
We further conduct ablation studies with different CBSQE variants the WMT19 EN-DE dataset.
Convergence efficiency. Better pseudo data could speed up training convergence. To compare the convergence of different methods, we plot the learning curve the WMT19 EN-DE validation set in Figure 4. For both sentence- and word-level tasks, CBSQE converges faster than DirectQE and BSQE, achieving the highest performance.
Effect of threshold c. To measure the effect of threshold e, we perform CBSQE with CBSQE will increase reference tokens, whatever their probabilities. Table shows that threshold matters for CBSQE.
that

---

A series of unsupervised QE works use various features estimate the adequacy and fluency of MT outputs. Etchegoyhen et al. (2018) use lexi- cal translation overlaps and language model cross- entropy scores. BERTScore (Zhang et al., 2019), YiSi (Lo, 2019) and Zhou et al. (2020) adopt to- ken similarities based on contextual embeddings. Fomicheva et al. (2020) define several QE features based translation probability, uncertainty quan- tification, and attention scores. Furthermore, SSQE (Zheng et al., 2021) utilizes TLM generation prob- abilities of masked MT words and explores Monte Carlo Dropout (Gal and Ghahramani, 2016) for quantifying uncertainty. On the other hand, pseudo data can be directly used for unsupervised QF. However, SSQE outperforms existing pseudo data methods due to noisy pseudo data. In this paper, aim improve the quality of pseudo data.
W4 M H
12
Bres00
DeestO Hare
2000
4000
6000
5000
2000
4000
6000
5000
Stop
Stop
(a) FI-MULT.
(b) Pearson.
Figure 4: Training step vs. (a) F1-MULT or (b) Pearson score of different methods on the WMT19 19 EN-DE QE validation set.
485
460
47.5
C2060 E1 Ma33
37.5
CIp0: T2- ML.I
47.0
Reatio
37.0
CBSQ2 Pearinn
405
3502 Norson
36.3
5502 Narson
0.1
A. 0:35
(a) Threshold
Combination weight
Figure 5: Performance of CBSQE on the WMT19 EN-
DE QE test set using different (a) threshold e and (b) combination weight A.
6.2 Constrained Machine Translation
There are two main ways to integrate constraints in NMT: constrained training and decoding. Song et al. (2019) and Dinu et al. (2019) train a vari- ant MT model by augmenting the data to include the constraints. Constraint-aware beam search al- gorithms have been widely used to introduce con- straints during inference (Hokamp and Liu, 2017; Hu et al., 2019). The above approaches rely on pre-defined constraints, while our work also de- fines desired constraints to generate better pscudo data. The closest work is (Lopes et al., 2019), which penalizes all tokens not present in transla- tions with the same reduction so that the automatic post-editing model will not over-correct the transla- tions. For comparison, we assign different rewards to reference tokens based on their positions and generation probabilities.
Tuning hyper-parameters. CBSQE involves three syper-parameters. Specifically, the tempera- ture determines how much probability is assigned to each position. We simply set 5 for all exper- iments. The threshold decides when to preserve reference words. The combination weight À de- cides the BAD ratio of pscudo translations. set = 1, the generated pseudo translations will be the same as references, and the BAD ratio equals O; if we set À = 0, we obtain real MT', and the BAD ratio equals TER. As shown in Figure 5, CBSQE outperforms baselines under the broad range of e and X.
6 Related Works
6.1 Machine Translation Quality Estimation
Most supervised QE methods transfer knowledge using the pre-training and fine-tuning strategy. Predictor-Estimatoo (Kim et al., 2017) and QE- Brain (Fan et al., 2019) pre-train the QE model with the word prediction task. The translation lan- guage modeling objective (Conneau and Lample, 2019) is also used for QE pre-training in (Kepler et al., 2019). As mentioned in (Cui et al., 2021), these pre-training objectives are far from the QE task, and the pseudo data method achieves remark- able results on several QE datasets. COMET (Rei et al., 2020) and UniTE (Wan et al., 2022) incorpo- rated the QE task with MT evaluation.
7 Conclusion
We present a constrained beam search algorithm to generate improved pseudo QE data. The pro- posed CBSQE preserves the main structure of ref- erences by increasing the generation probabilities of reference tokens according their original NMT probabilities and positions. Experiments show that we achieve remarkable performance in both su- pervised and unsupervised settings. The analysis further shows that CBSQE efficient, and each part of CBSQE contributes to the improvement.

---

Limitations
Meeting of the Association for Computational Lin- guistics and the 11th International Joins Conference Natural Language Processing (Volume I: Long Papers), pages 4528-4537, Online. Association for Computational Linguistics.
In this study, we only explore the pseudo data methods for the word- and sentence-level QE tasks with post-editing annotation. The proposed CBS algorithm could be effective in other evaluation tasks. For instance, CBS can be extended to other QE annotations with simple designs as introduced in (Geng et al.. 2022). CBS could also be ex- tended to the fully unsupervised QE task by us- ing back-translations of an unsupervised MT as pseudo parallel data; Some annotations for the translation metric are also based on fine-grained errors (Freitag et al., 2022), making it natural to ex- tend CBS to reference-basee based evaluation; Zhou et al. (2021) developed a token-level hallucination detec- tion method similar to BSQE, which can be fur- ther improved by the CBS algorithm; High-quality pseudo data could be also helpful to predict human feedback. Predicting human feedback is crucial for aligning language models with human intent (Ouyang al., 2022).
Rajen Chatterjee, Christian Federmann, Matteo Negri, and Marco Turchi. 2019. Findings of the WMT 2019 shared task on automatic post-editing. In Proceed- ings of the Fourth Conference on Machine Transla- tion ( (Volume 3: Shared Task Papers, Day 2), pages 11 11-28, Florence, Italy. Association for Computa- tional Linguistics.
Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Out, Luke 7ettle- moyer, and Veselin Stoyanov. 2020. Unsupervised lingull representation learning scale. In Pro- ceedings of the 58th Anmual Meeting of the Asso- ciation for Computational Linguistics, pages 8440- 8451, Online. Association for Computational Lin- guistics.
Alexis Conneau and Guillaume Lample. 2019. Cross- lingual language model pretraining. Advances neural information processing systems, 32.
The labels of some CBSQE pseudo data are still noisy. It could be further improved by developing powerful fine-graine tne-grained reference- based evaluation methods employing the self-distillation strategy. We have not tested the ability of CBSQE to gener- ate various pseudo data using the same parallel pair. Besides, there is potential risk of collapse under adversarial attacks for QE methods. We leave these explorations in the future.
Qu Cui, Shujian Huang. Jiahuan Li, Xiang Geng, Zaixi- ang Zheng, Guoping Huang, and Jiajun Chen. 2021. Directqe: Direct pretraining for machine translation quality estimation. In Proceedings of the AAAI Con- ference on Artificial Intelligence, volume 35, pages 12719-12727.
Jacob Devlin, Ming-Wor Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language under- standing. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Tech- nologies, Volume (Long and Short Papers), pages 4171-41888 Minneapolis, Minnesota. Association for Computational Linguistics.
A Acknowledgements
We would like to thank the anonymous review- ers for their insightful comments. Shujian Huang is the corresponding author. This work is sup- ported by National Science Foundation of China (No. 62376116 16, 62176120), the Liaoning Provin- cial Research Foundation for Basic Research (No. 2022-KF-66-02).
Georgiana Dinu, Prashant Mathur, Marcello Federico, and Yaser Al-Onaizan. 2019. Training neural ma- chine translation to apply terminology constraints. In Proceedings of the 57th Annual Meeting of the Asso- ciation for Computational Linguistics, pages 3063- 3068, Florence, Italy. Association for Computational Linguistics.
References
Thierry Etchegoyhen, Eva Martínez Garcia, and An- doni Azpeitia. 2018. Supervised and unsupervised minimalist quality estimators: Vicomtech's partic- ipation in the WMT 2018 quality estimation task. In Proceedings of the Third Conference on Machine Translation: Shared Task Papers, pages 782-787, Belgium, Brussels. Association for Computational Linguistics.
Jonas Belouadi and Steffen Eger. 2022. Uscore: An effective approach to fully unsupervised evaluation metries for machine translation. arXiv preprint arXiv:2202.10062.
Sumanta Bhattacharyya, Amirmohammad Rooshenas, Subhajit Naskar, Simeng Sun, Mohit lyyer, and An- drew McCallum. 2021. Energy-based reranking: Improving neural machine translation using energy- based models. In Proceedings of the 59th Annual
Kai Fan, Jiayi Wang, Bo Li, Fengming Zhou, Boxing Chen, and Luo Si. 2019. "bilingual expert" can find

---

translation errors. In Proceedings of the AAAI Con- ference on Artificial Intelligence, volume 33, pages 6367-6374.
Hyun Kim, Hun- Young Jung, Hongseok Kwon, Jong- Hyeok Lee, and Seung-Hoon Na. 2017. Predictor- estimator: Neural quality estimation based tar- get word prediction for machine translation. ACM Trunsactions on Asian and Low-Re Resouree Language Information Processing (TALLIP), 17(1):1-22.
Marina Fomicheva, Shuo Sun, Lisa Yankovskaya, Frédéric Blain, Francisco Guzmán, Mark Fishel, Nikolaos Aletras, Vishrav Chaudhary, and Lucia Spe- cia. 2020. Unsupervised quality estimation for neural machine translation. Transactions of the Association for Computational Linguistics, 8:539 555.
Hyun Kim, Joon-Ho Lim, Hyunki Kim, and Seung- Hoon Na. 2019. Qe bert: Bilingual bert using multi- task learning for neural quality estimation. In Pro- ceedings of the Fourth Conference on Machine Trans- lation.
Markus Freitag, Ricardo Rei, Nitika Mathur, Chi-kiu Lo, Craig Stewart, Eleftherios Avramidis, Tom Koemi, George Foster, Alon Lavie, and André F. T. Martins. 2022. Results of WMT22 metrics shared task: Stop using BLEU neural metrics are better and more robust. In Proceedings of the Seventh Conference on Machine Translation (WMT), pages 46-68, Abu Dhabi, United Arab Emirates (Hybrid). Association for Computational Linguistics.
Samuli Laine and Timo Aila, 2016. Temporal ensem- bling for semi-supervised learning, arXiv preprint 110.0246..
Chi-kiu Lo. 2019. YiSi a unified semantic MT quality evaluation and estimation metric for languages with different levels of available resources. In Proceed- ings of the Fourth Conference on Machine Transla- tion ( (Volume 2: Shared Task Papers, Day 1), pages 507-513, Florence, Italy, Association for Computa- tional Linguistics.
Yarin Gal and Zoubin Ghahramani. 2016. Dropout as a bayesian approximation: Representing model uncer- tainty in deep learning. In international conference on machine learning. pages 1050-1059, PMLR.
António Lopes, M. Amin Farajian, Gonçalo M. Cor- reia, Jonay Trénous, and André F. T. Martins. 2019. Unbabel's I' submission to the WMT2019 APE shared task: BERT-based encoder-decoder for automatic post-editing. In Proceedings of the Fourth Confer- ence Machine Translation (Volume 3: Shared Task Papers, Day 2). pages 123, Florence. Italy. As- sociation for Computational Linguistics.
Xiang Geng, Yu Zhang, Shujian Huang, Shimin Tao, Hao Yang, and Jiajun Chen. 2022. Njunlp's partic- ipation for the wmt2022 quality estimation shared task. WMT 2022, page 615.
Yvette Graham. 2015. Improving evaluation of ma- chine translation quality estimation. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 1804-1813.
Myle Ott. Sergey Edunov, Alexei Baevski, Angela Fan, Sam Gross, Nathan Ng, David Grangier, and Michael Auli. 2019. fairseq: A fast, extensible toolkit for sequence modeling. In Proceedings of NAACL-HLT 2019: Demonstrations.
Chris Hokamp and Qun Liu. 2017. Lexically con- strained decoding for sequence generation using grid beam search. In Proceedings of the 55th Annual Meeting of the Association for Computational Lin- guistics ( (Volume 1: Long Papers), pages 1535-1546, Vancouver, Canada. Association for Computational Linguistics.
Long Ouyang. Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. 2022. Training language models to follow instruc- tions with human feedback. Advances in Neural Information Processing Systems, 01:27730-27744
J. Edward Hu, Huda Khayrallah, Ryan Culkin, Patrick Xia, Tongfei Chen, Matt Post, and Benjamin Van Durme. 2019. Improved lexically constrained decoding for translation and monolingual rewriting. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computa- tional Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages Minneapolis, Minnesota. Association for Computa- tional Linguistics.
Kishore Papineni, Salim Roukos, Todd Ward, and Wei- Jing Zhu. 2002. Bleu: a method for automatic evalu- ation of machine translation. In Proceedings of the 40th annual meeting of the Association for Computa- tional Linguisties, pages 311-318.
Tharindu Ranasinghe, Constantin Orasan, and Ruslan Mitkov, 2020. TransQuest: Translation quality esti- mation with cross-lingus transformers. In Proceed- ings of the 28th International Conference on Com- putational Linguistics, pages 5070-5081, Barcelona, Spain (Online). International Committee on Compu- tational Linguistics.
Fabio Kepler, Jonay Trénous, Marcos Treviso, Miguel Vera, António Gois, M. Amin Farajian, António V. Lopes, and André F. T. Martins. 2019. Unbabel's 's par- ticipation in the wmm19 19 translation quality estimation shared task. In Proceedings of the Fourth Conference on Machine Translation.
Ricardo Rei, Craig Stewart, Ana C Farinha, and Alon Lavie. 2020. COMET: A neural framework for MT evaluation. In Proceedings of the 2020 Conference

---

on Empirical Methods in Natural Language Process- ing (EMNLP), pages 2685-2702, Online. Association for Computational I Linguistics.
Yu Wan, Dayiheng Liu, Baosong Yang, Haibo Zhang, Boxing Chen, Derek Wong, and Lidia Chao. 2022. UniTE: Unified translation evaluation. In Proceed- ings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume I: Long Pu- pers), pages 81 17-8127, Dublin, Ireland. Association for Computational Linguistics.
Matthew Snover, Bonnie DOIr, Richard Schwartz, Lin- nea Micciulla, and John Makhoul. 2006. A study of translation edit rate with targeted human annotation. In Proceedings of association for machine franslation in the Americas, volume 200. Cambridge, MA.
Ke Wang. Jiayi Wang. Niyu Ge, Yangbin Shi, Yu Zhao, and Kai Fan. 2020, Computer assisted translation with neural quality estimation and auotmatic post- editing. In Proceedings of the 2020 Conference Empirical Methods in Natural Language Processing: Findings, pages 21 2175-2186.
Kai Song, Yue Zhang, Heng Yu, Weihua Luo, Kun Wang, and Min Zhang. 2019. Code-switching for enhancing NMT with pre-specified translation. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computa- tional Linguistics: Human Language Téchnologies, Volume 1 (Long and Short Papers), pages 449-459, Minncapolis, Minnesota. Association for Computa- tional Linguisties.
Yiren Wang, Fei Tian, Di He, Tao Qin, ChengXiang Zhai, and Yan Liu. 2019. Non-autoregressivee machine translation with auxiliary regularization. In Proceedings of the AAAI conference on artificial in- telligence, volume 33, pages 5377-5384.
Lucia Specia. 2011. Exploiting objective annotations for minimising translation post-editing effort. In Pro- ceedings of the 15th Annual conference of the Euro- pean Association for Machine Translation.
Tianyi Zhang. Varsha Kishore, Felix Wu, Kilian Q Wein- berger, and Yoav Artzi 2019. Bertscore: Evaluating text generation with bert. International Confer- ence on Learning Representations.
Lucia Specia, Frédéric Blain, Marina Fomicheva, Chrysoula Zerva, Zhenhao Li, Vishrav Chaudhary, and André FT Martins. 2021. Findings of the wmn 2021 shared task on quality estimation. In Proceed- ings of the Sixth Conference on Machine Translation, pages 684-725.
Yuanhang Zheng, Zhixing Tan, Meng Zhang. Mieradil- ijiang Maimaiti, Huanbo Luan, Maosong Sun, Qun Liu, and Yang Liu. 2021. Self-supervised quality estimation for machine translation. In Proceedings of the 2021 Conference Empirical Methods in Nat- ural Language Processing, pages 3322-3334, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics.
Lucia Specia, Carolina Scarton, and Gustavo Henrique Paetzold. 2018, Quality estimation for machine trans- lation. Synthesis Lectures on Human Language Tech- nologies, 1162.
Chunting Zhou, Graham Neubig, and Jiatao Gu. 2019. Understanding knowledge distillation in non- autoregressive machine translation. arXiv preprint arXiv:1911.02727.
Shuo Sun, Francisco Guzmán, and Lucia Specia. 2020. Are we estimating or guesstimating translation qual- ity? In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 6262-6267, 0267, Online. Association for Computational Linguistics.
Chunting 7hou, Graham Neubig, Jiatao Gu, Mona Diab, Francisco Guzmán, Luke Zettlemoyer, and Marjan Ghazvininejad. 2021. Detecting hallucinated content in conditional neural sequence generation. In Find- ings of the Association for Computational Linguis- tics: ACL-LICNLP 2021, pages 1393-1404, Online. Association for Computational Linguistics.
Yuqing Tang, Chau Tran, Xian Li, Peng-Jen Chen, Na- man Goyal, Vishrav Chaudhary, Jiatao Gu, and An- gela Fan. 2020. Multilingual translation with exten- sible multilingual pretraining and finetuning, arXiv prepriat 2iv:2008.0044)
Lei Zhou, Liang Ding, and Koichi Takeda. 2020. Zero- shot translation quality estimation with explicit cross- lingual patterns. In Proceedings of the Fifth Con- ference on Machine Translation, pages 1068-1074, Online. Association for Computational Linguistics.
Yi-Lin Tuan, Ahmed El- El-Kishky, Adithya Renduchintala, Vishrav Chaudhary, Francisco Guzmán, and Lucia Specia. 2021. Quality estimation without human- labeled data. In Proceedings of the 16th Conference of the European Chapter of the Association for Com- putational Linguistics: Main Volume, pages 619-625, Online, Association for Computational Linguistics.
A Other Implementation Details
All experiments set the random seed as 1 except for fine-tuning. When fine-tuning, we run each experiment with different random seeds five times. We follow the setting of the WMT shared tasks to report the best results in Table 2. We also re- port average results in Table 7. We can observe similar phenomena to the main results from aver- age results. For SSQE, we follow the setting of
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. Advances in Neural Information Process- ing Systems 30.

---

Sent-level Test
Word-level Test
Dataset
Settings
Method
Pearsont
MCCT
FI-MULTY
FI-OK1
FI FI-BADT
XLMRQE
$2.06
38.75
42.64
92.16
46,29
DirectQE
55,95
42.26
45.87
92.77
49,45
WMT19 EN-DE
Supervised
BSQE
56.16
41.63
45.67
91.56
49,88
CBSQE
56.50
42.60
46.45
92.15
50,41
XIMRQE
59.38
44.76
50.63
73.30
69.08
DirectQE
63.52
48.99
54.62
74.61
73.21
WMT20 EN-ZH
Supervised
BSQE
64.44
48.71
54.82
74.32
73.76
CBSQE
65.29
49.44
55.22
74.81
73.81
XLMRQE
28.92
27.84
35.46
79.74
44.47
DireciQE
31,41
29.86
36.07
78.52
45.93
WMT21 EN-ZII
Supervised
BSQE
31.14
29.36
34.87
76.65
45.49
CBSQE
31.59
30.15
36.41
78.91
46.13
Table 7: Average results for supervised methods on different QE datasets.
hyper- peeemearaesters described in (Zheng et al., 2021). We provide code for pre-processing data and all source code supplementary materials. We list the hyper-parameters as follows.
WMT20/21.
For CBSQE Pseudo QE data, we manually tune hyper-parameters one by one. We test sev- cral groups of typer-parameterr (temperature=5; lambda=0. 15, 0.2; epsilon=0.25, 0.3, 0.35). Finally, set the temperature to 5. The À is set to 0.15 for WMT19 EN-DE and 0.20 for WMT20/21 EN-ZH. The is set to 0.35 for WMT19 EN-DE and 0.30 for WMT20/21 EN-ZH. For self-ensemble, we pre- diet the QE labels 40 times which is the same as SSQE.
A.1 Generating Pseudo Data
A.1.1 DirectQE
For the generator, follow the setting in Direc- (QE, and one NVIDIA V100 GPU is used train the model. The mask ratio is 30% for the WMT19 EN-DE direction and 35% for WMT20/21 EN-ZH direction, respectively. We use the Adam optimizer with 81 = 0.9, B2 = 0.999 to optimize model param- eters. The initial learning rate is set to 0.2, and the Noam learning rate schedule is equipped with 8000 warm-up steps. We update the parameters every 20 batches, and the maximum number of tokens is set 8000 batch. If the validation performance gets no improvements for the last 20 runs, we will perform early stopping.
During data generation, we set the beam size to 4 and set the batch size to 512 for both BS and CBS.
A.2 Pre-Training and Fine-tuning
For unsupervised experiments, three RTX 3090Ti GPUs are used to train the models. We set the learn- ing rate to le-5 and use the Adam optimizer with 31 = 0.9, Bg = 0.999 to optimize model parameters. The clip norm is set to 1.0. During training, we set the maximum number of tokens in batch to 1600 tokens, and the update frequency is set to 3. We perform 5 epochs for every experiment and choose the best valid checkpoint.
A.1.2 BSQE and CBSQE
For WMT19 EN-DE, train translation model with one NVIDIA V100 GPU. Adam optimizer parameters 31 and B2 are set to 0.9 and 0.98, re- spectively, to optimize model parameters, and the weight decay is set to le-4. We set the initial learn- ing rate to 5e-4 and use the inverse square root learning schedule with 4000 warm-up steps. The dropout rate is set to 0.3, and the penalty of label smoothing is set to 0.1. During training, the maximum number of tokens in a batch 4096 We perform
For supervised experiments, one NVIDIA V100 GPU is used to train the models. We use the Adam optimizer with 31 = 0.9, B3 0.999 to optimize model parameters, and the clip norm is set to 1.0. The learning rate is set to 2e-5 for the WMT19 EN-DE direction, 5c-6 for the WMT20 EN-ZH di- rection, and le-5 for the WMT21 EN-ZH direction. During training,
set
and set the update frequency early stopping if the validation performance does not improve for the last 15 runs. For WMT20/21 EN-ZH, we use the translation model provided by
the maximum number of sentences in a batch to 5, and the update frequency is set to 20. We perform early stopping if the vali- dation performance does not improve for the last 5

---

Sent-level Test
Dataset
Settings
Method
Pearsont
MAE,
RMSEL
WMT19 EN EN-DE
DirectQE
42.52
12.88
20.53
BSQE
46.30
23.46
27.78
Unsupervised
SSQR (w/ dropout)
45.04
668.56
668.78
CBSQE
48.47
20.93
25.30
XLMRQE
52.21
11.27
17.10
DirectQE
56.26
11.49
18.25
Supervised
BSQF
56.45
11.34
17.12
CBSQE
56.65
11.06
17.04
DirectQE
45.15
34.59
39.00
BSQE
54.99
14.33
17.85
OCLINO EENE PAAN
Unsupervised
SSQE (w/ dropout)
43.82
676.87
677.14
CBSQE
59.09
13.66
17.07
XLMRQE
60.55
17.34
DirectQE
63.59
12.97
16.40
Supervised
BSQE
64.71
12.95
16.27
CBSQE
65.42
12.85
16.16
DirectQE
20.35
20.82
26.03
NMNTE EN-ZII ZII
BSQE
21.22
43.41
49.28
Unsupervised
SSQE (w/ dropout)
21.40
641.26
641.83
CBSQE
27.41
43.35
49.71
XLMRQE
30.38
23.07
27.05
DirectQE
31.99
23.30
28.22
Supervised
BSQE
31.88
27.44
32.07
CBSQE
32.15
24.24
28.99
Table 8: Main results of MAE and RMSE metrics on different test
epochs for the WMT19 EN-DE direction and the WMT20 EN-ZH direction and 10 epochs for the WMT21 EN-ZH direction.
B Results of MAE and RMSE Metrics
Table 8 shows the results of MAE and RMSE met- rics, CBSQE achieves the best MAEs and RMSEs in many cases. Since SSQE is not trained to mini- mize MAE loss, its MAEs and RMSEs are abnor- mally high. As shown in (Graham, 2015), the MAE metric has counter-intuitive effect QE system rankings. Therefore, we provide these results only for reference.
Significance Test
As recommended in (Specia et al., 2021), we use William's 's test' to compute statistical significance Pearson. The results listed in Table 9 and 10.

---

Dataset
Scttings
Method
DirectQE
BSQE
SSQE (w/ dropout)
CBSQE
DirectQE
BSQE
0.1024
WMT19 EN-DE
Unsupervised
SSQE (w/ dropout)
0.1551
0.2878 0.0076
CBSQE
0.0128
0.0454
DirectQE BSQE
2.54E-6
WMT20 EN-ZH
Unsupervised
SSQE (w/ dropout)
0.2782
3.29E-6
CBSQE
1.85E-10
0.0007
1.69--11
DirectQE BSQE 0.3745
WMT21 EN-ZH
Unsupervised
SSQE (w/ dropout)
0.3306
0.4455 0.0001
CBSQE
0.0074
0.0140
Table 9: Results of William's 's test for unsupervised methods. The p-values are marked in bold if p < 0.05.
Dataset
Settings
Method
XLMRQE
DirectQE
BSQE
CBSQE
XLMRQE
DirectQE:
0.0049
WMT19 EN-DE
Supervised
BSQE
0.0041
0.4978
CBSQE
0.0067
0.4716
0.4634
XLMRQE
DirectQE
0.0492
WMT20 EN-ZH
Supervised
BSQE 0.0108
0.1810
CBSQE
0.0005
0.0266
0.0990
XLMRQE 0.3104
DirectQE
WMT21 EN-ZH
Supervised
BSQE
0.1892
0.3328 0.2581
CBSQE
0.1405
0.4340
Table 10: Results of William's 's test for supervised methods. The p-values marked in bold if 0.05.


Reviews:
[{"review_id": "u4pGKLKziM", "reviewer": "Reviewer_tS9S", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We would like to thank you for your time and valuable suggestions for writing. \n\n> R1: The writing is unclear...Nonetheless, owing to certain instances of unclear articulation, such as in the method section (although I can grasp the general gist), I hold reservations regarding the method's practicality and ease of adoption within the community.\n\nWe will polish the expressions of the method part and revise the paper carefully. We would like to clear the practicality as follows. Generally speaking, we improve the generation probabilities of some reference tokens during beam search. The adjustments happen only if the probabilities of any reference token are above a threshold so that the less fluent hypothesis will not be mistakenly rewarded. We want to preserve the main structure of references so that we can obtain accurate labels using TER tools. Therefore, CBSQE assigns larger improvements for closer reference tokens when adjusting. As shown in section 5.1, CBSQE successfully improves the pseudo data quality. Ablation studies further show that each part of the constraints is important for improvement. The whole algorithm is efficient (section 5.3) and open source (supplementary materials).", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. Better performances compared with the baselines.\n2. The main idea is simple and straightforward. ", "reasons_to_reject": "The writing is unclear: I suggest that the author proofread and modify the article's expression, especially the causal logic and the method part. I often have to guess the meaning of some sentences: E.g., in lines 194-198, \"Thus, we assign the same probabilities to ... To this end, we define the distance function ...\". This kind of portion is rather challenging for me to comprehend as I progress through it.\n\nOn the whole, I value the core concept of the method, which is straightforward and yields consistent improvements in both supervised and unsupervised scenarios. Nonetheless, owing to certain instances of unclear articulation, such as in the method section (although I can grasp the general gist), I hold reservations regarding the method's practicality and ease of adoption within the community.\n\n\n\n\n", "questions_to_authors": "", "paper_topic_and_main_contributions": "This paper target the \"false negative\" problem of pseudo data construction for quality estimation, i.e., the previous methods may ignore semantic equivalence and produce pseudo data that are labeled as \"BAD\" but actually they are reasonable translation. They propose an MT-based model equipped with a constrained beam search to alleviate this problem. The motivation makes sense to me. The experiments show that the method works better than baselines in both supervised and unsupervised settings, and the analysis shows that the proposed model results in better pseudo labels.", "excitement": 3, "reproducibility": 3, "ethical_concerns": "Yes", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Typos_Grammar_Style_and_Presentation_Improvements": "line 174-176: I can roughly understand your meaning, but I suggest saying it straightforwardly in case some misleading.\n\nThe expression of the method part, including the Algorithm1, needs to polish."}, "extra_text": {}}, {"review_id": "nKgaoYxpdB", "reviewer": "Reviewer_JQ3q", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 1, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your time and positive comments. We would like to highlight the contribution as follows. Pseudo data methods play an essential role in the QE area due to the lack of labeled QE data. However, existing pseudo data methods suffer from either noisy pseudo transaltions or labels. In this paper, we proposed a constrained beam search algorithm (CBSQE) that reconciles the pseudo label quality with pseudo translation quality. Analysis shows that CBSQE improves pseudo data quality. Therefore CBSQE outperforms baselines in both supervised and unsupervised settings.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "Paper is well formed with lots of experiments and have enough details of the experiment setups. It also covers different aspect like efficiency and abdication study.", "reasons_to_reject": "I don't have any reason to reject this paper", "questions_to_authors": "", "paper_topic_and_main_contributions": "The paper proposed a reranking approach to do quality estimation for Machine Translation output. A separate model is being trained to do quality estimation on the fly during the decoding process(beam search) and rank higher quality output higher in the beam.", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "i7pBuhfKGI", "reviewer": "Reviewer_Zcy9", "summary": "", "questions": "Q1: As quality estimation measures at the sentence level, in addition to Pearson correlation factor,  I would expect the mean absolute error (MAE) and the root-mean-square deviation (RMSE) as two other informative metrics usually used. Can the authors provide the results for these two measures?  \nQ2: Did the authors run a significance test on the obtained results? Such as T-test for instance?\n", "limitations": "", "rating": 3, "confidence": 3, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We would like to thank you for the constructive suggestions.\n>R1: In many cases the improvements are not very important.\n\nYes, for all supervised scenarios, the improvements seem incremental. As discussed in the paper, QE models tend to learn shortcuts (such as lexical artifacts, and fluency) from real QE datasets [1]. As a result,  it seems that the benefits of improved pseudo data are moderate after fine-tuning.\n\n>R2: Missing important metrics at the sentence level evaluation (MAE and RMSE).\n\n>Q1: As quality estimation measures at the sentence level, in addition to the Pearson correlation factor, I would expect the mean absolute error (MAE) and the root-mean-square deviation (RMSE) as two other informative metrics usually used. Can the authors provide the results for these two measures?\n\nAs shown in [2], the MAE metric has a counterintuitive effect on QE system rankings. Therefore, we didn't report MAE and RMSE metrics in the main results as many QE works do (e.g., SSQE, and the official WMT QE shared task paper[3]). We would like to add MAE and RMSE metrics to the final version for more information.\n\n>R3: It is not clear to me if the results are significant\n\n>Q2: Did the authors run a significance test on the obtained results? Such as T-test for instance?\n\nWe have performed the significance test on unsupervised results.\nAs recommended in [3], we use [William’s test](https://github.com/ygraham/mt-qe-eval) to compute statistical significance on Pearson.\nThe results are listed as follows, and the p-values are marked in bold if p < 0.05.\nCBSQE significantly outperforms baselines on all datasets.\n\n|  |  | WMT19 EN-DE |  |  |\n| :----: | :----: | :----: | :----: | :----: |\n|  | **DirectQE** | **BSQE** | **SSQE** | **CBSQE** |\n| **DirectQE** | - | - | - | - |\n| **BSQE** | 0.1024 | - | - | - |\n| **SSQE** | 0.1551 | 0.2878 | - | - |\n| **CBSQE** | **0.0128** | **0.0076** | **0.0454** | - |\n\n|  |  | WMT20 EN-ZH |  |  |\n| :----: | :----: | :----: | :----: | :----: |\n|  | **DirectQE** | **BSQE** | **SSQE** | **CBSQE** |\n| **DirectQE** | - | - | - | - |\n| **BSQE** | **2.54E-6** | - | - | - |\n| **SSQE** | 0.2782 | **3.29E-06** | - | - |\n| **CBSQE** | **1.85E-10** | **0.0007** | **1.69E-11** | - |\n\n|  |  | WMT21 EN-ZH |  |  |\n| :----: | :----: | :----: | :----: | :----: |\n|  | **DirectQE** | **BSQE** | **SSQE** | **CBSQE** |\n| **DirectQE** | - | - | - | - |\n| **BSQE** | 0.3745 | - | - | - |\n| **SSQE** | 0.3306 | 0.4455 | - | - |\n| **CBSQE** | **0.0074** | **0.0001** | **0.0140** | - |\n\nWe have not performed the significance test on supervised results due to the limited time. We would like to provide these results in the final paper. However, we suppose that CBSQE could not significantly outperform baselines in the supervised setting. The reason is the same as we discussed for R1.\n\n[1] Sun et al. Are we Estimating or Guesstimating Translation Quality? ACL2020.\n\n[2] Yvette Graham. Improving Evaluation of Machine Translation Quality Estimation. ACL 2015.\n\n[3] Specia et al. Findings of the WMT 2021 Shared Task on Quality Estimation. WMT2021.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- Overall the paper is clear and well written\n- The proposed method shows improvements over the baselines", "reasons_to_reject": "- In many cases the improvements are not very important\n- Missing important metrics at the sentence level evaluation (MAE and RMSE)\n- It is not clear to me if the results are significant", "questions_to_authors": "Q1: As quality estimation measures at the sentence level, in addition to Pearson correlation factor,  I would expect the mean absolute error (MAE) and the root-mean-square deviation (RMSE) as two other informative metrics usually used. Can the authors provide the results for these two measures?  \nQ2: Did the authors run a significance test on the obtained results? Such as T-test for instance?\n", "paper_topic_and_main_contributions": "This paper addresses the task of MT quality estimation.    \nThe authors propose to generate pseudo data using MT model with constrained beam search.\nThe proposed method (CBSQE) keeps the main structure of references by increasing the generation probabilities of the reference tokens with regards to to their original translation probabilities and positions.\nExperiments were conducted on the WMT19 (English-German)  shared task and the WMT20 and WMT21 (English-Chinese) shared tasks in both supervised and unsupervised scenarios.\nThe proposed method (CBSQE) was compared to several related work baselines: DirectQE, BSQE, the unsupervised method  SSQE in addition to a standard supervised baseline XLMRQE.\nThe results overall show improvements in most cases by the proposed method in both supervised and unsupervised scenarios.  ", "excitement": 3, "reproducibility": 2, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Typos_Grammar_Style_and_Presentation_Improvements": "- line 275 \"...based on\" instead of \"...base on\""}, "extra_text": {}}, {"review_id": "HqvaUBKYnx", "reviewer": "Reviewer_Ehgu", "summary": "", "questions": "", "limitations": "", "rating": 3, "confidence": 4, "soundness": 3, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your time and the positive evaluation of the soundness. We would like to discuss the following concerns in the final paper.\n\n> R1: The proposed method relies on a pre-trained MT model to create augmentation data, which is not applicable for a lot of cases (such as low-resource languages).\n\nIn many applications, we have access to the target MT model that needs to be evaluated. Following this trend, WMT QE shared tasks have adopted the white-box setting in recent years [1]. In the black-box setting, we can obtain a competitive multilingual MT model [2] for low-resource languages. \n\nThe parallel data is the bottleneck of pseudo data methods for low-resource languages. As shown in Figure 3, CBSQE still achieves competitive results with 100K parallel pairs which could satisfy many language pairs. We leave the exploration of fully unsupervised pseudo data methods in the future. Because it is quite heavy to include both supervised, unsupervised, and fully unsupervised settings in one paper. Considering that DirectQE and SSQE only focus on the supervised and unsupervised setting respectively.\n\n> R2: The calculation of HTER label relied on automatic scripts can hardly be scaled to other annotations such as MQM or DA. If it is not the case, then the author should experiment on other languages and annotations to verify the scalability.\n\nAs shown in the WMT2022 QE shared task paper [3], the advanced pseudo data methods for MQM (rank 1st on EN-DE sentence- and word-level, rank 2nd on Multilingual sentence-level) and DA (rank 2nd on English-Marathi word-level) obtain pseudo labels based on simple rules. For the MQM annotation, they sample tokens from top-2, 10, and 100 NMT probabilities to simulate minor, major, and critical errors. Then, the MQM scores can be calculated as the weighted sum of different errors, which is the definition of MQM. For the DA annotation, they just take the replaced ratio of reference tokens as the DA score.\n\n Although different annotations are proposed for different applications, they all reflect the quality of translations and share similar word-level tags. The analysis of the pseudo data quality (section 5.1) is based on word-level labels and NMT probabilities which are not dependent on the specific annotation. Hence, CBSQE generates improved pseudo translations and word-level labels for different annotations. The sentence scores of different annotations can be obtained with simple designs as introduced above. \n\n\n> R3: The contribution is focused on the constrained beam search for QE, which is somewhat less qualified for a long paper.\n\nThis paper highlights that better pseudo data concurrently need MT-like pseudo translations and accurate quality labels. To this end, it is essential to define the proper constraints of beam search. CBSQE needs to decide when to improve the probability of references and which reference tokens should be improved. CBSQE only improves the probability of references when the probability of any reference token is above a threshold so that the less fluent hypothesis will not be mistakenly rewarded. CBSQE uses a distance function to assign larger improvements for closer reference tokens from the right side so that we can preserve the main structure of references and therefore obtain accurate labels using TER tools. As shown in section 5.1, CBSQE successfully improves pseudo data quality. Ablation studies further show that each part of the constraints is important for improvement.\n\n> Writing: The description of 3.1 is a bit confusing and could be improved.\n\nWe will polish the expressions of the method part and revise the paper carefully. \n\n\n[1] https://www.statmt.org/wmt21/quality-estimation-task.html\n\n[2] Tang et al. Multilingual Translation with Extensible Multilingual Pretraining and Finetuning. arXiv:2008.00401.\n\n[3] Geng et al. NJUNLP’s Participation for the WMT2022 Quality Estimation Shared Task. WMT2022.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. The proposed data-augmentation method only manipulate the decoding process, without demanding the post-train process.\n2. It is enlightening to alleviating the deviance by constrained beam search.\n3. Results on both supervised and zero-shot setting are improved significantly compared with previous methods based on the same data.", "reasons_to_reject": "1. The proposed method relies on a pre-trained MT model to create augmentation data, which is not applicable for a lot of cases (such low-resource languages). \n2 The calculation of HTER label relied on automatic scripts can hardly be scaled to other annotations such as MQM or DA. If it is not the case, then the author should experiment on other languages and annotations to verify the scalability.\n3. The contribution is focused on the constrained beam search for QE, which is somewhat lesss qualified for a long paper.", "questions_to_authors": "", "paper_topic_and_main_contributions": "This paper proposes a new data-augmentation method for translation quality estimation (QE). The author performs constrained beam search on the MT model and manipulate the generation process according to MT probabilities. They make comparison with other data-augmentation methods in both supervised and zero-shot scenarios to verify their effectiveness.", "excitement": 3, "reproducibility": 4, "ethical_concerns": "No", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"Typos_Grammar_Style_and_Presentation_Improvements": "The description of 3.1 is a bit confusing and could be improved."}, "extra_text": {}}]

Author rebuttal:

Meta-review: The authors propose a method to create synthetic data for training an MT QE model using a machine translation (MT) model and constrained beam search. The main idea is simple (which is good!), straightforward and leads to quality improvements. The author might want to improve the writing at some points in the paper. The reviewers overall agree that this is solid work and suggested only minor changes.
Decision: Accept-Main