Rewriter-Evaluator Framework for Neural Machine Translation
Authors: Yangming Li
Kaisheng Yao
Venue:  (2021)
Keywords: Neural Machine Translation
Post-editing mechanism
Polish Mechanism
Proper Termination Policy

Abstract: Encoder-decoder architecture has been widely used in neural machine translation (NMT). A few methods have been proposed to improve it with multiple passes of decoding. However, their full potential is limited by a lack of appropriate termination policy. To address this issue, we present a novel framework, Rewriter-Evaluator. It consists of a rewriter and an evaluator. Translating a source sentence involves multiple passes. At every pass, the rewriter produces a new translation to improve the past translation and the evaluator estimates the translation quality to decide whether to terminate the rewriting process. We also propose a prioritized gradient descent (PGD) method that facilitates training the rewriter and the evaluator jointly. Though incurring multiple passes of decoding, Rewriter-Evaluator with the proposed PGD method can be trained with similar time to that of training encoder-decoder models. We apply the proposed framework to improve the general NMT models (e.g., Transformer). We conduct extensive experiments on two translation tasks, Chinese-English and English-German, and show that the proposed framework notably improves the performances of NMT models and significantly outperforms previous baselines.

Under review as a conference paper at ICLR 2021
REWRITER-EVALUATOR FRAMEWORK FOR NEURAL MACHINE TRANSLATION
Anonymous authors Paper under double-blind review
ABSTRACT
Encoder-decoder architecture has been widely used in neural machine translation (NMT). few methods have been proposed to improve it with multiple passes of decoding. However, their full potential is limited by lack of appropriate ter- mination policy. To address this issue, we present a novel framework, Rewriter- Evaluator. It consists of rewriter and evaluator. Translating a source sentence involves multiple passes. At every pass, the rewriter produces new translation to improve the past translation and the evaluator estimates the translation quality decide whether to terminate the rewriting process. We also propose a prioritized gradient descent (PGD) method that facilitates training the rewriter and the eval- uator jointly. Though incurring multiple passes of decoding, Rewriter-Evalle with the proposed PGD method be trained with similar time to that of train- ing encoder-decodeer models. We apply the proposed framework to improve the general NMT models (c.g.. Transformer). We conduct extensive experiments on two translation tasks, Chinese-Englise and Aiglish-Germanl and show that the proposed framework notably improves the performances of NMT models and sig- nificantly outperforms previous baselines.
INTRODUCTION
Encoder-decoder architecture (Sutskever et al., 2014) has been extensively used in neural machine translation (NMT) (Vaswani et al., 2017; Zhang al., 2019). Given source sentence, an encoder firstly converts it into hidden representations, which are then conditioned by decoder to generate the target sentence. Attention mechanism (Bahdanau et al., 2015) very effective in learning the alignment between source sentence and target sentence. Hence, attention mechanism is usually used the architecture to improve its capability, such as capturing long-distance dependencies.
Similar to traditional machine learning efforts (Zhang & Gildea, 2008), some recent approaches deep learning attempt to improve encoder-decoder architecture with multiple passes of decod- ing (Xia et al., 2017; Zhang et al., 2018; Geng et al., 2018). NMT refers this polish mecha- nism (Niehues et al., 2016). Under this scheme, more than one translations are generated for a source sentence and, except for the first translation, each of them is based on the translation from the previous decoding pass. While these methods have achieved promising results, they lack a proper termination policy to the multi-turn process. Xia et al. (2017); Zhang et al. (2018) adopt fixed number of decoding passes that can be inflexible in deciding the optimal number of decoding passes. Geng al. (2018) use reinforcement learning (RL) (Sutton al., 2000) automatically decide the optimal number of decoding passes. However, RL is unstable due to its high variance of gradient estimation and objective instability (Boyan Moore, 1995). Since these methods may have premature termination or over translation, their potential can be limited.
To address this problem, we propose a novel framework, newriter-Evaluatan in this paper. It con- sists of a rewriter and an evaluator. The translation process involves multiple passes. Given source sentence, at every pass, the rewriter generates a new target sequence aiming at improving the trans- lation from prior passes, and the evaluator measures the translation quality to determine whether to terminate the rewriting process. We also propose prioritized gradient descent (PGD) method that facilitates training the rewriter and the evaluator jointly. The essential idea is using a priority queue to improve sampling efficiency by collecting the translation cases that yield low scores from the evaluator for next-pass rewriting. The size of the queue is few times larger than the batch size.

---

Under review as a conference paper at ICLR 2021
Source Encoder
Estimator
Quality Score
Target Encoder
Evaluator
Source Sequence
Target Sequence
Source Encoder
Decoder
Target Sentence
Target Encoder
Rewriter
x
Figure 1: This illustrates the architecture of the propose
Although Newriter-Evaluator involves multiple decoding passes, training time using PGD method comparable to that of training an encoder-decoder (Bottou Bousquet, 2008) that doesn't have multiple decoding passes.
We apply Rewriter-Evaluate to improve the widely used NMT models, RNNSearch (Bahdanau et al., 2015) and Transformer (Vaswani et al., 2017). Extensive experiments have been conducted on two translation tasks, Chinese-Englins and English-German, to verify the proposed method. The results demonstrate that the proposed framework notably improves the performance of NMT models and significantly outperforms prior methods.
REWRITE--VVLUATTO
2.
ARCHITECTURE
The architecture of Rewriter-Evaluato is shown in Figure 2. Our framework consists of rewriter and an evaluator. The process of translating a source sentence x consists of multiple passes. At each iteration &: 2 assuming the translation from the prior iteration k is (k)-1), the rewriter produces translation (*)) and the evaluator estimates its quality score (*)) Formally, the k-th iteration of a translation process is defined as
(k) rewriter(x, z (R--))), (*)) evaluator z (*)
Initially, (0) and q (0) are respectively set as empty string and a large negative number.
In the following, we describe an architecture that shares the sentence encoders between rewriter and evaluator. However, the proposed method is not restricted this particular architecture.
At the k-th pass, the source sequence and the translation (k=1) from the previous pass respec- tively embedded by a source encoder SSE and a target encoder TTE,
h = S p (k-1)
(2 (k-)))
The rewriter z contains decoder g DEC to produce a new translation (A); z (k) (xxx (k (k-1)) = DBC (h, (k-1))
(3)
where g DEC can be any conditional language model (e.g., Transformer).
The evaluator & measures the translation quality with an estimator EST as
(k) o(xx z (k)) g EST (h, P.pp" (*)).
(4)
Here the estimator can be any text matching model, e.g., ESIM (Chen et al., 2016).
The above procedure repeatedly carried out until the following condition is satisfied:
g + 8 < (k-1)), 0,
(5)
or a certain number of iterations K > 0 is reached. the former case, we use (k--1) as the final translation. In the latter case, the last translation 2 (K) is accepted.
2

---

Under review as a conference paper at ICLR 2021
Algorithm 1: Optimization via Prioritized Gradient Descent
Rewriter y evaluator di training set T; batch size B: expected iterations E. Output: Well trained rewriter yo; well trained evaluator Φ. Initialize an empty priority queue with capacity C B x E.
Input:
while Models are not converged do Randomly sample a B-sized batch of training samples from for (x, y) E S do
Push quadruple (x,y, ("SOS, "EOS"), -00) into queue
Pop the samples that exceed that the capacity from queue Initialize an empty priority queue D of limited size C. Initialize an empty list to collect samples for training.
for (x, y, z((=(),1, r((k-1) E A do
Obtain rewrite (x) and evaluation score (8)), respectivly, using Eqs. (3) and (4).
10
Add sample y, y, 2(K-1), (x)) into list F.
11
Compute new quality score r using Eq. (8).
Push quadruple (x, y, ((()) into queue D.
13
Update rewriter with the samples in list F using Eq. (6). Update evaluator & with the samples in list F using Eq. (7).
15
D.
2.2 TRAINING CRITERIA
The rewriter d is trained with teacher forcing. To learn generating word position it uses ground truth of the prior time steps [yo. -i-- as follows
v (x, (k-1), [Vo. U1. II).)). J
Σ - log(m,(MA))
(6)
ISSN
where N is the length of The first symbol yo and the last symbol yn are "SOS" for sentence beginning and "EOS" for sentence ending, respectively.
For evaluator d, when presented with the ground truth and the predicted quality of current trans- lation from Eq. (4), incurs hinge loss as
Φ(x,)), J max(0, 1- q" + (1))).
The translation (k) can be generated via greedy search or beam search.
3 PRIORITIZED GRADIENT DESCENT METHOD
We introduce, in Algorithm 1, a prioritized gradient descent (PGD) method to bias the training towards rewriting poor translations with certain quality scores. Different from random sampling from the whole training set in the stochastic gradient descent (SGD), it uses priority queue to keep poorly translated cases for sampling in each mini-batch of the training.
The procedure starts with empty priority queue (1-st line) with capacity C' no greater than the product of mini-batch size B and expected iterations E (i.e., C' < x E). The priority queue ordered with quality score in ascending order with the top one corresponding to the lowest score. The quality score of a certain sample yx z (k)) is computed as
(k) = BLEU(Z (k) .y) + + (K)
where weight p is controlled by annealing schedule ما1 with being training epoch number. The first few epochs mainly rely on BLEU scores. The quality score is more balanced towards the evaluation score g with increasing number of training epochs.
Therefore, the samples with high quality rate that exceed capacity C are discarded (7-th line). The remaining cases are further rewriten using rewriter & get new translation (k) and quality scores
3

---

Under review as a conference paper at ICLR 2021
(()) from Eq. (8) (11-th 14-th lines). Eventually, train the rewriter & and the evaluator respectively using Eq. (7) and Eq. (6) (16-th and 17-th lines).
This method is designed to be both effective and efficient for training the models with Rewriter- Evaluator framework. For effectiveness, it keeps the low-quality translated samples in the queue for possible multiple passes of usages, facilitating the re-editing paradigm (Niehues al., 2016) in training. For efficiency, can obtain training time comparable to training an encoder-decoder architecture without multi-pass decoding. Assume the run-time cost for training sample is P and the training set has Q batches. Using parallel mini-batch computation, the training time for encoder- decoder about (P BB Q P x Q. Analogously, the total training time for the proposed method is about (P (B E))
E) * = P
APPLICATIONS
We apply Rewriter-Evaluator improve RNNSearch and Transformer, both of which are widely adopted in NMT. In the following, we denote the length of the source sentence as M and the length of translation k-th pass as Lk--
Improving RNNSearch. Here both encoders fSE and fTB are Bi-directional GRU (Chung al., 2014). We omit the details of RNNSearch and denote its hidden state at word position i as s; and its next word distribution on predefined vocabulary V as T. E RI.
The rewriter pointer generator (Gu al., 2016; See al., 2017) architecture. Words in the translated sentence are predicted sequentially from the predefined word vocabulary V or directly copied from the target candidate words. Firstly, for position i, pointer attention Th E R is computed on the j-th word in the previous translation (k--1) as
exp(BLa) Bij = vD lanh(W ns; + VDP) (np))""), Fij Eisighe exp(Big) 1 SJ LE-1-
(9)
Then, for position i, its word is selected from the predefined vocabulary in size [V] and the previous translation (kk-1), Hence, the probability n. ni(us] is computed
A nH lual + (1 Ax) Eosish m= n w= J.j
y/ E Ω n y z (k-1)
TYlu
yi
E
Ω 0
y: $ (k-1)
Eosish (t=))=1) w=w
y
$ S
ys E (k)
(10)
where weight X; is computed as hhe)
For evaluator d, the beginning, given context representations that are computed via Eq. (2), we apply co-attention mechanism (Parikh et al., 2016) to capture the semantic alignment between element i of source sequence and element of translation z (8)
ai = h? WED, 1sis M; Lu
exp(an.)
h, Σ
Pj
Eisish exp(oaa.)
(1I)
1SiSL:
exp(os.))
Pj (k)
Σ
exp(os)))
EisIsM exp(os.))
I515M
We then use pooling extract features and compute the quality score:
((
{
))
(+)
vT
I
h
1
Σ
Pj
(K)
(12)
N
Lk
ISi<M
I<t<L
where operation is the column-wise vector concatenation.
4

---

Under review as a conference paper at ICLR 2021
Improving Transformer. We keep the whole architecture (Vaswani al., 2017) unchanged and only modify ontutt format and mask matrices.
The input to Transformer is modified to x' concatenation of source sentence x, special symbol "ALIGN", and the last translation 2(A-1) (i.e., x' = ① ["ALIGN"] @ (*=-1)) We design the following mask matrix so that words in cannot attend to those in ((--1) and vice versa:
1MXM
0x
OMxLE-1 1ix
11xM
(13)
OF 1xLe-1
1
This arrangement the input mask also enables symbol "ALIGN" attend all of the words in both source sentence x and past translation (kk-1)
For rewriter at k-th pass, we adopt the same computation procedure in Eq. (9) and Eq. (10). To accomodate its sequential generation of a word at position here the hidden state s; is the output (k-1) is from Transformer for from Transformer for generating target word at position i - and Pj position j in the previous translation z (k-1))
The evaluator obtains a score using the representation h ALIGN from symbol "ALIGN" as
( = v
(14)
which vector Vg is a learnable parameter.
EXPERIMENTS
We have conducted extensive experiments translation tasks, Sehinese-English (Zh+EEn) and (En->De). The results demonstrate that Rewriter-Evaluato significantly improves the performances of NMT models (e.g., RNNSearch) and notably outperforms prior post-editing methods. We further conducted oracle experiments, ablation studies, and running time comparison verify the effectiveness of our framework.
5.1 SETTINGS
For Chinese-English translation, the training set consists of 1.25M sentence pairs extracted from LDC corpora. They are mainly chosen from LDC2002E18, LDC2003E07, LDC2003E14, Hansards portion of LDC2004T07, LDC2004T08, and LDC2005T06. We use NIST 2002 (MT02) dataset as the validation set, which has 878 sentences, and the NIST 2004 (MT04), NIST 2005 (MT05), and NIST 2006 (MT06) datasets as the test sets, which contain 1788, 1082 and 1664 sentences respectively. For Eiglish-German translation, we use WMT 2015 training data that contains 4.46M sentence pairs with 116.1M English words and 108. 9M German words. Following previous works, we segment words via byte pair encoding (BPE) (Sennrich et al., 2015). The news-test 2013 was used as dev set and the news-test 2015 test set. We measure the translation quality by BLEU metric (Papineni et al., 2002). For Zh->En, we adopt insenssiie BLEU score. For En->De, case-sensitive BLEU score is calculated by multi-bleu.pl script².
We train all models with sentences of length up to 50 words. We set the vocabulary size to 30K for Chinese--ngllise translation, and 50K for Ennlish translation. For RNNSearch and RNNSearch w/ Rewriter-Evalutte and the beam size in testing 10. Dropout rate is set as 0.5. For Transformer and Transformer w/ Rewriter- Evvauttrr,,,lutoor,, we use default settings in fairseq (Ott et al., 2019). The batch size is fixed as 80 and the expected iterations E set as The values of 8 in Eq. (5) and maximum iterations K are set as 0.01 and 6, respectively. We adopt Rmsprop (Graves, 2013).
the dimension of word embedding and hidden layer is 512,
5.2
MAIN RESULTS
Our models are compared with strong baselines: 1) Deliberation Network (Xia et al., 2017) adopt second decoder for polishing the raw sequence produced by the first-pass decoder; 2) ABD-
5

---

Under review as a conference paper at ICLR 2021
Zh->En (NIST)
En->De
Method
MT04
MT05
MT06
Avg.
(WMT'15
Deliberation Network (Xia et al., 2017)
40.56
37.67
37.20
38.48
26.01
ABD-NMT (Zhang et al., 2018)
41.20
38.07
37.59
38.95
26.26
Adaptive Multi-pass Decoder (Geng et al., 2018)
41.43
38.54
37.86
39.28
26.77
RNNsearch (Bahdanau et al., 2015)
40.41
36.57
36.01
37.66
25.08
w/ Rewriter- Evaluator
43.13
39.82
40.95
41.30
27.16
Our Work
Transformer (Vaswani et al., 2017)
47.11
47.58
46.70
47.13
26.45
w/ Rewriter-Evvlluttr
48.65
48.31
49.59
48.85
28.13
Table I: Experiment results of our models and all baselines. The numbers in bold indicate that the improvements are statistically significant with p-value p < 0.05 under t-test.
NMT (Zhang et al., 2018) utilize backward decoder to generate translation, and a forward decoder refines it with an attention model; and 3) Adaptive Multi-pass Decoder (Geng et al., 2018) integrate polishing mechanism into NMT model via RL.
For En translation, we adopt all the results of baselines as reported in Geng et al. (2018). For the results of ABD-NMT in Geng et al. (2018). Other performances obtained by our re-implementations. For RNNSearch, Transformer, and Rewriter-fivaluator, we implement them top of fairseq (Ott et al., 2019)².
En-+De,
The main results are presented in Table The proposed Rewriter-Evvluatte consistently improves RNNSearch and Transformer. Our best results are obtained by using Rewriter-Evalaator Trans- former. It's the first time that a NMT model with iterative rewriting consistently outperforms other post-editing based neural machine translation baselines on large translation tasks by sizeable mar- gins. We describe the improvements in detail in the following.
on RNNScarch. Improving RNNSearch with Rewriter-Evallator obtains relative improvements of 9.67% on NIST dataset and 8. 8.29% on WMT' dataset. These results confirm that framework is effective in improving translation quality of encoder-decoder mod- els (Sutskever et al., 2014; Bahdanau et al., 2015). BLEU scores of our framework with RNNSearch also consistently higher than methods in post-edit mechanisms. For example, compared with Adaptive Multi-pass Decoder, the best one of them, we achieve higher BLEU scores by relatively 5.14% and 1.46% on NIST and WMT' 15 dataset, respectively.
Rewriten-Evaluator on Transformer. Improving Transformer with Rewriter- Evaluator achieves relative improve- ments of 3.65% and 6.35% on NIST dataset and WMT' dataset, respec- tively. These improvements pecially significant, considering that Transformer itself consistently out- performs RNNSearch in our experi- ments. The progress of using Rewriter- Evaluator on Transform is very ap- parent when comparing to the post- editing methods in Xia et al. (2017); Zhang et al. (2018); Geng et al. (2018). For example, comparing with the best post-editing method of Adaptive Multi-
29
Oracle
28.57
ailan Nooms on Test
28
Evaluator
27.16
Average
26.34
26.54
26.79
26.51
26.35
26.62
25
Number Rewriting ltnntions
Figure 2: Oracle experiment on En-*De task with Rewriter-Evaluator on RNNSearch.
pass Decoder (Geng et al., 2018), Rewriter-Evaluato on Transformer improves BLEU scores size- ably by relative 24.36% on NIST dataset and 5.08% on WMT' 15 dataset.

---

Under review as a conference paper at ICLR 2021
5.3 ORACLE EXPERIMENT
One of the attractive features of our framework is its ability to select final translation results from translation candidates based on scores from evaluator. We plot the test BLEU scores versus rewriting turn k in Figure 5.2. The first iteration corresponds to the encoder-decoder result. BLEU scores from each iteration are improved over the first iteration but are consistently lower than using evaluator to select answers from them. For instance, iteration 3 obtains the highest BLEU score of 26.34 for individual iteration, but is worse than 27.16 from Evaluator. This strongly verifies the effectiveness of the evaluator. We also achieve an oracle result by selecting translations with the highest BLEU scores given ground truths. The oracle makes a higher BLEU score by 5.11%, indicating potential improvements from the further refinement of evaluator Φ
5.4 ABLATION STUDIES
Table shows ablation studies. The last row is from
S
K
Zh->En
En-+De
X
48.21
27.19
on Transformer.
48.02
27.45
0.0
47.99
27.23
Parameter Sharing. The encoders
0.1
48.65
27.87
parameters from Eq. (2) are shared in Eqs. (3) and (4) so that their repre-
1.0
48,26
27.51
0.0
48.53
27.71
sentations are consistently improving for both evaluator and rewriter. In- deed, the scores from not sharing pa- rameters, where rewriter and evalua- tor have their own encoders, drop by 1.32% and 3.34%.
0.1
47.82
27.15
2
48.55
27.56
48.71
27.82
8
48.87
28.09
A
0.01
6
48.85
28.13
Table 2: Ablation studies. S denotes parameter sharing, C denotes copy mechanism, and A denotes weight annealing.
Copy Mechanism in Rewriter. Ability of copying words from past translation via Eqs. (9) and (10) contributes to 1.73% and 2.42% BLEU score increase.
Annealing in Prioritized Training. Equation (8) uses with annealing to incorporate evaluator scores. Interestingly, only using BLEU scores in selecting training samples, i.e., p 0.0. decreases BLEU scores on test set by 1.76% and 3.20% (3-rd row). On the other hand, using fixed value (4-st and 5-st rows), the scores drop by 0.81% and 1.56%. We observe that evaluators are prone make wrong judgment initially, therefore adopting the annealing scheme that gradually increases 1.0 is indeed useful. The results also indicate the scores from evaluator contain complementary information from BLEU score.
Threshold 5. The threshold in Eq. (5) controls the tolerance of stoping the iterative rewrite process on sentences that may have low translation quality. With 8 set to 0.0, the Rewriter continues its editing until it cannot improve evaluation scores further. However, test set BLEU scores decline by 0.66% and 1.49% on both datasets. On the other hand, having a large tolerance, using 5 0.1, is catastrophic (7-th line) as many low-quality translation are not rewritten at all.
Maximum Number of Iterations K. Increasing the maximum number of iterations K in training and test is generally beneficial. However, as it consumes more time with larger K during testing, we set the maximum iterations K to 6 and expected iterations E to 3 in Algorithm 1.
5.5 RUNNING TIME COMPARISON
Table 3 shows the running times of different models. We use 8 GPUs with expected iteration number E 3, maximum iteration number 6, and beam size being 5. Results confirm that training times between
Evaluutto and encoder-decoder comparable.
Rumning time for evaluation using decoder model. Nevertheless, Table demonstrates that Rewriter-Eivllator significantly improves
is approximately times of an encoder-

---

Under review as a conference paper at ICLR 2021
Zh En (NIST)
En-sDe (WMT 15)
Method
Training
Evaluation (ave.)
Training
Evaluation
RNNSearch
2h17m
7m
8h12m
12m
RNNSearch w/ Rewriter-Evaluttor
2h39m
31m
9h03m
47m
Transformer
1h11m
9m
5h47m
14m
Transformer w/ Rewritr-Evvalutto
1h29m
36m
6h15m
53m
Table Running time comparison on NIST and WMT 15.
BLEU scores in comparison to encoder-decodet models. For instance, on NIST dataset, improv- ing RNNSearch via Rewriter-Evaluator increases the testing time by 4.43 times but significantly improves the BLEU score by 8.81%
6 RELATED WORK
Our work is closely related to recent efforts in end-to-end multi-pass decoding (Xia et al., 2017; Zhang et al., 2018; Geng et al., 2018). The models generate multiple target sentences for a source sentence and, except for the first one, each of them is based on the sentence generated in previous turn. For example, Xia et al. (2017) propose deliberation network that uses a second decoder to polish the raw sequence produced by the first-pass decoder. While these methods have achieved promising results, they lack proper termination policy to the multi-pass translation process. Zhang al. (2018) adopt predefined number of decoding passes, which is not flexible. Geng et al. (2018) incorporate post-editing mechanism into NMT model via RL. However, RL is notoriously unstable for training because of the high variance of gradient estimation.
An alternative line of research focuses on computer-assisted translation (CAT) (Barrachina et al., 2009) that collaborates existing machine translation technologies with human translators. In such a situation, quality estimation (QE) and automatic post-editing (APE) play important roles in reducing human efforts. Word-level QE (Kim & Lee, 2016; Martins et al., 2017; Fan et al., 2019) assigns a label of OK or BAD to every word in the translation. For example, work in Basu et al. (2018) measures the similarity of the source context of the target word with the context for which the word is retained. APE corrects typical and repetitive mistakes found in the generated sequence. For instance, work in Vu & Haffari (2018) interleaves generating and executing the edit actions to rectify errors. Most recently, some works explore multi-task learning of QE and APE (Chatterjee et al., 2018: Kim et al., 2017), which is analogous to us. While the translation quality has indeed been improved, these approaches heavily rely on extra handcraft annotation that is expensive and domain-specifie. Moreover, they essentially serve as the post-process modules in a pipeline-based translation system, instead of directly empowering machine translation models with mechanisms to estimate and improve the translation quality.
7 CONCLUSION
In this work, we present novel framework, Rewriter-Evalluttr, that aims at achieving proper ter- minations for multi-pass decoding. It consists of a rewriter and an evaluator. At each translation pass, the rewriter generates new translation to improve previous translations and the evaluator estimates the translation quality to determine whether to terminate the process. We also propose prioritized gradient descent method that biases the training samples toward rewriting those low- quality translations. This enables training have comparable training time to training encoder-decoder models that only have single pass de- coding. We have applied Evvauttoo to improve RNNSearch and Transformer. Extensive experiments have been conducted on two translation tasks, Chinese- Englishh and Gernis,, showing that the proposed framework has notably improved the performances of NMT models and has significantly outperformed previous methods.
that has multiple pass decodings to
8

---

Under review as a conference paper at ICLR 2021
REFERENCES
Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. Neural machine translation by jointly learning to align and translate. In ICLR, 2015.
Sergio Barrachina, Oliver Bender, Francisco Casacuberta, Jorge Civera, Elsa Cubel, Shahram Khadivi, Antonio Lagarda, Hermann Ney, Jesús Tomás, Enrique Vidal, et al. Statistical ap- proaches to computer-assisted translation. Computational Linguistics, 35(1):3-28, 2009.
Prasenjit Basu, Santanu Pal, and Sudip Kumar Naskar. Keep it or not: Word level quality estimation for post-editing, In Proceedings of the Third Conference Machine Translation: Shared Task Papers, pp. 759-764, 2018.
Léon Bottou and Olivier Bousquet. The tradeoffs of large scale learning. In Advances in Neural Information Processing Systems, pp. 161-168, 2008.
Justin A Boyan and Andrew W Moore. Generalization in reinforcement learning: Safely approxi- mating the value function. In Advances in neural information processing systems, pp. 369-376, 1995.
Rajen Chatterjee, Matteo Negri, Marco Turchi, Frédéric Blain, and Lucia Specia. Combining quality estimation and automatic post-editing to enhance machine translation output. In Proceedings of the 13th Conference of the Association for Machine Translation in the Americas (Volume 1: Research Papers), pp. 26-38, 2018.
Qian Chen, Xiaodan Zhu, Zhenhua Ling, Si Wei, Hui Jiang, and Diana Inkpen. Enhanced Istm for natural language inference. arXiv preprint arXiv:/609.06i0 2016.
Junyoung Chung, Caglar Gulcehre, KyungHyun Cho, and Yoshua Bengio. Empirical evaluation of gated recurrent neural networks on sequence modeling. arXiv preprint arXiv:1412.3555, 2014.
Kai Fan, Jiayi Wang, Bo Li, Fengming Zhou, Boxing Chen, and Luo Si. "bilingual expert" can find translation errors. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 33, pp. 6367-6374, 2019.
Xinwei Geng, Xiaocheng Feng, Bing Qin, and Ting Liu. Adaptive multi multi--ass decoder for neural machine translation. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pp. 523-532, 2018.
Alex Graves. Generating sequences with recurrent neural networks. arXiv:1308.0850 2013.
arXiv preprint
Jiatao Gu, Zhengdong Lu, Hang Li, and Victor OK Li. Incorporating copying mechanism in sequence-to-sequuecce learning. arXiv preprint 1rXiv:1603.0639) 2016.
Hyun Kim and Kong-Hyeok Lee. A recurrent neural networks approach for estimating the quality of machine translation output. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguisties: Human Language Technologies, pp- 494-498, 2016.
Hyun Kim, Jong- Hyeok Lec, and Seung- Seung-Hoon Na. Predictor-estimator using multilevel task learning with stack propagation for neural quality estimation. In Proceedings of the Second Conference on Machine Translation, pp. 562-5662666 568, 2017.
André FT Martins, Marein Junczys-Downn
Fabio N Kepler, Ramón Astudillo, Chris Hokamp, and Roman Grundkiewicz. Pushing the limits of translation quality estimation. Transactions of the Association for Computational Linguistics, 5:205-218, 2017.
Jan Niehues, Funah Cho, Thanh-l.e Ha, and Alex Waibel. Pre-translation for neural machine trans- lation. arXiv preprint arXiv: (61.005233 2016.
Myle Ott. Sergey Edunov, Alexei Baevski, Angela Fan, Sam Gross, Nathan Ng. David Grangier, and Michael Auli. fairseq: A fast, extensible toolkit for sequence modeling. In Proceedings of NAACI-HIL 2019: Demonstrations, 2019.

---

Under review as a conference paper at ICLR 2021
Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. Bleu: method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting on association for computational linguistics, pp. 311-315 Association for Computational Linguistics, 2002.
Ankur Parikh, Oscar Täckström, Dipanjan Das, and Jakob Uszkoreit. A decomposable attention model for natural language inference. arXiv preprint arXiv: 1606666666333. 2016.
Abigail See, Peter J Liu, and Christopher D Manning. Get to the point: Summarization with pointer- generator networks. arXiv preprint arXiv: HBNN::704.4366 2017.
Rico Sennrich, Barry Haddow, and Alexandra Birch. Neural machine translation of rare words with subword units. arXiv preprint arXiv: Arxiv:/508.0 TTt1::/50.079990 508.09999999 2015.
Iyya Sutskever, Oriol Vinyals, and Quoe V Le. Sequence sequence learning with neural networks. In Advances in neural information processing systems, pp. 3104-3112, 2014.
Richard S Sutton, David A McAllester, Satinder P Singh, and Yishay Mansour. Policy gradient methods for reinforcement learning with function approximation. Advances in neural informa- lion processing systems, pp. 1057-1063, 2000.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Advances in neural information processing systems, pp. 5998 5998-6008, 2017.
Thuy Vu and Gholamreza Haffari. Automatic post-editing of machine translation: A neural programmer-interpreeer approach. In Proceedings of the 2018 Conference on Empirical Meth- ods in Natural Language Processing, pp. 3048-004000 2018.
Yingce Xia, Fei Tian, Lijun Wu, Jianxin Lin, Tao Qin, Nenghai Yu, and Tie- Yan Liu. Deliberation networks: Sequence generation beyond one-pass decoding. In Advances in Neural Information Processing Systems, pp. 1784-1794, 2017.
Hao Zhang and Daniel Gildea. Efficient multi-pass decoding for synchronous context free gram- mars. In ACL, 2008
Wen Zhang, Yang Feng, Fandong Meng. Di You, and Qun Liu. Bridging the gap between training and inference for neural machine translation. arXiv preprint arXiv:1906.02440 2019.
Xiangwen Zhang, Jinsong Su, Yue Qin, Yang Liu, Rongrong Ji, and Hongji Wang. Asynchronous bidirectional decoding for neural machine translation. In Thirty-Second AAAI Conference on Artificial Intelligence, 2018.
10


Reviews:
[{"review_id": "nJRQbw0hI3Q", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "### Summary\nThis paper proposes a multi-pass generation process for NMT. It introduces an evaluator model that learns to score reference translations higher than model outputs, and serves as a policy to determine how many refinements to make during inference, and which samples to prioritize during training. Samples that have a low quality score are prioritized, such that the translation (rewriter) module is trained with a focus on low-quality inputs where rewriting can still yield improvements.\nThe performance of this rewriter-evaluator is evaluated on Chinese-to-English and English-to-German benchmark tasks with RNNs and Transformer architectures, and shows superior quality than previous works. Training time is reported to be comparable to a standard encoder-decoder, while inference is roughly 4 times slower. In an ablation analysis, the influence of individual components on output quality (such as a copy mechanism and hyperparameters, e.g. the maximum number of iterations) is compared. In addition, the relevance of the evaluator during inference is demonstrated in a comparison with an oracle sentence selection.\n\nTo summarize, I'd like to see this paper accepted if it addresses the questions below and makes an effort to increase reproducibility. The ideas are novel and the combination of training algorithm and architectural modifications seem effective. In seq2seq NLP applications where inference time is not a concern, the proposed methodology could improve quality in practice.\n\n### Strengths\n- The prioritized training with model outputs is novel. The idea is neat, and solves the problem of having to pick model outputs of different quality for training.\n- The proposed training and models show great improvement over previous benchmarks (disregarding inference effort) on two language pairs and with two different architectures.\n- The ablation shows the individual contribution of the proposed set of modifications and the performance under different hyperparameter settings.\n- Training and inference time requirements are analyzed.\n- The paper is written clearly and therefore easy to follow.\n\n### Weaknesses\n- Reproducibility: The code is not published, and half of the experiments are on non-freely available datasets (NIST LDC). The experiments on Zh-En could have been performed on a public benchmark like WMT 2017 as well.\n- Comparison with previous works: It is unclear whether these are Transformer-based or RNN-based, and how they relate to the implementation of this paper (e.g. if the baseline MLE scores are the same), and if hyperparameter were tuned equally well. In addition, inference effort in terms of rewriting iterations is not considered in the comparison.\n- Analysis: There are a few open questions (see below) that would strengthen the paper and highlight the effectiveness and the workings of the novel elements.\n\n### Details\n- Geng et al. 2018 do not report any issues with variance in their experiments, so I'd refrain from using generic RL statements to argue that their work is inferior.\n- Related work in Active Learning (see below), Curriculum Learning (e.g. Zhou et al. 2020 (https://www.aclweb.org/anthology/2020.acl-main.620.pdf), and Quality Estimation (e.g. Zho et al. 2020 (https://arxiv.org/abs/2005.03519)) often use model scores directly for measuring model competence/quality/confidence, so it would be insightful to replace q with model scores to analyze if the evaluator is learning anything beyond what the rewriter already captures. See also Lee et al. 2020 (https://arxiv.org/abs/2002.07233) for relating model scores to translation quality.\n- Eq 1 introduces only 2 arguments for z, but Eq. 6 feeds 3.\n- The parameters are confused for the losses in Eq. 6 and 7.\n- Algorithm 1: Why not directly pushing quadruples into A? There is no filter between l13 and l16.\n- For the analysis, it would be great to compare how many passes each training samples receives on average during training, and how it relates to how many passes are done in inference. This would help to understand the importance of seeing model outputs during training.\n- Does the baseline MT also use a pointer generator? If not, it would be good to add this for a more fair comparison.\n- With longer inputs, the rewriter also performs more computation than the normal MT Transformer. Therefore, I'd like to see a comparison to a simple MT baseline where the source is concatenated twice (as described in Sec. 4) as input to the encoder. If there is no benefit from doing that, the advantage must come from inserting previous outputs.\n- How are hyperparameters delta and K determined? And how large is the queue? Limiting the training to 3 epochs seems quite low.\n- The comparisons to previous work are inconsistent: It is not clear which scores come from reimplementations and which ones are taken from previous papers. The reimplementation should also directly be compared to the original paper to validate the quality.\n- Table 1 hides the number of iterations/rewrites. It would be great to add, to make the advantages of the proposed method shine better.\n- The ablation should be done on the dev set rather than the test set.\n\n\n### Related Work\n- The prioritized queue is related to works in Active Learning and Curriculum Learning. Relating to these works might inspire an improvement of the algorithm or some shared insights, e.g. Liu et al. 2018 (https://www.aclweb.org/anthology/K18-1033.pdf), Platanios et al. 2019 (https://www.aclweb.org/anthology/N19-1119.pdf). ", "extra_scores": {}, "extra_text": {}}, {"review_id": "cuMtEbU_GWG", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "In this paper, the authors complement iterative refinement for neural machine translation with an evaluator model that controls the termination of the translation process. Their approach is an alternative to the policy network used in [1].\n\nThe translation process is terminated when the predicted quality of the current translation is inferior to the previous one by a margin of $\\delta$ or more, or after a fixed number of iterations otherwise. To train the models, the authors propose a sampling mechanism that prioritizes low-quality translations.\n\nStrengths:\n\nThe proposed approach is fairly intuitive and improves upon two baselines over multiple datasets.\n\nThe paper is mostly clear and the authors give enough details to reproduce experiments, at least for En-De.\n\nWeaknesses:\n\nThe Zh-En training data is not publicly available.\n\nComparison to previous work is not that convincing because most use baselines that are significantly worse than Transformer. The oracle experiment is only conducted with RNNSearch.\n\nQuestions:\n\nIn Table 3, I assume you display training time per epoch. What is the training time to convergence? Given that the sampling process is auto-regressive (i.e. conditioned on previous tokens), how can your approach have a speed comparable to standard training?\n\nDid you train models without the priority queue, simply training over a fixed number of iterations for each example? If so, what are the results?\n\nWhy do further iterations sometimes decrease translation quality?\n\n[1] Geng et al. Adaptive multi-pass decoder for neural machine translation. EMNLP, 2018.", "extra_scores": {}, "extra_text": {}}, {"review_id": "VWhTpON9BXW", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "** Summary **\nIn this work, the authors proposed a rewriter-evaluator framework for neural machine translation (NMT). The translation is achieved by an iterative way: at the $k$-th iteration, a rewriter generates a translation sequence $z^{(k)}$ based on the source input $x$ and the output of previous iteration $z^{(k-1)}$. The evaluator is used to provide a score for $z^{(k)}$, used to indicate when to stop refinement. \nThe authors used priority queue to organize the training samples, where the sequences with low translation quality will be processed first. \nThe authors conduct experiments on NIST Zh->En and WMT’15 En->De.\n\n** Pros **\n(1) Proposed a new method for improving the NMT in an iterative way.\n\n** Cons **\n(1) Experimental results not convincing.\n(2) The method lacks a strong explanation why it works.\n\n** Clarify **\nI think the clarify of the current version should be greatly improved.\n1. What is the configuration of your Transformer and what does ``we use default settings in fairseq (Ott et al., 2019)’’ mean? Please give the specific number of layers, hidden dimensions, dropout, batchsize here. By the way, according to my knowledge, people always use Adam for optimization in NMT. Why do you choose Rmsprop?\n2. Where is Figure 5.2?\n\n** Significance **\n1. About the evaluator: In Section 2, I do not get how $q^{(k)}$ is obtained in detail. It is not enough for me to get Eqn.(4) only. If you mean that the detailed definition is in Eqn.(12), I do not think it is a good measure for quality estimation, since it only measures the similarity between source input the previous translation in some hidden space. That is, the evaluator is not well-defined.\n2. Experimental results not convincing. Why don’t you evaluate on WMT’14, where the baselines are provided in (Ott et al., 2019)?\n3. The sacreBLEU score should also be reported (at least, in the appendix), so that we can compare your results with more algorithms in a fair way.\n4. It is better to give several cases on how your method gradually improves the translation quality.\n\n\n** About the baseline **\nI am not satisfied with results of WMT’15 En-De as you reported in Table 1. \nThere are some pre-trained checkpoints on English->German, which is available at \nhttps://github.com/pytorch/fairseq/tree/master/examples/translation.\nI download the model `transformer.wmt16.en-de`, whose md5sum is (evaluated by the command line md5sum)\n``68f2ea37f8eb5431f1c6b92c44c6a7bf  model.pt``\n\nThe data is preprocessed by \n```\nBINHOME=`the place to store mosesdecoder and subword-nmt`\nSCRIPTS=$BINHOME/mosesdecoder/scripts\nTOKENIZER=$SCRIPTS/tokenizer/tokenizer.perl\nNORM_PUNC=$SCRIPTS/tokenizer/normalize-punctuation.perl\nREM_NON_PRINT_CHAR=$SCRIPTS/tokenizer/remove-non-printing-char.perl\n\ncat $fname | \\\nperl $NORM_PUNC $lang | \\\nperl $REM_NON_PRINT_CHAR | \\\nperl $TOKENIZER  -l $lang | \\\npython $BINHOME/subword-nmt/subword_nmt/apply_bpe.py -c $BPE_CODE  > ${fname}.bpe\n```\nAnd the translation is obtained by \n```\ncat wmt15.en.bpe | \\\npython3 $FAIRSEQ/interactive.py /tmp/wmt16.en-de.joined-dict.transformer \\\n--source-lang en --target-lang de \\\n--path /tmp/wmt16.en-de.joined-dict.transformer/model.pt \\\n--buffer-size 2000 --batch-size 128 --beam 5 --remove-bpe | \\\ngrep ^H | cut -f3- > out\n```\nThe multi-bleu/sacreBLEU for WMT’14 are 29.14/29.0, which are almost consistent with the results in (Ott et al., 2019) (i.e., 29.3 vs 28.6)\nHowever, the multi-bleu/sacreBLEU for WMT’15 are 31.65/31.8, which are much higher than your baseline and algorithm. Therefore, I think you should re-implement your method following the settings in (Ott et al., 2019) to get more convincing results. \n", "extra_scores": {}, "extra_text": {}}, {"review_id": "C8JLEBwGU8-", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper proposes a rewriter-evaluator framework for multi-pass decoding, where translation hypothesis at last iteration is refined by the rewriter and further evaluated by the evaluator to decide whether to terminate the iterative decoding process. Compared to previous studies, the evaluator offers this framework the capability of flexibly controlling the termination. The authors also propose a prioritized gradient decent algorithm that biases the training process to those low-quality translation samples. Experiments on NIST Zh-En and WMT15 En-De demonstrate that the proposed model significantly outperforms several strong baselines.\n\nPros:\n\n= The idea behind the rewriter-evaluator framework is easy to follow.\n\n= The proposed method achieves significant performance improvement against several multi-pass baselines on both Zh-En and En-De translation tasks.\n\n= The authors demonstrate that the proposed training algorithm has similar training time to the vanilla baseline, i.e. no training time loss (Table 3).\n\nCons:\n\n= Some model details are missing, and the NIST Zh-En training data is not publicly available so it’s hard to exactly replicate the experiments.\n\n= Although the framework enables flexible termination, the evaluator requires a threshold that has a large impact on translation quality and must be carefully tuned (Table 2).\n\n= Baselines and model optimization should be further improved to fully convincing readers.\n\nMy detailed comments are as follows:\n\n1.\tThe authors claim that their model can better handle the termination. One important experiment is to testify how many iterations the model uses for translating one sentence, and what factors could affect the iteration number, such as sentence length (would long inputs require more passes?). Particularly, it would be great to have an experiment to compare the termination difference between the proposed model and the Adaptive Multi-pass Decoder (Geng et al., 2018), and show evidence how the proposed one outperforms Geng et al., 2018 (i.e. RL-based model).\n\n2.\tThe authors didn’t provide full model details about how to combine source encoder outputs (x) and target encoder outputs (z) in the decoder part, i.e. more details about Ea. (4) are required. \n\n3.\tCompared to other baselines, the authors adopt the copy mechanism. Could you please provide an ablation study to justify its impact, such as retraining a model using the rewriter-evaluator framework without copying?\n\n4.\tFor Transformer, the authors concatenate the source input and target translation but disables cross-attention over them. Could you please give some explanation behind this practice? What if we allow the cross-attention here?\n\n5.\tUsing non-public dataset, like NIST Zh-En, is not suggested in my opinion. Other researchers might not be able to replicate this experiment at all. Running experiments with (C)WMT Zh-En would be a better alternative. Besides, using tokenized BLEU with ‘multi-bleu.perl’ is also not suggested nowadays. Use sacrebleu and report the signature, instead.\n\n6.\tThe results, in Table 1, for WMT15 En-De on newstest2015 are not convincing. Based on my own experience, a standard Transformer-base model can already achieve a tokenized BLEU score of ~29. I think this weak baseline comes from the fact that the authors train Transformer with batch size of 80 and optimize model parameters with RMSProp, as shown in experimental settings, Section 5.1. The authors should update their experimental training.\n\n7.\tIf I understand correctly, Algorithm 1 requires online decoding process, that is, performing greedy or beam search decoding during training to get real-time estimation for q and r. In my experience, the decoding is very time-consuming. However, results in Table 3 show that there is almost no training time difference! Could you please show some theoretical explanation about this? How did you perform decoding, or get z^k from z^k-1 in practice?\n\nIf the authors can address all my concerns, I’d like to update my scores. \n", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper builds upon recent iterative refinement approaches NMT with an evaluator model that controls the termination of the translation process, yielding a “rewriter-evaluator framework” for multi-pass decoding. Their approach is an alternative to the policy network used in Geng et al (EMNLP 2018). The main delta wrt previous studies is that the evaluator offers this framework the capability of flexibly controlling the termination. While the idea behind the rewriter-evaluator framework is sensible and well described, and the proposed method achieves significant performance improvement against reported baselines, reviewers pointed out some concerns with the baselines and model optimization details. More analysis of the termination procedure against the RL-based model of Geng et al. 2018 could shed some light on why the proposed approach is better. Some analysis testifying how many iterations the model uses for translating one sentence, and what factors could affect the iteration number, such as sentence length, would greatly improve the paper. A second weakness pointed out by reviewers is related to the results of WMT’15 En-De reported in Table 1, where the reported baseline numbers seem to be weaker than expected. As pointed out by one the reviewers, pre-trained checkpoints on English->German (available at https://github.com/pytorch/fairseq/tree/master/examples/translation) exist which achieve much higher sacre BLEU than the reported baseline. I found the authors’ answer not very convincing regarding this point. Therefore, I recommend rejection. I suggest the authors, in future iterations of their work, address some of the issues pointed out by the reviewers and re-implement their method following the settings in (Ott et al., 2019) to get more convincing results. 
Decision: Reject