Venue: ICLR 2021 Conference

Reviews:

final_score: rating: 6: Marginally above acceptance threshold
review_content: The paper presents an extension of the back-translation method which provides a means of leveraging monolingual data in NMT where the quality of the data generated by the back-translation model is controlled through a meta learning regime that trains the BT model jointly with the actual translation model.
The method is an interesting application of meta-learning to NMT and worth seeing the results. It is a well-written paper with a sound description of the method and evaluation. The motivation is the main weakness and a better discussion and comparison to related work would help clarify the applicability of the method.

Comments
- The motivation for the method is supported by two claims, the first one related to the upper bound on quality of the BT model trained on the parallel data, and the second on the quality of the pseudo-parallel data generated by the BT model used then to train the actual NMT model together with the original parallel data. There is on the other hand not any empirical support in either claim that these create a weakness, thus, grounding the motivation for the paper. 
In the former case, the method does not really modify in any matter the quality of the original parallel data, hence the discussion is irrelevant. In the latter case, especially, when using the pseudo-parallel data, there are many factors not discussed that affect the quality of the pseudo-parallel data generated by the BT model other the diversity, such as source domain drift and eg. generation of translationese. It is indeed limited the literature on this topic although doesn't validate that any claim should be supported either by references or within the study included in the paper. The second reference in this context is referred to as slow and expensive but isn't the meta learning also making the model much slower and more costly in a similar sense?
- Can the authors analyze or discuss how well the backward model is improving and how does it distinguish from the previous methods?

review_title: An interesting application of meta-learning in NMT, but with an unclear motivation

final_score: rating: 7: Good paper, accept
review_content: The paper describes a method to improve NMT training with backtranslation.

Rather than using a fixed t->s model to translate target monolingual data in order to augment the training set for the s->t model, the proposed approach first pretrains the t->s model as usual then jointly trains it with the forward s->t model using a meta-learning approach: the s->t model is trained on the syntetic backtranslated data and a "meta-validation" loss is computed on a paralled dataset, which is used to update the t->s model using REINFORCE.
The approach is similar to the DualNMT model by Xia et al, but rather than updating on monolingual data based on LM and reconstruction scores, it uses a reward based on the cross-entropy on parallel data.
The paper also proposes a way to adapt this method to a multi-lingual setting.

Experiments are performed on WMT-14 En->De and En->Fr, and on 4 IWSLT-2018 language pairs. The authors report small but consistent improvements. Additional analyses are also reported.

Overall the method seems valid, although it is described at a very high level and no code release is mentioned. In my experience successfully implementing RL-based method is strongly dependent on getting hyperparameters and implementation details right, so it could be hard to reproduce this work without the code or a more detailed description. Also it's not entirely clear what is being used as "meta-validation" data here, I suppose it's all the parallel training data, but the paper doesn't make it clear.

Minor issues: the "Tagged backtranslation" paper by Caswell et al. 2019 contrasts the claim that improvements with sampling backtranslation are due to increased diversity. It should be referenced as relevant work. The Xia et al., 2016 Dual NMT paper is referenced multiple times in the text but not in the bibliography section
review_title: Review

final_score: rating: 7: Good paper, accept
review_content: The paper proposed an interesting approach for back-translation. The idea is to update both forward model and backward models during training. The forward model is updated in a standard way using synthetic samples generated from the backward model. The backward model is updated using gradient of the forward model on meta-validation dataset (i.e., parallel data). Evaluation shows that Meta BT performs better than offline BT ((Edunov et al., 2018)  in both multilingual setup and standard  en-fr, en-de translation direction. In general, I find the approach is nice.
While the paper compared Meta-BT with offline BT (i.e., MLE as mentioned in the paper) and DualNMT, I think these two baselines are not sufficient to verify the claims made by the paper.

The paper claims that MetaBT allows to update the backward model unlike offline-BT and avoids expense of multiple iterations in iterative BT. Note that back-translation can be done on-the-fly (Artetxe et al., 2018, Conneau and Lample, 2019). Online back translation allows updating both forward and backward models using monolingual data during training. Thus, I think online BT should be a baseline for appropriate comparison. While MetaBT avoids multiple iterations of iterative BT, the evidence is not provided in the paper in terms of training time for MetaBT. The additional complexity of MetaBT lies in the update of the backward model per mini-batch and the computation of the Jacobian-vector products. As the author already mentioned that MetaBT has a large memory footprint, thus it’s slower to do one update in MetaBT. I wonder how MetaBT performs in comparison with iterative BT with 2 iterations with respect to BLEU scores and training time.

Can the authors provide similar plots in Figure 2 for En-De and in Figure 3 for En-Fr? With respect to MetaBT avoids overfitting. I think it would be nice to have some analysis on the samples generated by the backward model. In comparison to offline BT, does the backward model in MetaBT generate more diverse output?

With respect to the presentation of the paper, I think Figure 1 is a bit confusing to read. I was hoping to get the main idea from Figure 1 but it didn’t help at all.




**References**

Mikel Artetxe, Gorka Labaka, Eneko Agirre, Kyunghyun Cho. Unsupervised Neural Machine Translation. ICLR 2018. 
Guillaume Lample, Alexis Conneau. Cross-lingual Language Model Pretraining. NeurIPS 2019


**Post-response update**
Thanks authors for extra effort on semi-supervised experiments. I decided to increase the score to 6. 


review_title: Missing some baselines

final_score: rating: 6: Marginally above acceptance threshold
review_content: summary:

This paper applies techniques from meta-learning to derive and end-to-end update rule for a workflow involving backtranslation, specificically maximizing translation performance of the forward model, while updating the backward model to produce backtranslations that are maximally useful to improve the forward model's quality (as measured on a meta validation set). The approach is evaluated on WMT EN-DE and EN-FR and compared against a simple sampling strategy for backtranslation, and dual learning. In addition, the paper considers a multilingual setup where the translation direction is low-resource, and the initial backtranslation model is trained on a mix of parallel data from the language pair of interest, as well as auxiliary data with a high-resource, related source language. 

strengths:

+ the idea of end-to-end optimization of the backward model to maximally benefit training of the forward model is novel and interesting.

+ the mathematical derivation of the objective function is sound.

weaknesses:

- the empirical evaluation is not very convincing. Important information is missing, baselines are inexplicably weak, and some other simple baselines are missing. Specifically:

 -- what data sets do you use for meta validation? Is it one (or several) of the newstest sets? Since you're actually learning gradients on this data set, rather than just using it for early stopping or hyperparameter choice, I think you should consider using that data directly for training as one of the baselines. For example, Oravecz et al. (2019) report a 1.5-2 BLEU gain from fine-tuning on newstest2008-2017 for EN-DE in their submission to WMT.

 -- for the Edunov et al. 2018 baseline (sampling for back-translation), there is a gap of >6 BLEU between the best result they report (35 BLEU) and yours (28.73 BLEU). Some of this may be explainable by the fact that you use Transformer base rather than Transformer large, but even relatively speaking, Edunov et al. (2018) observe an improvement of ~4 BLEU with BT, while this paper only reports ~2. Authors discuss memory limitations as the reason why the did not train on Transformer Big, but they could conceivably use gradient accumulation to enable training even on limited hardware. Other reasons for the score discrepancy deserve discussion (for example, are BLEU scores reported tokenized or not?)
 
 -- please provide more detail how the high-resource data was used for the various baselines. You mention that you follow settings from previous work (Neubig & Hu, 2018; Wang et al., 2019a), but these actually use different techniques. I gather that you use some type (which?) of transfer learning for the forward models. Do you also apply these techniques to the backward models, and if so, is the initial model for meta back-translation initialized differently?
  
 -- the dual learning baseline bears some conceptual similarity to the proposed objective in that both backward and forward model are continuously improved. I'm surprised to see that it leads to worse performance than the baseline trained on parallel data only, but there is too little information to build trust in this result. Which language model was used for the additional rewards? What beam size (or sample size) was used for the backtranslation? What monolingual data was used, only target-languge data or also source-language data? If the latter, which data was used?
 
- I find the title problematic. I can see how it was coined as a combination of meta-learning and backtranslation, but it's misleading to a reader who doesn't know this intention. If we take "meta" to roughly express some self-referentiality (meta-learning is learning to learn; a meta-analysis is an analysis of analyses), is meta-backtranslation the backtranslation of backtranslations?

recommendation:

Overall, I vote for rejection. I like the core idea and could see this work being published eventually, but feel that the empirical evaluation needs to be strengthened before I would recommend acceptance. There are open questions about the evaluation setup, and if authors can answer these questions, this will let me better judge the empirical rigour.

typos:

mulltilingual -> multilingual


**post-response update**:

some of my concerns about the evaluation were resolved in the author response (for example regarding the meta-validation datasets) , and I have slightly increased my rating.
review_title: interesting idea; concerns about evaluation

Meta-review: This paper proposes a meta-learning-based technique to learn how to back-translate (generate a synthetic source-language translation of an observed target-language sentence) for the purpose of better optimising a source-to-target translation model. 

The approach is an interesting novel angle to jointly training the translation model and the back-translation component. Compared to techniques like UNMT and DualNMT, the approach offers reduced training time and a simpler formulation with fewer trainable components (and fewer hyperparameters). 

During the discussion phase the authors provided additional insight, clarifications, and results that improved our perception of the paper. I would personally appreciate if the authors would update their paper with the clarifications they made to points raised by R2, R3, and R4, especially on the details about meta-validation, the discussion about memory footprint, and the additional results on UNMT (and variants). 
Decision: Accept (Poster)