  Back-translation is an effective strategy to improve the performance of
Neural Machine Translation~(NMT) by generating pseudo-parallel data. However,
several recent works have found that better translation quality of the
pseudo-parallel data does not necessarily lead to better final translation
models, while lower-quality but more diverse data often yields stronger
results. In this paper, we propose a novel method to generate pseudo-parallel
data from a pre-trained back-translation model. Our method is a meta-learning
algorithm which adapts a pre-trained back-translation model so that the
pseudo-parallel data it generates would train a forward-translation model to do
well on a validation set. In our evaluations in both the standard datasets WMT
En-De'14 and WMT En-Fr'14, as well as a multilingual translation setting, our
method leads to significant improvements over strong baselines. Our code will
be made available.
