Unsupervised Neural Machine Translation with Generative Language Models Only

We show how to derive state-of-the-art unsupervised neural machine translation systems from generatively pre-trained language models . Our method consists of three steps : few-shot amplification , distillation , and backtranslation . We first use the zero-shot translation ability of large pretrained language models to generate translations for a small set of unlabeled sentences . We then amplify these zeroshot translations by using them as few-shot demonstrations for sampling a larger synthetic dataset . This dataset is then distilled by discarding the few-shot demonstrations and then fine-tuning . During backtranslation , we repeatedly generate translations for a set of inputs and then fine-tune a single language model on both directions of the translation task at once , ensuring cycle-consistency by swapping the roles of gold monotext and generated translations when fine-tuning . By using our method to leverage GPT-3 ’ s zero-shot translation capability , we achieve a new state-of-the-art in unsupervised translation on the WMT14 English-French benchmark , attaining a BLEU score of 42.1 . 1 INTRODUCTION . Recent work on generative pre-training has shown that with sufficient data and scale ( Kaplan et al. , 2020 ; Henighan et al. , 2020 ; Radford et al. , 2019 ) , large language models ( LMs ) acquire remarkable in-context metalearning abilities ( Brown et al. , 2020 ) . One of the most striking ways this capability manifests is via few-shot learning , where the model picks up patterns from multiple training examples placed in context . While few-shot prompting is flexible and enables strong performance on a diverse suite of NLP tasks to be coaxed out of generatively pre-trained LMs , its benefits are most pronounced with larger models , with commensurate training , inference , compute , and data costs . The desire to reduce these costs motivates our present work , which allows us to continue finetuning our models , obtaining more performance from smaller models and pushing our larger models even further , without resorting to few-shot prompting at test time or any additional supervision at train time . We target the domain of unsupervised neural machine translation ( NMT ) , which typically involves bootstrapping a weak translation model before amplifying its translation ability via backtranslation . Recent work in unsupervised NMT has been dominated by large encoder-decoder architectures where the bootstrap is implemented by denoising/autoencoding tasks ( e.g. , multilingual Cloze ( Devlin et al. , 2019 ; Conneau & Lample , 2019 ) , masked-span prediction ( Raffel et al. , 2020 ; Xue et al. , 2021 ) , reconstruction from corrupted inputs ( Wang et al. , 2019 ; Liu et al. , 2020 ) ) intended to produce strong encoders and aligned multilingual representations for decoding . In our present work , we show that generative language modeling alone can implement the entire unsupervised NMT pipeline , and derive state-of-the-art unsupervised NMT systems using only generatively pre-trained language models . We implement the bootstrap by first sampling a small number of zero-shot translations from GPT-3 . These are then used as few-shot prompts to sample a larger dataset of synthetic translations . The few-shot prompts are then discarded and the generated samples are distilled by fine-tuning the model on these synthetic data in the zero-shot format . This produces a language model aligned to our translation format and amenable to large-scale backtranslation . By using our method to leverage GPT-3 ’ s zero-shot translation capability , we achieve a new state-of-the-art in unsupervised translation on the WMT14 English-French benchmark , attaining a BLEU score of 42.1 . 2 BACKGROUND AND RELATED WORK . The modern approach to unsupervised neural machine translation typically involves encoderdecoder architectures jointly trained via denoising autoencoding / reconstruction tasks ( Vincent et al. , 2008 ; Conneau & Lample , 2019 ; Liu et al. , 2020 ; Ma et al. , 2020 ; Raffel et al. , 2020 ; Xue et al. , 2021 ; Wang et al. , 2019 ; Liu et al. , 2020 ; Song et al. , 2019 ) and backtranslation ( Sennrich et al. , 2016 ; Edunov et al. , 2018 ; Cotterell & Kreutzer , 2018 ) . This approach to unsupervised NMT is codified by Artetxe et al . ( 2018 ) and Lample et al . ( 2018 ) , although various ideas can be traced back further : unsupervised machine translation was framed as a deciphering task by Ravi & Knight ( 2011 ) and backtranslation was first introduced for machine translation as a method for data augmentation using target-side monolingual data by Sennrich et al . ( 2016 ) . Denoising autoencoding with a bilingual encoder can be viewed as a kind of latent bilingual lexicon induction , necessary for producing sufficiently aligned embeddings to kick-start backtranslation ; such techniques have been extensively studied in the context of machine translation ( Artetxe et al. , 2017 ; Klementiev et al. , 2012 ; Vulic & Moens , 2015 ; Hu et al. , 2017 ; Goyal et al. , 2016 ; Shen et al. , 2017 ) . At the same time , recent work on large-scale generative pre-training ( Kaplan et al. , 2020 ; Henighan et al. , 2020 ; Radford et al. , 2019 ) has demonstrated that with sufficient data and model scale , strong performance on a diverse suite of NLP tasks can be coaxed from transformer language models using few-shot prompts . Our present work unifies these two lines of research by using generative language modeling to simplify unsupervised NMT even further : we show how with sufficient scale , pre-training , and clever prompting , a single generative language model can implement the entire unsupervised neural machine translation pipeline , avoiding optimizations such as denoising autoencoding , auxiliary / adversarial losses in latent space , or ad-hoc bilingual dictionaries . Our reliance on large-scale generative pre-trainingis similar to prior work in unsupervised NMT which uses large-scale language modeling tasks on internet data as part of the bootstrap ( Conneau & Lample , 2019 ; Conneau et al. , 2020 ; Liu et al. , 2020 ) . The role of few-shot prompting and distillation in our method is related to recent work on unsupervised data augmentation using language models ( Anaby-Tavor et al. , 2020 ; Schick et al. , 2021 ; Kumar et al. , 2020 ; Papanikolaou & Pierleoni , 2020 ; Schick & Schütze , 2021 ; Yang et al. , 2020 ) and is also in the same spirit as recent work on selftraining and noisy-student training ( Mi et al. , 2021 ; Vu et al. , 2021 ; Xie et al. , 2020 ) . The few-shot distillation component of our method is similar to contemporaneous work by Wang et al . ( 2021b ) which uses few-shot prompting for unsupervised data augmentation , though they focus only on inference for text classification rather than generation for sequence-to-sequence tasks like machine translation and they do not study the phenomena of self-amplification nor few-shot data efficiency ( Section 6 ) as we do . 3 BACKTRANSLATION VIA LANGUAGE MODELING . Algorithm 1 Iterated backtranslation using a single generative language model Input : Source monotextMS ; target monotextMT ; number of iterations I ; number of samples per iteration J ; monotext formatter f ( · ) ; bitext formatter g ( · , · ) ; parameters θ of language model pθ ( · ) trained to complete outputs of f to outputs of g. Output : Final model parameters θ . 1 : for i = 1 to I do 2 : Bback ← ∅ 3 : for j = 1 to J do 4 : y ∼MS ∪MT 5 : x̃ ∼ pθ ( · | f ( y ) ) 6 : Bback ← Bback ∪ { 〈x̃ , y〉 } 7 : estimate θ by maximizing log pθ of g ( x̃ , y ) for 〈x̃ , y〉 ∈ Bback Backtranslation was first introduced in the context of machine translation as a method for data augmentation using target-side monolingual data ( Bojar & Tamchyna , 2011 ; Sennrich et al. , 2016 ; Poncelas et al. , 2018 ) , by sampling synthetic source-to-target data from another target-to-source translation model . In our present work , we cast machine translation as a language modeling task and jointly train and sample from a single language model for both source-to-target and target-to-source translation . Given bitext 〈seq1 , seq2〉 in languages L1 and L2 , we format the translation task as follows : [ L1 ] < seq1 > [ [ TRANSLATE ] ] [ L2 ] < seq2 > At test-time , the LM is prompted with [ L1 ] < seq > [ [ TRANSLATE ] ] [ L2 ] and we parse a candidate translation < sampledSeq > from the sampled completion . Backtranslation is implemented by reversing the roles of seq and sampledSeq and finetuning on the bitext 〈sampledSeq , seq〉 . We remark that in contrast to the interpretation of backtranslation as a wake-sleep algorithm ( Cotterell & Kreutzer , 2018 ) , where the forwards and backwards translators are trained alternately , we use a single language model for both forwards and backwards translation and train on both directions jointly at every iteration . There are various ways to train a model using backtranslation , e.g. , completely online ( interleaving minibatch gradient updates and sampling ) versus offline ( backtranslating the entire training dataset at each epoch ; potentially re-training the model from scratch after sampling new backtranslations ) . In practice , we find that data scaling of a model ’ s optimal test loss and BLEU score quickly saturates on backtranslations from previous versions of the model , and opt for a semi-online setup where we synchronously sample a relatively small number of L1-L2 and L2-L1 pairs before resuming training for a single epoch on the newly sampled data . We refer to this as a single iteration of backtranslation . Formally , Algorithm 1 describes our implemention of backtranslation using a single generative language model pθ ( · ) . We assume that pθ ( · ) has already been trained to complete formatted monotext ( [ L1 ] < seq1 > [ [ TRANSLATE ] ] [ L2 ] ) to formatted bitext ( [ L1 ] < seq1 > [ [ TRANSLATE ] ] [ L2 ] < seq2 > ) . 4 THE BOOTSTRAP : GENERATIVE PRE-TRAINING , FEW-SHOT AMPLIFICATION , AND DISTILLATION . [ en ] [ fr ] [ en ] [ fr ] [ en ] [ fr ] [ en ] [ fr ] prompt sample } few-shot prompting finetuning [ en ] I 'll pursue solitary pathways through twilit meadows with only this one dream : you come too . [ [ TRANSLATE ] ] [ fr ] Je m'éloigne seul dans des chemins obscurs , dans les champs de l'ombre , avec seulement ce seul rêve : tu viens aussi . [ en ] [ fr ] [ [ TRANSLATE ] ] Figure 1 : Illustration of our bootstrap procedure , which we call few-shot distillation . We use fewshot prompts sampled from GPT-3 to generate an initial dataset of synthetic translations from a generatively pretrained language model ( left ) . The few-shot examples are then discarded and the synthetic bitext reformatted for finetuning on the autoregressive language modeling objective ( right ) . The modern approach to unsupervised NMT is parametrized by a choice of initialization or bootstrap . The bootstrap has typically relied on some form of unsupervised cross-lingual representation learning , e.g. , bilingual dictionaries initialized from unsupervised cross-lingual word embeddings ( Lample et al. , 2018 ; Artetxe et al. , 2018 ) or multilingual masked language modeling followed by denoising autoencoding with a shared encoder and decoder ( Conneau & Lample , 2019 ) . In Section 3 , we formulated iterative backtranslation in terms of language modeling , assuming a language model which has already been trained to follow a particular instruction format for translation . To complete our procedure , we must supply such a language model . Unlike previous work on unsupervised NMT , we use language models from the GPT-3 family ( Brown et al. , 2020 ) which have been generatively pre-trained on a large corpus of Internet data . A key observation from the body of work around GPT-3 is that generative pre-training at scale induces strong in-context metalearning abilities , two special cases of which are ( 1 ) instruction following and ( 2 ) few-shot prompting : a sufficiently trained large language model benefits from both detailed natural language descriptions of tasks and , when given in-context examples , can achieve strong performance on a diverse suite of tasks ( e.g. , question-answering , natural language inference , translation . ) We implement the bootstrap by exploiting both of these abilities , by using natural language instruction to produce zero-shot translations and few-shot prompting during amplification .

Summary: The paper proposed an iterative finetuning method to adapt pretrained LMs to translation tasks. The framework refines the LM into translation models, which accepts both languages with a predefined format, and translate them into other language bidirectionally. The model is finetuned by backtranslation.  To generate translation counterparts of the given monolingual corpora during finetuning, the paper utilized prompting on LMs: gives several translations X1 Y1 X2 Y2 ... as contexts, then predict the successor Yn given the real input Xn, where Xk and Yk are souce/target sentences.  The context of the prompt X1 Y1 X2 Y2 is also generated by LM using another template, but this part can be replaced as a real examples.  Experiments show that the trained system achieves the better performance than existing similar methods, showing that the effectiveness of the proposed method. The paper provided some additional studies to support the main results.