Unsupervised Neural Machine Translation with Generative Language Models Only

We show how to derive state-of-the-art unsupervised neural machine translation systems from generatively pre-trained language models . Our method consists of three steps : few-shot amplification , distillation , and backtranslation . We first use the zero-shot translation ability of large pretrained language models to generate translations for a small set of unlabeled sentences . We then amplify these zeroshot translations by using them as few-shot demonstrations for sampling a larger synthetic dataset . This dataset is then distilled by discarding the few-shot demonstrations and then fine-tuning . During backtranslation , we repeatedly generate translations for a set of inputs and then fine-tune a single language model on both directions of the translation task at once , ensuring cycle-consistency by swapping the roles of gold monotext and generated translations when fine-tuning . By using our method to leverage GPT-3 ’ s zero-shot translation capability , we achieve a new state-of-the-art in unsupervised translation on the WMT14 English-French benchmark , attaining a BLEU score of 42.1 . 1 INTRODUCTION . Recent work on generative pre-training has shown that with sufficient data and scale ( Kaplan et al. , 2020 ; Henighan et al. , 2020 ; Radford et al. , 2019 ) , large language models ( LMs ) acquire remarkable in-context metalearning abilities ( Brown et al. , 2020 ) . One of the most striking ways this capability manifests is via few-shot learning , where the model picks up patterns from multiple training examples placed in context . While few-shot prompting is flexible and enables strong performance on a diverse suite of NLP tasks to be coaxed out of generatively pre-trained LMs , its benefits are most pronounced with larger models , with commensurate training , inference , compute , and data costs . The desire to reduce these costs motivates our present work , which allows us to continue finetuning our models , obtaining more performance from smaller models and pushing our larger models even further , without resorting to few-shot prompting at test time or any additional supervision at train time . We target the domain of unsupervised neural machine translation ( NMT ) , which typically involves bootstrapping a weak translation model before amplifying its translation ability via backtranslation . Recent work in unsupervised NMT has been dominated by large encoder-decoder architectures where the bootstrap is implemented by denoising/autoencoding tasks ( e.g. , multilingual Cloze ( Devlin et al. , 2019 ; Conneau & Lample , 2019 ) , masked-span prediction ( Raffel et al. , 2020 ; Xue et al. , 2021 ) , reconstruction from corrupted inputs ( Wang et al. , 2019 ; Liu et al. , 2020 ) ) intended to produce strong encoders and aligned multilingual representations for decoding . In our present work , we show that generative language modeling alone can implement the entire unsupervised NMT pipeline , and derive state-of-the-art unsupervised NMT systems using only generatively pre-trained language models . We implement the bootstrap by first sampling a small number of zero-shot translations from GPT-3 . These are then used as few-shot prompts to sample a larger dataset of synthetic translations . The few-shot prompts are then discarded and the generated samples are distilled by fine-tuning the model on these synthetic data in the zero-shot format . This produces a language model aligned to our translation format and amenable to large-scale backtranslation . By using our method to leverage GPT-3 ’ s zero-shot translation capability , we achieve a new state-of-the-art in unsupervised translation on the WMT14 English-French benchmark , attaining a BLEU score of 42.1 . 2 BACKGROUND AND RELATED WORK . The modern approach to unsupervised neural machine translation typically involves encoderdecoder architectures jointly trained via denoising autoencoding / reconstruction tasks ( Vincent et al. , 2008 ; Conneau & Lample , 2019 ; Liu et al. , 2020 ; Ma et al. , 2020 ; Raffel et al. , 2020 ; Xue et al. , 2021 ; Wang et al. , 2019 ; Liu et al. , 2020 ; Song et al. , 2019 ) and backtranslation ( Sennrich et al. , 2016 ; Edunov et al. , 2018 ; Cotterell & Kreutzer , 2018 ) . This approach to unsupervised NMT is codified by Artetxe et al . ( 2018 ) and Lample et al . ( 2018 ) , although various ideas can be traced back further : unsupervised machine translation was framed as a deciphering task by Ravi & Knight ( 2011 ) and backtranslation was first introduced for machine translation as a method for data augmentation using target-side monolingual data by Sennrich et al . ( 2016 ) . Denoising autoencoding with a bilingual encoder can be viewed as a kind of latent bilingual lexicon induction , necessary for producing sufficiently aligned embeddings to kick-start backtranslation ; such techniques have been extensively studied in the context of machine translation ( Artetxe et al. , 2017 ; Klementiev et al. , 2012 ; Vulic & Moens , 2015 ; Hu et al. , 2017 ; Goyal et al. , 2016 ; Shen et al. , 2017 ) . At the same time , recent work on large-scale generative pre-training ( Kaplan et al. , 2020 ; Henighan et al. , 2020 ; Radford et al. , 2019 ) has demonstrated that with sufficient data and model scale , strong performance on a diverse suite of NLP tasks can be coaxed from transformer language models using few-shot prompts . Our present work unifies these two lines of research by using generative language modeling to simplify unsupervised NMT even further : we show how with sufficient scale , pre-training , and clever prompting , a single generative language model can implement the entire unsupervised neural machine translation pipeline , avoiding optimizations such as denoising autoencoding , auxiliary / adversarial losses in latent space , or ad-hoc bilingual dictionaries . Our reliance on large-scale generative pre-trainingis similar to prior work in unsupervised NMT which uses large-scale language modeling tasks on internet data as part of the bootstrap ( Conneau & Lample , 2019 ; Conneau et al. , 2020 ; Liu et al. , 2020 ) . The role of few-shot prompting and distillation in our method is related to recent work on unsupervised data augmentation using language models ( Anaby-Tavor et al. , 2020 ; Schick et al. , 2021 ; Kumar et al. , 2020 ; Papanikolaou & Pierleoni , 2020 ; Schick & Schütze , 2021 ; Yang et al. , 2020 ) and is also in the same spirit as recent work on selftraining and noisy-student training ( Mi et al. , 2021 ; Vu et al. , 2021 ; Xie et al. , 2020 ) . The few-shot distillation component of our method is similar to contemporaneous work by Wang et al . ( 2021b ) which uses few-shot prompting for unsupervised data augmentation , though they focus only on inference for text classification rather than generation for sequence-to-sequence tasks like machine translation and they do not study the phenomena of self-amplification nor few-shot data efficiency ( Section 6 ) as we do . 3 BACKTRANSLATION VIA LANGUAGE MODELING . Algorithm 1 Iterated backtranslation using a single generative language model Input : Source monotextMS ; target monotextMT ; number of iterations I ; number of samples per iteration J ; monotext formatter f ( · ) ; bitext formatter g ( · , · ) ; parameters θ of language model pθ ( · ) trained to complete outputs of f to outputs of g. Output : Final model parameters θ . 1 : for i = 1 to I do 2 : Bback ← ∅ 3 : for j = 1 to J do 4 : y ∼MS ∪MT 5 : x̃ ∼ pθ ( · | f ( y ) ) 6 : Bback ← Bback ∪ { 〈x̃ , y〉 } 7 : estimate θ by maximizing log pθ of g ( x̃ , y ) for 〈x̃ , y〉 ∈ Bback Backtranslation was first introduced in the context of machine translation as a method for data augmentation using target-side monolingual data ( Bojar & Tamchyna , 2011 ; Sennrich et al. , 2016 ; Poncelas et al. , 2018 ) , by sampling synthetic source-to-target data from another target-to-source translation model . In our present work , we cast machine translation as a language modeling task and jointly train and sample from a single language model for both source-to-target and target-to-source translation . Given bitext 〈seq1 , seq2〉 in languages L1 and L2 , we format the translation task as follows : [ L1 ] < seq1 > [ [ TRANSLATE ] ] [ L2 ] < seq2 > At test-time , the LM is prompted with [ L1 ] < seq > [ [ TRANSLATE ] ] [ L2 ] and we parse a candidate translation < sampledSeq > from the sampled completion . Backtranslation is implemented by reversing the roles of seq and sampledSeq and finetuning on the bitext 〈sampledSeq , seq〉 . We remark that in contrast to the interpretation of backtranslation as a wake-sleep algorithm ( Cotterell & Kreutzer , 2018 ) , where the forwards and backwards translators are trained alternately , we use a single language model for both forwards and backwards translation and train on both directions jointly at every iteration . There are various ways to train a model using backtranslation , e.g. , completely online ( interleaving minibatch gradient updates and sampling ) versus offline ( backtranslating the entire training dataset at each epoch ; potentially re-training the model from scratch after sampling new backtranslations ) . In practice , we find that data scaling of a model ’ s optimal test loss and BLEU score quickly saturates on backtranslations from previous versions of the model , and opt for a semi-online setup where we synchronously sample a relatively small number of L1-L2 and L2-L1 pairs before resuming training for a single epoch on the newly sampled data . We refer to this as a single iteration of backtranslation . Formally , Algorithm 1 describes our implemention of backtranslation using a single generative language model pθ ( · ) . We assume that pθ ( · ) has already been trained to complete formatted monotext ( [ L1 ] < seq1 > [ [ TRANSLATE ] ] [ L2 ] ) to formatted bitext ( [ L1 ] < seq1 > [ [ TRANSLATE ] ] [ L2 ] < seq2 > ) . 4 THE BOOTSTRAP : GENERATIVE PRE-TRAINING , FEW-SHOT AMPLIFICATION , AND DISTILLATION . [ en ] [ fr ] [ en ] [ fr ] [ en ] [ fr ] [ en ] [ fr ] prompt sample } few-shot prompting finetuning [ en ] I 'll pursue solitary pathways through twilit meadows with only this one dream : you come too . [ [ TRANSLATE ] ] [ fr ] Je m'éloigne seul dans des chemins obscurs , dans les champs de l'ombre , avec seulement ce seul rêve : tu viens aussi . [ en ] [ fr ] [ [ TRANSLATE ] ] Figure 1 : Illustration of our bootstrap procedure , which we call few-shot distillation . We use fewshot prompts sampled from GPT-3 to generate an initial dataset of synthetic translations from a generatively pretrained language model ( left ) . The few-shot examples are then discarded and the synthetic bitext reformatted for finetuning on the autoregressive language modeling objective ( right ) . The modern approach to unsupervised NMT is parametrized by a choice of initialization or bootstrap . The bootstrap has typically relied on some form of unsupervised cross-lingual representation learning , e.g. , bilingual dictionaries initialized from unsupervised cross-lingual word embeddings ( Lample et al. , 2018 ; Artetxe et al. , 2018 ) or multilingual masked language modeling followed by denoising autoencoding with a shared encoder and decoder ( Conneau & Lample , 2019 ) . In Section 3 , we formulated iterative backtranslation in terms of language modeling , assuming a language model which has already been trained to follow a particular instruction format for translation . To complete our procedure , we must supply such a language model . Unlike previous work on unsupervised NMT , we use language models from the GPT-3 family ( Brown et al. , 2020 ) which have been generatively pre-trained on a large corpus of Internet data . A key observation from the body of work around GPT-3 is that generative pre-training at scale induces strong in-context metalearning abilities , two special cases of which are ( 1 ) instruction following and ( 2 ) few-shot prompting : a sufficiently trained large language model benefits from both detailed natural language descriptions of tasks and , when given in-context examples , can achieve strong performance on a diverse suite of tasks ( e.g. , question-answering , natural language inference , translation . ) We implement the bootstrap by exploiting both of these abilities , by using natural language instruction to produce zero-shot translations and few-shot prompting during amplification .

Summary: This paper proposes to solve the machine translation task without any bi-text data and using only a pre-trained generative language model to bootstrap the process. The entire method relies on using powerful pre-trained language models that have been trained on a large amount of data to extract initial boot-strap examples, and then fine-tune the model to solve the machine translation task using specially formatted prompts. The same language model is used to translate in both directions, and back-translation is used to improve results.