Language Model Prior for Low-Resource Neural Machine Translation (2020)

Abstract: The scarcity of large parallel corpora is an important obstacle for neural machine translation. A common solution is to exploit the knowledge of language models (LM) trained on abundant monolingual data. In this work, we propose a novel approach to incorporate a LM as prior in a neural translation model (TM). Specifically, we add a regularization term, which pushes the output distributions of the TM to be probable under the LM prior, while avoiding wrong predictions when the TM “disagrees” with the LM. This objective relates to knowledge distillation, where the LM can be viewed as teaching the TM about the target language. The proposed approach does not compromise decoding speed, because the LM is used only at training time, unlike previous work that requires it during inference. We present an analysis of the effects that different methods have on the distributions of the TM. Results on two low-resource machine translation datasets show clear improvements even with limited monolingual data.

## introduction neural machine translation (nmt) @xcite @xcite relies heavily on large parallel corpora @xcite and needs careful hyperparameter tuning, in order to work in low-resource settings @xcite . a popular approach for addressing data scarcity is to exploit abundant monolingual corpora via data augmentation techniques, such as back-translation @xcite . although back-translation usually leads to significant performance gains @xcite , it requires training separate models and expensive translation of large amounts of monolingual data. however, when faced with lack of training data, a more principled approach is to consider exploiting prior information. language models (lm) trained on target-side monolingual data have been used for years as priors in statistical machine translation (smt) @xcite via the noisy channel model. this approach has been adopted to nmt, with the neural noisy channel @xcite . however, neural noisy channel models face a computational challenge, because they model the "reverse translation probability" p(x|y). specifically, they require multiple passes over the source sentence x as they generate the target sentence y, or sophisticated architectures to reduce the passes. lms have also been used in nmt for reweighting the predictions of translation models (tm), or as additional context, via lm-fusion @xcite @xcite . but, as the lm is required during decoding, it adds a significant computation overhead. another challenge is balancing the tm and the lm, whose ratio is either fixed @xcite or requires changing the model architecture @xcite . in this work, we propose to use a lm trained on target-side monolingual corpora as a weakly informative prior. we add a regularization term, which drives the output distributions of the tm to be probable under the distributions of the lm. this gives flexibility to the tm, by enabling it to deviate from the lm when needed, unlike fusion methods that change the decoder's distributions, which can introduce translation errors. the lm "teaches" the tm about the target language similar to knowledge distillation @xcite . this method works by simply changing the training objective and does not require any changes to the model architecture. importantly, the lm is separated from the tm, which means that it is needed only during training, therefore we can decode faster than fusion or neural noisy channel. we also note that this method is not intended as a replacement to other techniques that use monolingual data, such as back-translation, but is orthogonal to them. we make the following contributions: ## background nmt models trained with maximum likelihood estimation, model directly the probability p(y|x) of the target sentence y given the source sentence x: modeling directly p(y|x) requires large amounts of parallel sentences to learn a good model and nmt lacks a principled way for leveraging monolingual data. in this section we review approaches that exploit prior information encoded in lms or the signal from the language modeling task. noisy channel model smt @xcite employs bayes' rule which offers a natural way for exploiting monolingual data, using a targetside lm based on the so called "noisy channel" model @xcite . instead of directly modeling p(y|x), it models the "reverse translation probability" p(x|y), by rewriting p(y|x) ∝ p(x|y) × p(y). it selects words that are both a priori likely with p(y i ) and "explain well" the input with p(x|y i ). this idea has been adopted to nmt with neural noisy channel, but it has two fundamental limitations. first, during decoding the model has to alternate between generating the output and scoring the input @xcite or perform multiple forward passes @xcite over x. and crucially, since the lm is part of the network it has to also be used during inference, which adds a computational constraint on its size. fusion gulcehre et al. (2015) proposed to incorporate pretrained lms in nmt, using shallowand deep-fusion. in shallow-fusion, the lm re-weights the tm's scores via log-linear interpolation: log p(y t ) =(1 -β) log p tm (y t |@xmath0, x) + β log p lm (y t |@xmath1 in deep fusion, they alter the model architecture to include the hidden states of a rnn-lm @xcite as additional features for predicting the next word in the decoder, which are weighted with a controller mechanism (i.e., gating). in both approaches, the tm and lm are first trained independently and are combined later. sriram et al. it is practically the same as shallow-fusion, but with the lm used also during training, instead of used just in inference, and interpolating with λ=1. fusion methods face the same computational limitation as noisy channel, since the lm needs to be used during inference. also, probability interpolation methods, such as shallow fusion or post-norm, use a fixed weight for all time-steps, which can lead to translation errors. gated fusion @xcite is more flexible, but requires changing the network architecture. other approaches transfer-learning is another approach for exploiting pretrained lms. @xcite , first proposed to use lms trained on monolingual corpora to initialize the encoder and decoder of a tm. @xcite extended this idea to transformer architectures @xcite . this approach requires the tm to have identical architecture to the lm, which can be a limitation if the lm is huge. @xcite used language modeling as extra signal, by training the decoder of a tm also as a lm on target-side monolingual data. @xcite replaced the source with a null token, while training on monolingual data. both, reported mixed results, with marginal gains. we propose to move the lm out of the tm and use it as a prior over its decoder, by employing posterior regularization (pr) @xcite . pr incorporates prior information, by imposing soft constraints on a model's posterior distributions, which is much easier than putting bayesian priors over all the parameters of a deep neural network. the first term is the standard translation objective l mt and the second is the regularization term l kl , which we interpret as a weakly informative prior over the tm's distributions p tm , that expresses partial information about y. l kl is defined as the kullback-leibler divergence between the output distributions of the tm and the lm, weighted by λ. this formulation gives flexibility to the model, unlike probability interpolation, such as in fusion methods. for example, postnorm multiplies the probabilities of the lm and tm, which is the same as applying a logical and operation, where only words that are probable under both distributions will receive non-negligible probabilities. this prevents the model from generating the correct word when there is a large "disagreement" between the tm and the lm, which is inevitable as the lm is not aware of the source sentence (i.e., unconditional). however, by using the lm-prior we do not change the outputs of the tm. l kl pushes the tm to stay on average close to the prior, but crucially, it enables the tm to deviate from it when needed, for example to copy words from the source. secondly, the lm is no longer part of the network. this means that we can do inference using only the tm, unlike fusion or neural noisy channel, which require the lm for both training and decoding. by lifting this computational overhead, we enable the use of large pretrained models lms (bert; @xcite , gpt-2; @xcite ), without compromising speed or efficiency. ## analysis the main results show that ls, that simply penalizes confidence, is a very effective form of regularization in low-resource settings. we conduct a qualitative comparison to test whether the improvements from the proposed lm-prior are due to penalizing confidence, similar to ls, or from actually using information from the lm. specifically, we evaluate each model on the @xmath2. in fig. 3 we plot for each model the density foot_2 over all its entropy values. first, we observe that the un-regularized "base" model generates very confident (low-entropy) distributions, which suggests that it overfits on the small parallel data. as expected, the ls regularization successfully makes the tm less confident and therefore more robust to over-fitting. for additional context, we plot the entropy density of the lm and observe that, unsurprisingly, it is the most uncertain, since it is unconditional. interestingly, the model trained with the lmprior emits more confident distributions than the "base+ls" model, although it also achieves significantly better performance. this clearly shows that the gains cannot be explained just from smoothing the distributions of the tm and suggests that the model indeed exploits information from the lm. next, we focus on the "base+postnorm" model and observe that it generates the most confident predictions. note that, this finding aligns with a similar analysis in the original paper, where it was shown that under postnorm the tm generates lowentropy distributions. however, even though this method might improve fluency, it can hurt translation quality in certain cases. as described in sec. 3, by multiplying the two distributions, only a small subset of words will have non-zero probability in the final distribution. this means that when there are "disagreements" between the tm and lm this can lead to wrong predictions. we illustrate this with a concrete example in fig. 4 . although the tm predicted the correct word, the multiplication with the lm distribution caused the model to finally make a wrong prediction. also, the final distribution assigns a relatively high probability to a word compared to plotting overlapping histograms. de: die republikaner im kongress drängen auf eine umfassendere neufassung der ozonregeln. en: republicans in congress are pushing for a broader rewrite of the ozone rules. broader 34.3% wider 22.4% larger 7.2% tm new 8.9% repeal 7.1% bill 3.7% lm more 44.8% wider 31.8% ("more"), which is not among the top predictions of neither the lm or the tm. by contrast, the lmprior does not change the tm's predictions, and the model has the flexibility to deviate from the prior. ## conclusions in this work, we present a simple approach for incorporating knowledge from monolingual data to nmt. specifically, we use a lm trained on targetside monolingual data, to regularize the output distributions of a tm. this method is more efficient than alternative approaches that used pretrained lms, because it is not required during inference. also, we avoid the translation errors introduced by lm-fusion, because the tm is able to deviate from the prior when needed. we empirically show that while this method works by simply changing the training objective, it achieves better results than alternative lm-fusion techniques. also, it yields consistent performance gains even with modest monolingual data (3m sentences) across all translation directions. this makes it useful for low-resource languages, where not only parallel but also monolingual data are scarce. in future work, we intend to experiment with the lm-prior under more challenging conditions, such as when there is domain discrepancy between the parallel and monolingual data. also, we would like to explore how to overcome the obstacles that prevent us from fully exploiting large pretrained lms (e.g., gpt-2) in low-resource settings.