The Curious Case of Hallucinations in Neural Machine Translation (2021)

Abstract: In this work, we study hallucinations in Neural Machine Translation (NMT), which lie at an extreme end on the spectrum of NMT pathologies. Firstly, we connect the phenomenon of hallucinations under source perturbation to the Long-Tail theory of Feldman, and present an empirically validated hypothesis that explains hallucinations under source perturbation. Secondly, we consider hallucinations under corpus-level noise (without any source perturbation) and demonstrate that two prominent types of natural hallucinations (detached and oscillatory outputs) could be generated and explained through specific corpus-level noise patterns. Finally, we elucidate the phenomenon of hallucination amplification in popular data-generation processes such as Backtranslation and sequence-level Knowledge Distillation. We have released the datasets and code to replicate our results.

## introduction neural machine translation (nmt) enjoys tremendous success, far surpassing the performance of previous statistical approaches in high-to-moderate resource settings @xcite . however, nmt suffers from well known pathologies such as coverage @xcite , mistranslation of named entities @xcite , etc. in terms of adequacy of the generated output @xcite , hallucinations are egregious mistakes that lie at the extreme end of nmt pathologies. such hallucinated outputs are characterized as being decoupled from the source sequence, despite being (fully or moderately) fluent in the target language @xcite . two main hallucination phenomena have been reported in the existing literature: however, a plausible theory to explain the generation of different types of hallucinations, including the above two results is still lacking in the nmt literature. @xcite posited that hallucinations could be happening due to decoder instability, however, their experiments to engineer solutions based on this proved inconclusive. in this work, we present a systematic study of different kinds of hallucinations, studying them through the lens of generalization, memorization and optimization in sequence to sequence models. our key contributions are as follows: our work connects hallucinations in nmt to the problem of generalization in deep learning. in this section, we briefly survey the two areas. ## categorizing hallucinations in nmt in this section we systematize the study of hallucinations by coining a few definitions to aid further analysis. firstly, we categorize hallucinations in nmt into two primary categories: source: das kann man nur feststellen , wenn die kontrollen mit einer großen intensität durchgeführt werden . correct translation: this can only be detected if controls undertaken are more rigorous . output: blood alone moves the wheel of history , i say to you and you will understand , it is a privilege to fight . source: 1995 das produktionsvolumen von 30 millionen pizzen wird erreicht . correct translation: 1995 the production reached 30 million pizzas . output: the us , for example , has been in the past two decades , but has been in the same position as the us , and has been in the united states . further, we classify a natural hallucination (nh) as belonging to one of the two types: both figures 1 and 2 show the tokenized input and output (hallucinated) examples from models trained in section 4.2, to illustrate the above two definitions. the above categorization of natural hallucinations excludes two other types of pathologies, discussed as hallucinations in @xcite , namely, generation of shorter outputs and copy of source to the output. the proposed categorization allows us to quantitatively disentangle the study of hallucinations from other nmt pathologies, without losing any generality. ## origins of hallucinations in this section, we propose and empirically validate two hypotheses in order to explain the two categories of hallucinations described in section 3. ## hallucination amplification in this section, we analyze how hallucinations caused due to corpus-level noise get amplified when a model trained on a noisy mt corpus is used for downstream data generation in algorithms such as sequence-level knowledge distillation (kd) @xcite and backtranslation (bt) @xcite . to analyze this, we need to compute nh at scale. so, firstly, we propose an automatic nh detection algorithm based on the analysis that hallucinations often occur in terms of oscillations or repeats of the target sequences. ## discussion in this section, we present a qualitative analysis of a few topics discussed in section 4, along with a discussion on some future research directions. mv source target 0.61 gerade plus gerade : gerade . ungerade plus ungerade : gerade . even plus even gives you even . odd plus odd gives you even . 0.65 also ist 2 ∧ 5 : 2 x 2 = 4 , 8 , 16 , 32 . so 2 ∧ 5 is 2 x 2 = 4 , 8 , 16 , 32 . 0.69 beweg dich ! nein ! beweg dich ! nein ! beweg dich ! nein ! move . no . move . no . move . no . 0.82 frau : sie bestanden darauf , ich würde lügen . ## conclusion in this work we demonstrated that memorized training samples are far more likely to hallucinate under perturbation than non-memorized samples, under an extension of the memory value estimator proposed in @xcite . we also showed that specific noise patterns in the training corpora lead to specific well-known hallucination patterns. finally, we demonstrated that these patterns can be amplified by popular data-generation processes such as backtranslation and sequencelevel knowledge distillation. due to the compute-intensive algorithms involved in our analysis, we conduct most of our experiments using the iwslt 2014 corpus. however, long-tailed phenomena are a characteristic of natural language and even scaling the size of the corpus doesn't alleviate the characteristic zipfian distribution of the occurrence of words/tokens in the nmt corpora; which, according to the central thesis of the long-tail theory @xcite , would lead to memorizations. similarly, noise in the form of invalid references is an artifact of the scale at which web-based corpora are collected and given that both hallucinations under perturbations and natural hallucinations are widely reported in large-scale nmt systems, our insights should be directly applicable to larger-scale models as well. we hope that our work serves as a useful step towards a detailed understanding of hallucinations in nmt and in other sequence to sequence models. among the numerous interesting directions for follow-up work, in future, we would like to explore learning-centric fixes to ameliorate the impact of memorization and corpus-level noise patterns in nmt training. mv source target 0.0 sie haben das gleiche gehirn , und das gleiche eeg . they have the same brain , and the same eeg . 0.08 und wir hatten keine ahnung , wo er war . so , recolonization is really very slow . 0.30 zum einen waren beide sehr real . for one thing , both were very real . 0.50 wir überqueren auf alu-leitern mit angefügten sicherheits-seilen . we cross on aluminum ladders with safety ropes attached .