Looking for a Needle in a Haystack: A Comprehensive Study of Hallucinations in Neural Machine Translation (2023)

Abstract: Although the problem of hallucinations in neural machine translation (NMT) has received some attention, research on this highly pathological phenomenon lacks solid ground. Previous work has been limited in several ways: it often resorts to artificial settings where the problem is amplified, it disregards some (common) types of hallucinations, and it does not validate adequacy of detection heuristics. In this paper, we set foundations for the study of NMT hallucinations. First, we work in a natural setting, i.e., in-domain data without artificial noise neither in training nor in inference. Next, we annotate a dataset of over 3.4k sentences indicating different kinds of critical errors and hallucinations. Then, we turn to detection methods and both revisit methods used previously and propose using glass-box uncertainty-based detectors. Overall, we show that for preventive settings, (i) previously used methods are largely inadequate, (ii) sequence log-probability works best and performs on par with reference-based methods. Finally, we propose DeHallucinator, a simple method for alleviating hallucinations at test time that significantly reduces the hallucinatory rate.

## introduction neural machine translation (nmt) is becoming increasingly accurate @xcite , particularly in high resource language pairs where parallel data is abundant. however, even the best systems available today may generate hallucinations. these are extremely pathological translations that contain content that is unfaithful to the source sequence. critically, a tiny fraction of these mistakes is all it takes to compromise user trust or safe deployment of nmt models in production. unfortunately, although the problem of hallucinations received some attention, research on this highly pathological phenomenon lacks solid ground. first, previous work used multiple and often overlapping definitions and categories of hallucinations which makes it hard to draw connections between observations made in different works @xcite @xcite . next, since hallucinations are extremely rare, previous work focused on settings in which the phenomenon is amplified, e.g. perturbing data either in training or at inference, or evaluating under domain shift @xcite @xcite @xcite @xcite . critically, the analysis on these works mostly relied on the adequacy of the automatic hallucination detection methods they proposed. however, it is not immediate whether these methods translate well to unperturbed settings. in this work, we set foundations for the study of nmt hallucinations. we take a step back from previous work and, instead of considering perturbed settings for which hallucinations are more frequent, we consider a natural scenario and face the actual problem of identifying a small fraction of hallucinations (a "needle") in a large number of translated sentences (a "haystack"). then, we provide a rigorous comparison among hallucination detection methods. apart from analysing those proposed in previous work (e.g., heuristics based on anomalous encoder-decoder attention), we also propose to use simple model uncertainty measures as detectors. for each of these methods, we select examples marked as hallucinations, put them together, and gather human annotations. as a result, we introduce a corpus of 3415 structured annotations for different nmt pathologies and hallucinations. we use this corpus for analysis and show that, in preventive settings where high recall is desirable, previously proposed methods are mostly inadequate, and filtering according to standard sequence logprobability performs the best. in fact, it performs on par with the state-of-the-art comet @xcite ## taxonomy of translation pathologies choosing a good taxonomy is a compromise between simplicity (which minimizes annotation effort) and comprehensiveness. thus, generic qual-ity assessment taxonomies, such as mqm @xcite , might be unfit or too complex when we focus only on critical errors and hallucinations. for hallucinations, in turn, previous work used multiple, often overlapping, definitions @xcite @xcite . the taxonomy we build here is rather general: it covers categories considered previously @xcite and others not reported before. for a broader discussion on the taxonomy of hallucinations in nmt, and how it differs from other natural language generation tasks, refer to @xcite . ## hallucination detection methods approaches to hallucination detection generally aim to find low-quality translations that may also satisfy additional constraints. previous work either relied only on quality filtering @xcite @xcite , or only on heuristics @xcite , or a combination of the two @xcite . we stick to this general form and consider different quality filters and heuristics. ## experimental setting model. we use transformer base @xcite from fairseq @xcite . data. we use the wmt2018 de-en news translation data excluding paracrawl @xcite -5.8m sentence pairs. we randomly choose 2/3 of the dataset for training and use the remaining 1/3 as a held-out set for analysis. for validation, we use the newstest2017 dataset. held-out data filtering. we are mainly interested in hallucinations produced for clean data. since our held-out data comes from the wmt2018 training dataset and thus can be noisy, we filter it using bicleaner, the filtering tool used in official releases of filtered paracrawl data @xcite @xcite . following previous work, we exclude examples with a score below 0.5 and end up with about 1.3m examples. all details on preprocessing, hyperparameters and implementation can be found in appendix a. ## hallucinations dataset to analyse the effectiveness of hallucination detection criteria, we pick a subset of examples that are likely to be hallucinations, and obtain fine-grained annotations from professional translators. ## analysing detection criteria in this section, we provide a comprehensive analysis of the performance of the heuristics and quality filters introduced in section 3. ## analysing hallucination pathologies in this section, we look at hallucination pathologies in isolation and show that the behavior of detection methods varies depending on the type of pathology. for example, for a given pathology, some methods may be specialised, whereas others may fail. for a similar analysis on other less severe translation errors, refer to appendix d. ## dehallucinator: overwriting hallucinations at test time in previous sections, we saw that hallucinations are more unstable than other translations: for them, generated mc-dropout hypotheses tend to vary greatly. this motivates us to look more closely into these hypotheses: are any of these translations not hallucinations? answering this question not only gives insight into the inner workings of hallucinating nmt models, but also leads to an interesting practical application -overwriting hallucinations at inference time. this is of utmost importance for production systems where hallucinations have a deeply compromising effect on user trust. whenever flagged, overwrite with better. intuitively, our idea is similar to hybrid pipelines when a machine-generated translation is first passed to a quality estimation system and then, if needed, is corrected by human translators. in our case, we first apply a hallucination detector and then, if a translation is flagged, we try to overwrite it with a better translation (figure 4 ). for this, we generate several mc-dropout hypotheses, score them with some measure, and pick the highest-scoring translation as a final candidate (in the spirit of reranking approaches @xcite @xcite ). the general pipeline above relies on the choice of a hallucination detector and a scoring measure. for the detector, we use the best of the analysed detectors, i.e. seq-logprob. 6 for the scoring measure, a natural choice would be a quality estimation system: by construction, these systems are designed to score translations according to quality. however, as we saw earlier, even the state-of-the-art comet-qe may fail (section 7.1). therefore, we compare two measures: comet-qe and seq-logprob. in this experiment, we randomly choose 200 translations from our dataset flagged by seq-logprob. for each, we generate 10 hypotheses with mc-dropout. then, for the overwritten translations we gather annotations according to our guidelines (section 5). the results are summarized in figure 4 . although we were concerned about comet-qe because of its low performance when ranking poor translations, we find that for choosing the best hypothesis, it is indeed appropriate and performs better than seq-logprob. we thus show results with comet-qe scores in the main text and with seq-logprob in appendix e. ## conclusions dealing with hallucinations is difficult. first, we had to take a step back from previous work and refuse procedures that amplify the problem, as these hinder the behavior of models in their standard settings. after that, we notice that work on detection often relies on assumptions that remained unquestioned (e.g., generic quality measures, targeted heuristics, anomalous attention being suitable detectors). through extensive experiments, we establish order in detection methods. surprisingly, despite introduction of several methods specifically targeted for hallucinations, what works best has always been at our disposal: standard sequence logprobability. this suggests that characteristics innate to a model can have a lot of value. in fact, such characteristics are the backbone of our dehallu-cinator, a lightweight approach that significantly alleviates hallucinations at test time. this leaves space for future research on model uncertainty, hallucination prevention, understanding where hallucinations come from, among others. for this, we release our corpus with structured annotations along with the model and its training data. altogether, this allows us to say that we provide solid ground for future study of hallucinations in nmt.