Optimal Transport for Unsupervised Hallucination Detection in Neural Machine Translation
Year: 2023

Abstract:
Neural machine translation (NMT) has become the de-facto standard in real-world machine translation applications. However, NMT models can unpredictably produce severely pathological translations, known as hallucinations, that seriously undermine user trust. It becomes thus crucial to implement effective preventive strategies to guarantee their proper functioning. In this paper, we address the problem of hallucination detection in NMT by following a simple intuition: as hallucinations are detached from the source content, they exhibit cross-attention patterns that are statistically different from those of good quality translations. We frame this problem with an optimal transport formulation and propose a fully unsupervised, plug-in detector that can be used with any attention-based NMT model. Experimental results show that our detector not only outperforms all previous model-based detectors, but is also competitive with detectors that employ external models trained on millions of samples for related tasks such as quality estimation and cross-lingual sentence similarity.

heading: 1 Introduction
text: Neural machine translation (NMT) has achieved tremendous success (Vaswani et al., 2017; Kocmi et al., 2022), becoming the mainstream method in real-world applications and production systems for automatic translation. Although these models are becoming evermore accurate, especially in high-resource settings, they may unpredictably produce hallucinations. These are severely pathological translations that are detached from the source sequence content (Lee et al., 2018; Müller et al., 2020; Raunak et al., 2021; Guerreiro et al., 2022). Crucially, these errors have the potential to seriously harm user trust in hard-to-predict ways (Perez et al., 2022), hence the evergrowing need to develop security mechanisms. One appealing strategy to address this issue is to develop effective on-the-fly detection systems.
In this work, we focus on leveraging the crossattention mechanism to develop a novel hallucination detector. This mechanism is responsible for selecting and combining the information contained in the source sequence that is relevant to retain during translation. Therefore, as hallucinations are translations whose content is detached from the source sequence, it is no surprise that connections between anomalous attention patterns and hallucinations have been drawn before in the literature (Berard et al., 2019; Raunak et al., 2021; Ferrando et al., 2022). These patterns usually exhibit scattered source attention mass across the different tokens in the translation (e.g. most source attention mass is concentrated on a few irrelevant tokens such as punctuation and the end-of-sequence token). Inspired by such observations, previous work has designed ad-hoc heuristics to detect hallucinations that specifically target the anomalous maps. While such heuristics can be used to detect hallucinations to a satisfactory extent (Guerreiro et al., 2022), we argue that a more theoretically-founded way of using anomalous attention information for hallucination detection is lacking in the literature.
Rather than aiming to find particular patterns, we go back to the main definition of hallucinations and draw the following hypothesis: as hallucinations— contrary to good translations—are not supported by the source content, they may exhibit cross-attention patterns that are statistically different from those found in good quality translations. Based on this hypothesis, we approach the problem of hallucination detection as a problem of anomaly detection with an optimal transport (OT) formulation (Kantorovich, 2006; Peyré et al., 2019). Namely, we aim to find translations with source attention mass distributions that are highly distant from those of good translations. Intuitively, the more distant a translation’s attention patterns are from those of good translations, the more anomalous it is in light of that distribution.
13766
Our key contributions are:
• We propose an OT-inspired fully unsupervised hallucination detector that can be plugged into any attention-based NMT model;
• We find that the idea that attention maps for hallucinations are anomalous in light of a reference data distribution makes for an effective hallucination detector;
• We show that our detector not only outperforms all previous model-based detectors, but is also competitive with external detectors that employ auxiliary models that have been trained on millions of samples.1

heading: 2 Background

heading: 2.1 Cross-attention in NMT models
text: A NMT model M defines a probability distribution pθ(y|x) over an output space of hypotheses Y conditioned on a source sequence x contained in an input space X . In this work, we focus on models parameterized by an encoder-decoder transformer model (Vaswani et al., 2017) with a set of learned weights θ. In particular, we will look closely at the cross-attention mechanism, a core component of NMT models that has been extensively analysed in the literature (Bahdanau et al., 2014; Raganato and Tiedemann, 2018; Kobayashi et al., 2020; Ferrando and Costa-jussà, 2021). This mechanism is responsible for computing, at each generation step, a distribution over all source sentence words that informs the decoder on the relevance of each of those words to the current translation generation step. We follow previous work that has drawn connections between hallucinations and cross-attention (Berard et al., 2019; Raunak et al., 2021), and focus specifically on the last layer of the decoder module. Concretely, for a source sequence of arbitrary length n and a target sequence of arbitrary length m, we will designate as Ω ∈ [0, 1]m×n the matrix of attention weights that is obtained by averaging across all the cross-attention heads of the last layer of the decoder module. Further, given the model M we will designate πM(x) := 1m [Ω(x)]
⊤ 1 ∈ △n as the source (attention) mass distribution computed by M when x is presented as input, where △n = {p ∈ Rn |p ≥ 0, 1⊤p = 1} is the (n− 1)- dimensional probability simplex.
1Our code and data to replicate our experiments are available in https://github.com/deep-spin/ ot-hallucination-detection.

heading: 2.2 Optimal Transport Problem and Wasserstein Distance
text: The first-order Wasserstein distance between two arbitrary probability distributions µ ∈ △n and ν ∈ △m is defined as
W (µ,ν) = inf γ∈Π(µ,ν) E(u,v)∼γ [c(u, v)] , (1)
where c : [n]× [m] → R+0 is a cost function,2 and Π(µ,ν) = {γ ∈ △n×m : γ1 = µ;γ⊤1 = ν}3 is the set of all joint probability distributions whose marginals are µ, ν. The Wasserstein distance arises from the method of optimal transport (OT) (Kantorovich, 2006; Peyré et al., 2019): OT measures distances between distributions in a way that depends on the geometry of the sample space. Intuitively, this distance indicates how much probability mass must be transferred from µ to ν in order to transform µ into ν while minimizing the transportation cost defined by c.
A notable example is the Wasserstein-1 distance, W1, also known as Earth Mover’s Distance (EMD), obtained for c(u, v) = ∥u−v∥1. The name follows from the simple intuition: if the distributions are interpreted as “two piles of mass” that can be moved around, the EMD represents the minimum amount of “work” required to transform one pile into the other, where the work is defined as the amount of mass moved multiplied by the distance it is moved.
Although OT has been explored for robustness (Paty and Cuturi, 2019; Staerman et al., 2021) and out-of-distribution detection (Wang et al., 2021; Yan et al., 2021; Cheng et al., 2022) in computer vision, the use of OT for anomaly detection in NLP applications remains largely overlooked.

heading: 2.3 The problem of hallucinations in NMT
text: Hallucinations are translations that lie at the extreme end of NMT pathologies (Raunak et al., 2021). Despite being a well-known issue, research on the phenomenon is hindered by the fact that these translations are rare, especially in highresource settings. As a result, data with hallucinations is scarce. To overcome this obstacle, many previous studies have focused on amplified settings where hallucinations are more likely to occur or are easier to detect. These include settings where (i) perturbations are induced either in the source
2We denote the set of indices {1, . . . , n} by [n]. 3We extend the simplex notation for matrices representing joint distributions, △n×m = {P ∈ Rn×m : P ≥ 0,1⊤P1 = 1}.
sentence or in the target prefix (Lee et al., 2018; Müller and Sennrich, 2021; Voita et al., 2021; Ferrando et al., 2022); (ii) the training data is corrupted with noise (Raunak et al., 2021); (iii) the model is tested under domain shift (Wang and Sennrich, 2020; Müller et al., 2020); (iv) the detectors are validated on artificial hallucinations (Zhou et al., 2021). Nevertheless, these works have provided important insights towards better understanding of the phenomenon. For instance, it has been found that samples memorized by an NMT model are likely to generate hallucinations when perturbed (Raunak et al., 2021), and hallucinations are related to lower source contributions and over-reliance on the target prefix (Voita et al., 2021; Ferrando et al., 2022).
In this work, we depart from artificial settings, and focus on studying hallucinations that are naturally produced by the NMT model. To that end, we follow the taxonomy introduced in Raunak et al. (2021) and later extended and studied in Guerreiro et al. (2022). Under this taxonomy, hallucinations are translations that contain content that is detached from the source sentence. To disentangle the different types of hallucinations, they can be categorized as: largely fluent detached hallucinations or oscillatory hallucinations. The former are translations that bear little or no relation at all to the source content and may be further split according to the severity of the detachment (e.g. strong or full detachment) while the latter are inadequate translations that contain erroneous repetitions of words and phrases. We illustrate in Appendix A the categories described above through examples of hallucinated outputs.

heading: 3 On-the-fly detection of hallucinations
text: On-the-fly hallucination detectors are systems that can detect hallucinations without access to reference translations. These detectors are particularly relevant as they can be deployed in online applications where references are not readily available.4

heading: 3.1 Categorization of hallucination detectors
text: Previous work on on-the-fly detection of hallucinations in NMT has primarily focused on two categories of detectors: external detectors and modelbased detectors. External detectors employ auxiliary models trained for related tasks such as quality
4As such, in this work, we will not consider metrics that depend on a reference sentence (e.g. chrF (Popović, 2016), COMET (Rei et al., 2020)). For an analysis on the performance of such metrics, please refer to Guerreiro et al. (2022).
estimation (QE) and cross-lingual embedding similarity. On the other hand, model-based detectors only require access to the NMT model that generates the translations, and work by leveraging relevant internal features such as model confidence and cross-attention. These detectors are attractive due to their flexibility and low memory footprint, as they can very easily be plugged in on a vast range of NMT models without the need for additional training data or computing infrastructure. Moreover, Guerreiro et al. (2022) show that model-based detectors can be predictive of hallucinations, outperforming QE models and even performing on par with state-of-the-art reference-based metrics.

heading: 3.2 Problem Statement
text: We will focus specifically on model-based detectors that require obtaining internal features from a model M. Building a hallucination detector generally consists of finding a scoring function sM : X → R and a threshold τ ∈ R to build a binary rule gM : X → {0, 1}. For a given test sample x ∈ X ,
gM(x) = 1{sM(x) > τ}. (2) If sM is an anomaly score, gM(x) = 0 implies that the model M generates a ‘normal’ translation for the source sequence x, and gM(x) = 1 implies that M generates a ‘hallucination’ instead.5

heading: 4 Unsupervised Hallucination Detection with Optimal Transport
text: Anomalous cross-attention maps have been connected to the hallucinatory mode in several works (Lee et al., 2018; Berard et al., 2019; Raunak et al., 2021). Our method builds on this idea and uses the Wasserstein distance to estimate the cost of transforming a translation source mass distribution into a reference distribution. Intuitively, the higher the cost of such transformation, the more distant—and hence the more anomalous— the attention of the translation is with respect to that of the reference translation.
4.1 Wass-to-Unif: A data independent scenario
In this scenario, we only rely on the generated translation and its source mass distribution to decide whether the translation is a hallucination or not. Concretely, for a given test sample x ∈ X :
5From now on, we will omit the subscript M from all model-based scoring functions to ease notation effort.
1. We first obtain the source mass attention distribution πM(x) ∈ △|x|;
2. We then compute an anomaly score, swtu(x), by measuring the Wasserstein distance between πM(x) and a reference distribution u:
swtu(x) = W (πM(x),u). (3)
Choice of reference translation. A natural choice for u is the uniform distribution, u = 1n · 1, where 1 is a vector of ones of size n. In the context of our problem, a uniform source mass distribution means that all source tokens are equally attended.
Choice of cost function. We consider the 0/1 cost function, c(i, j) = 1[i ̸= j], as it guarantees that the cost of transporting a unit mass from any token i to any token j ̸= i is constant. For this distance function, the problem in Equation 1 has the following closed-form solution (Villani, 2009):
W (πM(x),u) = 1/2 ∥πM(x)− u∥1. (4)
This is a well-known result in optimal transport: the Wasserstein distance under the 0/1 cost function is equivalent to the total variation distance between the two distributions. On this metric space, the Wasserstein distance depends solely on the probability mass that is transported to transform πM(x) to u. Importantly, this formulation ignores the starting locations and destinations of that probability mass as the cost of transporting a unit mass from any token i to any token j ̸= i is constant.
Interpretation of Wass-to-Unif. Attention maps for which the source attention mass is highly concentrated on a very sparse set of tokens (regardless of their location in the source sentence) can be very predictive of hallucinations (Berard et al., 2019; Guerreiro et al., 2022). Thus, the bigger the distance between the source mass distribution of a test sample and the uniform distribution, the more peaked the former is, and hence the closer it is to such predictive patterns.
4.2 Wass-to-Data: A data-driven scenario In this scenario, instead of using a single reference distribution, we use a set of reference source mass distributions, Rx, obtained with the same model. By doing so, we can evaluate how anomalous a given translation is compared to a model data-driven distribution, rather than relying on an arbitrary choice of reference distribution.
First, we use a held-out dataset Dheld that contains samples for which the model M generates good quality translations according to an automatic evaluation metric (in this work, we use COMET (Rei et al., 2020)). We use this dataset to construct (offline) a set of held-out source attention distributions Rheld = {πM(x) ∈ △|x| : x ∈ Dheld}. Then, for a given test sample x ∈ X , we apply the procedure illustrated in Figure 1:
1. We generate a translation ŷ = (y1, . . . , ym) and obtain the source mass attention distribution πM(x) ∈ △|x|;
2. We apply a length filter to construct the sample
reference set Rx, by restricting Rx to contain source mass distributions of Rheld correspondent to translations of size [(1− δ)m, (1+ δ)m] for a predefined δ ∈]0, 1[;6
3. We compute pairwise Wasserstein-1 distances between πM(x) and each element ri of Rx:
Wx = ( W1(πM(x), r1), . . . , (5)
W1(πM(x), r|Rx|) ) .
4. We obtain the anomaly score swtd(x) by averaging the bottom-k distances in Wx:
swtd(x) = 1 k ∑ si∈S si, (6)
where S is the set containing the k smallest elements of Wx.
Interpretation of Wass-to-Data. Hallucinations, unlike good translations, are not fully supported by the source content. Wass-to-Data evaluates how anomalous a translation is by comparing the source attention mass distribution of that translation to those of good translations. The higher the Wassto-Data score, the more anomalous the source attention mass distribution of that translation is in comparison to those of good translations, and the more likely it is to be an hallucination.
Relation to Wass-to-Unif. The Wasserstein-1 distance (see Section 2.2) between two distributions is equivalent to the ℓ1-norm of the difference between their cumulative distribution functions (Peyré and Cuturi, 2018). Note that this is different from the result in Equation 4, as the Wasserstein distance under c(i, j) = 1[i ̸= j] as the cost function is proportional to the norm of the difference between their probability mass functions. Thus, Wass-to-Unif will be more sensitive to the overall structure of the distributions (e.g. sharp probability peaks around some points), whereas Wass-to-Data will be more sensitive to the specific values of the points in the two distributions.
4.3 Wass-Combo: The best of both worlds With this scoring function, we aim at combining Wass-to-Unif and Wass-to-Data into a single detector. To do so, we propose using a two-stage process that exploits the computational benefits
6For efficiency reasons, we set the maximum cardinality of Rx to |R|max. If length-filtering yields a set with more than |R|max examples, we randomly sample |R|max examples from that set to construct Rx.
of Wass-to-Unif over Wass-to-Data.7 Put simply, (i) we start by assessing whether a test sample is deemed a hallucination according to Wass-to-Unif, and if not (ii) we compute the Wass-to-Data score. Formally,
swc(x) = 1 [swtu(x) > τwtu]× s̃wtu(x) (7) + 1 [swtu(x) ≤ τwtu]× swtd(x)
for a predefined scalar threshold τwtu. To set that threshold, we compute Wwtu = {swtu(x) : x ∈ Dheld} and set τwtu = PK , i.e τwtu is the K th percentile of Wwtu with K ∈ ]98, 100[ (in line with hallucinatory rates reported in (Müller et al., 2020; Wang and Sennrich, 2020; Raunak et al., 2022)).8

heading: 5 Experimental Setup

heading: 5.1 Model and Data
text: We follow the setup in Guerreiro et al. (2022). In that work, the authors released a dataset of 3415 translations for WMT18 DE-EN news translation data (Bojar et al., 2018) with annotations on critical errors and hallucinations. Our analysis in the main text focuses on this dataset as it is the only available dataset that contains human annotations on hallucinations produced naturally by an NMT model (we provide full details about the dataset and the model in Appendix A). Nevertheless, in order to access the broader validity of our methods for other low and mid-resource language pairs and models, we follow a similar setup to that of Tang et al. (2022) in which quality assessments are converted to hallucination annotations. For those experiments, we use the RO-EN (mid-resource) and NE-EN (low-resource) translations from the MLQEPE dataset (Fomicheva et al., 2022). In Appendix J, we present full details on the setup and report the results of these experiments. Importantly, our empirical observations are similar to those of the main text. For all our experiments, we obtain all modelbased information required to build the detectors using the same models that generated the translations in consideration.
7We also tested with a convex combination of the two detectors’ scores. We present results for this alternative approach in Appendix G.
8In order to make the scales of swtu and swtd compatible, we use a scaled s̃wtu value instead of swtu in Equation 7. We obtain s̃wtu by min-max scaling swtu such that s̃wtu is within the range of swtd values obtained for a held-out set.

heading: 5.2 Baseline detectors

heading: 5.2.1 Model-based detectors
text: We compare our methods to the two best performing model-based methods in Guerreiro et al. (2022).9
Attn-ign-SRC. This method consists of computing the proportion of source words with a total incoming attention mass lower than a threshold λ:
sais(x) = 1 n ∑n j=1 1 [ (Ω⊤(x)1)j < λ ] . (8)
This method was initially proposed in Berard et al. (2019). We follow their work and use λ = 0.2.
Seq-Logprob. We compute the length-normalised sequence log-probability of the translation:
sslp(x) = 1 m ∑m k=1 log pθ(yk | y<k,x). (9)

heading: 5.2.2 External detectors
text: We provide a comparison to detectors that exploit state-of-the-art models in related tasks, as it helps monitor the development of model-based detectors.
CometKiwi. We compute sentence-level quality scores with CometKiwi (Rei et al., 2022), the winning reference-free model of the WMT22 QE shared task (Zerva et al., 2022). It has more than 565M parameters and it was trained on more than 1M human quality annotations. Importantly, this training data includes human annotations for several low-quality translations and hallucinations.
LaBSE. We leverage LaBSE (Feng et al., 2020) to compute cross-lingual sentence representations for the source sequence and translation. We use the cosine similarity of these representations as the detection score. The model is based on the BERT (Devlin et al., 2019) architecture and was
9We compare with ALTI+ (Ferrando et al., 2022), a method thas was leveraged for hallucination detection concurrently to our work in Dale et al. (2022), in Appendix H.
trained on more than 20 billion sentences. LaBSE makes for a good baseline, as it was optimized in a self-supervised way with a translate matching objective that is very much aligned with the task of hallucination detection: during training, LaBSE is given a source sequence and a set of translations including the true translation and multiple negative alternatives, and the model is optimized to specifically discriminate the true translation from the other negative alternatives by assigning a higher similarity score to the former.

heading: 5.3 Evaluation metrics
text: We report the Area Under the Receiver Operating Characteristic curve (AUROC) and the False Positive Rate at 90% True Positive Rate (FPR@90TPR) to evaluate the performance of different detectors.

heading: 5.4 Implementation Details
text: We use WMT18 DE-EN data samples from the heldout set used in Guerreiro et al. (2022), and construct Dheld to contain the 250k samples with highest COMET score. To obtain Wass-to-Data scores, we set δ = 0.1, |R|max = 1000 and k = 4. To obtain Wass-to-Combo scores, we set τwtu = P99.9. We perform extensive ablations on the construction of Rheld and on all other hyperparameters in Appendix G. We also report the computational runtime of our methods in Appendix D.

heading: 6 Results

heading: 6.1 Performance on on-the-fly detection
text: We start by analyzing the performance of our proposed detectors on a real world on-the-fly detection scenario. In this scenario, the detector must be able to flag hallucinations regardless of their specific type as those are unknown at the time of detection.
Wass-Combo is the best model-based detector. Table 1 shows that Wass-Combo outperforms most

heading: DETECTOR AUROC ↑ FPR@90TPR ↓
text: other methods both in terms of AUROC and FPR. When compared to the previous best-performing model-based method (Seq-Logprob), Wass-Combo obtains boosts of approximately 4 and 10 points in AUROC and FPR, respectively. These performance boosts are further evidence that model-based features can be leveraged, in an unsupervised manner, to build effective detectors. Nevertheless, the high values of FPR suggest that there is still a significant performance margin to reduce in future research.
The notion of data proximity is helpful to detect hallucinations. Table 1 shows that Wass-to-Data outperforms the previous best-performing modelbased method (Seq-Logprob) in both AUROC and FPR (by more than 10%). This supports the idea that cross-attention patterns for hallucinations are anomalous with respect to those of good modelgenerated translations, and that our method can effectively measure this level of anomalousness. On the other hand, compared to Wass-to-Uni, Wass-toData shows a significant improvement of 30 FPR points. This highlights the effectiveness of leveraging the data-driven distribution of good translations instead of the ad-hoc uniform distribution. Nevertheless, Table 1 and Figure 2 show that combining both methods brings further performance improvements. This suggests that these methods may specialize in different types of hallucinations, and that combining them allows for detecting a broader range of anomalies. We will analyze this further in Section 6.2.
Our model-based method achieves comparable performance to external models. Table 1 shows that Wass-Combo outperforms CometKiwi, with significant improvements on FPR. However, there still exists a gap to LaBSE, the best overall detector.
This performance gap indicates that more powerful detectors can be built, paving the way for future work in model-based hallucination detection. Nevertheless, while relying on external models seems appealing, deploying and serving them in practice usually comes with additional infrastructure costs, while our detector relies on information that can be obtained when generating the translation.
Translation quality assessments are less predictive than similarity of cross-lingual sentence representations. Table 1 shows that LaBSE outperforms the state-of-the-art quality estimation system CometKiwi, with vast improvements in terms of FPR. This shows that for hallucination detection, quality assessments obtained with a QE model are less predictive than the similarity between crosslingual sentence representations. This may be explained through their training objectives (see Section 5.2.2): while CometKiwi employs a more general regression objective in which the model is trained to match human quality assessments, LaBSE is trained with a translate matching training objective that is very closely related to the task of hallucination detection.

heading: 6.2 Do detectors specialize in different types of hallucinations?
text: In this section, we present an analysis on the performance of different detectors for different types of hallucinations (see Section 2.3). We report both a quantitative analysis to understand whether a detector can distinguish a specific hallucination type from other translations (Table 2), and a qualitative analysis on a fixed-threshold scenario10 (Figure 3).
10We set the threshold by finding the 99th percentile of Wass-Combo scores obtained for 100k samples from the clean WMT18 DE-EN held-out set (see Section 5.4).
This analysis is particularly relevant to better understand how different detectors specialize in different types of hallucinations. In Appendix J, we show that the trends presented in this section hold for other mid- and low-resource language pairs.
Fully detached hallucinations. Detecting fully detached hallucinations is remarkably easy for most detectors. Interestingly, Wass-to-Unif significantly outperforms Wass-to-Data on this type of hallucination. This highlights how combining both methods can be helpful. In fact, WassCombo performs similarly to Wass-to-Unif, and can very easily separate most fully detached hallucinations from other translations on a fixed-threshold scenario (Figure 3). Note that the performance of Wass-to-Unif for fully detached hallucinations closely mirrors that of Attn-ign-SRC. This is not surprising, since both methods, at their core, try to capture similar patterns: translations for which the source attention mass distribution is highly concentrated on a small set of source tokens.
Strongly detached hallucinations. These are the hardest hallucinations to detect with our methods. Nevertheless, Wass-Combo performs competitively with the previous best-performing modelbased method for this type of hallucinations (SeqLogprob). We hypothesize that the difficulty in detecting these hallucinations may be due to the varying level of detachment from the source sequence. Indeed, Figure 3 shows that Wass-Combo scores span from a cluster of strongly detached hallucinations with scores similar to other data samples to those similar to the scores of most fully detached hallucinations.
Oscillatory hallucinations. Wass-to-Data and Wass-Combo significantly outperform all previous model-based detectors on detecting oscillatory hallucinations. This is relevance in the context of model-based detectors, as previous detectors notably struggle with detecting these hallucinations. Moreover, Wass-Combo also manages to outperform LaBSE with significant improvements in FPR. This hints that the repetition of words or phrases may not be enough to create sentence-level representations that are highly dissimilar from the non-oscillatory source sequence. In contrast, we find that CometKiwi appropriately penalizes oscillatory hallucinations, which aligns with observations made in Guerreiro et al. (2022).
Additionally, Figure 3 shows that the scores for oscillatory hallucinations are scattered along a broad range. After close evaluation, we observed that this is highly related to the severity of the oscillation: almost all non-detected hallucinations are not severe oscillations (see Appendix I).

heading: 7 Conclusions
text: We propose a novel plug-in model-based detector for hallucinations in NMT. Unlike previous attempts to build an attention-based detector, we do not rely on ad-hoc heuristics to detect hallucinations, and instead pose hallucination detection as an optimal transport problem: our detector aims to find translations whose source attention mass distribution is highly distant from those of good quality translations. Our empirical analysis shows that our detector outperforms all previous model-based detectors. Importantly, in contrast to these prior approaches, it is suitable for identifying oscillatory hallucinations, thus addressing an important gap in
the field. We also show that our detector is competitive with external detectors that use state-of-the-art quality estimation or cross-lingual similarity models. Notably, this performance is achieved without the need for large models, or any data with quality annotations or parallel training data. Finally, thanks to its flexibility, our detector can be easily deployed in real-world scenarios, making it a valuable tool for practical applications.

heading: Limitations
text: We highlight two main limitations of our work.
Firstly, instead of focusing on more recent NMT models that use large pretrained language models as their backbone, our experiments were based on transformer base models. That is because we used the NMT models that produced the translations in the datasets we analyze, i.e, the models that actually hallucinate for the source sequences in the dataset. Nevertheless, research on hallucinations for larger NMT models makes for an exciting line of future work and would be valuable to assess the broad validity of our claims.
Secondly, although our method does not require any training data or human annotations, it relies on access to a pre-existing database of source mass distributions. This can be easily obtained offline by running the model on monolingual data to obtain the distributions. Nevertheless, these datastores need not be costly in terms of memory. In fact, in Appendix J, we validate our detectors for datastores that contain less than 100k distributions.

heading: Acknowledgments
text: This work is partially supported by the European Research Council (ERC StG DeepSPIN 758969), by EU’s Horizon Europe Research and Innovation Actions (UTTER, contract 101070631), by the P2020 program MAIA (LISBOA-01-0247FEDER-045909), by the Portuguese Recovery and Resilience Plan through project C64500888200000055 (NextGenAI, Center for Responsible AI), and by the FCT through contract UIDB/50008/2020. This work was also granted access to the HPC resources of IDRIS under the allocation 2021- AP010611665 as well as under the project 2021- 101838 made by GENCI.

heading: A Model and Data Details
text: NMT Model. The NMT model used in Guerreiro et al. (2022) to create the hallucination dataset is a Transformer base model (Vaswani et al., 2017) (hidden size of 512, feedforward size of 2048, 6 encoder and 6 decoder layers, 8 attention heads). The model has approximately 77M parameters. It was trained with the fairseq toolkit (Ott et al., 2019) on WMT18 DE-EN data (excluding Paracrawl): the authors randomly choose 2/3 of the dataset for training and use the remaining 1/3 as a held-out set for analysis. We use that same held-out set in this work.
Dataset Stats. The dataset used in this paper was introduced in Guerreiro et al. (2022). It consists of 3415 translations from WMT18 DE-EN data with structured annotations on different types of hallucinations and pathologies. Overall, the dataset contains 118 translations annotated as fully detached hallucinations, 90 as strongly detached hallucinations, and 86 as oscillatory hallucinations.11 The other translations are either incorrect (1073) or correct (2048). Details on annotation, a high-level overview and other statistics can be found in the original paper. We show examples of hallucinations for each category in Table 3.12

heading: B Details on External Detectors
text: COMET. We use models available in the official repository13: wmt22-cometkiwi-da for CometKiwi and wmt20-comet-da for COMET.
LaBSE. We use the version available in sentence-transformers (Reimers and Gurevych, 2019).14

heading: C Performance of reference-free COMET-based models
text: Guerreiro et al. (2022) used the COMET-QE version wmt20-comet-qe-da, whereas we are using the latest iteration wmt22-cometkiwi-da (CometKiwi). CometKiwi was trained on human annotations from the MLQE-PE
11Some strongly detached hallucinations have also been annotated as oscillatory hallucinations. In these cases, we consider them to be oscillatory.
12All data used in this paper is licensed under a MIT License.
13https://github.com/Unbabel/COMET 14https://huggingface.co/
sentence-transformers/LaBSE
dataset (Fomicheva et al., 2022), which contains a high percentage of hallucinations for some language pairs (Specia et al., 2021; Tang et al., 2022). We show the performance of both these versions in Table 4. CometKiwi significantly outperforms the previous iteration of COMET-QE. This hints that training quality estimation models with more negative examples can improve their ability to adequately penalize hallucinations.

heading: D Computational runtime of our detectors
text: Our detectors do not require access to a GPU machine. All our experiments have been ran on a machine with 2 physical Intel(R) Xeon(R) Gold 6348 @ 2.60GHz CPUs (total of 112 threads). Obtaining Wass-to-Unif scores for all the 3415 translations from the Guerreiro et al. (2022) dataset takes less than half a second, while Wass-to-Data scores are obtained in little over 4 minutes.

heading: E Evaluation Metrics
text: We use scikit-learn (Pedregosa et al., 2011) implementations of our evaluation metrics.15

heading: F Tracing-back performance boosts to the construction of the reference set Rx
text: In Section 6.1 in the main text, we showed that evaluating how distant a given translation is compared to a data-driven reference distribution–rather than to an ad-hoc reference distribution– led to increased performance. Therefore, we will now analyze the construction of the reference set Rx to obtain Wass-to-Data scores (step 2 in Figure 1). We conduct experiments to investigate the importance of the two main operations in this process: defining and length-filtering the distributions in Rheld. Construction of Rheld. To construct Rheld, we first need to obtain the source attention mass distributions for each sample in Dheld. If Dheld is a parallel corpus, we can force-decode the reference translations to construct Rheld. As shown in Table 5, this construction produces results similar to using good-quality model-generated translations. Moreover, we also evaluate the scenario where Rheld is constructed with translations of any quality. Table 5 shows that although filtering for quality
15https://scikit-learn.org

heading: ABLATION AUROC ↑ FPR@90TPR ↓
text: improves performance, the gains are not substantial. This connects to findings by Guerreiro et al. (2022): hallucinations exhibit different properties from other translations, including other incorrect translations. We offer further evidence that properties of hallucinations—in this case, the source attention mass distributions—are not only different to those of good-quality translations but also to most other model-generated translations.
Length-filtering the distributions in Rheld. The results in Table 6 show that length-filtering boosts performance significantly. This is expected: our translation-based length-filtering penalizes translations whose length is anomalous for their respective source sequences. This is particularly useful for detecting oscillatory hallucinations.

heading: G Ablations
text: We perform ablations on Wass-to-Data and WassCombo for all relevant hyperparameters: the length-

heading: ABLATION AUROC ↑ FPR@90TPR ↓
text: Random Sampling 80.65 0.15 57.06 2.04 Length Filtering 84.20 0.15 48.15 0.54

heading: ABLATION AUROC ↑ FPR@90TPR ↓
text: filtering parameter δ, the maximum cardinality of R, |R|max, the value of k to compute the Wass-to-Data scores (step 4 in Figure 1), and the threshold on Wass-to-Unif scores to compute WassCombo scores. The results are shown in Table 7 to Table 10, respectively. We also report in Table 11 the performance of Wass-to-Data with a 0/1 cost function instead of the ℓ1 distance function.
On length-filtering. The results in Table 7 show that, generally, the bigger the length window, the worse the performance. This is expected: if the test translation is very different in length to those obtained for the source sequences in Rx, the more penalized it may be for the length mismatch instead of source attention distribution pattern anomalies.
On the choice of |R|max. Table 8 shows that increasing |R|max leads to better performance, with reasonable gains obtained until |R|max = 2000.

heading: ABLATION AUROC ↑ FPR@90TPR ↓

heading: ABLATION AUROC ↑ FPR@90TPR ↓
text: While this increase in performance may be desirable, it comes at the cost of higher runtime.
On the choice of k. The results in Table 9 show that the higher the value of k, the worse the performance. However, we do not recommend using the minimum distance (k = 1) as it can be unstable.
On the choice of threshold on Wass-to-Unif scores. Table 10 show that, generally, a higher threshold τ leads to a better performance of WassCombo. Wass-to-Unif scores are generally very high for fully detached hallucinations, a type of hallucinations that Wass-to-Data struggles more to detect. Thus, when combined in Wass-Combo, we obtain significant boosts in overall performance. However, if the threshold on Wass-to-Unif scores is set too low, Wass-to-Combo will correspond to Wass-to-Unif more frequently which may not be desirable as Wass-to-Data outperforms it for all other types of hallucinations. If set too high, fewer fully detached hallucinations may pass that threshold and may then be misidentified with Wassto-Data scores.
On the choice of Wass-to-Data cost function. Table 11 shows that using the ℓ1 cost function instead of using the 0/1 cost function to compute Wass-to-Data scores leads to significant improve-

heading: ABLATION AUROC ↑ FPR@90TPR ↓

heading: COST FUNCTION AUROC ↑ FPR@90TPR ↓
text: ments. This suggests that when comparing the source mass attention distribution of a test translation to other such distributions obtained for other translations (instead of the ad-hoc uniform distribution used for Wass-to-Unif scores), the information from the location of the source attention mass is helpful to obtain better scores.
On the formulation of Wass-Combo. To combine the information from Wass-to-Unif and Wassto-Data, we could also perform a convex combination of the two scores:
swc(x) = λswtd(x) + (1− λ)s̃wtu(x) (10)
for a predefined scalar parameter λ. In Table 12, we show that this method is consistently subpar to our two-pass approach. In fact, this linear interpolation is not able to bring additional gains in performance for any of the tested parameters λ when compared to Wass-to-Data.

heading: H Analysis against ALTI+
text: Concurrently to our work, Dale et al. (2022) leveraged ALTI+ (Ferrando et al., 2022), a method that evaluates the global relative contributions of both source and target prefixes to model predictions, for detection of hallucinations. As hallucinations are translations detached from the source sequence, ALTI+ is able to detect them by identifying sentences with minimal source contribution. In Table 13, we show that ALTI+ slightly outperforms Wass-Combo for fully detached hallucinations, but lags considerably behind on what comes to de-

heading: ABLATION AUROC ↑ FPR@90TPR ↓
text: tecting strongly detached and oscillatory hallucinations.
I Error Analysis of Wass-Combo
We show a qualitative analysis on the same fixedthreshold scenario described in Section 6.2 in Figure 4. Differently to Figure 3, we provide examples of translations that have not been detected by WassCombo for the chosen threshold.
Our detector is not able to detect fully detached hallucinations that come in the form of exact copies of the source sentence. For these pathological translations, the attention map is mostly diagonal and is thus not anomalous. Although these are severe errors, we argue that, in a real-world application, such translations can be easily detected with string matching heuristics.
We also find that our detector Wass-Combo struggles with oscillatory hallucinations that come in the form of mild repetitions of 1-grams or 2-

heading: METHOD AUROC ↑ FPR@90TPR ↓
text: grams (see example in Figure 4). To test this hypothesis, we implemented the binary heuristic top n-gram count (Raunak et al., 2021; Guerreiro et al., 2022) to verify whether a translation is a severe oscillation: given the entire Dheld, a translation is flagged as an oscillatory hallucination if (i) it is in the set of 1% lowest-quality translations according to CometKiwi and (ii) the count of the top repeated 4-gram in the translation is greater than the count of the top repeated source 4-gram by at least 2. Indeed, more than 90% of the oscillatory hallucinations not detected by Wass-Combo in Figure 4 were not flagged by this heuristic. We provide 8 examples randomly sampled from the set of oscillatory hallucinations not detected with Wass-Combo
in Table 14. Close manual evaluation of these hallucinations further backs the hypothesis above.

heading: J Experiments on the MLQE-PE dataset
text: In order to establish the broader validity of our model-based detectors, we present an analysis on their performance for other NMT models and on mid and low-resource language pairs. Overall, the detectors exhibit similar trends to those discussed in the main text (Section 6).

heading: J.1 Model and Data
text: The dataset from (Guerreiro et al., 2022) analysed in the main text is the only available dataset that contains human annotations of hallucinated translations. Thus, in this analysis we will have to make use of other human annotations to infer annotations for hallucinations. For that end, we follow a similar setup to that of (Tang et al., 2022) and use the MLQE-PE dataset (Fomicheva et al., 2022)— that has been reported to contain low-quality translations and hallucinations for NE-EN and RO-EN (Specia et al., 2021)— to test the performance of our detectors on these language pairs.
The NE-EN and RO-EN MLQE-PE datasets contain 7000 translations and their respective human quality assessments (from 1 to 100). Each translation is scored by three different annotators. As hal-
lucinations lie at the extreme end of NMT pathologies (Raunak et al., 2021), we consider a translation to be a hallucination if at least two annotators (majority) gave it a quality score of 1.16 This process leads to 30 hallucinations for NE-EN and 237 hallucinations for RO-EN. Although the number of hallucinations for NE-EN is relatively small, we decide to also report experiments on this language pair because the type of hallucinations found for NE-EN is very different to those found for RO-EN: almost all NE-EN hallucinations are oscillatory, whereas almost all RO-EN are fully detached.
To obtain all model-based information required to build the detectors, we use the same Transformer models that generated the translations in the datasets in consideration. All details can be found in Fomicheva et al. (2022) and the official project repository17. Moreover, to build our heldout databases of source mass distributions, we used readily available Europarl data (Koehn, 2005) for RO-EN (∼100k samples), and filtered Nepali Wikipedia monolingual data18 used in (Koehn et al.,
16We tried other methods to infer hallucinations from the annotations (e.g. average quality score below 5, at least one quality score of 1). The trends on performance were similar to those reported in this section.
17https://github.com/facebookresearch/mlqe 18Note that creating the datastore only requires access to monolingual data. If quality filtering is needed and references are not available, we suggest using quality estimation or

heading: Model-based Detectors
text: 2019) for NE-EN (∼80k samples).

heading: J.2 Results
text: The trends in Section 6.1 hold for other language pairs. The results in Table 15 establish the broader validity of our detectors for other NMT models and, importantly, for mid and low-resource language pairs. Similarly to the analysis in 6.1, we find that our detectors (i) exhibit better performance than other model-based detectors with significant gains on the low-resource NE-EN language pair; and (ii) can be competitive with external detectors that leverage large models.
The trends in Section 6.2 hold for other language pairs. In Section A, we remark that almost all NEEN hallucinations are oscillatory, whereas almost all RO-EN hallucinations are fully detached. With that in mind, the results in Table 15 establish the validity of the claims in the main-text (Section 6.2) on these language pairs: (i) detecting fully detached hallucinations is remarkably easy for most detectors, and Wass-to-Unif outperforms Wass-to-Data on this type of hallucinations (see results for ROEN); and (ii) our detectors significantly outperform all previous model-based detectors on detecting oscillatory hallucinations (see results for NE-EN), which further confirms the notion that some detectors specialize on different types of hallucinations (e.g Attn-ign-SRC is particularly fit for detecting fully detached hallucinations, but it does not work for oscillatory hallucinations).
cross-lingual embedding similarity models to filter low-quality translations.
ACL 2023 Responsible NLP Checklist

heading: A For every submission:
text: 3 A1. Did you describe the limitations of your work?
Limitations Section
A2. Did you discuss any potential risks of your work? Not applicable. Left blank.
3 A3. Do the abstract and introduction summarize the paper’s main claims? Introduction is Section 1
3 A4. Have you used AI writing assistants when working on this paper? ChatGPT; DeepL; Grammarly; Assistance purely with the language of the paper. Used throughout the paper.
B 3 Did you use or create scientific artifacts? 5 and Appendix
3 B1. Did you cite the creators of artifacts you used? 5 and Appendix
3 B2. Did you discuss the license or terms for use and / or distribution of any artifacts? 1 and Appendix
B3. Did you discuss if your use of existing artifact(s) was consistent with their intended use, provided that it was specified? For the artifacts you create, do you specify intended use and whether that is compatible with the original access conditions (in particular, derivatives of data accessed for research purposes should not be used outside of research contexts)? Not applicable. Left blank.
B4. Did you discuss the steps taken to check whether the data that was collected / used contains any information that names or uniquely identifies individual people or offensive content, and the steps taken to protect / anonymize it? Not applicable. Left blank.
3 B5. Did you provide documentation of the artifacts, e.g., coverage of domains, languages, and linguistic phenomena, demographic groups represented, etc.? 5 and Appendix
3 B6. Did you report relevant statistics like the number of examples, details of train / test / dev splits, etc. for the data that you used / created? Even for commonly-used benchmark datasets, include the number of examples in train / validation / test splits, as these provide necessary context for a reader to understand experimental results. For example, small differences in accuracy on large test sets may be significant, while on small test sets they may not be. 5 and Appendix
C 3 Did you run computational experiments? 5, 6, Appendix
3 C1. Did you report the number of parameters in the models used, the total computational budget (e.g., GPU hours), and computing infrastructure used? Appendix
The Responsible NLP Checklist used at ACL 2023 is adopted from NAACL 2022, with the addition of a question on AI writing assistance.
3 C2. Did you discuss the experimental setup, including hyperparameter search and best-found hyperparameter values? Appendix
3 C3. Did you report descriptive statistics about your results (e.g., error bars around results, summary statistics from sets of experiments), and is it transparent whether you are reporting the max, mean, etc. or just a single run? 6 and Appendix
3 C4. If you used existing packages (e.g., for preprocessing, for normalization, or for evaluation), did you report the implementation, model, and parameter settings used (e.g., NLTK, Spacy, ROUGE, etc.)? Appendix
D 7 Did you use human annotators (e.g., crowdworkers) or research with human participants? Left blank.
D1. Did you report the full text of instructions given to participants, including e.g., screenshots, disclaimers of any risks to participants or annotators, etc.? No response.
D2. Did you report information about how you recruited (e.g., crowdsourcing platform, students) and paid participants, and discuss if such payment is adequate given the participants’ demographic (e.g., country of residence)? No response.
D3. Did you discuss whether and how consent was obtained from people whose data you’re using/curating? For example, if you collected data via crowdsourcing, did your instructions to crowdworkers explain how the data would be used? No response.
D4. Was the data collection protocol approved (or determined exempt) by an ethics review board? No response.
D5. Did you report the basic demographic and geographic characteristics of the annotator population that is the source of the data? No response.

heading: Future Work
text: Nevertheless, research on hallucinations for larger NMT models makes for an exciting line of future work and would be valuable to assess the broad validity of our claims. Secondly, although our method does not require any training data or human annotations, it relies on access to a pre-existing database of source mass distributions. This can be easily obtained offline by running the model on monolingual data to obtain the distributions. Nevertheless, these datastores need not be costly in terms of memory. In fact, in Appendix J, we validate our detectors for datastores that contain less than 100k distributions