Optimal Transport for Unsupervised Hallucination Detection in Neural Machine Translation (2023)

Abstract: Neural machine translation (NMT) has become the de-facto standard in real-world machine translation applications. However, NMT models can unpredictably produce severely pathological translations, known as hallucinations, that seriously undermine user trust. It becomes thus crucial to implement effective preventive strategies to guarantee their proper functioning. In this paper, we address the problem of hallucination detection in NMT by following a simple intuition: as hallucinations are detached from the source content, they exhibit encoder-decoder attention patterns that are statistically different from those of good quality translations. We frame this problem with an optimal transport formulation and propose a fully unsupervised, plug-in detector that can be used with any attention-based NMT model. Experimental results show that our detector not only outperforms all previous model-based detectors, but is also competitive with detectors that employ external models trained on millions of samples for related tasks such as quality estimation and cross-lingual sentence similarity.

## introduction neural machine translation (nmt) has achieved tremendous success @xcite , becoming the mainstream method in real-world applications and production systems for automatic translation. although these models are becoming evermore accurate, especially in high-resource settings, they may unpredictably produce hallucinations. these are severely pathological translations that are detached from the source sequence content @xcite @xcite . crucially, these errors have the potential to seriously harm user trust in hard-to-predict ways @xcite , hence the evergrowing need to develop security mechanisms. one appealing strategy to address this issue is to develop effective on-the-fly detection systems. in this work, we focus on leveraging the crossattention mechanism to develop a novel hallucination detector. this mechanism is responsible for selecting and combining the information contained in the source sequence that is relevant to retain during translation. therefore, as hallucinations are translations whose content is detached from the source sequence, it is no surprise that connections between anomalous attention patterns and hallucinations have been drawn before in the literature @xcite @xcite . these patterns usually exhibit scattered source attention mass across the different tokens in the translation (e.g. most source attention mass is concentrated on a few irrelevant tokens such as punctuation and the end-of-sequence token). inspired by such observations, previous work has designed ad-hoc heuristics to detect hallucinations that specifically target the anomalous maps. while such heuristics can be used to detect hallucinations to a satisfactory extent @xcite , we argue that a more theoretically-founded way of using anomalous attention information for hallucination detection is lacking in the literature. rather than aiming to find particular patterns, we go back to the main definition of hallucinations and draw the following hypothesis: as hallucinationscontrary to good translations-are not supported by the source content, they may exhibit cross-attention patterns that are statistically different from those found in good quality translations. based on this hypothesis, we approach the problem of hallucination detection as a problem of anomaly detection with an optimal transport (ot) formulation @xcite . namely, we aim to find translations with source attention mass distributions that are highly distant from those of good translations. intuitively, the more distant a translation's attention patterns are from those of good translations, the more anomalous it is in light of that distribution. our key contributions are: ## on-the-fly detection of hallucinations on-the-fly hallucination detectors are systems that can detect hallucinations without access to reference translations. these detectors are particularly relevant as they can be deployed in online applications where references are not readily available. 4 ## unsupervised hallucination detection with optimal transport anomalous cross-attention maps have been connected to the hallucinatory mode in several works @xcite @xcite . our method builds on this idea and uses the wasserstein distance to estimate the cost of transforming a translation source mass distribution into a reference distribution. intuitively, the higher the cost of such transformation, the more distant-and hence the more anomalous-the attention of the translation is with respect to that of the reference translation. ## conclusions we propose a novel plug-in model-based detector for hallucinations in nmt. unlike previous attempts to build an attention-based detector, we do not rely on ad-hoc heuristics to detect hallucinations, and instead pose hallucination detection as an optimal transport problem: our detector aims to find translations whose source attention mass distribution is highly distant from those of good quality translations. our empirical analysis shows that our detector outperforms all previous model-based detectors. importantly, in contrast to these prior approaches, it is suitable for identifying oscillatory hallucinations, thus addressing an important gap in the field. we also show that our detector is competitive with external detectors that use state-of-the-art quality estimation or cross-lingual similarity models. notably, this performance is achieved without the need for large models, or any data with quality annotations or parallel training data. finally, thanks to its flexibility, our detector can be easily deployed in real-world scenarios, making it a valuable tool for practical applications.