Reference-free Hallucination Detection for Large Vision-Language Models (2024)

Abstract: Large vision-language models (LVLMs) have made significant progress in recent years. While LVLMs exhibit excellent ability in language understanding, question answering, and conversations of visual inputs, they are prone to producing hallucinations. While several methods are proposed to evaluate the hallucinations in LVLMs, most are reference-based and depend on external tools, which complicates their practical application. To assess the viability of alternative methods, it is critical to understand whether the reference-free approaches, which do not rely on any external tools, can efficiently detect hallucinations. Therefore, we initiate an exploratory study to demonstrate the effectiveness of different reference-free solutions in detecting hallucinations in LVLMs. In particular, we conduct an extensive study on three kinds of techniques: uncertainty-based, consistency-based, and supervised uncertainty quantification methods on four representative LVLMs across two different tasks. The empirical results show that the reference-free approaches are capable of effectively detecting non-factual responses in LVLMs, with the supervised uncertainty quantification method outperforming the others, achieving the best performance across different settings.

## introduction large vision-language models (lvlms) like llava @xcite , minigpt-4 @xcite have demonstrated remarkable capabilities in understanding and generating complex visual and textual content. however, an emerging concern with these models is their tendency towards hallucination @xcite @xcite . for instance, a model might describe an object or event in the generated text that *equal contribution. most existing methods of assessing hallucinations rely on external models. @xcite propose pope, which requires automated segmentation tools like seem @xcite to identify present and absent objects in images. faithscore @xcite utilizes a visual entailment model @xcite to validate the factuality, focusing on object attributes and their relationships. nonetheless, these approaches increase the computational cost and are restricted by the capabilities of these models. figure 2 : two data sources: self-generated and handcrafted data. the test model assigns a probability of 0.9 to "cat" in self-generated input, while in hand-crafted input, "dog" receives a probability of 0.1. reference-free methods are a promising approach to address this issue, primarily utilizing the model's own knowledge. they have attracted great research interest in large language models (llms), with typical methods including the uncertainty-based methods @xcite @xcite @xcite , consistency-based methods @xcite and supervised uncertainty quantification (suq) approach @xcite . however, their potential in the context of lvlms is unknown. they have not yet been systematically studied, possibly due to the complexity of multimodal data. we are committed to bridging this gap. in this work, we collect and implement a battery of state-of-the-art methods, including four metrics of uncertainty-based methods, four variants of consistency-based methods, and the supervised uncertainty quantification method. the experiments are conducted on five lvlms with different types, versions, and sizes on yes-and-no and open-ended tasks. the main contributions of this paper are summarized as follows: ## reference-free hallucination detection methods uncertainty-based methods have the hypothesis that when lvlms are uncertain about generated information, generated tokens often have higher ## experimental results and analysis the comprehensive results on open-ended tasks are presented in tables 1, 2 and 3. the numbers in the pink and yellow background, respectively, represent the best results for the uncertainty-based methods and consistency-based methods, while the numbers in bold indicate the overall best results. we utilize auc-pr @xcite , which stands for the area under the precision-recall curve, to objectively evaluate the effectiveness of each model. the higher the value of auc-pr, the stronger the ability of this method for hallucination detection. ## conclusion this paper systematically studies reference-free approaches for detecting the hallucination of lvlms. our results indicate that the suq method performs best as a supervised method, but it relies on sufficient training data, which could limit its applicability in certain scenarios. consistency-based methods outperform uncertainty-based methods; however, they require more computation and are more suitable for black-box models. uncertaintybased methods are particularly well-suited for tasks similar to yes-or-no questions.