Defining and Quantifying Visual Hallucinations in Vision-Language Models (2025)

Abstract: The troubling rise of hallucination presents perhaps the most significant impediment to the advancement of responsible AI. In recent times, considerable research has focused on detecting and mitigating hallucination in Large Language Models (LLMs). However, it’s worth noting that hallucination is also quite prevalent in Vision-Language models (VLMs). In this paper, we offer a fine-grained discourse on profiling VLM hallucination based on the image captioning task. We delineate eight fine-grained orientations of visual hallucination: i) Contextual Guessing, ii) Identity Incongruity, iii) Geographical Erratum, iv) Visual Illusion, v) Gender Anomaly, vi) VLM as Classifier, vii) Wrong Reading, and viii) Numeric Discrepancy. We curate Visual HallucInation eLiciTation, a publicly available dataset comprising 2,000 samples generated using eight VLMs across the image captioning task, along with human annotations for the categories as mentioned earlier. To establish a method for quantification and to offer a comparative framework enabling the evaluation and ranking of VLMs according to their vulnerability to producing hallucinations, we propose the Visual Hallucination Vulnerability Index (VHVI). In summary, we introduce the VHILT dataset for image-to-text hallucinations and propose the VHVI metric to quantify hallucinations in VLMs, targeting specific visual hallucination types. A subset sample is available at: https://huggingface.co/datasets/vr25/vhil . The full dataset will be publicly released upon acceptance.

## visual hallucination -an extensive categorization despite the rapid advances in generative ai, policymakers @xcite are primarily concerned with the issue of hallucinations. these occurrences of hallucinations pose a significant risk of eroding trust in technology. for instance, when google's bard ai "hallucinated" during its initial public demonstration, alphabet experienced a temporary loss of $100 billion in market value @xcite . the study of hallucinations for llms has recently attracted considerable attention @xcite . this paper delves into visual hallucination, a phenomenon notably prevalent in numerous recent vlms. given that this field is still emerging, it is imperative to initially comprehend, classify, and quantify these phenomena while establishing a benchmark. this will aid the scientific community in collectively addressing this issue. compared to recent research @xcite @xcite , which has primarily investigated object hallucination in vlms using limited data. this paper aims to provide a comprehensive categorization of vlm hallucinations. we defined eight categories of visual hallucination: when the model generates unrelated elements that bear no resemblance to the subject at hand, highlighting the non-deterministic nature of the model. ## ➠ alarming identity incongruity (ii) it's when the model can't differentiate between a person's real and fake identity traits, causing a mismatch with the predicted identity. ## dataset the rise of generative ai has fueled online misinformation, as highlighted by the eu (commission, 2022). to address this, we focus our visual hallucination dataset on the news domain. since accurate annotations require factually correct references, we use the new york times twitter handle (newyorktimes, 2024) as our trusted source, covering a decade (2011-2021) of multimodal data. nyt tweets, authored by professional journalists, ensure grammatical accuracy and avoid common twitter issues. we specifically selected image-containing tweets for studying visual hallucinations, applying rigorous filtering to remove duplicates, irrelevant content, non-english tweets, hashtags, and urls, retaining only original, relevant alphanumeric data. ## visual hallucination vulnerability index (vhvi) as vlm usage grows, their tendency to hallucinate lacks standardized evaluation. to address this, we introduce vhvi, a comparative spectrum for ranking vlms by hallucination susceptibility, specifically in image captioning. when defining vhvi, we take several factors into account. firstly, not all captions/answers generated by a vlm are hallucinated, so it is important to determine the ratio of actual hallucinated captions/answers with the total number of captions/answers. in this context, we consider u as the total number of captions/answers produced by a vlm. moreover, vlms can exhibit varying degrees of hallucination, including alarming, mild, and low types. for instance, if we have two vlms and their total number of generated hallucinations in terms of captions/answers are the same, but vlm1 produces significantly more alarming hallucinations than vlm2, we must rank vlm1 higher in terms of vhvi. this comparative measure is achieved using multiplicative damping factors, δ h , δ m , and δ l which are calculated based on µ ± rank x × σ. initially, we calculate the hvi for all the vlms, considering δ h , δ m , and δ l as one. with these initial vhvis, we obtain the mean (µ) and standard deviation (σ), allowing us to recalculate the hvis for all the llms. the resulting hvis are then ranked and scaled, providing a comparative spectrum as pre-sented in equation 1, similar to z-score normalization (wikipedia_zscore) and/or min-max normalization (wikipedia_min_max). having damping factors enables easy exponential smoothing with a handful of data points, 3/5 in this case. finally, for ease of interpretability, vhvi is scaled between 0 -100. please see figure 2 for the vhvi ranking of three vlms. ## conclusion the enthusiasm and achievements surrounding generative ai models have led to their widespread adoption, and this trend is only expected to flourish. however, one of the most significant challenges faced by these models today is hallucination. in light of this, the benchmark and visual hallucination vulnerability index (vhvi) will continue to serve the wider scientific community and aid policy-makers. benchmark and vhvi will be publicly open for further collaborative updates.