VADE : Visual Attention Guided Hallucination Detection and Elimination (2025)

Abstract: Vision Language Models (VLMs) have achieved significant advancements in complex visual understanding tasks. However, VLMs are prone to hallucinations—generating outputs that lack alignment with visual content. This paper addresses hallucination detection in VLMs by leveraging the visual grounding information encoded in transformer attention maps. We identify three primary challenges in this approach: the elective nature of visual grounding for certain tokens, the high-dimensional and noisy nature of attention maps, and the dynamic sequence length of attention on previous tokens. To address these, we propose VADE, a novel sequence modelling approach to effectively learn complex sequential patterns from high-dimensional and noisy attention maps for fine-grained hallucination detection and mitigation. VADE achieves an average PR-AUC of 80% in hallucination detection on M-HalDetect across four different model architectures and an 5% improvement in hallucination mitigation on MSCOCO.

## introduction large vision-language models (vlms) have ushered in a new era of image comprehension, achieving near-human performance on intricate tasks such as image captioning and visual question answering. this breakthrough was pioneered by the seminal work on clip @xcite , paving the way for several instruction-tuned vlm architectures @xcite @xcite ) that seamlessly integrate visual and textual information. however, akin to their large language model (llm) counterparts, vlms exhibit a concerning tendency for hallucination, generating coherent yet unsubstantiated outputs that lack grounding in the actual visual data. hallucinations in vlms majorly occur due to: (a) distributional shift, when they encounter data distributions different from their training data (b) data biases, where they learn and reflect biases present in their training data (c) lack of grounding, where insufficient grounding to the visual input data leads to fabricated content, specifically when the llm's language prior is stronger than the visual grounding. this work focuses on detecting hallucinations in vlms by analyzing the visual grounding of their outputs with respect to the input images. we leverage transformer's attention maps to assess visual grounding in vlms, as they provide interpretability and a fine-grained view without requiring additional reference data, tools, or knowledge. figure 1 illustrates the relationship between visual grounding signal and hallucinations for the generated image caption. the visual grounding signal used here is the aggregated attention weights pertaining to the input image tokens across all layers and heads for each generated text token. we clearly observe that the aggregated visual attention scores tend to be higher for accurately grounded tokens and drop significantly for hallucinated tokens. to empirically investigate this behavior, we conducted tests on four vlms within the context of image describing tasks on 1000 samples from m-haldetect dataset and false positives (suffix tokens, punctuation, etc.). @xcite containing fine-grained hallucination labels. as shown in figure 2a , we noticed a substantial proportion of hallucinated tokens had low visual attention scores (i.e., below the 20 th percentile of all visual attention scores). however, there are three major challenges in leveraging attention map based scores in detecting hallucinations: to tackle these challenges, we introduce our approach called visual attention guided hallucination detection and elimination (vade), a novel sequence learning paradigm to leverage the sequence of raw attention-maps corresponding to input/previous tokens for hallucination detection. specifically, at each time step, vade extracts attention weights from all layers and heads corresponding to the input image tokens, prompt tokens, and previously generated tokens. these attention weights are then used to construct a structured sequence of attention maps. remarkably, with access to a small amount of annotated data, vade can learn intricate patterns present in the attention maps, enabling it to distinguish between hallucinated and accurate segments. by formulating the task as a sequence learning problem, vade can leverage fine-grained temporal information across the previous tokens from their attention maps without any lossy aggregation operations, thereby reducing the impact of noise and sinks. we can further integrate the vade detector during decoding using rejection sampling strategy to mitigate hallucinations. the proposed method is architecture-independent, functioning across cross-attention and autoregressive architectures, applicable for diverse vlms. through extensive experiments on hallucination benchmarks, along with detailed ablation studies, we demonstrate that vade achieves superior and generalized hallucination detection and mitigation performance across various vlms. ## related work previous approaches have focused on analyzing token probabilities, logits, and entropy @xcite or using probing classifiers with hidden states @xcite . these methods often struggle to detect complex hallucinations. recent studies @xcite propose detection schemes through vlm fine-tuning with custom classification heads, but these lack cross-domain generalizability. hallucination mitigation strategies involve integrating detection into vlm sampling @xcite or decoding processes @xcite @xcite to improve candidate selection and probability calibration. some recent studies @xcite @xcite have explored leveraging attention weights for hallucination detection and mitigation. lookback lens @xcite handcraft a simple feature vector from aggregated attention weights and train a logistic regression classifier to detect contextual hallucinations for text summarization task in llms. opera @xcite , proposes to detect the knowledge aggregation patterns in vlm's selfattention maps (i.e. attention sinks) to detect the onset of hallucinated continuations. pai @xcite and ibd @xcite , propose to amplifying the attention weights assigned to image tokens to enhance visual grounding during inference for hallucination mitigation. these methods depending upon handcrafted features and manually identified emerging patterns have limited effectiveness and generalizability to detect complex and intricate cases. differing from existing works, our proposed method (vade), leverages the attention maps in their raw form to learn complex sequential patterns to determine visual grounding for vlms using small annotated data. vade, proves effective in distinguishing visual grounding signal from noise in attention maps caused by the presence outliers, sinks and null attentions. ## conclusion in this work, we conducted a comprehensive analysis of the properties of multi-head and multilayered visual attention patterns, evaluating their effectiveness in detecting hallucinations in vlms. our findings revealed that statistical measures, aggregation schemes, and feature selection strategies used to derive visual grounding signals from attention maps are suboptimal for detecting hallucinations. unlike conventional methods that rely on handcrafted features or attention aggregation, our proposed method vade learns complex sequential patterns from high-dimensional and noisy attention maps using a limited amount of annotated data. our extensive experiments across multiple vlms and datasets demonstrate the superiority of vade over existing techniques, achieving state-ofthe-art performance in both hallucination detection and mitigation tasks. furthermore, vade exhibits strong generalization capabilities, with minimal performance degradation when transferred to outof-distribution datasets, highlighting its potential for real-world applications.