Mitigating Hallucination in Large Vision-Language Models through Aligning Attention Distribution to Information Flow (2025)

Abstract: Due to the unidirectional masking mechanism, Decoder-Only models propagate information from left to right. LVLMs (Large Vision-Language Models) follow the same architecture, with visual information gradually integrated into semantic representations during forward propagation. Through systematic analysis, we observe that over 80% of the visual information is absorbed into the semantic representations. However, the model’s attention still predominantly focuses on the visual representations. This misalignment between the attention distribution and the actual information flow undermines the model’s visual understanding ability and contributes to hallucinations.To address this issue, we enhance the model’s visual understanding by leveraging the core information embedded in semantic representations. Specifically, we identify attention heads that focus on core semantic representations based on their attention distributions. Then, through a two-stage optimization paradigm, we propagate the advantages of these attention heads across the entire model, aligning the attention distribution with the actual information flow.We evaluate our method on three image captioning benchmarks using five different LVLMs,demonstrating its effectiveness in significantly reducing hallucinations. Further experiments reveal a trade-off between reduced hallucinations and richer details. Notably, our method allows for manual adjustment of the model’s conservativeness, enabling flexible control to meet diverse real-world requirements.

## introduction large vision-language models (lvlms) @xcite @xcite integrate large language models (llms) with visual encoders, aligning the extracted visual features with the semantic space of the llms to enable comprehensive understanding of visual content. however, they often suffer from hallucination @xcite @xcite , where the models may generate content that is inconsistent with the visual evidence, thereby severely limiting their reliability in realworld scenarios. recent studies have explored methods to mitigate hallucinations in lvlms. @xcite and @xcite attribute hallucinations to the influence of language priors, proposing contrastive decoding-based methods to suppress language priors. however, they do not explicitly enhance the model's visual understanding capabilities. other studies @xcite @xcite argue that lvlms exhibit biased mechanisms in visual attention distribution, and propose optimizations such as increasing the relative weight of key visual tokens. in this work, however, we point out that these approaches make suboptimal adjustments to the information flow, as they overlook the influence of information aggregation. in fact, the unidirectional masked generation process in transformer-based models can be regarded as a form of information flow, where information from earlier tokens in the input or generated sequence flows toward later tokens. since lvlms adopt this architecture, their understanding of visual content can be modeled as a process in which visual information flows into the semantic representations (since visual tokens are positioned before the textual tokens). based on this perspective, we follow the attention rollout @xcite to conduct an in-depth analysis of information flow within lvlms. the visualization results are shown in fig. 1 . specifically, we use f l i,j to represent the contributions of the i-th input embedding to the j-th representation in layer l. considering the residual connection in the forward propagation, we formulate the recursive relation as f @xmath0, where w attn is the attention weight after average pooling across multi-heads. the contributions of each input embedding were normalized to 1 across all representations. we perform layer-wise quantification of visual embeddings' contributions to the visual and semantic representations, respectively. as shown in the visualization, visual information progressively flows into semantic representations across layers. in the final layers, the majority of the visual information is integrated into the semantic representations. this indicates that during the forward pass of the lvlms, visual information is gradually encoded into the semantic representations. while prior works largely concentrate on optimizing the attention distribution to visual features, they neglect the critical insight that essential information has already been integrated into semantic representations, where attention refinement may be more impactful. to further validate this insight, we mask all visual representations and compare the consistency between the logits distribution of the vision-masked input and that of the regular (unmasked) input, as illustrated in fig. 2 . we observe that applying masking to visual representations starting from earlier layers leads to significant deviations in the model's outputs, whereas masking from later layers has minimal impact on the final outputs. this finding further supports our view that visual information has already been integrated into the semantic representations in the later layers. building upon the above findings, as the number of layers increases, textual representations progressively integrate more visual information and thus play an increasingly important role. consequently, lvlms should place greater attention on these integrated representations. to investigate this, we statistically analyze the attention allocation between semantic representations and others at each layer, as illustrated in fig. 3 . the model's attention to semantic representations is significantly lower than expected, accounting for only about one-fifth of the total attention. this phenomenon stems from the nature of supervision in lvlm training, which is primarily driven by soft supervision from endto-end data and lacks explicit guidance on the attention distribution. we argue that the model's attention distribution should align with the actual flow pattern of visual information. otherwise, hallucinations may arise, as the model does not sufficiently attend to regions enriched with visual information. to address the aforementioned challenge, we propose sevi (semantic-enhanced visual interpretation), a novel training-free approach that augments the model's attention to those semantic representations that have absorbed visual information. furtherly, we encourage the model to focus on the most meaningful representations in order to reduce the occurrence of hallucinations. several studies @xcite have revealed that contextual information tends to spontaneously concentrate in some tokens, forming anchor tokens that encapsulate core information. these tokens are typically characterized by receiving disproportionately high attention @xcite . therefore, our method focuses the model's attention on these core semantic representations at higher layers where information integration and aggregation occur. the core visual information contained within these representations provides more effective guidance for the model's understanding of visual inputs, thereby mitigating hallucination. specifically, we identify superior attention heads within the multi-head attention mechanism-those that attend to core semantic representations-and use their attention distribution as the target distribution to guide the optimization of the model's overall attention. to achieve this, we design a two-stage attention optimization paradigm, incorporating a smoothing mechanism to ensure stable performance improvements. we evaluate the effectiveness of our method in mitigating hallucinations by testing it with five lvlms: instructblip @xcite , llava-1.5 @xcite , llava-next @xcite , qwen2-vl-instruct @xcite , and qwen2.5-vl-instruct @xcite , on three image captioning benchmarks: chair @xcite , amber @xcite , and detailcaps @xcite . experimental results show that our approach significantly reduces hallucinations while supporting manual adjustment of the model's conservativeness, enabling a flexible trade-off between cautious outputs (fewer hallucinations) and comprehensive descriptions (richer details). our main contributions are as follows: various approaches have been proposed to mitigate hallucinations in lvlms, such as improving training data quality @xcite @xcite or designing specific data formats @xcite @xcite to enhance model reliability. however, these training-based methods often suffer from limited scalability. in recent years, the rapid evolution of lvlm backbones has brought increasing attention to training-free methods. early approaches @xcite @xcite primarily relied on post-processing techniques to correct hallucinated content after generation. however, the complexity of such pipelines and their dependence on external modules limit their practical usability. researchers have since shifted focus toward investigating the root causes of hallucinations, exploring targeted optimizations in both the decoding strategies and the decoding processes. some methods @xcite @xcite apply contrastive decoding algorithms to refine the logits, thereby reducing the likelihood of hallucinationrelated tokens. while these approaches improve the expressiveness of the model and enhance the transfer of visual information into textual form, they do not strengthen the model's intrinsic crossmodal understanding. other works have proposed enhancements to the attention mechanism, such as correcting visual positional bias @xcite , directly boosting visual attention @xcite , or redistributing attention weights @xcite . however, these methods primarily concentrate on visual attention and overlook the vision-integrated semantic representation. in contrast to these approaches, we analyze the information flow between the visual and textual modalities in lvlms and focus the model's attention on core information within semantic representations, thereby reducing hallucinations more effectively. ## preliminaries lvlms consist of a visual encoder and an llm backbone. high-dimensional features extracted by the visual encoder are concatenated with textual embeddings and jointly fed into the llm. subsequently, the visual and semantic features undergo cross-modal interaction through the self-attention layers within the llm. the cross-modal interaction in layer l simplified as when generating the (j+1)-th token, the model's attention to feature x l i at layer l across all heads is quantified by the weight value w i,j . a larger w i,j indicates that the model's current layer relies more heavily on the information from x l i when determining the generated content. therefore, we can intuitively infer that more important features x l i should generally correspond to larger attention weights w i,j . ## method we attribute the occurrence of hallucinations to a misalignment between the attention distribution of lvlms and the visual information flow. to address this, we enhance the visual interpretation capabilities of lvlms by leveraging semantic representations that encapsulate visual information. we first identify superior attention heads that focus on core semantic representations, and then propagate their strengths to other heads through a two-stage optimization paradigm, as illustrated in fig. 4 . ## conclusion we systematically analyze the information flow in lvlms and verify that visual information is indeed integrated into the semantic representations. however, the model's attention remains predominantly focused on the visual representation. this inconsistency impairs the model's visual understanding and contributes to hallucination. to address this, we propose semantic-enhanced visual interpretation (sevi), a method that guides the model's attention toward the core components of semantic representations through a two-stage optimization process. extensive experiments demonstrate that our approach significantly mitigates hallucinations. while our method optimizes the attention distribution in lvlms based on the underlying information flow, it does not directly enhance the flow mechanism itself. in future work, we plan to explore more efficient strategies for visual information propagation.