Visual Description Grounding Reduces Hallucinations and Boosts Reasoning in LVLMs

## Abstract

Abstract Large Vision-Language Models (LVLMs) often produce responses that misalign with factual information, a phenomenon known as hallucinations. While hallucinations are well-studied, the exact causes behind them remain underexplored. In this paper, we first investigate the root causes of hallucinations in LVLMs. Our findings reveal that existing mitigation techniques primarily reduce hallucinations for visual recognition prompts—those that require simple descriptions of visual elements—but fail for cognitive prompts that demand deliberate reasoning. We identify the core issue as a lack of true visual perception in LVLMs: although they can accurately recognize visual elements, they struggle to fully interpret these elements in the context of the input prompt and effectively link this recognition to their internal knowledge, which is critical for reasoning. To address this gap, we introduce Visual Description Grounded Decoding (VDGD), a simple, robust, and training-free method designed to enhance visual perception and improve reasoning capabilities in LVLMs. VDGD works by first generating a detailed description of the image and appending it as a prefix to the instruction. During response generation, tokens are sampled based on their KL divergence to the description, favoring candidates with lower divergence. Experimental results on multiple visual reasoning benchmarks and LVLMs demonstrate that VDGD consistently outperforms existing baselines 2% - 33%. Finally, we introduce VaLLu, a benchmark designed for comprehensive evaluation of the cognitive capabilities of LVLMs. Code is available here: https://github.com/Sreyan88/VDGD 1 1 1 Corresponding Author: sreyang@umd.edu. * denotes equal technical contribution.

## 1 Introduction

Large Language Models (LLMs) pre-trained on web-scale text data with the next token prediction objective implicitly compress world knowledge in their parameters (Zhao et al., ). These models learn general-purpose representations, which can then be aligned with the desired response characteristics (Zhang et al., ). This step generally involves fine-tuning on instruction-response pairs, also known as instruction tuning (IT) (Wei et al., ), and is followed by an optional step of reinforcement learning from human feedback (RLHF) (Bai et al., ). Such aligned LLMs can be engaged through text-based inputs, or prompts, directing them to perform various tasks, including those based on information retrieval and reasoning.

Recent advancements in the research community have demonstrated success in multi-modal alignment, whereby fine-tuning LLMs with multi-modal instruction-response pairs enables them to execute tasks that conform to the text prompt as well as additional multi-modal inputs (Yin et al., ). Since their introduction, the research community has extensively evaluated the capabilities of Large Vision-Language Models (LVLMs) across areas such as visual recognition (Wang et al., ), perception (Fu et al., ), and reasoning (Yue et al., ). Researchers have made significant efforts to enhance these capabilities by refining model architectures and training techniques (Zhang et al., ) or by implementing strategies to mitigate hallucinations (^2^22Hallucination refers to the mismatch between factual content and the model’s generated responses. Mitigation aims to reduce these discrepancies for more accurate outputs and improve task performance.) (Huang et al., ).

Among these approaches, training-free hallucination mitigation techniques stand out as a cost-effective and accessible method to enhance LVLM performance without the need for extensive computational resources. However, current hallucination mitigation techniques primarily focus on reducing hallucinations and improving the performance of visual recognition tasks (e.g., describing a scene, OCR, etc). Our extensive experiments show that these techniques fall short when applied to cognitive prompts requiring reasoning or knowledge extraction, indicating a significant gap in their ability to address hallucinations in more complex, reasoning-intensive scenarios.

To this end, we propose Visual Description Grounded Decoding (VDGD), a simple and novel training-free technique designed to improve LVLM reasoning by reducing hallucinations for cognitive prompts. Much like humans, who often write down their observations of an image in their own words and refer back to them while tackling complex reasoning tasks, we hypothesize that grounding the LVLM’s response generation in an explicit visual description can significantly aid its reasoning capabilities. Specifically, we first generate a description of the input image. Next, at every decoding step during auto-regressive response generation, we calculate the Kullback–Leibler divergence (KLD) between the top-k tokens in the current logit and all the logits of the image description. Finally, to predict the next word, we sample from the plausible candidates according to their KLD to the description, where lower KLD is given higher preference. By continuously referencing its descriptive summary of the visual content, the LVLM maintains alignment between the visual input and its generated responses, thereby reducing hallucinations. To summarize, our contributions are as follows:

- 1.
We conduct extensive controlled experiments to investigate the causes of hallucinations in LVLMs, categorizing them into distinct types. We demonstrate that existing hallucination mitigation techniques are effective only for visual recognition (VR) prompts—specifically, those that involve object recognition and image descriptions—while proving ineffective for cognitive prompts that require reasoning. We further show that these techniques are biased towards reducing language hallucinations in real-world scenes but fail to address hallucinations in non-real-world contexts and other categories.
- 2.
Identifying the core issue, we reveal that LVLMs suffer from a critical visual perception gap. Although these models can recognize visual elements and possess the reasoning skills needed for factual response generation, they struggle to fully perceive the visual information in the context of the prompt, resulting in hallucinated outputs for cognitive reasoning tasks.
- 3.
We introduce VDGD, a novel and training-free method that enhances LVLM reasoning by grounding response generation in visual descriptions. We evaluate VDGD on X benchmarks with prompts that require deliberate reasoning and how VDGD significantly outperforms existing techniques, improving performance by 2% - 33% by reducing hallucinations.
- 4.
Finally, we introduce VaLLu, a meticulously curated benchmark designed to evaluate the cognitive capabilities of LVLMs. Comprising 1,500 carefully selected instances, VaLLu is constructed by refining existing benchmarks and rigorously removing noisy and misleading examples identified through our thorough analysis. VaLLu emphasizes a balanced mix of diverse cognitive tasks, focusing exclusively on questions that require open-ended responses.

## 2 Related Work

Large Vision-Language Models In recent years, Large Vision-Language Models (LVLMs) have seen rapid advancements with the release of numerous new models. These developments have been driven by innovations in model architectures (Liu et al., ; Ye et al., ; Wang et al., ), training methods (Liu et al., ), alignment techniques (Chen et al., ), and data augmentation techniques (Liu et al., ; Wang et al., ). To assess the capabilities of these models, the research community has introduced a variety of benchmarks specifically designed to evaluate different aspects, including visual recognition and understanding in diverse scenarios (Liu et al., ; Kim et al., ), reasoning (Lu et al., ; Yue et al., ), knowledge-based information retrieval (Hu et al., ), and other specialized tasks.

Reducing Hallucinations to Improve Response Quality: Hallucination mitigation broadly aims to enhance the accuracy of model responses. While numerous studies have proposed methods to reduce hallucinations and improve response accuracy (Sicong Leng et al., ; Huang et al., ; Yin et al., ; Zhou et al., ) and evaluated their effectiveness (Wang et al., ; Guan et al., ), most focus on mitigating object hallucinations in visual recognition tasks. While these efforts are valuable, they fall short of assessing the true reasoning capabilities of LVLMs. Moreover, despite the growing focus on evaluation and mitigation, the underlying causes of hallucinations remain largely unexplored.

## 3 How Close Are We to Reliable LVLM Responses?

Experimental Setup for Analysis. We evaluate six LVLMs—LLaVA-v1, LLaVA-v1.5, LLaVA-v1.6, mPLUG-Owl2, InternLM-X, and CogVLM—all built on a 7B parameter language model. The models are tested across seven benchmarks: AMBER (visual recognition), SynthDoG (OCR), MMMU (expert-level reasoning), MathVista and MATH-Vision (mathematical reasoning), MMC (chart understanding), and MME and HallusionBench. To address potential limitations in existing evaluation metrics, which may not fully capture reasoning skills and often rely on string-matching techniques, we employ a combination of expert human evaluation and LLM-as-a-Judge (using GPT-4-turbo-2024-04-09). This paradigm has been extensively followed by prior studies in LLM evaluation (Zhou et al., ; Ghosh et al., ; Zheng et al., ) and hallucination evaluation (Sicong Leng et al., ; Yu et al., ; Liu et al., ). We devise an evaluation prompt that penalizes hallucinations and assigns scores from 1 to 5 based on factual correctness (see Appendix G). This approach also helps us standardize scoring across benchmarks and has a higher correlation ($\approx$0.97%) to expert human evaluations (see Section D). Quantitative results for all graphs in the paper are in Appendix L.

Preliminaries. As LVLMs advance and become more accessible, their application extends beyond merely generating image descriptions. Thus, to better understand visual perception and reasoning, we break down information processing by an LVLM into its constituent steps:

[linewidth=1pt, linecolor=black, leftmargin=1pt, rightmargin=1pt, innerleftmargin=10pt, innerrightmargin=10pt, innertopmargin=-3pt, innerbottommargin=2pt, backgroundcolor=gray!20, roundcorner=5pt]

$$ $\text{Visual Recognition}\xrightarrow[\text{Perception}]{}\text{Knowledge Extraction (optional)}\rightarrow\text{Reasoning (optional)}$ $$

Figure: Figure 1: Depending on the text instruction, an LVLM might be assessed on one or more different capabilities.

Figure 1 demonstrates that different prompts applied to a single image can assess various LVLM skills. Visual Recognition (VR) focuses on identifying visual elements and their relationships within the image, such as describing objects or specific details. Visual Perception (VP) extends beyond VR by interpreting and contextualizing these elements within the broader scene (Fu et al., ), essential for tasks requiring more than basic recognition. Prompts that demand knowledge-specific insights (also known as information-seeking prompts) engage in knowledge extraction (KE) learned from pre-training. Finally, prompts requiring reasoning involve combining visual data, textual prompts, and extracted knowledge to generate a response. While recognition depends on the vision encoder, knowledge extraction, and reasoning rely on the foundational language model. VP, implicitly learned during alignment (Ghosh et al., ; Zhou et al., ), bridges VR and higher cognitive functions, facilitating comprehensive understanding of the image in context to the prompt (Barrow et al., ; Torralba & Oliva, ; Hartley & Zisserman, ).

### 3.1 Do Hallucination Mitigation Techniques improve Cognitive Abilities?

Figure: Figure 2: (Left) Performance comparison of different LVLMs on various benchmarks. (Right) Performance comparison of different hallucination mitigation techniques applied to LLaVA-1.5.

Fig. 2 (left) compares the performance of LVLMs across 5 benchmarks. As we clearly see, while newer models that were trained using improved architectures and alignment training data boosted performance on the AMBER benchmark (for recognition), performance on other reasoning and information-seeking benchmarks remained stagnant.

Fig. 2 (right) compares the performance of LLaVA-1.5 across the same 5 benchmarks when employed with explicit hallucination mitigation techniques, with strategies ranging from improving model architectures to explicit response correction strategies, namely, Visual Contrastive Decoding (VCD) (Sicong Leng et al., ), OPERA (Huang et al., ), Woodpecker (Yin et al., ) and LURE (Zhou et al., ). As we clearly see, while all these methods boost performance on AMBER, performance on other reasoning and information-seeking benchmarks remains stagnant.

### 3.2 Do Mitigation Techniques Generalize Beyond Real-World Scenes?

Figure: Figure 3: (Left) Performance comparison of different LVLMs on various benchmarks when prompted to only describe the image. (Right) Performance comparison of different hallucination mitigation techniques applied to LLaVA-1.5.

As LVLMs are now being employed on a wide range of tasks, considerable research efforts are being made to evaluate the efficacy of LVLMs to reason on images beyond real-world scenes, i.e., charts (Liu et al., ) or math problems (Lu et al., ). However, how often do LVLMs hallucinate while describing images beyond real-world scenes? Fig 3 (left) compares the performance of various LVLMs across 5 benchmarks. MMMU and MathVista include images with mathematical figures and MMC includes charts from diverse sources. On the other hand, SynthDoG prompts the model for OCR and AMBER for image descriptions for real-world scenes. As we clearly see, compared to AMBER, models hallucinate more often when describing visuals beyond real-world scenes. Fig 3 (right) further shows that hallucination mitigation techniques improve performance on AMBER but rarely other benchmarks.

Figure: Figure 4: Types of Visual Recognition Hallucinations. We define Algo. 1 to divide VR hallucinations into 4 different categories automatically: Language, Vision, Style, and IT (explained further in Sec. 3.3). While language and vision hallucinations have been explored earlier, and methods to alleviate them have been proposed, we show for the first time that Style and IT hallucinations exist and existing methods fail to alleviate them.

### 3.3 Are all hallucinations equal?

Overview. In our detailed analysis of model responses on the AMBER dataset, we identified intriguing patterns in which hallucinations occur. Although the community has primarily focused on the broader topic of object hallucinations (Liu et al., ), our findings reveal that visual recognition hallucinations can be classified into several subcategories based on their specific occurrence patterns. Specifically, we perform the token-distribution analysis proposed by  (Lin et al., ).

Figure: Algorithm 1 Categorizing Visual Hallucinations

For a given instruction-response-image triplet, the instruction $i$ = {$i_{1}$, $i_{2}$, $\cdots$} and the image $v$ are first input to the aligned (or fine-tuned) model to obtain its response $r$ = {$r_{1}$, $r_{2}$,$\cdots$} via greedy decoding. Next, for each position $t$ in the response, a ‘context’ at this position is defined as to be $x_{t}$ = $i$ + {$r_{1}$, $\cdots$, $r_{t-1}$}. This “context” is then input to the base model (model from which the aligned model was instruction-tuned) to obtain its probability distribution for predicting the next token at position $t$, P${}_{\text{base}}$. We then define Base Rank as follows: Rank in P${}_{\text{base}}$ of the token at $t$ with the maximum P${}_{\text{align}}$ value. With the base rank denoted as $\eta$, the unshifted, marginal and shifted tokens are defined as when ($\eta$ $=$ 1), (1 $<$ $\eta$ $\leq$ 3) and ($\eta$ $>$ 3) respectively. Next, we use our judge (GPT-4) to extract the exact hallucinated phrases based on 3 types: object, relation, and action hallucinations. Finally, based on the Base Rank formulation and the hallucinated phrases, we propose Algorithm 1 to categorize identified hallucinations into 4 different categories based on their cause. The algorithm is further verbally described in Appendix K.4.

(1) Language Hallucinations: These are hallucinations that are caused when LVLMs over-rely on the language priors (or prior tokens in the response) rather than the input image. We attribute all hallucinated tokens that are unshifted as language hallucinations.

Figure: Figure 5: (Left) Frequency comparison of hallucination categories. (Right) Comparison for LLaVA-v1.5 with hallucination mitigation techniques. The top graph compares AMBER, and the bottom graph compares MMMU.

(2) Vision Hallucinations: Unlike language hallucinations, the tokens for vision hallucinations are generated by attending to the input image and are shifted or marginal tokens. These are hallucinations caused when the model fails to accurately recognize visual elements in an input image or inaccurate reasoning or knowledge extraction, among other factors.
(3) Style Hallucinations. These are hallucinations caused by style imitation. Open-source VLLMs are often trained on instruction-tuning data synthesized from proprietary closed-source models. Training on such datasets makes the VLLM learn to mimic the characteristics of responses in the IT dataset (Ghosh et al., ; Gudibande et al., ) (e.g., lengthy answers, a summary line at the end of the response, etc.). However, owing to the lack of sufficient knowledge to mimic its closed-source counterparts, open-source VLLMs might hallucinate facts while responding (Ghosh et al., ).
(4) Instruction Tuning (IT) Hallucinations. These are hallucinations caused by biases learned during the instruction tuning stage. LLMs are known to mimic the responses associated with the unfamiliar examples in the model’s instruction-tuning data (i.e., those unfamiliar to the pre-trained base model) (Ghosh et al., ). Formally, this can be defined as a distribution shift. As visual instruction tuning is an extreme case of such a distribution shift, IT hallucinations are a major cause of hallucinations in LLMs.

Performance Comparison. Fig. 5 (top;left) compares the frequency of different categories of hallucinations across various LVLMs for AMBER. Fig. 5 (top; right) displays the frequency of different hallucination categories for LLaVA-v1.5 alongside various hallucination mitigation techniques. Fig. 5 (bottom) offers a similar comparison for MMMU. Our key findings include: (1) Model-based improvements and mitigation strategies reduce language and vision hallucinations, but progress in style and IT hallucinations remains limited. (2) Specific mitigation techniques are more effective for certain types of hallucinations. For e.g., decoding-based methods like VCD and OPERA excel at reducing language hallucinations, whereas grounding approaches like Woodpecker are more effective for vision hallucinations. Decoding-based techniques benefit from low-confidence hallucinated tokens and uncertainty encoded in logits discussed next. More discussion is in Appendix K.1. (3) Existing methods fail to reduce hallucinations on reasoning benchmarks like MMMU. We attribute this to the algorithmic biases of these methods that are crafted to just reduce object hallucinations.

[linewidth=1pt, linecolor=black, leftmargin=1pt, rightmargin=1pt, innerleftmargin=10pt, innerrightmargin=10pt, innertopmargin=4pt, innerbottommargin=2pt, backgroundcolor=gray!20, roundcorner=5pt]
Key Takeaway:
Current hallucination mitigation techniques mainly improve LVLMs’ visual recognition skills but are less effective in improving other cognitive abilities, such as reasoning. Additionally, these improvements are largely limited to real-world scenes and specific types of hallucinations, leaving areas like non-real-world scenes largely unexplored.

## 4 The Visual Perception Gap: LVLMs can see but not perceive

It is evident that current hallucination mitigation techniques primarily enhance visual recognition but fail to improve other cognitive abilities, such as reasoning or knowledge extraction. In this section, we investigate the underlying cause of this limitation. Our analysis reveals that LVLMs often rely on language priors rather than attending to the input image when generating responses to reasoning prompts. This leads us to identify a critical issue: the visual perception gap. While LVLMs can accurately recognize visual elements and possess the necessary knowledge and reasoning skills to respond factually, they struggle to perceive and interpret these elements in relation to the input prompt. This gap in perception causes hallucinations, resulting in incorrect responses.

### 4.1 Visual Blind Spots: LVLMs Ignore the Input Image for Reasoning

Figure: Figure 6: Base Rank Comparison between AMBER and MATH-Vision datasets as a function of token position in responses (for CogVLM).

With findings from previous sections, we now assess the influence of the image on the LVLM response. Precisely, we compare the Base Rank (defined in Section 3.3) of tokens across datasets. A lower value would signify that the model solely responds using language priors, while a higher value would signify that the model pays attention to the image, i.e., it responds with tokens different from what the model would have responded with only language priors. Fig. 6 plots the Base Rank of tokens against the token position in response (average across the dataset). As we see, the Base Rank for AMBER, which prompts for an image description, is higher than Math-Vision, which requires it to reason or extract knowledge based on the image. This hints towards an alignment issue: different from AMBER, for Math-Vision LVLMs are unable to perceive the image before responding, under-relying on visual cues and over-relying on language priors for generating responses. In Appendix K.1 we explain why decoding-based mitigation techniques discussed in Section 3.3 do not work for reaosning in cognitive prompts.

### 4.2 Disentangling Visual Recognition from other Cognitive Skills

A key objective of alignment fine-tuning is to equip LVLMs with implicit visual perception skills. As outlined in Section 3, LVLMs should interpret images in context and respond accurately to information-seeking or reasoning prompts. To determine if hallucinations arise from the base LLM’s limitations or alignment gaps, we aim to isolate visual recognition from other cognitive skills and evaluate them separately. We use GPT-4 to rephrase prompts from visual information-seeking and reasoning benchmarks, modifying them so they can be answered without the image (examples in Appendix K.7 and prompt used in Appendix G).

Figure: Figure 7: (Left) Performance comparison of different LVLMs when prompted w/ original prompt vs rephrased prompts w/o image (-t). (Right) Performance comparison of different LVLMs for their ability to generate a faithful image description.

LVLMs are first evaluated on these text-only prompts and then on generating independent image descriptions (evaluation prompt in Appendix G). Figure 7 (left) compares the performance of LVLMs on the text-only prompts (marked with -t) against their original image-text versions, while Figure 7 (right) shows VR scores. Our results indicate that LVLMs excel in visual recognition and perform better on factual responses when using text-only prompts than with combined image-text inputs. This suggests that LVLMs have the necessary knowledge and reasoning skills but struggle to integrate these with visual perception in prompts requiring both. We identify this as a perception gap: LVLMs can recognize visual elements but fail to fully perceive them in a way that connects recognition to internal knowledge, which is essential for reasoning or information extraction. This challenge likely arises due to several factors: (i) training datasets that are largely composed of image-description pairs, (ii) alignment datasets that contain strong visual cues within the natural-language prompts.

### 4.3 Uncertainty in the Logit Space

Table: Table 1: Post-truncation probability statistics using the elbow method. All values are in the format: non-/hallucinated averaged across all tokens in the dataset.

To investigate the origins of hallucinations, we analyze the logit space of hallucinated tokens. Specifically, after applying the softmax operation to the logits, we rank them by their probability scores. We then use the elbow method (detailed in Appendix K.6) to truncate the distribution, obtaining the top-k tokens along with their corresponding probabilities. This truncation approach, based on the elbow method, effectively identifies the tokens most likely to be selected during multinomial sampling. Table 1 shows several statistics of the top-k probabilities for hallucinated tokens (we only consider the first token of each word). As evident from the value of k, standard deviation, and the average, the top-k probability space for hallucinated tokens is dominated by a set of low and equally confident tokens. Thus, each of these tokens has almost a similar chance of being sampled with multi-nomial sampling.

[linewidth=1pt, linecolor=black, leftmargin=1pt, rightmargin=1pt, innerleftmargin=10pt, innerrightmargin=10pt, innertopmargin=4pt, innerbottommargin=2pt, backgroundcolor=gray!20, roundcorner=5pt]
Key Takeaway: LVLMs generally demonstrate accurate recognition and cognition skills to respond and reason accurately. However, they struggle to effectively link they recognize to their internal knowledge during the reasoning process, which results in inaccurate responses. This issue underlines what we identify as the visual perception gap.

## 5 Visual Description Grounding Decoding (VDGD)

Figure: Figure 8: Illustration of our proposed VDGD method.

Main Motivation. To bridge the visual perception gap and reduce hallucinations for cognitive prompts, we propose Visual Description Grounding Decoding (VDGD), a simple, effective, and training-free hallucination mitigation technique. During decoding, VDGD first appends the image description (generated by the LVLM itself) as a prefix to the prompt’s text instruction, then grounds token generation by selecting the token that deviates the least from the description. Token deviation is measured using a novel formulation based on Kullback–Leibler divergence of the token with the description generated by the LVLM itself. Much like humans, who often write down their key observations in an image and refer to them while solving complex reasoning tasks, we hypothesize that grounding the LVLM’s responses in a detailed visual description can significantly enhance its reasoning process. Additionally, re-scoring the equally and low-confidence tokens (discussed in Section 1) based on deviation from the description can increase the confidence of the accurate token from several low-confidence tokens (Xu et al., ).

### 5.1 Methodology

Fig. 8 illustrates the VDGD methodology. For an input prompt, we first employ the LVLM to generate a description of the image accompanying the prompt (see Appendix G for the prompt). Next, we concatenate the generated description as a prefix to the original prompt.

Formally put, let the VLM, denoted as $p_{VLM}$, receive an input prompt of length $n$ symbolized as $X_{\text{pre}}=x_{1}\dots x_{n}$, with each $x_{i}$ representing a token from the vocabulary $\mathcal{V}$. The decoder’s objective is to produce responses of length $m$, represented as $X_{\text{res}}=x_{n+1},\dots,x_{n+m}$. During the decoding phase, the model iteratively decodes one token at a time, conditioning on the tokens that have been generated so far:

$$ $p_{VLM}(X_{\text{resp}}\mid X_{\text{pre}})=\prod_{i=n+1}^{n+m}p_{VLM}(x_{i} \mid x_{<i})$ (1) $$

Here, $p_{VLM}(x_{i}\mid x_{<i})$ denotes the probability distribution for the next token, $x_{i}$, given the previous tokens. Inspired by our findings in Section 4.3, we first apply a plausibility constraint to truncate the vocabulary space of the next token $x_{i}$ (Li et al., ). Essentially, this step cuts off tokens from the vocabulary $\mathcal{V}$ below the elbow and keeps only the top-k plausible tokens. This is done as follows:

$$ $p_{\text{VLM}}\left(x_{i}\mid x_{<i}\right)\begin{cases}p_{\text{VLM}}\left(x_ {i}\mid x_{<i}\right),&\text{ if }p_{\text{VLM}}\left(x_{i}\mid x_{<i}\right) \geq\alpha\max_{w}p_{\operatorname{VLM}}\left(w\mid x_{<i}\right)\\ -\infty,&\text{ Otherwise }\end{cases}$ (2) $$

where $\alpha$ is a hyper-parameter between [0,1]. Now let $\mathcal{V}_{K}^{x_{i}}$ be the set of top-K most plausible tokens from the vocabulary $\mathcal{V}$ after the truncation of $x_{i}$. Next, for each token $w_{k}\in\mathcal{V}_{K}^{x_{i}}$ in the top-K plausible tokens, we calculate the deviation of the token with the $n$ tokens of the prompt as follows:

$$ $\mathrm{KL}_{w_{k}}^{x_{i}}=\min_{1\leq j\leq n}\mathrm{KL}\left(\operatorname {one-hot}(w_{k})\|p_{\operatorname{VLM}}\left(\cdot\mid x_{<j}\right)\right).$ (3) $$

where $\mathrm{KL}$ is the Kullback–Leibler divergence and $\operatorname{one-hot}(w_{i})\in\mathrm{R}^{\mathcal{V}}$ is the one-hot representation of token $w_{k}$ where each value is 0 except the position corresponding to the token-id of $w$, which is 1. Next, we replace the value of $w_{k}$ in the logit with $-\mathrm{KL}_{w_{k}}^{x_{i}}$ and apply softmax on the logits to obtain a probability distribution. Thus, tokens with larger KLD (or a large deviation) to the input prompt are less likely to be sampled given the softmax operation upon KL divergence. With this distribution, we can now apply any sampling technique to sample the next token.

Table: Table 2: Performance comparison of VDGD with various baselines. VDGD outperforms by 2%-33%.

### 5.2 Experimental Setup

Datasets and Evaluation Metrics. For evaluation, we employ a variety of standard benchmarks focused on reasoning and information-seeking tasks. These include LLaVA-Bench, MM-Vet (Yu et al., ), MMBench (Liu et al., ), MME (Fu et al., ), MathVista (test-mini subset), MathVision, and MMMU (validation set). For each benchmark, we employ our proposed GPT evaluation technique described in Section 3. Beyond being more robust to the diverse formats of model generation, we also find GPT scores to have a higher correlation to expert-human evaluation ($\approx$0.97; Section D has more details with original benchmark evaluation metrics that has a correlation of $\approx$0.92). We are inspired by a wealth of prior studies in hallucination evaluation (Sicong Leng et al., ; Yu et al., ; Liu et al., ). All results are averaged across 3 runs.

Baselines. We compare VDGD against VCD, OPERA, Woodpecker, LRV, LURE, HALC (Chen et al., ) and PAI (Liu et al., ) hallucination mitigation techniques. Additionally, we compare two vanilla decoding methods (without any additional mitigation techniques): Vanilla-greedy with vanilla-greedy decoding and Vanilla-sampling with multinomial sampling-based decoding (top-p=0.5 and temperature=0.7). We employ greedy decoding for all methods as we find no difference in performance on sampling. We generate image descriptions for VDGD using the VCD mitigation technique. For LVLMs, we employ LLaVA-v1, LLaVA-v1.5, LLaVA-v1.6, mPLUG-Owl2, InternLM-X, CogVLM, Qwen2-VL (Wang et al., ) and CogVLM2 (Hong et al., ).

Proposed VaLLu Benchmark. Automatic evaluation of LVLMs, with powerful closed-source models, on multiple large benchmarks can be prohibitively expensive. Thus, we propose VaLLu, a benchmark that we build by amalgamating existing benchmarks.

Figure: Figure 9: Distribution of task types in VaLLu.

VaLLu comprises a total of 1500 instances, with 554 sourced from Oven, 535 from MMMU, 218 from MMC, 74 from MathVista, 60 from HallusionBench, 53 from MATH-Vision, and 14 from MME. We only include instances with instructions that require open-ended generations and do not include any binary Yes/No or Multi-choice questions. We control the size to ensure the evaluation is affordable while keeping the diversity of tasks and topics for comprehensive analysis. VaLLu is manually filtered for noisy examples commonly found in existing benchmarks (we discuss this further in Appendix K.3), annotated with additional meta-data (illustrated in Fig. 9) and paired with an expert-provided response.

### 5.3 Results and Analysis

Quantitative results. Table 9 compares the performance of VDGD with other baselines. As we clearly see, VDGD not only outperforms all baselines but is also the only method that alleviates and improves performance on VaLLu. Results on other full benchmarks are provided in Appendix L.6, together with scores for other aspects beyond factuality, where we show that VDGD does not compromise the depth and expressivity of responses. Ablation results in Appendix C demonstrate that removing KL divergence leads to drop in performance. Additionally, using only the description prefix without the image leads to performance drops, confirming that VDGD enhances reasoning through grounding rather than making the model rely completely on the description alone.

Qualitative examples. Fig 10 compares responses from vanilla greedy decoding with VDGD. Grounding response generation results in more factual responses. Appendix B illustrates 2 failure cases of VDGD where incorrect descriptions can lead VDGD to hallucinate.

Figure: Figure 10: Illustration of responses generated with VDGD compared with greedy decoding. More in Sec. K.2.

## 6 Conclusion, Limitations and Future Work

In this paper, we conduct extensive experiments to analyze hallucinations in LVLMs. Our findings reveal that although there has been considerable progress in reducing hallucinations related to visual recognition, reducing hallucinations for cognitive prompts has been limited. Further investigation into LVLM responses to cognitive prompts reveals a visual perception gap: despite having the required capabilities, LVLMs often produce hallucinations when responding to prompts. To address this issue, we introduce VDGD, a novel and training-free method to reduce hallucinations. Our results demonstrate that VDGD is the first method to effectively reduce hallucinations in responses that necessitate additional cognitive skills beyond just visual recognition.

As part of future work, we would like to work on the current limitations of our work, which include (1) Error accumulation in VDGD that comes from inaccurate image descriptions in the prefix, (2) the requirement of prompting the LVLM twice, (3) Analysis of our findings with better alignment techniques and methods like RAG.

## 7 Reproducibility Statement

We provide our code here: https://anonymous.4open.science/r/VDGD-1E04/. All codes will be open-sourced upon paper acceptance, including all checkpoints. All experimental details, including training parameters and hyper-parameters, are provided in Section 5.3. The Appendix has more comprehensive details about our experimental setup.

## Appendix A Appendix

In the supplemental material,

- 1.
B: We illustrate failure cases for VDGD.
- 2.
C: We provide results for ablation of VDGDs key components.
- 3.
D: We provide correlation between GPT scores and actual metrics.
- 4.
E: We provide a detailed explanation of computational complexity of VDGD algorithm described in section 5.
- 5.
F: We describe the impact of the paper on society and research communities.
- 6.
G: We provide the prompts used for object detection, description, rephrasing, noise removal and evaluations required for our experiments.
- 7.
H: We provide a detailed description of all the datasets utilized in this paper.
- 8.
I: We provide a detailed description of all the models we have experimented with.
- 9.
J: We provide a detailed description of all the prior-art hallucination mitigation techniques used as baselines.
- 10.
K: We provide details about:
•
K.1: We provide detailed explanation about Logit Space for VR and Cognitive Prompts.
•
K.2: We provide examples comparing generations from Greedy, Decoding, VCD and VDGD.
•
K.3: We provide examples of noisy samples in datasets.
•
K.4: We provide and in-depth explanation of all the details of Algorithm 1.
•
K.6: We explain the elbow method used for logit analysis.
•
K.7: We provide examples of rephrased prompts which do not require image input.
•
K.8: We provide details of computational resources used for this paper.
- 11.
L: We provide additional results for all the experiments described in the paper.

## Appendix B VDGD Failure Cases

Fig. 11 illustrates 2 failure cases of VDGD. We show that VDGD can lead to incorrect responses due to an incorrectly generated image description. However, we also argue that as LVLMs get better at describing images, the performance of VDGD decoding will improve.

Figure: Figure 11: Illustration of failure cases of VDGD.

## Appendix C VDGD Ablation Results

To demonstrate the effectiveness of VDGD, we conduct ablation studies on its key components: (i) VDGD (- VCD): Image descriptions are generated without using Visual Contrastive Decoding (VCD), (ii) VDGD (- KLD): KLD-based sampling is removed, and only the description is appended, (iii) VDGD (- Image): VDGD is applied without the original image input; only the image description is appended to the prompt. Results are presented on the VDGD benchmark in Table 3. Removing KL Divergence leads to a substantial performance drop. We notice a similar phenomenon by removing the input image with the steepest drop. This shows that the input image is still essential for the model.

Table: Table 3: Ablation results for VDGD. VDGD sees a drop in performance when the KLD decoding objective is removed and a major drop when no input image is provided.

## Appendix D Correlation between GPT scores and actual metrics

Background of Experts in Human Evaluation. For the expert human evaluation of model generations, we engaged a group of 5 PhD students with at least four years of research experience in computer vision. These experts were tasked with evaluating the outputs of the models for accuracy, relevance, and factual correctness. To ensure thorough evaluation, they were provided with internet access to verify factual information when necessary. The evaluation process was conducted in an anonymized format to maintain objectivity, with the experts working independently and without prior knowledge of the model specifics. We selected 500 samples or the size of the benchmark, whichever was lower.

## Appendix E VDGD Computational Complexity

Given an LVLM with a vocabulary size $V$, an input instruction $I$ with a description of length $M$ tokens, at each decoding step, the complexity for equation (2) is $O(V)$ and let the number of plausible tokens after this step be $K$, the complexity for equation (3) is $O(VMK)$. The total computational complexity of VDGD at each decoding step is $O(V)+O(VMK)\approx O(VMK)$.

## Appendix F Broader Impact

This paper addresses the broader impact of reducing hallucinations in Large Vision-Language Models (LVLMs), which are increasingly utilized in diverse applications such as automated content generation, assistive technologies, and decision-making systems. By focusing on the phenomenon of hallucination, particularly in responses to cognitive prompts, our work has the potential to enhance the reliability and safety of LVLMs in practical scenarios. The proposed Visual Description Grounded Decoding (VDGD) method, being training-free, offers an accessible and scalable solution that can be readily implemented across various models to mitigate risks associated with inaccurate or misleading outputs. Furthermore, the introduction of the VaLLu benchmark contributes to the field by providing a tool for the comprehensive evaluation of cognitive capabilities in LVLMs, promoting transparency and fostering further research in the area. Collectively, these contributions aim to improve user trust in AI systems and encourage responsible AI development that prioritizes factual accuracy and understanding.

## Appendix G Prompts

For image description benchmarks, we employ the prompt illustrated in Fig. 12. For non-image description benchmarks, we modify this prompt slightly (we modify the part of the prompt for extracting hallucinated phrases), and we illustrate the same in Fig. 16. The prompt used to generate image descriptions for VDGD is illustrated in Fig. 15. Correctness in the prompts refers to Factuality in all tables in Section L. We name it so after an ablation study on the terminology that works better for prompting the judge (GPT-4). All prompt evaluations on GPT-4 are done with a temperature of 0.7.

Figure: Figure 12: Prompt used to evaluate image description benchmarks using GPT-4-Turbo as described in Section 3.2.

Figure: Figure 13: Prompt used to evaluate the performance of LLMs on rephrased instructions using GPT-4-Turbo as described in Section 4.2.

Figure: Figure 14: Prompt used to identify the visual elements in images using Llama3 for the algorithm described in Section 3.3.

Figure: Figure 15: Prompt used to describe images using LVLMs for VDGD as described in Section 5.

Figure: Figure 16: Prompt used to evaluate generations from cognitive benchmarks in Table 9.

Figure: Figure 17: GPT evaluation prompt for evaluating informativeness of image descriptions generated by LVLMs.

Figure: Figure 18: Prompt used to evaluate generations for rephrased instructions on benchmarks like VaLLu, MMMU, MathVista, and SynthDoG using GPT-4-Turbo as described in Section 4.2.

## Appendix H Dataset Details

VaLLu We propose VaLLu benchmark which is sourced from Oven, MMMU, MMC, MathVista, HallusionBench, MATH-Vision and MME. This dataset is licensed under all the licenses of the original benchmarks that it was sourced from.

AMBER AMBER (Wang et al., ) is a benchmark for evaluating hallucinations in both the generative task and discriminative task of MLLMs. It provides comprehensive coverage of evaluations for various types of hallucination, including existence, attribute, and relation. Licensed under Apache-2.0 License.

MMMU MMMU (Yue et al., ) benchmark evaluates multimodal models on multi-discipline tasks and demands college-level subject knowledge and deliberate reasoning. It consists of 11.5k questions, spanning 30 subjects and 183 subfields, focusing on advanced perception and reasoning on domain-specific knowledge. Licensed under Apache-2.0 License.

MMC MMC (Liu et al., ) is a human-annotated benchmark comprising 9 distinct tasks to evaluate LVLM’s reasoning capabilities to comprehend visual charts. It comprises tasks such as chart information extraction, chart reasoning, contextual chart understanding, etc. It also offers two evaluation methods: GPT4 free-format general ability evaluation and multiple-choice QA format chart understanding. Licensed for research purposes.

MathVista MathVista (Lu et al., ) is a robust mathematical reasoning evaluation benchmark that consists of challenging tasks that require fine-grained, deep visual recognition and compositional reasoning. It consists of 6141 examples derived from 31 multimodal datasets involving mathematics. Licensed under CC-BY-SA-4.0.

MATH-Vision MATH-Vision (Wang et al., ) dataset is a collection of 3,040 high-quality mathematical problems with visual contexts sourced from real math competitions. The dataset spans 16 distinct mathematical disciplines and is graded across 5 levels of difficulty to evaluate the mathematical reasoning abilities of Multimodal Large Language Models(MLLM). Licensed under MIT License.

HallusionBench HallusionBench (Guan et al., ) is a comprehensive benchmark designed for the evaluation of image-context reasoning. The benchmark presents significant challenges to advanced large visual-language models (LVLMs), such as GPT-4V(ision), Gemini Pro Vision, Claude 3, and LLaVA- 1.5, by emphasizing nuanced understanding and interpretation of visual data. The benchmark comprises 346 images paired with 1129 questions, all meticulously crafted by human experts. Licensed under BSD-3-Clause.

LlavaBench LlavaBench (Liu et al., ) is a challenging evaluation suite designed to assess visual-language alignment and instruction-following capabilities of large visual-language models. It consists of two benchmarks: LLaVA-Bench (COCO), which includes 30 randomly selected images from COCO-Val-2014 paired with 90 questions covering conversational, descriptive, and complex reasoning tasks, and LLaVA-Bench (In-the-Wild), featuring 24 diverse images from various domains, including memes, paintings, and sketches, with 60 questions.

MM-Vet MM-Vet (Yu et al., ) is an evaluation benchmark designed to assess large multimodal models on complex multimodal tasks. It defines six core vision-language capabilities—Recognition, Knowledge, OCR, Spatial Awareness, Language Generation, and Math—and examines their integration across 16 emergent tasks. MM-Vet employs an LLM-based evaluator for open-ended outputs, providing unified scoring across diverse question types and answer styles. The benchmark offers insights into the capabilities of different large multimodal model paradigms and models beyond simple performance ranking.

MME Multimodal large language model Evaluation Benchmark (MME) (Fu et al., ) measures perception and cognition abilities on a total of 2 tasks(Perception and Congition) and 14 subtasks(Existence, Count, Position, Color, Poster, Celebrity, Scene, Landmark, Artwork, OCR, Commonsense Reasoning, Numerical Calculation, Text Translation and Code Reasoning). To prevent data leakage from use of public datasets for evaluation, the annotations of instruction-answer pairs are all manually designed.

Oven Open-domain Visual Entity Recognition (OVEN) (Hu et al., ) is a task where a model needs to link an image onto a Wikipedia entity with respect to a text query. OVEN challenges models to select among six million possible Wikipedia entities, making it a general visual recognition benchmark with the largest number of labels. Licensed under MIT License.

SynthDoG SynthDoG (Kim et al., ) is created by generating synthetic data which consists of various components: background, document, text, and layout. The background image is sampled from ImageNet, and the document is sampled from the collected paper photos. The text is sampled from Wikipedia. Layout is constructed using randomly stacks grids. The english variant of the dataset consists of 65.5k training and 500 validation entries. Licensed under MIT License.

## Appendix I Model Details

LLaVa-v1. LLaVa-v1 (Liu et al., )(^3^33[https://huggingface.co/liuhaotian/llava-llama-2-7b-chat-lightning-lora-preview](https://huggingface.co/liuhaotian/llava-llama-2-7b-chat-lightning-lora-preview)) is an open-source LVLM trained by fine-tuning LLaMA/Vicuna (13B) on GPT-generated multimodal instruction-following data. It is initially trained on 558K filtered image-text pairs from LAION/CC/SBU and then, finetuned on 80K GPT-generated multimodal instruction-following data. Licensed under CC BY-SA 4.0 DEED.

LLaVa-1.5. LLaVa-1.5 (Liu et al., )(^4^44[https://huggingface.co/liuhaotian/llava-v1.5-7b](https://huggingface.co/liuhaotian/llava-v1.5-7b)) builds on LLaVA-v1’s architecture by replacing linear projection design with a two-layer MLP. It also includes additional academic-task-oriented VQA datasets for VQA, OCR, and region-level perception, to enhance the model’s multimodal capabilities. Licensed under CC BY-SA 4.0 DEED.

LLaVa-1.6. LLaVa-1.6 (Liu et al., )(^5^55[https://huggingface.co/liuhaotian/llava-v1.6-vicuna-7b](https://huggingface.co/liuhaotian/llava-v1.6-vicuna-7b)) re-uses the pretrained connector of LLaVA-1.5 and is instruction tuned on 158K GPT-generated multimodal data, 500K academic-task-oriented VQA data, 50K GPT-4V data and 40K ShareGPT data. LLaVa-1.6 has improved reasoning, OCR, and world knowledge. Licensed under CC BY-SA 4.0 DEED.

mPLUG-Owl2. mPLUG-Owl2 (Ye et al., )(^6^66[https://huggingface.co/MAGAer13/mplug-owl2-llama2-7b](https://huggingface.co/MAGAer13/mplug-owl2-llama2-7b)) is the first Multi-modal Large Language Model that performs well in both pure-text and multi-modal scenarios. mPLUG-Owl2 is pretrained 400 million image-text pairs from: Conceptual Captions (CC3M/CC12M), COCO, Laio-en, COYO, DataComp and for instruction tuning 5 types of data are used: image captioning, mage question answering, region-aware QA, multi-modal instruct data and text-only instruct. Licensed under MIT License.

InternLM-X. InternLM-X (Zhang et al., )(^7^77[https://huggingface.co/internlm/internlm-xcomposer2-vl-7b](https://huggingface.co/internlm/internlm-xcomposer2-vl-7b)) is a vision-language large model based on InternLM2 (Cai et al., ) for advanced text-image comprehension and composition. InternLM-X is pre-trained on 1.1 billion images alongside 77.7 billion text tokens, including both public datasets and in-house concept data followed by supervised fine-tuning using multi-task training and instruction tuning. Licensed under Apache-2.0 License.

CogVLM. CogVLM (Wang et al., )(^8^88[https://huggingface.co/THUDM/cogvlm-chat-hf](https://huggingface.co/THUDM/cogvlm-chat-hf)) is a powerful open-source visual language model. CogVLM-17B has 10 billion vision parameters and 7 billion language parameters. CogVLM is pre-trained using 1.5B text-image pairs and instruction tuned on various visual question-answering datasets. CogVLM achieves state-of-the-art performance on 10 classic cross-modal benchmarks. Licensed under Apache-2.0 License.

Llama-2-7b. Llama-2-7b (Touvron et al., )(^9^99[https://huggingface.co/meta-llama/Llama-2-7b-chat-hf](https://huggingface.co/meta-llama/Llama-2-7b-chat-hf)) is an open source fine-tuned generative text model with 7 billion parameters. It is an auto-regressive language model that uses an optimized transformer architecture. The fine-tuned version use supervised fine-tuning (SFT) and reinforcement learning with human feedback (RLHF) for helpfulness and safety. Licensed under Llama 2 community license agreement.

Vicuna-7b-v1.5. Vicuna-7b-v1.5 (Zheng et al., )(^10^1010[https://huggingface.co/lmsys/vicuna-7b-v1.5](https://huggingface.co/lmsys/vicuna-7b-v1.5)) is a 7b parameter open-sourced model and fine-tuned from Llama 2 with supervised instruction fine-tuning and is an auto-regressive language model based on the transformer architecture. The training data is around 125K conversations from ShareGPT. Licensed under Apache-2.0 License.

InternLM2-7B. InternLM-7B (Cai et al., )(^11^1111[https://huggingface.co/internlm/internlm2-chat-7b](https://huggingface.co/internlm/internlm2-chat-7b)) InternLM2 is a open-source 7 billion parameter chat model with a 200k context window and achieves good performance in the“Needle-in-a-Haystack” test(on long-context tasks like LongBench and L-Eval) and is trained using Conditional Online RLHF (COOL RLHF). Licensed under Apache-2.0 License.

CogVLM2. CogVLM2 (Hong et al., )(^12^1212[https://huggingface.co/THUDM/cogvlm2-llama3-chat-19B](https://huggingface.co/THUDM/cogvlm2-llama3-chat-19B)) CogVLM2 is a new generation of visual language models based on Meta-LLaMA-3-8B-Instruct, offering significant improvements over previous versions. It supports 8K content length, image resolutions up to 1344x1344, and excels in both English and Chinese. CogVLM2 has achieved state-of-the-art performance on benchmarks like TextVQA, DocVQA, and MM-Vet. The open-source models, including CogVLM2 and CogVLM2-Video, are available on GitHub, contributing to advancements in image and video understanding. Licensed under Apache-2.0 License.

Qwen2-VL. Qwen2-VL (Wang et al., )(^13^1313[https://huggingface.co/Qwen/Qwen2-VL-7B-Instruct](https://huggingface.co/Qwen/Qwen2-VL-7B-Instruct)) Qwen2-VL is the latest iteration of the Qwen-VL model, featuring state-of-the-art performance in image and video understanding tasks. It excels in processing images of various resolutions and aspect ratios and supports long-form video understanding for tasks like question answering and content creation. Key architectural updates include Naive Dynamic Resolution and Multimodal Rotary Position Embedding (M-ROPE), improving its multimodal processing capabilities. Available in 2, 7, and 72 billion parameter variants. Licensed under Apache-2.0 License.

## Appendix J Baseline Details

VCD. Visual Contrastive Decoding (VCD) (Sicong Leng et al., ) is a training-free method that contrasts output distributions of original/unaltered inputs and distorted visual inputs. VCD hypothesizes statistical bias and unimodal priors as the two essential causes of object hallucinations and reduces the dependence on them. This results in the generated content being closely grounded to visual inputs resulting more accurate outputs. Licensed under Apache-2.0 License.

OPERA. Over-trust Penalty and a Retrospection-Allocation (OPERA) (Huang et al., ) is a training-free approach that hypothesizes that hallucinations are closely tied to the knowledge aggregation patterns in the self-attention matrix and that generation of new tokens by models is majorly dependent on a few summary tokens(over-trust issue) but not all of the previous tokens. This methodology introduces a penalty term on the model logits during beam-search decoding to mitigate the over-trust issue and implements a rollback strategy to retrospect the presence of summary tokens. Licensed under MIT License.

Woodpecker. Woodpecker (Yin et al., ) is a training-free method that identifies and corrects hallucinations in the generated text. Unlike other training-free methodologies, Woodpecker works after the model generation process. This method works by creating a visual knowledge base of Question-Answer(QA) pairs which contain object-level and attribute-level claims about the input image followed by an LLM correcting hallucinations.

LRV. LRV (Liu et al., ), or Large-scale Robust Visual Instruction tuning, proposes a dataset of visual instructions generated by GPT4, covering 16 vision-and-language tasks with open-ended instructions and answers. Each instance has positive instruction samples and negative instructions for more robust visual instruction tuning. As a result, a model trained on this dataset proves to be robust to hallucinations. Licensed under BSD-3-Clause license.

LURE. LVLM Hallucination Revisor (LURE) (Zhou et al., ) attributes object hallucination to three factors: co-occurrence, uncertainty, and object position, and develops an object hallucination revisor which converts potential hallucinations into accurate outputs. This method first creates a hallucination dataset by making modifications to the original data and trains the object hallucination revisor on this dataset. This method post-hoc rectifies object hallucinations in LVLMs by reconstructing less hallucinatory descriptions.

PAI. Pay Attention to Image (PAI) (Liu et al., ) is a training-free algorithm designed to reduce hallucinations in LVLMs. PAI intervenes during the inference process to make it more image-centric by amplifying attention to image tokens in the self-attention layers of LVLMs. By adjusting attention weights and subtracting logits of text-only inputs from multi-modal inputs, PAI mitigates the issue of “text inertia,” where outputs are overly influenced by context text. This method enhances image representation during generation, leading to more accurate, visually grounded outputs without requiring additional training or external tools.

HALC. HALC (Chen et al., ) is a novel decoding algorithm designed to mitigate object hallucinations (OH) in LVLMs. It integrates a robust auto-focal grounding mechanism to correct hallucinated tokens locally and a specialized beam search algorithm to reduce OH globally while preserving text generation quality. HALC can be easily integrated into any LVLM without additional training, addressing all three types of OH—existence, attribute, and relationship. Extensive experiments show that HALC outperforms state-of-the-art methods across multiple benchmarks.

## Appendix K Additional Details

### K.1 Logit Space for VR and Cognitive Prompts

From our in-depth analysis of the logit space, we see that language hallucinations in LVLMs have a unique characteristic – the probability of the most confident token in the vocabulary marginally decreased from the probability of the same token in the base LLM when prompted with the same prior context (the most confident token by the LVLM will also be the most confident token by the language-only base LLM as the token on a language hallucination). Thus, decoding-based mitigation methods like OPERA and VDD benefit from this unique property, and contrasting tokens help mitigate hallucinations.

### K.2 Qualitative Comparison of Greedy vs VCD vs VDGD Decoding

In this subsection, we illustrate several instances (figures 20, 22, 23, 21, 24) from VaLLu and compare their responses for LLaVa-1.5 with Greedy, VCD, and VDGD Decoding.

Figure: Figure 19: Qualitative example for VaLLu benchmark also showing the descriptions.

Figure: Figure 20: Qualitative example 1 for VaLLu benchmark.

Figure: Figure 21: Qualitative example 2 for VaLLu benchmark.

Figure: Figure 22: Qualitative example 3 for VaLLu benchmark.

Figure: Figure 23: Qualitative example 4 for VaLLu benchmark.

Figure: Figure 24: Qualitative example 5 for VaLLu benchmark.

### K.3 Noisy Examples in Existing Datasets

To maintain a high quality of the benchmark and eliminate noisy examples, we performed a manual review of the existing datasets. This was crucial to ensure that the data used in our paper was reliable. It was also essential for the validity of our research findings. We removed questions that did not have a definite ground truth answer. We also made sure that the questions were answerable using a combination of general knowledge and reasoning. This filtering was done by the paper authors and has been approved by our institution’s Institutional Review Board (IRB). We share a few examples (figures: 25, 26, 27, 28, 29, 30, 31, 32) of noise in the existing datasets below:

Figure: Figure 25: Noisy example 1 for MMMU benchmark.

Figure: Figure 26: Noisy example 2 for MMMU benchmark.

Figure: Figure 27: Noisy example 1 for MMC benchmark.

Figure: Figure 28: Noisy example 2 for MMC benchmark.

Figure: Figure 29: Noisy example 1 for MathVista benchmark.

Figure: Figure 30: Noisy example 2 for MathVista benchmark.

Figure: Figure 31: Noisy example 1 for HallusionBench benchmark.

Figure: Figure 32: Noisy example 1 for HallusionBench benchmark.

### K.4 Additional Details on Algorithm 1

In this Section, we describe our algorithm verbally.

Token Distribution Analysis. We perform the token-distribution analysis proposed by  (Lin et al., ). Specifically, for a given instruction-response-image triplet, the instruction $i$ = {$i_{1}$, $i_{2}$, $\cdots$} and the image $v$ is first input to the aligned (or fine-tuned) model to obtain its response $r$ = {$r_{1}$, $r_{2}$,$\cdots$} via greedy decoding. Next, for each position $t$ in the response, a ‘context’ at this position is defined as to be $x_{t}$ = $i$ + {$r_{1}$, $\cdots$, $r_{t-1}$}. This “context” is then input to the base model (model from which the aligned model was instruction-tuned) to obtain its probability distribution for predicting the next token at position $t$, P${}_{\text{base}}$. We then define Base Rank as follows: Rank in P${}_{\text{base}}$ of the token at $t$ with the maximum P${}_{\text{align}}$ value. With the base rank denoted as $\eta$, the unshifted, marginal and shifted tokens are defined as when ($\eta$ $=$ 1), (1 $<$ $\eta$ $\leq$ 3) and ($\eta$ $>$ 3) respectively.

Obtaining Hallucinated Phrases. Fig. 12 illustrates the prompt used for AMBER evaluation, where an LVLM is prompted for a description of an input image. In addition to scores, our judge, GPT-4, also returns the exact hallucinated phrases by the LVLM. We divide these hallucinated phrases into 3 types: Object Hallucinations – There are hallucinated visual elements which are objects. Relation Hallucinations – Action/Verb Hallucination – Hallucinated visual elements that are not objects but just actions or verbs, e.g., walking, etc. Relation Hallucination – Hallucinated visual elements that define relationships (spatial or other kinds) between objects.

Filtering visual elements in Hallucinated Phrases. We prompt LLaMa-3 with the image description output by the LVLM, to extract the phrases from the image description that are visual elements. This ensures filtering out stop-words and all other words that are not visual elements and thus aids our algorithm in getting a precise count of each category of hallucination.

Causal Categorization of Hallucinations. Next, based on the Base Rank formulation and the exact hallucinated phrases detected by our judge (GPT-4), we propose an algorithm (Algorithm 1) to automatically categorize identified hallucinations into the 4 different categories. Precisely, for every hallucinated phrase returned by the judge, we first check if the word in the phrase is a visual element. We do this using the list of visual elements output by LLaMa-3. Next, for the first token of each word of the hallucinated phrase, we check the Base Rank of the token.

- 1.
If the Base Rank is 0, we attribute the phrase to be a language hallucination. This is as simple as the phrase was forced to be generated by the base LLM itself without looking at the image.
- 2.
Next, if the Base Rank is more than 0 and the hallucination can be found in the top-k retrieved elements from the IT dataset the model was aligned using, we attribute it to be an IT hallucination (Ghosh et al., ). To retrieve top-k image-instruction pairs that are closest to the prompt image, we employ CLIP (Radford et al., ) similarity. Specifically, we calculate the similarity between the image embedding of the image associated with the input prompt and the text embedding of the description/response associated with the IT instances. Next, for every prompt, we retrieve the top-25 IT instances and search the hallucinated phrase returned by the judge (in descriptions/responses) using string matching. We show how varying values of k change the final counts for our hallucination categories in Appendix K.5.
- 3.
Next, if the Base Rank is more than 0, the hallucination is not an IT hallucination and the hallucination is a Relation Hallucination, we attribute it to be a style hallucination. LVLMs fine-tuned on datasets synthesized from stronger closed-source LVLMs generally learn to mimic their style. One such style is adding more details to an identified object, by relating it to other objects or world knowledge. Our in-depth analysis shows that LVLMs, striving for lengthier, more detailed responses to imitate style, hallucinate and generate wrongful relations for identified objects.
Our relation hallucinations returned by the judge precisely capture this, and thus we attribute relation hallucinations with a base rank of more than 0 to be Style hallucinations.
- 4.
Finally, if the Base Rank is more than 0, the hallucination is identified as an Obejct hallucination by the judge, we attribute it to be a Vision Hallucination. These are caused by ineffective recognition of objects in the image (due to low-resolution images, weak encoder, etc.)

### K.5 Results for various values of k for IT Hallucinations

Fig. 33 shows that even for higher and lower values of k, our hypothesis and claims stay unchanged as changing the value of k does not affect the overall frequency distribution of categories.

Figure: Figure 33: Frequency comparison of hallucination categories for top-k=10,25,40

### K.6 Elbow Method for Logit Analysis

The Knee/Elbow Detection Method operates by plotting the cumulative distribution of the sorted probabilities and seeking the point that maximizes the distance from the line connecting the first and the last points of this cumulative plot. Conceptually, this corresponds to finding the point of maximum curvature on the plot, indicative of the most pronounced change in the distribution’s progression. It offers a robust approach by identifying the point in a sorted probability vector where the rate of decrease in probabilities shows a significant bend or ’elbow’. This method is particularly useful when the probabilities are distributed such that the top few are similar and significantly higher than the rest, which then decrease sharply. The method proves to be effective in our experiments with multinomial sampling from a distribution, where it allows us to focus on the probabilities that correspond to the tokens that are most likely to be selected by sampling. This method ensures that the chosen cutoff $k$ dynamically adjusts to the specific characteristics of each probability distribution encountered, aligning with the overarching goal of maintaining sampling relevance and computational efficiency.

### K.7 Examples of Rephrased Prompts

To determine whether LVLM hallucinations stem from the limitations of the base LLM or from alignment issues, we ask GPT-4 to rewrite the instructions in prompts in such a way that the base LLM can answer the question without any image input as described in section 4.2. We share a few examples (figures: 34, 35, 36, 37, 38, 39, 40, 41) of rephrased prompts in the existing datasets below:

Figure: Figure 34: Rephrased prompt example for VaLLu benchmark.

Figure: Figure 35: Rephrased prompt example for VaLLu benchmark.

Figure: Figure 36: Rephrased prompt example for MMMU benchmark.

Figure: Figure 37: Rephrased prompt example for MMMU benchmark.

Figure: Figure 38: Rephrased prompt example for MathVista benchmark.

Figure: Figure 39: Rephrased prompt example for MathVista benchmark.

Figure: Figure 40: Rephrased prompt 1 for SynthDoG benchmark.

Figure: Figure 41: Rephrased prompt 2 for SynthDoG benchmark.

### K.8 Compute Infrastructure

All our analysis, inference and baseline experiments are conducted on a node of 4 NVIDIA RTX A6000 GPUs, with 128GB RAM and 10 CPU cores. The evaluations are conducted using gpt-4-turbo-2024-04-09 model.

## Appendix L Additional Results

### L.1 Comparison of various LVLMs on rephrased prompts

Table 4 provides quantitative values for rephrased prompts without image input for various LVLMs, detailed in section 4.2 of the main paper.

Table: Table 4: Performance comparison of various LVLMs on rephrased prompts without images as described in Section 4.2

### L.2 Comparison of various LVLMs on VaLLu benchmark with no image input.

Table 5 provides comparison of various LVLMs on VaLLU benchmark when prompted with no image but image descriptions are provided.

Table: Table 5: Comparison of LVLMs performance when prompted without an image but image descriptions are provided on VaLLu benchmark. The values correspond to the "Factuality" metric.

### L.3 Comparison of various LVLMs on image description

Table 6 provides quantitative values for faithful image description generations for various LVLMs on multiple benchmarks, detailed in section 4.2 of the main paper.

### L.4 Comparison of LVLMs on different capabilities on the proposed VaLLu benchmark

Table 7 provides a comparison of LVLMs on different capabilities on the proposed VaLLu benchmark.

Table: Table 6: Performance comparison of various LVLMs on popular benchmarks for image description. While AMBER has real-world scenes, others do not. Experiment described in Section 3.2.

Table: Table 7: Comparison of LVLMs on different capabilities on the proposed VaLLu benchmark for the "Factuality" metric.

### L.5 Comparison of post-truncation probability statistics on AMBER and MMMU benchmarks

Table 8 provides post-truncation probabilities for all LVLMs on AMBER and MMMU, this is an extension of the statistic shown in section 4.3 of the main paper.

Table: Table 8: Post-truncation probability statistics for various models using the elbow method. All values are computed across all hallucinated tokens in each dataset.

### L.6 Comparison of VaLLu with Mitigation Baselines

Table 9 provides quantitative values for comparison of VDGD and other baselines on the VaLLu benchmark, detailed in section 5.3 of the main paper.

### L.7 Comparison of MMMU with Mitigation Baselines

Table 10 provides quantitative values for comparison of VDGD and other baselines on the MMMU benchmark, detailed in section 3.2 of the main paper.

### L.8 Comparison of MathVista with Mitigation Baselines

Table 11 provides quantitative values for comparison of VDGD and other baselines on the MathVista benchmark, detailed in section 3.2 of the main paper.

### L.9 Comparison of Oven with Mitigation Baselines

Table 12 provides quantitative values for comparison of VDGD and other baselines on the Oven benchmark, detailed in section 3.2 of the main paper.

Table: Table 9: Performance comparison of various LVLMs on VaLLu benchmark. This is an extension of results shown in Table 9

Table: Table 10: Performance comparison of various LVLMs on MMMU benchmark (only questions tagged with open-ended generation). This is an extension of results shown in Figure 7.

Table: Table 11: Performance comparison of various LVLMs on MathVista benchmark (only questions tagged with open-ended generation). This is an extension of results shown in Figure 7.

Table: Table 12: Performance comparison of various LVLMs on Oven benchmark. This is an extension of results shown in Figure 7.

## Peer Reviews

confidence: 4
contribution: 2
initial_rating: 6
presentation: 3
questions: 1. How can hallucinatory results be mitigated using the proposed VDGD in the newly defined hallucination categories, compared to other decoding strategies? Analyses through section 3 and 4 are really intriguing, but the reviewer belives that there is significant gap to bridge such motivation and findings into the VDGD design.


2. The method of VDGD is limited to merely prefixing self-generated model response and relying on the first generated response that the model predicts (ironically this may include a lot of hallucinatory responses-even if limitation section mentioned this). Considering LLMs are more prone to hallucination snowballing rather than error correction, it is unclear where the performance gains are derived from. Unlike original contrastive decoding, VDGD cannot make logit corrections by counteracting with premature models and relies solely on vocabulary truncation.


3. Computational analyses should be conducted such as throughput or latency. Also, can this VDGD be seamlessly applied to beam search decoding? Then, how will be the result comparison?
soundness: 3
strengths: Through section 3 and 4, in this paper, the authors extensively explored hallucination issues across various benchmarks, models, and decoding strategies. The novel taxonomies beyond simple object hallucination are crucial to understand the current problems in hallucination research areas (particularly LVLM).
summary: In this paper, the authors argue that the current LVLMs and hallucination mitigation decoding strategies lack visual perception capabilities. Through extensive analyses, the authors delve into such deficiency across cognitive and real-world benchmarks and categorize more detailed hallucination taxonomies beyond mere object hallucination: Language Hallucination / Vision Hallucinations / Style Hallucinations / IT Hallucinations. At the end, the authors introduce a new decoding strategy, VDGD, which prefixes the model's detailed descriptive response to the given input and truncates grounding of its intermediate responses to refine the final answers.
weaknesses: Even with the new hallucination categories, and new findings, their approach, VDGD, lacks of analyzing its effectiveness on the new hallucination categories they defined and limited for its computational costs due to successive response generation.

---

confidence: 4
contribution: 3
initial_rating: 5
presentation: 4
questions: Please see weaknesses part.
soundness: 3
strengths: 1. The problem of cognitive reasoning in hallucination is interesting and seems to be under-explored in previous works.
2. The analysis is sufficient, solid, and easy to follow, yielding several interesting insights.
3. The proposed method, although appears to be very simple, is based on the analysis on "language grounding" in previous sections, which has a reasonable motivation of such design.
4. The method demonstrates consistent improvements across benchmarks.
summary: This paper focuses on the cognitive reasoning aspect of hallucination in large vision language models. Through a set of analysis and experiments, it demonstrates that the core blocker of this issue is the difficulty of linking recognized visual concepts to the internal knowledge of LLM. Therefore, the paper further proposes a simple method that per-appendes the image description to the text instruction as the full instruction so that the model can better leverage its reasoning capacity. Evaluation shows that this method can achieve consistent performance improvement on reasoning-focused benchmarks.
weaknesses: 1. This method seems to be limited in science domain, e.g., chart understanding and reasoning. The underlying assumption of the method is that the image can be sufficiently described by texts. It might hold true for science images, e.g., one can easily describe a chart by enumerating all the involved data or simply transforming the chart figure into a table. However, for natural scenes with complex object categories, attributes, and relations, it is almost impossible to fully represent the image with texts. The evaluated benchmarks seems to be focused on such kind of data.
2. Based on my first point, I may suspect that the essential reason of the performance improvement comes from that chart figure is more intuitive for human eyes while text descriptions of data is more suitable for LLM to understand. It may has little relation with **cognitive reasoning**.
3. Also, based on my first point, we'd better not simply regard such science data as reasoning, there can be other forms of reasoning in natural scenes according to some related works [1].
4. The analysis, though informative, takes too much space, and it may have overlap with previous works [2]. For example, categorization of hallucination types in this paper is essentially based on the **cause** of hallucination, which has been discussed in previous works.
5. Moreover, the the experiments and investigation of the proposed method seems to be limited. It is better to involve more ablation studies.
6. The related works is somehow limited. I understand it might be constrained by the space, but it's important to review and discuss related works about hallucination, reasoning, benchmarks, and so on [2] [3].

I will put my initial score as 5 and I hope the authors can resolve my concerns.

[1] Lai et. al. LISA: Reasoning Segmentation via Large Language Model

[2] Bai et. al. Hallucination of Multimodal Large Language Models: A Survey

[3] Liu et. al. A Survey on Hallucination in Large Vision-Language Models

---

confidence: 4
contribution: 2
initial_rating: 3
presentation: 3
questions: 1. In Sections 3.1 and 3.2, you mention that existing hallucination mitigation techniques improve performance on AMBER but not on other datasets. Could you provide quantitative results for this to show this?

2. In Section 3.3, the algorithm indicates that a base rank less than zero signifies a language hallucination. Since ranks are typically non-negative, could you explain how a rank can be less than zero in this context?

3. The algorithm seems to classify instances as information transfer (IT) hallucinations first, which might influence the distribution of hallucination types. How do you ensure that this approach doesn't skew the results, particularly the higher incidence of IT hallucinations compared to visual and style hallucinations?

4. In Section 4.1, you observe a decline in rank as the text prompt lengthens, suggesting reduced attention to image context. Given that longer prompts naturally introduce more textual context, how do you differentiate between the model's reliance on textual versus visual information in this scenario?

5. Your proposed GPT-based evaluation metric shows a correlation of 0.96 with human responses, while existing benchmarks have a correlation of 0.92. Considering this marginal difference, what advantages does your metric offer over traditional evaluation protocols?
soundness: 2
strengths: 1) The paper is well written and nicely presented.
2) The authors present a classification of different types of halluciantions
3) The authors recognize the gap between visual recognition capability of a model and ability to utilize it for cognitive tasks
4) The authors propose a training-free strategy to mitigate hallucinations
summary: This paper addresses the problem of hallucinations in LVLMs. Strictly speaking, the authors provide a way to understand the root cause of such hallucinations in LVLMs. They claim that existing approaches for hallucination mitigation only focus on the visual recognition aspect of the model, and do not dive further into understanding whether the model actually has cognitive skills, thus failing to mitigate hallucinations properly from such models. The authors first conduct a study to investigate the various causes of hallucinations in LVLMs. Then they introduce VDGD, a training-free method to mitigate said hallucinations from an LVLM, and finally propose VaLLu, a benchmark to evaluate cognitive reasoning capabilities of LVLMs.
weaknesses: 1) Sections 3.1 and 3,2 are not that informative about the failure of hallucination techniques on cognitive tasks. The claim in 3.1 that all methods boost performance on AMBER but not on other datasets is weak since even on AMBER, the relative performance increase is small, which is the same for other datasets as well. This goes against the claims of these two sections.

2) In section 3.3, in the algorithm, firstly, currently it says that it is a language hallucination if base rank is less than 0. How is a rank less than zero? I think it should be language hallucination if it does not fall inside the visual elements of the response. Moreover, I think you are using GPT-4 vision, and not just GPT-4 for this? Also, the visual content extraction itself will have hallucinations due to use of llama-3. Further, pushing everything first to IT hallucination definitely skews the outputs of visual and style hallucinations, it can be both. And so the results showing huge IT hallucination compared to the other two is misleading.

3) Experiment on 4.1 showing fall of rank as length of text prompt increases is nice, but it is also bound to happen since the textual context is getting added. This does not definitively prove that no image context is being attended to. Also, the rank difference between the two datasets is just 1. 

4) The gpt-type metric the authors propose is claimed to have high correlation with the human responses. But in appendix we see that the other correlations are also quite high, with a normal benchmarks having a 0.92 correlation compared to author's 0.96. This marginal difference is not significant enough to claim for a new type of evaluation protocol.

---

confidence: 4
contribution: 3
initial_rating: 5
presentation: 3
questions: ### Q1. While this does not affect my score, I believe the terms “perception” and “recognition” should be interchanged in the paper. Perception refers to the basic observation of visuals, while recognition is a more complex process based on what has been perceived. However, the paper appears to use these terms in reverse, which could cause some confusion for readers. 

Ref: https://en.wikipedia.org/wiki/Visual_perception
soundness: 3
strengths: ### S1. The paper is well-structured and comprehensive, providing a smooth overall flow.

### S2. The analysis is in-depth and offers interesting insights.

### S3. The VaLLu benchmark has the potential to serve as a comprehensive benchmark for evaluating hallucinations in models.
summary: The paper presents a comprehensive and extensive analysis on object hallucination. It proposes VDGD, a method that generates descriptions first, which are then used as prompts for a second inference. During decoding, KLD is calculated with the pre-generated descriptions to identify highly deviant candidates. The authors curate several datasets and introduce the VaLLu benchmark for a comprehensive hallucination evaluation.
weaknesses: ### W1. While the idea is simple and effective, a significant drawback is the latency. The proposed method requires generating long descriptions even for short responses (e.g., in Fig. 8, a single token output would typically be very fast in a baseline method, but the proposed method is much slower). Therefore, it is important to include a latency analysis (e.g., average inference time, throughput) compared to simpler decoding-based hallucination mitigation methods [1,2,3] and baseline LVLMs, especially since many recent training-free methods perform a single inference.
[1] Don’t Miss the Forest for the Trees: Attentional Vision Calibration for Large Vision Language Models, Arxiv 2024.  
[2] Seeing is Believing: Mitigating Hallucination in Large Vision Language Models via CLIP-Guided Decoding, Arxiv 2024.  
[3] RITUAL: Random Image Transformations as a Universal Anti-hallucination Lever in LVLMs, Arxiv 2024. 

### W2. Using the description from the first inference as a prompt for the second inference appears to act like a form of Chain-of-Thought (CoT) prompting. The authors should explicitly discuss how VDGD relates to or differs from Visual CoT methods [4,5,6], and compare several aspects (e.g., performance, computational requirements, applicability to different types of tasks). Comparisons or references to recent Visual CoT methods will strengthen the paper.
[4] Compositional Chain-of-Thought Prompting for Large Multimodal Models, CVPR 2024.    
[5] Beyond Embeddings: The Promise of Visual Table in Visual Reasoning, EMNLP 2024.   
[6] Visual Evidence Prompting Mitigates Hallucinations in Multimodal Large Language Models, OpenReview.  

### W3. While each individual analysis is interesting, the overall flow feels somewhat disjointed. The paper presents a categorization of hallucinations and provides extensive explanations for each category, but there are no corresponding experimental results or analysis showing how the proposed method specifically addresses or improves each of these hallucination categories. This lack of connection between the categorization and the method’s effectiveness on each type of hallucination weakens the coherence of the paper. The authors should include a specific analysis or set of experiments demonstrating how VDGD performs on each category of hallucination they've identified. This would help tie together the theoretical framework and the practical application of their method.

### W4. While the in-depth analysis is appreciated, the paper sometimes feels overloaded with content, which can distract from the core focus. At times, it is difficult to follow, and the connection between earlier sections and the methodology feels weak. The dense content also limits the space for method-related experiments, with only one experiment table included in the main paper. Most experiments have been relegated to the appendix, suggesting the need for better content management.

Meta-review: The submission addresses the problem of hallucination in large vision-language models. It identifies that a key contributor to hallucination is the difficulty of associating visual concepts with the internal knowledge of large language models. The authors introduce VDGD, a training-free approach to mitigate hallucination by adding image description to the text instruction for the inputs of VLMs. They also introduce a new benchmark to measure "cognitive reasoning" capabilities of such models. After rebuttal, the submission received three borderline accepts (6) and one accept (8). The AC agrees with the consensus reached by the reviewers, and recommends acceptance of the submission to ICLR 2025. The AC especially appreciates the detailed analysis on the sources of hallucination in VLMs, and encourages the authors to integrate all valuable reviewer feedback in the final version.

Decision: Accept (Poster)