Venue: NAACL 2025

# Mitigating Hallucinations in Multi-modal Large Language Models

via Image Token Attention-Guided Decoding

 Xinhao Xu\({}^{1,2}\) Hui Chen\({}^{2*}\) Mengyao Lyu\({}^{1,2}\) Sicheng Zhao\({}^{2}\)

**Yizhe Xiong\({}^{1,2}\) Zijia Lin\({}^{1}\) Jungong Han\({}^{2,3}\) Guiguang Ding\({}^{1,2}\) \({}^{1}\)School of Software, Tsinghua University, Beijing, China \({}^{2}\)BNRist, Tsinghua University, Beijing, China \({}^{3}\)Department of Automation, Tsinghua University, Beijing, China xxhthu18@gmail.com, jichenhui2012@gmail.com**

###### Abstract

Multi-modal large language models (MLLMs) integrate the inherent text generation capabilities of large language models with an understanding of other modalities, promising wide applications in open-ended tasks. Despite their success, they often generate plausible but incorrect content. This phenomenon, known as hallucination, significantly impacts their practical deployment. In this paper, we delve into the intrinsic characteristics of hallucination from the perspective of interaction between input and output tokens. We find that the hallucination typically occurs with attention reduction of output tokens to image tokens. Based on this observation, we introduce _image Token attention-guided Decoding (iTaD)_, a plug-and-play method which leverages MLLMs' internal representations to mitigate their hallucinations. We first define an _image token attention vector_ to measure the inter-layer differences in attention of output tokens to image tokens across different layers. Based on the vector, we design a novel layer selection strategy and conduct inter-layer contrastive decoding to highlight the progression in image understanding, thereby exploiting attention to image tokens to mitigate hallucinations. Extensive experiments well demonstrate _iTaD_'s effectiveness across different MLLMs and benchmarks.

+
Footnote †: Corresponding author

## 1 Introduction

Multi-modal large language models (MLLMs) Bai et al. (2023); Liu et al. (2023); Zhang et al. (2023); Ye et al. (2023) process inputs from language and other modalities to generate open-ended responses. Recent advancements in MLLMs, such as LLaVA-1.5 Liu et al. (2024), InvinctBLIP Dai et al. (2023), and MiniGPT-4 Zhu et al. (2024), have demonstrated their outstanding performance in a variety of visual tasks, such as object detection Wang et al. (2024); Zhang et al. (2023), image captioning Chen et al. (2018); Rohrbach et al. (2018), visual question answering Goyal et al. (2019), etc. Despite their success, MLLMs usually suffer from a severe hallucination issue Gunjal et al. (2024); Liu et al. (2024); Li et al. (2023), which refers to the phenomenon where MLLMs generate grammatically coherent yet factually incorrect content. Such issue should be carefully managed in scenarios like healthcare Wang et al. (2023), autonomous systems Chen et al. (2024); Yang et al. (2023), and robotics Wang et al. (2024), where the presence of incorrect outcomes is unacceptable and potentially disastrous. Therefore, mitigating hallucinations in MLLMs is crucial for enhancing their practical deployment and reliability in real-world scenarios.

Many efforts have been made to mitigate hallucinations in MLLMs Liu et al. (2024); Huang et al. (2024); Leng et al. (2024); Zhou et al. (2024); Wang et al. (2024); Favero et al. (2024). Early works typically employ additional training datasets or external knowledge bases Wang et al. (2024); Liu et al. (2024); Gunjal et al. (2024). Despite the exploitative depth and effectiveness, they necessitate substantial human and computational costs. Notably, recent studies have leveraged the inherent knowledge or internal representations of LLMs to mitigate their hallucinations during inference Chuang et al. (2024); Shi et al. (2024). Following this trend, some studies attempt to develop training-free MLLM hallucination mitigation methods Leng et al. (2024); Huang et al. (2024); Wang et al. (2024). Those studies mainly focus on applying perturbations to the input text or images from the input side Leng et al. (2024); Wang et al. (2024) or on identifying internal representations of the output tokens associated with hallucinations from the output side Huang et al. (2024). Despite their success, they rarely explore the interaction between output and input tokens. Moreover, as the most critical characteristic distinguishing MLLMs from LLMs, the image

understanding capability expressed in the internal representations of MLLMs is generally neglected in existing methods.

In this paper, we first investigate the interaction between output and input image tokens and their correlation with hallucination in MLLM. A typical MLLM architecture consists of a vision encoder, a vision-language alignment connector, and an LLM. The connector transforms the output embedding from the vision encoder into _image tokens_, aligning them with _text tokens_. Then the aligned _image_ and _text tokens_ are input into the LLM to generate _output tokens_. Intuitively, the attention weights during the LLM's decoding procedure should accurately reflect such interaction between the output tokens and the input image tokens. Therefore, we conduct experiments to explore the relationship between the attention weights of output tokens to input image tokens and hallucinatory outputs. Take the example in Figure 1 for instance, for a description generated for an image, we highlight hallucinatory segments, _i.e._, \(s_{2}\), \(s_{4}\), and \(s_{6}\). We calculate the mean value of the output tokens' attention weights to image tokens in each segment. Figure 1 shows that tokens in hallucinatory segments exhibit significantly lower attention weights to image tokens than those in non-hallucinatory segments. Given that the output tokens' attention weights to all input tokens (including image and text tokens) decrease as the output length increases, this observation is even more pronounced. Specifically, output tokens in hallucinatory segments, _i.e._, \(s_{2}\) and \(s_{4}\), show less attention to image tokens compared to subsequent tokens in non-hallucinatory segments, _i.e._, \(s_{3}\) and \(s_{5}\). Furthermore, we extend our experiments to verify our findings in descriptions generated by different MLLMs for 100 randomly selected images. Table 1 statistically indicates that the attention weights of output tokens in hallucinatory segments to image tokens are significantly lower than those in non-hallucinatory segments. This finding provides a novel insight from the internal attention perspective, revealing that **the hallucinatory segments in MLLMs are correlated with the reduction of attention weights to image tokens**. Therefore, we pose a question: _How can we leverage such correlation and attention differentiation to enhance the mitigation of hallucinations?_

Motivated by the observation, we advocate mitigating the hallucinations of MLLMs by exploiting the attention to image tokens from the model's internal representations. Our method, _image Token attention-guided Decoding (iTaD)_, leverages the inherent layer-level progression of MLLMs to extract and highlight the progression of output tokens' attention to image tokens, engaging the MLLMs with a more powerful capability of image understanding so as to enhance the mitigation of hallucination. Specifically, we construct an _image Token attention Vector (iTaV)_ for each decoding layer and establish a distance metric between two _iTaV_s. The metric is designed to quantify the difference in output tokens' attention to image tokens across different decoding layers. Then _iTaD_ selects the layer whose corresponding _iTaV_ is the most distant from that of the last layer and projects the selected layer's output internal representation into a probability distribution. Based on this layer selection strategy, we conduct inter-layer contrastive decoding, thereby highlighting the progression of attention to image tokens. The experiments show that our simple but effective improvement exhibits better performance

Figure 1: An example illustrating that hallucinatory outputs occur with attention weight reduction to image tokens. The hallucinatory segments are highlighted in bold, _i.e._, \(s_{2}\), \(s_{4}\) and \(s_{6}\). The chart on the right shows the mean value of the attention weights to _all_ input tokens versus to _image_ tokens in each segment.

 

compared to the original output distribution in the MLLM hallucination benchmarks.

In summary, our contributions are as follows:

1. We observe that hallucinations often occur with output tokens' attention reduction to image tokens, and propose an _iTaD_ method, which leverages MLLMs' internal representations and exploits the attention to image tokens to mitigate their hallucinations.
2. To implement _iTaD_, we introduce a novel intermediate layer selection strategy for interlayer contrastive decoding, which utilizes _iTaV_ to measure the inter-layer difference in image understanding and selects the intermediate layer whose _iTaV_ is the most distant from that of the last layer, thereby extracting and highlighting the progression of image understanding in MLLMs.
3. We conduct extensive experiments across different models and hallucination benchmarks. The results show that _iTaD_ achieves state-of-the-art performance, well demonstrating its effectiveness and superiority over baselines.

## 2 Method

In this section, we first briefly introduce inter-layer contrastive decoding, and then detail the proposed method _iTaD_.

### Inter-Layer Contrastive Decoding Process

Given the image-text input, we can obtain the aligned sequence of image and text tokens, denoted as \({\textbf{h}^{0}=\{h_{1}^{0},h_{2}^{0},\ldots,h_{L}^{0}\}}\), where \(L\) represents the length of the input tokens. \({\textbf{h}^{0}}\) can be directly input into the LLM to generate a response. Specifically, the LLM decodes \({\textbf{h}^{0}}\) through successive layers, each consisting of a multi-head attention (MHA) mechanism Vaswani et al. (2017) and a multilayer perceptron (MLP). Assuming the LLM is composed of \(N\) layers, with \({\textbf{h}^{0}}\) as the hidden state input to the first layer, for each position \(t\):

\[h_{t}^{n}=\textrm{MLP}(\textrm{MHA}(h_{t}^{n-1})),\quad n=1,2,\dots,N. \tag{1}\]

Following this pattern, we obtain \(h_{t}^{N}\), which is then projected into a \(|\mathcal{V}|\)-dimensional space through a linear projection layer, _i.e._, \(\texttt{Proj}(\cdot)\). \(\mathcal{V}\) denotes the vocabulary. Finally, the softmax function transforms the projection into a probability distribution for predicting the next token:

\[p_{N}(x_{t+1}|x_{<t+1})=\texttt{softmax}(\texttt{Proj}(h_{t}^{N})), \tag{2}\]

where \(x_{t+1}\in\mathcal{V}\).

Inter-layer contrastive decoding, represented by DoLa Chuang et al. (2024), adopts the _early exit_ mechanism Teerapittayanon et al. (2016), which utilizes the projection function \(\texttt{Proj}(\cdot)\) in Eq. (2) to transform \(h_{t}^{M}\) into a probability distribution:

\[p_{M}(x_{t+1}|x_{<t+1})=\texttt{softmax}(\texttt{Proj}(h_{t}^{M})), \tag{3}\]

where \(x_{t+1}\in\mathcal{V}\) and \(M\in[0,N)\). \(M\) represents the selected intermediate layer. Then it subtracts the distribution \(p_{M}\) from the original model's output distribution, _i.e._, \(p_{N}\) in Eq. (2), on the logit scale, and applies the softmax function to transform it into a probability distribution, as follows:

\[\hat{p}(x_{t+1}|x_{<t+1})=\\ \texttt{softmax}(\mathbb{I}(x_{t+1})\cdot\log\frac{p_{N}(x_{t+1} |x_{<t+1})}{p_{M}(x_{t+1}|x_{<t+1})}), \tag{4}\]

where

\[\mathbb{I}(x_{t+1})=\begin{cases}1&\mbox{if $x_{t+1}\in\mathcal{C}_{ t+1}$},\\ -\infty&\mbox{otherwise},\end{cases} \tag{5}\]

\[\mathcal{C}_{t+1}=\{x_{t+1}\in\mathcal{V}:\\ p_{N}(x_{t+1})\geq\alpha\max_{x_{t+1}^{{}^{\prime}}\in\mathcal{ V}}p_{N}(x_{t+1}^{{}^{\prime}})\}. \tag{6}\]

The constraint function \(\mathcal{C}_{t+1}\) and its penalized indicator function \(\mathbb{I}(x_{t+1})\) are introduced to avoid false positive cases, as illustrated in CD Li et al. (2023d). \(\mathcal{C}_{t+1}\) ensures that the probability of tokens generated are at least \(\alpha\) times the maximum token probability in \(p_{N}\), where \(\alpha\) ranges from 0 to 1. Eq. (4) penalizes tokens that violate \(\mathcal{C}_{t+1}\) and sets their probability to 0 while maintaining the values for all others.

### _iTaV_'s Construction and Distance Measurement

We focus on the attention weights of MHA in Eq. (1) at each step \(t\) in the \(n\)-th decoding layer for the head \(h\):

\[{\textbf{w}_{t,h}^{n}} =[u_{t,h,1}^{n},w_{t,h,2}^{n},...,w_{t,h,l_{t}}^{n}]\] \[=\texttt{softmax}\left(\frac{Q_{t,h}^{n}K_{t,h}^{n}}{\sqrt{d_{k} }}\right),\] (7) 

where \(Q^{n}_{t,h}\), \(K^{n}_{t,h}\), and \(d_{k}\) represent the corresponding query, key matrices, and the dimension of their product, respectively. \(l_{t}\) denotes the length of the sequence \(K^{n}_{t,h}\), _i.e._, the total length of the input and output token sequence. \(\textbf{w}^{n}_{t,h}\) includes attention weights from the query \(Q^{n}_{t,h}\) to each token, ranging from the 1st to the \(l_{t}\)-th position, and meets the condition \(\Sigma^{l_{t}}_{j=1}w^{n}_{t,h,j}=1\). Given that the maximum weight in multi-head attention usually indicates the strong confidence of models Huang et al. (2024), we select it for each step:

\[\hat{v}^{n}_{t,j}=\max_{n=1,2,...,H}w^{n}_{t,h,j},\quad j=1,2,...,l_{t}, \tag{8}\]

where \(H\) indicates the number of attention heads.

Assuming the image tokens span positions from \(I_{s}\) to \(I_{e}\), we select their corresponding attention weights \(\hat{v}^{n}_{t,j}\) and normalize them using the softmax function. Then, for the \(n\)-th decoding layer, the image token attention vector, _i.e., iTaV_, at step \(t\) can be derived as:

\[\textbf{iTav}^{n}_{t}=\texttt{softmax}([\hat{v}^{n}_{t,I_{s}},\hat{v}^{n}_{ t,I_{s}+1},...,\hat{v}^{n}_{t,I_{e}}]). \tag{9}\]

iTaV represents the attention weight distribution to image tokens. Thus, we use it to estimate the difference in image understanding across different layers by measuring the distance between their corresponding _iTaVs_ using Jensen-Shannon Divergence (JSD) Menendez et al. (1997):

\[\text{dist}(\textbf{iTav}^{i}_{t},\textbf{iTav}^{j}_{t})=\text{ JSD}(\textbf{iTav}^{i}_{t}||\textbf{iTav}^{j}_{t}). \tag{10}\]

### Image Token Attention-Guided Decoding

Inter-layer contrastive decoding leverages the incremental improvement of LLMs' internal representations as inputs propagate through layers, and we observe that MLLMs' hallucinatory outputs often occur with attention weight reduction to image tokens. Building on this insight, we propose a novel layer selection strategy for inter-layer contrastive decoding. Our method, _i.e., iTaD_, is designed to extract and highlight the incremental improvements in image understanding between layers, thereby effectively empowering inter-layer contrastive decoding to mitigate hallucinations.

Specifically, we select the target intermediate layer, _i.e._\(M\) in Eq. (3), by maximizing the distance between **iTaV\({}^{N}_{t}\)** and **iTaV\({}^{m}_{t}\)** for each step \(t\):

\[M=\max_{j\in\mathcal{M}}\text{dist}(\textbf{iTav}^{j}_{t},\textbf{iTav}^{N}_{ t}). \tag{11}\]

\(\mathcal{M}\) is a subset of \(\mathcal{N}=\{1,2,...,N-1\}\), and we select \(M\) from \(\mathcal{M}\) for efficiency. We propose that, among the layers in \(\mathcal{M}\), the hidden state of \(M\), denoted as \(h^{M}_{t}\), most effectively highlights the improvement of image attention achieved by \(h^{N}_{t}\). Subsequently, Eq. (4) explicitly extracts such improvement in image understanding in the last layer's hidden state output compared to the hidden states from intermediate layers. The selected layer \(M\) in our _iTaD_ maximizes the emphasis on this improvement, thereby exploiting attention to image tokens to enhance the final probability distribution \(\hat{p}\) and mitigate hallucinations in MLLMs.

Figure 2: A pipeline of our method. We first obtain iTaV for each layer and then select the layer \(M\) whose corresponding iTaV is the most distant from that of the last layer \(N\). Then we derive the final output distribution \(\hat{p}\) based on \(p_{N}\) and \(p_{M}\). As shown in the figure, \(\hat{p}\) predicts the correct token “trees” with the highest probability, whereas \(p_{N}\) assigns the highest probability to the plausible but factually incorrect token “spectators”.

 



## 3 Experimental Setup

### Models and Benchmarks

We select four remarkable MLLMs: LLaVA-1.5 Liu et al. (2024), InstructBLIP Dai et al. (2023), MiniGPT-4 Zhu et al. (2024), and mPLUG-Owl Ye et al. (2023). They comprise a 7B-parameter LLaMA Touvron et al. (2023) or Vicuna Chiang et al. (2023); Zheng et al. (2023). LLaVA-1.5 employs a linear projection as the vision-language alignment connector, while the others adopt Q-former Li et al. (2023). The number of image tokens for the four models is 576, 32, 32, and 65, respectively.

We employ three commonly used hallucination benchmarks, _i.e._, CHAIR Rohrbach et al. (2018), POPE Li et al. (2023), and GPT-4V OpenAI, 2023 assisted evaluation, to comprehensively evaluate the versatility of our method. The benchmarks encompass various task types, including image captioning and visual question answering (VQA), and are designed to evaluate the hallucinations in MLLMs from different perspectives like objects, attributes, positions, etc. The details of these benchmarks can be referred to in the Appendix.

### Implementation Details

Our main experiments are conducted on MSCOCO Lin et al. (2014). For CHAIR and GPT-4V assisted evaluation, we randomly select 500 images from the COCO validation set for evaluation following Huang et al. (2024). Furthermore, we repeat our experiments with different random seeds and report the mean value of the results over 5 runs and 3 runs on the CHAIR and GPT-4V assisted evaluation benchmark, respectively. Prompt details for all benchmarks are in the Appendix.

We compare our method with basic decoding methods: Greedy Search, Nucleus Sampling Holtzman et al. (2020), and Beam Search Sutskever et al. (2014). We set \(p=0.9\) for Nucleus Sampling and the number of beams to 5 for Beam Search. Additionally, we choose DoLa Chuang et al. (2024) and OPERA Huang et al. (2024) as our baselines, which are designed for mitigating hallucinations in LLMs and MLLMs, respectively. Given that OPERA is based on Beam Search, we apply DoLa and our _iTaD_ to Beam Search for fairness. Specifically, following Huang et al. (2024), we adopt the default hyper-parameter settings for OPERA considering its robustness on various settings of hyper-parameters and use \(\{0,2,4,6,8,10,12,14\}\) as indices for candidate pre-mature layers and 32 for the mature layer in DoLa. Besides, we provide a comparison with VCD Leng et al. (2024) in the Appendix.

For simplicity, we directly set \(\mathcal{M}\) to \(\{2,4,6,8,10,12,14\}\) for our _iTaD_ in this paper without additional tuning. Further analysis of \(\mathcal{M}\) is in the Appendix. We select the hyper-parameter \(\alpha\) by evaluating its performance on the CHAIR benchmark, utilizing an independently sampled subset of 500 images from the COCO validation set. The 500 images ensure no overlap with the test sets across all benchmarks. Finally, we set \(\alpha\) to 0.03, 0.05, 0.05, and 0.7 for LLaVA-1.5, InstructBLIP, MiniGPT-4, and mPLUG-Owl, respectively.

## 4 Main Results

### Results on CHAIR

Given CHAIR's sensitivity to sequence length, we set the maximum length of the output tokens to 512 and 64, respectively, for fair comparison. As presented in Table 2, _iTaD_ achieves state-of-the

\begin{table}
\begin{tabular}{l c c c c c c c c c c c c c c c} \hline \multirow{3}{*}{Method} & \multicolumn{4}{c}{LLaVA-1.5} & \multicolumn{4}{c}{InstructbLIP} & \multicolumn{4}{c}{MiniGPT-4} & \multicolumn{4}{c}{mPLUG-Owl} \\ \cline{2-13}  & \multicolumn{3}{c}{\(max\_|\_|=512~{}max\_|=64\)} & \multicolumn{3}{c}{\(max\_|=512~{}max\_|=64\)} & \multicolumn{3}{c}{\(max\_|=512~{}max\_|=64\)} & \multicolumn{3}{c}{\(max\_|=512~{}max\_|=64\)} & \multicolumn{3}{c}{\(max\_|=512~{}max\_|=64\)} \\ \cline{2-13}  & \multicolumn{3}{c}{\(C_{S}\downarrow\)} & \(C_{I}\downarrow\) & \(C_{S}\downarrow\) & \(C_{I}\downarrow\) & \(C_{S}\downarrow\) & \(C_{I}\downarrow\) & \(C_{S}\downarrow\) & \(C_{I}\downarrow\) & \(C_{S}\downarrow\) & \(C_{I}\downarrow\) & \(C_{S}\downarrow\) & \(C_{I}\downarrow\) & \(C_{S}\downarrow\) & \(C_{I}\downarrow\) & \(C_{S}\downarrow\) & \(C_{I}\downarrow\) \\ \hline Greedy & 47.8 & 13.7 & 20.4 & 6.7 & 57.3 & 23.9 & 29.1 & 13.5 & 31.8 & 10.7 & 22.0 & 7.6 & 71.0 & 24.8 & 33.4 & 13.0 \\ Nucleus & 52.2 & 15.9 & 24.9 & 8.5 & 56.2 & 25.6 & 30.2 & 14.8 & 33.6 & 11.2 & 23.4 & 8.5 & 76.6 & 28.1 & 38.1 & 16.0 \\ Beam Search & 50.0 & 14.5 & 19.4 & 6.3 & 56.4 & 16.2 & 23.6 & 8.0 & 33.9 & 10.7 & 23.2 & 7.9 & 75.8 & 25.4 & 31.4 & **12.1** \\ DoLa & 47.5 & 13.7 & 19.3 & 6.4 & 56.4 & 16.1 & 23.8 & 7.8 & 32.5 & 10.1 & 23.4 & 8.0 & 73.0 & 24.7 & 31.8 & 12.7 \\ OPERA & 47.2 & 13.5 & 19.1 & 6.5 & 53.4 & 15.3 & 24.0 & **6.2** & 27.8 & 9.7 & 21.8 & 7.7 & 73.8 & 25.1 & 31.1 & 12.4 \\ _ftaD_ & **45.4** & **13.4** & **19.0** & **6.2** & **53.2** & **14.7** & **22.2** & 7.5 & **26.4** & **9.6** & **20.7** & **7.5** & **70.0** & **24.5** & **29.5** & 12.4 \\ \hline \end{tabular}
\end{table}
Table 2: Results on the CHAIR benchmark. \(max\_|\) is the max output token length. \(C_{S}\) and \(C_{I}\) assess sentence-and image-level hallucinations, with lower values indicating fewer. Best results are bolded and second-best are underlined (same below).



art performance, surpassing baseline methods in nearly all metrics. Besides, it shows consistent effectiveness in generating both long and short responses. The standard deviations of experiments are in the Appendix.

It is observed that the superiority of _iTaD_ is particularly pronounced when the maximum output token length is set to 512. In this setting, _iTaD_ consistently exhibits the lowest \(C_{S}\) and \(C_{I}\) across all four MLLMs. The experimental results demonstrate that _iTaD_ excels specifically in mitigating hallucinations within lengthy sequences.

### Results on POPE

For the POPE benchmark, we report the average F1 scores across _random_, _popular_, and _adversarial_ splits. For mPLUG-Owl, we replicate the baseline methods and present our reproduction results. For all other models, we report the baseline results as presented in OPERA. The model temperature is set to 1 by default, following OPERA.

As shown in Table 3, _iTaD_ consistently outperforms the other methods across all four models, well demonstrating its effectiveness and superiority. It is important to note that POPE typically elicits responses beginning with Yes or No, which generally have limited semantic information. And _iTaD_'s effectiveness primarily lies on the single token Yes or No, which is limited and leads to smaller performance gains. Nonetheless, our _iTaD_ still yields consistent and considerable improvements for POPE across different models. More experiment details on A-OKVQA Schwenk et al. (2022) and GQA Hudson and Manning (2019) can be found in the Appendix.

### Results on GPT-4V Assisted Evaluation

Table 4 presents the results from the GPT-4V assisted evaluation. In this benchmark, we input image descriptions to GPT-4V from 2 different decoding methods each time. Table 4 shows the comparison results between Beam Search/OPERA and _iTaD_, respectively. It is observed that _iTaD_, while maintaining the detailedness of the response, significantly enhances its correctness.

Notably, GPT-4V can evaluate the attribute, location, and relation hallucinations of objects. The superior performance of _iTaD_ on the GPT-4V correctness score demonstrates its capability to mitigate such kinds of hallucinations. Besides, considering that the more detailed responses generated by Beam Search and OPERA exhibit hallucinations and showcase lower correctness scores, _iTaD_'s slight or even negligible decrease in the detailedness score compared to those methods is reasonable and does not affect the overall quality of the output. Given GPT-4V's human-like perception and reasoning capabilities, the results indicate that _iTaD_ can effectively mitigate hallucinations as perceived by humans.

## 5 Analysis

### Ablation Studies on _iTaV_

In this subsection, we investigate the performance of _iTaV_'s variations. Table 5 shows our _iTaV_ and its five variations, which differ in the concatenation of attention weights to specific tokens, _i.e._, the input to the softmax function in Eq. (9). Our proposed _iTaV_ selects the attention weights to only image tokens, _i.e._, (f) in Table 5, while the five variations select attention weights to only input text tokens, all text tokens, only input tokens, only output tokens, and all input and output tokens, respectively.

Table 5 shows that the proposed _iTaV_, _i.e._, (f), achieves the best overall performance. While certain variations may outperform ours on a few metrics, their effectiveness tends to fluctuate unpredictably across different models. In contrast, our

\begin{table}
\begin{tabular}{l c c c c c c c c} \hline \hline Method & \multicolumn{6}{c}{LLaVA-1.5 InstructBLIP MiniGPT-4 nPLUGo-owl} \\  & \(C\uparrow\) & \(D\uparrow\) & \(C\uparrow\) & \(D\uparrow\) & \(C\uparrow\) & \(D\uparrow\) & \(C\uparrow\) & \(D\uparrow\) \\ \hline Beam Search & 5.7 & 5.5 & 4.9 & **4.5** & 5.4 & **5.1** & 4.2 & 4.5 \\ _itnD_ & **6.8** & **5.7** & **5.5** & 4.4 & **6.5** & **5.1** & **5.2** & **4.6** \\ \hline \hline \multirow{2}{*}{Method} & \multicolumn{6}{c}{LLaVA-1.5 InstructBLIP MiniGPT-4 nPLUGo-owl} \\ \cline{2-7}  & \(C\uparrow\) & \(D\uparrow\) & \(C\uparrow\) & \(D\uparrow\) & \(C\uparrow\) & \(D\uparrow\) & \(C\uparrow\) & \(D\uparrow\) \\ \hline OPERA & 5.4 & **5.3** & 5.0 & **4.9** & 5.4 & 5.1 & 4.4 & **4.3** \\ _iTaD_ & **5.9** & **5.3** & **5.7** & **4.9** & **6.7** & **5.6** & **5.0** & 4.1 \\ \hline \hline \end{tabular}
\end{table}
Table 4: Results on GPT-4V assisted evaluation. \(C\) and \(D\) denote concreteness and detailedness; higher is better.

\begin{table}
\begin{tabular}{l c c c c} \hline \hline Method & LLaVA-1.5 InstructBLIP MiniGPT-4 nPLUGo-owl \\ \hline Greedy & 82.2 & 80.0 & 58.5 & 68.5 \\ Nucleus & 82.5 & 80.1 & 57.8 & 70.1 \\ Beam Search & 84.9 & 84.4 & 70.3 & 69.2 \\ DoLa & 83.2 & 83.4 & 72.8 & 68.8 \\ OPERA & 85.4 & 84.8 & 73.3 & 67.6 \\ _iTaD_ & **85.5** & **85.2** & **75.5** & **72.3** \\ \hline \hline \end{tabular}
\end{table}
Table 3: The average F1 scores \(\uparrow\) across _random_, _popular_ and _adversarial_ splits on the POPE benchmark.

 

_iTaV_ consistently delivers superior results across different models. Notably, we observe that selecting attention weights to only output tokens, _i.e._, (d), results in the poorest overall performance among all five variations. It can be attributed to its overemphasis on self-generated tokens during the generation procedure, which has a negligible and even negative effect in mitigating hallucinatory outputs that are unfaithful to the input. Overall, selecting attention weights to image tokens to construct _iTaV_ is crucial for the success of _iTaD_, and the experimental results further validate the rationale behind our definition of _iTaV_.

### Ablation Studies on the Selection of \(M\)

Figure 3 shows the ablation studies on \(M\) by selecting different layers to construct \(\hat{p}\). Following Chuang et al. (2024) and Huang et al. (2024), we use LLaVA-1.5 on 2 distinct validation sets, each with 500 randomly selected images from the COCO validation set. Given the candidate set \(\mathcal{M}\) in Eq. (11), we set \(M\) statically to each layer in \(\mathcal{M}\), respectively. Besides, we test the performance of three dynamical selection strategies for \(M\), _i.e._ (1) randomly selecting from \(\mathcal{M}\), (2) setting \(M\) to \(\min_{j\in\mathcal{M}}\text{dist}(\textbf{iTaV}^{j}_{t};\textbf{iTaV}^{N}_{t})\), and (3) setting \(M\) to \(\max_{j\in\mathcal{M}}\text{dist}(\textbf{iTaV}^{j}_{t};\textbf{iTaV}^{N}_{t})\) (adopted in our method).

Figure 3 shows that the model's performance varies when we set \(M\) statically to different layers in \(\mathcal{M}\). Although some specific layers can yield better performance compared to the dynamic layer \(\max_{j\in\mathcal{M}}\text{dist}(\cdot,\cdot)\) on one validation set, their performance is sensitive to the data distribution. For example, layer 4 and layer 8 achieve superior results on the _val_ set #1 but exhibit limited performance on the _val_ set #2, where layer 6 shows the best performance among all the static layers. On the contrary, our dynamic selection method, represented by \(\max_{j\in\mathcal{M}}\text{dist}(\cdot,\cdot)\), is robust to the data distribution and shows consistently superior performance across different validation sets. Additionally, the performance of setting \(M\) to \(\min_{j\in\mathcal{M}}\text{dist}(\cdot,\cdot)\) performs the worst compared to the other two dynamic strategies. It well demonstrates the reasonableness of our selection strategy in Eq. (11), which highlights the last layer's improvement of image understanding compared to the intermediate layers, thereby significantly mitigating the attention reduction to image tokens.

### Cross-Dataset and Other Benchmark Validation

To validate the generalizability of _iTaD_, we conduct extensive experiments across different benchmarks, _i.e._, MME Fu et al. (2023) and GPT-4 assisted hallucination evaluation Zhao et al. (2023), and different datasets, _i.e._, the official MME dataset and VG-100K Krishna et al. (2017) (for GPT-4 assisted hallucination evaluation). Details on these benchmarks and datasets are in the Appendix.

\begin{table}
\begin{tabular}{l c c c} \hline \hline Model & Beam Search & OPERA & _iTaD_ \\ \hline MiniGPT-4 & 745.9 & 757.4 & **772.3** \\ mPLUG-Owl & 1189.4 & 1175.0 & **1259.7** \\ \hline \hline \end{tabular}
\end{table}
Table 6: Results on the MME benchmark. We report MME scores, with higher scores indicating fewer hallucinations.

Figure 3: Ablation studies on the intermediate layer \(M\) for constructing \(\hat{p}\). We report the average value of \(C_{S}\) and \(C_{I}\) on 2 distinct validation sets. The original LLaVA-1.5 achieves 31.9 on set #1 and 32.2 on set #2, respectively.

 

Table 6 demonstrates the effectiveness of _iTaD_ on the MME benchmark, while OPERA occasionally shows a decrease in performance compared to Beam Search. Figure 4 shows that _iTaD_ generates fewer hallucinatory sentences or words compared to baseline methods. Specifically, it achieves up to 6.4% improvement in HSR on MiniGPT-4 over OPERA, and about 10.7% enhancement in HWR on mPLUG-Owl over DoLa. Although _iTaD_ slightly reduces MLLMs' output sequence length, _i.e._, SPI and WPI, it can be attributed to the reduction of additional hallucinatory content and does not compromise the output quality. The experiments in this section span diverse benchmarks and datasets, demonstrating _iTaD_'s robust and consistent effectiveness across various scenarios. Moreover, these results highlight _iTaD_'s promising potential for a wide range of applications.

### Selection of Hyper-Parameter \(\alpha\)

To investigate the robustness of the selected \(\alpha\) for different data distributions, we repeat experiments on 3 validation sets, as shown in Table 7. Following Huang et al. (2024), each set includes 500 images randomly selected from the COCO validation set.

The selected \(\alpha\) values of 0.05 for MiniGPT-4 and 0.7 for mPLUg-Owl consistently yield superior performance across validation sets. Moreover, the relative effectiveness of hyper-parameters on one set generally indicates their performance on other sets. For example, _iTaD_ performs better on MiniGPT-4 with \(\alpha=0.4\) compared to 0.03 across all sets. Although the optimal \(\alpha\) might vary by model, the results indicate that it can be effectively selected using a small set with 500 images randomly selected from the COCO validation set, as illustrated in the implementation details.

### Latency

In Table 8, we evaluate the impact of _iTaD_ on decoding latency, comparing it to OPERA on NVIDIA A40 GPUs. It is observed that _iTaD_ increases the inference latency by a factor of 1.08 to 1.18, whereas OPERA results in a multiple-fold increase. These findings demonstrate that our method can mitigate hallucinations with only minimal and even a negligible increase in inference latency compared to OPERA. Furthermore, we analyze the latency-performance trade-off in the Appendix.

### Text Quality Analysis

Following HA-DPO Zhao et al. (2023), we calculate 1-gram and 2-gram fluency as repetition metrics, where higher values indicate less repetitive generation. As shown in Table 9, the impact of _iTaD_ on 1-gram and 2-gram fluency varies across models but does not lead to a significant increase in repetition. In most cases, it reduces repetitive gen

Figure 4: Results on GPT-4 assisted evaluation. Lower HSPI, HWPI, HSR, or HWR indicate fewer hallucinations.

\begin{table}
\begin{tabular}{l c c c c c c} \hline \hline \multirow{2}{*}{Dataset} & \multicolumn{3}{c}{MiniGPT-4} \\  & \multicolumn{3}{c}{} & \multirow{2}{*}{Dataset} & \multicolumn{3}{c}{mPLUg-Owl} \\  & 0.03 & & & & 0.6 & 0.7 & 0.8 \\ \hline _val_ set #1 & 27.6 & 25.5 & **24.6** & _val_ set \#1 & 72.7 & **72.6** & 74.0 \\ _val_ set \#2 & 28.0 & 27.4 & **27.2** & _val_ set \#2 & 71.2 & **67.0** & 72.4 \\ _val_ set \#3 & 24.4 & 23.6 & **22.8** & _val_ set \#3 & 70.9 & **70.0** & 71.7 \\ \hline \hline \end{tabular}
\end{table}
Table 7: The robustness of the selected \(\alpha\). We present the \(C_{S}\) results for different \(\alpha\) values on 3 distinct validation sets.

 

eration. Additionally, the CHAIR evaluation script calculates recall values, which we summarize in Table 10. We observe that _iTaD_ generally maintains recall comparable to or even higher than the original Beam Search. These experimental results demonstrate that _iTaD_ can mitigate hallucinations in MLLMs while preserving the quality of the output text. More qualitative and text quality analysis can be referred to in the Appendix.

## 6 Conclusion and Future Work

In this paper, we introduce _iTaD_, a plug-and-play method to mitigate hallucinations in MLLMs. _iTaD_ is motivated by the observation that hallucinatory outputs in MLLMs typically occur with attention reduction to image tokens. To address this, we first define the image token attention vector (_iTaV_) to measure the distance in image understanding across different layers. Then we leverage the inherent layer-level progression of MLLMs to extract and highlight the improvement in image understanding and derive the output token distribution \(\hat{p}\), thus exploiting the attention to image tokens to mitigate hallucinations. Extensive experiments demonstrate that _iTaD_ achieves state-of-the-art results and consistently exhibits superior performance across different models, datasets, and benchmarks. This paper focuses on hallucinations in visual and language modalities, where most research has concentrated. Our future work will explore hallucinations in other modalities, such as video and audio. Given the complexity of their origins, further research is critical for a more thorough understanding of the underlying causes.

### Limitations

The proposed _iTaD_ is predominantly empirical, presenting a solution to mitigate hallucination by employing attention to image tokens. While its effectiveness has been extensively demonstrated, a thorough investigation into the underlying causes of the attention reduction to image tokens has not yet been conducted. We believe that future work should focus on exploring the intrinsic mechanisms behind this attention misalignment during large model pre-training or fine-tuning processes and developing techniques to address this issue, which would be a valuable contribution to the field. Additionally, this work explores attention to all image tokens collectively, without investigating the differences in attention across image regions under varying contexts. Future research could conduct a more detailed investigation into the variations in attention to specific image tokens as influenced by the context, and its relation with hallucinatory output.

## Acknowledgment

This work was supported by National Natural Science Foundation of China (Nos. 62441235, 62271281, 62021002). It was also supported by Tsinghua University - Beijing Jingdong Century Trading Co., Ltd. Joint Research Center for Smart Retail Technology.

## References

* Bai et al. [2023] Jinze Bai, Shuai Bai, Shusheng Yang, Shijie Wang, Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, and Jingren Zhou. 2023. Qwen-vl: A frontier large vision-language model with versatile abilities. _CoRR_, abs/2308.12966.
* Chen et al. [2018] Hui Chen, Guiguang Ding, Zijia Lin, Sicheng Zhao, and Jungong Han. 2018. Show, observe and tell: Attribute-driven attention model for image captioning. In _Proceedings of the Twenty-Seventh International Joint Conference on Artificial Intelligence, IJCAI 2018, July 13-19, 2018, Stockholm, Sweden_, pages 606-612. ijcai.org.
* Chen et al. [2024] Long Chen, Oleg Sinusaki, Jan Hunermann, Alice Karnsund, Andrew James Willmott, Danny Birch, Daniel Maund, and Jamie Shotton. 2024. Driving with llms: Fusing object-level vector modality for explainable autonomous driving. In _IEEE International Conference on Robotics and Automation, ICRA 2024, Yokohama, Japan, May 13-17, 2024_, pages 14093-14100. IEEE.
* Chiang et al. [2023] Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph E. Gonzalez, Ion Stoica, and Eric P. Xing. 2023. Vicuna: an open-source chatbot impressing gpt-4 with 90%* chatgpt quality.
* Chuang et al. [2024] Yung-Sung Chuang, Yujia Xie, Hongyin Luo, Yoon Kim, James R. Glass, and Pengcheng He. 2024. Dola: Decoding by contrasting layers improves factuality in large language models. In _The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024_. OpenReview.net.
* Crandes et al. [2018]

\begin{table}
\begin{tabular}{l c c c c} \hline Method & LLaVA-1.5 & INstructBLIP & MiniGPT-4 & mPLUG-Owl \\ \hline Beam Search & **79.2** & 75.3 & **58.9** & 69.3 \\ OPERA & 78.4 & 74.8 & 57.8 & 72.9 \\ _ iTaD_ & 78.9 & **75.4** & **58.9** & **73.4** \\ \hline \end{tabular}
\end{table}
Table 10: The recall metric \(\uparrow\) on the CHAIR benchmark.

 

Wenliang Dai, Junnan Li, Dongxu Li, Anthony Meng Huat Tiong, Junqi Zhao, Weisheng Wang, Boyang Li, Pascale Fung, and Steven C. H. Hoi 2023. Instructible: Towards general-purpose vision-language models with instruction tuning. In _Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10 - 16, 2023_.
* Favero et al. (2024) Alessandro Favero, Luca Zancato, Matthew Trager, Siddharth Choudhary, Pramuditha Perera, Alessandro Achille, Ashwin Swaminathan, and Stefano Soatto. 2024. Multi-modal hallucination control by visual information grounding. In _IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2024, Seattle, WA, USA, June 16-22, 2024_, pages 14303-14312. IEEE.
* Fu et al. (2023) Chaoyou Fu, Peixian Chen, Yunhang Shen, Yulei Qin, Mengdan Zhang, Xu Lin, Zhenyu Qiu, Wei Lin, Jinrui Yang, Xiauw Zheng, Ke Li, Xing Sun, and Rongrong Ji. 2023. MME: A comprehensive evaluation benchmark for multimodal large language models. _CoRR_, abs/2306.13394.
* Gera et al. (2023) Ariel Gera, Roni Friedman, Ofir Arviv, Chulaka Gunasekara, Benjamin Sznajder, Noam Slonim, and Eyal Shmarch. 2023. The benefits of bad advice: Autocontrastive decoding across model layers. In _Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), ACL 2023, Toronto, Canada, July 9-14, 2023_, pages 10406-10420. Association for Computational Linguistics.
* Goyal et al. (2019) Yash Goyal, Tejas Khot, Aishwarya Agrawal, Douglas Summers-Stay, Dhruv Batra, and Devi Parikh. 2019. Making the V in VQA matter: Elevaring the role of image understanding in visual question answering. _Int. J. Comput. Vis._, 127(4):398-414.
* Gunjal et al. (2024) Anisha Gunjal, Jihan Yin, and Erhan Bas. 2024. Detecting and preventing hallucinations in large vision language models. In _Thirty-Eighth AAAI Conference on Artificial Intelligence, AAAI 2024, Thirty-Sixth Conference on Innovative Applications of Artificial Intelligence, IAAI 2024, Fourteenth Symposium on Educational Advances in Artificial Intelligence, EAAI 2014, February 20-27, 2024, Vancouver, Canada_, pages 18135-18143. AAAI Press.
* Holtzman et al. (2020) Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, and Yejin Choi. 2020. The curious case of neural text degeneration. In _8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020_. OpenReview.net.
* Huang et al. (2024) Qidong Huang, Xiaoyi Dong, Pan Zhang, Bin Wang, Conghui He, Jiaqi Wang, Dahua Lin, Weiming Zhang, and Nenghai Yu. 2024. OPERA: alleviating hallucination in multi-modal large language models via over-trust penalty and retrospection-allocation. In _IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2024, Seattle, WA, USA, June 16-22, 2024_, pages 13418-13427. IEEE.
* Hudson and Manning (2019) Drew A. Hudson and Christopher D. Manning. 2019. GQA: A new dataset for real-world visual reasoning and compositional question answering. In _IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2019, Long Beach, CA, USA, June 16-20, 2019_, pages 6700-6709. Computer Vision Foundation / IEEE.
* Ji et al. (2023a) Ziwei Ji, Nayeon Lee, Rita Frieske, Tiezheng Yu, Dan Su, Yan Xu, Etskuo Ishii, Yejin Bang, Andrea Madotto, and Pascale Fung. 2023a. Survey of hallucination in natural language generation. _ACM Comput. Surv._, 55(12):248:1-248:38.
* Ji et al. (2023b) Ziwei Ji, Zihan Liu, Nayeon Lee, Tiezheng Yu, Bryan Wile, Min Zeng, and Pascale Fung. 2023b. RHO: reducing hallucination in open-domain dialogues with knowledge grounding. In _Findings of the Association for Computational Linguistics: ACL 2023, Toronto, Canada, July 9-14, 2023_, pages 4504-4522. Association for Computational Linguistics.
* Krishna et al. (2017) Ranjaya Krishna, Yuke Zhu, Oliver Groth, Justin Johnson, Kenji Hata, Joshua Kravitz, Stephanie Chen, Yannis Kalantidis, Li-Jia Li, David A. Shamma, Michael S. Bernstein, and Li Fei-Fei. 2017. Visual genome: Connecting language and vision using crowdsourced dense image annotations. _Int. J. Comput. Vis._, 123(1):32-73.
* Leng et al. (2024) Sicong Leng, Hang Zhang, Guanzheng Chen, Xin Li, Shijian Lu, Chunyan Miao, and Lidong Bing. 2024. Mitigating object hallucinations in large vision-language models through visual contrastive decoding. In _IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2024, Seattle, WA, USA, June 16-22, 2024_, pages 13872-13882. IEEE.
* Li et al. (2023a) Bo Li, Yuanhan Zhang, Liangyu Chen, Jinghao Wang, Fanyi Pu, Jingkang Yang, Chunyuan Li, and Ziwei Liu. 2023a. MIMIC-IT: multi-modal in-context instruction tuning. _CoRR_, abs/2306.05425.
* Li et al. (2023b) Junnan Li, Dongxu Li, Silvio Savarese, and Steven C. H. Hoi. 2023b. BLIP-2: bootstrapping language-image pre-training with frozen image encoders and large language models. In _International Conference on Machine Learning, ICML 2023, 23-29 July 2023, Honolulu, Hawaii, USA_, volume 202 of _Proceedings of Machine Learning Research_, pages 19730-19742. PMLR.
* 16, 2023_.
* Li et al. (2023d) Xiang Lisa Li, Ari Holtzman, Daniel Fried, Percy Liang, Jason Eisner, Tatsunori Hashimoto, Luke Zettlemoyer, and Mike Lewis. 2023d. Contrastive decoding: Open-ended text generation as optimization. In 

Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), ACL 2023, Toronto, Canada, July 9-14, 2023_, pages 12286-12312. Association for Computational Linguistics.
* Li et al. [2023e] Yifan Li, Yifan Du, Kun Zhou, Jinpeng Wang, Wayne Xin Zhao, and Ji-Rong Wen. 2023e. Evaluating object hallucination in large vision-language models. In _Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, EMNLP 2023, Singapore, December 6-10, 2023_, pages 292-305. Association for Computational Linguistics.
* ECCV 2014
- 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part V_, volume 8693 of _Lecture Notes in Computer Science_, pages 740-755. Springer.
* Liu et al. [2024a] Fuxiao Liu, Kevin Lin, Linjie Li, Jianfeng Wang, Yaser Yacoob, and Lijun Wang. 2024a. Mitigating hallucination in large multi-modal models via robust instruction tuning. In _The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024_. OpenReview.net.
* Liu et al. [2024b] Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. 2024b. Improved baselines with visual instruction tuning. In _IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2024, Seattle, WA, USA, June 16-22, 2024_, pages 26286-26296. IEEE.
* 16, 2023_.
* Menendez et al. [1997] M.L. Menendez, J.A. Pardo, L. Pardo, and M.C. Pardo. 1997. The jensen-shannon divergence. _Journal of the Franklin Institute_, 334(2):307-318.
* OpenAI [2023] OpenAI. 2023. GPT-4 technical report. _CoRR_, abs/2303.08774.
* December 9, 2022_.
* November 4, 2018_, pages 4035-4045. Association for Computational Linguistics.
* ECCV 2022
- 17th European Conference, Tel Aviv, Israel, October 23-27, 2022, Proceedings, Part VIII_, volume 13668 of _Lecture Notes in Computer Science_, pages 146-162. Springer.
* Shi et al. [2024] Weijia Shi, Xiaochuang Han, Mike Lewis, Yulia Tsvetkov, Luke Zettlemoyer, and Wen-tau Yih. 2024. Trusting your evidence: Hallucinate less with context-aware decoding. In _Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies: Short Papers, NAACL 2024, Mexico City, Mexico, June 16-21, 2024_, pages 783-791. Association for Computational Linguistics.
* Shuster et al. [2021] Kurt Shuster, Spencer Poff, Moya Chen, Douwe Kiela, and Jason Weston. 2021. Retrieval augmentation reduces hallucination in conversation. In _Findings of the Association for Computational Linguistics: EMNLP 2021, Virtual Event / Punta Cana, Dominican Republic, 16-20 November, 2021_, pages 3784-3803. Association for Computational Linguistics.
* Sutskever et al. [2014] Ilya Sutskever, Oriol Vinyals, and Quoc V. Le. 2014. Sequence to sequence learning with neural networks. In _Advances in Neural Information Processing Systems 27: Annual Conference on Neural Information Processing Systems 2014, December 8-13 2014, Montreal, Quebec, Canada_, pages 3104-3112.
* Teerapittayanon et al. [2016] Surat Teerapittayanon, Bradley McDanel, and H. T. Kung. 2016. Branchynet: Fast inference via early exiting from deep neural networks. In _23rd International Conference on Pattern Recognition, ICPR 2016, Cancun, Mexico, December 4-8, 2016_, pages 2464-2469. IEEE.
* Touvron et al. [2023a] Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothee Lacroix, Baptiste Roziere, Naman Goyal, Eric Hambro, Faisal Azhar, Aurelien Rodriguez, Armand Joulin, Edouard Grave, and Guillaume Lample. 2023a. Llama: Open and efficient foundation language models. _CoRR_, abs/2302.13971.
* Touvron et al. [2023b] Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmime Babaei, Nikolay Bashlykov, Soumya Batra, Prajiwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cristian Canton-Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu, Wenyin Fu, Brian Fuller, Cynthia Gao, Vedanuj Goswami, Naman Goyal, Anthony Hartshorn, Saghar Hosseini, Rui Hou, Hakan 

Inan, Marcin Kardas, Viktor Kerkez, Madian Khabsa, Isabel Klounann, Artem Korenecy, Punit Singh Koura, Marie-Anne Lachaux, Thibaut Lavril, Jensya Lee, Dian Liskovich, Yinghai Lu, Yuning Mao, Xavier Martinet, Todor Mihaylov, Pushkar Mishra, Igor Molbyog, Yixin Nie, Andrew Poulton, Jeremy Reizenstein, Rashi Rungta, Kalyan Saladi, Alan Schelten, Ruan Silva, Eric Michael Smith, Ranjan Subramanian, Xiaoqing Ellen Tan, Binh Tang, Ross Taylor, Adina Williams, Jian Xiang Kuan, Pavin Xu, Zheng Yan, Hiyan Zarov, Yuchen Zhang, Angela Fan, Melanie Kambadur, Sharan Narang, Aurelien Rodriguez, Robert Stojnic, Sergey Edunov, and Thomas Scialom. 2023b. Llama 2: Open foundation and fine-tuned chat models. _CoRR_, abs/2307.09288.
* Vaswani et al. (2017) Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In _Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, December 4-9, 2017, Long Beach, CA, USA_, pages 5998-6008.
* Wang et al. (2024a) Ao Wang, Hui Chen, Lihao Liu, Kai Chen, Zijia Lin, Jungong Han, and Guiguang Ding. 2024a. Yolov10: Real-time end-to-end object detection. _CoRR_, abs/2405.14458.
* Wang et al. (2024b) Bin Wang, Fan Wu, Xiao Han, Jiahui Peng, Huaping Zhong, Pan Zhang, Xiaoyi Dong, Weijia Li, Wei Li, Jiaqi Wang, and Conghui He. 2024b. VIGC: visual instruction generation and correction. In _Thirty-Eighth AAAI Conference on Artificial Intelligence, AAAI 2024, Thirty-Sixth Conference on Innovative Applications of Artificial Intelligence, IAAI 2024, Fourteenth Symposium on Educational Advances in Artificial Intelligence, EAAI 2014, February 20-27, 2024, Vancouver, Canada_, pages 5309-5317. AAAI Press.
* Wang et al. (2024c) Jiaqi Wang, Zihao Wu, Yiwei Li, Hanqi Jiang, Peng Shu, Enze Shi, Huawen Hu, Chong Ma, Yiheng Liu, Xuhui Wang, Yincheng Yao, Xuan Liu, Huaqin Zhao, Zhengliang Liu, Haixing Dai, Lin Zhao, Bao Ge, Xiang Li, Tianming Liu, and Shu Zhang. 2024c. Large language models for robotics: Opportunities, challenges, and perspectives. _CoRR_, abs/2401.04334.
* Wang et al. (2023) Sheng Wang, Zihao Zhao, Xi Ouyang, Qian Wang, and Dinggang Shen. 2023. Chatcad: Interactive computer-aided diagnosis on medical image using large language models. _CoRR_, abs/2302.07257.
* Wang et al. (2024d) Xinntong Wang, Jingheng Pan, Liang Ding, and Chris Biemann. 2024d. Mitigating hallucinations in large vision-language models with instruction contrastive decoding. In _Findings of the Association for Computational Linguistics, ACL 2024, Bangokok, Thailand and virtual meeting, August 11-16, 2024_, pages 15840-15853. Association for Computational Linguistics.
* Xu et al. (2024) Xinhao Xu, Hui Chen, Zijia Lin, Jungong Han, Liking Gong, Guoxin Wang, Yongjun Bao, and Guiguang Ding. 2024. Tad: A plug-and-play task-aware decoding method to better adapt llms on downstream tasks. In _Proceedings of the Thirty-Third International Joint Conference on Artifical Intelligence, IJCAI 2024, Jeju, South Korea, August 3-9, 2024_, pages 6587-6596. ijcai.org.
* Yang et al. (2023) Fan Yang, Xinhao Xu, Hui Chen, Yuchen Guo, Yuwei He, Kai Ni, and Guiguang Ding. 2023. Gpro3d: Deriving 3d bbox from ground plane in monocular 3d object detection. _Neurocomputing_, 562:126894.
* Ye et al. (2023) Qinghao Ye, Haiyang Xu, Guohai Xu, Jiabo Ye, Ming Yan, Yiyang Zhou, Junyang Wang, Anwen Hu, Pengcheng Shi, Yaya Shi, Chenliang Li, Yuanhong Xu, Hehong Chen, Junfeng Tian, Qian Qi, Ji Zhang, and Fei Huang. 2023. mplug-owl: Modularization empowers large language models with multimodality. _CoRR_, abs/2304.14178.
* Yin et al. (2023) Shukang Yin, Chaoyou Fu, Sirui Zhao, Tong Xu, Hao Wang, Dianbo Sui, Yunhang Shen, Ke Li, Xing Sun, and Enhong Chen. 2023. Woodpecker: Hallucination correction for multimodal large language models. _CoRR_, abs/2310.16045.
* System Demonstrations, Singapore, December 6-10, 2023_, pages 543-553. Association for Computational Linguistics.
* Zhang et al. (2023) Shilong Zhang, Peize Sun, Shoufa Chen, Min Xiao, Wenqi Shao, Wenwei Zhang, Kai Chen, and Ping Luo. 2023b. Gpt4roi: Instruction tuning large language model on region-of-interest. _CoRR_, abs/2307.03601.
* Zhang et al. (2023c) Yue Zhang, Yafu Li, Leyang Cui, Deng Cai, Lemao Liu, Tingchen Fu, Xinting Huang, Enbo Zhao, Yu Zhang, Yulong Chen, Longvue Wang, Anh Twun Luu, Wei Bi, Freda Shi, and Shuming Shi. 2023c. Siren's song in the AI ocean: A survey on hallucination in large language models. _CoRR_, abs/2309.01219.
* Zhao et al. (2023) Zhiyuan Zhao, Bin Wang, Linke Ouyang, Xiaoyi Dong, Jiaqi Wang, and Conghui He. 2023. Beyond hallucinations: Enhancing lvlms through hallucination-aware direct preference optimization. _CoRR_, abs/2311.16839.
* 16, 2023_.
* Zhou et al. (2024)Yiyang Zhou, Chenhang Cui, Jaehong Yoon, Linjun Zhang, Zhun Deng, Chelsea Finn, Mohit Bansal, and Huaxiu Yao. 2024. Analyzing and mitigating object 

hallucination in large vision-language models. In _The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024_. OpenReview.net.
* Zhu et al. (2024) Deyao Zhu, Jun Chen, Xiaoqian Shen, Xiang Li, and Mohamed Elhoseiny. 2024. Minigpt-4: Enhancing vision-language understanding with advanced large language models. In _The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024_. OpenReview.net.

## Appendix A Related Work

Multi-modal large language modelsMany efforts to extend LLMs' capabilities to other modalities have led to the emergence of MLLMs Zhang et al. (2023a); Bai et al. (2023); Dai et al. (2023); Liu et al. (2024b, 2023); Li et al. (2023a). LLAVA Liu et al. (2023, 2024b), InstructBLIP Dai et al. (2023), MiniGPT-4 Zhu et al. (2024), and mPLUG-Owl Ye et al. (2023) are four representative MLLMs. They have significant differences in the training data, fine-tuning parameters, and the particular designs of the visual encoders and connectors. We conduct experiments on all four models to demonstrate the consistent effectiveness of _iTabD_.

Hallucinations in large models"Hallucination" is primarily defined by the NLP community, which refers to the models' outputs that are plausible but incorrect or nonsensical Ji et al. (2023a); Zhang et al. (2023c). Many works mitigating hallucinations in LLMs involve extra data or training costs Ji et al. (2023b); Shuster et al. (2021); Ouyang et al. (2022), while some recent works leverage LLMs' internal representations during inference, with minimal or no extra training or data costs Li et al. (2023c); Chuang et al. (2024).

In MLLMs, hallucinations typically occur when the model's output contradicts the input image Liu et al. (2024a); Li et al. (2023e). Similar to LLMs, many efforts introduce additional data and training costs Wang et al. (2024b); Liu et al. (2024a); Zhou et al. (2024). A few works discuss solutions during inference Long et al. (2024); Huang et al. (2024); Wang et al. (2024d). However, those studies concentrate on the input and output independently, neglecting the interaction between them. Moreover, they tend to yield significantly increased inference latency. On the contrary, our _iTaD_ is inspired by the observed reduction in attention weights to input image tokens associated with hallucinatory outputs. It leverages the MLLM's internal representations to mitigate hallucinations during the inference stage, without incurring additional training or data costs, and results in only a minimal and even negligible increase in inference latency.

Decoding strategies Decoding strategies are critical for MLLMs, and can significantly affect their output quality. Basic decoding strategies include Greedy Search, Beam Search Sutskever et al. (2014), Nucleus Sampling Holtzman et al. (2020), etc. Moreover, there exists a range of incremental works that can be integrated with and effectively optimize those basic strategies. The contrastive decoding series Li et al. (2023d); Gera et al. (2023); Shi et al. (2024); Chuang et al. (2024); Xu et al. (2024) serve as a prime example of such works, which utilize the likelihood difference between two probabilities to produce higher-quality texts.

Despite their achievements, that series of methods bypass attention-based approaches, either ineffective at mitigating MLLM hallucinations or impractical in real-world scenarios. VCD Lang et al. (2024) doubles inference costs by processing both noisy and original images, while DoLa Chuang et al. (2024) does not explore MLLM properties and selects layers based solely on token probabilities, resulting in limited performance in MLLMs. In contrast, our proposed _iTab_ adopts a novel attention-guided layer selection strategy for inter-layer contrastive decoding and distinguishes itself by investigating the properties of MLLM hallucinations while incurring almost no extra inference cost. It fills a critical gap, offering a highly effective and practical solution.

## Appendix B Experimental Details

### Benchmarks

CHAIR Rohrbach et al. (2018) is a widely used evaluation tool for assessing object hallucinations in image captioning tasks. Specifically, it evaluates the hallucination by counting the number of objects that appear in the generated captions but not in the ground truth. CHAIR\({}_{S}\) (denoted as \(C_{S}\)) and CHAIR\({}_{I}\) (denoted as \(C_{I}\)) are the two major metrics in the CHAIR benchmark, which assess sentence-level and image-level hallucinations, respectively. They are formulated as follows:

\[C_{S}=\frac{|\{\text{hallucination objects}\}|}{|\{\text{all mentioned objects}\}|}\] \[C_{I}=\frac{|\{\text{captions w/ hallucination objects}\}|}{|\{ \text{all captions}\}|}\] 

where a lower \(C_{S}\) or \(C_{I}\) indicates fewer hallucinations.

POPE Li et al. (2023) assesses hallucinations by querying MLLMs about the presence of objects in an image and is a widely adopted benchmark in visual question answering (VQA) tasks. It consists of an equal number of positive and negative samples. According to the strategy for constructing negative samples, the POPE test has three different splits: _random_, _popular_, and _adversarial_, comprising approximately 9,000 image-question pairs in total. In each split, POPE judges the model's answers, where responses typically start with Yes or No, and calculates the F1 score as the major metric. A higher F1 score indicates fewer hallucinations.

Following Yin et al. (2023) and Huang et al. (2024), the GPT-4V assisted evaluation benchmark prompts GPT-4V OpenAI (2023) with instructions to compare two sets of descriptions for an image. Then it assigns a score between 1 and 10 for each description based on its correctness and detailedness, respectively, with higher scores indicating better performance. The strength of this benchmark lies in its capability to assess hallucinations in attributes, positions, etc., while other benchmarks like CHAIR and POPE fail.

MME Fu et al. (2023) calculates the scores of various sub-tasks, where higher scores indicate fewer hallucinations. GPT-4 assisted hallucination evaluation Zhao et al. (2023) comprises six metrics, _i.e._, SPI, WPI, HSPI, HWPI, HSR, and HWR, which represent the number of sentences, words, hallucinatory sentences, hallucinatory words per image, and the ratios of hallucinatory sentences and words, respectively.

### Prompts for All Benchmarks

By default, we use the same prompt templates for LLaVA-1.5, InstructBLIP, and MiniGPT-4 as those described by Huang et al. (2024)1. For mPLUG-Owl, we follow the template in Yin et al. (2023)2, which starts with the instruction "The following is a conversation between a curious human and AI assistant. The assistant gives helpful, detailed, and polite answers to the user's questions." For all benchmarks requiring image descriptions, _i.e._, CHAIR, GPT-4V and GPT-4 assisted evaluation, we query different MLLMs with "Please describe this image in detail." For the POPE and MME benchmarks, we tailor the prompts with specific questions. In the POPE benchmark, we query MLLMs with an essay question formatted as "Is there an

\begin{table}
\begin{tabular}{l} \hline \hline GPT-4V Prompt \\ \hline You are required to score the performance of two AI assistants in describing a given image. You should pay extra attention to the hallucination, which refers to the part of descriptions that are inconsistent with the image content, such as claiming the existence of something not present in the image or describing incorrectly in terms of the counts, positions, or colors of objects in the image. Please rate the responses of the assistants on a scale of 1 to 10, where a higher score indicates better performance, according to the following criteria: \\
1: Accuracy: whether the response is accurate with respect to the image content. Responses with fewer hallucinations should be given higher scores. \\
2: Detailedness: whether the response is rich in necessary details. Note that hallucinated descriptions should not count as necessary details. Please output the scores for each criterion, containing only two values indicating the scores for Assistant 1 and 2, respectively. The two scores are separated by a space. Following the scores, please provide an explanation of your evaluation, avoiding any potential bias and ensuring that the order in which the responses were presented does not affect your judgment. \\
[Assistant 1] \\
[End of Assistant 1] \\
[Assistant 2] \\
[End of Assistant 2] \\ Output format: \\ Accuracy: <Scores of the two answers> \\ Reason: \\ Detailedness: <Scores of the two answers> \\ Reason: \\ \hline \hline \end{tabular}
\end{table}
Table 11: The prompt used in the GPT-4V assisted benchmark.

 

<object> in the image?" to determine whether an MLLM can identify the object's presence in a given image. In the MME benchmark, the question format is similar to that used in the POPE benchmark, which also elicits answers starting with 'Yes' or 'No'.

The prompt for the GPT-4V assisted benchmark, detailed in Table 11, leverages GPT-4V's image understanding capabilities to evaluate the correctness and detailedness of two descriptions. In our experiments, Assistant 1 and Assistant 2 are initially set to Beam Search/OPERA and our _iTAD_, respectively, with their roles randomly swapped to prevent any bias associated with the order of presentation in GPT-4V. Table 12 displays the prompt used in the GPT-4 assisted benchmark. In this benchmark, we employ the VG-100K dataset Krishna et al. (2017), which includes detailed ground-truth descriptions of all objects. GPT-4 is prompted with these descriptions that detail various object attributes such as quantity, color, and location, to judge and revise the descriptions generated by MLLMs. The performance metrics HSPI, HWPI, HSR, and HWR are then calculated based on GPT-4's judgment.

### Selection of the Hyper-Parameter \(\alpha\)

Due to the distinct answer format of the POPE and MME benchmarks, which starts with 'yes' or 'No', we set the hyper-parameter \(\alpha\) for these benchmarks differently from our default settings described in Section 3.2. Specifically, for LLaVA-1.5, InstructionBLIP, MiniGPT-4, and mPLUG-Owl, \(\alpha\) is set to 0.9, 0.3, 0.9, and 0.03, respectively. The value of

\begin{table}
\begin{tabular}{l} \hline \hline GPT-4 Prompt \\ \hline Please help me judge if the comment of this image is hallucination or correct. \\ I will give you a list of region description of a image. The format is [x1, y1, x2, y2]: region description, where [x1, y1, x2, y2] is the bounding box of the region. Highly overlapping bounding boxes may refer to the same object. This is the ground truth information of the image. Your judgement should base on this information. However, this information only describe the objects in the region of image, so it cannot describe the subjective part of the image, e.g., atmosphere, style, emotion. In that case, you can return “Cannot judge”. \\ Also, I will give you a list of comments of the image for you to judge if it is hallucination. Please give a judgement one by one along with the reason. \\ You should pay extra attention to the hallucination, which refers to the part of comments that are inconsistent with the descriptions, specially claiming the existence of something not present in the descriptions. \\ If a comment is hallucination, please help me rewrite it. When rewrite the comment, sound like you are looking at the image directly. Each rewritten comments should compose a description about the image which is correct, detailed, smooth and has strong readability. If not hallucination (correct or cannot judge), keep the original comment. \\ Your output should be: \\ Judgement: \\
1. hallucination or correct or cannot judge: \textless{reason}> \\
2. ... \\ Revised Sentences: \\
1. ... \\
2. ... \\ Here are the region descriptions of the image: \\ \{} \\ Here is the comment for you to judge if it is hallucination and revise: \\ \{} \\ \hline \hline \end{tabular}
\end{table}
Table 12: The prompt used in the GPT-4 assisted benchmark.

 

\(\alpha\) for each MLLM is determined by evaluating its F1 score performance of POPE on a separate set of 100 randomly selected images from the COCO validation set, ensuring no overlap with the test images used in the POPE and MME benchmarks.

### License for Scientific Artifacts

LLaVA-1.5 Liu et al. (2024) and Instruct-BLIP Dai et al. (2023) are subject to the Llama 2 Community License. MiniGPT-4 Zhu et al. (2024) is licensed under BSD 3-Clause License. mPLUG-Owl Ye et al. (2023) is subject to the Apache 2.0 License. MSCCOC Lin et al. (2014) and A-OKVQA Schwenk et al. (2022) are licensed under CC BY 4.0. MME Fu et al. (2023), VGG-100K Krishna et al. (2017), and GQA Hudson and Manning (2019) are subject to the MIT license. All usages of scientific artifacts in this paper obey the corresponding licenses.

## Appendix C Additional Results Compared to Baselines

### Comparison with VCD

We compare _iTaD_ with VCD Leng et al. (2024) on the CHAIR benchmark, as shown in Table 13. Although VCD can effectively mitigate hallucinations in MLLMs, our _iTaD_ consistently outperforms VCD across all performance metrics. The experiment results highlight _iTaD_'s superior performance compared to VCD.

### Cross-Dataset Validation on the POPE Benchmark

To further validate the generalizability of the proposed _iTaD_, we conduct extensive experiments on A-OKVQA Schwenk et al. (2022) and GQA Hudson and Manning (2019) for the POPE benchmark. Table 14, along with Table 3, demonstrates that our method consistently outperforms Beam Search and OPERA on the POPE benchmark across different datasets.

## Appendix D Additional Quantitative Analysis of _iTaD_

#### Performance on LLaVA-NeXT

The LLaVA-NeXT series3 adopts the 'AnyRes' strategy, selecting grid configurations based on image sizes, which naturally leads to varying image token proportions. Based on this, we randomly sample two sets of 500 images from the MSCOCO validation set with grid configurations of {1\(\times\)2, 2\(\times\)1} (2 grids) and {2\(\times\)2} (4 grids), respectively. The average number of image tokens for the two sets is <1600 and >2000, respectively. Experimental results on CHAIR are shown in Table 15.

Footnote 3: [https://github.com/LLaVA-VL/LLaVA-NeXT](https://github.com/LLaVA-VL/LLaVA-NeXT)

_iTaD_ demonstrates robust and effective hallucination mitigation across varying image token proportions. When the grid configuration is {2\(\times\)2} (average number of image tokens >2000), its effectiveness is comparable to or slightly more notable than when the grid configuration is {1\(\times\)2, 2\(\times\)1} (average number of image tokens <1600).

### Standard Deviations of Experiments

Table 16 presents the standard deviations in the performance of the four MLLMs across 5 evaluation sets, each comprising 500 images randomly selected from the COCO validation set. It is noted that although the value of \(C_{S}\) and \(C_{I}\) varies across different evaluation sets, _iTaD_ exhibits consistent improvements upon Beam Search, which demonstrates robustness to variations in data distribution.

\begin{table}
\begin{tabular}{l c c c c c c} \hline \hline \multirow{2}{*}{Method} & \multicolumn{3}{c}{ALEAVA-1.5} & \multicolumn{3}{c}{InstructBLIP MinistiOPT-4} \\ \cline{2-8}  & \(C_{S}\downarrow\) & \(C_{I}\downarrow\) & \(C_{S}\downarrow\) & \(C_{I}\downarrow\) & \(C_{S}\downarrow\) & \(C_{I}\downarrow\) \\ \hline Beam Search & 50.0 & 14.5 & 56.4 & 16.2 & 33.9 & 10.7 \\ VCD & 47.9 & 13.7 & 54.6 & 15.5 & 32.2 & 10.1 \\ _iTaD_ & **45.4** & **13.4** & **53.2** & **14.7** & **26.4** & **9.6** \\ \hline \hline \end{tabular}
\end{table}
Table 13: Comparison with VCD on the CHAIR benchmark. We bold the best results and underline the second-best results (same below).

\begin{table}
\begin{tabular}{l c c c c} \hline \hline \multirow{2}{*}{Method} & \multicolumn{3}{c}{A-OKVQA} & \multicolumn{3}{c}{GQA} \\ \cline{2-5}  & MiniGPT-4 & mPLUG-Owl & MiniGFPT-4 & mPLUG-Owl \\ \hline Beam Search & 69.94 & 68.72 & 69.37 & 69.13 \\ OPERA & 70.84 & 68.37 & 70.10 & 68.76 \\ _iTaD_ & **72.33** & **70.77** & **70.86** & **70.73** \\ \hline \hline \end{tabular}
\end{table}
Table 14: Results on the POPE benchmark for A-OKVQA and GQA. We report the average F1 scores † across _random_, _popular_ and _adversarial_ splits.

 



### Analysis of the Candidate Set \({\cal M}\)

In our paper, we set \({\cal M}\) to \(\{2,4,6,8,10,12,14\}\) by default, without any tuning. Table 17 presents the analysis results for \({\cal M}\). To explore the impact of different layer depths, we adjust \({\cal M}\) to \(\{16,18,20,22,24,26,28\}\). It can be observed that using shallow layers for \({\cal M}\) is critical for the effectiveness of the proposed _iTaD_ while deeper layers tend to degrade performance and yield hallucinations. This performance degradation is consistent with the results discussed in Chuang et al. (2024), which empirically demonstrates that statically selecting deeper layers for contrastive decoding results in performance below that of the baseline, _i.e._, Beam Search in our experiment. Furthermore, we expand \({\cal M}\) to \(\{1,2,3,4,5,6,7,8,9,10,11,12,13,14\}\) to investigate the impact of the size of set on the performance of _iTaD_. Although this adjustment doubles the additional memory overhead required to store \(h^{n}_{t}\), the experiment results show that it only slightly improves the performance of _iTaD_. On the contrary, the setting of \({\cal M}\) in our paper effectively balances performance with efficiency.

### Impact of Output Length on Latency

Table 8 shows the average latency per token for generating descriptions of 50 images, which is independent of the dataset size and empirically consistent with the result calculated on more images (_e.g._, 200 images). Table 18 shows the impact of output sequence length on LLaAVA-1.5's inference latency (ms/token). As the output lengthens, _iTaD_'s latency increase diminishes, showing a decreasing factor from 1.11 to 1.09 and 1.08.

### Latency-Performance Trade-Off

The primary source of the increased latency with _iTaD_ compared to Beam Search is the calculation of _iTaV_ and their JSD for intermediate layers in \({\cal M}\) and the final layer. We can achieve a performance and efficiency trade-off by adjusting the size of \({\cal M}\), as shown in Table 19. It is observed that the selected \({\cal M}=\{2,4,6,8,10,12,14\}\) in this paper effectively balances performance and efficiency.

### Text Quality Evaluation

Following OPERA Huang et al. (2024), we calculate PPL (Perplexity, a classical metric in NLP without using reference text) using LLaMA-2-7b and LLaMA-2-13b Touvron et al. (2023b) to evaluate text quality, respectively. Table 20 shows that _iTaD_ has a minor effect on text quality while largely maintaining the higher text quality of generated text by Beam Search compared to text generated by Greedy Search and Nucleus Sampling.

## Appendix E Qualitative Analysis

Figure 5, 6, and 7 showcase the qualitative results of LLaVA-1.5, InstructBLIP and MiniGPT-4, and mPLUG-Owl, respectively. In these figures, MLLMs are prompted to generate descriptions for the images. We bold and highlight the hallucinatory segments in the Beam Search outputs in red. These hallucinations often include echolalic, repetitive sentence structures, such as the second example of

\begin{table}
\begin{tabular}{c c c c c c c c} \hline \hline Method & \multicolumn{2}{c}{LLaVA-1.5} & \multicolumn{2}{c}{Instruct@ILP} & \multicolumn{2}{c}{MiniGPT-4} & \multicolumn{2}{c}{mPLUG-Owl} \\ \cline{2-7}  & \(C_{S}\downarrow\) & \(C_{I}\downarrow\) & \(C_{S}\downarrow\) & \(C_{I}\downarrow\) & \(C_{S}\downarrow\) & \(C_{I}\downarrow\) & \(C_{S}\downarrow\) & \(C_{I}\downarrow\) \\ \hline Beam Search & 50.0 (\(\pm\)2.70) & 14.5 (\(\pm\)1.00) & 56.4 (\(\pm\)1.47) & 16.2 (\(\pm\)0.68) & 33.9 (\(\pm\)2.92) & 10.7 (\(\pm\)1.01) & 75.8 (\(\pm\)0.66) & 25.4 (\(\pm\)0.89) \\ _iTaD_ & 45.4 (\(\pm\)3.00) & 13.4 (\(\pm\)0.99) & 53.2 (\(\pm\)2.97) & 14.7 (\(\pm\)1.48) & 26.4 (\(\pm\)1.83) & 9.6 (\(\pm\)0.45) & 70.0 (\(\pm\)2.52) & 24.5 (\(\pm\)1.52) \\ \(\Delta\) & **-4.6 (\(\pm\)2.29)** & **-1.1 (\(\pm\)0.19)** & **-3.2 (\(\pm\)2.24)** & **-1.5 (\(\pm\)0.95)** & **-7.5 (\(\pm\)2.47)** & **-1.1 (\(\pm\)1.10)** & **-5.8 (\(\pm\)2.20)** & **-0.9 (\(\pm\)0.74)** \\ \hline \hline \end{tabular}
\end{table}
Table 16: Mean value and standard deviations from experiments on the CHAIR benchmark across 5 evaluation sets. \(\Delta\) represents the improvement of _iTaD_ compared to Beam Search.

\begin{table}
\begin{tabular}{l c c c} \hline \hline \(max\_l\) & 32 & 64 & 128 \\ \hline Beam Search & 52.5 (\(\times\)1.00) & 54.3 (\(\times\)1.00) & 55.7 (\(\times\)1.00) \\ _iTaD_ & 58.1 (\(\times\)1.11) & 59.3 (\(\times\)1.09) & 60.4 (\(\times\)1.08) \\ \hline \hline \end{tabular}
\end{table}
Table 17: Analysis results of the candidate set \({\cal M}\).



mPLUG-Owl, and references to objects that frequently co-occur with the object in the image but are absent, such as the second example of Instruct-BLIP. It can be observed that our _iTaD_ effectively mitigates such hallucination issues while still yielding high-quality and informative responses.

\begin{table}
\begin{tabular}{l c c c} \hline \hline PPL\(\downarrow\) & Greedy & Nucleus & Beam Search & _iTaD_ \\ \hline LLaMA-2-7b & 6.32 & 8.61 & **5.66** & 6.01 \\ LLaMA-2-13b & 6.15 & 8.56 & **5.49** & 5.92 \\ \hline \hline \end{tabular}
\end{table}
Table 20: The evaluation results for the quality of the generated text, where a lower PPL score indicates higher text quality. The experiments are conducted using LLaVA-1.5.

Figure 5: Qualitative results of LLaVA-1.5. We bold and highlight the hallucinatory segments in red.

 

The image depicts a group of sheep grazing on a lush green lawn in a residential area. There are three sheep in the scene, with two of them standing close to each other near the center of the lawn, while the third one is slightly off to the side. **no addition to the sheep, there are several cars parked around the area. One car is positioned closer to the center of the lawn, while the others are located further away from the main group of sheep. A truck can also be seen in the background, adding to the variety of vehicles in the scene. Overall, the image captures a peaceful moment in a residential area where animals and vehicles coexist in harmony.**

Figure 6: Qualitative results of InstructBLIP and MiniGPT-4. We bold and highlight the hallucinatory segments in red.

 

Figure 7: Qualitative results of mPLUG-Owl. We bold and highlight the hallucinatory segments in red.