Intervening Anchor Token: Decoding Strategy in Alleviating Hallucinations for MLLMs
Authors: Feilong Tang
Zile Huang
Chengzhi Liu
Qiang Sun
Harry Yang
Ser-Nam Lim
Venue: ICLR 2025 Poster (2025)
Keywords: Multimodal large language models; Hallucination
Primary area: foundation or frontier models, including LLMs

Abstract: Multimodal large language models (MLLMs) offer a powerful mechanism for interpreting visual information. However, they often suffer from hallucinations, which impede the real-world usage of these models. Existing methods attempt to alleviate this issue by designing special decoding strategies that penalize the summary tokens. However, these methods lack analysis of the relationship between hallucination and summarization mechanism of LLMs. Interestingly, we find that penalizing summary tokens is not necessary: merely intervening the query-key parameters variance, without costing extra inference time, still alleviates hallucinations. Specifically, we explore the causes of hallucinations by analyzing localized self-attention patterns called ``anchor" tokens and define the attention localization degree of the model as token propagation probabilities. Our analysis reveals that over-propagation of anchor tokens occurs when the distribution of eigenvalues of the query and key matrices has a non-zero mean and a polarized variance, leading to excessive dependence on anchor tokens while neglecting vision information and describes the image content with hallucination. Based on the observation, we propose a versatile plug-and-play decoding strategy, Dynamic Token Propagation Mechanism (TAME), to alleviate excessive propagation by dynamically intervening the eigenspectrum variance of the attention weight, thereby alleviating hallucinations without relying on complex decoding strategies. Extensive experiments reveal a correlation between the eigenspectrum and hallucinations across various MLLMs, and show that TAME reduces the percentage of hallucinated objects.

Published as a conference paper at ICLR 2025
INTERVENING ANCHOR TOKEN: DECODING STRAT- EGY IN ALLEVIATING HALLUCINATIONS FOR MLLMS
Feilong Tang 1.2g , Zile Huang 22, 12,, 'HKUST, "Everlyn AI, University of Toronto, *University of Liverpool, SUCF tangfeilone fongllllngl gma com, sernam@uct. edu
Chengzhi Liu", Qiang Sun Harry Yang Ser-Nam Lim 2.5
ABSTRACT
Multimodal large language models (MLLMs) offer a powerful mechanism for interpreting visual information. However, they often suffer from hallucinations. which impede the real-world usage of these models. Existing methods attempt alleviate this issue by designing special decoding strategies that penalize the summary tokens. However, these methods lack analysis of the relationship be- tween hallucination and the summarization mechanism of LLMs. Interestingly, we find that penalizing summary tokens is not necessary: merely intervening in the query-key parameters variance, without costing extra inference time, still al- leviates hallucinations. Specifically, explore the causes of hallucinations by analyzing localized self-attention patterns called "anchor" tokens and define the attention localization degree of the model token propagation probabilities. Our analysis reveals that over- wee-prooaaattion of anchor tokens occurs when the distribu- tion of eigenvalues of the query and key matrices has non-zero mean and polar- ized variance, leading to excessive dependence on anchor tokens while neglecting vision information and describing the image content with hallucination. Based on the observation, we propose versatile buu-and--paa decoding strategy, Dynamic Token Propagation Mechanism (TAME), to alleviate excessive propagation by dynamically intervening in the eigenspectrum variance of the attention weight, thereby alleviating ballucinations without relying on complex decoding strategies. Extensive experiments reveal correlation between the cigenspectrum and hallu- cinations across various MLLMs and show that TAME reduces the percentage hallucinated objects. Code released at https///gthbbbcoom//////////////
INTRODUCTION
Recent advancements in multi-modal large language models (MLLMs) (Xue et al. 2025 Zhang et al. 2023a Liu et al. 2024d Bai et al. 2023 Dai al. 2023a: Liu et al. 2024c Dong et al. 2024 have propelled general purposs foundation models to unprecedented capabilities. These ad- vancements have equipped MILMs with the ability to process images inputs, enabling highly dynamic and contextually rich interactions. The advanced functionality of MLLMS allows them be adept variety of vision-related tasks(Black al. 2023 Zhang al.) 2023b Li et al. 2024a), while seamlessly handling more complex tasks such as content comprehension (Lai et al., 2024) and generation (Geng al., 2024). Despite their remarkable versatility, MLLMs often suffer from hallucinations. Specifically. these models tend to generate fabricated or incorrect outputs in re- sponse to user-pre -soviddo images and prompts, often producing irrelevant nonsensical information, or misidentifying objects in terms of colors, quantities, or locations that do not exist in the image.
Various approaches (Wang et 2024a Yin et al. 2023 Zhou et al., 2023) have been proposed to mitigate hallucinations in MLLMs. These methods often incur substantial additional costs, including the annotation budget for extra instruction data for training (Liu et al.) 2023a), or the integration of external knowledge or models. Conversely, other approaches focus on decoding strategy optimiza- tion to penalize the knowledge aggregation patterns, avoiding training but doubling or even tripling inference time. OPERA (Huang et al. 2024) introduces a penalty-based re-decoding approach to alleviate the over-trust summary token issue. Contrastive Decoding (CD) strategies adjust logits for
"Equal Contribution. Work done during internship Everlyn AI.

---

Published as a conference paper at ICLR 2025
Aguregasion flow
Systmm Tokone SVS
Vasal Teleeza VIS
Insouction Tokera INS
Genenaal Tokene GEN
Attention Score
Distribttion flow
Layer
Layer 16
Layer 31
(a) Token Propagation Pattern
(b) The Attention Maps Across Different Layers During the Decoding Process of L-7.
Figure 1: (a): Illustration of the token propagation pattern in the Self-attention Query-Key Matrix, where anchor tokens (highlighted in orange boxes) aggregate and distribute knowledge. (b): The attention maps across layers during the decoding process of LLa show that in initial lay- ers, attention is evenly distributed. However, in deeper layers, attention aggregates mainly towards system, instruction, and generated tokens, while attention to image tokens becomes sparse. Detailed attention allocation is provided in Appendix [A] These findings indicate that anchor tokens, poten- tially caused by the QK parraaaesssrs, attract most of the attention and contribute to hallucinations.
next-token prediction using contrastive techniques. Vision CD applies Gaussian noise (Leng et al., 2024) or ablates visual inputs (Favero et al. 2024) to amplify language priors, while Instruction CD (Wang et al., 2024b Jiang et al.] 20024 introduces noise by adding random words, contradictory commands, or truncated instructions. Despite their effectiveness, these methods incur computational burdens and impede the deployment of MLLMs on personal devices. Furthermore, the relationship between hallucination and the inherent summarization mechanism of LLMS remains unexplored.
Recent studies (Wang et al. 2023 Pang et al., 2024) have shown that token information flow ag- gregates to a few "anchor tokens", from which the model extracts information to make predictions, facilitating tokens interaction patterns and in-context learning, as illustrated in Figure (a). How- ever, (Huang et al. 2024 Wu et al. 2024) empirically found that hallucinations stem from an over-reliance partial anchor tokens. Specifically, the limited anchor tokens cannot retain the rich visual information provided by the entire context. During the transmission of information between anchor tokens, the visual information becomes attenuated as the length of the generated text in- creases. Their findings suggest that subsequent tokens neglect the initial visual input leading to hallucinations caused by the model bias. Up until now, discussions about anchor tokens have been conducted independently, each with slightly different interpretations. As a result, our understanding of the blessing and curse of anchor tokens remains elusive.
To delve deeper into this phenomenon, we analyze the attention maps of the first, middle, and final layers during the decoding process of a model response as illustrated in (b) As attention can be regarded as token mixer, in the shallow layers, attention scores are more uniformly distributed across different tokens. Whereas in the deeper layers, system prompts display vertical strong lines that take up most of the attention scores (which we call localized attention). Our statistical analysis reveals a highly imbalanced attention distribution: in the deep layers, attention is focused on these anchor tokens, leading to significantly reduced attention results in the model generating content inconsistent with the actual facts in images. Based on these observations, we propose the following two key research questions: (QI) When are tokens localized or uniform? (Q2) How does anchor tokens affect the generation of hallucinations?
the image tokens themselves. This
In this paper, we characterize self-attention token patterns through the attention weight matrix to investigate the root causes of hallucinations. First, we define the concept of anchor tokens through token propagation probability (Section 2. which describes the likelihood of specific input token propagating its information to other tokens within the information flow of LLM. Our rigorous statis- tical analysis reveals that hallucinatory captions tend to exhibit higher token propagation probabili- ties. Then, we demonstrate that the propagation pattern of anchor tokens can be characterized by the eigenspectrum of the attention weight matrix (Section and 4 by (Bao et al., 2024). Specifically, proper-propagation of anchor tokens enhances expressivity when the query-key eigenspectrum has a non-zero mean and a small variance. However, over werrprrpaggttio triggers hallucinations when the variance becomes polarized. To alleviate this issue, we propose versatile plug and-play decoding strategy, Dynamic Token Propagation Mechanism (TAME), which reduces the over- propprrpagttion of anchor tokens through dynamically intervening in the eigenspectrum variance (Section 5). Interest-
2

---

Published as a conference paper at ICLR 2025
ingly, find that penalizing summary tokens is not necessary: merely intervening in the query-key parameters variance, without incurring extra inference time, still alleviates hallucinations. Lastly, with extensive experiments, observe a correlation between the eigenspectrum and hallucinations in various MLLMs, and demonstrate that TAME reduces the percentage of hallucinated objects.
2 TOKEN PROPAGATION PROBABILITY
This section scrutinizes the root causes of hallucinations in vision- saiinnlanguage models through com- prehensive statistical analyses of token propagation probability and hallucination. We also provide a rigorous theoretical explanation that complements our empirical findings on hallucinations.
Notations. MLLMs generate text in auto-regressive manner by progressively predicting the probability distribution of the next token. In this section, we represent the input as X = (21,22 E R*xT where z; E Rh is the embedding of the i-th token and T is the number of tokens. The correct answer is denoted as y, and the
sequence, consisting of N tokens, represented Z = {21.22,.. 22,..................... 2N) E RaxN Specifically, the probability of generating the i-th token 2; is modeled as p(z X), where 1 < S N and sei represent the sequence of previously generated tokens before the i-th token. Several decoding strategies are developed based on p. including Greedy Decoding and Beam Search. The decoded token is concatenated to the end of the original input text for the next round of generation, continuing until the process concludes.
During autoregressive generation, the model employs a self-attention mechanism to capture depen- dencies between tokens. At the f-th layer, for each attention head, the self-attention defined as:
= (X WoxX'11
=
Ut = -
Vd
where Wv Raxh represents the value weight matrix, W WQK = WK E RTxT is the combined query-key weight matrix, and Vd is temperature scaling factor. S denotes the softmax function. At the é-th layer, U represents the updated token embeddings after applying the value matrix Wy and attention scores the input embeddings in X
Uniform vs. Localized Softmax. We employ Sparsemax (Martins & Astudillo 2016), a piecewise linear alternative to Softmax, to streamline the computation of Gaussian moments while preserving the attention structure of the original Softmax. To linearize S(K)); where e RR is input vector, we perform Taylor expansion the origin, yielding:
= V.S( 1 - 5(0,,)) 1
where et and 6 represent the expansion coefficients. We then approximate S using a piecewise linear function S. as follows:
(2. + =((s)) This indicates that the i-th input token is activated when 8; = (É. + c [0. Otherwise, S(x), 7/ effectively preventing the input token 2; from contributing to the self-attention mech- anism. Building on this, we quantify the likelihood of activation for the i-th token in the Softmax function to estimate the extent to which its information propagates to other tokens.
S(k), x max(0, min(1, (cf, + +1) = =
(3)
Definition 1 (Token Propagation Probability). Suppose that W Wok is independent of X. For each i E [T], the token propagation probability of the i-th token is defined as:
P: P (ō, E [0,
(4)
where k- = X WWKKTTK/VVV and the randomness originates solely from the input tokens X.
When only few P; significantly greater than zero, the softmax function behaves localized, meaning the self-attention mechanism (Eq. [ is dominated by a few anchor tokens. Conversely, uniform softmax, which produces similar A values, results in equal contributions from most tokens.
Based on the definition the Token Propagation Probability Pi. we compare the distributions of p1 between hallucinatory and non-hallucinatory captions (see Appendix C.3 for details). As shown in Figure 2 hallucinatory captions tend to exhibit higher token propagation probabilities, which suggests a stronger association between object hallucination and higher propagated tokens.
3

---

Published as a conference paper at ICLR 2025
InstrueRRL IP
2-0.1
Interatee Prrpassee th
10
Frequencc
30
04
92
Information Propogation Probability
Bdiaive ponnigoniizz
pooiico
$ 5-5
{ -0
cederyy Coptinn
Frequency
92-
02-
Informorion Propagation Probability
taken postiun
Balative
Figure 2: (Left): Comparison of token propagation probability between hallucinatory and non- hallucinatory captions generated by LLaVA-1.5 and InstructBLIP models. (Right): The plots of the Token Propagation Probability p(0) for varying & and w. The horizontal axis indicates relative token position = i/T ( (i: token index, T: number of tokens) by (Bao et al. 2024)..
WHEN ARE TOKENS LOCALIZED OR UNIFORM?
Assumption (Gaussian Token Distribution). Assume that the tokens (x1)121 are independent and identically distributed (i.i.d.) random vectors frow a multivariate Gaussian distribution:
X (p, for all t t 1,
(5)
where y is the mean vector, and Σ is the covariance matrix.
To derive Ps. note that o; is linear combination of multiple random variables. By the Central Limit Theorem, be approximated as a normal distribution with mean and variance:
hr E [ u = Var (8.). 64 N( v').
(6)
Proposition 1. i E [T]. the mean y² and variance v² with the input & X W WOKXT/ Vd as:
Suppose that Wok is symmetric and independent from X. Under Assumption 7 for
tr(W) Vd
(W2) + o(1),
+ +o();
=2 d
(7)
where W denotes the weighted covariance matrix W WoKE. tr(W) represents the trace of W. and are constants, with c1 + and C2
2;2 7
72 +
12
When W qk is asymmetric, we redefine the token propagation probability using the symmetrized matrix (W QK + WTx) /2 in the following proposition. The tr(W) equals the sum of its eigenval- tr(W) -hh Wt, where w1, .., , Wi, are the eigenvalues of W.
Proposition Since p² and v' depend on the relative token location i/T,, we extend i/T continu- ously to 0 E [0, 1), and thus extend token propagation probability to p [0, 10 [0, 1 as:
(0
2)
(0
1) 1
p(0)
E 2
(8)
(())
(())
where o- = tr( V tr (W2), w = (t2))////// Vd, with ranges & E [d vdl and w E (0, oo).
<(0) /2 (202 + 72) denotes the error function.
Remark: When W is independent of X, ó and w can be considered independent variables, as the eigenspectrum scale of tr (W2) can be adjusted within the bound once the eigenspectrum of W
4

---

Published as a conference paper at ICLR 2025
is given. & measures the variance spread across eigenvalues, while represents the eigenvalue scale relative to the matrix dimension, indicating correlation strength between dimensions.
Figure 2 iillustrttes p(0) with different ø and w, leading to several key observations: (i) Localization. p(0) concentrates on fewer tokens as w increases. By contrast, p(0) behaves relatively uniformly regardless of for small (6) (ži) Location focus. For small w, ø increases, p(0) assigns weight to late-site tokens. Conversely, for negative Ø, it focuses on early-site tokens. When w increases, p(0) localizes around the middle of the sequence for sufficiently large Φ. (iii) Vanishing propagation. As
increases, p(0) diminishes to zero for any E [0, 1] regardless of Φ.
Proposition 3. p(0) satisfies the following properties by (Bao et al.
2024
1. (Tokens Localized) When dw + significantly deviates from zero, such that |r| >> 2, the signal propagation probability p(0) will concentrate specific positions in the sequence.
2. (Tokens Uniform) With w held as a finite value, as |Ø] approaches zero, p(0) approaches
constant value for any 0 [0, 1].
3. (Vanishing Propagation) With o fixed as a finite value, as w increases indefinitely, p(0) diminishes
to zero for all E [0, 1).
Remark. Proposition indicates that the behavior of the token propagation probability p(0) is closely tied to the interaction between w and Φ. As dh = tr(W)/vd, we focus on the eigenspectrum of W, where the eigenvalues (W;)kE[M) are considered as samples from a distribution with mean tr(W)= - = W; and scale tr(W3) - uu (assuming W is real diagonalizable).
Firstly, the condition Secondly, since dw tr(W)/Vd + > 2, it follows that tr(W) > 2vd, meaning tr tr(W) significantly different from zero. Combining these insights, we conclude that localizes when the eigenspectrum concentrates around non-zero mean. This localization is more likely when the embedding dimension d is large, allowing the eigenvalue sum tr(W) to remain significantly non- zero while the scale ir( (W2) stays close zero (i.e., each eigenvalue is close zero). Therefore, increasing the embedding dimension d facilitates attention localization. Conversely, according to Proposition 3]s assertion of uniformity, as 6 approaches zero, p(0) varies less across different token positions 0. In this limit, when tr (W) + 0, p becomes uniform across positions.
0 indicates that the scale tr(W2) approaches zero (see Proposition. 2
A1: When are tokens localized or uniform?
Localization: becomes localized when tr (W2) is close to zero while tr(W)] is signifi- cantly different from zero; i.e., the eigenspectrum of concentrates around non-zero mean.
Uniform: / becomes uniform when tr(W) is close to zero while tr (W2) remains finite, iee, the eigenspectrum of W has zero mean with finite variance.
Vanishing: P uniformly tends to zero when tr (W2) is sufficiently large; i.e., the eigenspec- trum of W has an infinitely large variance.
4 How DOES ANCHOR TOKEN AFFECT MLLMS?
Propee-Propagation of Anchor Tokens Enhances Expressivity: In self-attention blocks, rank collapse (Dong et al, 2021) indicates that the output matrix U in Eq. converges to a rank-1 matrix as L - 0o, i.c., lim A z1 where z is a non-zero vector and 1 is all-ones matrix. In this scenario, the attention matrix becomes uniform, causing the attention distributions of all input tokens to converge the same value. This prevents the model from distinguishing between different input information, resulting in a gradual loss of diversity and expressiveness. (Dong et al.) 202Th linked uniformity the spectral properties of the weight matrix W, demonstrating that when the E norm |Wokll, ||, of the matrix is large, the convergence to a rank- 1 matrix slows down. This implies that when attention is appropriately propagated across a few anchor tokens, the localized attention distribution can guide the model to more effectively capture subtle feature differences, thereby leading to better expressivity. The connection between || WoKll || and tr(W) is given as:
tr)( < ||Woll2 < ||Wokllp S ||
where the first inequality is due to the bound ( and the Cauchy-Schward inequality, it is sufficient to increase |tr[W] under fixed tr (W2) to enhance expressivity.
5

---

Published as a conference paper at ICLR 2025
Plopaptiin Entorry Ex800
T+40
72
15
0.55
83
646
01
0.
0.4
0.6
88
Eigenvalue Scale
Eigenvalue Mean
Figure (Left)): Propagation Entropy under different token length a |o ||x||XXX 1l2- (Right): The box plots of the averaged Token Propagation Probability with different value of Eigenvalue Scale and Eigenvalue Mean. The experiments for Eigenvalue Scale are conducted with fixed Eigenvalue Mean, and vice versa. With stronger Scale smaller Mean, the p increases.
Over-Propagate of Anchor Tokens Triggers Hallucinations: In self-attention blocks, the av- erage Shannon entropy of the columns of the attention matrix A (see Eq. decreases, iee, limg-+o Havg (At) + O where Havg (-) measures the uniformity of the attention distribution. Intu- itively, low attention entropy leads to localized attention. This concept is similar to ours. (Zhai et al.) 2023 Bao et al. 2024) propose that low attention entropy leads to instability in transformer training and inference. They advocate for avoiding overly concentrated attention and demonstrated that the lower bound of attention entropy is unimodal function of || qulll- In MLLMs, when token tropy approaches zero, the attention distribution becomes overly concentrated, meaning the model excessively relies on few anchor tokens. This results in generated outputs depending more the summarized information of these anchor tokens, rather than on the integrated information from visual and linguistic tokens in the context in Figure This demonstrates that propagationn of anchor tokens leads to hallucinations. Therefore, we next investigate the properties of propagation entropy Ent(p). We show the next theorem that Ent(p) is directly connected to || WVkl22
Theorem (Propagation Entropy). Let a = I[W T T12 XX and = exp (-ov T-1
The propagation entropy Ent(p) holds that:
a² TT(T-11 Ent(p) = a log(1 (T (T))) 1 1)3
(10)
where Ent(p) represents that lower entropy increases the likelihood of over-propagation of anchor tokens, following unimodal pattern a, and vanishing as || Qk l2 or oa as illustrated in Figure Left). Propagation entropy increases with [[Wokllz up to peak, then decreases, being lowest at extreme values of [Wokll, 2l2. If tr(W)) | is moderate, propagation entropy stays near the peak. To miligate over-propugation of anchor tokens, it is sufficient to control (W2) under a fixed tr(W), constrain the eigenspectrum using the inequality:
| /tr (W2) 2 ||Wqklll
We aim observe the correlation between the eigenspectrum and the hallucinations trig- gered by the over- wer-propagatton of anchor tokens. As illustrated in Figure 3 (Right), p localizes with smaller scales and larger means, which is consistent with the conclusion above. As the eigenvalue scale increases, the mean of p decreases, with smaller scales yielding wider, more variable distribu- tions, while larger scales lead to more stable, smaller values. Conversely, smaller means result in more concentrated p distributions, whereas larger means cause greater dispersion and variability in token propagation. The results indicate that intervening in the scale and mean W-eigenspectrum could be effective way mitigate over peerppropggttion propagation vanishing.
Remark:
A2: How Does Anchor Token Affect the MLLM Models?
Enhance Expressivity: Maximizing (t()) while keeping tr (W2) fixed allows anchor to- kens to maintain complexity and capture subtle feature differences.
Trigger Hallucinations: Polarization of tr (r( with fixed tt(W) reduces propagation tropy, causing over- vve-propaagtton of anchor tokens and triggering ballucinations.

---

Published as a conference paper at ICLR 2025
5 INTERVENING ANCHOR TOKEN PROPAGATION
The previous analysis demonstrates that anchor tokens significantly impact the expressive capabil- ity and hallucination phenomena of MLLMs through the eigenspectrum properties of the attention weight matrix. Specifically, moderate propagation of anchor tokens enhances the model's expres- sivity, while over proppagatnn leads to hallucinations. These findings highlight that controlling the eigenspectrum of the query-key weight matrix (W) can effectively regulate the propagation inten- sity of anchor tokens. Compared to intervening in W = W Wok Σ, we choose to directly adjust the eigenspectrum of W WoK- This is because W introduces the covariance matrix Σ, which remains unchanged during inference, making it sufficient to adjust Wok to flexibly control attention propa- gation. Hence, merely intervening the W Wok helps the model avoid triggers for hallucinations.
To achieve this goal, we propose a versatile plug-and-play decoding strategy, Dynamic Token Prop- agation Mechanism (TAME), to alleviate excessive propagation by dynamically intervening the eigenspectrum variance of the attention weight, thereby reducing hallucinations without relying on complex decoding strategies. Hence, the WQk reparameterized as:
7 Wok = +1+ log(1) + E) WQK. QK.
(11)
where 77 denotes tr(W2 Qk), and controls the eigenspectrum scale of WQk. { is a small constant like 10-6 The logarithm prevents excessive scaling when is near 0, decreasing the total scaling when v) is large and increasing it when 1) is small, thus adjusting the variance of Wok. Our method degrades to original weight when 7 is O. The algorithm is provided in Appendix E
EXPERIMENTS
6.1
SETUP
Models. For our evaluation, we choose four of the most exemplary MLLMs: InstructBLIIP DDii et al. 2023b). MiniGPT-4 (Zhu et 2023), V-- (Liu et al., (024b), and Shikra (Chen et al., 2023a). These MLLM models can be broadly categorized into two groups The first group, com- prising InstructBLIP and MiniGPT-4, utilizes the Q-former (Li et al. 2023b) to effectively bridge the vision and text modalities by representing images with only 32 tokens. In contrast, the second group. which includes VL. and Shikra, employs linear projection layers to align the features of both modalities, requiring a larger number of image tokens 256 or even 576 as input for the MLLMs. Additionally, all these MLLM models incorporate robustly pretrained vision encoder, such as CLIP (Radford et al., (02T)) or EVA (Fang et al.] 2023), alongside a pretrained language model like LLaMA (Touvron et al. 2023a) or Vieuna (Chiang et al. 2023).
Baselines and Benchmark. Since decoding strategies in a training-frree manner, we compare seven decoding methods: Sampling (Top-p=1) Decoding, Greedy Decoding, Visual Contrastive Decoding (VCD) (Leng et al. 22024), Instruction Contrastive Decoding (ICD) (Wang et al. 2024b). Beam Search Decoding (Sutskever) (2144), beam-search-baseee OPERA (Huang et al., [2244, , and SID (Huo) et al. 2024). The proposed TAME can be seamlessly integrated into different decoding strategies, simply replaces the variance of the self-attention Wok parameters of L.M. We evaluate TAME's performance of mitigating hallucinations on both long descriptions and simplified VQA answers. i.e., Caption Hallucination Assessment with Image Relevance (CHAIR) (Rohrbach et al. 2018) and
Polling- based Object Probing Evaluation (POPE) (Li et al. 2023c)).
6.2 QUANTITATIVE RESULTS
CHAIR Evaluation on Hallucinations. The CHAIR metric specialized evaluation tool de- veloped to assess the issue of object hallucination in image captioning tasks. Specifically, CHAIR measures the extent of object hallucination a given image description by calculating the proportion of objects mentioned in the description that are not present in the corresponding ground-truth label set. The evaluation consists of two distinct dimensions: CHAIRs, which operates at the sentence level, and CHAIR,, which operates at the image level. These metrics are represented as Cs and Cr, and their detailed formulations are provided in Appendix C.2 We perform the CHAIR evaluation on the MSCOCO dataset (Lin et al. 2014), which includes over 300,000 images and annotations for 80 object categories. Following the Baseline method, we randomly select 500 images from the valida- tion set of COCO 2014 and prompt variousMI.LMMs with the query "P lease describe this
7

---

Published as a conference paper at ICLR 2025
Table 1: CHAIR hallucination evaluation results four MILMM models. Denote CHAIRS Cs and CHAIRI as C₁. Smaller values indicate less hallucinations. TAME as a plug-and-play method.
New Tokens:
Methods
Cg
Cz
Cs
InstractB1
C2
Shikra Ca
Cy
C5
MiniGIPT4
Ca
HLVA-IS
Cy
Ce
TestruciRL.P
Cr
Cs
Shikm
Cz
Ca
MiniGPT4
Cy
Sampling
16.8
+LAME
47.7
15.9
53.1
21.7
45.4
12.8
30.7
17.9
19.5
7.1
28.7
12:2
25.3
9.5
18.8
6.7
Greedy
09.5
14.1
57.2
15.8
47.1
13.9
35.7
25.5
22.6
7.2
30.0
14.5
22.0
7.0
24.2
$.2
+TAME
47.3
14.1
54.7
14.8
42.0
11.7
33.2
18.4
20.1
28,9
19.2
6.9
46.6
20.2
STAME
45.2
14.0
52.0
14.7
43.1
11.2
29.4
15.8
21.2
7.3
29.2
14,0
17.7
6.9
17.9
5.6
VCD
31.7
15.6
31.0
10.7
48.0
14.0
30.4
14.2
23.6
8.6
30.0
11.2
27.00
10.4
22.0
10.6
MTAME
43.8
14.1
44.6
13.5
24.8
9.9
47.3
22.5
+TAME
44.5
13.6
42.8
13.5
45.9
12.2
28.6
12.7
19.4
7.6
29.4
7.6
25.3
10.2
19.2
7.8
OPERA
46.4
13.0
47.1
12.4
46.4
12.7
26.4
10.7
21.88
8.2
28.4
9.7
22.6
12.8
22.6
6.2
12.2
42.3
12.4
44.8
13.8
28.55
11.7
20.7
76.0
20.8
11.7
23
10.7
+TAME
32.2
9.6
36.5
9.2
35.8
10.0
22.4
8.6
17.5
7.3
21.9
8.4
23.6
10.7
18.5
8.1
H6P)
WPI
HSP)
WPI
SPI
SPI HWP!
S21
HWW
HWR
HSR
FTWR
HWR
ISR
InSTuCILLIP
MinKGPPT44
LLaVA-1.
Stikea
OPERA TAME (Oun))
OPERA
SED
Beam Scarch
Greraly
Figure 4: GPT-4 assisted hallucination evaluation (Zhao et al. 2023 analyze six key metrics: sentences per image (SPI), words per image (WPI), hallucinated sentences per image (HSPI), hallucinated words per image (HWPI), hallucinated sentences ratio (HSR), and hallucinated words ratio (HWR). Note that larger SPI and WPI, smaller HSPI, HWPI, HSR and HWR are better. Larger radar indicates better performance.
results on VG- dataset
image in detail" to generate descriptions. To ensure a fair evaluation, we impose two dif- ferent maximum token limits, as the length of generated sequences can significantly affect CHAIR scores (Cs and Cγ) (Li et al., 2023c). As shown Table [ our TAME obviously surpasses all of baselines decoding methods in both Cs and C₁ metries. Especially LLaVA-1 our method achieves approximately a 27.1% improvement on SID. TAME consistently performs well in both long and short description generation.
GPT-4 Assisted Evaluation. While CHAIR a robust metric for assessing object-existence-leee hallucinations, it is limited in detecting other forms of hallucination, such those involving object attributes, locations, or relationships. HalluBench (Zhao et al., 2023) represents a more advanced benchmark, utilizing detailed object-level descriptions from the VG dataset (Krishna al., 2017) as ground-truth, and relying on GPT-4 to evaluate hallucinations in generated descriptions. In this process, the detailed object-level descriptions are compiled into comprehensive but unordered summary of the image, and GPT-4 is carefully prompted to assess hallucinations in the descriptions generated by MILMs, sentence by sentence. MLLMs are prompted with "P "Please describe this image in detail," with the maximum token limit set to 512.
As illustrated in Figure 4 our TAME method demonstrates a significant reduction the occurrence of hallucinated sentences and words when generating descriptions for each image. Specifically, TAME achieves a 34.3% improvement over greedy decoding in terms of the hallucinated sentence ratio (HSR), and a 12.7% reduction in the number of hallucinated words per image (HWPI) com- pared to OPERA. This suggests that TAME, as a plug-in, effectively mitigates hallucinations issues, potentially by alleviating model biases caused by excessive propagation of anchor tokens. Further- more, we observe a slight reduction in the length the output sequences generated by MLLMs when using TAME, which may be attributed the omission of extraneous hallucinated content.
POPE Evaluation on Hallucinations. The POPE method evaluates hallucination issues in MLLMs, with particular focus object hallucination, similar CHAIR. POPE employs essay-
8

---

Published as a conference paper at ICLR 2025
(A) Propugation Entrpyy Fnd(s)
(B) Fig. spectr seale (r(
(C) CHAIRS
CHAIRS C
k00
6:31
0:00
Layers
Lagers
Controlling scale
Controlling scale T
Figure 5: Ablation study of our proposed TAME. (A) Propagation entropy Ent(p) under varying y. (B) Eigenspectrum scale Ent(p) under varying 7. The shaded areas represent the error bounds at each layer. (C) Comparison of CHAIRS scores between TAME and Baselines across different y. (D) Comparison of CHAIRS scores between TAME and Baselines across different In (C), the baselines refer to the models without TAME when y = 0, while in (D), it corresponds to T = 1, where
rvd = vd. (D) demonstrates that 7 is not relevant to mitigating hallucinations.
question format to prompt the model with queries such as "Is there <object> in the image?" to assess whether the model can accurately identify the presence of a specific object in the image. The evaluation is conducted across three distinct splits: the "random" split, where objects are randomly selected from the entire dataset; the "popular" split, which evaluates the recognition of frequently occurring objects; and the "adversarial" split, which assesses the ability of model to detect objects closely related to those present in the image. We apply the POPE evaluation to four MLLM models, and the average Fl scores are presented in Table 7 While TAME, as plug-in, effectively enhances performance among the tested decoding strategies. Our approach effectively mitigates hallucinations in both longer sequences and short binary classification tasks by dynami- cally adjusting token propagation, reducing over wer-propagation of anchor tokens.
6.3 ABLATION STUDY
Intervening the Eigenspectrum Variance. We aim to investigate the relationship between the eigenspectrum and token propagation dynamics. Figure (B) illustrates how reparameterization
allows us to control the variance of the eigenspectrum. Our empirical analysis shows that the eigenspectrum typically exhibits large scale, and Figure (A) and (B), indicating that when the eigenspectrum is positioned toward the right of the unimodal distribution of propagation entropy, reducing its scale effectively preserves higher propagation entropy throughout the auto-regressive generation process. As result, the model demonstrates reduced hallucination rates, highlighted in Figure
5 (C). This confirms the correlation between the eigenspectrun and hallucinations across various MLLMs, demonstrating that simply adjusting the query-key parameter variance, without incurring additional inference time, can still mitigate hallucinations effectively.
Temperature vd of Self-Attention. We set T as a control parameter to adjust the temperature vd in attention matrix, i.e., TVd. d. This scaling affects the sensitivity of model to inputs. Figure (D) illustrates as increases, CHAIR scores decrease within range, then slightly rise at higher values, but do not perform better than the original baseline. This indicates that while adjusting r influences hallucinations, the effect is not simply due temperature regulation. Instead, TAME effectively reduces hallucinations at specific T values, as shown in Figure ( In other words, the effectiveness of TAME lies in its dynamic intervention in the eigenspectrum variance of the attention weight matrix, rather than merely adjusting the temperature parameter.
Case Analysis. We analyze the attention maps generated by OPERA models to evaluate the impact of TAME on modality alignment in image captioning and visual question answering tasks. Figure 6 illustrates two cases where the attention maps reveal the distribution of attention scores assigned to generated textual tokens within the input image-text sequence during the output generation phase of the MLLM. Our findings show that the OPERA- LLaVA-1.5 model tends to overemphasize the context of the text, which may lead to hallucinations. However, with TAME integrated, the model focuses more on the image, indicating a stronger align- ment between image and text modalities. One possible explanation is that by controlling the prop- agation of anchor tokens and mitigating internal hallucination issues caused by over pro-proppggtion, TAME redirects the MLLM'ss attention, resulting in greater focus the image tokens.
and

---

Published as a conference paper at ICLR 2025
Captioning Task
VQA Task
Proegf:
OPERA TAME (Durs)
OPERA
thene driing table the imaech
OPERA- (Ound)
OPERA
wable the iage the
the segge oie kutn suaurnd alts.
bolking commellesss
OPTRA
Ganerctdd hoire
OPERA
Generated
15440 Oumn
Weet Here
Gonercted tolane
-TAMlsbei Verl Ioan
Sanartte folkan
Reg atggiigl Anrotonice
O @
Figure 6: Comparison of attention map between TAME and OPERA-LLaVAS VA-I. 1.5 at different tasks. The blue box region is labeled with the image attentions that can be improved by ours TAME.
7 RELATED WORK
Multimodal Large Language Foundation Models. Recent advancements in computational re- sources have significantly boosted research into large-scale foundational models integrated with multi-modal learning (Xu et al. 2024a; Zheng et al., 2024 Hu et al., 2024b a Zhang et al. 20224b] Peng et al., 2024b]a). Leveraging open-
large language models like LLaMA (Touvron al,,
2023a b) and Vicuna (Chiang ct al. 2023), MLLMs (Huang et al. 2023 Chen et al. [023bb Young
ct al.j 2024; Elhoushi et al.) 2024 Ye et al., 2024 Li et al. 2024bb Zhang et al. 2024a Liu et al,
2025a Wei & Zhang (224)) can understand and generate wide range of content more effectively by combining information from multiple modalities, such as text, images, and audio. Models like CLIP and BLIP align text and image features well, while LLaVA (Liu et al.) 2024c), InstructBLIP (Dai ct al. 2023b) and MiniGPT-4 (Zhu et al.) 2023) take this further, enabling users to interact with these systems using images and text prompts. However, they suffer from severe hallucination problems.
Decoding Strategy in LLMs. The hallucination (Ji et al. 2023
Liu et al., 2023b 2024a) in
MLLMs refers the case where the generated text answer does not reflect the true contents of the provided images but rather relies on the internal knowledge of the models. Selecting decoding strategies in language models crucial, as determines how models generate text. Top-k sam- pling decoding (Fan et al. 2018) selects from the top-k most probable tokens, promoting diversity but occasionally resulting in less coherent text. Recent studies propose various decoding strate- gies (Chuang et al. [2023 Kim al. 2024 Li et al., 2025 Huang et al. 2024| Wang al. 20244b Chen et al. 2024; Leng et al. 2024). They emphasize that hallucinations stem from an over-reliance on anchor tokens, causing subsequent generations to focus more summarizing the anchor token information rather than utilizing the full context of preceding visual and linguistic tokens. However, these methods lack analysis of anchor tokens in the decoding strategy and often improve perfor- mance at the cost of doubling or even tripling inference time. In this paper, our approach analyzes the relationship between hallucinations and anchor tokens, effectively controlling the generation of hallucination--riggeeee anchor tokens without incurring additional training, data, or inference time.
CONCLUSION & LIMITATION
In this paper, we explore the causes of hallucinations in MLLMs by analyzing the propagation pat- terns of anchor tokens within the attention mechanism. Our findings reveal that over-propagation of anchor tokens, driven by polarized variance in the eigenspectrum of the QK-parameters, leads to hallucinations by causing the model to neglect visual information. To address this, we propose boug-and-pla decoding strategy, Dynamic Token Propagation Mechanism (TAME), which dy- namically intervenes the eigenspectrum variance to mitigate excessive propagation. Our approach effectively alleviates hallucinations without complex decoding strategies and incurring extra infer- ence time. Experiments show superiority in reducing hallucinations on various MLLMs.
Limitation: We clarify the limitations of our proposed TAME: (i): TAME cannot solve all types of hallucination phenomena in MLLMs. This is understandable because our scheme does not require any additional costs or modifications to the MLLM structure, and therefore has certain limitations in dealing with hallucination problems. (ti): Why anchor tokens affect model performance remains elusive. Although the proper propagation of anchor tokens related to avoiding rank collapse and reducing hallucinations, we need extra effort to fully understand this mechanism in MI..Ms.
10

---

Published as a conference paper at ICLR 2025
REFERENCES
Jinze Bai, Shuai Bai, Shusheng Yang, Shijie Wang, Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, and Jingren Zhou. Qwen-vl: A frontier large vision-language model with versatile abilities. arXiv preprint arXiv:2308. 00008.//900 /2966, 2023.
Han Bao, Ryuichiro Hataya, and Ryo Karakida. Self-attention networks localize when qk- eigenspectrum concentrates. arXiv preprint wrXiv:2402.02098, 2024.
Kevin Black, Michael Janner, Yilun Du, llya Kostrikov, and Sergey Levine. Training diffusion models with reinforcement learning. arXiv preprint arXiv:2305. 2023.
Nicolas Bouulager-Lowwaddwssk Yoshua Bengio, and Pascal Vincent. Audio chord recognition with recurrent neural networks. ISMIR, pp. 335-340. Curitiba, 2013.
Keqin Chen, Zhao Zhang, Weili Zeng, Richong Zhang, Feng Zhu, and Rui Zhao. Shikra: Unleashing multimodal Ilm's referential dialogue magic. arXiv preprint arXiv:2306. 15/95, 2023a.
Lin Chen, Jisong Li, Xiaoyi Dong, Pan Zhang, Conghui He, Jiaqi Wang, Feng Zhao, and Dahua Lin. Sharegpt4v: Improving large multi-modal models with better captions. arXiv preprint arXiv:2311.1299, 2023b.
Zhaorun Chen, Zhuokai Zhao, Hongyin Luo, Huaxiu Yao, Bo Li, and Jiawei Zhou. Hale: Object hallucination reduction via adaptive focal-contrast decoding. arXiv preprint arXiv:2403.00425. 2024.
Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph E Gonzalez, et al. Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality. See hilps://vicuna. Imsys. org (accessed 14 April 2023), 2(3):6, 2023.
Yung-Sung Chuang, Yujia Xie, Hongyin Luo, Yoon Kim, James Glass, and Pengcheng He. Dola: Decoding by contrasting layers improves factuality in large language models. arXiv preprint arXiv:2309. 03883, 2023.
Wenliang Dai, Junnan Li, Dongxu Li, Anthony Meng Huat Tiong, Junqi Zhao Weisheng Wang, Boyang Li, Pascale Fung, and Steven Hoi. Instructblip: Towards general-purpose vision-language models with instruction tuning, 2023a.
Wenliang Dai, Junnan Li, Dongxu Li, Anthony Meng Huat Tiong, Junqi Zhao, Weisheng Wang, Boyang Li, Pascale Fung, and Steven Hoi. Instructblip: Towards purposs vision-language models with instruction tuning, 2023b. Utthttps::// http:///axxvvv... org/ 2005//500000300050000000 06500
Xiaoyi Dong, Pan Zhang, Yuhang Zang. Yuhang Cao, Bin Wang. Linke Ouyang, Xilin Wei, Songyang Zhang, Haodong Duan, Maosong Cao, et al. Intemlnn-xcomposer2 Mastering free- form text-image composition and comprehension in vision-language large model. arXiv preprint arXiv:2401. 16420, 2024.
Yihe Dong, Jean-Baptiste Cordonnier, and Andreas Loukas. Attention is not all you need: Pure attention loses rank doubly exponentially with depth. In International Conference on Machine Learning. pp.
PMLR, 2021.
Mostafa Elhoushi, Akshat Shrivastava, Diana Liskovich, Basil Hosmer, Bram Wasti, Liangzhen Lai, Anas Mahmoud, Bilge Acun, Saurabh Agarwal, Ahmed Roman, et al. Layer skip: Enabling early exit inference and self-speculative decoding. arXiv preprint arXiv:2404. 16710, 2024.
Angela Fan, Mike Lewis, and Yann Dauphin. Hierarchical neural story generation. arXiv preprint arXiv:1805.00 04833, 2018.
Yuxin Fang. Wen Wang, Binhui Xie, Quan Sun, Ledell Wu, Xinggang Wang, Tiejun Huang, Xinlong Wang, and Yue Cao. Eva: Exploring the limits of masked visual representation learning at scale. In Proceedings of the IEEE/CVE Conference on Computer Vision and Pattern Recognition, pp- 19358-19369 2023.
11

---

Published as a conference paper at ICLR 2025
Alessandro Favero, Luca Zancato, Matthew Trager, Siddharth Choudhary. Pramuditha Perera, Alessandro Achille, Ashwin Swaminathan, and Stefano Soatto. Multi-modal hallucination con- trol by visual information grounding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 14303-14312 14312, 2024.
Chaoyou Fu, Peixian Chen, Yunhang Shen, Yulei Qin, Mengdan Zhang, Xu Lin, Jinrui Yang. Xiawu Zheng, Ke Li, Xing Sun, et al. Mme: A comprehensive evaluation benchmark for multimodal large language models. arXiv preprint arXiv:2306. 13394, 2023.
Zigang Geng, Binxin Yang, Tiankai Hang, Chen Li, Shuyang Gu, Ting Zhang, Jianmin Bao, Zheng Zhang, Houqiang Li, Han Hu, et al. Instructdiffusion: generalist modeling interface for vision tasks. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 22709-12720 2024.
Borjan Geshkovski, Cyril Letrouit, Yury Polyanskiy, and Philippe Rigollet. The emergence of clus- ters in self attention dynamics. Advances in Neural Information Processing Systems, 36, 2024.
Alex Graves. Sequence transduction with recurrent neural networks. arXiv:1211.3711, 2012.
arXiv preprint
Danna Gurari, Qing Li, Abigale Stangl, Anhong Guo, Chi Lin, Kristen Grauman, Jiebo Luo, and Jeffrey P Bigham. Vizwiz grand challenge: Answering visual questions from blind people. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 3608-3617, 2018.
Ming Hu, Peng Xia, Lin Wang. Siyuan Yan, Feilong Tang, Zhongxing Xu, Yimin Luo, Kaimin Song, Jurgen Leitner, Xuelian Cheng, et al. Ophnet: A large-scale video benehmark for ophthalmic surgical workflow understanding. In European Conference Computer Vision, pp. 481-500. Springer, 2024a.
Ming Hu, Kun Yuan, Yaling Shen, Feilong Tang, Xiaohao Xu, Lin Zhou, Wei Li, Ying Chen, Zhongxing Xu, Zelin Peng, et al. Ophelip: Hierarchical reureeed learning for oph- thalmic surgical video-language pretraining. arXiv preprint arXiv:2411.15422 2024b.
Qidong Huang, Xiaoyi Dong, Dongdong Chen, Weiming Zhang, Feifei Wang, Gang Hua, and Neng- hai Yu. Diversity- aware meta visual prompting. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pp. 10878-10887, 2023.
aware
Qidong Huang, Xiaoyi Dong, Pan Zhang, Bin Wang, Conghui He, Jiaqi Wang. Dahua Lin, Weiming Zhang, and Nenghai Yu. Opera: Alleviating hallucination in multi-modal large language models via over-trust penalty and datrospectionaalloaate allocation. In Proceedings of the IEEE/CVE Conference Computer Vision and Pattern Recognition, pp. 13418- 13418-13427, 2024.
Drew A Hudson and Christopher D Manning. Gqa: A new dataset for real-world visual reasoning and compositional question answering. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 6700-6700 2019.
Fushuo Huo, Wenchao Xu, Zhong Zhang, Haozhao Wang. Zhicheng Chen, and Peilin Zhao. Self-introspective decoding: Alleviating hallucinations for large vision-language models. arXiv preprint arXiv:2408.020.0 2024.
Ziwei Ji, Nayeon Lee, Rita Frieske, Tiezheng Yu, Dan Su, Yan Xu, Etsuko Ishii, Ye Jin Bang. Andrea Madotto, and Pascale Fung Survey of hallucination in natural language generation. ACM Computing Surveys, 55(12):1-38, 2023.
Chaoya Jiang, Haiyang Xu, Mengfan Dong, Jiaxing Chen, Wei Ye, Ming Yan, Qinghao Ye, Ji Zhang, Fei Huang, and Shikun Zhang. Hallucination augmented contrastive learning for multimodal large language model. In Proceedings of the IEEE/CVF Conference on Computer Vision and Patlern Recognition, pp. 27036-27046, 2024.
Jean Kaddour. The minipile challenge for data-efficient language models, 2023.
12

---

Published as a conference paper at ICLR 2025
Vladimir Karpukhin, Barlas Oguz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. Dense passage retrieval for open-domain question answering. arXiv preprint arXiv:2004.04900 04906, 2020.
Taehyeon Kim, Joonkee Kim, Gihun Lee, and Se- Young Yun. Instructive decoding: Instruction- tuned large language models are self-refiner from noisy instructions. In The Twelfth International Conference on Learning Representations, 2024.
Ranjay Krishna, Yuke Zhu, Oliver Groth, Justin Johnson, Kenji Hata, Joshua Kravitz, Stephanie Chen, Yannis Kalantidis, Li-Jia Li, David A Shamma, et al. Visual genome: Connecting lan- guage and vision using crowdsourced dense image annotations. International jourraaainddouurall of computer vision, 123:32-73, 2017.
Xin Lai, Zhuotao Tian, Yukang Chen, Yanwei Li, Yuhui Yuan, Shu Liu, and Jiaya Jia. Lisa: Rea- soning segmentation via large language model. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9579-9589, 2024.
Sicong Leng, Hang Zhang, Guanzheng Chen, Xin Li, Shijian Lu, Chunyan Miao, and Lidong Bing. Mitigating object hallucinations large vision-language models through visual contrastive de- coding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recogni- tion, pp. 13872 13882, 2024.
Brian Lester, Rami Al-Rfou, and Noah Constant. The power of scale for parameter-efficient prompt tuning. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Pro- cessing, pp. 1044-3059, 2021.
Bo Li, Yuanhan Zhang, Dong Guo, Renrui Zhang, Feng Li, Hao Zhang, Kaichen Zhang, Yanwei Li, Ziwei Liu, and Chunyuan Li. Llava-onevision: Easy visual task transfer. arXiv preprint arXiv:2408. 03326, 2024a.
Bohao Li, Rui Wang, Guangzhi Wang, Yuying Ge, Yixiao Ge, and Ying Shan. marking multimodal iims with generative comprehension. arXiv preprint arXiv:2307.16125, 2023a.
Bench-
Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. In International conference on machine learning, pp. 19730-19742. PMLR, 2023b.
Yanwei Li, Yuechen Zhang, Chengyao Wang, Zhisheng Zhong, Yixin Chen, Ruihang Chu, Shaoteng Liu, and Jiaya Jia. Mini-gemini: Mining the potential of multi-modalig vision language models. arXiv preprint arXiv:2403. 18814, 2024b.
Yifan Li, Yifan Du, Kun Zhou, Jinpeng Wang, Wayne Xin Zhao, and Ji-Rong Wen. Evaluating object hallucination in large vasion-language models. arXiv preprint S0W:23055.0355. 2023c.
Yingcong Li, Yixiao Huang, Muhammed E Ildiz, Ankit Singh Rawat, and Samet Oymak. Mechanics of next token prediction with self-attention. International Conference on Artificial Intelligence and Statistics, pp. 685 6933.69..6 PMLR, 2024c.
Yulong Li, Yuxuan Zhang, Feilong Tang, Mian Zhou, Zhixiang Lu, Haochen Xue, Yifang Wang, Kang Dang, and Jionglong Su. Beyond words: Aurallim and signmst-c for precise sign language production and bidirectional accessibility. arXiv preprint arXiv:2501.00765, 2025.
Tsung-Y Yi Lin, Michael Maire, Serge Belongie, James Hays, Pictro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco: Common objects context. In Computer Vision-ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part V 13, pp. 740-755. Springer, 2014.
Chengzhi Liu, Zile Huang. Zhe Chen, Feilong Tang, Yu Tian, Zhongxing Xu, Zihong Luo, Yalin Zheng, and Yanda Meng. Incomplete modality disentangled representation for ophthalmic disease grading and diagnosis. arXiv preprint arXiv:2502. 1724, 2025a.
Fuxiao Liu, Kevin Lin, Linjie Li, Jianfeng Wang, Yaser Yacoob, and Lijuan Wang. Aligning large multi-modal model with robust instruction tuning. arXiv preprint arXiv:2306. 14565, 2023a.
13

---

Published as a conference paper at ICLR 2025
Fuxiao Liu, Kevin Lin, Linjie Li, Jianfeng Wang, Yaser Yacoob, and Lijuan Wang. Mitigating hal- lucination large multi-modal models via robust instruction tuning. The Twelfth International Conference Learning Representations, 2023b.
Hanchao Liu, Wenyuan Xue, Yifei Chen, Dapeng Chen, Xiutian Zhao, Ke Wang, Liping Hou, Rongjun Li, and Wei Peng. A survey on hallucination in large vision-language models. arXiv preprint arXiv:2402. 00253, 2024a.
Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. Improved baselines with visual instruction tuning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recogni- tion, Pp. 02296-26306 2024b.
Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning, Advances in neural information processing systems, 36, 2024c.
Yexin Liu,, Zhengyang Liang, Yueze Wang, Muyang He, Jian Li, and Bo Zhao. Seeing clearly, an- swering incorrectly: A multimodal robustness benchmark for evaluating milms on leading ques- tions. arXiv preprint arXiv:2406.106 2024d.
Yuan Liu, Haodong Duan, Yuanhan Zhang, Bo Li, Songyang Zhang, Wangbo Zhao, Yike Yuan, Jiaqi Wang, Conghui He, Ziwei Liu, et al. Mmbench: your multi-modal model an all-around player? arXiv preprint Niiv:2307.06281. 2023c.
Yuan Liu, Haodong Duan, Yuanhan Zhang, Bo Li, Songyang Zhang, Wangbo Zhao, Yike Yuan, Jiaqi Wang, Conghui He, Ziwei Liu, et al. Mmbench: Is your multi-modal model an all-around player? In European Conference on Computer Vision, pp. 216-233. Springer. 2025b.
Andre Martins and Ramon Astudillo. From softmax to sparsemax: A sparse model of attention and multi-label classification. International conference on machine learning, pp. 1614-1623. PMLR, 2016.
Stephen Merity, Caiming Xiong, James Bradbury, and Richard Socher. Pointer sentinel mixture models. arXiv preprint 1609,0780 2016.
Jianhui Pang, Fanghua Ye, Derek F Wong. and Longyue Wang. Anchor-based large language mod- els. arXiv preprint arXiv:2402.0766 2024.
Zelin Peng, Zhengqin Xu, Zhilin Zeng, Yaoming Wang, Lingxi Xie, Qi Tian, and Wei Shen. Parameter-efficient fine-tuning in hyperspherical space for open-vocabulary semantic segmen- tation. arXiv preprint arXiv:2405. 18840, 2024a.
Zelin Peng. Zhengqin Xu, Zhilin Zeng, Lingxi Xie, Tian, and Wei Shen. Parameter efficient fine- tuning via cross block orchestration for segment anything model. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 3743-3752 2024b.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pp- 8748-882 8763. PMLR, 2021.
Anna Rohrbach, Lisa Anne Hendricks, Kaylee Burns, Trevor Darrell, and Kate Saenko. Object hallucination in image captioning. arXiv preprint arXiv:1809.02I11 2018.
Jiashuo Sun, Jihai Zhang, Yucheng Zhou, Zhaochen Su, Xiaoye Qu, and Yu Cheng. Surf: Teach- ing large -anguage models to selectively utilize retrieved information. arXiv preprint 240.:23000..488.. 2024.
I Sutskever. Sequence to sequence learning with neural networks. arXiv preprint 10X 2014.
Feilong Tang, Zhongxing Xu, Qiming Huang, Jinfeng Wang, Xianxu Hou, Jionglong Su, and Jingxin Liu. Duat: Dual-aggregation transformer network for medical image segmentation. In PRCV, 2023.
14

---

Published as a conference paper at ICLR 2025
Feilong Tang, Zhongxing Xu, Zhaojun Qu, Wei Feng, Xingjian Jiang, and Zongyuan Ge. Hunting attributes: Context prototype-awac aware learning for weakly supervised semantic segmentation. In CVPR, 2024.
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie- Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models. arXiv preprint arXiv: 139771, 2023a.
Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaci, Niko- lay Bashlykov, Soumya Batra, Prajiwal Bhargava, Shruti Bhosale, et al. Llama 2: Open founda- tion and fine-tuned chat models. arXiv preprint arXfv:2307.09288. 2023b.
Bin Wang, Fan Wu, Xiao Han, Jiahui Peng, Huaping Zhong, Pan Zhang, Xiaoyi Dong, Weijia Li, Wei Li, Jiaqi Wang, et Vige: Visual instruction generation and correction. In AAAI, 2024a.
Lean Wang. Lei Li, Damai Dai, Deli Chen, Hao Zhou, Fandong Meng. Jie Zhou, and Xu Sun. Label words are anchors: An information flow perspective for understanding in in-context learning. arXiv preprint arXiv:2305. /4160, 2023.
Xintong Wang, Jingheng Pan, Liang Ding, and Chris Biemann. Mitigating hallucinations in large vision-language models with instruction contrastive decoding. arXiv preprint 2881::2403.18755, 2024b.
Jinfeng Wei and Xiaofeng Zhang. Dopra: Decoding over-accumulation penalization and re-
allocation in specific weighting layer. In Proceedings of the 32nd ACM International Conference on Multimedia, pp. 7065-7074, 2024.
Kai Wu, Boyuan Jiang. Zhengkai Jiang. Qingdong He, Donghao Luo, Shengzhi Wang, Qingwen Liu, and Chengjie Wang. Noiseboost: Alleviating hallucination with noise perturbation for mul- timodal large language models. arXiv preprint arXiv:2405. 20081, 2024.
Xinyu Xiong, Zihuang Wu, Shuangyi Tan, Wenxue Li, Feilong Tang, Ying Chen, Siying Li, Jie Ma, and Guanbin Li. Sam2-unet: Segment anything 2 makes strong encoder for natural and medical image segmentation. arXiv preprint arXiv:2408. 08870, 2024.
Zhongxing Xu, Feilong Tang, Zhe Chen, Yingxue Su, Zhiyi Zhao, Ge Zhang, Jionglong Su, and Zongyuan Ge. Toward modality gap: Vision prototype learning for weakly-supervised semantic segmentation with clip. arXiv preprint arXiv:2412.1963 2024a.
Zhongxing Xu, Feilong Tang, Zhe Chen, Zheng Zhou, Weishan Wu, Yuyao Yang, Yu Liang, Jiyu Jiang, Xuyue Cai, and Jionglong Su. Polyp-mamba: Polyp segmentation with visual mamba. In MICCAI, 2024b.
Haochen Xue, Feilong Tang, Ming Hu, Yexin Liu, Qidong Huang, Yulong Li, Chengzhi Liu, Zhongxing Xu, Chong Zhang, Chun-Mei Feng. et al. Mmre: A large-scale benchmark for understanding multimodal large language model in real-world conversation. arXiv preprint arXiv:2502. 19903 2025.
Qinghao Ye, Haiyang Xu, Jiabo Ye, Ming Yan, Anwen Hu, Haowei Liu, Qi Qian, Ji Zhang, and Fei Huang. mplug-ow12: Revolutionizing multi-modal large language model with modality collabo- ration. In Proceedings the the IEEF/VF Conference on Computer Vision and Pattern Recognition, pp. 13040-13051, 2024.
Shukang Yin, Chaoyou Fu, Sirui Zhao, Tong Xu, Hao Wang. Dianbo Sui, Yunhang Shen, Ke Li, Xing Sun, and Enhong Chen. Woodpecker: Hallucination correction for multimodal large lan- guage models. arXiv preprint 1rXiv:2310.16044 2023.
Alex Young, Bei Chen, Chao Li, Chengen Huang. Ge Zhang, Guanwei Zhang, Heng Li, Jiangcheng Zhu, Jianqun Chen, Jing Chang, et al. Yi: Open foundation models by 01. ai. arXiv preprint arXiv:2403.. 04652, 2403.0462,
2024.
15

---

Published as a conference paper at ICLR 2025
Tianyu Yu. Yuan Yao, Haoye Zhang, Taiwen He, Yifeng Han, Ganqu Cui, Jinyi Hu, Zhiyuan Liu, Hai-Tao Zheng, Maosong Sun, et al. Rlhf-v: Towards trustworthy milms via behavior alignment from fine-grained grained correctional human feedback. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 13807-13816, 2024.
Shuangfei Zhai, Tatiana Likhomanenko, Etai Littwin, Dan Busbridge, Jason Ramapuram, Yizhe Zhang, Jiatao Gu, and Joshua M Susskind. Stabilizing transformer training by preventing attention entropy collapse. In International Conference on Machine Learning, pp. 40770-40880 PMLR, 2023.
Pan Zhang, Xiaoyi Dong Bin Wang. Yuhang Cao, Chao Xu, Linke Ouyang, Zhiyuan Zhao, Shuan- grui Ding, Songyang Zhang, Haodong Duan, Hang Yan, et al. Internlm-xcompont A vision- language large model for advanced text-image comprehension and composition. arXiv preprint arXiv:2309. 15/12, 2023a.
Pan Zhang, Xiaoyi Dong, Yuhang Zang, Yuhang Cao, Rui Qian, Lin Chen, Qipeng Guo, Haodong Duan, Bin Wang. Linke Ouyang, Songyang Zhang, Wenwei Zhang, Yining Li, Yang Gao, Peng Sun, Xinyue Zhang, Wei Li, Jingwen Li. Wenhai Wang. Hang Yan, Conghui He, Xingcheng Zhang, Kai Chen, Jifeng Dai, Yu Qiao, Dahua Lin, and Jiaqi Wang. dneemllm-composerr versatile large vision language model supporting long-contextual input and output. arXiv preprint arXiv:2407.03330, 2024a.
A
Shilong Zhang, Peize Sun, Shoufa Chen, Min Xiao, Wenqi Shao, Wenwei Zhang, Yu Liu, Kai Chen, and Ping Luo. Gpt4roi: Instruction tuning large language model on region-of-interest. arXiv preprint ar arXiv:2307.03600 2023b.
Xiaofeng Zhang, Fanshuo Zeng, Yihao Quan, Zheng Hui, and Jiawei Yao. Enhancing mul- timodal large language models complex reason via similarity computation. arXiv preprint arXiv:2412.09811 09817. 2024b.
Xinqiao Zhao, Feilong Tang, Xiaoyang Wang, and Jimin Xiao. Sfc: Shared feature calibration in weakly supervised semantic segmentation. In AAAI, 2024.
Zhiyuan Zhao, Bin Wang, Linke Ouyang, Xiaoyi Dong, Jiaqi Wang, and Conghui He. Beyond hal- lucinations: Enhancing Ivlms through ballucination-awwe aware direct preference optimization. arXiv preprint arXiv:2311 16839, 2023.
Mingzhe Zheng, Yongqi Xu, Haojian Huang, Xuran Ma, Yexin Liu, Wenjie Shu, Yatian Pang, Fei- long Tang, Qifeng Chen, Harry Yang, et al. Videogen-of-thonee A collaborative framework for multi-shot video generation. arXiv preprint arXiv:2412.002259, 2024.
Yiyang Zhou, Chenhang Cui, Jachong Yoon, Linjun Zhang, Zhun Deng, Chelsea Finn, Mohit Bansal, and Huaxiu Yao. Analyzing and mitigating object hallucination large vision-language models. arXiv preprint arXiv:2310.00754, 2023.
Yiyang Zhou, Zhiyuan Fan, Dongjie Cheng, Sihan Yang, Zhaorun Chen, Chenhang Cui, Xiyao Wang, Yun Li, Linjun Zhang, and Huaxiu Yao. Calibrated self-rewarding vision language models. arXiv preprint arXiv:2405. 240::.465..22 14622, 2024.
Deyao Zhu, Jun Chen, Xiaoqian Shen, Xiang Li, and Mohamed Elhoseiny. Minigpt-4: En hancing vision-language understanding with advanced large language models. arXiv preprint Stt::2304.10592 2023.
16

---

Published as a conference paper at ICLR 2025
APPENDIX
A ATTENTION ALLOCATION
We compute and present the average attention values from the Generated token to both visual tokens and system prompt tokens across different Transformer layers in the pre-trained LLaVA-1.5-7B model. As shown in Figure (light purple trends), attention to visual tokens decreases progressively the layers deepen. In the shallow layers, the attention distribution is more balanced, with the Generated tokens focusing on both previous output tokens and visual tokens. However, in the deeper layers, the model shifts its focus primarily the system prompt tokens, reducing attention visual tokens.
These observations indicate significant redundancy in the visual tokens, particularly in deeper lay- ers, where they contribute less to the output of model. The shift in attention towards the system prompt suggests that as the model processes information through deeper layers, it relies more on the structured prompts than the visual input, revealing potential inefficiencies in visual token utilization.
Attention Allocation
10a
System Tokens
Average Attention
Visual Tokens
Instruction Tokens
100
Generated Tokens
10-1
10-2
10
20
25
30
Transformer Layers
Figure 7: Comparison between the average attention allocation for different types of token across transformer layers.
B DETAILS ABOUT BASELINE
Sampling- Sampling-Decoding: Sampling Decoding generates the next words by randomly selecting from the output probability distribution. Specifically, Top-k sampling (Fan al. 2018) chooses from the top-k most probable tokens, which introduces diversity into the generated text but can sometimes result less coherent outputs.
Greedy-Decoding: The key distinction between the Screedy-Decoding strategy and the "Origi- nal" strategy lies the decoding method used during the generation of image descriptions. In the "Grredy--ecoddigg approach, the model opts for greedy decoding instead of sampling, aiming to produce the most deterministic and consistent output. This strategy is employed to examine the possible relationship between the occurrence of hallucinations and the sampling technique.
Beam-Search: Beam-Searcht Beam Search Gooulanger-Lewandorn
et al. 2013 Graves 2012 Sutskever 2014) is an advanced decoding technique that maintains fixed number of hypotheses at each step. allowing for the exploration of multiple potential paths to identify a more optimal sequence. Specifically, with designated beam size Nbeams Beam Search retains Nbeam candidate sequences, each represented by a decoded sequence x Neeum accompanied by a corresponding beam score. When generating the token 3t, each candidate hypothesis selects Nbcam possible tokens based
17

---

Published as a conference paper at ICLR 2025
on the top Nbeam probabilities from the logits. Finally, the decoding process outputs the hypothesis that achieves the highest beam score.
VCD: Vision Contrastive Decoding(VCDD manipulates vision inputs by introducing Gaussian noise or directly ablating specific visual features to amplify language priors. By adding Gaussian noise, VCD subtly perturbs the visual data, making the model less confident in relying solety on visual cues and encouraging it to integrate contextual language information more effectively. Direct ablation involves removing or masking certain parts of the visual input, forcing the model to infer missing information based on linguistic context rather than defaulting to learned statistical biases. These manipulations create a contrast between the original and distorted inputs, enabling VCD identify and suppress tendencies toward ballucination.
ICD: Instruction Contrastive Decoding (ICD) methods introduce various types of noisy instructions, including random words, contradictory directives, truncated words, and irrelevant information. For instance, adding random words disrupts instruction coherence, making it difficult for the model to understand the intended meaning. Contradictory directives force the model to depend more on accurate visual inputs instead of relying solely on learned language patterns. Similarly, truncated or incomplete instructions reduce clarity, compelling the model to infer missing information from context.
OPERA: Over-trust Penalty and Retrospection -alloctionnllocaatioo strategy (OPERA) penalizes the 'Over- Trust Logit'- - -a measure the model's over-reliance on certain tokens- in the beam score. By applying this penalty, OPERA effectively alleviates the aggregation patterns that contribute to hal- lucinations. This adjustment forces the model to distribute its attention more evenly across rele- vant tokens, including those representing actual visual content, thereby enhancing the accuracy of the generated descriptions. Although effective, decoding-based methods require iterative decoding, which incurs computational burden and impedes MILM's deployment on personal devices.
SID: Self-snnrrs Decoding (SID) is a simple and effective method designed to reduce hal- lucination issues Large Vision-Language Models (LVLMs) without relying on additional data, knowledge, or training. SID employs a Context and Text-aware Token Selection (CT2S) strategy, which retains only unimportant visual tokens in the early layers of the model, thereby adaptively enhancing text- base information during the autoregressive decoding process. This strategy en- sures that multimodal know ledgledge guides the model to generate contextually relevant outputs at early stages, rather than aimlessly generating hallucinated content.
C DETAILS OF EVALUATION
C.I DETAILS OF GPT-4 EVALUATION
We generally adopt the GPT-4 evaluation framework introduced in HalluBench (Zhao et and apply it to the VG dataset. Each image in the VG (Krishna et al. (017)) dataset includes compre- hensive ground- -rrth descriptions of all visible objects. Since GPT-4 cannot directly process image data, we incorporate these ground- truth descriptions into the input prompt to help GPT-4 understand the image content. Then, when provided with description generated by an MILM in response to the prompt "Pleass descri be this image in detail, 1, GPT-4 is tasked with eval- uating whether each sentence in the MLLM's description contains hallucinated information. This evaluation is highly stringent, with GPT-4 marking any descriptions from the MLLM as halluci- nations if they differ from the ground-truth details regarding quantity, color, location, activity, or direction.
2023
Metrics. There are six metrics considered, which include:
The number of sentences per image (SPI). It reflects the detailedness of MLLM's description at the sentence level.
The number of words per image (WPI). It reflects the detailedness of MLLM'S description the word level.
The number of hallucinated sentences per image (HSPI). It reveals the hallucination degree of MLLM's description at the sentence level. Any sentences that contain hallucinated contents are taken into calculation.
18

---

Published as a conference paper at ICLR 2025
The number of hallucinated words per image (HWPI). It reveals the hallucination degree of MLLM's description at the word level. Any words related with hallucinated contents are taken into calculation.
The ratio of hallucinated sentences (HSR). The average ratio of hallucinated sentences in all sen- tences of MILM's 's descriptions different images.
The ratio of hallucinated words (HWR). The average ratio of hallucinated words in all words of MLLM's descriptions on different images.
Prompt. As shown in Table 2 our adopted GPT-4 prompt generally based on HalluBench ZZao
al. 2023
C.2 DETAIL OF EVALUATION SCORE
Denoted as Cs and C₁ these two variants can be formulated as the average results of { hallucinated objects } }1
Cs { all mentioned objects } } Cr 1 captions w/ hallucinated objects }
{ all captions
where the integration of CHAIRs and CHAIR, enables a thorough and detailed analysis of object hallucination issues in image captioning.
C.3 COMPARISON OF TOKEN PROPAGATION PROBABILITY BETWEEN HALLUCINATORY AND NON-HALLLCCINTTRY CAPTIONS
The objects in this experiment are based on the 80 object labels annotated in (Rohrbach al. 2018) from the COCO dataset, and the image descriptions are generated by MiniGPT-4 based on inference results from 5000 images in the COCO 2014 train dataset.
C.4 DETAILS GPT-4v EVALUATION
Following YYn et al. 2023). we perform dual evaluation on GPT-4V(ision) comparing Beam search decoding with our proposed TAME decoding. Given a trained MLLM model and an im- age, we generate two descriptions using the prompt "Please describe this image in detail," one with Beam search and the other with TAME, We then use the prompt shown in Table 10 to ask GPT-4V to rate these two descriptions based on the image on scale from 0 to 10, focusing on two aspects: Accuracy and Detailedness. The accuracy reflects the consistency between the description and the given image. If GPT-4V thinks any content in this description is inconsistent with the given image, namely higher hallucinations, will get lower score. The detailedness reflects the degree of expressive ability, i.e., how comprehensive does the description characterize the image. The prompt used for GPT-4V, listed in T instructs it to ignore any bias from the sequential or- der and to pay special attention objects mentioned in the MLLM's descriptions that do not appear in the image, including incorrect colors, positions, or relationships. GPT-4V thoroughly analyzes the MLLM's descriptions, leveraging its strong, human-like capabilities.
D DISCUSSION
D.1 TOKEN PROPAGATION PROBABILITY AGAINST ATTENTION SCORE
The distinction between Token Propagation Probability and the Attention Score in transformer mod- els is pivotal for understanding token interactions within the self-attention mechanism of large lan- guage models (LLMs).
Attention scores, fundamental to the scaled dot-product attention mechanism, measure the similarity between query and key vectors, dictating how much one token should focus on another. These scores calculated for every token pair and normalized via the softmax function, yielding probabilistic distribution over attention weights that sum to 1. Although all tokens participate in the attention calculation, those with lower attention weights inherently contribute less to the decisions of model.
On the other hand, token propagation probability estimates the likelihood that a token's information influences the next layer in the network. Unlike attention scores, which do not have inherent
19

---

Published as a conference paper at ICLR 2025
Table 2: The prompt used for GPT-4 evaluation.
GPT-4 Prompt
Please help me judge the comment of this image is hallucination or correct. I will give you list of region description of image. The format is [x1, yl, x2, y2]: region descrip- tion, where [x1, y1, x2, y2] is the bounding box of the region. Highly overlapping bounding boxes may refer to the same object. This is the ground truth information of the image. Your judgement should base on this information. However, this information only describes the objects in the region of image, it cannot describe the subjective part of the image, e.g., atmosphere, style, emotion. In that case, you can return "Cannot judge".
Also, I will give you a list of comments of the image for you to judge if is hallucination. Please give judgement one by one along with the reason.
You should pay extra attention to the hallucination, which refers the part of comments that are inconsistent with the descriptions, specially claiming the existence of something not present in the descriptions.
If a comment hallucination, please help me rewrite it. When rewrite the comment, sound like you are looking at the image directly. Each rewritten comments should compose a description about the image which is correct, detailed, smooth and has strong readability. If not hallucination (correct or cannot judge), keep the original comment.
Your output should be: Judgement:
1. hallucination or correct or cannot judge: <reason>
Revised Sentences:
2.
Here are the region descriptions of the image:
Here is the comment for you to judge if it is hallucination and revise: 0
threshold, propagation probability introduces threshold-like behavior by selectively filtering to- kens. a token's activation (or indicator function) is low, it indicates that information of the token is not substantially propagated, effectively discarding less relevant tokens. This metric offers deeper insight into how likely information of a token is to be utilized in subsequent computations, making it important tool for analyzing cross-layer information flow in LLMs.
While attention scores are computed directly via softmax without any explicit threshold, propaga- tion probability offers more robust metric for selectively filtering tokens based their relevance. This mechanism helps mitigate the model's focus on irrelevant tokens, effectively regulating which tokens exert influence across the network. More importantly, token propagation probability not only captures token-wise interactions within a single layer but also models cross-layer information flow, which is critical in autoregressive-based large language models (MLLMs). This provides a theoret- ical foundation for understanding the phenomenon of over-propagation, a key factor contributing to hallucination in MLLMS
D.2 COMPARISON OF SUMMARY TOKENS IN OPERA AND ANCHOR TOKENS
We provide clear definitions for Summary Tokens in OPERA and Anchor Tokens below.
Summary Tokens are a specific type of token in LLMs that primarily serve to aggregate critical information from sequence during generation and provide global guidance for subsequent token
20

---

Published as a conference paper at ICLR 2025
Table 3: The prompt used for GPT-4V(ision) evaluation. evaauuutaionaaa
GPT-4V(ision) Prompt
You are required to score the performance two AI assistants in describing a given image. You should pay extra attention the hallucination, which refers to the part of descriptions that are in- consistent with the image content, such as claiming the existence of something not present in the image describing incorrectly in terms of the counts, positions, colors of objects in the image. Please rate the responses of the assistants on a scale of to 10, where a higher score indicates better performance, according to the following criteria:
1. Accuracy: whether the response is accurate with respect to the image content. Responses with fewer hallucinations should be given higher scores.
2. Detailedness: whether the response is rich in necessary details. Note that hallucinated descriptions should not count as necessary details.
Please output the scores for each criterion, containing only two values indicating the scores for Assistant and 2, respectively. The two scores are separated by space. Following the scores, please provide explanation of your evaluation, avoiding any potential bias and ensuring that the order in which the responses were presented does not affect your judgment. [Assistant 1]
O
[End of Assistant 1]
[Assistant 2]
O
[End of Assistant 2]
Output format:
Accuracy: Scores of the two answers
Reason:
Detailedness: Scores the two answers
Reason:
generation. They reflect the "Aggregation Pattern" inherent to LLMs, enabling the model syn- thesize global context for generating coherent outputs. However, excessive reliance on the global information provided by summary tokens may cause the model to overlook original contextual or visual modality inputs, leading to hallucinations.
key tokens with high propagation probability in the attention mechanism, pecially in multimodal tasks, where they emphasize information interaction between multimodal tokens. However, the over-propagation of anchor tokens can lead to an overemphasis ized information, causing the generated content to disproportionately focus on specific objects or concepts while neglecting broader contextual cues. This imbalance in attention distribution can contribute to the emergence of hallucinations.
Anchor Tokens
local-
D.3 ATTENTION MECHANISM
In this section, we discuss concurrent research on understanding the attention mechanism, which is widely used in computer vision (Tang et al. 2024, 2023 Zhao et al., 2024 Xu et al. 2024b) Xiong et al. 2024). (Geshkovski et al.) 2024) argued that trained multi-layer self-attention networks exhibit layer-wise dynamics akin to the Kuramoto model, where token embeddings converge to few "leader" tokens based on the structures determined by the self-attention parameter matrices. (Li et al., 2024c) proved that the learning dynamics of a single-layer self-attention network produce query-key parameter matrix that captures token-pair frequencies. (Bao al.) 2024) controlled the eigenspectrum variance through regularization dynamics, explicitly steering attention towards localization and thereby preventing the two failure modes of rank collapse and entropy collapse.
21

---

Published as a conference paper at ICLR 2025
By integrating current mainstream MLLM models with the inherent summarization mechanisms of LLMs, we are the first work to explore the promising direction studying the implicit bias of atten- tion through parameter eigenspectra. We discuss the strong correlation between over-propagee verrprooaggatio and hallucinations, highlighting that reducing the propagation probability of anchor tokens can ef- fectively alleviate the hallucination problem. Our method, TAME, addresses over-propagation by intervening in the eigenspectrum of the query-key parameter matrix, without requiring additional training inference time.
D.4 SOCIETAL IMPACTS
TAME does not pose any potential social harm. On the contrary. it has the potential to significantly advance the development of multimodal large language models (MLLMs). TAME provides inspira- tion for the research community, encouraging the exploration of more efficient solutions to mitigate the hallucination issue in MLLMs without incurring additional costs. It not only avoids any nega- tive social impact but also promotes the progress of multimodal Al assistants. These methods can achieve better generalization across different types MLLMs. Currently, although MLLMs rely on large language models, they still lack modules that resemble human brain functions, which need to be developed at the architectural level.
IMPLEMENTATION DETAILS
Basically, the hyperparameter gamma of TAME is set to the default value of 1. Other parameters use the default settings, same as the Baseline. Experiments are performed on NVIDIA H20/H100
GPUs.
To accelerate computation, we adopt the power method to approximate the eigenspectrum variance of the current matrix, with Algorithm P providing brief implementation sketch. In practice, fp32 precision typically required to ensure numerical stability. We experimented with applying the TAME method as plug-and-play approach to key and query weights, and confirmed its robustness across different configurations. Applying TAME to all layers is the simplest and most effective solution, performing well in practice without introducing any additional overhead.
Algorithm Pseudo-code of TAME PyTorch-like style.
Parameters:
W: Wei ght macrix of shape (bs, gamma: Hyperpazameter shape 1)
s]
Isitialize gamma as a tunable hyperparametes, with a default value of
Compute the prace of W'2 eta using Monte Carlo estimation srace_squaredeettinn
for in range toun_samples)
idx random integer N. # Randomly select row/colamn index estimate - (W(idx, idx)) "2 f Take the diagonal element and squaze it trace_squared_est iap.ee.aaamttteeedddddd fest imata)
eta sum
sun_samples # Average the estimates
arnamically intervening W according to the computed trace of W12
hat gamma log (eta xi))
Return reparameterized weight matrix W_hat return kat
F MORE RESULTS
EVALUATION ON CHALLENGING BENCHMARKS
We conduct rigorous evaluation of VCD, OPERA, and RAG (Vanilla-RAG (Karpukhin et al.
2020). SURF-RAG (Sun et al, 2024) as well as RLHF (RLHF-V (Yu et al.) 2024). CSR (Zhou
al. 2024
six popular MILM benchmarks and four additional ones, shown in the table 4
22

---

Published as a conference paper at ICLR 2025
For details, SEED-Bench et 2023a) consists of 19k multiple choice questions with human notations, while spanning 12 evaluation dimensions. GQA (Hudson & Manning 2019) incorporates novel evaluation metries suite focused on consistency, grounding, and plausibility, establishing rigorous standard for assessing in vision-language tasks. Vizwiz (Gurari et al.] 2018) examines certain perception capability, like knowledge and relation. MMR (Fu et al., (023) contains 14 meticulously designed subtasks that challenge the models' interpretative and analytical skills. MM- Bench (Liu et al., (2025b) also examines LVLMs on general perception capabilities using wide range of tasks. POPE is an assessment methodology designed to scrutinize object hallucination in LVLMs.
Vanilla-RAG concatenates the Top-N image-caption pairs from the database, which have the highest CLIP score similarity to the test image, before appending the questions and images for the LVL.Ms to respond. SURf-RAG is a self-refinement framework that teaches LVLMs to selectively utilize retrieved information. RLHF-V collects fine-grained paragraph-Ievel corrections from humans on hallucinations and performs dense direct preference optimization using human feedback. CSR en- ables the model to self-improve by iteratively generating candidate responses, evaluating the reward for each response, and curating preference data for fine-tuning.
For the RAG and RLHF methods, our approach seamlessly integrates as a plug-and module within their frameworks, without incurring extra inference time. By combining RAG with TAME, external knowledge retrieval enhances the accuracy of generated outputs, ensuring better alignment with the input context. Similarly, when integrated with RLHF, our method leverages human feed- back to guide the model in producing outputs that are more faithful to the actual content. Exper- imental results demonstrate that our approach delivers comprehensive improvements across both frameworks, further validating its effectiveness.
We conduct additional experiments evaluate improvements in factuality metrics and the effective- ness of data-driven instruction tuning on model performance. Our evaluations focus on benchmarks such as GQA, VizWiz, MME, and POPE, which test real-world knowledge QA and multimodal understanding tasks, as shown in the table below. The results demonstrate that instruction tuning with LRV-Iinstruction-inetue finttuned (Liu et al. (023b) moderately enhances performance by leveraging high-quality image-text pairs for task-specific fine-tuning, improving the model's 'ss alignment with real-world knowledge. However, integrating TAME further amplifies these gains by dynamically mitigating hallucinations and strengthening factual alignment, resulting in significant improvements across all benchmarks. This combination achieves a higher degree of correctness and consistency in generated outputs, validating that TAME effectively complements instruction tuning plug-and- play enhancement, improving both accuracy and robustness in real-world multimodal tasks.
F.2 EXTENDING TAME TO SINGLE- LLMs
We extend our experiments to single-modal L.MMs. As shown in 5 we conduct on the Wikitext- 103 (Merity et al. 2016) and MiniPile (Kaddour 2023) datasets, to assess the scalability and con- sistency of TAME's s impact. TAME was integrated as a plug-and-play enhancement across three distinct model configurations, including the BLOOM (LLaMA architecture with ALiBi) (Lester) et al. 202T) and OpenLLaMA (Touvron al.) 2023a). The results showed that TAME consistently improved perplexity (PPL) across all architectures and parameter sizes, demonstrating its effective- ness in enhancing model performance. These findings further support that TAME is generalizable mechanism, suitable for both multimodal and single-modal L.LMs, broadening its applicability.
F.3 GPT-4V ASSISTED EVALUATION
We further resort to GPT-4Vision, a strong multi-modal assistant that can easily handle the in- put from vision, language, and voice modality. Typically, we randomly sample 500 images from MSCOCO's validate set and ask different MILM models to describe these images. For fair compar- ison, we follow (Yin et al. 2023) and compare the answers obtained from two decoding methods at the same time, i.e., providing the image and both the answers to GPT-4V and prompting it to give judgement from 0-10 for each. The prompt emphasizes mitigating the impact of the sequential order fed to GPT-4V and, additionally, paying special attention to the objects mentioned in answers but not appear in the image. It includes instances where the objects are represented in an incorrect form of colors, positions, or relationships.
23

---

Published as a conference paper at ICLR 2025
GQA
SEED
VisWiz
MME
MMEA
MMB
POPER
LLaVA-L.5-7B
60.4
58.1
49.0
278.33
245.00
64.2
80.7
VCD
61.0
58.9
50.8
293.00
268.33
61.4
83.3
VCD w/ TAME
61.7
59.4
51.6
295.67
275.67
61.5
84.0
OPERA
62.0
59.6
52.4
290.33
251.67
64.8
83.4
OPERA w/ TAME
62.5
60.7
52.9
294.33
256.00
65.3
83.8
Vanilla-RAG
60.2
49.6
264.33
255.33
84.7
Vanilla-RAG w/ TAME
61.8
50.7
272.33
259.67
86.5
SURF-RAG
62.4
54.3
268.67
253.00
85.6
SURE-RAG w/ TAME
62.9
55.2
270.00
258.33
87.1
RLHF-V
62.3
59.3
53.7
283.33
263.67
63.6
85.2
RLHE-V w/ TAME
62.8
59.8
54.2
272.33
266.33
64.2
86.
CSR
61.9
60.0
53.4
285.67
264.67
65.2
86.0
CSR w/ TAME
62.4
60.6
53.9
294.00
271.00
66.9
87.0
LLaVA-1.5-13B
64.1
53.3
68.2
86.5
OPERA
64.0
55.6
68.9
87.2
OPERA w/ TAME
65.7
56.2
68.8
89.0
ALAWA-NeXT-Mistral--7
62.9
52.6
88.2
OPERA
52.9
67.2
OPERA TAME
64.6
54.0
67.4
90.4
mPLUG-OWl-77
66.7
63.5
57.1
310,33
281.67
68.2
89.1
ENV-Instruction-fiiee
67.5
64.2
58.0
315.00
285.33
69.5
90.0
-.n-nmstruciion-ffeeuunddd w/ TAME
68.9
65.8
59.2
322.33
291.00
71.0
91.5
Table Comparison of methods on different benchmarks. SEED' refers SEED image evaluation, VisWiz refers to image refers to VisWiz VQA, POPER referss to POPE Random, MME refers to MME Object-level Hallucination Existence Count.
WikiText-103
MiniPilc
Model
#Params
PPL
Model
#Params
PPL
BLOOM
71M
29.9
BLOOM
160M
25.8
BLOOM w/ TAME
71M
29.0
BLOOM w/ TAME
160M
25,3
OpenLLaMA
71M
27.4
OpenLLaMA
160M
25.9
LaE w/ TAME
71M
26.9
Ope w/ TAME
160M
24.9
BLOOM
160M
27.6
BLOOM
430M
20.6
BLOOM w/ TAME
160M
26.0
BLOOM w/ TAME
430M
19,3
LaMALLaMA
160M
22.5
OpenLLaMA
430M
19.6
LaMALLaMA w/ TAME
160M
21.3
OpenLLaMA w/ TAME
430M
19.4
Table 5: Perplexity (PPL) comparison on WikiText-103 and MiniPile datasets using BLOOM and OpenLLaMA architectures with and without TAME across varying parameter sizes.
As demonstrated in T TAME delivers improvements of up to 25.7% compared Beam Search decoding and enhances performance over oatte-of-the-are methods by as much as 6.8%, all while maintaining the level of detail in responses. Given that GPT-4V's perceptual and reasoning capa- bilities closely approximate human judgment, the evaluation results from GPT-4V provide a strong indication the effectiveness of TAME in reducing hallucinationss from human-centric perspective.
F.4 EVALUATING ON DIVERSE HALLUCINATION TYPES
The table pressenss our experimental results on the hallucination subset of the MME dataset, which includes object-level hallucinations and attribute-level hallucinations. By comparing the perfor- mance of various methods, we demonstrate the broad applicability of TAME. As plug-and-play decoding strategy, TAME achieves significant performance improvements across multiple methods. By dynamically adjusting the eigenspectrum variance of the attention weight matrix, TAME fectively mitigates the over-propagation of anchor tokens, thereby enhancing model performance
24

---

Published as a conference paper at ICLR 2025
Table 6: GPT-4V assisted hallucination evalua- tion (Huang et al. 2024] results on MSCOCO. Two aspects are verified, i.e., correctness (C)
Table POPE hallucination evaluation results for four MLLM models, presenting the average Fi-score calculated across the random, popu- lar, and adversarial splits of POPE.
and detailedness (D). Higher correctness/de- tailedness indicates less hallucinations.
Methods
InsmucBLIPP
MiniGPT-4
LLaVA-15
Shikrn
Methods
InstrnctBLiP IP
MiniGPT4
Beam
77.9
75.1
80.8
80.1
+TAMK
78.1
$1.9
74.7
83.3
82.5
Beam
5.68
5.35
5.47
5.16
5.53
5.55
TAME
$4.5
79.4
84.0
83.7
+TAME
6.24
5.78
6.68
5,20
6.39
5.82
ICD
$2.3
74.4
82.2
S1.7
OPERA
6.19
5.67
6.62
5.18
6.23
5.79
OPERA
82.7
76.6
83.4
82.6
81.8
+TAME
6.45
5.80
6.84
5.46
6.37
5.80
+TAME
84.9
79.2
83.8
82.3
SID
6.26
5.73
6.54
5.21
6.25
5.84
SID
77.0
+TAME
5.78
6.89
5.24
6.42
5.95
+TAME
$2.5
79.3
84.1
83.9
multiple dimensions, including object existence, count estimation, position alignment, and color consistency.
Object-level
Attribute-level
Decoding
Total Scores
Existence
Counrt
Positiont
Colort
LLaVA-1.5-7B
163.67
114.66
104.00
141.00
523.33
Greedy
184.00
95.33
112.00
157.67
549.00
+ VCD
172.67
120.33
129.67
155.00
561.33
+ OPERA
174.67
115.66
110.67
141.00
542.00
+ OPERA w/ TAME
176.00
118.33
113.00
143.00
550.33
+ SID
182.00
127.00
116.00
139.00
564.00
+ SID w/ TAME
193.00
137.33
139.00
164.67
634.00
Table 8: Evaluation results on the hallucination subset MME. max-tokens=512.
F.5 LAYER-WIE EVALUATION OF TAME: EXPLORING ITS IMPACT ON TOKEN PROPAGATION
As analyzed in Figure and Appendix A early, middle, and late layers exhibit significant dif- ferences token propagation patterns. Early layers (1)) primarily focus on extracting low-level features and token initialization, middle layers (416) emphasize multi-modal alignment and feature aggregation, while late layers (131) are responsible for high-level reasoning and final output genera- tion. These differences play critical role in the model's behavior and error generation, especially in hallucination prone scenarios.
We evaluate TAME by applying it different combinations of layers (lo lıs and (31). ). shown in the table In this experiment, the baseline is OPERA (Exp I). The results from Exp II-IV demon- strate the incremental impact of TAME when applied to individual layers: Early layers (lo, Exp II): Applying TAME at this stage slightly reduces hallucinations by refining token initialization and propagation, but its overall impact limited due the lack of deeper-layer information. Middle layers (116, Exp III): Incorporating TAME at this stage significantly improves multi-modal align- ment, optimizing feature integration and enhancing response generation accuracy. Late layers (131, Exp IV): Applying TAME here substantially reduces errors, as this stage directly influences final reasoning and output generation.
The results from Exp V-VIII further demonstrate the cumulative effect of applying TAME to mul- tiple layers: to + 116 + lai (Exp VIII): Applying TAME across layers achieves the best overall performance, with the fewest hallucinations and the highest accuracy across all tasks, albeit at slightly higher inference cost.
This phenomenon highlights the importance of holistically optimizing token propagation across the model. Early layers provide foundational improvements, middle layers optimize multi-modal representations, and late layers ensure high-level reasoning accuracy. The experiments suggest that integrating TAME across more layers significantly reduces error-prone responses, as it dynamically mitigates peepaaropagttion at different stages of the model.
25

---

Published as a conference paper at ICLR 2025
Exp
to
116
131
MMB
GQA
CHAIRs
CHAIR,
64.8
62.0
46.4
13.0
II
/
64.9
62.0
46.2
12.7
III
64.8
62.1
45.8
12.6
IV
64.9
62.3
45.9
12.7
V
65.1
62.3
44.2
12.6
VI
V
65.3
62.6
44.4
12.5
VII
65.2
62.4
43,9
12.3
VIII
65.3
62.5
41.3
12.2
Table 9: Ablation study of applying TAME on different layers. 4o layer O; (16 layer 16; (31 layer
31;
F.6 COMPARISON OF OUTPUT TEXT LENGTH ACROSS DECODING METHODS
We conduct detailed evaluations of the generated output length across different decoding methods, as shown the table The results indicate that while TAME effectively reduces hallucinated content, it maintains even slightly increases the output length when integrated with decoding strategies like VCD, OPERA, or ICD. Specifically, on the COCO dataset, TAME achieves a balanced reduction in hallucinations without sacrificing detail richness, demonstrated by the minimal variation slight increase in the average length.
Method
Length
LLaVA-1.5
100.6
+ VCD
100.4
VCD w/ TAME
100.9
+ OPERA
98.6
OPERA TAME
98.4
+ ICD
106.3
+ ICD w/ TAME
110.1
Table 10:
Comparison of the hallucination mitigation performance across different methods in terms of output length.
F.7 TEXT QUALITY EVALUATION.
To assess the overall quality of generated text comprchensively. we adopt PPL (Perplexity. a classical metric in NLP without using reference text), and resort GPT-4 to assess the grammar, fluency, and naturalness of generated text. We randomly select 1,000 images in MSCOCO and verify on LLa 7B model. The average results are listed above, where PPLI and PPL2 are calculated by pretrained gpt2 and gpt2-medium respectively. From the results in Table [1] we discover that TAME can generally keep the quality of generated text from various aspects. Besides, we test TAME on two popular MLLM benchmark, i.e., MME and MMBench (Liu et al. (023c), using VLa 7B model. Table [2 shows that TAME can maintain and even improve MLLM's performance on both MILM benchmarks.
26

---

Published as a conference paper at ICLR 2025
Table 11: The evaluation results for the quality of generated text. We calculate PPL,, and PPL.2 with gp(2 and gp12-medium in the Huggingface model zoo, respectively. The ratings of grammar, fluency, and naturalness are provided by GPT-4.
PPL.1 4
PPL.2
Grammar t
Fluency
Natural 1
Greedy
12.72
10.27
9.58
9.01
8.52
+TAME
12.63
10.04
9.59
9.12
8.57
Beam Search
11.11
8.89
9.54
8.95
8.55
+TAME
11.04
8.79
9.58
8.98
8.56
OPERA
11.67
9.31
9.54
8.93
8.53
+TAME
11.56
9.17
9.60
8.98
8.52
SID
11.70
9.35
9.49
9.06
8.47
+TAME
11.62
9.30
9.55
9.09
8.49
Table 12: TAME generally improves the MLLM's performance on popular MLLM benchmark.
Greedy
Beam
OPERA
OPERA + TAME
MMBench
64.3
64.4
64.4
65.2
MME 1510.7
1504.3
1515.4
1523.0
27

---

Published as a conference paper at ICLR 2025
G
DETAILED PROOFS
Theorem (Propagation Entropy). Let WW |22|XXX 122, and exp (-ov --1
(-O0/=)
The propagation entropy Ent(p) holds that:
Ent(p) = a log(1 (T + 0² T (T 1)3
where Ent(p) represents that lower entropy increases the likelihood of over pver-propagation of anchor 0 or oa as illustrated in Figure 3 Left). Propugation entropy increases with [ Wokllp ll up to peak, then decreases, reaching its lowest point extreme values of [[Worr l|2- If [[]r([W] moderate, propagation entropy stays near the peak. To mitigate over-propagation of anchor tokens, is sufficient to control tr (W2)
tokens, following unimodal pattern 6, and vanishing as [[okkl
under fixed tr(W): ||=>>| (W2) 2 [[ 2
Proof. Let a E R.T denote the i'th row of A, a = A, From the assumptions it holds that ||a|| S a. p(a) in Eq. denote the softmax propagation probabilities given by:
P.
The entropy still follows the classical probability- definition. For the pi propagation entropy can be written as:
T
e
6:a
Ent(a) = Σ log
log
Z
Z
We are minimizing the entropy based on Ps. while subject to the quadratic constraint on a:
min Ent(a) s.t. ||a||2 < o
The Lagrangian function is defined to handle the constraint:
C(a, A) Ent(a) 1 1(1aa|1- o2).
To find all saddle points, solve the system of equations:
OC(u, A)
O, OC(u, A)
0
Ou
Based on the final equation shown and your reference to the work of (Zhai et al. 2023). the propa- gation entropy is derived and represented as:
o TVT(T-
Ent(a") = log 1+ (T
1)e
1 1
To maintain degree of localization and avoid rank collapse, as discussed in Eq 9 we added constraint Ent(a*), such that Ent' (a*) = Ent(((") Therefore, the final propagation entropy is given by:
8² TT 118
Ent' (a") log (1 - - 1)3) + 1 (T- 1)3
where B = exp (-o
The first inequality due to the
inequality as:
T-1
|Σー1|| (rlll//rr((e))) = ||>~||>|||]| ||Wokllr 2 ||Wok]|2.
28

---

Published as a conference paper at ICLR 2025
H VISUALIZATION
In this section, we present visualization that illustrates the relationship between the attention map, token propagation probability, and the generated tokens. The visualization reveals that tokens with exceptionally high propagation probabilities trigger a series of hallucinations in the subsequent gen- erated text.
Attention Map for Generated Tokens
Hag
High
Token Propagation Probability
Figure 8: Visualization of attention map and token propagation probability for generated tokens. The words in hallucinated captions are marked red.
I MORE QUALITATIVE RESULTS
In this section, we provide additional visual comparisons to further demonstrate the effectiveness of TAME, These visualizations offer clear evidence of how TAME improves the model's performance
29

---

Published as a conference paper at ICLR 2025
Shikra (Beam Search):
scane park next to each other on read. the motoroyde arrange in row with one ocoupy the majority of the frame on the elf side and the others gradually decrease size. thhre be three motoroydle total, with one prominenlly postion and the other on the right side in background there be few chair and bench scatter around, the chair be place in various postiion indude
imge fescure
motoraydl
middle
the left foreground. another the middle.. right
Instruction: Please describe this image in detail
the bench side the scene near trighthe motoraydle additionally there be person stand on the far loft sidi dasse to the chair.
the towards
Shikra (OPERA):
The scone feature group of people dirt bike, Iine noxt to other and
A AMA 3AA AM
EA AMAA AMA A AM
prominently on ther matornyclo. the front and the other too on cither side. there be two more people one near center and another the side the the matortycla be arrange
for three person position
center and the other two towards the right side of the soene A chair be also Visble the middle of the scene, possibly for rast or for the official.
Shikra (Beam Search+ TAME):
The image festure red train with the word howrah wiiiee on station the
side park
position
track,
pecupy grificant portion the scene the station sppear be busy area with mu tiple people scatter throughout the scone, likey wail for their train or attend to other task, the presence of several individual suggest that train station bustie activity. .
Input Image
Shikra (OPERA+ TAME):
rid motorsyclo track. they wear and appear participate race. the matoraycle ha position closaly togethar. with one on the left, another in the middle and the third on.
image depict group
Shikra (Beam bearch):
The image feature large umarella
car park on
Instruction: Please describe this image in detail
a street, the umbroola be position above the car provide protaction from the elemant- the car be park next a building. and there saveral the car there be multiple people walk around area. some of tham be carry handhag. with one pansnn hold a handbag near the center of the spone and another person carry a handbagg peclestrian and parked car creats lusty urban atmasphore.
park
vicinity addition
the right
Shikra (Opera):
The image depict
with large umbrella couch place dewallk. umbrella position above couch, provide shade and protaction from the clament the couch be surround by several potted plant add touch of groenary to
Shikra (Beam Search+ TAME):
The image feature car cover in plastic. likey to protect from the alement the be on the side of the street and appaar to convartible the and there be person stand nearby. the soane also inclue a few other car park the background.
park next build
Input Image
Shikra (Opera+ + TAME):
The soene festure an umbrella cover black car park on the side a busy city strect the umbralo be postiin above the car, protect from the there be several other park in vidinly. include one the right edge image.
Figure 9: Qualitative example.
by reducing hallucinations and enhancing attention localization. As shown in Figure Figure I0] Figure []] and Figure S, our proposed TAME not only mitigates hallucinations in large language models but also ensures that the generated text remains of high quality. By comparing the outputs of models with and without TAME, we highlight the significant differences in how the attention mechanism processes input tokens and distributes weights across the sequence.
á
30

---

Published as a conference paper at ICLR 2025
Shikra (Beam Search):
partially undernesth pillo.. dag octupy the majortty soane, with hody pocttion acroes the wich and longrs the bed the bed appear be cover blankec
age
flully dog
Instruction: Please describe this image in detail
on give the dog inviting with the dog enjoy relexing moment bed
comforable
Shikra (OPERA):
the image feature arge oideril dog on a bed with nead rest on piliow. the bed be cover in plaid blanket, give a cody and comfortable appearanze
portion the occupy reach edge
af frame.
Shikra (Beam Search TAME):
the image feature large black
couch, rest head
gillow:
ocoupy significant portion of space. there two pillow the couch, one the left side and another on the
Shikra COPERA+TAMES
the image depict black dog sleep comfortably on couch the dog onzapy a large portion of the couch, w th head rest on add textura the allow postton, one behind dog 's head and anotner on the rihht add the couch. the cazy arrangement the pillow and couch provide relexing dog.
pillow: the couch have plaid
background there
Input Image
Shikra (Beam Search):
the in image show two men play a video game in dimnly it room each hord a niteeno wi controlier and actively engage the the
be glass, and they looth thersselves they compete. in the room there be a chair and couch potteci plant, bowl, and a cup place in the beckground the setting
their
Instruction: Please describe this image in detail
atmasphere for the friend enjoy their gaming togesher.
Shikra (OPERA):
the image dep oi two younng men stand in living room play video game nintenco will console, both man hold wi they position close to eacn other with one man the other on the right sice of the room the living room be furnish couch the beckground and cheir neer by there be two
the right side, add a touch of greenery y to the space vese can also be see on the right side of the room, further enhance the decar
Shikra (Beam Search + TAME):
the image dep two young men stand in living room, play vieep gama rintendo consalo. both man be hold wi engege game.
wear glass.
Input Image
Shikra (OPERA + TAME):
men stand iving actively engage in the game. they be position close to each other with one man on the left side and the other on the right side of the
the image feeture
geme together.
Figure 10: Qualitative example.
31

---

Published as a conference paper at ICLR 2025
Shikra (Beam Search):
the mage feature yellow st on desk, postion in front montor the cas cccupy large portion of the desk, and it orange for stand out agains: the wooden surface include keyboard. mouse. and add borelly, there be multiple book scatter around the desk in verious orientation give the scene outeered appearance a chair be situaee the desk, add the overell workspece environment.
the desc, there
Instruction: Please describe this image in detail
the background.
Shikra (Opera):
the image feature large orange tasby cat top of wooden desk, the cat be pos too near the center of the desk and appear to be look at the camera, the desk be dutter with varicus teem indiude computer.
phone.
side remote, and several desk
book. some can spot others be seatter around the right side. a be also present in the background, mount on the well above the desk.
Shikra (Beam Search TAME):
the image teature a large orange tanby cat sit on a wooden dask, look direcily at the camera. the cat position towards the center the desk, gocupy signfficatt portion
scene, desk clutter with various item inclde computer monitor
Input Image
Shikra (OPERA -OPER++TAME): TAME):
the image tsatura orange tabby car st wonden dask surround varicty objost the be the background : scene, face towards the right. there ane be pocition above the other and locate to the left the cat.
of
Shikra (Beam Search):
the image cantura naseoal game progress, with group people
uniform on the
man
possibly a coach. be crouch down next young boy who be weaar red hat they be both hold baseball glove prepore to make a
the background there be bench the left on right. the
and thair be socupy by a spectator. the scene so include baseball bet locate on the for right side. suggest that be ready for swing
empty.
Instruction: Please describe this image in detail
Shikra (OPERA):
mage shows baseball player sitting bench with oasebail glove his hend, watching a smmll child besebell uniform run up him. The child waa ring beseball cap and nolding besebell bat. The basebal player wearing baseball uniform with a
boseball glove. The crowd sackground made up of people watching the game.
Shikra (Beam Search+ + TAME):
325
of people. include baseball unitorm and appear to basebal player while boy be aiso present on the fiedd they seem to be engage in a corvers sation or discuss something relate the game. there be several people
the image festure beseball field with grass
boy
background, some whoon weer basebal uniform: a besebelll glove can be see on the ground. indicate thas the player be likely prepare for a game or have finish
Shikra (OPERA TAME):
Input Image
the image feature young boy bascball unilarm stand on basetal
ind indicatt involvement in the sport the boy be stand front of a baseball field, possibly wait for nis turn play. in the background und, there several
wear baseball
people, possibly
spectator to foroground,
the game. .sme them stand while others be far in the background the scene capture the escrtmment and anticipation of basecal game.
Figure 11: Qualitative example.
32


Reviews:
[{"review_id": "Z1TlZo9o7i", "reviewer": "Reviewer_UP4R", "summary": "This paper explores the role of anchor tokens in causing hallucinations in LLMs, and proposed Dynamic Token Propagation Mechanism (TAME) based on eigenspectrum variance of the attention weight. The results show that TAM can be integrated with existing decoding method including Beam/VCD/ICD/SID and improve the performance consistently on InstructBLIP, MiniGPT-4, LLaVA-1.5, and Shikra.", "questions": "1. in Table 1: the baselines with max token 512 generate very high hallucination (>30%). Is it aligned with user's experience? At least I feel Gemini and ChatGPT's numbers should be much better than that.\n\n2. I feel the proposed method can be used for any LLM based method. Why does the title claim it only for multimodal LLMs?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "1. Simple method which is very easy to use\n2. Good insights from eigenspectrum variance\n3. Constantly improve the decoding methods of InstructBLIP, MiniGPT-4, LLaVA-1.5, and Shikra.", "weaknesses": "1. I hope the paper could explore additional approaches to reduce hallucinations, such as Retrieval-Augmented Generation (RAG), Reinforcement Learning from Human Feedback (RLHF), improvements in factuality metrics, and high-quality, data-based instruction tuning.\n\n2. The paper currently uses the a few simple eval sets like MS COCO. I would suggest incorporating more challenging benchmarks for a more rigorous assessment.\n\n3. The writing could benefit from refinement. For instance, many citations are not in the correct format.\n\n4. The paper only reports metrics for hallucination. But it is not clear the new predictions do better or worse in other metrics. Sometimes less hallucination may lead to less details or not as friendly to users to read.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "qKVoMfdLqm", "reviewer": "Reviewer_kinG", "summary": "This paper investigates the causes of hallucinations in LVLMs through the attention weight matrix of anchor tokens. Then this paper demonstrates the propagation pattern of anchor tokens by the eigenspectrum of the attention weight matrix. The authors further propose a versatile plug-and-play decoding strategy named Dynamic Token Propagation Mechanism (TAME) to reduce the over-propagation of anchor tokens through dynamically intervening in the eigenspectrum variance. Extensive experiments show that TAME improves hallucination metrics across multiple MLLMs.", "questions": "Please see the weakness.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "1.\tThis paper provides a solid theoretical framework by analyzing the relationship between LVLM hallucinations and attention patterns.\n2.\tThis paper introduces a novel plug-and-play decoding strategy that dynamically adjusts the eigenspectrum variance of attention weights to mitigate hallucinations without adding inference time.\n3.\tExtensive experiments show that TAME improves hallucination metrics across multiple MLLMs.\n4.\tTAME can be integrated into various decoding strategies without requiring additional training or data.", "weaknesses": "1.\tAlthough this paper provides an additional theoretical framework, the impact of anchor tokens on the LVLM hallucination was first proposed by OPERA [1].\n2.\tTAME primarily addresses object hallucinations but may not be as effective for other hallucination types, such as incorrect attributes or relations (e.g., limited performance improvement on the MME benchmark). Experiments on more types of hallucination should be conducted.\n3.\tIt would be interesting to study applying TAME to some layers rather than all layers through experiments.\n\n[1] OPERA: Alleviating Hallucination in Multi-Modal Large Language Models  via Over-Trust Penalty and Retrospection-Allocation", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "qqWerAzPfi", "reviewer": "Reviewer_tkGb", "summary": "MLLMs encounter hallucination issues, where the generated text does not match the provided visual content. Previous methods have attempted to mitigate hallucinations by designing special decoding strategies, such as penalizing summary tokens, but they lack an analysis of the relationship between the model’s hallucinations and the summary mechanism. In this paper, a study and analysis of the causes of hallucinations in MLLMs are presented. A general decoding strategy, TAME, is proposed to reduce the excessive propagation of anchor tokens by dynamically intervening in the variance of feature spectra. Experiments demonstrate the correlation between feature spectra and hallucinations in MLLMs, as well as the effectiveness of TAME.", "questions": "See weakness.", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The motivations are strong, with in-depth analysis and derivation of the mechanisms behind hallucinations in MLLMs.\n2. The proposed method, TAME, is a simple and effective plug-and-play decoding strategy.\n3. The experiments are thorough, validating both the causes of hallucinations mentioned in the paper and the effectiveness of the proposed method.", "weaknesses": "1. The introduction mentions summary tokens but does not define them, leaving unclear the distinction between summary tokens and anchor tokens.\n2. There are some typos in the text that need to be checked:\n   1. Line 301: \"Conversly\" -> \"Conversely\"\n   2. Line 333: \"7,3\" -> \"7.3\"", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ikRrc9eMgt", "reviewer": "Reviewer_59hN", "summary": "This paper proposes a new decoding method to address hallucination issues in multimodal large language models (MLLMs). The authors identify that excessive propagation of “anchor tokens”, influenced by polarized variance in the attention mechanism’s eigenspectrum, contributes to hallucinations by sidelining crucial visual information. To mitigate this, the paper introduces the Dynamic Token Propagation Mechanism (TAME), a plug-and-play decoding strategy that adjusts the eigenspectrum variance in the attention weights, limiting over-propagation without adding to inference time. Extensive experiments demonstrate that TAME effectively reduces hallucinations across multiple MLLMs and achieves great improvements over existing methods like Greedy Decoding and Visual Contrastive Decoding (VCD). The method is simple-yet-effective and easy to implement.", "questions": "1. In the qualitative results, it appears that the proposed method tends to make the model’s output shorter, which is expected, as the goal is to reduce hallucinated content. However, from another perspective, could this lead to insufficient perception of the image by the model, potentially missing some details? If so, how could we measure and balance this trade-off?\n\n2. The example in Figure 6 is somewhat unclear, as it’s difficult for reviewers to determine which parts of the image or which specific patches correspond to the enhanced visual tokens. Could the authors provide a visualization of the relevant visual tokens mapped onto the image?", "limitations": "", "rating": 8, "confidence": 5, "soundness": 4, "presentation": 3, "contribution": 4, "strengths": "1. The paper is well-written and clear motivated. The authors begin by introducing hallucinations and related work, gradually diving deeper by focusing on the phenomena of token information flow and information aggregation, ultimately leading to the proposal of TAME. \n\n2. This paper is built on a detailed and clear theoretical foundation. Although the proposed method appears simple, the step-by-step theoretical derivation strongly supports it. \n\n3. The proposed method is simple-yet-effective, which obviously surpasses all baselines on CHAIR, POPE and GPT-4 assisted hallucination evaluation. Meanwhile, the method introduces no additional computation overhead, which is superior to existing contrastive decoding based methods.", "weaknesses": "1. Some of the color schemes in the figures, such as Figures 4 and 5, are prone to causing confusion. It is recommended to use colors with higher contrast and more distinct differentiation.\n\n2. The paper begins its analysis with the phenomena of token information flow and token information aggregation, gradually developing a method to mitigate hallucinations. However, it does not yet discuss the underlying causes or triggers of these phenomena, only relying on empirical analysis. Nevertheless, the analytical approach used in the paper is still worth learning from.\n\n3. The paper provides limited explanation of the proposed method. Although the method is simple (just one single formula), a smoother transition from the theoretical derivation is needed, along with a clear description of its application scenarios.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)