Look Twice Before You Answer: Memory-Space Visual Retracing for Hallucination Mitigation in Multimodal Large Language Models

## Abstract

Abstract Despite their impressive capabilities, Multimodal Large Language Models (MLLMs) are susceptible to hallucinations, especially assertively fabricating content not present in the visual inputs. To address the aforementioned challenge, we follow a common cognitive process - when one’s initial memory of critical on-sight details fades, it is intuitive to look at them a second time to seek a factual and accurate answer. Therefore, we introduce Mem ory-space V isual R etracing ( MemVR ), a novel hallucination mitigation paradigm that without the need for external knowledge retrieval or additional fine-tuning. In particular, we treat visual prompts as supplementary evidence to be reinjected into MLLMs via Feed Forward Network (FFN) as “key-value memory”, when the model is uncertain or even amnesic about question-relevant visual memories. Comprehensive experimental evaluations demonstrate that MemVR significantly mitigates hallucination issues across various MLLMs and excels in general benchmarks without incurring added time overhead, thus emphasizing its potential for widespread applicability. 1 1 1 Code can be available at https://github.com/1zhou-Wang/MemVR .

## 1 Introduction

Multimodal Large Language Models (MLLMs), due to their formidable capacity to comprehend visual inputs, have emerged as indispensable tools in computer vision (Koh et al., ) and natural language processing (Tu et al., ) to tackle numerous visual tasks and facilitate complex visual question answering (VQA). Nevertheless, MLLMs still exhibit certain limitations, i.e., the so-called “hallucination” phenomenon (Huang et al., ; Zheng et al., ). Specifically, MLLMs frequently generate descriptions inconsistent with user-provided visual inputs, such as incorrectly outputting non-existent things in the image or conflicting judgments. This flaw poses significant risks to the reliability of MLLMs as trustworthy assistants (Yu et al., ), particularly in safety-critical applications (Zou et al., ) (e.g., clinical healthcare (Lin et al., ) and autonomous driving scenarios (Ding et al., )). While the exact reasons for hallucinations in MLLMs are not fully understood, one possible factor could be the imbalance between their understanding of visual and textual information.
Typically, MLLMs encode images into vision tokens via CLIP (Radford et al., ), which are fed along with text tokens into LLMs for decoding. LLMs excel at text comprehension but struggle with visual information perception and memory, where differences in information density between modalities may cause inconsistencies during decoding.

Numerous methods have been proposed to mitigate hallucinations in MLLMs, and general studies can be broadly categorized into three streams: (i) Retrieval-Augmented Generation (RAG) (Shuster et al., ; Caffagni et al., ) that incorporates knowledge from external databases to mitigate the problem of “hallucination”, as well as (ii) through extra Fine-tuning (Yu et al., ) to enhance the self-consistency of generation, and (iii) Contrastive Decoding (CD) strategy, which do not involve extra training. Specifically, RAG and fine-tuning patterns typically employ external knowledge retrieval or robust instruction-tuning datasets to post-hoc debias (Yang et al., ; Liu et al., ), which inevitably introduces substantial computational overhead or storage requirements. For example, some approaches(Yin et al., ; Yu et al., ) have fine-tuned models using high-quality visual instructions generated by advanced automated annotation tools including GPT-4 (OpenAI, ).

CD-based methods (Li et al., ; Shi et al., ) represent a simpler and more efficient way to mitigate hallucinations than RAG and fine-tuning based methods. Particularly, CD-based hallucination mitigation methods usually modulate the logits of the next token prediction through contrast manner or penalty mechanisms. As illustrated in Figure 2 (a), DoLa (Chuang et al., ) enriches factual knowledge via layer-wise contrasting and reduces the generation of incorrect facts in LLMs. In MLLMs, VCD (Leng et al., ) amplifies the language priors by adding Gaussian noise to the visual inputs, thereby reducing over-reliance on statistical biases and single-modal priors through contrasting output distributions from original and distorted visual inputs as in Figure 2 (b). This perturbation of original inputs requires task-specific design, inevitably doubling inference costs. More critically, contrastive distributions are agnostic to visual and instructional nuances, which may not always amplify the intended hallucinations, occasionally introducing potential noise into CD.

Figure: Figure 2: The conventional paradigm for hallucination mitigation, employing contrastive decoding methods such as DoLa (a) and VCD (b), is compared to our proposed MemVR strategy (c).
Refer to caption: x2.png

**Table 1: Efficiency Comparisons for generating different length tokens, using an NVIDIA-A40 GPU. Inference time (ms) of different methods is recorded.**
| Method | 20-Token Len | 50-Token Len | 80-Token Len |
| --- | --- | --- | --- |
| LLaVA-1.5 | 1880.3  $\downarrow$$\times$1.0 | 3617.6  $\downarrow$$\times$1.0 | 5256.6  $\downarrow$$\times$1.0 |
| + VCD | 4537.4  $\uparrow$$\times$2.4 | 7690.8 $\uparrow$$\times$2.1 | 11569.3 $\uparrow$$\times$2.2 |
| + OPERA | 6242.7  $\uparrow$$\times$3.3 | 12672.3  $\uparrow$$\times$3.5 | 19247.2  $\uparrow$$\times$3.7 |
| + MemVR | 1861.7  $\uparrow$$\times$1.0 | 4000.9  $\uparrow$$\times$1.1 | 5545.5  $\uparrow$$\times$1.1 |

In this work, we delve into the challenges of hallucination mitigation in MLLMs and address the shortcomings of CD-based approaches. Our research is grounded in a common cognitive process: when the initial memory of certain critical visual details fades, it is intuitive to look at them for the second time to search for the accurate answer (O’regan, ; Ballard et al., ; Horowitz & Wolfe, ). Different from visual contrastive decoding strategies that alleviate hallucinations by diminishing language priors (Leng et al., ; Qu et al., ; Park et al., ), we propose a novel Memory-space Visual Retracing (MemVR) method that mitigates hallucinations through supplementing visual evidence, akin to the two sides of a coin. MemVR, as shown in Figure 2 (c), is an architecture-agnostic, plug-and-play solution that re-injects visual features into an intermediate layer suffering from vision-related memory lapse with only one regular inference. This novel hallucination mitigation paradigm in terms of efficiency, and its inference cost and performance are optimal compared to previous studies as listed Table 1. It’s a game-changer for effectiveness and efficiency. Through extensive experiments on multimodal hallucination benchmarks, as well as GPT-4o(^2^22GPT-4o-2024-08-06: https://platform.openai.com/docs/models/gpt-4o.) evaluations, we show the comprehensive performance improvements of MemVR in hallucination mitigation and general capabilities. Our contributions can be summarized as follows:

- ❶
We propose MemVR, a novel training-free hallucination mitigation paradigm that effectively alleviates hallucinations in MLLMs. In contrast to previous methods, which primarily focus on eliminating biases of language priors, MemVR seeks to replenish question-relevant visual clues towards more evidential responses, which signifies the other side of the coin.
- ❷
We design a dynamic premature layer injection strategy with visual retracing in MLLMs, mimicking human intuitive thinking to revisit image features for self-consistency and credible answers when pivotal memories are scrambled. Furthermore, we theoretically demonstrate that visual retracing can effectively diminish hallucinations from an information-theoretic perspective.
- ❸
Comprehensive experimental results demonstrate the effectiveness of MemVR in mitigating hallucinations and enhancing general cognitive and perceptual performance, as well as its high efficiency. Our research will make a substantial contribution to trustworthy multimodal intelligence.

## 2 Related Work

MLLMs and Challenges.
In recent years, MLLMs have made remarkable progress, particularly as they have evolved from the foundations laid by Vision Language Models (VLMs). Early based on BERT-style language decoders (Devlin, ), which achieved initial cross-modal integration by combining visual and textual data (Li et al., ). Leveraging open-source Large Language Models (LLMs) such as LLaMA families (Touvron et al., ), MLLMs (Alayrac et al., ; Wu et al., ) have demonstrated enhanced adaptability across a range of visual language tasks, leading to a more profound ability to interpret the world. Models like LLaVA (Liu et al., ), Qwen-VL (Bai et al., ), and GLM4V (Wang et al., ) have further advanced this field, enabling users to interact with these agents using both image and text prompts. These models adhere to two critical training phases: pre-training feature alignment and instruction fine-tuning, ensuring they better comprehend the format of instruction inputs (Yin et al., ). However, despite their impressive performance in many areas, MLLMs still suffer from hallucination issues. Thus, in this paper, we primarily conducted experiments and analysis on these three representative models.

Hallucinations in MLLMs.
Before LLMs, hallucination in NLP was mainly seen as generating nonsensical or deviant content. In MLLMs, “hallucination” is defined as the model generating content inconsistent with the provided image. This issue stems significantly from inadequate alignment among modalities. Several methods have been explored to mitigate hallucinations in MLLMs. Early efforts focused on fine-grained modality alignment (Rohrbach et al., ) and reducing co-occurrence biases (Kim et al., ) in small-scale VLMs, but these approaches struggle to scale with MLLMs. More recent strategies involve hallucination-targeted datasets for fine-tuning (Gunjal et al., ), post-hoc revisors (Zhou et al., ), and adopting RLHF (Yu et al., ). While effective, these methods are resource-intensive. CD-based approaches (Chuang et al., ; Leng et al., ) adjust the decoding distribution to mitigate hallucinations, but it does not consistently improve performance. Compared with them, our MemVR stands as “a paradigm of effectiveness and efficiency” in hallucination mitigation, effortlessly enhancing performance without extra training.

## 3 Methodology

In this section, we first formulate the generation process of MLLMs to facilitate a clearer understanding of our MemVR. Moreover, we introduce our hypothesis about the causes of hallucinations and discuss visual retracing and its dynamic strategy. Further, we conduct theoretical analysis.

### 3.1 MLLMs Generation Process

Typically, MLLMs are composed of a visual encoder ${C}_{v}$, a text embedding layer, $L$ number of transformer layers, and an affine layer $\zeta(\cdot)$ which predicts the distribution of the next token, with diverse modalities as inputs, e.g., images and text. Regardless of specific architectural variations, MLLMs commonly employ ${C}_{v}$ to extract vision tokens from the raw image and project it into modality-shared feature space via an MLP or Q-Former module (Wadekar et al., ). Aligned vision tokens are represented as $X_{v}=\{x_{1},x_{2},\dots,x_{n_{v}}\}$ and used as part of the LLM input alongside the text tokens $X_{q}=\{x_{n_{v}+1},x_{n_{v}+2},\dots,x_{n_{v}+n_{q}-1}\}$ that are embedded from tokenized text input by the embedding layer. Subsequently, the vision and text tokens are concatenated as the final input sequence and we denote it as $\{x_{i}\}_{1}^{t_{n}-1}$ where $t_{n}=n_{v}+n_{q}$, which is then fed into successive transformer layers. We denote the output of the $l$-th layer as $h_{t}^{(l)}$. Then, the vocabulary head $\zeta(\cdot)$ is used to predict the probability of the next token $x_{t}$ among the vocabulary set $\mathcal{X}$ as follows,

$$ $p(x_{t}\mid x_{<t})=\mathrm{softmax}(\zeta(h_{t}^{(L)}))_{x_{t}},x_{t}\in \mathcal{X}.$ (1) $$

### 3.2 What Causes Hallucinations

A hypothesis on the cause of hallucinations. Informed by the phenomenon of catastrophic forgetting (Zhai et al., ) in MLLMs, we argue that the capabilities of LLMs to comprehend and memorize different modalities are quite distinct. Taking image and text inputs as an example, since an image possesses a much higher information density than a piece of text, it is reasonable to assume that LLMs struggle to understand and memorize vision tokens compared to text tokens, prone to fantasies.

Uncertainty quantification. Following the DoLa (Chuang et al., ), we compute the probability of the next token via the vocabulary head $\zeta$ on each layer during reasoning. Then, we introduce an entropy-based metric (Farquhar et al., ) to quantify the output uncertainty as $u=\sum-p_{k}\log p_{k}/\log{K}$, where $\{p_{k}\}_{i=1}^{K}$.
With uncertainty, we make an important assumption.

###### Assumption 3.1 .

LUFH: The Lower the Uncertainty, the Fewer the Hallucinations to be generated.

We define $\gamma$ as the threshold that separates high uncertainty from low uncertainty in predictions. The candidate layer with uncertainty exceeding $\gamma$ is termed a premature layer. The proof of Assumption 3.1 (LUFH) is provided in Section 4.
Based on the hypothesis, we aim to complement visual evidence in MLLMs to eliminate the hallucination caused by visual forgetting. In the following, we discuss our motivation, how MemVR is implemented, and why it can work in Section 3.3 and 3.4.

###### Assumption 3.1 .

### 3.3 Relationship Between Hallucinations and Uncertainty

As findings of Chen et al. () in LLMs: “incorrect tokens generally exhibit higher entropy than correct ones”, we also observe this phenomenon in MLLMs (visualization cases are shown in Appendix C.3). This implies the effectiveness of entropy-based metrics for detecting hallucinations. In this work, we use uncertainty as the metric. We present our in-depth findings in this section.

Figure: Figure 3: Uncertainty of different early layers to predict the next token. Rows denote indices of the early layers, and column names are decoded tokens in each step. Uncertainty distribution is dynamic.
Refer to caption: x3.png

Finding #1: In the context of tokens involving objects, attributes or relations, uncertainty is high.
We conduct preliminary analysis with 32-layer LLaVA-1.5-7B. Specifically, we compute the uncertainty in the output distributions $p^{(l)}(\cdot\mid x_{<t})$ of early exiting layers. Figure 3 shows the uncertainty scores of different early layers when decoding the answer, we can observe that the computed uncertainty remains relatively high in later layers when predicting key entity objects, attributes, or relations, such as wooden, filled, and food in Figure 3. This phenomenon suggests that LLM is still uncertain about its predictions in the last few layers and may inject more factual knowledge into the predictions. On the other hand, when predicting function words and those tokens copied from the question, e.g., image, a, with, we observe that the uncertainty becomes very low from the middle layers. This finding implies that the model is deterministic for easy-to-predict tokens at the intermediate layer and keeps the distribution of outputs almost constant at higher layers, however, it is more uncertain for difficult-to-predict key tokens and may constantly change its predictions until the final layer.

Finding #2: The uncertainty distribution of different tokens is dynamic during the generation.
As can be seen in Figure 3, when visual factual knowledge is required for the prediction of the next token, such as wodden, filled, food, most layers of LLaVA are of higher uncertainty, and appear to shift the prediction of the later layers. The prediction of the next tokens such as quantity, target category, color, relation, etc. requires evident visual knowledge, but not for function words or commas

From a qualitative perspective, intermediate values of uncertainty reflect the thinking or revision process of MLLMs, while too high uncertainty implies that the model is confused and relies on random guesses. For complex input images, the uncertainty of the output distributions tends to rise, which can propagate through subsequent layers, resulting in incorrect predictions, or gibberish.

Figure: Figure 4: The illustration of how dynamic premature layer injection works.
Refer to caption: x4.png

### 3.4 Visual Retracing and Its Dynamics

Motivated by the above findings, we propose re-injecting visual evidence during elevated uncertainty in the model’s reasoning. This strategy treats visual tokens as anchors to recalibrate off-target predictions and reduces uncertainties in object, attribute, relationship tokens. Experimental results also demonstrate that our method reduces uncertainty and alleviates hallucinations as shown in Figure 9.
We term this schema of re-injecting visual evidence as “visual retracing” that is to be elaborated further in Section 3.4.1. Further, we design a dynamic injection strategy, detailed in Section 3.4.2, ensuring timely visual evidence when generating uncertain visual-reliant tokens (Figure 4).

#### 3.4.1 FFN with Visual Retracing

We introduce the implementation of our proposed memory-space visual retracing method. Previous study (Geva et al., ) has found that FFN acts as a key-value memory storing factual knowledge. Inspired by the fact that FFN executes analogous retrieval from its key-value memory, we consider “visual retracing” to serve as a simplified and efficient information re-retrieval process (Jie et al., ). Concretely, given an hidden token $x\in\mathbb{R}^{d}$ and
vision features $z_{v}=(z_{v,1},\dots,z_{v,n_{v}})^{\top}\in\mathbb{R}^{n_{v}\times d^{\prime}}$, $\mathcal{K}_{\propto}\left({z}_{v,i}\right),\mathcal{V}_{\propto}\left({z}_{v,
i}\right)\in\mathbb{R}^{d}$ are the key and value corresponding to ${z}_{v,i}$, visual retracing follows,

$$ ${\operatorname{FFN}}_{\circlearrowright}^{(l)}({x}\propto z_{v})=(1-\alpha) \operatorname{FFN}^{(l)}(x)+\alpha\underline{\Lambda}_{\circlearrowright}^{(l) }({z}_{v}\mid{x}),$ (2) $$

where $\alpha$ denotes the injection ratio of visual memory (proportional to image complexity), FFN comprises two FC layers with non-linear activation in between and can be formulated as $\operatorname{FFN}({x})=\phi\left({x}{W}_{1}\right){W}_{2}^{\top}$, $\phi$ is activation function like ReLU or SiLU (Liu et al., ). Separately, the weight matrices can be rewritten as: ${W}_{1}=\left({k}_{1},{k}_{2},\ldots,{k}_{m}\right),{W}_{2}=\left({v}_{1},{v}_
{2},\ldots,{v}_{m}\right)^{\top}$, in which ${k}_{i}\in\mathbb{R}^{d}$ and ${v}_{i}\in\mathbb{R}^{d}$ are entries of key and value, respectively. Thus, FFN can be interpreted as using input ${x}$ as the query to calculate its similarity with keys to search for matching values. Analogously, we consider a simple and efficient retrieval process for visual retracing on $l$-th premature layer as,

$$ $\underline{\Lambda}_{\circlearrowright}^{(l)}({z}_{v}\mid{x})=\sum\nolimits_{i =1}^{n_{v}}\phi\left(\left\langle{x},\mathcal{K}_{\propto}\left({z}_{v,i} \right)\right\rangle\right)\cdot\mathcal{V}_{\propto}\left({z}_{v,i}\right).$ (3) $$

From the perspective of FFN, we regard the $\left(\mathcal{K}_{\propto}\left({z}_{v,i}\right),\mathcal{V}_{\propto}\left({
z}_{v,i}\right)\right)$ as new key-value entries to supplement vision-related evidence. $\mathcal{K}_{\propto}$ and $\mathcal{V}_{\propto}$ are key functions for aligning the dimension between vision features $z_{v}$ and the hidden token $x$. In MLLMs, $\mathcal{K}_{\propto}$ and $\mathcal{V}_{\propto}$ are regarded as visual projectors, e.g., MLPs or Q-Former (Li et al., ), for aligning features of different modalities.

#### 3.4.2 Dynamic Premature Layer Injection

To magnify the effectiveness of visual retracing, the optimal premature layer should ideally be the layer most uncertain about probable answers to visual questions. In practice, we consider that the uncertainty of a candidate layer exceeding the threshold $\gamma$ warrants visual retracing. Inspired by the fact that early exit patterns (Teerapittayanon et al., ; Elbayad et al., ; Schuster et al., ) have proven effective in directly employing the language heads $\zeta$ to the hidden states of the middle layers, even without a special training process (Kao et al., ), we compute the uncertainty of the next token probability on the early layers for reasoning. As our Finding #2, we utilize layer-specific uncertainty to allow for dynamic premature layer injection at each time step as illustrated in Figure 3.

Figure: Algorithm 1 Dynamic Injection Strategy

For MLLMs with different numbers of layers, we partition the layers into several (typically three) buckets according to the total layers for sensible visual retracing triggers, as detailed in Appendix C.1. This dynamic injection strategy identifies desirable premature layers for visual retracing based on output uncertainty, thus better leveraging insights from different layers.

Static Fixed Layer Injection. In addition to the dynamic premature layer injection strategy, another more straightforward strategy worth considering is to perform a brute-force experiment on all possible early layers using a validation set and selecting the layer with the best average performance. We refer to this simple strategy as MemVR-static.
However, the MemVR-static approach presents two limitations: (I) it requires more extensive hyperparameter tuning across layers, and (II) the optimal layer is highly sensitive to data distribution, necessitating an in-distribution validation set. In contrast, our proposed dynamic layer injection strategy mitigates these challenges by reducing the layer search space and improving robustness without depending on in-distribution validation. Empirical comparisons between our MemVR using dynamic and static strategies are provided in Section 4.2 and Table 4.

### 3.5 Theoretically Understanding Why MemVR Works

In order to gain further insight into the reasons behind the effectiveness of MemVR in mitigating hallucinations and its robust performance on general benchmarks, we attempt to explain these phenomena via the following three theorems from an information-theoretic perspective.

###### Theorem 3.1 .

Let ${H}_{vq}$ be the hidden states of FFN and $\hat{H}_{vq}$ be after reinjection of visual evidence $Z_{v}$. MemVR enhances Mutual Information (MI) between $\hat{H}_{vq}$ and $Z_{v}$:
$I(\hat{H}_{vq};Z_{v})\geq I(H_{vq};Z_{v})$.

The reinjection of $Z_{v}$ at intermediate layers of the model facilitates the replenishment of critical visual information, which may have been lost or distorted through earlier layers. This process increases MI between the hidden states and the visual features, ensuring adequate visual context is preserved.

###### Theorem 3.2 .

Let $Y$ be the target output dependent on hidden states. If MI between ${H}_{vq}$ and ${Z}_{v}$ increases, then conditional entropy $H(Y\mid{H}^{(l)}_{vq})$ decreases with
$H(Y\mid\hat{H}_{vq})\leq H(Y\mid H_{vq})$.

According to Theorem 3.1, 3.2, and DPI (Cover et al., ), MemVR improves the quality of the representations $H_{vq}$ and reduces the uncertainty in the output $Y$. As a result, the probability of hallucinations decreases, which is consistent with the observed findings of hallucination mitigation.

###### Theorem 3.3 .

Within the Information Bottleneck (IB) framework, the loss of objective function, represented by the notation $\mathcal{L}(T)$, is optimized by MemVR, which is defined as $\mathcal{L}(\hat{H}_{vq})\leq\mathcal{L}(H_{vq})$, where $\mathcal{L}(H_{vq})=I(H_{vq};X_{vq})-\beta I(H_{vq};Y)$ is IB loss, and $\ beta$ is a trade-off parameter.

Anchored in the information bottleneck framework, MemVR optimizes the delicate balance between retaining relevant information from multimodal inputs and compressing non-essential details, thereby safeguarding the predictive performance of the hidden representations for the target output $Y$.

The theoretical underpinnings of MemVR are supported by the Data Processing Inequality (Cover et al., ) and the contraction properties of stochastic mappings in deep neural networks, as shown in various studies on the Information Bottleneck Principle (Achille & Soatto, ). By enhancing mutual information and reducing the uncertainty in hidden states, MemVR effectively mitigates hallucinations while preserving computational efficiency. Detailed proofs are in Appendix B.

**Table 2: Performance evaluation on POPE. The best results in each scenario are bolded for clarity. We report the averages under the three settings, e.g., *Random*, *Popular*, and *Adversarial* to show the robustness of the different methods directly. Green denotes improvement, and Red means degradation.**
| Method | MSCOCO | A-OKVQA | GQA |  |  |  |
| --- | --- | --- | --- | --- | --- | --- |
| %Accuracy | %F1 Score | %Accuracy | %F1 Score | %Accuracy | %F1 Score |  |
| LLaVA1.5-7B | 81.38  $\uparrow$0.0 | 79.65  $\uparrow$0.0 | 79.13  $\uparrow$0.0 | 79.10  $\uparrow$0.0 | 79.00  $\uparrow$0.0 | 79.13  $\uparrow$0.0 |
| + VCD (Leng et al., ) | 84.66  $\uparrow$3.3 | 84.52  $\uparrow$4.9 | 80.99  $\uparrow$1.8 | 82.30  $\uparrow$3.2 | 81.74  $\uparrow$2.7 | 82.16  $\uparrow$3.0 |
| + OPERA (Huang et al., ) | 84.77  $\uparrow$3.4 | 85.46  $\uparrow$5.8 | 84.27  $\uparrow$5.1 | 84.08  $\uparrow$5.0 | 84.03  $\uparrow$5.0 | 83.83  $\uparrow$4.7 |
| + MemVR (Ours) | 87.00  $\uparrow$5.7 | 85.87  $\uparrow$6.2 | 86.21  $\uparrow$7.0 | 86.64  $\uparrow$7.5 | 85.25  $\uparrow$6.2 | 85.59  $\uparrow$6.4 |
| Qwen-VL-10B | 83.79  $\uparrow$0.0 | 81.13  $\uparrow$0.0 | 84.74  $\uparrow$0.0 | 83.27  $\uparrow$0.0 | 84.41  $\uparrow$0.0 | 82.66  $\uparrow$0.0 |
| + VCD (Leng et al., ) | 84.27  $\uparrow$0.4 | 82.12  $\uparrow$1.0 | 84.09  $\downarrow$0.7 | 82.53  $\downarrow$0.7 | 83.73  $\downarrow$0.7 | 82.75  $\uparrow$0.1 |
| + OPERA (Huang et al., ) | 84.93  $\uparrow$1.1 | 83.41  $\uparrow$2.3 | - | - | - | - |
| + MemVR (Ours) | 84.07  $\uparrow$0.3 | 81.55  $\uparrow$0.4 | 86.43  $\uparrow$1.8 | 85.56  $\uparrow$2.3 | 85.69  $\uparrow$1.3 | 84.53  $\uparrow$1.9 |

**Table 3: Results on the hallucination subset of MME (including commonsense reasoning, existence, count, position, color scores). The best are in bold. MemVR achieves dramatic improvements.**
| Method | Commonsense QA(Reasoning) | Object-level Hallucination | Attribute-level Hallucination | Total Scores |  |  |
| --- | --- | --- | --- | --- | --- | --- |
| Existence | Count | Position | Color |  |  |  |
| LLaVA1.5-7B | 110.71  $\uparrow$0.0 | 175.67  $\uparrow$0.0 | 124.67  $\uparrow$0.0 | 114.00  $\uparrow$0.0 | 151.00  $\uparrow$0.0 | 676.05 |
| + VCD (Leng et al., ) | 112.86  $\uparrow$9.9 | 184.66  $\uparrow$9.0 | 138.33  $\uparrow$13.6 | 128.67  $\uparrow$14.6 | 153.00  $\uparrow$2.0 | 717.52 |
| + OPERA (Huang et al., ) | 115.71  $\uparrow$5.5 | 180.67  $\uparrow$5.0 | 133.33  $\uparrow$8.6 | 123.33  $\uparrow$9.3 | 155.00  $\uparrow$4.0 | 708.04 |
| + MemVR (Ours) | 121.42  $\uparrow$18.5 | 190.00  $\uparrow$14.3 | 155.00  $\uparrow$30.3 | 133.33  $\uparrow$19.3 | 170.00  $\uparrow$19.0 | 769.75 |
| Qwen-VL-10B | 106.40  $\uparrow$0.0 | 155.00  $\uparrow$0.0 | 127.67  $\uparrow$0.0 | 131.67  $\uparrow$0.0 | 173.00  $\uparrow$0.0 | 693.74 |
| + VCD (Leng et al., ) | 104.33  $\downarrow$2.1 | 156.00  $\uparrow$1.0 | 131.00  $\uparrow$3.3 | 128.00  $\downarrow$3.6 | 181.67  $\uparrow$8.6 | 701.00 |
| + OPERA (Huang et al., ) | 104.33  $\uparrow$2.2 | 165.00  $\uparrow$6.9 | 145.00  $\uparrow$4.8 | 133.33  $\uparrow$1.6 | 180.00  $\uparrow$7.0 | 727.66 |
| + MemVR (Ours) | 120.00  $\uparrow$13.6 | 185.00  $\uparrow$30.0 | 145.00  $\uparrow$17.3 | 123.33  $\downarrow$8.3 | 185.00  $\uparrow$12.0 | 758.33 |

###### Theorem 3.1 .

###### Theorem 3.2 .

###### Theorem 3.3 .

## 4 Experiments

This section details the evaluation of our MemVR across three MLLMs on seven benchmarks.

### 4.1 Experiment Setup

Datasets and Metrics.
To rigorously assess the effectiveness of our proposed method, we conduct a comprehensive set of experiments across two benchmarks specifically designed to evaluate hallucination mitigation and five general-purpose benchmarks to gauge the general performance:

- ❶
Hallucination benchmarks: Polling-based Object Probing Evaluation (POPE) (Li et al., ), and Caption Hallucination Assessment with Image
Relevance (CHAIR) (Rohrbach et al., );
- ❷
General-purpose benchmarks: VizWiz-VQA (Gurari et al., ), MLLM Comprehensive Evaluation (MME) (Fu et al., ), Multimodal Benchmark (MMBench) (Liu et al., ), Multimodal Veterinarian (MM-Vet) (Yu et al., ), LLaVA-Bench (in-the-wild) (Liu et al., ).

More detailed information on these various benchmarks can be obtained from the Appendix C.1.

Backbones and Baselines.
To evaluate our method, we utilize three well-known MLLMs: LLaVA-1.5 (Liu et al., ), Qwen-VL (Bai et al., ), and GLM4V (Wang et al., ). Further, We compare our methods with classic training-tree SOTA methods designed to mitigate object hallucination, including visual contrastive decoding SOTA VCD Leng et al. (), OPERA (Huang et al., ) based on overconfidence penalty and hindsight allocation. As Dola (Chuang et al., ) is layer-wise contrastive decoding for LLMs and performs poorly in MLLMs, it will not be shown in the experiment. Experimental results are obtained and benchmarked using unified implementation.

Implementation Details.
Greedy search is used as the default decoding strategy in MemVR for all benchmarks. For benchmarks, annotation questions are adapted to MLLM templates. For POPE, COCO, A-OKVQA, and GQA are used, while MMBench_DEV_EN is used for MMBench. MM-Vet is assessed using MM-Vet Online Evaluator, and gpt4-1106-preview is used for LLaVA-Bench. CHAIR uses images from COCO Val2014 with the query ”Please describe this image in detail”. In MemVR, do_sample=False, temperature=0, threshold=0.75, beam=1. All settings of the compared method follow the default configurations from the original papers. More details are in Appendix C.1.

Figure: Figure 5: Results on MMBench. MemVR enhances comprehensive performance on diverse tasks.
Refer to caption: x5.png

**Table 4: Results of MemVR-static and MemVR-dynamic on MME. Static-# indicates fixation on layer # for visual retracing. MemVR-dynamic achieves optimal performance improvements.**
| Strategy | Static-3 | Static-7 | Static-11 | Static-15 | Static-19 | Static-23 | Static-27 | Static-31 | Dynamic |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Preception | 1508.08 | 1500.48 | 1526.40 | 1529.07 | 1513.96 | 1500.29 | 1510.47 | 1510.34 | 1512.80 |
| Cognition | 364.64 | 347.14 | 362.86 | 352.14 | 350.36 | 357.86 | 355.71 | 346.43 | 383.92 |
| Total Score | 1872.72 | 1847.62 | 1889.26 | 1881.22 | 1864.32 | 1858.15 | 1866.18 | 1856.77 | 1896.72 |

### 4.2 Quantitative Results

##### Key Finding: MemVR consistently outperforms the baselines in mitigating hallucinations and improving overall accuracy across various scenarios.

Figure: Figure 6: Comparison of different injection ratios $\alpha$ on cognition scores.
Refer to caption: x6.png

##### Q1: How does MemVR performance on hallucination benchmark?

We conduct POPE, CHAIR, and MME evaluations, as shown in Table 2, Table 3 and Table 5, our MemVR obviously surpasses all of the compared baselines. For the results of POPE evaluation, we observe that our proposed MemVR presents robust effects. The performance of MemVR surpasses the baseline results by large margins, i.e., average up to +7.0% accuracy and +7.5% F1 score on A-OKVQA dataset under Random, Popular, and Adversarial settings.
As showcased in Table 5, our MemVR achieves up to 15.6% improvements on CHAIRI compared with vanilla LLaVA-1.5-7B. For MME subset evaluations (encompass both object-level and attribute-level hallucinations), results in Table 3 show that MemVR achieves a uniform improvement in handling object-level and attribute-level hallucinations, as well as commonsense reasoning. Existence, Count, and Color scores all achieve dramatic improvements (Existence score in Qwen-VL up 30). On the contrary, Position scores are relatively low, which suggests weak position reasoning capability in MLLMs.

**Table 5: CHAIR hallucination evaluation results of LLaVA. Small values correspond to fewer hallucinations.**
| Model | Length | CHAIRS $\downarrow$ | CHAIRI $\downarrow$ | Recall $\uparrow$ |
| --- | --- | --- | --- | --- |
| LLaVA-1.5 | 100.6 | 50.0  $\uparrow$0.0 | 15.4  $\uparrow$0.0 | 77.1  $\uparrow$0.0 |
| + VCD | 100.4 | 48.6  $\downarrow$1.4 | 14.9  $\downarrow$0.5 | 77.3  $\uparrow$0.2 |
| + OPERA | 98.6 | 47.8  $\downarrow$2.2 | 14.6  $\downarrow$0.8 | 76.8  $\downarrow$0.3 |
| + MemVR | 99.6 | 46.6  $\downarrow$3.4 | 13.0  $\downarrow$2.4 | 80.8  $\uparrow$3.7 |

##### Q2: How well does MemVR perform on general-purpose benchmarks?

We evaluate the performance of MemVR on general-purpose benchmarks, i.e., VizWiz, MME, MMBench, MM-Vet, and LLaVA-Bench. Appx. C summarizes the results on MMBench, highlighting MemVR’s comparative performance relative to SOTA methods. As shown in Table 6, MemVR consistently outperforms competing models. Besides, MemVR achieves a significant improvement in overall performance listed in Table 7, with an average increase of 6.1% in OCR and spatial awareness tasks, demonstrating superior generalization capabilities. These results indicate that compared with CD-based methods, MemVR excels in hallucination mitigation and delivers competitive performance on general-purpose benchmarks. More complete results are in Appendix C.

**Table 6: LLaVA-Bench evaluation results.**
| Model | Convs $\uparrow$ | Detail $\uparrow$ | Complex $\uparrow$ | All $\uparrow$ |
| --- | --- | --- | --- | --- |
| LLaVA-1.5 | 58.8  $\uparrow$0.0 | 52.1  $\uparrow$0.0 | 74.6  $\uparrow$0.0 | 63.4  $\uparrow$0.0 |
| + VCD | 57.8  $\downarrow$1.0 | 50.8  $\downarrow$1.3 | 77.9  $\uparrow$3.3 | 59.1  $\downarrow$4.3 |
| + OPERA | 59.5  $\uparrow$0.7 | 49.6  $\downarrow$2.5 | 78.6  $\uparrow$4.0 | 59.8  $\downarrow$3.6 |
| + MemVR | 63.8  $\uparrow$0.5 | 52.6  $\uparrow$0.5 | 77.9  $\uparrow$3.3 | 64.0  $\uparrow$0.6 |

##### Q3: How efficient is MemVR ?

MemVR operates dynamically based on the uncertainty, which employs visual retracing when the uncertainty exceeds threshold $\gamma$ on the early layer. If the uncertainty remains low across all layers—indicating that the model is highly confident in its generated results—MemVR is not triggered. This mechanism ensures efficient inference without extra computational overhead. Compared with VCD and OPERA, they need inference twice or the rollback strategy leads to exponentially added overheads, our MemVR only once regular inference.

Figure: Figure 7: MME evaluation results under different mixing ratios of noise and visual features.
Refer to caption: x7.png

##### Q4: How does visual uncertainty affect MemVR ?

To examine the impact of visual uncertainty on MemVR ’s performance, we introduced Gaussian noise (McHutchon & Rasmussen, ) into the extracted visual features. We gradually increased the noise level to assess how MemVR ’s performance would respond, using MME as the benchmark and LLaVA as the backbone model. As illustrated in Figure 7, both the perception and cognition scores declined as the noise step increased. At high noise, the performance fell significantly below the baseline. This demonstrates that MemVR is sensitive to the quality of visual features and is efficient at understanding shallow information.

Figure: Figure 8: Results of GLM4V-9B. MemVR enhances comprehensive performance on diverse tasks.
Refer to caption: x8.png

**Table 7: MM-Vet evaluation results with multiple complicated multimodal tasks, where gn denotes language generation, kw means knowledge, sp denotes spatial awareness, and rec is recognition.**
| Method | gn_kw_rec | rec | ocr_sp | ocr | ocr_sp_rec | ocr_kw_rec | ocr_gn_sp | Total |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
| LLaVA1.5-7B | 18.1  $\uparrow$0.0 | 67.6  $\uparrow$0.0 | 17.7  $\uparrow$0.0 | 48.3  $\uparrow$0.0 | 60.0  $\uparrow$0.0 | 21.2  $\uparrow$0.0 | 10.0  $\uparrow$0.0 | 31.1  $\uparrow$0.0 |
| + VCD | 19.2  $\uparrow$1.1 | 62.2  $\downarrow$5.4 | 15.8  $\downarrow$1.9 | 29.2  $\downarrow$20.0 | 42.5  $\downarrow$17.5 | 17.5  $\downarrow$3.7 | 60.0  $\uparrow$50.0 | 30.2  $\downarrow$1.1 |
| + OPERA | 21.8  $\uparrow$3.7 | 61.9  $\downarrow$5.7 | 21.5  $\uparrow$3.8 | 51.7  $\uparrow$3.4 | 56.2  $\downarrow$3.8 | 11.2  $\downarrow$10.0 | 30.0  $\uparrow$1.4 | 32.0  $\uparrow$0.9 |
| + MemVR | 19.5  $\uparrow$1.4 | 70.3  $\uparrow$2.7 | 23.8  $\uparrow$6.1 | 48.3  $\uparrow$0.0 | 58.8  $\downarrow$1.2 | 21.2  $\uparrow$0.0 | 30.0  $\uparrow$20.0 | 32.4  $\uparrow$1.3 |

Figure: Figure 9: Visualisation of uncertainty across layers without and with MemVR. MemVR effectively reduces uncertainty after 8th layer, contributing to hallucination mitigations as Assumption 3.1.
Refer to caption: x9.png

##### Q5: How does layer uncertainty assist with visual retracing?

MemVR leverages layer entropy to trigger visual retracing dynamically. We compared the performance of fixed retracing layers with dynamic retracing layers and found that dynamic retracing outperformed fixed retracing, as demonstrated in Table 4. Under all experimental conditions, dynamic retracing achieved the highest total score on the MME evaluation using LLaVA. Furthermore, we analyzed the injection ratio for MemVR, with the optimal injection ratio observed around 0.15, as shown in Table 4. This indicates that a balanced level of $\alpha$ is crucial for maximizing performance.

##### Q6: Can MemVR adapt to other MLLMs?

Definitely, MemVR is designed to be flexible and compatible with various architectures. A key advantage of MemVR lies in its ability to function without requiring modifications to the Transformers library, facilitating smooth integration into both older and cutting-edge models. We conducted extensive experiments across multiple benchmarks with LLaVA, Qwen-VL, and GLM-4V, and all three models outperformed baselines.

### 4.3 Qualitative Study: What types of faults can our method address?

Figure: Figure 10: A case study in long text generation. MemVR effectively mitigates hallucinations.
Refer to caption: x10.png

In addition to evaluating single-word question-answering (QA) benchmarks, we further explore the models’ ability to generate comprehensive long-text descriptions for various tasks. As depicted in Figure 10, our MemVR excels in accurately identifying question-relevant details within images. In contrast, as discussed in Appendix C, Qwen-VL-Chat exhibits occasional difficulties in producing detailed image descriptions when applied to VCD. These shortcomings become evident in scenarios requiring a nuanced interpretation of image content. This suggests that MemVR demonstrates superior adaptability across different architectures, enabling more reliable long-text generation.

### 4.4 Limitations and Further Discussions

While MemVR demonstrates significant promise in improving the performance in MLLMs, it is not without limitations. One major challenge lies in the variability of MLLM architectures. Different MLLMs employ varying FNNs, activation functions, and knowledge systems, making it difficult to identify the optimal hyperparameters—such as injection ratios $\alpha$, and premature layer for each specific model. This requires considerable effort in model-specific tuning, which may limit the scalability of MemVR without further automation or standardization in hyperparameter selection.

Additionally, although our research focused on MLLMs with image inputs, MemVR is theoretically applicable to LLMs with different modalities. However, we have yet to explore its performance on inputs such as voice, point clouds, 3D meshes, or video frames. This opens up an exciting avenue for future work, where we plan to extend MemVR ’s framework to these diverse input formats and assess its efficacy across a broader range of tasks. Furthermore, we will explore integrating MemVR into the training procedures of MLLMs, rather than limiting its application to inference, to evaluate whether this could lead to even greater improvements in model performance and generalization.

## 5 Conclusion

This paper proposes a novel training-free paradigm to mitigate hallucination, named MemVR. In contrast to previous CD-based methods, which primarily focus on
eliminating biases of language priors, MemVR seeks to replenish question-relevant visual clues towards more evidential answers, which signifies the other side of the coin. Our experiments, conducted on seven benchmarks, demonstrate the effectiveness of MemVR in mitigating hallucination and improving general performance.

## Appendix A Ethic Considerations

We list some key ethical considerations of our method:

Bias and fairness. By injecting visual features from CLIP into the Feed-Forward Network (FFN) layers of LLMs, there’s a potential for inherited biases from the original models. CLIP, like many pre-trained models, may contain biases in how it represents certain objects, scenes, or demographics. These biases could propagate, affecting performance on different types of data (e.g., gender, ethnicity, cultural contexts). It’s important to evaluate how MemVR performs across diverse datasets and ensure that it doesn’t reinforce harmful stereotypes or disproportionately fail for certain groups.

Misuse of Enhanced Models. As MemVR aims to improve the long-text generation and overall performance of VLMs, enhanced models could be misused to generate deceptive or misleading content, such as deepfakes or disinformation. It’s important to consider whether there are safeguards in place to prevent malicious use of these improved models in scenarios like automated misinformation campaigns or unethical surveillance.

Data Privacy. If the benchmarks used for evaluating MemVR include datasets with personally identifiable information or sensitive content, care should be taken to ensure data privacy. Models should be evaluated on publicly available, anonymized, or ethically sourced datasets to avoid violating privacy laws or ethical norms.

## Appendix B Theoretical Analysis of MemVR in MLLMs

In Multimodal Large Language Models (MLLMs), hallucinations often arise due to insufficient alignment between visual inputs and the model’s internal representations. This paper provides a rigorous theoretical analysis demonstrating that re-injecting visual features into the intermediate layers of MLLMs mitigates hallucinations and enhances representation capability.

We demonstrate that MemVR increases Mutual Information (MI) between the hidden states and visual tokens, decreasing the conditional entropy of outputs given the hidden state for fidelity to the visual input. We begin by defining the relevant variables and information-theoretic concepts that will be used throughout the proof as, $X_{vq}$ denote concatenated tokens of text and vision, with probability distribution $p({X}_{vq})$; ${Z}_{v}$ means visual (image) features, with probability distribution $p({Z}_{v})$; The output hidden states of the Transformer model at layer $k$, defined recursively as: $H_{vq}^{(k)}=f^{(k)}({H}_{vq}^{(k-1)},\mathbf{1}_{k=m}Z_{v})$, where $\mathbf{1}_{k=m}$ is the indicator function that equals 1 when $k=m$ (the layer where ${Z}_{v}$ is rejected) and 0 otherwise, and $Y$ denotes the target output of MLLMs.

The probability of hallucination can be expressed as:

$$ $P_{\text{hallucination}}=P(Y\neq Y^{*}\mid X_{vq}),$ $$

where $Y^{*}$ is the ground truth output. According to information theory, a higher conditional entropy $H(Y\mid X_{vq})$ indicates greater uncertainty of $Y$ given $X_{vq}$, which increases the probability of hallucination.

Information Flow of Visual Features.
In a standard Transformer model, the initial input $X_{vq}$ undergoes multiple layers of processing. As the number of layers increases, the initial visual information may gradually diminish (information forgetting). In the absence of MemVR, the MI between the hidden states and the visual features $Z_{v}$ tends to decrease with depth:

$$ $I(H_{vq}^{(l)};Z_{v})\leq I(H_{vq}^{(l-1)};Z_{v}),$ $$

for $l>1$. This inequality indicates that in deeper layers, $H_{vq}^{(l)}$ contains less vision-related information.

###### Theorem B.1 .

Assume that each Transformer layer acts as a deterministic or stochastic mapping with the Markov property. Then, the mutual information between the hidden states and the visual features decreases with depth:

$$ $I(H_{vq}^{(l)};Z_{v})\leq I(H_{vq}^{(l-1)};Z_{v}).$ $$

###### Proof.

Each Transformer layer can be modeled as a stochastic mapping (Markov kernel) that processes the input hidden states. Specifically, $H_{vq}^{(l)}$ is a function of $H_{vq}^{(l-1)}$, possibly incorporating additional inputs such as $Z_{v}$ at specific layers.

According to the Data Processing Inequality (DPI) (Cover et al., ), if $A\rightarrow B\rightarrow C$ forms a Markov chain, then:

$$ $I(A;C)\leq I(A;B).$ $$

In this context, consider $A=Z_{v}$, $B=H_{vq}^{(l-1)}$, and $C=H_{vq}^{(l)}$. Since $H_{vq}^{(l)}$ is generated from $H_{vq}^{(l-1)}$ without direct access to $Z_{v}$, we have the Markov chain $Z_{v}\rightarrow H_{vq}^{(l-1)}\rightarrow H_{vq}^{(l)}$. Applying DPI yields:

$$ $I(Z_{v};H_{vq}^{(l)})\leq I(Z_{v};H_{vq}^{(l-1)}).$ $$

Thus, mutual information between the hidden states and the visual features does not increase with depth.
∎

Visual Retracing in MLLMs. We reinject vision tokens $Z_{v}$ on $l$-th layer ($ahead_layer\leq l<L$):

$$ $\hat{H}_{vq}^{(l)}=\operatorname{FFN}_{\circlearrowright}^{(l)}(H_{vq}^{(l)} \propto Z_{v}).$ $$

MemVR ensures that after the $l$-th layer, $\hat{H}_{vq}^{(l)}$ again contains question-aligned visual information.

###### Theorem B.2 .

Let ${H}_{vq}$ be the hidden states of FFN and $\hat{H}_{vq}$ be after reinjection of visual evidence $Z_{v}$. MemVR enhances Mutual Information (MI) between $\hat{H}_{vq}$ and $Z_{v}$:
$I(\hat{H}_{vq};Z_{v})\geq I(H_{vq};Z_{v})$.

###### Proof.

We aim to show that reinjecting $Z_{v}$ at layer $l$ increases the mutual information between the hidden states and $Z_{v}$ conditioned on $X_{vq}$.

By the definition of conditional mutual information:

$$ $I(\hat{H}_{vq}^{(l)};Z_{v}\mid X_{vq})=\mathbb{E}_{X_{vq}}[I(\hat{H}_{vq}^{(l) };Z_{v}\mid X_{vq}=x)].$ $$

Similarly,

$$ $I(H_{vq}^{(l)};Z_{v}\mid X_{vq})=\mathbb{E}_{X_{vq}}[I(H_{vq}^{(l)};Z_{v}\mid X _{vq}=x)].$ $$

Given $\hat{H}_{vq}^{(l)}=\operatorname{FFN}^{(l)}_{\Diamond}(H_{vq}^{(l)}\propto Z_{
v})$ denotes the hidden states after utilizing MemVR on $l$-th, reinjection of $Z_{v}$ introduces a direct dependency between $\hat{H}_{vq}^{(l)}$ and $Z_{v}$ beyond what is present in $H_{vq}^{(l)}$.
Since $\operatorname{FFN}^{(l)}_{\Diamond}$ is a deterministic function that incorporates $Z_{v}$, the mutual information $I(\hat{H}_{vq}^{(l)};Z_{v}\mid X_{vq})$ is at least as large as $I(H_{vq}^{(l)};Z_{v}\mid X_{vq})$. $\hat{H}_{vq}^{(l)}$ retains all information in $H_{vq}^{(l)}$ and additionally incorporates information from $Z_{v}$. Thus, MemVR ensures that:

$$ $I(\hat{H}_{vq}^{(l)};Z_{v}\mid X_{vq})\geq I(H_{vq}^{(l)};Z_{v}\mid X_{vq}).$ $$

By directly incorporating $Z_{v}$ into the computation of $\hat{H}_{vq}^{(m)}$, MemVR ensures that the hidden states retain more information about the visual features relative to the original hidden states $H_{vq}^{(m)}$, thereby increasing $I(\hat{H}_{vq}^{(m)};Z_{v}\mid X_{vq})$, enhancing the representation capability and utilizing visual information.
∎

###### Theorem B.3 .

Let $Y$ be the target output dependent on hidden states. If MI between ${H}_{vq}^{(l)}$ and ${Z}_{v}$ increases, then conditional entropy $H(Y\mid{H}^{(l)}_{vq})$ decreases, leading to a lower probability of hallucinations:

$$ $H(Y\mid\hat{H}_{vq}^{(l)})\leq H(Y\mid H_{vq}^{(l)}).$ $$

###### Proof.

We aim to show that an increase in mutual information between $\hat{H}_{vq}^{(l)}$ and $Z_{v}$ conditioned on $X_{vq}$ leads to a decrease in the conditional entropy $H(Y\mid\hat{H}_{vq}^{(l)})$. According to the definition of conditional entropy, we have,

$$ $\displaystyle H(Y\mid\hat{H}_{vq}^{(l)})=H(Y)-I(Y;\hat{H}_{vq}^{(l)}),$ $\displaystyle H(Y\mid H_{vq}^{(l)})=H(Y)-I(Y;H_{vq}^{(l)}).$ $$

From Theorem B.2: $\hat{H}_{vq}^{(l)}$ contains more information about $Z_{v}$, i.e., $I(\hat{H}_{vq}^{(l)};Z_{v})\geq I(H_{vq}^{(l)};Z_{v})$. There is $I(Y;\hat{H}_{vq}^{(l)})\propto I(\hat{H}_{vq}^{(l)};Z_{v})$, thus we have $I(\hat{H}_{vq}^{(l)};Y)\geq I(H_{vq}^{(l)};Y)$. Then, we assume a dependency between $Z_{v}$ and $Y$, i.e., $I(Z_{v};Y)>0$, and subtract the inequalities, have:

$$ $\displaystyle H(Y\mid\hat{H}_{vq}^{(l)})$ $\displaystyle=H(Y)-I(Y;\hat{H}_{vq}^{(l)})$ $\displaystyle\leq H(Y)-I(Y;{H}_{vq}^{(l)})$ $\displaystyle=H(Y\mid H_{vq}^{(l)}).$ $$

Thus, MemVR reduces the conditional uncertainty of the target output given the intermediate embedding, thereby mitigating the probability of hallucinations and improving the model’s predictive capability.
∎

###### Theorem B.4 .

Within the Information Bottleneck (IB) framework, reinjecting $Z_{v}$ at layer $m$ optimizes the objective function:

$$ $\mathcal{L}(\hat{H}_{vq}^{(m)})\leq\mathcal{L}(H_{vq}^{(m)}),$ $$

where the IB objective is defined as:

$$ $\mathcal{L}(H)=I(H;X_{vq})-\beta I(H;Y),$ $$

and $\beta$ is a trade-off parameter.

###### Proof.

The Information Bottleneck (IB) objective aims to find a representation $H$ that maximizes the mutual information with the target $Y$ while minimizing the mutual information with the input $X_{vq}$. The optimization objectives before & after MemVR are as follows:

$$ $\displaystyle\mathcal{L}$ $\displaystyle=I(H_{vq}^{(l)};X_{vq})-\beta I(H_{vq}^{(l)};Y),$ $\displaystyle\mathcal{L}_{\Diamond}$ $\displaystyle=I(\hat{H}_{vq}^{(l)};X_{vq},Z_{v})-\beta I(\hat{H}_{vq}^{(l)};Y),$ $$

where $I(\hat{H}_{vq}^{(l)};X_{vq},Z_{v})=I(\hat{H}_{vq}^{(l)};X_{vq})+I(\hat{H}_{vq}
^{(l)};Z_{v}\mid X_{vq})$. The gap in the objective function is:

$$ $\displaystyle\Delta\mathcal{L}$ $\displaystyle=\mathcal{L}_{\Diamond}^{(l)}-\mathcal{L}^{(l)}$ $\displaystyle=[I(\hat{H}_{vq}^{(l)};X_{vq})+I(\hat{H}_{vq}^{(l)};Z_{v}\mid X_{ vq})-\beta I(\hat{H}_{vq}^{(l)};Y)]-[I(H_{vq}^{(l)};X_{vq})-\beta I(H_{vq}^{(l )};Y)]$ $\displaystyle=[I(\hat{H}_{vq}^{(l)};X_{vq})-I(H_{vq}^{(l)};X_{vq})]+I(\hat{H}_ {vq}^{(l)};Z_{v}\mid X_{vq})-\beta[I(\hat{H}_{vq}^{(l)};Y)-I(H_{vq}^{(l)};Y)].$ $$

To ensure that $\mathcal{L}_{\Diamond}^{(m)}\leq\mathcal{L}^{(m)}$, we require: $\Delta\mathcal{L}\leq 0.$ We define the changes in mutual information. Let $\Delta I_{X}=I(H_{vq}^{(l)};X_{vq})-I(H_{vq}^{(l-1)};X_{vq})$, $\Delta I_{Y}=I(H_{vq}^{(l)};Y)-I(H_{vq}^{(l-1)};Y)$. Note that $I(H_{vq}^{(l)};Z_{v}\mid X_{vq})\geq 0$. For $\Delta I_{X}$, the change in mutual information between $H_{vq}^{(l)}$ and $X_{vq}$ depends on how much additional information from $Z_{v}$ affects the dependence on $X_{vq}$. We denote the maximum possible increase as $\Delta I_{X}^{\max}$. For $\Delta I_{Y}$, From Theorem B.2, $\Delta I_{Y}\geq 0$, and suppose we can establish a minimum increase $\Delta I_{Y}^{\min}>0$. $I(H_{vq}^{(l)};Z_{v}\mid X_{vq})$ represents supplement information about $Z_{v}$ in $H_{vq}^{(l)}$ that is not already explained by $X_{vq}$, and we denote this maximum as $I_{\max}^{Z\mid X}$.

To satisfy this inequality, choose $\beta$ such that:

$$ $\Delta\mathcal{L}\leq 0\Rightarrow\beta\Delta I_{Y}\geq\Delta I_{X}+I(H_{vq}^{ (l)};Z_{v}\mid X_{vq}).$ (4) $$

Upper Bound on $\Delta I_{X}$ and $I(H_{vq}^{(l)};Z_{v}\mid X_{vq})$ as $\Delta I_{X}\leq\Delta I_{X}^{\max}$, $I(H_{vq}^{(l)};Z_{v}\mid X_{vq})\leq I_{\max}^{Z\mid X}$. Lower Bound on $\Delta I_{Y}$ as: $\Delta I_{Y}\geq\Delta I_{Y}^{\min}>0$.
Then, we derive the condition with error bounds, for $\Delta\mathcal{L}\leq 0$, it suffices that:

$$ $\beta\Delta I_{Y}^{\min}\geq\Delta I_{X}^{\max}+I_{\max}^{Z\mid X}\Rightarrow \beta\geq\frac{\Delta I_{X}^{\max}+I_{\max}^{Z\mid X}}{\Delta I_{Y}^{\min}}.$ (5) $$

This condition provides a lower bound for $\beta$ to ensure that reinjecting $Z_{v}$ at layer $m$ decreases the IB objective function. By adhering to this condition, MemVR optimizes the IB objective, balancing the trade-off between the compression of input information and the preservation of relevant information for prediction.
∎

By reducing the IB objective function, the model focuses more on information relevant to predicting $Y$ while compressing irrelevant information.
The enhanced mutual information with $Y$ reduces the likelihood of generating hallucinated outputs not supported by the visual input.

Error Bounds Provide Guarantees. The upper and lower bounds on mutual information changes ensure that, under specific conditions (e.g., the selection of $\beta$), theoretical improvement holds.

Estimating the Bounds.

- •
$\Delta I_{Y}^{\min}$ requires knowledge of how much additional information about $Y$ is gained by reinjecting $Z_{v}$. It can be estimated based on the mutual information $I(Z_{v};Y)$ and the effectiveness of $H_{vq}^{(m)}$ in capturing information relevant to $Y$.
- •
$\Delta I_{X}^{\max}$ can be bounded based on the capacity of $H_{vq}^{(m)}$ to represent $X_{vq}$. Specifically, it relates to how much additional information $H_{vq}^{(m)}$ can encode about $X_{vq}$ beyond what was already captured in $H_{vq}^{(m-1)}$.
- •
$H(Z_{v})$ is bounded by the entropy of the visual features, as mutual information cannot exceed the entropy of $Z_{v}$.

Through detailed mathematical derivations and the inclusion of upper and lower error bounds, we have established that:

- (a)
Increased Mutual Information: Reinjecting visual features at an intermediate layer increases the mutual information between the model’s embeddings and the visual input.
- (b)
Reduced Conditional Entropy: MemVR reduces the conditional uncertainty of the target output given the intermediate embedding, enhancing the model’s predictive accuracy and mitigating hallucination phenomena caused by the forgetting of visual information.
- (c)
Optimization within IB Framework: Within the Information Bottleneck framework, MemVR optimizes the objective function, provided certain conditions on the mutual information changes are met and appropriate choices of the trade-off parameter $\beta$ are made.

These theoretical findings provide strong support for the practice of MemVR in MLLMs to improve their performance and reliability.

###### Theorem B.1 .

###### Proof.

###### Theorem B.2 .

###### Proof.

###### Theorem B.3 .

###### Proof.

###### Theorem B.4 .

###### Proof.

## Appendix C Additional Experiments, Results, and Discussions

### C.1 Benchmarks

In this appendix, we provide additional details into the benchmarks referenced in the main paper.
To evaluate hallucinations, we employ the following five benchmarks:

CHAIR Rohrbach et al. () evaluates how well the generated captions align with the content of the given image.
CHAIR consists of two versions: CHAIR_S, which measures the inaccuracies at the sentence level, and CHAIR_I, which evaluates at the object level within the sentence by comparing the number of false objects to the total number of objects.
For evaluation, we use the val2014 split of the MSCOCO Lin et al. () dataset, which includes annotations for 80 object categories.
We randomly select 500 images from the entire dataset and used the prompt “Please describe this image in detail.” for the MLLM.

Polling based Object Probing Evaluation (POPE) Li et al. () is a VQA-based metric proposed to assess hallucinations in MLLMs.
This metric evaluates the MLLM’s response to the prompt “Is [object] is in this image?” To emphasize that this is a binary VQA task, we appended the prompt with “Please answer yes or no.”
To select objects referenced in the question prompt, we followed three different sampling options: random, popular, and adversarial.
We evaluated performance across all sampling options.

MLLM Evaluation (MME) Fu et al. () evaluates the capabilities of MLLMs, dividing the evaluation into two major categories: perception and cognition.
The perception category includes fine-grained tasks such as existence, count, location, rough color, poster, celebrity, scene, landmark, artwork identification, and OCR.
The cognition category includes tasks like commonsense reasoning, numerical calculations, text translation, and code reasoning.
All questions in this benchmark are structured to be answered with a simple yes or no.

Using the LLaVA-Bench Liu et al. (), we further demonstrated how well our proposed method maintains the language model performance.
This benchmark involves posing various situational questions, such as dialogue, detailed descriptions, and complex reasoning, to randomly selected images from the MSCOCO val2014 dataset.
A total of 60 questions are used to assess whether the model faithfully follows the instructions.
The generated answers are evaluated by comparing them to the responses of a text-only GPT-4 model.

### C.2 Reproducibility

##### Implementation details.

We employed greedy search as the default decoding strategy across all benchmark evaluations. For the hallucination benchmarks (POPE and CHAIR) and general-purpose benchmarks (MME, VizWiz-VQA, MMBench, MM-Vet, and LLaVA-Bench (in-the-wild)), questions from the annotation files were used as prompts, formatted to fit the chat templates of each respective MLLM. Specifically, we utilized the COCO, A-OKVQA, and GQA datasets for POPE evaluation, and MMBench_DEV_EN for MMBench. In the MM-Vet evaluation, we used an online evaluator powered by OpenAI GPT-4 to assess generated results, while for LLaVA-Bench (in-the-wild), we employed OpenAI’s model gpt4-1106-preview via API. For CHAIR, a randomly sampled image set from the COCO Val2014 dataset was used across all three models, with the prompt ”Please describe this image in detail.” We sampled three different sets of images using different random seeds and evaluated performance by calculating the mean and standard deviation of the results. All MemVR tests were conducted using a greedy decoding approach, with do_sample=False, temperature=0, threshold=0.75, and beam=1. For VCD tests, we set do_sample=True, temperature=1, noise_step=500, and the plausibility constraint hyperparameter $\lambda$ to 0.1, while $\alpha$, which controls the degree of contrastive emphasis, was set to 1, following the default parameter settings from the original code and literature. OPERA tests were configured with beam=5, sample=True, scale_factor=50, threshold=15, num_attn_candidates=5, and penalty_weights=1. Due to OPERA’s reliance on older versions of Torch and Transformers, it was incompatible with Qwen and GLM models, and thus experiments involving these models were not conducted. Additionally, our method introduces two hyperparameters: the informative layer l for activation calculations and the factor $\lambda$ to control the influence of entropy on the next token probability distribution. To map the hidden states from selected layers l to vocabulary tokens, we chose intermediate layers based on the model’s depth (e.g., layers 5 to 16 for vicuna-7b, which has 32 layers), and we set $\lambda$ as a fixed value (e.g., 0.75). All parameter settings adhered to the default configurations specified in the respective papers and code repositories.

##### Experimental Code.

To promote transparency and ensure the reproducibility of our work, we will release all experimental code, datasets, and detailed tutorials necessary for replicating our experiments. Our goal is to make it straightforward for researchers and practitioners to reproduce our results, regardless of their technical background. Additionally, by providing comprehensive documentation and clear guidelines, we aim to facilitate the extension of our method to other models and architectures, enabling the broader research community to explore its potential applications and improvements. We believe that open and reproducible research is essential for advancing the field and fostering collaboration.

##### Computational Resources.

Our experiments were conducted on eight A40 and four A800 GPUs.
The computational bottleneck was not the numerical accuracy values but the collection of potential hallucinatory factors for analytical purposes, including logits and attention values for each head and layer.

### C.3 Case Study

This case study aims to evaluate and present various benchmark cases across multiple domains systematically.

Figure: Figure 11: A case study comparing the levels of hallucination among various baselines
Refer to caption: x11.png

Figure: Figure 12: A case study comparing the levels of hallucination among various baselines
Refer to caption: x12.png

Figure: Figure 13: A case study comparing the levels of hallucination among various baselines
Refer to caption: x13.png

Figure: Figure 14: A case study comparing the levels of hallucination among various baselines
Refer to caption: x14.png

Figure: Figure 15: A case study comparing the levels of hallucination among various baselines
Refer to caption: x15.png

Figure: Figure 16: A case study comparing the levels of hallucination among various baselines
Refer to caption: x16.png

Figure: Figure 17: A case study comparing the levels of hallucination among various baselines
Refer to caption: x17.png

Figure: Figure 18: A case study comparing the levels of hallucination among various baselines
Refer to caption: x18.png

Figure: Figure 19: A case study comparing the levels of hallucination among various baselines
Refer to caption: x19.png

### C.4 Additional Experiments and Results

**Table A1: Time cost for generating tokens. All based on LLaVA1.5-7B**
| Strategy | 1-Token Len | 5-Token Len | 10-Token Len | 20-Token Len | 30-Token Len | 50-Token Len | 80-Token Len |
| --- | --- | --- | --- | --- | --- | --- | --- |
| Greedy | 661.7 | 897.9 | 1273.1 | 1880.3 | 2501.8 | 3617.6 | 5256.6 |
| Sample | 786.8 | 1056.2 | 1314.9 | 1998.5 | 2568.5 | 3593.0 | 5587.0 |
| VCD Sample | 1747.74 | 2767.52 | 4027.07 | 4537.42 | 5031.39 | 7690.77 | 11569.3 |
| Opera Beam | 1566.1 | 3094.9 | 4166.4 | 6242.7 | 8436.9 | 12672.3 | 19247.2 |
| MemVR Sample | 750.8 | 1197.6 | 1780.5 | 2339.2 | 2631.7 | 3718.0 | 6011.0 |
| MemVR Greedy | 775.1 | 974.2 | 1337.5 | 1861.7 | 2742.8 | 4000.9 | 5545.5 |

**Table A2: Results on LLaVABench (in-the-wild) dataset. Best-performing method per model size and dataset is highlighted in bold; arrows indicate improvement or degradation over the baseline.**
| Method | LLaVABench (in-the-wild) |  |  |  |
| --- | --- | --- | --- | --- |
| Average | All_1 | All_2 | All_3 |  |
| LLaVA1.5-7B | 64.80  $\uparrow$0.0 | 63.40  $\uparrow$0.0 | 80.20  $\uparrow$0.0 | 50.80  $\uparrow$0.0 |
| + VCD (Leng et al., ) | 63.20  $\downarrow$1.6 | 59.10  $\downarrow$4.3 | 82.00  $\uparrow$1.8 | 48.50  $\downarrow$2.3 |
| + OPERA (Huang et al., ) | 64.30  $\downarrow$0.5 | 59.80  $\downarrow$3.6 | 83.30  $\uparrow$3.1 | 49.80  $\downarrow$1.0 |
| + MemVR (Ours) | 65.17  $\uparrow$0.4 | 64.00  $\uparrow$0.6 | 80.20  $\uparrow$0.0 | 51.30  $\uparrow$0.5 |
| Qwen-VL-Chat | 68.50  $\uparrow$0.0 | 70.40  $\uparrow$0.0 | 79.30  $\uparrow$0.0 | 55.80  $\uparrow$0.0 |
| + VCD (Leng et al., ) | 53.77  $\downarrow$14.7 | 41.00  $\downarrow$29.4 | 85.30  $\uparrow$6.0 | 35.00  $\downarrow$20.8 |
| + OPERA (Huang et al., ) | - | - | - | - |
| + MemVR (Ours) | 69.50  $\uparrow$1.0 | 69.50  $\downarrow$0.9 | 82.00  $\uparrow$2.7 | 57.00  $\uparrow$1.2 |
| GLM-4V-9B | 75.30  $\uparrow$0.0 | 88.40  $\uparrow$0.0 | 73.00  $\uparrow$0.0 | 64.50  $\uparrow$0.0 |
| + VCD (Leng et al., ) | 74.23  $\downarrow$1.1 | 86.70  $\downarrow$1.7 | 72.80  $\downarrow$0.2 | 63.20  $\downarrow$1.3 |
| + OPERA (Huang et al., ) | - | - | - | - |
| + MemVR (Ours) | 76.73  $\uparrow$1.4 | 88.90  $\uparrow$0.5 | 74.80  $\uparrow$1.8 | 66.50  $\uparrow$2.0 |

**Table A3: Results on MM-Vet dataset. Best-performing method per model size and dataset is highlighted in bold; arrows indicate improvement or degradation over the baseline, where lower values indicate better performance.**
| Method | Total |
| --- | --- |
| LLaVA1.5-7B | 31.1  $\uparrow$0.0 |
| + VCD (Leng et al., ) | 30.20  $\downarrow$0.9 |
| + OPERA (Huang et al., ) | 32  $\uparrow$0.9 |
| + MemVR (Ours) | 32.4  $\uparrow$1.3 |
| Qwen-VL-Chat | 49.0  $\uparrow$0.0 |
| + VCD (Leng et al., ) | 34.60  $\downarrow$14.4 |
| + OPERA (Huang et al., ) | - |
| + MemVR (Ours) | 49.6  $\uparrow$0.6 |
| GLM-4V-9B | 63.4  $\uparrow$0.0 |
| + VCD (Leng et al., ) | 59.40  $\downarrow$4.0 |
| + OPERA (Huang et al., ) | - |
| + MemVR (Ours) | 65.0  $\uparrow$1.6 |

**Table A4: Results on Vizwiz dataset. Best-performing method per model size and dataset is highlighted in bold; arrows indicate improvement or degradation over the baseline, where lower values indicate better performance.**
| Method | Accuracy |
| --- | --- |
| LLaVA1.5-7B | 50.00  $\uparrow$0.0 |
| + VCD (Leng et al., ) | 44.90  $\downarrow$5.1 |
| + OPERA (Huang et al., ) | 50.76  $\uparrow$0.8 |
| + MemVR (Ours) | 51.50  $\uparrow$1.5 |
| Qwen-VL-Chat | 66.05  $\uparrow$0.0 |
| + VCD (Leng et al., ) | 34.54  $\downarrow$31.5 |
| + OPERA (Huang et al., ) | - |
| + MemVR (Ours) | 66.36  $\uparrow$0.3 |
| GLM-4V-9B | 57.39  $\uparrow$0.0 |
| + VCD (Leng et al., ) | 48.04  $\downarrow$9.4 |
| + OPERA (Huang et al., ) | - |
| + MemVR (Ours) | 58.00  $\uparrow$0.6 |

**Table A5: Results on CHAIRS dataset. Best-performing method per model size and dataset is highlighted in bold; arrows indicate improvement or degradation over the baseline, where lower values indicate better performance.**
| Method | CHAIRS |  |  |  |
| --- | --- | --- | --- | --- |
| Cs | Ci | Recall | Len |  |
| LLaVA1.5-7B | 47.60  $\uparrow$0.0 | 13.30  $\uparrow$0.0 | 80.60  $\uparrow$0.0 | 99.70  $\uparrow$0.0 |
| + VCD (Leng et al., ) | 55.00  $\uparrow$7.4 | 15.80  $\uparrow$2.5 | 77.40  $\downarrow$3.2 | 101.20  $\uparrow$1.5 |
| + OPERA (Huang et al., ) | 47.60  $\uparrow$0.0 | 13.50  $\uparrow$0.2 | 79.00  $\downarrow$1.6 | 93.20  $\downarrow$6.5 |
| + MemVR (Ours) | 46.60  $\downarrow$1.0 | 13.00  $\downarrow$0.3 | 80.80  $\uparrow$0.2 | 99.60  $\downarrow$0.1 |
| GLM-4V-9B | 40.40  $\uparrow$0.0 | 9.00  $\uparrow$0.0 | 72.70  $\uparrow$0.0 | 218.20  $\uparrow$0.0 |
| + VCD (Leng et al., ) | 42.20  $\uparrow$1.8 | 9.60  $\uparrow$0.6 | 72.80  $\uparrow$0.1 | 239.80  $\uparrow$21.6 |
| + OPERA (Huang et al., ) | - | - | - | - |
| + MemVR (Ours) | 39.40  $\downarrow$1.0 | 9.00  $\uparrow$0.0 | 70.70  $\downarrow$2.0 | 214.00  $\downarrow$4.2 |
| Qwen-VL-10B | 6.80  $\uparrow$0.0 | 5.30  $\uparrow$0.0 | 53.40  $\uparrow$0.0 | 17.60  $\uparrow$0.0 |
| + VCD (Leng et al., ) | 13.00  $\uparrow$6.2 | 12.30  $\uparrow$7.0 | 47.90  $\downarrow$5.5 | 115.70  $\uparrow$98.1 |
| + OPERA (Huang et al., ) | - | - | - | - |
| + MemVR (Ours) | 4.80  $\downarrow$2.0 | 3.30  $\downarrow$2.0 | 52.30  $\downarrow$1.1 | 15.00  $\downarrow$2.6 |

**Table A6: Results on MMBench dataset. Best-performing method per model size and dataset is highlighted in bold; arrows indicate improvement or degradation over the baseline, where lower values indicate better performance.**
| Method | MMBench-Dev-EN |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- |
| AR | CP | FP-C | FP-S | LR | RR | Overall |  |
| LLaVA1.5-7B | 72.86  $\uparrow$0.0 | 75.68  $\uparrow$0.0 | 58.04  $\uparrow$0.0 | 63.48  $\uparrow$0.0 | 28.81  $\uparrow$0.0 | 51.30  $\uparrow$0.0 | 62.80  $\uparrow$0.0 |
| + VCD (Leng et al., ) | 60.30 | 68.58 | 51.75 | 53.24 | 18.64 | 48.70 | 54.21 |
| + OPERA (Huang et al., ) | 69.85 | 75.00 | 56.64 | 66.21 | 28.81 | 53.04 | 62.80 |
| + MemVR (Ours) | 71.86  $\uparrow$1.2 | 76.69  $\uparrow$1.0 | 57.34  $\downarrow$0.7 | 64.16  $\uparrow$0.9 | 31.36  $\uparrow$2.5 | 56.52  $\uparrow$5.2 | 63.75  $\uparrow$0.9 |
| GLM-4V-9B | 88.44  $\uparrow$0.0 | 86.49  $\uparrow$0.0 | 69.93  $\uparrow$0.0 | 85.67  $\uparrow$0.0 | 66.10  $\uparrow$0.0 | 85.22  $\uparrow$0.0 | 82.39  $\uparrow$0.0 |
| + VCD (Leng et al., ) | 86.43 | 85.47 | 68.53 | 84.64 | 61.86 | 81.74 | 80.58 |
| + OPERA (Huang et al., ) | - | - | - | - | - | - | - |
| + MemVR (Ours) | 88.94  $\uparrow$0.5 | 86.49  $\uparrow$0.0 | 70.63  $\uparrow$0.7 | 86.01  $\uparrow$0.4 | 66.10  $\uparrow$0.0 | 85.22  $\uparrow$0.0 | 82.65  $\uparrow$0.3 |
| Qwen-VL-10B | 60.30  $\uparrow$0.0 | 71.28  $\uparrow$0.0 | 45.45  $\uparrow$0.0 | 62.80  $\uparrow$0.0 | 28.81  $\uparrow$0.0 | 38.26  $\uparrow$0.0 | 56.53  $\uparrow$0.0 |
| + VCD (Leng et al., ) | 34.67 | 52.36 | 20.28 | 55.63 | 11.86 | 22.61 | 39.18 |
| + OPERA (Huang et al., ) | - | - | - | - | - | - | - |
| + MemVR (Ours) | 61.31  $\uparrow$1.0 | 71.28  $\uparrow$0.0 | 44.06  $\downarrow$1.4 | 62.80  $\uparrow$0.0 | 27.97  $\downarrow$0.8 | 38.26  $\uparrow$0.0 | 56.44  $\downarrow$0.1 |

### C.5 Supplement Implement Detail

The code of VCD Leng et al. () is also released. However, the result of VCD evaluated in our experiments (e.g. POPE and MME benchmarks) is lower than the original paper. Therefore, we report the results in the original paper.

## Appendix D Examples of capability integrations

**Table A7: Six samples on MM-Vet benchmark requiring different capability integrations.**
|  |  |
| --- | --- |
| Question: Which room is bigger, the double garage or the living room? | Question: How many gallons of supreme gasoline can I get with $50? |
| Ground Truth: Double garage | Ground Truth: 13.6 $\mid$ 13.7 |
| Required Capabilities: OCR, Spatial Awareness, Math | Required capabilities: OCR, Math |
|  |  |
| Question: Which car is on the parking spot 33? | Question: Is this apple organic? |
| Ground Truth: No $\mid$ Empty | Ground Truth: Yes |
| Required Capabilities: Recognition, OCR, Spatial Awareness | Required capabilities: Recognition, OCR |
|  |  |
| Question: What will the girl on the right write on the board? | Question: Which are producers in this food web? |
| Ground Truth: 14 | Ground Truth: Phytoplankton & Seaweed |
| Required capabilities: Recognition, OCR, Spatial Awareness, Math | Required Capabilities: OCR, Knowledge, Spatial Awareness |

### D.1 Others

##### Q: How can the model understand information directly from the vision encoder, especially if it has a different vision system?

To ensure that MemVR is adaptable across diverse vision systems, we conducted experiments on multiple VLM architectures, including LLaVA, which utilizes a Visual-Instructional-Tuning framework with different sizes of ViT-based CLIP models, Qwen-VL-Chat, which employs a Q-Former-like architecture for visual processing, and ChatGLM-4v-9B, which integrates a large pre-trained visual encoder. These architectures encompass a broad range of vision models, providing confidence that MemVR is applicable to most VLMs in use today.

##### Artifacts and licenses

We report a list of licenses for all datasets and models used in our experiment in Table A8.
We strictly follow all the model licenses and limit the scope of these models to academic research only.

**Table A8: License information for the scientific artifacts.**
| Data Sources | URL | License |
| --- | --- | --- |
| MSCOCO 2017 | [Link](https://cocodataset.org/) | CC BY 4.0 |
| ADE20K | [Link](https://groups.csail.mit.edu/vision/datasets/ADE20K/) | BSD-3-Clause |
| VQA Val | [Link](https://vizwiz.org/tasks-and-datasets/vqa/) | CC BY 4.0 |
| LLaVA-bench-in-the-wild | [Link](https://huggingface.co/datasets/liuhaotian/llava-bench-in-the-wild) | Apache-2.0 |
| ImageNet | [Link](https://image-net.org/) | [Custom License](https://image-net.org/download.php) |
| MMBench | [Link](https://github.com/open-compass) | Apache-2.0 |
| Software Code | URL | License |
| LLaVA | [Link](https://github.com/haotian-liu/LLaVA) | [Llama Community Licence](https://ai.meta.com/llama/license/) |
| Qwen-VL | [Link](https://github.com/QwenLM/Qwen-VL) | [Tongyi Qianwen Licence](https://github.com/QwenLM/Qwen-VL?tab=License-1-ov-file) |
| GLM-4V | [Link](https://github.com/THUDM/GLM-4) | [THUDM GLM-4 Licence](https://github.com/THUDM/GLM-4?tab=Apache-2.0-1-ov-file) |
| GPT-4V/4O | [Link](https://chatgpt.com/) | [OpenAI Term of Use](https://openai.com/policies/terms-of-use/) |

## Peer Reviews

confidence: 3
contribution: 2
initial_rating: 6
presentation: 3
questions: 1. In formula (2), the feedforward neural network comprises two fully connected layers. As MEMVR is a training-free method, how are the weight matrices W1 and W2 obtained? If they utilize weights from a pre-trained projector within the MLLM, how can their effectiveness be ensured?
2. In formula (3), K and V are regarded as visual projectors. What is their relationship with the keys and values in the attention mechanism? This seems to differ from the kv cache approach, leading to some ambiguity in the expression.
3. The injection ratio α is a manually tuned hyperparameter. How should the validation set be selected during the tuning process? Does the parameter range differ across scenarios such as healthcare and transportation?
4. In line 279, it is mentioned that details about the dynamic injection strategy will be presented in Appendix C.1; however, C.1 only discusses benchmarks. Due to the unclear introduction of the dynamic injection strategy, I still have some questions. If the authors could provide a detailed explanation of this method, I might consider increasing my score.
soundness: 3
strengths: 1. The writing quality is high. The paper provides a clear introduction to the hallucination problem in MLLMs, analyzes its causes, presents solutions, and offers a theoretical understanding of the methods used, with clear language and logical coherence.
2. High reusability. Due to the training-free nature of MEMVR, it can be applied in a plug-and-play manner across different MLLMs. The method's simplicity allows for easy adaptation with minimal costs.
3. Sufficient experimentation. The authors validate their approach on both hallucination benchmarks and general-purpose benchmarks, proving its adaptability across various scenarios.
summary: This paper proposes MEMVR, a training-free hallucination mitigation paradigm for MLLMs. MEMVR mimics human patterns of image understanding by revisiting image features to enhance comprehension of image content, utilizing layer-specific uncertainty for dynamic premature layer injection. The authors validate their approach against hallucination benchmarks as well as general-purpose benchmarks, demonstrating its high efficiency.
weaknesses: 1. The explanation of the specific process of MEMVR is not sufficiently clear; the discussion of the dynamic injection strategy is brief and potentially confusing.
2. Concerns about usability: The method requires manual tuning of hyperparameters, which may necessitate multiple rounds of adjustments to achieve optimal performance in different scenarios.

---

confidence: 3
contribution: 3
initial_rating: 5
presentation: 2
questions: - Could the authors clarify how the uncertainty threshold was selected, and how sensitive MEMVR is to this parameter across different datasets?

- Are there specific scenarios where MEMVR fails to reduce hallucinations, could you provide a few concrete examples of failure cases and limitation analysis on them?

- Would MEMVR function effectively if extended to other modalities (e.g., audio, 3D data)? Or would it require significant adjustments?
soundness: 3
strengths: - Originality: MEMVR’s approach to hallucination mitigation is innovative, leveraging visual token reinjection without additional training or external databases. This is a straightforward solution that addresses a critical issue in multimodal models.

- Efficiency: The proposed method minimizes computational overhead by using a single inference pass and only invoking reinjection when needed, making MEMVR potentially applicable to real-world scenarios.

- Empirical Performance: MEMVR demonstrates strong results across multiple benchmarks, including improvements over Visual Contrastive Decoding (VCD) and OPERA, showing its effectiveness in reducing hallucinations and enhancing model alignment with visual content.
summary: This paper proposes a hallucination mitigation technique called MEMVR, Memory-space Visual Retracing for MLLMs. The approach reinjects visual tokens as “key-value memory” into intermediate layers when high uncertainty is detected, to reinforce alignment between visual and textual information without additional fine-tuning or external data. The author used experimental results to show that MEMVR improves hallucination mitigation across various benchmarks, and the method operates efficiently by only triggering visual reinjection when needed.
weaknesses: - Dependence on Threshold-based Mechanism: MEMVR's use of an uncertainty threshold for reinjection may lead to sensitivity to specific datasets or model architectures, potentially affecting its generalizability. The adaptability of this mechanism across different models without manual tuning remains uncertain.

- Section C.3 Analysis: Section C.3 highlights only the successes of MEMVR. Including instances of failure would provide a more balanced analysis. Without external data as used by other methods, MEMVR might risk reinforcing biases in the original model by reinjecting similar visual features. Addressing these issues and examining cases where MEMVR underperforms would offer a more comprehensive understanding of its limitations and inform future improvements.

- Presentation Style: The Q&A format in Section 4.2 on quantitative results lacks logical structure and clarity. It would be more effective to present this section in a structured format that clearly outlines the key findings, for example: “Q2: How well does MEMVR perform on general-purpose benchmarks?” Can be simply phrase as “MEMVR performance on general-purpose benchmarks.”

- The paper provides valuable insights but needs more attention to detail in presenting results. Specific issues include:
In Table 2, Qwen-VL-10B + OPERA's best F1 score on MSCOCO is not bolded.
Table 7 shows LLaVA1.5-7B + OPERA and LLaVA1.5-7B + VCD have top scores on gnkwrec and ocrgnsp, respectively, but these are not bolded. Additionally, a value for LLaVA1.5-7B + OPERA is listed as +1.4 instead of +20.
Table 6 reports MemVR’s improvement on Convs as +0.5 instead of +5.0.

---

confidence: 4
contribution: 3
initial_rating: 5
presentation: 1
questions: See weakness.
soundness: 3
strengths: 1. The paper uses concise and intuitive charts to illustrate the limitations of current methods based on Contrastive Decoding, while also clearly presenting the key design features of the proposed MEMVR framework.

2. Two statistical findings are presented that visually demonstrate the uncertainty present in large language models (LLMs) during multi-layer decoding. The authors provide initial insights into the causes of this uncertainty, adding depth to the analysis.

3. The proposed MEMVR is introduced as a training-free paradigm aimed at mitigating hallucinations in multimodal large language models (MLLMs).
summary: Despite their impressive capabilities, Multimodal Large Language Models (MLLMs) are susceptible to hallucinations, especially assertively fabricating content not present in the visual inputs. To address this issue the authors introduce a hallucination mitigation paradigm named Memory-space Visual Retracing (MEMVR). They design a dynamic premature layer injection strategy with visual retracing in MLLMs, mimicking human intuitive thinking to revisit image features for self-consistency and credible answers when pivotal memories are scrambled. Comprehensive experimental results demonstrate the effectiveness of MEMVR.
weaknesses: The paper’s explanation for the occurrence of hallucinations in MLLMs is not sufficiently persuasive. While the authors identify uncertainty in the decoding process, they fail to provide adequate evidence that this uncertainty stems from the forgetting of visual features, subsequently leading to hallucinations. To strengthen this claim, I recommend that the authors conduct additional experiments, such as directly masking input image tokens to simulate the effect of forgetting. They could then test whether MEMVR is able to recover visual features through the "look twice" mechanism while utilizing the normal image tokens during the recovery process.

Poor Presentation. The presentation of the proposed method is convoluted and lacks clarity. The authors unnecessarily complicate straightforward concepts by introducing numerous "unusual" symbols and formulas. While this may create an impression of sophistication, it significantly increases the reading difficulty for the audience. For instance, in Equation (2), the symbol 
 typically denotes proportionality; however, the intended meaning remains unclear. Furthermore, the relationship between the content of Equation (2) and its explanation in Equation (3) is separated by a substantial amount of text, making it laborious for readers to follow. Additionally, the terms 
 and 
 mentioned in line 256 lack proper definitions, leading to confusion—if they represent trainable parameters of the FFN, it is unclear why they are treated as keys and values. There are multiple instances of such issues throughout the manuscript, indicating that it is not a well-presented piece of work. Here are some suggested modifications:
1. Provide a clear definition and explanation for each symbol when it is first introduced, especially for non-standard usage like $\propto$ in Equation (2).
2. Reorganizing the text to keep related equations and their explanations closer together, particularly for Equations (2) and (3).
3. Clearly define the terms $k$ and $v$ and explain the rationale behind treating them as keys and values if they are indeed FFN parameters.
4. Recommending the inclusion of a notation table or glossary to help readers keep track of symbols and their meanings throughout the paper.
5. Try to review the entire manuscript to identify and simplify overly complex formulations without losing essential information.

---

confidence: 5
contribution: 2
initial_rating: 3
presentation: 1
questions: See Weaknesses.
soundness: 2
strengths: 1.  MEMVR is a training-free approach that enhances the model's ability to retrieve question-relevant visual information within the FFN layers, incurring minimal additional inference overhead.
2. The method demonstrates effectiveness in various benchmarks.
3. The authors provide a thorough analysis of the relationship between model uncertainty and the tendency to hallucination.
summary: The paper introduces a novel approach named Memory-Space Visual Retracing (MEMVR) to address the challenge of hallucinations in Multimodal Large Language Models (MLLMs). Hallucinations in this context refer to the models' tendency to generate content that is inconsistent with the provided visual inputs, which can compromise their reliability in various applications. MEMVR is designed to mimic a human cognitive process where one would revisit visual details to seek accurate answers when initial memories fade. It operates by reinjecting visual features as "key-value memory" into the model's Feed Forward Network (FFN) layers when the model exhibits uncertainty or memory lapses regarding visual details. This method does not require external knowledge retrieval or additional fine-tuning. The paper presents comprehensive experimental evaluations demonstrating that MEMVR significantly reduces hallucination issues across various MLLMs and performs well in general benchmarks without incurring additional time overhead.
weaknesses: 1. The paper presents a method for hallucination mitigation in MLLMs by re-focusing on visual details relevant to the textual prompt. While the concept of improving model performance through prompt-related visual cues is not entirely new, with prior works like API[1], AGLA[2] exploring similar ideas. Despite some innovation in the design, the contributions do not seem to offer substantial new value or advancements in the field.
2. The paper's writing style can be overly embellished, making it less intuitive in certain parts, which may not clearly convey the intended message.
3. The phrase "signifies the other side of the coin" at line 114 is unclear and requires further explanation to understand its intended meaning.
4. While the authors claim at line 499 that MEMVR is flexible and compatible with various architectures, they also mention at line 514 that adapting the method to other structures is challenging. It raises the question of whether MEMVR can be generalized to models like LLaVA-next and InstructBlip.
5. The analysis of uncertainty in section 3.2 is based on the entropy of the decoder's output distribution, but the impact of uncertainty on the method, as shown in Figure 7, is analyzed by adding noise to the images. The claim that the method is unrelated to the biases of language priors may be questionable, as the entropy-based uncertainty measure could be connected to these biases. Additionally, it is unclear how the analysis of uncertainty in section 3.2 is directly related to the proposed method, beyond its application in the Dynamic Injection Strategy.
6. There is a typo error in line 390 where "Preception" should likely be "Perception." 
7. I recommend that the authors restructure the paper, including both figures and text, to express their ideas more clearly and logically, rather than attempting to embellish the research to make it appear more sophisticated. 

[1] Attention Prompting on Image for Large Vision-Language Models.

[2] AGLA: Mitigating Object Hallucinations in Large Vision-Language Models with Assembly of Global and Local Attention.

Meta-review: The paper received mixed feedback, with reviewers acknowledging its innovative "look twice" mechanism for mitigating hallucinations in MLLMs but raising concerns about its clarity and broader impact. The proposed method offers a training-free approach that dynamically injects visual information during inference, which some reviewers found the general idea promising. However, key issues and inconsistencies in presentation/explanation as well as uncertainty measures weakened the overall impression. While the authors addressed many points through revisions and experiments, there is insufficient strong support for its acceptance. The AC recommends reject.

Decision: Reject