Look Twice Before You Answer: Memory-Space Visual Retracing for Hallucination Mitigation in Multimodal Large Language Models

Authors: Xin Zou, Yizhou Wang, Yibo Yan, Sirui Huang, Kening Zheng, Junkai Chen, Chang Tang, Xuming Hu
Primary area: foundation or frontier models, including LLMs
Keywords: Hallucination mitigation, MLLMs, visual retracing, training-free

Abstract:
Despite their impressive capabilities, Multimodal Large Language Models (MLLMs) are susceptible to hallucinations, especially assertively fabricating content not present in the visual inputs. To address the aforementioned challenge, we follow a common cognitive process - \textit{when one's initial memory of critical on-sight details fades, replenishing visual memory is essential to seek a factual and accurate answer.} Therefore, we introduce Memory-space Visual Retracing (MemVR), a novel hallucination mitigation paradigm that without the need for external knowledge retrieval or additional fine-tuning. In particular, we treat visual tokens as supplementary evidence to be reinjected into MLLMs via Feed Forward Network (FFN) as “key-value memory” at the middle trigger layer, \textit{i.e.}, when the model is uncertain about visual memories in the layer. Comprehensive experimental evaluations demonstrate that \modelname significantly mitigates hallucination issues across various MLLMs and excels in general benchmarks without incurring added time overhead, thus emphasizing its potential for widespread applicability.

Decision: reject
Reviewer scores: 6, 5, 5, 3

Reviews and discussion:
<thread>
creation date:
1729585187374

thread_type:
official_review

<thread_object>
creator: reviewer_ysel

<content>
summary:
The paper introduces a novel approach named Memory-Space Visual Retracing (MEMVR) to address the challenge of hallucinations in Multimodal Large Language Models (MLLMs). Hallucinations in this context refer to the models' tendency to generate content that is inconsistent with the provided visual inputs, which can compromise their reliability in various applications. MEMVR is designed to mimic a human cognitive process where one would revisit visual details to seek accurate answers when initial memories fade. It operates by reinjecting visual features as "key-value memory" into the model's Feed Forward Network (FFN) layers when the model exhibits uncertainty or memory lapses regarding visual details. This method does not require external knowledge retrieval or additional fine-tuning. The paper presents comprehensive experimental evaluations demonstrating that MEMVR significantly reduces hallucination issues across various MLLMs and performs well in general benchmarks without incurring additional time overhead.

strengths:
1.  MEMVR is a training-free approach that enhances the model's ability to retrieve question-relevant visual information within the FFN layers, incurring minimal additional inference overhead.
2. The method demonstrates effectiveness in various benchmarks.
3. The authors provide a thorough analysis of the relationship between model uncertainty and the tendency to hallucination.

weaknesses:
1. The paper presents a method for hallucination mitigation in MLLMs by re-focusing on visual details relevant to the textual prompt. While the concept of improving model performance through prompt-related visual cues is not entirely new, with prior works like API[1], AGLA[2] exploring similar ideas. Despite some innovation in the design, the contributions do not seem to offer substantial new value or advancements in the field.
2. The paper's writing style can be overly embellished, making it less intuitive in certain parts, which may not clearly convey the intended message.
3. The phrase "signifies the other side of the coin" at line 114 is unclear and requires further explanation to understand its intended meaning.
4. While the authors claim at line 499 that MEMVR is flexible and compatible with various architectures, they also mention at line 514 that adapting the method to other structures is challenging. It raises the question of whether MEMVR can be generalized to models like LLaVA-next and InstructBlip.
5. The analysis of uncertainty in section 3.2 is based on the entropy of the decoder's output distribution, but the impact of uncertainty on the method, as shown in Figure 7, is analyzed by adding noise to the images. The claim that the method is unrelated to the biases of language priors may be questionable, as the entropy-based uncertainty measure could be connected to these biases. Additionally, it is unclear how the analysis of uncertainty in section 3.2 is directly related to the proposed method, beyond its application in the Dynamic Injection Strategy.
6. There is a typo error in line 390 where "Preception" should likely be "Perception." 
7. I recommend that the authors restructure the paper, including both figures and text, to express their ideas more clearly and logically, rather than attempting to embellish the research to make it appear more sophisticated. 

[1] Attention Prompting on Image for Large Vision-Language Models.

[2] AGLA: Mitigating Object Hallucinations in Large Vision-Language Models with Assembly of Global and Local Attention.

questions:
See Weaknesses.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730635410865

thread_type:
official_review

<thread_object>
creator: reviewer_tpgc

<content>
summary:
Despite their impressive capabilities, Multimodal Large Language Models (MLLMs) are susceptible to hallucinations, especially assertively fabricating content not present in the visual inputs. To address this issue the authors introduce a hallucination mitigation paradigm named Memory-space Visual Retracing (MEMVR). They design a dynamic premature layer injection strategy with visual retracing in MLLMs, mimicking human intuitive thinking to revisit image features for self-consistency and credible answers when pivotal memories are scrambled. Comprehensive experimental results demonstrate the effectiveness of MEMVR.

strengths:
1. The paper uses concise and intuitive charts to illustrate the limitations of current methods based on Contrastive Decoding, while also clearly presenting the key design features of the proposed MEMVR framework.

2. Two statistical findings are presented that visually demonstrate the uncertainty present in large language models (LLMs) during multi-layer decoding. The authors provide initial insights into the causes of this uncertainty, adding depth to the analysis.

3. The proposed MEMVR is introduced as a training-free paradigm aimed at mitigating hallucinations in multimodal large language models (MLLMs).

weaknesses:
The paper’s explanation for the occurrence of hallucinations in MLLMs is not sufficiently persuasive. While the authors identify uncertainty in the decoding process, they fail to provide adequate evidence that this uncertainty stems from the forgetting of visual features, subsequently leading to hallucinations. To strengthen this claim, I recommend that the authors conduct additional experiments, such as directly masking input image tokens to simulate the effect of forgetting. They could then test whether MEMVR is able to recover visual features through the "look twice" mechanism while utilizing the normal image tokens during the recovery process.

Poor Presentation. The presentation of the proposed method is convoluted and lacks clarity. The authors unnecessarily complicate straightforward concepts by introducing numerous "unusual" symbols and formulas. While this may create an impression of sophistication, it significantly increases the reading difficulty for the audience. For instance, in Equation (2), the symbol 
 typically denotes proportionality; however, the intended meaning remains unclear. Furthermore, the relationship between the content of Equation (2) and its explanation in Equation (3) is separated by a substantial amount of text, making it laborious for readers to follow. Additionally, the terms 
 and 
 mentioned in line 256 lack proper definitions, leading to confusion—if they represent trainable parameters of the FFN, it is unclear why they are treated as keys and values. There are multiple instances of such issues throughout the manuscript, indicating that it is not a well-presented piece of work. Here are some suggested modifications:
1. Provide a clear definition and explanation for each symbol when it is first introduced, especially for non-standard usage like $\propto$ in Equation (2).
2. Reorganizing the text to keep related equations and their explanations closer together, particularly for Equations (2) and (3).
3. Clearly define the terms $k$ and $v$ and explain the rationale behind treating them as keys and values if they are indeed FFN parameters.
4. Recommending the inclusion of a notation table or glossary to help readers keep track of symbols and their meanings throughout the paper.
5. Try to review the entire manuscript to identify and simplify overly complex formulations without losing essential information.

questions:
See weakness.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730678682770

thread_type:
official_review

<thread_object>
creator: reviewer_it3w

<content>
summary:
This paper proposes a hallucination mitigation technique called MEMVR, Memory-space Visual Retracing for MLLMs. The approach reinjects visual tokens as “key-value memory” into intermediate layers when high uncertainty is detected, to reinforce alignment between visual and textual information without additional fine-tuning or external data. The author used experimental results to show that MEMVR improves hallucination mitigation across various benchmarks, and the method operates efficiently by only triggering visual reinjection when needed.

strengths:
- Originality: MEMVR’s approach to hallucination mitigation is innovative, leveraging visual token reinjection without additional training or external databases. This is a straightforward solution that addresses a critical issue in multimodal models.

- Efficiency: The proposed method minimizes computational overhead by using a single inference pass and only invoking reinjection when needed, making MEMVR potentially applicable to real-world scenarios.

- Empirical Performance: MEMVR demonstrates strong results across multiple benchmarks, including improvements over Visual Contrastive Decoding (VCD) and OPERA, showing its effectiveness in reducing hallucinations and enhancing model alignment with visual content.

weaknesses:
- Dependence on Threshold-based Mechanism: MEMVR's use of an uncertainty threshold for reinjection may lead to sensitivity to specific datasets or model architectures, potentially affecting its generalizability. The adaptability of this mechanism across different models without manual tuning remains uncertain.

- Section C.3 Analysis: Section C.3 highlights only the successes of MEMVR. Including instances of failure would provide a more balanced analysis. Without external data as used by other methods, MEMVR might risk reinforcing biases in the original model by reinjecting similar visual features. Addressing these issues and examining cases where MEMVR underperforms would offer a more comprehensive understanding of its limitations and inform future improvements.

- Presentation Style: The Q&A format in Section 4.2 on quantitative results lacks logical structure and clarity. It would be more effective to present this section in a structured format that clearly outlines the key findings, for example: “Q2: How well does MEMVR perform on general-purpose benchmarks?” Can be simply phrase as “MEMVR performance on general-purpose benchmarks.”

- The paper provides valuable insights but needs more attention to detail in presenting results. Specific issues include:
In Table 2, Qwen-VL-10B + OPERA's best F1 score on MSCOCO is not bolded.
Table 7 shows LLaVA1.5-7B + OPERA and LLaVA1.5-7B + VCD have top scores on gnkwrec and ocrgnsp, respectively, but these are not bolded. Additionally, a value for LLaVA1.5-7B + OPERA is listed as +1.4 instead of +20.
Table 6 reports MemVR’s improvement on Convs as +0.5 instead of +5.0.

questions:
- Could the authors clarify how the uncertainty threshold was selected, and how sensitive MEMVR is to this parameter across different datasets?

- Are there specific scenarios where MEMVR fails to reduce hallucinations, could you provide a few concrete examples of failure cases and limitation analysis on them?

- Would MEMVR function effectively if extended to other modalities (e.g., audio, 3D data)? Or would it require significant adjustments?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730686360702

thread_type:
official_review

<thread_object>
creator: reviewer_ezb5

<content>
summary:
This paper proposes MEMVR, a training-free hallucination mitigation paradigm for MLLMs. MEMVR mimics human patterns of image understanding by revisiting image features to enhance comprehension of image content, utilizing layer-specific uncertainty for dynamic premature layer injection. The authors validate their approach against hallucination benchmarks as well as general-purpose benchmarks, demonstrating its high efficiency.

strengths:
1. The writing quality is high. The paper provides a clear introduction to the hallucination problem in MLLMs, analyzes its causes, presents solutions, and offers a theoretical understanding of the methods used, with clear language and logical coherence.
2. High reusability. Due to the training-free nature of MEMVR, it can be applied in a plug-and-play manner across different MLLMs. The method's simplicity allows for easy adaptation with minimal costs.
3. Sufficient experimentation. The authors validate their approach on both hallucination benchmarks and general-purpose benchmarks, proving its adaptability across various scenarios.

weaknesses:
1. The explanation of the specific process of MEMVR is not sufficiently clear; the discussion of the dynamic injection strategy is brief and potentially confusing.
2. Concerns about usability: The method requires manual tuning of hyperparameters, which may necessitate multiple rounds of adjustments to achieve optimal performance in different scenarios.

questions:
1. In formula (2), the feedforward neural network comprises two fully connected layers. As MEMVR is a training-free method, how are the weight matrices W1 and W2 obtained? If they utilize weights from a pre-trained projector within the MLLM, how can their effectiveness be ensured?
2. In formula (3), K and V are regarded as visual projectors. What is their relationship with the keys and values in the attention mechanism? This seems to differ from the kv cache approach, leading to some ambiguity in the expression.
3. The injection ratio α is a manually tuned hyperparameter. How should the validation set be selected during the tuning process? Does the parameter range differ across scenarios such as healthcare and transportation?
4. In line 279, it is mentioned that details about the dynamic injection strategy will be presented in Appendix C.1; however, C.1 only discusses benchmarks. Due to the unclear introduction of the dynamic injection strategy, I still have some questions. If the authors could provide a detailed explanation of this method, I might consider increasing my score.
</content>
</thread_object>

</thread>
<thread>
creation date:
1732500737958

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
General Response (1)

comment:
We appreciate all reviewers’ constructive feedback and positive remarks on the paper. 

**To the best of our knowledge, we are the first to mitigate hallucinations and improve general performance in MLLMs with only one regular inference, without incurring added time overhead.** Moreover, MemVR can be extended to **more modalities** instead of being limited to images, and videos. Its high efficiency also allows MemVR to be applied to responsiveness-sensitive domains e.g. healthcare and embodied intelligence.

Here are the characteristics of different methods.
﻿
| Method | Training-free  | Visual Hallucination Mitigation | Generalization | Extended to more modalities | Efficiency | Enhanced Components | 
| -------- | -------- | -------- | -------- | -------- | -------- | -------- |
| DoLa (ICLR'24) | ✅ | - | - |  - | ✅ | logits |
| VCD  (CVPR'24) | ✅ | ✅ | -  |  - | - | visual input, logits |
| OPERA (CVPR'24) | ✅ | ✅ | -  |  ✅ |  - | attention matrix |
| HALC [3] (ICML'24) | ✅ | ✅ | -  |  - | - | visual input, logits |
| MVP [4] | ✅ | ✅ | -  |  - | - | visual input, logits |
| VACoDe [5] | ✅ | ✅ | - | -  | - | visual input, logits |
| SID [6] | ✅ | ✅ | -  |  - | - | text input, logits |
| API [1] (ECCV'24) | ✅ | - | ✅  |  - | - | visual input |
| AGLA [2] | ✅ | ✅ | -  | - |  - | visual input, logits |
| **MemVR (ours)** | ✅ | ✅ | ✅ | ✅ |  ✅ | **hidden states** |
﻿

[1] Attention Prompting on Image for Large Vision-Language Models.

[2] AGLA: Mitigating Object Hallucinations in Large Vision-Language Models with Assembly of Global and Local Attention.

[3] HALC: Object Hallucination Reduction via Adaptive Focal-Contrast Decoding, ICML, 2024.

[4] Look, Compare, Decide: Alleviating Hallucination in Large Vision-Language Models via Multi-View Multi-Path Reasoning, 2024. 

[5] Vacode: Visual augmented contrastive decoding, 2024.

[6] Self-introspective decoding: Alleviating hallucinations for large vision-language models, 2024.
</content>
</thread_object>

</thread>
<thread>
creation date:
1733110671835

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
General Response to All Reviewers

comment:
We are truly grateful for the invaluable time and detailed feedback provided by all the reviewers. 

**It is encouraging to see that **Reviewers eZB5, iT3w, tPgc** has recognized the significant contributions and positive aspects of our manuscript**,  such as `writing quality is high`, and `high reusability`, `sufficient experimentation` (Reviewer eZB5);  `MemVR’s approach is innovative`, and `efficiency` (Reviewer iT3w); `good contribution`, and `clearly illustrate the limitations of CD-based methods`, and `clearly presenting`, and `depth analysis` (Reviewer tPgc).

**We have provided detailed responses to each reviewer’s feedback**. In this general response, we outline the major revisions made to our new manuscript based on the valuable suggestions offered by the reviewers. (Please check the changes in blue font in our revision) We hope these responses adequately address any potential concerns from the reviewers, the AC, and SPC.

- **Presentation**: Based on the feedback from **Reviewers eZB5, iT3w, tPgc**, our revisions include ① modifying the method part of Section 3 to more clearly expound the specific process of MemVR, simplify overly complex formulations, and ② updating Section 4.2 on quantitative results to enhance logical presentation and understanding, and ③ revising some figure mistakes on Table 2, 6, 7.

- **Experiments**: According to the feedback from **Reviewers eZB5, iT3w, tPgc**, our revisions include ① adding results and analysis of experiments for hyper-parameters (on the comments and Appendix C) and ② including comparisons and analysis of the bad cases with baseline (see Appendix C), and ③ conducting additional experiments by masking input images with randomly generated noise, offers strong evidence for the effectiveness of the "look twice" mechanism. (on the comments to Reviewer tPgc)

---
**For the feedback from Reviewers YSEL**

Our comments include ① presenting **clear difference and novelty** of our methods (see **General Response (1)** below), ② adding extra experiments on LLaVA-next and InstructBlip requested by Reviewer YSEL (**in fact, we have conducted experiments on 4 different models across 7 benchmarks, more comprehensive than previous methods**), and ③ clarifying a few points regarding all other concerns by Reviewer YSEL, *e.g.*, the phrase "signifies the other side of the coin" at line 114, and the analysis of uncertainty in section 3.2, and concept of “visual prompts”.

We have made sincere efforts to address all the issues raised. **However**, in the last **Reviewer YSEL** claimed that the concept of "visual prompt" lacks context, stating that the writing is **extremely poor** without any other feedback, insisting on the 3 rating.

 **Reviewer YSEL's statement appears to be maliciously biased**.  We hope that the AC and other Reviewers will be able to make a fair judgment. Thank you!

___

We are deeply grateful for the recognition and suggestions from all reviewers. We still respectfully welcome further discussions from all reviewers.

Best regards,

All Authors
</content>
</thread_object>

</thread>


Metareview:
metareview:
The paper received mixed feedback, with reviewers acknowledging its innovative "look twice" mechanism for mitigating hallucinations in MLLMs but raising concerns about its clarity and broader impact. The proposed method offers a training-free approach that dynamically injects visual information during inference, which some reviewers found the general idea promising. However, key issues and inconsistencies in presentation/explanation as well as uncertainty measures weakened the overall impression. While the authors addressed many points through revisions and experiments, there is insufficient strong support for its acceptance. The AC recommends reject.