R e L oop: “Seeing Twice and Thinking Backwards” via Closed-loop Training to Mitigate Hallucinations in Multimodal understanding (2025)

Abstract: While Multimodal Large Language Models (MLLMs) have achieved remarkable progress in open-ended visual question answering, they remain vulnerable to hallucinations. These are outputs that contradict or misrepresent input semantics, posing a critical challenge to the reliability and factual consistency. Existing methods often rely on external verification or post-hoc correction, lacking an internal mechanism to validate outputs directly during training. To bridge this gap, we propose ReLoop, a unified closed-loop training framework that encourages multimodal consistency for cross-modal understanding in MLLMs. ReLoop adopts a ring-shaped structure that integrates three complementary consistency feedback mechanisms, obliging MLLMs to “seeing twice and thinking backwards”. Specifically, ReLoop employs the frozen Consistency Feedback Plugin (CFP), comprising semantic reconstruction, visual description, and an attention supervision module for attention alignment. These components collectively enforce semantic reversibility, visual consistency, and interpretable attention, enabling the model to correct its outputs during training. Extensive evaluations and analyses demonstrate the effectiveness of ReLoop in reducing hallucination rates across multiple benchmarks, establishing a robust method for hallucination mitigation in MLLMs. We will release our source code and data in the camera-ready version. The code is available at: https://github.com/ZiyanHuang11/Reloop-hallucinations.

## introduction in recent years, mllms @xcite @xcite have demonstrated significant progress in bridging vision and language, addressing tasks such as visual question answering (vqa), image captioning, and instruction adherence. however, a fundamental difficulty that * equal contribution persists is hallucination, where the generation of outputs that are inconsistent with or unsupported by visual inputs @xcite . hallucinations are especially common in open-ended vqa circumstances, in which unclear or underspecified questions can result in factual mistakes. these hallucinations span diverse categories, including object, attribute, relation, and event. figure 1 illustrates that a singular image of "a dog grasping an orange frisbee" can elicit various forms of hallucination: a fictitious "cat" (object), an incorrectly identified "green" frisbee (attribute), an erroneous spatial relation "under the paw" (relation), or a temporal misrepresentation "sleeping" (event). these errors are semantically plausible yet visually unfounded, posing major challenges for trustworthiness and safety of mllms across critical applications, including medical decision-making @xcite , robotic perception @xcite , and autonomous navigation @xcite . existing works @xcite @xcite often regard hallucination as an output-level anomaly that is corrected post hoc, overlooking its underlying cause. in practice, hallucinations frequently arise from misalignment between the input, visual content, and the model's latent reasoning. without an internal supervision mechanism, models may produce fluent yet ungrounded answers. we argue that hallucination stems from the model's inability to validate its own output across modalities and recommend injecting this ability directly into training. we subsequently derive inspiration from human cognitive processes. when answering visual questions, individuals rarely rely on a single forward guess. instead, after answering, they may reassess the question's intent, examine the visual scene, and refine conclusions-especially in the face of ambiguity or uncertainty. however, most models operate in a unidirectional manner, mapping (q, @xmath0. as a result, once the model makes a prediction, there is no structured way to assess whether it actually understood the question, if the answer aligns with the visual evidence, or whether the model attended to the right regions in the image. to address this issue, we propose reloop, a cognitively inspired unified training framework that encourages multimodal consistency for crossmodal understanding in mllms. reloop implements a feedback-driven closed-loop supervision process, allowing the model to reassess its predictions and validate their consistency with the original input through multi-level supervision during training. specifically, after mllms produce an answer from the image-question pair, reloop enables the model to recapture input semantics and assess internal consistency via: a consistency feedback plugin (cfp), comprising two frozen modules: (1) cfp-lang reconstructs the question q * from (a, i) to supervise semantic alignment, and (2) cfp-vis generates a description i * to assess factual grounding. in parallel, an attention supervision module extracts the model's token-to-image attention map h and compares it with an entropy-based pseudoground truth. all signals are integrated as differentiable losses in the overall optimization objective. this design encourages the model to "see twice and think backward"-first look to answer (q, @xmath1, look twice to reassess (a, @xmath2, i * , h), and finally to correct ( q * , i * , h ≈ q, i, h pseudo ). reloop bridges the gap between perception and output. it turns the black-box understanding process into an interpretable, feedback-aware loop that continuously refines the model's internal representations. our key contributions can be summarized clearly as follows: ## experimental setup training data. we curate 30k high-quality {i, q, a} from llava-instruct-150k. to simulate hallucination supervision, we generate contrastive examples by perturbing key semantics (e.g., objects, attributes, relations, event), enabling fine-grained control over hallucination types. details can be found in appendix a.1 a.2. evaluation benchmarks and metrics. we evaluate reloop on a broad range of hallucination and multimodal understanding benchmarks, including pope (li et al., 2023b), chair (rohrbach et al., 2018), amber (wang et al., 2023), mmhal-b (sun et al., 2023), hallusionbench (guan et al., 2024), faith/faiths (jing et al., 2024), and mme (fu et al., 2023). full definitions can be found in appendix a.3 a.4. baselines. we use minigpt-4 as the baseline model in experiment 6.1 and compare against llava-1.5 variants trained with llava-rlhf (sun et al., 2023), ha-dpo (zhao et al., 2023), povid (zhou et al., 2024), and visual contrastive decoding (vcd) (sicong leng, 2023). for robustness analyses, we adopt llava-1.5 + reloop as the canonical setting and report stress tests under noisy external supervision and nonsensical answers (