Title: Hallucination at a Glance: Controlled Visual Edits and Fine-Grained Multimodal Learning
Venue: NeurIPS 2025
Accepted: yes
Keywords: MLLMs
Abstract: Multimodal large language models (MLLMs) have achieved strong performance on vision-language tasks but still struggle with fine-grained visual differences, leading to hallucinations or missed semantic shifts. We attribute this to limitations in both training data and learning objectives. To address these issues, we propose a controlled data generation pipeline that produces minimally edited image pairs with semantically aligned captions. Using this pipeline, we construct the Micro Edit Dataset (MED), containing over 50K image-text pairs spanning 11 fine-grained edit categories, including attribute, count, position, and object presence changes.
Building on MED, we introduce a supervised fine-tuning (SFT) framework with a feature-level consistency loss that promotes stable visual embeddings under small edits. We evaluate our approach on the Micro Edit Detection benchmark, which includes carefully balanced evaluation pairs designed to test sensitivity to subtle visual variations across the same edit categories.
Our method improves difference detection accuracy and reduces hallucinations compared to strong baselines, including GPT-4o. Moreover, it yields consistent gains on standard vision-language tasks such as image captioning and visual question answering. These results demonstrate the effectiveness of combining targeted data and alignment objectives for enhancing fine-grained visual reasoning in MLLMs. Code and datasets are publicly released at https://github.com/Relaxed-System-Lab/hallu_med.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces a lightweight, training-free framework for detecting hallucinations in multimodal large language models (MLLMs), particularly vision-language models. The proposed method, Single-shot Hallucination Detection (SHD), leverages spatial attention maps and token-level entropy to determine whether the generated text is grounded in the input image. Rather than relying on costly retraining or dataset-specific tuning, SHD performs hallucination detection in a single forward pass using only internal model signals. The authors evaluate SHD across six hallucination types and multiple state-of-the-art MLLMs, showing competitive or superior performance compared to more complex baselines. The work contributes a novel, interpretable, and efficient technique for improving the reliability of multimodal LLM outputs in real-world settings.\n\nSTRENGTHS AND WEAKNESSES:\nQuality\nThe proposed method, SHD (Single-shot Hallucination Detection), is a well-engineered, training-free solution that leverages intrinsic model signals—namely token entropy and cross-modal attention—to detect hallucinations.\n\nThe experiments are extensive, covering six different hallucination categories and multiple state-of-the-art MLLMs (e.g., MiniGPT4, LLaVA, Otter). The authors use a recently proposed benchmark (POPE) and ablation studies to validate individual components.\n\nThe paper reports competitive results against several strong baselines, including detect-then-rank models, supervised discriminators, and gradient-based techniques—despite SHD requiring no additional tuning or training.\n\nClarity\nThe paper is generally well-organized and clearly written. The motivation for a lightweight, interpretable hallucination detector is well-argued in the introduction.\n\nThe authors provide intuitive visualizations of the entropy maps and cross-attention distributions to support interpretability.\n\nEach of the six hallucination categories is explained with clarity, and examples are included for illustration.\n\nSignificance\nHallucination detection in multimodal LLMs is an increasingly important area, particularly as vision-language models see growing deployment in safety-critical domains.\n\nThe proposed method has practical significance: being lightweight, interpretable, and model-agnostic, it can be applied in real-world applications without the need for costly retraining.\n\nThe ability to detect hallucinations in a single forward pass makes SHD particularly relevant for latency-sensitive systems and scalable inference.\n\nOriginality\nThe work is original in its framing of hallucination detection as a signal derived from token entropy and cross-attention, without the need for ground-truth labels or external discriminators.\n\nUnlike many prior works that rely on retraining or multiple forward passes (e.g., question answering consistency checks, multi-step verification), SHD provides a conceptually novel and computationally efficient approach.\n\nQUESTIONS:\n1.  How well does SHD generalize to hallucination detection tasks beyond the POPE benchmark and short-form VQA-style prompts? For instance, how would it perform in document-grounded generation, image captioning, or complex reasoning scenarios involving multiple modalities?\n2.  How does SHD handle borderline or ambiguous cases? Are there examples where the model flags correct outputs as hallucinations (false positives)?\n3. Even a qualitative comparison or brief discussion in related work would help position SHD among broader hallucination mitigation strategies.\n\nLIMITATIONS:\nPartially.\n\nThe authors include a limitations section that briefly acknowledges the dataset scope (POPE) and the reliance on internal model signals (token entropy, cross-modal attention). However, several critical aspects are under-addressed and would benefit from deeper reflection:\n\nSuggestions for Improvement\nBias Amplification & Model Dependence:\nSince SHD depends entirely on internal attention and entropy signals, which are products of the underlying model's learned biases, the method may inherit or even reinforce spurious correlations and visual/textual biases. A discussion on how this might affect fairness or performance across diverse demographic attributes, image styles, or non-English content would be valuable.\n\nDeployment Misuse or Overtrust:\nSHD is lightweight and attractive for production settings, but it may create overconfidence in model safety if not properly calibrated. The authors should caution against using SHD as a standalone truth evaluator, especially in sensitive domains (e.g., healthcare, legal AI), and emphasize the need for human oversight.\n\nScope of Applicability:\nThe method has only been evaluated on static images and short-form QA. Authors should note that applying SHD to other modalities (e.g., video, audio, document OCR) may not yield reliable results without adaptation or retraining, even if SHD is training-free in its current form.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 5, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-10"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper points out that current MLLMs struggles at fine-grained visual differences, which may further lead to inaccurate predictions or hallucinations. To address this, the authors designed an automated data generation pipeline, yielding the Micro Edit Dataset (MED). This dataset comprises over 50K image-text pairs across 11 fine-grained edit categories, specifically designed to highlight subtle visual changes.  Then, the author validate the effectiveness of the dataset with different MLLMs like Qwen 2.5-VL. The results demonstrate that the dataset not only enhances MLLM performance on the dedicated MED benchmark but also improves performance on other VL tasks.\n\nSTRENGTHS AND WEAKNESSES:\n# Strengths\n1. The identification of existing MLLMs' weakness in recognizing subtle visual changes is significant. Current SOTA MLLMs remain considerably behind human-level recognition abilities (e.g., GPT-4.1-2025-04-14 at 54% versus human performance at 95%), highlighting a crucial direction for future MLLM optimization.\n2. The dataset generation pipeline is well-designed, incorporating various filtering and editing techniques to ensure the production of a high-quality dataset.\n3. The authors rigorously validated the MED across diverse MLLMs, utilizing both the MED benchmark and other VL benchmarks. This extensive validation confirms the dataset's utility in enhancing MLLM recognition capabilities.\n\n# Weakness\n1. Some details are not clear. See Questions.\n\nQUESTIONS:\n1. I am curious about the scalability of the Micro Edit Dataset (MED). If the dataset size were continuously scaled up, would performance improvements persist?\n2. The authors employed LoRA for fine-tuning the model. What would be the impact of not using LoRA? Additionally, the hyper-parameters vary significantly across different base models. What might explain this discrepancy?\n3.  I noticed that CLIP loss is incorporated into all presented equations. However, most existing MLLMs do not typically integrate CLIP loss during SFT. Is CLIP loss indeed utilized in the paper's SFT process?\n4. The paper claims to use a feature-level consistency loss. How is this implemented within the proposed method? If I understand correctly, the SFT loss only requires the model to distinguish differences between two input images.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 5, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-29"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper tackles hallucinations in large VLMs, a problem of high interest in LM research. The authors hypothesize the root of hallucinations of large VLMs is inadequate training data requiring fine-grained visual understanding. The authors propose the Micro Edit Dataset, which provides both a training and benchmarking portion for improving and evaluating performance on fine-grained visual differences. The authors perform extensive fine-tuning on their dataset based on risk minimization over hallucination. They show that the SFT procedure results in higher performance across a number of standard VLM benchmarks.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\nThe problem is clearly of high importance, and the visual quality of paired images seems high. While not totally realistic visually, the resulting performance gains seem clear to validate the pipeline to be built upon as parts of larger VLM fine-tuning pipelines. The formulation of SFT as empirical risk minimization for hallucination makes sense, and results in nice performance gains.\n\nWeaknesses:\nThere are artifacts of the text-based difference caption generation in multiple of the presented examples. For instance, the texts include phrases like \"are not mentioned,\" \"is described,\" \"in the second caption.\" This raises questions about potential false negatives during benchmarking due to selecting the next best choice that describes an image instead of a caption.\n\nAs the dataset is a main contribution of the paper, more extensive qualification behind design choices (e.g. similarity thresholding, failure cases, ablations on backbone LLMs) would help strengthen the usefulness of the MED.\n\nSome of the images are qualitatively obviously synthetic. The performance on the MED-Real set is much higher than analogous performance on the synthetic set (admittedly the real set is small).\n\nQUESTIONS:\nCan the authors comment on why performance seems to be a large margin higher on the MED-Real set vs the main MED benchmark? Are there any particular reasons stemming from the candidate data distribution, or is this an artifact of the model training domain? In this case, how can it be verified that the performance gains on fine-tuning on the MED were not primarily due to bridging the domain gap between real and synthetic?\n\nCan the authors comment on the quite frequent artifacts of the texts referring to samples as texts? Will this not introduce false negatives in benchmarked models?\n\nCan the authors demonstrate qualitative or quantitative ablations over the data curation pipeline, showing why specific similarity thresholding, etc. necessary to curate the best dataset?\n\nOverall I have a positive impression of this work, but addressing my questions would encourage me to improve my rating. Thanks!\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper works on fine-grained visual reasoning in Multimodal Large Language Models (MLLMs), particularly their tendency to hallucinate or miss small but meaningful visual differences (e.g., object presence, count, spatial changes). The authors identify two key limitations: insufficient training data and weak alignment objectives.\n\nTo address these, they propose:\n1. Micro Edit Dataset (MED): A large-scale dataset (~50K image-text pairs) of minimally edited image pairs and aligned captions spanning 11 fine-grained semantic edit types.\n2. Controlled Visual Editing Pipeline: Generates image pairs using editing models like Gemini Flash 2.0, and captions via Qwen-VL and Qwen3-32B.\n3. Micro Edit Detection (MED) Benchmark: A 165-question benchmark across the same 11 edit types, designed to evaluate sensitivity to small differences.\n4. Feature Consistency Regularization: A fine-tuning objective that encourages stable visual embeddings for minor edits, thereby improving semantic alignment.\n\nThey fine-tune open-source models (e.g., Qwen2-VL-7B, LLaVA) and show significant improvements on the MED benchmark and generalization to other tasks (e.g., POPE, MMStar, BLINK). Their best model (Qwen2.5-VL-7B) matches or surpasses GPT-4o and Claude 3.7 Sonnet on fine-grained reasoning tasks.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1. Well-motivated and timely problem: Hallucination in MLLMs is a known but under-addressed issue, and the paper identifies a specific failure mode and addresses it directly.\n2. High-quality dataset creation: the MED dataset is carefully constructed using various tools for editing, filtering, and alignment. The dataset and benchmark can be a solid contribution. \n\nWeaknesses:\n1. Finetuning with specific learning objective on a specific dataset are expected to gain improvements, but it is not clear how the fine-tuned models perform on general benchmarks. \n2. the MED-Real set only has 35 image pairs, which might to small to verify real-world applicability.\n3. Other families of open-source models to be evaluated, e.g., BLIP-2 / BLIP-3.\n\nQUESTIONS:\nSee weaknesses.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper is targeting a core-issue in Large Vision-Language Models(LVLM) -- Fine-grained understanding of image content. First, the authors have shown that even in some most advanced LVLMs, such as GPT4o, there still exists issues with detail understanding. And they conclude these errors into a range of types. Then the authors construct a dataset for fine-grained image learning by their proposed pipeline. This pipeline uses image-editing to subtly modify two images and ask questions related to the original and edited images. Then the author argued that the models trained to describe the differences between two similar images are not doing well in the downstream description task. Inspired by this observation, the author proposed a new supervision training method to overcome this problem.  In this way, the model can learn to discreminate the fine-grained differences. The author also proposed a new loss function wich included the contrastive loss to train the LVLMs and show the performance superiority over other methods.\n\nSTRENGTHS AND WEAKNESSES:\nWeaknesses:\n1. This data generation pipeline involved too many models: from open source to close source, from LVLM to LLM; acrossing different platform; including text generation and image generation models. This whole pipeline is quite complicate and thus hard to replicate your work because of the involving of too many models. \n2. Though I personally do not have any problem with leveraging synthetic data in training models, I still doubt the quality of the proposed datasets. I think the authors should release more quantitative results to evluate their prompt template and instruction quality.\n\n\nStrength:\n1. For the MED-Benchmarks, this work manually verified and categorize the question types. They also do the examination of data contamination for the splits.\n2. This work does not simply train a downstream models on the curated datasets but verified the generalizing problem and minimize the performance gap in downstream tasks by proposing a new training method. The authors suggest that the performance gap of the existence LVLMs might be caused by the noisy training data which has a binomial probability distribution for giving a good quality description of the paired images.  And thus this work proposing a clean supervision training pipeline to mitigate such noise. \n3. The curated dataset is quite valuable to the research community.\n\nQUESTIONS:\n1. In the data generation process, the pipeline uses a QwenVL model to generate instruction and filtering the images from the raw dataet. Then, the authors use the gemini model to generate the image pairs from the generated instructions. Since the instruction generation process is very important for the quality of generated images in the proposed MED dataset, do you consider any other quality control method for the instructions. From my understanding, QwenVL is indeed very powerful, but is far from perfect. A Qwen2.5VL-72B model can also have hallucination during generating instructions.\n\n2. The authors use another data filtering module powered by CLIP models to varify the generated images quality compared to the original images. However, from my experiences with many image generative models, only semantic filtering is not enough for generated image quality control. In fact, the author should consider introducing some basic image quality evaluation method to control the qulity(i.e. SSIM). Furthermore, the authors only do CLIP embedding similarity check between the original image and the edited image but not between the edited image and the instruction. Why is that?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 4, "IMPACT": 4, "ORIGINALITY": 3, "DATE": "2025-07-03"}