Title: Seeing is Believing? Mitigating OCR Hallucinations in Multimodal Large Language Models
Venue: NeurIPS 2025
Accepted: yes
Keywords: OCR

MLLM

hallucination
Abstract: Recent advancements in multimodal large language models (MLLMs) have enhanced document understanding by integrating textual and visual information. However, existing models exhibit incompleteness within their paradigm in real-world scenarios, particularly under visual degradation (e.g., blur, occlusion, low contrast). In such conditions, the current response paradigm often fails to adequately perceive visual degradation and ambiguity, leading to overreliance on linguistic priors or misaligned visual-textual reasoning. This difficulty in recognizing uncertainty frequently results in the generation of hallucinatory content, especially when a precise answer is not feasible. To better demonstrate and analyze this phenomenon and problem, we propose KIE-HVQA, the first benchmark dedicated to evaluating OCR hallucination in degraded document understanding. This dataset includes test samples spanning identity cards, invoices, and prescriptions, with simulated real-world degradations and pixel-level annotations for OCR reliability. This setup allows for evaluating models' capacity, under degraded input, to distinguish reliable visual information and answer accordingly, thereby highlighting the challenge of avoiding hallucination on uncertain data. To achieve vision-faithful reasoning and thereby avoid the aforementioned issues, we further introduce a Group Relative Policy Optimization (GRPO)-based framework featuring a novel reward mechanism. By incorporating a self-awareness of visual uncertainty and an analysis method that initiates refusal to answer to increase task difficulty within our supervised fine-tuning and reinforcement learning framework, we successfully mitigated hallucinations in ambiguous regions. Experiments on Qwen2.5-VL demonstrate that our 7B-parameter model achieves a ~28% absolute improvement in hallucination-free accuracy over GPT-4o on KIE-HVQA and there is no significant performance drop in standard tasks, highlighting both effectiveness and robustness. This work advances the development of reliable MLLMs for real-world document analysis by addressing critical challenges in visual-linguistic alignment under degradation.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper aims to address MLLM’s OCR hallucination issue in degraded document understanding. The authors propose a dataset named KIE-HVQA, consisting of various documents with real-world degradation and pixel-level annotation. For annotation, they tag the degraded area (blur, occlusion, low contrast) using special tokens like `<part_occluded>` and `<occluded>`, following the correct content. They also propose a new RL-based KIE MLLM, which is trained using GRPO and self-designed rewards. The authors conduct experiments to validate their methods’ effectiveness.\n\nSTRENGTHS AND WEAKNESSES:\n> Strengths:\n\n1. The authors discover a novel perspective: the hallucination of OCR on degraded documents. This is interesting and can encourage the community to pour more attention into this underexplored issue.\n2. The KIE-HVQA dataset is a valuable contribution to the community. There is a lack of document analysis datasets with such real-world, character-level degradation. This pioneering effort fills this gap and potentially contributes to broader applications in the document field beyond this paper’s task.\n3. The KIE-HVQA includes diverse data sources, including receipts, documents, and ID cards. The authors generate ID card images using GPT-4o to avoid privacy issues. This is an effective and important approach to producing privacy-sensitive data.\n4. In Sec. 4.2, the authors generate CoT data using an iterative approach, which is reasonable for producing high-quality CoT data.\n\n> Weaknesses:\n\n1. In Sec. 3.2, L141-142, the authors only say they use GPT and Qwen2.5-VL-72B to evaluate the degraded results, without giving a concrete evaluation process. Why are two models used here? How do they operate? The authors should provide more details.\n2. In Sec. 3.3, how do the authors implement the measurement of legible and degraded character accuracy? The authors provide only definitions without implementation details.\n3. In Fig. 3, the bottom-right panel “3. Supervised Fine-Tuning” is unclear and confusing. What is the role of Qwen2.5-VL here? SFT is a training procedure. If it takes the QA pair as input, it looks more like a data generation procedure, which should not occur during training. In addition, what does the “CoT” output by Qwen2.5-VL mean? I think the SFT model is trained using the CoT data generated during “2. Data Generation”, and the CoT here makes me confused. Can the authors provide some clarifications and make Fig. 3 clearer?\n4. The title of Sec. 4.2 is “Cold-start Initialization”. As far as I know, cold-start initialization refers to supervised fine-tuning (SFT) on certain data. Yet, this sub-section mainly describes the CoT data generation process rather than the training process. If the authors aim to describe data generation, then they should change the title. Also, it seems that the authors have omitted the illustration of SFT (despite its presence in Fig. 3) since Sec. 4.3 is about RL training. An SFT-related sub-section should precede RL training.\n5. In Sec. 4.3, the authors define three rewards (legible character, partially obscured but human-recognizable characters, unrecognizable characters), but do not provide their detailed implementations. Algorithm 1 appears to compute rewards; however, the definitions inside do align with the three rewards (the outputs’ names are not_clear_metric, final_metric, and clear_metric, which are different from the above three). The authors should give necessary details for reward computing.\n6. For Algorithm 1, the CALCULATE_METRICS function simply outputs one similarity. However, in line 11 of this algorithm, this function outputs three values. This inconsistency is confusing.\n7. In experiments, what prompts are used in testing the closed-source models (e.g. GPT, Claude)? Are the prompts the same as those used for training the proposed model to ensure fairness?\n8. In Table 1, the proposed model exhibits better performance than other models not trained on the proposed dataset. However, for recognition of those occluded texts, what is the rationale behind the outperformance of the proposed models? Recognizing visual text mainly depends on the ability of the visual encoder. Since the authors did not specifically improve visual modeling and text is occluded, performance on these texts should be close. Can the authors provide a deeper analysis of such outperformance?\n9. In L322-323, what does the format reward refer to? I did not found any definition of the format reward in either the methodology section or the experiment section.\n10. typo: L106: MLMMs -> MLLMs; L145: We -> we; L307: KIE-HVQA -> KIE-HVQA. L320: setting -> Setting\n\nQUESTIONS:\nFor rebuttal questions, please see weaknesses. The major concern of this paper lies in the lack of many critical model implementation and experimental details. If the authors can address these issues, I will consider raising my score.\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 5, "CLARITY": 2, "IMPACT": 3, "ORIGINALITY": 4, "DATE": "2025-06-21"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes the first benchmark specifically designed to evaluate OCR hallucination in degraded images, covering diverse document types such as identity cards, invoices, and prescriptions. To address this real-world challenge, the authors further introduce a Group Relative Policy Optimization (GRPO)-based training framework, incorporating a novel reward mechanism to fine-tune Multimodal Large Language Models (MLLMs). Experimental results demonstrate the effectiveness of the proposed framework on the newly introduced benchmark.\n\nSTRENGTHS AND WEAKNESSES:\nStrength\n1.\tThe paper proposes the OCR benchmark with degraded documents—a challenging yet realistic task setting that is crucial for enhancing the reliability of MLLMs.\n2.\tThe proposed GRPO-based training framework effectively improves model performance on the newly introduced benchmark.\n\nWeakness\n1.\tThe benchmark lacks essential details. It would be better to provide comprehensive dataset documentation by including key statistics (e.g., size, category distribution) and a clear explanation of the dataset construction process. Additionally, incorporating human expert validation can improve the annotation quality and enhance the benchmark's overall credibility. \n2.\tAs illustrated in Figure 2, the benchmark introduces numerous novel tokens (e.g., <part_occluded>) that are unseen during MLLM pretraining. Consequently, existing MLLMs are inherently incapable of generating these tokens. So the authors should provide more details about how to control comparison fairness between the trained model and the original model on generating unseen tokens.\n3.\tThe proposed GRPO-based framework only modifies the reward design without introducing significant architectural or algorithmic innovations. So the contribution of this paper from a technical perspective should be strengthened. \n4.\tThe method does not show improvement on traditional OCR benchmarks, which raises concerns that the observed gains on the proposed benchmark may result from overfitting to specific data patterns. So it would be better to validate the model’s effectiveness and generalization ability with broader experiments.\n\nQUESTIONS:\nPlease refer to the listed weakness.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 3, "CLARITY": 2, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper presents KIE-HVQA, the first benchmark for measuring OCR hallucinations in degraded document images with character-level annotations. A 7B model is trained via supervised CoT followed by GRPO with a reward that penalizes hallucinations. The system achieves ~28% higher hallucination-free accuracy than GPT-4o on KIE-HVQA while matching OCR baselines on standard tasks, combining a novel benchmark and RL method for robust document understanding\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n1. First, the benchmark itself fills a clear gap: existing OCR/VQA suites rarely test degraded inputs or explicitly label which characters are uncertain. By supplying pixel-level reliability masks and diverse degradation types, KIE-HVQA offers the community a concrete target for measuring and analysing hallucinations in safety-critical settings such as ID verification and medical forms.\n2. Second, the technical solution is well motivated and experimentally validated. The GRPO framework, with its uncertainty-aware reward, yields large gains over strong proprietary and open-source baselines without eroding general OCR accuracy.\n\n**Weaknesses**\n1. Despite its realism, the dataset is modest in scale and partially synthetic, relying on GPT-4o-generated documents; it is unclear whether 2 400 samples cover the full variability of real-world paperwork or support robust generalisation to unseen layouts and languages.\n2. The reward formulation assumes oracle-level degradation labels during training, practical deployment would lack such pixel-wise ground truth, so the paper should discuss how performance degrades when degradation detection itself is noisy or learned.\n3. Finally, the evaluation omits specialised vision-only OCR engines or hybrid pipelines that could be combined with LLM reasoning; without these baselines it is hard to disentangle the benefit of the new learning paradigm from that of simply adding more OCR-centric supervision.\n\nQUESTIONS:\n1. Have you tested the model on entirely unseen corporate or governmental documents to verify generalisation beyond the synthetic portion?\n2. How does hallucination suppression change if degradation masks are obtained from an automated detector rather than ground truth?\n3. Could the authors report results for a system that first runs a state-of-the-art OCR engine, then feeds text and images to the LLM? This would clarify the specific advantage of GRPO.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces KIE-HVQA, the first benchmark explicitly designed to evaluate OCR hallucinations in multimodal large language models (MLLMs) under degraded visual conditions. The benchmark includes degraded document images with pixel-level OCR reliability annotations, enabling precise assessment of hallucination behavior. To address this challenge, the authors propose a novel reinforcement learning framework using Group Relative Policy Optimization (GRPO), combined with a new reward function that accounts for visual uncertainty. The approach incorporates both supervised fine-tuning and reinforcement learning to encourage models to recognize and appropriately respond to degraded visual information. Experiments on Qwen2.5-VL show substantial improvements over state-of-the-art models like GPT-4o, especially in hallucination-prone settings, while preserving general OCR performance.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n- The paper presents a rigorous and thorough experimental setup, including well-designed benchmarks, precise annotation strategies, and a custom reward function tailored to OCR degradation. The results clearly demonstrate strong gains in hallucination suppression and robustness across diverse degradation types.\n- This work tackles a critical and underexplored problem in OCR image understanding, vision-language hallucination in degraded inputs. It offers not only a novel dataset (KIE-HVQA) but also a method to mitigate OCR errors.\n- The proposed GRPO-based framework, with a multi-objective reward function, is a novel approach to mitigating OCR hallucinations. The emphasis on \"vision-faithful reasoning\" and incorporating uncertainty-driven refusal behaviors directly into the training process via reinforcement learning is innovative and well-aligned.\n\n**Weakness**\n- While the paper effectively demonstrates the problem and its proposed solution, it lacks a comprehensive comparison with other potential or existing mitigation techniques for hallucination in MLLMs (beyond just showing that other models hallucinate). A discussion or comparison with methods that might approach hallucination from different angles (ex., uncertainty quantification, confidence estimation, or alternative training paradigms) would provide a broader context and further highlight the unique advantages of the proposed GRPO-based approach.\n- The reinforcement learning approach is adapted from prior work (ex., DeepSeek-R1), and while the integration with OCR-specific reward design is well-executed, it builds incrementally on known RL optimization principles.\n- Lack of comparison and evaluation on other widely used datasets like TextVQA[1], ST-VQA[2], OCR-VQA[3]. These datasets contain OCR images and bounding box data along with them.\n\n[1] Singh, A., Natarajan, V., Shah, M., Jiang, Y., Chen, X., Batra, D., Parikh, D. and Rohrbach, M., 2019. Towards vqa models that can read. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (pp. 8317-8326). \\\n[2] Biten, A.F., Tito, R., Mafla, A., Gomez, L., Rusinol, M., Valveny, E., Jawahar, C.V. and Karatzas, D., 2019. Scene text visual question answering. In Proceedings of the IEEE/CVF international conference on computer vision (pp. 4291-4301). \\\n[3] Mishra, A., Shekhar, S., Singh, A. K., and Chakraborty, A., \"OCR-VQA: Visual Question Answering by Reading Text in Images,\" 2019 International Conference on Document Analysis and Recognition (ICDAR), pp. 947-952\n\nQUESTIONS:\nPrior works[3] have tried to address a similar problem of models not focusing on visual details. Under settings like TextVQA[1], STVQA[2], have the authors done any experiments or have any thoughts as to how the authors' proposed method would work?\n\n[1] Singh, A., Natarajan, V., Shah, M., Jiang, Y., Chen, X., Batra, D., Parikh, D. and Rohrbach, M., 2019. Towards vqa models that can read. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (pp. 8317-8326). \\\n[2] Biten, A.F., Tito, R., Mafla, A., Gomez, L., Rusinol, M., Valveny, E., Jawahar, C.V. and Karatzas, D., 2019. Scene text visual question answering. In Proceedings of the IEEE/CVF international conference on computer vision (pp. 4291-4301). \\\n[3] Hegde, S., Jahagirdar, S. and Gangisetty, S., 2023. Making the v in text-VQA matter. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 5580-5588).\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-03"}