Title: ViCrit: A Verifiable Reinforcement Learning Proxy Task for Visual Perception in VLMs
Venue: NeurIPS 2025
Accepted: yes
Keywords: Visual reasoning; Vision-Language Model; Visual captioning; Reward Model; Visual Hallucination
Abstract: Reinforcement learning (RL) has shown great effectiveness for fine-tuning large language models (LLMs) using tasks that are challenging yet easily verifiable, such as math reasoning or code generation. However, extending this success to visual perception in vision–language models (VLMs) has been impeded by the scarcity of vision-centric tasks that are simultaneously challenging and unambiguously verifiable. To this end, we introduce \textbf{ViCrit} (\textit{Visual Caption Hallucination Critic}), an RL proxy task that trains VLMs to localize a subtle, synthetic visual hallucination injected into paragraphs of human-written image captions. Starting from a 200-word captions, we inject a single, subtle visual description error—altering a few words on objects, attributes, counts, or spatial relations—and task the model to pinpoint the corrupted span given the image and the modified caption. This formulation preserves the full perceptual difficulty while providing a binary, exact-match reward that is easy to compute and unambiguous. Models trained with the \textbf{ViCrit Task} exhibit substantial gains across a variety of VL benchmarks. Crucially, the improvements transfer beyond natural-image training data to abstract image reasoning and visual math, showing promises of learning to perceive rather than barely memorizing seen objects. To facilitate evaluation, we further introduce \textbf{ViCrit-Bench}, a category-balanced diagnostic benchmark that systematically probes perception errors across diverse image domains and error types. Together, our results demonstrate that fine-grained hallucination criticism is an effective and generalizable objective for enhancing visual perception in VLMs.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nViCrit is a proxy task to enhance visual perception of Vision-Language Models (VLMs). The objective is to teach a VLM to spot and correct a visual hallucination that was synthetically injected in an existing image caption by prompting a proprietary VLM such as GPT-4. The correctness of the generated caption is easily verifiable by exact matching, and this allows for optimization of the target VLM with GRPO, using an additional format reward to incentivize thinking. The authors also propose a new benchmark called ViCrit-Bench, where they show that Qwen2.5-VL-Instruct (7B and 72B) achieve state-of-the-art performance after training with the ViCrit task.\n\nSTRENGTHS AND WEAKNESSES:\n**STRENGTHS:**\n1. ViCrit introduces an intuitive objective that can act as the reward signal for reinforcement learning. The procedure to build ViCrit data is simple (although it may require proprietary VLM to work the best) and should be scalable given that image captioning data is relatively simple to obtain compared to other kinds of annotations, *e.g.* bounding boxes, segmentation masks, etc. The other contribution of this work, ViCrit-Bench, is well-documented and appears to be a challenging benchmark even for proprietary VLMs.\n\n2. The effectiveness of the ViCrit task is exemplified by the strong performance achieved by Qwen2.5-VL after reinforcement learning. Notably, ViCrit enhances performance across different benchmarks other than ViCrit-Bench, such as visual hallucination and mathematical reasoning.\n\n**WEAKNESSES:**\n1. Experiments are carried out exclusively on Qwen2.5-VL. Despite its solid results, we urge to apply ViCrit on different VLMs (for instance, LLaVA-1.5 [1]) to test whether it can generalize to other models.\n\n2. *(minor)* Table 2 shows encouraging results overall. However, ViCrit severely hurts Qwen2.5-VL performance on \"Text\" hallucinations. The authors should discuss this point and provide insights on the possible causes.\n\n3. *(minor)* Appendix E readily compares ViCrit against an SFT baseline, given the same amount of data. I appreciate it, and yet I would have included additional experiments to further isolate the role of GRPO and the incentive to reason: what if we simply fine-tune a VLM with SFT to generate the non-hallucinated caption, given the hallucinated caption as prompt? This ties up ViCrit and SFT also on the kind of data being used, and not only on the amount.\n\n**References**\n\n[1] Liu, Haotian, et al. \"Improved baselines with visual instruction tuning.\" CVPR 2024.\n\nQUESTIONS:\nI think that this paper is valuable, and I encourage the authors to address the weaknesses listed above, in particular **W1**.\n\nMoreover:\n\n- Did the authors try to build ViCrit annotations starting from shorter captions (e.g. 100 words rather than 200)? Has this even had an impact on the final performance?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 4, "IMPACT": 4, "ORIGINALITY": 3, "DATE": "2025-06-19"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces a novel task called the ViCrit task, which enables models to detect visual hallucinations in captions and locate the corresponding errors. The authors also construct a challenging benchmark, ViCrit-Bench, for hallucination detection, which includes four types of visual perception. Using this task and dataset, they employ RL to train the inference model ViCrit, improving the model's performance in hallucination mitigation, mathematics, VQA, and counting tasks.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\n1). The proposed new task is interesting, and the construction of the dataset is reasonable. The ViCrit model, trained using RL, shows improvements across various tasks.\n\n2). The paper is well-written, with a clear format and easy readability.\n\nWeaknesses:\n\n1). The paper does not clearly explain the strong motivation for introducing this dataset. Common VQA datasets are also verifiable, and it is unclear what advantages this dataset offers over existing ones.\n\n2). The experimental comparisons are unfair, as there is no comparison with other reasoning models such as Vision-R1, R1-Onevision, etc.\n\n3). The ViCrit task primarily focuses on visual perception. Further experimental analysis is needed to demonstrate how training on this task can enhance the model's generalization ability in reasoning tasks, such as mathematics.\n\nQUESTIONS:\n1). Could you provide a detailed data comparison analysis and compare your dataset with standard VQA datasets to demonstrate its advantages?\n\n2). Could you provide comparisons with other reasoning models, such as Vision-R1, R1-Onevision, etc., to demonstrate the performance advantages of your method?\n\n3). Could you provide more detailed experiments and analysis to explain why training on visual perception tasks improves performance and generalization in reasoning tasks such as mathematics?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a novel perception task, called ViCrit, which requires VLMs to predict challenging corrupted words in the original detailed captions. This task is proven to be effective for training VLMs with RL methods. Extensive evaluations on both general multimodal benchmarks and hallucination examination benchmarks demonstrate the effectiveness of the proposed method. The authors also introduce a benchmark, ViCrit-Bench, to validate the reconstruction accuracy of corrupted words in captions.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n- This paper proposes ViCrit, a novel perception task that prompts models to predict corrupted words in original captions. This approach is simple to implement and useful for training VLMs with RL strategies.\n- The authors have conducted extensive evaluations on both general and hallucination benchmarks, demonstrating the effectiveness of the proposed method.\n -  A new benchmark, ViCrit-Bench, has also been introduced to evaluate the reconstruction accuracy of corrupted words in detailed captions.\n\nWeakness\n - Limited RL training details. Since unstable training is a common issue in RL, key details (e.g., rollout number, reward curve) are crucial for the research community, especially in the VLM domain.\n - No training recipe is provided, making reproducibility challenging.\n - For ViCrit-Bench, human performance metrics are missing. These would help estimate the benchmark’s approximate upper bound.\n - Lack of benchmark examples or case studies. Without concrete cases, it is difficult to assess the benchmark’s difficulty or practical validity.\n\nQUESTIONS:\nIn the section 4.1, the images are categorized into four domains. I think for vision understanding the GUI images are also important and common for VLMs. Why this domain was not considered in your data preparation?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 5, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces a novel vision-centric reinforcement learning task, ViCrit, which requires VLMs to identify subtle visual hallucinations within image captions. This task is designed for efficient and accurate evaluation and aligns well with reinforcement learning training frameworks. Building on this task, the paper further proposes ViCrit-Bench as a diagnostic benchmark to assess the fine-grained visual perception capabilities of vision-language models across four distinct visual domains.\n\nSTRENGTHS AND WEAKNESSES:\nStrength：\n1.The paper proposes a novel vision-centric reinforcement learning task, ViCrit, which effectively detects fine-grained visual hallucinations in image captions and offers strong practical value and evaluation efficiency.\n2.Building upon the ViCrit task, the authors develop ViCrit-Bench, a benchmark covering four visual domains that systematically diagnoses the visual perception ability of multimodal models.\n\nWeakness：\n1.The paper lacks detailed information on the GRPO parameter settings used in ViCrit training and does not provide visualized training metrics, which affects reproducibility and methodological transparency.\n2.The experiments only compare model performance before and after ViCrit training, without comparing against other tasks or datasets, making it difficult to fully demonstrate the unique advantages of the ViCrit task.\n\nQUESTIONS:\n1. Could the authors provide details on the GRPO parameter settings used during training for the ViCrit task, as well as visualized curves of relevant metrics during the training process?\n\n2. The paper only compares model performance before and after training on the ViCrit task. Could the authors also present comparisons where the same model is trained on other  tasks or datasets, to better demonstrate the advantages of ViCrit?\n\n3. Have the authors conducted experiments on other base models to validate the adaptability and effectiveness of the ViCrit training across different model architectures?\n\nLIMITATIONS:\nthe paper may have the following limitations:\n1.It lacks sufficient information related to the reinforcement learning training process.\n2.It does not provide performance comparisons of the model on other tasks or datasets, limiting the understanding of its generalizability.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-03"}