ViPER: Empowering the Self-Evolution of Visual Perception Abilities in Vision-Language Models
Authors: Juntian Zhang
Song Jin
Chuanqi Cheng
Yuhan Liu
Yankai Lin
Xun Zhang
Yufei zhang
Fei Jiang
Guojun Yin
Wei Lin
Rui Yan
Venue: ICLR 2026 Poster (2026)
Keywords: Vision-Language Models
visual perception
self-evolution
reinforcement learning
Primary area: applications to computer vision, audio, language, and other modalities

Abstract: The limited capacity for fine-grained visual perception presents a critical bottleneck for Vision-Language Models (VLMs) in real-world applications. Addressing this is challenging due to the scarcity of high-quality data and the limitations of existing methods: supervised fine-tuning (SFT) often compromises general capabilities, while reinforcement fine-tuning (RFT) prioritizes textual reasoning over visual perception. To bridge this gap, we propose a novel two-stage task that structures visual perception learning as a coarse-to-fine progressive process. Based on this task formulation, we develop ViPER, a self-bootstrapping framework specifically designed to enable iterative evolution through self-critiquing and self-prediction. By synergistically integrating image-level and instance-level reconstruction with a two-stage reinforcement learning strategy, ViPER establishes a closed-loop training paradigm, where internally synthesized data directly fuel the enhancement of perceptual ability. Applied to the Qwen2.5-VL family, ViPER produces the Qwen-Viper series. With an average gain of 1.7\% on seven comprehensive benchmarks spanning various tasks and up to 6.0\% on fine-grained perception, Qwen-Viper consistently demonstrates superior performance across different vision-language scenarios while maintaining generalizability. Beyond enabling self-improvement in perceptual capabilities, ViPER provides concrete evidence for the reciprocal relationship between generation and understanding, a breakthrough to developing more autonomous and capable VLMs.

Published as a conference paper at ICLR 2026
VIPER: EMPOWERING THE SELF-EVOLUTION OF VI- SUAL PERCEPTION ABILITIES IN VISION-LANGUAGE MODELS
Juntian Zhangl.2, ,ong Jin Chuanqi Cheng', Yuhan Liu', Yankai Lin', Xun Zhang", Yufei Zhang". Fei Jiang', Guojun Yin', Wei Lin"; Rui Yan "Galling School of Artificial Intelligence, Renmin University of China, MMeituan, MBZUUA, *Wuhan University izhangjjntian jinsong8, chengchuangi, yuhan. liueeruc...u.. edu, on
ABSTRACT
The limited capacity for fine-grained visual perception presents a critical bottle- neck for Vision-Language Models (VLMs) in real-world applications. Address- ing this challenging due to the scarcity of high-quality data and the limitations of existing methods: supervised fine-tuning (SFT) often compromises general capabilities, while reinforcement fine-tuning (RFT) prioritizes textual reasoning over visual perception. To bridge this gap, propose novel two-stage task that structures visual perception learning as a coarse-to-fine progressive process. Based on this task formulation, we develop ViPER, a self- boltsrootttrppinnggggg frame- work specifically designed to enable iterative evolution through self-critiquing and self- seff-prediction. By synergistically integrating image-level and instance-level re- construction with a two-stage reinforcement learning strategy, ViPER establishes a closed-loop training paradigm, where internally synthesized data directly fuel the enhancement of perceptual ability. Applied to the Qwen2.5-VV family, ViPER produces the Qwen- Viper series. With an average gain of 1.7% on seven com- prehensive benchmarks spanning various tasks and up to 6.0% fine-grained perception, Qwen-Viper consistently demonstrates superior performance across different vision-language scenarios while maintaining generalizability. Beyond enabling self-improvement in perceptual capabilities, ViPER provides concrete evidence for the reciprocal relationship between generation and understanding, a breakthrough to developing more autonomous and capable VLMs.
INTRODUCTION
Multimodal Large Language Models (MLLMs) have exhibited strong capabilities and wide-ranging application value across diverse domains (Caffagni et al. (024)). In contrast to Large Language Models (LLMs), MILMSs extend beyond text to multiple modalities, enabling them to handle more complex, information-ride tasks and establishing them critical for embodied Al and world mod- els (Yin et al. 2024). As an important type of MLLMs, Vision-Language Models (VLMs) are typically architected with visual encoder to process pixel-level information and language model backbone for semantic understanding, establishing a connection between the two modalities (Li al. 2024 Wang et al. 2025b). Consequently, the overall performance of VLM depends the coordinated functioning of its visual encoder and language backbone.
Recent progress, such ol Kil-Kishky 2024) and Deepseek-R1 (Ren al. 2025), has advanced the reasoning ability of I.LMs, leading to increased focus on "slow thinking" " [Wang et al. (002cc). Ac- cordingly, the field has led to the development of multimodal reasoning models (Huang et al. [2025a] Li et al. 2025). Although Chain-of- Thought methods (Wei et al.) (002)) have improved the reason- ing ability of these models, the expressive limitations of the text present inherent constraints (Lyu et al., 2024: Stechly et al. 2024). For many vision-language tasks, especially those demanding fine tine-grained perception, performance cannot be attributed solely the power of the language model
Corresponding author.

---

Published as a conference paper at ICLR 2026
backbone. These tasks hinge critically on sophisticated visual understanding coupled with language reasoning, a dual requirement that poses a significant challenge for current VLMs (Shao et al. 2024] Hu et al. 2024). This interdependence means that post-training efforts which enhance only the vi- sual or linguistic component in isolation yield marginal improvements, underscoring the need for approaches that enable their co-evolution.
Several recent studies have proposed task-specific architectures to address the challenges of fine- grained visual perception. However, these approaches often exhibit notable limitations. widely adopted strategy (Udandarao et al.] 2024; Hou et al. 2025 Zhang et al. 2025c Cao & Ou 2025 in-
volves scaling up training data by distilling knowledge from superior models. This paradigm is computationally inefficient due to the high cost of data synthesis, and large-scale distillation can lead to marginal performance gains while compromising generalization, resulting in an unsustain- able trade-off. In contrast, another line of research employs reinforcement learning (RL. within "ainkiing-wttiimmgee" paradigm where performance is improved through iterative tool use and
multi-step interaction (Zheng et al.. 2025b; Shao al. 2024 Hu et al. 2024 Gao et al. 2024 Zhou ct al. 2024). While this improves system-level outcomes, it introduces substantial latency due to multi-round interactions. Moreover, the reasoning process often becomes superficial, emphasizing tool manipulation over visual comprehension, and fails to improve the model's underlying percep-
tual capabilities on downstream tasks (Su et al. 2025a Zhang et al. 2025a Sun al. 2024).
To address these challenges, we formulate the enhancement of visual perception as a two-stage structured process. The first stage cultivates holistic image reasoning and static scene understanding via self-critical caption refinement, teaching the model to "see widely". The second stage focus on fine-grained perception and dynamic change awareness by inferring visual operations from image differences, training the model to "focus accurately". This deliberate coarse-to-fine progression structures the model's learning trajectory from global understanding to local precision.
Correspondingly, we introduce ViPER, self-evolutionarry framework that concretizes the two-stage task by integrating data synthesis and the RL method. ViPER consists of two components: an auto- mated data synthesis module and a closely integrated two-stage RI. method. By incorporating both image-level and instance-level reconstruction, ViPER's data synthesis establishes a bidirectional mapping between visual and textual modalities at bual-granularities, intrinsically using the genera- tion process to solidify perceptual understanding. The self-synthesized data is seamlessly channeled into tightly coupled two-stage RL process. This integration creates self-reinforcing loop where generation and learning are intertwined, eliminating the need for external bootstrapping and thereby establishing self-evolving paradigm.
Based on ViPER, we constructed the Viper/OK dataset and yield Owen-Viper series perceptually enhanced from Qwen2.5-VL. Experiments revealed that during training, the models spontaneously developed a "thinking tthinking-with-imass -ith--maaee capability and learned to redirect attention to critical details. On seven comprehensive benchmarks encompassing single-image, multi-image, and hallucination tasks, the Qwen-Vipe series demonstrated consistent gains over the baseline. Notably, on fine- grained perception tasks, the 3B and 7B models achieved significant improvements of up to 4.4% % and 6.0%, respectively. These results validate that ViPER enables VLMs to undergo self-evolution on perception-centrcc tasks.
In summary, our main contributions are threefold:
I. We designed a progressive two-stage task that guides VLMs from holistic scene reasoning to fine-grained visual understanding, establishing a structured learning paradigm for perceptual self- improvement.
II. We propose ViPER, seff-evolutionary paradigm that bootstraps VLM perception through closed loop of data construction and RFT, producing ViperIOK and the Qwen Viper models. !
III. We demonstrate that Qwen-Viper achieves consistent improvements on perteption-inteniive tasks, revealing the reciprocal relationship between generation and understanding by practice.
intensive
We open-source the implementation codes and data at http:://jtttbhccomm//mss/////19p
2

---

Published as a conference paper at ICLR 2026
2
RELATED WORK
2.1
VISION-LANGUAGE MODEL
The rapid advancement of LLMs has catalyzed substantial progress in VLMs Yang et al. 2025 Liu al. 2024a|b) Representative proprietary models include GPT-4o (Hurst et al., 2024). Gemini 2.5 (Comanici et al. 2025). and Claude3 (Anthropic) (024). Among open-source mod- els, LLaVA (Liu et al. 2023) has gained widespread adoption through its three-stage architecture, which comprises Vision Transformer (VIT) (Dosovitskiy et al., (020), connector module and an LLM. Subsequent research has largely extended and refined this architecture. For example, LLaVA-One Vision (Li ct al. 2024) introduced a dynamic resolution mechanism that adaptively ad- justs the number of visual tokens based on the input image resolution. Similarly, Qwen2-VL. (Wang et al. (0244) also adopts this encoding strategy and incorporates the M-RoPE positional encoding mechanism, enabling unified processing of ID text, 2D images and 3D video data. Its successor, Qwen2.5-VL (Bai et al. 2025) further integrates sparse attention mechanisms within the visual en- coder architecture. DeepSeck-Ve VL [L et al. (2024) optimizes encoding through dual encoders that separate visual and textual information for various downstream tasks. The Intern VL series (Zhu et al. 2025 Wang et al., 2025b) proposed using thumbnail to integrate global image information without substantially increasing the number of tokens.
2.2
REINFOR LEARNING-ENHAANCE MULTI-MODAL REASONING
The integration of reinforcement learning (RL) has substantially enhanced the reasoning capabilities LLMs, as demonstrated by models such DeepSeek-RR R1 (Guo al.) 2025) and Kimi-K1.5 (Tea
et al. 2025). This progress is driven by effective RL frameworks, including DPO (Rafailov et al.) PPO (Schulman et al., 2017). GRPO (Guo et al., 2025), GSPO (Zheng et al. 2025a). and DAPO (Yu et al.) (025a). Recent work has successfully extended these methodologies to VLMs, leading to promising results in systems such as Vision-R R1 (Huang et al.] 2025a) and VLM-RI (Shen
2023
ct al. 2025). Alongside end-to-end RL training paradigm. significant research direction focuses on tool -uggenedde methods for multi-modal reasoning (Zhang et al.) 2025b Huang et al. 2025b Su
et al. 2025b). The success of multi-tury tool-calling in advanced agents such as Open 03 (Ope-
nAI 2025) has inspired numerous efforts to equip models with specialized visual tools. Represen- tative works include DeepEyes (Zheng et al.) 2025b), Chain-of-Focus (Zhang et al. (025d)), and
Mini-o3 (Lai et al. 2025) aiming to improve the models' ability to dynamically invoke tools for solving complex reasoning tasks.
In contrast to these methods, our proposed method ViPER integrates data construction with post- training process to form a closed-loop cycle. The framework employs guallgrraularrtyy reconstruc- tion processes to promote understanding by generation, enabling self-driven evolution without ex- ternal supervision or large-scale data.
3 METHODOLOGY
In this section, we first formalize the two-stage tasks in Section 3.11 Then we elaborated on the pro- posed self-evolutionary framework ViPER in Section $3.2 and introduced its two key components: the data synthesis module and the coupled two-stage reintorcement learning strategy.
3.1 TASK FORMULATION
Caption Self-Refining: The first stage involves the self-refinement of captions. This task is designed train unified VLM revise errors or biases in its own generated textual descriptions through deconstructing visual information and self-reflection.
Specifically, for each training sample (I, Cg, R) E D. the model performs the task of self- correction. It takes the image and the initial caption C which is generated by itself with pa- rameters do as input, analyses inaceurcy in the original caption and outputs a set of refinement actions:
Ryped S(T,Cy;0)
3

---

Published as a conference paper at ICLR 2026
Original Gastion
Image
Imaget
Image Reconstruct: Qeen Quen-Inage
Secudent cossert leaturgg io mund cala aced ittaal leare
The owent in 180, croary tooding with pale yollow
Oryifeed details
I2T
?
Inasuurte ontribunes
The cate placed in squarr, godd codecolleeed
Spotia
The plaus on te cate neds Navaa Tokyyer.
Reconstruct Chni
"Ti wate pardiceed
Coption
I2T
Fiteet Enity shaped tag
Move ewayy the ovek shoee with parted aut Tokyy'
Visual Operations
Irathance
the lith
Tokyy' glottd
cream leyer near the right
rollouts
roliouts
spdate
Parmol rourd
Porect reand
2WV
Sampling
Ledene
Qwan2 JVL
Sampling
Caption Self-Refining
Vieual Operation Predicting
Figure 1: An overview of ViPER. The upper part illustrates the two-stage data synthesis framework, whose component is bidirectional vision-language mapping module composed of VLM and a diffusion model. The lower part depicts the corresponding two-stage RL process: the first stage focuses on caption self-refinement, and the second stage on visual-operation prediction. The Qwen2.5-Ve VL models are trained on ViperI0K constructed by the ViPER framework and result in Qwen- Viper, thereby achieving substantial celf-improvement
The goal of the training is to minimize the discrepancy between the predicted set of refinement points Rpred and the ground-truth set R by adjusting the parameters 0. This is formulated as minimizing the expectation of a discrepancy metric 5 RR Rpeea) over the training distribution:
min E LL.CC (ō (R, Rpeca].
Through this process, the VLM learns to iteratively refine own generated captions through an image-based reasoning process. This task is designed to enhance the model's ability for self- reflection and correction in visual perception, without relying on external correction mechanisms.
The second phase aims to train the VLM to predict visual opera- tions based on a pair of highly similar images with slight differences in visual information. The reconstructed image is generated by applying an editing operation to the original image, and the instruction itself is generated by the same VLM with parameters 0j. For each training sample (Torig: Trecon: Ops) E DD the VLM predicts the visual operations based on careful perception and reasoning across the image pair:
Visual-Operation Predicting:
OpSpred f (Forig Ireconi 8),
(3)
Where Opspred E T is the model's 's predicted visual operations. The objective of the training process to minimize the discrepancy between the model's predicted visual operations and the actual visual operation instructions:
min E. Omen (5 (Ops, OpSprea):
(4)
0
Where 8 TTTT R+ is a text sequence difference metric. Through this process, the VLM leamns to predict accurate visual operation instructions based on the differences between image pairs, thereby enhancing the model's 's ability to focus on key information and understand visual operations. The two-stage tasks, encompassing both static scene understanding and dynamic visual reasoning, not only enhance the model's image-based planning and reasoning abilities but also strengthen its fine-grained visual perception.
4

---

Published as a conference paper at ICLR 2026
3.2 VIPER: THE SELF-EVOLUTIONARY FRAMEWORK
In this section, we introduced the unified framework ViPER, which integrates the data synthesis pro- cess with a corresponding two-stage RL strategy, and elaborated on how enables self-evolutionary enhancement of perception capabilities.
3.2.1 THE DATA SYNTHESIS MODULE
A central part of ViPER is an automated data synthesis module, which functions as an integrated system for two-stage training data generation. This system seamlessly connects the upstream Cap- tion Self-Refining task to the downstream Visual-Operation Predicting task. The structure of this module is illustrated in the upper section of Figure
The first-stage data synthesis is centered on an image-level reconstruction process that creates a closed-loop feedback mechanism for the VLM. The VLM first generates static description from the original image. A diffusion model then reconstructs an image based on this description, which, due to the inherent information loss in visual-to-text conversion, exhibits local discrepancies from the original. These visual differences serve as a form of visual feedback, guiding the VLM to critique and refine its initial description by rectifying errors in object attributes, text, and spatial relationships. Thus, the diffusion model acts not merely as a generator, but as a critic that enables the VLM iteratively optimize its visual grounding.
In contrast, the second-stage synthesis shifts the focus to instance-level reconstruction. Leveraging the refined caption from the upstream task, the VLM selects hard entities and generates correspond- ing visual operation instructions through a hand-crafted heuristic rules, as detailed in Appendix A diffusion model then executes these instructions edit the original image, producing recon- structed version. The instructions used naturally serve as ground truth for the Visual-Operation Predicting task, where the VLM learns to infer the operation from the image pair. Crucially, by concretizing instructional intent into observable visual variations, the generative model enables the VLM to learn fine-grained visual reasoning from self-induced scene changes.
The introduction of the generative model imbues the process with profound significance: It exter- nalizes the VLM's internal reasoning into concrete visual snapshot. This act of mapping critical cognitive steps back into the visual modality provides tangible "cognitive anchor" for the VLM. Effectively, it endows the model with a form of image imagination, allowing it to perceive, critique, and refine its own understanding by confronting the visual consequences of its own textual descrip- tions. Specifically, we selected the Qwen2.5-VL-77 (Bai et al.] 2025) model for the VLM in the framework, maintaining consistency with the trained baseline, while the diffusion models in two stages are implemented using Qwen-Image (Wu et al. (2025a) and OmniGen2 (Wu et al. 2025b), respectively. In line with the closed-loop design, the VLM within the data synthesis module is dy- namically updated from training checkpoints, enabling a co-evolution of the model and its training data. Based on our framework. we constructed a 10K perception crreetiioninteenivee vision-land languge dataset ViperI0K, including 7K Caption Self- Self-Refining data and 3K Visual- Operation P Predicting data. The detailed implementation and instructions for the dataset construction are thoroughly presented the Appendix A to ensure reproducibility and to facilitate future research within the community.
3.2.2 TWO-STAGE REINFOR LEARNING
To align with the progressive cognitive demands of the two-stage task, we designed a phased rein- forcement learning approach, shown in the lower part of Figure []] Since all training data is self- synthesized by the model undergoing training, distribution shift from heterogeneous data sources eliminated. This self-sourcing strategy renders the entire RL process free from any cold-start re- quirement. The training proceeds sequentially: the first phase utilizes the Caption Self-Refining data, followed by the second phase focused on the Visual- SxuualOperatton Predicting task. unified reward computation mechanism is applied consistently across both stages.
Reward mechanism: For each model output O, we split it into a set of text sequences and use the BGE-M3 model (Chen et al., 2024a) to calculate the semantic similarity matrix between these sequences and the ground truth. The reward is applied as follows:
{
if O matches the expected format
Riormat
(5)
otherwise
5

---

Published as a conference paper at ICLR 2026
Σn
[max]= (im 93) 2 27]
x (Zin (1 maxx== sim 9:) L(si))
Reorrest
N
Σ ((s:)
(6)
(7) is the set of text se- quences obtained by splitting the output O, L(s;) is the character length of the split sentence si sim(s;, 95) is the semantic similarity between the sentence s; and the ground truth 9j. and is the similarity threshold, set to 0.85. The weight coefficients wy and we are set as wy 0.05 and w. 0.95.
R R= wy Riormat + We Recrrect.
Here, = (91, 92............ }M is the ground truth set, S = ( SN
Optimization process: We adopt a variation of Group Relative Policy Optimization (GRPO) al- gorithm, which has been proven effective in tasks involving mathematically structured reasoning chains (Ren et al., 2025). Following DAPO, we decouple the lower and higher clipping range to promote the diversity of the system well as avoiding entropy collapse, and remove the KL penalty term to to encourage bolder policy updates. Specifically, for each input question q. we first sample set of outputs for, 02...., ... ). and then apply the reward function in Equation 7 to compute the reward score for each output. The policy model is optimized by maximizing the following objective:
I(0) = Elg ~ P(Q), (o:) Tean(Olo)
1
min (t), clip (r), 1 Glow + Chigh) Are/3
(8)
G
los
where,
(0) = To(or, 0r.e)))
Ar, R, mean (R))..)
(9)
Teaa (0i. 10 0i,2))
std (d,((R))
EXPERIMENTS
4.1 EXPERIMENTAL SETUP
We selected Qwen2.5-VL-33 VLL-B and Qwen2.5-VL-7B as base models and performed two-stage RL training on Viper10K, resulting in Qwen Viper series. Identical hyperparameters were used for both stages: a batch size of 128, rollouts per prompt at temperature 1.0, and rewards computed by the BGE-M3 model following the Equation More details are presented in Appendix A.1
4.2
MAIN RESULTS
4.2.1
BENCHMARKS
In our work, we use seven multimodal benchmarks for evaluation: (1) MMStar (Chen et al. 2024b) emphasizes visual dependency, ensuring that answer inference requires visual grounding while min- imizing potential data leakage. (2) RealWorIdQA (X.AI 2024) focuses image understanding and verifiable reasoning in real-world scenarios. (3) MME-RW(en) (Zhang et al.) 2025e} is the English subset of MME-RW, improving complexity for real-world applications.
previous benchmarks in scale, resolution, and task (4) BLINK (Fu et al., 2024) examines core visual per- ception abilities such as depth estimation, visual correspondence, and multi-view reasoning. (5)
MANTIS Eval (Jiang et al. (024)) is designed to assesse multi-image reasoning, including refer- ence resolution, comparison, and temporal understanding. (6) HallusionBench (Guan et al. 2023)
evaluates model robustness against visual and language hallucinations. (7) CRPÉ (relation) (Wang jet al., 2024b) evaluates models on subject-peedicate--he entities and their relations. The staties of each benchmark are detailed in the Appendix B
structures, requiring recognition both
4.2.2 EXPERIMENTAL RESULTS
We evaluated our method on seven diverse lisnunnlanguage benchmarks encompassing single-image multi-image, and hallucination tasks. The selected benchmarks include both general and reality

---

Published as a conference paper at ICLR 2026
Table I: Experimental results multiple benchmarks emcompassing single-image, multi-image and hallucination tasks, A 1 denotes the absolute gain of ViPER over the base model.
Model
MMStar
RealWoridQA
MME-RW
BLINK
Mantis
Hallusion
CRPE (relation)
Overall
(en)
(val)
Bench(Avg)
GPT Huust
75.4
LLaVA-OneWision-..555 Li 2024
37.7
55.6
IntemVL22551 20240
50.1
57.5
44.2
42.0
60.9
49.3
IntemVL.3. IB
IntemW1.2.5-2i(Chenn
M124g
Inlem 2hu
64.3
Qwen2.5.VL:3B
2025
65.4
Owen: Viper
2.3
1.5
0.7
MiniCPM-VZL V2.6 Yao 2024
65.0
LLaVA-One
2024
Intem e M024
IntemVL3-88
76.3
Qwen2.5-VL-7Bi Bai
2025
68.5
+Viper1OK (SFT)
64.5
68.2
57.9
56.8
75.6
76.1
64.6
Qwen-Viper-7B
66.2
71.4
59.0
57.6
77.6
A
2.9
Table 2: Performances across different domains. A denotes the absolute gain of ViPER over the
base model.
Coarse Perception
Fine-grained Perception
Instance Reasoning
Logic Reasoning
Science Technology
Model
Math
Qwen2.5-VL-33
68.8
48.4
62.4
56.4
62.0
37.2
Qwen- Qwen-Viper-3B
70.0
64.4
39.2
2.0
Qwen2,5-VL. LL-77B
73.6
55.6
73.2
69.6
66.8
44.4
Owen-Viper-7B
75.2
61.6
74.8
70.4
67.2
48.0
1.6
3.6
perception-oriented visual question answering (VQA) tasks, aligning with our method's focus. As shown in Table []] ViPER yielded average performance gains of 1.7% and 1.6% across all bench- marks for the 3B and 7B models, respectively.
To analyze the specific capability improvements, we conducted fine-grained evaluations across six subdomains, and results shown in Table 2 The improvements were most pronounced in Fine- grained Perception, where Qwen- Vinen-Vipe--BBBB and Qwen- Viper-7B achieved gains of 4.4% and 6.0% respectively, underscoring ViPER's primary effect on detailed visual understanding(e.gg object lo- cation/counting, attribute recognition). Stable gains were also observed in Coarse Perception and In- stance Reasoning, confirming broader perceptual strengthening. Notably, despite direct training on knowledge tasks, a significant 3.6% average improvement emerged in the Science & Technology domain for the 7B model. This compeliiggy suggests that enhanced perception enables more accu- rate integration of visual cues with parametric knowledge, demonstrating that model capabilities are deeply intertwined rather than modular.
Results on hallucination benchmarks further support this observation. Compared to the base models, Qwen- Viper exhibited lower hallucination rates, indicating that enhanced visual perception enables more faithful processing of image information and partially mitigates the negative impact of linguis- tic priors. Furthermore, performance gains on multi-imin multi-image benchmarks confirm that the perception enhancements are fundamental and generalizable to out-of-domain tasks, enabling more effective understanding and reasoning across complex visual contexts.
"The data of all subdomains from MMStar.

---

Published as a conference paper at ICLR 2026
4.3
IN-DEPTH ANALYSIS
4.3.1
INFLUENCE OF COLD-START
Conventional RL methods often rely on high-quality cold-start data (Ren et al. 2025 Lee et al., 2024 Yu et al. 2025b). Integrating cold-start phase with RL training typically leads to more stable and significant improvements compared to direct RL fine-tuning. However, owing to the self- bootstrapping nature of the ViPER method, the data used for RL training is entirely self-generated. thereby eliminating the distribution discrepancy typically introduced by external models.
We conducted two experimental setups: three-stage supervised SFT RL process that includes a cold-start phase and a direct two- stage RL process without cold-start. In the cold-start stage, we used the Gemini-2.5-Pro model generate IK chain-of-thought data samples containing caption refinement and vi- sual operation prediction annotations, which were then used to SFT of Qwen2.5-VL.
As shown in Figure 2 although the reward curve of the training process without cold-start began significantly lower level compared to that with cold-start, its reward growth trend Figure 2: Reward curves with/without cold-start.
caught up with and surpassed the latter after 300 steps, eventually converging to a marginally higher final reward. The experimental phenomenon indicates that incorporating a cold-start phase in ViPER does not enhance model performance and may even constrain its potential for exploration and self- evolution. Thus, the proposed ViPER method effectively eliminates the dependency on high-quality cold-start data.
4.3.2 TWO-STAGE RL VS. MIXED RL
We further analyze the advantages of the two-stage RL strategy. common practice in reinforce- ment learning is to mix data from different types of tasks during training, typically to prevent the model from overfitting to specific task and suffering from performance degradation on others.
Our two-stage task follows a coarse-to-fing progres- sive process: the Caption Self- Refining stage focuses on global image information processing and empha- sizes the understanding of static scenes, while the Visual-Operation Predicting stage targets more fine- grained visual cues and highlights the comprehen- sion of dynamic changes. Therefore, multi-stage RL approach that incrementally enhances the model presents intuitively sound training paradigm.
Cyee Aipnn
29.3
To evaluate the training strategy, we compared our two-stage RL approach against a mixed RL process, where data from both stages were randomly blended during training on the Viper1OK dataset. As illus- trated in Figure 3 the two-stage RL strategy sig- nificantly outperforms the mixed training approach, demonstrating the necessity of the proposed phased training strategy.
Figure 3: Improvements of the two-stage RL vs. the mixed RL across six domains. The two-stage strategy outperforms the mixed RL.
To further investigate the effects before and after the two-stage training, we conducted more in- depth analysis. For the Caption Self-Refining stage, we intriguingly observed the emergence of visual thinking during training. We visualized the word cloud of the chain-of-thought tokens after training. as shown in Figure 4a The trained model spontaneously produced high-frequency opera- tion verbs such "scan" "zoom in" "look closely at" and "focus on", demonstrating "thinking- with-imagen reasoning pattern. For the Visual- Operation Predicting stage, we tracked changes in the model's 's attention distribution for identical visual input before and after training analyze the
8

---

Published as a conference paper at ICLR 2026
DOD
mitted
Iocated
ta
stated
S
resting
Cleal
doesn
desc
filled
oiig
Look C ose Ty
D
2008
innted
so
Lookingg
Look
MLS
sed
starring wi
ncans
Laen
nnnni ittffee
les
lusin
can
Lilling
sittinge
o
shaped
points
missiing
moving
compairing
tar
misses
Look closely
omi Ltee
rect
t Leareads
Eranning
ns
eads
aate mi Asiienntiii fiede matches
(a) Word cloud of chain-of- tokens from the RL training. Verbs indicating visual operations are highlighted with col- ored boxes.
(b) Attention heatmaps for the set of input im- ages. The top two pancls show the attention heatmaps from Qwen2.5-VL VL-7B, while the bottom corresponds to those from Qwen- Viper-
model after Caption
Figure Visualization of word cloud and attention.
impact of this training stage through the lens of attention mechanisms. The results are presented in Figure 4b] The model after the second stage showed significantly increased concentration on critical information, effectively translating the image operations described in the first-stage reasoning chains into shifts in attention patterns. This shift enables the model to spontaneously focus on and perform detailed analysis of local regions.
4.3.3 ABLATION STUDY
To validate the necessity of each stage, we conducted ablation studies. Specifically, we per- formed RI training solely with the Caption Self- Seff-Refining stage and separately with only the Visual- Operation Predicting stage, then compared their results against the fully trained Qwen- Viper model that underwent both stages. The detailed experimental outcomes are presented in Figure
The ablation results confirm that while each stage contributes uniquely, their integration is key to significant gains. Training solely on the Caption Self-Refining stage fosters founda- tional, global visual reasoning capability. By learning to generate and critique holistic de- scriptions, the model builds a robust scaffold for scene understanding, leading to balanced but moderate gains. In contrast, exclusive train- ing on the Visual-Open -oeeaa-Oprratinn Predicting drives localized perceptual sensibility, which forces the model to master fine- -nn-irdined attribute and relationship analysis, resulting in sharper im- provements on corresponding tasks. Crucially, the full framework's superiority stems from a coordinated mechanism: the first stage establishes reliable global context, which in turn primes and grounds the intensive local analysis of the sec- ond stage. This coherent progression from global to local enables a more profound and integrated self-evolution than either stage could achieve in isolation.
Ovind 541-35
Ownnd 2V4-334 CSA
Owand 5VL-2-51.333
Drmmettrs
Figure 5:
Ablation results Caption Self- Refining and Visual-Operation Predicting Tasks.
4.3.4 MECHANISMS FOR MITIGATING NOISES FROM GENERATIVE MODELS
ViPER's framework centrally leverages diffusion models, which are employed not only for caption- to-image generation but also for highly challenging fine-grained visual editing tasks. This reliance

---

Published as a conference paper at ICLR 2026
imposes stringent requirements on the precision of diffusion models and consequently establishes critical performance bottleneck. Current multimodal generative models often exhibit limitations in eestruction-followiin capabilities, particularly in fine-grained compositional generation tasks, which may lead to data collapse and compromise training stability. To address the potential data noise introduced by generative models, we implement dual-mechanism strategy:
First, for fine-grained editing tasks, we introduce VLMMass-udge to filter generated outputs that fail meet specified requirements. Implementation details and corresponding prompts are documented in Appendix A.4 Nevertheless, while VLM-as-Judge effectively ensures semantic fidelity, provides inadequate control over non non-semantic attributes such lighting conditions and textural patterns, As result, reconstructed images often inherit style transfer effects and textural artifacts characteristic of the generative model. To circumvent the impact of such generator-specífic artifacts, we strategically design the reward function in the reinforcement learning phase to neutralize their influence. Although the VLM may occasionally produce self-critical content containing hallucinatory refinements due to biases introduced during data construction, it will never generate the same hallucinatory content during training because the absence reconstruted images. recall based on semantic similarity. Consequently, these
The reward of RL focuses exclusively minimally hallucinated refinements remain unmatched by the model's s rollouts, thereby yielding zero advantage estimates and gradient updates. This design systematically addresses the impact of data noise on the VLM's training process.
CONCLUSION
To break through the bottleneck of VLMs in perception-intensive tasks, we introduce self- evolutionary framework ViPER, which establishes closed-loop cycle of data construction and reinforcement fine-tuning. ViPER is built around novel two-stage perceptual task that leverages image-level reconstruction and instance-level reconstruction to bootstrap visual perception and com- prehension. This framework addresses the scarcity of high-quality perceptual data by enabling the model autonomously generate its own training samples, while the subsequent post-training phase continuously refines the model's capabilities using the self-synthesized data. In this way, ViPER facilitates VLMs' self-evolutionary enhancement of visual perception, moving beyond the limita- tions of text-centric reasoning approaches. Extensive experiments demonstrate the effectiveness of ViPER and provide insights into its self-consistent design, strongly validating the role of generation in advancing perceptual understanding.
10

---

Published as a conference paper at ICLR 2026
REFERENCES
Anthropic The claude 3 model
family:
Opus.
sonnet, haiku.
Technical re-
port, Anthropic PBC. 2024.
URL
https:/ /ww. cdn. anthropic com/
de$ba9bo1ccanl
Cawudeaaadde_aaa3______ Claude p. pdf
Model Card.
Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, et al. Qwen2. 5-vl technical report. arXiv preprint urXiv:2502.13922 2025.
Davide Caffagni et al. The revolution of multimodal large language models. In Findings of the Asso- ciation for Computational Linguistics: ACL 20241, 2024. URL https::/accanthhlll
/rgg/024.. Findings-acll.. 8077
Yuqi Cao and Minghao Ou Ame: Aligned manifold entropy for robust vision-language distillation. arXiv preprint arXiv:2508.0866 2025. URL BBll:ntpes///axxt..orr////b///// 08644
Jianlyu Chen, Shitao Xiao, Peitian Zhang, Kun Luo, Defu Lian, and Zheng Liu. M3-embedding: Multi-linguality, multi-functionaaliy, multi-gra -uati-granullrity text embeddings through self-knowledge distillation. In Findings of the Association for Computational Linguistics ACL 2024, pp. 2318- 2335, 2024a.
Lin Chen, Jinsong Li, Xiaoyi Dong, Pan Zhang, Yuhang Zang. Zehui Chen, Haodong Duan, Jiaqi Wang, Yu Qiao, Dahua Lin, and Feng Zhao. Are we on the right way for evaluating large vision-language models? In A. Globerson, L. Mackey, D. Bel- grave, A. Fan. U. Paquet, J. Tomezak, and C. Zhang (eds.), Advances in Neural In formation Processing Systems. volume 37, pp. 27056-27087. Curran Associates. Inc., 2024b. URL 22le/2f8ee6a33 66l2447776422226 orr.r///66600ceee5555533eeeeeer0
https//eeccpeeeeeeseeaaaeereerr files/paper/20247
Zhe Chen, Weiyun Wang. Yue Cao, Yangzhou Liu, Zhangwei Gao, Erfei Cui, Jinguo Zhu, Shen- glong Ye, Hao Tian, Zhaoyang Liu, et al. Expanding performance boundaries of open-source multimodal models with model, data, and test-time scaling. arXiv preprint a1X05.241..0527,, 2024c.
Gheorghe Comanici, Eric Bieber, Mike Schaekermann, Ice Pasupat, Noveen Sachdeva, Inderjit Dhillon, Marcel Blistein, Ori Ram, Dan Zhang, Evan Rosen, et al. Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capa- bilities. arXiv preprini arXiv:2507.06266 2025.
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, al. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010. 1929, 2020.
Ahmed El-Kishky. Openai ol system card. ArXiv, abs/2412.16720,, 2024. //axxiv.rrg/ab//222.2. 16720
URL
https:
Xingyu Fu, Yushi Hu, Bangzheng Li, Yu Feng, Haoyu Wang, Xudong Lin, Dan Roth, Noah A. Smith, Wei-Chiu Ma, and Ranjay Krishna. Blink: Multimodal large language models can see but not perceive. ArXiv, abs/2404. 12390, 2024.
Zhi Gao et al. Multi-modal agent tuning: Building a vlm-driven agent for efficient tool usage. arXiv:2412.15606, 2024.
Tianrui Guan, Fuxiao Liu, Xiyang Wu, Ruiqi Xian, Zongxia Li, Xiaoyu Liu, Xijun Wang, Lichang Chen, Furong Huang, Yaser Yacoob, Dinesh Manocha, and Tianyi Zhou. Hallusionbench: An advanced diagnostic suite for entangled language hallucination and visual illusion in large vision- language models. 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). pp. 14375-14385, 2023.
11

---

Published as a conference paper at ICLR 2026
Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-rl: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.122453 2025.
Yujie Hou al. Vision-language model knowledge distillation method for iqa. arXiv preprint arXiv:2507. 15680, 2025.
15680
Yuxin Hu et al. Visual sketchpad: Sketching visual chain of thought for multimodal Ims. NewrlPS, 2024.
Wenxuan Huang, Bohan Jia, Zijie Zhai, Shaosheng Cao. Zheyu Ye, Fei Zhao, Zhe Xu, Yao Hu, and Shaohui Lin. Vision-rl: Incentivizing reasoning capability in multimodal large language models. arXiv preprint arXiv:2503.06749) 2025a.
Zeyi Huang, Yuyang Ji, Anirudh Sundara Rajan, Zefan Cai, Wen Xiao, Haohan Wang, Junjie Hu, and Yong Jae Lee. Visualtoolagent (vista): A reinforcement learning framework for visual tool selection. arXiv preprint arXiv:2505. 20289, 2025b.
Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Os- trow, Akila Welihinda, Alan Hayes, Alec Radford, et al. Gpt-4o system card. arXiv preprint arXiv:2410.21276. 2024.
Dongfu Jiang, Xuan He, Huaye Zeng, Cong Wei, Max W.F. Ku, Qian Liu, and Wenhu Chen. Mantis: Interleaved multi-image instruction tuning. Trans. Mach. Learn. Res., 2024, 2024. URL https
//openzeview. wwt. wort:/foummaaa dUVaJa
Xin Lai, Junyi Li, Wei Li, Tao Liu, Tianjian Li, and Hengshuang Zhao. Mini-o3: Scaling up reasoning patterns and interaction turns for visual search. arXiv preprint arXiv:250900796) 2025.
Hyung Won Lee al. Rlaif: Scaling reinforcement learning from human feedback with ai feedback. In ICLR 2024, 2024. URL httLtttts///peezeeeeee
Bo Li, Yuanhan Zhang, Dong Guo, Renrui Zhang, Feng Li, Hao Zhang, Kaichen Zhang, Peiyuan Zhang, Yanwei Li, Ziwei Liu, et al. Llava-onevision: Easy visual task transfer. arXiv preprint WwXiv:2408.03322 2024.
Yunxin Li, Zhenyu I. Liu, Zitao Li, Xuanyu Zhang, Zhenran Xu, Xinyu Chen, Haoyuan Shi, Shenyuan Jiang, Xintong Wang, Jifang Wang, et al. Perception, reason, think, and plan: survey on large multimodal reasoning models. arXiv preprint 25t.::::22000022222 2025.
Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, et al. Deepseek-v3 technical report. arXiv preprint arXiv:2412.19437 2024a.
Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. 36:34892- 34916. 2023. URL htt.:s/s/pppoeeeennnnneeeoososos www.:eiii.......u............. 2023/ fi 1e/ 6dc f2
.dff
Shihong Liu, Samuel Yu, Zhiqiu Lin, Deepak Pathak, and Deva Ramanan. Language models as black-box optimizers for vision-language models. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition, pp. 12687-12697, 2024b.
Haoyu Lu, Wen Liu, Bo Zhang, Bingxuan Wang, Kai Dong, Bo Liu, Jingxiang Sun, Tongzheng Ren, Zhuoshu Li, Hao Yang. Yaofeng Sun, Chengqi Deng, Hanwei Xu, Zhenda Xie, and Chong Ruan. Deepscek-vl: Towards real-world lingon-language understanding, 2024. URL https:
Qiao Lyu et al. Towards faithful model explanation in nlp: A survey. Computational Linguistics, owg. 2C24
2024. URL httLtttts///caatt..att
pdd
OpenAl Openai o3 and o4-mini system card. Technical report, OpenAL, April 2025. URL 000-00ddc-789 - 77775-8800--777583222222
03-and-o4-mini-sye o System Card.
12

---

Published as a conference paper at ICLR 2026
Rafael Rafailov, Archit Sharma, Eric Mitchell, Stefano Ermon, Christopher D. Manning, and
Chelsea Finn. Direct preference optimization: Your language model secretly a reward model. arXiv preprint arXiv:2305./8290, 2023. URLhtts::
wwwpsiiiivrrrrooooooo abs 2305 18290
Zehui Ren et al. Deepseck-rl: Incentivizing reasoning capability in Ilms via reinforcement learning. Nature, 2025, doi: 10. 101033//41386--05549422222
John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv preprint arXiv:/707.0n0 1707.06347, 2017.
Hao Shao et al. Visual cot: Advancing multi- language models with step-by-step visual rea- soning, arXiv:2403. 16999, 2024.
Haozhan Shen, Peng Liu, Jingcheng Li, Chunxin Fang, Yibo Ma, Jiajia Liao, Qiaoli Shen, Zilun Zhang, Kangjia Zhao, Qianqian Zhang, et al. VIm-rl: A stable and generalizable rl-style large vision-language model. arXiv preprint arXiv:2504. 07615, 2025.
Kacper Stechly et al. Chain of thoughtlessness? analysis of cot in planning. In NeurlPS, 2024.
Alex Su, Haozhe Wang, Weiming Ren, Fangzhen Lin, and Wenhu Chen. Pixel reasoner: In- centivizing pixel-space reasoning with curiosity-driven reinforcement learning. arXiv preprint arXiv:2505. 15966 2025a.
Zhaochen Su, Linjie Li. Mingyang Song, Yunzhuo Hao, Zhengyuan Yang, Jun Zhang, Guanjie Chen, Jiawei Gu, Juntao Li, Xiaoye Qu, et al. Openthinkimg: Learning to think with images via visual tool reinforcement learning. arXiv preprint arXiv:2505.08617. 20255..
Kai Sun, Yushi Bai, Ji Qi, Lei Hou, and Juanzi Li. Mm-math: Advancing multimodal math eval- uation with process evaluation and ine-grained classification. arXiv preprint arXiv:2404.055: 05091, 2024.
Kimi Team, Angang Du, Bofei Gao, Bowei Xing, Changjiu Jiang, Cheng Chen, Cheng Li, Chenjun Xiao, Chenzhuang Du, Chonghua Liao, et al. Kimi kl. 5: Scaling reinforcement learning with Ilms. arXiv preprint arXiv:2501.12599 2025.
Vishaal Udandarao et al. Active data curation effectively distills large-scale multimodal models. In CVPR, 2024. URL http:https//axiiio org/ abs///as..222/ae/./ 18674
Haozhe Wang. Chao Qu, Zuming Huang, Wei Chu, Fangzhen Lin, and Wenhu Chen. VI- rethinker: Incentivizing self-reflection of vision-language models with reinforcement learning. arXiv preprint arXiv:2504. 20Xx::25004088877,, 2025a.
Peng Wang. Shuai Bai, Sinan Tan, Shijie Wang. Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, et al. Qwen2-vl: Enhancing lasounelanguage model's 's perception of the world at any resolution. arXiv preprint arXiv:2409. /2 1919, 2024a.
Weiyun Wang, Yiming Ren, Haowen Luo, Tiantong Li, Chenxiang Yan, Zhe Chen, Wenhai Wang. Qingyun Li, Lewei Lu, Xizhou Zhu, et al. The all-seeing project v2: Towards general relation comprehension of the open world. arXiv preprint arXiv:2402.1949 19474, 2024b.
Weiyun Wang, Zhangwei Gao, Lixin Gu, Hengjun Pu, Long Cui, Xingguang Wei, Zhaoyang Liu, Linglin Jing, Shenglong Ye, Jie Shao, et al. Internv13. 5: Advancing open-source multimodal models in versatility, reasoning, and efficiency, arXiv preprint arXiv:2508. 18265, 2025b.
Yiping Wang, Qing Yang. Zhiyuan Zeng, Liliang Ren, Liyuan Liu, Baolin Peng. Hao Cheng, Xuehai He, Kuan Wang, Jianfeng Gao, et al. Reinforcement learning for reasoning in large language models with one training example. arXiv preprint arXiv:2504.20571, 2025c.
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. Chain-of-thought prompting elicits reasoning in large language models. Advances in 2022.
neural information processing systems,
Chenfei Wu, Jiahao Li, Jingren Zhou, Junyang Lin, Kaiyuan Gao, Kun Yan, Sheng Sheng-ming Yin, Shuai Bai, Xiao Xu, Yilei Chen, et al. Qwen-image technical report. arXiv preprint arXiv:2508.022 023244 2025a.
13

---

Published as a conference paper at ICLR 2026
Chenyuan Wu, Pengfei Zheng, Ruiran Yan, Shitao Xiao, Xin Luo, Yueze Wang. Wanli Li, Xiyan Jiang. Yexin Liu, Junjie Zhou, et al. Omnigen2: Exploration to advanced multimodal generation. arXiv preprint wwXiv:250.188// 2025b.
X.AI. Realworidga. Blog post, November 2024. URL htttp:///x.al. gwwo// 1.5v
An Yang, Anfeng Li, Baosong Yang. Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al. Qwen3 technical report. arXiv preprini arXiv:2505.09388, 2025. bRtpss:/://axiii wwtp:iii.rrrraoooooo abs//ab//20/ 2505. 09388
Yuan Yao. Tianyu Yu, Ao Zhang, Chongyi Wang, Junbo Cui, Hongji Zhu, Tianchi Cai, Haoyu Li, Weilin Zhao, Zhihui He, et al. Minicpm-v: A gpt-4v level milm on your phone. arXiv preprint arXiv:2408. 01800, 2024.
Shukang Yin, Chaoyou Fu, Sirui Zhao, Ke Li, Xing Sun, Tong Xu, and Enhong Chen. A survey on multimodal large language models. National Science Review, 2024. URL https::/arxix:
brg/abs/ aww.aa/........ 2306. 13549
Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Weinan Dai, Tiantian Fan, Gaohong Liu, Lingjun Liu, al. Dapo: An open-source IIm reinforcement learning system at scale. arXiv preprint arXiv:2503. 14476, 2025a.
Tianyu Yu, Haoye Zhang, Qiming Li, Qixin Xu, Yuan Yao, Da Chen, Xiaoman Lu, Ganqu Cui, Yunkai Dang, Taiwen He, et al. Rlaif-v: Open-source ai feedback leads to super gpt-4v trust- worthiness. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 19995,19995, 2025b.
Di Zhang, Jingdi Lei, Junxian Li, Xunzhi Wang, Yujie Liu, Zonglin Yang. Jiatong Li, Weida Wang, Suorong Yang. Jianbo Wu, Peng Ye, Wanli Ouyang, and Dongzhan Zhou. Critic-v: Vlm crities help catch vlm errors in multimodal reasoning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 9050-9061 June 2025a.
Haoji Zhang, Xin Gu, Jiawen Li, Chixiang Ma, Sule Bai, Chubin Zhang, Bowen Zhang, Zhichao Zhou, Dongliang He, and Yansong Tang. Thinking with vidcos: Multimodal oull-uugeette reinforcement learning for long video reasoning. arXiv preprint arXiv:2508. 04410600004....... 2025b.
Xin Zhang et al. Hkd4vlm: A progressive hybrid knowledge distillation framework for robust mul- timodal hallucination and factuality detection vims. arXiv preprint arXiv:2506. / /3038, 2025c.
URL httthttps://aae //axxii o
13038
Xintong Zhang. Zhi Gao, Bofei Zhang, Pengxiang Li, Xiaowen Zhang, Yang Liu, Tao Yuan, Yuwei Wu, Yunde Jia, Song-Chun Zhu, et al. chain-of-focuss Adaptive visual search and zooming for multimodal reasoning via rl. arXiv preprint arXiv:2505. 15436, 2025d.
YiFan Zhang, Huanyu Zhang, Haochen Tian, Chaoyou Fu, Shuangqing Zhang, Junfei Wu, Feng Li, Kun Wang, Qingsong Wen, Zhang Zhang, Liang Wang, and Rong Jin. MME-realworld: Could your multimodal LLM challenge high-resolution real-world scenarios that are difficult for humans? In The Thirleenth International Conference on Learning Representations, 2025e. URI wpw.eeeeeveveeeeeeeeee htttps://ppenrev ew net wow.ooommmoooaageeeoooooo
https: /
Chujie Zheng. Shixuan Liu, Mingze Li, Xiong-Hui Chen, Bowen Yu, Chang Gao, Kai Dang, Yuqiong Liu, Rui Men, An Yang, Jingren Zhou, and Junyang Lin. Group sequence policy op- timization. 2025a. URL http:/tttt///ate http://wwwwwrr////......... 18071
Ziwei Zheng, Michael Yang, Jack Hong, Chenxiao Zhao, Guohai Xu, Le Yang, Chao Shen, and Xing Yu. Deepeyes: Incentivizing" thinking with images" via reinforcement learning. arXiv preprint arXiv:2505.14362. 2025b.
Qiji Zhou, Ruochen Zhou, Zike Hu, Panzhong Lu, Siyang Gao, and Yue Zhang. Smage-of-thoure prompting for visual reasoning refinement in multimodal large language models. arXiv preprint ar0iv:2405. 13872, 2024.
Jinguo Zhu, Weiyun Wang. Zhe Chen, Zhaoyang Liu, Shenglong Ye, Lixin Gu, Hao Tian, Yuchen Duan, Weijie Su, Jie Shao, et al. Internv1:: Exploring advanced training and test-time recipes for open-source multimodal models. arXiv preprint urXiv:2504.10479 2025.
14

---

Published as a conference paper at ICLR 2026
A IMPLEMENTATION DETAILS
A.1 EXPERIMENTAL SETUP
We implemented the two-stage reinforcement learning based on the verl 0.3.1 The entire training process was conducted on a single node equipped with eight A100 80GB GPUs, leveraging Fully Sharded Data Parallel (FSDP) for efficient large-scale training. Key hyperparameters used in the reinforcement learning process are summarized Table
Table 3: Key experimental settings for the RL process
Module
Parameter
Value
batch.size
128
data
max.prompt.length
10240
max.response.eenthh
4096
optim.lr
10-6
siih
64
actor
micro batch size per gpu
4
k1 loss
False
temperature
1.0
sampling
rollout.n
5
num-gpus
8
hardware
memory-per.gpu
80 GB
A.2 HEURISTIC RULES OF VISUAL OPERATIONS
For the data construction process of the visual-operation predicting task, we designed heuristic rules for the VI.M. First, predefined four visual operation tasks toolkits, specifying their applica- ble scenarios and rules. These correspond to four common error types in fine-grained perception scenarios for VLMs: 1. Omission of small-scale details; Confusion of spatial relationships; 3. Overemphasis on primary subjects while neglecting contextual backgrounds; 4. Incorrect fine- grained attribute discrimination;
The model selects the most suitable editing task upon thorough and meticulous analysis of the input image:
Prompt for Heuristic Rules of Visual Operations
Role and Core Principle
You are a creative and insightful visual analyst. Your task is to analyze the provided image and determine the most suitable category subtle edit for creating challenging VQA (Visual Question Answering) data.
The edit category you choose must the most suitable to content the image, and support fine-grained changes that require careful observation to notice.
Task Description
First, conduct a thorough analysis of the provided image. Describe all the details. Next, refer the "Editing Toolkit" below.
Based on your analysis, select the most suitable task category from the toolkit.
Finally, articulate your reasoning for this choice, explaining why this category superior to others for creating a subtle and effective edit on this specific image.
Editing Toolkit
15

---

Published as a conference paper at ICLR 2026
Removing/Adding Details: If the image contains many elements or objects with complex ingredients, consider removing or adding one of the small-scale objects.
Changing Spatial Relationships: an entity has a well-defined spatial relation to its surroundings and can be feasibly repositioned, then alter its location to disrupt and reconstruct the spatial context.
Editing Background: If the image is clearly separated into foreground and back- ground, and the background is rich with elements, consider making edits to ob- ject there.
If the image contains an object with a property that is easy to change, and the change would be subtle enough to require close inspection to notice, consider editing one of the object's properties.
Attribute Tuning:
Output Format
Your output MUST follow the ISON structure below. Do not include any other commentary or introductory text.
Image Analysis:
(Your detailed analysis of the image in natural language.]
Edit Task: [The name of the task category you chose from the toolkit.] Reasoning: [Your detailed justification for why this task category is the most suitable for this image, explaining how it enables subtle and effective edit and why it is superior to other options.]
For the given visual operation tasks, we provide advanced and detailed task descriptions, and require the model to generate precise visual-operation instructions that are optimally tailored to the image content.
A.3 PROMPT IN DUAL-LEVEL RECONSTRUCTION
We provide the key prompts used for dual-level reconstruction during the data synthesis process below to ensure full reproducibility of the pipeline:
Prompt for Caption Self-Refining
You are an expert AI assistant specializing in high-fidelit image caption evaluation. Your task is analyze original image, its caption, and reconstructed image that generated based only on that caption. Your goal is to identify how the caption fails to accu- rately and completely describe the original image.
The core of your analysis is to compare the original image with the reconstructed image. The differences between these images are a direct result of inaccuracies or omissions in the caption. For every significant difference you find, you must describe the corresponding flaw in the caption.
Analysis Criteria:
Analyze the caption's flaws based on the visual discrepancies between the original and constructed images:
Omitted Details: Does the reconstructed image lack important objects, elements, or background details that are present in the original image? This indicates the caption omitted these details.
Misidentified Entities: Are objects, people, scenes in the reconstructed image fundamentally different from the original? (e.g.. dog instead of a cat, city instead of forest). This indicates the caption misidentified something.
Inaccurate Spatial Relationships: Is the position arrangement of elements in the reconstructed image incorrect compared to the original? This indicates the caption described spatial relationships inaccurately.
16

---

Published as a conference paper at ICLR 2026
Unmentioned or Incorrect Text: Is there text visible in the original image that is missing or garbled in the reconstructed image? This indicates the caption either omitted or incorrectly transcribed the
Input:
Original Image:
The ground-truth image
Reconstructed Image: The image generated from the caption
Current Caption: The text caption being evaluated
Output Requirements:
Your response must single JSON object. This object must contain single key, "refinemett", which is list of strings. Each string in the list must be a concise and precise description of single factual error or significant omission you identified in the caption.
Focus exclusively on factual errors and omissions that cause visual discrepancies.
Crucially, do not describe the differences between the images in your output. In- stead, describe the flaw in the caption that caused the difference.
Do NOT comment writing style, phrasing, tone, or subjective quality.
If the caption is perfectly accurate and the reconstructed image is faithful repre- sentation of the original, return a JSON object with an empty list: "refinement": [].
Prompt for Generating Refined Captions
Role
You are an extremely meticulous and precise text editor.
Task
Your sole task to revise an "Original Caption" based on list of "Refinement Sugges- tions". Your goal is to produce fluent, coherent "Refined Caption" that fully incorporates all suggestions.
Rules
Must Integrate All Suggestions: You must accurately reflect every point from the "Refinement Suggestions" in the new caption.
No New Information: You are strictly forbidden from adding any new details that not present in either the original caption the suggestions.
Preserve Correct Information: All accurate information from the original caption that is not targeted by suggestion must be retained.
Single Text Output: The final result must be single, flowing descriptive para- graph, not list of changes.
Strict Output Format: Your response must contain only the final refined caption text. Do not include any preambles (c.g., "Here is the refined caption:"), explana- tions, or Markdown formatting.
Input Data
Original Caption: (original.caption)
Refinement Suggestions: (suggestions str)
Please directly output the refined caption:
17

---

Published as a conference paper at ICLR 2026
Prompt for Generating Visual Editing Instructions
You are a professional image editing instruction generator. Follow these steps:
1. First, carefully observe the image and identify all entities and elements in the image.
2. Analyze the cognitive difficulty of each entity (considering complexity, abstraction level, recognition difficulty, etc.).
Determine the entity with the highest cognitive difficulty and suitable to edit.
4. Based the given task description and specific image content, generate targeted editing instruction.
Return JSON format with these fields:
"entity to edit": the entity chosen to be remove
ioaiting.instructioon" Comprehensive editing instruction (see requirements below) Requirements for editing instructions:
Must be concise, clear, and directly usable as prompt for image editing/generation models
Must comply with the given task description
Avoid lengthy explanations, focus direct editing commands
Task Description:
[task.description)
Please analyze this image and generate corresponding editing instructions.
A.4 DATA QUALITY ASSURANCE
Precise and diverse visual editing instructions pose significant challenges for current open-source image editing models, which are prone uncontrollable errors fine-grained editing tasks. To address this, we have introduced dual-validation mechanism combining CLIP scores and LLM-as- a-judge. The CLIP score serves an initial filter to eliminate editing results that deviate significantly from the original image, while the LLM-as-a-judde employs stricter validation rules to ensure that the final edits strictly adhere the instructions while maintaining high consistency with the original image.
Prompt for Checking Instance-Level Reconstruction
Role
You are an image validation specialist, and you specialize in rigorously assessing edited images ensure they precisely adhere the given instructions.
The original image {
The edited image [image2)
Please evaluate the edited image compared to the original image, focusing only on semantic changes.
Editing instruction {edit instruction)
What to evaluate:
18

---

Published as a conference paper at ICLR 2026
Does the edit accurately reflect the user's instruction in terms of content and objects?
Strict Adherence:
No Unintended Semantic Changes: Are there any new objects, removed objects (that were not supposed to be removed), changes in the attributes of existing objects that were not part of the instruction?
What to ignore:
Minor, global variations in brightness, contrast, color saturation. Consider these acceptable side effects unless the instruction was specifically about changing them.
Decision:
the edit is semantically accurate and clean, set "is_valid" to true.
If the edit fails to follow the instruction or introduces unintended semantic changes, set "is_valid" to false and provide brief explanation in the "reason" field.
Please return your judgment only in JSON format as follows: vis_vd boolean, "reason": "A brief explanation if is_ valid is false, otherwise this field can be omitted.") }
19

---

Published as a conference paper at ICLR 2026
B BENCHMARK DETAILS
To provide a clearer understanding of the datasets used in our experiments, we present more detailed descriptions of the seven multimodal benchmarks:
(1) MMStar (Chen et al. 2024b) is designed to address the limitations of weak visual dependency and unintentional data leakage in previous benchmarks. It contains 1,500 human-curated samples that require genuine visual reasoning and are resistant to memorization effects. The benchmark spans six core capabilities and 18 fine-grained axes, ensuring a balanced and rigorous evaluation of large VLMS.
(2) RealWAR X.AI
provides more than 700 images from real-world scenarios, includ- ing those captured from vehicles and other everyday contexts. Each image is paired with a question and verifiable answer, making it concise yet practical dataset to assess real-world grounding and reasoning in multimodal models.
2024)
(3) MME-RW (Zhang et al. 2025e) currently one of the largest manually annotated multimodal benchmarks, comprising tens of thousands of high-quality images collected from public datasets and the Internet. Expert annotators generated diverse question-answer pairs covering challenging real-world subtasks. Compared with previous datasets, it features higher-resolution images and significantly greater task diversity. MME-RW (en) denotes its English subset, where all questions and answers are provided in English.
(4) BLINK (Fu et al. 2024) reformulates 14 classic computer vision tasks into 3,807 multiple- choice questions, paired with single multiple images. 'he benchmark assesses core perceptual abilities, including relative depth estimation, visual correspondence, and multi-view reasoning.
(5) Mantis Eval (Jiang et al. 2024) is benchmark specifically designed to evaluate multi-image reasoning in multi-modal models. It consists of carefully curated test cases that require reasoning across multiple images, including tasks such reference resolution, cross-image comparison, and temporal understanding. By focusing on multi-image contexts, the benchmark provides a systematic way to assess whether models can integrate information beyond single-image perception.
(6) HallusionBench (Guan et al. 2023) is built to evaluate multimodal reasoning under intluccinationinndeiin conditions. It contains 346 images paired with 1,129 expert-designed ques- tions, many organized into control structures that allow fine-grained analysis of logical consistency and error types.
(7) CRPE (Wang et al. 2024b) provides structured evaluation of object recognition and relation comprehension. The benchmark consists of single-choice questions divided into four subsets: Ex- istence, Subject, Predicate, and Object. This division enables systematic testing of whether models can detect the presence of objects, identify entities, and capture subject- proectt-rrdiccte-ojjee relations, offering insight into fine-grained relational reasoning abilities. In experiments, we specifically adopt the relation subset to evaluate the subject- prspeett-prddicte-oojeeee reasoning.
20

---

Published as a conference paper at ICLR 2026
C
DATA SAMPLE
Caption Self-Refining
Original Image & Reconstructed Image
Original Caption
decadent dessert featuring a round cake placed at the center of a wooden serving tray, embellished with intricate layers and vibrant toppings. The cake is covered in rich, creamy frosting with a pale yellow hue displaying a rustic, textured finish reminiscent of hand-spread icing. Embedded along the sides slices of ripe pink fruit, possibly figs, their interior pulp and outer skin visible through the frosting. Atop the cake sits an eye-catching glossy orange segment, likely a fruit compote or poached persimmon, crowned by small edible flower decorations with earthy green and burgundy tones. Surrounding the centerpiece is a white whipped cream spiral arranged in flower-like shape, with petite, oval-shaped decorative plaque reading "Muletokyo" nestled among the toppings. To the side of the cake, sitting on the same wooden tray, clear glass filled with amber-colored liquid, potentially tea juice, offering a refreshing pairing to the indulgent treat. The background reveals an outdoor setting with hints of vegetation, characterized by thin branches and clusters of leaves, alongside dark glass panel that reflects faint shapes and shadows.
Refinement
The cake is placed on a square, gold-colored cake board, which is not mentioned in the caption.
The 'small edible flower decorations' described on top of the orange fruit are the fruit's natural calyx, not separate decorations.
The decorative plaque on the cake reads 'Myuke Tokyo', not 'Muletokyo',
A slice of red fruit, possibly raspberry, is visible the whipped cream next to the persimmon, but is not mentioned.
The cake positioned on the left side of the wooden tray, not the center.
21

---

Published as a conference paper at ICLR 2026
Visual-Operation Predicting
Original Image Reconstructed Image
M vin
DI
Divin
Ol vin
ONI
Divin
Oik cr Din
Dhr Cr Uin
Wuin
Diii
G Divin
Visual-Operation Instructions
Add small chopped green scallions scattered on top of the sesame-covered meat pastries and on the paper lining in the basket.
22

---

Published as a conference paper at ICLR 2026
D CASE STUDY
In this section, present concrete cases to visually demonstrate the significant improvements achieved by our method. For each example, we provide responses from three models: Gemini-2.5- Pro (by API, Qwen2.5-VL-78 and Qwen- Viper-BB. For the latter two models, visualize their attention distributions during the response generation process, using attention heatmaps as a lens to observe the operational mechanism of the ViPER method.
Case Study of Visual Counting
Question: How many garages does the first building on our right have?
Gemini-2.5-pro
Determining Garage Count
I'm zeroing in on the user's query: specifically, how many garages are visible on the first structure to the right. My focus is thoroughly examining the image to isolate and enumerate them.
Focusing On Specifics
Now, I'm focusing on the specifics. I've pinpointed the correct structure and need to accurately count the visible garages. Counting them carefully, I've identified five distinct garage doors on the ground floor. My next step is to formulate the answer concisely and confidently, incorporating brief descriptive confirmation for clarity.
Refining The Response
Now, I'm taking more analytical approach. I've broken down the user's question into distinct steps: identifying the building, counting the garages, and crafting clear answer. I've confirmed that the target building the most prominent on the right with five garage doors. I' I'm focusing ensuring my response is both direct and detailed to instill trust. Based on the image, the first building on the right has five garages.
23

---

Published as a conference paper at ICLR 2026
Qwen2.5-VL-7B
The first building the right side of the image has four garage doors. These clearly visible at the bottom of the building, just above the street level.
Qwen-Viper-7BCC
I'll start by examining the image, focusing on the structure on the right side of the street. The image is view down a residential street with parked cars and buildings on both sides. My goal is to determine how many garages the first building the right has.
I'm looking at this building, which appears to be a multi-story structure, likely an apartment complex similar type of building. It's light grayish- brown color. As my eyes its facade, see what looks like a row white doors. These are clearly visible and are located directly beneath to be identical in size and shape.
of windows the ground floor. They evenly spaced and appear
Next, I'll do a quick visual count of these doors. Starting from the left, I see one, two, three, four, five, six, seven, and cight of them. They are all aligned in straight line, making simple count. This building on the right has total of eight white garage doors.
24

---

Published as a conference paper at ICLR 2026
The question specifically asks for the number of garages the first building the right, and can confidently state that there are eight of them.
Case Study of Fine-grained Perception
Question: What is the color scheme of the costumes worn by the men in the parade? Options: A: Yellow, red, and gold, B: Red, blue, and black, C: Green, orange, and white, D: Black, white, and gray
alamy
a
a
alamy stock photo
KFBYXP
Gemini-2.5-pro
Based on the image, the primary colors visible in the costumes are:
Red (outer robes, tassels on hats)
Blue (sashes/belts)
Yellow/Gold (inner ganments/trim)
Black (main body of the tunic/vest, hats) White (gloves, neck trim)
Comparing this to the options:
A: Yellow, red, and gold (Missing prominent black and blue)
B: Red, blue, and black (These are all very prominent colors)
C: Green, orange, and white (Green and orange are not major colors) D: Black, white, and gray (Missing prominent red, blue, and yellow)
The most accurately descriptive option focusing the major, dominant colors is B: Red, blue, and black, as these three are clearly primary components of the main garment layers. However, since Yellow is also very prominent, let's look closer. The yellow is significant color block. The options are bit restrictive.
If we have to choose the best fit: Red, blue, and black all very strong, dominant colors.
25

---

Published as a conference paper at ICLR 2026
Red, blue, and yellow/gold are also very strong, dominant colors.
Given the choices, Red, blue, and black captures three of the most contrasting and major colors in the main body of the uniform. However, the yellow parts are considered "gold," A: Yellow, red, and gold would also be strong contender, focusing on the brightest colors, but it misses the prominent black and blue.
Let's stick with the most prominent and contrasting large color blocks: Red, blue, and black appear to be the dominant features.
The correct option is B: Red, blue, and black.
Qwen2.5-VL-7B
Attention Weight (Normalizee)
aaam
0.6
a
alamy stock photo
K7
0.0
The color scheme of the costumes worn by the men in the parade is:
A: Yellow, red, and gold.
Qwen- (iven-iipr-7B((uur))
26

---

Published as a conference paper at ICLR 2026
1.0
0.8
Attention Weight (Normalizee)
0.4
alamy
0.2
a
alamy stock photo
I'll start by examining the image of the group of men in what appears to be a traditional parade. My goal is to determine the color scheme of their costumes from the options provided.
First, I'll look at the central figure on the left. He's wearing hat with red tassel, red and balck robe, and blue belt. The next man to his right is wearing similar outfit but with a red hat and red and black top. As my eyes scan the group, I can this pattern continues. The colors that stand out are red, black, and blue, along with some golden elements like the trims.
Next, I'll compare these colors against the given options. Option A mentions yellow, red, and gold, which misses the black and blue main colors. Option B includes red, blue, and black, which a perfect match for what I'm seeing. Option C lists green, orange, and white, none of which are present. And option D describes black, white, and gray, which are not part of the costumes at all.
After carefully reviewing the image, the most fitting description is clearly B: Red, blue, and black. That's my final answer.
E RESULTS ON VIRL39K
Table Comparison between Qwen2.5-VL-7B and the version RFT on ViRL39K data, A denotes the variations in accuracy.
MME-RW
BLINK
Mantis Eval
Hallusion Bench(Avg)
CRPE (relation)
Model
MMStar RealWorldQA
Overall
(en)
(val)
Qwen2.5-VL.y
63.9
68.5
57.4
56.4
75.1
52.9
76.4
64.4
Dwen2.5sVL-7BViRLL99
57.1
57.0
64.3
0.5.
0.4,
0.31
0.6
0.9,
0.8t
0.41
0.1.
To compare the training data synthesized by the ViPER framework with existing multimodal rea- soning datasets, fine-tuned Qwen2.5-VL-7B using the comprehensive ViRL39K dataset (Wang) et al., 2025a) via the GRPO algorithm. ViRL39K rovides a curated collection of 38,870 verifi- able QAs for Vision-Language RI. training, with over 80% math chart/diagram reasoning
27

---

Published as a conference paper at ICLR 2026
questions, The results summarized in Table 4 show that the fine-tuned model did not achieve perior performance on the evaluated benchmarks. Although modest improvements were observed tasks, the model slightly underperformed the baseline single-image and multi-image benchmarks emphasizing real-world perception. This outcome highlights two key insights: 1.The reasoning paradigms derived from mathematical and chart-oriented tasks exhibit limited generalizability to perception-intensive visual tasks; 2.It underscores the targeted efficacy of our proposed method in specifically enhancing visual perceptual capabilities.
on hallucination-orienne
F DATA AVAILABILITY STATEMENT
All image data employed in this study are under formal authorization for academic research pur- poses, ensuring full compliance with legal and ethical standards regarding copyright protection and privacy preservation.
28


Reviews:
[{"review_id": "Ft1LDntpuF", "reviewer": "Reviewer_8UBN", "summary": "This paper proposes ViPER, a self-bootstrapping training framework to improve fine-grained visual perception in VLMs without relying on external curated data. The core idea is a two-stage, coarse-to-fine framework: (1) Caption Self-Refining, in which a VLM captions an image, a diffusion model redraws the image from that caption, and the VLM critiques and then refines its caption by comparing the redraw with the original; (2) Visual-Operation Predicting, in which the model infers the edit applied to an image pair, thereby learning to attend to small, local changes. These yield a self-synthesized training set, Viper10K, and feed a coupled two-stage RL process, producing the Qwen-Viper variants from Qwen2.5-VL bases. On seven benchmarks encompassing single-image, multi-image, and hallucination tasks, Qwen-Viper shows ~1.6–1.7% average gains and up to 6.0% on fine-grained perception axes, with ablations supporting the need for both stages and for two-stage (vs. mixed) RL.", "questions": "- How sensitive are your results to the choice of BGE-M3? \n- Have you run a supervised-only baseline on the same data/budget to quantify the gain from RL?\n- Can you report results on at least one non-Qwen VLM (e.g., LLaVA/InternVL)?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- This work presents a coarse-to-fine framework that separates holistic image captioning from localized visual-operation prediction. This design cultivates both scene-level understanding and region-specific editing reasoning, forming an executable synthesis pipeline.\n- The system forms a closed loop that uses mismatches between the initial captions and the redrawn images as supervision, enabling label-free improvement.\n- Across seven benchmarks covering multi-image tasks and hallucination diagnostics, the models improve consistently with largest gains on fine-grained perception up to +6.0% on the 7B variant. \n- Ablations confirm that the two-stage design surpasses mixed RL approaches, with both stages providing complementary benefits.", "weaknesses": "- Results are limited to Qwen2.5-VL (3B/7B) while framework generality across other architectures/sizes is not demonstrated.\n- Data synthesis relies on Qwen-Image and OmniGen2 to reconstruct/edit images without  cross-generator report or real-edit robustness tests, leaving a risk of generator-specific artifacts/shortcuts.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "zbTBffelXc", "reviewer": "Reviewer_YstH", "summary": "The paper introduces ViPER, a self-evolutionary framework designed to enhance visual perception capabilities in Vision-Language Models. The authors identify a critical bottleneck in VLMs: the limited fine-grained visual perception, which is difficult to address with traditional methods like supervised fine-tuning and reinforcement fine-tuning. To overcome this challenge, they propose a novel two-stage task formulation for visual perception learning, structured as a coarse-to-fine process. This approach is implemented in the ViPER framework, which integrates a self-bootstrapping mechanism for iterative self-critiquing and self-prediction, allowing the model to evolve by generating its own training data. The framework employs a dual-granularity reconstruction process—image-level and instance-level—and integrates a two-stage reinforcement learning strategy. ViPER was applied to enhance the Qwen2.5-VL model, producing the Qwen-Viper series, which showed improvements across various benchmarks.", "questions": "Please refer to Weakness.", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "The paper presents a clear and well-structured approach to enhancing visual perception in Vision-Language Models, with an innovative self-evolutionary framework, ViPER, supported by robust experimental design and comprehensive benchmarks. The detailed illustrations and methodical experimental setup effectively demonstrate the framework's performance improvements.", "weaknesses": "* While the paper demonstrates the effectiveness of ViPER in improving visual perception, it primarily compares the Qwen-Viper models against a limited set of benchmarks. Including comparisons with more diverse and recent methods would provide a broader context for the proposed approach.\n\n* Although the paper conducts some ablation studies, a more thorough analysis of the individual components of the ViPER framework—such as the specific impact of the data synthesis module or the two-stage reinforcement learning—would help clarify the contributions of each part to the overall improvements.\n\n* The method relies heavily on self-generated data, which could potentially lead to issues with scalability and efficiency, especially for larger datasets. Future work could explore optimizations to reduce computational overhead and improve the model’s scalability without sacrificing performance.\n\n* The study focuses heavily on visual perception tasks, but the integration of visual understanding with textual reasoning could be further explored. The interplay between these two components could be better examined to understand their combined impact on multimodal reasoning tasks.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "StEETHIdbm", "reviewer": "Reviewer_vNi5", "summary": "This paper introduces a self-bootstrapping paradigm that enhances VLMs in recognizing fine-grained details and capturing dynamic differences. The core contribution is a two-stage RL optimization algorithm used to bootstrap the base model. The first stage requires the model to self-reflect on its initial caption and perform refinements. The second stage then requires the model to predict the visual operations based on the refined information. Evaluation across several benchmarks proves the method’s effectiveness in improving fine-grained perception and mitigating hallucinations. Ablation studies further validate the necessity of the two-stage structure over mixed RL, confirming that a cold start is unnecessary. Finally, the authors introduce an automated online data construction system designed for training this model.", "questions": "* How many RL steps do you train for each stage?\n* Maybe the author can try to report the results on MMVIP[1] and Inst-IT[2] benchmarks to further evaluate the fine-grained, instance-related perception capabilities.\n\n\n[1] Eyes Wide Shut? Exploring the Visual Shortcomings of Multimodal LLMs\n\n[2] Inst-IT: Boosting Multimodal Instance Understanding via Explicit Visual Prompt Instruction Tuning", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "* The idea of self-bootstrap VLMs through a two-stage training paradigm is interesting and novel, which improves the model in fine-grained visual perception.\n* The ablation studies are extensive, validating the key design choices of VIPER.\n* The results on several multimodal benchmarks show the effectiveness of the proposed training paradigm.", "weaknesses": "* This method heavily relies on a diffusion model, which is used not only for caption-to-image generation but also for image editing. This places high demands on the diffusion model's accuracy and subsequently becomes a bottleneck. The instruction-following capability of multimodal generative models, particularly for tasks requiring fine-grained compositional generation, is currently not very satisfactory.\n* The paper's data construction pipeline is not very clear, and some points remain ambiguous even after reviewing the appendix. For example, it is unclear how the refined caption is obtained.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "FKMtgOwU79", "reviewer": "Reviewer_56QP", "summary": "The paper presents ViPER, a self-supervised loop that lets a VL model bootstrap its own visual-perception skills without human labels or external teachers. Key technical novelty is an internal critic that detects its own failures, converts them into executable image-editing prompts, synthesises hard negatives, and continues fine-tuning itself.", "questions": "Please refer to weaknesses.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- first fully self-driven loop that converts its own perception failures into targeted training images.\n- The algorithm is presented through a modular pipeline, illustrated with a step-by-step running example, and accompanied by released code and 2.1 M generated images that together ensure reproducibility.\n- By enabling 3–8 B parameter models to outperform 10× larger proprietary counterparts without additional labels, ViPER offers an immediately practical and conceptually new route toward continual self-improvement of visual perception abilities.", "weaknesses": "- The quality of viper10k data is constrained by the performance of the models used for its creation, potentially leading to a self-reinforcing loop of model-centric biases rather than ground-truth visual reasoning.\n- The paper contrasts VIPER's efficiency with the computationally inefficient nature of large-scale SFT/distillation. However, the proposed framework introduces a high computational requirement for data generation.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)