Pixel Reasoner: Incentivizing Pixel Space Reasoning via Curiosity-Driven Reinforcement Learning
Authors: Alex Su
Haozhe Wang
Weiming Ren
Fangzhen Lin
Wenhu Chen
Venue: NeurIPS 2025 poster ()
Keywords: Vision-Language Models
Reinforcement Learning
Primary area: deep_learning
TL;DR: We introduce a novel reasoning paradigm -- pixel-space reasoning. We identified the learning trap when cultivating this ability and proposed a curiosity-driven RL approach to address it.

Abstract: Chain-of-thought reasoning has significantly improved the performance of Large
Language Models (LLMs) across various domains. However, this reasoning process has been confined exclusively to textual space, limiting its effectiveness in visually intensive tasks. To address this limitation, we introduce the concept of pixel-space reasoning. Within this novel framework, Vision-Language Models (VLMs) are equipped with a suite of visual reasoning operations, such as zoom-in and select-frame. These operations enable VLMs to directly inspect, interrogate, and infer from visual evidences, thereby enhancing reasoning fidelity for visual tasks. Cultivating such pixel-space reasoning capabilities in VLMs presents notable challenges, including the model’s initially imbalanced competence and its reluctance to adopt the newly introduced pixel-space operations. We address these challenges through a two-phase training approach. The first phase employs instruction tuning on synthesized reasoning traces to familiarize the model with the novel visual operations. Following this, a reinforcement learning (RL) phase leverages a curiosity-driven reward scheme to balance exploration between pixel-space reasoning and textual reasoning. With these visual operations, VLMs can interact with complex visual inputs, such as information-rich images or videos to proactively gather necessary information. We demonstrate that this approach significantly improves VLM performance across diverse visual reasoning benchmarks. Our 7B model, Pixel-Reasoner, achieves 84% on V* bench, 74% on TallyQA-Complex, and 84% on InfographicsVQA, marking the highest accuracy achieved by any open-source model to date. These results highlight the importance of pixel-space reasoning and the effectiveness of our framework.

Pixel Reasoner: Incentivizing Pixel-Space Reasoning with Curiosity-Driven Reinforcement Learning
Alex Su Haozhe Wang w WWeiiing Ren Vt Fangzhen Lin Wenhu Chen V1
University of Waterloo HKUST USTC* Vector Institute
Project Page: https: /tt:://iige ger- ai aib. .aa.lii...i...... github. io/Pixel -eaxxeeeReeaoer/
Abstract
Chain-of-thought reasoning has significantly improved the performance of Large Language Models (LLMs) across various domains. However, this reasoning pro- cess has been confined exclusively to textual space, limiting its effectiveness in visually intensive tasks. To address this limitation, we introduce the concept of rea- soning the pixel-space. Within this novel framework, Vision-Language Models (VLMs) are equipped with suite of visual reasoning operations, such as zoom-in and select-frame. These operations enable VLMs directly inspect, interrogate, and infer from visual evidences, thereby enhancing reasoning fidelity for visual tasks. Cultivating such pixel-space reasoning capabilities in VLMs presents no- table challenges, including the model's initially imbalanced competence and its reluctance to adopt the newly introduced pixel-space operations. We address these challenges through two-phase training approach. The first phase employs instruc- tion tuning on synthesized reasoning traces to familiarize the model with the novel visual operations. Following this, reinforcement learning (RL) phase leverages a curiosity-driven reward scheme to balance exploration between pixel-space reason- ing and textual reasoning. With these visual operations, VLMs can interact with complex visual inputs, such as information-rich images or videos to proactively gather necessary information. We demonstrate that this approach significantly improves VLM performance across diverse visual reasoning benchmarks. Our 7B model, Pixel-Ree Fixel-Reasoner, achieves 84% V* bench, 74% on TallyQA-Complex, and 84% on Infographics marking the highest accuracy achieved by any open-soure model to date. These results highlight the importance of pixel-space reasoning and the effectiveness of our framework.
Introduction
Recent advancements have demonstrated remarkable progress in developing complex reasoning
abilities in Vision-Language Models (VLMs). Leading models, such as OpenAl GPT40/GPT-
ol Hurst et al.) 2024 Jaech et al., 2024a], Gemini-2.5 (Team et al., 2023], VL-Rethinker [Wang et al. 2025 achieve superior performance on various multimodal reasoning benchmarks like Math [Lu et al. [202). MMMU (Yue et al.) [024], MEGA-
[Chen et al. 2024], etc. A common paradigm underpinning these of VLMs involves processing multimoall queries to extract relevant cues, followed by reasoning process (CoT Wei et al. 2022 conducted purely in the textual format.
Despite their success, the prevailing textual reasoning paradigm faces inherent limitation: relying solely on text tokens to express intermediate reasoning steps can constrain the depth and accuracy achievable by Vision-Language Models (VLMs) on visually intensive tasks. The lack of direct interaction with visual inputs-suce as drawing lines/marks, highlighting regions, or zooming
'These authors contributed equally and are listed alphabetically. Haozhe as Project Lead. Corresponding to: dlwImma3 45 44566@ 14516@gmail.com gmail.com,
wenhuchen uwaterloo.ca
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

in hinders the model's 's ability to interact with the information-rich images. As result, VLMs often struggle to capture fine-gre aeeeggrandd visual details, including tiny objects, subtle spatial relationships. small embedded text, and nuanced actions in videos.
Pixel Reasoner
leamingg
image shows sctup whore The qustion may likey located
Quandiun: Where to buy a may this haaed un lop??
pelicy
the tabli, so clesr
bypas Plae--ppace Ressering
Warm-Start Indruction Tusing lay the foundation elfcctidy
The mug is on the right upper comorr need to look closer at the logo on it.
Carimnk--Drisvn EI recoungss the explaration and practice Space Reasering.
Anower: This mug is from VaweddSturbuccsll
Figure 1: Illustration of Pixel Reasoner. When asked a visually-rich question, Pixel-Rease -Reasoner first inspects the visual inputs. Then it itera- tively refines its understanding and evolve reasoning by leveraging visual operations, such ZOOM-1 IN for images and SELECT-FRAMES for videos, ultimately arriving at conclusion.
The Learning Trap. Our approach combines warm- start instruction tuning phase and curiosity-driven RI. phase overcome the learning trap.
Figure 2:
These limitations motivates a fundamental rethinking of how VLMs engage with the visual modality during reasoning more seamlessly. This leads us to pose the research question:
Can VLMs perform reasoning steps more directly within the visual modality itself, leveraging computational visual manipulations as actions to guide reasoning?
We introduce the concept of pixel-space reasoning, proposing novel paradigm where reasoning is not exclusively confined to verbalized format but actively incorporates operations applied directly to the visual inputs. Rather than solely translating visual observations into textual cues, the model could actively manipulate and interact with the visual information throughout the reasoning process 'SELECT-FRAME' These visual operations serve as
employing operations like 'ZOOM-IN', integral steps within its reasoning chain, empowering the model to inspect, interrogate, and infer from visual evidence with enhanced fidelity. We frame this problem developing VLM endowed with a suite of visual operations. This novel framework involves strategically selecting and applying appropriate visual operations to the visual inputs, progressively refining its understanding, evolving its reasoning and ultimately arriving at a conclusion. To instill this novel capability of pixel-space reasoning, we follow the common post-training paradigm of instruction tuning and reinforcement learning (RL). However, cultivating pixel-space reasoning presents significant challenges.
Firstly, existing VLMs exhibit limited zero-shot proficiency in executing pre-defined visual operations, thus requiring meticulous instruction tuning to establish a foundational understanding of these new visual operations. This initial training phase must also preserve the model's inherent self-correction abilities, thereby preparing for -ndd--snd- in the subsequent RL phase.
Secondly, the warm-started model exhibits significant disparity in proficiency between its well- established textual reasoning and its emergent pixel-space reasoning capabilities, which creates "learning trap" that impedes the effective acquisition of pixel-space reasoning. On one hand, the model's initial incompetence in visual operations garners more negative feedback than textual reasoning. On the other hand, significant portion of training queries may not strictly necessitate visual operations, allowing the model to bypass these under-developed skills. These factors trap the cultivation of pixel-space reasoning, causing the premature cessation of efforts utilize visual operations and improve pixel-space reasoning.
To address these challenges, our approach combines warm-start instruction tuning phase and reinforcement learning phase. For instruction tuning, we synthesize 7,500 reasoning traces that facilitate the cultivation of both mastery over visual operations and self-correction capabilities. Following this meticulous warm-start instruction tuning, our RL approach leverages a curiosity-drie -driven reward scheme to balance the exploration and exploitation of pixel-space reasoning to incentivize the pixel-level reasoning. The RL phase collect another 7,500 examples from several public image and video datasets [Feng et al. 2025
Xu et al. 2025 Our final model Fixe-Reaaooner built on

---

top of Qwen2.5-VL-7B Bai et al. 2025 is able to show significant improvement across several visual reasoning benchmarks (with information-ric rich images/video) like V* [Wu and Xiel [024]] TAllyQA [Acharya et al. 2019 MVBench [Li et al. 2024a] and Infographies VQA [Mathew et al.
2.21]. On these benchmarks, Pixel-Reasoner shows best known open-source performance and
even exceeds proprietary models like Gemini-2.5-Pro [Team et al., [024a] and GPT-4o [Hurst et al.,
2024). We further conduct comprehensive ablation studies to provide insights into how our framework
effectively cultivates pixel-space reasoning. Our contributions are listed as follows:
We introduce the concept pixel-space reasoning for the first time.
2. We identified learning trap when cultivating this novel reasoning ability.
We proposed a novel two-staged wost-training approach, featuring meticulous instruction tuning stage, and a puriosity-driveer RL stage.
We achieved state-of-the offfth---arttt results on visually-intensive benchmarks with sixel-space reasoning.
Problem Formulation
We introduce pixel-space reasoning. novel paradigm enabling models integrate operations directly applied to visual inputs, rather than solely relying on textual reasoning. Formally, consider a vision-language query x = [V, L]. where V represents visual inputs (e.g., images or videos) and L is the textual query. A model To constructs solution
[y...... yal via an iterative reasoning process both pixel and textual space. At each step t, the model generates a reasoning segment V/ ~ =0(-(x,, yt-1) conditioned on the initial query and the set of all preceding reasoning steps Yt-1 |1 .i...... 3/ -1. Unlike the predominant textual reasoning paradigm, pixe-spaace reasoning allows each reasoning step y/: to be one of two types:
Textual Thinking:
Steps that involve reasoning purely within the textual domain, like calculating equation or domain knowledge to derive conclusion, etc.
Visual Operations: Steps that activate visual operations to directly manipulate or extract infor- mation from the visual inputs. A visual operation V/ involves invoking predefined function f. yielding an execution outcome e, = fwt). For instance, a model might generate y to trigger a select_f fame operation, fsr, with specified arguments (c.g., "target frame") in ye retrieve visual tokens e, for a particular frame. The reasoning step is then updated to y + concat(y, ex). incorporating the execution outcome e₁ for subsequent reasoning.
This iterative reasoning process concludes when a designated end token is generated. We aim to cultivate pixel-space reasoning via reinforcement learning (RI.), where the objective is to optimize Vision-Language Model (VLM) policy To that maximizes the expected reward over a dataset D:
max E [(xx y) D.y mo(x|x)]
common approach for r(x, y) is to adopt binary correctness reward, which assesses the correctness of the generated solution y for given query (DeepSeek-Al et al., 2025 Liu et al. 2025
r(x, y) {
if the solution contains the correct answer query x, otherwise.
In this work, we focus on two types of visual inputs: images and videos. We specifically consider two types of visual operations: ZOOM-IN for inspecting details within a specified region of target image, and SELECT-FRAME for analyzing specific frames in a video sequence. Detailed protocols for these visual operations are provided in the appendix.
3 Warm-Start Instruction Tuning
We aim to cultivate a novel pixel-space reasoning paradigm leveraging existing Visual-Language Models (VLMs). However, instruction-tunel models such as Qwen22.5VLLIInstruc exhibit limited zero-shot proficiency in executing novel visual operations (as shown the analysis the appendix), likely due to their absence in standard training data. To lay the groundwork for utilizing visual operations in subsequent reinforcement learning, we describe in this section our approach to data curation and instruction tuning.
Collect Seed Datasets. Our data curation pipeline is designed to collect high-quality pixel-space reasoning trajectories. These trajectories are intended to serve expert demonstrations for policy,

---

Homerre Ingester
Menic
(a) Direct Dittilation free CPT
(b) We orglay Template-Rood Synthaii complii cantrol
peanc "lepaving tngick
Direct Distillation from GPT40 may generate "bypassing trajec- Figure 4: We synthesize self- tories" where the model ignores the visual operations and performs textual correction trajectories by inserting reasoning. We thus adopt a template-based synthesis strategy.
Figure 3:
erroneous reasoning segments.
showcasing the effective utilization of visual operations. Therefore, we first select three datasets: SAIB [Kirillov et al.] 2023]. FineWeb [Ma et al. 2024] and STARQA (Wu et al. 2024). These datasets offer diverse modalities and contents, spanning natural scenes with extensive segmentation masks, diverse web-pages, and real-world videos requiring situated reasoning. Across all three datasets, their high visual complexity provides rich visual information for fine-grained analysis, and their explicit annotations serve as crucial reference visual cues for trajectory synthesis. A detailed description of these datasets can be found in the appendix.
Localize Reference Visual Cues. To ensure that the visual operations are genuinely necessary for resolving the lision-language queries, selected or synthesized queries that specifically require the localization of fine- fine-grained visual cues within the rich visual information. The Fine and STARQA datasets already provide vision-language queries paired with reference visual cues for answers. For the SA1B dataset, we first leveraged GPT-4o to identify specific target visual details prompted GPT-4o within an image, such as small objects particular attributes. Subsequently, to generate a natural language query based on the identified detail and the corresponding image, formulating a fine-grained visual question that necessitates locating that specific cue.
Synthesize Expert Trajectories. Based on the curated lision-language queries requiring fine-grained visual analysis, we then synthesize expert trajectories using GPT-4o. As illustrated in Fig. 3 (a), we observed that direct distillation from GPT-4o sometimes resulted in "bypassing trajectories" ". In these cases, GPT-4o could occasionally bypass erroroneous visual operations and arrive at the correct final answer solely through its textual reasoning capabilities. Such trajectories pose a risk of mislcading the policy by ignoring the problematic outcomes of executing visual operations.
To mitigate this issue and ensure complete control over the synthesized trajectories, we employ a template-ben based synthesis approach. As shown in Fig. 33 (b), this template structures a pixel-space reasoning trajectory as a sequence: initial analysis of the entire visual input, followed by triggering specific visual operations to extract fine-grained details, subsequent analysis of these detailed visual cues, and ultimately arriving at the final answer. To synthesize a trajectory according to this template, we utilize the reference visual cue associated with each vision-language query. We first prompt GPT-4o to generate a textual description summarizing the entire visual input. Then, leveraging the reference visual cue, prompt GPT-4o for a more detailed textual analysis focusing specifically on that cue. By composing these textual thinking segments and incorporating the visual operation targeting the reference visual cue, obtain pixel-space reasoning trajectory that effectively interleaves textual reasoning with required visual operations.
In addition to these basic single-pass trajectories that help the policy understand the effective utilization of visual operations, we also synthesize error-induced self-correction trajectories. These are designed to preserve and foster the policy's 's ability to properly react to unexpected inputs or errors during exccution. As illustrated in Fig. 4 we synthesize such trajectories by deliberately choosing incorrect improper visual cues, such an irrelevant video frame or overly large image regions, for reaching the correct answer. We then insert the visual operations and textual thinking segments for these distracting visual cues before introducing the correct reference visual cues, thus simulating self-correction behaviors in error-induced trajectories.
Warm-Start Instruction Tuning, We include two primary types of pixel-space reasoning trajectories in our training data: single-pass and error-induced self-correction trajectories. We also include textual reasoning trajectories for vision-language queries that do not necessitate fine-grained visual analysis. This mixed data composition allows the policy to adaptively employ pixel-space reasoning only when necessary. We employ the standard Supervised Fine- Tuning (SFT) loss for training. However,

---

Aueeageet Curiosit Velue
Curiosity Training
Reazoning (83832
0.7
8,006
0,04
Fa00 Pare space Space
0.5
Efficiency Penalty over Trinning Steps
Auarages Punalty value
0.0020
Emideney Sanstyy
-3.0025
-3.0000
0.0
20 25
20
30
10
15
30
Steps 3X1
Staps (x16)
Figure 5: Requires Incentives to Explore Pixel- space Reasoning. Without proper incentives, the pol- icy learns to bypass the nascent pixel-space reasoning, resulting declining RaPR.
Figure 6: The Training Trend of our Curiosity- -Diven Reward Scheme. We leverage curiosity bonus to en- courages exploration and efficiency penalty to punish excessive visual operations.
apply loss masks to tokens that represent either execution outputs from visual operations the specifically designated erroneous visual operations within the self-correction trajectories. Masking the erroneous operations prevents the policy from learning execute the incorrect actions.
4 Curiosity-Driven Reinforcement Learning
The warm-started model typically suffers from disparity in its capabilities: proficient textual reasoning versus nascent pixel-space reasoning. This inherent imbalance creates a "learning trap" that impedes the development of pixel-space reasoning, stemming from two synergistic issues. Firstly, the model's initial limited mastery over visual operations frequently leads to failure incorrect outputs, resulting in higher incidence of negative feedback compared to text-mediated reasoning. Secondly, significant portion of training queries does not rigorously demand visual processing for correct response. allowing the model to ignore the outcomes of visual operations or default to its stronger textual reasoning. This interplay fosters a detrimental cycle where initial failures discourage further attempts, leading to the premature abandonment of exploring and mastering visual operations. As shown in Fig. when training the Warm-Start Model with standard RI. [DeepSeek-Ae Al et al., 2025 Wang al. 2025 without proper incentives, the policy learns to bypass the nascent visual operations.
To break this cycle, we propose vuriosity-driverd reward scheme to incentivize sustained exploration of pixel-space reasoning, inspired by curiosity-driven exploration in conventional RL [Pahhkk et al. 2017]. Instead of relying solely on extrinsic rewards for correctness, this curiosity bonus specifically incentivizes the act of attempting pixel-space operations. By intrinsically rewarding such active practice, aim to bolster the model's nascent visual skills and counteract the discouragement of exploration that arises from early operational failures and the associated negative feedback. This mirrors how a child, driven by curiosity, might repeatedly attempt a difficult motor task, learning from each attempt, rather than immediately defaulting to an easier, already mastered skill.
Specifically, we formalize this objective as constrained optimization problem. Let 1 1PR(y) denotes the indicator function of response utilizing pixel-space reasoning, and (y) represent the number of visual operations. The goal is maximize the expected correctness outcome, subject to two critical constraints meticulously designed to cultivate the pixel-space reasoning:
max IE ()xx,)) D.y Tely x)
subject to RaPR(x) E [PR(Y) [I9())))>~<o>>/b no (ybx) 2 H. H. N
(2)
Here, the first constraint concerns the Rate of Pixel-space Reasoning (RaPR) (pronounced "rapper") triggered for query x. We mandate that this rate, averaged over rollouts y of query x, must be no less than predefined threshold H. This encourages the policy to consistently attempt pixel-space reasoning across a significant proportion queries, acting as directive to explore this less familiar reasoning path. The second constraint imposes an upper bound N on the number of visual operations
5

---

used in any individual response. This ensures that while exploration is encouraged, it remains computationally efficient and does not lead overly complex or protracted visual processing for individual responses.
This constrained optimization problem can be transformed into an unconstrained problem via La- grangian Relaxation [Lemaréchal, [001], resulting in a single reward function. This technique commonly employed in constrained RL [Achiam al. 2017 Wang et al. 2022 2023]. The transformation yields the following modified reward function r' (', y), detailed in the appendix:
r' (xx y) - r(x, y) + 0: Veuriosity (x) + 3 Tpenatty (y), where Feuriosity () (H RaaRRRP(()))) (x). 0)- (y)
(3)
(4)
"penalty (y) = min(N nxo (y).
(5)
The modified reward incorporates two additional terms. The first term Feuriosity (x, y) serves as the core of our curiosity mechanism. It provides an intrinsic reward that directly encourages the model to satisfy its "curiosity" about pixel-space operations, especially for queries where it has low history of attempting them. Akin to infants curious about and exploring unseen environments interactions, this term credits response y a bonus for employing pixel-space reasoning when the adoption of pixel-space reasoning, RaPR(x) (x). is below target threshold H. This curiosity bonus effectively lowers the activation energy for trying the visual operations, making the model more "inquisitive" and willing to venture into less certain reasoning paths. The second term, penaltyy (y), acts as an efficiency penalty at the response level, penalizing redundancy in visual operations by considering the number of visual operations performed, n. (y), relative to desired maximum N.
novel
The coefficients a 2 and 8 2 0 are non-negative I. Lagrangian multipliers. These multipliers can be tuned automatically, for instance, via dual gradient descent [Bishop and Nasrabadi] 2006 Wang et al., 2020), or set as pre-defined hyperparameters [Wang et al. 2022]. In our experiments, we adopt the latter approach for simplicity. We provide concrete example in the appendix to illustrate how these hyperparameters reflect our desired properties of the policy.
This reward scheme offers a dynamic reward mechanism that automatically tune the exploration bonus as training proceeds. As illustrated in Fig. 6 the curiosity bonus will naturally diminishes as the policy explores more on pixel-space reasoning. This prevents the policy policy from reward hacking - overly relying on the exploration bonus regardless of final correctness.
5 Experiments
In this section, we first outline the training and evaluation settings. We then examine the effectiveness of pixel-space reasoning, and study the key factors for cultivating pixel-space reasoning.
Training Data and Evaluation Settings. Utilizing the data curation pipeline outlined in Section 3 we assembled a dataset of 7,500 trajectories for warm-start instruction tuning. This dataset includes 5,500 pixel-space reasoning trajectories synthesized using GPT-4o, spanning domains such as images, webpages, and videos. We also include 2,000 text-space reasoning trajectories to balance the use of visual operations. During RL, we construct 15,000 queries from our SFT dataset, Infograph-
ieVQA [Mathew al., 202T). and publicly available datasets [Xu et al. 2023 Wu et al. 2024 Refer to the appendix for comprehensive view of the dataset compositions.
We evaluated our model and other baselines on four representative multimodal benchmarks using greedy decoding: TallyQA, V*, InfographicVQQA. and MVBench. This selection offers a wide spectrum of visual understanding tasks, from fine-grained object recognition high-level reasoning in both static and dynamic scenarios. Specifically, V* (V-Star) [Wu and Xie 2024) evaluates multimodal large language models (MLLMs) on their ability to process high-resolution, visually complex images and focus on fine-grained visual details. Tally [Acharya et al. 2019) consists of questions that require reasoning over object quantities, often demanding the model to locate, differentiate, and tally objects across complex scenes. MVBench [Li al. 2024a] is a comprehensive benchmark designed to evaluate multimodal large language models (MLLMs) on their temporal understanding capabilities across 20 challenging video tasks, necessitating reasoning beyond static image analysis. InfographicVOS. VQA [Mathew et al.. 2021 evaluates the model's ability to understand complex infographic images that blend textual and visual content, including charts, diagrams, and annotated images. Success on this benchmark requires parsing layout, reading embedded text, and linking visual elements with semantic mcaning.

---

Table Our main results on the four evaluated benchmarks.
Model Metric
Size
V Bench
TallyQA-complex Acc
MVBench-tess Acc
InfoVQA-test
Acc
ANLS
Models w/o Tools
GPT-4o
62.8
73.0
64.6
80.7
Demini-2.0-Flast
73.2
73.8
86.5
Gemini-2.5-Pro
79.2
74.0
84.0
Qwen2.5-VI.
70.4
68.6
63.8
80.7
Video-R1
7B
51.2
42.6
63.9
67.9
LongLLava
13B
68.5
64.6
54,6
65.4
Gemma3
27B
62.3
54.3
56.8
59.4
Models with Tools
Visual Sketchpad (GPT-4o)
80.4
WV--nhance (GPT-4V)
81.2
PaLI-3-VPD
5B
70.9
SEAL
55B
76.6
Ours (Initialized from Qwen2.5-VI--78
Pixel RReasoner
84.3
73.8
67.8
84.0
Ablation Baselines (Ablated from Pixel Reassner)
Warm-Star Model (w/o RL)
7B
79.0
67.9
59.0
74.3
RL w/o Curiosity
80.7
RL w/o Warm-Start
72.2 81.7
65.6
81.2
RL w/o Correction-Dass
7B
80.1
69.8
63.6
78.2
Compared Models and Implementation. We compare against a wide range of models.
Models without Tools: We include GPT-4o [Hurst et al.
2024] and Gemini-2.00 2.0-Fin-.0--lash [Team]
let al., 2024b] and Gemini-2.5-PPy [Team al., 2024a These models do not have access to tools and simply answer with chain-of-thought. We include Qwen2.5-VL. [Bai et al.] [2025] Gemma3 [Team al. 2025 to show the general VLMs' performance. We also compare with RL-based VLM model Video-RI Feng et al.) [025] due to the similar algorithm. We further include Long [Wang et al. 2024 because aims scale up image input deal with high-resolution images (V*) and long video sequence (MVBench).
Models with Tools: We include Visual Sketchpad [Hu et al. 2024a]. which empowers the GPT- 4o to use different tools like zoom-in, depth, etc. We also include Instruction-Guide Visual Masking [Zheng et al. 2024), which highlights desired region in given image. Finally, we add
(VPD) (Hu et al., 2024b). which aims to distill tools reasoning into closed-source VLMs like PaL.I These models are specialized in V* Bench. We include SEAL [Wu and Xie 2024] from original V* Bench paper, which utilizes visual guided search tool to augment high-resolution image understanding.
Pixel was trained on 8 A800(80G) GPUs, using Open-RI I and OpenRLHF for instruc- tion tuning and reinforcement learning respectively. We adopt GRPO [DeepSeek-AI et al., (2025) with selective sample relay due to vanishing advantages [Wang et al. 2025]. We include training details in the appendix, and will release code, models, and data to support reproducibility.
5.1 Main Results
achieves the highest open-source results across all four bench- marks. Remarkably, Pixel-Reasoner, at mere 7B parameters, not only surpasses substantially larger open-source models like the 27B Gemma3 across all benchmarks, but also outperforms specialized models that depend on external tools, such as IVM-Enhance (GPT-4V). Furthermore, Tixel-Reasoner" exceptional capabilities extend to outperforming leading proprietary models, evidenced by its significant 5.1 percentage point lead over Gemini-2.5-Pro on V-star Bench (84.3 vs 79.2), and achieving the overall highest scores amongst all models listed. We observe that RL training
Table I shows that

---

Pize-space Reasuring
Operatcons Error Rale
Compatence Gep beriveen Twu Beesuring Cxpabllios
64
Ra turm Concrrrnenniit Hodd
Carterty
(Np Do saergge Cormetsens
6:22
Be Ser--hee Modul
-6.3
fakun bat of
Peedifi Cuaniity
Rauhetee Rovere
nJ Zor Moot Hode
-4.00
alo Durtos hrnd
15 Wheo 3416)
25
30
10
1.5 20
25
30
Stngs (c100
Steps (4S0)
Figure 7: Trainig Dynamics Ablation Baselines. During RI. training, different baselines show different trends in triggering pixel-space reasoning (Left), and the error rate of utilizing visual operations (Middle). Our curiosity-dri driven reward scheme effectively cultivates pixel-space reasoning by actively practicing and enhancing this nascent ability, evidenced by the narrowed gap in return between the reasoning paradigms (Right).
is monumental to the great results. Our ablation model "Warm-Start Model (w/ RL)" has lower performance than the original checkpoint Qwen2.5-VL on many benchmarks. After RL training, the performance skyrockets to SoTA level. This reflects the necessity of RL training to cultivate pixel-Ievel reasoning.
The performance profoundly underscores the significant potential of our proposed pixcl-space reasoning paradigm. This potential is further highlighted when comparing Pixel-Reasone with the ablation baselines, "RL w/o Curiosity" and "RL. w/o Warm-Start". Due to insufficient incentives and limited proficiency in utilizing visual operations, these baselines ultimately default to text-space reasoning, resulting in a significant performance drop of 2.5 points on average across benchmarks. These empirical gains verify the effectiveness of pixel-space reasoning b enabling the model to proactively engage with visual operations, this new reasoning paradigm facilitates a more precise visual understanding and consequently, stronger reasoning capability.
5.2
Key Factors for Cultivating Pixel-Space Reasoning
This section investigates two critical factors in fostering pixel-space reasoning through RL: first, the policy's 's proficiency in utilizing visual operations lays the foundation for RL, and second, the role of incentives in encouraging the adoption of pixel-space reasoning during RL. To gain deeper insights into the results in Tab. [ we analyze the training dynamics of various baselines, shown in Fig. 7 Specifically, the left panel shows the proportion of training rollouts that employ pixel-space reasoning strategies, while the middle panel indicates the error rate associated with the execution of visual operations. The right panel compares the expected correctness between the reasoning paradigms, highlighting the disparity in the two capabilities over training time. In addition, the appendix provides concrete examples of trajectories that bypass pixel-space reasoning, illustrating "the learning trap."
Effective Utilization of Visual Operations Requires Instruction Tuning. A crucial finding is that meticulous warm-start instruction tuning is essential for enhancing the policy's 's mastery of visual operations and its capacity for self-correction. To demonstrate this, we analyze the RL training dynamics originating from three distinct instruction-tuncd models: (a) The Warm-Start Model that undergoes the proposed warm-start instruction tuning phase. (b) The No-Correction Model is tuned using single-pas pass expert trajectories but without the error-induce self-corrective trajectories. (c) The Zero-Shot Model is Qwen2.5- 20ven2.5-VL-IInttru VL-Instruct with zero-shot prompts of available visual operations.
The training dynamics reveal distinct outcomes:
The Zero-Shot Model (orange lines) commences with a low RaPR of approximately 20%, which progressively declines during RL and reaches zero. This initial low propensity to trigger visual operations provides insufficient practice on visual operations. Consequently, the model receives lower expected returns from its nascent pixel-space reasoning compared to its more established textual reasoning, leading to a diminishing RaPR. This illustrates how limited initial proficiency in visual operations can create a detrimental cycle. hindering the development of pixel-space reasoning. Its error rate for visual operations also remains low due their minimal usage.
The No- No-Correction Model (blue lines), trained solely on single-pass expert trajectories, initially exhibits an increase in RaPR, suggesting a propensity attempting visual operations. However, this trend is quickly overshadowed by significant and persistent rise in the failure rate of
8

---

these operations. This elevated error rate points to the model's inability to effectively respond to unexpected or erroneous outcomes from visual tasks. This deficiency stems directly from the absence of error-induced self-correction trajectories during its instruction-tuning phase. Consequently, the policy increasingly relies on pixel-space reasoning while simultaneously ignoring the outcomes of visual operations and favoring textual reasoning. Interestingly, we observe that the resulting reasoning trajectories involve error messages from visual operations but can still arrive at correct answer. This indicates reward hacking: the policy earns curiosity bonus by superficially executing visual operations, and meanwhile it also earns correctness reward by essentially relying textual reasoning to arrive at the final answer.
The Warm-Start Model (purple lines as "PixelReasoner") ) in constrast, serves as the foundation for RL and is uppropriately incentivized, it enables the successful cultivation of pixel-space reasoning without exhibiting excessive error rates in visual operations. This underscores the importance of the comprehensive instruction tuning provided by the warm-start phase.
Cultivation of Novel Reasoning Capabilities Requires Incentives. To evaluate the impact of incentives, we compare the RL training dynamics starting from the Warm-Start Model, both with and without curiosity-driven incentives.
Standard RL without curiosity (grey lines). The curve shows a consistent decrease in the utilization of visual operations (RaPR), from around 0.55 to O in 240 gradient steps. This decline occurs because, without a specific impetus to explore, the policy favors its more developed textual reasoning over the initially less competent pixel-space reasoning. The failure rate of visual operations remains low as their usage diminishes.
Our Model (purple lines), which also starts from the Warm-Start Model but incorporates curiosity-driven exploration bonus, demonstrates a more complex and ultimately successful trajectory. Initially, Pixel -Reasonee exhibits a decrease in RaPR in the first 50 gradient steps, and then plateaus for around 150 steps. During this stage, the policy is compelled by the curiosity bonus to continue exploring mixel-space reasoning, despite its relative inferiority compared to textual reasoning (as shown in Fig. / (Right)). Not until 200 gradient steps, the policy starts to effectively leverage the benefits of pixel-space reasoning. Its RaPR proactively and substantially increases, accompanied by a low and stable failure rate for visual operations. This indicates that the combination of the robust Warm-Start instruction tuning and the curiosity-drive incentive allows the policy to not only explore but also master the new pixel-space reasoning capability. Also note that of visually intensive tasks in training queries. We provide curves of test sets in the appendix.
exhibit relatively high RaPR of 80% due the high proportion
Related Work
Post-Training for Vision-Languann Models. Post-training techniques, such as instruction tuning and reinforcement learning, are critical for adapting large Vision-Language Models (VLMs) to complex tasks beyond initial pre-training. LLaVA [Luu et al.
2023]. Llava-OV (Li et al. 2024b), Infinity-MM
[Gu et al. 2024]. and CommoTT-Vi [Guo et al. 2024] has shown that scaling instruction tuning
datasets and increasing task diversity significantly enhances VLM generalization across various multimodal benchmarks.
Recently, a growing body of work applies RL to the multimodal domain [Deng et al. 2025 Huang
al. 2025, Feng et al.] (0255. These approaches typically employ multi-stage pipelines, starting with SFT on costly data distillation and then applying RL to further refine the model's 's reasoning capabilities. VL--Rehinneer (Wang et al. 2025) investigates more direct RL approaches to foster slow-thinking in VLMs, and introduced selective sample replay (SSR) to counteract the vanishing
advantages problem in GRPO.
Vision-Language Models with Tools. Recent research has explored augmenting VLMs with external tools or enabling them to perform pixel-level operations on inputs. Distiillaitltii (VPD) [Hu et al., 2024b) focus on training models to effectively utilize tools or distill tool-based reasoning. Visual Sketchpad (Hu 2024a] equips models, such as GPT-4o, with tools like depth perception and Python plotting. Models like 03 [Jaech et al. 2024b) demonstrate an ability to "think with images" by dynamically applying operations like zooming or flipping to improve visual understanding. Specific tools such as Cerrnctioo-Guided Visual Masking
(Qi et al.
2023],
(Zheng et al. 2024 and visual guided search (Wu and Xie. 2024 has been integrated into these frameworks.

---

7 Conclusion
In this paper, we show how to incentivize the pixcl-space reasoning from an existing vision-language models for the first time. Our warm-start instruction tuning and curiosity RL are both essential to achieve the state-of-the-ar performance. However, our work is currently still limited to two primary operations, which is insufficient for broader tasks. Our framework is easily extensible to other operations like depth map, image search, etc. In the future, the community can work together to enrich the visual operations to enhance the pixel-space reasoning in VLMs.
10

---

References
Aaron Hurst, Adam Lerer, Adam Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Os- trow, Akila Welihinda, Alan Hayes, Alec Radford, al. Gpt-4o system card. arXiv preprint arXiv:2410.21276, 2024.
Aaron Jacch, Adam Kalai, Adam Lerer, Adam Richardson, Ahmed El-Kishky, Aiden Low, Alec Helyar, Aleksander Madry, Alex Beutel, Alex Carney, et al. Openai ol system card. arXiv preprint arXiv:2412.16720, 2024a.
Gemini Team, Rohan Anil, Sebastian Borgeaud, Jean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew Dai, Anja Hauth, Katie Millican, al. Gemini: a family of highly capable multimodal models. arXiv preprint arXiv:2312./1805, 2023.
Haozhe Wang. Chao Qu, Zuming Huang, Wei Chu, Fangzhen Lin, and Wenhu Chen. Vl-rethinker: Incentivizing self-reflection of vision-language models with reinforcement learning. arXiv preprint 2888: 2025.
Pan Lu, Hritik Bansal, Tony Xia, Jiacheng Liu, Chunyuan Li, Hannaneh Hajishirzi, Hao Cheng, Kai-Wei Chang, Michel Galley, and Jianfeng Gao. Mathvista: Evaluating mathematical reasoning of foundation models in visual contexts. arXiv preprint arXiv:2310.02255, 2023.
Xiang Yue, Yuansheng Ni, Kai Zhang, Tianyu Zheng, Ruoqi Liu, Ge Zhang, Samuel Stevens, Dongfu Jiang, Weiming Ren, Yuxuan Sun, et al. Mmmu: A massive multi-discipline multimodal under- standing and reasoning benchmark for expert agi. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9556-9567 2024.
Jiacheng Chen, Tianhao Liang, Sherman Siu, Zhengqing Wang, Kai Wang, Yubo Wang, Yuansheng Ni. Wang Zhu, Ziyan Jiang, Bohan Lyu, et al. Mega-bench: Scaling multimodal evaluation to over 500 real-world tasks. arXiv preprint arXiv:2410. 10563, 2024.
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824 24837, 2022.
Kaituo Feng, Kaixiong Gong, Bohao Li, Zonghao Guo, Yibing Wang, Tianshuo Peng, Benyou Wang, and Xiangyu Yue. Video-r1: Reinforcing video reasoning in milms. arXiv preprint arXiv:2503.21776, 2025.
Guowei Xu, Peng Jin, Hao Li, Yibing Song, Lichao Sun, and Li Yuan. Llava-cot: Let vision language models reason step-by-step, 2025. /ttpsstss///a orgg httttp/s//aaxiiooooo/////1111101 10440
Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, et al. Qwen2. 5-vl technical report. arXiv preprint arXiv:2502. 13923, 2025.
Penghao Wu and Saining Xie. V Guided visual search as a core mechanism in multimodal Ilms. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 13084-13094, 2024.
Manoj Acharya, Kushal Kafle, and Christopher Kanan. Tallyqa: Answering complex counting questions. In Proceedings of the AAAI conference on artificial intelligence, volume 33, pages 8076-8084 2019.
Kunchang Li, Yali Wang, Yinan He, Yizhuo Li, Yi Wang, Yi Liu, Zun Wang, Jilan Xu, Guo Chen, Ping Luo, et al. Mvbench: A comprehensive multi-modal video understanding benchmark. In Proceedings of the IEEE/CVE Conference Computer Vision and Pattern Recognition, pages 22195-222066 2024a.
Minesh Mathew, Viraj Bagal, Rubèn Pérez Tito, Dimosthenis Karatzas, Ernest Valveny, and CV Jawa- har. Infographicvga. 2022 ieee. In CVF Winter Conference on Applications of Computer Vision (WACV), pages 2582-2599 2021.
11

---

Gemini Team, Petko Georgiev, Ving lan Lei, Ryan Burnell, Libin Bai, Anmol Gulati, Garrett Tanzer, Damien Vincent, Zhufeng Pan, Shibo Wang. et al. Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context. arXiv preprint arXiv:2403.05550, 2024a.
DeepSeek-AI, Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, Xiaokang Zhang. Xingkai Yu, Yu Wu, Z. F. Wu, Zhibin Gou, Zhihong Shao, Zhuoshu Li, Ziyi Gao, Aixin Liu, Bing Xue, Bingxuan Wang, Bochao Wu, Bei Feng. Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, Damai Dai. Deli Chen, Dongjie Ji, Erhang Li, Fangyun Lin, Fucong Dai, Fuli Luo, Guangbo Hao, Guanting Chen, Guowei Li, H. Zhang. Han Bao, Hanwei Xu, Haocheng Wang. Honghui Ding, Huajian Xin, Huazuo Gao, Hui Qu, Hui Li, Jianzhong Guo, Jiashi Li, Jiawei Wang. Jingchang Chen. Jingyang Yuan, Junjie Qiu, Junlong Li, J. L. Cai. Jiaqi Ni, Jian Liang, Jin Chen, Kai Dong. Kai Hu, Kaige Gao, Kang Guan, Kexin Huang, Kuai Yu, Lean Wang, Lecong Zhang. Liang Zhao, Litong Wang. Liyue Zhang, Lei Xu, Leyi Xia, Mingchuan Zhang. Minghua Zhang, Minghui Tang. Meng Li, Miaojun Wang. Mingming Li. Ning Tian, Panpan Huang, Peng Zhang, Qiancheng Wang. Qinyu Chen, Qiushi Du, Ruiqi Ge, Ruisong Zhang, Ruizhe Pan, Runji Wang. R. Chen, R. Jin, Ruyi Chen. Shanghao Lu, Shangyan Zhou, Shanhuang Chen, Shengfeng Ye, Shiyu Wang, Shuiping Yu, Shunfeng Zhou, Shuting Pan, S. Li, Shuang Zhou, Shaoqing Wu. Shengfeng Ye, Tao Yun, Tian Pei, Tianyu Sun, T. Wang. Wangding Zeng, Wanjia Zhao, Wen I Liu, Wenfeng Liang, Wenjun Gao, Wenqin Yu, Wentao Zhang, W. L.. Xiao, Wei An, Xiaodong Liu, Xiaohan Wang, Xiaokang Chen, Xiaotao Nie, Xin Cheng, Xin Liu, Xin Xie, Xingchao Liu, Xinyu Yang, Xinyuan Li, Xuecheng Su, Xuheng Lin, X. Q. Li. Xiangyue Jin, Xiaojin Shen, Xiaosha Chen, Xiaowen Sun, Xiaoxiang Wang, Xinnan Song, Xinyi Zhou, Xianzu Wang. Xinxia Shan, Y. K. Li, Y. Q. Wang. Y. X. Wei, Yang Zhang, Yanhong Xu, Yao Li, Yao Zhao, Yaofeng Sun, Yaohui Wang, Yi Yu, Yichao Zhang, Yifan Shi, Yiliang Xiong. Ying He, Yishi Piao, Yisong Wang, Yixuan Tan, Yiyang Ma, Yiyuan Liu, Yongqiang Guo, Yuan Ou, Yuduan Wang. Yue Gong, Yuheng Zou, Yujia He, Yunfan Xiong, Yuxiang Luo, Yuxiang You, Yuxuan Liu, Yuyang Zhou, Y. X. Zhu, Yanhong Xu, Yanping Huang, Yaohui Li, Yi Zheng, Yuchen Zhu, Yunxian Ma, Ying Tang, Yukun Zha, Yuting Yan, Z. 7. Ren, Zehui Ren, Zhangli Sha, Zhe Fu, Zhean Xu, Zhenda Xie, Zhengyan Zhang, Zhewen Hao. Zhicheng Ma, Zhigang Yan, Zhiyu Wu, Zihui Gu, Zijia Zhu, Zijun Liu, Zilin Li, Ziwei Xie, Ziyang Song, Zizheng Pan, Zhen Huang, Zhipeng Xu, Zhongyu Zhang, and Zhen Zhang. Deepscek-rl: Incentivizing reasoning capability in Ilms via reinforcement learning, 2025. 12948
URL URLLhttps///arivvorrr///
Zichen Liu, Changyu Chen, Wenjun Li, Penghui Qi, Tianyu Pang. Chao Du Wee Sun Lee, and Min Lin. Understanding le-zooo-like training: A critical perspective. arXiv preprint urXiv:2503.20783, 2025.
Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C. Berg, Wan-Yen Yen Lo. Piotr Dollár, and Ross Girshick. Segment anything, 2023. URL https:///axxi www.aaiww.orggab//2200000 23044024333
Xueguang Ma, Shengyao Zhuang, Bevan Koopman, Guido Zuccon, Wenhu Chen, and Jimmy Lin. Visa: Retrieval augmented generation with visual source attribution, 2024. URL https //arxiv.org/abs/2412..1
Bo Wu, Shoubin Yu, Zhenfang Chen, Joshua B Tenenbaum, and Chuang Gan. Star: A benchmark for situated reasoning in real-world videos, 2024. URL /ttpp::/ //app::///axvv....//// http:/aaivvv.gr.oo.. wwtp/bb////0...... 2405.09111
Deepak Pathak, Pulkit Agrawal, Alexei Efros, and Trevor Darrell. Curiosity-driven exploration by self-supervised prediction. In International conference on machine learning, pages 2778-2787. PMLR, 2017.
Claude Lemaréchal. Lagrangian relaxation. Computational combinatorial optimization: optimal or provably near-optimal solutions, pages 112-156, 2001.
Joshua Achiam, David Held, Aviv Tamar, and Pieter Abbeel. Constrained policy optimization. In International conference on machine learning, pages 22-31. PMLR, 2017.
Haozhe Wang, Chao Du, Panyan Fang, Shuo Yuan, Xuming He, Liang Wang, and Bo Zheng. Roi- constrained bidding via curriculum- bayesian reinforcement learning. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 4021-4031, 2022.
12

---

Haozhe Wang. Chao Du, Panyan Fang, Li He. Liang Wang, and Bo Zheng. Adversarial constrained bidding via minimax regret optimization with causality-awazd aware reinforcement learning. In Proceed- ings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 2314-2325, 2023.
Christopher M Bishop and Nasser M Nasrabadi. Puttern recognition and machine learning, volume 4, Springer, 2006.
Haozhe Wang, Jiale Zhou, and Xuming He. Learning context-aware task reasoning for efficient meta-reinforcement learning. arXiv preprint 2rXiv:2003.01373, 2020
Gemini Team, Petko Georgiev, Ving Ian Lei, Ryan Burnell, Libin Bai, Anmol Gulati, Garrett Tanzer, Damien Vincent, Zhufeng Pan, Shibo Wang, et al. Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context. arXiv preprint arXiv:2403.055300 2024b.
Gemma Team, Aishwarya Kamath, Johan Ferret, Shreya Pathak, Nino Vieillard, Ramona Merhej, Sarah Perrin, Tatiana Matejovicova, Alexandre Ramé, Morgane Rívière, et al. Gemma 3 technical report. arXiv preprint arXiv: 19786, 2025.
Xidong Wang. Dingjie Song, Shunian Chen, Chen Zhang, and Benyou Wang. Longllava: Scal- ing multi-modal Ilms 1000 images efficiently via hybrid architecture. arXiv preprint arXiv:2409.0280 0289999889, 2024.
Yushi Hu, Weijia Shi, Xingyu Fu, Dan Roth, Mari Ostendorf, Luke Zettlemoyer, Noah A Smith, and Ranjay Krishna. Visual sketchpad: Sketching as a visual chain of thought for multimodal language models. In The Thirty-eighth Annual Conference Neural Information Processing Systems, 2024a.
Jinliang Zheng, Jianxiong Li, Sijie Cheng, Yinan Zheng, Jiaming Li, Jihao Liu, Yu Liu, Jingjing Liu, and Xianyuan Zhan. Instruction-guider visual masking. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024.
Yushi Hu, Otilia Streteu, Chun-Ta Lu, Krishnamurthy Viswanathan, Kenji Hata, Enming Luo, Ranjay Krishna, and Ariel Fuxman. Visual program distillation: Distilling tools and programmatic reasoning into vision- models. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pages 5590-9601, 2024b.
Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. Advances in neural information processing systems, 36:34892-3490 2023.
Bo Li, Yuanhan Zhang. Dong Guo, Renrui Zhang, Feng Li, Hao Zhang, Kaichen Zhang, Peiyuan Zhang, Yanwei Li, Ziwei Liu, al. Llava-onevision: Easy visual task transfer. arXiv preprint wrXiv:2408.033266, 2024b.
Shuhao Gu, Jialing Zhang, Siyuan Zhou, Kevin Yu, Zhaohu Xing, Liangdong Wang, Zhou Cao, Jintao Jia, Zhuoyi Zhang, Yixuan Wang, et al. Infinity- mm:: Scaling multimodal performance with large-scale and high-quality instruction data. arXiv preprint arXiv:2410. 18558, 2024.
Jarvis Guo, Tuney Zheng, Yuelin Bai, Bo Li, Yubo Wang, King Zhu, Yizhi Li, Graham Neubig, Wenhu Chen, and Xiang Yue. Mammoth-vl: Eliciting multimodal reasoning with instruction tuning at scale. arXiv preprint arXivv24/2.05237 2024.
Yihe Deng, Hritik Bansal, Fan Yin, Nanyun Peng, Wei Wang, and Kai- Bei Chang. Openvithinker: An early exploration to complex vision-language reasoning via iterative self-improvement. arXiv preprint arXiv:2503. 17352, 2025.
Wenxuan Huang, Bohan Jia, Zijie Zhai, Shaosheng Cao, Zheyu Ye, Fei Zhao, Zhe Xu, Yao Hu, and Shaohui Lin. Vision-r1: Incentivizing reasoning capability in multimodal large language models, 2025.
06749
Ji Qi, Ming Ding, Weihan Wang, Yushi Bai, Qingsong L.v, Wenyi Hong, Bin Xu, Lei Hou, Juanzi Li, Yuxiao Dong, and Jie Tang. Cogcom: visual language model with of saaii-of--mniiulation reasoning, 2025, URL htthst:::: /ttp///xww..orr//////0/./... 04236
13

---

Aaron Jaech, Adam Kalai, Adam Lerer, Adam Richardson, Ahmed El-Kishky, Aiden Low, Alec Helyar, Aleksander Madry, Alex Beutel, Alex Cameyy, al. Openai ol system card. arXiv preprint 14..22222.1660,, 2024b.
14

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
We have double-checked that the claims accuratedly reflect them.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results
be expected to generalize other settings.
is fine to include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
We discuss limitations in conclusion and include a section for in the appendix.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification. asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low images are taken in low lighting. Or speech-to-text system might be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question:
For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
15

---

We detail the assumptions and problem formulation in Section 2. We provide derivations of constrained policy optimization in the appendix.
Justification:
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proof's can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: we include detailed information in the experiment section and in the appendix.
Guidelines:
The answer NA means that the paper does include experiments. If the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility, In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
16

---

Answer: [Yes]
Justification:
We will make code, data and models public.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.cc
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
/ //nips.cc/pbbll httoll//c/iiiee//oeuuibbiie
wwi.iiissooooooiioccooooo for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data,
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification:
We confirmed they are provided in experiment section and in the appendix.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core the paper to a level detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined other appropriate information about the statistical significance of the experiments?
Answer: [No]
Justification: Due the high computational cost of our experiments, we did not perform multiple runs and thus did not report eтror bars. Nonetheless, we used fixed seeds and consistent settings across experiments to ensure stable and representative results.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
17

---

It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative етror rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification:
We confirmed we include this information.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics //teur/p/////rpppp
Answer: [Yes]
Justification: We have reviewed and fully adhered to the NeurIPS Code of Ethics throughout our research process, including data usage, experimental design, and reporting. No ethical concerns were identified the course of this work.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. if there is special consid- eration due to laws or regulations their jurisdiction..
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: Our work has potential positive societal impacts by advancing the capabilitics of multimodal models, which could benefit applications such as assistive technologies and education. However, like other general-purpos. AI systems, also poses risks of misuse, such as generating misleading or harmful content. We encourage responsible use and open discussion regarding the deployment such technologies.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
18

---

Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), faireess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper deseribe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA].
Justification: Our work does not involve the release of pretrained language models, image generators, or scraped datasets that pose high risk of misuse. Therefore, no specific safeguards are required.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g.. code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: All datasets and code assets used in our work are properly cited in the paper, along with their respective licenses. We ensured compliance with the terms of use of each asset, and included license details where applicable.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
19

---

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paper swi thcode. com com/daaasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. If this information is available online, the authors encouraged to reach out the asset's s creators.
13. New assets
Question: Are assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: We introduce new assets in the form of code and data used in experiments. These assets are accompanied by clear documentation, including usage instructions, data schema descriptions, and license information.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/mmodel as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: Our work does not involve any research with human subjects crowdsourced participants.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: Our work does not involve any research with human subjects crowdsourced participants, and therefore IRB approval is not required.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
20

---

for Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if' applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, originality of the research, declaration is not required.
Question:
Answer: [Yes]
Justification:
We detail in Section 3 how we employ GPT4o to synthesize training data.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non-standard components.
Please refer our LLM policy what should or should not be described.
21

---

Appendix
A Limitations
In this work, pose fundamental rethinking of vision- langunlan reasoning, and introduce the concept of pixel-space reasoning. While we show the effectiveness of our approach to cultivating pixel-space reasoning, this improvement is still bottleneck-ed by limited data that spans actoss tasks and contents. In addition, we focus on two specific visual operations to handle primary media formats of images and videos. In the future, endeavor to include more visual operations and examine the effectiveness of pixel-space reasoning on more diverse collections tasks.
B Derivations of Curiosity-Driven Reward
The primary objective is maximize the expected correctness outcome, formalized constrained optimization problem. Let r(x, y) be the original correctness reward for a query x and response The policy generating responses denoted by (y
The optimization problem is:
[r(((x,))) r(xx
x) x~ D,y m
max 0
(6)
subject to
Ci(0:)) = RaPR(x) H2
(7)
C2(y) = lyo 0
where:
RaPR(x) = EE [Ip no (ylx) is the Rate of Pixel-space Reasoning for query x.
H is predefined minimum threshold for RaPR(x).
nyo (y) is the number of visual operations in response y.
N is a predefined upper bound on (y)
Constraint is expectation-level constraint for given query x, while constraint each individual response y.
applies to
To incorporate these constraints into the objective, a common technique is the method of Lagrangian Relaxation. For a maximization problem, this typically involves subtracting terms proportional to the constraint violations (when constraints r(x, y. If we rewrite our constraints as 91 (6: x) = H RaPR(x) S 0 and gy(y) (yo - N s 0, the standard Lagrangian modification to the per-instance reward would be:
written glz) < 0) from the original objective function
Lagrangian y, 0) r(x,y) - > (H - RaPR(x)) - Au (y)
where d1₁ A2 2 0 are Lagrange multipliers. The overall optimization objective would then be to maximize E ('Lagrangian (x, 8)] with respect to 0, and to minimize with respect the multipliers.
However, directly applying this standard formulation has two problems. Firstly, this formulation has an over-satisfaction issue. The term - ->2 (22 (y) N) would provide positive reward if ny.(y) < N (i.e., the constraint is "over-satisfied"", potentially encouraging the policy to use far fewer visual operations than necessary. Secondly, the term -A1 (H RaPR(x)) operates on the expectation-level and does not properly reward individual responses y ~ πe.
Therefore, we adopt the following modified reward function:
r' (x( y) r(x,y) + a max RaPR(x),0) IrR(y) + + min(N n,, 0)
(10)
where 0, 2 are fixed hyperparameters.
This formulation offers several benefits. Firstly, the clipping mechanism addresses the over- satisfaction issue while preserving equivalence to the original constrained objective (Wang et al. 2022). The clipping ensures the penalties are active only when the respective constraints are violated, otherwise the penalties zero, thus avoiding over-statisfaction.
22

---

Secondly, this structure allows a, 3 to be treated as fixed hyperparameters. In standard Lagrangian methods (Eq. D. multipliers are often dynamically adjusted: for example. Karush-Kuus Tucker (KKT) conditions imply that multipliers for inactive constraints (those satisfied with slack) are zero. The clipping zeros out the penalties when constraints are satisfied, thereby obviating the need for dynamic adjustment of ox / based on constraint satisfaction levels.
In addition, the inclusion of the indicator 1pR (y) converts the query-level expectation constraint into a response-level reward. Intuitively, this term acts as a targeted incentive: it rewards the specific behavior of engaging in pixel-space reasoning precisely when the average rate of such reasoning below the desired threshold. The multiplier 2 0 scales this incentive. It provides implicit penalty for missing out on the potential bonuses the policy could have earned by employing pixel-space reasoning.
C Data and Training Details
WEOng DROK
Seed Data
Localize Reference Vieual Cues
Expert Trajectory Synthesis
cboo, 24:
Keadycing sticee arager The rrage shows sounee street- Iccusso fer sig teartt...
citing roomt..
Anslycing shhul valoo The video shows
1u LEcLOIe mnn picking up food.
Sir mnjestery
<ay2gee chews soane the
sdoo siivis mon siting in
will soces 20AL1E005--map-
r00re Non well ftommss uedect Paan men peiding up sox....
prosped imagee foctoes on the whe.tiieeeeeee sigttoeed...
camage> image smoot. Now will zoom
tradenory
Se Delf-comatto Soljecttrv <ollecs The video shons u man sittkng in rocrs rooms./oow moappo The solecctd thmmee don't other
namos 20
Video
piospedt Image dheen't comain the targel, willt wmrgge The eroeced image rouubos on tic aggne card..
frant Jheee dijertaggoddmaeeeeeee The selected trarnes foooedss wan gick ng kood.
[15.4|
Figure 8: A detailed illustration of our data generation pipeline.
C.1
Protocols of Visual Operations
We include two primary visual operations: cropping an image and selecting frames from video.
This operation allows the model zoom in specific region of image by providing bounding box. The imput includes two-dimensional bounding box bbox_2d-2 list of numeric coordinates [z 31, 22, 22, V/l constrained within the image dimensions and target_image index indicating which image to operate on (indexed from 1, where 1 refers to the original image). This operation helps the model focus fine-grained details.
CropImage
SelectFrames This operation enables the model select subset of frames from video. The input target_frames a list of integer indices specifying which frames to extract from a 16-frame sequence, with limit of more than 8 frames. This allows the model to focus key temporal moments relevant to the query.
C.2 Instruction Tuning Data
Details of Seed Datasets We selected datasets based on two key attributes: high visual complexity requiring fine-grained analysis, and the presence of explicit annotations that can as targets anchors for visual operations. Based on these criteria, our data sources include:
SA1B Kirillov al. 2023 : large-scale dataset of high-resolution natural scenes offering rich
visual detail and complexity.
23

---

FineWeb [aa et al. 2024]: Consists of webpage screenshots paired with Question-Answering Answering
(QA) instances and preeise bounding box annotations for answer regions, offering explicit spatial targets for visual analysis.
STARQA [Wu et al. 2024]: Provides video data with QA pairs and annotated temporal windows indicating relevant visual contents for answers, offering both visual and temporal context for potential video-specific operations.
Detailed Data Pipeline Illustration. As the Fig. Tig.Bdde deppccs, after we obtain reference visual cues from seed data, we input both the whole HR image or video and the corresponding localized reference visual cues to gpt. Then we template- based method extract whole visual input analysis and local detailed analysis before we concatenate the whole analysis, localized reference visual cue and the partial analysis to form the single-pass trajectory. We utilize the reference visual cue get the wrong visual cues to insert in the obtained single-pass trajectories to get self-correction trajectory.
Single-pass and Self-correction Data Synthesis Details
Category
Trajectory Type
Proportion
single-pass
30%
Recrop once
20%
Image
Recrop twice
20%
Further zoom-in
30%
single-pas-
90%
Video
Reselect
10%
Table 2: Self-correction trajectory types and corresponding proportions.
Here single-pass means no error is inserted in the trajectory. Recrop once means we randomly select a bbox that has no intersection with the reference visual cue and insert before the correct visual operation. Recrop twice means we randomly select 2 bboxes that have intersection with the reference visual cue and insert them sequentially before the correct visual operation. Fur ther zoom-in means select inaccurate bbox that contains the reference visual cue but is excessively larger than it, and we insert it before the correct visual operation. Reselect means we sample frame indexes that have intersection with the reference visual cue's frame indexes, and insert it before the correct visual operation.
C.3 Training Details
Implementation Details. For Instruction Tuning, we adapt the Open-RI code to implement SFT loss with loss masks. For RL, we implement based on OpenRLHF. We adopt GRPO [DeepSeck-A] al. 2025) with selective sample replay [Wang al. 2025 because we witness significant issues of vanishing advantages. As shown in Fig. 9 our reward scheme incorporates curiosity bonus and efficiency penalty in addition to correctness rewards, which provides more variance in rewards. However, the ratio of queries that suffer from reward uniformity steadily increases to 90% as training progresses, leading to drastic plunge performance evidenced by the ratios of "response-ai all- incorrect" queries. During RL training, we employed a near on-policy RL paradigm, where the behavior policy was synchronized with the improvement policy after every 512 queries, which we define as episode. The replay buffer for SSR persisted for the duration of each episode before being cleared. For each query, we sampled 8 responses. The training batch size was set to 256 query-response pairs. Our 7B model is trained on 4 8 sets of A800 (80G) for 20 hours.
Training Hyperparameters. For Instruction Tuning, we use a batch size of 128. The learning rate is 1 with 10% warm up steps. For RL, we set employ cosine learning rate schedule with initial learning rate le and 3% warm up iterations. During RL training, we sample trajectories per training query and set hyperparameters to
0.5, 3 0.05, H 0.3, and 1. This configuration reflects our objectives: the threshold H = 0.3 encourages the policy to utilize pixel-space reasoning in approximately 30% of responses generated for a given query, while N promotes efficiency by favoring responses that require at most one visual operation. Under these parameters, response can receive maximum exploration bonus of approximately
24

---

resporues
05-
my
0.1
10
4.7
mw
0.301
-0.25
5.4
MM
-0.32
6.2
20
Steps 10251
Steps 10:.11
Stape 1x351
Figure 9: Trainig Dynamics of RL without SSR. The ratio of reward uniformity steadily saturates to 90%.
The video shows a person interacting with a table that has lamp and other objects it. The lamp turned on, indicated the bright light emits. The person adjusts the lamp and Interacts with other Items on the table. \nknNow some frames analyze.evisu. operations "name::: \'se frammfe"""'', t'argumentsy" ": (V (""arget_frames' [2, 4[)\\n=/visual operation
The person Interacts with lighting device among other objects. the lighting device on point? (B) yes (C) no
In the selected frames, the lamp is clearly the light visible and illuminating the scene. The person is seen interacting with the lamp, and the light prominent feature in the frames. Cormes./n\nnnboooo (bowee/b))))
Figure 10: Pixel RRasoner effectively captures key frames that are most informative for solving the query.
0.5 (0.3 1/8) x 0.0875, while each additional visual operation beyond the first incurs penalty of -0.05.
D Additional Analysis
D.1 Statistics
Owen2.5-VL-Instruun shows limited Zero-Shot Ability in utilizing novel visual operations. We include the protocols of visual operations in the system prompt for Qwen2.55VVL-Instmu Initially, invokes visual operations in 20.2% training rollouts, where 40.6% of them incurs error and 36.2% leads incorrect answer. This results 23.2% average accuracy when utilizing pixel-space reasoning. in contrast to 49.5% average accuracy when utilizing textual reasoning.
RaPR of Pixel- Mexel-Reasse Reasoner the Evaluation Benchmarks. Our Tixel-Reasoner adaptively triggers pixel-space reasoning with a portion of 78.53% on V-Star, 57.78% on TallyQA-Complex, 58.95% Infographies VQA, and 66.95% on MVBench.
D.2 Case Analysis
We show successful trajectories Fig. T1 and Fig. TO We also provide concrete examples to show the failure modes when the policy suffers from "the learning trap".
25

---

Understand the given visual information and the user query. Determine if In which movie does Tony Stark create an armour save his life?
The image is an infographic Uiled \'Everyhhin You Need to Know Before Watching Avengers: infinity War-Y" provides timeline Mervel Cinematic Universe, chronological order moviee, characters, a, and events. The intographio also highlights the Infinity Stones and their signifcance. New will zoom look clearer the section about Stark
The crapped Image focuses on the title and the beginning of the intographic. The section about TTooy Stark creating an armer to save his litel is not visible in this cropped image. noed to zoom further to get a clcarer view the relevant part cart.oviuull_ooe oop.ratoopoooooooooo tramel" Vicnop. Imagel: anguueents) * toox 20V: 11012. 300. 1387, 6001. Margget, Umagel </vis aot oporations
ATLINOAS
Rreualooperationc/ 'name h'arop_mmmag Image!. "argumente": (1 (V"bbok_2e0 201": ": (225, 100, 675, 300), Vttargt, Image\":ll=/vviaaa httpv/ilopppppaatiooooooo
EVERYTHING BEFORE
WICHIS
AVENUERS
The cropped image focuses on the Vtran Man'" section. The text states: \'Tony Stark creates armor to save his life, then evalving into Iron Man. The yoar 2008 also mentioned, which correspands the movie Y'rrnn Man. www''nnnnn/oonnnn Vacked]rro Mar)
RResoner effectively identifies the relevant region within the infographic and performs targeted analysis, either by refining the crop or directly answering the query.
Figure 11:
Failure mode 1: Hallucination. In the example below, the policy hallucinates as if the operations are successfully executed.
The video shows table with various items, including vegetables and bowl. The person is seen interacting with the vegetables, specifically placing them into the bowl. At one point, the person is seen shuffling some letters on the table. The video does not clearly show the sequence order of the letters throughout the video. Now I will select
frames look clearer the sequence of the letters.
Execution arror:max() arg is an empty sequence
The cropped video frames show the person continuing to prepare the salad and shuffle the letters. The letters appear be in different arrangement compared the beginning of the video. \n'n) \boxed(B)
Failure mode No-Reaction, the example below, the policy ignores the error message and proceeds with textual reasoning.
The video shows person in a kitchen. The person interacts with objects on the stove, including a pot. At one point, the person appears to handle an object on the counter and then moves away from the stove area. The sequence of actions suggests the person might have placed item down or completed an action involving the counter. \n\nNow will select some frames to look clearer at which object was put down by the person.
Execution error:max: m) is empty sequence
It seems there was an issue with selecting frames. Let me describe the scene instead: AnA nThe person in the video is standing near the stove with a pot on it. After interacting with the pot, the person moves to the counter area. It appears the person may have placed object (like box counter before moving away. The object is not visible the provided frames, but the sequence of actions suggests it was placed there. AnAn' Mboo\bDoxd((C)
/aper/notebook) down the
E Prompts
E.1
Prompts Used in RL Training
During RL training, we append the following prompt after the user query.
(question)
\nNnGuidelines:
26

---

it is beneficial to employ the given visual operations (tools). For video, can look closer by "eamect For an image, look closer by "crop_image". Reason with the visual information step by step, and put your final answer within Wboxed().
E.2 Prompts Used in Data Synthesis
E.2.1 Prompt for Question-answer Pair Generation for SA1B
Since SAIB lacks question-answer pairs and corrisponding annotations and some pictures SAIB have little content, prompt gpt-4o to first determine if the image is information-ric If yes, gpt-4o needs to use zoom-in tool to first crop small part the image, and then ask a question about objects in the small region. Otherwise, gpt-4o should reply Not valid. Here is the prompt for gpt-4o:
You are an expert in generating questions about small details in image. You will be given HR image. First determin if the image is information-rich image. If it is not, return 'Not valid'. If it is, choose small region both cropped image and whole image. Generate a question about objects in the small region. The question should be about the small object or its color, material. Also generate 4 choices. One of them is the correct answer. Others ask about the color of person's shoes, there should either be only one person or you specify which person you are referring to. Please make sure the object is small. Don't ask about questions related to the cropped image. For example, don't ask 'What is the color of the frame in the cropped image?' because the cropped image will not provided. Put the question in the following format: <question>
use crop image tool zoom According
wrong. should not be ambiguous. For example you
QUESTION HERE
</question>
Here is an example question:
<question>
question:What is the color of the person's 's shoes?
choices:
Red
B: Blue
C: Green
D: Yellow
correct_answer:
</question>
<question>
question:What is the child on the crosswalk holding?
choices:
Ice cream
Ball
C: Book
D: None
correct_answer:
</question>
Here is the tool description (tool_description). For each tool call, return json object with function name and arguments within <ttol_call><ttoo]caal XML tags:
<tool_call>
[["name": <function-name> "arguments": <args-json-objectt))
</tool_call>
Stop generating after call tool.
Here is the image.
E.2.2 Prompts for Expert Trajectory Synthesis
For SAIB dataset:
You expert in generating trajectories involving image cropping and answering
27

---

questions. You will be given an image and one cropped part of it and question. First, you need to briefly analyze the whole image, then generate: "Now will zoom in to look clearer at 'query object or text'." " Then you need to analyze the cropped part answer the question. Put your answer choice \boxed(}.
Here is example:
question: What the price mentioned for renting the single house?
choices:
A: 9, 9,000 Baht
B: 10,000 Baht
8,500 Baht
D: 12,000 Baht
Analyzing the whole image: The image shows lively street scene with people celebrating, possibly during festival. There is pickup truck with people on it, others walking around. signboard with text is visible in the background, which seems to contain information about renting or selling house.
Now I will zoom in look clearer the text the signboard.
Analyzing the cropped part: The cropped image focuses on the signboard. The text on the signboard mentions "SALE RENT SINGLE HOUSE" and specifies the price for renting as **9, 800 Baht**.
\boxed(A)
Here is the question, image and cropped part: [text)
For Fineweb dataset:
You expert in generating trajectories involving image cropping and answering questions. will be given an image and one cropped part it and a question. First you need to briefly analyze the whole image, then generate: "Now will zoon in to look clearer at the part about 'query'. Then you need to analyze the cropped part and answer the question. Put your answer in \boxed[]. Final answer should be text from article. Don't 't change the original text include irrelevant text from the article. answer should be one sentence.
Here are some examples:
question: What the key responsibilities of leader?
Analyzing the whole image: The document appears to be an article titled "Top 7 Skills Leadership Training Should Teach Managers." It discusses various aspects of leadership training, including leadership essentials, change management, performance coaching,
conflict management. The article emphasizes the importance of leadership skills in managing teams effectively.
Now I will zoom in to look clearer at the part about "key responsibilities of leader."
Analyzing the cropped part: The cropped part focuses on "Leadership Essentials," which outlines the basics of leadership, including understanding the role of leader and the key responsibilities of leader.
\boxed(building relationships, setting expectations, delegation, and developing a goal-oriented approach.)
question: Who the first SEC championship in football?
Analyzing the whole image: The document is Wikipedia article titled "SEC Championship Game." It provides an overview of the Southeastern Conference (SEC) Football Championship Game, including its history, format, results, and notable
28

---

moments. The article also includes a table summarizing the results of all SEC Championship games since its inception in 1992.
Now I will zoom in to look clearer at the part about "who won the first SEC championship in football."
Analyzing the cropped part: The cropped section includes table of results from Championship games. The first game, held in 1992, lists Alabama
defeating #12 Florida with a score of 28-21 at Legion Field in Birmingham, Alabama.
(babem(Alabama)
Here is the question and image: [text]
For STARQA dataset:
You are expert in generating trajectories involving frame selection and answering questions. You will be given 16 images (video frames) in chronological order and several selected frames from them and question. First you need to briefly analyze the whole video, then generate: "Now will select some frames to look clearer at 'query object or text'." Then you need to analyze the selected frames and answer the question. Put your answer choice in \boxed().
Here are some examples:
question: why did the woman take the measuring spoons away from the boy?
choices:
A: do not need it anymore feeding
C: finish eating the piece
D: so can take picture
E: wants to play with it
Analyzing the video:
The video shows a woman and a boy in a kitchen setting. The boy is sitting on the counter, holding measuring spoons, while the woman appears to be engaged in baking cooking activity. The woman interacts with the boy, guiding him as they work with ingredients like flour and eggs. Toward the end, woman takes the measuring spoons away from the boy.
Now I will select some frames to look clearer at why the woman took the measuring spoons away from the boy.
Analyzing the selected frames:
In the selected frames, the woman is seen taking the measuring spoons from the boy. The boy appears to have finished using the spoons add ingredients to the bowl. The woman likely takes the spoons proceed with the next step the cooking process.
\boxed(A]
question: Which object was put down by the person?
choices:
A: The cup/glass/bottte.
B: The clothes.
C: The bag.
The book
Analyzing the video:
The video shows person entering room and sitting at table. The person appears to be holding sandwich and book. She places the book the table, eats the sandwich, and then picks up the book again to read. Toward the end of the video, the person leaves the table, leaving the book behind.
Now I will select frames look clearer which object put down the
29

---

person.
Analyzing the selected frames:
In the selected frames, the person is seen entering the room holding a sandwich and a book. She places the book on the table before eating the sandwich. The book remains on the table the person continues her activity and eventually leaves the room.
\boxed(D)
Here is the question and video: (text)
30


Reviews:
[{"review_id": "Xfm7UWpOG2", "reviewer": "Reviewer_wysE", "summary": "The paper introduces Pixel Reasoner, a novel VLM framework that enables pixel-space reasoning, meaning the model actively performs visual operations (like zooming in or selecting frames) as part of its reasoning process, rather than relying exclusively on CoT reasoning. The proposed method first synthesizes expert visual reasoning trajectories, including self-correction paths, to teach the model how and when to use visual operations effectively. Then, curiosity-driven RL employs a carefully designed reward structure to overcome the inherent bias of the model toward text-based reasoning. The reward encourages the use of visual operations while penalizing excessive or redundant operations, mitigating what the paper term a \"learning trap\", i.e., the model’s tendency to default to textual reasoning because it is initially more proficient.", "questions": "See weaknesses above. Not sure whether this is non-trivial, but could the authors provide more analysis or statistics on how often pixel-space reasoning is truly necessary versus when it is artificially invoked to meet the rate constraint? Or perhaps check what happens in examples from existing benchmarks in which pixel reasoning is not necessary?", "limitations": "A limitations section exists, but I could not find a broader impacts section.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n- The concept of pixel-space reasoning is a significant and timely contribution that opens new directions in VLM reasoning.\n- The paper is methodologically rigorous and combines instruction tuning with curiosity-driven reinforcement learning.\n- The experiments are comprehensive, covering multiple benchmarks, and the performance gains against Qwen2.5-VL-7B are consistent. The demonstrated performance, especially surpassing most of the open-source and proprietary models, indicates that the proposed methods meaningfully improve practical VLM capabilities.\n\nWeaknesses:\n- The scope of visual operations is currently narrow. While the framework is extensible, the demonstrated significance is somewhat confined to the selected benchmarks and the two operations presented.\n- Some components, such as curiosity-driven exploration and instruction tuning, are adapted from existing RL literature and VLM fine-tuning pipelines with no clear technical contributions, though the paper clearly motivates these design choices.\n- The SA1B dataset, while large and visually diverse, is inherently noisy because it is generated through automatic segmentation at scale. For example, it is not clear whether the fine-grained visual cues used in the instruction tuning phase are consistently reliable. Although the work attempts to mitigate this by filtering for high-information regions using GPT-4o, this can similarly introduce biases or systematically exclude challenging cases. There is a risk that the model might learn brittle or superficial strategies with limited generalization. \n- Could the fixed-rate constraint on pixel-space reasoning (RaPR) lead to over-regularization? If there exist queries that do not require pixel-space reasoning at all, yet the constraint enforces a minimum rate across the dataset, then the model may superficially boost RaPR to meet the target without genuinely improving reasoning quality, e.g., by adding visual operations even when they are irrelevant.\n- Will the data be released? The submission did not contain any data samples for reviewers to assess data quality.\n- In Table 1, some cells have no reported numbers. While it is reasonable in some cases, the Models with Tools have results for only one benchmark, which makes it challenging to evaluate the performance gains against these baselines. Also, SEAL [Wu and Xie, 2024] h is not included as a baseline, and related work does not explain the differences of this work with existing visual search models. \n- The absence of qualitative examples in the main paper is striking. Given the paper’s focus on a new reasoning modality, it would be valuable to see more concrete reasoning trajectories that illustrate when pixel-space reasoning is essential, when it fails, and when it might be unnecessarily invoked. \n\nComments:\n- Figure 3 fonts are too small. An actual, fully developed example would help understand the dataset better. Figure 8 is also unreadable.\n- Figure 7 highlights that pixel-space reasoning does not naturally emerge as a dominant strategy without curiosity-driven incentives. This suggests that the pixel-space reasoning capability is not intrinsically self-reinforcing. This raises an intriguing question: can we develop more intrinsic mechanisms, where the model autonomously values pixel-space operations based on their long-term utility or epistemic uncertainty, rather than needing hard-coded curiosity bonuses? Such directions could potentially lead to more robust, adaptive, and agent-like reasoning behaviors in future vision-language models.", "was_revised": true, "final_justification": "The rebuttal satisfactorily addressed all major concerns. The authors clarified the reasoning behind the selected set of visual operations, provided a detailed explanation of the RaPR (dynamic) mechanism, and resolved all concerns regarding baselines and data quality. My final recommendation has been updated to reflect the novelty and significance of operationalizing pixel-space reasoning, supported by the clear author responses.", "rebuttal": "We sincerely thank the reviewer for their detailed, insightful, and constructive feedback. We are encouraged that they recognized our work as a \"significant and timely contribution\" and appreciated the rigor of our methodology and the meaningful performance gains our model achieves.\n\nWe would like to begin by highlighting what we see as the core contribution of this work: to formally identify, operationalize, and incentivize pixel-space reasoning. We believe this is a distinctive capability that underpins the powerful visual analysis seen in frontier models like OpenAI o3. Our paper reveals a key challenge in acquiring this new ability and proposes an effective framework to overcome it. We will frame our responses to the specific points below with this central contribution in mind.\n\n\n### **Addressing Weaknesses**\n\n**1. On the Scope of Visual Operations:**\nWe agree with the reviewer that the current set of visual operations is focused, a point we also acknowledged in the limitation section. This was a deliberate choice for this initial work, and we wish to clarify our rationale:\n- The space of potential visual operations is vast (e.g., rotation, line drawing, segmentation, and more). It is neither feasible nor necessarily productive to try and enumerate and implement them all.\n- Rather than attempting an exhaustive implementation, we selected ZOOM-IN and SELECT-FRAME as they are fundamental and broadly representative operations for interacting with static images and dynamic videos, respectively. These two operations cover the most of the use cases. The other operations like segmentation, etc are not contributing much values to our primary goal, i.e., to show the importance of the pixel-space reasoning paradigm.\n- Our framework is designed to be extensible. Based on the current core set, we envision future work designing bespoke operations for specific downstream tasks, such as line drawing for geometric problem-solving or fine-grained segmentation for autonomous driving, etc.\n\nTherefore, while the scope of the demonstrated operations is focused, we believe this does not detract from the paper's main contribution: revealing the importance and challenges of pixel-space reasoning and providing a viable method to unlock this capability in VLMs.\n\n**2. On the Technical Contribution of the Training Pipeline:**\nWe appreciate the reviewer's point that we indeed \"stand on the shoulders of giants\". We will refine the contribution stated in line 83 by clarifying that we innovatively address the challenges of learning pixel-space reasoning by proposing the approach, but not directly inventing instruction-tuning from scratch.\n\n**3. On the Quality of the SA1B Dataset and Potential for Brittle Strategies:**\nWe thank the reviewer for raising this important point about data quality. We share the concern regarding the potential noise in automatically generated datasets and took several deliberate steps to mitigate this risk, which we would like to clarify:\n- **Diverse Data Sources:** We did not rely solely on SA1B. Our instruction tuning data is curated from a variety of sources, including FineWeb and STARQA, to provide rich textual and contextual diversity (Lines 117-119).\n- **Controlled Synthesis:** Our data generation process utilizes a template-based synthesis approach (Figure 3, Lines 137-141). This provides strong structural control over the reasoning traces, ensuring that the generated visual operations are logically sound and contextually necessary for the task. This moves beyond simple noisy labels and enforces a coherent reasoning structure.\n- **Empirical Generalization:** The strongest evidence against the model learning brittle strategies lies in its performance. Our model demonstrates consistent and significant generalization across a wide range of benchmarks and media formats (images and videos), many of which have distributions distinct from the training data.\n\nThese factors together ensure that the model learns robust and generalizable reasoning skills rather than superficial shortcuts. We thank the reviewer for highlighting this concern, and we will consider more measures to facilitate data quality assurance.\n\n**4. On the Fixed-Rate Constraint (RaPR) and Potential Over-Regularization:**\nThis is a crucial question, and we thank the reviewer for giving us the opportunity to clarify a nuanced but central aspect of our method. There may be a misunderstanding that the `RaPR` constraint is a rigid, dataset-wide mandate. In fact, it is a dynamic reward bonus to that monitors at the query level and rewards at the response level. \n\nHere is a more precise breakdown of how it functions:\n- RaPR monitors the query-level exploration. It tracks the proportion of rollouts that uses visual operations within the same query group. \n- A curiosity bonus is added to a response's reward if and only if two conditions are met: (1) the response contains a visual operation, and (2) the current query-level RaPR is too low, i.e., below the target ratio.\n- **The model is never forced to use a visual operation.** On the one hand, if the model determines that visual operations are not beneficial for a query and consequently explores no such actions in its rollouts, no penalty is incurred. On the other hand, if the model finds visual operations useful occasionally and its exploration satisfies the curiosity, no further reward bonus is given, thus preventing the model from adding gratuitous operations.\n\n- The curiosity bonus acts as a temporary learning scaffold, primarily to help the model escape the initial \"learning trap.\" As Figure 6 shows, this reward signal diminishes as the model learns the intrinsic value of the actions. The primary reward for reasoning correctness always remains dominant.\n\nTherefore, there is no permanent mandate that could lead to over-regularization. We will revise the methodology section to make this dynamic, query-level exploration mechanism clearer to prevent misunderstanding.\n\n\n**5. On Data Release:**\nWe are fully committed to open science. The data,code,models were released (as stated in Line 253), but rebuttal disallow any form of link sharing. So we will share the link after the paper decision is out on the Openreview platform. We also appreciate the suggestion to include data samples and will add representative examples to the appendix in our revision.\n\n**6. On Table 1 and Missing Baselines:**\n- **SEAL Baseline:** We thank the reviewer for this point and would like to gently clarify that SEAL is included in Table 1 and cited in our related work (Line 355). To improve clarity, we will explicitly name \"SEAL\" in our discussion of \"visual guided search\" in Line 355 of the related work section.\n- **Missing Tool-Use Model Results:** The empty cells for some tool-using models exist because the original papers for these models did not report performance on those specific benchmarks. Adapting these highly specialized visual search models to the diverse reasoning benchmarks we use is a significant engineering effort beyond the scope of this work. We believe the current table provides a fair comparison on the visual search benchmark (V-Star) where results are available.\n- **Distinction from tool-using models:** Our work focuses on enabling a new reasoning paradigm where the model directly interacts with visual inputs to guide its internal thought process. This is distinct from direct tool-use or visual search, where the goal is typically to retrieve external information. The objective of our visual operations is to improve the quality of reasoning itself. We will clarify this distinction further in the related work section.\n\n\n**7. On the Absence of Qualitative Examples in the main paper:**\nWe agree that qualitative examples are essential for a paper introducing a new reasoning ability. Due to page constraints, we placed these examples, including both successes and failures, in the appendix. In our revision, we will restructure the paper to move several key qualitative examples into the main body for better visibility.\n\n### **Addressing Comments and Questions**\n\n* **Figure Readability:** We agree and will revise Figures 3 and 8 for the final version to improve readability.\n\n* **Intrinsic Mechanisms for Pixel-Space Reasoning:** We are grateful for this insightful and thought-provoking question. The reviewer's observation that pixel-space reasoning is not \"intrinsically self-reinforcing\" perfectly captures the motivation behind our work. This is precisely the \"learning trap\" we aimed to solve. Our use of curiosity-driven RL is a pragmatic and effective method to provide an initial \"intrinsic motivation\" and bootstrap the learning process, allowing the model to discover the long-term utility of these actions.\n\nWe wholeheartedly agree that developing more sophisticated intrinsic mechanisms is an exciting and important frontier for research. We believe our work is a crucial first step in this direction. We will add a discussion of this promising future direction to our conclusion to inspire further research in the community.\n\n* **Broader Impacts Section:** We apologize for the confusion. A broader impacts statement was included in the NeurIPS checklist (Lines 703-710). We will add a formal \"Broader Impacts\" section to the appendix for better visibility.\n\nWe thank the reviewer again for their time and their thoughtful, high-quality feedback. This review has provided us with clear, actionable guidance that will undoubtedly help us improve the quality and clarity of our paper."}, {"review_id": "ItCbqiTFzT", "reviewer": "Reviewer_1tA7", "summary": "This paper presents pixel reasoner, a framework to train multimodal LLM to do complex multimodal reasoning with two tools: zoom-in and select-frame. The training recipe contains two stages, SFT and RL. Specifically, the authors tackle numerous challenges they face and propose novel and effective solutions for each stage, including the warm-start instruction tuning and the curiosity-driven RL. The model achieves great results on V*, TallyQA, MVBench, and InfographicVQA, outperforming all other open-source models. Extensive ablation experiments show the effectiveness of each step in the training recipe.", "questions": "1. What is the number of tokens & tool-calls per query for each task? It is interesting to see these methods' efficiency trade-offs.\n\n2. What other operations besides zoom-in and select-frame might be useful for multimodal reasoning? How would you change the algorithm if there are more tools?", "limitations": "yes", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 4, "significance": 4, "originality": 3, "strengths_and_weaknesses": "Strength:\n1. The motivation is clear, and the method is effective.\n2. The meticulous instruction tuning stage and the RL stage are very informative to other researchers. The paper is well-written and clearly shows the importance of these tricks and the mathematical intuition behind them.\n3. SOTA performance on a wide range of challenging tasks.\n\nWeakness:\nIn Table 1, the VPD model numbers are for TallyQA, not V*. Also, they are not tool-use models (they are trained with tool-use trajectories). The authors should correct these.", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their valuable feedback and insightful questions. \n\n\n**1. Efficiency Trade-offs (Tokens & Visual Operations)**\n\nThank you for this excellent question about the efficiency of our method. We reported RaPR (rate of pixel-space reasoning) and tokens per query-response for each benchmark to clarify the computational trade-offs.\n\n| Benchmark | RaPR | Avg. Tokens |\n| :--- | :--- | :--- |\n| V* (Visual Search) | 0.78 (150/191) |  153.8 |\n| TallyQA (Visual Counting) | 0.43 (1389/3200) | 151.2 |\n| MV-Bench (Video Reasoning) | 0.94 (3802/4032) |  215.4 |\n| InfographicsVQA (Doc Reasoning) | 0.21 (593/2816) |  161.5 |\n\nAs shown, tasks requiring fine-grained detail, like video analysis (MV-Bench) and visual search (V*), trigger visual operations most frequently. In contrast, tasks like InfographicsVQA, where the evidence is often easier to found in the single page infographics, have a much lower rate of visual opeartion. These observations also demonstrate PixelReasoner's ability to allocate computation efficiently and autonomously.\n\n**2. Extending with More Visual Operations**\n\nThis is a great question about the extensibility of our framework. Additional operations like **line-drawing** for geometric problems, **region highlighting/masking** for grounded reasoning, or **optical character recognition (OCR)** for document understanding would be highly beneficial. Our framework is designed to be easily extensible to new tools through three core mechanisms:\n\n* **Skill Acquisition:** New operations are incorporated by fine-tuning the model on instruction-following datasets that demonstrate the tool's syntax and function.\n* **Agnostic Exploration:** Our curiosity-driven reward mechanism is tool-agnostic. It naturally encourages the model to explore when and how a new tool can be used to solve a task, without needing manual reprogramming.\n* **Task-Specific Rewards:** The primary adaptation is in the task-success reward function. For example, a GUI navigation task could use an **Intersection-over-Union (IoU)** reward for object grounding, while a document task might use a **layout-aware edit distance**. This modular design allows our method to easily incorporate new operations for a wide range of multimodal problems.\n\n\n**3. Correction in Table 1**\nWe sincerely thank the reviewer for identifying the error in Table 1 and the imprecise description of the VPD model. We will correct them accordingly in our revision."}, {"review_id": "cYOXXJWvqe", "reviewer": "Reviewer_RbYh", "summary": "This paper proposes an interesting idea: leveraging pixel-space reasoning (via visual operations) to improve visual reasoning capabilities when tackling complex visual understanding tasks such as InfographicsVQA. The approach is concise and well-motivated. However, several important technical details are missing, making it hard to follow. The authors are encouraged to improve the writing and evaluate the proposed method on more challenging benchmarks—such as ChartQAPro, which emphasizes visual reasoning.", "questions": "Please carefully address the concerns raised in the “Weaknesses” section above.", "limitations": "No", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "**Strengths:**\n\n* The idea of pixel-level reasoning via a set of image operations is reasonable and appears to be inspired by a recent blog post from OpenAI \\[1]. The authors should cite this blog and credit the original source accordingly.\n\n* The proposed synthetic dataset also represents a valid contribution to the research community.\n\n\\[1] [https://openai.com/index/thinking-with-images/](https://openai.com/index/thinking-with-images/)\n\n---\n\n**Weaknesses:**\n\n1. **Limited experimental results:**\n   The primary concern is that the authors fail to demonstrate the effectiveness of the proposed approach on tasks requiring highly accurate visual reasoning, such as the ChartQAPro benchmark.\n\n   \\[1] *ChartQAPro: A More Diverse and Challenging Benchmark for Chart Question Answering*, arXiv:2504.05506\n\n2. **Lack of discussion on the limitations of the synthetic dataset:**\n   The reviewer is also concerned about how the authors plan to construct a scalable visual reasoning dataset for such a challenging task.\n\n3. **Insufficient analysis of the 'learning trap':**\n   As one of the core contributions of the paper, the concept of a 'learning trap' lacks systematic theoretical analysis and deeper discussion.\n\n4. **Concerns regarding ablation studies:**\n   The ablation results show that the error-correcting data significantly improves performance. However, it remains unclear why this data enhances the model’s visual reasoning or tool invocation capabilities. Further analysis is strongly encouraged.\n\n5. **Advantages of using RL for pixel reasoning**. The necessity or reason for using RL should be further explained. Does the model's pixel-space reasoning capability necessarily require learning through RL, or is RL a more efficient approach?", "was_revised": true, "final_justification": "The authors fail to address the raised major concerns, such as limited experimental results.", "rebuttal": "We sincerely thank the reviewer for constructive suggestions. Below, we address the reviewer's specific questions.\n\n**1. Regarding the inclusion of the ChartQAPro benchmark.**\n\nWe thank the reviewer for bringing the interesting ChartQAPro benchmark to our attention. We agree that evaluating our method on such tasks is a valuable direction. However, we did not include it in the current submission for two primary reasons:\n\n* **Timing:** The ChartQAPro benchmark was released in mid-April 2025. Given the NeurIPS submission deadline in May, we didn't know about this new work at the moment of submission.\n* **Scope:** As we state in our paper (L34, L85), our primary research goal is to demonstrate the significance of pixel-space reasoning in visually-intensive, general-purpose scenarios. We therefore prioritized benchmarks like InfographicsVQA, V-Star, MVBench, which include visually-rich scenarios like videos, high-resolution scenes and infographics.\n\n**2. Regarding the limitations of the synthetic dataset.**\n\nWe appreciate the reviewer's insightful comment on the limitations and scalability of our synthetic dataset. We acknowledge that its current form is focused on demonstrating the core capabilities on visually-rich tasks and does not yet cover complex, multi-step numerical or logical reasoning often required by chart-based VQA.\n\nTo address this and to outline a path toward a more comprehensive dataset, we will add a discussion on this limitation to the paper. We will also propose a concrete pipeline for extending our synthetic data generation to tasks like ChartQA, which includes the following steps:\n1.  Leverage a state-of-the-art VLM (e.g., GPT-4o) to identify key visual elements and tabular data within a chart.\n2.  Generate question-answer pairs that require targeted reasoning about these identified visual cues.\n3.  Synthesize both expert demonstration and self-correction trajectories based on these generated pairs, teaching the model to ground its reasoning in specific visual regions.\n\n**3. Regarding the analysis of the 'learning trap'.**\n\nWe thank the reviewer for highlighting the need for a deeper analysis on the 'learning trap'. \n\nIn our current work, we provided a deep empirical analysis of this \"learning trap\" problem in Section 5.2. Our experiments demonstrate that without instruction tuning and intrinsic rewards, the model defaults to its well-established textual reasoning abilities and fails to effectively learn the novel pixel-space reasoning skill, which is precisely the \"learning trap\" problem. We also show that the fundamental reason for this problem is the significant proficiency gap between the two capabilities.\n\nTo make this clearer, we will revise the manuscript to explicitly frame Section 5.2 as an empirical investigation into the 'learning trap'. \n\n**4. Regarding the ablation study on self-correction data.**\n\nWe thank the reviewer for raising this point, as it gives us the opportunity to clarify the precise role of the self-correction data. We believe the potential confusion arises from our choice to use a negative framing in the paper (L302). To emphasize the data's impact, we described the model’s failures without it, rather than its benefits with it.\n\nTo state the mechanism directly and positively: the self-correction trajectories are designed to teach the model to handle errors from the visual operations. When a visual operation yields an unexpected or erroneous outcome, this data provides explicit demonstrations of how to proceed with a corrective action, rather than leading to model's bypassing of these errors from visual operations. \n\nIn our revision, we will rephrase this section to state this positive mechanism more explicitly, ensuring the contribution of the self-correction data is unambiguous.\n\n**5. Regarding the advantages of using RL for pixel-space reasoning.**\n\nWe thank the reviewer for requesting a more thorough justification for using Reinforcement Learning (RL). As we note in L52, our approach follows the standard paradigm of post-training with instruction tuning and RL to instill new, complex abilities. The reason this two-stage process is necessary is that instruction tuning alone is insufficient for this task.\n\nAs our results in Table 1 empirically demonstrate, a model trained with only instruction tuning is fragile and struggles to generalize. RL is not just a means of making learning more efficient; it is *necessary* for developing a robust reasoning policy. RL enables the model to explore the vast space of possible visual operations and learn to assign credit to actions over long sequences. The intrinsic reward signal encourages the model to move beyond simply mimicking expert demonstrations and to discover for itself which policies lead to correct answers, making it more resilient to novel situations not seen during supervised fine-tuning. We will clarify this motivation in the revised manuscript."}, {"review_id": "t0XP6b2Zv8", "reviewer": "Reviewer_jxhz", "summary": "This paper introduces the concept of pixel-space reasoning, a novel paradigm for Vision-Language Models (VLMs) that enables direct interaction with visual inputs through operations such as zoom-in and select-frame. Unlike traditional chain-of-thought reasoning confined to textual space, this approach allows VLMs to inspect and manipulate visual content directly during the reasoning process, improving their ability to handle visually intensive tasks. To overcome challenges in training VLMs to effectively use these new capabilities—such as imbalanced proficiency between text-based and visual reasoning—the authors propose a two-phase training framework: (1) warm-start instruction tuning using synthesized reasoning traces, followed by (2) curiosity-driven reinforcement learning (RL) that incentivizes exploration of pixel-space reasoning while maintaining efficiency.", "questions": "1. Generalization Beyond Zoom-In and Select-Frame \n\nThe paper demonstrates success using zoom-in and select-frame operations. How would the framework perform if extended to include additional visual operations (e.g., drawing, color filtering, object highlighting)? Are there plans or preliminary results indicating the scalability of the method to more complex visual tools?\n\n2. Error Analysis and Failure Cases \n\nThe paper includes some analysis of failure rates in visual operation execution. Could you provide concrete examples of cases where the model failed to utilize pixel-space reasoning effectively, and what insights these failures offer for future improvements?", "limitations": "The authors have adequately addressed the limitations of their work. They acknowledge the current restriction to two visual operations and suggest that the framework is easily extensible. They also discuss the need for further research into richer visual operations and broader task coverage. However, the paper could benefit from a more detailed discussion of failure cases.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "## Strengths\n- Originality \n\nThe core idea of pixel-space reasoning is novel and addresses a clear gap in current VLMs—namely, their reliance on textual reasoning even when visual information is critical.\n\n- Clarity \nThe paper is well-written and structured, making it easy to follow the problem formulation, methodology, and results.\nFigures and examples are used effectively to illustrate the reasoning process and training dynamics.\n\n## Weaknesses:\n\n- Limited Exploration of Visual Operations \n\nThe study currently focuses only on zoom-in and select-frame operations. While these are useful, broader visual manipulations (e.g., drawing, masking, depth estimation) could further enhance the model's capabilities and generalization.", "was_revised": true, "final_justification": "borderline accept", "rebuttal": "We thank you for your thoughtful questions and constructive feedback. Below, we address the questions.\n\n**1. On Generalization Beyond Zoom-In and Select-Frame**\n\nWe thank the reviewer for requesting us for clarification on this crucial question. We agree that the current set of operations is focused, a deliberate choice we acknowledge in the paper's limitations section. We would like to clarify our rationale.\n\n1.  **Rationale for a Focused Scope:** The space of potential visual operations is vast, including rotation, drawing, filtering, segmentation, and more. Rather than attempting an exhaustive implementation, which is neither feasible nor necessarily productive for an initial investigation, we strategically selected **ZOOM-IN** and **SELECT-FRAME**. These were chosen because they are fundamental primitives for interacting with static images and dynamic videos, respectively. This focused set allowed us to rigorously study the core challenges of the pixel-space reasoning paradigm in a controlled yet broadly representative setting. These two operations cover the most of the use cases. The other operations like segmentation, etc are not contributing much values to our primary goal, i.e., to first establish the importance and viability of this paradigm.\n\n2.  **Extensibility of the Framework:** Our framework is designed from the ground up to be extensible. We envision future work building upon this foundation by designing bespoke operations tailored to specific downstream tasks (e.g., line drawing for geometric problem-solving, or fine-grained segmentation for autonomous driving). The core mechanisms for incorporating new operations would remain consistent:\n    * **Skill Acquisition via Instruction Tuning:** The model would be fine-tuned on instruction-following datasets to learn the syntax and semantics of any new visual operation.\n    * **Exploration via Curiosity:** The curiosity-based reward mechanism is agnostic to the specific operation and would continue to encourage the model to explore the utility of the new tool.\n    * **Task-Specific Rewards:** The primary adaptation would be in the task-success reward function. For example, extending to a GUI grounding task would involve incorporating an Intersection-over-Union ($IoU$) reward, while a document understanding task might require a novel layout-aware reward metric.\n\nIn summary, while the scope of the *demonstrated* operations is focused, this does not detract from the paper's main contribution: revealing the importance of pixel-space reasoning and providing a viable, scalable method to unlock this capability in VLMs.\n\n\n**2. On Error Analysis and Failure Cases**\n\nWe appreciate the reviewer's request for a more detailed discussion of failure cases. As noted, we provide a qualitative analysis in Appendix D.2. The primary failure modes observed were:\n \n1.  **Hallucination:** In some cases, the model pretends the visual operation is performed successfully and hallucinate with a conclusion and continue its reasoning.\n\n2.  **No-Reaction:** Another failure mode is the model's inability to properly react to the errors from visual operations. The model noted the errors and then proceed with textual reasoning without attempting to fix the errors.\n\n**Insights and Future Work:** \nThis paper is a preliminary study of pixel-space reasoning and it follows the common setting of using only outcome rewards. A key insight from analyzing these failures is that **intermediate supervision** could benefit more accurate and reliable pixel-space reasoning. Currently, the model relies primarily on the outcome reward, so the correctness of intermediate steps, e.g., correcting errors from visual operations, will not be properly encouraged. This points to a clear path for future improvement: developing methods to provide denser supervisory signals for the intermediate pixel-space reasoning steps. This could involve designing reward shaping for handling errors from visual operations, or providing more signals for the correctness of visual operations, e.g., rewarding the grounding of a relevant visual cues."}]

Author rebuttal:

Meta-review: The paper proposes pixel-space reasoning for VLMs, showing that models can perform visual operations like zoom-in or select-frame as part of their reasoning, and introduces a two-stage training process combining instruction tuning with curiosity-driven RL. The main strengths highlighted by the reviewers are the novelty of the pixel-space reasoning paradigm, the strong empirical results across diverse benchmarks, the methodological rigor of combining instruction tuning with RL to overcome the learning trap, and the overall clarity and presentation. Weaknesses raised include the narrow scope of visual operations, the lack of deeper theoretical analysis of the learning trap, concerns about data quality in synthetic traces, missing baselines, and a need for more qualitative examples. Reviewers also asked for more results on benchmarks like ChartQAPro and deeper discussion on efficiency trade-offs and failure modes.

During rebuttal, the authors provided detailed clarifications on the curiosity bonus mechanism, explained the extensibility of visual operations, and committed to correcting errors in baselines and tables. They also added new experimental results on ChartQAPro showing strong open-source performance and competitive generalization against proprietary models, which addressed the main criticism of limited evaluation. Some reviewers remained cautious about the scope and dataset limitations, but they acknowledged that the responses were clear and that the additional experiments strengthened the contribution. Overall, the consensus after rebuttal is that the novelty, methodological soundness, and strong empirical evidence outweigh the weaknesses. I recommend accept.
Decision: Accept (poster)