ProxyThinker: Test-Time Guidance through Small Visual Reasoners
Authors: Zilin Xiao
Jaywon Koo
Siru Ouyang
Jefferson Hernandez
Yu Meng
Vicente Ordonez
Venue: ICLR 2026 Poster (2026)
Keywords: decoding-time algorithms
visual reasoning
Primary area: applications to computer vision, audio, language, and other modalities
TL;DR: ProxyThinker enables large vision-language models to inherit slow-thinking visual reasoning skills from smaller RFT models at inference time.

Abstract: Recent advancements in reinforcement learning with verifiable rewards have pushed the boundaries of the visual reasoning capabilities in large vision-language models (LVLMs). However, training LVLMs with reinforcement fine-tuning (RFT) is computationally expensive, posing a significant challenge to scaling model size. In this work, we propose ProxyThinker, an inference-time technique that enables large models to inherit the visual reasoning capabilities from small, slow-thinking visual reasoners without any training. By subtracting the output distributions of base models from those of RFT reasoners, ProxyThinker modifies the decoding dynamics and successfully elicits the slow-thinking reasoning demonstrated by the emerged sophisticated behaviors, such as self-verification and self-correction. ProxyThinker consistently boosts performance on challenging visual benchmarks on spatial, mathematical, and multidisciplinary reasoning, enabling untuned base models to compete with the performance of their full-scale RFT counterparts. Furthermore, our implementation efficiently coordinates multiple language models with parallelism techniques and achieves faster inference compared to previous decoding-time methods, paving the way for the practical deployment of ProxyThinker. Code is available at https://github.com/MrZilinXiao/ProxyThinker.

Published as a conference paper at ICLR 2026
PROXYTHINKER: TEST-TIME GUIDANCE THROUGH SMALL VISUAL REASONERS
Zilin Xiao Jaywon K Siru Ouyang I Jefferson Hernandez', Yu Meng?, Vicente Ordonez' Rice University UUniversity of Illinois Sirbana-Champaae University of Virginia {zilin, vicenteor) wrieeererrrieeeeeeee
ABSTRACT
Recent advancements reinforcement learning with verifiable rewards have pushed the boundaries of the visual reasoning capabilities in large vision-language models (LVLMs). However, training LVLMs with reinforcement fine-tuning (RFT) is computationally expensive, posing significant challenge to scaling model size. In this work, we propose PROXYTHINKER an inference-tin time technique that enables large models to inherit the visual reasoning capabilities from small, slow-thinking visual reasoners without any training. By subtract- ing the output distributions of base models from those of RFT reasoners, PROX- YTHINKER modifies the decoding dynamics and successfully elicits the slow- thinking reasoning demonstrated by the emerged sophisticated behaviors such as self-verification and self-correction. PROXYTHINKER consistently boosts perfor- mance on challenging visual benchmarks on spatial, mathematical, and multi- disciplinary reasoning, enabling untuned base models to compete with the per- formance their full-scale RFT counterparts. Furthermore, our implementation efficiently coordinates multiple language models with parallelism techniques and achieves faster inference compared to previous decoding-ting time methods, paving the way for the practical deployment of PROXYTHINKER. Code is available at https: / / thub. com Zi ittaa//i/iiio/////oiiillii
INTRODUCTION
Recent advances in large language models have led to the development of systems capable of ex- tended reasoning and deliberation, often referred to "slow-thinking" models, such as OpenAI- ol (Jaech et al., 2024), DeepSeck-RI (Guo et al., 2025), and QwQ (Team, 2025). Unlike "fast- thinking" models such GPT-4o (Hurst et al., 2024), "slow-thinking" " models usually engage in multi-step self-reffection to produce an answer that resembles the thorough thinking process that hu- mans make before producing final answer for non-trivial problems. These models have achieved remarkable success in complex problem-solving benchmarks, particularly in mathematical and sci- entific reasoning domains (Shao et al., 2024b; Zeng al., 2025a; Yu al., 2025). Recent research has also extended such reflective reasoning to multimodal tasks (Huang et al., 2025; Deng et al., 2025; Yang et al., 2025; Zhou al., 2025; Wang et al., 2025a), pushing large vision-language models (LVLMs) toward greater performance in scenarios that require structured and contextual understanding across modalities.
Many of the most effective "slow-thinking" models rely reinforcement learning with verifiable rewards (RLVR) (Face, 2025; Su et al.. 2025; Wei et al., 2025a), a reinforcement fine-tuning (RFT) framework that encourages the model to generate intermediate reasoning steps that lead to a correct answer for automatically verifiable tasks. While effective, this approach computationally intensive and resource-dmmanding First, the process often requires maintaining multiple model copies when using algorithms such as Proximal Policy Optimization (PPO) (Schulman et al., 2017) or Group Relative Policy Optimization (GRPO) (Shao et al., 2024a), which significantly increases memory usage. Second, the training process typically alternates between rollout and optimization phases, resulting in significant complexity and extensive training time.
Due to these high training costs, prior work has rarely applied RFT to LVLMs with more than bil- lion parameters. Recent research findings (Shah al., 2025; Yue al., 2025b; Gandhi al., 2025;

---

Published as a conference paper at ICLR 2026
RFT Training
Reasoning Expert (Small)
(Small)
KAAN
Reasoning Expert (Large)
Base Model (Small)
Test- time
Queries
Training Queries
Base Model (Large)
Unaffordable RFT Training
Model
(Large)
Illustration of PROXYTHINKER design. Without training, we migrate the reasoning be- haviors learned by the small visual reasoner by capturing the token-level logit difference between small, reasoning expert after reinforcement fine-tuning (RFT) and small, base model. The logit difference can effectively guide large base model to become large reasoning expert at test time.
Figure/ 1:
Liu et al., 2025a; Wang et al., 2025c) suggest that RFT does not teach new knowledge beyond the capabilities of the base model, but rather elicits and amplifies reasoning behaviors that are already in- cluded in the sampling distributions of the base model. In this work, we introduce PROXY THINKER, simple yet effective inference- sicrence-time method that allows for efficient transfer of visual reasoning pabilities without incurring any training costs, illustrated in Figure 1. Motivated by the line of work that explores secoding-steering of language models (Liu al., 2021; Li al., 2023b; O'Brien & Lewis, 2023; Leng et al., 2024; Liu et al., 2024a), we propose using the difference between the last- token logits from a reasoning Expert model after RFT training and those from a non Con-FFTAA Amateur model to represent the reasoning abilities induced by RFT. Such difference could steer a larger Base model toward the slow-thinking reasoning pattern. To investigate the effectiveness of PROX- YTHIIKER, we conduct experiments tasks using larger base models with 32B and 72B parameters. Quantitative results show significant improvements on benchmarks such as MathVista (Lu et al., 2024). MathVerse (Zhang et al., 2024), VathVision (Wang et al., 2024a), MMMU-Pro (Yue et al., 2025a) and R1-One VisionBench (Yang et al., 2025). For example. on the Math Vision test split, we improved the accuracy of Qwen2.5- -L--32B-Instruue (Bai et al., 2025) from 38.4% to 40.8% by integrating Open V.Teiniihiee (Deng al., 2025) as reasoning expert, despite the latter's poor accuracy of 25.3%. This even surpasses the 40.5% achieved by the full-scale RFT model VL-Rethinker-322B (Wang et al., 2025a).
mathematical, multi-disciplinary and complex reasoning
To reduce the computation overhead of running multiple models, we implement our system on top of vLLM (Kwon et al., 2023), fully exploiting modern parallelism techniques. Our optimized scheduling of logits computation across models yields 38x speedup over earlier open-sourced decoding-time time steering methods. Ablation studies show that our method works robustly without any hyperparameter tuning achieve substantial gains. We further conducted comprehensive analysis to show emerging reasoning behaviors in ProXyThInkER hoping to shed light on future research work in decoding algorithms that enhance reasoning abilities.
METHODOLOGY
2.1 PRELIMINARIES
Vision-Language Model (VLM) Decoding. A VLM defines conditional probability distribution Pe over output sequences, parameterized by model weights 0, and conditioned on both a textual prompt [T1,..., "r,] and set of input images I = [ ... It]. The model autoregressively
generates a response [/i....: yml according to:
( x,
(y IxIII yes).
2

---

Published as a conference paper at ICLR 2026
What the area of the room?
Question:
+ off--al (3af+++))
Final answer:
D.3ofb a) 1 B.8
larpe, Bape model
Wisit does it seem right
let' dounte check the choires
Current Thinking
te + of a] 3ola -4 to bu(b- The coneect choice
Whlt Hinwwer tutt
(Bido)
Simpiity eaprassee endion:
deb 4ol8 [1)
expression
glven
Answer: B
a) Sala-s 6)
small Amatour model
Arenie
Walt,
The
-
dirided into large rectangle smaller rectandes simplely the exprassinn: Total arma - 2nh 4 + - a² Jab Sab This, the
1 Thus Answe
-11-1- Fixal Thus Anssier
Not Howevor Bir
Wast However But
Figure 2: PROXYTIINKER with case study from MathVision. When provided with the same incor- rect thinking process, both the large and small base models tend to finalize the answer prematurely in the decoding process. The small reasoning expert shows signs of self-verification behaviors, e.g., assigning high probabilities to "Wait", "However", "But". However, its limited capacity confines it to shallow reasoning, such as restating answer choices. Through logits manipulation, PROXY- THINKER transfers this reasoning behavior to the large base model, effectively triggering accurate self-correction and leading to the correct option.
Decoding- Treoeing-Time Algorithm refers to a technique that modifies language model (LM) output dis- tribution at inference time as a means to improve generation quality and control without training. DExperts (Liu et al., 2021) first proposes to steer the output of an LM toward desirable attributes, such as reducing toxicity and controlling sentiment, with a pair of expert and anli-expert LMs, encouraging safe continuation and penalizing toxic completions. Contrastive Decoding (Li al., 2023b) improves open-ended text generation quality by contrasting the predictions of a large expert LM against those small amateur LMM. The intuition behind this is that if both a big and small model are likely to produce an undesirable token (e.g., a generic, repetitive word), that token score suppressed, whereas tokens favored by the expert but not the amateur are boosted. We include additional related work in $6. In contrast to these approaches, our motivation lies in transferring the reasoning abilities of a small visual reasoner, which is orthogonal to their goals and contributions.
2.2 PROXYTHINKER:
WITH TEST-TIME GUIDANCE
There is increasing evidence that reinforcement fine-tuning (RFT) does not impart fundamentally new knowledge into a base model, but rather amplifies reasoning behaviors that the base model was already capable of in principle (Yue et al., 2025b). Or in other words, RFT shifts the probability mass of a model toward token sequences that exhibit structured, "slow-thinking" reasoning strategies, such branching into sub-cases, backtracking after contradiction (Swamy al., 2025), and self- checking intermediate answers (Gandhi et al., 2025). These reasoning strategies are reflected in the high activation of relevant tokens at specific stages of the reasoning process.
In the upper part of Figure 2, we present an example from the Math Vision dataset, where we pro- vide three different vision-language models (VLMs) with the same incorrect reasoning process. Both large and small base models tend to directly provide an answer after reading the reasoning process, whereas a small RFT-trained expert exhibits reflective reasoning strategies. However, due to its limited model capacity, this reflective behavior remains shallow and largely restricted to re- stating answer options. We therefore ask: Can the reasoning skills acquired during RFT be directly transferred to a larger model via logits delta? By combining the reasoning patterns of the small RFT expert with the enhanced capacity of the large base model, we anticipate that such a transfer could deepen the model's reasoning behaviors and improve performance the lower part of Figure 2, we observe that applying logits delta successfully elicits the effective reflection of the large base model and ultimately leads to a correct option.
reasoning-intnnivve reasoning- intensive tasks. In
Formally, consider a pretrained VLM V or Base model, which we wish to adapt toward improved "slowwthinking" reasoning behavior without updating its parameters. Given set of input images
3

---

Published as a conference paper at ICLR 2026
} and text prompt Z<t with f being the current decoding time step, v produces a logit vector over the vocabulary, conditioned jointly on both modalities. The goal is to guide V to behave as if it had undergone RFT while avoiding any parameter updates to V. To achieve this, we introduce two auxiliary models: a small, base Amateur model t/n and its RFT counterpart, reasoning Expert model 1/1, which is much cheaper the RFT method. The proposed PROX YHHNKER modifies the output distribution of V at inference time using logit shift computed from the difference between the output logits of 1 and too
I=
than tuning the large model V itself with
At each decoding step L, we condition V/ 11, and vp on the shared image set I and the current text prefix Z<t to compute logits zy, 2 and 2 respectively. A hyperparameter a E IR controls the influence of this difference signal, which we will discuss its impact in $4.1. The adjusted distribution from PROXYTHINKER model T is given by:
p (2, Tet: = softmax (zw (z, Ier. I) + (Zo,, (2, Z) 2 (z; Te I))]
(2)
A token I2 is then sampled from this adjusted distribution and appended to the input sequence 2- forming Zgr, used as the next-step input for all three models - V, tho, and 1/ in the subsequent decoding iteration. This feedback loop continues autoregressively until the end of the generation.
2.3 IMPLEMENTATION
In practice, using multiple models inevitably introduces computational overhead. Prior decoding-time algorithms have naively em- ployed coarse-grained pipeline parallclism, where the exccution of different models is se- quential and synchronous. For instance, the expert model must wait for the base model to complete all pipeline stages before proceeding. This leads a significant amount idle time in the multi-GPU pipeline, as shown in Figure 3.
Methods
Optimized
B
B
GPU
B
B
GPU
B
A
B
E
GPU
B
B A
B
E
B
A
B
A
GPU
B
B
B
A
GPU
B
B
A
GPU
B
B
A
We address this by first implementing collab- orative decoding with multiple models in the vLLM framework (Kwon al., 2023), which enables us to leverage modern inference fea- tures such KV cache, tensor parallelism, and continuous batching, corresponding to the Ours implementation in Figure 3. Furthermore, we find that small models typically exhibit marginal returns in speedup the tensor parallel size in- creases (Wu et al., 2025). Therefore, we apply tensor parallelism to the large base model and asyn- chronously run the small expert and amateur models in separate tensor- marolll process groups to further increase throughput. Logits from different models are synchronized via collective communi- cation right before sampling. This optimized scheduling of multiple language models, indicated in Optimized in Figure 3, minimizes GPU idle time and significantly reduces inference time, as shown in $4.4.
Time
Base
Expart
A Amatour
Idle
Figure 3: Comparison between PROXYTHINKER implementation and previous methods.
EXPERIMENTS
In this section, we first introduce the evaluation setup, including models and benchmarks used and evaluation strategies. We next report the effectiveness of PROXYTHINE THINKER from these perspectives: mathematical and multi-disciplinary reasoning.
3.1 EXPERIMENT SETUP
Benchmarks and Evaluation Setup. To quantitatively assess PROXYTHINKER'S reasoning ca- pabilities across various domains, evaluate it several established benchmark datasets. For math-related reasoning, we use Verse (Zhang et al., 2024), MathVista (Lu et al., 2024), and Math Vision (Wang al., 2024a). Specifically, adopt the testmini splits of Verse and
4

---

Published as a conference paper at ICLR 2026
Table 1: Performance (Aceuracy %) on mathematical and nulti-disciplinary reasoning benchmarks. is set to 0.5 for PROXYTHINKER methods. RI-Bench stands for RI-Onevision-Bent Overall best THIXKERINKER method is marked with light blue. Ceiling performance of full-scale RFT expert highlighted with gray. A shows the average relative improvement in % compared the base model. indicates an RFT reasoning expert. and a's denotes a PROxY THINKKER variant.
Math Vista
Math Verse
Math Vision
MMMU
RI- Bench
Model
Expert
A
Pro
Qwen2.5-VL-77
68.2
46.3
25.1
36.9*
32.1
OpenVLThinkerss
70.2
47.9
25.3
39.4
32.9*
ThinkLite-LL-7B
75.1
50.7
32.9
41.1*
39.0"
VI-RRthinner--7B
74.9
54.2
32.3
41.7
47.2"
Wen2.5VVL-32B
74.7
53.8"
38.4
49.5t
49.4"
0.0
Qwen2.5-VL-33
OpenVL.Thinker-7B
77.4 (42.7)
53.8 (0.0)
40.8 (42.4)
51.8 (42.3)
53.0 (43.6)
+2.2
Qwen2.5-VL-32B
ThinkLite-VL-7B
77.6 (+2.9)
56.0 (+2.2)
38.8 (40.4)
51.7 (42.2)
49.7 (40.3)
+1.6
Qwen2.5-VI-32B
VL-Rethinker-7B
78.1 (+3.4)
55.3 (+1.5)
39.2 (+0.8)
52.8 (+3.3)
52.5 (43.1)
+2.4
VL-Rethinker-322B
78.8
56.9
40.5
50.6
50.8*
+2.4
Qwen2.5-VI--22B
74.8
55.1
38.1
51.6
50.4
0.0
Owen2.5-VL-722
OpenVLThinker 7B
77.8 (43.0)
56.4 (+1.3)
36,2 (-1.9)
52.4 (+0.8)
50.4 (0.0)
+0.6
Qwen2.5-VL-72B
ThinkLite-VL.1) VL-7BB
78.7
(43.9)
57.2 (42.1)
40.4 (+2.3)
51.7 (40.1)
50.2 (:0.2)
+1.6
Qwen2.5-VI-722
VL-Rethinker-7B
78.1 (43.3)
58.6 (43.5)
39.5 (+1.4)
53.1 (+1.5)
54.4 (44.0)
+2.7
VI-RRethinke-72BB
80.3
61.7
43.9
55.9
57.9"
+5.9
indicates results from Wang et al. (2025a) where we adopt the same evaluation protocol.
indicates reproduced results by us because the original authors did not conduct such evaluation.
Math and the full test split of MathVision. For assessing multi-diseiplinary understanding and reasoning, we employ the MMMU-Pro (Yue et al., 2025a) overall split and the RI RI-Onevision- Bench (Yang al., 2025) dataset. Following prior work, we use VLMEvalKit (Duan et al., 2024) to perform the extract-and-score procedure on Math Verse with gpt popt-4o-mins as judge. For the remaining datasets, we apply exact matching and rule-based grading function from their official benchmark toolkits to extract answers from model outputs and compare them to the ground truth. Unless stated otherwise, we report Pass@1 I accuracy across all benchmarks using greedy decod- ing following previous works (Wang et al., 2025a; Deng al., 2025) and set the hyper-parameter = 0.5.
Model Selection. PROXYTHINKER employs three distinct types of models, each serving a unique role: a large Base model to be steered, a compact reasoning Expert model, and a compact Amateur model. We use Models, while Qwen2.5- VL--B- VL-7B-Inntruct serves as the Amateur Model across all RFT Experts. In later sections, PROXYTHINKER-333 32B and 72B base models, respectively. Based on differing training paradigms and data selec- tion strategies, we include these public models the compact reasoning experts: OpenVLThinker 7B (Deng et al., 2025), ThinkLite- VL-7B (Wang et al., 2025b) and VL-Rethinker-75 (Wang et al., 2025a). We refer Appendix A.1 for details of the used benchmarks and reasoning expert models.
-L-332B-Instruuu and Swwn2.5-VVL-72B-Inne
(Bai et al., 2025) as Base
and
correspond to methods using
3.2 MAIN RESULTS ON MATHEMATICAL AND MULTI-DISCIPLINARE REASONING
To further investigate the generality and scalability of PROXYTHINKER, we examine whether similar effects can be observed widely used mathematical and multi-disciplinary reasoning tasks using 32B and 72B models with different types of RFT reasoning experts. The prompt template for each reasoning expert is attached in Appendix A.3. We report these results in Table 1. To explore the upper bound of our method, we also use two larger models, VL-Rethinker-325 and VL-Rethinker- 72B, which are directly trained via RFT, ceiling performance references.
5

---

Published as a conference paper at ICLR 2026
Mahneess A00/NN
158
06
20.3
Mantiiaaion
58.4
0.0
0.1
0.2
0.3
0.4
0.5
0.8
0.7
0.5
0.0
1.0
1.1
1.2
1.5
1.4
1.5
0.0 0.1
0.2
0.3 0.4
0.5
0.8
0.7
0.5
0.8
1.0
1.1
1.4
Proxy
Contrastive Dacoding
Eest
Figure 4: Hyper-parameter search results of on MathVerse and MathVision benchmarks with VL-Rethinker-77 as the expert model. Red dotted lines provide the ceiling performance of VL- Rethinker-32B, full-scale RFT reasoning expert.
PROXYTHINKER provides consistent improvements on nearly all benchmarks. With the ex- ception of the MMMU validation set, we observe consistent performance improvements across all benchmark tasks and model-expert combinations. For example, using OpenVLThinker-778 as an expert improves Swen2.5-VL-3222 11-22B-Insstret'ss Math Vision test accuracy from 38.4% to 40.8%, sur- passing even the fully RFT-trained VL-Rethinkerr-320 (40.5%). This improvement unlikely due to knowledge transfer, as Open VLThinker-7B achieves only 25.3% Math Vision. Rather, it suggests that the reasoning patterns of the small expert have been effectively extracted and applied to enhance the large base model's 's reasoning abilities that otherwise would require full- scalescale RFT to activate. We discuss the domain-agnostic transfcrability of PROXYTHINKER in Appendix B.I, providing strong evidence that
enables the transfer of general visual reasoning prior.
Quality of RFT expert generally determines the degree of Consistent with intuition, stronger RFT expert tends provide more structured reasoning paths, enhancing the base model's reasoning abilities more effectively. VL-Rethhnker- -BB the most competitive of the experts, achieves the best overall results with both Qwen2.5-VL-32B and 72B. To gain deeper insights into the role of RFT experts, we discuss in Appendix B.2 on how underper- forming experts may introduce negative effects PROXYTIINKER.
improvement.
maintains competitive scalability across model sizes. At the 32B scale, we observe that PROXYTHINKER achieves similar average relative improvement to full RFT models at the 32B scale. This indicates that Proxy Thinker can extract and transfer structured reasoning behaviors almost effectively direct RFT training this size. Notably, PROXYTHINKER can even surpass the RFT ceiling on certain tasks. At the 72B scale, Proxy Thinker continues to yield consistent gains across benchmarks smaller in magnitude but still positive demonstrating that the approach remains effective even as the base model grows.
ANALYSIS
In this section, we further analyze PROXYTHINKER both qualitatively and quantitatively, with focus on hyperparameter sensitivity, the type of RFT expert, the emergence of transferred reasoning behaviors, and the computational overhead of multiple model inference.
4.1 HYPERPARAMETER SENSITIVITY
The only hyperparameter in PROXYTHINKER, a Equation 2, regulates the scaling of the log- its difference applied to the base model's logits, where a smaller results in the logits closer to the original base model. To validate the robustness, we run experiments with Qwen2.5-VL-32B. Instruct with VL-Rethinker-7B as the expert model, across Math Vision and MathVerse benchmarks. We show a sweep of results on a E 1.5] in increments of 0.1 in Figure 4. To demonstrate the advantage of PROXYTHINKER design, we additionally include modified contrastive decoding baseline shown as the blue curve the sweep, which relies on the base model distribution Shaee
expert distribution zexpe and performs decoding by sampling from Zhuse Zespert-

---

Published as a conference paper at ICLR 2026
Oward 3al--aa VL fldto 72
50
50
70
70
pass
600
50
50
40
40
80
30
Number Samples
Number of Samples k
Figure 5: Pass@k k curve of base model, VL--Rethnner-778 RFT expert and PROXYTHINKER.
We found PROXYTIINKER to be highly robust to a, as values between 0.1 to 1.0 could yield no- ticeable improvements over the base model (a 0.0). Specifically, we highlight that the results reported in Table 1 with the default setting of VL-Rethinker- 7B, the best a yields 40.3% on Math Vision and 57.2% on Math Verse, surpassing the 39.2% and 55.1% reported in Table 1, and even reaching the ceiling performance of VL-Rethinker- 32B, a full-scale, large reasoning expert. In addition, we observe that as a approaches 1.5, the perfor- mance gradually converges toward that of the small expert, which aligns with our intuition: larger amplifies the influence of the logits delta, resulting in the decoding process being increasingly dominated by the small reasoning expert. Finally, the performance gap between PROXYTIINKER and contrastive decoding indicates that the Amateur model in PROXYTHINE THINKER is critical compo- nent. plays a key role in calibrating the logits distribution introduced by the trantere small visual reasoning expert and effectively leveraging the capabilities of the large base model.
0.5 do not reflect the best performance. For
4.2 MEASURE THE REASONING CAPACITY BOUNDARY BY PASS @k
Recent studies (Yue et al., 2025b) argue that single-pass greedy decoding reflects only average- case performance of a model, and advocate for pass (Chen et al., 2021) as a better measure of reasoning capacity boundary. These studies found that while RFT improves sampling efficiency, it lowers the reasoning boundary: as i increases, RFT models produce less diverse outputs, leading to lower pass@ & scores compared to base models.
To investigate whether PROXYTHINKER inherits this phenomenon, we first compare the pass @ of Owen2.5-VVL-BB-nnnmmee (base model) and VL-Rethinker-77B (reasoning expert) on Vishonsion subset on the left of Figure 5 with evaluation details in Appendix A.2. Echoing prior findings, VL-Rethinker-7B outperforms the base model at k S pressed as k grows, indicating reduced reasoning boundary brought by RFT training. We then compared Qwen2.5- VL-33B- Instructt (base model), VL--Rethinker-322 (full-scale RFT expert), and ProxyThinker-32B w/ VL-Rethinker--B shown on the right of Figure 5. ProxyThinker outperforms both at & = 1, though the base model dominates as increases. In terms curve trends, the PROX- YTHINKER curve consistently lies between those of the base model and the reasoning expert for 2 2, indicating that PROXYTHINKER is able to transfer the efficient sampling behavior of the expert while partially retaining the diverse exploration capability of the base model.
pass@ but sup-
4.3 EMERGING REASONING BEHAVIORS
In addition to the example in Figure we present a qualitative Math Vision test case Figure showcasing the reason- ing process of three models to examine the emerging reasoning abilities of PROX- YTHINKER. We find that the large base model makes a commonsense error and fails to reach the correct answer, while the small reasoning expert remains confined to shallow reasoning patterns. In contrast,
Table 2: Reasoning pattern statistics Math of Base, Expert, and PROXYTHINKER with relative gains Base. Base model is Qwen2.5- VL-32B and Ex- pert model is Open W..Thinkiee..
Model
Backtracking
Verification
Subgoal
Base Expert
203
1206
2980
805 (+296.6%)
1585 (+31.48)
2427 (-18.5%)
Proxy FroxyThinker
482 (+137.4%)
1736 (+43.9%)
2998 (+0.6%)
7

---

Published as a conference paper at ICLR 2026
Four of the numbers 1,3,4,5 and written into the boxes so that the calculation is correct. Which number was not used?
Problem:
Response:
To
solve
the
problem
###
Step
6
Verify the solution...
G/Hkydel
egсap
the athis
Wart
Wat
Givertty
A (Top-3): ToSolector
Determin
20 (Daalglinaataaee
arantest twor
WAIT
Gwatt MC16
Gidentiyy
4 ArPlecCaas awls
_solve
Grvait
waii
Verlyy
|A| (Top-3):
Jooore
6140
2017
C4644
Successful Solf-Reflection
Nolsy Duo Agreament Between Expert and Amateur
Self-Reflection Attempts
Figure 6: Case study presenting the top-3 tokens of between small reasoning expert VOpenVLThinkeer
and amateur model (Qwen2.5-VL-7B) per decoding step. At early decod- ing steps, A tends to be noisy and creates a minor shift over byte-level tokens (shown in those random non-ASCII characters) in the vocabulary. In later decoding steps, A becomes structured and steers the response to self-reflection patterns. A indicates the magnitude of top-3 logits.
PROXYTHINKER successfully combines the strengths of both: inherits the explicit "thinking pro- cess" tags (<think>) from the small reasoning expert, while adopting the step-by-ster reasoning style and even providing an interpretable, visualized solution.
To quantitatively analyze emerging reasoning behaviors, report statistics 3,040 Math Vision samples in Table 2 on different reasoning patterns, such as backtracking, verification and sub-goal branching. Motivated by Gandhi et al. (2025), employ 9pt-4o-mini soning trajectories from the large base model, the small reasoning expert, and PROXYTHINKES it- self Vision 3040 samples. We observe that while large base models exhibit subgoal-oriented reasoning patterns, their performance on backtracking and verification remains limited. In contrast, small visual reasoners trained with RFT show significant improvements these aspects strength that Proxy Thinker successfully inherits. Notably, Proxy Thinker also retains the subgoal planning ability, which appears to diminish in small RFT models.
judge to classify rea-
4.4 INFERENCE OVERHEAD
Running multiple language models at in- ference time inevitably incurs overhead, which has been particularly problematic in previous decoding-time algorithms be- cause of their naive implementation men- tioned in $2.2. To investigate the effi- ciency of PROXYTHINKER, Table 3 re- ports runtime and accuracy Math Vi- sion testmini split using Qwen2.5-Vi 32B-Instruct as the base model and VL- Rethinker-7B as the expert model under several implementation variants. We also ran the full-scale reasoning RFT expert with vLLM as reference. All evaluations were con- ducted on 8 A100 PCIe 40GB GPUs. The results show that our implementation delivers close to 33x speedup over the Huggingface implementation, which was adopted by all previous decoding- steering methods. With the optimized tensor parallel group design, the runtime duration could be cut by another 13%, yielding total speedup of 38x, which closely matches the duration of di- rectly running a 32B RFT expert. We discuss the computational overhead in depth in Appendix to demonstrate PROXYTHINKER practical solution in terms of memory and compute.
Table Inference Statistics of PROXYTHINKER.
Method
Duration (s)
Acc (%)
PROXYTHINKER
Huggingface Ours Implementation
19133
40.78
578
41.44
Optimized TP
501
41.44
Full-scale RFT expert
VL.-Rethinker-322
451
41.77
8

---

Published as a conference paper at ICLR 2026
MathVision Set Sample #10
Four people can seated square table, How many people most could seated we pushed four tables this kind tagether ane <image
To solve the problem, let's analyze the seating arrangement step by step.
Stap Determine the number usable sides
When four tables are joined rew, there inner sides that are shared and usable scating.
The total number of usable sides therefore $ 16 13
<image1>
Reasoning from
V0D Final Answer: \bowed/1 13)
<think>...
<thinks
##8# Step 2: Consider Arrangement Four Tables Row When four square tables pushed together in row, they form longer rectangular table. The arrangement looks like this:
Okay, so need figure out how many people can be scated four square tables pushed together row. table have four tables, just need multiply
seat four people. the number of tables by the number of people each seot. That would be 4 tables times people per table.
</thinks
... the total numbee of people that can be seated is 4 16.
#de Final Answer:
The maximum number of people that can be seated when four square tables are pushed together in row is: \boxed(1 D).
<answer> 16</answer>
Reasoning from OpenVLThinker-7B
Reasoning from ProxyThinker-328
Figure 7: Case study presenting the reasoning process of large base model (Qwen2.5-VL-32) Instruct), small reasoning expert (OpenV VLThinker-7B)) and PROXYTHINKER, Bold in the reasoning process highlights typical reasoning patterns. Red bold denotes wrong intermediate reasoning steps that ultimately lead wrong prediction, while green bold indicates correct reasoning steps.
5 CASE STUDY: TEMPORAL ANALYSIS OF THE LOGIT DELTA A
To better understand the mechanism of how the logit delta A Rexpert zamateur affacts the model behaviour, we perform a qualitative temporal analysis of along a single reasoning process from PROXYTHINKER, shown in Figure 6. For each decoding step compute A, sort its components, and visualize the top-3 tokens together with their absolute magnitudes |
In the early part of the trajectory (the tokens "To solve the problem, ... ")., the expert and base model make very similar predictions: the top-3 tokens under A are mostly generic or even uninterpretable, and the corresponding bars for | are close zero. This indicates that PROXYTHINE the large model's 's distribution essentially unchanged when the two smaller models produce aligned logits. As the reasoning progresses, the structure of A, becomes more pronounced: the highest- scoring tokens are variations of Wair, with noticeably larger | Here, the expert initiates an internal self-reflection step. In the final segment, the magnitude A, increases sharply and the dominant tokens shift to Verify, Identify, and similar verbs, corresponding an explicit verification sub- -oal. These dynamics show that is not static list of "reasoning tokens", but context-dependent steering signal whose direction changes over the course of the reasoning trace.
leaves
RELATED WORK
Large Vision-Language Models (LVLMss) Large Vision-Language Models (LVLMMs) (Wang et al., 2024b; Bai et al., 2025; Chen et al., 2024a; Zhu et al., 2024: Li et al., 2023a; Ye et al., 2023; Awadalla al., 2023; Chen et al., 2024b) have demonstrated significant capabilities multimodal understanding through a series of architectural advances. The Qwen2-VL (Wang et al., 2024b) and its extension introduced efficient visual tokenization through dynamic resolution mechanisms. Despite these advances, current LVLMs continue to struggle with complex reasoning tasks requiring spatial understanding and mathematical reasoning, highlighting the need for novel approaches to enhance reasoning capabilities without increasing computational demands.

---

Published as a conference paper at ICLR 2026
Improving Visual Reasoning with Reinforcement Learning. Reinforcement learning has become an approach to enhance reasoning capabilities in LLMs and VLMs. OpenVLThinker 7B (Deng et al., 2025) demonstrates how reasoning capabilities can be integrated into LVLMs through an iterative self-improvement process combining supervised fine-tuning and reinforcement learning. The R1 R1-style frameworks (Wang et al., 2025a;b; Huang al., 2025; Zhang et al., 2025; Shen et al., 2025a; Li et al., 2025; Wei et al., 2025c) apply Group Relative Policy Optimization (GRPO) to the visual domain, demonstrating that slow-thinking reasoning abilities can be transferred to multimodal contexts. Variants like Visual-RFTss (Liu et al., 2025b) design task-specific reward and VisualThinker-P RI-Zero (Zhou et al., 2025) explore applying the "Aha moment" of DeepSeek- R1 (Guo et al., 2025) to visual reasoning without supervised fine-tuning. Similar work (Wei et al., 2025b) made successful attempts transferring reasoning behaviours from LLMs to VLMs at train- ing time. And variations (Shen et al., 2025b; Yao et al., 2025) of policy optimization method have been proposed in post-training literatures for visual reasoning. While these approaches yield notable improvements in visual reasoning benchmarks, they require substantial computational resources, making it harder to scale. In contrast, no further training and adds minimal computation, allowing larger VLMs to gain reasoning skills efficiently.
is a test-time decoding method that requires
Decoding-time Algorithms for Language Model. Decoding Decoding-time algorithms steer LLMs at in- ference without expensive retraining. Contrastive decoding method (O (O'Brien& & Lewis, 2023) has been applied to maximize the differences between the likelihoods of expert and amateur models to improve reasoning. CAL (Xiao et al., 2024) and VCD (Leng et al., 2024) extend this approach to multimodal settings by subtracting logits from original and perturbed images. DoLa (Chuang al., 2024) targets the pervasive issue of LLM hallucinations generating text not supported by factual knowledge. RAST (Ouyang al., 2025) studies the reasoning transfer in the context of text-only tasks. DeRa (Liu et al., 2024b) and MOD (Shi et al., 2024) approximate geometric mixtures of aligned and base models or linearly combine expert predictions across objectives. In the line of these decoding-time methods, of a reward-aligned RFT expert and a non-RIT base model, eliciting the reasoning ability learned during RFT with no additional training. A similar training-frre technique was proposed recently to improve visual reasoning through the lens of model merging (Chen et al., 2025b).
leverages the difference between last-token logits
7 CONCLUSION
We present PROXYTHINKER, a simple yet effective decoding-time algorithm for transferring visual reasoning capabilities from small visual reasoning models. PROXYTHINKER leverages the token- level logits difference between an RFT expert and an amateur model to effectively steer a large model's generation toward "slow-thinking" multi-step reasoning behaviors. Through extensive ex- periments on vision-centric and multimodal reasoning tasks, we demonstrate that PROXY THINNER can consistently enhance performance across model sizes, including substantial improvements on spatial, mathematical, and multi multi-disciplinary reasoning benchmarks. We believe PROXY THINKER provides a promising direction for efficient reasoning transfer in large language models and offers insights into the understanding of how RFT influences model behavior.
Acknowledgements. This work was supported by an NSF CAREER Award No. 2201710, by the Ken Kennedy Institute Rice University, and in part by NSF Campus Cyherinfrastructure grant NSF OAC-2019007 and by Rice University's Center for Research Computing (CRC). We thank the anonymous reviewers for their constructive feedback.
REPRODUCIBILITY STATEMENT
Experimental setups, including benchmarks, evaluation protocols, and model roles (Base / Expert / Amateur), are detailed in $3.1. Main results across mathematical and multi-disciplinary reasoning appear in Table I with analysis in $3.2, while efficiency considerations and multi-mode scheduling are shown in Fig. 3 and summarized in Table 3. Additional implementation details, benchmark/- model descriptions, and prompts are provided in Appendix A.I and A.3. Pass k evaluation details are in Appendix A.2. A repository with inference scripts, evaluation harnesses, configuration files, and implementation is included in the abstract to reproduce Tables 1, 3, and Figures 1-5.
10

---

Published as a conference paper at ICLR 2026
REFERENCES
Art of Problem Solving. Aime problems and solutions wiki artofproblemsolving homm/wiki/nndeen httpa//inne...pp/Ainaaaa.. Problems_ and Solutions. Source of problems used in AL-MO/aimoo-vaiidatonn-iii Accessed 2025- 11-20.
https:/ //
Anas Awadalla, Irena Gao, Josh Gardner, Jack Hessel, Yusuf Hanafy, Wanrong Zhu, Kalyani Marathe, Yonatan Bitton, Samir Gadre, Shiori Sagawa, et al. Openflamingo: An open- source framework for training large autoregressive vision-language models. ArXiv preprint, abs/2308.01 01399, 2023.
Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, et al. Qwen2. 5-vl technical report. ArXiv preprint, abs/2502. 13923, 2025.
Hardy Chen, Haoqin Tu, Fali Wang. Hui Liu, Xianfeng Tang, Xinya Du, Yuyin Zhou, and Cihang Xie. Sft or rl? an early investigation into training rl-like reasoning large vision-language models. ArXiv preprint, abs/2504. 1468, 2025a
Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde De Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, et al. Evaluating large language models trained on code. ArXiv preprint, 0bss/207.003344 2021.
Shiqi Chen, Jinghan Zhang, Tongyao Zhu, Wei Liu, Siyang Gao, Miao Xiong, Manling Li, and Junxian He. Bring reason to vision: Understanding perception and reasoning through model merging. ArXiv preprint, Abss/2005005464 2025b.
Yijie Chen, Yijin Liu, Fandong Meng, Yufeng Chen, Jinan Xu, and Jie Zhou. Enhancing cross-tokenizer knowledge distillation with contextual dynamical mapping. ArXiv preprint, abs/2502.11104 2025c.
Zhe Chen, Weiyun Wang, Yue Cao, Yangzhou Liu, Zhangwei Gao, Erfei Cui, Jinguo Zhu, Shen- glong Ye, Hao Tian, Zhaoyang Liu, al. Expanding performance boundaries of open-source multimodal models with model, data, and test-time scaling. ArXiv preprint, abs/2412.05271, 2024a.
Zhe Chen, Weiyun Wang, Hao Tian, Shenglong Ye, Zhangwei Gao, Erfei Cui, Wenwen Tong. Kongzhi Hu, Jiapeng Luo, Zheng Ma, et al. How far are we to gpt-4v? closing the gap to commercial multimodal models with open-source suites. Science China Information Sciences, 67 (12):220101, 2024b.
Yung-Sung Chuang, Yujia Xie, Hongyin Luo, Yoon Kim, James R. Glass, and Pengcheng He. Dola: Decoding by contrasting layers improves factuality in large language models. In The Twelfth International Conference on Learning Representations, ICIR 2024, Vienna, Austria, May 7-1 11, 2024. OpenReview.net, 2024.
Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, et al. Training verifiers to solve math word problems. arXiv preprint arXiv:21 14168, 2021.
Yihe Deng, Hritik Bansal, Fan Yin, Nanyun Peng, Wei Wang, and Kai-Wei Chang. Openvlthinker: An early exploration to complex vision-language reasoning via iterative self-improvement. ArXiv preprint, abs/2503. 17352, 2025.
Haodong Duan, Junming Yang, Yuxuan Qiao, Xinyu Fang, Lin Chen, Yuan Liu, Xiaoyi Dong, Yuhang Zang, Pan Zhang. Jiaqi Wang, et al. VImevalkit: An open-source toolkit for evaluat- ing large multi-modas modaliy models. In Proceedings of the 32nd ACM international conference on multimedia, pp. 11198 1201, 2024.
Hugging Face. Open rl: A fully open reproduction of deepseek-r1, 2025.
Kanishk Gandhi, Ayush Chakravarthy, Anikait Singh, Nathan Lile, and Noah D Goodman. Cogni- tive behaviors that enable self celfovino reasoners, or, four habits of highly effective stars. ArXiv preprint, abs/2503.01307 2025.
11

---

Published as a conference paper at ICLR 2026
Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. The llaaa 3 herd of models. arXiv preprint arXi::240.22178., 2024.
Daya Guo, Dejian Yang. Haowei Zhang. Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-rl: Incentivizing reasoning capability in Ilms via reinforcement learning. ArXiv preprint, 0oos/2501.12944 2025.
Yunzhuo Hao, Jiawei Gu, Huichen Will Wang, Linjie Li, Zhengyuan Yang. Lijuan Wang, and Yu Cheng. Can MLLMs reason in multimodalit?? EMMA: An enhanced multimodal reason- ing benchmark. In Forty-secont International Conference on Machine Learning, 2025.
Wenxuan Huang, Bohan Jia, Zijie Zhai, Shaosheng Cao, Zheyu Ye, Fei Zhao, Zhe Xu, Yao Hu, and Shaohui Lin. Vision-rl: Incentivizing reasoning capability in multimodal large language models. ArXiv preprint, abs/2503.06749, 2025.
Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Os- trow, Akila Welihinda, Alan Hayes, Alec Radford, et al. Gpt-4o system card. ArXiv preprint, abs/2410.21276, 2024.
Aaron Jaech, Adam Kalai, Adam Lerer, Adam Richardson, Ahmed El-Kishky, Aiden Low, Alec Helyar, Aleksander Madry. Alex Beutel, Alex Carney, et al. Openai ol system card. ArXiv preprint, abs/2412. 2024.
Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. Scaling laws for neural language models. ArXiv preprint, abs/2001 .08361, 2020.
Woosuk Kwon, Zhuohan Li, Siyuan Zhuang. Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, and Ion Stoica. Efficient memory management for large language model serving with pagedattention. In Proceedings of the ACM SIGOPS 29th Symposium on Operating Systems Principles, 2023.
Sicong Leng, Hang Zhang, Guanzheng Chen, Xin Li, Shijian Lu, Chunyan Miao, and Lidong Bing. Mitigating object hallucinations in large vision-language models through visual contrastive de- coding. In JEEF/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2024, Seattle, WA, USA, June 16-22, 2024, pp. 13872-13882. IEEE, 2024. doi: 10.1109/CVPR5233.. 2024.01316.
Bo Li, Yuanhan Zhang, Liangyu Chen, Jinghao Wang, Fanyi Pu, Jingkang Yang, Chunyuan Li, and Ziwei Liu. Mimic-it: Multi-modal in-context instruction tuning. ArXiv preprint, abs/2306.05425, 2023a.
Lin Li, Wei Chen, Jiahui Li, and Long Chen. Relation-rl: Cognitive chain-of-thought guided rein- forcement learning for unified relational comprehension. ArXiv preprint, abs/2504. 11122544.4444 14642, 2025.
Xiang Lisa Li, Ari Holtzman, Daniel Fried, Percy Liang, Jason Eisner, Tatsunori Hashimoto, Luke Zettlemoyer, and Mike Lewis. Contrastive decoding: Open-ended text generation optimization. In Anna Rogers, Jordan Boyd-Graber and Naoaki Okazaki (eds.), Proceedings of the ōl Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). pp. 12286- 12312, Toronto, Canada, 2023b. Association for Computational Linguistics. doi: 10.18653/v1/ 3023.acl-long.66
Hunter Lightman, Vineet Kosaraju, Yura Burda, Harri Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. Let's verify step by step. arXiv preprint arXiv:2305.20050 2023.
Alisa Liu, Maarten Sap, Ximing Lu, Swabha Swayamdipta, Chandra Bhagavatula, Noah A. Smith, and Yejin Choi. DExperts: Decoding-time controlled text generation with experts and anti- experts. In Chengqing Zong, Fei Xia, Wenjie Li, and Roberto Navigli (eds.), Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pp. 6691-6706, Online, 2021. Association for Computational Linguistics. doi: 10. 10.1885510 1/2021 accl- 522g.
12

---

Published as a conference paper at ICLR 2026
Alisa Liu, Xiaochuang Han, Yizhong Wang. Yulia Tsvetkov, Yejin Choi, and Noah A. Smith. Tuning language models by proxy. In First Conference on Language Modeling, 2024a.
Tianlin Liu, Shangmin Guo, Leonardo Bianco, Daniele Calandriello, Quentin Berthet, Felipe Hlinares-Lóped Jessica Hoffmann, Lucas Dixon, Michal Valko, and Mathieu Blondel. Decoding- time realignment of language models. In Forty-first International Conference on Machine Learn- ing, ICML 2024, Vienna, Austria, July 21-27, 2024. OpenReview.net, 2024b.
Zichen Liu, Changyu Chen, Wenjun Li, Penghui Qi, Tianyu Pang. Chao Du, Wee Sun Lee, and Min Lin. Understanding r1 rl-zero-like training: A critical perspective. ArXiv preprint, abs/2503.20783, 2025a.
Ziyu Liu, Zeyi Sun, Yuhang Zang, Xiaoyi Dong, Yuhang Cao, Haodong Duan, Dahua Lin, and Jiaqi Wang. Visual-rft: Visual reinforcement fine-tuning. ArXiv preprint, abs/2503.0177 2025b.
Pan Lu, Hritik Bansal, Tony Xia, Jiacheng Liu, Chunyuan Li, Hannanch Hajishirzi, Hao Cheng, Kai-
Wei Chang, Michel Galley, and Jianfeng Gao. Mathvista: Evaluating mathematical reasoning of foundation models in visual contexts. In The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024. OpenReview.net, 2024.
Sean O' Brien and Mike Lewis. Contrastive decoding improves reasoning in large language models.
ArXiv preprint, abs/2309.09) 00910999999......00..0....1.0,11 2023.
Siru Ouyang, Xinyu Zhu, Zilin Xiao, Minhao Jiang, Yu Meng, and Jiawei Han. RAST: Reasoning activation in LLMs via small-model transfer. In The Thirty-ninth Annual Conference on Neu- ral Information Processing Systems, 2025. URL http:ht:t/////eeeeeeeee /ttp:////peeeeeeeeeeeeee/.. net forum?id
2swy1B4Wnt
Jeff Rasley, Samyam Rajbhandari, Olatunji Ruwase, and Yuxiong He. Deepspeed: System op- timizations enable training deep learning models with over 100 billion parameters. In Rajesh Gupta, Yan Liu, Jiliang Tang, and B. Aditya Prakash (eds.), KDD '20 The 26th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, Virtual Event, CA, USA, August 23-27, 2020, pp. 3505-3506. ACM, 2020.
John Schulman, Filip Wolski, Prafulla Dhariwal. Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. ArXiv preprint, abs/1707.06347, 2017
Darsh J Shah, Peter Rushton, Somanshu Singla, Mohit Parmar, Kurt Smith, Yash Vanjani, Ashish Vaswani, Adarsh Chaluvaraju, Andrew Hojel, Andrew Ma, et al. Rethinking reflection in pre- training. ArXiv preprint, abs/2504.0400 2025.
Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song. Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Y Wu, et al. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. ArXiv preprint, abs/2402.03300, 2024a.
Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Mingchuan Zhang, Y.K. Li, Y. Wu, and Daya Guo. Deepseekmath: Pushing the limits of mathematical reasoning in open language models, 2024b.
Haozhan Shen, Peng Liu, Jingcheng Li, Chunxin Fang, Yibo Ma, Jiajia Liao, Qiaoli Shen, Zilun Zhang, Kangjia Zhao, Qianqian Zhang, et al. Vlm-rl: A stable and generalizable r1-style large vision-language model. ArXiv preprint, abs/2504.07615, 2025a.
Junhao Shen, Haiteng Zhao, Yuzhe Gu, Songyang Gao, Kuikun Liu, Haian Huang, Jianfei Gao. Dahua Lin, Wenwei Zhang, and Kai Chen. Semi-off-policy reinforcement learning for vision-language slow-thinking reasoning. In The Thirty-ninth Annual Conference on Neural Information Processing Systems, 2025b. URL https: http:///pppeeeeeeeeee net forum?id= zIFuLXUAuS.
Ruizhe Shi, Yifang Chen, Yushi Hu, Alisa Liu, Hanna Hajishirzi, Noah A. Smith, and Simon S. Du. Decoding-time language model alignment with multiple objectives. In Amir Globersons, Lester Mackey, Danielle Belgrave, Angela Fan, Ulrich Paquet, Jakub M. Tomezak, and Cheng Zhang (eds.), Advances in Neural Information Processing Systems 38: Annual Conference on Neural Information Processing Systems 2024, NeurIPS 2024, Vancouver, BC, Canada, December 10 15, 2024, 2024.
13

---

Published as a conference paper at ICLR 2026
Yi Su, Dian Yu, Linfeng Song, Juntao Li, Haitao Mi, Zhaopeng Tu, Min Zhang, and Dong Yu. Expanding rl with verifiable rewards across diverse domains. ArXiv preprint, abs/2503.23829, 2025.
Gokul Swamy, Sanjiban Choudhury. Wen Sun, Zhiwei Steven Wu, and J Andrew Bagnell. All roads lead to likelihood: The value of reinforcement learning fine-tuning. ArXiv preprint, abs/2503.01067, 2025.
Qwen Team. Qwq-32b: Embracing the power of reinforcement learning, 2025.
Haozhe Wang, Chao Qu, Zuming Huang, Wei Chu, Fangzhen Lin, and Wenhu Chen. Vl-rethinker: Incentivizing self-reflection of rision-language models with reinforcement learning. ArXiv preprint, abs//ab4. 008377 2025a.
Ke Wang, Junting Pan, Weikang Shi, Zimu Lu, Houxing Ren, Aojun Zhou, Mingjie Zhan, and Hongsheng Li. Measuring multimodal mathematical reasoning with math- vision dataset. Amir Globersons, Lester Mackey, Danielle Belgrave, Angela Fan, Ulrich Paquet, Jakub M. Tomezak, and Cheng Zhang (eds.), Advances Neural Information Processing Systems 38: Annual Con- ference on Neural Information P Processing Systems 2024, NeurlPS 2024, Vancouver, BC, Canada, December 10 15, 2024, 2024a.
Peng Wang. Shuai Bai, Sinan Tan, Shijie Wang. Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, et al. Qwen2-vl: Enhancing lisionllanuuag model's 's perception of the world at any resolution. ArXiv preprint, abs/2409.12199 2024b.
Xiyao Wang, Zhengyuan Yang, Chao Feng, Hongjin Lu, Linjie Li, Chung-Ching Lin, Kevin Lin, Furong Huang, and Lijuan Wang. Sota with less: Mcts-guided sample selection for data-efficien visual reasoning self inff-improveettt.. ArXiv preprint, 0as/2504.09934 2025b.
Yiping Wang. Qing Yang. Zhiyuan Zeng, Liliang Ren, Lucas Liu, Baolin Peng, Hao Cheng. Xuehai He, Kuan Wang, Jianfeng Gao, et al. Reinforcement learning for reasoning in large language models with one training example. ArXiv preprint, 25ss22504.00511, 2025c.
Tong Wei, Yijun Yang, Junliang Xing, Yuanchun Shi, Zongqing Lu, and Deheng Ye. Gtr: Guided thought reinforcement prevents thought collapse in rl-based vlm agent training. ArXiv preprint, abs/2503.08525. 2025a.
Yana Wei, Liang Zhao, Jianjian Sun, Kangheng Lin, Jisheng Yin, Jingcheng Hu, Yinmin Zhang, En Yu, Haoran Lv, Zejia Weng, Jia Wang, Chunrui Han, Yuang Peng, Qi Han, Zheng Ge, Xiangyu Zhang, Daxin Jiang, and Vishal M. Patel. Open vision reasoner: Transferring linguistic cognitive behavior for visual reasoning, 2025b. URL /tttps://axxiv.orr/t wwwps//rggwaa/////poppaaaaaaaaap 250.. 05255.
Yichen Wei, Yi Peng, Xiaokun Wang, Weijie Qiu, Wei Shen, Tianyidan Xie, Jiangbo Pei, Jianhao Zhang, Yunzhuo Hao, Xuchen Song, et al. Skywork r1 Multimodal hybrid reinforcement learning for reasoning. ArXiv preprini, abs/2504. 166566 2025c.
Yize WU, Ke Gao, and Yanjun Wu. Easyspec: Layer-parallel speculative decoding for efficient multi-gpu utilization. ArXiv preprint, abs/2502.02493,, 2025.
Xin Xiao, Bohong Wu, Jiacong Wang, Chunyuan Li, Xun Zhou, and Haoyuan Guo. Seeing the im- age: Prioritizing visual correlation by contrastive alignment. In Amir Globersons, Lester Mackey, Danielle Belgrave, Angela Fan, Ulrich Paquet, Jakub M. Tomezak, and Cheng Zhang (eds.), Ad- vances in Neural Information Processing Systems 38: Annual Conference on Neural Information Processing Systems 2024, NeurlPS 2024, Vancouver, BC, Canada, December 10 15, 2024, 2024.
Yi Yang, Xiaoxuan He, Hongkun Pan, Xiyan Jiang, Yan Deng, Xingtao Yang, Haoyu Lu, Dacheng Yin, Fengyun Rao, Minfeng Zhu, et al. R1-onevision: Advancing generalized multimodal rea- soning through cross-modal formalization. ArXiv preprint, 10./25033.161.. 2025.
Huanjin Yao, Qixiang Yin, Jingyi Zhang, Min Yang, Yibo Wang. Wenhao Wu, Fei Su, Li Shen, Minghui Qiu, Dacheng Tao, and Jiaxing Huang. RI RI-share VL: Incentivizing reasoning capabilities of multimodal large language models via share-GRPO. In The Thirty-ninth Annual Conference Neural Information Processing Systems, 2025. URL https: httptt::://ppeeeeeeeeeee wpwprevieeeeeeeeeeee/ee net forum? dd=kMP IT TXXXKC.
14

---

Published as a conference paper at ICLR 2026
Qinghao Ye, Haiyang Xu, Guohai Xu, Jiabo Ye, Ming Yan, Yiyang Zhou, Junyang Wang, Anwen Hu, Pengcheng Shi, Yaya Shi, et al. mplug-owl: Modularization empowers large language models with multimodality. ArXiv preprint, ibs/2304.14178, 2023.
Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Tiantian Fan, Gaohong Liu, Lingjun Liu, Xin Liu, et al. Dapo: An open-source IIm reinforcement learning system at scale. ArXiv preprint, abs/2503.14477 2025.
Xiang Yue, Yuansheng Ni, Tianyu Zheng, Kai Zhang, Ruoqi Liu, Ge Zhang, Samuel Stevens, Dongfu Jiang, Weiming Ren, Yuxuan Sun, Cong Wei, Botao Yu, Ruibin Yuan, Renliang Sun, Ming Yin, Boyuan Zheng. Zhenzhu Yang, Yibo Liu, Wenhao Huang, Huan Sun, Yu Su, and Wenhu Chen. MMMU: A massive multi-discipline multimodal understanding and reasoning benchmark for expert AGI. In IEEE/CVF Conference on Computer Vision and Pattern Recog- nition, CVPR 2024, Seattle, WA, USA, June 16-22, 2024, pp. 9556-9567. IEEE, 2024. doi: D0.1099CVVP5222333224000000
Xiang Yue, Tianyu Zheng, Yuansheng Ni, Yubo Wang. Kai Zhang, Shengbang Tong, Yuxuan Sun, Botao Yu, Ge Zhang. Huan Sun, Yu Su, Wenhu Chen, and Graham Neubig. MMMU-pro: A more robust multi-discipline multimodal understanding benchmark. In Wanxiang Che, Joyce Nahende, Ekaterina Shutova, and Mohammad Taher Pilehvar (eds.), Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume I: Long Papers), pp- 15134-15186, Vienna, Austria, 2025a. Association for Computational Linguistics. ISBN 979-8- 89176-251-0. doi: 10. 10153/86611220 w0w.o/a.cl........ 736.
Yang Yue, Zhiqi Chen, Rui Lu, Andrew Zhao, Zhaokai Wang, Shiji Song, and Gao Huang. Does inforcement learning really incentivize reasoning capacity in lims beyond the base model? ArXiv preprint, abs/2504.13837, 2025b.
Weihao Zeng. Yuzhen Huang, Qian Liu, Wei Liu, Keqing He, Zejun Ma, and Junxian He. Simplerl- zoo: Investigating and taming zero reinforcement learning for open base models in the wild. ArXiv preprint, abs/2503. 18892, 2025a.
Weihao Zeng, Yuzhen Huang, Qian Liu, Wei Liu, Keqing He, Zejun Ma, and Junxian He. Simplerl- zoo: Investigating and taming zero reinforcement learning for open base models in the wild. In Second Conference on Language Modeling, 2025b.
Jingyi Zhang, Jiaxing Huang, Huanjin Yao, Shunyu Liu, Xikun Zhang, Shijian Lu, and Dacheng Tao. R1-vl: Learning to reason with multimodal large language models via step-wise group relative policy optimization. ArXiv preprint, abs/2503. 12937, 2025.
Renrui Zhang, Dongzhi Jiang. Yichi Zhang, Haokun Lin, Ziyu Guo, Pengshuo Qiu, Aojun Zhou, Pan Lu, Kai-Wei Chang, Yu Qiao, et al. Mathverse: Does your multi-modal I1m truly see the diagrams in visual math problems? In European Conference Computer Vision, pp. 169-186. Springer, 2024.
Yaowei Zheng, Junting Lu, Shenzhi Wang, Zhangchi Feng, Dongdong Kuang, and Yuwen Xiong. Easyrl: An efficient, scalable, multi-modality rl training framework. https::/giie /ttpt/gttbt/ttt///// com/ Siiyouga/EasyR1,22 2025.
Hengguang Zhou, Xirui Li, Ruochen Wang, Minhao Cheng, Tianyi Zhou, and Cho-Jui Hsieh. R1- zero's" aha moment" in visual reasoning on non-sft model. ArXiv preprint, abs/2503.05132, 2025.
Deyao Zhu, Jun Chen, Xiaoqian Shen, Xiang Li, and Mohamed Elhoseiny. Minigpt-4: Enhancing vision- Laiguage understanding with advanced large language models. In The Twelfth Interna- fional Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024. OpenReview.net, 2024.
15

---

Published as a conference paper at ICLR 2026
A IMPLEMENTATION DETAILS
General Conversation Template
blin_starl>systen
You are a helpful assistant.<limg
> endi>
biim_starti> wwtpe//wioosttttiiiiaeeeeeaaaaae
end
equery__qesttonn
(instruction prompt) <lim <lim_end|>
fim__stat>>assistann
Instruction Prompt for ThinkLite-VL
You FIRST think about the reasoning process as an internal monologue then provide the final answer. The reasoning process MUST BE enclosed within <think> </ think tags. The final answer MUST put in \boxedf.
Instruction Prompt for OpenVLThinker
Your final answer MUST BE put between <answer> </answer>.
Instruction Prompt for VL-Rethinker
Please reason step by step, and put your final answer within \boxedf.
Figure 8: Prompt templates of different RFT experts in $3.2.
A.1 INTRODUCTION FOR BENCHMARKS AND REASONING EXPERT MODBLS
To rigorously evaluate multimodal reasoning, we adopt several recent benchmark datasets designed to probe different dimensions of model understanding.
Math (Zhang al., 2024) contains 3,940 samples (testmini split) and tests model's ability to interpret diagrams and solve multi-subject visual math problems presented in varying multimodal formats.
Math Vista (Lu et al., 2024) provides 1,000 examples (testmini split) of visual mathematical reason- ing tasks drawn from real-world diagrams, charts, and images, with both open-ended and multiple- choice formats.
Vision (Wang et al., 2024a) includes 3,040 visual math problems across 16 mathematical dis- ciplines, sourced from real competition problems and annotated five distinct difficulty levels.
MMMU (Yue et al., 2024) supplies validation split of 900 samples covering a broad range of college-level disciplines, designed to assess deliberate, multi-disciplinary multimodal understand- ing.
MMMU-Pru (Yue et al., 2025a) comprises 5,190 items across Standard setting (4- and 10-option MC) and a Vision-only setting where questions are embedded directly in images, designed to stress integrated dosull-exttu understanding across diverse academic subjects by filtering out text-only- solvable items and augmenting candidate options.
EMMA (Hao al., 2025) offers 2,788 multimodal problems spanning math, physics, chemistry, and coding, targeting organic cross mods- reasoning that cannot solved by treating modalities independently.
16

---

Published as a conference paper at ICLR 2026
MathVerse Answer Extraction Prompt
am providing you response from model to math problem, termed 'Model Response: You should extract the answer from the response as 'Extracted Answer'. Directly output the extracted answer with explanation Ankn
MathVerse Answer Score Prompt
Below are two answers to math question. Question is [Question], [Standard Answer] is the standard answer to the question, and [Model_answel is the answer extracted from a model's output to this question. Determine whether these two answers are consistent.
Please note that only when the [Model_answel _answer] completely matches the [Standard Answer] means they consistent. For non-multiple-chiic questions, if the meaning is expressed in the same way, it is also considered consistent, for example, 0.5m and 50cm.
they are consistent, Judement is 1; they are different, Judement O.
Figure 9: Math Verse extraction and scoring prompt for gpt-4o-mini judge.
E1-Onevision-Bentn (Yang et al., 2025) consists of 942 samples organized into five academic do- mains and five difficulty levels, offering a structured framework to benchmark model performance across educational subject areas.
In addition, we choose the following public models as reasoning experts based on differing training paradigms and data selection strategies:
Open VLThinker-BB (Deng al., 2025): Enhances reasoning through iterative alternation be- tween SFT and RFT stages using progressively challenging questions.
ThinkLite-VL-7B (Wang et al., 2025b); Improves training efficiency via Monte Carlo Tree Search (MCTS)-guided data selection to achieve data-efficient RFT paradigm.
VL-Rethinker-7B (Wang et al., 2025a): Addresses the vanishing advantage issue through Selec- tive Sample Replay and enforces self-reflection using the Forced Rethinking technique.
A.2 PASS @ EVALUATION DETAILS
For the pass@k & evaluation in $4.2, we use a temperature of 0.6 and top-p value of 0.95, allowing maximum generation of 4,096 tokens during generation. To prevent the model from randomly guessing the multiple-choice answer through repeated sampling, we filtered the Math Vision dataset subset where no multiple-choice problems included, resulting in 114 samples.
We adopt an extended version of the pass lass@k k metric originally proposed for code generation by (Chen al., 2021). Given a dataset D and particular problem z; D, generate n indepen- dent outputs from the model, where 2 k and record & as the number of outputs that pass the mathematical checker. The pass@k score is then estimated using the following unbiased formula:
pass @k Ee,ND
This estimator provides a low-variance measurement the proportion of problems solved correctly within & attempts, applicable for all l <
17

---

Published as a conference paper at ICLR 2026
MMMU Validation Set Sample #898
controversia groups may mislabe <image induude:
<image
Swee2.5-VL-322B-1
VL-Rethinker-78
ProxyThinker-322
To determine the correct answer, let's analyze the image and the options provided:
To determine the correct answer, let's analyze the image and the options provided:
To determine the correct answer, let's analyze the image and the aptions provided:
1. Image Analysis:
1. Image Analysis: image shows group people gathered circle, with one individual standing
Image Analysis:
The image shows group of people sitting circle, dressed white clothing, which often associa ted with certain sp apirttual religious groups.
image group people sitting circle, dressed white, with one person standing the center. This setup reminiscent spiritual religious gathering.
center, possibly leading meditation spiritual practice. The attire and the setting suggest spiritual communal gathering, which is characteristic of certain in New Age spiritual movements.
2. Option Analysis:
A. Scientology and the Hare Krishna:
2. Option Anelysis:
-A. Scientology and the Hare Krishna: However, they not typically associated with...
The image could be consistent with the Hare Krishna movement, as they aften gather in circles for kirtan (chanting) and meditation.
2. Option Analysis:
A. Scientology and the Hare Krishna:
Neither fits the description of the
B. the Peoples Temple and Heaven's Gate: ... image could be associated with such groups due to the organized and somewhat cult like appearance...
image well.
B. the Peoples Temple and Heavens
Given the analysis, the group in the image most consistent with the Hare Krishna movementt which often mislabelled or associated with controversial groups.
Gate:
This description aligns more closely with the image.
Given the and lyiis, the group image is most closely associated with the controversial groups known for like behavior tragic endinngs \boxed(B)
Therefore, correct answer \boxed(B).
\boxed(A)
Figure 10: A kmowledge-intensive test case from the MMMU Val set with reasoning trajectories from Qwen2255-V..32B-Innmin VIL-Rethinker-7B and PROXYTHINKER-325
A.3 PROMPT TEMPLATES
We provide the prompt templates different reasoning experts in Figure 8. For Math Verse evalua- tion, we employ the prompt template in Figure 9 to first extract the answer and then score the answer using gpt- 1ec--4o-min as judge, following VLMEvalKit (Duan et al., 2024).
A.4 FAILURE CASE ANALYSIS
achieves consistent improvements across multiple -nntoninn-itttsvveee datasets, we observe that struggles to deliver statistically significant gains on certain knowledge- intensive benchmarks, such MMMU (Yue et al., 2024) limitation also present in full-scale RFT experts. To illustrate this, we present a test case from the MMMU Val set in Figure 10, comparing the reasoning processes of the large base model (Qwen2.5-4 S-VL-33BBIIntteee expert (VL-Rethinker-7BB and PROXYTHINKER. The results show that the small reasoning expert fails to accurately validate the knowledge content of answer choices, likely due its limited model capacity. This type knowledge verification is particularly challenging to learn via reinforcement learning with verifiable rewards. As result, ProxyThinker inherits this limitation well.
Although
the small reasoning
18

---

Published as a conference paper at ICLR 2026
B ADDITIONAL ANALYSIS
B.1
DOMAIN-AGNOSTIC REASONING ACTIVATION
Table 4: Performance (Accuracy %) on EMMA and RI-Bench benchmarks. Best scores in bold.
EMMA Overall
R1-Bench
Model
Expert
Overall
Math
Physies
Biology
Chemistry
Qwen2.5-VI.-7B
21.5
32.1
31.2
31.3
52.2
43.8
OpenVLThinker-7B
24.9
32.9
29.4
34.2
50.0
35.2
Qwen2.5-Ve -VL-32BB
31.1
49.4
41.9
51.1
61.9
61.0
Qwen2.5-VL-322 VL-322B
Open OPenVLThinker-77 7B
35.3
53.0
46.2
55.4
69.4
62.9
To further substantiate the claim that PROXYTHINKER reasoning transferability is domain-agnostic, we provide a detailed breakdown of its performance on EMMA and RI-OneVision-Bente (R1- Bench). The latter one covers reasoning- tasks across four distinct disciplines: Math, Physics, Biology, and Chemistry. Table 4 reports the per-domain accuracies for the base model, the expert model and PROXYTHINKER.
Open WeThinkeee---... despite being relatively under-aligned visual reasoner, exhibits only marginal gains over the smaller Qwen2.5-VL-7h baseline, with improvements only in the Physics subject. However, when PROXYTHINKER is applied, we observe consistent improvements across every do- main. This validates that ProxyThinker's transfer ability is not restricted specific subject but instead activates domuin-agnosiic reasoning capabilities. Such findings provide concrete evidence that PROXYTHINEE induces structural shifts the output distribution of experts, enabling system- atic cross-domain gains.
B.2 QUALITY OF RFT EXPERT
Table 5: Impact of expert quality PROXYTHINKER. Performance (Accuracy %) across three mathematical reasoning benchmarks.
Model
Expert
Amateur
MathVista
Math
Math Vision
Qwen2.5-VL-32B
38.4
VL.-Rethinker-7B
74.9
54.2
32.3
Qwen2.5- VL-32B
VL-Rethinker-7B
Qwen2.5-VL-7B
78.1
55.1
39.2
VLAA-Thinker:33B Qwen2.5-VL-32B
61.0
36.4
24.4
VLAA- Thinker-3B 3B Qwen2.5-VL-3B
51.5
49.2
37.8
To gain deeper insights into the role of RFT experts, we further examine how under perfor-erforming experts may introduce negative effects PROXYTHINKER. In main experiments, we primarily choose 32B/72B models as the large base and 7B models as reasoning experts, since 7B is gener- ally the smallest scale capable of supporting effective RFT-based visual reasoning. Smaller models tend to struggle with capturing consistent reasoning patterns, leading to suboptimal transfer in the PROXYTHINKER setting.
Table 5 reports results with VLAA-Thinkers Using "LAA-Thinker: as an expert not only fails improve performance but also significantly degrades results on Math Vista (from 61.0 to 51.5), yielding performance even worse than the 3B expert alone. This demonstrates that the quality and scale the RFT expert are crucial: a poorly aligned or undersized expert can misguide the base model and introduce negative transfer effects.
(Chen et al., 2025a), an early visual reasoning expert.
B.3 LANGUAGE-ONIA BENCHMARKS
We additionally evaluate PROXYTHINKER a language-onle setting in Table to verify that the method is not vision-specifie. All RFT reasoning models used expert model taken from
19

---

Published as a conference paper at ICLR 2026
Table 6: Performance (Aceuracy %) on language-only mathematical reasoning benchmarks. FronkThinker is applied to base LLMs using small RFT reasoning experts as proxies. All RL models are taken from SimplcRL-Zoo (Zeng et al., 2025b). A shows the average relative improvement in % compared to the base model. Full-scale RFT expert is highlighted with gray.
Model
Expert
Math500
AIME'24
GSM8K
A
Qwen2.5-1.5B-R.
9.2
0.0
28.4
Qwen2.5-7B-RL
74.4
10.0
90.9
1lama3.1-8B-R.
9.4
0.0
13.3
Llama3.1 70B
22.2
3.3
42,4
0.0
1 1 70B + ProxyThinker-8B
1-BB-RL
27.0 (+4.8)
3.3 (0.0)
52.9 (+10.5)
+5.1
Qwen2.5-14B
66.4
3.3
90.4
0.0
Qwen2.5-14B ProxyThinker-1.5B
Qwen2.5-1.5B-RI.
73.4 (+7.0)
10.0 (46.7)
92.9 (+2.5)
+5.4
Qwen2.5-14B
78.4 (+12.0)
10.0 (46.7)
94.6 (+4.2)
+7.6
Qwen2.5-32B
68.8
6.7
92.0
Qwen2.5-32B ProxyThinker-1.5
Qwen2.5-1.5B-RI.
75.2 (+6.4)
13.3 (46.6)
94.2 (+2.2)
+5.1
Qwen2.5-32B ProxyThinker-7B Qwen2.5-7B-R
81.6 (+12.8)
16.7 (+10.0)
94.7 (+2.7)
+8.5
Qwen2.5-32B-RL
81.8 (+13.0)
16.7 (+10.0)
95.5 (+3.5)
+8.8
SimpleRL-Zov (Zeng al., 2025b). The decoding setup strictly follows $3.1 in the main paper. We select the Math500 (Lightman et al., 2023), AIME24 (Art of Problem Solving) and GSMSK (Cobbe et al., 1)21) test split preliminary testbeds to evaluate
We find that PROXYTHINKER yields consistent gains on all base language models and benchmarks. For Llama 3.1 70B model (Grattafiori et al., 2024), observe noticable gains Math500 and GSM8K, resulting in a solid average relative gain. This also indicates that beyond the Qwen model series. For Qwen2.5-32B model, draw another observation which can not be seen in VLM setting due to the absence of effective small expert models upgrading the expert from 1.5B to 7B further boosts overall performance essentially match the fully RL-fine- tuned 32B ceiling. This echoes the conclusion we have in the main paper: a stronger RFT expert tends to provide more structured reasoning paths, enhancing the base model's reasoning abilities more effectively. Overall, these results confirm that PROXYT THIXYTH is generic method that is effective beyond VLM.
works
C COMPUTATIONAL OVERHEAD ANALYSIS
Setup and assumptions. We analyze the computational overhead of PROXYTHINKER by com- paring the training FLOPs of small vs. large visual reasoners and the additional inference FLOPs introduced by the collaborative decoding with Amateur and Expert models. Following the reports of VL-Rethinker (Wang et al., 2025a), training VL-Rethinker-77B uses 2 x 8 A800 (80GB) GPUs hours, while VL-Rethinker-332B uses 10 8 A800 (80GB) GPUs for 60 hours. assume peak bf16 throughput of an A800 (80GB) GPU to be 312 TFLOPS and conservative training-ting time FLOPs utilization 0.5 to account for communication and system overheads.
Training FLOPs. Under the above assumptions, the training FLOPs are
7B: 16 (#GPUs) x 28,800 3.12 10t4 FLOPS x 0.5 7.19 x 10t9 FLOPs,
32B: 80 (#GPUs) 216,000 3.12 10 FLOPS 0.5 2.70 10 FLOPs.
Hence, training a 32B reasoner costs approximately
2.70 10 7.19 x 10 ~ 2.62 x 20 FLOPs
more than training 7B reasoner.
20

---

Published as a conference paper at ICLR 2026
Inference FLOPs for collaborative decoding. According to scaling-law estimates (Kaplan al., 2020), the forward pass of a Transformer requires roughly 2x (number of parameters) FLOPs per generated token.' Therefore, the per-token decoding cost of two 7B models (Amateur and Expert) is
2 x 2 x 10 2.8 10 FLOPS/token.
Let Auain 2.62 x 10 be the extra training FI.OPs of 32B over 7B. The number of tokens 7 after which the accumulated collaborative-decoding overhead matches Arain satisfies
Atrain
T 2.8 1010 9.37 x 102D tokens.
Thus, in terms of FLOPs, the additional inference overhead of PROXYTHINKER-33 does not out- weigh the training cost until roughly 9.37 x 10 generated tokens, i.e. around 93 billion tokens. Importantly, PROXYTHINKER -32B and VL.-Rethinker-322 achieve comparable accuracy, while the former avoids the full training budget of 32B reasoner.
Latency in practice. FLOPs do not map one-to-one to wall-clock latency. With our cus- tomized vLLM-based implementation for multi-model collaborative decoding. Table 3 shows that ProxyThinker-322 incurs only an ~ 11% latency overhead on the same hardware (from 451 s to 501 s), indicating that the practical runtime penalty is small relative to the savings from avoiding 32B training.
Memory overhead. As a modei collaborative decoding approach, PROXYTHINKER entails additional memory to host multiple model weights. Using DeepSpeed (Rasley et al., 2020) Memory Profier, we measure the GPU memory required to load bfloat1 6 weights:
Model Size
7B
32B
72B
ProxyThinker-32B
ProxyThinker-72B
Memory (GiB) 15.31
64.44
138.86
(+47.6%%
169.48 (+22.0%)
The overhead introduced by PROXYTUINKER is approximately fixed in absolute terms, hence its relative percentage decreases with the base model size (e.g.. +22% at 72B). This is negligible com- pared with the resources needed to fully train an RFT expert: e.g., EasyR1 (Zheng al., 2025) estimates that training 32B RFT model with AMP requires at least 16 x 80GB GPUs (== 960 GB of device memory). Moreover, modern inference frameworks typically reserve most GPU memory for KV caches, so the extra weight memory is less constraining in practice. Prior collaborative- decoding multi-model methods (e.g., contrastive decoding (Li et al., 2023b), ProxyTuning (Liu et al., 2024a)) exhibit similar memory characteristics.
Takeaways and limitations. PROXYTHINKER an inference-time strategy that trades modest multi-model decoding overhead for avoiding the substantial training cost of scaling single rea- soner. There must exist tipping point where cumulative inference cost exceeds the avoided training budget, and our conservative calculation places this far beyond typical academic inference volumes. Additionally, the requirement of simultancously hosting multiple models leads to fixed memory premium whose relative impact diminishes for larger base models. Finally, not all academic groups can access clusters with 80 high-end GPUs. From a systems and economic perspective, PROXY- THINKER offers a practical path high reasoning quality with substantially lower upfront compute.
D LIMITATIONS
While PROXY THINKER demonstrates strong empirical performance and practical scalability, several limitations remain. First, our method relies on access to both an RFT expert and an amateur model sharing the same vocabulary and preferably sharing the same model architecture. This requirement may limit applicability in settings where high-quality RFT experts are unavailable or where model architectures are not easily aligned. Existing cross-model alignment methods, such as CDM (Chen et al., 2025c), might help mitigate this issue. In addition, our current experiments focus primarily
We deli deliberately ignore the modest dependence sequence length and note that modern KV cache designs further reduce effective compute per token, our estimate conservative (i.c., overestimate).
21

---

Published as a conference paper at ICLR 2026
visual reasoning benchmarks. The effectiveness of PROXYTHINKER in other domains, such as natural language-only reasoning or real-world embodied tasks, remains to be thoroughly explored.
E THE USE OF LARGE LANGUAGE MODELS (LLMs)
We employed large language models (LLMs) only for language refinement, such as correcting gram- mar and enhancing clarity. The LLM was not involved in generating ideas, conducting analysis, or contributing substantive content. All conceptual framing, methodological design, results, and inter- pretations were carried out solely by the authors.
22


Reviews:
[{"review_id": "Ihnr5dDc4f", "reviewer": "Reviewer_x1re", "summary": "The paper proposes a training-free method to improve the visual reasoning of large vision-language models (LVLMs). \nThe core problem it addresses is the immense computational cost of full-scale Reinforcement Fine-Tuning (RFT) on large models. \nThe paper proposes to use logit delta to transfer the \"slow-thinking\" reasoning behaviors (like self-correction) from the small expert to the large base,", "questions": "1. Could you elaborate on the Pass@k trade-off? Do you view the loss of reasoning diversity (worse Pass@k performance) as an acceptable price for the gain in Pass@1 performance? Does this imply RFT-style reasoning (at least as captured by this method) inherently prunes other valid reasoning paths?\n\n2. Could you elaborate on why a simple logic delta could work so well, and what might be the premise of the success achieved from applying this logit delta? Following on Weakness #2, what is in the delta $(z_{\\text{expert}} - z_{\\text{Amateur}})$? Have you analyzed its composition? For instance, does the delta vector's direction change over a reasoning trace, perhaps pushing toward \"sub-goal\" tokens at the start and \"verification\" tokens near the end? A temporal analysis of the delta's top-k components would be fascinating.\n\n3. The knowledge bottleneck (Appendix A.4) is a key limitation. How do you see this method performing on tasks that are a tight mix of reasoning and knowledge? Is there a risk that the small, knowledge-poor expert could misguide the large, knowledge-rich base model into a factually incorrect but \"well-reasoned\" answer?\n\n4. Given that a poor expert causes \"negative transfer\" (Table 5), how should one practically determine if a small expert is \"good enough\" to be a PROXYTHINKER? Its own standalone accuracy seems unreliable (e.g., the OpenVLThinker-7B had only 25.3% on MathVision but gave a positive boost). Is there a \"floor\" of capability or a metric (beyond final accuracy) that predicts positive transfer?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The paper spot on a valuable problem of training cost of RL. \n\n2. The paper delivers a surprising and useful finding that the reasoning behavior can be transfered from small expert model to large base model, alleviating the burden of training cost. \n\n3. The authors have thoroughly addressed the practical viability of using three models at inference. By leveraging vLLM and optimized tensor parallelism, they demonstrate a ~38x speedup over a naive implementation. Their system adds only a minor latency overhead (~11%) compared to running a single large model, making it a genuinely practical and economical solution for deploying advanced reasoning capabilities.", "weaknesses": "1. A Significant Trade-off in Reasoning Diversity (Pass@k): The paper's claim of \"striking a balance\" in reasoning exploration (Sec 4.2) is an oversimplification. The data in Figure 5 clearly shows that while Pass@1 performance (greedy decoding) is improved, the Pass@k performance for $k>4$ drops below that of the unguided base model. This suggests the guidance narrows the large model's reasoning diversity, forcing it down the single \"slow-thinking\" path favored by the small expert. This trade-off—improving the single best answer at the cost of exploratory reasoning capacity—is a major weakness that is not sufficiently discussed.\n\n2. Unexplored Mechanism of Action: The paper's central hypothesis—that the logit delta $(z_{\\text{expert}} - z_{\\text{Amateur}})$ isolates a scalable, model-agnostic \"reasoning vector\"—is demonstrated that it works but not why it works. The ablation in Figure 4 proves that subtracting the amateur model is critical, but the paper provides no deeper analysis into the nature of this delta. Is it merely up-weighting a shared vocabulary of \"reasoning tokens\" (e.g., \"Wait,\" \"let's check\"), or is it manipulating a deeper, abstract representation of the reasoning process? The simplicity of the mechanism makes this lack of explanation a significant gap.\n\n3. The method's effectiveness is strictly bounded by the small expert's capabilities.\n- As admitted in Appendix A.4, the method fails on knowledge-intensive tasks (like MMMU) where the small expert lacks the requisite knowledge. It cannot transfer knowledge, only reasoning patterns, and a correct pattern applied to incorrect knowledge still fails.\n\n- As shown in Appendix B.2, a low-quality expert results in negative transfer, actively harming the large model's performance. This implies the method is not a universal booster; it requires a small expert that is both knowledge-sufficient for the domain and has been RFT-trained to a high-quality \"floor.\"", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "EKsQT6S3Hz", "reviewer": "Reviewer_cUEK", "summary": "The paper proposes PROXYTHINKER, a training-free decoding method that transfers “slow-thinking” visual-reasoning behaviors from a small RFT-trained expert VLM to a larger base VLM by adding a logits delta at every generation step. The approach consistently improves Pass@1 on visual math and multi-disciplinary benchmarks and sometimes approaches or even surpasses a fully RFT-trained large model. The authors further implement a multi-model scheduling scheme in vLLM to reduce the overhead of running three models at inference.", "questions": "# Questions:\n\n1. Did you try to apply the proposed method on language-only reasoning tasks? What are the challenges and performance?\n\n2. Did you observe any negative side effects (e.g. increased repetition, verbosity, or unnatural reasoning) from adding the logit shift?\n\n3. On MMMU/MMMU-Pro the gains are negligible. Could you provide a detailed diagnosis?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "# Strengths\n\n1. Practical inference-time approach: The method requires no additional training of the large model, addressing the high cost of RFT for VLMs. It is simple to implement (just logit arithmetic) and can leverage existing small RFT models.\n\n2. Empirical gains on visual reasoning tasks: ProxyThinker consistently improves accuracy on spatial and math reasoning benchmarks. In many cases the base VLM closes most of the gap to a fully RFT-trained model. For example, applying a small visual “OpenVLThinker-7B” expert to Qwen2.5-32B raises MathVision accuracy from 38.4% to 40.8%, slightly exceeding the 40.5% achieved by the full-scale RFT model.", "weaknesses": "# Weaknesses\n\n1. Overall, the novelty of the proposed method is under par. ProxyThinker closely mirrors existing logit-guidance techniques. The existence of current methods, like DExperts, Proxy-Tuning and DoLa, makes the ProxyThinker not novel enough.\n\n2. Weak justification for VLM-specific focus. The authors motivate the work by the expense of RFT on large VLMs, but they do not identify any modality-specific challenge that makes ProxyThinker inherently necessary for vision. The technique appears generic and could equally apply to language-only reasoning tasks.\n\n3. Although the gains are statistically significant, they are small (often only a few percentage points). Crucially, ProxyThinker requires simultaneous inference of multiple models (base + expert + amateur), which increases serving complexity, latency, and cost; in many settings this overhead may not be justified by the marginal improvements over fully RFT-trained baselines.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "MA8RWyaGnM", "reviewer": "Reviewer_nfVY", "summary": "This paper proposes PROXYTHINKER, a novel and training-free decoding strategy designed to elicit multi-step reasoning from large vision-language models without expensive training. The method operates at inference time, steering a large base model's generation by applying a guidance signal derived from the logit difference between a small, RFT expert model and its non-RFT amateur counterpart. Experiments compellingly demonstrate that this approach effectively transfers the reasoning skills, enabling a 32B model to achieve performance on complex visual reasoning benchmarks that is comparable to, and in some cases surpasses, its fully RFT-trained equivalent, showcasing a highly efficient path to enhancing model reasoning.", "questions": "See Weaknesses", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The paper is exceptionally well-written, clearly articulated, and easy to follow. The motivation is strong and well-grounded, directly addressing the significant and timely challenge of improving the scalability of reinforcement learning for large-scale VLMs.\n\n2. The core idea is highly intuitive and logically sound. It builds upon the key insight that RFT methods like GRPO often do not introduce new external knowledge but rather reshape the model's output distribution to elicit a step-by-step reasoning process. The authors astutely hypothesize that if this capability is already latent within a large instruction-tuned model, it can be unlocked via a direct, inference-time decoding control mechanism. This work provides a feasible and effective solution that validates this hypothesis with impressive results.\n\n3. This research could pioneer a new and more scalable paradigm for building powerful reasoning systems. It suggests a practical \"division of labor,\" where intensive and costly reinforcement learning is focused on creating compact, specialized \"expert\" models. Larger, more general models could then be trained primarily with SFT and efficiently endowed with these specialized reasoning skills at inference time through interaction, paving a more sustainable path for model development.", "weaknesses": "1. Lack of Principled Analysis on Expert Model Selection: The paper's primary contribution relies on guidance from a small \"expert\" model, yet the criteria for selecting this expert seem somewhat ad-hoc. While the authors experiment with three public models chosen based on \"differing training paradigms and data selection strategies\" (line 259), the paper falls short of addressing a crucial question: What properties define an optimal expert for the PROXYTHINKER framework? A deeper investigation is needed into the essential characteristics an expert must possess. For instance, is it absolute accuracy, the diversity of reasoning paths, or the magnitude of the logit shift post-RFT? A more solid contribution would involve discussing these factors and perhaps even exploring how one might intentionally train a more effective, targeted expert model specifically for this guidance role. Such an analysis would significantly enhance the paper's impact and provide more concrete guidance for the community.\n\n2. Limited Generalizability of Base Models: The empirical validation exclusively uses models from the Qwen2.5-VL series as the base model. While these are strong models, this narrow selection raises concerns about the generalizability of the proposed method. It is plausible that the success of PROXYTHINKER is partially contingent on the specific latent reasoning capabilities inherent in the Qwen architecture and its instruction-tuning process. The paper's claims would be substantially more robust if the authors demonstrated effectiveness on other diverse and widely-used LVLM families (e.g., InternVL / LLaVA-OV). Without such experiments, it remains an open question whether this is a universally applicable decoding technique or one that works particularly well for a specific class of models.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "fJdnj36I2W", "reviewer": "Reviewer_rAEa", "summary": "This paper introduces ProxyThinker, a test-time decoding method that transfers visual reasoning abilities from small RFT models to LVLM without any additional training. By applying the token-level logits difference between a small RFT \"expert\" and its non-RFT \"amateur\" counterpart to the large base model's output distribution, ProxyThinker effectively elicits \"slow-thinking\" behaviors such as self-verification and backtracking. The method achieves consistent improvements on challenging benchmarks, often matching or surpassing full RFT-trained models. Analyses show that ProxyThinker, not only enhances quantitative performance but also induces interpretable reasoning behaviors.", "questions": "See above weaknesses.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "1. This paper presents clear motivation and method. The method is not complex but effective, which is easy to follow.\n\n2. The system implementation is carefully engineered on top of vLLM, with tensor-parallel scheduling and asynchronous multi-model inference that achieve up to 38× speedup.\n\n3. Experiments are thorough across five visual reasoning benchmarks and two model scales (32B/72B).  The results are consistent and reported with clear baselines and ablations.", "weaknesses": "1. The paper begins by arguing that RFT does not teach new knowledge but merely amplifies existing reasoning patterns.  However, ProxyThinker itself inherits exactly those amplified patterns without introducing new reasoning capabilities. The proposed issue is not well resolved.\n\n2. The related-work section omits some concurrent or near-concurrent works on multimodal reasoning with RL or decoding-time control, such as\n\n- Open Vision Reasoner: Transferring Linguistic Cognitive Behavior for Visual Reasoning (Wei et al., 2025)\n\n- R1-ShareVL: Incentivizing Reasoning Capability of Multimodal Large Language Models via Share-GRPO (Yao et al., 2025)\n\n- Semi-Off-Policy Reinforcement Learning for Vision-Language Slow-Thinking Reasoning (Shen et al., 2025)\n\n3. While ProxyThinker improves on math- and spatial-reasoning benchmarks, gains on knowledge-intensive datasets such as MMMU-Pro are marginal or even negative with weaker experts.  This suggests the transferred signal mainly benefits structural reasoning rather than factual grounding.\n\n4. ProxyThinker requires simultaneous execution of three large models: a base model plus two auxiliary 7B-scale models (expert and amateur).  This setup substantially increases both memory footprint and compute parallelism requirements, which could be prohibitive for typical deployment environments where even a single 32B LVLM may require multiple GPUs.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)