Mixture-of-Visual-Thoughts: Exploring Context-Adaptive Reasoning Mode Selection for General Visual Reasoning
Authors: Zejun Li
Yingxiu Zhao
Jiwen Zhang
Siyuan Wang
Yang Yao
Runzhou Zhao
Jun Song
Bo Zheng
zhongyu wei
Venue: ICLR 2026 Poster (2026)
Keywords: visual reasoning
adaptive reasoning
multimodal large language models
Primary area: foundation or frontier models, including LLMs
TL;DR: We introduce an mixture-of-visual-thoughts paradigm that unifies different visual reasoning modes within a model and guides it to adaptively select the appropriate mode based on context, achieving consistent gains across various scenarios.

Abstract: Current visual reasoning methods mainly focus on exploring specific reasoning modes. Although improvements can be achieved in particular domains, they struggle to develop general reasoning capabilities. Inspired by this, we propose a novel adaptive reasoning paradigm, $\underline{\text{M}}$ixture-$\underline{\text{o}}$f-$\underline{\text{V}}$isual-$\underline{\text{T}}$houghts (**MoVT**), which unifies different reasoning modes within a single model and guides it to select the appropriate mode based on context. To achieve this, we introduce **AdaVaR**, a two-stage $\underline{\text{Ada}}$ptive $\underline{\text{V}}$isu$\underline{\text{a}}$l $\underline{\text{R}}$easoning learning framework: different modes are unified and learned during the supervised cold-start stage, and the mode selection capability is induced via an RL process with a carefully designed AdaGRPO algorithm. Extensive experiments show that AdaVaR effectively guides the model to learn and differentiate multiple modes and perform context-adaptive mode selection, achieving consistent improvement across various scenarios, highlighting MoVT as an effective solution for building general visual reasoning models.

Published as a conference paper at ICLR 2026
Misture-of-Visval-TTouuuss EXPLORING CONTEXT- ADAPTIVE REASONING MODE SELECTION FOR GENERAL VISUAL REASONING
Zejun Li Yingxiu Zhao 2.31 Jiwen Zhang', Siyuan Wang", Yang Yao
Runzhou Zhao2.3 Jun Song2.3+, Bo Zheng?. Zhongyu Wei
Fudan University Alibaba Group Holding Limited FFutuee Living Lab of Alibaba
*University of Southern California
Shhnnghai Innovation Institute
fzejun1120, zywei @ edu.cr isong. aeeaallaaaaaaaaaaaaaaaaaaaaaaaaa com
http://ttaab..oo/uuuto// Futtu/uuuuueeeeee wwwinggLab/mixttne caaetre---viiilaalle wrisual-thoughts
ABSTRACT
Current visual reasoning methods mainly focus on exploring specific reasoning modes. Although improvements can be achieved in particular domains, they strug- gle to develop general reasoning capabilities. Inspired by this, we propose novel adaptive reasoning paradigm, Mixture-of- fies different reasoning modes within a single model and guides it to select the appropriate mode based on the context. To achieve this, we introduce AdaVaR, a two-stage Adaptive Visual Reasoning learning framework: different modes are unified and learned during the supervised cold-start stage, and the mode selection capability is induced via an RL process with a carefully designed AdaGRPO al- gorithm. Extensive experiments show that AdaVak effectively guides the model to learn and differentiate multiple modes and perform context-adaptive mode se- lection, achieving consistent improvement across various scenarios, highlighting MoVT as an effective solution for building general visual reasoning models.
(MoVT), which uni-
INTRODUCTION
Recent research has revealed the reasoning potential of large language models (LLMs): by guiding LLMs to generate chain-of-the tf-tuhhhtt (CoT) rationales in various forms before providing an answer, their performance on complex problems can be substantially improved (Kojima al. 2022 Wei et al. 2022, Wang et al. 2023 Yao et al. 2024b). Furthermore, the reasoning capability of LLMs can further enhanced through reinforcement learning (RL), ultimately pushing the frontier of
model capability (Guo et al. 2025 Kimi-Team et al. 2025 Yu et al. 2025.
Motivated by these advancements, preliminary explorations of reasoning in multi-modal contexts have emerged. As illustrated in Figure [a] existing visual reasoning methods fall into two types based on the form of CoTs (referred to as "reasoning mode" in this paper): (1) Text-based reason- ing, which is consistent with LLMs, where large vision-langee reasoning process directly in natural language (Yang et al. 2025 Huang et al., 2025 Meng et al.) 2025);
models (LVLMs) represent the
reasoning. which anchors the reasoning process in the visual con- text. Typically, this involves guiding LVLMs to generate structured outputs (c.g.. bounding boxes or points) that align textual concepts in the reasoning process with specific image regions (Chen al.
(2)
2023 Lei et al. 2024: Fan et al. 2025). Additionally, several works propose to introduce associated local visual information into the generated sequence. enabling the model to focus on specific regions
for subsequent inference (Li et al. 2025 Shao et al. 2024a Zheng et al. 2025 OpenAl et al., 2025).
Different reasoning modes impose distinct inductive biases, yielding varying strengths and weak- nesses across domains. As shown in Figure Tb text-based models excel at abstract reasoning (e.g., mathematics) but are more prone to hallucinations due to overthinking and language bias. In con- trast, visually-grounded models are better at leveraging visual information, curbing hallucination,
Equal contribution; Work done during the internship at Future Living Lab of Alibaba. Corresponding authors.

---

Published as a conference paper at ICLR 2026
our Adapehe Model AAR
Test-liased Models Purbur
Vissally orounded Models
Question
lext-based Reasoner
Deuctyo
the question zaking for
Performance Gaint+ Selnn++DDron (Drapp)
+123
+134
position two objects. The clock is located...
Wart, maybe I made mistake,, I will check the
image again, clock appears the left.. Looking at the options, Option A.., Option D..
-12
In this image, is the diock the left to right side of the laptop?
Therefare, the final answer A.
Dwen2.5-VL-78 Baseline
Visually Grounded Reasoner
31
First, we check the shelf (25, 122, 125) and find the clock (81, 23, 102,
-12
Options:
which
11
A.
is on the left of the image. Then find laptop
D. Right
[371, 423, 256) [ the table. Base
-20
Unknown
the position of two objects, the answer is
(Visual Search)
POPE (tallucination)
WeMttt [Math)
MathhVista (Math Generaa
(a) Rollout examples of two reasoning modes.
(b) Performance of different reasoning modes.
Figure 1: Comparison between two reasoning modes. (b) Performance gains (positive values) and drops (negative values) of Qwen2.5-VL. --L-7BB S--------------- based reasoning models relative to the base model.
and handling problems with clear object information, yet show limited gains mathematical bench- marks (since abstract concepts like length, size, etc. cannot be grounded to provide useful informa- tion). Overall, no single mode dominates across all tasks. This raises critical question: can integrate the complementary strengths of different modes to build general visual reasoning model?
In light of this, we propose novel visual reasoning paradigm, Mixture-of-Visual- Visuul-Thouuhhts:: the model is able to reason in different modes and adaptively choose the appropriate one based on the context. However, the greatest challenges lie in (i) how uniformly represent different reason- ing modes and enable unified model to learn them: and (ii) how to develop the capability for context-adaptive mode selection. To tackle these challenges, we introduce Ada VaR, an Adaptive Visual Reasoning framework with a two-stage training paradigm: supervised cold-start learning of different modes, and an adaptive RL to allow LVLMs explore and acquire mode selection skills.
Specifically, VdR begins with a cold-start stage to unify multiple reasoning modes within a model. We define a uniform sequence format for the reasoning paths of different modes and distin- guish them with mode-specific prefix tokens. This allows us to mix reasoning data across modes and train the model through a unified supervised fine-tuning (SFT) stage. Although different reasoning modes can be effectively learned, guiding the selection of appropriate modes via SFT remains chal- lenging, since we cannot pre-estimate the relative performance of different modes for each problem.
Therefore, to address the second challenge, subsequent RL stage is introduced to induce context- adaptive mode selection skills while enhancing the reasoning abilities. The idea is straightforward: encourage the model explore different modes for each question and reward the mode that is more likely to yield correct answers. Nonetheless, standard GRPO (Shao et al., (024b) poses obstacles to achieving this goal. First, the policy model may under-explore reasoning modes without explicit guidance (c.g., producing rollouts from only one mode per question). Second, the rollout- level advantage in GRPO fails explicitly capture preferences among different modes. To address these issues, we propose AdaGRPO, which steers exploration and comparison between reasoning modes through three key components: (1) Prefix-guided mode exploration: by fixing the mode prefix, we compel the policy model to explore evenly across multiple modes for the same sample. (2) Adaptive advantage mechanism: besides the rollout-level advantage, we introduce a mode-relative advantage to explicitly guide the model towards optimal mode selection. (3) Curriculum-baan scheduling: we initiate training with easier data to learn coarse- coarse-grained distinctions between modes, then gradually transition to harder questions for fine- ine-grained mode selection capability.
data
Extensive experiments demonstrate that Ada VaR enables the learning of different reasoning modes and adaptxte mode selection capabilities. As shown in Figure Tb our VaR-7BB achieves consistent improvements across scenarios. In terms of average performance over various bench- marks, Ada VaR-3B matches Qwen2.5-VL-7B. and Ada AdR-aR-7B surpasses GPT-4o. Our code, mod- els, and data will be open-sourced to the community. The contributions of this work are threefold:
Unlike existing works that focus on specific reasoning modes, propose a novel visual reasoning paradigm, MoVT: unifying different modes within a model and guiding it to select the appropriate one based context, pushing the frontier of general visual reasoning.
2

---

Published as a conference paper at ICLR 2026
We introduce AdaVaR, two-stage adaptive reasoning training framework: different modes are unified and learned during the supervised cold-start stage, and adaptive selection of ap- propriate modes is learned during the RL process using the proposed AdaGRPO algorithm.
Extensive experiments demonstrate the effectiveness of AdaVaR in guiding L.V.Ms to learn and distinguish different reasoning modes in a unified manner, and to dynamically select modes based on context. Our adaptive reasoning method brings general improvements across various scenarios and exhibits considerable potential for future exploration.
2 RELATED WORKS
2.1 LANGUAGE REASONING MODELS
Early work reveals the reasoning potential of LLMs, well-designed prompts (Kojima et al., 2022)
and in-context samples (Wei et al. 2022) can be adopted to guide LLMs to generate CoTss, enhancing the ability to solve complex questions. Subsequent research focuses on designing complex forms of CoT
Yao ct al. 2024b Besta et al., (024) and introducing mechanisms like majority voting (Wang
et al.
2023), reflection (Madaan al. [202]] Shinn et al. (2023), and step-level search (Lample al.
2022
Xie et al., 2023 Tian et al. (024). Recently, works represented by DeepSeek-RR (Guo et al,,
2025
have demonstrated that the reasoning abilities of LLMs can be effectively incentivized and
expanded under highly scalable RL environment (Qwen Team 2025
Yu et al. 2025).
2.2
VISION-LANGUAGE REASONING MODELS
Text-Based Reasoning Models Mainstream visual reasoning models directly express thinking processes in natural language. Early methods first learn reasoning patterns via SFT, building datasets through complex distillation (Xu et al.] 2024] Thawakar et al. (2025) collective search frame- works (ao et al., 2024a), and then enhance the reasoning capability with search-based mecha- nisms (Xu et al. 2024; Thawakar et al.) 2025) DPO (Zhang et al. 2024b Dong et al., 2025
Recent works mainly follow DeepSeek-RR RI, using high-quality data to compute verifiable rewards
and employing GRPO incentivize the reasoning abilities (Liu et al. 2025 Ma et al. 2025 Peng jet al.] (023) Meng et al., [025 Xu et al. (025). Meanwhile, A cold-start SFT phase before RL.,
trained on high-quality data, bootstrap the model's reasoning and enable more effective explo-
ration during RL (Yang et al. 2025 Huang et al. 2025 Wei et al. 2025 Chen et al. 2025a).
VisuallyGGrundder Reasoning Models Another line of research proposes to explicitly ground the reasoning process onto images. The prevailing approach is to introduce bounding box coordinates in CoTs, either through prompting (Mitra et al. 2024) or SFT (Chen et al. 2023 Wu et al. (025c)). Furthermore, the located regions can be cropped, zoomed, and then fed back into the model to
assist it in referring to fine- ane-grained visual information for subsequent reasoning (Li et al., 2025 Shao et al. 20244] Shen et al.] [024 OpenAl al.) (025). In addition. tool-based (Zhou et al., 2024) and visual prompting (Lei et al. 2024 Yang et al. 2023) methods can be used to supplement images with coordinate systems or segmentation maps, which can be utilized by LVLMS for explicit grounding. Recent works further demonstrate that grounded reasoning abilities can be activated and enhanced through RL by designing rewards (Sarch et al.) 2025 Fan et al. 2025) and methods to guide grounded rollout generation (Cao et al.
et al. 2025).
Existing models typically focus on one of the above modes and narrow domain: text-based rea- mode is mainly used for object-
soning targets mathematical problems, while the centric tasks. Inspired by this, we propose an adaptive reasoning paradigm, MoVT, that integrates the complementary strengths of different modes to build general-purpose visual reasoning model.
ADAVAR: AN ADAPTIVE VISUAL REASONING FRAMEWORK
This section details how the proposed AdR framework endows LVLMs with adaptive reasoning abilities. First, to integrate multiple reasoning modes in unified model, introduce reasoning paradigm in $ 3. to represent reasoning paths from different modes in a unified manner, merging mode selection and reasoning into single autoregressive generation process. Building this, the
3

---

Published as a conference paper at ICLR 2026
model undergoes an SFT phase learn multiple reasoning modes simultaneously (as described in $B.2). Based on the SFT model, we design an RL algorithm (as detailed in $33.3 to help the model acquire the ability to select the appropriate mode according to the provided context.
3.1 REASONING MODE DEFINITION AND UNIFICATION
As introduced in the introduction, we regard different forms of CoT as distinct reasoning modes. In this paper, we mainly consider the integration of two commonly applied visual reasoning modes: (i) Text-based reasoning. without introducing additional inductive bias, represents all reasoning processes only in language; (ii) Visually-groundee
reasoning requires models to generate struc- tural information, such as coordinates, to align key concepts-primanty objects-with corresponding regions in the image during reasoning. In this work, we adopt the widely used bounding- -oo------oo------ enhanced format (Li et al. 2025 Fan et al. (2055), guiding the model to describe objects in the form of "object (Z1 V/, 22, 1/_
explicitly align the reasoning process with the image.
Reasoning Mode Unification To enable the model to learn and distinguish different modes within unified framework, define uniform format of reasoning sequences: each mode is assigned unique prefix token, which is placed at the beginning of reasoning paths to serve as an in-context indicator, guiding the model to differentiate between modes. We define the reasoning template
System: You are helpful assistant. You need to first think about the reasoning process and then answer the question raised by the user, in the format of <think>> reasoning process here </think>
<answer>> answer here </answer>. You have two modes of thinking:
1. Grounded Reasoning: .... begin your response with <ground> when using this mode.
2. Text-based Reasoning: .... begin your response with <text> when using this mode.
User: prompt. Assistant: moode prefix>> <think> </think> <answer>> </answer>>
The system prompt specifies the reasoning requirements and introduces the different reasoning modes (omitted for brevity; the full prompt is provided in Appendix [B.]]. The model response includes the mode prefix and the reasoning process. Leveraging the autoregressive property, gener- ating reasoning paths in this format can be naturally divided into two steps, i.e., P(a, m/i, g) P(mli,g) q) P(a, f/m, i, q): (1) P(mli, q) selecting the reasoning mode by generating the mode prefix m based on the input image i and question q. and (2) P(a, t/m, i, generating the corre- sponding thinking process t and answer based on the selected mode m and the input context q- These two steps can be accomplished sequentially within a single sequence generation process.
3.2 STAGE 1: COLD-START MODE LEARNING WITH SET
Building on the unified reasoning format, we adopt an SFT stage as a cold start to enable the base model, Qwen2.5-VL (Bai et al.) 2025), learn reasoning abilities across different modes simulta- neously. To this end, we mix expert reasoning trajectories from different modes: (i) For text-based reasoning, we follow DeepSeek-RI (Guo et al. 2025) to construct data by distilling text-based visual reasoning model (Ma et al., 2025) and applying rejection sampling. (ii) Regarding visually- grounded reasoning, we directly utilize high-quality SFT data constructed in existing works (Li) et al., 2025). The proportion of the two parts of data is controlled to 1:1 to prevent introducing undesired bias on mode selection. Please refer to Appendix B.2 for more details.
B.2
3.3 STAGE ADAPTIVE MODE SELECTION WITH RL
Although Stage helps the model learn reasoning abilities in different modes, it is difficult to learn mode selection under the supervised learning paradigm because we cannot pre-construct effective supervision on which mode to select. Moreover, recent studies have shown that the reasoning abili- ties acquired during the SFT stage can be further enhanced through RL (Yang et al.) 2025 Wei al.) 2025). Therefore, we introduce an RL stage, which simultancously improves the reasoning ability and guides the model to learn adaptive mode selection based on the proposed AdaGRPO algorithm.
Absolute coordinates of the top-left and hottom-right corners, which aligned with Qwen2.5-VT.
4

---

Published as a conference paper at ICLR 2026
Vanilla GRPO
Rollout
Aa
A
As
A
Question
Policy Model
Reward Function
exc.ee Finm
leutl Advantage
Rollout Ozu
Age
A
thinking process
w/ preftx
<tants
Rollout of
Rollout-level Advantage
4
As
AdaGRPO
Rollout
Agu
A
Aq
Ap
Model
Reward Function
GRD Rollout of GRD Rollout of
Mode Relative Advantage
Aa
Ar
sgrounds
Ap
mode prefix
thinking process
RL Environment
Reasoning Rollouts
Rewards
Advantages
Token-wise Advantages
Figure 2: Demonstration GRPO and AdaGRPO. In AdaGRPO, we use mode prefixes to guide exploration across different modes (TXT and GRD respectively represent text-based and visually- grounded reasoning), calculate both rollout-level advantages {A, 12n and mode-relative preference (A,,A.) , and explicitly guide mode selection and the enhancement of reasoning abilities.
3.3.1 RL ALGORITHM: ADAGRPO
Revisiting GRPO For better illustration on the design rationale of AdaGRPO, review the de- tails of GRPO (Shao al. 2024b). shown in the upper part of Figure the old policy model Tone generates 2n reasoning rollouts {01, 02,....
...: 02n for question q, obtaining rewards {ri, F2, ... P2n)
which are utilized to optimize the current policy model Ta with the following objective:
(O) E [g P(Q), {0
Toai (O(4)]
2n
losl
Talo, volo,1q.0j.c AJ. clip To To(o;lq,0,.c
)A11) (1)
min
2n
Teaas (0y1g, 03,ce)
(Toad (0,19, (0,l9.0).<t))
1=1
where 6 is the clipping hyper-parameter and the KL penalty is omitted for brevity. Aj.t = Aj (fr..( 1) are the rollout-level advantages estimated by comparing rewards of all rollouts.
sid( (Tin....Tnn))
Although effective, the vanilla GRPO is limited in the adaptive reasoning scenarios terms of: (i) the policy model, especially after SFT, may exhibit preferences for certain reasoning modes. As result, the generated 2n rollouts could come from same mode, leading to uneven exploration on different reasoning modes; (ii) GRPO merely considers rollout-wise advantages without explicit modeling on preference between different reasoning modes to guide mode selection.
Building on the insights above, we propose AdaGRPO, variant of GRPO that encourages explo- ration across reasoning modes and guides mode selection with mode-relative advantage estimation.
Prefix-Guided Mode Exploration First, AdaGRPO enforees uniform exploration different modes during the rollout sampling process. The unified reasoning format defined in Section 3.] allow us to guide the model perform reasoning in specific mode by fixing the corresponding mode prefix in the input sequence. We divide the 2n rollouts into two sub-groups: {o} ) ~ To (Oli, q, m.) consists of n text-based reasoning rollouts with prefix m. = <text>, and To (Oli, 9, m.) consists of visually-grounded rollouts with prefix m, <ground>.
We follow the reward setup utilized in DeepSeek-RI (Guo et al. 2025), which includes format rewards and accuracy rewards. The former controls the output format, while the latter assigns a reward of 1 or 0 based on rule- based evaluation of the produced answer. The same reward functions are applied to all rollouts from different modes {of...., of of, ..., " support subsequent computation of advantages both between reasoning modes and among rollouts.
Reward Function
Adaptive Advantage Calculation To explicitly guide the mode selection, we characterize the mode-relative advantage based on the rewards {r,}ý=1 and {ry} corresponding to the two sub-groups of rollouts {o]] and {o)))=1- Following the Gaussian assumption in GRPO,
5

---

Published as a conference paper at ICLR 2026
obtain two normal distributions of rewards for the two sub-groups fry P. and fry ) f=11 Pu. then we estimate the advantage of mode A over mode with the probability that a rollout sampled from mode A outperforms rollout sample from mode B (and vice versa), formulated as:
A, =PxxP..y (X Y); A, = PX <
where P. N (wt mean( ( (1y ) Var((r;))==)) Var((r, ((,,)(==11)
(2) Furthermore, we can reasnonably assume that and Y are independent, so X - N(u M4,07 df). ). In this way, A, and A, can be directly calculated with the cumulative distribution function (CDF) & of the standard Gaussian distribution as A. Φ
P. (Mo mean (m,))==+)),+
= 1 A..
As in GRPO, we also compute rollout wwiee advantages {45} However, unlike GRPO, we assign different advantages different tokens within the same sequence. As illustrated in the lower part of Figure 2 mode-relative advantages, A, and A.. are assigned to mode prefix tokens, guiding models 2n select the better mode. Rollout-wise advantages { are applied tokens the thinking processes, enhancing the reasoning capability of the policy model. The specific formulation is:
(K for Elogle An (oEE)))) Ar
if Oja E otherwise
A3,4
Aj
51 meani (T) TT raall
(3)
std(( (P2
In summary, the optimization objective of the proposed AdaGRPO is formulated as:
I JAdaGRPO(O) I [i.q ~ P(L, Q), {o; ~ Testa (Olg, mz), {o,) ~ Toaa (0 9, m.)]
logl
oo 9,
min
AJ,,, clip
c, 1,1+e) A3e0)
Oj
Ne (o oc1cl)
Te (0, 0),(e)
3.3.2 DATASET COLLECTION
For RL, we collect a diverse dataset consisting of two parts: (1) existing datasets with verifiable answers, including Geo170K (Gao al., 2023), OmniCount (Mondal et al. 2025), and MM- Eureka (Meng et al. 2025); (2) curated subset from the SFT data of LLaVA On-One Vision (Li et al.] 2024) and Intern VL (Chen al. 2024b). focusing on math, OCR, object counting, science, ground- ing, and document-related tasks. The subset is filtered based on the answer verifiability and diffi- culty, and dowplsampled balance different tasks. Please refer Appendix B.2 for more details.
Curriculum Learning To help the model progressively learn reasoning and mode selection in complex scenarios, we design two data mixing strategies: binary mixture, which includes only the OmniCount and Gco170K (with Geol70K being relatively simple geometry problems); and diverse mixture, which consists of all remaining data beyond binary mixing, covering multiple tasks and featuring higher difficulty. Our model first learns from the binary mixed data, then from the diverse mixed data, with both the difficulty and task distribution progressing from simple to complex.
4 EXPERIMENTS
4.1 EXPERIMENTAL SETUP
Implementation Details Based on the proposed framework, we develop two reasoning mod- els, AdaVaR 3B and VaR- which are constructed from Qwen2.5-VL-3B and Qwen2.5-VL. 7B (Bai et al. (2025), respectively. Please refer Appendix B.3 for detailed hyperparameter settings.
Unlike existing works that focus on specific domains, we comprehen- sively evaluate the reasoning abilities of Ada across diverse scenarios. For mathematical reason- Verse (Zhang et al. 2024a). and WeMath (Qiao et al.) 2024). additional, for general-purpose scenarios, we
Evaluation Benchmarks
ing, consider Math Vista (Lu et al.] (023).. Math Vision (Wang et al. (024), Verse
adopt V (Wu & Xie. 2023). SpatialScore (Wu al. 2025b), and MMStar (Chen et al. 2024a)

---

Published as a conference paper at ICLR 2026
Evaluation results on various benchmarks. indicates results reproduced in this paper. Within each model group with different parameter sizes, the best and second-best results are high- lighted in bold and underlined, respectively. We use background colors distinguish text-based, visually-grounded and our adaptive reasoning models. "Ace." short for accuracy.
Table 1:
Math-Oriented
General Scenarios
Model
Average Acc.
MathVista testmini
MathVision
Math vision-only
WeMath strict
MMStar
V
POPE SpatialScore hard
SET Close source Models
GPT-4o 2024
63.8
30.4
39.9
42.9
64.7
66.0
86.9
30.6
53,20
MLVA-OV-TE
62.6
12.6
17.7
75.4
15.6
Intem Xn 2015
70.5
28.5
39.8
37.5
66.3
91.1
12.9
Reusoning
Qwen2.5.
Qwen2.5.VL. VL 2025
62.3
21.2
31.2
21.2*
53.1*
73.5*
87.2*
16.2
45.74
LMM-RI 2025
63.2
25.2
37.3*
29.2"
55.0*
74.9*
86.7*
18.9"
48.80
VLAA-Thinker
2025a
61.0
24.4
36.4
33.8
$4,8*
56.0*
87.8*
19.4
46.70
GRTT-3B 2025
59.8
21.2*
28.4
29.1
52.7*
72.3*
83.0"
19.7
45.80
ViGoRL
55.2*
AdaVaR-3B (Orss)
69.8
24.5
35.2
33,8
59.3
77.0
88.2
18.9
$0.84
Reasoning Models based Swen22.5VVL-78
Qwen2.5-VV VL.-BB
2025
68.2
25.1
41.1
31.2
60.3
78.0*
87.8*
15.22
50.90
VLAA-Thinker: 75 21755a
68.0
26.4
48.2
41.5
62.6*
78.5*
86.2*
22.3*
54,20
MM-Bureka
72.6
28.1
45.4
36.9
27.1"
52.52
OVR-7B 2024
72.1
46.4"
54.6
44.6
62.7
62.5*
83.2
20.0"
55.26
Orsta-78
72.5
28.2"
42.9*
31,8"
78.0"
16.4"
52,04
DeepPyes
2125
20.1
26.6
40.7
32,7
61.3*
90.1
87.9*
20.3*
53.72
Chain-of Tocus 2025
63.1
22.9*
32,1
33.1
59.34
88.0
88.4
20.6*
50.94
VIGoRI-7BB
63.5"
23.3*
32.6"
36.3*
19:5
50.53
AdaVaR. 7B (Oars)
74.4
28.5
43.0
44.8
63.0
83.4
89.0
20.4
55.82
to evaluate the capabilities of models visual search, spatial reasoning, and general perceptual reasoning, respectively, while POPE (Li al. (023) is used to diagnose hallucination risk.
During evaluation, our AdaValt model naturally incorporates mode-switching mechanism: the model gets stuck in repetitive logic and cannot conclude answer (between <answer>> and </an- swer>) while reasoning in a particular mode, we switch to another mode and try again. More evaluation details, including descriptions of the compared models, included in Appendix B3.
4.2 OVERALL PERFORMANCE
Table T presents thorough evaluation of visual reasoning models. Several insights be gleaned: (1) As argued in Section I models based on single reasoning mode tend to become experts in specific domains. Text-based models, such as TLAA--hinker-3BB and OVR-7B, achieve notable improvement on mathematical problems but show significant degradation in V and POPE. In con- trast, visually-grounder models consistently improve performance on V* and control hallucination, but struggle with math DeolemssDeeepyess makes slight improvement while other methods fail to maintain the original mathematical capability of Qwen2.5-VL. (2) Ada VaR-3B and Vda on the other hand, combine the strengths of two modes in adaptive manner, and are the only models that outperform Qwen2.5- VL across all datasets. Specifically, AdaVah achieves the best perfor- mance Math Vista, WeMath, and POPE, while excelling MMStar, Vision, and V*. (3) In terms of the overall performance measured by average accuracy, AdaVaR achieves the best results in both the 3B and 7B groups. Notably, Ada VaR33B reaches level very close Qwenn.55VVL-7. while Ada VaR-7B surpasses GPT-4o. In summary, our adaptive reasoning framework provides an effective and feasible solution for building general visual reasoning models.
4.3 DELVING INTO ADAPTIVE VISUAL REASONING
Beyond the overall improvement, we delve into the mechanism of adaptive reasoning based on Ada VaR-3B and several ablated models, aiming to answer the questions raised in Section [
4.3.1 QUESTION 1: CAN WE UNIFY DIFFERENT REASONING MODES IN ONE MODEL?
As shown in Table after SET and RL, reasoning modes induce different prediction patterns in terms of performance: the text-based mode excels at math, while the grounded mode is better at object-centric tasks. Please also cases in Appendix C.8 for qualitative comparison.
7

---

Published as a conference paper at ICLR 2026
Table 2: Performance of AdaVaR with different reasoning modes various stages. AdaVaR with- out subscript performs adaptive reasoning. Ada VaR with the subscripts G and T denote the per- formance using visually-groune grounded mode and text-based mode, respectively. GRD% indicates the proportion of times VdR chooses the grounded mode in the corresponding dataset.
Math-Oriented
General Scenarios
Average Acc.
Model
MathVista
Vashn
MathVerse
WeMath
MMStar
V+
POPE:
SpaSeore
Qwen2.5.V1.s
62.3
21.2
31,2
21.1
53.1
73.5
86.1
16.2
45.6
Stage SFT-based Models
AdaVaR SFF-38
611.4
23.2
33.0
24.5
54.7
75.5
89.1
16.8
47.8
MdVaR-SFF--3B
56.4
20.3
28.5
29.5
53.2
75.5
89.2
46.3
AdaVaR-SFT-3Br
62.5
22.9
31.4
31.1
55.9
65.6
83.1
17.8
46.3
GRD%
31%
5%
1%
78%
90%
98%
94%
96%
Stage Rhase Models
AdaVaR-3B
69.8
24.5
35.2
33.8
59.3
77.1
88.2
18.9
50.8
AdaVaR-38,
66.1
49.3
AdaVaR-3By
68.1
23.0
34,9
32.6
58.5
58.6
84.6
17.7
47.3
Upper Bound
74.8
30.9
41.7
38.3
67.6
78.5
95.6
27.8
56.8
GRDS
04%
Baseline Models based
Single Reasoning Mode
Grounded- SFT-RL
Grounded-SFT-RL
62.6
22.2
34.4
33.0
55.4
77.0
87.4
18.4
48.7
Text-SFT-RL
67.2
23.5
36.1
32.2
56.0
70.7
85.6
18.8
48.8
MiX-SPT-RL
65.0
23.2
33.6
28.8
56.2
76.9
85.4
18.6
48.5
Mode Mode 5RD
70
Accuracy Raward
0.30
0.6
Accuracy
0.25
0.5
GRD%
9.35
3.30
0.
Binary Mixture
Diverse Mixture
58
0.50
500
1000
3500
2000
2502
500
1000
1500
2000
2500
Training Step
Training Steps
(a) Training rewards math problems.
(b) Performance evolution of different modes on MathVista.
Figure 3: Math-related training dynamics and evaluation metrics during Stage 2. ADA, TXT, and GRD are short for the three reasoning modes: adaptive, text-based, and grounded, respectively.
Comparison with Single-Mode Baselines Furthermore, we construct two single-mode baselines, Grounded-SFT-RR and SFT-RL, under the same training settings: using SFT data correspond- ing each mode and RL data consistent with VdR We find that: (1) The difference between the single-mode baselines and the respective modes of AdaVaR is minimal, suggesting that consolidat- ing both modes into one model does hinder the improvement of either individual mode; (2) Both baselines perform worse than AdR indicating that, compared to simply training with diverse data, adaptive reasoning is more effective for developing general-purpose reasoning capabilities.
Necessity of Mode-Specific Prefix Another question is whether need distinguish between different modes. To investigate this, we remove the prefix token and mix the SFT data from two modes together, creating Mix-SFT-RL. Its performance is even inferior the single-mode baselines. This suggests that simply mixing data from different modes is insufficient, and that the mode-speciff prefix token introduced in AdaVaR helps the model differentiate between modes and facilitate uni- form exploration across modes during the RL stage, resulting in a better overall performance.
The Upper Bound Adaptive Reasoning Table 2 also presents the upper bound of adaptive reasoning: the performance when a prediction is considered correct if either mode yields the correct answer. The upper bound clearly surpasses Qwen2.5-VL. mathematical problems. where text-based reasoning some problems that the text-based mode cannot. This phenomenon demonstrates the complementary nature between modes and highlights the great potential of the adaptive MoVT paradigm.
and single-mode models. Even in dominant, the grounded mode solve
8

---

Published as a conference paper at ICLR 2026
Table 3: Ablation study, Ada-Adv and PG-Exp are abbreviation for adaptive advantage and prefix- guided exploration, respectively. The definition of diverse mixed data is provided in Section 3.3.
Model
Math-Oriented
General Scenarios
Averuge Acc.
Vista
Mathnn
VathVerse
WeMath
MMStur
V-
POPE
SpaScore
AdaVaR-3B
69.8
24.5
35.2
33.8
59.3
77.1
88.2
18.9
50.8
w/o Ada-Adv PG-Exp
66.3
23.8
34,9
31.3
56.6
75.4
89.1
19.7
49.6
w/o Ada-Adv
68.4
24.7
34.4
33.7
58.9
77.4
88.0
17.4
50.3
w/o Diverse Mixed Data
67.4
23.7
33.1
33.4
57.3
76.4
82.1
18.5
49.0
w/o Curricllu Learning
66.8
24.4
34,9
33.4
57.8
76.9
88.2
18.5
50.1
Gruully-Gruundd Mode Taxt-based Mode Rate
Perception
01.2%
Fine-Grained Perception
Instance Reasoning
64.0%
36.0%
59.3%
40.0%
100.0%
Logical Rea soningg
23.4%
71.4%
26.8%
73.4%
100.0%
Math
Science & Technology
AdaVaR-3B
AdaVaR-3B w/o Ada-Adv
AdaVaR-3B w/o Ada-Adv PG-Exp
Figure 4: Proportion of each mode selected by different models across categories in MMStar.
4.3.2 QUESTION 2: CAN ADAVAR SELECT APPROPRIATE MODE?
Comparing the SFT and RL results in Table 2 we find that (1) the SFT model exhibits unexpected behavior: although the text mode performs significantly better on Math it still chooses the grounded mode 31% of the time, resulting worse performance than the text mode alone. This also supports our argument in Section D it is hard to guide the model to learn effective mode selection during the SFT stage. (2) After Stage 2, AdaVaR exhibits reasonable mode selection: it prefers the text mode for math problems and the grounded mode for V and POPE tasks (as well as fine-gr fine-grained choices shown in Figure 4 its performance consistently surpasses single-mode reasoning across all benchmarks, indicating that the RL stage helps the model acquire effective mode selection abilities.
Learning Curves of Mode Selection How is the mode--eleccionn capability acquired? Figure 3 lustrates the learning process using the mathematical scenario as an example. It is observed that the relative performance across modes (as reflected by the relative reward levels of two modes in Fig- ure 3ab guides the mode-selection preference, which is illustrated by the GRD% curve in Figure 3bb The RL process can be divided into three phases, which are delineated in the figure using different background colors in Figure 3 (1) Exploration phase: at the beginning, performance estimates for different modes obtained via RL sampling are still unstable, and the GRD mode even outperforms the TXT mode for period, with the choice of mode also fluctuating. (2) Stabilization phase: as the text mode demonstrates a clear advantage in RL. sampling, the adaptive model gradually tends to select the text mode. However, does not yet achieve optimal performance, and the adaptive perfor- mance is slightly weaker than the TXT mode. (3) Refinement phase: by introducing more difficult and diverse data, the capabilities of each mode steadily improve, and the model learns which types of math problems benefit from choosing the GRD mode, ultimately pushing the ADA mode to reach optimal performance. Please refer to Appendix C.3 for the training dynamics in other domains, and to Appendix C.T for a more comprehensive analysis of the learned mode selection behavior.
Effectiveness of AdaGRPO Table 3 and Figure 4 present the results of ablation experiments lated to key designs in AdaGRPO. (T) When both Ada-Ady and PG-Exp are removed, which is equivalent to the original GRPO, observe that RL reinforces the SFT model's unreasonable pref- erence for the grounded mode, resulting in overall inferior performance. (2) With PG-Exp included, the model is able fully explores different modes and compares rollouts between them. In datasets where the advantages between modes are clear, such as V* and Math Vision, the model makes rea- sonable mode selections and achieves good performance. (3) With further introduction of adaptive advantage, AdaVaR learns more refined mode selection abilities, leading to improvements on com- plex datasets like Math Vista and MMStar, as well as further boosting overall performance.
Training Data Matters Apart from demonstrating the importance of AdaGRPO, the results in Table 3 demonstrate that diverse data enables the model to learn more generalizable reasoning and

---

Published as a conference paper at ICLR 2026
mode selection abilities. Additionally, the curriculum learning strategy is also highly effective during the RL phase. Both factors contribute to the improved model performance.
CONCLUSION
This paper presents an adaptive visual reasoning paradigm, MoVT, for building general visual rea- soning models. We propose Vda framework that unifies the reasoning capabilities of different modes within single model through SFT, and employ tailored RL algorithm, AdaGRPO, to train the model choose the appropriate mode based on the context. Experimental results demonstrate that, unlike existing models that excel only in specific domains, AdaVaR achieves consistent im- provements across multiple scenarios- with Ada VaR-7B surpassing GPT-4o in average performance.
ETHICS STATEMENT
all data sources used in this paper are open-source and available for research purposes. We strictly comply with the corresponding terms of use to ensure our data can be used for scientific research. The original datasets have been widely used in related studies and are confirmed to contain no harmful content. Building on these datasets, we performed certain reconstruction and filtering steps, detailed in Appendix [8.2] this process only involves modifying the problem format and providing additional information for filtering. Through compling-based spot checks, we have verified that this data construction process does not introduce harmful content, and our filtering is entirely for research purposes, without any discriminatory bias.
As described in Appendix B. 2
Our reasoning models, Ada VaR-3B and Ada currently focus English scenarios, and aim to expand to more languages in the future. Our work and artifacts are designed with the principle of universality and fairness, without any preference for specific demographic groups.
REPRODUCIBILITY STATEMENT
We have taken rigorous steps ensure that relevant researchers can easily reproduce this work. First, our approach is built upon the open-source Qwen2.5-VI- tion we describe our VdR framework in detail, including how represent reasoning gen- eration sequences in unified way (Section 3.1), how to perform cold-start SFT (Section 3.2). and how to train the model's mode selection capability via RI. (Section 3.3 Furthermore, we provide extensive additional details, including the implementation details and formulas of Ada- GRPO (Section 3.3.I), detailed model prompts (Appendix [B.], data construction details (Sec- tion 3.3.2 and Appendix (B.2), specific training parameter settings (Appendix B.3). and evalua- tion details (Appendix [B44). In summary, we believe researchers can readily reproduce the find- ings of this paper. Our data, model weights, and code are available at https: triithubb... com/ Nnture-Living LiiiinngLab/iiittueeeeeeeeeeeeee wft.:/ttto------uuull----------- thougghtttt
model. Second, in Sec-
ACKNOWLEDGMENT
This work was supported by Alibaba Group through Alibaba Research Intern Program. The sup- port was also provided by AI for Science Program, Shanghai Municipal Commission of Economy and Informatization BEA-GZZ-RGGN--BB..E (2033YFF1204800) Computational resources were provided in part by CFFF.
and National Key R&D Program China
REFERENCES
Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, Humen Zhong, Yuanzhi Zhu, Mingkun Yang, Zhaohai Li, Jianqiang Wan, Pengfei Wang, Wei Ding. Zheren Fu. Yiheng Xu. Jiabo Ye, Xi Zhang, Tianbao Xie, Zesen Cheng, Hang Zhang. Zhibo Yang. Haiyang Xu, and Junyang Lin. Qwen2.5-vl technical report, 2025. URL httpt:tpp//..........
org/ / 13923
10

---

Published as a conference paper at ICLR 2026
Maciej Besta, Nils Blach, Ales Kubicek, Robert Gerstenberger, Michal Podstawski, Lukas Gian- inazzi, Joanna Gajda, Tomasz Lehmann, Hubert Niewiadomski, Piotr Nyczyk, et al. Graph of thoughts: Solving elaborate problems with large language models. In Proceedings of the AAAI Conference on Artificial Intelligence volume 38, pp. 17682-17690, 2024.
Meng Cao, Haoze Zhao, Can Zhang, Xiaojun Chang, Ian Reid, and Xiaodan Liang. Ground- rl: Incentivizing grounded visual reasoning via reinforcement learning. arXiv preprint arXiv:2505.20272 2025.
Hardy Chen, Haoqin Tu, Fali Wang, Hui Liu, Xianfeng Tang, Xinya Du, Yuyin Zhou, and Cihang Xie. Sft or ri? an early investigation into training rl-like reasoning large vision-language models. arXiv preprini arXiv:2504.1 1468, 2025a.
Keqin Chen, Zhao Zhang, Weili Zeng, Richong Zhang. Feng Zhu, and Rui Zhao. Shikra: Unleashing multimodal I1m's 's referential dialogue magic. arXiv:2306. 15195, 2023.
Liang Chen, Lei Li, Haozhe Zhao, Yifan Song, and Vinci. R1-v: Reinforcing super generalization ability in vision-language models with less than $3. 2025b. Accessed: 2025-02-02.
Lin Chen, Jinsong Li, Xiaoyi Dong, Pan Zhang, Yuhang Zang, Zehui Chen, Haodong Duan, Jiaqi Wang, Yu Qiao, Dahua Lin, et al. Are we on the right way for evaluating large vision-language models? Advances in Neural Information Processing Systems, 37:27056-27087, 2024a.
Zhe Chen, Weiyun Wang, Hao Tian, Shenglong Ye, Zhangwei Gao, Erfei Cui, Wenwen Tong, Kongzhi Hu, Jiapeng Luo, Zheng Ma, al. How far are we gpt-4v? closing the gap to commercial multimodal models with open-source suites. Science China Information Sciences, 67 (12):220101, 2024b.
Yuhao Dong, Zuyan Liu, Hai-Long Sun, Jingkang Yang, Winston Hu, Yongming Rao, and Ziwei Liu. Insight-v: Exploring long-chain visual reasoning with multimodal large language models. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 9062-9072, 2025.
Yifan Du, Zikang Liu, Yifan Li, Wayne Xin Zhao, Yuqi Huo, Bingning Wang, Weipeng Chen, Zheng Liu, Zhongyuan Wang, and Ji-Rong Wen. Virgo: A preliminary exploration on reproducing ol- like mllm. arXiv preprint wrxiv:250100900 2025.
Yue Fan, Xuchai He, Diji Yang, Kaizhi Zheng, Ching-Chen Kuo, Yuting Zheng, Sravana Jyothi Narayanaraju, Xinze Guan, and Xin Eric Wang. Grit: Teaching milms to think with images, 2025. URLpttps: :tap::://araa //tpxaaavvvv//////////ee.///.ava org abb http:/abb/225....0 2550p/1557959
Jiahui Gao, Renjie Pi, Jipeng Zhang, Jiacheng Ye. Wanjun Zhong, Yufei Wang. Langing Hong, Jianhua Han, Hang Xu, Zhenguo Li, and Lingpeng Kong. G-llava: Solving geometric problem with multi-modal large language model, 2023.
Daya Guo, Dejian Yang. Haowei Zhang, Junxiao Song. Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-rl: Incentivizing reasoning capability in lims via reinforcement learning. arXiv preprint arXiv:2501.119455 2025.
Wenxuan Huang, Bohan Jia, Zijie Zhai, Shaosheng Cao, Zheyu Ye, Fei Zhao, Zhe Xu. Yao Hu, and Shaohui Lin. Vision-rl: Incentivizing reasoning capability in multimodal large language models, 2025. URLpttps:: http::////aaivvloo////////////// 2553
Hugging Face. Open rl: A fully open reproduction of deepseek-rl, January 2025. URL https www.t....ccooo..i...coo http:///wuuugnggggaaa///eeeeeeee open-ri
Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Os- trow, Akila Welihinda, Alan Hayes, Alec Radford, et al. Gpt-4o system card. arXiv preprint arXiv:2410.212777 2024.
Omri Kaduri, Shai Bagon, and Tali Dekel. What's 's in the image? deep-dive into the vision of vision language models. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 14549-14558, 2025.
11

---

Published as a conference paper at ICLR 2026
Kimi-Team, Angang Du, Bofei Gao, Bowei Xing. Changjiu Jiang, Cheng Chen, Cheng Li, Chenjun Xiao, Chenzhuang Du, Chonghua Liao, et al. Kimi k1. 5: Scaling reinforcement learning with llms. arXiv preprint arXiv:2501.12599, 2025.
Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. Large language models are zero-shot reasoners. Advances in neural information processing systems, 35:22199-22213, 2022.
Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng. Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, and Ion Stoica. Efficient memory management for large language model serving with pagedattention. In Proceedings of the ACM SIGOPS 29th Symposium on Operating Systems Principles, 2023.
Guillaume Lample, Timothee Lacroix, Marie-Anne Lachaux, Aurelien Rodriguez, Amaury Hayat, Thibaut Lavril, Gabriel Ebner, and Xavier Martinet. Hypertree proof search for neural theorem proving, Advances in neural information processing systems, 35:26337-26349, 2022.
Xuanyu Lei, Zonghan Yang, Xinrui Chen, Peng Li, and Yang Liu. Scaffolding coordinates to pro- mote vision- Sainu-la coordination in large multi-mode models, 2024.
Bo Li, Yuanhan Zhang, Dong Guo, Renrui Zhang, Feng Li, Hao Zhang, Kaichen Zhang, Peiyuan Zhang, Yanwei Li, Ziwei Liu, and Chunyuan Li. -leviioonvision: Easy visual task transfer, 2024.
URL httpthhtt/aaattt
wwt.:oggg/aao.oooooooo...oo.es/s http//bs//4405 03326
Jia LI, Edward Beeching, Lewis Tunstall, Ben Lipkin, Roman Soletskyi, Shengyi Costa Huang, Kashif Rasul, Longhui Yu, Albert Jiang, Ziju Shen, Zihan Qin, Bin Dong, Li Zhou, Yann Fleureau, Guillaume Lample, and Stanislas Polu. Numinamath. [Battps::/huggggggaae http://thhu.cccm/pppoottttn.....
5] a aimo-progress-pee ze/bl ob/maa//aaaaao....//. ma http://aannneerot/t//oeeooo wtw.a//mennnnnatteeeeae..e.e 2024.
Yifan Li, Yifan Du, Kun Zhou, Jinpeng Wang, Wayne Xin Zhao, and Ji-Rong Wen. Evaluating object hallucination in large vision-language models. arXiv:2305.10355, 2023.
Zejun Li, Ruipu Luo, Jiwen Zhang, Minghui Qiu, Xuanjing Huang, and Zhongyu Wei. Vocot:
Unleashing visually grounded multi-step reasoning large multi-modal models, 2025. URL
attps://aarxtn www.xxxv..ooggaaoooo abs///////// 2405. 11.00.66999. 19119
Ziyu Liu, Zeyi Sun, Yuhang Zang, Xiaoyi Dong, Yuhang Cao, Haodong Duan, Dahua Lin, and Jiaqi
Wang. Visual-rft: Visual reinforcement fine-tuning. arXiv preprint urXiv:2503.0/785 2025.
Pan Lu, Hritik Bansal, Tony Xia, Jiacheng Liu, Chunyuan Li, Hannaneh Hajishirzi, Hao Cheng, Kai- Wei Chang, Michel Galley, and Jianfeng Gao. Mathvista: Evaluating mathematical reasoning of foundation models in visual contexts. arXiv preprint arXiv:231000255. 2023.
Yan Ma, Linge Du, Xuyang Shen, Shaoxiang Chen, Pengfei Li, Qibing Ren, Lizhuang Ma, Yuchao Dai, Pengfei Liu, and Junjie Yan. One rl to see them all: Visual triple unified reinforcement learning. arXiv preprint arXiv:2505.18129 2025.
Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, et al. Self-refine: Iterative refinement with self-feedback. Advances in Neural Information Processing Systems, 36:46534-46594, 2023.
Fanqing Meng, Lingxiao Du, Zongkai Liu, Zhixiang Zhou, Quanfeng Lu, Daocheng Fu, Tiancheng Han, Botian Shi, Wenhai Wang, Junjun He, Kaipeng Zhang, Ping Luo, Yu Qiao, Qiaosheng Zhang, and Wenqi Shao. Mm-cureka: Exploring the frontiers of multimodal reasoning with rule- based reinforcement learning, 2025. https:: www.a.iiiv.ommmmm wttp:awwww......//aa//aaaaaaae/a
absp///ss2//5 07365
Chancharik Mitra, Brandon Huang, Trevor Darrell, and Roei Herzig. Compositional chain-of- thought prompting for large multimodal models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 14420-14431, 2024.
Anindya Mondal, Sauradip Nag, Xiatian Zhu, and Anjan Dutta. Omnicount: Multi-label object counting with comantic-geomette priors. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pp. 19537-19545, 2025.
12

---

Published as a conference paper at ICLR 2026
OpenAl, Aditya Ramesh, Aidan Clark, Aleksandra Spyra, et al. Thinking with images, 2025. URL
plttps://openate hwtpaaaa.ccccnnaaaaa//aaaaaaaaaa
Yingzhe Peng, Gongrui Zhang, Miaosen Zhang, Zhiyuan You, Jie Liu, Qipeng Zhu, Kai Yang, Xingzhong Xu, Xim Geng, and Xu Yang. Lmm-rl: Empowering 3b Imms with strong reasoning 2025.
abilities through two-stage rule-based rl. arXiv preprint
Runqi Qiao, Qiuna Tan, Guanting Dong, Minhui Wu, Chong Sun, Xiaoshuai Song, Zhuoma GongQue, Shanglin Lei, Zhe Wei, Miaoxuan Zhang, et al. We-math: Does your large multi- modal model achieve human-like mathematical reasoning? arXiv preprint arXiv:2407.01284 2024.
Qwen Team. QwQ-32B: Embracing the power of reinforcement learning, March 2025. URL httpss:/qweet http:///wwwnnn.......... wtt..ooooobllgoe.. gwglo/////wq-20 32b/
Gabriel Sarch, Snigdha Saha, Naitik Khandelwal, Ayush Jain, Michael Tarr, Aviral Kumar, and Katerina Fragkiadaki. Grounded reinforcement learning for visual reasoning. arXiv preprint arXiv:2505.2367%, 2025.
Hao Shao, Shengju Qian, Han Xiao, Guanglu Song. Zhuofan Zong, Letian Wang, Yu Liu, and Hong- sheng Li. Visual cot: Unleashing chain-of- of-thhuutttht reasoning in multi-modal language models. arXiv preprint arXiv:2403.1699) 2024a.
Zhihong Shao, Peiyi Wang. Qihao Zhu, Runxin Xu, Junxiao Song. Xiao Bi. Haowei Zhang, Mingchuan Zhang, YK Li, Yang Wu, et al. Deepseekmath: Pushing the limits of mathemati- cal reasoning in open language models. arXiv preprint arXiv:2402.0338 2024b.
Haozhan Shen, Kangjia Zhao, Tiancheng Zhao, Ruochen Xu, Zilun Zhang, Mingwei Zhu, and Jian- wei Yin. Zoomeye: Enhancing multimodal Ilms with human-like zooming capabilities through tree-based image exploration. arXiv preprint 2rxiv:2411..6044 2024.
Haozhan Shen, Peng Liu, Jingcheng Li, Chunxin Fang. Yibo Ma, Jiajia Liao, Qiaoli Shen, Zilun Zhang, Kangjia Zhao, Qianqian Zhang, et al. Vlm-rl: A stable and generalizable rl-style large vision-language model. arXiv preprint arXiv:2504.0766 07615, 2025.
Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. Retlexion: Language agents with verbal reinforcement learning. Advances in Neural Information Processing Systems, 36:8634-86555 2023.
Omkar Thawakar, Dinura Dissanayake, Ketan More, Ritesh Thawkar, Ahmed Heakl, Noor Ahsan, Yuhao Li, Mohammed Zumri, Jean Lahoud, Rao Muhammad Anwer, et al. Llamav-ol: Rethink- ing step-by-step visual reasoning in llms. arXiv preprint arXiv:2501.06188 2025.
Ye Tian, Baolin Peng, Linfeng Song, Lifeng Jin, Dian Yu, Lei Han, Haitao Mi, and Dong Yu. Toward self-imprrvemmnt of lims via imagination, searching, and criticizing. Advances Neural Information Processing Systems, 37:52723-5222 527448 2024.
Ke Wang, Junting Pan, Weikang Shi, Zimu Lu, Mingjie Zhan, and Hongsheng Li. Measuring multi- modal mathematical reasoning with math-vision dataset, 2024.
abs/2402. 14804
Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, Sharan Narang, Aakanksha Chowdh- ery, and Denny Zhou. Self-consistency improves chain thought reasoning in language models, 2023.
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoe Le, Denny Zhou, et al. Chain-of-thouuie thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824-24837 2022.
Yana Wei, Liang Zhao, Jianjian Sun, Kangheng Lin, Jisheng Yin, Jingcheng Hu, Yinmin Zhang, En Yu. Haoran Lv, Zejia Weng, et al. Open vision reasoner: Transferring linguistic cognitive behavior for visual reasoning. arXiv preprint 25Xiv=2507.05255, 2025.
13

---

Published as a conference paper at ICLR 2026
Haoning Wu, Xiao Huang, Yaohui Chen, Ya Zhang, Yanfeng Wang, and Weidi Xie. Spa- tialscore: Towards unified evaluation for multimodal spatial understanding. arXiv preprint arXiv:2505.17012, 2025a.
Haoning Wu, Xiao Huang, Yaohui Chen, Ya Zhang, Yanfeng Wang, and Weidi Xie. Spa- tialscore: Towards unified evaluation for multimodal spatial understanding, 2025b. URLhctps:
/
Penghao Wu and Saining Xie. V*: Guided visual search as core mechanism in multimodal llms. arXiv preprint urXiv:2312. 14135, 2023.
Qiong Wu, Xiangcong Yang, Yiyi Zhou, Chenxin Fang, Baiyang Song, Xiaoshuai Sun, and Ron- grong Ji. Grounded chain-of-thouggh for multimodal large language models. arXiv preprint arXiv:2503. 12799, 2025c.
Yuxi Xie, Kenji Kawaguchi, Yiran Zhao, James Xu Zhao, Min- Yen Kan, Junxian He, and Michael Xie. Self-evaluation guided beam search for reasoning. Advances in Neural Information Process- ing Systems, 36:41618 416500 2023.
Guowei Xu, Peng Jin, Ziang Wu, Hao Li, Yibing Song. Lichao Sun, and Li Yuan. Llava-cot: Let vision language models reason step-by-step. arXiv preprint 4rX4:2411.10440, 2024.
Liangyu Xu, Yingxiu Zhao, Jingyun Wang, Yingyao Wang, Bu Pi, Chen Wang, Mingliang Zhang, Jihao Gu, Xiang Li, Xiaoyong Zhu, al. Geosense: Evaluating identification and application of geometric principles in multimodal reasoning. arXiv preprint arXiv:2504.12591 2025.
Jianwei Yang, Hao Zhang, Feng Li, Xueyan Zou, Chunyuan Li, and Jianfeng Gao. Set-of-mark prompting unleashes extraordinary visual grounding in gpt-4v, 2023.
Yi Yang, Xiaoxuan He, Hongkun Pan, Xiyan Jiang, Yan Deng, Xingtao Yang, Haoyu Lu, Dacheng Yin, Fengyun Rao, Minfeng Zhu, Bo Zhang, and Wei Chen. RI-onevision: Advancing general- ized multimodal reasoning through cross-modal formalization, 2025. URL https://arxiv.
org/abs/ 2503. 10615
Huanjin Yao, Jiaxing Huang, Wenhao Wu, Jingyi Zhang, Yibo Wang, Shunyu Liu, Yingjie Wang, Yuxin Song, Haocheng Feng, Li Shen, et al. Mulberry: Empowering milm with ol-like reasoning and reflection via collective monte carlo tree search. arXiv preprint arxiv:2412./883/9 2024a.
Shunyu Yao. Dian Yu, Jeffrey Zhao, Izhak Shafran, Tom Griffiths, Yuan Cao, and Karthik Narasimhan. Tree of thoughts: Deliberate problem solving with large language models. Ad- vances in Neural Information Processing Systems, 36, 2024h.
Qiying Yu, Zheng Zhang. Ruofci Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Weinan Dai, Tiantian Fan, Gaohong Liu, Lingjun Liu, Xin Liu, Haibin Lin, Zhiqi Lin, Bole Ma, Guang- ming Sheng, Yuxuan Tong, Chi Zhang. Mofan Zhang, Wang Zhang, Hang Zhu, Jinhua Zhu, Jiaze Chen, Jiangjie Chen, Chengyi Wang, Hongli Yu, Yuxuan Song, Xiangpeng Wei, Hao Zhou, Jingjing Liu, Wei-Yo Ying Ma, Ya-Qin Zhang, Lin Yan, Mu Qiao, Yonghui Wu, and Mingx- uan Wang. Dapo: An open-source 11m reinforcement learning system at scale, 2025. URL
https::/arxxv..rggee wttpxii..rr//bb//553333 14476
Renrui Zhang, Dongzhi Jiang, Yichi Zhang, Haokun Lin, Ziyu Guo, Pengshuo Qiu, Aojun Zhou, Pan Lu, Kai-Wei Chang, Peng Gao, and Hongsheng Li. Mathverse: Does your multi-modal I1m truly see the diagrams in visual math problems?, 2024a. URL 2403. 4624
Ruohong Zhang, Bowen Zhang, Yanghao Li, Haotian Zhang, Zhiqing Sun, Zhe Gan, Yinfei Yang, Ruoming Pang, and Yiming Yang. Improve vision language model chain-of-thought reasoning. arXiv preprini arXiv:2410.1 161.6999,.6 2024b.
Xintong Zhang, Zhi Gao, Bofei Zhang, Pengxiang Li, Xiaowen Zhang, Yang Liu, Tao Yuan, Yuwei Wu, Yunde Jia, Song-Chun Zhu, and Qing Li. Chain-of focus: Adaptive visual search and zoom- ing for multimodal reasoning via rl, 2025. URL htt.hhtts:/// h/apps:///rse v. wttg:br////0....../ 11.00..554455
14

---

Published as a conference paper at ICLR 2026
Ziwei Zheng, Michael Yang. Jack Hong, Chenxiao Zhao, Guohai Xu, Le Yang, Chao Shen, and Xing Yu. Deepeyes: Incentivizing "thinking with images" via reinforcement learning. 2025.
URL httptss/////aii.oorrtt org/ 2bs52555.13362
Qiji Zhou, Ruochen Zhou, Zike Hu, Panzhong Lu, Siyang Gao, and Yue Zhang. Image-of-thought prompting for visual reasoning refinement in multimodal large language models. arXiv preprint arXiv:2405.13872. 2024.
Jinguo Zhu, Weiyun Wang. Zhe Chen, Zhaoyang Liu, Shenglong Ye, Lixin Gu, Hao Tian, Yuchen Duan, Weijie Su, Jie Shao, et al. Internv13: Exploring advanced training and test-time recipes for open-source multimodal models. arXiv preprint arXiv:2504. 10479, 2025.
15

---

Published as a conference paper at ICLR 2026
A
APPENDIX ROADMAP
To assist the reader in navigating our supplementary materials, we provide brief overview of the Appendix below. Appendix provides supplementary implementation details, including:
Detailed prompts in Appendix B.1
How to collect the training data (Appendix B.2
Training details in Appendix|B..
Details of the evaluation process in Appendix B.4
Appendix presents supplementary experiments and analysis:
Appendix C.1 analyzes the mode-selection behavior learned by Ada including fine-
grained selection, the impact of images on mode selection, and attention visualization.
Appendix explores whether the adaptive reasoning ability can be induced through SFT.
A detailed error analysis provided in Appendix
Computational efficiency analysis is conducted in Appendix
Appendix presents training dynamics during the two stages.
The scalability of MoVT and AdaVaR, as well the direction for future extension, are discussed in Appendix C.6
Appendix
C.7
provides additional ablation studies on several designs.
A comprehensive qualitative case study is included in Appendix C.8
B SUPPLEMENTED IMPLEMENTATION DETAILS
B.1 DETAILED PROMPTS
Here we provide the complete prompt template used in our adaptive reasoning model:
System: You are a helpful assistant. The user asks question related to an image, you need to solve Please first think about the reasoning process in the mind and then provide the user with the answer. The reasoning process and answer are enclosed within <think>> </think>>and <answer> <lanswer>>tags, respectively, i.e., reasonnnn process here </think>> <an- answ>a here </answer>. You have two modes of thinking, and you should choose the appropriate one based on the question:
1. Grounded Reasoning: Use this mode when you need to locate specific objects in the visual input. In your reasoning path, identify key objects and provide their corresponding bounding box coordinates in the format 'object[x1, y1, x2, y2]'. When using this mode, begin your re- sponse with the tag <ground>.
2. Text-hased Reasoning: Use this mode for general reasoning based solely textual thoughts.
No object localization or coordinate output is required in this mode. When using this mode, begin your response with the tag <text>>
Choose the mode that best fits the task, and structure your response accordingly.
User: {QUESTION]. Please first select the appropriate reasoning mode based the question, using <ground> <text>>to indicate the type, then follow the corresponding format to output the reasoning process, and finally provide the correct answer according to the user requirement. <mode prefix>> <think> ... </think> <answer>> </answer>>
Assistant:
where the (QUESTION) will be replaced by the input question. Please note that have added post prompt after each question to emphasize mode selection. in order to prevent the model from ignoring the system prompt due the long input sequence of images and questions.
16

---

Published as a conference paper at ICLR 2026
B.2 DETAILS ON TRAINING DATA COLLECTION
B.2.1 SFT DATA COLLECTION
Visunlly-Grounded Reasoning Data Regarding the visually- grouuded mode, we utilize the data constructed in VoCoT (Li et al. (025), we find that the dataset lacks multiple-choice samples, to avoid potential bias induced by such a distribution gap, we employ GPT-4o to rewrite a subset into multiple-choice format (by providing the original question and the correct option). To control the total size, we down-sample the GQA subset, resulting in a final dataset of 119K examples.
During the re-formulation process, we strictly adhere to copyright regulations. We would like to clarify that VoCoT data released under the Apache 2.0 license, which permits free reconstruction, modification, and creation of derivative datasets. We strictly adhere these requirements and will include the appropriate VoCoT attributions in our future open-son open-soureing process, clearly stating in documentation that our dataset is derived from VoCoT with modifications.
For quality assessment, we sample 100 reconstructed samples to check for formatting errors or the generation of false-negative options. None of the samples exhibited any issues, which is primarily due to the simplicity of the reconstruction task and GPT-4o's 's strong instruction-following capability. Therefore, we believe the quality of the reconstructed data is reliable.
Text- based Reasoning Data For the text-based mode, we aim to avoid introducing other induc- tive bias; therefore, instead of using R1 On-Onneeiiinn
approach (Yang al.) (025) built from DeepSeek-R1, we distill Orsta (Ma et al. 2025). reasoning model obtained by applying RL di- rectly to Qwen2.5-VL. The specific procedure as follows: We randomly sample total of 250K questions from R R1-One al. 2024), and Virgo (Du et al.) (025). For each question, Orsta generates eight responses with temperature of 1. We then use a rule-based method to determine correctness and retain only the correct instances; if multiple reasoning traces are correct, we randomly select one during training. We also observe that Orsta might skip the reasoning process and directly output an option, so we distinguish reasoning data from direct- annwer- data based on whether the model's 's reply contains the <think>> and <answer>> tags, and apply different system prompts for these two data types. After the rejection sampling, we obtain 115K reasoning examples and 95K direct-answer examples.
(Yang et al. 2025), Ilava-CoT (Xu et al.) 2024), NuminaMath1.5 (L)
B.2.2 RL DATA COLLECTION
Diffe Filtering Since overly simple questions cannot provide effective supervision during the RL process, we filter them out from the RL data. Following prior work (Ma et al., 2025 Zheng et al. 2025), adopt a model-based evaluation strategy: questions that answered correctly in all 8 random trials by Qwen2.5- are considered too simple and are excluded.
RL Data Collection As introduced in Section 3.3.2 our RL dataset consists of two parts: (1) existing data with verifiable answers, including math-related Geol 10K (Gao et al.] 2023) and MM- Eureka (Meng al., 2025), as well OmniCount (Mondal et al.) 2025) for object counting. We further filter MM-Eureka following the process: we first perform uniform sampling across the data sources (capping any category exceeding 1,500 instances at 1, 1,500) and remove overly simple ques- tions using the difficulty-based filtering method described in the previous paragraph, ultimately re- taining 13K examples. (2) Additionally constructed data for general scenarios. The questions and answers come from the SFT data used in LLaVA-One Vision (Li et al. 2024) and IntervLL (Zhu al., 2025). We retain only the subsets that are related to mathematics, OCR, object counting, STEM, grounding, and table/document understanding. For this part of data, we use an offline Qwe-2.5-V1L-72B (Bai et al.) 2025) model to assign task categories to each samples, using the same categorization scheme in Orsta (Ma et al. 2025).
To ensure that answers could be verified by a rule-based reward function, we first use Qwen2.55 VL-72B to determine the answer type. Depending on the task and question type, we collect and evaluate responses from multiple models via offline experiments, and, guided by the question type, augment some input queries with prompts specifying the required answer type. Ultimately, retain only those questions that could be judged accordingly. Finally, after filtering by difficulty to remove overly easy questions, we retain 22K examples.
17

---

Published as a conference paper at ICLR 2026
Table 4: The detailed training hyper- of Ada.
Configuration
Stage SFT
Stage RL
Model Initialization
Qwen2.5-V Inn-strctt
Stagel
Trainable Modules
MLP + LLM
All Modules AdamW
Optimizer
AdamW
Optimizer Hyperparameters
B1 0.9. 83 = 0.95, 1 81 = 0.9, A2 = 0.95, 1e-6
Global batch size
256
32
Peak learning rate
5e-6
le-6
Learning rate scheduler
Cosine
Linear
Training Epochs
Warm-up ratio
0.01
0.0
Weight decay
0.1
0.0
Max Sequence Length (SFT)
4096
Max Prompt Length (RL)
2048
Max RL Completion Length (RL)
2048
KL Penalty Coefficient (RL)
0.04
Generation Temperature (RL.
0.9
# Rollouts per Sample (RI.)
8
Numerical precision
bíloat16
bfloat
GPU Usage
8 NVIDIA A100
32 NVIDIA A100 zero3
DeepSpeed Configure
zero3
Training Time
29h
Multi-Modal Data Processing Since our model is based on Qwen2.5-VI. (Bai et al. 2025 our
data processing methods strictly follow those of Qwen2.5-VL: we apply the same dynamic resolu- tion strategy for patchifying images and use the corresponding tokenizer for tokenizing text.
Summary on Data Efficiency Our VaR framework operates under data-efficient setting, in terms of (1) Data sources: We rely entirely on existing datasets, and the processes of data recon- struction and filtering do not involve any additional human annotation. (2) Data volume: We use 329K samples in the SFT phase and 55K in the RL phase, totaling 394KK-comparable to other RI. methods such as OVR-7B (Wei et al. 2025) (428K), and significantly less than the SFT data used for mainstream LVLMs, such 1.8M samples used in LanA-OnnVisioon (Li et al. 2024).
B.3 TRAINING DETAILS
We list the hyper-parameter settings in Table and here we discuss the rationale behind several key parameter choices.
The Number of Training Epochs in the SFT Stage We follow the commonly adopted setup in existing works (Yang et al., 2025) Chen al. 2025a) to train one epoch for SFT. We validate the rational behind this setting in Appendix C.5
The Number of Rollouts Generated per Sample the RL stage, "# Rollouts per Sample" refers to the number of rollouts generated for each question. Generating more rollouts per sample can help provide better RL supervision signals and improve performance, but it also incurs higher compu- tational costs. Therefore, ensure fair comparison with other models in Table 1, we set it to 8, which corresponds to n = in Figure 2 and the equations in Section 3.3 meaning that the policy model generates 4 rollouts with each reasoning mode, for total of 8. Similarly, for fair com- parison with ablated models, when training the ablated models that are based on a single reasoning mode in Section 4.3
we set the number of rollouts generated in GRPO 8.
Coefficient for KL Penalty For brevity, the KL penalty are omitted in equations in Section 3.3 In practice, we introduce the KI. penalty term to preserve the knowledge of reasoning modes learned during the SFT phase. The coefficient is set to 0.04, following the settings used in existing works and frameworks (Chen et al. 2025bb
Shen et al. 2025 Hugging Face 2025).
During training, we implement monitor to track the performance of different modes on various types of data, allowing us to better observe the relative strengths and weaknesses between modes, the detailed training dynamics provided in Appendix C.5
Training Dynamics Monitor
18

---

Published as a conference paper at ICLR 2026
Algorithm Mode-relative advantage calculation.
Require: r[1.22n] # rewards of all rollouts where r[1 n) and r[n + 1 2n] correspond to the
text-based mode and the visually-grounded mode, respectively.
Require: ((x) CDF of the standard Gaussian distribution (0,)
Ensure: Ag
# mean and standard deviation of text-based mode
2: Me f- mean(r[1 rl), o, < std(r[1 : n!)
3: mean and standard deviation of visually- -aroulyegg mode
4: Me & mean(r(n + 2nl), o, + std[r(n 2n])
5: # Advantage text-based mode over visually-groundee mode
6: A, Φ
Voz+o?
7: # Advantage of risually-grounde mode text-based mode
8: Ag = =1-1,
9: return Ar, Av
Algorithm for Advantage Calculation Specifically, in Algorithm T we provide the pseudo code for caleulating the mode-relative advantage A, and A, in Section 3.3.1
B.4 EVALUATION DETAILS
we use the testmini split in Math- Vista (Lu et al., 2023), the test split Math Vision (Wang al. 2024), the vision-only portion of the testmini split in Math Verse (Zhang et al. (0024)). the val split in MMStar (Chen et al. (024a), the test split in V* (Wu Xie 2023). the testmini split WeMath (Qiao et al. 2024). all four splits of POPE (Li al. 2023), and the hard split of SpatialScore (Wu et al. 2025a). ). For the eight benchmark datasets used in this paper, the evaluated model is required to produce structured outputs (e.g., option letters numbers), we prompt the model to generate the desired format, and evaluate the results offline using sule-based procedures. Except for WeMath, for which we report the strict accuracy as specified in the original paper (Qiao al. 2024
Benchmark Details As indicated in the header of Table [
Average Schema Since each benchmark represents different domain, but the number of samples in each is inconsistent, all benchmarks are weighted equally for the average accuracy. We aim for the average performance to comprehensively reflect capabilities across multiple domains.
Mode-Switching Mechanism Specifically, the phrase "getting stuck in repetitive logic" men- tioned reffe to the situation where the mode reaches the maximum number of tokens (8192, significantly longer than the typical reasoning length of Ada VaR) without generating an- swer in the form <answer><<answerr> Please note that our mode-switching mechanism does not leverage any additional information, ensuring a fair comparison with other models; rather, mode switching is new feature enabled by our MoVT reasoning paradigm.
Compared Models As discussed in Section T the compared models can be divided into two groups: text-based models including LMM-RI Peng al. 2025), VIAAA-hhinker (Chen et al. 2025a), MM-Fureka (Meng et al. (025). OVR-7B (Wei et al. 2025 J and Orsta-7B (Ma et | as well as the souulll-grronded models including GRIT (Fan et al. 2025). ViGoRL (Sarch et al. 2025), DeepEyes (Zheng et al. 2025), and Chain-of-Focus (Zhang et al. 2025). Since most com- pared models focus on specific domains and do not provide evaluation results across all benchmarks, conduct evaluation to fill in the missing results for comprehensive comparison. Specifically, to ensure a fair comparison, we do not retrain the corresponding models; instead, we strictly follow the original papers, using the officially released pretrained checkpoints and evaluating them with the recommended parameters according to their respective evaluation protocols.
Evaluation Environment and Cost All evaluation is conducted single node with 8 A100 GPUs, accelerated by the vLLM framework (Kwon et al. 2023). As measured in Appendix C.4.2 complete evaluation of the 3B-scale model across all benchmarks requires 17.7 GPU hours
19

---

Published as a conference paper at ICLR 2026
Table 5: The proportion of mode selected by AdaVaR-3B across different image types in MMStar.
Mode Rate
Natural Images
Art
Document
Diagram
Geometry
Others
Grounded Text-based
92.20% 7.80%
39.40% 60.60%
8.10% 91.90%
4.50% 95.50%
0% 100%
0%
100%
Virually-Greunned Rate
Text-based Rate
electronics energy mechanical eng.
100.0%
100.0% 0%
geography earth science & agriculture 27.8%
72.4%
96.6%
blology chemistry & physics
92.5%
97.9%
numeric and calculation
93.8%
97.9%
statistical reasoning
98.8%.
100.0%
geometry
100.0%
100.0%
code sequence reasoning
97.4%
100.0%
diagram reasoning
91.8%
100.0r
reasoning
60.4%
39.6%
59.4%
40.6%
single-Instance reasoning
74.7%
25.3%
77.8%
22.2%
cross-Instance attribute reasoning
73.0%
27.0%
53.9%
46.1%
cross-Instance relation reasoning
33.9%
66.1%
30.6%
69.4%
localization
97.5%
100. 0%
object counting
95.7%
90.2%
recognition
94.1%
83.9%
16.1%
Image style quallty
83.3%
16.7%
65.4%
34.6%
Image emotlon
93.5%
90.3%
Image and topic
95.0%
95.7%
0%
20%
40%
60%
80%
100%
0%
20%
40%5
60%
80%
100%
AdaVa R-3B
AdaVaR-
Figure 5: Proportion of each mode selected by different models across sub-categories MMStar.
(on A100), making frequent performance monitoring during evaluation highly resource-intensee Therefore, only track Math Vista performance, shown in Figure
B.5 LLM USAGE
Our of LLMs in this paper mainly comprises two parts: writing assistance and tools for data construction. During writing, we use GPT-4o to help polish the exposition. In the data construction process, described in Appendix [B.2] employ Qwen2.5-VL. and GPT-4o assist with data reconstruction, annotation, and filtering.
C SUPPLEMENTARY EXPERIMENTAL RESULTS AND ANALYSIS
C.1 DELVING INTO MODE-SELECTIR BEHAVIORS
In the main text, we have discussed mode selection at the dataset level and the coarse- coarse-grained task level within MMStar. This section provides additional analysis complement that discussion.
C.1.1 MODE SELECTION THE TINE-GRAINED TASK LEVEL
Figure illustrate the fine-grained mode selection tendencies of Ada VaR-3B and Ada We observe that, since the 7B model has richer knowledge, Ada Ada VaR-7B prefers the text-based mode for knowledge-related categories, such as geography & earth science agriculture. This also highlights the necessity of the RL process:
need sample during RL to better estimate the strengths and weaknesses of each mode, which are different for different models (there is no universal ground truth of mode selection that is applicable for all models).
20

---

Published as a conference paper at ICLR 2026
Table 6: Comparison between Ada VaR and simple mode ensemble strategies.
Model
MathVista
MathVision
Vathee
WcMath
MMStar
POPE
V*
SpaScore
Avg. Acc
AdaVaR-3B
69.8
24.5
35.2
33.8
59.3
77.1
88.2
18.9
50.8
Expert Pick
68.1
23.0
34,9
32.6
56.3
76.4
Max TXT)
68.1
23.0
34.9
32.6
58.5
76.4
88.2
18.5
50.1
70
120
50
80-
40
20
10
Attention on text tokens
20
Attention text tokens
Attention on image tokens
Attention image tokens
0.5
022
06
0a
1.
3.5
62
64
Di
5.
Attention when Generating <text>
Attention when Generating <ground>
Figure 6: Distribution of attention mass over tokens (blue) and image tokens (orange) when
generating the <text> (left) and <ground> (right) mode prefixes on MMStar.
C.1.2 THE IMPACT OF IMAGE TYPES ON MODE SELECTION
Furthermore, we conduct an analysis from the perspective of images. We employ GPT-5 annotate the image types in MMStar and analyze their impact on mode selection. As presented in Table image type also has significant impact on mode selection: (i) for natural images, the model tends to use the grounded mode to precisely locate and analyze key objects; (ii) when images contain more abstract harder-to- lo-caliize concepts- such as artistic styles in artwork, content in documents and diagrams, or angles in geometry problems-the the model tends to favor the text-based reasoning mode.
C.1.3 ATTENTION VISUALIZATION FOR MODE SELECTION
To better understand how MoVT selects between the text-based and grounded modes, visualize the attention patterns used when predicting the mode prefix. Concretely, for each sample we probe the attention scores from the token immediately preceding the mode prefix (i.c.. the token that "de- cides" the mode) to all previous tokens. Following (Kaduri et al. 2025), we select the middle 25% of layers and average the attention weights across these layers and across all attention heads.
We first aggregate the attention mass over all image tokens and all text tokens on the MMStar dataset. As shown in Figure when generating either the <text> or <ground> prefix, the model places the vast majority of attention on text tokens (0.97 in average) rather than image tokens (0.03 average). This is consistent with the nature of MMStar, where the question type is primarily encoded in the textual prompt and multiple questions be asked about the same image.
We further examine how attention is distributed over different textual tokens. Figure 7 visualizes token-level attention maps for representative examples from six benchmarks. We observe that the at- tention concentrates on tokens that define the question type, such as the interrogative words "which", "what"; key mathematical terms like perimeter", "area" and object-oriented concepts of "back- ground", "bus" and "person".
C.1.4 COMPARISON WITH NAIVE MODE ENSEMBLE STRATEGIES
To validate the effectiveness of the mode selection learned by VaR, compare it with two naive strategies to use external knowledge for mode ensemble at the dataset level: (1) Expert pick: selection based on expert knowledge-hert we choose the text-based mode for math benchmarks and the grounded mode for others. Note that, in practical applications, even experts would find difficult to make reasonable judgments mixed-task datasets like MMStar and SpatialScore. (2) Max(GRD, TXT): for each dataset, we select the mode that achieves higher accuracy. Note that this strategy uses additional information from the dataset, making it inherently unfair for comparison.
21

---

Published as a conference paper at ICLR 2026
Case from Math
0.10
0.28
0.17
0.20
ocfricter
roctongle
Case from MathVision
in
cm
6f
figure
Case from SpatialScore
9.71
9.16
9.06
9.21
0.41
0:45
0.39
0.20
0.62
0.22
9.53
Which
option
describe
the
objett releltonship
In
the
Image
conectly
MMStar
0:34
0.13
0.34
2.14
0.07
9.05
0.12
0.42
What
the
main
Teature
In
the
teckimuud
of
the
Image
Case from VStar
0.711
0.13
0.34
0.40
0.17
0.23
0.32
0.37
0.2
0.10
0.04
0.35
0.39
13
the
blue
kugggee
on
left
rght
side
0f
the
805
Case from POPE
0.82
0.15
petaah
Image
Figure 7: Token-level attention heatmaps over the question when generating the mode prefix.
Table 7: Performance of adaptive reasoning models.
Math-Oriented
General Scenarios
Average Acc.
Model
Vathe
WeMath
MMStar
POPE
SpuScore
Qwen2.5-VL-3B
62.3
21.2
31.2
21.1
$3.1
73.5
86.1
16.2
45.6
Stage SFT-based Models
MaVaR-SFF-33
60.4
23.2
33.0
29.5
54.7
75.5
89.1
16.8
47.8
GRID
31%
5%
1%
78%
905%
98%
94%
96%
Stuge RL-bused Models
AdaVaR-3B
69.8
24.5
35.2
33.8
59.3
77.1
88.2
18.9
50.8
GRD%
2%
0%
51%
99%
100%
59%
SFFT-Guided Aduptive Reasoming Model
-dT-taaative-38
59.1
21.2
30.7
29.1
$2.8
77.5
84.5
17.6
47.2
GRD
86%
90%
59%
100%
100%
100%
100%
100%
The results in Table 6 show that both strategies- with additional information-ant outper- formed by the selection made by AdaVaR. This indicates that: (i) mode selection at the individual sample level is necessary, especially for diverse queries in datasets like MMStar and in real-worke applications; and (ii) our model is capable of making highly reasonable and effective mode choices.
C.2 CAN THE ABILITY TO ADAPTIVELY SELECT MODES BE LEARNED VIA SFT?
To further validate the claim made in Section 1 that it is difficult control the model's ability to choose among different modes during the SFT stage-and to demonstrate the necessity of the RL stage, we conduct an supplementary experiment. Without relying on RL at all, we train the model using two stages of SFT. The first stage is identical to AdaVaR Stage I and is used to learn the reasoning capabilities of different modes; the second stage is based on sampling from the model trained in Stage to offline estimate the relative performance of the different modes.
Specifically. using the same RL data as Ada VaR, estimate for each question with AdaVaR-SFT by running inference eight times with each of the two modes (temperature=1), estimating the accuracy of both modes, and retaining the rollouts of the mode with higher accuracy; if the accuracies are equal, we keep both. We then aggregate the reasoning trajectories corresponding to the estimated better mode(s) across all questions to serve as the SFT data for the second stage.
As presented in Table 7 SFT-Adaptive-3B indicates the SFT-guided adaptive reasoning baseline. can be seen that SFT- SFT-Adaptive-3B exhibits a clear preference for the grounded mode; according to the evaluation results, this is unwarranted, because its performance its overall performance are even worse than the model after Stage 1 (except on POPE and VStar, which are dominated by the grounded mode). A possible reason that, shown in Figure 3a after
mathematical tasks and
22

---

Published as a conference paper at ICLR 2026
Table 8: Proportions of different response types from Ada across different datasets.
Response Type
MathVista
MathVision
Mathe
WeMath
MMStar
VStar
POPE
SpaScore
Correct
69.80%
24.50%
35.20%
33.80%
59.30%
77.10%
88.20%
18.90%
Selection Error
4.80%
6.30%
7.20%
9.10%
8.80%
2.60%
2.90%
10,50%
Reasoning Error
25.40%
69.20%
57.60%
57.10%
31.90%
20.30%
8.90%
70.60%
SF((S+RRF)
15.89%
8.34%
11.11%
13.75%
21.62%
11.35%
24.58%
12.95%
Table 9: Distribution of selection errors across different question types and selected modes.
Selected Mode
Science
Perception
Math
Logical Reasoning
Instance Reasoning
Total
GRID Mode
68
TXT Mode
33
14
12
12
71
Total
36
43
14
24
22
139
SFT the performances of the two modes have not yet stabilized (the text mode still has substantial room for improvement). Due to the limitations of the SFT framework, we can only perform offline sampling, which further reinforces this preference, reflected in SFT-Adaptive-3B.
Overall, as an offline learning framework, SFT finds it is difficult to both strengthen different modes and guide mode selection, and it cannot correct unreasonable biases that emerge during iteration (and may even exacerbate them). In contrast, RI,, as an online learning framework, uses dynamic sampling and estimation effectively help the model enhance its reasoning ability and assess the rel- ative merits of the modes. Therefore, the RL stage in the Ada framework is necessary. However, as promising future direction, worth exploring multi-round SFT+RL iterative framework to further improve the model's capabilities.
C.3 ERROR ANALYSIS
First, we categorize errors into two types: (i) mode selection error (SE): the model fails to select the mode that could have produced the correct answer; (ii) reasoning error (RE): neither mode is able to produce the correct answer. As shown in Table across all datasets, mode selection errors constitute only a small portion of the total errors (all less than 1/4).
Furthermore, we investigate the distribution of mode selection errors across different question types and image types in MMStar. Results in Table 9 and Table 10] show that: (1) most errors occur in domains where one mode has a clear advantage, such as perception tasks and natural images, where the grounded mode performs better, and science mode holds the advantage. (2) These exceptions illustrate that the complementarity between the two modes exists not only at the domain level, but also at the sample level. As result, optimal mode selection strategy cannot be predetermined and must be learned through the AdaGRPO algorithm.
math problems, where the text-based
Overall, results indicate: (i) the context-adaptive mode selection in MoVT is reasonable, and (ii) our Ada VaR framework is capable of learning effective selection strategies, (iii) while still leaving room for further optimization.
C.4 ANALYSIS OF THE COMPUTATIONAL EFFICIENCY
C.4.1 TRAINING EFFICIENCY: GRPO VERSUS ADAGRPO
First, as stated in Section 3.3. and Appendix B.3 to ensure a fair comparison between GRPO, Ada- GRPO generates n 4 rollouts per mode (8 per sample), while GRPO generates 2n 8 rollouts per sample. In this section, we verify the training efficiency both theoretically and empirically.
From theoretical perspective, the main computational cost is incurred during rollout generation and the calculation of the training objective: (i) In rollout generation, simply generating rollouts separately for each mode would require two KV cache computations, leading to additional cost. However, in our design, the two modes share the same input sequence except for the mode-specific prefixes. By sharing the KV cache across modes, achieve nearly the same computational cost as
23

---

Published as a conference paper at ICLR 2026
Table 10: Distribution of selection errors across different image types and selected modes.
Selected Mode
Natural Images
Diagram
Art
Geometry
Document
Others
Total
GRD Mode
59
4
5
0
0
0
68
TXT Mode
46
6
6
4
1
71
Total
67
30
11
6 4
1
139
Table 11: Comparison between the training efficiency GRPO and AdaGRPO. Theoretical FLOPs estimated within single optimization step: based on Qwen2.5-VL-33 using image of size 1024x768, a question of 260 tokens, and KV cache enabled. Empirical wall-clock time is measured by running 100 steps during the training process and averaging the results.
Algarithm
Theoretical FLOPs
Empinical Wall-lock Time (scconds)
Rollout Generation
Reward Calculation
Advaniuge Calculation
Objective Forward
Total
Rollout Generation
Reward Calculation
Advantage Calculation
Objective Forward
Total
AdaGRPO
20.31T
0
111.9T
132.2T 134.4T
18.45
1.400-02
2.20F-04
1.12
19.58
GRPO
20.47T
0
113.9T
19.18
1,20E-02
1.10E-03
1.17
20.36
GRPO. (ii) Regarding the training objective, both algorithms require forward passes of the policy model and reference model on 8 rollouts, so their computational costs are also nearly identical.
For empirical validation, we implement FLOPS tracker using the deepspeed library and monitor the wall-clock time during training. As presented in Table [1] both theoretical computation and actual execution statistics show that the efficiency of AdaGRPO and GRPO is nearly identical (with approximately 4% additional overhead).
C.4.2 INFERENCE EFFICIENCY
As for inference, theoretically, as described in Section 3.1 our Ada model naturally integrates mode selection and reasoning into a single sequence generation step. During inference, unlike the training phase, there is need to estimate mode-wise advantages. Therefore, similar single-mode reasoning models, AdaVaR only requires a single call to the "modellgeneratee) method for each sample. Empirically,
compare the inference efficiency of AdaVaR and single-mode baselines in Table [2] As illustrated, the inference efficiency of different reasoning models is very similar on average, demonstrating that our framework does introduce additional inference cost.
C.5 TRAINING DYNAMICS
Loss Curve in the SFT Stage Figure saoshows the training loss during the SET stage, demonstrat- ing that the training process has already converged after epoch.
Rewards in the RL Stage Figure shows the training dynamic of the two reasoning modes across different task during Stage 2 (the diverse mixture), illustrating the relative strengths and weakness between two modes. Based on the comparative performance of the two modes, find that the text-based mode predominates on Math and Science, while the grounded mode excels on Detection and Grounding; for the other tasks, the modes perform similarly. Overall, these results align with our expectations and further guide the learning of adaptive mode selection across tasks.
Rollout Length in the RL Stage Figure 9b iblustrrtssss the change in rollout length for two modes during the RL process. It can be observed that the lengths of rollouts in two modes are relatively close, with a difference of less than 20 tokens.
C.6 DISCUSSION ON SCALABILITY AND FUTURE WORK
Although this paper demonstrates that the adaptive reasoning paradigm MoVT is feasible and effective solution for building derscore its substantial potential. In this section, we discuss directions for future extension.
visual reasoning capabilities, our results further un-
24

---

Published as a conference paper at ICLR 2026
Table 12: Inference time (seconds) per sample across different models and datasets, measured with
8 A100 GPUs and accelerated by the vLLM framework (Kwon et al. 2023p.
Model
Vath
MathVision
MathVerse
WeMath
MMStar
POPE
Vo
SpaScore
Avg.
VdRVaR-3B
0.32
0.95
0.45
0.48
0.18
0.07
0.25
0.87
0.36
TexI-SFT-RL
0.32
0.91
0.44
0.53
0.21
0.11
0.23
0.74
0.37
Grounded- SF--RL
0.31
0.85
0.39
0.4
0.25
0.27
0.84
0.34
samples
1000
3040
3940
1400
Task: Math
Task: Chart
Task: Counting
0.5
0.4
Accuracy Reward
Accuracy Reward
0.4
0.3
0.3
0.3
0.2
0.1
0.1
Mode
Madi
0.0
Mode
0.0
400
800
200
400
600
800
1000
2000
Step
Step
Task: Science
Detection
Others
Accuracy Reward
0.4
Accuracy Reward
0.3
GRD Mode
0.2
Made
0.1
TXT Mode
0.0
GRDD Modic
0.0
GRD Mode
0.0
200
600
800
1000
1000
Step
Step
Stop
Task; Grounding
Task: Document
Task:
Accuracy Reward
0.5
0.6
Accuracy Roward
0.5
GRD Pade
TXT Mode
0.2
0.2
TXT Mode
0.1
TXT Mode
0.1
0.0
GRD Madic
0.0
GRD Mods
0.0
200
400
Step
Step
Step
Figure 8: Accuracy reward curves for different modes across different tasks during RL.
C.6.1 THE SCALABILITY OF MOVT TO MORE REASONING MODES
In the main paper, we instantiate MVoT with two mainstream reasoning modes: text-based mode and grounded mode. This design already validates the effectiveness MoVT unified paradigm for adaptive mode selection. Then, to naturally extend MVoT to support richer reason- ing patterns, we consider two complementary directions illustrated Figure To width extension by adding more reasoning modes and depth extension by composing modes over multiple steps.
Width Extension Beyond the current text-based and grounded modes in Figure 10 a), MoVT can incorporate additional complementary modes by assigning each mode dedicated prefix. For exam- ple, as shown in Figure we may introduce tool-use mode, where the model is encouraged to invoke external image-related tools before reasoning, e.g., <tool> <think>
</ttink>>>> no-think mode, where the model directly produces an answer
< /naswer>> without explicit intermediate reasoning, e.g., <answer> <..</ <..<//nswer.. By constructing cor- responding SFT data and performing RL to compare the new modes against existing ones under the MT framework, new reasoning skills can be seamlessly integrated into the model while preserv- ing the unified mode-selection mechanism.
<answer>
25

---

Published as a conference paper at ICLR 2026
180
Mode TXT
Mode GRD
160
Response Length
Logs
0.8
0.7
120
0.6
100
Binary Mixture
Diverse Mixture
1000
500
1000
1500
2000
2500
Training Step
Training Step
(a) Training loss in the SFT stage.
(b) Response lengths during RL stage.
Figure 9: Supplementary training curves.
Mode
cleat> <thinks ... </thinio
<anower> </answer>>
canpwer... </answer>
<thinlo..<tthink
ggoundd <hhinlo.</thiilo
</answer>
cthinko.s/thinkk
(a) Current Formulation of MoVT
s/anowero
Text-lased Mode
<think>..s/hiikke canswer>., dannwer>>
<lext> <thinlo..<thhikk>
<answer...
</answer>
Grounded
<ground> chinlo..</hiilll
spround> <think.../ttinkk
Question
canswer... Jannwwe>>
canswer... </answere
Question
<tool </hinko..</thinke answers.c/anonn s/anower>
Tool-use
mode
<answer... </answer>
canswer> _.ower>>.... Janwers
Doithink Mode
Depth Extension of MoVT
Figure 10: Extensions of MoVT to additional reasoning modes.
Depth Extension MoVT can also be extended in depth by allowing multi-step mode selection and reasoning, as depicted in Figure [0]c). Concretely, at each step the model can either select mode (e.g., text-based grounded), performs reasoning within that mode or optionally decides to terminate by producing an answer based on the current information. This yields a flexible controller that can realize traditional single-mode reasoning patterns constructed via sequential mixing and mode switching (c.g., grounded - text-based, or text-based -> grounded). Under this formulation, inference and training can be further enhanced by search strategies and RL objectives that explicitly explore the space of multi-step mode sequences.
special case but with more complex reasoning
C.6.2 THE SCALABILITY OF THE ADAVAR LEARNING FRAMEWORK
The Ada learning framework can also be naturally adapted to the two extension directions in- troduced in Appendix. While the SFT extension only requires constructing the corresponding data, here we primarily discuss the extension strategy for the AdaGRPO algorithm.
Scalability Prefix-Guided Mode Exploration For width extension to k>2 modes, we can improve efficiency by selecting j<ck modes to explore for each sample: (i) randomly selecting modes in the early training stages, and (ii) once the model develops certain mode selection capability, selecting the top-j modes based the model's predicted probabilities. For depth extension, intervene after the end of each thought process (i.c., after the </think> token) by append all feasible prefixes construct rollouts with hierarchical and thorough exploration.
26

---

Published as a conference paper at ICLR 2026
Table 13: The impact of data proportions among different modes during the SFT stage, Each cell is presented in the format 'accuracy / proportion of selecting the grounded mode'.
Modal
SET Raiio
MothVista
MathVision
MatiNose
WeMath
MMS
POPF
SpatialSeve
Avp. Ace
5-V1-25
62.3
21.2
31.2
21.1
53.1
73.5
86.1
16.2
45.6
SPT-kosed Models
AdVVR--SFTE
GRD-XX--1:
804/315
2
33.0/ 1%
29.5/ 75%
$4.7/ 90%
75.5/
89.1 1979
16.8/965
47.8
AdEVaR-SHI SFT-38
57.1/52% GRD4XXT=311
21.5/ 10%
31.2/35 39
29.47 84%
51.4/ 9906
75.3 190%
99.6/ 100%
17.01 1008
46.5
AdvVR--SFT-T
GRDD/XTT113
61.0/219
23.1/39
32.5/0%
50.0/400 /40%
$5.0 75%
75.0/958
8 /81%
16.52 9256
47.2
RE-bosod Models
AdvVaR-3B
GRD/TXT=I:
6%
24.510%
35.2/0%
33.8/00
59.3/51%
77.1 7999
88.27 100%
18.97
50.8
3B
GRD:XXI=33
66.8/66
24.1
14.8/0%
32.4/25
57.2/ 64%
76.8/1000 190%
83.97
17.4/ 30%
McVaR-38
GRD
24.8/09
34.3/0%
/458
76./
16.5 447%
Table 14: The impact of mode prefix formatting. For Ada VaR models, each cell is presented in the format 'accuracy/ proportion of selecting the grounded mode'.
Model
Mode Prefx
Mathiisia
Muth Vision
Math
WeMath
MMStar
V*
POPE
SpatialScoes
Avg. Ace
Qwen2.5-V LB
21.2
31.2
21.1
53.1
73.5
86,2
AdMA5R-3B
<lest>, <ground>
24.5
35.//0%
338/0%
77.17 79999
100%
18.91
50.8 /5.6/55%
AdNaR 3B
<model>,, <mode2.>
09.6/2%
238/0%
33.000%
/0
39.5/45%
77.5/ 100%
89.07
100%
50.7
although Ada achieves substantial performance gains, there remains a gap to the upper bound of adaptive reasoning, indi- cating that the model's mode-selection capability can be further improved. In the future work, can explore to construct richer datasets, or introduce a dedicated reasoning mechanism for mode selection itself-having having the model first reason about which mode use and then reason about the problem to strengthen mode selection.
Advancing Mode Selection As shown in Table 2 and Appendix C.3
C.6.3 EXTENSION TO WIDER DOMAINS
Extension Multilingual Context As stated in Ethics Statement, following mainstream visual reasoning models, our work primarily focuses on English scenarios. The main limitation prevent- ing thorough exploration of multilingual visual reasoning is the lack of high-quality data, both for training and evaluation. In future work, we aim to explore several key directions: (1) how to effi- ciently construct multilingual reasoning-oriented datasets; (2) the cross-lingual transferability and generalization of reasoning capabilities; and (3) how to develop adaptive reasoning methods for multilingual scenario, in order to extend MoVT and VaR to a broader range of languages.
Extension to Other Modalities Our method primarily focuses on images. The improvements observed on video and multi-view 3D scenes in SpatialScore indicate that our approach can be ex- tended to additional modalities. When dealing with diverse modalities-sucl as video, 3D data, and medical imaging- believe adaptive reasoning will offer even greater value, as different modali- ties often require distinct reasoning modes. In future work, we aim to explore grounded reasoning methods -igh-sinnnsi scenarios, expand the repertoire of reasoning modes, and investigate the transferability of these modes across different modalities.
C.7 SUPPLEMENTARY ABLATION STUDY
C.7.1 THE 1:1 DATA RATIO BETWEEN TWO MODES DURING SFT
we chose the 1:1 ratio to avoid introducing a preference for any specific mode during the SFT stage. To validate this design, Table [3] we experiment with different data ratios under the premise of keeping the total amount of data constant.
As stated in Section 3.2
Several findings can be gleaned: (1) Higher proportion of a specific mode in the SFT data makes the model more inclined to select that mode: this tendency is particularly evident in the SFT model. After RL, different models converge toward similar trends, but the influence of the SFT stage still persists. (2) Such preference may be effective on specific datasets (e.g.. the preference for the text mode improves performance on WeMath); however, from the perspective of generalization (average performance), although all biased settings outperform the baseline, none perform well as the 1:1 setting. (3) In addition, a major limitation of applying a skewed data ratio is that, in practical applications, facing various queries, it is difficult to know in advance which mode would be better.
27

---

Published as a conference paper at ICLR 2026
Table 15: Ablation study on the SFT stage.
Model
Math Vista
Math Vision
MathVerse
WeMath
MMSta
V*
POPE
SpaScore
Avg. Acc
Qwen2.5 VL-3B
62.3
21.2
31.2
21.1
53.13
73.52
$6,16
16.24
45.6
VaRVaR-3B
69.8
24.5
35.2
33.8
59.3
77.1
88.2
18.9
50.8
w/o SFT
63.2
21.7
31.3
26,9
54.1
73.2
82.4
18.6
46,5
Table 16: The impact of different system prompts. Each cell is presented in the format 'accuracy / proportion of selecting the grounded mode"..
procegt
pronus
MutiViit
Math/ssion
MathVene
WcMalh
MMSSar
POPE
SpaSesrc
Aug.
Owen225 3B
62.3
21.2
31,2
55.1
16.2
45.6
AdaVaR--38
full full think calv
fell
69.5//26
24.5/0% 70/
35.2/0%
158/00
59.3/516
77.1/905
88.2 10005
189
50.3
AdaVuR-3B 38
think think only
69.1
36./08
150/19
57.8/596
100%
88.6 1009
50.5
pousppt
69
36.1/06
332/0%
58.4/52%
77.0/ 100%
88.3/ 100%
18.5/596
50.3
In summary, although the RL phase helps reduce the sensitivity the data in the SFT stage, an unbiased design between different modes is optimal for achieving superior general performance.
C.7.2 THE FORMAT OF MODE PREFIXES
Currently, we utilize two mode prefixes with explicit semantics. To investigate the sensitivity to the format of prefixes, we conduct an ablation experiment, using semantically neutral prefixes, namely <model> and <mode2>, As presented in Table 14] models with different prefix formats show similar performance, indicating that, within our mode unification design, different prefix formats have minimal impact, and the model is able learn distinct reasoning modes after SFT.
C.7.3 THE EFFECTS OF SYSTEM PROMPT DESIGN
In our framework, we learn different reasoning modes during the SFT stage, where supervision primarily come from the SFT data, and the system prompt serves as an auxiliary guide. To verify the role of the system prompt, we conduct two ablation studies, aiming to address two questions: (1) Can different reasoning modes be elicited solely through the system prompt without relying on SFT data? and (2) Does the system prompt require detailed descriptions of the modes?
For the first question, we ablate Vda-aR by skipping the SFT stage. Results in Table 15 show that: (i) Even provided with the description of the grounded mode, the model does not elicit the ability to perform grounded reasoning during the RL phase-only 0.03% of rollouts contain coordi- nates. (ii) In evaluations, the model shows no clear distinction between modes and fails to generate responses in the grounded reasoning format, achieving only a marginal improvement over Qwen2.5- VL-3B. Therefore, these results show that it difficult to directly elicit multiple distinct reasoning modes merely by designing system prompts, which highlights the necessity of the SFT phase in our framework regardless of whether it is applied to Qwen2.5 S-VL or other LVLMs.
To investigate the sensitivity to the system prompt, experiment with two variants: full, which includes both the thinking instructions and the descriptions of the modes, and think-only, which re- moves the mode descriptions. Table 16 presents the results obtained using different system prompts during training and evaluation: (i) Our model, that is trained with the full prompt, can directly trans- fer to the think-only prompt and still maintain the capabilities of both reasoning modes, performing reasonable mode selection. (ii) When training with the think-only prompt, the model achieves per- formance nearly on par with that of the model trained using the full prompt. These results indicate that, after SFT, the model exhibits low sensitivity to the prompt. When extending to more modes, there is no need to design complex prompts, which enhances the scalability of our framework.
C.7.4 DATA ORDER FOR CURRICULUM LEARNING
Results in Table B validate the effectiveness of the easy-to-hard data order. In this section, we investigate whether a hard-to-easy order works. As illustrated in Table the model trained with hard-to-easy data order consistently underperforms compared to the easy-to-hane setting, validating the rationality of our design.
28

---

Published as a conference paper at ICLR 2026
Table 17: The impact of data order in curiculum learning.
Model
Data Order Math
Vishvn
Verse
WeMath
MMStar
V*
POPE
SpaScore
Avg. Acc
AdaVaR.3BB
easy hard
69.8
24.5
35.2
33.8
59.3
77.1
88.2
18.9
50.8
VaR--BB
hard-to-easy
68.1
24.2
35.2
32.3
58.5
76.4
87.3
18.1
50.0
Table 18: Robustness against image noise. For Ada VaR, each cell is presented in the format "accu- racy/ proportion of selecting the grounded mode".
Model
Gaussian Blur
Marh
MathVision
MattiVesee
WeMath
MMSttr
V
POPE
SpaScore
Avg. Ace
VLL
No
62.3
21.2
31,2
21.1
53,1
73.5
86,1
16.2
45.6
AdMVRR-33
69.8/29
24.5/0%
35.//09
33.8/0%
39.3/51%
100%
/8.9/595
AdiVRR-3R
68.8/2%
33.7/09
58.3/459
/999/
88.2/ 98%
18.5/
SFT-SL
67.2
23.52
36.1
32.19
56
70.7
85.6
18.8
48.8
GRD-SFT-RL.
No
62.6
22.17
34,4
33
55.4
77
87.4
18.4
48.7
C.7.5 ROBUSTNESS AGAINST IMAGE NOISE
To investigate whether the adaptive mode selection ability in Ada VaR is sensitive to image noises, we conduct experiment by applying Gaussian blur to images during evaluation, we set the radius of blur to 0.5, in order to prevent excessive degradation of image information. The results are shown in Table 18 find that: (1) introducing noise does not significantly affect mode selection; (2) the overall performance of Ada VaR shows only a slight decline; (3) even with added noise, Ada VaR-3B still maintains& clear advantage over baselines. These supports the robustness of our model.
C.8 QUALITATIVE ANALYSIS
To illustrate the characteristics of different reasoning modes and how our Ada model select the appropriate mode, provide examples across multiple datasets, including maage-rllated questions, the reasoning trajectories of different modes, and the model's mode selections.
General Notation and Findings In this section, all figures follow the same notation scheme: we underline the correct answers and visualize the images the bounding boxes generated by the visually-grounded grounded mode; different colors indicate the correspondence between the bounding boxes the images and the text; "TXT Mode" and "GRD Mode" are respectively short for the text-based mode and the visually-groudded mode. By examining cases across different datasets, clear differences in the reasoning trajectories produced by the two modes, which also validates our discussion in Section 4.3.1 the two modes induce distinct reasoning patterns. In general, text-based reasoning is better at constructing complex, abstract logic, whereas the grounded mode tends to locate and search for key information and analyze it.
observe
Math Vista includes both math-oriented and general-scenane questions. As presented in Figure 11 we make several observations: (1) For more general scenarios for example, the first case the grounded mode, which places greater emphasis on visual information, performs better; (2) On tasks with relatively simple solution methods, such as the second case, the grounded mode also succeeds; (3) For problems requiring complex and abstract logic, like the third case, the textual mode is superior; (4) Moreover, when the grounded mode recognizes that there requiring localization for instance, the geometry problem in the fourth case-ic it also exhibits a certain degree of abstract reasoning ability.
Math Vista
concepts
Math-Oriented Benchmarks Figure shows examples from three other math-oriented datasets. We observe pattern: in these datasets, for nouns referring to abstract concepts (c.g., numbers), the grounded reasoning mode may invoke the "find" mechanism yet still cannot locate a concrete corresponding bounding box in the image. This corroborates our assumption that performing visual grounding is difficult abstract scenarios. In such cases, the grounded mode can handle simple questions (e.g., the fourth case), but fails on those requiring step-by-step computation, such as the first case. Another observation is that the grounded mode is weak at numerical computation. For example, in the second and third cases it produces correct initial derivations but makes mistakes during the numerical calculations; in the second case, it even gets stuck in a repetitive generation loop and fails to solve the corresponding equation.
29

---

Published as a conference paper at ICLR 2026
V* V and POPE are representative of tasks that hinge fine-grained object-related details. V* in particular tests the model's ability to localize small objects. We provide several cases in Figure [3] As seen in the first two cases, text-based reasoning in such scenarios yields reasoning that lacks interpretability: without coordinates, users cannot tell whether the model actually localized the target object. Text-based reasoning is also prone to hallucinations; for example, in the second model produces incorrect information: it fails to find the bicycle and assumes in advance that the dog is to the left of the bicycle. In contrast, grounded reasoning can precisely locate the target object in such complex images, and the correct coordinates help the model answer the question correctly.
the
POPE
Considering the hallucination task POPE, the third case Figure 13 shows that text- based reasoning mode struggles to locate the target object (especially when it is small, this limitation example, the
also evident in V*). Moreover, text-based reasoning can lead to overthinking- misunderstanding of sandwich in the fourth case, and the introduction of irrelevant information and a repetitive generation loop in the fifth case. In contrast, the grounded mode thoroughly checks the objects in the image and provides correct answers.
MMStar As a benchmark including samples across different scenarios, We find that in many scenarios, both reasoning modes can arrive the correct answer, so Figure 14 we present several examples where the two modes differ. According to the first three examples, we can see that the text- based reasoning mode is better suited to tasks requiring complex abstract logic and knowledge-foo instance, the folding problem in the first example and the painting-style knowledge in the second. The third example shows that the grounded mode tends to leverage information visible in the image, such the colorless liquid, but cannot further infer additional information based "acetone". The last two cases show that the grounded mode better at capturing on attributes of specific objects and making comparisons between target objects.
SpatialScore Focusing on spatial reasoning in general scenes, Figure 15 provides several exam- ples. The first case relatively common one from a third-person perspective (camera viewpoint). Both reasoning modes answer correctly, and the grounded mode further indicates the specific lo- cations of the objects. The second case further requires reasoning about spatial relations from the first-person perspective of a specific object, which calls for some imagination. Thus, although the grounded mode correctly locates the corresponding object, it lacks further imaginative reasoning and does not answer correctly as the textual mode does. In addition, the third case reveals current limitation of the grounded mode in multi-image scenarios: it cannot distinguish between different images using coordinates, which is direction for future improvement. The fourth and fifth cases illustrate the grounded mode's s advantage on more conventional spatial reasoning problems: by lo- cating key objects and combining common sense, both ultimately yield correct reasoning outcomes.
Extra Qualitative Results on MMStar We provide two additional qualitative examples from the common-sens, reasoning subset of MMStar Figure [6] These cases require diverse focus: not only grounding relevant objects in the image but also performing abstract common-sense reasoning. Rather than relying on single mode, our model dynamically selects the most suitable mode for each question.
30

---

Published as a conference paper at ICLR 2026
MathVista General
Question: donut than half caten? Choices: (A) yes; (B)
TXT Mode
GRD Mode
<think>To determine the donut more than
eaten, need visually assess the portion of the denut that bas bocn consumed.
<think> Find the donut [0, 116, 392, 397]. Check ih donut
donut appeurs have significant portion more than half caten. The visible part of the donut shows large gap, small
missing, suggesting
116, 392, 397] more than catem. The <tthink>
small portion. <ithink> <answer> As/unswer-
indicating that
<answer> <Janswer>
MathVista Diagram
Trephic livil
TXT Mode
GRD Mode
Kiler wiilee
<hink-Too determine which organism would most affected if there was shortage plants, need look at the food chain depicted in the image. Plants are at the base the temestrial food chain, they consumed by grasshoppers. If there is shortuge of plants, the primary comsumers, which the grasshoppers, would be most affected because they rely directly on plants options provided (A)
<think> Find the plant [252, 328, Chock the organism that it connected to, got the organism 644, 422, 794]. The grasshopper [244, 644, 422, 794] connected plant [252, 900, 328, 966), So answer grasshopper. </think> <answer> </answer>
Smale
Harring
Zispanntaan
Grasshupper Snake Mouse (D) Hawk Since the grassbopper is dircctly dependent
lamrsiiall fesd shate
aquatoo fiod chate
plants for food. shortage plants would most directly affect the grasshopper <ithikk> <answer> AS/answer>
Question: Which organism would be most affected there was shortage of plants?
Choices: (A) Grasshopper; (IB Snake; (C)
Mouse; (D) Hawk
MathVista Logic
Question:
Choose the answer for the missing picture. (GT Answer: A)
TXT Mode
<think> To solve this, lot's observe the pattern in the given images. Each image is an eight equal parts, different color. The question asks for the missing picture that continues this pattem.
octagon divided
color pattern alternaess between yellow
The first row has pattern where the yellow part alternates with a different color, now follows same pattern.
The third now should continue this alternating pattem. Looking at the options: A has a yellow part the top left. hus yellow part in the top left. has yellow part in the top left. has yellow yellow part in the top left. has yellow part in the top left.
in top left. has
CHOOSE ANSKER
Since the pattern consistent the question asks the missing picture, correct
should maintain the alternating pattern. </tink> canswer Asanswwer
GRD Mode
puttern missing picture. missing picture answer
<>thnk>> <answer> </answer>
Math Vista Geometry
TXT Mode
To the length of hypotenuse trigonometric relationships. Given that one of the angles 60re 1) and the adjacent side this angle is 4, can use the cosine function. which relates the adjacent side the hypotenuse right triangle. The cosine an angle defined as the ratio the length of the adjacent side to the length of the hyposcmuse. Therefore, we have: [[ (oos(60"ccir)) ffre
right triangle,
h
60
Vfrec (reettefajaaceeeee (ttxtt (hypttenne) 0.51). So
We know "Ccos(60 To solve for rearnang the equation: {[ h ffae (0.5) =8 <hhinks canswer-D</anssee
Question:
the equation: 0.5 trac
Find ShS the triangle.
Choices: (A) 4.62
GRD Mode
triangle, the side opposite 60' angle shorter leg, which given as 4. The hypotemuse is labeled ft', Using the properties of 30-60-90 triangle, the ratio sides opposite 30", 60', 90" angles I 13: Therefore, side opposite 60' angle (which 4) half hypotenuse. This relationship expressed as l/2. Solving for 'f' gives à 4 </tink>> <answer> D </answer>
<think>
(B) 5,66
(C) 6.93
(D)
Figure 11: Cases from Math Vista. The pointing- finger icon indicates the mode selected by AdaVal
31

---

Published as a conference paper at ICLR 2026
MathVision
TXT Mode
GRD Mode
-think-Firsst we start with the number 48. We subtract from
Then 28: 28 37. Next, we subtract from
<hinkk> Find the number that more than 22. The answer
Question:
37: 37-6- 6- 31. Finally, we add 30 31; 31 So, number
Which number do you have to write in the last daisy?
52. <ithink> <answer> <lanswer
in the last daisy is 61 </think
GT Answer: 61
<answer-61 1isseer-1--aasseerr klanswer>>
MathVision
TXT Mode
Mode
Sthink-Sinnee AB BC, angle
<thinks To find the value of we to use properties parallel lines and the
124
angle
angles in triangle is 180
angles. I. We find that ([ ('triangle ABC isosceles with BC 1).
degrees. Therefore, angle BAC
angle angle ABC 180
degrees. Given angle BAC 124
angles langll BAC \) and langle ACB !) are equal. 2. The sum angles angle property: I 124"scire + + 180% Y Mf 124% 56% 1
Question:
degrees, we substitute and
(BverlineeBC]S is parallel to the segment through SAS, and SAB-
solve for angle ACB, 124
angle ACB angle ACB - 180.
BCS. What is the number of degrees represented by $x$?
Simplifying. angle
180% 1)
Therefore, angle ACB 28
Please provide your final answer using a single number.
degrees. Since angle ACD x,
56/heiire / 56% 2 56
dogrees, */think> <answer-26 <answer-2B-/answeer
56" /22 56rc
5e'"eire... (the model get stuck in loop)
Answer: 28
MathVerse
TXT
GRD Mode
Nasttagler are CSC 0.22
<think--innee the triangles are similar, ratios correpoondng
<think Since the trinngles similar, the ratios of corresponding sides are equal. Therefore, we can set up the proportion 245
equal. Therefore, we set up the proportion v "2xe6 B.). Solving this
W
30
10
propurtion give
Solving this proportion for (X)), we get (Xx 6). Substituting VX
(xi). Cross-multiplingg gives: 10(x
Question:
6) 8(2x 63 Expunding both
into V2x +6),, find (2(6)
According to the question shown in the image, please first conduct reasoning, and then answer the question und provide the correct option letter, e.g., A, B, C, at the end.
sides:
181). Therefore, (WR 181).
Rearnnging terms to isolate ((0)
However, question asks
If
16x 10x ( 6x
(WR)) which is Vxx 6i).
I[ 2 Now, substituting
29)
Subatituting 1(x 6) into ((x 61),
into ((x 6)): ([ + 22 6 6-8
Therefore, correct answer is D.~/tinks>
we get
Therefore,
8i)-ithinkss
GT Answer:
<answer- Canswer-Be/answee B'/answer>
<answer> Dasanwe-
WeMath
cm
Question: As shown in the diagram, which the following line segments is perpendicular AB?
5
4
Choices: A. AB; B. AC; C. No correct GT Answer::
TXT Mode
GRD Mode
<think> In the diagram, given that C right angle, which means that AC perpendicular to AB. Therefore,
<thinks Find the line segment that is perpendicular to According diugram, perpendicular to AR. Therefore, the answer is B. </think> Sunswer'> <lanswer>
scgment that is perpendicular AB is AC.//hink>> sanswer- BBtaerser>///>>>>
Figure 12: Cases from other math-oriented benchmarks.
32

---

Published as a conference paper at ICLR 2026
V*-Visual Search
Question: Is the silver vehicle on the left right side of the red vehicle? (A) right (B) left
GT Answer:
TXT Mode
GRD Mode
<think- The image shows parking lot with vehicles. The silverr vehicle on the the red vehicle, <ithink>
<think>> Find red vehicle
nin
[630, 22, 706, 50). Think the car the left of
86, 128] to the right of the vehicle
(630, 22, 706, 50).
left, </think>
canswer Ba/answer>>
<answer? </answee>
V*-Visual Search
Mode
GRD Mode
<hinkk image walking dog on poth. The bicycle visible the image, but hased on the question, it implied that dog on the side the bicycle if it were present.
person
<thinkc> Find bicyale [238, 796. 302, 862). Think the dog 1862,
900, 1144| is to the
However, since question about the dog's position relative the bicycle, and the image does not infer
left of the bicycle
[238,
862), The answer
show bicycle, based on the question's context. right of the image, as it is walking away viewer right side of the image. <ithinks- Ganswer-B<lansweer B-daanwwr>>
right. <ithink>
sanswers A
dog clearly
stanswer
Question: Is the dog on the left or right side of the bicycle? (A) right (I) left
GT Answer:
POPE-Hallucination
Question: Is there in the image? GT Auswer: Yes
TXT Mode
GRD Mode
the image. The image shows red double-docker bus with people top, and the background includes street and vehicles cars visible in the image. <ithink> <unswer-no-stanwwe
<think- There
<think Find the car 258). It exists, The answer is yes. sanswer> <lanswer-
buildings.
ARROZ
POPE-Hallucination
Question: there sandwich image?
GT Answer: Yes
TXT Mode
GRD Mode
<think- There dog in the image, which type of sandwich. However, the question asks if there is sandwich in the image. The image shows a hot dog with toppings, hut is not sandwich the traditional sandwich typically made with
<think> Find the sandwich 280, 460), It exists. answer yes. <ithinks <answer yes
slices of bread, and the hot dog single item. Therefore, the image does not contain a sandwich traditional sense. <ithnk- <answerrnor/mmwwr>
PollHHallucination
TXT Mode
GRD Mode
image. image photograph drinking from with orange slices on table. The question is asking image. image
<think-Theree
<hinkk
doesn't exist, The question ask about 'is' relation, there no car. So the answer no. withink> sanswerr </unswert-
about the presence is photograph of child drinking from cup with orange slices a table. The question asking about presence car, which not in image. The image is photograph of child drinking from cup with orange slices on a table ...dmeeel gess stuck loop)
not present
Question: Is there a car image? GT Auswer: No
Figure 13: Cases from V and POPE. Please zoom in for better view of small objects in images.
33

---

Published as a conference paper at ICLR 2026
MMStar-Puzzle
Question: Which cube identical to the unfolded net? GT Answer: D
TXT Mode
2 3 4
options find one that matches it The unfolded net has the following sequence of oumbers: I, 3,4, Now, examine each option: A. This cube has the sequenice 3, 5. 3, which does match the unfolded not. B. This cube has the sequence 4, which does not the unfolded
<think-LLet's unulyze unfolded
<think> Find the
unfolded net (118, 12, 303, 257].
5
Chock the cube that has the pattorn, got the 349, 269, 480). The cube [151, 349, 269, 480j is So
the sequence which does not match the unfolded net. D. This cube has the sequenee 2, which does not match unfolded net. This cube has the sequence net. None of the options match unfolded net exactly. However, unfolded net, seems be a sequence of numbers that might be repeated or rearranged. 4, which
which does not match the unfolded
closely
sequence
the answer is </think>
correct sequence unfolded corresponds option D. </think> <answer=D<lanswer.
<answer>
clanswer>
MMStar-Image Style
Question: Identify the art style image?
Options: oil paint,
art, C: Baroque, watercolor. GT Answer: Yes
TXT Mode
GRD Mode
<think> The image appears to have soft, blended quality that
characteristic of watercolor painting. The colors are not sharp and the brushstrakes visible, which typical watercolor techniques. The vibrant colors and the way the light and shadow are depicted also suggest watercolor style. paint would vector art would be digital representation with clean lines and no blonding. and Baroque art would have a more detailed and dynamic style with focus
<think> It painting. So the unswer oil paint. <itink-
more opaque and
look,
canswer>
and <ihink> <unswer-DD<insweer
<lanswer>
MMStar-OCR & Knowledge
Question: The object shown in this figure? Options: A: Is colorless liquid with sweet, fruity odor, B: Has boiling point of 56.05'C, C: Is used as solvent for many organic compounds.
TXT Mode
GRD Mode
<think> The image shows a bottle labeled "Acetene," which is common
<think Find bottle [192, 70, 399, 364]
various applications. Acetone known sweet, fruity and is used solvent for many organic compounds. has boiling point
The bottle [192, 70, 399, 364] is coleriess liquid [192, 70, 399,
56.05"C. Therefore, the correct answer should reflect these properties. A: colorless liquid with boiling point of 56.05"C- This also trac for secionc. C: Is used solvent many organic compounds
fruity oder
for
Has
364].
acetone. None are correct - This incorrect because all the options correct for acctone. </hink>> <answerr <answer-C</answee Caanswer>>
options
colorless liquid.
<ithinks <answer>
<lanswer>
MMStar-Object Attribute
Question: What the primary color of the football in the image?
Options: A: Marcon, B: Red, C: Brown, D: Oranges
Mode
TXT Mode
<thinic> Find
[364,
317, 390. 362]. The football [364, 317, 390, 362] brown.
<think> The image shows football the hands player. The foothall clearly visible and distinct. The color foothall in the image is marcon, <itinks> A s/nnswer>
So the is brown.
</tinkk> <inswer> <lanswerr
new robacco
MMStar-Cross Instance Reasoning
Question: Which object bigger, the Christmas tree or the hotel lobby?
Options: A: They are about the same size, Hotel lobby, C: Christmas tree
TXT Mode
GRD Mode
<think> The image shows large Christmas tree the foreground, which appears to be background. The hotel lobby is room with concierge desk and sign, which is soallee in comparison the Christmas tree. Therefore. the Christraas tree is bigger than the hotel lobby <ithnkk <answer Sanswer<C</nsswerr Cc/answer>
<thinks the Christmas tree (19, 10, 224, 443), The question ask which one is bigger, the hotel lobby l0, 0, 335, 224, 443), botel lobby [0, 0, 335, 499] is bigger than the [19,
lobby [0, 335, 499).
taller than hotel lobby
Christmas
443). So the
botel lobby, <ithink> <answor> B </answor>
Figure 14: Cases from MMStar.
34

---

Published as a conference paper at ICLR 2026
SpatialScore-Spatia Relation (third person)
Question: Where is the blind located with respect coffee table? Sclect from the following choices. (A) left (B) right
TXT Mode
GRD Mode
<think> Find the coffee
"think- The blind located the right side the image, as it is on the right wall of the room. The coffee table is on the side of the image. Therefore, the blind the right the coffee table,<tthink <ithink> BemmwerrBB//nswer
table [186, 566, 552, 794].
Think the blind [109, 252, the right
or to the left of the coffee
[186,
The answer is right. <ithink--
sllaswwrr
DpatialScore-Spatiee Relation (first person)
Question: Ifl stand person's 's position facing where facing, is the printer the left right me? Options: on the left: (B) right
TXT Mode
GRD Mode
<think The image shows person sitling desk with laptop and some papers. printer is typically located on the right side of the desk when facing person. Therefore. you were standing at the person's position facing where it facing, printer would be right you. <hthink> "answer- B <lanswer-
<think-- Find person [312, 107,
537, 496). Think the printer (257,
186, 367. 276] is to the left of
the right of the person [312, 107.
537, 496]. The
left.
<ithink> <answer>> A <lanswer*>
SpatialScore- Cross Image Reference
Question: point is circled the image, labeled with REF change camera position lighting and shoot second image, Which point on the second image corresponds to the point in the first image?
Options: A. Point A; B. Point B; C. Point C
TXT Mode
GRD Mode
<think- The question asks for the point on the second image that corresponds to the point "REF" in the first image is located in the lower right section of the image. In the second image. the point labeled "REF' conresponds to "REP" the secand image Point C. <ithink> Ceasseerr-C/anseerrrr
<think- Find the point in the first image, labeled with REF, Think the point in the secoed image
image. Observing images, point labeled "REF"
lower right
Therefore, point that
corresponds to it. The answer </think> <answer- B <lanswer>
SpatialScore-Spatial Relation (3D world)
TXT Mode
GRD Mode
<think The question asking for the locasion of the objects "Lous
<think Find the shop plaque 1134, 257, 300. 305]. Find the sign [228, 102, 588, 240).
Vuitton" shop plaque and "VIA RODEO' sign. In image, the "Lous Vuitten" shop plaque located the building, and the
RODEO
RODDODD DR
"VIA RODEO' sign on pole in front the building. Since question asks for the higher location, need to compare the positions of these two objects. The "Lous Vaitton" shop plaque location, and "VIA RODEO" sign is on pole, which is also a 3D location. However, relative heights positions 2D sense, which not directly applicable here. The question is representation of the sign. Since the question is about the sign in the image, and sign
The question ask the object that has higher location. Compare the
the building, which
locations of the objects, the shop plaque 257, 300, 305] is at higher location than the [228, 102, 588, 240|- So the plaque. <ithink>
Question:
Consider
question implics comparison
real-world locations of the objects. Which object has higher location?
likely asking for the sign the image, which
compare signs image. The "VIA RODEO" sign 20 representation of the sign in image, and
representation,
shop
A "I "Lous Vuitton" plaque
"VIA RODEO" sign
the "Lous Vuitton" shop plaque in the image, <ithink SanswarrBB)
SpatialScore-Objett Facing
Question: Which side plain the grey facing? Options: (A) the right side; (B) the left side
TXT Mode
GRD Mode
<think> The image shows airplane with its nose facing left
sthink> Find the grey truck [440,
inage. The grey trick is positioned to the right of the airplane.
497, 360). Think which side the airplane (76, 154, 590, 322] is the truck [440, 333, 497, 360] facing. The answer
Therefore, truck facing right
the airplanncc <ithink sanswer> <lanswrr>
is left. <ithink> canswer- <lanswer-
Figure 15: Cases from SpatialScore.
35

---

(51, Published as a conference paper at ICLR 2026
MM Common Sense
Question: Where
holding her belly image? Cholces: With both hands; With her right hand, C: With her left hand
TXT Mode
GRD Mode
<think> The image shows a woman holding apple her right hand her belly in her left hand. This indicates that she holding her belly with her left band <ithin? </unswer>
<think> Find the woman (26, 12, 280, 471j. Find object holding her belly, got
301, 255, 465j. Check the hand her left hand using right holding her belly. So the answer D. <ithink>
right hand,
<answer>
MMStar- Common Sense
TXT Mode
GRD Mode
image shows facing each other, which are both prodators
tiger
<thinks Find the two creatures, 161, 223] and tiger
respective ecosystems. They are known to competc for the same resources, such as food territory, which is competitive relationship. Parusitic relationhips involve one organism beaefiting the expense the other, which not depicted here. Symbiotic relationships can be muttullitii,, commensalistic, parasitic, the image does not show any of these. Tereffrre, relationship between creatures in nature is compelitive <itink>> Bnewee-B</answee
[180, 28, 223]. The relationship between two creatures [0, 50, 161.
223] and [180, 28, 419, 223] competitiee. So
Question: In nature, what's
the answer is competitive. </think>
relationship between these creatures?
Choices: (A) Predatory relationships;
<answor> B </answer>
(B) Competitive relationships:
(C) Parasitic relationships;
(D) Symbiotic relationships.
Figure 16: Cases from the common sense reasoning task in MMStar.
36


Reviews:
[{"review_id": "JdXKTfUoDH", "reviewer": "Reviewer_kCu9", "summary": "This paper proposes to augment reasoning models by allowing them to adaptively select  their reasoning patterns. In particular, it first prompts the models to choose between two reasoning modes, either focusing more on grounding objects or pure textual reasoning. A modified version of GRPO algorithm is then developed to train the models for mode selection, which splits the general rollouts and rewards into two types based on the reasoning modes. The method shows strengths in several types of reasoning scenarios, and experimental results validate its usefulness in selecting the right reasoning patterns.", "questions": "(1) Are there other important reasoning modes? How can the proposed method incorporate more diverse reasoning modes?\n\n(2) Please consider additional experiments on the effects of prompt design (related to questions above). It would also be useful to apply the same prompts on existing state-of-the-art.\n\n(3) Is the proportion of grounding mode sufficient to evaluate mode selection? Please justify.\n\n(4) How would the model choose between the two modes when answering visual questions with diverse focuses?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "(1) It is an intuitive choice to adopt diverse reasoning modes for solving different problems, the proposed method demonstrates the advantages of explicit mode selection.\n\n(2) The proposed method demonstrates consistent improvement across multiple datasets.\n\n(3) The paper provides extensive analysis on the model behavior during training and contribution of individual components.", "weaknesses": "(1) One concern I have is about the potentially limited scalability of the method. Specifically, solving real-life reasoning problems requires a collection of skills, while the method simplifies reasoning into textual and visual (grounding-based) reasoning. It is unclear how it can scale to larger domains in the current format, i.e., with the prompt listing all possible reasoning modes and explicit data/reward splitting in the training algorithm.\n\n(2) Related to the previous comment. How sensitive is the method to prompt design? Does prompting with more diverse reasoning modes work? What is the level of details needed to specify each reasoning mode? Does the prompt work for other models (in addition to Qwen2.5-VL)?\n\n(3) The paper evaluates mode selection based on two relatively extreme scenarios, i.e., math vs general, and largely ignores the variety of reasoning types in visual questions (e.g., probing the physical properties of multiple objects, comparing multiple objects, etc). It would be reasonable to further evaluate mode selection based on question types (e.g., on benchmarks with annotations like GQA).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "r1Pd9RmCgF", "reviewer": "Reviewer_n9fj", "summary": "This paper addresses the limitation of existing visual reasoning models that focus on single reasoning modes and lack generalizability. It proposes a novel adaptive visual reasoning paradigm called Mixture-of-Visual-Thoughts (MoVT), which unifies two common reasoning modes (text-based and visually-grounded) into a single model and enables context-adaptive mode selection. To implement MoVT, the authors design the AdaVaR framework with a two-stage training process: a supervised cold-start stage (SFT) to unify and learn multiple reasoning modes, and a reinforcement learning (RL) stage using the proposed AdaGRPO algorithm to induce mode selection capability. Extensive experiments on 8 benchmarks (covering math-oriented and general scenarios) show that AdaVaR models (AdaVaR-3B and AdaVaR-7B) achieve consistent performance improvements, with AdaVaR-7B surpassing GPT-4o in average accuracy. The paper's main contributions include: proposing the MoVT paradigm for general visual reasoning, developing the two-stage AdaVaR framework, and validating the effectiveness of AdaGRPO and the overall method through experiments.", "questions": "*To facilitate discussions during the Rebuttal phase, authors are advised to respond point-by-point (indicating the question number).*\n\n(1) For the 1:1 data mixing ratio in the SFT stage, could you provide ablation experimental results comparing different ratios (e.g., 2:1, 1:2, 3:1) and explain why 1:1 is optimal for mode unification and subsequent RL training?\n\n(2) Regarding the KL penalty coefficient of 0.04 in the RL stage, could you detail the tuning process (e.g., tested coefficient values, performance changes with different coefficients) and explain how you determined that 0.04 balances mode selection bias and reasoning accuracy?\n\n(3) For the mode-switching mechanism, could you provide a quantitative definition of \"getting stuck in repetitive logic\" (e.g., number of consecutive repeated tokens, threshold for logical loops) and present ablation results comparing model accuracy with and without this mechanism?\n\n(4) For reproduced results of baseline models (marked with \"*\"), could you provide detailed training settings (e.g., learning rate, batch size, number of epochs) and compare them with the original papers? If there were discrepancies between reproduced and original results, how did you resolve them?\n\n(5) In the MathVista-General case (Figure 7), the GRD mode incorrectly answered the donut question. Could you analyze the root cause (e.g., bounding box localization error, flawed visual reasoning) and provide evidence (e.g., localization results, intermediate reasoning steps)?\n\n(6) In the Ethics Statement, you claim to use open-source data from VoCoT (Li et al., 2025). Could you provide specific proof of authorization (e.g., VoCoT's license text, confirmation of compliance with terms) to address potential copyright concerns?\n\n(7) In the Reproducibility Statement, you mention open-sourcing code, models, and data after review. Could you provide a clear timeline (e.g., date of release after acceptance) and a temporary link to sample data or a preprint repository to allow reviewers to verify key implementations?\n\n(8) Regarding AdaGRPO's convergence, could you supplement theoretical analysis (e.g., proof of convergence to a stable policy) or empirical evidence (e.g., convergence curves of policy loss over training steps) to confirm the algorithm's stability?\n\n(9) For extending MoVT to more reasoning modes (e.g., direct answering), could you provide preliminary experimental results (e.g., performance after adding a <direct-answer> mode) to support the claim of scalability?\n\n(10) Could you compare the training cost (e.g., GPU hours, energy consumption) of AdaVaR with other models (e.g., GPT-4o, Qwen2.5-VL-7B) and explain whether the performance improvement justifies the increased computational cost?\n\n(11) Could you measure the inference latency of AdaVaR (including mode selection time) and compare it with single-mode models? Additionally, do you have optimization strategies (e.g., early mode selection) to reduce latency for edge device applications?\n\n(12) For the RL data filtering threshold of 8 (Qwen2.5-VL-7B answering correctly in 8 random samples), could you conduct a sensitivity analysis (e.g., testing thresholds 5, 10) and analyze how different thresholds affect data distribution and model performance?\n\n(13) For SFT data rewritten by GPT-4o into multiple-choice format, could you provide results of manual quality assessment (e.g., error rate of options, bias in difficulty) and compare model performance trained on original vs. rewritten data?\n\n(14) Could you supplement statistical significance analysis for experimental results (e.g., p-values via t-test, confidence intervals) to confirm that AdaVaR's performance improvements over baselines are not due to random fluctuations?\n\n(15) For the 100% GRD% in POPE, could you analyze the dataset distribution (e.g., proportion of tasks requiring object localization) and conduct experiments on a modified POPE subset (with non-localization tasks) to verify whether the preference is context-adaptive or dataset-specific?\n\n(16) Regarding AdaVaR-7B's greater preference for text-based mode in knowledge-related categories, could you provide more data (e.g., GRD% across more knowledge categories) and analyze whether this is due to the 7B model's richer pre-trained knowledge or differences in the RL process?\n\n(17) Could you supplement an ablation experiment on \"mode-specific prefixes\" (comparing performance with and without prefixes) to confirm their necessity for mode differentiation?\n\n(18) Could you detail the processing of multi-modal data (images+text questions) in SFT and RL stages (e.g., image preprocessing steps, visual-text fusion methods) to clarify the multi-modal integration process?\n\n(19) Could you supplement experiments on model robustness (e.g., testing with noisy/occluded images) and compare AdaVaR's robustness with single-mode models to assess its practicality in real-world scenarios?\n\n(20) Do you plan to conduct user studies to evaluate the readability of reasoning processes and their impact on trust in model answers? If so, could you outline the study design (e.g., number of participants, evaluation metrics)?\n\n(21) Could you correct the formatting errors in the reference list (e.g., completing OpenAI (2025)'s title and authors, abbreviating long author lists) to improve professionalism and ease of verification?\n\n(22) Could you correct the typo in AdaGRPO's optimization objective formula and provide a clear derivation to avoid misleading readers during implementation?\n\n(23) In Algorithm 1, could you specify the order of rewards in \\(r[1..2n]\\) (e.g., \\(r[1..n]\\) for text-based rollouts) to ensure correct implementation of advantage calculation?\n\n(24) Could you confirm the exact version of the Qwen2.5-VL base model (e.g., Instruct vs. base) and verify that the pre-training data is consistent across all compared models to rule out base model advantages as a factor in performance gains?\n\n(25) Could you supplement experiments on low-resource conditions (e.g., training with 50% of original data) and compare AdaVaR's performance with single-mode models to demonstrate its data efficiency?\n\n(26) Could you clarify the weighting scheme for average accuracy across benchmarks (e.g., equal weighting, size-based weighting) and show how different schemes affect the average accuracy ranking of AdaVaR and baselines?\n\n(27) For the SFT stage's 1 training epoch, could you provide the training loss curve and ablation results with more epochs (e.g., 2, 3) to confirm that 1 epoch is sufficient for convergence and avoids overfitting?\n\n(28) For the RL stage's 8 rollouts per sample, could you conduct an ablation experiment (e.g., testing 4, 16 rollouts) and explain how the number of rollouts affects mode comparison accuracy and computational cost?\n\n(29) Could you provide complete training dynamics (e.g., accuracy, reward, GRD%) for all benchmark categories (e.g., POPE, V*) to allow a comprehensive understanding of mode selection learning across tasks?\n\n(30) For qualitative analysis cases (e.g., Figure 7: MathVista-Geometry), could you supplement clear visual details of input images (e.g., labeled side lengths of the triangle) to allow reviewers to verify the correctness of the model's reasoning?\n\n(31) Could you clarify the relationship between \"reasoning mode\" and \"CoT\" (e.g., whether modes are CoT variants) to avoid confusion with existing CoT literature?\n\n(32) Could you supplement zero-shot generalization experiments on unseen tasks (e.g., video-based reasoning) to validate AdaVaR's claim of \"general visual reasoning\"?\n\n(33) For curriculum learning, could you conduct an ablation experiment on data order (e.g., training on diverse mixture first) to confirm that the \"easy-to-hard\" order is optimal?\n\n(34) Could you visualize the mode selection decision process (e.g., attention weights on mode prefixes, feature importance of question/image) to make the mechanism more transparent?\n\n(35) Could you test AdaVaR's performance across different image types (e.g., natural images, diagrams, documents) and analyze whether mode selection adapts appropriately to each type?\n\n(36) Could you control the length of reasoning processes across modes (e.g., enforcing similar lengths) and re-evaluate performance to rule out reasoning length as a confounding factor?\n\n(37) Could you test smaller models (e.g., AdaVaR-1B) and analyze the relationship between model size and adaptive reasoning performance to clarify the method's scalability?\n\n(38) Do you have plans to extend AdaVaR to cross-lingual visual reasoning? If so, could you outline preliminary steps (e.g., training data preparation, mode selection adaptation for non-English questions)?\n\n(39) Could you supplement long-term training experiments (e.g., 2x the original training steps) to show whether AdaVaR avoids mode collapse and maintains stable performance?\n\n(40) Could you provide a systematic error analysis (e.g., breakdown of errors by mode, task type) to identify AdaVaR's current limitations and directions for improvement?", "limitations": "", "rating": 4, "confidence": 5, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "(1) The proposed MoVT paradigm addresses a critical gap in existing visual reasoning research by integrating complementary strengths of text-based and visually-grounded modes, breaking the domain limitation of single-mode models and providing a feasible solution for building general visual reasoning models.\n\n(2) The two-stage AdaVaR framework (SFT+RL) is practically designed: the SFT stage solves mode unification via a uniform sequence format with mode-specific prefixes, while the RL stage (with AdaGRPO) targets mode selection, ensuring the model both learns multiple modes and adapts to context.\n\n(3) The experimental design is relatively comprehensive: the paper evaluates on diverse benchmarks (math, visual search, hallucination control, etc.), compares with multiple single-mode baselines, and conducts ablation studies on key components (Ada-Adv, PG-Exp), providing sufficient evidence for the method's effectiveness.", "weaknesses": "(1) In the SFT stage of AdaVaR, the paper sets the data mixing ratio of text-based and visually-grounded reasoning to 1:1 but provides no justification for this ratio. It neither explains why 1:1 is superior to other ratios (e.g., 2:1, 1:2) nor conducts ablation experiments to verify whether different ratios affect mode learning or subsequent RL-based mode selection. This unsubstantiated setting makes it unclear whether the ratio is optimal or merely an empirical choice, reducing the methodological rigor.\n\n(2) The paper mentions that the KL penalty term is omitted in the AdaGRPO formula (Section 3.3.1) but uses a KL penalty with a coefficient of 0.04 in training (Appendix A.3). However, it fails to explain the basis for choosing the 0.04 coefficient—for example, whether it was determined through experimental tuning (e.g., testing coefficients like 0.01, 0.02, 0.05) or based on prior work, and how different KL coefficients influence the model's mode selection bias and reasoning accuracy. The lack of this information impairs the reproducibility of the algorithm.\n\n(3) The unified reasoning format uses <text> and <ground> as mode prefixes (Section 3.1), but the paper does not explore the impact of prefix design on model performance. It neither tests whether longer prefixes (e.g., <text-based-reasoning>) or more distinguishable prefixes (e.g., <txt> vs. <grd>) affect the model's ability to distinguish modes nor verifies whether prefixes introduce unintended inductive biases (e.g., <ground> being more salient than <text>, leading the model to prefer it). This oversight leaves potential factors influencing mode selection unaddressed.\n\n(4) In the evaluation benchmarks (Section 4.1), the paper omits several widely used visual reasoning benchmarks, such as CLEVR (for compositional visual reasoning) and GQA (for structured visual question answering). Excluding these benchmarks makes it impossible to evaluate the model's performance on compositional or structured reasoning tasks, limiting the generalizability of its claim to \"general visual reasoning.\"\n\n(5) For some comparison models (e.g., Qwen2.5-VL-3B, LMM-R1), the paper marks results with \"*\" indicating reproduction but provides no details on the reproduction process. It does not specify whether the reproduction strictly followed the original paper's training settings (e.g., learning rate, batch size), whether there were discrepancies between reproduced results and original results, or how such discrepancies were resolved. This opacity raises concerns about the fairness of comparisons between AdaVaR and these baseline models.\n\n(6) The mode-switching mechanism (Section 4.1) is described as \"switching to another mode if the model gets stuck in repetitive logic,\" but the paper provides no quantitative definition of \"getting stuck in repetitive logic.\" It neither clarifies how many consecutive repeated tokens or logical loops constitute \"being stuck\" (e.g., 3 repetitions vs. 5 repetitions) nor verifies through ablation experiments whether this mechanism actually improves performance (e.g., comparing accuracy with and without mode switching). The lack of clear criteria makes the mechanism vague and difficult to reproduce.\n\n(7) In qualitative analysis cases (e.g., Figure 7: MathVista-General), the visually-grounded (GRD) mode incorrectly answers the donut question (\"No\") while the text-based (TXT) mode answers correctly, but the paper does not analyze the root cause of the GRD mode's error. It fails to clarify whether the error stems from incorrect bounding box localization, flaws in visual-information-based reasoning, or mode selection bias, weakening the depth of the qualitative analysis.\n\n(8) The Ethics Statement claims all data sources are open-source but provides no specific proof of authorization for using VoCoT (Li et al., 2025) data. It neither cites VoCoT's license type (e.g., MIT, CC BY-NC) nor confirms that data reuse complies with VoCoT's terms of use. This omission raises concerns about potential copyright issues with the data.\n\n(9) The Reproducibility Statement mentions that code, models, and data will be open-sourced after review, but the current version provides no preprint link, temporary repository, or sample data. This prevents reviewers from verifying key implementations (e.g., advantage calculation in AdaGRPO) or preliminary results, hindering the assessment of reproducibility.\n\n(10) The paper provides no theoretical analysis of AdaGRPO's convergence. It does not prove whether the algorithm can converge to a stable policy, how the three key components (e.g., prefix-guided exploration) affect convergence speed, or under what conditions the algorithm can avoid mode collapse (e.g., always selecting one mode). The lack of theoretical guarantees makes the algorithm's stability during long-term training questionable.\n\n(11) The paper lists extending MoVT to more reasoning modes (e.g., direct answering) as a future direction (Section B.4) but provides no preliminary experimental evidence. It neither tests whether AdaVaR can still learn mode selection after adding a <direct-answer> mode nor analyzes whether increasing the number of modes exacerbates exploration difficulty or reduces performance. This leaves MoVT's claimed scalability unsupported.\n\n(12) The paper does not discuss the model's training efficiency. For example, AdaVaR-7B is trained using 32 NVIDIA A100 GPUs for 29 hours (Table 4), but it does not compare this training cost (e.g., GPU hours, energy consumption) with other models (e.g., GPT-4o, Qwen2.5-VL-7B). This omission makes it impossible to evaluate whether AdaVaR's performance improvements are worth the increased computational cost, limiting its practical application value.\n\n(13) The paper does not mention the inference latency introduced by adaptive mode selection. It neither measures whether the mode selection process (generating prefixes, evaluating mode advantages) increases inference time compared to single-mode models nor discusses optimization strategies to reduce latency (e.g., early mode selection). This is a critical oversight for practical applications (e.g., edge devices with latency constraints).\n\n(14) In RL data filtering (Section 3.3.2), the paper removes \"easy questions that Qwen2.5-VL-7B can answer correctly in 8 random samples\" but provides no basis for choosing 8 as the threshold. It neither tests whether other thresholds (e.g., 5, 10) retain more useful data or reduce noise nor analyzes the distribution (e.g., task type, difficulty) of retained and removed data to confirm whether filtering improves data quality.\n\n(15) For SFT data, the paper uses GPT-4o to rewrite a subset of VoCoT data into multiple-choice format (Appendix A.2) but does not evaluate the quality of the rewritten data. It neither conducts manual verification to check whether GPT-4o introduces errors (e.g., incorrect options) or biases (e.g., overly simple options) nor compares model performance trained on original vs. rewritten data to verify the impact of rewriting.\n\n(16) The experimental results lack statistical significance analysis. When reporting accuracy values (e.g., Table 1), the paper provides no error bars, confidence intervals, or p-values. For example, it claims AdaVaR-7B's average accuracy surpasses GPT-4o (55.82 vs. 53.20) but does not confirm whether this difference is statistically significant (e.g., via t-test). This makes it impossible to determine whether performance improvements stem from the method itself or random fluctuations.\n\n(17) The paper reports GRD% (proportion of choosing the grounded mode) across datasets (Table 2) but does not analyze why the model strongly prefers the grounded mode in some datasets (e.g., POPE: 100% GRD%). It neither rules out overfitting to the POPE data distribution (e.g., all POPE tasks require object localization, leading the model to always select GRD) nor confirms whether this preference is true context adaptation rather than dataset-specific bias.\n\n(18) The paper notes that AdaVaR-7B (7B parameters) is more inclined to select the text-based mode for knowledge-related categories (e.g., geography) compared to AdaVaR-3B (3B parameters) (Section B.1) but does not analyze the root cause of this difference. It neither explores whether the 7B model's richer pre-trained knowledge reduces its reliance on visual grounding nor verifies whether the RL process for the 7B model is more likely to prioritize text-based reasoning, leaving the impact of model size on mode selection unaddressed.\n\n(19) The ablation experiment (Table 3) tests the impact of Ada-Adv, PG-Exp, diverse mixed data, and curriculum learning but omits ablation of the \"mode-specific prefix\" (a key component of mode unification in Section 3.1). It does not compare model performance with and without prefixes to confirm whether prefixes are essential for mode differentiation, weakening the completeness of ablation analysis.\n\n(20) The paper does not provide details on how multi-modal data (images+text questions) is processed in the SFT and RL stages. It neither clarifies whether images are preprocessed identically across modes (e.g., resolution, feature extraction) nor explains how the model fuses visual and textual information during mode-specific reasoning (e.g., whether visual features are weighted differently in GRD vs. TXT modes), leading to ambiguity in the multi-modal integration process.\n\n(21) The paper does not evaluate the model's robustness to image noise or occlusion. It neither tests whether AdaVaR's mode selection adapts appropriately when input images contain noise (e.g., Gaussian blur) or object occlusion (e.g., a partially covered donut) nor compares robustness with single-mode models. This is critical for assessing the model's practicality in real-world scenarios where images are often imperfect.\n\n(22) The paper does not conduct user studies to evaluate the readability or utility of generated reasoning processes. It neither assesses whether humans can easily understand the GRD mode's bounding box-integrated reasoning or the TXT mode's textual logic nor analyzes whether reasoning processes enhance trust in the model's answers (a key factor for real-world adoption), limiting the assessment of the method's practical value.\n\n(23) The reference list contains inconsistent formatting and incomplete entries. For example, OpenAI (2025) (Section 1.10) lacks authors and a full title (only \"Thinking with images\" and a URL), while some entries (e.g., DeepSeek-AI et al., 2025) have overly long author lists that could be abbreviated. This reduces the paper's professionalism and makes it difficult for readers to locate and verify cited works.\n\n(24) The optimization objective formula of AdaGRPO (Section 3.3.1, Equation in Section 3.3.1) contains a typo: the denominator repeatedly includes \\(\\pi_{\\theta_{old}}(o_{j}|i,q,o_{j,<t})\\), causing confusion about the correct formula. The paper does not correct this typo or provide a clear derivation, which may mislead readers and hinder implementation.\n\n(25) Algorithm 1 (Appendix A.3) for mode-relative advantage calculation uses \\(r[1..2n]\\) but does not specify the order of rewards (e.g., whether \\(r[1..n]\\) corresponds to text-based or visually-grounded rollouts). This ambiguity could lead to incorrect implementation of the advantage calculation, undermining reproducibility.\n\n(26) The paper initializes AdaVaR based on Qwen2.5-VL (Section 3.2) but does not specify the exact version (e.g., Qwen2.5-VL-Instruct vs. base Qwen2.5-VL) or confirm whether the pre-training data of Qwen2.5-VL is consistent across all compared models. If AdaVaR uses a more capable pre-trained base model (e.g., the Instruct version with better initial reasoning), performance gains could stem from the base model rather than the proposed framework, distorting the evaluation of the method's effectiveness.\n\n(27) The paper does not evaluate the model's performance under low-resource conditions. It neither tests how AdaVaR performs when training data is reduced (e.g., 50% of the original data) nor compares it with single-mode models to determine whether adaptive reasoning maintains its advantage in data-scarce scenarios. This limits the assessment of the method's data efficiency.\n\n(28) The paper calculates average accuracy across benchmarks (Table 1) but does not specify the weighting scheme. It neither states whether all benchmarks are weighted equally nor whether weighting is based on dataset size, nor analyzes how different weighting schemes affect average accuracy (e.g., if MathVista has more samples, size-based weighting could inflate the average). This ambiguity makes fair comparison of AdaVaR's overall performance with other models difficult.\n\n(29) The SFT stage uses 1 training epoch (Table 4), but the paper provides no justification for this choice. It neither tests whether more epochs (e.g., 2, 3) improve mode learning or cause overfitting nor analyzes the SFT stage's training loss curve to confirm that 1 epoch is sufficient for convergence.\n\n(30) The RL stage generates 8 rollouts per sample (4 text-based, 4 visually-grounded) (Appendix A.3), but the paper does not explain why 8 is chosen. It neither tests other rollout counts (e.g., 4, 16) to see if more rollouts improve mode comparison accuracy nor checks if fewer rollouts reduce computational cost without performance loss.\n\n(31) The paper provides training dynamics for math tasks (Figure 3) but only partial curves for other tasks (Figure 6). It does not present detailed training dynamics (e.g., accuracy, reward, GRD% over steps) for all benchmark categories (e.g., POPE, V*), making it difficult to fully understand how the model learns mode selection across different task types.\n\n(32) In qualitative analysis cases (e.g., Figure 7: MathVista-Geometry), the paper does not provide clear visual details of input images (e.g., the side lengths of the right triangle). Reviewers cannot verify whether the model's reasoning (e.g., using cosine or 30-60-90 triangle properties) is correct, weakening the credibility of qualitative results.\n\n(33) The paper uses \"reasoning mode\" and \"chain-of-thought (CoT)\" interchangeably (Section 1.10) but does not clearly define their relationship. It neither distinguishes whether a \"reasoning mode\" is a type of CoT nor clarifies if it is a broader concept, leading to confusion with existing CoT literature (e.g., whether MoVT's modes are compatible with other CoT variants).\n\n(34) The paper does not evaluate zero-shot generalization to unseen tasks. It neither tests AdaVaR on visual reasoning tasks not included in training (e.g., video-based reasoning, medical image reasoning) nor verifies whether it can adaptively select modes for new tasks, which is essential for validating the \"general visual reasoning\" claim.\n\n(35) The ablation experiment (Table 3) does not test the impact of data order in curriculum learning (e.g., training on diverse mixture first, then binary mixture). It does not confirm whether the \"easy-to-hard\" order is truly optimal or if other orders (e.g., hard-to-easy) yield better performance, limiting the analysis of curriculum learning's effectiveness.\n\n(36) The paper does not explain how the model makes mode selection decisions for individual samples. It neither provides a breakdown of the factors influencing mode selection (e.g., question complexity, image content richness) nor visualizes the decision process (e.g., attention weights on mode prefixes), making the mode selection mechanism opaque.\n\n(37) The paper does not discuss the model's adaptation to different image types (e.g., natural images, diagrams, documents). It neither tests whether AdaVaR adjusts mode selection appropriately for different image types (e.g., preferring TXT for diagrams, GRD for natural images) nor compares performance across image types, limiting the assessment of its generalizability across visual domains.\n\n(38) The paper does not control for the length of reasoning processes across modes. It neither ensures that reasoning processes in TXT and GRD modes have similar lengths (to avoid performance differences due to longer reasoning) nor analyzes how reasoning length affects accuracy, leading to potential confounding factors in performance comparisons.\n\n(39) The paper does not explore the trade-off between model size and performance. It neither tests whether smaller models (e.g., AdaVaR-1B) can still benefit from adaptive reasoning nor analyzes whether the performance gain of AdaVaR-7B over AdaVaR-3B is proportional to the increase in parameters, limiting the understanding of the method's scalability with model size.\n\n(40) The paper does not address the model's performance in cross-lingual visual reasoning. It only focuses on English scenarios (Ethics Statement) and does not test whether AdaVaR can adapt mode selection for non-English questions (e.g., Chinese, Spanish) nor compare cross-lingual performance with single-mode models, limiting its applicability in multilingual contexts.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "AtOBBlfWu8", "reviewer": "Reviewer_NdeY", "summary": "This paper proposes a novel paradigm, Mixture-of-Visual-Thoughts (MoVT), and a corresponding two-stage training framework, AdaVaR, to build a general-purpose visual reasoning model. The core innovation lies in unifying different reasoning modes,specifically, text-based reasoning and visually-grounded reasoning within a single model and enabling it to adaptively select the appropriate mode based on the input context.Extensive experiments demonstrate that models trained with AdaVaR achieve consistent and superior performance across diverse benchmarks (mathematical, spatial, object-centric), outperforming specialized single-mode models and even surpassing GPT-4o on average accuracy.", "questions": "See weakness.", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1.The paper is well written and easy to follow\n2.The paper introduces the  \"Mixture-of-Visual-Thoughts\" paradigm, which systematically integrates different visual reasoning modes within a unified model and enables context-adaptive mode selection. \n3.The designed AdaVaR framework employs a logical two-stage training process. Through specific techniques such as mode prefixes, prefix-guided exploration, and mode-relative advantages, it successfully achieves multi-mode learning and adaptive selection. Experiments demonstrate consistent performance improvements.", "weaknesses": "1.The Prefix-guided Mode Exploration mechanism in AdaGRPO enforces a rigid, equal exploration of the two predefined reasoning modes (text-based and visually-grounded) for each input sample. While this hard constraint effectively prevents mode collapse, it introduces a fundamental limitation: the exploration space is artificially confined to a discrete set of predetermined patterns. I concern this design may prematurely restrict the model's policy search to this predefined mode space, thereby limiting its capacity to discover potentially superior strategies that fall outside these rigid categories. For instance, the optimal strategy for a given problem might involve a hybrid or sequential application of both modes, or even an entirely novel reasoning path not encapsulated by either predefined mode.\n2.I think the classification of reasoning into strictly \"text-based\" and \"visually-grounded\" modes is conceptually limiting. we don't need to force the model to choose a single, rigid strategy at the start of its reasoning process, which is an unnatural constraint.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "yCnR765PWY", "reviewer": "Reviewer_Yq9o", "summary": "The paper proposes Mixture‑of‑Visual‑Thoughts (MoVT), an adaptive reasoning paradigm that unifies text‑based and visually‑grounded chains of thought within a single LVLM and learns to select the appropriate mode per instance. The approach is instantiated via AdaVaR, a two‑stage pipeline: (1) SFT that teaches both modes using mode‑prefix tokens (e.g., <text>, <ground>), followed by (2) RL with AdaGRPO, an extension of GRPO introducing prefix‑guided mode exploration, a mode‑relative advantage signal for selection, and a curriculum from simpler, clearer tasks to more diverse ones. Experiments on eight benchmarks (covering math, spatial reasoning, hallucination, visual search, and general perception) show consistent gains over strong specialists and base models; the AdaVaR‑7B variant reportedly surpasses GPT‑4o on average, with ablations and an “oracle upper bound” analysis illustrating the complementarity of modes and the value of adaptive selection.", "questions": "- How would AdaGRPO’s mode‑relative advantage generalize beyond two modes—pairwise comparisons, a shared latent score, or another efficient scheme? Any preliminary results or complexity analysis?  \n\n- What fraction of errors are due to wrong mode choice versus within‑mode reasoning errors? Are there identifiable categories (e.g., deceptive diagrams) where the gate is brittle?  \n\n- Have you tried the framework with non‑Qwen LVLMs? Any architectural prerequisites for effective selection?  \n\n- Can you discuss multi-step or long-context reasoning—can the model switch modes dynamically inside a single reasoning chain? \n\n- There is limited interpretability of the mode selection. The paper does not sufficiently explain why the model chooses one mode over another beyond empirical observation", "limitations": "", "rating": 8, "confidence": 3, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "- Clear, novel problem framing offering a compelling path toward generalist multimodal reasoning. \n\n- AdaGRPO is purpose‑built for mode selection, combining (i) prefix‑guided exploration to avoid mode collapse, (ii) a mode‑relative advantage estimator to directly train the gating decision, and (iii) a curriculum that stabilizes learning before exposing the agent to noisy, diverse tasks. \n\n- Strong empirical results and breadth. Evaluation on eight varied benchmarks demonstrates robust, cross‑domain improvements; ablations show each component’s necessity. \n\n- Well‑presented and easy to follow.", "weaknesses": "- Scalability to k>2 reasoning modes (e.g. symbolic, tool‑augmented, variable CoT length) could be explored; naive exploration would scale rollout cost linearly with k and may become prohibitive without hierarchical or more efficient routing.  \n\n- The current setup enforces a single mode for the entire reasoning process, whereas many tasks might benefit from intra‑problem switching (e.g., ground first, then abstract reasoning).  \n\n- Prefix‑guided exploration requires n rollouts per mode (vs. 2n from a single policy in vanilla GRPO), increasing cost; quantitative training FLOPs, wall‑clock, and inference latency comparisons could be elaborated upon.  \n\n- The 1:1 SFT mix is reasonable but not properly analyzed; understanding sensitivity to data quality/quantity and skewed ratios (e.g., 3:1) would aid reproducibility and deployment.  \n\n- More systematic characterization of when mode selection fails (and why), interpretability of the gate’s decisions, and comparisons to simple ensembling would strengthen the paper.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)