Think or Not? Selective Reasoning via Reinforcement Learning for Vision-Language Models
Authors: Jiaqi WANG
Kevin Qinghong Lin
James Cheng
Mike Zheng Shou
Venue: NeurIPS 2025 poster ()
Keywords: Vision-Language Models
Reinforcement Learning
Primary area: applications
TL;DR: Teach multimodal models to think or not to think- this is a question.

Abstract: Reinforcement Learning (RL) has proven to be an effective post-training strategy for enhancing reasoning in vision–language models (VLMs). Group Relative Policy Optimization (GRPO) is a recent prominent method that encourages models to generate complete reasoning traces before answering, leading to increased token usage and computational cost. Inspired by the human-like thinking process—where people skip reasoning for easy questions but think carefully when needed—we explore how to enable VLMs to first decide *when reasoning is necessary*.
To realize this, we propose \ours, a two-stage training strategy:
**(i)** a supervised fine-tuning (SFT) stage with a simple yet effective “**thought dropout**” operation, where reasoning traces are randomly replaced with empty thoughts. This introduces a think-or-not format that serves as a cold start for selective reasoning;  **(ii)** a GRPO stage that enables the model to freely explore when to think or not, while maximizing task-aware outcome rewards.
Experimental results show that \ours can *reduce the completion length by up to **90%** compared to vanilla GRPO, without sacrificing performance or even improving it*. Further evaluations across LLM (GSM8K), VLM (CLEVR, Super-CLEVR, GeoQA), and Agentic (AITZ) tasks—covering a range of reasoning difficulties under both 3B and 7B models—consistently reveal that the \textit{model progressively learns to bypass unnecessary reasoning steps as training advances}.
These findings shed light on the path toward human-like reasoning patterns in RL approaches.
Our code is available at https://github.com/kokolerk/TON.

Think or Not? Selective Reasoning via Reinforcement Learning for Vision-Language Models
Jiaqi Wang Kevin Qinghong Lin
James Cheng'
Mike Zheng Shou
The Chinese University of Hong Kong
Show Lab, National University of Singapore
Abstract
Reinforcement Learning (RL) has proven to be an effective post-training strategy for enhancing reasoning in Dision-languee models (VLMs). Group Relative Policy Optimization (GRPO) is recent prominent method that encourages mod- els to generate complete reasoning traces before answering, leading to increased token usage and computational cost. Inspired by the human-like thinking pro- cess wess-where people skip reasoning for easy questions but think carefully when needed- pioneer how enable VLMs to first decide when reasoning is nec- essary. To realize this, we propose TON, a two-stage training strategy: (i) a supervised fine-tuning (SFT) stage with simple yet effective "thought dropout" operation, where reasoning traces are randomly replaced with empty thoughts. This introduces think-or-not format that serves as cold start for selective reason- ing; (ii) GRPO stage that enables the model to freely explore when to think or not, while maximizing task-aware outcome rewards. Experimental results show that TON can reduce the completion length by up to 90% compared to vanilla GRPO, without sacrificing performance or even improving it. Further evaluations across LLM (GSM8K), VLM (CLEVR, Super-CLEVR, CLEVR, GeoQA), and Agentic (AITZ) coves-covering a range of reasoning difficulties under both 3B and 7B reveal that the model progressively learns to bypass unneces- sary reasoning steps as training advances. These findings shed light on the path toward human-like reasoning patterns in RL approaches. Our code is available at https:: / thub. com/xokolerk/ION
1 Introduction
"To think or not to think, that is the question."
Reinforcement learning (RL) has recently emerged as dominant post-supervised fine-tuning (SFT) strategy in llsgua-language models (VLMs) [1, 2, 3, 4]. Methods like GRPO [5] have shown promising results in enhancing reasoning capabilities through KL-divergence losses based rule-driven rewards. However, these approaches often lead to unnecessarily long and redundant reasoning processes due to their reliance full-length generative trajectories [6. 8). To address this inefficiency, attempt to shorten reasoning chains with rule-based reward penalties [9, 10, 11] during the pre-training phase or introduce external control mechanisms, such in very recent Qwen3 [12]. Nonetheless, more natural and scalable solution is to enable the model to decide when to think mirroring how humans modulate cognitive effort in response to task difficulty.
works
In this work, we begin by presenting empirical evidence that thinking is not always necessary. In AITZ [13], observe that 51% of questions be answered correctly even when the entire reasoning trace omitted, resulting in significant savings in thought tokens. This finding underscores
Equal contribution.
@ Corresponding authors.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Q: As shown the figure, AB CD, and EF intersects AB and
Q: As shown the figure, triangle A inserthed triangle cirele angle OAB 35.00, then the degree ungle is
CD at points E, F. angle 1 - 50.0, then the degree of angle 2 is
-tank--Stxx AB padidd to CD, cmn use the proporty that aumopuediigg ungla fimmal by conovenal
<thinke in the giver fguue thangle ABC is inmaiibui in a cinlle wane O... we cnnalude that undl ACB zsgle AGB - 905 <ithink X -annwer--90 90%coassme
GRPO Thinkiar
paraleel lines equal Tharefoe, angle equal sngle hecmsse Thansirs, ongle 2 180) 50 - 130 degrees.
GRPO
the giver prbllm, with
trangle ABC insetheot
canswer- Sanswer-110-aaaneee 130-/answer
eatier O... Sinx seple ACB is
TON Naw- Thint
TON Thinking 2611 toluns
angle AOLL, angle ACB - 1/2 110"
Easy
18 tolees
+ Sanswer- 3atapp----/a/aee--
Hard
v -answer-SS 55hc/aasme"
cithink
For simple queries, explicit reasoning is oonecessary-modeeee like GRPO that always "think" incur redundant computation. Right: For more complex geometric problems, step-by-step reasoning is essential to arrive at the correct answer. Our proposed TON framework learns adaptively invoke reasoning only when needed- skiipiin for easy cases while engaging deeper inference for harder tasks.
Figure 1: Illustrating the "think or not think" trade-off. Left:
the potential of selective reasoning strategies to improve efficiency without sacrificing accuracy. Secondly, by exploring a simple prompting strategy - allowing the model to skip reasoning steps for easier queries we observe that even math-enhanced VLMs struggle adaptively omit redundant thought generation. Instead, they tend to default to a conservative approach, producing full reasoning traces regardless of task difficulty. This suggests that the ability "think not" is not solely determined by reasoning capacity, but should instead be treated as a distinct snill-one that should be explicitly activated through format-following supervised fine-tuning (SFT) stage.
Motivated by the above observations, we introduce TON (i.e., Think-or-Now Not). a two-stage training framework featuring simple yet effective "thought dropout" approach. This method explicitly replace reasoning traces with minimal "\n \n" delimiter and employs SFT to train the model that reasoning can be skipped-thereby enabling the possibility of bypassing reasoning. A subsequent GRPO stage further refines this selective-reasoning policy via self-exploration, rewarding answers without introducing extra regularization. As illustrated in Figure generates reasoning sequences regardless of task difficulty. In contrast, our method, TON, adaptively allocates reasoning based on the complexity of the task. For simple tasks (left). TON can bypass unnecessary reasoning and directly provide the answer, reducing 90% token usage. For more hard problems (right), it still engages in detailed, step-by-step reasoning arrive at the correct solution. To the best of our knowledge, TON is the first work to study "when to think" in VLM.
vanilla GRPO consistently
Built top of TON, we using the Qwen-2.5. LL series and conduct extensive evaluations actoss the LLM benchmark (GSM8K [14]), vision-language tasks spanning counting (CLEVR [15], SuperCLEVR [16)) well mathematical reasoning (GeoQA [17]), and the agent task like mobile agent navigation (AITZ which collectively cover a spectrum of reasoning levels and diverse task settings. Overall, find that TON achieves substantial reductions in completion length without compromising performance cuttinn 87% of tokens on CLEVR and 65% on GeoQA. Notably, on the multi-step navigation task AITZ, TON reduces the average task-level output length from 3.6K to 0.9K tokens. Moreover, we observe that omitting reasoning traces can even improve performance: on GeoQA, TON outperforms the vanilla GRPO baseline by up to 17% in accuracy, demonstrating "free-lunch" effect where shorter reasoning outperforms or matches longer trajectories. Comprehensive ablation studies further reveal that the skip-thought ratio increases progressively with reward improvements during training, suggesting the model learns to selectively bypass unnecessary reasoning steps in an adaptive manner.
2 Related Works
Reinforcement Learning for Vision-Language Models. Most VLMs start with SFT on large collections of instruction data to acquire broad foundational knowledge [18, 19, 13, 20]. To further improve performance, recent work has adopted post-training paradigm that leverages human feed- back [21, 22, 23]. RL from human feedback (RLHF) fits a reward model on preference annotations and refines the policy via Proximal Policy Optimization (PPO) [24. 25, 22, 26]. Direct Preference Optimization (DPO) [27] streamlines this workflow by recasting policy updates as a binary classifi- cation task, aligning model outputs distributions with human preferences without a reward module.

---

Beyond these methods, Group Relative Policy Optimization (GRPO) [5] blends offline and online learning: it samples groups of thinking process, uses Answer verification (such as Math verifier) as reward feedback, and computes relative advantages within each group. By avoiding value function, GRPO provide an elegant solution by promoting diverse reasoning paths and improved answer quality. Despite a series of GRPO follow-up works [28, 29, 9], all of these approaches assume that every question demands a full thinking-leading to lengthy decoding. In contrast, our work focuses on "when to think" instead of "how to think": we introduce a selective reasoning policy that learns to skip unnecessary "think" phases, boosting inference efficiency without sacrificing accuracy.
Thinking in Language Models. From early Chain-of-Thouggh [30, 31, 32] prompting recent
reasoning-intensse reinforcement learning approaches [5, 23, 33, 34], reasoning has emerged as core dimension in the development of language models. Most existing work emphasizes how to enhance reasoning capabilities, often resulting in increasingly lengthy and complex thought processes [7, 35, 10] while relatively few studies address the efficiency of reasoning. For instance, [36] proposes a long2short strategy to compress decoding length, [37] encourages models to output "I don't 't know" to terminate unproductive reasoning, and [38] introduces a token- seeen--udgt--wwadd reasoning policy. While these approaches offer promising insights into controlling reasoning length, we argue for a more foundational perspective: rather than deciding how to reason once the process has started, models should first determine whether reasoning is necessary at all. Simple questions may be answered directly without any explicit reasoning, while complex questions may require maintaining full reasoning trajectory [8, 6, 9]. In this work, we explore the selective reasoning paradigm within VLMS by introducing simple yet effective method- thought-dropout We validate its effectiveness on tasks such as Counting, Math, and further extend it to more practical agentic settings.
3 Preliminary
Task Definition. We formalize the vision-language reasoning environment as a Markov Decision Process (MDP) defined by tuple (V, S, x, r), covering a wide range of rision-languan tasks. Here, V denotes the visual context (e.g., an image). Q is a language-basee based query or question posed about the visual input. The model, governed by policy æ, takes the input pair (V, Q) and generates predicted answer S. The environment provides a scalar reward function r(-) based on the model's s response O. A correct prediction, e.g., O matches the ground truth answer S*, yields positive reward, while an incorrect one yields zero. The objective in this environment is to learn an adaptive policy wo, parameterized by 0, that maximizes the expected reward, enabling the model to reason selectively and efficiently across diverse input settings.
The reward function r(-) can be either model-based [24, 26] or rule-based, as recently demonstrated in [5, 23], which is typically categorized into two types: format re- wards ry and outcome rewards ro- While the outcome rewards are usually carefully designed based on different tasks or requests in previous works [5, 1, 2, 9], the format reward ry, is al- ways shared in the same. Given the response O, it should follow the required HTML tag format <think>? T is the predicted answer. This formulation requires the model think before deriving the answer and makes it easy to parse both the reasoning process and the final outcome (e.g., via regular expression).
Reward Function.
< SaaanSSaaarss where T is the reasoning process (i.e., a thought) and S
4 TON: Selective Reasoning via Policy Optimization
Observation. In practice, humans do not require explicit reasoning for all tasks-many can be completed intuitively, Similarly, models can often produce correct answers to simple questions without explicit thinking. As illustrated in figure the percentages of correct and incorrect sam- ples under different setups with and without the thinking process in inference (see Appendix A for overall perfor- mance). We find that 52. 1 1% of answers remained correct without "think," " and 14.5% were even correct only without it implying that explicit thinking is not always necessary.
wo think correct
wo think incurrect
think correct
53.15
25.6%
w think incorrect
14.3%
Figure 2: Accuracy comparison of
with v.s. without "thinking" dur-
ing SFT using Qwen-2.5-VL-3B on the AITZ task.
3

---

A straightforward idea is to prompt the model to decide whether to "think" or not (we prompt the model skip thinking in the simple questions in Sec. 5.4). However, shown in our experiments (Figure 5d and Appendix G.7), the model still tends to generate the full reasoning process without any no-think try. This suggests that the ability to decide whether to think is not solely governed by reasoning capability, but should instead be treated as separate skill-on that must be explicitly trained through format-following during the supervised fine-tumi finetuning (SFT) stage. These observations motivate us to activate this ability early in the SFT stage and develop TON, which enables selective reasoning by automatically switching between "think" and "non-think" modes.
4.1 First SFT stage: Thought Dropout
In the initial stage, the model is typically fine-tuned on "think-answer" formatted data, where the "think" contains high-quality reasoning traces serve as cold start. To extend this predefined reasoning ability to selective reasoning, we view "think" vs. "non-think" as part of the output format itself by dropping the "think" component during training.
However, difficult to determine which samples should be skipped, as different models exhibit varying reasoning capabilities. Therefore, we begin with random dropout and allow the model to learn to decide for itself during the second RL stage (Sec.4.2). To this end, we propose "Thought Dropout" that randomly injecting empty "thought" segments, requiring only minor code changes:
Algorithm 1 Pscudo-code for thought Bropught_droooutttt
def thought thought_drapous propout (thought, drapout__prob) if random, random () dropout_prob:
thought = "An\n" return thought
This approach injects both the answer format and the skip-thought format as prior knowledge before the second RL stage.
Where do Thoughts come from? Given a policy operating in an environment (V, S, n. r), key challenge is how to curate high-quality cold-start "thought" data without relying external models, such as closed-source APIs. A naïve approach is to run multiple inference passes and retain only successful cases based answer matching but we find this to be less effective. To address the scarcity of high-quality "thought" data, we instead adopt a reverse thinking strategy: leveraging the base model π itself to self-generate rich corpus of thought sequences. Specifically, given the visual context V, textual query Q, and ground-truth answer S*, we prompt the policy my to deduce the corresponding intermediate thought follows:
T To (V)
Specially, we generate intermediate thoughts with the following prompts:
Prompt for Reverse Thinking
Based on the following question and image, generate a thought process to explain how to derive the answer from the inputs.
Image: (Image) )o Question: (Question] Answer: (Answer)
Do not output the answer, only generate the reasoning process. Formulate your outputs using concise language.
In this way, we curate sufficient thought data without relying on external models. These serve as our cold-start training corpus, enabling to apply the Thought Dropout strategy during SFT to activate the model's ability to bypass thoughts.
4.2 Second RL stage: Group Relative Policy Optimization
Although SFT teaches the skip-thought format, still leaves central question unresolved: when should thoughts be skipped or retained? Ideally, the model should learn to explore this decision

---

Thought
Answer
02
Correct
Correct
03
03
Incorrect
Incorrect
GRPO
TON
Emply
04
diversity (01, 1,,5)
Figure 3: Illustration of the responses from GRPO and TON. @1 is the question and (o1,-.. os ) the generated responses containing thoughts T (circle) and answers S (triangle). TON can sample from the empty think Ta thus enhancing the response diversity over the vanilla GRPO.
its own this end, we adopt reinforcement learning via GRPO to enhance the model's 's ability to explore this decision as part of its reasoning process.
Given an image E V and text query q Q. GRPO samples N candidate responses with variations {0i, 02. ON) from the policy To and evaluates each response o, using reward function r(-), which measures the quality of the candidate in the context of the given question. To determine the relative quality of these responses, GRPO normalizes the rewards by computing their mean and standard deviation and subsequently derives the advantage as:
A r(o,) mean (eaa(r(()), r(o)).... r((x))
std(r(o1), r(o))...,))).
where A, represents the advantage of the candidate response o; relative to other sampled responses. GRPO encourages the model to generate responses with higher advantages within the group by updating the policy using the following objective:
N
(an = Elfor ~ To
{minla, Aii 3,) A.] 11D KL. KIl (3)
To(ozlu.9)
To(o; U, 9)
0; Tosa 10(v,
clip
Tourd (0s/0: 12/0;40 (.
+ e, -e)
How does TON impact GRPO? As illustrated in Fig. 3, our TON allows the model choose "empty- think" Trna during the inference step, thus resulting in a significant variation in the distribution between the non-think (o, Trn))) and think responses (o, T) by TON compared to both think
ones (o, ~ ')) generated by vanilla GRPO. Unlike previous works like DAPO [28] emphasize on advantage distribution A, by dynamic sampling in the sparse reward space, our TON shifts the focus to the latent distribution space of responses (o:lu, q)), thus enhancing the diversity of the terms and [ in Eq.
How to design Rewards? To support GRPO training across diverse settings, it is crucial to carefully examine reward design choices. We consider main types of matching:
(i) Discrete Matching. For tasks with deterministic, categorical or numerical outputs -.g. classi- fication, counting, math problems-oee we use a binary value reward ra(s,g) 1(s 9): if the predicted answer s matches the ground-truth g. we assign Ta = 1: otherwise, Ta = 0.
(ii) Continous Matching. For tasks producing continuous outputs- e.g., spatial coordinates in UI navigation or object grounding we allow tolerance region. Given predicted point p = [zr and ground-truth box b = [21, V1. 22, y2l. we define:
[.. 1, p lies inside b,
(p. b)
0. otherwise.
If only ground-truth point p' is available,
distance threshold 0:
1, | 1p-p"ll2 50
re(p,
0, otherwise.
In practice, we sum the applicable components to form outcome reward: r. = rattre This simple yet flexible scheme can cover classification, numeric reasoning, and grounding. See Appendix B for details adapting these rewards alongside the format reward to individual downstream tasks.
5

---

Table 2:
Performance comparison between TON and vanilla GRPO. Acc. is the accuracy the test set.
Time is the RL training time. Length is the average competition length at the end of training.
Counting (CLEVR)
Math (GeoQA)
Acc (t)
Time ()) Qwen-2.5-
Len. ())
Acc (1)
Time (L)
Len. (U)
Acc (f)
Time (L)
Len. ())
Qwen-2.5-VL-3B
Qwen-2.5-VIL-7B
Baseline
64.0
306
36
924
38
939
GRPO
93.5
1h 44m
227
37
2h 50m
272
44
3h 4m
1069
TON
98.5
57m
28
51
2h 4m
96
61
2h 34m
112
Gain
+5.0
-47m
87%
+
46m
65%
+17
-29m
-90%
5
Experiments
In this section, we conduct experiments on various benchmarks to evaluate our approach. Mainly, we design the experiments to study the following key questions:
Q1: Compared to vanilla GRPO, how does TON impact performance and efficiency?
Q2: Is there correlation between TON's skipping behavior and the strength of reasoning ability (e.g., different model sizes or single model under different iterations)?
Q3: Do we really need SFT with thought dropout? Can we rely solely on prompt following if the base model is strong enough?
5.1 Benchmarks and Settings
To evaluate the effectiveness and generalization ability of our approach on the below settings:
Benchmarks. We evaluate TON on three vision-language benchmarks, including the general benchmark CLEVR [15] (3D object counting), agent benchmark AITZ [13] (mobile navigation), and the math benchmark GeoQA [17] (middle school math questions) as illustrated in Table 1, spanning spectrum of reasoning lev- els from simple to complex. To benchmark the model's
Table 1: Summary of benchmark used in our evaluation.
Benchmark
Model OOD
Type
Difficulty
Answer
Thought len.
GSMSK [14]
Maih
Number
CLEVR [155]
VLM
Counting
Easy
Integrate
586
Super-CLEVR [16]
Counting
Easy
Integrate
GeoQA [17]
VLM
Math
Hard
Number
1652
AITZ. [13]
Agent
GUI
Mediam
Action (x)
283
AITZ (OOD)
Agont
/
GUI
Modium Action (X)
283
(OOD) performance, we Super-CLEVe [16] to supplement the CLEVR. AITZ comprises four test domains: we train on the [General] and test on the remaining OOD domains: (Web shopping, Google apps, Install]. We remove the choices in GeoQA and ask the model generate the answer, enhancing the reasoning complexity. AITZ includes action thought annotations, which we utilize directly, while applying reverse thinking generate thoughts for SFT on CLEVR and GeoQA. More benchmark details refer to Appendix E.
also evaluate
Training details. We conduct our experiments using Qwen-2.5- Swen-2.5-VL--Intrruee VL-Innstrucct3BB7BB [39] as the base model. All experiments are conducted utilizing NVIDIA H20 GPUs. We train 100 steps for both CLEVR and AIT7, and 300 epochs for GeoQA, given its higher reasoning difficulty level. See setup details in Appendix F. We leverage vLLM [40] to accelerate GRPO training. We add the SFT stage before GRPO as the baseline the agent task with the same setting TON because observe that directly applying GRPO would cause the coordinate reward during the training process, considering its complex output format. For simplicity, we set the dropout probabilities to 50% and examine the impact of different dropout ratios selected from f20% 50%, 80%} in Sec 5.3.
For evaluation, we test all the datasets under the greedy strategy. In CLEVR and GeoQA tasks, where answers are numerical, we measure accuracy by comparing the predicted number to the ground truth. In the AITZ task, where answers are structured as 'SON-formatted actions, report step-level and task-level metrics, including type accuracy (correct action type) and exact accuracy (correct action type and click coordinates) following [18].

---

Training rewards
(h) Completion length
Skip think ratio
(d) Output len. with think
Figure 4: Training metries comparison between TON and GRPO on GcoQA. (a) Training rewards, (b) Completion length over training steps, (c) Ratio of non-think outputs to total samples at each step for TON, and (d) Average completion length of think outputs across training.
Table 3: Out-of-domain (OOD) performance comparison between our method TON and GRPO on the the AITZ multi-step mobile navigation. 'Type' is the action type accuracy and 'Exact' requires both the type and value to be correct exactly. 'Avg.' is the average accuracy of all domains. "Task-level thought's' is the average output lengths all OOD domains. Step-level accuracy reported.
IID
OOD
Think?
General exact
Google apps exact
Weh
Install
Avg
Task-level
type
type
type
exact
type
exact
type
exact
Thought's Ien.
Qwen-2.5-VL-3B
0.01
0.01
0.01
0.01
0
0.01
2132
W. SFT
X
0.39
0.11
0.11
0.12
0.51
0.19
0.17
0.17
0.16
0.15
712
SFT
0.12
0.53
0.56
0.14
0.58
0.14
GPRO
0.74
0.6
0.72
0.57
0.7
0.5
0.81
0.65
0.74
0.59
3664
Ours
0.74
0.74
0.56
0.72
0.75
0.39
922
Gain
+0.0
+0.0
+0.02
-0.01
+0.02
+0.0
-0.03
-0.01
+0.01
+0.0
-2742
Qwen-2.5-VL-7B
0.28
0.14
0.26
0.1
0.33
0.13
0.39
0.16
0.31
0.13
3304
GRPO
0.64
0.22
0.73
0.32
0.15
0.62
0.23
0.65
0.23
3272
W. TON
Ours
0.74
0.54
0.62
0.23
0.68
0.47
0.73
0.55
0.89
0.45
908
Gain
+0.1
+0.32
+0.09
+0.32
23661
5.2 Q1: Performance and Efficiency Comparison between TON and GRPO
In Table 2, we present TON on the CLEVR and GeoQA benchmarks under both 3B and 7B settings, with the performance, time consumption, and the average completion length at the RL stage. We find that TON effectively reduces the average of the completion length by up 90% while achieving comparable even superior performance compared to GRPO with maximum of 17 Acc. gains. This imply that skipping unnecessary reasoning can lead better performance. The reduction of the completion length decreases the decoding time when generating samples, thus simultaneously shortening the training time. Figure 4a & 4b show the reward and completion length curves where TON remains the rewards on par with vanilla GRPO while the completion length reduces significantly. Appendix G.2 G.I shows the entire metrics during training.
Multi-step Navigation and OOD Testing, In Table 3, we evaluate TON's performance on AITZ multi-step mobile navigation, we also assessed its generalization capabilities on OOD test sets using a greedy decoding strategy. Table 3 summarizes the step-level type match accuracy and exact match accuracy for both IID (general) and OOD (Google Apps, web shopping, and install) domains on AITZ, with detailed training visualization in Appendix G.3. Overall, TON demonstrates comparable OOD generalization performance to GRPO, while significantly reducing the task-level output length from 3K to 0.9K (70% token saving). This highlights the strong potential of TON to substantially reduce completion length without compromising performance. See Appendix G.4 for the OOD performance on other benchmarks.
Adapt TON framework to the text-only setting. Furthermore, extend our study the LLM domain and present the corresponding experiments and results in Figure 6(b) on the LLM benchmark GSM8K. The findings indicate that TON significantly reduces response length while maintaining high accuracy, demonstrating the generalizability of TON across different modalities.
5.3 Q2: Skip Thought Ratio Analysis
Beyond the performance change and completion length reduction achieved by TON, we further investigated the evolution of the skip ratio in "Thought dropout' during the training step. Figure 4c
7

---

Complattn JargT
Completion length
(b) Skip think ratio
(c) Training rewards
(d) Prompting v.s. SET
Figure 5: Further Analysis of TON on the AITZ benchmark. (a)()(c) is the average completion length, skip thought ratios, and the reward under different dropout probabilities. (d) Prompting (hybrid) does not reduce the completion length, while TON using SFT can effectively reduce it.
0 Zeroshorri acc L TON acc Li Skip ratar(h))
Passg Accurncy
Avernge think lengths
105
98.5 08
939.0
928.0
342.0
Accuracy (%)
90
85
75
64.0
70
65
50.5
45
Diffiellty aware dropout
Random dropout (ours)
38.0
Qwen
Counting CLEVR
AFTZ
Gool34
GRPO
TON
Accuracy and average think lengths of TON rrinndd on Qwen2.5. 1.5B-dittil deepeelll on GSM8K.
(e) Accuracy comparison difficulty- dropout and
(a) The difficulty-awart classification
random
across from the zero-short accuracy.
dropout during SFT on GeoQA.
Figure 6: Further Analysis of TON. (a)(c) give the in-depth analysis of the effectiveness of our thought dropout and the generalization across the task difficulty. (b) shows the superiority of TON on the text-only domains.
illustrates the percentage of skip ratio in the generated samples at each step on GcoQA. We observed an increasing trend in the skip ratio during the training process with the increase in training reward. A similar trend is observed across three benchmarks in Figure 21 in the Appendix G.6. This phenomenon suggests that the model progressively internalizes the reasoning process-learning to skip explicit thoughts while still producing accurate answers. Moreover, Figure 4d illustrates the length of these outputs generated with 'think'. TON maintain comparable lengths to the vanilla GRPO, indicating that the TON model can choose not to think but remains diligent when deeper reasoning is necessary.
Thought dropout ratio ablation. We experiment with the impact of different thought dropout ratios of 20%, 50%, and 80% during the SFT stage. Figure 5a & 5b show the completion lengths and the skip ratio during the training process on AITZ. Figure 5c shows close reward curve of these three variants. Refer more metrics on Appendix G.5. Although the dropout ratios differ, TON consistently exhibits an increasing skip ratio as training progresses. Notably, the 20% setting shows a rapid increase in skip rate, while the higher 80% setting remains relatively stable throughout training. This motivates to start with lower dropout probability for further investigation. TON can then be dynamically optimized according to reward signals-decrees
the dropout ratio when performance
is high and increasing it when performance drops.
Deep analysis of the difficulty-awan dropout and random dropout (ours). As shown in Fig- ure 6(a), we further assess the task difficulty both qualitatively (via intuitive reasoning complexity) and quantitatively (by base VLM zero-shot accuracy), grouped as: 60-100 (easy), 40-60 (medium), and 0-40 (hard). Our findings indicate that our TON maintains a high skip ratio for easy counting queries, allowing it to bypass unnecessary thinking processes, while keeping a low skip ratio for harder, more professional, knowledge-intee Furthermore, we do the difficulty-aware ablation as shown in Figure 6(c), where we drop thoughts for only the easy samples (those answered correctly by the base VLM). To investigate this, we then implement difficulty-awan dropout strategy and compare its accuracy under our TON training framework where random dropout slightly outperforms difficulty-awee awaree dropout (51.0% vs. 50.5% in
questions to fully utilize its thinking capabilities.
8

---

8
G
Query: photos in the google photo apps.
- Tie scaae ae
-G.a Coagk Molec up friat
rdla Clc Ranwapli silee New wadde J.N paklerr
Conterr Umeul. Staeer
conter Tee S0t LV". Naral Caale
Cecoe Sear SOOLLL (P:, Torsl Cacten
GRPO
BV ouilor
92 vilvor
viler
cellant 3.7% 155) ctatee poloes
STATES TASK COMPLETTE
TON
Norc ooster
Sier: jostie (340 (1edly
Not.
Figure 7: Comparison between GRPO and TON on Agent setting AITZ [13]. TON adaptively skips unnecessary think steps during multi-step mobile navigation, achieving greater decoding efficiency compared to GRPO while maintaining task accuracy (saving 60% tokens this case).
TON Accuracy). The results suggest that hand-crafted heuristics for task difficulty may introduce noise or unintended bias, potentially interfering with the learning process while our random dropout offers simpler, unbiased alternative that generalizes well across tasks.
5.4
Q3: Emprical Verfication of SFT Significance in TON
In addition to incorporating the skip-think format during the SFT stage as in TON, we explored simpler alternative: modifying the prompt to encourage the model to automatically omit reasoning steps, enabling direct GRPO training without the need for a separate SFT stage. The wybrid-thought prompt is defined as follows:
Prompt for Hybrid Thinking
conversation between User and Assistant. The user asks a question, and the Assistant solves it. The assistant evaluates whether the question is simple enough answer directly. If simple, the output formatted as chhnns--nnnchhiin--cnssweeer If the question is difficult, the assistant needs to first think then answering the question. The output is formatted as <think> reasoning process here <Wthink><asswe>>> answer here <lanswer>,
here<lanswer>>
The assistant is encouraged to use <think>>\nne<thhink> while maintaining accuracy.
Figure 5d shows the completion length of GRPO using the hybrid prompt, vanilla GRPO (with full-think prompt), and TON throughout the training process on AITZ. Appendix G.7 presents similar trends across three benchmarks, revealing only minor differences in completion length between the hybrid prompt and vanilla GRPO. Moreover, we observe only 2 'skip' cases in GeoQA and none in AITZ among all samples generated by GRPO during both training and inference. We attribute this to the model's tendency to play it safe by generating long and detailed reasoning, consistent with its ingrained behavioral patterns learned during pre-training or SFT. Since the model does not produce thiptthought outputs, applying additional reward to these outputs has no effect, resulting in zero contribution throughout training. These findings highlight the necessity our SFT stage with thought dropout (Sec. 4.1) establish the desired format-following behavior.
5.5 Qualitative Examples
Figure 7 compares GRPO and TON on the AITZ benchmark for multi-step mobile navigation. While GRPO generates verbose reasoning at every step. TON adaptively skips unnecessary thinking, reducing token usage by 60% without sacrificing task accuracy. This demonstrates TON's efficiency in handling world, long-hoo honghhorizon procedural agent tasks. Table 4 further illustrates TON's ability

---

Table 4: Illustration between Thinking and Non-Thinking modes from CLEVR [15]. TON demonstrates selective activation of reasoning-engagnin thought only when needed-whereas GRPO generates reasoning traces
cases indiscriminately. The full outputs are Table
Counting examples from CLEVR [15]
Question (GT)
How many items are there in the im- age? Fanswer>-<</answeee
How many items are there the image?
<answer>&</answer>>
GRPO's outputs
<thinks The image shows three objects
<think> count six distinct objects in picture: a large green matte cylinder, blue matte cylinder, ...
Green Cylinder: Transparent (metal-
lie) 2. Teal Sphere: Transparent (metal- lie)
I also note the size and color variety of
each object as described by the visual cues. </think>
</think>
<answer>3</anwwer>
<answer> <answer>l0<<answer 10</answe>>>>>>>>
TON's outputs
<think>\ninetthink>
<think Okay, let's check this out. The user provided image with
<answers3</anwer>>
First, need
1. "A small purple rubber sphere..."... ".. small gray rubber cube..." Third.
Wait, that's eight distinct entries... <Ahinks
<answer>> </answer>
to selectively activate reasoning. Unlike GRPO, which consistently generates detailed thought traces, TON omits reasoning for simple questions that can be answered glance, while producing accurate and focused reasoning for complex scenarios involving visual occlusion.
Conclusion
present TON, simple yet effective two-stage training framework that enables vision-language models to learn when to reason-intrrouuing selective reasoning controllable and trainable behavior. By combining thought dropout during supervised fine-tuning with reward-guided refinement via GRPO, TON significantly reduces completion length (up to 90%) without sacrificing- and in some cases improving performance across diverse reasoning tasks. Our findings challenge the assumption that full reasoning traces always beneficial and pave the way for more efficient, human-like reasoning strategies in both multimodal intelligence and reinforcement learning.
7
Acknowledgements
This research supported by the National Research Foundation, Singapore under its Al Singapore Programme (AISG Award No: AISG3-RP-2022-003
We sincerely thank Dongchi Huang for his invaluable guidance the code and for providing essential computational resources. We also appreciate Binghui Xie's insightful discussion on topic selection and idea suggestions. Additionally, we are grateful to Qiguang Chen and Yuxuan Wan for their thoughtful and constructive feedback on this paper. Finally. we extend our gratitude to Xiaojun Guo and Qixun Wang for their valuable advice on visual reasoning and the GRPO series methods.
References
[1] Zilin Huang, Zihao Sheng, Yansong Qu, Junwei You, and Sikai Chen. Vlm-rl: A unified vision language models and reinforcement learning framework for safe autonomous driving, arXiv
10

---

preprint arXiv:2412.15544 2024.
[2] Zhengxi Lu, Yuxiang Chai, Yaxuan Guo, Xi Yin, Liang Liu, Hao Wang, Han Xiao, Shuai Ren, Guanjing Xiong, and Hongsheng Li. Ui-rl: Enhancing action prediction of gui agents by reinforcement learning, 2025.
[3] Run Luo, Lu Wang, Wanwei He, and Xiaobo Xia. Gui-rl: A generalist r1 r1-style vision-language action model for gui agents, 2025.
[4] Hardy Chen, Haoqin Tu, Fali Wang, Hui Liu, Xianfeng Tang, Xinya Du, Yuyin Zhou, and Cihang Xie. Sft or rl? an early investigation into training r1-like reasoning large vision-language models, 2025.
[5] Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Y Wu, et al. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402. 20t::2402.033000, 2024.
[6] Jinyan Su, Jennifer Healey, Preslav Nakov, and Claire Cardie. Between underthinking and overthinking: An empirical study of reasoning length and correctness in 1lms, 2025.
[7] Qiguang Chen, Libo Qin, Jiaqi Wang. Jinxuan Zhou, and Wanxiang Che. Unlocking the capabilities thought: A reasoning boundary framework to quantify and optimize chain-of- thought, 2024.
[8] Yuyang Wu, Yifei Wang, Tianqi Du, Stefanie Jegelka, and Yisen Wang. When more is less: Understanding chain-of-thoun tf-touhht length in Ilms, 2025.
[9] Xingyu Chen, Jiahao Xu, Tian Liang, Zhiwei He, Jianhui Pang, Dian Yu, Linfeng Song, Qiuzhi Liu, Mengfei Zhou, Zhuosheng Zhang, Rui Wang, Zhaopeng Tu, Haitao Mi, and Dong Yu. Do not think that much for 2+3=? on the overthinking ol-like Ilms, 2024.
[10] Haotian Luo, Haiying He, Yibo Wang, Jinluan Yang. Rui Liu, Naiqiang Tan, Xiaochun Cao, Dacheng Tao, and Li Shen. Adarl: From long-cot to hybrid-cot via bi-level adaptive reasoning optimization, 2025.
[11] Yi Shen, Jian Zhang, Jieyun Huang, Shuming Shi, Wenjing Zhang, Jiangze Yan, Ning Wang, Kai Wang, and Shiguo Lian. Dast: hifficulty-adaptii dow-thinking for large reasoning models, 2025.
[12] An Yang, Baosong Yang. Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, Keming Lu, Keqin Bao, Kexin Yang, Le Yu, Mei Li, Mingfeng Xue, Pei Zhang, Qin Zhu, Rui Men, Runji Lin, Tianhao Li, Tingyu Xia, Xingzhang Ren, Xuancheng Ren, Yang Fan, Yang Su, Yichang Zhang, Yu Wan, Yuqiong Liu, Zeyu Cui, Zhenru Zhang, and Zihan Qiu. Qwen2.5 technical report. arXiv preprint MrXiv:2412.15115 2024.
[13] Jiwen Zhang, Jihao Wu, Yihua Teng, Minghui Liao, Nuo Xu, Xiao Xiao, Zhongyu Wei, and Duyu Tang. Android in the zoo: arXiv:2403.02713 2024.
for gui agents. arXiv preprint
[14] Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. Training verifiers to solve math word problems, 2021.
[15] Justin Johnson, Bharath Hariharan, Laurens Van Der Maaten, Li Fei-Fei, C Lawrence Zitnick, and Ross Girshick. Clevr: A diagnostic dataset for compositional language and elementary visual reasoning. In Proceedings of the IEEE conference computer vision and pattern recognition, pages 2901-2910, 2017.
[16] Zhuowan Li, Xingrui Wang, Elias Stengel-Eskin, Adam Kortylewski, Wufei Ma, Benjamin Van Durme, and Alan I. Yuille. Super-clevr: A virtual benchmark to diagnose domain robustness visual reasoning. In Proceedings of the IEEF/CVF conference on computer vision and pattern recognition, pages 14963-1493 14973, 2023.

---

[17] Jiaqi Chen, Jianheng Tang, Jinghui Qin, Xiaodan Liang, Lingbo Liu, Eric P Xing, and Liang Lin. Geoqa: A geometric question answering benchmark towards multimodal numerical reasoning. arXiv preprini arXiv:2 1055. 2021.
[18] Kevin Qinghong Lin, Linjie Li, Difei Gao, Zhengyuan Yang, Shiwei Wu, Zechen Bai, Weixian Lei, Lijuan Wang, and Mike Zheng Shou. Showui: One vision-language-actoon model for gui visual agent. arXiv preprint arXiv:241 1. 17465, 2024.
[19] Kanzhi Cheng. Qiushi Sun, Yougang Chu, Fangzhi Xu, Yantao Li, Jianbing Zhang, and Zhiyong Wu. Seeclick: Harnessing gui grounding for advanced visual gui agents. arXiv preprint arXiv:2401. 10935, 2024.
[20] Wenyi Hong. Weihan Wang. Qingsong Lv, Jiazheng Xu, Wenmeng Yu, Junhui Ji, Yan Wang, Zihan Wang, Yuxiao Dong. Ming Ding, et al. Cogagent: A visual language model for gui agents. arXiv preprint arXiv:23/2.089 2023.
[21] OpenAl. Gpt-4 technical report, 2023.
[22] Gemini Team, Rohan Anil, Sebastian Borgeaud, Yonghui Wu, lean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, et al. Gemini: family of highly capable multimodal models. arXiv preprint arXiv:2312. 1805, 2023.
[23] DeepSeek-Ar AI, Daya Guo. Dejian Yang, Haowei Zhang. Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang. Xiao Bi, Xiaokang Zhang, Xingkai Yu, Yu Wu, Z. F. Wu, Zhibin Gou, Zhihong Shao, Zhuoshu Li, Ziyi Gao, Aixin Liu, Bing Xue, Bingxuan Wang. Bochao Wu, Bei Feng, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, Damai Dai, Deli Chen, Dongjie Ji, Erhang Li, Fangyun Lin, Fucong Dai, Fuli Luo, Guangbo Hao, Guanting Chen, Guowei Li, H. Zhang, Han Bao, Hanwei Xu, Haocheng Wang, Honghui Ding, Huajian Xin, Huazuo Gao. Hui Qu, Hui Li, Jianzhong Guo, Jiashi Li, Jiawei Wang. Jingehang Chen, Jingyang Yuan, Junjie Qiu, Junlong Li, J. L. Cai, Jiaqi Ni, Jian Liang, Jin Chen, Kai Dong, Kai Hu, Kaige Gao. Kang Guan, Kexin Huang, Kuai Yu, Lean Wang, Lecong Zhang, Liang Zhao, Litong Wang. Liyue Zhang, Lei Xu, Leyi Xia, Mingchuan Zhang, Minghua Zhang, Minghui Tang, Meng Li. Miaojun Wang. Mingming Li, Ning Tian, Panpan Huang, Peng Zhang, Qiancheng Wang, Qinyu Chen, Qiushi Du Ruiqi Ge, Ruisong Zhang, Ruizhe Pan, Runji Wang, R. J. Chen, R. L. Jin, Ruyi Chen, Shanghao Lu, Shangyan Zhou, Shanhuang Chen, Shengfeng Ye, Shiyu Wang, Shuiping Yu, Shunfeng Zhou, Shuting Pan, S. S. Li. Shuang Zhou, Shaoqing Wu, Shengfeng Ye, Tao Yun, Tian Pei, Tianyu Sun, T. Wang. Wangding Zeng, Wanjia Zhao, Wen Liu, Wenfeng Liang, Wenjun Gao, Wenqin Yu, Wentao Zhang, W. L. Xiao, Wei An, Xiaodong Liu, Xiaohan Wang. Xiaokang Chen, Xiaotao Nie, Xin Cheng, Xin Liu, Xin Xie, Xingchao Liu, Xinyu Yang, Xinyuan Li, Xuecheng Su, Xuheng Lin, X. Q. Li, Xiangyue Jin, Xiaojin Shen, Xiaosha Chen, Xiaowen Sun, Xiaoxiang Wang. Xinnan Song, Xinyi Zhou, Xianzu Wang, Xinxia Shan, Y. K. Li, Y. Q. Wang. Y. X. Wei, Yang Zhang, Yanhong Xu, Yao Li. Yao Zhao, Yaofeng Sun, Yaohui Wang, Yi Yu, Yichao Zhang, Yifan Shi, Yiliang Xiong, Ying He, Yishi Piao, Yisong Wang, Yixuan Tan, Yiyang Ma. Yiyuan Liu, Yongqiang Guo, Yuan Ou, Yuduan Wang, Yue Gong, Yuheng Zou, Yujia He, Yunfan Xiong, Yuxiang Luo, Yuxiang You, Yuxuan Liu, Yuyang Zhou, Y. X. Zhu, Yanhong Xu, Yanping Huang, Yaohui Li, Yi Zheng, Yuchen Zhu, Yunxian Ma, Ying Tang, Yukun Zha, Yuting Yan, Z. Z. Ren, Zehui Ren, Zhangli Sha, Zhe Fu, Zhean Xu, Zhenda Xie, Zhengyan Zhang, Zhewen Hao, Zhicheng Ma, Zhigang Yan, Zhiyu Wu, Zihui Gu, Zijia Zhu, Zijun Liu, Zilin Li, Ziwei Xie, Ziyang Song, Zizheng Pan, Zhen Huang, Zhipeng Xu, Zhongyu Zhang, and Zhen Zhang. Deepseek-rl: Incentivizing reasoning capability in llms via reinforcement learning, 2025.
[24] John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347. 2017.
[25] OpenAI. Gpt-4v, 2023.
[26] Leandro von Werra, Younes Belkada, Lewis Tunstall, Edward Beeching, Tristan Thrush, Nathan Lambert, Shengyi Huang, Kashif Rasul, and Quentin Gallouédec. Trl: Transformer reinforce- ment learning. https:/ ptttps:/gtti //github. cww www.///gggiggac/eeeeeeeoooeeeeee 2020.
12

---

[27] Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model is secretly a reward model. Advances in Neural Information Processing Systems, 36, 2024.
[28] Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Tiantian Fan, Gaohong Liu, Lingjun Liu, Xin Liu, et al. Dapo: An open-source Ilm reinforcement learning system at scale. arXiv preprint ar (rX.x:25033144777 2025.
[29] Zichen Liu, Changyu Chen, Wenjun Li, Penghui Qi, Tianyu Pang, Chao Du, Wee Sun Lee, and Min Lin. Understanding rl rl-zero-like training: A critical perspective. arXiv preprint arXiv:2503.20783 2025.
[30] Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoe V Le, Denny Zhou, et al. ofain-of-thoughe prompting elicits reasoning in large language models. Advances neural information processing systems, 35:24824 24837, 2022.
[31] Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. React: Synergizing reasoning and acting in language models. In International Conference on Learning Representations (ICLR), 2023.
[32] Zhengyuan Yang. Linjie Li, Jianfeng Wang. Kevin Lin, Ehsan Azarnasab, Faisal Ahmed, Zicheng Liu, Ce Liu, Michael Zeng, and Lijuan Wang. Mm-react: Prompting chatgpt for multimodal reasoning and action. arXiv preprint arXiv:2303./138/ 2023.
[33] Hao Bai, Yifei Zhou, Mert Cemri, Jiayi Pan, Alane Suhr, Sergey Levine, and Aviral Kumar. Digirl: Training in-the-wild device-control agents with autonomous reinforcement learning. arXiv preprint arXiv:2406. 1896, 2024.
[34] Zehan Qi, Xiao Liu, lat Long long, Hanyu Lai, Xueqiao Sun, Wenyi Zhao, Yu Yang, Xinyue Yang, Jiadai Sun, Shuntian Yao, al. Webrl: Training IIm web agents via self-evolving online curriculum reinforcement learning, arXiv preprint arXiv:241 1.04411..23,,,, 2024.
[35] Izzeddin Gur, Hiroki Furuta, Austin Huang, Mustafa Safdari, Yutaka Matsuo, Douglas Eck, and Aleksandra Faust. A real-world webagent with planning, long context understanding, and program synthesis. arXiv preprint arXiv:2307.1288. 2023.
[36] Kimi Team, Angang Du, Bofei Gao, Bowei Xing, Changjiu Jiang, Cheng Chen, Cheng Li, Chenjun Xiao, Chenzhuang Du, Chonghua Liao, et al. Kimi kl. 5: Scaling reinforcement learning with llms. arXiv preprint arXiv:2501.12599 2025.
[37] Zirui Zhao, Hanze Dong, Amrita Saha, Caiming Xiong, and Doyen Sahoo. Automatic curricu- lum expert iteration for reliable Ilm reasoning. arXiv preprint arXiv:4410.07627, 2024.
[38] Tingxu Han, Zhenting Wang, Chunrong Fang, Shiyu Zhao, Shiqing Ma, and Zhenyu Chen. Token-budget-aware 11m reasoning, 2025.
[39] Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang. Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, et al. Qwen2. 5-vl technical report. arXiv preprint arXiv:2502.13923, 2025.
[40] Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, and Ion Stoica. Efficient memory management for large lan- guage model serving with pagedattention. In Proceedings of the ACM SIGOPS 29th Symposium on Operating Systems Principles, 2023.
[41] Christopher Rawles, Alice Li, Daniel Rodriguez, Oriana Riva, and Timothy Lillicrap. Android in the wild: A large-scale dataset for android device control. arXiv preprint arXiv:2307. 10088, 2023.
[42] Yaowei Zheng, Richong Zhang, Junhao Zhang, Yanhan Ye, Zheyan Luo, Zhangchi Feng, and Yongqiang Ma. Llamafactory: Unified efficient fine-tuning of 100+ language models. arXiv preprint arXiv:2403. 2024.
13

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
The main claims made in the abstract and introduction accurately reflect the paper's contributions and scope.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification: Refer Section C.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model sell-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
14

---

Justification:
We do not have the theoretical result.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: Refer to Section for implementation. We will release the code and models when this paper is published.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility, In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
15

---

Answer: [NA]
Justification:
We will publish our code when the paper is published.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please the NeurlPS code and data submission guidelines (https: /ttpss////ipsppeeeeeeeeesseessss cc/
publ / guides
des CoodeSuomi ss issiooooll liyy for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https: //niss. cc/ cec/publice wwwp///gu///////eee//eee/sssssss /Codee///eeaa CodeS GrodeSubmii Submi ms ssiissiinn ciiinoooiicy for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: Refer to Section 5.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [No]
Justification: We fixed the random seed and selected the best performance.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, at least for the experiments that support main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
should be clear whether the error bar is the standard deviation the standard error of the mean.
16

---

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of enrors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: Refer Section 5.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g.. preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https: /neuripuupp..// htcp///blccccthhhscleeeeeeeee
Answer: [Yes]
Justification:
The research conform with the NeurlPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., if there is a special consid- eration due to laws or regulations their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification:
We discuss the broader impacts.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to
17

---

generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [Yes]
Justification: This paper describes safeguards that have been put in place for the responsible release of data or models that have a high risk.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided relcasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: We cite all papers that produced the code package or dataset.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g.. website). the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paper ixperstt swi thoode, com/ datasets has curated licenses for some datasets. Their licensing guide help determine the license of a dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New assets
18

---

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: The new assets introduced the paper are well documented and the documen-
tation is provided.
Guidelines:
The answer NA means that the paper does release new assets.
Researchers should communicate the details the dataset/code/model as part their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with
human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact core methodology, scientific rigorousness, originality of the research, declaration is not required.
19

---

Answer: [Yes]
Justification:
The paper describes the usage of LLMs.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non non-standard components.
Please refer to our LLM policy (https: / neurrps....... cc/ Conferences// / 2025/
LLM) for what should or should not described.
20

---

Appendix
Contents
Motivation Experiments
22
Rewards for Downstream Tasks
22
C Limitations
22
Broader Impact
22
E Dataset
22
F Setup
23
G Experiments
24
G.1 TON Math-GeoOL
24
G.2 TON on Counting-CLEVR
G.3 TON on Mobile Agent-AITZ
25
G.4 OOD Performance of TON on CLEVR.
26
G.5 Different Thought Dropout Probabilities
................................
27
G.6 Skip-thought Ratio on Different benchmarks
27
G.7 Prompt v.s. SFT on different benchmarks
28
G.8 Visualization Examples
29
G.9 Prompt for AITZ task
H Supplementary
32
H.1 Comprehensive Comparison of Length, Rewards, and Skip Ratio Across Three Benchmarks
32
H.2 Reward for length rather than SFT
32
H.3 More cases
33
21

---

A Motivation Experiments
Table presents the performance the agent VQA with and without the think source during the SFT stage, as well as with and without the think format in the GRPO reward function. Using the think source results in higher performance but longer output lengths, while excluding it leads lower performance with shorter outputs.
Table 5: Qwen2.5-VL-3B on the agent dataset (Android-in seeet--intth-zeee (oo) with/ without think before the answer by SFT and vanilla GRPO. acc is the test accuracy and len the output length of step-level.
SFT
GRPO
acc
len
acc
len
w think
0.67
181
0.74
88
wo think 0.39
29
0.58
29
B Rewards for Downstream Tasks
General VQA: r = ry Tu. where ry is the format reward and ry = if the response follows the think answer format, otherwise 0, ra is the concrete reward that if the predicted answer is equal to the ground truth number.
Agent VQA: r ry think answer format, otherwise O, ra the concrete reward that ra = the predicted action type equal to the ground truth action sypee.g..clice. predicted coordinates when the action type is click. In this paper, we use the normalized coordinates ranging from 0-1 and set 0 0.14 following [19].
+ Te. where ry is the format reward and ry if the response follows the
press _ome, Te is the continues reward for the
Math VQA: rf + ra, where ry the format reward and ry if the response follows the think answer format, otherwise 0, ra is the concrete reward that rd if the predicted answer is equal to the ground truth number.
C Limitations
Due to computational resources, our current work focuses smaller-sized visual-language models like 3B and 7B, the proposed method has not been evaluated on even larger models 235BB. We implement TON on the open-domain VLMMs; however, without access to the source code of proprictary VLMs like GPT-4o, the proposed method has not been implemented on them.
D Broader Impact
In this paper, propose simple yet effective method TON, cooperate SFT and RL stages by thought dropout. We improve the vanilla GRPO's performance by sampling minor code changes to teach the model reason during the RL exploration stage selectively. This enables deeper understanding of RL in VLMs, inspiring flexible injection prior knowledge into the SFT stage instead of manually creating rule-based rewards. For social impact, this work has certain impact on the RL research in the VLM and LLM.
E
Dataset
General VQA. The CLEVR dataset [15] designed generate complex multi-step questions based on synthetic images, assessing model's 's true reasoning ability. is diagnostic dataset that includes 100,000 rendered images and approximately one million automatically generated questions, which 853,000 are unique. The dataset features challenging questions involving counting, comparison, logical reasoning, and memory storage, while the images depict simple 3D shapes. contrast to the original CLEVR dataset, Kuper-CLEVE [16] introduces more complex visual components and
22

---

offers better control over the factors contributing to domain shift. For our experiments, we select subset of 1,000 datasets that contain only counting problems for training. We evaluate the model's performance on test sets by selecting 200 samples from CLEVR that were not seen in the training set, as well as 200 counting problems from the out-of-distribution Super-CLEVR dataset.
Math VQA. GeoQA [17] is a large-scale geometric question answering dataset that contains 4,998 geometric problems collected from real math exams in Chinese middle school. Each problem is accompanied by annotated programs illustrating the solution process. While this dataset features multiple-choie questions, we increase the difficulty in this paper by removing the answer choices and requiring the model to generate the answers directly. We select subset of Ik problems that involve computing angles and side lengths for training and test the model on this training set.
GUT Agent. AITZ [13] dataset designed for the graph user interface (GUT) navigation task derived from the large-scale mobile benchmark Android-in-the-wille (AITW [41]). It features a unique annotation called chain-of-action thought (CoAT), establishing a connection between the understanding of screen layouts and UI elements-ann and cognition, which involves action decision-makinn The AITZ dataset includes 2,504 operational trajectories that encompass 18.6K real-world intentions. Additionally, is categorized into five subsets based on application domains: General, Install, GoogleApps, Single, and WebShopping. We train the model using the General domain with dataset of randomly selected 1k examples and evaluate its performance on the corresponding test sets, as well as on other out-of-distribute domains.
perception
F
Setup
We use Llamafactory [42] for the SFT stage with full parameters, and the training time is no longer than 15 minutes for both Qwen2.5-VL-3B/77B models. We set 0 = 0.14 following [19]. We use vLLM [40] and the zerol perol_no_oppe _e__no__ppiie mo_optimizer GRPO settings to optimize further:
Table 6: Training Parameters for the first SFT of TON
Parameter
Value 2048
cutoff len
per der_ceevcee__ train batch size
siaiieet_accumulattinn sttppp
learning_ret rate
1.0e-5
Ir_ it_scheduler_tye type
cosine
warmup_rand
0.1
epoch
2
Table Training Parameters for the second GRPO stage of TON in general/agent
Parameter
Value
max_prompt_lenghi
4096
max max_completion__
2048
per_ per_device_traide Liree__rri_______ train_batch______ size
gradient_accumulation
2
learning_rat rate
le-6
Ir ir_scheduler_type
constant
bf16
true
0.04
attn inteimppllmenaiionn
flash attention 2
min_pixels
3136
max_pixels
501760
temperature
1.0
num num_ gene__eeetaaions
8
step
100
23

---

Table 8: Training Parameters for the second GRPO stage of TON in math
Parameter
Value
max_ wwtppptoooonnapepe
4096
max max_coopletionn lenggttionn_ngghhh
2048
per ver_device_trane batch_stcte size
1
steps
2
learning rate
le-6
Ir jr_scheduler_tye
constant
bfl6
truc
0.04
attn implementation
_aat__atteniio__ 2
min pixelixels
3136
max_pixels
501760
temperature
1.0
num generetienn
step
300
G Experiments
G.1 TON on Math-GeoQAA
Figure 8 & 9 illustrate the progression of various variables throughout the training process.
traintcomplttion length
0.4
200
0.3
trais/los
traininward
train
3b ratt
TOlLmat mato_mati_.e
6003
0004
0.1
(00
0.3
6.05
100
Figure 8: TON and GRPO visualization during the training process on Qwen2.5-ViL VL-BBB on GeoQA.
24

---

traiinccanlertion length
200
202
180
mathless
631
225
Figure 9: TON and GRPO visualization during the training process Qwen2.5-VL-7B on GeoQA.
G.2 TON on Counting-CLEVR
Figure 10 illustrates the progression of various variables throughout the training process.
traleieaants foraat reward Geeoo 36 toL do
toanncomolltionn length Lrtnt.ggreral
train/eeaardeacccryyy reword -coPtLamedd
310
100
0.00
tronghhte Vap
100
300
2.01
6085
Ww
Nr
6.00
Figure 10: TON and GRPO visualization during the training process Qwen2.5-VL-3BB on CLEVR.
G.3 TON on Mobile Agent-AITZ
Figure 11 & 12 illustrate the progression of various variables throughout the training process.
25

---

60
trainicomleeion langth
train
trainineaadd
GRR0
WV
6.00
6100
M
Figure 11: TON and GRPO visualization during the training process Qwen2.5-VL-33 on AITZ.
twih//maardi value rwward 6800_24.70 10
train/jesarttiffrrn reword
www.:eaanniriypee rwwaed
5800 74.75
1.903
0.99
3.
8.303
0.4
8303
Tollggooo stay
Toak Toln
crinstompletion leegth
300
30
30
20
1.30
60
Ma
82
100
303
200
300
Figure 12: TON and GRPO visualization during the training process on Qwen2.5-VL-7B on AITZ.
G.4 OOD Performance of TON CLEVR
Table compares the IID and OOD performance of TON and vanilla GRPO, TON demonstrates superior performance in both IID and, particularly, OOD scenarios under easy reasoning tasks, helping to avoid overfitting to the training set of vanilla GRPO.
26

---

Table 9: Qwen2.5-VL-3B on the IID domain CLEVR and OOD domain Super-CLEVR.
CLEVR acc
Super-CLEVR acc
think
base SFT
64
57.3
88.5
13.17
GRPO
93.5
51.9
TON
ours
98.5
62.79
G.5 Different Thought Dropout Probabilities
Figure 13 illustrates the progression of various variables throughout the training process under different dropout probabilities.
hype,
0.125
0.7
trainico wetpiooopplooooieeeeee
traivless 30.00.36
trann/rwward
2010
traeegooy ste
0.4
Figure 13: GRPO visualization during the training process Qwen2.5-VL-3B AITZ under dropout probabilities 20%, 50%, 80%.
G.6 Skip-thought Ratio Different benchmarks
Figure 21 illustrates the skip-thought ratios under TON throughout the training process under different VQA benchmarks.
1.0
3.0
0.02
699
1/
0.33
5kip Batto
0.7
SKB Balla
6.7
644--
062
0.55
03-
106
Chunk Steps
Chunk Steps
Chunk Steps
(a) Counting-CLEVE
(b) AITZ
(c) GeoQA
Figure 14: Skip Ratio of the output thinking during our TON training on three benchmarks.
27

---

G.7
Prompt v.s. SFT on different benchmarks
Figure 15 & & 17 illustrate the progression of various variables throughout the training process between injecting the skip-thought during the prompt and the SFT stage.
Toy, de
32
20
100
bank "On_zber_30
TOU, dh
3012
010
0.0
0.03
3008
3002
0.3
Figure 15: hybrid prompt SFT visualization during the training process Qwen2.5-VL-3B clevr.
twin://wwrrt/eauu reward
Tha n/tewards lormat rowserd
tran//eeerdi typa, rewward
Iebrid orcrer aut 30 CEPO and TON DA 36
Fylvvi poer SWOLELL L.
babrd prenst ESPO TOX
0.5
32
0.7
0.5
00
1335
trantsorphettnn length
franjem
traninwoard
vobrid.roopp PU TON ugpn
7/0rstprempt.ze
50
0.52
80
100
100
Figure 16: hybrid prompt v.s. SFT visualization during the training process on Qwen2.5-VL-3B on AITZ.
28

---

cainy Hormat FRoard
Braticoor plation JeaKEh GIND
TON
04
0.4
193
M
300
train
1,DNLSSPO, 6890 207
0433
0.2
0032
0.3
003
Figure 17: hybrid prompt v.s. SFT visualization during the training process on Qwen2.5-VL-3B on GeoQA.
G.8 Visualization Examples
Q: As shown in the figure, triangle ABC is the inscribed triang le of circle O, angle OAB 35.0, then the degree of angle ACB is ()
C
<thinks
given problem, trtangle
circle circle
given angle 35" inseribed
right pooperties ang subtended the conaider
directly dealing with semictecle rather
Since
center the circle, 04
radil
the circie, making trtangle ADB isoselles triangle with OA 0B. The angle at the center (angle ADB) twice the angle the circamferene (ungle ACI)), because the angle the camterr is twite the angle ei the cirzimfammma when they subtund the weme are, Given thei ungle UAS 35", and knowing that DA 08, angle OBA alsu 35", Therefore, angle ADB a 180" 35" 35 130". Since angle ACB is half angle AOB, angle ACB = 1/2 n10" 55". </think>
<answer>55 55 </answer>
Q: As shown in the figure, AB // CD, and EF intersects AB and CD at points E, F, angle 50.0, then the degree of angle is ()
ose the propartty shat cureepundingg anylos formeed bay inansverral snith two paraliel liner equal. Tharsfasa, angile equal anple bocauses they are cornespanding anglat- Glouss that 50 degroos, fullows anple
achiink siinn parailai
dagresss hum, sinse angll usseriur anaple criacogle two opposice incesion anqias, whlich also angle AF degress cattefy
Add cuuall analee
itstif- Jicneaat pincc Thersfoon,
angle airvadl depeaeoe
dagress
afininks
Task: Install the Facebook app
Task: Set
a alarm for 12pm
7:00
12:00
<thinkk
<ihinkke
The time set in the popup curvently wrong: it at 12:00 AM. Possible actions are to clidi on the "yot" indicator and thes adjust the minutes
The usser wannde destaii Pacebood But the susinani- eoneess- wwwwe--wwwweeeeee dhe wwwheee-----eeeeeeeeeeeeeeeeeeee saply selitch dusppedde incorrect entry. Rosetbll actionn
to
noenah cop type "arrbbonk appll and thon for-th. de-------------- conooci-the
12:00 collowed by clicking /lliikks
app"
inctaliation aathinis
<answers- acction 'value' None, "positionn: 10.51, 8.71 >//as//wserr>
<thinks BekiinkeAnnns/thiime wtwss/thinnee-nnseerrr 'action's 'TYPE', 'value': 'facebook app 'position's <one)
Figure 18: Examples of TON on math VQA and GUI agent VQA settings.
29

---

Counting examples from CLEVR [15]
Question
many items are there in the im- age?
How many items are in the image?
GRPO's outputs
<think>-The image shows three objects: large green cylinder from 1919, large sphere, and a large golden sphere. Both teal and golden spheres appear to be in a translucent state, indicating that they are metallic.
<think> count six distinct objects in the picture: large green matte cylinder, blue matte cylinder, a large green reflec- live sphere, large
malte cube, small blue matte cube, a light blue rubber object, a shiny silver cube, large reflective blue ob- ject, silver or dark green reflective sphere, shiny purple metallic cylinder, and a blue matte sphere. observe the reflective sur- face properties of the objects around the central large green sphere. II also note the size and color variety of each object as scribed by the visual cues, </think> <answer>> 10esser>0<//aswwr>>>>>
The green cylinder is plain and opaque, without any reflective properties. 1. Green Cylinder: Tran (metallic)
2. Teal Sphere: Transparent (metallic)
Golden Sphere: Transparent (metal- lie) The question seems to ask for the count of items, and considering that all three objects are distinguishable from each other as separate items in the im- age, the count is three. </think> <answer>?</answer
TON's oulputs
<think>innne/think>
<khink>Okay, let's check out. user provided a image with several objects and their properties. Each entry starts with "A" followed by attributes like size, color, ma- terial, shape, rotation, 3D coordinates, and pixel coordinates.
<answer>3</answer>
First, need to count how many items there Let me list each entry one by "A small purple rubher sphere..." That's the first item. 2. large yellow rubber cube..." " Second item. 3. "A small gray
rubber cube..." Third. 4. "A large green
rubber cylinder... Fourth. 5. "A small blue
metal cube..." Fifth. 6. "A small green
metal sphere..." Sixth. 7. "A large brown
metal cube..." Seventh. 8. "A small green rubber cylinder..." Eighth. Wait, that's eight distinct entries. Each unique combinations attributes. can't see any duplicates. Each has different values for color, size, material, rotation, or coordinates, so they're separate. So yes, there are eight items in total. </think> <answer> 8 </answer>
Table 10: Counting example from CLEVR [15]. Although the question is simple, the two con- ditioned images differ significantly in difficulty: the left image is clearly casier, while the right involves greater complexity due to object occlusion. TON demonstrates selective activation of reasoning- engaging thought only when needed-whereal GRPO generates reasoning traces for both cases indiscriminately.
30

---

G.9
Prompt for AITZ task
AITZ System Prompt
You are an assistant trained to navigate the mobile phone. Given a task instruction, screen observation, and an action history sequence, output the next action and wait for the observation.
Here the action space:
1. "CLICK': Click element, value is not applicable and the position [x.y] is required.
2. 'TYPE": Type a string into an element, value is string to type and the position not applicable.
3. 'SCROLL UP:: Scroll up for the screen,
4. 'SCROLL DOWN': Scroll down for the screen.
5. SCROLL LEFT': Scroll left for the screen.
6. 'SCROLL RIGHT': Scroll right for the screen.
7. PRESS BACK:: Press for returning the previous step, value and position not applicable.
8. 'PRSSS HOME': Press for returning to the home screen, value and position are not applicable.
9. 'RESS ENTER": Press for submitting the input content, value and position are not applicable.
10. 'STATUS TASK COMPLETE': Indicate the task is completed, value and position are not applicable.
Format the action dictionary with the following keys: ('action': "ACTION TYPE', 'value': 'element', 'position': [x.y]]
value position is not applicable, set as 'None'. Position represents the relative coordinates the screenshot and should be scaled to range of 0-1.
Think Prompt
Please first thinks about the reasoning process in the mind and then provides the user with the action. The reasoning process and answer are enclosed within <think> </think> and <action> </action> tags, respectively, i.e., <think> reasoning process here </think><action> action here </action>
Overall Prompt
(AITZ System Prompt]
(Think Prompt)
Task: (Task)
<image>
Action history: (Action history)
**xtxt **
31

---

H
Supplementary
H.1 Comprehensive Comparison of Length, Rewards, and Skip Ratio Across Three Benchmarks
We present a comprehensive comparison of length, rewards, and skip ratio across three benchmarks. The results reveal consistent trend: TON reduces completion length and increases the skip ratio as rewards increase during training.
trann//waarddcccurrry reward CCrPiggmmnal
train/rwwardkttppe reward
train//ewaadd/accurryyy reward
100 ort..
1000 moo.
26
2.5
201
58
24
33
26
36
32
24
531
train stap
train//cchal.cooo
tratn//fchal. step
20
200
20
40
60
208
20
23
208
(a) Counting-CLan CLEVR
(b) AITZ
(c) GeoQA
Figure 19: Rewards of the output during our TON training three benchmarks.
vaanicampler.om Jength
trainicampietcom on length
vrainicampiee Jength Garo.ynat
-Ci00_genorl
170
18
130
10
Track Jchull
traini k00p
Tradn Mllal
200
20
(a) Counting-CLEEN
(b) AITZ
(c) GeoQA
Figure 20: Completion length of the output during our TON training on three benchmarks.
3.0
1/
0.9
5kip Patte
6.75
Sdip Rate
0.23
0.55
64
0.62
64
0.55
64
Steps
Steps
Steps
(a) Counting-CLEV'
(b) AITZ
(c) GcoQA
Figure 21: Skip ratio of the output thinking during TON training three benchmarks.
H.2 Reward for length rather than SFT
We give the reward r = 1 for the model if it outputs <think>>llnn<tthinkk.. and otherwise O. We observe that the length reward remains at O during the first 100 steps. The visualization of the entire training process is shown below, highlighting our proposed thought dropout in the SFT stage.
32

---

MM
0.56
toln//ewardstypp.cm reward
train/oomplltion Jengtii
trllvirward
2.6
03
M
WM M
Figure 22: Length penalty rewards and completion length on AITZ datasets on Qwen2.5-VL-3B.
0.85
2.54
2.000
3.23
2.22
0.5
2.81
04
train
Figure 23: Length penalty rewards and completion length AITZ datasets Owen2.5-VLL3B
H.3
More cases
We give more cases to show the effectiveness and efficiency of our proposed TON.
33

---

Seggested
T
Query: Uninstall Messages.
<oleabo later whiid- cactisn Santice daictio
sbed://antibaakmmeeeeee (lccke)) 'alee: Masage, posiion?
-Al /actiee Taethen Manegys Clottoo
Soooe www.eeoeeeeeeeetter///e/e Newr, jontint 077))"ecne"*
justiss??
Nosj
Figure 24: TON applied to the AITZ -fi-triiiitiitttion domain: the task is to uninstall the messaging apps. TON performs well without extensive reasoning.
Messages
Query: Uninstall Messages.
Kalead later whiit cactiar' Caction
www.-rbaaaaaae--oeaoooo lactien? (0.25 6.33--action
ractle Tartkon puntice?
MLL 0.700)
Figure 25: TON applied to the AITZ disttribiitttnoonnn domain: the task is to uninstall the messaging apps. TON performs well without extensive reasoning.
34

---

Table 11: Counting example from SuperCLEVR [16]. Although the question is out-of-distriuttinn, the performance of TON and vanilla GRPO differs significantly their outputs. The output from vanilla GRPO is excessively lengthy (over 500 tokens), focusing more reasoning rather than providing direct answer. contrast, TON delivers a concise response, effectively bypassing the lengthy reasoning process.
Counting examples from SuperCLEVR [16]
Question
How many objects are either small green wagons or buses?
How many other objects are there of the same color as the matte car?
GRPO's outputs
"<think>, let's tackle this question step by step. The user asking how many objects are either šmall green wagon- sür buses. need to check each object in the image provided., let's parse the image. There are six objects listed: 1. A large blue rubber scooter with specific coordinates and rotation.2. A small yellow rubber tails.3. small brown rubber bicy- cle.4. A large red rubber tandem bicy- cle.5. small gray metal dirt bike.6. A large gray metal school bus. wws.stuttionn is about two categories: small green wag- ons and buses. Let's break down each wart.with Small green wagons? Looking at the list, there's 's a šmall yellow rubber carWhich is wagon. The color here is yellow, not green. So that doesn't 't fit. Are there any other small green wag- ons? The other objects are different colors and types. The only green oh- ject mentioned is the first one, but it's a scouter, not a wagon. So no small green wagons exist in the scene., buses? The sixth object is large gray metal school bus. Since the question says" "
<think>, let's see. The question ask- ing how many other ohjects are there of the same color as the matte car, First, I need to figure out which object is the matte caratt the image, each object has color mentioned. Let me check each en- try:1. A large yellow rubber SUV.... A small blue metal bicycle...3. A large pur- ple metal sedan...4. A large brown metal cylinder...5.
(wagon) with de-
large yellow metal school bus...6. small yellow metal bicycle...7. large purple metal se small yellow rubber dirt bike..... A large yel- low metal school bos A large purple metal articulated train..., the matte car isn't 't listed here. All the colors mentioned are either yellow, blue, purple, brown, or gray. The matte material isn't specified for any of them. Hmm, maybe matterefers material being non-shiny, but the given data, materials are either metal rub- ber. So perhaps the matte car is one the rubber ones. But the problem says matte, so maybe it's a mistake, or maybe the user intended fubberinstead of matte",
the
TON's outputs
<lhinks-innn/llinnk
<think>-nluc/hhink>
<answer>1 I</answe>>
<answer33</aswee>>
35

---

As shown in the figure, AB parallel CD, straight line EF intersects AB point E, intersects CD at point F, bisects angle BEF, and it intersects CD point G, angle 1 50.0, then angle 2 is equal to ()
TON:
D
GRPO:
<think>/atti solve for angle need to use the properties of parallel lines and angle bisectors. (n\nn, Since AB is is
parallel
angle BEF angle BFG supplementary angles linear pair-ina Given that angle is 50 degrees, and since EG bisects angle BEF, angle BEG must be half angle degrees. Since EG bisects angle BEF, angle BEG angle FBG same angle because they are corresponding angles
they
BER Therefore, degrees. (n5. Now, angle transversal EF intersecting the parallel lines AB and CD. Therefore, angle 2 = degrees. /n/ps//h////////////////// mnnnsweeerrr /n40\ns/answer>
BEG
by
Figure 26: Comparison between TON and vanilla GRPO in GeoQA
Q: As shown the figure, AB ia tangent to circle at point B, the extended Line of intersects circle 0 point Connect BC, if angle 36.0, then angle C is equal to
TON:
A
GRPO:
B
angle C, need properties circles and tangents, tntas **]dentify <think lnTo solve for the tangent
given information: the measure of angle Cn) **Use the property of the tangent:**11 Since AB is tangent to the circle at point B, the radius triangle OAB, the sum of angles is 180 degrees. In So, angle 0AB angle A angle OBA 150 degrees In degrees 180 degrees. Simplifying, get angle radius, triangle subtends the Therefore, angle DCB angle OBC.)nia5. **Determine angle COB:**|p The central the inscribed angle measure of the inscribed angle that subtends the same arc.in So, angle COB = 2 angle CBA. |nin6, **gelate angles in triangle OBC:Av In triangle OBC, the sum of the angles 180 degrees. angle
point B,jn Angle 36 degrees.In
find
perpendicular
Therefore, angle OBA go degrees.\ning.. **Deermmine angle OAB:arip In
degrees 54 degrees./n)n. **]enntif angle COB: isosceles with OB OC.In
Substituting the known values: angle OAB
radius
The
of the central angle twice
So, angle 0CB angle write: angle
angle OBC, degrees. in Substituting angle COB 2 angle CBA, we get: 2 angle OCB + 2 angle CBA 180 degrees. In Simplifying, get: angle OCB
degrees.
angle
angle
degrees 180 degrees. In This simplifies
angle OCB
degrees 180 degrees.la Solving for angle OCB, we get: 2 angle OCB 108 degrees. Therefore, angle OCB 54 degrees. nSjo, the measure of angle C is degrees. degrees.)ns//thnnks cannsaseer>> (154 (n54)///ans/// /n/answer>>
Figure 27: Comparison between TON and vanilla GRPO in GeoQA
36

---

As shown the figure, diameter of circle crosses the midpoint G chord EF, angle DCF 20.0, 0, then angle EOD is equal to
TON:
F
D
GRPO:
angle EOD, need properties circles and angles. (nini. **Identify <think> lnTo solve for the CD (1) is the diameter of the circle. in -V W) is the midpoint the chord the given information:**la
WV EF WD-1n W Wangle DCP 20*1]ccic (1)22 **Understand the implications of the midpoint: midpoint VV V0. V 10 perpendicular W VO. This means that ViC Vaangle go*Vicire W1mming. *sUse the inscribed angle t The angle subtended by an are at the center of the circle twice the angle subtended the
Since
GIG
Therefore, WM Wangle EOD (U (the central angle) twice 110 laangle ECF W (the inscrided in... *Determine the measure WV Wangle ECF (0 Since VV Vlangle DCP 20-l/eire WD) and \I( Vangle CGE gor)]eire 11), we can find (U
point the circumference..e
WI Wangle
Wiln Wangle
(\cree (ldre- 7cclin Whavas.
*Callullaee W Vangle EOD 10:**n Using the relationship between the central angle and the inscribed
angle:ia Wi' Vaanle EOD Vitmmes llangle ECP Wttimes Veire 1eirtt
WiaajaThus, the
measure angle VV BOD 1U VVC Vicre \U-loin</thinks www.://knnnannssnnanaa htwp://wwwwwtoot/////ss/eeee
Figure 28: Comparison between TON and vanilla GRPO in GcoQA
37

---

G. In the provided figure, the city government is making plans to construct an overpass with height AB measuring 6.0. If angle ACB represents the angle be tween the slope of the overpass and the ground BC, the sine angle AC
is equal to 3. 3.0/5.0, what the length slope AC?
B
length slope 3:4, we can set up a right triangle where the height (AB) is 6.0 unityd horizontal distance. Since the ratio
calculated using Pytha
.
this find the length of BC.
**Identify the ratio and relate to the sides trlangle:**
length
1) length
V-
(( AB !) units, the 3:4, we express !) (( Q):
(d) (4)
Solving M )):
(frc(4433) (times \text units)
*AAplly the Pythagorean theorem to find the hypatenuse V( AC 1):**
M
AC (sqrt(AB<2 BC*2)
V
Substituting the known values:
M
AC (sqrt(6.002 8.0*2)
V]
Calculate the squares:
M[
6.0^2 36.0 Iquad [text(and) lquad 8.0*2 64.0
V]
Add these values:
M
AC = (sgrt(36. 64.0) (sqrt(100.0) 10.0 \text( units)
V
Therefore, the length of the slope AC is 10.0 units. /ntis://hiinlooccmweeer///////
Figure 29: Comparison between TON and vanilla GRPO in GeoQA
38

---

In the provided figure, city government making plans to construct overpass with height AB measuring 6.0. If angle ACB represents the angle between the slope AC of the overpass and the ground BC, and the sine of angle ACB is equal to 3. 3.0/5.0, what is the length of slope AC?
B
GRPO
C
<think>
To solve for the length of the slope AC, we can use the given information about the slope and the vertical height. The slope AC is in the ratio 3.0:4.0, and the vertical height AB is 6.0 units.
1. "'dentify the components of the right triangle:"
The vertical height (AB) 6.0 units.
The ratio of the horizontal length (BC) to the vertical length (AB) is 3.0:4.0, so we can denote BC as V( \frra{3344) itimes AB \).
"Calculate BC:"*
1( BC Mrac(3X44 Wimes 6.0 = 4.5 !) units.
Use the Pythagorean theorem find AC:**
The Pythagorean theorem states I( AAC22 AB12 BO12 Substitute the known values: V AC*2 6.0º2 4.5*2 ().
((AC22 36.0 20.25 56.25 )). ((AC isqrt(56.22))) 25) 7.5 units.
Therefore, the length of the slope AC is 7.5 units.
</think>
<answer>
7.5
<lanswer>
Figure 30: Comparison between TON and vanilla GRPO in GeoQA
39


Reviews:
[{"review_id": "vnOQoWvH40", "reviewer": "Reviewer_WjY5", "summary": "This paper proposes a method to enable selective reasoning for VLMs. The\nproposed method works by using a simple \"random thought dropout\" approach during\nsupervised fine-tuning and incentivize \"selective thought dropout\" during RL\npost-training (e.g., GRPO). Experiments on models from 3B to 7B demonstrate the\neffectiveness of the proposed method.", "questions": "see weaknesses", "limitations": "yes", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "# Strengths\n\n* The studied problem is interesting. The motivation is clear.\n\n* The overall writing of this paper is good.\n\n* Both out-of-distribution data and in-distribution data is involved in the evaluation.\n\n# Weaknesses\n\n* The proposed thought dropout mechanism dring SFT is applied randomly without\nconsidering task difficulty or reasoning necessity. This could interfere with\nthe model’s reasoning capability and the ability to calibrate when reasoning is needed. It is suggested to\nadd more discussion about this potential drawback (although the RL stage might be able to fix, it could still introduce potential negative effects). More principled approaches,\nsuch as difficulty-aware or confidence-based dropout might be also helpful.\nThese adaptive approaches are not explored or discussed. In addition, the optimal dropout rate is likely to depend on the difficulty distribution of the SFT dataset. A discussion on how to align dropout rates with dataset characteristics is missing and would improve the soundness of the proposed method.\n\n* Although the scope of this paper is limited to VLMs, the method itself is\ngeneric and could easily be applied to text-only LLMs. However, there are many\nexisting works on adaptive reasoning on LLMs, such as those mentioned in the\nrelated work section, but none are included in the experimental comparison.\nThis limits the clarity of the paper’s contribution comparing to prior works.\n\n* All results are based solely on Qwen2.5-VL 3B and 7B. There is no evaluation\non other model families (e.g., InternVL3 and LLaMA 3.2 vision) or larger models.\nIt is unclear whether the method scales to stronger models or transfers across\ndifferent VLM architecturesa, which weakens the generality of the proposed\nmethod and the contribution. Is the injection of the thought dropout harder for\nlarger models as they have stronger reasoning capability and better inherent\ncalibration of when reasoning is needed?", "was_revised": true, "final_justification": "Please see the comments for the rebuttal.", "rebuttal": "> We appreciate your valuable feedback and are glad to hear that you found the selective reasoning problem interesting. We value this rebuttal opportunity to clarify issues and address your concerns.\n> \nPlease find our detailed responses to your specific questions below. For clarity, we use the following notations:\n\n- W – Weakness\n- Q - Question\n\n\n\n# **W1: Consider Thought Dropout with task difficulty.**\n\n---\n\nThank you for emphasizing the importance of task difficulty—this is indeed a crucial factor, we also value this most, we appreciate the opportunity to discuss it in greater depth.\n\n**1. How to measure Task Difficulty?**\n\n\n- **By Answer Correctness**: \n A more direct approach is to classify each sample based on *whether the base VLM answers it correctly*. We then drop thoughts for only the “easy” samples (those answered correctly by the base VLM). To investigate this, we implemented a difficulty-aware dropout strategy and compared its accuracy under our TON training framework.\n\n| Qwen2.5-VL-3b (GeoQA) | TON acc(↑) |\n| --- | --- |\n| Difficulty-aware dropout | 50.5 |\n| Random dropout (ours) | **51.0** |\n\nSomewhat surprisingly, the accuracy of the difficulty-aware dropout version is similar to, and even slightly lower than, our proposed random dropout method. This outcome may be attributed to the challenge of precisely defining “task difficulty”. Human-defined heuristics introduce potential noise and risk interfering with model behavior.\n\nTherefore, we adopt **a conservative and principled approach**: we treat all samples equally, allowing the model to acquire format-following skills during SFT and to self-explore and learn effectively during the GRPO stage. This strategy ensures that the model’s improvements are intrinsic and not confounded by external definitions or interventions.\n\n**2. Despite Random Dropout in SFT, the Model Adapts to Task Difficulty in the RL Stage**\n\nAs shown in Supplementary Section A.1, even with random thought dropout during SFT, our TON framework enables the model to adapt its skip-thinking behavior during GRPO based on task difficulty and reasoning complexity.\nWe analyze performance across a range of benchmarks—including 3D object counting, GUI-based mobile agent navigation, and geometric math reasoning—spanning a spectrum of task difficulties. Task difficulty is assessed both qualitatively (via intuitive reasoning complexity) and quantitatively (by base VLM zero-shot accuracy), grouped as: \n\n- 60-100 (easy)\n- 40-60 (medium)\n- 0-40 (hard). \n\nThe results of the final skip-ratios and our TON accuracy are as follows:\n\n| Types | zero-shot acc | skip ratio | TON acc(↑) |\n| --- | --- | --- | --- |\n| [easy] Counting | 64.0 | 98% | 98.5 |\n| [middle] AITZ | 58.0 | 79% | 74.0 |\n| [hard] GeoQA | 38.0 | 70% | 51.0 |\n\nOur findings indicate that our **TON maintains a high skip ratio for easy counting queries, allowing it to bypass unnecessary thinking processes, while keeping a low skip ratio for harder, more professional, knowledge-intensive questions to fully utilize its thinking capabilities.**\n\n**3. Possible future extension:**\nWe acknowledge that the skip-thinking mechanism may introduce the risk of the model bypassing necessary complex reasoning steps, and we plan to address it in future work.\n\n**(i)** a potential direction is to incorporate auxiliary score such as visual perplexity (PPL) scores as a proxy for estimating task difficulty. \n\n**(ii)** Another possibility is to evaluate whether the model can answer a question correctly both with and without the reasoning trace. If so, the reasoning may be redundant and thus safely skippable.\n\n# **W1: Discussion of dropout rates with dataset characteristics.**\n\n---\n\nWe have conducted a dataset-aware analysis of dropout behavior as part of our experimental investigation. Specifically,\n\n- [**Same dataset, different ratio**] In **lines 255-263**, we present the ablation study and insights into selecting dropout rates of 20%, 50%, and 80%. Our findings indicate **a correlation between the dropout ratio and the zero-shot performance across the datasets.**\n\n- [**Different dataset, different ratio**] In **Appendix G.5 and G.6**, we analyze the training curves of these dropout ratios across datasets of varying reasoning difficulties.\nwe observe that, **despite differing dropout ratios, TON consistently shows an increasing skip ratio as training progresses**. Notably, the lower dropout ratios demonstrate a rapid increase in skip rates, while the higher ratios remain relatively stable throughout training. TON can dynamically adjust according to reward signals—decreasing the dropout ratio when performance is high and increasing it when performance declines. Thus, we recommend starting with a lower dropout probability for further investigation, such as a 20% ratio for more difficult tasks (i.e., zero-shot accuracy < 40) and an 80% ratio for easier tasks (i.e., zero-shot accuracy > 60). \n\nWe thanks for your suggestions. These insights enhance our deeply understanding of TON, and we will highlight them in our revision. \n\n\n\n# **W2: Would be better to have Discussion between Adaptive reasoning methods in LLMs and TON.**\n\n---\n\nThanks for your suggestion and for mentioning our work with these adaptive reasoning methods.\n\nFirstly, it is necessary to clarify the key differences between our TON to prior adaptive reasoning methods. \n\n**(i) How to think *v.s.* When to think:** Adaptive reasoning methods assume **reasoning is always needed** but optimize for *shorter* paths. Our TON tackles a more fundamental problem: **whether to think or not.** Figure 4(d) shows that the reasoning length of TON for those unskipped questions is still similar to the vanilla GRPO.\n\n**(ii)** **Orthogonality and Compatibility.** TON’s *think-or-not* decision is **orthogonal** to length-adaptive methods. For instance, TON first determines whether reasoning is necessary; if it is, adaptive reasoning methods can then optimize the extent of that reasoning. This synergy holds great promise for future research.\n\n**(iii)** We further **include the representative adaptive reasoning method DAPO** [1], which apply the adaptive reasoning with dynamic sampling and the normalziation of the token numbers within the group; We apply it on VLMs for comparison on GeoQA, and the results are as below: Despite both variants yield signifcant improvement over original GRPO, our TON with minimal design, which reduces the length greatly while retaining the high accuracy.\n\n|  | acc(↑) | think len (↓) |\n| --- | --- | --- |\n| GRPO | 37.0 | 272 |\n| DAPO | 48.7 | 296 |\n| TON (ours) | 51.0 | 96 |\n\n[1] “DAPO: An open-source llm reinforcement learning system at scale.” arXiv preprint arXiv:2503.14476 (2025).\n\n\n# **W3: Would be better to apply TON on other or larger models.**\n\n---\n\nThanks for your suggestion. We add the experiments on **InternVL3-2B and 8B as follows**. Our TON can retain comparable accuracy while reducing the think length significantly, particularly, it reduces length further on a stronger model (8B), which **demonstrates the generalization of TON across different backbones.**\nDue to time limitations, more results on the bigger VLM, such as Qwen2.5-VL-32B, will be included in our next version.\n\n| Models | Metrics  | Zero-shot | GRPO | TON |\n| --- | --- | --- | --- | --- |\n| InternVL3-2b | acc(↑) | 28.7 | 31.9 | **32.8** |\n|  | think len(↓) | 1204 | 1275 | **364** |\n|  InternVL3-8b | acc(↑) | 45.5 | 65.1 | **65.4** |\n|  | think len(↓) | 750 | 724 | **341** |\n\n---\n\nWe appreciate your questions and comments very much. Please let us know for any further questions."}, {"review_id": "ty8mAobXzM", "reviewer": "Reviewer_vGCC", "summary": "Considering current vision-language model using RL to enhance reasoning ability causing redundant response, the author is motivated to enable models to learn selective reasoning—knowing when to reason and when to skip it—rather than indiscriminately generating reasoning traces, this paper proposes TON, a two-stage method: SFT+GRPO. In the first stage, the author teaches the model to skip reasoning appropriately by prompting model to self-generate reasoning trace given image, question, answer and later randomly replacing reasoning traces with empty “\\n\\n” using to construct dataset. In the second stage, the author designs discrete matching and continuous matching reward to optimize model’s selective reasoning ability. The primary contribution of this work lies in introducing “selective reasoning” as a trainable capability for vision-language models, significantly reducing redundancy in model responses.", "questions": "（1）Clarification of the Two-Stage Design Rationale (SFT + GRPO)\nThe paper proposes a two-stage framework for selective reasoning, beginning with SFT using thought dropout and followed by a GRPO-based reinforcement learning stage. However, the motivation for structuring the method in this specific two-stage manner is not fully explained. In particular, it remains unclear whether random dropout may lead to skip complex reasoning after SFT. Could the authors provide more conceptual reasoning or design analysis supporting this choice? Including additional ablation experiments comparing dropout strategies would help clarify this point. Addressing this question could significantly improve the paper’s Clarity score.\n\n（2）Generalizability of Reward Design in the GRPO Stage\nDoes the reward design satisfy the case when outcomes may not be easily judged by binary or discrete metrics? It may be more convincible to comment on the adaptability of reward design to other situation? For instance, would the current selective reasoning policy be viable in tasks lacking deterministic rewards? If the approach can be extended or generalized to such settings, it would enhance the broader applicability of selective reasoning and justify raising the Significance rating.\n\n（3）Deeper Analysis of “Without Think but Correct” Cases in Ablation\nIn the early-stage ablation where responses with and without “think” are compared, a significant proportion of cases are correctly answered without reasoning while being incorrect with it. This is intriguing but underexplored. Could the authors analyze these cases more deeply—are the errors in the “think” version due to hallucinated reasoning, misaligned chains of thought, or memory interference? Furthermore, after applying TON, how do such cases behave? Are they handled better by skipping reasoning, and if so, what insights can we draw about the model’s internal reasoning reliability? A targeted analysis would shed light on whether selective reasoning improves not just efficiency but also robustness. This could positively impact the Clarity score if addressed with empirical evidence or case studies.\n\n（4）The specificity of vision-language tasks compared to only language task\nThe proposed method does not appear to differentiate significantly between pure-text and vision-language scenarios. While the perspective of exploring reasoning length in vision-language tasks is novel, the method itself does not explicitly leverage the unique characteristics of vision-language reasoning. Could the authors consider incorporating task-specific aspects of vision, or provide a comparison that highlights how this approach differs from existing methods for reducing reasoning length in pure language models? Addressing this could support an increase in both the Significance and Originality scores.", "limitations": "Yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 2, "originality": 2, "strengths_and_weaknesses": "This paper presents a technically sound framework with comprehensive experiments across multiple tasks and model scales. The use of thought dropout and reinforcement learning is well-motivated and evaluated through thorough ablations and clear metrics. However, the construction of dataset by randomly dropout reasoning trace may lead to model skip some complex reasoning after SFT which can’t be optimized effectively by GRPO with task-specific two kinds of reward designs.\nThe paper is well-organized and clearly communicates its motivation and methodology. By implementing ablation experiments of with/without thinking on AITZ task, the paper introduces its motivation of selective reasoning. And the experiment part, especially the ablation of 3 questions well suits the intuitive consideration of the “think or not” mechanism. The only slight shortcoming lies in the implementation of selective reasoning — the rationale for adopting a two-stage approach requires further clarification.\nThe work addresses reducing reasoning overhead in vision-language models by reinforcing selective reasoning. This has practical value for efficient inference in real-world applications.\nThe idea of teaching models when to reason, rather than how, is a novel and insightful shift. Although similar ideas exist in adaptive computation, the integration into multimodal RL training is new and well-executed.", "was_revised": false, "final_justification": "", "rebuttal": "> We are very encouraged by your positive response! Below we have addressed your questions:\n> \n\nPlease find our detailed responses to your specific questions below. For clarity, we use the following notations:\n\n\n- W – Weakness\n- Q - Question\n\n\n\n\n# **Q1: Rationale of Two-stage.**\n\n---\n\nThe two-stage framework originated from our exploratory experiments in **Section 5.4--Emprical Verfication of SFT Significance in TON**. \n- We initially attempted to utilize only the GRPO stage without the preceding SFT stage: **(1)** *hybrid prompts where prompt the models to skip think if it is confidence enough* and **(2)** *Non-think rewards give reward 1 if it skip the think*. Unfortunately, these attempts did not induce skipping;  \n- Fig. 5d shows a skip ratio of 0, and the training curve in Appendix. G indicates that non-think rewards also remained at 0. We analyzed this phenomenon and attributed it to the *VLMs defaulting to a conservative, fully reasoning approach*, resulting in a nearly 100% think rate.\n\nThis motivated us to **inject the non-think prior during the SFT stage using thought dropout**, which we found necessary to break this bias toward full reasoning.\n\n# **Q1: Rationale of Random dropout.**\n\n---\n\nThank you for raising this question. Below, we outline the motivation and supporting analysis behind our use of random dropout for reasoning traces:\n\n**1. Random vs. Difficulty-aware Dropout**\nWe also experimented with a difficulty-aware dropout strategy, where reasoning is dropped only for \"easy\" samples (those correctly answered by the base VLM). The comparison on GeoQA is as follows:\n\n| Method                   | TON Accuracy (↑) |\n|--------------------------|------------------|\n| Difficulty-aware dropout | 50.5             |\n| Random dropout (ours)    | **51.0**         |\n\nSurprisingly, random dropout performs slightly better. This suggests that hand-crafted heuristics for task difficulty may *introduce noise or unintended bias, potentially interfering with the learning process*. **Random dropout offers a simpler, unbiased alternative that generalizes well across tasks.**\n\n**2. Adaptive Behavior in GRPO**\n\nDespite random dropout during SFT, our model learns to adaptively skip or retain reasoning in GRPO based on task difficulty. For example:\n\n| Task Type      | Zero-shot Acc | Skip Ratio | TON Acc (↑) |\n|----------------|----------------|-------------|--------------|\n| [Easy] Counting | 64.0           | 98%         | 98.5         |\n| [Medium] AITZ   | 58.0           | 79%         | 74.0         |\n| [Hard] GeoQA    | 38.0           | 70%         | 51.0         |\n\nTON maintains a high skip ratio for easy tasks while preserving reasoning for more complex queries, demonstrating that **the model internalizes task difficulty and adjusts its behavior accordingly—even without explicit difficulty labels in SFT.**\n\nIn summary, random dropout is a simple yet effective mechanism to encourage flexible, adaptive reasoning while avoiding the heuristic-based supervision. We will expand this discussion in the revised paper.\n\n\n\n\n# **Q2: Can TON be scalable to tasks without binary or deterministic rewards?**\n\n---\n\nTON primarily addresses **selective reasoning in VLMs** by introducing a generic thought-dropout method, due to its generic and concise method, it do not add any regularization on the reward definitions. Thus, the TON framework can still be adapted to uncertain environments in the following ways:\n\n1. **Adapting reward beyond binary/discrete metrics:** In such cases, we can still define formal reward instead of binary metrics, such as in Mobile navigation, regrading the action grounding, we could use the absolute L1 distance between click coordinate and box as continuous reward measurement.\n2. **For these tasks without deterministic rewards:** This being an open challenge in RL training, a potential solution could be training a critic model or adopting powerful models to predict rewards to supervise training. \n\nBut we want to emphasize that as TON does not introduce explicit reward regularization; thus, TON could be flexibly adapted to these scenarios. We would like to explore this extension in future work.\n\n# Q3: Deeper Analysis of think and How TON improve it\n\n---\n\nThanks for raising insightful question regarding the our finding. Follow your suggestion, we analyze 10 examples in Appendix G.8 and Supp. A.3 with the outputs of zero-short, grpo and our TON on 3 tasks.  We have the following obseravations:\n\n- **The model may generate hallucinated reasoning that leads to incorrect answers.**  \n   As shown in **Table 10**, even on simple 3D counting tasks (e.g., \"How many items are in the picture?\"), the model may produce meaningless or incorrect reasoning steps, ultimately resulting in wrong answers. \n\n- **TON exhibits greater robustness with respect to the presence or absence of reasoning.** As shown in **Figures 15 and 16**, TON is capable of providing correct answers even without an explicit reasoning process. We attribute this to two key factors:  \n\n**(i)** TON can selectively skip reasoning steps, thereby **reducing the likelihood of hallucinations or logical errors.**  \n\n**(ii)** For many VLMs, this suggests that **generating a correct answer along with a valid reasoning trace is more difficult than generating the answer alone.**\n\n\n\n**How TON improve this issue?**\n- **TON can skip the reasoning process to alleviate incorrect reasoning, allowing it to provide direct answers for easier questions.** In Table 5, models outputs a lengthy but incorrect reasoning process (~1k), which hinders the model's ability to answer correctly. In contrast, TON directly provides the correct answer with minimal reasoning.\n- Insights about the model’s internal reasoning reliability: **for simpler questions, skipping the reasoning process reduces the probability of errors, offering a more efficient pathway to correct answers.**\n\nThese analyses provide a deeper understanding of the behavior and advantages of our TON framework. We will incorporate this discussion into the revision.\n\n\n# Q4: Differentiate TON from existing length adaptive reasoning methods in LLM\n\n---\n\n    \nWe thank you for raising such insightful comments. \n\nFirstly, it is necessary to clarify the key differences between our TON to prior adaptive reasoning methods.\n\n(i) **How to think v.s. When to think**: Adaptive reasoning methods assume reasoning is **always needed but optimize for shorter paths**. Our TON tackles a more fundamental problem: **whether to think or not**. Figure 4(d) shows that the reasoning length of TON for those unskipped questions is still similar to the vanilla GRPO.\n\n(ii) **Orthogonality and Compatibility**. TON’s think-or-not decision is **orthogonal to** length-adaptive methods. For instance, TON first determines whether reasoning is necessary; if it is, adaptive reasoning methods can then optimize the extent of that reasoning. This synergy holds great promise for future research.\n    \nIn the future, we would like to include more visual-centric designs, such as extend our ‘dropout’ to visual inputs, let model to skip redundant visual tokens as well.\n\n---\n\nWe appreciate your questions and comments very much. Please let us know for any further questions."}, {"review_id": "Tvm6RenyZT", "reviewer": "Reviewer_g1TJ", "summary": "This paper introduces TON, a two-stage training framework for vision-language models (VLMs) that instills selective reasoning ability by combining supervised fine-tuning with “thought dropout” and reinforcement learning. TON, evaluated on multiple benchmarks, substantially reduces reasoning trace length, while sometimes improving accuracy over GRPO. The approach demonstrates strong out-of-distribution generalization.", "questions": "Could the authors clarify the statement in L217 regarding why directly applying GRPO leads to a zero coordinate reward? The explanation is unclear.", "limitations": "yes", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "[Strength]\n1. The idea to equip VLMs with selective reasoning is practical and well-motivated.\n2. TON achieves a significant reduction in average reasoning length without sacrificing performance, surpassing vanilla GRPO.\n3. The reward design with continuous matching is straightforward yet effective across benchmarks.\n\n\n[Weaknesses]\n1. Applying random dropout of reasoning traces during SFT may harm the model’s reasoning on complex queries, such as GeoQA, where reasoning steps are critical.\n2. This paper lacks a detailed analysis of which query types the model learns to skip reasoning for after training. Providing quantitative insights into these categories would clarify the model’s selectivity and failure modes.", "was_revised": true, "final_justification": "I maintain my BR rating. Using purely random dropout to decide whether to remove reasoning traces is unreasonable, since it may harm the model’s reasoning capability when the reasoning trace for a complex question is dropped. Besides, the provided evidence, showing that random dropout outperforms difficulty-aware dropout, comes from a single dataset and is neither comprehensive nor general enough to rule out bias. Without substantial and diverse experiments to prove this, the methodology remains flawed, and I cannot recommend it for acceptance.", "rebuttal": "> Thank you for recognizing the value of our work and for providing constructive and thorough feedback. We value your insights and hope our response addresses your comments.\n> \n\nPlease find our detailed responses to your specific questions below. For clarity, we use the following notations:\n\n- W – Weakness\n- Q - Question\n\n \n# **W1: Rationale for Random Dropout of Reasoning Traces**\n \n---\n\n\nThank you for your comments on the complex reasoning case, we also value this most. We appreciate the opportunity to discuss it in greater depth.\n\n**1. Does adding the reasoning difficulty supervision make it better?**\n\n\n\n- **By Answer Correctness**: \n We tried a direct approach to classify each sample's difficulty based on *whether the base VLM answers it correctly*. We then drop thoughts for only the “easy” samples (those answered correctly by the base VLM). To investigate this, we implemented a difficulty-aware dropout strategy and compared its accuracy under our TON training framework.\n\n| Qwen2.5-VL-3b (GeoQA) | TON acc(↑) |\n| --- | --- |\n| Difficulty-aware dropout | 50.5 |\n| Random dropout (ours) | **51.0** |\n\nSomewhat surprisingly, the accuracy of the difficulty-aware dropout version is similar to, and even slightly lower than, our proposed random dropout method. This outcome may be attributed to the challenge of precisely defining “task difficulty”. Human-defined heuristics introduce potential noise and risk interfering with model behavior.\n\nTherefore, we adopt **a conservative and principled approach**: we treat all samples equally, allowing the model to acquire format-following skills during SFT and to self-explore and learn effectively during the GRPO stage. This strategy ensures that the model’s improvements are intrinsic and not confounded by external definitions or interventions.\n\n**2. Despite Random Dropout in SFT, the Model Adapts to Task Difficulty in the RL Stage**\n\nAs shown in Supplementary Section A.1, even with random thought dropout during SFT, our TON framework enables the model to adapt its skip-thinking behavior during GRPO based on task difficulty and reasoning complexity.\nWe analyze performance across a range of benchmarks—including 3D object counting, GUI-based mobile agent navigation, and geometric math reasoning—spanning a spectrum of task difficulties. Task difficulty is assessed both qualitatively (via intuitive reasoning complexity) and quantitatively (by base VLM zero-shot accuracy), grouped as: \n\n- 60-100 (easy)\n- 40-60 (medium)\n- 0-40 (hard). \n\nThe results of the final skip-ratios and our TON accuracy are as follows:\n\n| Types | zero-shot acc | skip ratio | TON acc(↑) |\n| --- | --- | --- | --- |\n| [easy] Counting | 64.0 | 98% | 98.5 |\n| [middle] AITZ | 58.0 | 79% | 74.0 |\n| [hard] GeoQA | 38.0 | 70% | 51.0 |\n\nOur findings indicate that our **TON maintains a high skip ratio for easy counting queries, allowing it to bypass unnecessary thinking processes, while keeping a low skip ratio for harder, more professional, knowledge-intensive questions to fully utilize its thinking capabilities.**\n\n**3. Possible future extension:**\nWe acknowledge that the skip-thinking mechanism may introduce the risk of the model bypassing necessary complex reasoning steps, and we plan to address it in future work.\n**(i)** A potential direction is to incorporate auxiliary scores such as visual perplexity (PPL) scores as a proxy for estimating task difficulty. \n**(ii)** Another possibility is to evaluate whether the model can answer a question correctly both with and without the reasoning trace. If so, the reasoning may be redundant and thus safely skippable.\n\n\n# **W2: Would be better to provide quantitative insights into the skip ratios and query types.**\n\n---\n\nThank you for your valuable suggestion. To address this, we categorize query types based on their originating benchmarks. For example, we identify query types such as:\n\n- **Counting** (e.g., \"How many [objects]?\")  \n- **Mobile Agent** (e.g., \"Download a music app\")  \n- **Math Reasoning** (e.g., \"What is the degree of the angle?\")\n\nThese query types naturally differ in their reasoning complexity. To quantify this, we assess task difficulty using the base VLM’s accuracy in a zero-shot setting:\n\n- **Easy**: 60–100%\n- **Medium**: 40–60%\n- **Hard**: 0–40%\n\nWe then analyze the model’s skip-thinking behavior during the GRPO stage and observe how the skip ratio correlates with query type and difficulty. \n\n| Types | zero-shot acc | skip ratio | TON acc(↑) |\n| --- | --- | --- | --- |\n| [easy] Counting | 64.0 | 98% | 98.5 |\n| [middle] AITZ | 58.0 | 79% | 74.0 |\n| [hard] GeoQA | 38.0 | 70% | 51.0 |\n\nOur findings indicate that our **TON maintains a high skip ratio for easy counting queries, allowing it to bypass unnecessary thinking processes, while keeping a low skip ratio for harder, more professional, knowledge-intensive questions to fully utilize its thinking capabilities.**\n\n# **Q1: Why does directly applying GRPO lead to a zero coordinate reward in L217**\n\n---\n\nThere is because in AITZ benchmark, we require the model to output well structural action format, must adhere to JSON (lines 222-224),`{action: 'ACTION_TYPE', value: 'ELEMENT', position: '[x,y]'}`, which is more complex than the math problems with a numerical answer (e.g., 5). \n\nWe find that open-source models without SFT format following cannot directly follow such complex instructions, making it difficult to extract answers and determine consistency with the ground truth, resulting in large failures. While RL stage is not good for format following and is better for generalization, thus without SFT, direclty GRPO cannot obtain reward to support training.\n\n---\n\nWe appreciate your questions and comments very much. Please let us know for any further questions."}, {"review_id": "R2CMOOAE0P", "reviewer": "Reviewer_bSRJ", "summary": "This paper studies whether the model should conduct chain-of-thought reasoning or not in the multimodal LLM. The idea is quite simple and straightforward: replacing the thinking process with an empty thinking token. The motivation is the author found that in some cases, the thinking process might hurt the performance. Especially there are a high percentage where the model answered the questions incorrectly with the thinking mode on, but it can answer correctly without thinking mode. This motivated the author to study the thinking dropout approach. This is activated in the SFT stage. The GRPO algorithm is also updated with the thinking dropout process. This simple approach reduces the completion length by up to 90% and achieves quite good performance on counting (CLEVR) and Math(GeoQA) tasks.", "questions": "This is a good paper. I have several questions:\n\n1. For the GRPO part, I didn't fully grasp how to update the GRPO algorithm to adapt the change of thinking dropout. Specifically, since the format reward is being added, how to handle the format reward for the sequence without thinking?\n2. This approach should also be applicable to the text only LLM. I wonder what is the reason for studying this in the MLLM domain? Why not also studying this in the text only LLM domain and observe the performance improvement?\n3. For the reverse thinking, how to verify the correctness of the thinking process?\n4. An naive question for L141, why can't we just use the third party (closed source APIs) to generate the thinking process as the starting point?", "limitations": "Yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 4, "significance": 4, "originality": 4, "strengths_and_weaknesses": "Strength:\n1. The paper is well written. It is very easy to follow the whole paper. It is actually a joy to read this paper.\n2. The proposed approach is very well motivated. Especially the Figure 2, which demonstrates the necessity for enabling the thinking dropout approach.\n3. The approach is quite simple and straightforward. Frankly speaking, it just several line changes in the data processors. I am quite surprising about the simplicity of this approach and eager to try it out by myself.\n4. The proposed approach achieves quite good performance in both efficiency and accuracy.\n\nWeakness:\nFrankly speaking, i didn't observe significant weakness of this work. The only thing that can improve is the clarity for how to update the GRPO. But the rest looks quite good to me. I would love to see this paper being accepted to this conference. It would be great if the author could also release the implementation.", "was_revised": false, "final_justification": "", "rebuttal": "> We are very encouraged by your positive feedback and your recognizing the value of our proposed method! Below we address your concerns:\n> \nPlease find our detailed responses to your specific questions below. For clarity, we use the following notations:\n\n- W – Weakness\n- Q - Question \n\n\n\n\n# W1: Would be great if the author could also release the implementation.\n\n---\n\nCertainly! We value the impact of open-sourcing and are committed to releasing our models, code, and data to benefit the community.\n\n# Q1: How to handle the format reward for the sequence without thinking?\n\n---\n\nGreat question! Format specification is an important factor that influences model behavior. In our approach, we standardize both “thinking” and “non-thinking” modes into a unified format:\n\n`<think>reasoning process</think><answer>predicted answer</answer>`\n\n- For the **non-thinking format**, we use the structure `<think>\\n\\n</think>`, where the content inside `<think>` is replaced with `\\n\\n`.\n- In contrast, the **thinking format** retains the structure `<think>reasoning process</think>`, where the content explicitly represents the model’s reasoning process.\n\nThis unified formulation enables supervised fine-tuning (SFT) to effectively incorporate thought dropout with minimal changes. Furthermore, it facilitates the transition to the GRPO stage by allowing reuse of the same format.\n\n\n# Q2: Why study this in the MLLM domain? Can we extend this in the LLM domain?\n\n---\n\nThank you for your suggestion regarding extending our study to the LLM domain.\n    \n**Why we choice MLLM as a primary testbed:**\n- We believe that vision tasks are more intuitive and can often be resolved at a glance. In contrast, text-based tasks typically provide more comprehensive or professional information. Visual tasks tend to be more redundant; often, **a quick glance is sufficient to determine whether further thought is necessary**, akin to human cognition. There is why we choice MLLM to study this problem.\n\n**Extension of TON to the LLM Domain:**\nWe appreciate your suggestion to extend our study to the LLM domain. In response, we have adapted our TON framework to the LLM benchmark GSM8K. Below, we present the corresponding experiments and results.\n\nThe findings indicate that TON significantly reduces response length while maintaining high accuracy. This demonstrates the **generalizability of TON across different modalities.**\n\nWe will include the experimental setup and incorporate further discussion on this extension in our revised submission.\n\n\n| GSM8K | Pass@1 (↑) | Avg. Think len (↓) |\n| --- | --- | --- |\n| Qwen2.5-1.5B-distill-deepseek-r1 | 78.7 | 939 |\n| grpo | 81.4 | 978 |\n| ton | **82.3** | **342** |\n\n# **Q3: For the reverse thinking, how to verify the correctness of the thinking process?**\n\n---\n\nGood question! An intuitive approach to verifying the correctness of a thinking process is to *prepend it as a prefix and observe whether it enables the base model to generate the correct answer*. If the model succeeds under this condition, we can treat the thinking process as valid.\n\nHowever, based on our investigation, **the primary objective of the SFT stage is to instill format-following behavior**—such as the ability to either generate a complete reasoning trace or omit it entirely. Our intention is **not to intervene in the model’s behavior through external verification or enforcement**. This is why we adopt a reverse thinking strategy derived from the base model itself, without introducing a more powerful auxiliary model.\n\nWe aim for the base model to self-explore during the GRPO stage, allowing us to observe the genuine gains achieved without relying on external guidance or supervision.\n\n\n# **Q4: Why not use third-party (closed-source APIs) to generate the thinking process as the starting point?**\n\n---\n\nYes, this is certainly possible. One could use third-party closed-source APIs to generate the thinking process simply by replacing the SFT training corpus with externally generated reasoning traces.\n\nHowever, **our focus is on enabling the model to develop its own reasoning abilities without external assistance**. By doing so, we ensure that *any improvements observed during the GRPO stage stem from the model itself, rather than being distilled from a stronger external model.* This setup allows for a clearer evaluation of the model's intrinsic capacity for reasoning and self-improvement.\n\n---\n\nWe appreciate your questions and comments very much. Please let us know for any further questions."}, {"review_id": "8jPSPRG89E", "reviewer": "Reviewer_PYs6", "summary": "This paper introduces an adaptive training framework that lets Vision-Language Models flexibly produce either short or long chains of thought (CoT). The method uses a two-stage pipeline: **(1) supervised fine-tuning (SFT)** to establish two basic reasoning patterns followed by **(2) guided reinforcement preference optimization (GRPO)** to adjust CoT length to task demands. Experiments show that the framework simultaneously shortens unnecessary reasoning steps and improves overall performance.", "questions": "**Polished reviewer questions (with citations added)**\n\n1. **Scope of application.**\n   The framework is tested only on VLM reasoning tasks, yet recent length-adaptive methods in *text-only* settings—e.g., **ThinkLess** and **LASER**—report that LLM CoT traces can easily reach thousands of tokens, making compression arguably more urgent there. Could the authors clarify why the vision-language modality was chosen as the primary testbed, and whether the approach generalises to purely textual reasoning? ([arxiv.org][1], [arxiv.org][2])\n\n2. **Source of performance gains.**\n   The stated goal is to remove redundant reasoning steps; in principle this should *preserve* accuracy, not improve it. What specific mechanisms (e.g., better credit assignment in GRPO, reduced exposure bias) lead to the observed accuracy gains? Please provide ablations that disentangle efficiency improvements from accuracy improvements.\n\n3. **Positioning against related work.**\n   Please situate the contribution with respect to existing length-adaptive approaches such as **ThinkLess**, **LASER**, **AdaCoT**, and other “think-less” / difficulty-aware reward-shaping methods. What is the key novelty—reward design, the two-stage SFT + GRPO pipeline, or the adaptation to VLMs? A concise related-work discussion with explicit citations would help readers see the incremental contribution. ([1，2])\n\n[1]: \"ThinkLess: A Training-Free Inference-Efficient Method for Reducing Reasoning Redundancy\"\n[2]: \"Learn to Reason Efficiently with Adaptive Length-based Reward Shaping\"", "limitations": "yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 2, "originality": 3, "strengths_and_weaknesses": "**Strengths**\n\n* Delivers strong performance on all reported benchmarks.\n\n**Weaknesses**\n\n* The motivation for “over-dense” chains of thought is not fully substantiated. The evaluated tasks never exceed ≈ 1 k tokens of reasoning, so it is unclear why such CoTs are considered excessively long.", "was_revised": true, "final_justification": "The rebuttal addressed most of my concerns effectively. Consequently, I have revised my rating upwards to a 4.", "rebuttal": "> We appreciate that you found our method to have strong performance. We value your insights and hope our response addresses your comments.\n> \n\nPlease find our detailed responses to your specific questions below. For clarity, we use the following notations:\n\n- W – Weakness\n- Q - Question \n\n\n# **W1: The motivation for “over-dense” chains of thought is not fully substantiated.**\n\n---\n\nWe would like to clarify that our motivation is **not** primarily aimed at optimizing over-dense chains of thought (i.e., making reasoning more efficient), but rather at addressing a more fundamental question:  \n**\"When should the model think?\"** — i.e., identifying whether explicit reasoning is necessary.\n\nAs shown in **Figure 2**, we conduct our motivation study on the AITZ benchmark, which does not exhibit excessively long CoT traces. Despite this, we observe that a non-trivial portion of cases (7.6%) still fail. More notably, **52.1% of the answers remain correct even without any reasoning**, and **14.5% are correct only when reasoning is skipped**. This implies that explicit reasoning is **not always required** for optimal performance.\nFurther evidence and expanded analysis supporting this motivation are provided in **Appendix A**.\n\nWe appreciate your insightful comment and agree that applying our framework to settings with long and potentially over-dense CoT is a valuable future direction. We plan to explore this in subsequent work.\n\n# **Q1: Why choose multimodal as the testbed? Can TON transfer to LLMs?**\n\n---\n\n- We believe that vision tasks are more intuitive and can often be resolved at a glance, so it would be a good testbed to study think-or-not, while language-centric benchmarks are more focused on knowledge, which makes them difficult to articulate in detail. In contrast, text-based tasks typically provide more comprehensive information, thus would be better for long-cot. Visual tasks tend to be more redundant; often, a quick glance is sufficient to determine whether further thought is necessary, akin to human cognition.\n- TON is being generic, and we adapt our TON to the text-only datasets based on LLMs. Here, we give the experiments on the text-only tasks, and the results are as below. The results show that our TON reduces the length greatly while retaining the high accuracy. We have included these experiment settings in our revision.\n\n| gsm8k | pass@1 (↑) | think len (↓) |\n| --- | --- | --- |\n| Qwen2.5-1.5B-distill-deepseek-r1 | 78.7 | 939 |\n| grpo | 81.4 | 978 |\n| ton | **82.3** | **342** |\n\n\n# **Q2: What specific mechanisms of TON lead to accuracy gains? Considering that redundant reasoning, in principle, should preserve accuracy**\n\n---\n\n\n**First**, we challenge the assumption that *redundant reasoning steps should preserve accuracy*. This is **not always true**. In fact, *saying more does not always mean saying it correctly*. As demonstrated in **Table 6** and **Figures 15 and 16**, lengthy but flawed reasoning processes can lead to incorrect answers by increasing the likelihood of hallucinations or logical errors.\n\n**Second**, we attribute our TON' effictiveness to two reasons.\n\n- **(i) TON effectively skips incorrect or redundant thinking processes, alleivating hallucinations.** **Fig. 2** shows that 52.1% of answers remained correct without the \"think\" process, and 14.5% were only correct when \"think\" was omitted. This suggests that explicit reasoning is not always necessary. **Sup. A.3** gives examples on GeoQA where the reasoning processes are either incorrect or redundant, leading to wrong answers. Our TON can skip these misleading reasoning steps and output answers directly and automatically.\n- **(ii) Introducing the non-think enlarge diversity on the distribution space of responses when GRPO sampling. Fig. 3** illustrates that TON can introduce empty thinking into the GRPO algorithm, thereby increasing response diversity within each group. Unlike previous works like DAPO emphasizes advantage distribution $A_i$ by dynamic sampling in the sparse reward space, our TON shifts the focus to the latent distribution space of responses $\\pi_\\theta(o_i|v,q)$, thus enhancing the diversity of the terms $\\alpha, \\beta$ in equation 4.\n\nTogether, these mechanisms explain how TON improves accuracy—**not by blindly preserving all reasoning, but by selectively reasoning only when it adds value.**\n\n\n# **Q3: What is the novelty of TON compared to length-adapted related works?**\n\n---\n\n**(i) TON v.s. length-adapted methods:** Length-adapted methods assume **reasoning is always needed** but optimize for shorter paths. Our TON tackles a more fundamental problem: **whether to think or not.** Figure 4(d) shows that the reasoning length of TON is similar to the vanilla GRPO.\n\n**(ii)** **Orthogonality and Compatibility.** TON’s *think-or-not* decision is **orthogonal** to length-adaptive methods. For instance, TON first determines whether reasoning is necessary; if it is, adaptive reasoning methods can then optimize the extent of that reasoning. This synergy holds great promise for future research.\n\n\n(iii) Lastly, we want to friendly remind that the works you suggested: Thinkless (21 May 2025), LASER (21 May 2025), AdaCoT (17 May 2025) mentioned above **were released after the NeurIPS2025 full paper submission (15 May 2025) deadline.** meaning that we are concurrent works. We would love to add a discussion of length-adaptive methods in related works in the revision.\n\n---\nWe appreciate your questions and comments very much. Please let us know for any further questions."}]

Author rebuttal:

Meta-review: This paper studies the issue of long CoT preference for large VLMs, which wastes computation and increases latency. This work further proposed a two-stage training framework combining with SFT and GRPO. The main novel designs are (1) SFT with random thought dropout and (2) selective reasoning reward to avoid unnecessary reasoning. Evaluations are conducted across diverse vision-language tasks under both 3B and 7B models, validating the effectiveness of the proposed method.

The main strengths are (1) the problem and solution to control when to think for VLMs are clear and practical, (2) the empirical gains in both efficiency and accuracy is significant, (3) the paper is well-written and easy to follow. The major weaknesses and concerns are (1) random thought dropout may hurt complex reasoning and the solution should be principled, (2) the comparisons to ThinkLess/LASER/AdaCoT and for text-only evaluations should be clearer, (3) the clarity on GRPO rewards & training strategy needs to be improved, (4) the sensitivity of drop-out rate needs forther ablations.

After rebuttal and discussions, most of the concerns are well addresses. The remaining concerns are that (1) difficulty-aware dropout ablation performs worse, (2) empirical results are limited to Qwen2.5-VL-3b with GeoQA. AC also checked authors' final remarks on the concerns, and believe that these concern are not major reasons for rejecting the paper. For difficulty-aware dropout ablation performs worse, AC agreed with authors that it is not core contribution of this paper but a complimentary ablation to analyze heuristic-free random dropout. For model&dataset selection, this paper also conducted experiments with Qwen2.5-VL-7b and CLEVR. AC didn't find strong evidence to support reviewers' claim on this point. Considering the overall quality of the paper and the concerns are well addressed, AC recommended accepting this paper.
Decision: Accept (poster)