To Think or Not To Think: A Study of Thinking in Rule-Based Visual Reinforcement Fine-Tuning
Authors: Ming Li
Jike Zhong
Shitian Zhao
Yuxiang Lai
Haoquan Zhang
Wang Bill Zhu
Kaipeng Zhang
Venue: NeurIPS 2025 spotlight ()
Keywords: Visual Reinforcement Fine-Tuning
explicit thinking
overthinking
Primary area: applications
TL;DR: We study the thinking process in visual reinforcement finetuning

Abstract: This paper investigates the role of explicit thinking process in rule-based reinforcement fine-tuning (RFT) for multi-modal large language models (MLLMs). We first extend \textit{Thinking-RFT} to image classification task, using verifiable rewards for fine-tuning~(FT). Experiments show {Thinking-RFT} significantly outperforms supervised FT and yields a cross-dataset generalization effect. We then rethink and question whether explicit thinking in RFT is always necessary and beneficial. Challenging the convention that explicit thinking is crucial for the success of RFT, we introduce \textit{No-Thinking-RFT}, exploring RFT without thinking by introducing a simple equality accuracy reward. We evaluate No-Thinking-RFT on six diverse tasks across different model sizes and types. Experiment results reveal four key findings: \textbf{(1).} Visual perception tasks do not require thinking during RFT, as No-Thinking-RFT consistently outperforms or matches Thinking-RFT across model sizes and types. \textbf{(2).} Models with limited capabilities struggle to generate high-quality CoT for RFT, making Thinking-RFT less effective than No-Thinking-RFT. \textbf{(3).} There are inconsistencies between the answers in the thinking tags and answer tags for some responses of Thinking-RFT, which show lower average accuracy than the overall accuracy. \textbf{(4).} The performance gain of No-Thinking-RFT mainly stems from improved learning during no thinking FT and the avoidance of inference overthinking, as evidenced by the partial gains from appending empty thinking tags at inference time of Thinking-RFT. We hypothesize that explicit thinking before verifiable answers may hinder reward convergence and reduce performance in certain scenarios. To test this, we propose \textit{Think-After-Answer}, which places thinking after the answer to mitigate this effect for experimental verification. Lastly, we conduct a pilot study to explore whether MLLMs can learn when to think during RFT, introducing an \textit{Adaptive-Thinking} method. Experiments show that model converges to either thinking or not depending on model capability, achieving comparable or better performance than both Thinking and No-Thinking-RFT. Our findings suggest MLLMs can adaptively decide to think or not based on their capabilities and task complexity, offering insights into the thinking process in RFT.

To Think or Not To Think: A Study of Thinking in Rule-Based Visual Reinforcement Fine-Tuning
Ming Li"
Jike Zhong
Shitian Zhao'
Yuxiang Lai
Haoquan Zhang
Wang Bill Zhu
Kaipeng Zhang!*
Shanghai AI Laboratory
University of Southern California
Emory University
Chinese University of Hong Kong
1m164036216 mtw.:/362211@mmii...om
zhangkaipengg@pjlab.comm org.cc
Project code:
Abstract
This paper investigates the role of explicit thinking process in rule-based reinforce- ment fine-tuning (RFT) for multi-modal large language models (MLLMs), We first extend Thinking-RIT to image classification task, using verifiable rewards for fine-tuning (FT). Experiments show Thinking-RRe significantly outperforms supervised FT and yields a cross-dataset generalization effect. We then rethink and question whether explicit thinking in RFT is always necessary and beneficial. Challenging the convention that explicit thinking is crucial for the success of RFT, introduce No-Thinking-BR RFF,, exploring RFT without thinking by introducing simple equality accuracy reward. We evaluate No-Thinking-Be RFT on six diverse tasks across different model sizes and types. Experiment results reveal four key findings: (1). Visual perception tasks do not require thinking during RFT, as No- Thinking-RIi RFF consistently outperforms or matches Thinking-RFT across model sizes and types. (2). Models with limited capabilities struggle to generate high- quality CoT for RFT, making Thinking-RFT less effective than No-Thinking-Re (3). There are inconsistencies between the answers in the thinking tags and answer tags for some responses of Thinking RFT, which show lower average accuracy than the overall accuracy. (4). The performance gain of No- No-TTinkiing-Be RFT mainly stems from improved learning during no thinking FT and the avoidance of inference overthinking, as evidenced by the partial gains from appending empty thinking tags inference time of Thinking -FF. We hypothesize that explicit thinking before verifiable answers may hinder reward convergence and reduce performance in cer- which places thinking
tain scenarios. To test this, we propose Think-Afler-Answer after the answer to mitigate this effect for experimental verification. Lastly, we con- duct a pilot study to explore whether MILMS can learn when to think during RFT, introducing an Silaptive-Thnnkiee to either thinking or not depending on model capability, achieving comparable better performance than both Thinking and No- fo-Thinking- RFT. Our findings suggest MILMs can adaptively decide to think or not based on their capabilities and task complexity, offering insights into the thinking process in RFT.
method. Experiments show that model converges
1 Introduction
Recently, rule-based reinforcement fine-tuning (RFT) has made significant progress and achieved better performance than traditional supervised fine-tuning (SFT) 19 68 26 RFT leverages verifiable
*Corresponding Author: zhangkaipeng @ pjlab.org.cn
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Response Length
Accuracy Reward
Teed-211
T0XCAmm Feeeo21
CalechiEl
Dear www.taatteeee
0.5
100
0.5
80
0.4
60
40
Epoch
(a) Response Length Curves
(b) Accuracy Reward Curves
Figure I: Thinking is less essential: Classification RFT exhibits sharp drops in response length at specific steps, accompanied by significant rises in accuracy reward.
rewards for training, encouraging models to engage in a thinking process before answering for solution exploration [19]. The explicit thinking is widely believed as a key factor in RFT's success, and many works multi-modal RFT
82] 24 aim reproduce the length-increasing and 'aha moment' effects seen in Deepseek-RI 19). However, a critical question emerges as RFT being widely used: Is explicit
thinking always necessary and beneficial for small-sizee cooll-sized-modee RFT? Recent studies (2 64 65
suggest that reasoning offers limited gains on commonsense tasks and overthinking may even harm inference performance. While these findings offer insight into model reasoning, they focus solely on inference, leaving the impact of explicit thinking during RFT unclear and largely unexplored. Furthermore, RFT typically requires significantly more fine-tuning time and GPU memory than SFT due to the generation of multiple lengthy responses [19]. Consequently, the role of explicit thinking during RFT warrants further exploration, considering both accuracy performance and computational efficiency.
In this paper, investigate the thinking process rule-based RFT for MLLMs across different tasks and model sizes. We begin with case study exploring closed-form MLLM image classification. Motivated by the success of rule-based RFT 19 68 77 in LLM fine-tuning, we extend thinking-based RFT (Thinking--o) RFFT) to few-shot classification fine-tuning, Thinking-REE RFT fine-tunes MLLMs using class labels as verifiable answers for reward calculation. Extensive experiments show that Thinking- RFT performs much better than SFT on both in-domain learning and new-class generalization. Additionally, we observe a afree-lunch phenomenon: mne-tuning MLLMs on one dataset with Thinking- RFT improves performance on other datasets, despite shift in data distribution and entirely different class names. This phenomenon validates that rule-based RFT can effectively teach models the fundamentals of image classification rather than simply memorizing [
We then revisit the role of thinking process in rule-based RFT, key factor in the success of Deepseck-R R1 [19]. Unlike the gradual increase in response length of math problems observed in [19], classification RFT exhibits sharp drops in response length at specific steps, accompanied by significant rises in accuracy reward (Figure D. Our finding that thinking is less essential for classification, prompting models to adopt minimal reasoning, aligns with recent studies (2 showing that overthinking during inference can hinder performance on certain tasks. To this end, we propose No-Thinking RFT, a rule-based RFT approach without explicit thinking process. No-Thinking RFT utilizes direct-answer instruction prompt and removes the format reward. It implements strict equality accuracy reward, granting a score of solely when the output precisely matches the labels, thereby effectively discouraging reasoning during fine-tuning. Notably, in few-shot classification task, No-Thinking-RRF RFF outperforms Thinking-Re while achieving substantially shorter fine-tuning and inference time (Sec 4.4) and faster convergence (Sec 4.5) than Thinking-RFF RFT.
Next, we evaluate No-Thinking-RFF on general visual reasoning tasks, including math, spatial reasoning, puzzles, referring grounding, and detection, across different model sizes. Results reveal four important findings: (1): Thinking is unnecessary for visual perception tasks during RFT. Across all test model sizes, No-Thinking-RE RFT matches or outperforms Thinking-RFP for these tasks, showing that excluding thinking RFT can preserve or improve performance while enhancing training and inference efficiency. (2). Models with limited capability (e.g.. 2B) tend to converge to produce trivial reasoning under Thinking-RFT (refer to Figure 15 ~ Figure 20 for examples), leading to worse
performance and longer fine-tuning time compared to No- No-Thinking- -FF.. (3): For mid-sized models

---

(e.g., 7B), Thinking- RFT often produces inconsistencies between the content in thinking tags and answer tags (see Figure Figuure I4]. We observe frequent mismatches comparing answers from the thinking tags and answer tags, with the average accuracy amongst inconsistent responses lower than the overall accuracy (Figure(), which suggests that maintaining response consistency could improve performance. (4): The performance gain of No- Thhinking-RPP over Thinking-RFF mainly stems from improved fine-tuning geeitunng and the avoidance of inference overthinking. When we test Thinking- Riin in no-thinking mode by appending empty thinking tags during inference, performance improves on perception tasks but still lags behind No- No-Thinking-RFT This suggests that both enhanced learning during no thinking tnne-tunnng and reduced inference overthinking contribute the gain.
We further study why No- No-Thinking--a RFT outperforms Thinking Moanking-RFTiii in certain scenarios (e.g.. 2B model on perception tasks), hypothesizing that the explicit thinking before verifiable answers may hinder the learning process during RFT, slowing accuracy reward convergence and reducing accuracy. To test this, we propose Think-AAier-AAswerr verifiable answers. Experiment results support our hypothesis: in tasks where No-Thinking-RaN RFT outperforms Thinking-RIF Thinking-RIT However, it still underperforms No-Thinking-RLL RFT, suggesting that reasoning, even when placed after answers, can remain detrimental during RFT in certain scenarios.
variant where reasoning occurs after generating
Think-After-Ansse
converges faster and achieves higher accuracy than
Finally, we conduct a pilot study to investigate whether MLLMs can learn adaptively decide when to think during RFT. We introduce Adqptive-Thinkkinnn models are prompted to first assess if problem required reasoning or thinking. If reasoning was deemed necessary, the model would generate a thinking process before answering; otherwise, it would answer directly. Our experiments revealed that models consistently converged to a single response strategy (either always thinking or never thinking). Notably, Adaptive- Thinkkig achieves comparative or better performance than the better one of Thinking-RFF RFF and No-Thinking-RR RFF, and the final response strategy always corresponded to one suited for that specific model size and task complexity. For example, on math tasks, the 2B model converged to the non-thinking answering response, whereas the 7B model consistently adopted the thinking process before answering. These findings suggest that MLLMs may possess the capability to learn whether to employ reasoning based on their inherent abilities and task complexity during RFT.
Our contributions can be summarized as follows:
We extend Thinking-RF to few-shot MILM classification and reveal cross datased transfer across datasets.
We show several important findings about No-Thinking-Rri and Thinking-RFF via experiments on six tasks with 2B-7B models.
Through Liiik-Aftt--Ansst and adaptive-Thinkiie confirm that deferring or omitting CoT speeds convergence without harming accuracy and explore adaptive thinking strategy.
2 RFT Methods: Think vs. No Think
2.1 Thinking-RFT
to employ Group Relative Policy Optimization (GRPO) as the RL algorithm for optimization in our study, since it is the most widely used one. We refer readers to Appendix for brief introduction of the technical details of GRPO.
Optimization Algorithm. We follow Deepseek-R1 19 59
Instruction prompt. Following Deepseek-R1 [59] we utilize a prompt that encourages models to first engage in a thinking process before generating the final answers. The prompt is designed as: (Question/ Please output the thinking process in <think> </think> and final answer in <answer> </answer> tags. Here /Question| will be replaced by each specific question.
To clearly examine the RFT thinking process and promote the generalizability of our findings, we employ the simplest binary reward function for our study. The reward function composed with two parts: format and accuracy reward. The format reward Riomat to check the responses follow the format correctly. Riomat = 1 if the response format is correct, and 0 if it incorrect. The accuracy reward Raccuracy checks whether the answer in the answer tag matches the ground truth. For example, Rueauracy verifies the extracted answer against the correct choice in multi-choice problems, class labels in classification tasks, and numeric results in math problems.
Reward Function.
Raceuracy = 1 if the extracted answer matches correctly, and 0 otherwise.
3

---

Training Strategy. To study explicit thinking during RFT clearly, we adopt R1-zero [9 training for all methods in our study, i.e, we apply RL to all base models without any SFT, following [0].
2.2 No-Thinking-RFT
We explore rule-based RFT without a thinking process and propose No- No-Thinking-RR RFT method. The instruction prompt and reward functions are designed as below:
Instruction prompt. Instead of the prompt in Thinking-RFY which encourages models to think before answering, the prompt in the No-Thinking-RR method prohibits the model from thinking. The prompt is designed as: {Question) Please directly output the answer.
Reward Function. No-Thinking-Ree RFT eliminates the format reward and relies solely on the accuracy reward rather than combining two rewards. The accuracy reward Raceuracy checks whether the model's output matches the ground truth exactly. Raccuracy if the model response matches the ground truth and otherwise. The equality-based matching reward forces the model to bypass any thinking process and output only the answers, which are significantly shorter than the typical reasoning responses in Mhinking-FFT, As result, training No- No-Thinking-RR RRT requires substantially less GPU memory, and its training and inference times are markedly shorter than those of Thinking- -RF (Sec. (Sec.
3 General Visual Reasoning
In this section, we introduce results on general visual reasoning, including spatial understanding, math, puzzle, referring grounding. and detection across different model sizes. We report the results of spatial understanding, math, and puzzle tasks with 2B and 7B Qwen22-LLInstruet (72 models in main text, and leave the results of referring grounding, detection tasks and other different model types (e.g.. Intern VL2.5 [5] and Qwen2-5- Swwn2-5-VL-IInstruce VL-Instruct [1] in Appendix G We also discuss offline DPO, No- No-Thinking-RRF RFT, and Thinking- RFT in Appendix HH.2 We first introduce Think-
and Adaptive- Thinking methods and then report experimental results.
3.1 Think-After-Answeer & Adaptive-Thinking
Think-Affer-Answer. We explore the reason that No-Thinking-RRFT outperforms Thinking-RFT under specific tasks and model sizes. As shown in Sec. 4.5 Thinking-RF7 converges slower than No-Thinking-Rel RFT. Therefore, we hypothesize that for the 2B model, the explicit "think" process is detrimental, whereas for the 7B model, it is not universally required, as verifiable answers are already conditioned on thinking. To verify this hypothesis, we propose Think-After- Answer by letting MI.LMs first answer the questions and then output brief reasoning process, therefore alleviating the negative impact of explicit thinking on verifiable answers during RFT, as now the thinking is conditioned on answers. The prompt is provided in Appendix and the format and accuracy reward do not change. If the hypothesis holds, the convergence speed of Think- Afswer should be faster and the final performance should be higher than Thinking-RIT in cases where No-Thinking-RRi outperforms Thinking-RFT.
Adaptive-Thinking. We propose an Adaptive- Adaptive-Thinking method that lets MLLMs decide whether to think by themselves during RFT. Specifically, we prompt MLLMs to first determine whether problem requires thinking, and then either output reasoning process if needed or directly output an answer not (refer to Appendix[F.. F..3 for the prompt). A response in either thinking format or direct answering format will receive format reward of 1. The accuracy reward is not changed as before.
3.2 Experimental Setup
We focus on three main visual or multi-modal tasks: visual perception (classifiction and spatial grounding), multi-modal math reasoning, and visual puzzle reasoning in the main text. For fair comparison, we set hyperparameters the same for all methods. We provide a brief overview of the experimental settings below, with detailed implementation described in Appendix [D.
Visual Perception. We follow [2 to fine-tune models on SAT dataset
[57 epochs and then test on
CVBench dataset (69]. We also include classification results for comparison. The number of rollout

---

Table Results of Thinking-RFI Think Shink-After-Answer, No- No-Thinking-RFFi RFT, and Adaptive- Adaptive-Thinking
on CVBench.
Model
Method
Overall
Count
Relation
Depth
Distance
FT Time
Thinking-RFT
70.36
66.12
83.38
68.50
63.67
599
Think-Afler-Answe
73.65
68.65
82.15
74.83
69.83
408
2B
No-Thinking-Ke RFT
76.76
69.67
84.46
80.67
73.83
139
Adaptive-Thinking
77.03
69.92
86,31
76.50
76,83
208 m
Thinking-RF"
80.36
66.24
90,92
87.33
80.00
651
Lank-Afier-Answar
81.61
66.75
90.46
87.33
85.83
428
No-Thinking-RR -FFT
80.67
66.50
92.15
83.83
83.67
155 m
Seppiive-Thinking
81.65
67.38
90.31
88.00
84.67
639 m
Table 2: Performance comparison of PuzzleVQA and AlgoPuzzIe across 2B and 7B models.
2B
7B
Method
PuzzleVQA
AlgoPuzzleVQA
PuzzleVQA
AlgoPuzzleVQA
Thinking-RFF RFT
52.50
27.72
66.60
24.78
Think-After-Answer
64.70
26.94
80.45
28.11
No-Thinking-RFF
70.85
29.17
80.65
29.39
Adaptive-Thinking
75.45
27.94
85.05
29.00
is set to 4 for both 2B and 7B models. The number of rollout is set to for both 2B and 7B models. 3 is set to 0.04, learning rate is set to 10 The temperature is set to 1 for all experiments.
Multi-Modal Math Reasoning, We utilize the Math-40K [6] as the find-tuning data and fine-tune models epoch. Then we test the fine-tuned models on both Math Vista 43) and Math Vision [71]. The maximum length of input prompt and response are set to 4096 and 512 respectively. The number of rollout is set to 8 for 2B model, and 4 for 7B model. ] is set to 0.04, learning rate is set to 1 x 10-6,
Visual Puzzle Reasoning. We follow the code of 7 generate a training dataset with 6.5k data and fine-tune models 2 epochs. We then test fine-tuned models on PuzzleVQA 77 as in-domain testing, and on AggouzzleVQAA [17] as out-of-domain testing. set to 0.04, learning rate is set to 1 x 10-6, 6 The batch size is set to per GPU and we use 2-step gradient accumulation during training.
3.3 Results & Findings
We compare the results of four different thinking strategies here and show examples of responses in 2 25 We report the results of visual perception on Table math reasoning on We analyze these results and discuss several findings below.
Table 3 and puzzle reasoning on Table 2
Finding 1: For small models with weak ability (e.g., 2B size), Thinking-RFT may lead to trivial reasoning and inferior performance compared with No-Thinking, We observe that for 2B models, by using much less fine-tume tine-tuning time, No-Thinking-Rel -FTT outperforms Thinking-RFF -FFT in most benchmarks by large margin, including even Mathvista and except only Math Vision. We find that for Mathvista, No- Thinking-Rte RFT outperforms Thinking RFT on all tasks except math world problems (MWP). It is reasonable that Thinking-Rii RFT outperforms No- No-Thinking-RFP in MWP and Math Vision since these problems need much computation for obtaining final answers and it hard to directly output correct answers. When looking deeper into model responses under Thinking-RFF we find that most reasoning is trivial, i.c., it does not contribute meaningfully to the final answer, especially for complex reasoning tasks (refer Figures Figure 20 for qualitative examples). These results suggest that when the capability of models is poor, the thinking process during RFT may converge to trivial solutions and finally lead to inferior performance compared with No- No-Thinking.
Visual perception and puzzle tasks do not need thinking. We find that No- No-Thinking- RFT outperforms Thinking- -Raiinng--FTT--- on spatial reasoning, classification (7B results in Table 12], and puzzle tasks across both 2B and 7B models, while Thinking-RFT performs better on math reasoning with the 7B model. We notice that although puzzle tasks typically require complex reasoning, No-Thinking-RFF RFT outperforms Thinking-RFP on both 2B and 7B models. This may because these tasks rely visual rather than linguistic reasoning. and language- based reasoning might cause
Finding 2:
5

---

Table Accuracy (%) of 2B and 7B models on Math Vista sub-tasks and Math Vision overall. TQA: Textbook QA, VQA: Visual QA, Geo: Geometry, MWP: Math Word Problem, FQA: Figure QA.
Model
Method
Vist
Mithnn
Overall
TQA
VQA
MWP
FQA
Thinking-RFT Aift--fffcr--weee
44.90
46.84
36.87
37.02
52.15
50.19
16.45
48.50
51.27
44.69
36.54
50.54
14.80
2B
No-Thinking- RRF
48.80
52.53
46.37
38.94
49.46
55.39
13.49
Adaptie-Thinking
50.20
50.00
51.96
37.98
52.69
56.88
13.16
Thinking-RFF
64.60
65.19
59,22
60.58
68.28
68.40
21.71
Think-After-A
62.00
63.92
58.10
58.65
63.98
64.68
21.71
7B
No-Thinking RFT
59.10
65.19
58.66
55.77
56.45
60.22
18.09
adlptive-Thinnkie
64.70
64.56
56.42
59.13
74.73
67.66
21.05
Oveuuil thd doumess Arymo Accs incina cieee The Arawer famponas
0.5
0.4
0.3
0.1
Mathibte Huthiase CLS omo
citerrl
Hathvista
Mathviaaion
Cyilnnh
(a) Distribution the Inconsistency
(b) Test Accuracy
Figure Comparison of inconsistency proportion and test accuracy among different datasets.
hallucinations, or the 7B model's puzzle-solving capability may still be limited. In summary, visual perception and puzzle reasoning tasks do not need reasoning among model sizes, and RFT without thinking beneficial to both accuracy performance and computational efficiency.
Finding 3: There are inconsistencies between thinking and answer content of some responses. We use GPT-4o [25] to extract answers from the thinking tag and compare them with those in the answer tag (see Appendix E. for details). We find inconsistencies in some responses between the thinking and answer tags. Figure 2 visualizes the proportion of inconsistent responses per dataset and their respective accuracies in both tags. We also show examples of inconsistent responses in Fgsurre]-Fiuurel 14, We found that 1). Inconsistencies are more common in reasoning tasks such puzzles than perception tasks like classification. 2). For inconsistent responses, accuracy of answers in answer tags much higher than in thinking tags, and both are below the overall accuracy. consistent responses could improve performance,
These results suggest that encouraging particularly on reasoning tasks. We leave strategies for enhancing consistency to future work.
Finding 4: Performance gain of No-Thinle Te-Thiinknn-RFP over Thinking-RFT on certain tasks stems from improved learning during RFT and avoidance inference overthinking. We test the No-Thinking inference mode of Thinking-RFT by appending empty thinking tag during inference and show the results on Table 17 We find that while appending empty thinking tags during inference could improve performance Thinking-RIF RFT. This suggests that the gain of No- No-Thinkking-Re primarily stems from two factors: improved learning during fine-tuning and the avoidance of overthinking during inference.
CVBench and puzzle tasks, its performance is still far behind No-
Finding 5: Explicit thinking before verifiable answers may leads slower reward convergence and inferior performance. We observe that Think-After-Anwwer outperforms Thinking-REi in perception and puzzle tasks. We also visualize the accuracy-rewar curves on the SAT and Puzzle achieves faster reward convergence and higher final accuracy than Thinking-RFTI These results indicate that placing explicit CoT after verifiable answers during RFT can alleviate its negative impact, improving both convergence speed and performance, thus validating our hypothesis that explicit thinking before answers may lead to slower reward convergence and reduced performance perception and puzzle tasks.
datasets in Figure 3 (2B) and Figure g (7B), where Mhink-After-Amee
Finding 6: MILMMS can adaptively learn whether to think on task level during RFT on task level. We find that Adaptive-Thinking performs comparably to or better than the stronger of Thinking-

---

Table Parameter Difference (L, Norm)
Dataset
Method
Visual
Language
MLP_V
norm V
attn_V V
MLP
norm L
attn L
DTD
Thinking 0.267
0.434
0.206
0.001
0.158
0.372
0.000
0.153
No- No-Thinking
0.237
0,402
0,184
0.001
0.143
0.379
0,000
0.154
SAT
Thinking 0.503
0.712
0.398
0.003
0.311
0.614
0.000
0.256
No-Thinking
0.518
0.718
0.398
0.003
0.327
0.704
0.000
0.281
Thinking 0.415
0.735
0.364
0.002
0.210
0.691
0.000
0.270
No- No-Thinking
0.731
0.931
0.581
0.004
0.455
0.880
0.000
0.388
SAT 28
Puzzle 28
0.8
0.6
0.4
9.2
Stop
Stop
1.5k
200
400
Thinking:577
TTéreAfarkkussrr
-Tinkingg
(a) SAT 2B Accuracy Reward
(b) Puzzle 2B Accuracy Reward
Figure 3: Comparison of accuracy reward convergence speed on SAT dataset over steps of Thinking-
RFT, Think-After-mee Anser,, adn No-Thinking-Ru across and 7B models.
and No- No-Thinking-RFF Figure shows the response length trend curves during RFT, and Table IO summarizes the statistics of test response types. We observe that models initially produce mix of thinking and No-Thinking responses, but gradually converge a single response type either think or not think and matching the petter-performing strategy. For instance, the 2B model converges to No-Thinking response, while the model converges thinking response for math tasks. These results suggest that MILMs can learn whether to think during RFT based on model capacity and task complexity. Despite promising results, the same task can present problems requiring varied responses. would operate the problem level, refinement for future work.
Ideally,
3.4 Parameter Change during RL
We compare the changes in parameters by computing the L, norm of parameter difference. Our analysis focuses on three aspects: 1. Modality: changes between visual and language components, Module: changes in different modules, 3. Layer: changes across different layers.
SAT, DTD, and Math datasets using Thinking and No-Thinking-RFFI Modality- and Module-level results are given in the table below. We directly discuss layer-level findings as the table is too large.
We analyzeQwen2VL-22
Modality-Level Language weights drift more than visual weights for aatasst-and the No- Thinking strategy raises visual drift slightly. This implies that the reward gradient mainly applies to the language modality.
2. Module-L.evel: MLP is the dominant changed component for both visual and language parts, and attention blocks change less but still contribute near 15-20%.
Layer-Level: With Thinking, weight drift grows toward deeper layers as reward prokppropsgates through the whole reasoning chain high-level semanties. With No-Thinking, drift peaks in early- mid layers and then declines, indicating that low-level features are reshaped so shallow forward path already produces the reward token.
Case Study on Image Classification Tasks
In this section, we introduce the experiment results on classification. Recent works 81 show that the classification abilities of MLLMs 42
36
12
13
49 6)
poor due to pre-training data limitations

---

Table 5: No-Thinking-RFT outperforms Thinking-RFT in 10 out of 11 datasets under few-shot learning results. S.C.: StanfordCars dataset. F.A.: FGVCAireraft dataset.
Catteh101
Flowers102
OxfordPats
ET
Food1101
ImageNer
SUN297
UCFIOL
Average
DTD
S.C.
FAA.
Qwen2VL.
54,79
45.6
77.54
70,8
66.22
42.75
62.21
SFT
93.91
71,336
75.16
75.75
41.60
96.87
85,80
71.13
41.66
63.81
60.15
70.65
'Tinking-RFF
98.09
69.92
49.46
88. 94
92.24
86.56
87.24
80.24
84.57
82.1
74.41
81.25
No-Thinking-RFT
98.46
73.52
58.02
90.78
92.31
91.6
86.13
92.5
86.72
83.82
74.41
84.39
and SFT with large-scale data could improve performance 8 However, acquiring large-scale datasets incurs substantial cost and computational overhead [84] Few-shot learning thus emerges as a compelling alternative, previously validated in contrastive vision-language models (VLMs) [54, 83
but its application to MILLMs remains underexplored. We report the results of few-shot learning, free-lunch phenomenon, convergence and efficiency comparison in this section, the results of base-to-new setting in Appendix open-set classification in Appendix [.2] the results of 7B
84
38 30].
Qwen2-VL-Instrruc [72] in Appendix F33 and examples of model response in Appendix
4.1 Experimental Setup
Setting. In this paper, mainly focus closed-form classification for MLLMs, where sub- set class names is provided for selection. The question format [Question] (Instruction prompt), where {Question} and (Instruction prompt} will be replaced by the specific ques- tions and instruction prompt of each dataset and method respectively.
we conduct experiments The names, questions, statistics of each dataset are provided in Appendix [ Due computational resource and model input limit, we select subset of class labels the choice list. For few-shot classification, we sample 40% labels including the gt labels form choice list question.
Datasets. Following 83
84
11 public classification benchmarks.
(72 as the base model, and fine-tune all parameters, following (2. 3) All training is conducted in A100 GPUs The batch size is set to I per GPU and we use 2-step gradient accumulation during training 3 60). All images are resized to 328x328 resolution and no data augmentation is used. For inference test, first extract answers from the answer tag and then verify if class names are in answers. If the answer tag does not exist in model responses we directly verify if class names are model responses, following (8I). The temperature is set to 1.0 for all experiments. The random seed is set to 100 for all experiments.
Implementation Details. We utilize (wen2-VL--2B-Instie
More details about dataset construction and implementation details provided in Appendix D
4.2 Few-Shot Learning Results
In this subsection, we present the results of few-shot learning. We train models on 4-shot setting and report the accuracy results in Table 2B models and Table 7B models. For 2B models, observe that Thinking-Ri RRF markedly surpasses SFT most datasets, resulting in a notably higher average accuracy than SFT, which implies that rule-base RFT can let model learn better downstream knowledge than SFT. We further discover that No- Thinking-RRt RFT outperforms Thinking-REt -FFT in 10 out of 11 datasets, ultimately achieving 3.14% higher average accuracy compared to Thinking-RFT For 7B models, the performance gap between Thinking-Ro RFT and No- Thinking- RFT narrows but remains. These results indicate that RFT without the thinking process effectively enhance model's performance on classification than Thinking-RFT.
4.3 Free-Lunch Phenomenon
In this section, we discuss the free-lunch phenomenon. Previous work in few-shot contrastive VLM fine-tuning has demonstrated that, when fine-tuned on a specific dataset, the performance of VLMs on other datasets is drastically degraded, a phenomenon known as catastrophic forgetting 83 30 However, we will show that Thinking-RRF and No- No-Thinkinng-Ree can enhance the performance of MLLMs other datasets when fine-tuned on one specific dataset. We visualize the improvement of

---

Improvement on each datenet.
Improvement on each detenet.
teageset
6.11
Ca K27
Caltech1O U.A.
1827
11.0
EuroSAT
Xm
130
EXroSAT +0
11.00
300
0331
Onferdflowers
Oaterdflowers
DafordPoos 841
23.01
GefardPoct 11.77
733
180
SUN297 11:0
641
130
100
1370
1110
1099
SUNDE7 (3:00
12:0
06.6
12:0
180
1200
SoVCAlrcrat
FöVCAtreaat
Inageeet
Cottech
Airedar
Poweress
Deterthets
stenbrotcane
vorcainnett
Colteching
Powtrox
(a) Thinking-RFT
(b) RFFT
Figure 4: Free-lunch phenomenon: Both Thinking-RFT and No-Thinking-RIL RFT lead to cross-dataset
accuracy gains. Red: increase; blue: decrease; x-axis: test dataset; y-axis: fine-tuning dataset.
Table 7: Comparison of Thinking-RFT and No-Thinking-RFI with 2B and 7B models on fewshot
classification. DTD: DescribableTTexture.
Model
Method
DTD
EuroSAT
OxfordFlowers
StanfordCars
Average
Thinking-RFT
77.90
53.17
93.91
84.19
77.29
7B
Think-After-nsseee
76.29
62.95
94.84
89.32
80.85
Shaptive-Thinking
79.60
56.62
96.86
89.92
80.75
No-Thinking-RRr
80.56
58.91
94.24
94.02
81.93
Thinking-RFT and No- No-Thinking-RFF compared with zero-shot Dwven2VL-instruct-BB in Figure 4 We find that when fine-tuned on specific dataset, both Thinking-RFT and No-Thinking-RFF yield improvements on other datasets in most instances, despite variations in data distribution and even completely different class lists. These results indicate that the application of rule-based verifiable signals and reward loss for model fine-tuning can effectively compel models to acquire essential classification knowledge instead of memorizing. This equips them to achieve superior performance on entirely new datasets. We discuss more about the free lunch phenomenon in Appendix F.4]
4.4 Efficiency Comparison
In this subsection, we compare the training and inference efficiency of SFT, Thinking- and No-Thinking-RFTT using the ImageNet dataset as case study. The results are presented in Table We find that Thinking-RFF requires significantly more time for both training and inference compared SFT and No-Thinking-RFT attributable the necessity of generating multiple lengthy responses during fine-tuning and long reasoning response before answers during inference. In contrast, SFT optimizes only label tokens during fine-tuning, and No- No-Thinking-RFF compels the model to output only the ground truth labels during this phase, which significantly reduces the time required. During the inference phase, both methods are designed output solely class labels, resulting in considerably reduced inference time.
Table 6: Thinking-RFi requires much more time (in mins) for finetuning and in- ference SFT and No- No-Thinking-PP RFT.
Method
FT
Infer
SFT Thinking-RFT No-Thinking-RFT
35
20
1587
30
26

---

(5) DescribableTetture Acc
Reward
(b) Testribableexxtues Test Ace (c) OxfordFlowers Ace Reward
(d) OxfordFlowers Test Ace
Figure 5: Comparison of accuracy reward convergence speed and test accuracy over steps between
Thinking-RFi and No-Thinking-RFI No-Thinking-RRI converges faster than Thinking-RFT.
4.5 Convergence Comparison
In this subsection, we discuss the comparison in convergence rates between Thinking-RFi and No- Thinking-RFT. We illustrate the accuracy reward at each training step and examine the test accuracy at select intervals on the DescribableTTextrre and OxfordFlowers datasets. The results are shown in Figure 5 We observe that No- Trinhiiiniii--re exhibits faster convergence speed compared to Thinking-Rii RFT, as evidenced by a higher accuracy reward and significantly higher test accuracy the early stages of training (within the first 30 steps). These findings imply that slower convergence speed and lower accuracy reward of Thinking--i RFT leads to inferior performance than No- No-Thinking-B
5 Related Works
Rule-based RFT has recently achieved significant progress in LLMs 19 26 68 and show superior performance compared with SFT. To extend this success to MILMs, numerous studies have been proposed 41
46 3 aiming to replicate phenomena observed in Deepseek-R1, such as increasing response length and emergence of 'aha' moments. Although these initial efforts offer encouraging outcomes, none have thoroughly investigated the role of thinking processes during RFT. Recent research into LLM and MLLM inference suggests that overthinking can degrade performance [27 64], providing valuable insights into model reasoning. However, these studies exclusively examine the inference stage and do not explore the impact of thinking during the fine- tuning process. As a result, the role of the explicit thinking process in RFT remains unclear. this paper, we study the effect of explicit thinking during RFT and conduct extensive experiments across different model sizes and tasks. More detailed related works are provided in Appendix
60
82
24
Conclusion
In this paper, we conduct comprehensive study on the explicit thinking in RFT. We begin by extending Thinking-RFi to classification. Experiments show that Thinking-RFi performs much better than SFT on both base-to-new and few-shot settings. Furthermore, we observed free-lunch phenomenon with classification RFT, wherein the performance of a model fine-tuned on one dataset improves on other datasets. We then delve into the thinking process of RFT. Inspired by recent research on inference overthinking and observations regarding the decreasing response length of classification RFT, we introduce No- Bo-TThnnking-Ree RFT, which compels the model to bypass thinking. We conduct extensive experiments on various visual reasoning tasks across different model sizes to evaluate Thinking and No- Thinking-RRF RFT. The results reveal four key findings about the inconsistency in model responses, the impact and the necessity of thinking across different tasks and model sizes, and the stem of No-Thinking--B RFT's performance gain. Finally, propose Thinking-AAfe Mainkin--Ater-Answwa After-Answer to verify the hypothesis that explicit thinking before verifiable reward may hinder fine-tuning and Adaptive-Thinkin to explore whether MLLMs can learn when to think or not during RFT.
Limitations This study has two main limitations. First, computational constraints restricted our experiments to models 7B and smaller; thus, the applicability of our findings to larger models remains unverified. Second, our adaptive-Thinkinn No-Thinking) at the task level, rather than adapting at the more granular problem level. Designing methods to achieve this problem-leve adaptation is a direction for future work.
method converges to single response mode (thinking or
10

---

Acknowledgments
This paper is supported by the National Key R&D Program of China No. 16.2022200160101.
References
[1] S. Bai, K. Chen, X. Liu, J. Wang, W. Ge, S. Song, K. Dang, P. Wang, S. Wang, J. Tang, H. Zhong, Y. Zhu, M. Yang, Z. Li, J. Wan, P. Wang, W. Ding, Z. Fu, Y. Xu, J. Ye, X. Zhang,
T. Xie, Z. Cheng, H. Zhang, Z. Yang, H. Xu, and Lin. Qwen2.5-vl technical report. arXiv preprint arXiv:2502.13923 2025.
[2] L. Bossard, M. Guillaumin, and L. Van Gool. Food-101-mining discriminative components with random forests. Computer Vision-ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part VI 13, pages 446-461. Springer, 2014.
[3] I.. Chen, I.. Li, H. Zhao, Y. Song, and Vinci. R1-v: Reinforcing super generalization ability in vision-language models with less than S3. httpp::/ggttuu..cmm Agent/R1--e 2025. Accessed: 2025-02-02.
[4] X. Chen, J. Xu, T. Liang, Z. He, J. Pang, D. Yu, L. Song, Q. Liu, M. Zhou, Z. Zhang, et al. Do not think that much for 2+ 3=? on the overthinking of ol-like 1lms. arXiv preprint arXiv:2412.21 187, 2024.
[5] Z. Chen, W. Wang. Y. Cao, Y. Liu, Z. Gao, E. Cui, Zhu, S. Ye, H. Tian, Z. Liu, et al. Expanding performance boundaries of open-source multimodal models with model, data, and test-time scaling, arXiv preprint arXiv:2412.05227, 2024.
[6] Z. Chen, W. Wang, Y. Cao, Y. Liu, Z. Gao, Cui, Zhu, S. Ye, H. Tian, Z. Liu, et al. Expanding performance boundaries of open-source multimodal models with model, data, and test-time scaling. arXiv preprint arXiv:2412.05227, 2024.
[7] Y. K. Chia, V. T. Y. Han, D. Ghosal, L. Bing, and S. Poria. Puzzlevqa: Diagnosing multi- modal reasoning challenges of language models with abstract visual patterns. arXiv preprint arXiv:2403. 13315, 2024.
[8] T. Chu, Y. Zhai, J. Yang, S. Tong, S. Xie, D. Schuurmans, Q. V. Le, S. Levine, and Y. Ma. Sft memorizes, rl generalizes: A comparative study of foundation model post-training. arXiv preprint arxiv 2025.
[9] M. Cimpoi, S. Maji, I. Kokkinos, S. Mohamed, and A. Vedaldi. Describing textures in the wild. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 3606-3613, 2014.
[10] A. Cuadron, D. Li, W. Ma, X. Wang, Y. Wang, S. Zhuang, S. Liu, L. G. Schroeder, T. Xia, H. Mao, et al. The danger of overthinking: Examining the reasoning-action dilemma in agentic tasks. arXiv preprint arXiv:2502.0823: 08235, 2025.
[11] C. N. De Sabbata, T. R. Sumers, B. AlKhamissi, A. Bosselut, and T. L. Griffiths. Rational metarcasoning for large language models. arXiv preprint arX55:2410.05566, 2024.
[12] DeepMind. Gemini 2.0 flash experimental. httpe://beepinn..ggg////eeellllp gemini/flash/ 2024. Accessed: 2024- 12-25.
[13] DeepMind. Gemini 2.0 flash thinking. https: https//bbeentnllloooolee//eeooos
haati//faaahhhhiiinggee 2025. Accessed: 2025-01-21.
[14] J. Deng, W. Dong, R. Socher, L.-J. J. Li, K. Li, and L. Fei-Fei. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition, pages 248-255. 2255. leee, 2009.
[15] A. Dosovitskiy, I. Beyer, A. Kolesnikov, D. Weissenbomn, X. Zhai, T. Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly, Uszkoreit, and N. Houlsby. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint 20Xv:20100.1192.2200
11

---

[16] L. Fei-Fei, R. Fergus, and P. Perona. Learning generative visual models from few training ex- amples: An incremental bayesian approach tested on 101 object categories. In 2004 conference on computer vision and pattern recognition workshop, pages 178-178. IEEE, 2004.
[17] D. Ghosal, V. T. Y. Han, C. Y. Ken, and S. Poria. Are language models puzzle prodi- gies? algorithmic puzzles unveil serious challenges in multimodal reasoning. arXiv preprint arXiv:2403.080 038644 2024.
[18] X. Guan, L. L. Zhang, Y. Liu, N. Shang, Y. Sun, Y. Zhu, F. Yang. and M. Yang. rstar- math: Small Ilms can master math reasoning with self-evolved deep thinking. arXiv preprint Ntt.:2501.04599, 2025.
[19] D. Guo, D. Yang. H. Zhang, J. Song, R. Zhang, R. Xu, Q. Zhu, S. Ma, P. Wang. X. Bi, et al. Deepseek-rl: Incentivizing reasoning capability in Ilms via reinforcement learning. arXiv preprint MXXi::2001.12445 2025.
[20] K. He, X. Zhang, S. Ren, and J. Sun. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). pages 770-778, 2016.
[21] Helber, Bischke. A. Dengel, and D. Borth. Eurosat: A novel dataset and deep learning benchmark for land use and land cover classification. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 12(7):2217-2266, 2019.
[22] A. G. Howard, M. Zhu, B. Chen, D. Kalenichenko, W. Wang, T. Weyand, M. Andreetto, and H. Adam. Mobilenets: Efficient convolutional neural networks for mobile vision applications. In arXiv preprint 17X4411704.04861, 2017.
[23] J. Hu. Reinforce++: A simple and efficient approach for aligning large language models. arXiv preprint 2rXiv:2501.03262. 2025.
[24] W. Huang, B. Jia, Z. Zhai, S. Cao, Z. Ye, F. Zhao, Z. Xu, Y. Hu, and S. Lin. Vision-rl: Incentiviz- ing reasoning capability in multimodal large language models, arXiv preprint ar arXiv:2503.06749, 2025.
[25] A. Hurst, A. Lerer, A. P. Goucher, A. Perelman, A. Ramesh, A. Clark, A. Ostrow, A. Welihinda, A. Hayes, A. Radford, et al. Gpt-4o system card. arXiv preprint arXiv:2410.21276, 2024.
[26] A. Jaech, A. Kalai, A. Lerer, A. Richardson, A. El-Kishky, A. Low, A. Helyar, A. Madry,
A. Beutel, A. Carney, et al. Openai ol system card, arXiv preprint arXiv:24/2.16720 2024.
[27] D. Jiang, R. Zhang, Z. Guo, Y. Li, Y. Qi, X. Chen, L. Wang, J. Jin, C. Guo, S. Yan, et al. Mme-cot: Benchmarking chain-of-thought in large multimodal models for reasoning quality, robustness, and efficiency. arXiv preprint arXiv:2502.0880 1, 2025.
[28] B. Joshi, Z. Liu, S. Ramnath, A. Chan, Z. Tong, S. Nie, Q. Wang. Y. Choi, and X. Ren. Are machine rationales (not) useful to humans? measuring and improving human utility of free-text rationales. Xiviv preprint arXiv:2305.07095, 2023.
[29] Y. Kang, X. Sun, L. Chen, and W. Zou. C3ot: Generating shorter chain-of- thougtt without compromising effectiveness. In Proceedings of the AAAI Conference Artificial Intelligence, volume 39, pages 24312-24320, 2025.
[30] M. U. Khattak, H. Rasheed, M. Maaz, S. Khan, and F. S. Khan. Maple: Multi-modal prompt learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recogni- tion, pages 1911222, 2023.
[31] J. Krause, M. Stark, J. Deng, and L. Fei-Fei. 3d object representations for fine-grained categorization. In Proceedings of the IEEE international conference on computer vision workshops, pages 554-561, 2013.
[32] A. Krizhevsky. I. Sutskever, and G. E. Hinton. ImageNet classification with deep convolutional neural networks. Advances in Neural Information Processing Systems (NeurIPS), pages 1097-1105, 2012.
12

---

[33] X. Lai, Z. Tian, Y. Chen, Y. Li, Y. Yuan, S. Liu, and J. Jia. Lisa: Reasoning segmentation via large language model. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9579-9589, 2024.
[34] Y. Lai, J. Zhong, M. Li, S. Zhao, and X. Yang. Med-rl: Reinforcement learning for generalizable medical reasoning in vision- language models. arXiv preprint (tXXi::25033/9933 2025.
[35] Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner. Gradient-based learning applied to document recognition. Proceedings of the IEEE, 86(11):2278-2324, 1998.
[36] J. Li, D. Li, C. Xiong, and S. Hoi. Blip: Bootstrapping language-image pre-training for unitied vision-language understanding and generation. In International conference on machine learning, pages 12888-12 -2888-12900. PMLR, 2022.
[37] M. Li, Q. Li, and Y. Wang. Class balanced adaptive pseudo labeling for federated semi- supervised learning. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pages 16292-16301, 2023.
[38] M. Li, J. Zhong, C. Li, L. Li, N. Lin, and M. Sugiyama. Vision-language model inn-tuning via simple parameter--fffiient modification. arXiv preprint arXiv:2409. 16711888..6, 2024.
[39] H. Lightman, V. Kosaraju, Y. Burda, H. Edwards, B. Baker, T. Lee, J. Leike, Schulman, I. Sutskever, and K. Cobbe. Let's verify step by step. In The Twelfth International Conference on Learning Representations, 2023.
[40] R. Liu, J. Geng, A. J. Wu, I. Sucholutsky, T. Lombrozo, and T. L. Griffiths. Mind your step (by step): Chain-of-thought can reduce performance tasks where thinking makes humans worse. arXiv preprint urXiv:2410.21333, 2024.
[41] Z. Liu, Z. Sun, Y. Zang, X. Dong, Y. Cao, H. Duan, D. Lin, and J. Wang. Visual-rft: Visual reinforcement fine-tuning, arXiv preprint arXiv:2503.01785, 2025.
[42] H. Lu, W. Liu, B. Zhang, B. Wang, K. Dong, B. Liu, J. Sun, T. Ren, Z. Li, H. Yang, et al. Deepseek-vl: towards real-world vasion-language understanding. arXiv preprint arXiv:2403.05525, 2024.
[43] Lu, H. Bansal, Xia, J. Liu, Li, H. Hajishirzi, Cheng, K.-W. Chang, M. Galley, and J. Gao. Mathvista: Evaluating mathematical reasoning of foundation models in visual contexts. arXiv preprint 20.iv:23100022555 2023.
[44] S. Maji, E. Rahtu, J. Kannala, M. Blaschko, and A. Vedaldi. Fine-grained visual classification of aircraft. arXiv preprint 1rxiv:1306.515. 2013.
[45] Mao, J. Huang, A. Toshev, O. Camburu, A. L. Yuille, and K. Murphy. Generation and comprehension of unambiguous object descriptions. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1 11-20, 2016.
[46] Meng, L. Du, Z. Liu, Z. Zhou, Q. Lu, Fu, B. Shi. W. Wang. He, K. Zhang, al. Mm- eureka: Exploring visual aha moment with rule-based large-scale reinforcement learning. arXiv preprint arXiv:2503.0m 00666..65...... 2025.
[47] Y. Meng, M. Xia, and D. Chen. Simpo: Simple preference optimization with a reference-free reward. Advances in Neural Information Processing Systems, 17:124198-22425, 2024.
[48] M.-E. Nilsback and A. Zisserman. Automated flower classification over large number of classes. In 2008 Sixth Indian conference on computer vision, graphics & image processing, pages 722-729. IEEE, 2008
[49] OpenAl. Gpt-4o system card. 2024. Accessed: 2024-09-26.
pdf
[50] L. Ouyang, J. Wu, X. Jiang, D. Almeida, C. Wainwright, P. Mishkin, C. Zhang, S. Agarwal, K. Slama, A. Ray, al. Training language models to follow instructions with human feedback. Advances in neural information processing systems, 015277300-27744 2022.
13

---

[51] O. M. Parkhi, A. Vedaldi, A. Zisserman, and C. Jawahar. Cats and dogs. In 2012 IEEE conference on computer vision and pattern recognition, pages 3498-350S, IEEE. 2012.
[52] A. Paszke, S. Gross, F. Massa, A. Lerer, J. Bradbury, G. Chanan, T. Killeen, Z. Lin, N. Gimelshein, L. Antiga, et al. Pytorch: An imperative style, high-performance deep learning library. Advances in neural information processing systems, 32:8026 80066-807,.11 2019.
[53] Y. Peng, G. Zhang, M. Zhang. Z. You, J. Liu, Q. Zhu, K. Yang, X. Xu, X. Geng, and X. Yang. Lmm-rl: Empowering 3b Imms with strong reasoning abilities through two-stage rule-based rl. arXiv preprint S..iv:22033075333 2025.
[54] A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, Mishkin, J. Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pages 8748-8763. PmLR, 2021.
[55] A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark, G. Krueger, and I. Sutskever. Learning transferable visual models from natural language supervision, 2021.
[56] R. Rafailov, A. Sharma, E. Mitchell, C. D. Manning, S. Ermon, and C. Finn. Direct preference optimization: Your language model is secretly a reward model. Advances in Neural Information 2023.
Processing Systems,
[57] A. Ray, J. Duan, R. Tan, D. Bashkirova, R. Hendrix, K. Ehsani, A. Kembhavi, B. A. Plummer, R. Krishna, K.-H. Zeng. et al. Sat: Spatial aptitude training for multimodal language models. arXiv preprint arXiv:2412.07755, 2024.
[58] J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov. Proximal policy optimization algorithms. arXiv preprint arXiv: 1707.06777 2017.
[59] Z. Shao, P. Wang, Q Zhu, R. Xu, Song, X. Bi, H. Zhang, M. Zhang, Y. Li, Y. Wu, et al. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint ar arXiv:2402.03300 033000000,, 2024.
[60] H. Shen, P. Liu, J. Li, C. Fang, Ma, Liao, Q. Shen, Z. Zhang, K. Zhao, Q. Zhang, al. Vim-rl: A stable and generalizable rl rl-style large vision-language model. arXiv preprint arXiv:2504.00 07615, 2025.
[61] W. Shi. Z. Hu, Y. Bin, J. Liu, Y. Yang. S.-K. Ng. L. Bing, and K.WW Lee. Math-llava Bootstrapping mathematical reasoning for multimodal large language models. arXiv preprint arXiv:2406. 7794, 2024.
[62] K. Simonyan and Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556, 2014.
[63] K. Soomro, A. R. Zamir, and M. Shah. Ucf101: A dataset of 101 human actions classes from videos in the wild. arXiv preprint arXiv:1212.0402 2012.
[64] 7. Sprague, F. Yin, J. D. Rodriguez, D. Jiang, M. Wadhwa, P. Singhal, X. Zhao, X. Ye, K. Mahowald, and G. Durrett. To cot or not to cot? chain-of-thought helps mainly on math and symbolic reasoning. arXiv preprint arXiv:2409.11 12183, 2024.
[65] Y. Sui, Y.-N. Chuang, G. Wang. J. Zhang, T. Zhang, J. Yuan, H. Liu, A. Wen, H. Chen, X. Hu, et al. Stop overthinking: A survey on efficient reasoning for large language models. arXiv preprint arXiv:2503. 16419, 2025.
[66] C. Szegedy, W. Liu, Y. Jia, P. Sermanet, S. Reed, D. Anguelov, D. Erhan, V. Vanhoucke, and A. Rabinovich. Going deeper with convolutions. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 1-9, 2015.
[67] M. Tan and Q Le. Efficientnet: Rethinking model scaling for convolutional neural networks. In International Conference on Machine Learning (ICML), pages 6105-6114, 2019.
[68] K. Team, A. Du, B. Gao, B. Xing, C. Jiang, C. Chen, C. Li, Xiao, C. Du, C. Liao, et al. Kimi k1. 5: Scaling reinforcement learning with 1lm.. arXiv preprint arXiv:2501.12599, 2025.
14

---

[69] P. Tong, E. Brown, P. Wu, S. Woo, A. J. V. IYER, S. C. Akula, S. Yang, J. Yang. M. Middepogu,
Z. Wang, et al. Cambrian-1: A fully open, vision-centric exploration of multimodal 1lms. Advances in Neural Information Processing Systems, 37:87310-87356, 2024.
[70] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and
I. Polosukhin. Attention is all you need, 2023.
[71] K. Wang, J. Pan, W. Shi, Z. Lu, H. Ren, A. Zhou, M. Zhan, and H. Li. Measuring multimodal mathematical reasoning with math- dataset. Advances in Neural Information Processing Systems, 37:95095-951 169, 2024.
[72] P. Wang. S. Bai, S. Tan, S. Wang. Z. Fan, J. Bai, K. Chen, X. Liu, J. Wang, W. Ge. Y. Fan, K. Dang, M. Du, X. Ren, R. Men, D. Liu, C. Zhou, Zhou, and J. Lin. Qwen2-vl: En- hancing ussion-languaa model's perception of the world at any resolution. arXiv preprint arXiv:249..1219, 2024.
[73] Wang. L. Z. Shao, R. Xu, D. Dai, Y. Li, D. Chen, Wu, and Z. Sui. Math-shepherd: Verify and reinforce Ilms step-by-step without human annotations. arXiv preprint arXiv:2312.08923 2023.
[74] J. Wei, X. Wang, D. Schuurmans, M. Bosma, F. Xia, E. Chi, Q V. Le, D. Zhou, et al. Chain-of- thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824 248377
24837, 2022.
[75] Y. Wu, Y. Wang, Z. Ye, T. Du, S. Jegelka, and Y. Wang. When more is less: Understanding chain-of thouuht length in Ilms. arXiv preprint arXiv:2502.07266. 2025.
[76] Xiao, Hays,
Ehinger, A. Oliva, and A. Torralba. Sun database: Large-scale scene recognition from abbey to zoo. In 2010 IEEE computer society conference computer vision and pattern recognition, pages 3485-3492 IEFE, 2010.
[77] T. Xie, Z. Gao, Q. Ren, H. Luo, Y. Hong, B. Dai, J. Zhou, K. Qiu, Z. Wu, and C. Luo. Logic-rl: Unleashing IIm reasoning with rule-based reinforcement learning. arXiv preprint W5Xi:22502.1476, 2025.
[78] Y. Xu, X. Guo, Z. Zeng, and C. Miao. Softcot: Soft chain-of-thought for efficient reasoning with 1mss. arXiv preprint 2212 2025.
[79] L. Yu, P. Poirson, S. Yang, A. C. Berg, and L. Berg. Modeling context in referring expressions. In Computer Vision-ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11-14, 2016, Proceedings, Part II 14, pages 69-85. Springer, 2016.
[80] Y. Yue, 7. Chen, R. Lu, A. Zhao, Wang, S. Song, and G. Huang. Does reinforcement learning really incentivize reasoning capacity in llms beyond the base model? arXiv preprint arXiv:2504. 13837, 2025.
[81] Y. Zhang. A. Unell, X. Wang. D. Ghosh, Y. Su, L. Schmidt, and Yeung-Levy, Why are visually- grounded language models bad at image classification? arXiv preprint 18845 2024.
[82] H. Zhou, X. Li, R. Wang, M. Cheng, T. Zhou, and C.-J. Hsieh. R1-zero's ss" aha moment" in visual reasoning on a 2b non-sft model. arXiv preprini arXiv:2503.05/33 2025.
[83] K. Zhou, J. Yang, C. C. Loy. and Z. Liu. Conditional prompt learning for vision-language models. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 16816 168652,, 2022.
[84] K. Zhou, J. Yang. C. C. Loy, and Z. Liu. Learning to prompt for vision-language models. International Journal of Computer Vision, 13009):2337-2348, 2022.
15

---

Contents
Introduction
2
RFT Methods: Think No Think
2.1
Thinking-RFT
2.2
No-Thinking-RRB -FI
3
General Visual Reasoning
3.1
Afink-Afte-AAsseer & Adaptive-Thinkig
3.2
Experimental Setup
3.3
Results & Findings
3.4
Parameter Change during RL
4
Case Study Image Classification Tasks
Experimental Setup
Few-Shot Learning Results
Free-Lunch Phenomenon
8
Efficiency Comparison
4.5
Convergence Comparison
10
Related Works
10
6
Conclusion
10
Group Relative Policy Optimization
18
B Detailed Related Works
18
B.1 MLLM for Image Classification
18
B.2 RL for Post Training
19
B.3 Rule-Based Reinforcement Fine-tuning
19
B.4
Overthinking in LLMs and MLLMs
19
Dataset Statisties and Details
19
Implementation Details
20
D.I
Dataset Construction for Classification
20
D.2
Implementation Details for Classification
20
D.3
Implementation Details for More Diverse Tasks
E
Prompt
21
Few-shot Prompt
Thinking-After-Answe
22
E.3
Adaptive-Thinking
22
16

---

More Experimental Results on Classification
22
Base-to-New Results
22
Open-set Classification Comparison
23
Results of Classification with 7B Models
24
More Discussion About Free- Lunch Phenomenon
24
More Experimental Results on More Diverse Tasks and Model Types
25
G.1
Experiments on More different Models
25
G.2
Referring Expression Comprehension Results
25
G.3
Experiments on Few-Shot Detection
26
G.4
Experiments of Thinking-Ri -FT inference with no-thinking tags.
26
More Discussion
26
Limited Gains in Puzzle Tasks
26
1.2
Discussion between offline DPO, No- Thinking-Ree RFT, and Thinking-Ree RF
27
Examples of the Model Response
28
17

---

Table 8: Statistics and questions for each Dataset.
Dataset
Classes
Train
Val
Test
Question
ImageNet
1,000
1.28M
N/A
50,000
"What type of object in the photo?"
Caltech101
100
4,128
1,649
2,465
"What type of object is in the photo?"
OxfordPets
37
2,944
736
3,669
"What type of object is in the photo?"
StanfordCars
196
6,509
1,635
8,041
"What type of object in the photo?"
Flowers
102
4.093
1,633
2.463
"What type of object is in the photo?"
Food101
101
50,500
20,200
30,300
"What type of object is in the photo?"
FGVCAircraft
100
3,334
3,333
3,333
"What type of aircraft is in the photo?"
SUN397
397
15,880
3,970
19,850
"What type of object is in the photo?"
DTD
47
2,820
1.128
1.692
"What type of texture is in the photo?"
EurOSAT
10
13,500
5,400
8,100
"What type of object is in the centered satellite photo?"
UCFI01
101
7,639
1,898
3,783
"What actions is the person performing in the photo?"
A Group Relative Policy Optimization
employ Group Relative Policy Optimization (GRPO) as the RL algorithm for optimization. Unlike SFT, which optimizes models through token-level losses, RL-based methods like GRPO utilize policy gradients, calculated from reward loss, for optimization. This encourages reasoning by exploring much larger solution space [19].
We follow Deepseek-R1
Let Q be the qestion set, uuw be the policy model and {01, (2, ... .o } be group of response from Thay for question Let Te., denote the frozen reference model. The GRPO algorithms aim to optimize model IT by the following objective:
JGRPO (0) = E B ~TR
a
To(osla) Te.a (0:1g) A, clip
To(0:14) Tean (oslg))
min
(.00))1111+10 (,1+) A; (Toll(
e,
G
where e and / are clipping (KL) penalty, respectively. Here, A;
and the coefficient controlling the Kullback-Leible mean( (f1.).....G is the advantage using the group reward
snd (ri rr..........cl
is the KL divergence loss to prevent current model Ty deviating too much from reference model Te. GRPO eliminates the critic model in PPO by estimating the relative advantage by sampling group of responses ) and normalizing their rewards within the group to compute relative advantage, which is more computationally efficient (59].
fri, P2, rrch, and Dxz(Fel(TTe) = mad (erlogg) [g) - log [aelor/4)
(a(o))))
B Detailed Related Works
B.1
MILMM for Image Classification
Image classification at the core of machine learning research, driving many fundamen- tal advancements in theoretical understanding and practical applications. Early CNN-based models perform classification by hierarchically extracting spatial features through convolutional layers and mapping them class probabilities. Vision transformers largely follow the same pipeline but replace convolution with self-attention (70) for global feature modeling. CLIP (55) classifies images by computing the similarity between the image embedding and text embeddings of category descriptions and then selecting the closest match. More recently, the advent of MLLM enabled new paradigm for image classification that leverages both vision encoders and LLMs. Unlike traditional classifiers, MLLMs take images and natural language prompts as input and generate text-based outputs, making them more interpretable, flexible, and user-friendly. However, MLLM for image classification is still an underexplored area. Recent work [81] shows that MILMs perform poorly at classification but can be improved with SFT. In this work, we take a different angle and investigate a novel approach: using rule-based RI. to fine-tune MLLM for classification. We systematically study its effectiveness compared SFT and show its advantage and potential improvements.
35
32
(66
20 37 22 67
15
18

---

B.2 RL for Post Training
Reinforcement learning (RL) has become an important technique used in LLM and MLLM post- training. Introduced by InstructGPT [50] as Reinforcement Learning from Human Feedback (RLHF), RL was used to align LLM and MILM output with human's 's preference. Due to the computation
inefficiency of PPO (58] used in RLHF, some offline RL algorithms 47 566] and value model free
RL algorithms
59 23
were introduced in the post training stage. Besides, researchers also explored
how to build a good reward model in terms of robustness and reward density [73] 18 39 Recently, DeepSeek-RI [9] applied the rule-based reward to the reinforcement training of LLM, proving the huge potential of RL terms of incentivizing LLM's reasoning ability. Inspired by DeepSeek-RI. RI, we applied reinforcement learning the classification task with rule-based reward function, and analyzed its generalizability and learning efficiency compared to supervised fine-tuning.
B.3 Rule-Based Reinforcement Fine-tuning
Rule-based reinforcement fine-tuning (RFT) has recently achieved much process in large language To transfer this success MLLMs, numerous studies have been proposed [4T 34 53). These works aim to replicate phenomena observed in Deepseek-Re -els such as increased response length and the emergence of "aha" moments. However, the role of the thinking process in RFT has not been explored in depth in these works. In this paper, we investigate its impact across different model sizes and tasks.
models 19
26
68
60
82
24
46
B.4 Overthinking in LLMs and MILMS
Recent advancements in sophisticated reasoning abilities enabled by techniques such as Chain-of- Thought (CoT) prompting [74] have marked significant milestone in the development of large language models (LLMs) and multimodal LLMs (MLLMs). CoT allows models to generate inter- mediate reasoning steps when solving complex problems, thereby improving both transparency and performance. However, this capability has also introduced a notable challenge referred to as the "overthinking phenomenon" (65)
Overthinking describes the tendency LLMs and MLLMs to produce unnecessarily verbose, redundant, and computationally expensive reasoning chains, even for simple queries. This behavior can hinder practical deployment and, in some cases, degrade performance (40 4 10
75
28
78, [T]. For example, [4)] shows that CoT can harm accuracy on tasks where extra deliberation impairs human performance. [4] quantifies overthinking in powerful LLMs and proposes pruning strategies, [10] shows excessive internal reasoning degrades success of ALM-based agents and (27 demonstrates that reasoning offers limited benefits on commonsense tasks. Additionally, [28] shows that overthinking can negatively impact MILM inference accuracy and (88 enhances CoT reasoning by representing its steps in a continuous space which makes CoT more efficient and improve performance.
While these studies provide valuable insights into model reasoning behavior, they focus exclusively on inference. The effect of explicit reasoning during reinforcement fine-tuning (RFT), however, remains largely unexplored.
C Dataset Statisties and Details
Following CoOp 84
we conducted extensive experiments on 11 public classification benchmark datasets to evaluate the effectiveness of the proposed CLIPFit. The datasets are ImageNet [14),
1altech101 [16). OxfordPets [5]] StanfordCars BI,, Flowers102 48), Food101 (2, FGVCAireraft
SUN397 76). DTD 9 EuToSAT (21). and UCF101
63 The dataset statistics and the
questions for each dataset are shown Table 8
For general reasoning dataset, CVBench a benchmark for spatial reasoning and SAT is a training dataset for spatial reasoning. PuzzleVQA is a dataset for visual puzzle reasoning. Mist is for general math reasoning and Vision is for competition math reasoning.
The examples of these datasets can be found in Appendix
19

---

Table 9: Comparison of Qwen2VL instruct, SFT, Thinking-RFI, and No-Thinking-RFF in the base- to-new generalization setting. No-Thinking: No- No-Thinking-RFi Base: base class accuracy. New: new class accuracy. H: harmonic mean accuracy. No- No-Thinkin:: No-Thinking-REE
(a) Average over 11 datascts.
(b) ImageNet.
(c) Caltech101.
Base New
II
Base
New
II
Base
New
II
Qwen2VL
62.1 66.27 64.12
Qwen2VL
61.56 74.9 67.58
Qwen2VL
88.83 92.9 90.82
SFT
67.4 70.73 69.03
SFT
27.78 47.78 35.13
SFT
93.87 93.01 93.44
Thinking-RFY RFT 81.17 79.15 80.15
Thinking-RRi RFT 88.12 90.01 89.05
Thinking-RRI RFT 97.74 95.2 96.45
No-Thinking
83.42 81.88 82.64
No-Thinking
88.97 90.66 89.81
No-Thinking
97.93 95.63 96.77
(d) DescribableTextures
(c) EuroSAT.
(f) Food101.
Base
New
I
Base
New
II
Base
New
I
Qwen2VL
60.99 61.34
61.17
Qwen2VL
54.52 63.54 58.69
Qwen2VL
74.07 80.42 77.12
SFT
71.98 71.41 71.7
SFT
91.55 77.87 84.16
SFT
74.27 77.82 76.0
Thinking-RIF
RRTT
77.42 67.82 72.3
Thinking-RFF
58.09 69.33 63.22
Thinking-RFI
87.29 87.56 87.42
No-Thinking
77.42 70.37 73.72
No-Thinking
66.43 74.13 70.07
No-Thinking
88.77 88.93 88.85
(g) OxfordFlowers.
(h) OxfordPets.
(i) StanfordCars.
Base
New
H
Base
New
H
Base
New
H
Qwen2VL
61.1 60.99 61.05
Qwen2VL
75.59 91.79 82.9
Qwen2VL
43,81 33.15 37.74
SFT
97.77 94.95 96,34
SFT
84.06 86.28 85.15
SFT
74.54 69.68 72.03
Thinking-RIi RFT
87.78 74.97 80.87
Thinking-
RFT
83.28 94.49 88.53
Thinking-
RFT
82.08 75.74 78.78
No-Thinking
88.71 76.73 82.29
No-Thinking
86.64 95.5 90.85
No-Thinking
91.13 87.04 89.04
(i SUN397.
(k) UCFI01.
(l) FGVCAircraft.
Base
New
H
Base
New
H
Base
New
H
Qwen2VL
56.97 65,8 61.07
Qwen2VL
69.6 64.62 67.02
Qwen2VL
36.07 39.47 37.7
SFT
27.39 37.8 31.77
SFT
59.95 63.93 61.87
SFT
38.23 57.53 45.94
Thinking-RIe RFT
81.03
82.52 81.77
Thinking-RFi RFT
79.47 74.95 77.14
Thinking-RFF
RFFT
70,53 58.07 63.69
No- Thinking
83.18 84.14 83.66
No-Thinking
80.47 79.18 79.82
No- Ro-nhing
68.01 58.31 62.79
D Implementation Details
D.1 Dataset Construction for Classification
Due to computational resource and model input limit, we select subset of class labels as the choice list. For few-shot classification, we sample 40% of labels including ground truth labels to form the classification list in question. For base-to-new classification, we sample 80% of labels including ground truth labels to form the classification list in question. If the length of the final choice list is less than 30, we include all class labels in the choice list, and if the length of the final choice list is larger than 100, include 100 class labels in the choice list.
D.2 Implementation Details for Classification
We implement our code in Pytorch [52]. We utilize Siwen--V-2B--A
722 as the base model, and fine-tune all parameters during training, following [8.] All training is conducted in 8 A100 GPUs. The batch size set to per GPU and we use 2-step gradient accumulation during training. All images are resized to 328x328 resolution with no data augmentation. We first extract answers from answer tags (<answer> ...</answer)) and then verify class names are in answers. If the answer tag does not exist in model responses we directly verify if class names are in model responses, following [1]. Both the maximum prompt length and maximum response length are set to 1024 for all datasets except StanfordCars dataset. The maximum prompt length and maximum response length are set to 1024 for StanfordCars dataset are set to 2048 and 1024, respectively. The number of rollout is set to 4 for both 2B and 7B models. 3 is set to 0.04, learning rate is set to x 10-6.
20

---

D.3 Implementation Details for More Diverse Tasks
Visual Perception. We follow 82 to fine-tune models on SAT dataset 57 epochs and then test on CVBench dataset [69]. We also include classification results for comparison. The maximum length of prompt and response both set to 1024. The number of rollout is set to 4 for both 2B and 7B models. The number of rollout is set for both 2B and 7B models. 3 is set 0.04, learning rate is set to 1 x 10-6. The batch size is set to 1 per GPU and we use 2-step gradient accumulation during training.
Multi-Modal Math Reasoning, We utilize the Math-40K 66 as the find-tuning data and fine-tune models epoch. Then test the
models on both Math Vista 43) and Math Vision [71] The maximum length of input prompt and response are set to 4096 and 512 respectively. The number of rollout is to 8 for 2B model, and 4 for 7B model. |3 set 0.04, learning rate is set x 10-6. The batch size is set to 1 per GPU and we use 2-step gradient accumulation during training.
Visual Puzzle Reasoning. We follow the code of 7 to generate training dataset with 6.5k data and fine-tune models epochs. We then test fine-tuned models on PuzzleVQA (7) as in-domain testing, and on AlgoPuzzle QA (17] as out-of-domain (OOD) testing. 3 is set to 0.04, learning rate is set to 1 X 10-6, The batch size is set to per GPU and we use 2-step gradient accumulation during training. The maximum length of prompt and response is both set to 1024. The number of rollout is set to 8 for 2B model, and for 7B model.
E
Prompt
E.1
Few-shot Prompt
We provide the prompt for thinking answer extract here. We use classification as the example, and for other datasets, the only difference is the few-shot examples.
Prompt:
will give you question, answer and the model response. The model response is The thinking process is in <think-
in the form
mgg>..</hinnking> and the answer <answer>...<ansvee
I want to to extract the answer in thinking tags and see if it is the same as the answer in answer tag. For multichoice problem, extract the choice (e.g.. A,B,C) that corresponding to the answer meaning of thinking process in thinking tag. If there is no answer in thinking tag, output None. Please output three items answer in thinking tag: answer answer tag: consistency with answer tag: Below is some examples:
Question: Whut type of texture is in the photo?
Please choose one from list | perforated, marbled, zigzagged, waffled, meshed, flecked, sprinkled, matted, woven, grooved, gauzy, bumpy, polka-dotted, potholed, veined, frilly, grid, bubbly, cobwebbed, scaly, pleated, crosshatched, wrinkled, freckled, porous, stained, swirly, chequered, spiralled, fibrous). answer: veined
model response: <think> The texture in the photo appears to be a series of parallel lines or grooves running along the surface of the leaf. This suggests grooved texture. </think> <answer> grooved <Janswer>
answer in thinking tag: grooved answer in answer tag: grooved
consistency with answer tag: Yes
Question: What type of texture is in the photo?
Please choose one from list fibrous, striped, chequered, swirly, studded. lacelike, knitted. freckled, crosshatched, wrinkled, cobwebbed, grooved, waffled, grid, scaly, bubbly, sprinkled, paisley, spi- ralled, interlaced, porous, veined, blotchy, pitted, zigzagged, crystalline, flecked, potholed, cracked, honeycombed].
answer: potholed
model response:<thinks The photo shows a road with visible damage, including cracks and pot- holes. The texture appears to be charucterized by these irregularities and depressions in the surface </think> <answer> potholed </answer>
answer in thinking tag: None
answer in answer tag: potholed
21

---

Table 10: Comparison of Adaptive-Thinking model response type among 2B and 7B models on Math Vista, Math Vision, CVBench, PuzzleVQA (Puzzle), and AlgoPuzzleVOs (AlgoPuzzle).
Model
Response Form
MathVista
Math Vision
CVBench
Puzzle
AlgoPuzzle
w/ Thinking 2B
0%
0%
0%
0%
0%
w/o Thinking
100%
100%
100%
100%
100%
w/ Thinking 100%
100%
99.6%
0%
0%
7B
w/o Thinking
0%
0%
0.4%
100%
100%
consistency with answer tag: None
Question: What lype of texture is in the photo?
Please choose one from list [ dotted, knitted, woven, honeycombed, bubbly, lacelike, swirly, interlaced. spiralled, marbled, zigzagged, pleated, stratified, gauzy, chequered, fibrous, paisley, lined, striped, studded, grid, crosshatched, crystalline, blotchy, stained, wrinkled, frilly, scaly, waffled, perforated]. answer: spiral
The photo shows a pattern with concentric circles that twist and turn spiral-like manner. This suggests that the texture is related to the spiral pattern, </think> <answer> swirly </answer>
model response: <think>
answer in thinking tag: spiral
answer in answer tag: swirly
consistency with answer lag: No (two more examples)
E.2 Thinking-After-Ansse
The prompt of Think-After-Answer is designed as: [Question] Please first output the answer in <answer> </answer> tags and then output brief reasoning process <reason> </reason> tags.
E.3
Adaptive-Thinking
We provide more discussions about Adaptive-Thinie Thinking method in this subsection.
Instruction Prompt. (Question). Please first identify whether this problem requires intermediate thinking or calculation. If the problem requires thinking or calculation, output the thinking and calculation process inside <think> </think> tags and the final answer inside <answer> <lanswer> tags. If no thinking or calculation is required, directly output the final answer inside <answer> </answer> lags. Your ouiput should follow one of two cases: (1) '<answer> <lanswer> ', (2) '<think> </think> <answer>... ... </unswer>'.
Format Reward. Both the thinking format (<think>... </think> <answer> </answer>) and the direct-answer format (<answer> ... </answer>) will be assigned a reward of 1.
Accuracy Reward. The accuracy reward not changed as Thinking-RFII
More Experimental Results on Classification
F1 Base-to-New Results
In this subsection, we introduce the results on the base-to-new generalization setting, Following [84]. We split each dateset into two disjoint groups: the base class dataset and the class dataset. This experimental setup is designed to assess the model's ability to acquire downstream knowledge while simultaneously demonstrating generalization novel information. All the methods are fine-tuned on the base class dataset and evaluated on both the base and new class test dataset. We conduct 4-shot experiments. The results are shown in Table
As demonstrated in Table 9 our proposed Thinking RFT framework notably surpasses SFT in performance. In detail, Thinking-RFT exhibits an approximately 14% higher accuracy for base classes and 9% increase for new class accuracy, leading to an aggregate improvement of in the harmonic mean accuracy. These results demonstrate the effectiveness of rule-base reinforcemnet
22

---

Response Length
Response Length
120
100
80
80
60
40
20
20
500
1.54
500
1.5k
TT Toikkiing
MMppte TTikkin
(a) SAT 2B Respomse Length
(b) SAT 7B Response Length
3B Bexporse Langth on Math
78 Response Length on Math
500
2.59
2.56
(c) Math4Uk 2B Response Length
Matb40k 7B Response Length
Response Longth
7B Response Longth
200
50
50
Stagle
200
400
600
800
200
400
600
800
Twikking
Adaptte-hhindingg
Sirkkinnngg Tainknn
(0) PuzzleVQA 2B Response Length
(f) PuzzieVQA 7B Response Length
Figure 6: Comparison of response length trend on SAT, Math40k, and PuzzleVQA datasets over steps of Thinking-RFT, No- No-Thinking-RFT and Adaptive-Thinkinn across 2B and 7B models.
fine-tuning in image classification. However, we also find that SFT can surpass Thinking-RFF -FFT in certain datasets, such as OxfordFlowers and EuroSAT, indicating that SFT may hold advantages in specific scenarios. We also observed that SFT significantly underperforms on the ImageNet and SUN397 datasets. The likely reason is that the prompts in these two datasets are very lengthy, preventing SFT from effectively memorizing classification knowledge, thereby resulting in poor performance.
The proposed No- No-Thinking-RTe demonstrates superior performance, in both base class and new class average accuracy, resulting 2.5% enhancement in average harmonic mean accuracy, compared with Thinking-RFi RFF. These findings suggest that omitting the thinking process during fine-tuning allows cule-based RL to achieve improved classification performance and enhanced generalization capabilities than with thinking process.
F.2 Open-set Classification Comparison
We present the results of open-set classification using Thinking-Rii RFT and No- Thinking- RFT in this subsection. Unlike closed-fone classification, open-set classification is not well-defined problem, which is much harder or even unrealistie for some datasets, since synonyms, plural forms, and name partially missing will be judged as incorrect. For example, in the StanfordCars dataset (31, the model can hardly output the correct year of the car in images. Therefore, we selected five datasets with class names that are relatively straightforward for the model to output and compare the few-shot
23

---

Table 11: Comparison open-set Few-shot learning results.
ImageNet
Caltech101
Food101
Flowers 102
OxfordPets
Average
Qwen2VL 46.57
62.96
57.79
48.44
47.40
52.63
Thinking Thinking-RFT RFT
54.84
79.07
73.51
67.64
89.94
73.0
No-Thinking-RFT 56.45
86.29
71.99
71.21
86.07
74.40
Table 12: Comparison of Thinking- and No-Thinking-RFT with 2B and models on fewshot
classification. DTD: DescribableTTextures
Model
Method
DTD
EuroSAT
OxfordFlowers
StanfordCars
Average
Thinking-RFi
69.92
49.46
86.56
80.24
71.55
2B
No-Thinking-RRl 73.52
58.02
91.6
92.5
78.91
7B Thinking-Re
77.90
53.17
93.91
84.19
77.29
Think-After-Answar
76.29
62.95
94.84
89.32
80.85
Adaptive- Thinking
79.60
56.62
96.86
89.92
80.75
No- Thinking-RR 80.56
58.91
94.24
94.02
81.93
learning performance between Thinking-RFT and No-Thinking-RFT The results are shown Table 11 No-Thinking-RFF outperforms Thinking-RUP RFFT on three datasets among five datasets, ultimately achieving 1.4% improvement average accuracy over Thinking-RFT)
F.3 Results of Classification with 7B Models.
We further conduct experiments of classification on DescribableTexxtures EuroSAT, OxfordFlowers, StanfordCars datasets using -wen-VL-2-7B models on and report the results Table 12 As shown in Table 12, compared with 2B models, the performance gap between Thinking-RFF RFF and No- Thinking-RFF narrows but remains. These results suggest that image classification does require explicit thinking, and that excluding the explicit thinking process can enhance both performance and computational efficiency.
F.4 More Discussion About Free-Lunch Phenomenon
In this subsection, give discussion about the free-lunch phenomenon. Despite improvement in most cases for cross dataset improvement in Figure 4 it is also noted that such improvements can be negative certain instances. For example, ane-tuning on the EuroSAT dataset could result in diminished performance on the OxfordPets dataset. This outcome is understandable, given that the knowledge required for the EuroSAT dataset significantly divergent from that of the OxfordPets dataset, and the classification knowledge from the EuroSAT dataset may even be detrimental to the classification of the OxfordPets dataset.
We further illustrate the comparative improvement of No- Thinking--nFF over Thinking- -RTT by testing 11 datasets, using a model that was fine-tuned one specific dataset. We also showcase the comparative improvement achieved by using a standard training prompt over a No-Thinking Prompt (which directs Thinking-RFT shown Figure 7 We can find that No- No-Thinking-RRe has better cross-dataset generalization ability than Thinking-RFi except OxfordPets and FGVCAircraft datasets. This suggests that utilizing an equality reward without engaging in any thinking process can enhance the model's cross-dataset generalization ability, with the exceptions being the OxfordPets and FGVCAircraft datasets. For these datasets, the thinking process during fine-tuning appears to be important for cross- cross-dataset performance. These findings are consistent the results of using different inference prompt strategies of Thinking-RFI as shown in Figure (b). We find that using normal prompt for in-domain fewshot test performs better than No-Thinking prompt among all datasets. This reasonable given that the normal prompt is the one used for fine-tuning. However, it is surprising to discover that using the normal prompt for cross-dataset testing results in lower performance many cases compared to the No- Thinking prompt, except OxfordPets and FGVCAireraft datascts. These findings suggest
immediately produce the answer during inference). The results

---

Improvement No- wootthinknng-ee AL compared with CLS
Imagenet
CAt
Ca Rechtog
S14
0:6
123
011
131
231
401
Saroday
4.1
144
000
031
10
Fondt01
Oxfondd lowers
631
611
6.1
GofordFlowar
0.00
104
10M
StanfordCars
10:9
410
14M
9111
13.0
14:0
1011
StanfordCars
(10)
4.0
443
652
0.9
UCP101
640
111
UCPL01
roVCArcruft
Callectii
Poaterss
Bernndr
AVD
Insegemet
Calbectol
Kamskar
Fowetall
DefortPret
(a) No- Thinking--RR RFT vs Thinking- RFT
(b) Normal prompt vs Noinking prompt
Figure 7: Visualization of improvement No- No-Thinking-RFT on different datastes compared with Thinking-RFF (Left) and improvement of Thinking- compared with Thinking- -RFT with No- Thinking prompt (Right). The accuracy improvement is marked as red, and decrease is marked as blue
SAT 70
Puzzle 7B
0.8
0.6
Reward
Puzzle Reward
Figure 8: Comparison of accuracy reward convergence speed on Puzzle dataset steps of Thinking- RFT, Think-After-Annswer, adn No-Thinking-Pe across 2B and 7B models.
that while Thinking-RFT fine-tuning may enable model to learn good cross-dataset classification generalization ability, over-thinking during inference can potentially diminish this ability.
G More Experimental Results on More Diverse Tasks and Model Types
G.1 Experiments on More different Models
We report more experimental results on more diverse model types. We conduct experiments on Intern VL2.5-IB and VL2.5-4B [3], and Doven----V--nnsructt-e
and Qwen2-5- -eon--5-V--Inssee 5--VL-Instruct 7B [.. The experimental results are shown in Table [4]and Table [3] The results are almost similar to which in the main text, i.e., visual and puzzle tasks do not need thinking during RFT.
G.2 Referring Expression Comprehension Results
We follow VLM-RI 60 to conduct experiments the referring expression comprehension task. The experiment setting and implementation is the same as VLM-RI [0)] We follow (60) to use the training splits Refcoco/+/g [79] 45) as the training data, and the val split Refeoco/+/g for in-domain evaluation and test split of LISA- LISA-Grounding
33 for out-of-domain (OOD) evaluation.
25

---

Table 13: Results of Thinking--RR RFT and No-Thinking-REE RFT on CVBench across different models.
Model
Setting
Overall
Count
Relation
Depth
Distance
Thinking-RFT 68.57
62.82
80.46
75.33
56.50
IntemVVL-2.5-BB
No-Thinking-RFF
70.55
65.48
77.08
78.67
62.00
Thinking-Re -FFT
79.34
71.32
91.08
87.17
69.33
Intem 2
MooTThnnking-Re
79.76
70.94
89.85
86.50
73.67
Thinking-REE 76.76
70,43
86.46
79.67
71.67
Qwen- VL-2.5-3B
No-Thinking-Pe
79.83
70.18
89.54
83.67
78.17
Table 14:
Performance comparison of PuzzleVQA and AlgoPuzZIeVQA on Qwven--5-VI-lnsruutt-7 models under three settings.
Model
Method
PuzzleVQA
AlgoPuzzleVQ
Thinking-RFT No-Thinking-RF7
77.10 86.45
27.72
Qwen-VL.-2.577
26.11
to report the evaluation results of first 600 steps. The results are shown in Table 15 We observe that No-Thinking-RFF generally achieves better performance than VLM-RI on the Re/COCO/+/g in-domain tests but performs worse on the LIS LJSA-Grounding out-of-domain (OOD) test. These results suggest that for referring grounding tasks, reinforcement fine-tunin fine-tuning (RFT) without explicit thinking can improve in in-domain performance while potentially degrading OOD generalization. However, also that as fine-tuning progresses, the OOD performance of No- No-Thinking-RRFT improves and consistently surpasses that of SFT.
We follow VLM-RI [60]
G.3 Experiments on Few-Shot Detection
We follow Viusal-RFT 41 to conduct experiments on few-shot object detection with MLLMs. We follow 41 to select eight categories from the COCO dataset with 4 images per category. to construct training dataset. We then follow [41 train the Qwen2-VL VL-2B innttuttttttt-2 and L0we2-VVL-25 instruct-7B models for 100 steps (50 epochs). The results are shown in Table [6 No-Thinking-RFT consistently outperforms Thinking-Ro among both 2B and 7B models.
G.4 Experiments of Thinking-RRi RFT inference with no-thinking tags.
We further conduct experiments to investigate where the performance gain of No- No-Thinking-RFI over Thinking-Ri on perception and puzzle tasks originates. We explore the No-Thinking inference mode of Thinking- by appending inference. The results are shown in Table I7 We find that while appending empty thinking tags during inference could improve performance on CVBench and puzzle tasks, its performance is still far behind No- Toinhiinnnneee RFT. Moreover, appending empty thinking tags during inference will decrease performance math tasks. These results suggest that the performance gain of No fo-Thinking Thinking-RFT RFT over Thinking-RE -FT on perception and puzzle tasks primarily stems from two factors: improved learning during fine-tuning and the avoidance of overthinking by bypassing inference.
empty thinking tag (<thinking> </thinking>) during
H More Discussion
H.1 Limited Gains in Puzzle Tasks
In this subsection, we discuss the explanation accounts for limited puzzle gains. As pragmatic proxy, we evaluated the o4-mini model, whose API supports optional in-context visual reasoning (e.g., cropping). On PuzzleVQA, performance rises from 84.30% (no visual reasoning) to 86.48% (with visual reasoning). a gain of 18% accuracy. This suggests that visual-context learning can improve puzzle performance.
26

---

Table 15: Performance comparison of Refcoco/+/g and LISA ground datasets.
Training method
Evaluation Dataset
100
200
300
400
500
600
SFT
Refcocoval
88.7
88.85
88.7
88.25
88.85
88.7
VLM-RI
RefcocOyal
88.7
88.7
89.4
89,25
90
90.55
No- No-Thinking- -FFTT
RefcocOval
90.29
90.85
90.52
90.88
90.85
90.83
SFT
Refcoco+
82.55
82.15
81.85
81.9
82.3
82.25
VLM-RI
Refcoco
82.6
81.9
82.8
83.35
83.6
84.3
No- No-Thinking-Pe RFT
Refcoco+
83.71
84.39
84.76
85.02
85.28
85.24
SFT
Refcocogval
85,65
85.95
85,85
85.6
85.95
85,95
VLM-RI
Refcocog.al
85.95
85.05
85,45
85.65
87.15
87.1
No- No-Thinking--ee
Refcocog.al
85.70
86.15
86.70
86.60
86.97
86.91
SFT
LISA- IISA-Grouniing
55.91
56.51
55.66
55.18
55.66
54.82
VLM-RI
LISA USS-Grounding
61.82
61.27
61.64
62.6
61.88
63.14
No- Thinking--Rri
LISA Lroondnun
57.90
57.24
58.99
59.65
59.95
61.76
Table 16: Performance comparison of 4-shot results on COCO dataset of 8 categories.
fire hydrant
stop sign
bys
train
cat
deg
hed
toilet
Models
mAP
Qwen2-VL-22B
Zero-Shot
19.6
19.0
15.8
25.8
18.4
29.9
23.2
14.6
9.8
Thinking-RT RFT
40.6
30,0
40.6
45.7
35.0
60.9
44,9
24.6
43.1
No-Thinking-RFT
43.72
32.29
44.64
47.38
43.43
61.59
$2.94
24.81
42.71
Qwen2-VVL. --V177
Zero-shot
43.0
35.0
43.3
37.1
36.7
$7.3
50.3
37.4
47.1
Thinking-RFi RFT
54.3
44.3
59.8
52.0
46.0
72.7
62.8
41.9
55.0
No-Thinking-RFF
54,99
49.10
58.72
53.37
45.52
72.35
63.64
46.41
50.79
H.2
Discussion between offline DPO, No-Thinking-RFF, and Thinking-RFT
In this subsection, we discuss the difference between offline DPO, Jo-Thinking-RFTI and Thinking- RFT.
(i) Although No-Thinking--ei RFTT removes the "think" phase, list-wise DPO algorithm should not be expected to reproduce similar effect due to the fundamental difference between DPO and GRPO. We discuss the detailed difference below:
(a) Sampling strategy. List-wise DPO selects one positive and several negatives offline. In contrast, No-Thinkee No-Thinking-Rt RFFT samples responses online every training step: the mix (N - 1) but varies with task difficulty
of positives and negatives not fixed and model competence (from 0 N to N 0). Thus while removing thinking, No- Thinking-RFF RFT continues to explore and learn from fresh errors, whereas DPO does not.
(b) Gradient update rule. No-Thinking- RFT computes a policy-gradiens using verifi- able rewards (R = 1 for an exact match, R. 0 otherwise) and group-advantage formulation, i.e. GRPO. Listwise DPO, by contrast, minimises a cross-entropy loss over pre-computed list, functioning more like contrastive SFT. This distinction yields different learning dynamics.
Because of these differences. listwise DPO should not be expected to match the empirical behaviour of No-Thinking-Re
(ii) We also implemented preliminary DPO and listwise DPO baselines quantify the per- formance difference. We evaluated two regative-sampling strategies: one using samples generated by Qwen2-VL (DPO model) and the other using randomly generated samples (DPO random). Experiments were carried out on the FGVCAircraft classification dataset and the MathVQA dataset. The results shown in Table T8 indicate that both DPO vari- ants under both sampling strategies perform substantially worse than Thinking-REF RFT and
27

---

Table 17: Comparison of Adaptive-Thinking model response type among 2B and 7B models on Math Vista, Math Vision, CVBench, PuzzleVQA (Puzzle), and AlgoPuzzleVOA (AlgoPuzzle).
Model
Response Type
Math Vista
Math Vision
CVBench
Puzzle
AlgoPuzzle
w/ Thinking
44,90
16.45
70.36
52.50
24.78
28
w/o Thinking
41.9
15.79
71.72
62.90
27.50
No- No-Thinking-RFF
48.80
13.16
76.76
70.85
29.17
w/ Thinking
64.60
21.71
80.36
66.60
24.78
7B
w/o Thinking
61.20
22.04
81.62
76.85
27.95
No- Thinking-Re RFT
59.10
18.09
80.67
80.65
29.39
Table 18: Performance comparison of DPO, Listwise DPO, Thinking-RIT, and No- No-Thinking-RR
Model
FGVC
Math Vista
Math Vision
DPO random
46.23
40.15
12.3
DPO model
51.25
40.32
14.4
Listwise DPO random
47.19
40.78
12.45
Listwise DPO model
51.36
42.66
15.1
Thinking- RFTT
74.41
44.90
16.45
No- No-Thinking-RFI
74.41
48.80
13.49
No-Thinking-RPe RFT. This finding highlights the importance of online sampling and policy- gradient optimization in RFT. Moreover, model-generated negative samples consistently outperform random negatives in both DPO and listwise DPO settings, demonstrating that the quality of negative samples is critical DPO training.
I Examples of the Model Response
In this subsection, we delve into the content of the thought process in Thinking- Thinking-RFT among different model sizes and tasks. Specifically, we show We compare the response examples of four different thinking strategies in Figure Figure[[]~Figure 25 the examples of trivial reasoning responses of 2B models Figure T5 ~ Figure 20 the inconsistent responses of 7B models in Figure FFFuurr for 2B model responses, the content found in the "thinking" tags somewhat trivial, such as "This a photo <class>" (classification) or "To find the area of the overlap between the two squares, need to find the length of the diagonal of the smaller square." (Math Vision) which offers little to no benefit towards arriving at the final answer, or they might already represent the final answers themselves without any reasoning process. For 7B models, the content within the thinking tag is significantly more meaningful and contributes more effectively to the reasoning process leading to the final answers. However, frequently observe inconsistencies in the responses, where the content within the thinking tag diverges significantly from that in the answer tag. For example, the reasoning may support choice while the final answer provided is choice A. To quantify this issue, GPT-4o to extract the answers from thinking tag and answer tag and then calculate the proportion of such inconsistencies across each dataset. The detailed results are presented in Figure 2 reveals that reasoning tasks are particularly prone to inconsistencies between the thinking and answer tags. Notably, as illustrated Figure 2] the average accuracy of the answer tags in inconsistent responses is substantially higher than that of their corresponding thinking tags. This observation expected, given that the answer tags are directly optimized via an accuracy- based reward signal, whereas the thinking tags receive no explicit supervision. Additionally, we find that the average accuracy of answer tags inconsistent responses lower than the overall average accuracy across all responses. This indicates that maintaining internal consistency between reasoning and final answers conducive improved model performance.
Typically,
Our analysis
28

---

3
2
7
3
4
4
Question: What is the missing number of the part denoted with
question mark? Please choose your answer from: ['1', '2', '7', '5']
Ground Truth: 5.
Response from Thinking-RFT:
<think> The pattern in the rows and columns is as follows:
First row: 3, 2,
Second row: 3, 4,
Third row: 4, 7
The pattern in the rows and columns that each number in the second row is the sum of the corresponding numbers in the first and third rows.
So, for the second row:
7
-4+1=5
Therefore, the missing number in the second row So, the answer is 5. </think> I
8
Question: What is the missing number of the part denoted with question mark? Please choose your answer from: [7, 4, 5,
Ground Truth: 7.
Response from Thinking-RFT:
<think> The pattern the shapes that the number of sides of each shape increases by one from the bottom to the top. The shapes are triangle (3 sides), hexagon (6 sides), and an octagon (8 sides). Following this pattern, the next shape should have sides. The number in the center of the octagon is 8, so the missing number in the center of the pentagon should be Therefore, the missing number is </think>
<answer>5</answer>
Response from No-Thinking-RFi -F::
Figure 9: Examples of the inconsistency enror of fine-tuned Qwen- VL-2-7B model on Puzzle VQA. The correct answer part is marked green, and wrong answer part marked as red.
29

---

Question:
How many walls are in the image? A. 1 B. 3 C.O
Ground Truth: A.
Response from Thinking-RFT:
<think> The image shows window with curtains on both sides. The curtains are hanging from a rod that is attached to the wall. There is visible wall in the image, only the window and the curtains. Therefore, the number of walls the image is zero.</think> <answer> (A) </answer>
Question: How many rugs are in the image? A.3 B. C.2 Ground Truth: B.
Response from Thinking-RFT:
<think> The image shows living on the floor in front of the coffee table, and another rug on the floor in front of the fireplace. Therefore, there are two rugs in the image.</tink>> <answer> (B) </answer>
with various items and decorations. There is rug
Figure 10: Examples of the inconsistency error of fine-tuned DwennVVL-2-77 model on CVBench. The correct answer part is marked green, and wrong answer part is marked red.
30

---

Hint: Please answer the question requiring integer and provide the final value, c.g., 1, 2, 3, at the end.
Question:
Question: what is the total volume of the measuring cup? (Unit: g)
Ground Truth: 1000.
Response from Thinking-RFT:
<think> The measuring cup has scale marked in milliliters (mL). To convert milliliters to grams, we need know the density of the substance being measured. For flour, the density is approximately 0.5 g/mL. The total volume of the measuring cup is 1000 mL. Therefore, the total volume in grams is 1000 ml. 0.5 g/ml. 500 g. </think>
<answer> 1000 </answer>
12
11
10
3
8
6
Question: Hint: Please answer the question and provide the correct option letter, e.g., A, B,
C, D. at the end.
Question: What time is shown? Answer by typing time word, not number. It ( past
six.
Choices:
(A) half
(B) quarter
(C) clock
(D) quarter to
(E) quarter past
Ground Truth: B.
Response from Thinking-RFT:
<think The clock shows time where the minute hand is pointing the 3, which represents
15 minutes past the hour. The hour hand is between the 6 and the 7, which means it is 6 </think> <answer> </answer>
o'clock. Therefore, the time 6:15,
Figure 11: Examples of the inconsistency error of fine-tuned Qwen- VL-2-7B model on Math Vista. The correct answer part is marked green, and wrong answer part is marked as red.
31

---

Question: Hint: Please answer the question and provide the correct option letter, e.g., A, B, C. D. E, F at the end.
Peter rides his bike along cycle path in park. He starts at point S and rides in the direction of the arrow. At the first crossing he turns right, then at the next left, and then again to the right and then again to left. Which crossing does he not reach?
Emaggl>>hhocees A. A B. B C. D.
Ground Truth: D.
Response from Thinking-RFT.
<think> The arrow indicates the direction Peter rides his bike. He starts point and follows the path indicated by the arrow. He turns right at the first crossing, then left at the next, right again, and finally left. This means he will pass through the crossings labeled B, C. and D. The crossing labeled A is not mentioned in his path. Therefore, the crossing he does not reach is A.</think> <answer> </answer>
B
Question: Hint: Please answer the question and provide the correct option letter, e.g., A, B, C, D. E, F the end.
(very small) ball is kicked off from point A on square billiard table with side length m. After moving along the shown path and touching the sides three times indicated, the path ends in point B. How long is the path that the bal travels from A to B? (As indicated on the right: incident angle emergent angle.)
Choices: A. B. 2/ 13 C. 8 D. 4/3 E. 2 (V2 v3)
Ground Truth: B. Response from Thinking-RFT:
<think The path of the ball is a series right angles, so we can break down into segments of equal length. Each segment is the hypotenuse of right triangle with legs of length meter (since the side length of the square is 2 meters). Using the Pythagorean theorem, the length of each segment is I + = /2 meters. Since there are four segments, the total length of the path is 4v2 meters. However, we can simplify this to 2/2 4v2 meters. Therefore, the correct answer is D. </think <answer> </answer>
Examples of the inconsistency error of fine-tuned Owen-VLL2-77 model Math Vision. The correct answer part is marked as green, and wrong answer part is marked as red.
Figure 12:
32

---

You are playing a Tower of Hanoi game with 3 rods and 5 disks of various diameters, which can slide onto any rod. You are given the starting and ending configuration of the game as shown in the top and the bottom of the image, respectively. The game has the following rules: Only one disk may be moved taking the upper disk from one of the stacks and placing it on top of another stack or on an empty rod: and iii) No disk can be placed on top of disk that is smaller than What the minimum number of moves required to from the starting to the ending configuration? Please choose your answer from: 6, 4, 5.
Question:
time; ii) Each move consists of
Ground Truth: 5.
Response from Thinking-RFT:
<think> The Tower of Hanoi problem is classic puzzle that requires moving stack of disks from one rod to another, following specific rules. The minimum number of moves required to solve the puzzle can be calculated using the formula 2" 1, where n is the number of disks.
In this case, we have 5 disks, so the minimum number of moves required is 2 =
However, the options provided are 6, 2, 4, and 5. It seems there might be misunderstandding in the question or the options. The correct answer based on the formula 31, but it is not listed among the options.
Let's re-examine the image to see if there's any additional information or a different inter- pretation of the problem that might lead to different answer.
Upon closer inspection, the image shows specific starting and ending configuration, but does not provide enough information to determine the exact number of moves required to solve the puzzle. The image does not show the intermediate steps or the specific rules applied to reach the ending configuration.
Given the options provided, it seems there might be misunderstanding or an error in the question. The correct answer based on the formula is 31, but not listed among the options.
Therefore, the minimum number of moves required to go from the starting to the ending configuration is 31, but this is not listed among the options. Final answer: 31 (not listed among the options) >//hink>> <answero5</answer
Tarch
Question:
The image shows the calendar of a month of particular non-leap year. Which day of the week
February 25 of that year?
Please choose your answer from: Friday, Sunday, Monday, Tuesday.
Ground Truth: Friday.
Response from Thinking-RFT:
<think> The calendar shows that February 25 is Friday. Therefore, the day of the week for February 25 of that year Friday, </think>
<answer> Tuesday </answer>
Figure 13: Examples of the inconsistency error of fine-tuned Qwen- VL-2-7B model on AlgoPuz-
zle VQA. The correct answer part is marked green, and wrong answer part is marked red.
33

---

Question:
What type of texture is in the photo?
Please choose one from list matted, grid, crosshatched, frilly, flecked, zigzagged, freckled, waffled, chequered, porous, striped, fibrous, sprinkled, perforated, crystalline, cracked, banded, lacelike, veined, polka-dotted, paisley, scaly, pleated, smeared, dotted, cobwebbed, wrinkled, gauzy, interlaced, knitted].
Ground Truth: wrinkled.
Response from Thinking-RFT:
<think> The photo shows a close-up of a fabric with smooth, continuous surface. There are no visible patterns, lines, textures that would suggest grid, crosshatched, frilly, flecked, zigzagged, freckled, waffled, chequered, porous, striped, fibrous, sprinkled, perforated, crystalline, cracked, banded, lacelike, veined, polka-dotted, paisley, scaly, pleated, smeared, dotted, cobwebbed, wrinkled, gauzy, interlaced, or knitted texture. The fabric appears to be soft and smooth, with a consistent, even texture. Therefore, the most appropriate texture to describe this fabric is "pleated" </think>> <answer> wrinkled </answer>
Question:
What type of texture is in the photo?
Please choose one from list smeared, bubbly, fibrous, lacelike, frilly, potholed, spiralled, chequered, flecked, paisley, stained, meshed, wrinkled, zigzagged, polka-dotted, matted, freckled, lined, stratified, banded, cracked, sprinkled, gauzy, striped, marbled, crosshatched, swirly, pitted, grid, veined].
Ground Truth: striped.
Response from T'hinking-FFT:
<think <think> The photo shows pattern of vertical lines. The lines straight and evenly spaced, creating consistent and repetitive visual effect. There are no visible curves, ripples, other irregularities in the lines. Therefore, the texture in the photo is best described "lined." </think> <answer> striped </answer>
Figure 14: Examples of the inconsistency emrrr of fine-tuned Owen-VL-2-78 model classification DTD dataset. The correct answer part is marked as green, and wrong answer part is marked as red.
34

---

4
5
Question: How many cubes have been taken from the block? Ground Truth: 4.
Response from Thinking-RFT: <think
To solve this, we need to determine the missing number in the pattern. Let's examine the numbers and their positions.
</think>
<answer>4<<answer>
2
Question: What is the missing number of the part denoted with a question mark? Please choose your answer from: [2, 3, 4, 1]
Ground Truth: 2.
Response from Thinking-RFT: <think>
To solve this, we need to determine the missing number in the sequence. Let's analyze the pattern sequence: 2, 3, 1, 1, 1,
</think>
<answer>
</answer>
Figure 15: Examples of the trivial reasoning of fine-tuned Qwen-VLe VL-2-2B model on PuzzleVOS
35

---

Question: Alice has 10 segments of chains of different lengths as shown in the image. The total length of all the segments combined is 19 pieces. She has saw machine with which closed piece can be cut opened. She also has a welding machine with which an open piece be closed. Each
takes 5 minutes and each welding takes 5 minutes. Initially, she has 3 segments each with I open piece as shown in the image. All the other pieces are closed. She now wants make the longest possible necklace using all the available 19 pieces. Each piece in the necklace would be connected to exactly two other pieces. This would require cutting open some pieces and then joining all the resulting segments together. What is the minimum time in which she can create the necklace?
Please choose your answer from: 40, 41, 35,
Ground Truth: 35.
Response from Thinking-RFT: <think>
To create the longest possible necklace, Alice needs to maximize the number of segments she use. She has 10 segments, and she wants use all 19 pieces. The goal is to maximize the number of segments used while ensuring that each segment connected to exactly two other segments. </think>
Sanswer-35</answe
Question: A Rubik's Cube has six different coloured panels: red, green, blue, yellow, orange, and grey. The initial state of the cube in terms of the different colour positions in its six faces is shown in the image. To represent the movements of the cube we use six letters: U for Up. for Down, for Left, R for Right, F for Front, for Back. These letters used in sequence where you need to perform each letter in the sequence from left to right. Each letter tells you move that face clockwise by 90 degrees. number 'n' immediately after letter denotes that you need to move that face clockwise by 90 degrees. For example, 'U R3' would mean rotating the up face degrees clockwise and then rotating the right face 270 degrees clockwise. You perform the move sequence 'B B2' starting from the state shown in the image. What would be the number small * red squares the left face after completing the move sequence?
Please choose your answer from: 3, 7, Ground Truth: 3.
Response from Thinking-RFT. <think
To solve this problem, we need to understand the sequence of moves and the effect each move has the cube's state. The sequence is 'B B2', which means rotating the left face clockwise by 90 degrees twice. We need to calculate the final state of the cube after these moves.
</think>
aanswrr><</aswee
Figure Examples of the trivial reasoning fine-tuned Qwen VL-222BB model on AlgoPuzZIeVQA.
36

---

Figure 21-25
Problems 10, and
Question: Hint: Please answer the question requiring a floating-pon number with two decimal places and provide the final value, e.g., 1.23, 1.34, 1.45, at the end.
Question: In Fig. 21-25. the particles have charges Q1 -92 100nC and ds Q4 200nC, and distance 5.0 cm. What is the x component of the net electrostatic force on particle 3?
Ground Truth: 0.17.
Response from Thinking-RFT:
<think>
To find the x component of the net electrostatic force particle 3, need find the net force acting on particle 3 and then divide it by the charge of particle 3.
</think>
<answer> canswer>100</answe 100</answe>>
Question: Hint: Please answer the question requiring an integer answer and provide the final value, e.g., 1, 2, 3, the end.
Question: What the difference between the highest and the lowest time required to import
Ground Truth: 4. Response from Thinking-RFI.
<think
To determine the difference between the highest and the lowest time required to import goods in Mexico, we need to analyze the data provided the plot. </think>
<answer<<<answer>>
Figure 17: Examples of the trivial reasoning of fine-tuned Qwen-VLe VL-2-2B model on Math Vista.
37

---

Question: Hint: Please answer the question and provide the correct option letter, e.g., A, B, C, E, F the end.
Two squares with side 1 have a common vertex, and the edge of one of them lies along the diagonal of the other. What the of the overlap between the squares?
<imagel>Choices: A. v2 -
B.
C.
-2-1
2
D. +1
E. v3 V2
Ground Truth: A.
Response from Thinking-RRi RFF: <think>
To find the area of the overlap between the two squares, we need to find the length of the diagonal of the smaller square.
</think>
<answe>>B</nswwr>
Jol
Question: <imagel>
In the picture above we see a cube two different positions.
The six sides of the cube look like this:
<image2>
Which side is opposite <image3>?
<image4>
Ground Truth: C.
Response from Thinking-RFT.
<think>
To determine which side is opposite to <image3>, we need to identify the opposite side of the cube <image4>.
</think>
<answer-D</anwwen
Figure 18: Examples of the trivial reasoning of fine-tuned Qwen-VL-2-21 model on Math Vision.
38

---

Question: How many chests are in the image? 3. B. C. 2.O
Ground Truth: B. Response from Thinking-RFT: <think>
The image shows bed with nightstand, a curtain, and toy chest. There is only chest in the image.
</think>
<answer> </answer>
Question: How many walls in the image? A. 1 BB. 0 3 Ground Truth: E.
Response from Thinking-RFT: <think>
The image shows a classroom with a blackboard, desks, and windows. The walls are painted in light color, and there are no visible windows on the back wall. Therefore, the correct answer (A) 1.
</think>
<answer> </answer>
Figure 19:
Examples of the reasoning of fine-tuned Qwen-VL--2-2B model CVBench. Unlike the trivial responses in reasoning tasks, the thinking process in CVBench is not trivial.
39

---

Championdealers.com
Question: What type of object is in the photo? Please choose one from caesar caesar_salad, Question: What type of object the photo? Please choose from list [ 2012 Acura TL Sedan, 2012 Chrysler Town and Country Minivan.... Response from l'hinking-FFT:
Question: What type texture is in the photo? Please choose from list spiralled, knitted, sprinkled...)
Response from Thinking-RF"
<think> The texture in the photo appears to be woven. </think>
<think> The object in the photo a 2012 Ford Fiesta Sedan. </think>
<unswer>Wovnnaanswer
<answer> 2012 Ford Fiesta Sedan</answer>
Question:
What type of aircraft is the photo? Please choose from list MD-90, Beechcraft 1900, DHC-6, MD-87, 777-300, CRJ-900...]
Question: What type of object the photo? Please choose one from list [ okapi, bass, revolver, cup. nautilus, ...] Response from Thinking-RFT.
Response from Thinking-RFF RFT: <think> The aircraft in the photo is an An-12. </think> <answer> An- 12</answer>>
<think> The object in the photo is a Joshua tree. </think> Siaswer>Joshua_tteee/aass
Question: What type of object is in the photo? Please choose one from list [ minia- ture prre_pinscher, yorkshire_terriee Response from Thinking-RFT.
list [ grilled__hhese____ lasagna,.. ...]
Response from Thinking-RFT:
<think>The dog in the photo appears to be Leonberger, breed known for its long, thick coat and large size. </think> <answer>l LewneeeeeLobbeege//nnswerrr
<think> The photo shows a dish of paella, which is traditional Spanish dish made with rice, seafood, vegetables, and spices.</thinks canswer>Paella</anseer
Figure 20: Examples of Thinking-RFT fine-tuned model responses across six datasets. The thinking content appears not very beneficial to deriving the final answers.
40

---

::11 15
Question: Hint: Please answer the question and provide the correct option letter, e.g., A, B,
C, D, the end.
Question: What time shown? Answer by typing a time word, not a number. is ( past
three.
Choices:
(A) half
(B) quarter
(C) clock
(D) quarter to
(E) quarter past
Ground Truth: B.
Response from 2B Thinking-RF's <think>
To determine the correct time shown on the clock, let's analyze the image:
The clock red and has the time displayed 3:15.
The time is shown as "quarter to" because it is 15 minutes past the hour.
</think>
<answer> D </answer>
Response from 2B <reason> The time shown is quarter past, as indicated by the "15" in the clock. The word "quarter" is used to describe the position of the hour hand between the two halves of the clock. Therefore, the correct option is E. </reason>
<answer> E </answer>
Response from 2B No-Thinking-RFFI D) Response from 2B Response from 7B Thinking-RFT: <think> The time shown the clock is 3:15. To determine the time, we look at the minute hand, which is pointing at 3, and the hour hand, which between 3 and 4. Since the minute hand is 15 minutes past the hour, it is 15 minutes past 3. Therefore, the correct answer is "quarter past" because 15 minutes quarter of hour. </think> <answer> B </answer>
<answer> E </answer>
Response from 7B Think-AAfer-Ansswerr <answer> B </answer>
<reason> The time shown on the clock 3:15. Since the minute hand is on the 3. which represents 15 minutes past the hour, the correct answer is "quarter past" or "quarter to". However, since the question asks for time word, "quarter past" is the appropriate choice. </reason>
Response from 7B No-Thinking-RFT: B Response from 7B Adaptive-Thinking <think> The time shown is 3:15. The minute part 15, which is quarter an hour. Therefore, time is minutes past 3. </think> <answer> B </answer>
Figure 21: Comparison of the model response of different fine-tuned model on Math Vista.
41

---

Question: Malaika is skiing on a mountain. The graph below shows her elevation, in meters, above the base of the mountain she skis along trail. In total, how many seconds does she spend at an elevation between 4 and meters?
Ground Truth: 8.
Response from 2B Thinking-Rie RFT: <think>
To find out how many seconds Malaika spends at an elevation between and meters, we need to find the area under the curve between these two values. </think>
<answer> <answer>10<<anssee 10</answer>
Response from 2B Think-After-Anwweer <answer> 10</ansser
<reason>She spends 10 seconds between 4 and 7 meters. The graph shows that she is at an elevation between and 7 meters for seconds. </reason>>>
Response from 2B No-Thinking-RFF7 10
Response from Adaptive-Thinking: <answer> 10 </answer>
Response from 7B Thinking-RFT: <think> The graph shows the elevation of Malaika over time. To find out how many seconds she spends at an elevation between 4 and meters, we need to look at the portion of the graph where the elevation is between and 7 meters. This is the area under the curve between the horizontal lines 4 meters and 7 meters. By examining the graph,
that Malaika spends approximately 6 seconds elevation between 4 and 7 meters. </think> <answer> </answer>
Response from 7B Think-A itte-Answer: <answer><6</answer>
<reason>The graph shows that Malaika's 's elevation is between 4 and 7 meters for 6 seconds. This is the time when the line is above the 4-meter mark and below the 7-meter mark on the graph. </reason>
Response from 7B No-Thinking-RFFI. 6
Response from 7B Adaptive Thinking: <think> The graph shows Malaika's 's elevation over time. To find the time she spends an elevation between and meters, we need look at the portion of the graph where the elevation is between 4 and 7 meters. This portion of the graph is between the time of seconds and 12 seconds. Therefore, Malaika spends seconds at an elevation between 4 and 7 meters. </think> <answer> </answer>
Figure 22: Comparison of the model response of different fine-tuned model on Math Vision.
42

---

Question: How many light switchs are in the image?
A. 1 B. OC 3
Ground Truth: A.
Response from 2B Thinking-RFT: <think>
To determine the number of light switchs the image, let's analyze the visible parts of the room:
1. The wall the left side has several light switchs. The wall on the right side has several light switchs. The ceiling has several light switchs. The floor has no visible light switchs.
3.
Based on this analysis, there are light switchs in the image. </think>
<answer> (C) </answer>
Response from 2B Shink-Alter-Answerr <answer>l</answe>>
<reason=There is only one light switch visible in the image, which is located on the wall to the right of the garage </reasos
Response from 2B No-Thinking-RFFT (A)
Response from 2B Adaptive-Thinking <answer> I/anser>></nnswr>>>>>>
Response from 7B Thinking-RF'T
<hhink>>Thrre are two light switches visible in the image, one on the wall near the ceiling and another on the wall near the floor. Therefore, the total number of light switches is 2.</think> <answer>(A) k</answer> Response from 7B
<answer> (A) </answer>
<reason> There is one light switch visible the wall in the image. </reason>
Response from 7B No-Thinking-RFT (A)
Response from 7B Adaptive-Thinkinn
<think There are two light switches in the image, one on the wall near the ceiling and another the wall near the floor. </think> <answer> A </answer>
Figure 23: Comparison of the model response of different fine-tuned model on CVBench.
43

---

?
Question:
What is the size the missing part denoted by a question mark? Please choose your answer from: ['large", 'medium', 'small']
Ground Truth: large.
Response from 2B Thinking-RFT:
<think>
To determine the size of the missing part denoted by the question mark, let's analyze the sizes of the shapes in the image.
</think>
<answer>larg<</answer Response from 2B <answer>omall<<nnwerr>>
<reason><Sine the shapes are arranged in way that the larger shapes are on the left and the smaller shapes are on the right, the missing part, which is the question mark, should be of the same size the square, which is medium in size. Therefore, the missing part is small.</reasee </reaon>>
Response from 2B No-Thinking-RIF RF:: medium
Glaptive-Thinkine <answer> large </answer> Response from 7B Thinking-RFT:
Response from
<think> The pattern in the image seems to be that the shapes are getting larger as we move from left to right and top bottom. The smallest shapes are the hexagon, square, and triangle, and the largest shapes are the large triangle and square. The missing shape is in the middle row, second column, which should be the medium-sized square. Therefore, the size of the missing part denoted by question mark is medium </think> <answer> large</answer> Response from 7B Think-Afe Afte--fftr--.er.:
<answer> large </answer> <reason>> The question mark is placed below the large triangle, indicating that the missing part should be of the same size as the large triangle. </reason>
Response from 7B No-Thinking-RFIT large Response from 7B Adaptive- Thinkking <answer> large </answer>
Figure 24: Comparison of the model response of different fine-tuned model on PuzzleVQA.
44

---

5
13
10
20
22
19
21
18
15
16
The board shown in the image is a sliding puzzle of # tile dimensions. It has 24 numbered tiles and one unoccupied (open) position. Tiles in the the open position can be moved by sliding them horizontally or vertically, respectively. All tiles always stay and move inside the red boundary wall, as shown the image. A move is defined as moving the open position by one tile unit in any available direction. You start from the board position shown in the image and perform exactly moves. How many unique final board positions can you reach?
Question:
or column of
Please choose your answer from: 3, 4,
Ground Truth: 5.
Response from 2B Thinking-REE
<think>To determine the number of unique final board positions that can be reached by performing exactly 2 moves, we need to analyze the possible moves that can be made from the given starting position and the constraints imposed by the sliding puzzle rules. Response from 2B Think-After-Answerr
<answer>5</answer
After performing 2 moves, the board can reach 5 unique final positions: (I, 1), (I, 2). (1,3), 3), (1, 4), and (1. 5). These positions are reached by sliding the open position in the first row and column of the board. The other positions are not reachable because the board cannot move the open position in the first row or column of the board. Therefore, there are unique final board positions. </reason>
Response from 2B Te-Thhinnng-FFFT Response from 2B adaptive-Thinking <answer> 5 </answer>
Response from 7B Thinking-RFTI <think> The puzzle a 5x5 sliding puzzle with 24 numbered tiles and one empty space. We need to find the number of unique final board positions after exactly moves. Let's analyze the possible moves:
Moving the open position to the left or right by one tile unit.
2. Moving the open position up or down by one tile unit.
Since the puzzle 5x5, there are possible moves for each direction (left, right, up. down). Therefore, there are total of 8 possible moves. However, we need to perform exactly 2 moves. The number of ways choose moves out of 8 given by the combination formula C(n, =)=!! (k!(n-k)!), where is the total number of moves and k is the number of moves we want choose. So, C(8, 2) = 8!/ (2!(8-2)!) = 28. However, we need to consider that some of these combinations might result in the same final board position. For example, moving the open position to the left and then to the right is the same as moving it to the right and then the left. Therefore, To do this, we can use the concept permutations. The number of permutations of items taken 2 at time is given by Pnn, k) n! / (n-k)!, where is the total number of items and the number of items to choose. So, P(8, 2) = 8! (8-2)! = 56. However, we need to divide this by 2 to account for the duplicates mentioned earlier. Therefore, the number of unique final board positions is 28 / 2= 14. So, the answer is 14.</think> <answer>5</answerr Response from 7B Think-Afee Saak--Afte-AAnssan
need to remove these duplicates.
<answer> 4 </answer>
<reason> The open position can move to any of the four adjacent tiles (up. down, left, right) in one move. Since there are 24 tiles and each tile be in one of the 25 positions (including the open position), there are 24 possible starting positions for the open position. After the first move, there are 4 possible positions for the open position (since can move any of the four adjacent tiles). Therefore, there are 24 4 96 possible final board positions. However, since the open position can move in any of the four directions, there are 4 4 16 possible final board positions. </reason>
Response from 7B No-Thinking-RFFI. 4
Response from 7B Adaptive- Slaptive-Thinking: <answer> 4 </answer>
Figure 25: Comparison of the model response of different fine-tuned model AlgoPuzzleVOA.
45

---

NeurIPS Paper Checklist
1. Claims
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Question:
Answer: [Yes]
Justification:
All contributions are clearly stated in abstract and Sec.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results
be expected to generalize other settings.
is fine to include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
The limitations about this paper is discussed Sec.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification. asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low images are taken in low lighting. Or speech-to-text system might be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
46

---

Justification:
This paper does not contain theoretical claims.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification: All implementation details stated in Sec. D
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make clear how to reproduce that algorithm. (b) the contribution is primarily a new model architecture, the paper should describe the architccture clearly and fully.
(c) If the contribution is new model (e.g., a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. the case of closed-source models, may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
47

---

reproducible, they Answer: [Yes]
Justification:
We will release all code, data, and models.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips. cc/
public/guides. Boolicc/guides//oddsbbbb..ssss
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https for more details.
/ //nips httpcc/cpplllgggiiueee /ttpg/ddde////ubbbiiiimmmm
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: All training and dataset details discussed in Sec. D and Sec. 4.1
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [No]
Justification: Given the high cost of fine-tuning, we do not report error bars. Please note that in Sec. and Sec. 3 spent huge resources for comparison of thinking and -h-ihiin study during RFT across 17 benchmarks and datasets, which makes it prohibitively to run each experiments for multiple times.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results accompanied by error bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
48

---

The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative emrr rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification:
We have include compute resouces information in Sec. D and Sec. 4.1
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics
Answer: [Yes]
Justification:
The research in this paper is with the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. if there is a special consid- eration due to laws or regulations their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: There is no social impact of this work.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
49

---

The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of MI..
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
This paper poses such risks.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Lieenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: All used datasets and other assets are cited and with proper license.
Guidelines:
The answer NA means that the paper does not existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
50

---

If this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
No new assets released in this paper.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/mmoel as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question:
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
16. Declaration of LLM usage
51

---

Question: Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods this research? Note that if the LLM used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Answer: [NA]
Justification: This paper does not involve LLMs any important, original, non-standard components
Guidelines:
The answer NA means that the method development in this research does involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy for what should or should be described.
52


Reviews:
[{"review_id": "5glPrkxjY1", "reviewer": "Reviewer_2mLo", "summary": "The paper investigates the usefulness of ‘thinking’ during RL training for small MLLMs (sizes 2B, 7B). Their main findings are (a) thinking during RL finetraining outperforms SFT (b) however, thinking actually seems to hinder training. Instead, doing direct RL on actions (without thinking) is more sample efficient and requires considerably less wall-time  (c)  They find that adaptive thinking RL can perform marginally better (compared to No-Thinking RL). On some reasoning tasks, like mathematical problems, they do observe a gain for Thinking RL.", "questions": "1. How likely is that the model is only learning to ‘format’ the answer correctly to be parsable by the chosen parser than learning new capabilities?\n2. Reasoning training on small LLMs often requires some sort of warmup; did the authors try any sort of SFT warmup; or generally provided any curriculum for thinking?", "limitations": "Limitations should be in the main paper, and more thorough.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "### Strengths\n\n1. Paper is well-written.  \n2. Experiments are conducted across range of datasets.  \n3. Paper has done a relatively ok job of disentangling some key hypotheses through creative experimentation.\n\n### Weaknesses\n\nNote to authors: Weaknesses are important to address. Either through including novel results, and/or including relevant discussion (e.g., in limitations), and/or convincing the reviewer that their critique is faulty. Improvements are things that I believe would be helpful to improve the paper but are not necessary to address (though addressing enough of them could mean I improve my score).\n\n1. The paper’s limitations section does not acknowledge its limitations in satisfactory way, and ought to be included in the main body.  \n2. The image classification datasets used are all highly popular ones. The paper does not discuss the ramifications of this. The dataset memorization could explain why no-thinking does better. I would ideally like the paper to test on novel synthetic image datasets (with potentially novel classes / objects) to mitigate this risk. Or if the authors don't think this is a risk, I would like to hear their explanation.  \n3. Image classification experiments results in the main paper are only presented for 2B model. 7B model results should also be in the main paper (currently table 10 in appendix) and done for all models. It seems gap shrinks significantly for 7B models between thinking / no-thinking RL (on OxfordFlowers it goes from 5 points to \\<.5 points).  \n4. The image classification setup reduces the number of classes. I imagine the number of classes as potentially a factor in model performance and would be interested in some analysis of this aspect. Would be interesting to understand how \\# of classes impacts model performance. Closest experiment to this is open-set classification experiments with results in Table 9 in which the gap shrinks significantly.  \n5. I am somewhat concerned that the RL training setup used in the paper is not optimal; no SFT warmup is used, no curriculum is used etc. So, another plausible explanation of the paper’s results could be that RL training was not done well. It is somewhat surprising to me that on reasoning tasks, thinking does not help (on classification tasks, it sort of makes sense that reasoning would be a distraction considering it is a recall based task).   \n6. Related to 5: \tLine 274-276 present two interesting hypothesis to explain why 7B model does not benefit from thinking on puzzle tasks. It would be interesting to try and disentangle which hypothesis is true. One way could be to give the LLM tools to perform visual reasoning in-context (e.g., zoom / crop etc.) and see if that helps model’s performance improve. “This may be because these tasks rely on visual rather than linguistic reasoning, and language-based reasoning might cause hallucinations, or the 7B model’s puzzle-solving capability may still be limited.”\n\n### Improvements\n\n1. The work provides prompts for all the settings which is good. It would be good to collect them within a single appendix section (instead of being spread throughout the appendix).   \n2. Create a distinct subsection in appendix where details of all datasets (and ideally one/two samples from the dataset) are provided. Some of the datasets were novel to me.  \n3. The appendix has many interesting results; would be helpful if authors could include some sort of table of content and preamble to the appendix to help readers navigate it better.  \n4. Doing some sort of analysis of what model parts change through the RL training would be insightful; e.g., are the changes concentrated in the vision encoder part or the LLM part?  \n5. Most of the results in the paper are presented in the form of table; it might be better to present them in terms of spider chart / radar plot where feasible. This presentation would be more amenable to a quick skim.", "was_revised": true, "final_justification": "Authors provided a comprehensive rebuttal with some results investigating why no-thinking RFT outperforms thinking-RFT. Specifically, 'Analyzing which model parts change during RL.' part of rebuttal I believe presents interesting findings.", "rebuttal": "Thank you for your  thoughtful feedback. We truly appreciate your suggestions for improving paper. Since we cannot revise paper during rebuttal, we will incorporate your comments in the next version. Below are our responses (R) to the weaknesses (W), improvement (I) , and questions (Q).\n\n**W1: The limitation section should be better addressed in the main text.**\n\n**R:** Due to page limitations, we placed it in the appendix. We will include this in the main text and provide a more thorough limitation discussion by incorporating the reviewers' comments in the next version.\n\n---\n\n**W2: Dataset memorization risks; test on novel synthetic data.**\n\n**R:** We agree that contamination is a legitimate concern. Following your suggestion, we used GPT-4 to create two *synthetic class groups* with novel class names absent from existing datasets, each accompanied by a brief GPT-4-generated description:\n\n- **Fantasy objects** e.g. *Frostleaf*:  Serrated emerald leaf edges coated in prismatic rime-ice crystals\n- **Mathematical shapes** e.g. *Ellipse*: horizontal ellipse outline centered on a plain background\n\nThen using FLUX‑1‑dev we synthesized 4 training and 20 test images per class (30 classes total) and fine‑tuned Qwen2VL‑2B. The results are shown below. No‑Thinking‑RFT remains decisively ahead on both synthetic datasets, indicating that its advantage is **not** due to memorization.\n\n| Model           | Fantasy   | Math      |\n| --------------- | --------- | --------- |\n| Qwen2VL-2B      | 15.5      | 30.99     |\n| Thinking-RFT    | 30.99     | 43.33     |\n| No-Thinking-RFT | **55.33** | **49.33** |\n\n---\n\n**W3: Only 2B results are shown in main text; 7B results in appendix should be included.**\n\n**R:** Due to page limit, we included 7B model classification results in the appendix. We provide a more detailed comparison of 7B model below and will include this table into the main text.\n\n| Method             | DTD       | EuroSAT   | Flowers   | S.C.      | Average   |\n| ------------------ | --------- | --------- | --------- | --------- | --------- |\n| Thinking-RFT       | 77.90     | 53.17     | 93.91     | 84.19     | 77.29     |\n| Think-After-Answer | 76.29     | **62.95** | 94.84     | 89.32     | 80.85     |\n| Adaptive-Thinking  | 79.60     | 56.62     | **96.86** | 89.92     | 80.75     |\n| No-Thinking-RFT    | **80.56** | 58.91     | 94.24     | **94.02** | **81.93** |\n\n---\n\n**W4: Study how the number of classes affects performance**\n\n**R:** The number of class names in the choice list reflects task difficulty, as models perform better with fewer options. We tested varying sizes on StanfordCars (20, 40, 60, 78 [# in paper], 100) and UCF-101 (20, 40 [# in paper], 60, 80, 100). The results are shown below. Fewer class choices boost accuracy for both models and shrink the Thinking- vs. No-Thinking gap.\n\n| Dataset | Model           | 20    | 40    | 60    | 78 / 80 | 100   |\n| ------- | --------------- | ----- | ----- | ----- | ------- | ----- |\n| DTD     | Thinking-RFT    | 85.60 | 69.92 | 65.23 | 63.42   | 57.92 |\n|         | No-Thinking-RFT | 86.53 | 73.52 | 70.79 | 70.15   | 65.33 |\n| S.C.    | Thinking-RFT    | 96.52 | 93.78 | 86.24 | 80.24   | 80.59 |\n|         | No-Thinking-RFT | 97.16 | 94.70 | 93.91 | 92.50   | 90.88 |\n\n---\n\n**W5: concerns about RL setup, without SFT warmup or curriculum.**\n\n**R:** We agree that the training setting (Zero-RL) might not be optimal. However, our aim is to study thinking in RFT, not to introduce a new SOTA algorithm. For this purpose, we believe Zero-RL is the most appropriate choice, for the following reasons:\n\n- **Focus on the fundamental contrast.** Zero-RL provides the cleanest environment to study the core question—*The role of thinking in RFT across different model sizes (2/7 B)*—without additional factors that could mask or amplify either side.\n- **Precedent and accessibility.** Many prior works adopt zero-RL (e.g., MMEureka, VisualThinker-R1-Zero, and VLM-R1); repeating that setup ensures our findings are directly comparable and reproducible across domains.\n- **Variability of CoT-SFT data.** CoT SFT hinges on data quality; uneven or low-quality corpora can even decrease results, while high-quality data are not universally obtainable. Relying on them would limit the scope of the study.\n- **SFT is a capability booster.** High-quality SFT boosts capability much like scaling model. To study thinking across different ability levels, it is sufficient to vary model size.\n- **Curriculum learning (CL) reduces generality.** Findings tied to a specific CL may not generalize, and different curricula can produce diverging results. Zero-RL removes this variance, making our conclusions broadly applicable.\n\nAccordingly, we adopt Zero-RL as our training paradigm, consistent with other RFT studies [1].\n\nFor completeness, we ran two small ablations on MathVista: a SFT warm-up and a CL schedule.\n\n**1. SFT warm-up**\n We generated 20k CoT data with GPT-4o to fine-tune 2B model, then trained both Thinking and No-Thinking-RFT from this SFT checkpoint. Results are below. With SFT warm-up, both methods improve, but No-Thinking-RFT still leads.\n\n| Model           | w/ SFT | w/o SFT |\n| --------------- | ------ | ------- |\n| Thinking-RFT    | 45.52  | 44.90   |\n| No-Thinking-RFT | 49.21  | 48.80   |\n\n**2. Curriculum learning (CL)**\nWe generated 10 outputs per question using Qwen2VL-2B and assigned difficulty based on average accuracy: easy (0–0.33), medium (0.33–0.66), hard (0.66–1.0). CL stages: Stage 1 = easy, Stage 2 = medium, Stage 3 = hard\n\nThis CL strategy even yields a slightly lower result for Thinking-RFT: 44.46 (w/ CL) vs. 44.90 (w/o CL).\n\n---\n\n**W6: Use visual-context fine-tuning (FT) to determine which explanation accounts for limited puzzle gains.**\n\n**R:** We appreciate this suggestion. However, visual-context FT experiments are outside the scope of our paper on RFT thinking. Given limited rebuttal time, we cannot implement it effectively.\n\nAs a pragmatic proxy, we evaluated the o4-mini model, whose API supports optional in-context visual reasoning (e.g., cropping). On PuzzleVQA, performance rises from 84.30% (no visual reasoning) to 86.48% (with visual reasoning), a gain of +2.18% accuracy. This suggests that visual-context learning can improve puzzle performance.\n\nWe will add this result to the paper, acknowledge that tool-using FT for puzzleVQA as an limitation and leave it to future work.\n\n---\n\n**I1: Group all prompts in a single appendix section.**\n\n **R:** We will collect the prompts in Appendix E.4 and G into a single section in the next version.\n\n---\n\n**I2: Add a appendix subsection with dataset details**\n\n**R:** We will consolidate the dataset introduction sections in the appendix into a single unified section, and will include several representative samples from each dataset.\n\n---\n\n**I3: A table of contents and preamble in the appendix.**\n\n **R:** We will add an organisation guide section and a table of content and preamble to the appendix in the next version.\n\n---\n\n**I4: Analyzing which model parts change during RL.**\n\n**R:** This is an insightful point. We compare the changes by computing the L₂ norm of parameter difference. Our analysis focuses on three aspects:\n\n1. **Modality**: changes between visual and language components\n2. **Module**: changes in different modules\n3. **Layer**: changes across different layers\n\nWe analyzeQwen2VL-2B on SAT, DTD, and Math datasets using Thinking and No-Thinking-RFT. Since we cannot upload images, Modality- and module-level results are given in the table below. We directly discuss layer-level findings as the table is too large.\n\n(i) **Modality-Level:** Language weights drift more than visual weights for all dataset—and the No-Thinking strategy raises visual drift slightly. This implies that the reward gradient mainly applies to the language modality.\n\n(ii) **Module-Level:** MLP is the dominant changed component for both visual and language parts, and attention blocks change less but still contribute near 15–20%.\n\n(iii) **Layer-Level:** With Thinking, weight drift grows toward deeper layers as reward back-propagates through the whole reasoning chain to high-level semantics. With No-Thinking, drift peaks in early-mid layers and then declines, indicating that low-level features are reshaped so a shallow forward path already produces the reward token.\n\n| Dataset | Method      | Visual | Language | MLP_V | norm_V | attn_V | MLP_L | norm_L | attn_L |\n| ------- | ----------- | ------ | -------- | ----- | ------ | ------ | ----- | ------ | ------ |\n| DTD     | Thinking    | 0.267  | 0.434    | 0.206 | 0.001  | 0.158  | 0.372 | 0.000  | 0.153  |\n|         | No-Thinking | 0.237  | 0.402    | 0.184 | 0.001  | 0.143  | 0.379 | 0.000  | 0.154  |\n| SAT     | Thinking    | 0.503  | 0.712    | 0.398 | 0.003  | 0.311  | 0.614 | 0.000  | 0.256  |\n|         | No-Thinking | 0.518  | 0.718    | 0.398 | 0.003  | 0.327  | 0.704 | 0.000  | 0.281  |\n| Math    | Thinking    | 0.415  | 0.735    | 0.364 | 0.002  | 0.210  | 0.691 | 0.000  | 0.270  |\n|         | No-Thinking | 0.731  | 0.931    | 0.581 | 0.004  | 0.455  | 0.880 | 0.000  | 0.388  |\n\n---\n\n**I5: Consider using radar plots instead of tables where feasible.**\n\n**R:** We will use spider chart and radar plot to present results such as Table 3, 4, and 5 in the next version to make the paper more amenable.\n\n---\n\n**Q1: Is the model just learning to format rather than gaining new capabilities?**\n\n**R:** Whether RFT teaches new skills is an open problem. Our models outperform zero-shot baselines, suggesting real learning. [1] finds RFT boosts pass@1 while leaving high-k pass@k unchanged, hinting it may just rank the correct answer first. \n\n---\n\n**Q2: SFT warmup or any curriculum for small-model training?**\n\n**R:** We ran a brief SFT warm-up and a CL schedule ablation; see W5 response.\n\n---\n**References**\n\n [1] Does reinforcement learning really incentivize reasoning capacity in llms beyond the base model? arXiv 2025."}, {"review_id": "7EVzfkpdbl", "reviewer": "Reviewer_jHwD", "summary": "The paper proposes No-Thinking-RFT, replacing the reward of RFT with an equality accuracy award. The No-Thinking-RFT is compared against RFT first in an image classification task, followed by a suite of visual reasoning tasks. Further analyses, including Think-After-Answer and Adaptive-Thinking are included to strengthen the paper's comprehensive analysis on thinking in RFT. Overall findings suggest that some tasks benefit from RFT, but often times No-Thinking-RFT is better --- creating a new paradigm to consider in training large multimodal models.", "questions": "In addition to weaknesses, I have a few questions:\n- Were the datasets reported in the paper the only datasets tested? If so, is there a pre-registration (i.e., on OSF) to indicate this? This would greatly increase the findings. \n- It would be good to discuss your results a bit more via a discussion section. Currently, it is a bit hard to contextualize the results of the work into the existing research landscape. It is also necessary to include a discussion of limitations and future directions to help support future work.", "limitations": "No - based on my reading, I did not find any limitations discussed in the paper. While there is limited negative societal impact for such technical work, one idea could be that if we remove thinking, the model traces could be less interpretable, or model behavior could potentially be more brittle given the more fixed equality training metric.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 4, "originality": 3, "strengths_and_weaknesses": "Strengths: \n- Timely paper on up to date methodologies. \n- Analyses seem thorough. I especially appreciated the think-after-answer and adaptive-thinking experiments, which add validity to the authors' claims. \n- Explanation for why the convergence is faster and why performance increases both make a lot of sense; creating a cohesive and believable set of takeaways.\n\nWeaknesses:\n- Framing could be stronger in the introduction and (especially) the abstract. I think the writing lacks motivation for why the problem is important. \n- Some missing citations: On overthinking harming performance [1], on adaptive selection of thinking/no thinking [2].\n\n[1] Liu, R., Geng, J., Wu, A. J., Sucholutsky, I., Lombrozo, T., & Griffiths, T. L. (2024). Mind your step (by step): Chain-of-thought can reduce performance on tasks where thinking makes humans worse. ICML 2025.\n[2] De Sabbata, C. N., Sumers, T. R., & Griffiths, T. L. (2024). Rational metareasoning for large language models. arXiv preprint arXiv:2410.05563.", "was_revised": true, "final_justification": "During the rebuttal period, the authors have improved the paper writing significantly, contextualizing their findings to a broader audience. While there are still some limitations to the breadth of tasks evaluated, limiting the score, these cannot reasonably be completed within the limited time frame of the rebuttal. I consider all my points addressed and support acceptance.", "rebuttal": "We sincerely appreciate your time and effort in reviewing our paper. We are encouraged that you found our analysis both thorough and cohesive. Below are our responses (R) to the weaknesses (W) and questions (Q) mentioned in your review.\n\n------\n\n**W1: Framing could be stronger in the introduction and (especially) the abstract.**\n  \n**R:** We appreciate this observation and will strengthen both the abstract and Introduction section by foregrounding the practical gap that motivates our study, drawing on claims below:\n\n- **Why the question matters.** Rule-based reinforcement fine-tuning (RFT) has recently surpassed supervised fine-tuning (SFT) on a range of tasks (l. 32–34), yet it is *computationally expensive* because it generates multiple long responses (l. 41–43). If explicit thinking is *not* always necessary (l. 36–40), practitioners could reduce latency, GPU hours, and energy costs without sacrificing accuracy.\n- **Untested assumption.** Explicit chain-of-thought is widely assumed to be the key to RFT’s success (l. 34–36), but prior work showing “overthinking” harms performance focuses only on inference-time reasoning, not on the **training** process (l. 38–40). Whether thinking is beneficial *during RFT* therefore remains unclear.\n- **Our contribution.** We introduce *No-Thinking-RFT*, a version of RFT that removes the thinking step by using a strict equality reward and direct-answer prompts (l. 60–64). Across tasks and model sizes, this variant outperforms Thinking-RFT while converging faster and requiring shorter fine-tuning and inference (l. 64–65), thus addressing both accuracy and efficiency.\n\n------\n\n**W2: Some missing citations: On overthinking harming performance [1], on adaptive selection of thinking/no thinking [2].**\n \n **R:** Thank you for these valuable references. We will add a subsection in the related work to discuss prior studies on overthinking, and we will be sure to cite these two papers in the next version.\n\n------\n\n**Q1: Were the datasets reported in the paper the only datasets tested? If so, is there a pre-registration (i.e., on OSF) to indicate this? This would greatly increase the findings.**\n \n **R:** Yes, the datasets reported in the paper are the only ones we tested. Although we did not carry out a formal *pre-registration*, every dataset choice strictly follows established practice in prior works rather than post-hoc selection. For example:\n\n- **Few-shot classification:** we follow *CoOp* [1] and use the standard suite of eleven datasets for training and evaluation; this protocol is adopted by nearly all recent papers in few-shot image classification.\n- **Spatial understanding:** we follow *VisualThinker-R1-Zero* [2], training on the SAT dataset and testing on CVBench.\n- **Mathematical VQA:** we follow *Math-LLaVA* [3], training on Math-40K and evaluating on the MathVista and MathVision datasets.\n\nNo additional datasets were tried or discarded and we purely followed the experimental setting in previous works to conduct experiments for our study. To increase transparency, we will create an OSF record that time-stamps the complete experimental protocol.\n\n------\n\n**Q2: It would be good to discuss your results a bit more via a discussion section. Currently, it is a bit hard to contextualize the results of the work into the existing research landscape.**\n \n **R:**  Thank you for the suggestion. We will reorganize the manuscript so that, after introducing the research questions that motivate the study, the experimental sections will present the results, followed by a dedicated **Discussion** section that interprets how the findings answer each question. In particular, we plan to discuss the following topics/questions (here we provide a short discussion section; we will provide a more lengthy discussion in the next version):\n\n1. **Are longer chain-of-thought reasoning processes always beneficial?**\n   Quite a few recent works on multimodal RFT have tried to reproduce the “length-increasing” effect (l. 34-36) originally reported in Deepseek-R1, treating lengthy chain-of-thought traces as essential for multimodal problem solving. However, our results show this is not always true. As *finding 1* indicates (l. 258), the model’s reasoning is often trivial—adding little to the final answer—especially on complex tasks when capacity is limited (2 B). Thus, the token budget devoted to CoT may be better spent on answer tokens or larger batch sizes when model capacity is tight.\n\n2. **Does model size mediate the gains of thinking- vs. no-thinking-RFT?**\n   Larger models are less prone to incoherent reasoning, improving thinking-RFT results. Yet, as Tables 3 & 4 and *finding 2* show, no-thinking-RFT still outperforms thinking-RFT on vision-centric and puzzle-related tasks, while generally lagging on math tasks—regardless of model size. Hence, although capacity influences each method’s absolute performance, the overall trend remains: task type largely determines which variant excels.\n\n3. **When should we choose thinking-RFT over no-thinking-RFT (and vice-versa)?**\n   Choice depends on task, budget, and requirements. Vision-centric and puzzle tasks benefit more from no-thinking-RFT, whereas reasoning-heavy tasks like math gain from thinking-RFT. Because it skips explicit reasoning, no-thinking-RFT runs faster and is more compute-friendly. Conversely, when explicit reasoning aids performance—e.g., in medical reasoning—thinking-RFT is preferable.\n------\n\n**Q2 & Limitation: It is also necessary to include a discussion of limitations and future directions to help support future work.**\n\n **R:** Thank you for the suggestion. Due to page limit, we placed the limitation section in Appendix A.1. We will include the limitation section in the main body of the paper and provide a more thorough discussion by incorporating the reviewers’ comments in the next version.\n\n------\n\n**References**\n\n[1] Learning to Prompt for Vision-Language Models, IJCV 2022\n\n[2] Math-LLaVA: Bootstrapping mathematical reasoning for multimodal large language models, EMNLP 2024\n\n[3] R1-Zero’s “Aha Moment” in Visual Reasoning on a 2B Non-SFT Model, arXiv 2025"}, {"review_id": "6CBI2wskZ6", "reviewer": "Reviewer_wtZh", "summary": "This paper first extends Thinking-RFT to visual tasks and then investigates whether explicit thinking is always necessary and beneficial in RFT. This is motivated by Deepseek R1 shows that explicit thinking can help model improve the performance. The authors propose No-Thinking-RFT on 2B and 7B models, demonstrating that explicit thinking does not consistently improve performance during RFT in visual perception tasks.", "questions": "1. The authors mention Deepseek-R1 as the motivation; however, I think Deepseek-R1 does not emphasize the benefits of explicit thinking specifically in small-scale models. Since the experiments presented in this paper focus exclusively on models up to 7B, the research question \"Is explicit thinking always necessary and beneficial for RFT?\" feels overly broad. The research question should be refined explicitly to address smaller-scale models (e.g., below 7B) or include additional experiments with larger models to strengthen the generality of the conclusion.\n2. In some tasks, such as EuroSAT, the No-Thinking-RFT method underperforms compared to SFT. Do authors consider these cases as the outliers? Hope to see more insights and explanations regarding these discrepancies.\n3. Do you think the Thinking-RFT has poor performance is because of the low-quality explicit thinking? Also, an open question is whether more fine-grained or nuanced reward designs for the thinking process could potentially improve Thinking-RFT’s performance. An ablation study or additional analysis would great to have.\n\nNote: I'm happy to increase my score if the authors can address my concerns in weakness and questions sections.", "limitations": "The experiments lack error bars, confidence intervals, or multiple runs with different random seeds to ensure robustness and statistical significance of the results.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths:\n- The paper extends the question of explicit thinking to multimodal LMs, providing valuable insight into scenarios where explicit thinking during RFT may not always be beneficial, particularly highlighting that for small models explicit thinking can sometimes negatively impact performance.\n- The authors also introduce an Adaptive-Thinking approach, enabling the model to learn autonomously when explicit thinking should be applied, which is an interesting contribution.\n\nWeaknesses:\n- The authors should include a section in related work about the literature on the negative impacts of explicit thinking (e.g. cot & overthinking) This would provide necessary context and better frame the motivation for their proposed No-Thinking-RFT method. (e.g. https://arxiv.org/abs/2503.16419 (survey paper as a good reference), https://arxiv.org/abs/2410.21333 (task2)).\n- The summary of contributions in the introduction is lengthy. It would be more effective and reader-friendly if presented in a concise and clear short sentences, highlighting the key contributions explicitly.\n- The implementation details section needs more clarity and completeness. Important hyperparameters such as temperature, random seeds, and other settings are important for reproducing the results should be explicitly mentioned.\n- Figures 2 and 3 have readability issues—the font size is too small, and the colors (particularly blue, representing decreases in Figure 2) are not clearly distinguishable. Authors should improve these visualizations to better show their results and enhance readability.\n- Section 4 highlights several critical findings and contains substantial insights. This section might be better placed earlier in the paper (e.g., before the case study on classification tasks). Reorganizing the structure to prioritize main results and findings would enhance readability and clarity in presenting the core contributions.", "was_revised": true, "final_justification": "I changed my score to 4 because the authors addressed most of my concerns and provided sufficient related work.", "rebuttal": "We sincerely appreciate your time and effort in reviewing our paper. We are encouraged that you found our insight into the explicit thinking process of RFT valuable. Below are our responses (R) to the weaknesses (W) and questions (Q) mentioned in your review.\n\n------\n\n**W1: Related–work omits negative impacts of explicit thinking.**\n\n**R:** Thank you for the suggestion. We will add a subsection in *Related Work* that surveys the emerging literature on the drawbacks of chain-of-thought and thinking. We will be sure to cite and discuss *Stop Overthinking* [1] and *Mind Your Step* [2] in the related work of the next paper version. This will better frame the motivation for No-Thinking-RFT.\n\n------\n\n**W2: Contribution summary is too long.**\n\n**R:** Thank you for the suggestion. We have written three more concise sentences to summarize our contributions and will include them in the next version of the paper.\n\n1. We extend *Thinking-RFT* to few-shot MLLM classification and reveal a cross-dataset transfer across datasets.\n2. We show several important findings about *No-Thinking-RFT* and Thinking-RFT via experiments on six tasks with 2B–7B models.\n3. Through *Think-After-Answer* and *Adaptive-Thinking* we confirm that deferring or omitting CoT speeds convergence without harming accuracy and explore adaptive thinking strategy.\n\n------\n\n**W3: Implementation details incomplete.**\n\n**R:** Due to space limitations, we briefly outline the implementation details in the main text and provide the full version in Appendix E. In the final version, we will move these details into the main text, including all parameter settings such as temperature (1.0 for all experiments) and random seed (100).\n\n------\n\n**W4: Figures 2 and 3 are hard to read.**\n\n**R:** We apologize for the difficulty in readability. To improve clarity, we will enlarge the fonts, switch to a color-friendly “RdBu” palette, and use dotted outlines to highlight negative deltas for better contrast in the next version.\n\n------\n\n**W5: Section 4 (findings) should appear earlier.**\n\n**R:** Thank you for the insightful suggestion. We will reorganize the paper by moving Section 4 earlier, beginning with the research questions, then presenting the main findings of Section 4, and finally proceeding to the task-specific case studies. This restructuring will allow readers to grasp the key results before delving into the detailed analyses.\n\n------\n\n**Q1: Deepseek-R1 doesn’t prove thinking helps small models; yet your study stops at ≤ 7B, so “Is explicit thinking always necessary?” seems too broad.**\n\n**R:** **(i)** We would like to respectfully clarify that our motivation is *not* that Deepseek-R1 demonstrates benefits for small models, but that many recent multi-modal RFT works with 2B–7B models implicitly treat lengthy chain-of-thought as essential and try to reproduce the “length-increasing” effect (l. 34–36). Given the considerable time and cost of Thinking-RFT, we evaluate its necessity by exploring and comparing it to No-Thinking-RFT.\n\n**(ii)** Due to resource constraints, we cannot conduct experiments with larger model sizes such as 32B. We agree the original wording was overly broad and will revise the research question to sentence below in the next paper version.\n\n> *“Is explicit thinking always necessary and beneficial for **small-sized-model** RFT?”*\n\n------\n\n**Q2: Explanation of why No-Thinking-RFT underperforms SFT on EuroSAT.**\n\n**R:** We view EuroSAT as an outlier since No-Thinking-RFT exceeds SFT on most datasets and by a large margin on average. The likely reason is the pre-training distribution of Qwen2-VL. EuroSAT consists of remote-sensing images, a domain we guess may be under-represented in the model’s pre-training corpus. Consequently, the model struggles to obtain positive rewards during RFT and the reward signals become extremely sparse, which impedes both Thinking- and No-Thinking-RFT learning processes. In contrast, SFT optimizes directly against ground-truth labels, avoiding the sparse-reward issue.\n\n------\n\n**Q3: Thinking-RFT has poor performance because of the low-quality explicit thinking? More fine-grained reward designs for the thinking process could potentially improve Thinking-RFT’s performance.**\n\n**R: (i)** We attribute the weaker performance to the low quality of the explicit thinking. Evidence includes the much slower convergence of Thinking-RFT relative to No-Thinking-RFT (Figures 5 and 8), indicating that the reasoning steps impede learning, and the frequent inconsistencies between thoughts and final answers (Finding 3 in Sec. 4.3), which suggest confused or ill-formed reasoning.\n\n**(ii)** Finer-grained rewards could indeed improve performance. In Finding 3 we show that inconsistently tagged answers have lower accuracy than the overall average, implying that enforcing consistency may boost results. However, designing such rewards is non-trivial and challenging. A simple strategy is to use an auxiliary LLM to evaluate reasoning quality, but this approach is both time-consuming and susceptible to reward hacking, making it impractical. \n\nAs an initial experiment, we designed a reward for the StanfordCars dataset that requires answers in the \\<answer\\> tag to also appear in the \\<thinking\\> tag; this preliminary approach yielded an almost 3-percentage-point boost in classification accuracy. Nonetheless, creating general, robust, and fine-grained reward functions and evaluating effectiveness remains challenging, so we leave that for future work.\n\n------\n\n**Limitation about no error bars or multiple seeds.**\n**R:** We would like to clarify that, given the high cost of fine-tuning, we did not report error bars, which is also adopted in most previous MLLM and LLM training papers. We report the mean and standard deviation over three runs for both Thinking-RFT and No-Thinking-RFT using Qwen2VL-2B on CVBench. The results are as follows:\n\n- **Thinking-RFT:** 70.66 ± 0.30\n- **No-Thinking-RFT:** 76.93 ± 0.25\n\nThese findings are largely consistent with those reported in the original paper.\n\n------\n\n**References**\n\n[1] Stop Overthinking: A Survey on Efficient Reasoning for Large Language Models.\n\n[2] Mind Your Step (by Step): Chain-of-Thought can Reduce"}, {"review_id": "jqT6MNBkdN", "reviewer": "Reviewer_rKqb", "summary": "The author investigates whether the \"think\" process is essential during the Rejection-Fine-Tuning (RFT) stage. Models that undergo Reinforcement Learning from Verifier Feedback (RLVR) typically show significant improvements in reasoning performance, a gain often attributed to the establishment of a stable thinking process. However, the author finds that even when the model is not required to output a \"think\" process during training, it can achieve performance comparable to conventional RLVR. The topic explored in this paper is very interesting, and the experiments are comprehensive.", "questions": "Please refer to the weakness", "limitations": "N/A", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 4, "originality": 3, "strengths_and_weaknesses": "### Strength\n- I quite like the motivation of this paper.\n- The paper explores many aspects and challenges the community's belief that the \"think\" process is the primary source of gains from RFT. The experiments are solid. \n\n\n### Weaknesses\n\n1.  The use of a 4-shot training setting (i.e., 4 examples per class), especially when sufficient data is available, seems somewhat \"toy-like.\" This small-data regime is also not the typical scenario where Supervised Fine-Tuning (SFT) is claimed to be most effective.\n\n2.  Although this work largely argues that the \"think\" process is not essential, it can also be seen as fundamentally challenging the RFT process itself. The current characteristics of RFT can be broken down into the following points:\n    - The model samples both \"think\" and \"answer\".\n    - A verifiable reward that is not easily \"hacked.\"\n    - Learning from a set of positive and negative samples.\n    \n    If the \"think\" process is removed from RFT, the model only needs to sample the \"answer.\" For training samples with verifiable rewards (common in domains like math or code), constructing answers is straightforward. For example, given the problem \"1+1 = ?\", we can easily generate answers from the range [1-10], treating \"2\" as the positive sample and the rest as negative samples. By applying a list-wise DPO algorithm, one could achieve a similar effect to \"no-think RFT.\" Since offline DPO does not involve online sampling, its cost is much lower than that of \"no-think RFT.\" This could potentially lead to a scenario where **offline DPO ≥ no-think RFT ≥ RFT**. I believe this comparative analysis would be very valuable, as this transitive relationship might suggest a new RL paradigm.\n\n3.  There is another difference between \"no-think RFT\" and RFT that needs consideration. \"no-think RFT\" can essentially be considered a learning process with one positive example versus multiple negative examples. In contrast, RFT can generate diverse positive examples by combining various \"think\" processes with the correct answer. In \"no-think RFT,\" however, the positive example is unique since it only consists of the final answer. This analytical experiment is also missing from the paper.\n\n4.  Section 4.1 (L229-230) directly assumes that the \"explicit\" think process is harmful. However, based on the experimental data, this conclusion only seems to hold true for the small model (2B). In the experiments with the 7B model, although \"no-think\" scores higher than \"think,\" the \"adaptive think\" mode, which clearly converges to a thinking pattern, achieves the highest score. This results in a relationship of `think < no-think < adaptive-think`. Or, given the small differences, it could be interpreted as `think ≈ no-think ≈ adaptive-think`. In either case, for the 7B model, one cannot conclude that the explicit \"think\" process is harmful.\n\n5.  The significant differences between the 2B and 7B models—in their convergence trends for \"adaptive think\" and in the perceived harm of the explicit \"think\" process—are noteworthy. This raises the question of which of this paper's conclusions can be generalized to larger 32B or 72B models. If the trends for 2B and 7B were similar, we could be more optimistic that the paper's conclusions have good scaling properties.\n\n6.  The \"free lunch phenomenon\" sounds strange. Isn't this just about performance transfer? I don't see this as a particularly novel finding. What is the author trying to convey in this section?\n\n7.  Figure 6: A figure belonging in the main text is placed in the appendix?", "was_revised": true, "final_justification": "My major concerns have been well addressed, thus I would like to raise my score.", "rebuttal": "Thank you very much for the time and effort in reviewing our paper. We are encouraged that you found the motivation compelling and the experiments solid. We truly appreciate your thoughtful discussion of DPO and RFT.\n Below are our responses (R) to the weaknesses (W) mentioned in your review.\n\n------\n\n**W1: The use of a 4-shot training setting (i.e., 4 examples per class) seems somewhat \"toy-like.\"**\n\n**R: (i)** We would like to respectfully clarify that few-shot image classification is a important multi-modal task for exploring the fine-tuning of vision-language models under limited labeled data, and has been widely studied in prior work [1, 2, 3]. This setting is intentionally designed to emphasize data efficiency and fine-tuning under minimal supervision. Therefore, it should not be regarded as a toy-like experimental setup. \n\n**(ii)** We would like to respectfully clarify that we did not claim SFT is most effective in the few-shot setting. As stated in lines 151–159, [4] observed that existing MLLMs struggle with classification, but performance can be improved through SFT with large-scale labeled datasets. However, such large-scale SFT is costly and time-consuming, particularly for RFT, which may require months of GPU hours on millions of labeled images. This exceeds our computational resources. Therefore, we focus on data-efficient strategies and explore few-shot fine-tuning instead.\n\n------\n\n**W2: Discussion betwen offline DPO, No-Thinking-RFT, and Thinking-RFT.**\n\n**R:** This is an insightful point, and we sincerely thank the reviewer for raising it. \n\n**(i)** Although No-Thinking-RFT removes the “think” phase, a list-wise DPO algorithm **should not** be expected to reproduce similar effect due to the fundamental difference between DPO and GRPO. We discuss the detailed difference below:\n\n1. **Sampling strategy.** List-wise DPO selects one positive and several negatives *offline*. In contrast, No-Thinking-RFT samples responses *online* at every training step; the mix of positives and negatives is not fixed at $1:(N−1)$ but varies with task difficulty and model competence (from $0:N$ to $N:0$ ). Thus while removing thinking, No-Thinking-RFT continues to explore and learn from fresh errors, whereas DPO does not.\n2. **Gradient update rule.** No-Thinking-RFT computes a policy-gradient using verifiable rewards ($R=1$ for an exact match, $R=0$ otherwise) and a group-advantage formulation, i.e. GRPO. Listwise DPO, by contrast, minimises a cross-entropy loss over a pre-computed list, functioning more like contrastive SFT. This distinction yields different learning dynamics.\n\nBecause of these differences, listwise DPO **should not** be expected to match the empirical behaviour of No-Thinking-RFT.\n\n**(ii)** We also implemented preliminary DPO and listwise DPO baselines to quantify the performance difference. We evaluated two negative-sampling strategies: one using samples generated by Qwen2-VL (DPO model) and the other using randomly generated samples (DPO random) as suggested by the reviewer.  Experiments were carried out on the FGVCAircraft classification dataset and the MathVQA dataset. The results shown in the table below indicate that both DPO variants under both sampling strategies perform substantially worse than Thinking-RFT and No-Thinking-RFT. This finding highlights the importance of online sampling and policy-gradient optimization in RFT. Moreover, model-generated negative samples consistently outperform random negatives in both DPO and listwise DPO settings, demonstrating that the quality of negative samples is critical to DPO training.\n\nWe will add this discussion in the next version of paper.\n\n| Model               | FGVC  | MathVista | MathVision |\n| ------------------- | ----- | --------- | ---------- |\n| DPO random          | 46.23 | 40.15     | 12.3       |\n| DPO model           | 51.25 | 40.32     | 14.4       |\n| Listwise DPO random | 47.19 | 40.78     | 12.45      |\n| Listwise DPO model  | 51.36 | 42.66     | 15.1       |\n| Thinking-RFT        | 74.41 | 44.90     | 16.45      |\n| No-Thinking-RFT     | 74.41 | 48.80     | 13.49      |\n\n------\n\n**W3: Positive-sample diversity differs between Thinking-RFT and No-Thinking-RFT.**\n\n **R:** This is a valuable observation and we thank the reviewer for pointing it out. During fine-tuning, Thinking-RFT can indeed produce multiple reward-positive trajectories (and many more negatives) for the same verifiable answer, whereas No-Thinking-RFT yields only one positive sample—the exact answer. Such a disparity could lead the Thinking-RFT model to generate more diverse outputs at inference. To test this as an extra analytical experiment , we ran each fine-tuned model five times on MathVista with Qwen2-VL-2B and recorded mean accuracy, its standard deviation, and pass@5 accuracy in Table below. We can find that Thinking-RFT exhibits both a higher gain of pass@5 and a larger 5 run standard deviation than No-Thinking-RFT, confirming greater output diversity.\n\n| Model           | pass@1 | Mean $\\pm$ std (5 runs) | pass@5 |\n| --------------- | ------ | ----------------------- | ------ |\n| Thinking-RFT    | 44.90  | 44.84$\\pm$ 0.45         | 46.70  |\n| No-Thinking-RFT | 48.80  | 48.74$\\pm$ 0.15         | 48.90  |\n\n------\n\n**W4: Your claim that explicit thinking is harmful relies mainly on the 2B results; for 7B the ranking is think < no-think < adaptive-think (or roughly equal across settings), so the data do not justify calling explicit thinking harmful for larger models.**\n\n **R:** We apologize for the confusion. Lines 229-230 contain a wording mistake: we did not intend to claim that explicit thinking is harmful for *all* model sizes. We want to use Think-After-Answer to show that explicit thinking is detrimental for the 2B model (Finding 1 that 2B No-Thinking-RFT surpasses Thinking-RFT) and not universally necessary, though not harmful, for the 7 B model (Finding 2 that 2B No-Thinking-RFT, Thinking-RFT, and Adaptive-Thinking have very similar results). We will revise the sentence to:\n\n> *“For the 2B model, the explicit ‘think’ process is harmful; for the 7B model it is not universally necessary.”*\n\n------\n\n**W5: Uncertain scalability to $\\ge$14B models.**\n\n **R:** We acknowledge this limitation that the trend of Adaptive-Thinking differs for 2B and 7B models. Because of GPU constraints we cannot currently run higher-scale experiments such as Qwen2-VL-32B; our study therefore focuses on small (2B) and medium (7B) checkpoints, the sizes most practitioners can fine-tune. We already flag this as a limitation and explicitly leave larger-model validation as future work for teams with the necessary computational resources.\n\n------\n\n**W6: Free-lunch phenomenon sounds trivial.**\n\n **R:** We would like to respectfully clarify that observing performance *improvements* on *other distinct* datasets after fine-tuning on a single dataset is **not** trivial in few-shot image classification. Prior work [2] reports severe *catastrophic forgetting* problem in few-shot image classification learning: for example, CoCoOp fine-tuned on ImageNet reduces DTD accuracy from 76.4% to 45.7%. We observe a similar drop with Qwen2-VL SFT. By contrast, both Thinking-RFT and No-Thinking-RFT *avoid* forgetting and instead *increase* accuracy on the other datasets. This finding contrasts sharply with previous few-shot learning results and is therefore striking. We want to use this finding to convey that RFT not only avoids catastrophic forgetting but also teaches the model the underlying essence of classification rather than memorising a single dataset (Section 3.3). To avoid confusion, we will rename this finding the *'cross-dataset generalisation effect'*.\n\n------\n\n**W7: Figure 6 belongs in the main text but in Appendix.**\n\n **R:** Thank you for pointing that out. Due to page limitations, we placed Figure 6 in the Appendix. We will move it to the main text in the next version.\n\n------\n\n**References**\n\n[1] Learning to Prompt for Vision-Language Models, IJCV 2022.\n\n[2] Conditional Prompt Learning for Vision-Language Models, CVPR 2022.\n\n[3] GraphAdapter: Tuning Vision-Language Models With Dual Knowledge Graph, NeurIPS 2023.\n\n[4] Why are visually grounded language models bad at image classification? NeurIPS 2024."}]

Author rebuttal:

Meta-review: This paper analyzes the effects of thinking in RL fine-tuning of MLLMs. The paper shows that RL outperforms SFT, yet thinking can often hinder final performance. The paper is a timely close look at the effects of thinking in MLLMs, and for practitioners has high utility. There were some weaknesses wrt ablations and datasets as well as some writing concerns, but a vast majority of these appear to have been addressed very well.

I therefore recommend the paper is accepted for spotlight.
Decision: Accept (spotlight)