Reason-RFT: Reinforcement Fine-Tuning for Visual Reasoning of Vision Language Models
Authors: Huajie Tan
Yuheng Ji
Xiaoshuai Hao
Xiansheng Chen
Pengwei Wang
Zhongyuan Wang
Shanghang Zhang
Venue: NeurIPS 2025 poster ()
Keywords: Multimodal
Reinforcement Fine-Tuning
Visual Reasoning
Primary area: applications

Abstract: Visual reasoning abilities play a crucial role in understanding complex multimodal data, advancing both domain-specific applications and artificial general intelligence (AGI). Existing methods enhance Vision-Language Models (VLMs) through Chain-of-Thought (CoT) supervised fine-tuning using meticulously annotated data. However, this approach may lead to overfitting and cognitive rigidity, limiting the model’s generalization ability under domain shifts and reducing real-world applicability. To overcome these limitations, we propose Reason-RFT, a two-stage reinforcement fine-tuning framework for visual reasoning. First, Supervised Fine-Tuning (SFT) with curated CoT data activates the reasoning potential of VLMs. This is followed by reinforcement learning based on Group Relative Policy Optimization (GRPO), which generates multiple reasoning-response pairs to enhance adaptability to domain shifts. To evaluate Reason-RFT, we reconstructed a comprehensive dataset covering visual counting, structural perception, and spatial transformation, serving as a benchmark for systematic assessment across three key dimensions. Experimental results highlight three advantages: (1) performance enhancement, with Reason-RFT achieving state-of-the-art results and outperforming both open-source and proprietary models; (2) generalization superiority, maintaining robust performance under domain shifts across various tasks; and (3) data efficiency, excelling in few-shot learning scenarios and surpassing full-dataset SFT baselines. Reason-RFT introduces a novel training paradigm for visual reasoning and marks a significant step forward in multimodal research.

Reason-RFT: Reinforcement Fine- Tuning for Visual Reasoning of Vision Language Models
Huajic Tan Yuheng Ji Xiaoshuai Hao Xiansheng Chen
Pengwei Wang"', Zhongyuan Shanghang Zhang 12,55
1 State Key Laboratory of Multimedia Information Processing, School of Computer Science, Peking University Beijing Academy of Artificial Intelligence Institute of Automation, Chinese Academy of Sciences 4 School of Artificial Intelligence, University of Chinese Academy of Sciences
Abstract
Visual reasoning abilities play crucial role in understanding complex multimodal data, advancing both domain-specific applications and artificial general intelligence (AGI). Existing methods improve Vision-Language Models (VLMs) reasoning via Chain-of-Thought (Co'T) supervised fine-tuning, using meticulously annotated training data enhance visual reasoning capabilities. However, this training paradigm may lead to overfitting and cognitive rigidity, restricting the model's s generalization ability transfer visual reasoning skills under domain shift and limiting its real-world applicability. To address these limitations, we propose Reason-RFT, the first two-stage reinforcement fine-tuning framework for visual reasoning: (1) Supervised Fine- Fine-Tuning (SFT) with curated CoT data activates the reasoning potential VLMs, followed by (2) Group Relative Policy Optimization response pairs, significantly enhancing the capability to address ubiquitous domain shift visual reasoning tasks. To evaluate the visual reasoning capabilities of Reason-RFF we reconstructed a comprehensive dataset encompassing visual counting, structural perception, and spatial transformation, serving as benchmark for systematic assessment across three core dimensions. Experimental results demonstrate three key advantages: (1) Performance Enhancement: achieving state-of-the-ar results across multiple tasks, outperforming mainstream open souncs and proprietary mod- els; (2) Generalization Superiority: consistently maintaining robust performance in addressing domain shift in typical visual reasoning tasks, outperforming alternative paradigms; (3) Data Efficiency: excelling in few-shot learning scenarios while surpassing full-dataset SFT baselines. Reason-RFT introduces a rebust training paradigm in visual reasoning, and please refer project website: RReason-FFT
(GRpO)-based reinforcement learning that generates multiple
Introduction
Visual reasoning is pivotal for understanding complex multimodal data and advancing artificial general intelligence (AGI) [[2), making it central focus in intelligent systems research. Recent advancements in image recognition (3)]) interactive security [10 and scene understanding [13]
[4] have enabled transformative applications in healthcare [5) 16). robotics [1/24], I and autonomous driving (25 Consequently, enhancing visual reasoning capabilities has garnered significant attention from both industry and academia for its potential to drive transformative advancements.
Researchers have explored two primary categories of methods to enhance visual reasoning capabilities: (1) neural-symbolic methods (32436), which integrate symbolic reasoning with neural networks to improve interpretability and modularity, and (2) Supervised Fine-Tuning (SFT) based on vision- language models (VIMss) [3 38), which utilize end-to-end training to strengthen reasoning abilities.
Equal contribution.
Project leader.
Corresponding author: shanghang @ @ku.edu.cn
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Oniy SFT-bused Training
VLMS
Gop
Bots
Question: For the pair of simlla figures, Rid the area of the green
Rigarein dimagees Choirss:
[A] 12.5
(B) 20.0
Scr
Reason RFT AR
Grendy Reward Siraiilicttion
Transient Adaptation Gap
Reasoning Redundanry
Scn
(1) 01,9
from Madel
Potlekketer's start by underiandre
COT Reasoning Data
from
emtiiittals oue indercanding the odalsenage betenes the discuiate the
Stace the redarphs seedes the
their poaten alfec aqua
Question + Images
grivi
Egures...
divitr problems
swead joerth core fedex
VLMs
Now, left caleubre
the fggre scingshe antamed diseadisc
"he irea of the green figar ca
which 25.54. The arean e 52 car
Question + Images
exy" (5 mnj car
Hooeeer original andiantes that
sineghtinn wark 30 as USS and 12.5
caisutadion
VLMs
Mas to and ecale bome thee the ama choudd be 1500) sasse Yepuliyee 15t
"hhs the
125 ae', minec
COT Reasoning Process
amagpsdda la dis >Allaak wanwere Ueaoeeeer
GRPO
choke (V)) sythiaes
Activation
Figure I: Overview of Reason-RFT. Compared to traditional SFT-based methods, our proposed Reason-RFT framework demonstrates superior generalization in visual reasoning tasks, excelling in reasoning improvement, domain shift adaptability, and data efficiency.
However, both approaches face significant limitations. Yeural-symbolit methods are hindered by high complexity and strong reliance on program generation, while SFT is constrained by its dependence on high-quality Chain-of -ai--of-hoooe Thought (CoT) annotated data and meticulously designed data mixing strategies, leading to issues such as overfitting, cognitive rigidity, and limited adaptability to domain shift. These challenges reduce their effectiveness in real-world applications.
Recent advances such GPT--ol DeepSeek-RI 339 and Kimi-1.5 (40] show that reinforcement learning (RL) during post-training enhances reasoning in coding and mathematics. RL offers a dynamic alternative SFT by enabling exploration and feedback-driven optimization, which can improve performance with limited labeled data. However, pure RL methods often lack robustness to domain shifts such as changes in visual appearance or configuration, limiting their generalization capacity in real-world visual reasoning scenarios.
To address this, we propose Reason-RFT, the first two-stage reinforcement fine-tuning framework designed to enhance generalization in visual reasoning tasks. First, we employ SFT with CoT reasoning activate the model's potential reasoning capabilities, using high-quality domain-specific visual reasoning dataset tailored to stimulate related reasoning abilities. Subsequently, we further enhance reasoning potential through Group Relative Policy Optimization (GRPO), demonstrating that Reason-RFT improves robustness under distribution shifts by enhancing the model's 's reasoning capabilities. To evaluate its effectiveness, we constructed a high-quality dataset covering visual counting, structure perception, and spatial transformation, serving as a benchmark for evaluating three core capabilities of visual reasoning. Extensive experiments highlight three key advantages of Reason-RF* (1) Performance Improvement: It achieves strong results on visual reasoning tasks, including visual counting, structure perception, and spatial transformation, outperforming mainstream VI.Ms; (2) Enhanced Generalization: It consistently exceeds both SFT-only and RL- only baselines under domain shift conditions, as demonstrated through comprehensive evaluations; (3) Data Efficiency: It reaches over 90% of the SFT-only performance while using less than 5% of the data. These results underscore the effectiveness and efficiency of Reason- RFF, establishing it as a robust framework for advancing visual reasoning. Our main contributions are summarized as follows.
We introduce Reason-RFT, a two-stage reinforcement fine-tuning framework that significantly enhances the visual reasoning capabilities of VLMs by effectively combining the complementary strengths of SFT-based and Ra-based methods.
We provide a systematic analysis of SFT-based and RL-based paradigms visual reasoning tasks, identifying the limitations of SFT and the advantages of RI. in improving reasoning ability, handling domain shifts, and achieving data-cfficient learning.
2

---

We reconstruct a comprehensive dataset spanning three core domains: visual counting, structure perception, and spatial transformation, serving as benchmark for evaluating visual cognition. geometric understanding, and spatial generalization.
Extensive experiments validate the proposed framework, demonstrating its practicality and effectiveness, and providing a new perspective for reinforceement-driven multi-modal training.
2
Related Work
Visual Reasoning Visual reasoning is core challenge in advancing AGI, requiring models to
perform complex cognitive tasks grounded visual perception [ 41.50 underpins a wide range
of applications, including visual counting [1], geometric problem-solving [4 52
55 visual
transformation reasoning [56] 57,, scientific analysis [58
59 and robotic task planning 17
60
Traditional approaches rely program generation [36, 62 63 or neural-symbolic reasoning 32
35., while recent advances in VLMs leverage large language models (LLMs) to enhance reasoning capabilities, For instance, VLaCA- (37 employs multi-stage SFT with CoT prompting [64], and Insight-V 665 integrates SFT with RL. DeepSeek-RI Matsseek-RR-eerr R--eero [66] further introduces rule-guided RL framework that substantially improves reasoning performance. Building upon the DeepSeek-RI [66], our work provides a comparative analysis of SFT-based and RL-based paradigms, demonstrating the advantages of RI RI-style methods in enhancing visual reasoning.
Post-Traie post-Training Post-Trase Training a crucial phase for enhancing the performance of LLMs and VLMs, bridging pre-trained models and their real-world applications [7/770] It primarily involves two and RL 73 SFT adapts pre-trained models to specific tasks using task-oriented datasets, often formatted as instructions. Research like FLAN (78) highlights the importance of diverse instruction- tuning datasets for improving zero-shot performance, while iterative processes, such Llama 3. is six-round strategy [79], integrate rejection sampling, synthetic data, and human annotations. RL aligns models with human preferences or task specific goals through feedback mechanisms. Reinforcement Learning from Human Feedback (RLHF)
methodologies: SFT
[74) refines models using human preference data, as seen in Llama 3.1 79 and Nemotron-4 88 which use reward modeling techniques like DPO [81] and RPO
80 For example, TÜLU3 82 employs length-
normalized DPO, while DeepSeek- V3 [33 combines rule-based and model-based reward systems. Recently, DeepSeek-R1 (66) achieved significant text reasoning improvements through pure RL (84. Our work first adapts RI methodologies to VLMs, enhancing visual reasoning, and systematically compares SFT-based and RL-based paradigms in visual reasoning tasks.
3 Methodology
In this section, we introduce Reason-RFT, a novel two-stage training strategy to enhance the reasoning capabilities of VLMs in complex visual reasoning tasks. As shown in Fig. [ the framework comprises two stages: (1) SFT-based Visual Reasoning Activation, which uses SFT with high-quality CoT reasoning data to activate the model's domain-specific reasoning capabilities, and (2) RL-based Reasoning Enhancement, which employs the GRPO algorithm with rule-based rewards to further push the upper limits of the model's 's reasoning potential.
3.1 STAGE 1:
SFT-based Reasoning Activation
In the initial stage, we employ SFT on structured visual reasoning dataset containing step-by-step reasoning processes. This stage trains the model decompose complex tasks into logical steps. Each sample is represented as (æ, r, a), where x denotes the input images, is the question, is the reasoning steps, and a the final answer. The training objective maximizes the likelihood of generating both r and a given (r, q):
LSFT -E (a
Σ log (o g.g.
(1)
where DD denotes the dataset, y represents the concatenated sequence of r and a, and my denotes the model's s token distribution. The resulting model सeaт used initialize the subsequent stage, providing stable foundation for RL-based reasoning enhancement.
3

---

Activation
(prablem), and S2: RI-based Enhancement
Visual Data
Problem
XL
Reference Model
Reward Evaluation
SPT for Limited Steps
Policy Model
Format Reward
Pretrained VLM
Test Pronage
Sampled Candidate Responses
GRPO
Accuracy Reward
Initial Policy Model
Policy Gradient Update
Format Reward
Mathematical Accuracy Reward
Penctiin-hased Accuracy
(Itnal
ttere) </an
to formst
Accuracy
Prompt:
Prompt: spatial visual nesuning expert contre simage-
Prompt: You
counting
Giren
you should
eupertt.
cimages
fustingpoint
many
GT casser (as esample) sarwerr
GT: camwer 3.3 corppl) comovrrs
Homer o7 ciy 1
detveriine
on www.orolliiiiinnnooo lppaiteni
achieve the fimll state.
trndorattions that
Rs.. puiditgrowan Partial rosard
Figure 2: Framework of Reason-RFTs RFF.. Reason-RFT adopts two-stage training paradigm for visual reasoning. The first stage applies SFT with CoT reasoning to establish strong task-specific priors. In the second stage, GRPO is employed to further enhance reasoning capability and generalization.
3.2 STAGE 2: RL-based Reasoning Enhancement
In the second stage, we refine #cor using GRPO, leveraging RL for its efficiency and scalability. Unlike Proximal Policy Optimization (PPO), which requires computationally expensive value net- work, GRPO calculates relative advantages by comparing rewards within group of sampled actions, reducing computational overhead and simplifying optimization. This makes GRPO particularly suitable for visual reasoning tasks.
Sampling Action Groups For each input state (xr, q), GRPO samples group of actions {a,, a2,...,aa.!. from the current policy Tg. initialized from TCoT- The sampling process is:
a, (a x, x).), for ..,.......
This strategy ensures diverse responses, promoting exploration and preventing premature convergence.
Reward Evaluation. Each sampled action a; receives reward R(a,) based on verifiable criteria, yielding a reward set {r1, T1........ .... J. .G.. For visual reasoning tasks, the reward R(a;) is composed of format reward Riomma (a:), which enforces structured outputs, and an accuracy reward (a,)
which measures correctness. This formulation balances structural alignment and factual precision in reasoning. The reward function is defined as:
R(a;)) Rfoemar (a:) Race (a:).
Policy Update with Relative Advantage Rewards are normalized within the sampled group to compute relative advantages (A1, A2, ... Ac), AG)
defined
4; - mean (r1, r2, r2,...,rG. std(ri, P2. ... rG]
(4)
Based on these ad advantages, the policy is updated to reinforce actions with positive advantages and reduce the probability of less effective ones. To maintain training stability, the update is constrained by minimizing the KI. divergence between the updated and reference policies.
3.3 Reward Design for Visual Reasoning Tasks
For the diverse requirements of visual reasoning tasks, including visual counting, structure perception, and spatial transformation, our reward design integrates two essential components: format reward and accuracy reward. The format reward is uniformly applied across all tasks, ensuring that the

---

model strictly adheres to a structured response format for consistency. For the accuracy reward, we carefully tailor the design the specific characteristics of each task, as shown in Fig. 2 creating task-specific reward mechanisms to evaluate the correctness of the model's responses.
Format Reward This component ensures structured and interpretable responses by requiring the model to adhere to a predefined template: reasoning within <think> and </think> and the final answer within <answer> and </ /nswer>> A reward of is only given for strict adherence.
This component evaluates the correctness of the model's responses, ensuring alignment with ground truth across diverse visual reasoning tasks. To address task diversity, design tailored reward mechanisms for discrete-valued mathematical, and function- based problems. Each mechanism is crafted to handle the unique characteristics of its problem category, enabling precise and fair evaluation. Below, we introduce the three reward forms.
Accuracy Reward
Discrete-valued Type This reward type applies to visual counting and structure perception tasks, where answers are discrete values (e.g., multiple-choice or integer-based responses). The accuracy reward (ac) is defined as:
(ac) = -(.. 0, otherwise,
f1.
aprett dg
(5)
where apeed is the predicted answer and agt is the ground truth. This discrete reward penalizes deviations from ground truth, ensuring precision in tasks requiring unambiguous answers.
Mathematical Type This reward type is designed for structure perception tasks involving numerical answers, such floating-polt point values tolerance-basee based evaluation account for minor numerical deviations, which is defined as:
expressions. It uses
Racee (a.)
cos
apreed
agl
c1 X [ag)
+1
(6)
(E2 f1)
agl
where Opred is the predicted answer, agi is the ground truth, is the tolerance threshold for an exact match (e.g., 0.05), and 62 is the upper bound for partial rewards (e.g., 0.20). If Mppred agrl < é₁ x lagel. the reward is 1 (exact match); if (4pred dgl >> x |dge]. the reward (incorrect). This formulation ensures smooth transitions between full and partial rewards, enabling fair evaluation of numerical accuracy.
Function-based Type This reward type is designed for spatial transformation tasks requiring sequence of transformation functions. The accuracy reward Ruee(a;) evaluates the alignment between the predicted sequence Tpred and the ground truth Tg., computed as:
len(T S + a den(T//o/// + 3 Ien(T prot) prod
Race (o.)
peal
max(len (peaa)) (Tn))
where Tf is the subset of transformation steps with complete matches (w/ function, object,
T/to/"" are the subsets with partial and only-function matches (w/ function and pred is the subset with only-function matches. or and the weighting coefficients for partial matches. This formulation ensures nuanced evaluation for flexible adjustment of partial match contributions.
and value), pred
object, w/ function and value), Tif
4 Experiments
We design experiments to investigate the following key research questions:
RQ1: How effective is Reason-RFT in reasoning, generalization, and data efficiency?
RQ2: Why is the STAGE 1 of SFT with CoT reasoning necessary?
RQ3: Why is the STAGE 2 of reinforcement fine-tuning necessary?
RQ4: How does reward design affect Reason-RFT's performance?
RQ5: What training dynamics emerge during reinforcement fine-tuning, and how they shape the reasoning behavior of (eason-RFT"
5

---

4.1 Experimental Details
Datasets In this paper, we comprehensively evaluate the visual reasoning capabilities of our method by leveraging six existing datasets, enhanced through subtask categorization, error arror-prone data filtering, and dataset restructuring. Detailed protocols for data filtering and restructuring are provided in Sec. Specifically, we define three task categories follows.
Visual Counting a multimodal reasoning task evaluating the integration of linguistic, visual, and mathematical skills by solving arithmetic problems in 3D block-based scenes. Specifically, we filtered and corrected 35K samples from CLEVR- -Mth [T] for training and 1K test samples for in-domain (ID) evaluation. To assess generalization under domain-shiff (DS), constructed 1K new samples using 3D assets from Super-CLEVE CLEVR [5T, including two subsets: direct arithmetic (DS-D) and mixed arithmetic (DS-M). Refer to the Appendix Sec. A. for details.
Structure Perception a structural reasoning task requiring models to analyze relationships in geometries, imaging structures, chart layouts, and architectural designs. We filtered 4.5K training samples and 820 ID test samples from Geol 70K (52) and Math360K [55], along with 800 samples from Geometry3K [85] to evaluate DS adaptability. See the Appendix Sec. A.2
spatial-visual reasoning task requiring models to infer single- multi-step transformations by analyzing initial and final states of 3D scenes from multiple perspectives (e.g., center, left, right). We generated 100K samples using TRANCE [56). covering four difficulty levels, and selected 60K for training and 6K for testing through specific filtering process. For DS evaluation, identical scenes are rendered from left/right viewpoints (DS-L/R) to test perspective-change robustness. Details can be found in the Appendix Sec. A.3 A.3
Spatial Transformation
Evaluation Metrics We use accuracy-rate (Acc) as the primary metric [66] For numerical answers, correctness is verified by mathematical equivalence to the ground truth. For multiple-choice questions, we perform a string match. For function-type sequences, we use stepwise multi-level evaluation.
Implementation Details We utilize Qwen2-VL-2B and Qwen2-VL-7B 87 as the backbone models for our experiments. Our implementation is built on the open-source frameworks Open-RI [88] and
vLLM [89], ensuring reproducibility and scalability. All experiments were conducted on a cluster of servers, each equipped with 8 A800 GPUs. For further details, see the Appendix Sec. B
Training Paradigms and Baselines To assess the performance and generalization of different training strategies, we compare: (1) SFT-based methods-ANS-SSE which fine-tunes answer generation, and COT-SFT, which uses supervised learning with CoT reasoning; and (2) RL-baaed metthods-RResso-RFFI--2
which applies RI without reasoning activation stage, and Reason- RFT, which uses limited CoT data for reasoning activation before RL training. For comprehensive experiments, we use Owen2-VL-Instrrue [87] as the base model (both 2B and 7B variants). In addition, we also select the most advanced open-source models [90-94] and the proprictary models 995 96 as baselines to evaluate the performance of different paradigms.
4.2
Overall Evaluation of Reason-RFT Framework (RQ1)
To evaluate Reason-REF RFT, we evaluate Reason- RFT using 2B- and 1B-parameter models on three visual reasoning tasks. The results are summarized as follows.
Strong reasoning performance across ID tasks. As shown in Tab. [ Reason-RFT achieves performance comparable to better than both SFT- and RL-based methods across all tasks. In visual counting, setting. In structure perception, Reason- outperforms most open-source and proprietary baselines in the 7B setting and remains competitive with top models such as IntermVL-2.5-8B [92]. In spatial transformation, Reason-RFT matches exceeds SFT-based methods while consistently outperforming all baselines. These results demonstrate that Reason-REF -FTT effectively integrates the strengths of both SFT and RL in structured reasoning tasks.
achieves the best performance among all models in the 7B
Superior generalization under DS. Under DS settings, Reason-RIT shows substantial gains over both traditional baselines and alternative training paradigms. In visual counting, it outperforms ANS- SFT by 10.95% (2B) and 13.93% (7B). In structure perception, Reason- achieves the highest performance in the 2B model, with 6.93% gain over CoT-SFT, and remains highly competitive in

---

Table I: Results three visual reasoning tasks. The best results among different training paradigms are highlighted in bold, while the second-ber results are underlined. "ID" denotes in-domain test data, and "DS" denotes domain-shift test data.
Visual Counting
Structure Perception
Spatial Transformation
ID
DS-D
DS-M
AVG
ID
DS
AVG
ID
DS-L
DS-R
AVG
Pruprictury Models
GPT-40-2024-08-06
50.18
42.55
Semini--1.5PPa
[96
61.80
41.20
26.40
43.13
50.12
48.38
49.45
26.22
18.76
19.88
22.77
Open-Source Models
Owwa22.-VV--BB-tnss
X0
75.90
50.93
4.40
43.74
36.75
37.44
37.09
8.57
8.26
8.31
8.42
Phi-3.5- 1is..--Viiion--8-Innt
01
21.40
18:27
15.22
36.83
50.25
43.54
7.42
4.02
5.33
Dwwa2.55-VL-7B--00
50
74.60
46.00
2.80
41.13
44.00
45.61
44.80
19.63
13.12
13.42
16.45
IntemVL-22.5 [92
93.50
47.48
47.32
51
7.19
6.63
6.91
Ilamm-3.22 3.2. 1i-vionn
93
10.30
9.60
9.20
9.70
13.25
20.85
17.30
8.22
8.40
9.03
8.47
120
7.35
Qwea2VL.-2B-nnsse
Zero Shot
82.40
42.67
03.00)
41.69
25.86
20.63
23.225
3.78
4.60
4.67
4.35
ANS-SFT
96.20
51,07
5.20
50.82
51.34
22.50
36.92
77.39
49.24
50.33
58.99
+ CoT-SFT
85.50
49.73
36.80
57.34
43.05
25.25
34.1
64.37
43.19
42.86
50.14
Season-RFT-Zem
98.40
58.00
5.20
53.87
47.68
32.50
40.09
42.13
34.07
33.41
33.74
Reason-RFT
96.80
60.00
28.40
61.77
49.03
33.13
74.61
64.05
64.08
67.58
Owea2VL.-7B. Brea2VL-7BBInstruuc
Zeno Zeno-Shot
98.60
54.53
4.80
52.64
43.30
43.88
43.59
13.53
12.72
12.78
13.01
ANS-SFT
95.00
42.53
8.00
48.51
51.34
25.38
38.36
82.19
54.29
54.83
63.77
+ CoT-SFT
87.30
45.33
33.60
55.41
50.49
33.00
41.75
81.31
47.99
47.80
59.00
Reuson: RET-Zero
99.40
63.60
21.20
61.40
55.00
54.75
54.88
67.67
57.20
56.15
60..3
+ Reason-RFT
95.60
56.13
35.60
62.44
59.27
49.25
54.26
79.97
59.36
58.61
65.98
the 7B model. Most notably, in the spatial transformation task, the 2B Reason- RFT model surpasses GPT-4o (95] by 31.7%, showcasing remarkable generalization under DS.
High training efficiency. To evaluate data efficiency during training, we train all meth- ods on the spatial transformation task and monitor intermediate ID and DS performance (Fig. B. Additional results for visual count- ing and structure perception are provided Appendix Sec. C In the 2B model, Reason- RFT achieves 70% of the final performance Reason- RFT-Zero using only 3% of the train- ing data (1,600 samples), and reaches 82.5% with 9% of the data. In the 7B model, it achieves over 92% of Geason-RFT-Zeero' per- formance using just 3% of the data. These sults confirm that Reason-RFT exhibits strong training efficiency in both ID and DS scenar- ios. The data-efficient nature of Reason-RFT renders it particularly effective for few-shot learning, offering significant potential for ap- plications where labeled data is limited.
(a)
(4)
Figure 3: Results of different methods and model sizes on spatial transformation task across training.
4.3 Effect of STAGE on Initialization (RQ2)
To investigate the role of CoT-SFT in initialization, we compare four baselines across three tasks. The results in Tab. reveal the following: (I) Consistent performance gains from CoT-SFT. Across all three tasks and both 2B and 7B model scales, Reason-REt consistently outperforms Reason-RFP Zero following stage 1 reasoming activation. This improvement is particularly notable when the model small and the task involves complex output structures. For example, in the spatial transformation wask- requires function-like serialized outputs-thh the 2B Reason-RFT model surpasses Reason- RFT-Zero by 33.84%. (2) Smaller models benefit more from CoT-SFT priors. In the visual counting task under the DS-M setting, the 2B model with CoT-SFT outperforms Reason- Macooo-RFF---- RFT-Zero by 31.6%.
7

---

Although the gap narrows in the 7B model, CoT-SFT still yields substantial gains. This indicates that pure RL-based methods struggle to adapt from direct arithmetic mixed arithmetic reasoning under DS, whereas CoT-SFT provides effective inductive priors for such adaptation. Moreover, under the same amount of CoT-SFT data, the model still underperforms its counterpart, highlighting the increased reliance of smaller models on CoT-SFT for acquiring reasoning capabilities.
4.4 Effect of STAGE 2 Generalization (RQ3)
To evaluate the impact of RL in stage 2, we compare the gen- eralization performance of CoT-SFT and Reason-RFT across three visual reasoning tasks under DS. As shown in Tab. aand Fig. the results reveal the following: Reinforcement fine- tuning improves generalization beyond CoT-SFT. Across all domain-shift settings, Reason-RFT RFT consistently outperforms CoT-SFT, demonstrating that reinforcement learning signifi- cantly enhances model robustness. For instance, the visual counting task, the 7B Reason-RFT RFT achieves a combined DS-D and DS-M score 12.8% higher than CoT-SFT. The improve- ment is even more pronounced in structure-sensitiv tasks such as spatial transformation, where the 2B Reason-RIT exceeds CoT-SFT by 21.04% average across DS-L and DS-R. These results indicate that CoT-SFT alone yields limited generaliza- tion, while reinforcement fine-tuning enables better adaptation to compositional and layout-dependent variations.
Spatia
Commaaieee Shift
Sarie A.
Qpea799-00 Corr Shet)
Dwre791 20 NNCAFT
(M COTOFT
Figure 4: Results of DS u.s. ID on spatial transformation task.
4.5 Exploration on Reward Design (RQ4)
Table 2: Results of different format reward
strategies the spatial transformation task.
Format Reward. In DeepSeek-RI 339 the format reward enforces the of <think> and <answer> tokens to structure reasoning in textual tasks. To better support visual reasoning, we extend this with
Sctting
ID
DS-L
DS-R
AVG
Bree22L--BBInnttee
Reason Reason-RFT-2 ReTo
42.13
34.07
33.41
33.74
<summary> and <caption> tokens to incorporate visual observations via caption-style prompts. As shown in Tab. 2 this improves Geason-RFFT-Zero but has limited effect Reason-RFT. We attribute this to Reason-RRt -RFTs prior CoT supervision, which likely helps it internalize caption-like structures in stage 1, reducing the benefit of explicit tags. In contrast, Reason-RFT-ZZ benefits more from such structural cues, indicating greater sensitivity to format-level guidance.
+ visual
42.01
36.05
35.97
Reason-RFT
74.61
64.05
64.08
69.33
+ visual tokeus
71.99
60.13
59.87
65.99
Owen2VL-7BBInsteee
Reason-RFT-Zere
67.67
57.2
56.15
62.17
+ visual tokens
70.28
59.52
57.01
64.27
Reason RFT
79.97
59.36
58.61
69.48
+ visual tokens
79.85
58.71
57.98
69.09
Accuracy Reward. We explore accuracy reward de- sign in the spatial transformation task, which requires predicting transformation sequences in structured format. The formulation in Eq. introducess coeffi- cients and control tolerance for partial matches. We test three settings: (1) = 0, B = 0 (exact match only). (2) a 0.50, = 0.25 (partial credit), and (3) -= -0.25, 3 = -0.50 (penalized partial matches). Results on 2B and 7B models (Tab. show that: (I) partial credit improves ID performance but harms generalization, suggesting "soft rewards" reduce ro-
Table 3: Results of different accuracy reward strategies the spatial transformation task. Setting
9
i
DS-L. DS-R
AVG
Qwen2VL-2B-Instree
Baseline
0
0
74.61
64.05
64.08
69.33
(a)
0.50
0.25
79.18
56.36
55.45
67.54
(h)
-0.25
-0.50
73.69
64.41
64.72
69.13
Qwen2VL-7B-
Baseline
79.97
58.61
69.48
(a)
0.50
0.25
80.89
53.20
52.61
66.90
(b)
75.03
bustness; (2) penalizing partial matches improves generalization under domain shift, indicating "hard rewards" better support serialized reasoning.
4.6 Training Dynamics and Reasoning Behavior Analysis (RQS)
Greedy Reward Stratification. This phenomenon captures the model's tendency, particularly in Reason-RFT--Zen to prioritize casier-to-optimize rewards (e.g., format reward) before addressing more challenging objectives (e.g., accuracy reward). shown in Fig the reasoning token length initially drops, then gradually increases and stabilizes. This dynamic correlates with the format
8

---

Response Token Length
Accuracy Reward
14
Maxiisum Point
20
Rapld
1.30
300
432
700
60
102
(C)
Figure 5: Analysis of Greedy Reward Stratification. The model's reasoning token length first decreases, then gradually rises and stabilizes during Reason-RFT RFT-Zero training. The peak of the format reward coincides with the accuracy reward's rapid growth phase.
Roarting hepare unall
Figure 6: Illustration of the Transient Adap- tation Gap. (a) shows a sharp drop and recov- ery in both IID and DS test performances within the early training steps. (b) shows a case study of the prediction result on early step.
Figure 7: Analysis of Reasoning Redundancy. (a) shows the reasoning token length curves for Reason-RFT-Zeero and Reason-RFT during train- ing. (b) displays their accuracy reward, with both paradigms converging to similar accuracy rate.
reward quickly reaching a plateau, followed by a sharp rise in the accuracy reward. We infer that the model initially simplifies its outputs to adapt rapidly to structured format expectations, and only later allocates learning capacity improve reasoning correctness.
Transient Adaptation Gap. This refers to a temporary performance degradation observed in the early training phase of Reason-RFTT-Zero. When the model shifts from directly predicting answers to producing structured reasoning traces, it experiences a brief adaptation bottleneck-marke by sharp decline and gradual recovery accuracy. Fig. 6 (a) illustrates this drop within the first 100 steps on the visual counting task under both ID and DS settings. A case analysis in Fig, (b) further reveals that forcing structured reasoning prematurely may lead incorrect outputs, highlighting the sensitivity of early-stage training to reasoning format constraints.
Reasoning Redundancy. This phenomenon concerns the discrepancy in reasoning token length between models trained with and without CoT activation. In the structure perception task, Reason- RFT and Crason-RFFT-Zer attain similar accuracy, yet the former generates significantly longer reasoning traces (Fig. 7). This likely stems from Reason-RFT's use of CoT data distilled from advanced models (e.g., GPT-4o), which encourages verbose reasoning during stage 1. In the absence of penalties or length control in reinforcement fine-tuning, such verbosity persists. By contrast, ReF-Zero converges to more concise reasoning through reward-driven exploration. We hypothesize that these longer chains in Reason- may introduce unnecessary computational overhead or reflect overthinking relative to task complexity.
Conclusion
In this paper, we propose Reason-RFT. a novel reinforcement fine-tuning framework that enhances the generalization capabilities of visual reasoning models. By integrating SFT with CoT reasoning activation data and GRPO-based reinforcement learning, Reason-RFT effectively mitigates key challenges such as overfitting and cognitive rigidity, thereby improving cross-domain transferability and real-world applicability. To support systematic evaluation, we reconstruct a comprehensive dataset covering visual counting, structure perception, and spatial transformation tasks, establishing a robust benchmark for assessing model performance across diverse scenarios. Extensive experiments demonstrate the effectiveness Reason-RFT, providing valuable insights for advancing visual reasoning research and introducing new paradigm in multimodal learning.

---

Acknowledgments
This work was supported by the National Natural Science Foundation of China (62476011), and the National Science and Technology Major Project (No. 20222D0117800;
References
[1] Adam Dahlgren Lindström and Savitha Sam Abraham. Clevr-math: dataset for com- positional language, visual and mathematical reasoning. arXiv preprint arXiv:2208.053356 2022.
[2] OpenAL Leanning to reason with 1lms.
learning- teasoooo----------ee-----oo-ooee creason-with 11ms/ 2024. Accessed: 2025-03-02.
[3] Maria MP Petrou and Costas Petrou. Image processing: the fundamentals. John Wiley & Sons, 2010.
[4] Huajie Tan, Guoqing Xiang, Xiaodong Xie, and Huizhu Jia. Joint frame-level and block rate-pereeption optimized preprocessing for video coding. In Proceedings of the 6th ACM International Conference on Multimedia in Asia, pages 1-12002.
[5] Qizhe Zhang, Mengzhen Liu, Lichen Li, Ming Lu, Yuan Zhang, Junwen Pan, Qi She, and Shanghang Zhang. Beyond attention or similarity: Maximizing conditional diversity for token pruning in milms. arXiv preprint arXiv:2506. 10967, 2025.
[6] Yuheng Ji, Yue Liu, Zhicheng Zhang, Zhao Zhang, Yuting Zhao, Gang Zhou, Xingwei Zhang, Xinwang Liu, and Xiaolong Zheng. Advlora: Adversarial low-rank adaptation of vision- language models. arXiv preprint arXiv:2404. 1.0004..3444 13425, 2024.
[7] Yuheng Ji, Yue Liu, Zhicheng Zhang, Zhao Zhang, Yuting Zhao, Xiaoshuai Hao, Gang Zhou, Xingwei Zhang, and Xiaolong Zheng. Enhancing adversarial robustness of vision-language models through low-rank adaptation. In Proceedings of the 2025 International Conference on Multimedia Retrieval, pages 550-559, 2025.
[8] Yuheng Ji, Xingwei Zhang, Gang Zhou, Xiaolong Zheng, and Daniel Dajun Zeng. Learning hash subspace from large-scale multi-modal pre-training: A clip-based cross-modal hashing framework. In China Conference on Command and Control, pages 514-526. Springer, 2023.
[9] Huaihai Lyu, Chaofan Chen, Yuheng and Changsheng Xu. Egoprompt: Prompt pool learning for egocentric action recognition. arXiv preprint arXiv: 033666 2025.
[10] Jianan Mu, Huajie Tan, Shuai Chen, Min Cai, Jing Ye, Huawei Li, and Xiaowei Li. Config- urable and high-level pipelined lattice-based post quantum cryptography hardware accelerator design. In 2023 IEEE 32nd Asian Test Symposium (ATS), pages 1-6. IEEE, 2023.
[11] Jianan MU, Huajie Tan, Jiawen Wu, Haotian Lu, Chip-Hong Chang, Shuai Chen, Shengwen Liang, Jing Ye, Huawei Li, and Xiaowei Li. and 2-d pe array. In 2023 Design, Automation & Test in Europe Conference & Exhibition (DATE), pages 1-2. IEEE, 2023.
efficient ntt design with one- bank sram
[12] Songran Bai, Yuheng Ji, Yue Liu, Xingwei Zhang, Xiaolong Zheng, and Daniel Dajun Zeng. Alleviating performance disparity in adversarial spatiotemporal graph learning under zero-inflated distribution. In Proceedings of the AAA/ Conference on Artificial Intelligence, volume 39, pages 1143--1444, 2025.
[13] Marius Cordts, Mohamed Omran, Sebastian Ramos, Timo Rehfeld, Markus Enzweiler, Rodrigo Benenson, Uwe Franke, Stefan Roth, and Bernt Schiele. The cityscapes dataset for semantic urban scene understanding. In IEEE conference on computer vision and pattern recognition. pages 3213-3223 2016.
[14] Jihan Yang, Shusheng Yang, Anjali W Gupta, Rilyn Han, Li Fei-Fei, and Saining Xie. Thinking in space: How multimodal large language models see, remember, and recall spaces. arXiv preprini 24 411,, 2024.
[15] Li-Ming Zhan, Bo Liu, Lu Fan, Jiaxin Chen, and Xiao-Ming Wu. Medical visual question answering via conditional reasoning. ACM International Conference on Multimedia, pages 2345-2354, 2020.
10

---

[16] Jiazhen Pan, Che Liu, Junde Wu, Fenglin Liu, Jiayuan Zhu, Hongwei Bran Li, Chen Chen, Cheng Ouyang, and Daniel Rueckert. Medvlm-rl: Incentivizing medical reasoning capability of vision- vision-language models (vlms) via reinforcement learning. arXiv preprint arXiv:2502. 19634444 2025.
[17] Yuheng Ji, Huajie Tan, Jiayu Shi, Xiaoshuai Hao, Yuan Zhang, Hengyuan Zhang, Pengwei Wang, Mengdi Zhao, Yao Mu, Pengju An, et al. Robobrain: A unified brain model for robotic manipulation from abstract to concrete, In Proceedings of the Computer Vision and Pattern Recognition Conference, pages 1724-1734, 2025.
[18] BAAI RoboBrain Team, Mingyu Cao, Huajie Tan, Yuheng Ji, Minglan Lin, Zhiyu Li, Zhou Cao, Pengwei Wang. Enshen Zhou, Yi Han, et al. Robobrain 2.0 technical report. arXiv preprint wrxiv22577.02029 2025.
[19] Jiaming Liu, Mengzhen Liu, Zhenyu Wang, Lily Lee, Kaichen Zhou, Pengju An, Senqiao Yang, Renrui Zhang, Yandong Guo, and Shanghang Zhang. Robomamba: Multimodal state space model for efficient robot reasoning and manipulation. arXiv e-prints, pages arXiv-2406, 2024.
[20] Huajic Tan, Xiaoshuai Hao, Cheng Chi, Minglan Lin, Yaoxu Lyu, Mingyu Cao, Dong Liang, Zhuo Chen, Mengsi Lyu, Cheng Peng, et al. Roboos: A hierarchical embodied framework for cross-embodiment and multi-agent collaboration. arXiv preprint S.Xiv:250500367. 2025.
[21] Enshen Zhou, Qi Su, Cheng Chi, Zhizheng Zhang, Zhongyuan Wang, Tiejun Huang, Lu Sheng, and He Wang. Code-as-monitor: Constraint-aware visual programming for reactive and proactive robotic failure detection. In Proceedings of the Computer Vision and Puttern Recognition Conference, pages 6919-6929, 2025.
[22] Enshen Zhou, Jingkun An, Cheng Chi, Yi Han, Shanyu Rong, Chi Zhang, Pengwei Wang, Zhongyuan Wang, Tiejun Huang, Lu Sheng. et al. Roborefer: Towards spatial referring with reasoning in vision- Asionnlanguage models for robotics. arXiv preprint arXiv:2506.0433
2025.
[23] Dingzhe Li, Yixiang Jin, Yuhao Sun, Hongze Yu, Jun Shi, Xiaoshuai Hao, Peng Hao, Huaping Liu, Fuchun Sun, Jianwei Zhang, et al. What foundation models can bring for robot learning in manipulation: A survey. arXiv preprint arXiv22404. 18201, 2024.
[24] Zirui Song, Guangxian Ouyang, Mingzhe Li, Yuheng Ji, Chenxi Wang, Zixiang Xu, Zeyu Zhang. Xiaoqing Zhang, Qian Jiang, Zhenhao Chen, et al. Maniplvm-rl: Reinforcement learning for reasoning in embodied manipulation with large vision-language models. arXiv preprint arXiv:2505. 16517, 2025.
[25] Xiaoshuai Hao, Yunfeng Diao, Mengchuan Wei, Yifan Yang, Peng Hao, Rong Yin, Hui Zhang, Weiming Li, Shu Zhao, and Yu Liu. Mapfusion: A novel bev feature fusion network for multi-modal map construction. Information Fusion, page 103018, 2025.
[26] Xiaoshuai Hao, Mengchuan Wei, Yifan Yang, Haimei Zhao, Hui Zhang. Yi Zhou, Qiang Wang, Weiming Li, Lingdong Kong, and Jing Zhang. Is your HD map constructor reliable under sensor corruptions In Advances in Neural Information Processing Systems, 2024.
[27] Xiaoshuai Hao, Ruikai Li, Hui Zhang. Dingzhe Li, Rong Yin, Sangil Jung, Seung-In Park, ByungIn Yoo. Haimei Zhao, and Jing Zhang. Mapdistill: Boosting efficient baser hd map construction via camera-lidar fusion model distillation. In European Conference on Computer Vision, pages 166-183, 2024.
[28) Xiaoshuai Hao, Hui Zhang, Yifan Yang. Yi Zhou, Sangil Jung, Seung-In Park, and ByungIn Yoo. Mbfusion: A new multi-modal bev feature fusion method for hd map construction. In IEEE International Conference on Robotics and Automation, pages 15922-15928. 2024.
[29] Xiaoshuai Hao, Guanqun Liu, Yuting Zhao Yuheng Ji, Mengchuan Wei, Haimei Zhao, Lingdong Kong, Rong Yin, and Yu Liu. Msc-bench: Benchmarking and analyzing multi- sensor corruption for driving perception. arXiv preprint arXiv:2501.01037 2025.
[30] Yuting Zhao, Yuheng Ji, Xiaoshuai Hao, and Shuxiao Li. Fastrsr: Efficient and accurate road surface reconstruction from bird's eye view. arXiv preprint arXiv:2504.09535, 2025.
[31] Xiaoshuai Hao, Yuting Zhao, Yuheng Ji, Luanyuan Dai, Peng Hao, Dingzhe Li, Shuai Cheng, and Rong Yin. What really matters for robust multi-sensor hd map construction? arXiv preprint arXiv:2507.00 250::0140..444 01484, 2025.
11

---

[32] Artur d' Avila Garcez, Marco Gori, Luis C Lamb, Luciano Serafini, Michael Spranger, and Son N Tran. Neural-symbolic computing: An effective methodology for principled integration of machine learning and reasoning. arXiv preprint arXiv: 1905. 060806 2019.
[33] Saced Amizadeh, Hamid Palangi, Alex Polozov. Yichen Huang, and Kazuhito Koishida. Neuro-symbolic visual reasoning: Disentangling. In International Conference Machine Learning, pages 279-290, Pmlr, 2020.
[34] Minkyu Choi, Harsh Goel, Mohammad Omama, Yunhao Yang, Sahil Shah, and Sandeep Chin- chali. Towards neuro-symbolic video understanding, In European Conference on Computer Vision, pages 220-236. Springer, 2024.
[35] Mingyu Zhang, Jiting Cai, Mingyu Liu, Yue Xu, Cewu Lu, and Yong-Lu Li. Take step back: Rethinking the two stages in visual reasoning. In European Conference on Computer Vision,
pages 124-141. Springer, 2024.
[36] Tanmay Gupta and Aniruddha Kembhavi. Visual programming: Compositional visual reason- ing without training. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14953-14962, 2023.
[37] Guowei Xu, Peng Jin, Li Hao, Yibing Song, Lichao Sun, and Li Yuan. Llava-ol: Let vision language models reason step-by-step. arXiv preprini arXiv:241 10440, 2024.
[38] Omkar Thawakar, Dinura Dissanayake, Ketan More, Ritesh Thawkar, Ahmed Heakl, Noor Ahsan, Yuhao Li. Mohammed Zumri, Jean Lahoud, Rao Muhammad Anwer, et al. Llamav-ol: Rethinking step-by-step visual reasoning in llms. arXiv preprint arXiv:2501.0611
2025.
[39] Daya Guo. Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang. Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-rl: Incentivizing reasoning capability in Ilms via reinforcement learning. arXiv preprint arXiv:2501.1294 2025.
[40] Kimi Team, Angang Du, Bofei Gao, Bowei Xing, Changjiu Jiang, Cheng Chen, Cheng Li, Chenjun Xiao, Chenzhuang Du, Chonghua Liao, et al. Kimi k1. 5: Scaling reinforcement learning with Ilms. arXiv preprint arXiv:2501. 12590, 2025.
[41] Pan Lu, Hritik Bansal, Tony Xia, Jiacheng Liu, Chunyuan Li, Hannaneh Hajishirzi, Hao Cheng, Kai-Wei Chang, Michel Galley, and Jianfeng Gao. Mathvista: Evaluating mathematical reasoning of foundation models in visual contexts. arXiv preprint arXiv:2310.0225: 2023.
[42] Xiaoshuai Hao, Zhu, Srikar Appalaraju, Aston Zhang, Wanqian Zhang, Bo Li, and Mu Li. Mixgen: A new multi-modal data augmentation. In IEEE/CVF winter conference on applica- tions of computer vision, pages 379-389, 2023.
[43] Yue Ma, Yulong Liu, Qiyuan Zhu, Ayden Yang, Kunyu Feng. Xinhua Zhang. Zhifeng Li, Sirui Han, Chenyang Qi, and Qifeng Chen. Follow-your-motion Video motion transfer via efficient spatial-temporal decoupled finetuning. arXiv preprint arXiv:2506.00 05207, 2025.
[44] Yue Ma, Hongyu Liu, Hongfa Wang, Heng Pan, Yingqing He, Junkun Yuan, Ailing Zeng, Chengfei Cai, Heung- Yeung Shum, Wei Liu, al. Follow-your-ent emoji: controontrll and expressive freestyle portrait animation. In SIGGRAPH Asia 2024 Conference Papers, pages 1-12, 2024.
[45] Yue Ma, Xiaodong Cun, Yingqing Hc,, Chenyang Qi, Xintao Wang. Ying Shan, Xiu Li, and Qifeng Chen. Magicstick: Controllable video editing via control handle transformations. arXiv preprint 1rxiv:2312.03044 2023.
[46] Yue Ma, Yingqing He, Xiaodong Cun, Xintao Wang, Siran Chen, Xiu Li, and Qifeng Chen. Follow your pose: Pose-guided text-to-video generation using pose-free videos. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 4117-4125, 2024.
[47] Yue Ma. Yali Wang. Yue Wu, Ziyu Lyu, Siran Chen, Xiu Li, and Yu Qiao. Visual knowledge graph for human action reasoning in videos. In Proceedings of the 30th ACM International Conference on Multimedia, pages 4132-4141 2022.
[48] Zexuan Yan, Yue Ma, Chang Zou, Wenteng Chen, Qifeng Chen, and Linfeng Zhang. Eedit: Rethinking the spatial and temporal redundancy for efficient image editing. arXiv preprint arXiv:2503.100
2025.
[49] Kunyu Feng, Yue Ma, Xinhua Zhang, Boshi Liu, Yikuang Yuluo, Yinhan Zhang, Runtao Liu, Hongyu Liu, Zhiyuan Qin, Shanhui Mo, et al. follow-yourr-nstrretiin comprehensive milm agent for world data synthesis. arXiv preprint arx.vv 2025.
12

---

[50] Yikuang Yuluo, Yue Ma, Kuan Shen, Tongtong Jin, Wang Liao, Yangpu Ma, and Fuquan Wang. soooo---our--hpppp Shape-awan aware image editing via trajectory- guuded region control. arXiv preprint arXiv:2508.08/33 2025.
[51] Zhuowan Li, Xingrui Wang. Elias Stengel-Eskin, Adam Kortylewski, Wufei Ma. Benjamin Van Durme, and Alan I. Yuille. Super-clevr: A virtual benchmark to diagnose domain robustness in visual reasoning. In IEEE/CVE conference on computer vision and pattern recognition, pages 14963-14973, 2023.
[52] Jiahui Gao, Renjie Pi, Jipeng Zhang. Jiacheng Ye, Wanjun Zhong, Yufei Wang. Lanqing Hong, Jianhua Han, Hang Xu, Zhenguo Li, et al. G-llava: Solving geometric problem with multi-modal large language model. arXiv preprint arXiv:2312. 1370, 2023.
[53] Mehran Kazemi, Hamidreza Alvari, Ankit Anand, Jialin Wu, Xi Chen, and Radu Soricut. Geomverse: A systematic evaluation of large models for geometric reasoning. arXiv preprint urXiv:2312.12241 2023.
[54] Renrui Zhang, Xinyu Wei, Dongzhi Jiang, Ziyu Guo, Shicheng Li, Yichi Zhang, Chengzhuo Tong. Jiaming Liu, Aojun Zhou, Bin Wei, et al. Mavis: Mathematical visual instruction tuning with an automatic data engine. arXiv preprint arXiv:2407.08739, 2024.
[55] Wenhao Shi, Zhiqiang Hu, Yi Bin, Junhua Liu, Yang Yang. See- Bee-Kiong Ng. Lidong Bing, and Roy Ka-Wei Lee. Math-llava: Bootstrapping mathematical reasoning for multimodal large language models. arXiv preprint arXiv:2406. 17294, 2024.
[56] Xin Hong, Yanyan Lan, Liang Pang, Jiafeng Guo, and Xueqi Cheng. Transformation driven visual reasoning. In IEEE/CVF Conference on computer vision and pattern recognition, pages 6903-6912, 2021.
[57] Yuheng Ji, Yipu Wang, Yuyang Liu, Xiaoshuai Hao, Yue Liu, Yuting Zhao, Huaihai Lyu, and Xiaolong Zheng. Visualtrans: A benchmark for real-world visual transformation reasoning. arXiv preprint arXiv:2508. 04043, 2025.
[58] Pan Lu, Swaroop Mishra, Tanglin Xia, Liang Qiu, Kai- Rei Chang, Song-Chur Zhu, Oyvind Tafford, Peter Clark, and Ashwin Kalyan, Learn to explain: Multimodal reasoning via thought chains for science question answering. Advances in Neural Information Processing Systems, pages 2507-2521, 2022.
[59] Aniruddha Kembhavi, Mike Salvato, Eric Kolve, Minjoon Seo, Hannaneh Hajishirzi, and Ali Farhadi. A diagram is worth a dozen images. European Conference on Computer Vision, pages 235-251, 2016.
[60] Yingdong Hu, Fanqi Lin, Tong Zhang, Li Yi, and Yang Gao. Look before you leap: Unveiling the power of gpt-4v in robotic vision-language planning. arXiv preprini 1..7422 2023.
[61] Peng Hao, Chaofan Zhang, Dingzhe Li, Xiaoge Cao, Xiaoshuai Hao, Shaowei Cui, and Shuo Wang. Tla: hangill-angguag-accttiin model for contact-rich manipulation. arXiv preprint arXiv:2503.068 00008,885,.00 2025.
[62] Justin Johnson, Bharath Hariharan, Laurens Van Der Maaten, Judy Hoffman, Li Fei-Fei,
C Lawrence Zitnick, and Ross Girshick. Inferring and executing programs for visual reasoning.
In IEEE international conference computer vision, pages 2989-2998, 2017.
[63] Dídac Surís, Sachit Menon, and Carl Vondrick. Vipergpt: Visual inference via python execution for reasoning. In IEEE/CVF International Conference on Computer Vision pages
11 11888-11898 2023.
[64] Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, pages 24824-24837, 2022.
[65] Yuhao Dong, Zuyan Liu, Hai-Long Sun, Jingkang Yang. Winston Hu, Yongming Rao, and Ziwei Liu. Insight-v: Exploring long-chain visual reasoning with multimodal large language models. arXiv preprint arXiv:2411. 14.32222, 2024.
[66] Daya Guo, Dejian Yang. Haowei Zhang, Junxiao Song. Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-rl: Incentivizing reasoning capability in 1lms via reinforcement learning. arXiv preprint 2rXiv:2501.12248 2025.
13

---

[67] Yingbo Tang, Shuaike Zhang, Xiaoshuai Hao, Pengwei Wang, Jianlong Wu, Zhongyuan Wang. and Shanghang Zhang. Affordgrasp: In-context affordance reasoning for open-vocabulary task-oriented grasping in clutter. arXiv preprint arXiv:2503.0000
2025.
[68] Lingfeng Zhang, Xiaoshuai Hao, Qinwen Xu, Qiang Zhang, Xinyao Zhang, Pengwei Wang, Jing Zhang, Zhongyuan Wang. Shanghang Zhang, and Renjing Xu. Mapnav: A novel memory representation via annotated semantic maps for vlm-based vision-and- lad---aa-- navigation. arXiv preprint arXiv:2502. 13451, 2025.
[69] Komal Kumar, Tajamul Ashraf, Omkar Thawakar, Rao Muhammad Anwer, Hisham Cholakkal, Mubarak Shah, Ming- Hsuan Yang. Phillip HS Torr, Salman Khan, and Fahad Shahbaz Khan. Llm post-training: A deep dive into reasoning large language models. arXiv preprint arXiv:2502.2132/1 2025.
[70] Tianzhe Chu, Yuexiang Zhai, Jihan Yang, Shengbang Tong, Saining Xie, Dale Schuurmans, Quoc Le. Sergey Levine, and Yi Ma. Sit memorizes, rl generalizes: A comparative study of foundation model post-training. arXiv preprint arXiv:2501.17161 2025.
[71] Jason Wei, Xuezhi Wang. Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoe Le, Denny Zhou, et al. thhougt prompting elicits reasoning in large language models. Advances in neural information processing systems, pages 24824 -4237,, 2022.
[72] Ke Wang, Houxing Ren, Aojun Zhou, Zimu Lu, Sichun Luo, Weikang Shi, Renrui Zhang, Linqi Song, Mingjee Zhan, and Hongsheng Li. Mathcoder: Seamless code integration in Ilms for enhanced mathematical reasoning. arXiv preprini arXiv:2310.0373: 2023.
[73] Daniel M Ziegler, Nisan Stiemnon, Jeffrey Wu, Tom B Brown, Alec Radford, Dario Amodei, Paul Christiano, and Geoffrey Irving. Fine-tuning language models from human preferences. arXiv preprint arXiv: 190.0099,, 2019.
[74] Long Ouyang. Jeffrey Wu, Xu Jiang. Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback. Advances in neural information processing systems, pages 27730-27744, 2022.
[75] Zhiqing Sun, Sheng Shen. Shengeao Cao, Haotian Liu, Chunyuan Li, Yikang Shen, Chuang Gan, Liang- Yan Gui, Yu-Xiong Wang, Yiming Yang, et al. Aligning large multimodal models with factually augmented rlhf. arXiv preprint arXiv:2309.. 14525, 2023.
[76] Haipeng Luo, Qingfeng Sun, Can Xu, Pu Zhao, Jianguang Lou, Chongyang Tao, Xiubo Geng, Qingwei Lin, Shifeng Chen, and Dongmei Zhang. Wizardmath: Empowering math- ematical reasoning for large language models via reinforced evol-instruct. arXiv preprint arXiv:2308.0955 2023.
[77] Simon Zhai, Hao Bai, Zipeng Lin, Jiayi Pan, Peter Tong, Yifci Zhou, Alane Suhr, Saining Xie, Yann LeCun, Yi Ma, et al. Fine-tuning large vision-language models as decision-making agents via reinforcement learning. Advances in Neural Information Processing Systems, pages 110935-119971, 2025.
[78] Jason Wei, Maarten Bosma, Vincent Y Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M Dai, and Quoe V Le. Finetuned language models are zero-shot learners. arXiv preprini arXiv: 20t.::222000155.... 2021.
[79] Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. The llama 3 herd of models. arXiv preprint arXiv:2407.2 0777833, 2024.
[80] Bo Adler, Niket Agarwal, Ashwath Aithal, Dong H Anh, Pallab Bhattacharya, Annika Brundyn, Jared Casper, Bryan Catanzaro, Sharon Clay, Jonathan Cohen, al. Nemotron-4 340b technical report. arXiv preprini arXiv:2406.J / 1704, 2024.
[81] Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model is secretly reward model. Advances in Neural Information Processing Systems, pages 33728-53741 2023.
[82] Nathan Lambert, Jacob Morrison, Valentina Pyatkin, Shengyi Huang, Hamish Ivison, Faeze Brahman, Lester James V Miranda, Alisa Liu, Nouha Dziri, Shane Lyu, et al. T ulu 3: Pushing frontiers in open language model post-training. arXiv preprint 241152241..15124, 2024.
14

---

[83] Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, et al. Deepseek-v3 technical report. arXiv preprini arXiv:2412. 19437, 2024.
[84] Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Y Wu, et al. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint (.Xix:22402.333000 2024.
[85] Pan Lu, Ran Gong, Shibiao Jiang, Liang Qiu, Siyuan Huang, Xiaodan Liang, and Song- -huun Zhu. Inter-gps: Interpretable geometry problem solving with formal language and symbolic reasoning. arXiv preprint S.dv:2205.041166 2021.
[86] Kaichen Zhang, Bo Li, Peiyuan Zhang, Fanyi Pu, Joshua Adrian Cahyono, Kairui Hu, Shuai Liu, Yuanhan Zhang, Jingkang Yang, Chunyuan Li, et al. Lmms-eval: Reality check on the evaluation of large multimodal models. arXiv preprint arXiv:2407112722 2024.
[87] Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang. Wenbin Ge, Yang Fan, Kai Dang, Mengfei Du, Xuancheng Ren, Rui Men, Dayiheng Liu. Chang Zhou, Jingren Zhou, and Junyang Lin. Qwen2-vl: Enhancing vision- language model's perception of the world at any resolution. arXiv preprint arXiv:2409.12191, 2024.
[88] Huggingface. open-rl: Fully open reproduction of deepseek-rl. huggingfacce/open-rr 2025. [Online; accessed: 2025-01-22 -2-24.
https://github.ccm com/
[89] Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, and Ion Stoica. Efficient memory management for large language model serving with pagedattention. In ACM SIGOPS 29th Symposium on Operating Systems Principles, 2023.
[90] Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang. Shijie Wang, Jun Tang, Humen Zhong. Yuanzhi Zhu, Mingkun Yang. Zhaohai Li, Jianqiang Wan, Pengfei Wang, Wei Ding, Zheren Fu, Yiheng Xu, Jiabo Ye, Xi Zhang, Tianbao Xie, Zesen Cheng, Hang Zhang, Zhibo Yang, Haiyang Xu, and Junyang Lin. Qwen2.5-vl technical report. arXiv preprint arXiv:2502. 13923, 2025.
[91] Marah Abdin, Jyoti Aneja, Hany Awadalla, Ahmed Awadallah, Ammar Ahmad Awan, Nguyen Bach, Amit Bahree, Arash Bakhtiari, Jianmin Bao, Harkirat Behl, al. Phi-3 technical report: A highly capable language model locally on your phone. arXiv preprint 10XXv:2404.14200 2024.
[92] Zhe Chen, Jiannan Wu, Wenhai Wang. Weijie Su, Guo Chen, Sen Xing, Muyan Zhong, Qinglong Zhang, Xizhou Zhu, Lewei Lu, et al. Internvl: Scaling up vision foundation models and aligning for generic visual-linguistic tasks. In JEEE/CVF conference on computer vision and paltern recognition, pages 24 1418-24 2024.
[93] Meta AI. Llama 3 at connect 2024: Vision for edge and mobile devices, 2024. Accessed: 2025-02-15.
[94] Pravesh Agrawal, Szymon Antoniak, Emma Bou Hanna, Baptiste Bout, Devendra Chaplot, Jessica Chudnovsky, Diogo Costa, Baudouin De Monicault, Saurabh Garg, Theophile Gervet, et al. Pixtral 12b. arXiv preprint arXiv:2410.07073, 2024.
[95] Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, et al. Gpt-4o system card. arXiv preprint ar.iv:2410..112700 2024.
[96] Gemini Team, Petko Georgiev, Ving Ian Lei, Ryan Burnell, Libin Bai, Anmol Gulati, Garrett Tanzer, Damien Vincent, Zhufeng Pan, Shibo Wang, et al. Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context. arXiv preprint 1:Xi::2403.05530 2024.
[97] Xiang Yue, Yuansheng Ni, Kai Zhang, Tianyu Zheng, Ruoqi Liu, Ge Zhang, Samuel Stevens, Dongfu Jiang, Weiming Ren, Yuxuan Sun, et al. Mmmu: A massive multi musti-discipline multi- modal understanding and reasoning benchmark for expert agi. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9556-9567, 2024.
ai. [98] Grok-1.5 Team. 115 vision preview. http:://...a//nessgrook-- 5v 2024. [Online; accessed: 2024-04-12].
15

---

[99] Ke Wang, Junting Pan, Weikang Shi, Zimu Lu, Houxing Ren, Aojun Zhou, Mingjie Zhan, and Hongsheng Li. Measuring multimodal mathematical reasoning with math-vision dataset. Advances in Neural Information Processing Systems, 77:95095-95169 2024.
[100] Tuomo Hiippaaa, Malihe Alikhani, Jonas Haverinen, Timo Kalliokoski, Evanfiya Logacheva, Serafina Orekhova, Aino Tuomainen, Matthew Stone, and John A Bateman. Ai2d-rst: multimodal corpus of 1000 primary school science diagrams. Language Resources and Evaluation, 55:661-688, 2021.
[101] Pan Lu, Swaroop Mishra, Tanglin Xia, Liang Qiu, Kai-Wei Chang, Song-Chun Tafjord, Peter Clark, and Ashwin Kalyan, Learn to explain: Multimodal reasoning via thought chains for science question answering. Advances in Neural Information Processing Systems, 35:2507-2521, 2022.
Chun Zhu, Oyvind
[102] Yanbin Wei, Shuai Fu, Weisen Jiang, Zejian Zhang, Zhixiong Zeng, Qi Wu, James Kwok, and Yu Zhang. Gita: Graph to visual and textual integration for vision-language graph reasoning. Advances in Neural Information Processing Systems, 37:44-72, 2024.
[103] Yew Ken Chia, Vernon Toh Yan Han, Deepanway Ghosal, Lidong Bing, and Soujanya Poria. Puzzlevqa: Diagnosing multimodal reasoning challenges of language models with abstract visual patterns. arXiv preprint arXiv:2403. 13315, 2024.
[104] Pan Lu, Liang Qiu, Jiaqi Chen, Tony Xia, Yizhou Zhao, Wei Zhang, Zhou Yu, Xiaodan Liang, and Song-Chun Zhu. Iconqa: A new benchmark for abstract diagram understanding and visual language reasoning. arXiv preprini
(tfi::2/10.132//// 2021.
[105] Chi Zhang, Feng Gao, Baoxiong Jia, Yixin Zhu, and Song-Chun Zhu. Raven: A dataset for relational and analogical visual reasoning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 5317-5327, 2019.
[106] Jiaqi Chen, Jianheng Tang, Jinghui Qin, Xiaodan Liang, Lingbo Liu, Eric P Xing, and Liang Lin. Geoqa: A geometric question answering benchmark towards multimodal numerical 2021.
reasoning. arXiv preprint
16

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: See Section D
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results be expected to generalize other settings.
is fine include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
See Appendix Section Limitations and Societal Impact.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification. asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low images are taken in low lighting. Or speech-to-text system might be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't 't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
17

---

Justification:
The paper does not include theoretical results or proofs but focuses on archi- tectural innovation and empirical evaluation.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proof's can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification: See Section 4
Guidelines:
The answer NA means that the paper does include experiments. If the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility, In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
18

---

Answer: [Yes]
Justification:
See supplementary material.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips.co cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (c.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
//nips.com wttpcccpppbccccceee guwdes/dde/eeee Coooeooodeesssseeeeeee http:///d///uuddddd//eee/e/ee/s/
ssiniiiiccypyy for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Does the paper specify all the training and test details (c.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary understand the results?
Question:
Answer: [Yes]
Justification:
See Section Implementation details and Appendix Sec.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, appendix, as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: We use five various seeds to train the model and report the average accuracy of them to avoid randomness.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the enror bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
19

---

It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative enror rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification: See Section Implementation details.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary failed experiments that didn't make into the paper).
9. Code ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics http:p/p///aarrr.....o/oop/////a
Answer: [Yes]
We have read the NeurIPS Code of Ethics, and our paper does not have these problems.
Justification:
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification:
See Appendix Limitations and Societal Impact.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
20

---

The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility MI..
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: The paper poses no such risks.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Lieenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: CC-BY 4.0.
Guidelines:
The answer NA means that the paper does existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
21

---

If this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: The paper does not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/mmoel as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question:
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
16. Declaration of LLM usage
22

---

Question: Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods this research? Note that if the LLM used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, or originality the research, declaration is not required.
Answer: [No]
Justification: The core methodology and experiments do not involve LLMs. Any language editing assistance does not impact the scientific contributions.
Guidelines:
The answer NA
that the method development in this research does involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy http:///eearrpsscc/ooeeeeeeeeeee for what should or should be described.
23

---

Appendix
This supplementary material provides additional details on the proposed method and experimental results that could not be included in the main manuscript due to page limitations. Specifically, this appendix is organized as follows.
Sec. A provides more details on the evaluation reasoning tasks and discusses how we collected, filtered, and reconstructed high-quality dataset.
Sec.
B
outlines the models and training processes, providing more detailed experimental specifics.
Sec.
presents comprehensive experimental results.
Sec.
details the pipeline of CoT date generation.
Sec.
presents detailed composition of diffenernt mixed CoT datasets.
Sec.
shows the comparison of CoT quality before and after RL.
Sec.
includes more visualization cases.
Sec.
introduces the limitations of our Reason-RFT and its societal impact.
A Details of Evaluation Reasoning Tasks
A.1 Visual Counting
Task Definition Visual Counting is a multi- modal reasoning task that evaluates the integra- tion linguistic, visual, and mathematical capa- bilities by requiring models solve arithmetic problems in dynamic visual scenes composed of 3D blocks with diverse attributes, including color, size. material, and shape. The task con- sists of four distinct reasoning types: 1) Sub- traction, which involves counting objects after removing specified subset based on given at- tributes; 2) Addition, where models must com- pute totals after inserting new objects with de- fined quantities and properties; 3) Adversarial, a challenging variant designed as trick questions in which operations are performed objects while the query targets an unrelated or unaffected subset, testing the model's robustness against deceptive scenarios; and 4) Multi-Hop, which requires sequential reasoning through multiple addition or subtraction steps to arrive at the final count. This task challenges models to perform attribute-based reasoning in dynamic visual contexts, emphasizing cross-modal understanding and reasoning capabilities. Some examples are shown in Fig. g
In-Domain
Oft-a--Domain
Prompe: You are visuel countica expart. Given cimage frone you, renove
Prosupt: You are dissal countng repert Gleen the cinsgeo froat of
obactt are thare the table?
hoom theme table?
Figure 8: The sample of Visual Counting.
one set of
Dataset Preparation For In-Domain (ID) dataset, we refined the original dataset from CLEVR- Math [ by filtering out low-quality or incorrect samples using GPT-4o, resulting clean dataset comprising 35K training samples and 1K test samples. These samples are categorized into four specific types: subtraction, addition, adversarial, and multihop-subtraction (DS) generalization, we extended CLEVR-Math by enhancing the diversity of objects through the incorporation of 3D assets from Super-CLEVR (51). which leads to the creation Super-Clas CLEVR- Math, advanced benchmark with IK test samples designed to assess model generalization under increased complexity. These test samples are also divided into four task types: addition, subtraction, intrractoonmultihonn and addition-suutraatioon The test samples are further categorized into four task types: addition, subtraction, subtraction wwnittiioommnitopp..oo and addition-subtractie constitute the DS-D subset, while addition-subtraction forms the DS-M subset. Notably, the mixed cuditionnsubbracttion type introduces a novel category consisting multi-steps of both addition and subtraction, which is not present in CLEVR-Math, further elevating the benchmark's challenge.
evaluate Domain-Shift
The first three
Reward Design Following the reward methodology of DeepSeek-RI [39], we define two dis-
tinct reward functions: Format Reward and Accuracy Reward. The Format Reward is assigned

---

a value of if the response adheres to the predefined template structure, specifically in the form of <think>... </think><<nsser>>..... ...</ answer>; otherwise, assigned a value of 0. The Accuracy Reward is assigned value of 1 if the numerical counting result in the response is correct; otherwise, it is assigned a value of 0. This dual-reward mechanism ensures both structural compliance and numerical accuracy in model responses.
A.2 Structure Perception
Task Definition Structure Perception represents complex class of visual mathematical rea- soning tasks, which focuses on assessing the model's capacity determine geometric struc- ture relationships and perform calculations in- volving angles, lengths, areas, and other geomet- ric properties. The task includes problems such as identifying congruent or similar shapes, calcu- lating perimeters and areas, determining angles between lines or shapes, and solving problems related to geometric transformations (c.g., ro- tations, translations, and reflections). By com- bining mathematical rigor with visual reasoning, this task challenges models to demonstrate deep understanding of geometric principles in
In-Domain
Dot-an-Dommin
Angle?
Volume
Figure 9: The sample of Structure Perception.
both abstract and real-world scenarios. Some examples are shown in Fig. g
Dataset Preparation For the ID dataset, we provide GeoMath-4K5, a dataset specifically designed for geometric problem solving, which is constructed based on Math360K [55] and Geol 70KK [52). To ensure data quality, we employed GPT-4o filter out incorrect samples and removed those with answers that were neither numerical nor included in the provided options, thereby streamlining the validation process during training and testing. This refinement process resuited in a curated dataset consisting of 4.5K training samples and 820 test samples. For DS evaluation, we selected 800 samples from GeOmetry3K (85) (including 400 multiple-choice and 400 open-ended questions) to comprehensively assess the model's generalization capabilities on geometry reasoning.
Reward Design We maintain the same Format Reward as used in the Visual Counting task above. The Accuracy Reward is extended support the evaluation of both multiple-choie questions and mathematical expressions, ensuring comprehensive assessment across various problem types. Specifically, mathematical reward type is designed for Structure Perception tasks involving numerical answers, such as floatingpooit values or -oommtttd expressions. It uses a tolerance- based evaluation to account for minor numerical deviations. The accuracy reward Race (a.) is defined as:
[ppred Ggdl e1 X [agi]
Race (a.)
cos
+
(8)
(E2 - 61) log.
where Opred is the predicted answer, ag is the ground truth, is the tolerance threshold for an exact match (e.g., 0.05), and 22 the upper bound for partial rewards (e.g.. 0.20). [apred - agt E1 X lagl the reward is 1 (exact match); if [Opred Agl > <2 x lagil. the reward is 0 (incorrect). This formulation ensures smooth transitions between full and partial rewards, enabling fair numerical evaluation.
A.3 Spatial Transtormation
Task Definition Spatial Transformation is a spatial-visual reasoning task designed to infer single-step or multi-step transformation actions by analyzing the initial and final visual states from multi- ple perspectives (e.g., center, left, right). The task utilizes transformation functions, including Shange_sizer change_color,
Saange_shape,,,____ and Sraage__postie inn, to modify object properties such size, color, material, shape, and position using predefined values. This task evaluates the model's ability to reason about spatial relationships and object transformations across diverse viewpoints in dynamic visual scenarios. Some examples are shown in Fig. 10
aaange_materiall
Dataset Preparation We generated 100K samples using the environment and configuration from Trance (56). with each sample comprising initial object attributes, front-view image of initial state,
25

---

<thlin
Out-of-Domain Views
follows metal cubo isttal state bocomes meddum and keepiss the other obiets unchanged. fiiii
Inttal
Chanpeite medium).
OatrgeNaerrall nubberi. ChangyPacthinn hebind) spantwers
Function List
Left-View
ChangeGotor (obj, valuej
ChangePosition (ob), value]
Final Right-Vies
ChangeShape value)
Initial Center-View
Final Center-View
@ 1_0
Prompt: spatial visual reasoning Goeen center-view <tenages
ChangeMaterial
view <inages of insttal
the inft-vlew <images or the right-view <imagrs) final state, and the (Function List) be used. should determine change
Changedizz (ohg, value)
sequence tranffoormatoons
Figure 10: The sample of Spatial Transformation.
and images of final state captured from front, left, and right perspectives. ensure high data quality, we implemented rigorous filtering process: (I) removing samples containing occluded invisible objects either the initial or final states, (2) eliminating redundant actions within the transformation sequences, and (3) consolidating multi-step displacement actions, which collectively ensure the uniqueness and correctness the solutions. The refined dataset consists of 60K training samples and 6K test samples. For the training set, constructed the Trans-Centee Center-60K dataset using the Center-Center configuration, which pairs front-view initial and final state images. For ID evaluation, derived the Maans-Center-6 dataset from the 6K test samples under the same Center-Center configuration. To evaluate DS generalization, we constructed two additional datasets: Trans-Left-66K (DS-L) and Trans-Right-6K (DS-R), leveraging the Center-Left and Center-Righ configurations assess the model's 's generalization capabilities in spatial reasoning under viewpoint conditions.
Reward Design For the Format Reward, we adopted the same formulation as used in the Visual Counting task. As for the Accuracy Reward, a specialized design was developed to evaluate the sequence of transformation functions. Function-based type is designed for Spatial Transformation tasks requiring a sequence of transformation functions. The accuracy reward (a,c(a;) evaluates the alignment between the predicted sequence Tpod and the ground truth Tgt. computed as:
(ac(a:) len(TS+ovy) + len(T prod
len(TT pead)
peot
(9)
max(Ien((praa) (en))
where Tffoot" is the subset of transformation steps with complete matches (w/ function, object, and value), T/4o/" are the subsets with partial and only-function matches (w/ function and object, w/ function and value), Tuf is the subset with only-function matches. and 3 are the weighting coefficients for partial matches. This formulation ensures nuanced evaluation, rewarding both exact and partially correct responses while allowing flexible adjustment of partial match contributions.
System Prompts For the Spatial Transformation task, we designed two versions of the system prompt. The first version specifies the answer output format using the <think> and <answer> tags, while the second version includes additional outputs <summary> and <caption> for experiments on exploration of format reward design in the main paper. These two versions illustrated in Fig. and Fig. 12 respectively.
B Details of Models and Training
We utilize Qwen2- Owen2-VLL-2B and Qwen2-VL-7B 87 as the backbone models for our experiments. Our implementation is built on the open-source frameworks Open-R1 88 and vL..M (89), ensuring reproducibility and scalability. All experiments were conducted on cluster of servers, each equipped with 8 A800 GPUs. For the Visual Counting task and Spatial Transformation task. trained
26

---

Table 4: Detailed configuration for each training stage of Reason-RFT. The table presents the train- ing parameters for the 2B model and 7B model across three visual reasoning tasks. The parameters marked with correspond to Visual Counting Structure Perception / Spatial Transformation.
Qwen2-VL-22
Qwen2-VL
Stage-1
Stage-2
Stage-1
Stage-2
Model Duta
Dataset
CoT dataset
Non-CoT dataset
CoT dataset
Non-CoT dataset
#Samples
1.6K
35K/ 4.5K 60K
1.6K
35K/ / /4.5K/ 60K x
Trainable Part
Full Model
Full Model
Full Model
Full Model
#Tunable Parameters
2.21B
2.21B
8.29B
8.29B
Per-device Batch Size
1
1 2
Gradient Accumulation
2
2
IR:: (v2 1 .
1x10-5
x10-6
x10-6
1
Epoch
1 1/5/1
1/5/1*
Optimizer
AdamW
AdamW
AdamW
AdamW
Deepspeed
Zero3
Zero3
Zero3
Zero3
Training
Weight Decay
0.1
0.0
0.1
0.0
Warmup Ratio
0.03
0.00
0.03
0.00
LR Schedule
Cosine
Cosine
Cosine 32768
Cosine
Max Seq. Length
32768
4096
4096
Max Compl. Length
512
512
Num. of Compl.
8
4
GPU Nums
x 8
1 1x8
1x8 8
1 x 8
the models for epoch each on their respective training datasets, ensuring sufficient exposure to task-specific patterns while avoiding overfitting. For the Structure Perception task, due to its GeoMath training dataset consisting of a relatively small number of training samples (a total of 4,500), we extended the training duration to 5 epochs to allow the models to fully capture the underlying structural and geometric relationships. In the Reason-RFT training pipeline, all models underwent an initial CoT activation stage with 1,600 samples before proceeding to the RL phase. More details on training process of each models are shown in Tab. 4
C More Experiment Results
Exploration on COT Activation Data To investigate the impact of differently com- posed CoT activation data on Reason-RFT] we construct two distinct datasets: mixed domain-speciff dataset, which integrates relevant yet distinct data from in-domain tasks, and mixed general-domain dataset,
Table 5:
Results of various mixed CoT activation
datasets on the Structure Perception task.
Setting
CoT Activation Data
ID
DS
AVG
Baseline
GeoMath-only data
59.27
49.25
54,26
(w)
Mixed Specifie-Domain data
50.61
45.35
48.02
(b)
Mixed General- Doorran--Dmannd data
42.51
40.25
41.38
encompassing broader range of visual reasoning tasks (e.g., graph topology, visual puzzles). The detailed dataset composition is provided in Appendix Sec. E Using these datasets, we perform Reason- training on Structure Perception task, with the results detailed Tab. [3] From this, two key points emerge: (1) As the proportion of in-domain training data decreases, the model's s performance on specific tasks declines; (2) Models trained on more diverse visual reasoning domain data may also exhibit reduction in domain-specifee performance.
Results on Different Backbones We further validate the effectiveness of Reason-RFT stronger or alternative vision-language backhones. We report results on three visual reasoning tasks: Visual Counting (TI). Structure Perception (T2), and Spatial Transformation (T3) in the combined Tab. 6 Reason-RPP RFT achieves the strongest averages across backbones and tasks, with especially large margins on domain-shifted splits while keeping in-domain (ID) performance near ceiling. On Qwen2.5- V-LL3B. for T1 (Visual Counting) ID is already saturated (99.0 with vs. 98.8 with Reason-RE -FF)), yet ReessnnRFFT markedly improves robustness on DS: +9.2 on DS-D (68.7 vs. 59.5 vs. COT-SFT) and +5.6 on DS-M (54.8 vs. 49.2 vs. COT-SFT), with an especially large +44.0 over Reason- RFT-Zero on DS-M (54.8 vs. 10.8), yielding the best TI AVG (74.1); for T2 (Structure Perception) it is best on both ID/DS (59.00 0/56.6)).6.... beating COT-SFT by 42.9 (ID) and -7.2 (DS) and RFF- by +4.2 (ID) and +2.1 (DS), indicating that RI. enhances stepwise structural reasoning rather than overfitting; for T3 (Spatial Transformation) ANS-SFT
27

---

Table 6: Results different backbones across three tasks. Best is bold; second-best is underlined.
"ID" denotes in-domain; "DS- ** denotes domain- shifted splits. Missing results are shown as
Visual Counting (T))
Structure Perception (T2)
Spatial Transformation (T3)
Backbone
Method
ID
DS-D
DS-M
AVG
DS-R
Zero Shot
75.9
50.9
4.4
43.7
36.8
37.4
37.1
8.6
8.3
8.3
8.4
97.
51.5
53.0
31.8
42.4
91.1
47.0
61.6
Owen2,5-VLL33
CST-SFT
89.2
59.5
49.2
66.0
56.1
49.4
52.7
81.6
46.1
44.2
57.3
FF--on-FTT-ero
99.0
58.9
10.8
56.2
548
54.5
54.6
68.5
49.5
48.0
55.3
Renson-KFT
98.8
68.7
54.8
74.1
59.0
56.6
57.8
86.
54,4
Zero-Shoe
79.30
51.20
5.10
45.20
ANS-SFT
96.90
IntermV1.3.28
CaT-SFT
88.90
60.10
Reason- Recson-RFT-Zue RFT-Zero
98.90
50
12.30
56.87
Renson-FFT
99.10
69.30
55.90
74.93
attains the top ID (91.1) but is less robust, whereas Reason- RFT trades modest -4.4 on ID (86.7) for substantial DS gains of 48.2 on DS-L (55.2 vs. 47.0) and +7.6 on DS-R R (54.4 vs. 46.8). delivering for T1, Reason-RF7 is best on
the best AVG (65.4) and a superior ID/DS Pareto. On Intern ID/DS-D/DS-M (99.10/69) 80/55.00550000 90) and AVG (74.93), improving over ANS-SFT, CoT-SFT, and Reason-RFT-Zero by 123.16, +8.53, and +18.06, respectively, with the largest domain-shift margin on DS-M of +43.60 over Reason-RFT-Zero (55.90 vs. 12.30). Taken together, these trends across architecturally distinct backbones indicate that the benefits of Reason- are backbone- aggossii, improving both coherence- driven reasoning and out- distiiiiiinttionn reliability.
Evaluation on General Benchmarks Al- though Reason-RF7 is primarily designed to enhance domain-specific visual rea- soning abilities, we conduct a thorough evaluation on general benchmarks to ver- ify whether our approach compromises the model's general reasoning capabili- ties. Tab. 7 presents the results on four widely adopted datasets: MMMU [97]. RealWorldQA 198), Math [99]. and A12D [10]. Across all tasks and model scales, Reason-RFT consistently maintains or even improves general performance. For
Table Evaluation results on general benchmarks.
General
Method
MMMU RealWorldQA
Math Vision
A12D
Qwen2VI-2B-Instruut
Zero-Shot
39.89
61.31
12.50
72.50
+ SNS--FFT
40.56
48.76
15.79
68.20
+ -CT-SFFT
34,00
37.78
12.99
65.36
RResonn--F--2 RFT-Zoro
39:30
42.81
13.00
24.61
Reasou-RFT RFT
41.14
53.06
14.82
Qwen2VL.77-Insttee
Zero-Shot
67.19
16.30
+ SFT
42.66
48.10
9.12
78.30
C.T.SFT
44.67
36.46
15.30
73.25
+ Reason- RFT-Zero
46.44
45.10
10.86
75.28
RRTT
50.04
61.31
17,60
81:70
instance, on the 2B model, Reason-RFT achieves the highest scores on MMMU
(41.14) and AI2D (75.24) (75.24), outperforming both zero-shot baselines and other supervised fine-tuning approaches such as ANS-SFT and COT-SFT. Notably, it also improves performance on the challenging MathVision task ( (14.82), demonstrating its robustness in spatial reasoning. For the larger 7B model, Reason-RFT again surpasses ANS-SFT and CoT-SFT by large margins, particularly on WorldQA (61.31) and MMMU (50.04), while maintaining strong results on A12D (81.70). These results suggest that Reason-RFT not only scales effectively with model size but also introduces no observable performance degradation on general benchmarks. In summary, the empirical evidence supports that Reason-RFT RFT enhances domain-speciie langal-e reasoning capabilities. This confirms the robustness and transferability of our method, making it a strong alternative to conventional fine-tuning paradigms.
reasoning while preserving if not enhancing general
Performance at Different Training Steps Fig. 13 and Fig. 14 illustrate the ID and DS performance of all the training methods across three visual reasoning tasks, evaluated at various training sample sizes. This analysis helps us understand how each method scales with training data. More detail evaluation results for each subset of three tasks are in Tab, -0- Tab. [7 We systematically varied the number of training samples, from minimal substantial, allowing us identify performance thresholds and data efficiency for each method in both ID and DS contexts. Key findings from this analysis include: Data Efficiency of Reason- RFT: Reason-RF7 demonstrates exceptional data efficiency, achieving approximately 70% of the performance of Ceason-RFT-Zer with only 3% of the training data (1,600 samples), and 82.5% with just 9%. Robust Generalization to DS scenarios: In the 7B model, Reason-RFT achieves over 92% of Reasom-RFT-Zeroos 's performance using just 3% of the training data, showcasing its strong generalization capabilities. Comparison Across Methods: Reason-RFF RFFT consistently outperforms other methods, particularly in data-constrained scenarios,
28

---

indicating its suitability for applications with limited data availability. Performance Saturation: As training sample size increases, some methods experience performance plateaus, suggesting that beyond certain point, additional data yields diminishing returns.
In conclusion, the evaluation of performance across different training samples not only highlights the strengths of Reason-RFT in terms of data efficiency and generalization but also provides crit- ical insights into the performance dynamies of various methods. These findings are essential for practitioners aiming to maximize performance while effectively managing training resources.
D More Details on CoT Data Construction
This section expands the pipeline of CoT generation by detailing both the automated and manual components used to construct our CoT-SFT corpus.
(1) Automated Generation. We instantiate CoT drafts using reasoning-guided prompt templates such as "Let's break down the problem step by step... and "To answer this, we need to consider... ". Templates are combined with model prompting (GPT-4o [95] and Gemini-Pre (96) under temperature- controlled sampling (7'=0.7, top-k=50, top-p=0.9). To increase coverage and depth, we insert hand-crafted, subtask-specific few-shot exemplars that bias toward explicit intermediate justifications and erro-cheeking behavior.
(2) Automated Filtering. Each generated CoT is screened by two criteria:
Lenghh range. For each subtask 8, we compute a target trajectory length Lg from a balanced mixture of 50% human-written and 50% model-generated samples:
L 2 (T. [hummm + I.g T.model)).
where Īhuuan and model are computed as sample means over their respective sets after basic de- duplication. A candidate with length L; retained iff
0.6 < L. S 1.4
where L; is measured in tokens by training-time tokenizer; the factors 0.6 and 1.4 were selected via a small pilot study to trim outliers while preserving diversity.
Inconsistency. We discard the responses that contradict the known ground truth, including self- inconsistent counts, incompatible algebraic steps, or reasoning that invalidates later conclusions.
For reference, the empirical trajectory-Iengee length statistics (mean μ4, stdev σ)) across tasks are:
Task
ji (tokens)
(tokens)
Visual Counting
30
Structural Perception
180
80
Spatial Transtormation
400
120
where y and a are computed over the curated pool after automated filtering and before manual review.
(3) Human Verification. We randomly sample 10% of CoT drafts from eachtask for manual review, focusing on (i) step-to-step coherence, (ii) logical validity, and (iii) alignment between the reasoning chain and the final answer. Typical failure modes include: (i) correct final answer supported by an incorrect chain (c.g., deriving triangle area via the Pythagorean theorem); (ii) internal contradictions, such stating "There are 3 red blocks the left and 2 on the right" and later concluding the total is A follow-up quality audit found that, prior to human verification, approximately 3.8% of samples contained critical logical flaws; after verification, the residual error rate fell below 1 1%, indicating high post-cleanup reliability.
Discussion. Automated generation with calibrated sampling and subtask-specift few-shots provides diverse yet structured CoTs; the length- and consistency-batt filters remove overly terse/verbose or self-contradictory drafts; targeted human verification further suppresses high-severity errors. Together, these stages yield CoT-SFT dataset with improved coherence and faithfulness, while maintaining scalability and reproducibility.
29

---

E Detail on Mixed CoT Datasets
As shown in Tab. we presents a comprchensive overview of the datasets utilized for all of our visual reasoning experiments, categorized into three experimental groups. All of them are CoT-annotated by GPT-4o [95] The Main Experiment section includes three large-scale datasets: (35,000 samples) for quantitative analysis, Structure- Serreptre-Perceptiion (4,500 samples) for structural understanding, and Opatial-Transformmnn capabilities. For Ablation Studies, two mixed-domain subsets were constructed: (1) The Mixed General-Domaii set comprises 11 CoT-annotated datasets spanning scientific reasoning (A12D [59], ScienceQA (TOT), topological graph problems (GVL-QA series (102)), ) and pattern recognition (Puzzle VQA [103), IconQA [104], Raven (105)). (2) The Mixed Specific-Domall set focuses exclusively on geometric reasoning, featuring GeoQA [106), Geom Verse [53]. and Geometry3K (85)
(60,000 samples) assessing spatial reasoning
with progressively complex problem structures, All datasets were standardized to ensure training
compatibility.
Table 8: Datasets Overview for Visual Reasoning Tasks
Dataset Name
Samples
Reasoning Type
Description
Main Experiment
Visual-Coune Counting
35,000
Visual Counting
Full dataset for visual counting task
Structure-Pereepaa
4,500
Structure Perception
Full dataset for structural perception tasks Full dataset for spatial transformation tasks
Tontill-ransformation
60,000
Spatial Transformation
Ablation Experiment (Mixed General- Domeri)
A12D 59
1,467
Scientific Reasoning
Scientific diagram interpretation
ScienceQA
2,112
Scientific Reasoning
Science question answering Graph connectivity problems Cyele detection in graphs Hamiltonian path problems General topology questions Graph matching tasks Visual puzzle solving
GVL.QA-comeettvvil
102
1,199
Topological Reasoning
GVy-QA-cycle
102
1,194
Topological Reasoning
GVI.QA-hamiton
102
1,158
Topological Reasoning
VUI.QA-topology
102
1,070
Topological Reasoning
GVLQAA-mtchhin
102
1,193
Topological Reasoning
PuzzleVQA 1033
1,618
Pattem/Puzzie
IconQA 1004
5,270
Pattern/Puzzle
Icon-based question answering Raven's 's Progressive Matrices
Raven [0S
982
Pattern/Puzzle
Ablation Experiment (Mixed Opecific-Domaing
GeoQA 106
1,500
Geometric Reasoning
Geometric problem solving
Geom Verse 53
2,841
Geometric Reasoning
Advanced geometry challenges Comprehensive geometry problems
Geometry3K 85
3,794
Geometric Reasoning
F
Comparison of CoT Quality Before and After RL
Setting. We compare the Qwen2VL-3B model trained with Reason-RFT (Stage 2, post-RL) against the same backbone trained with only CoT-SFT (Stage 1, pre-RL) on the Structure Perception task. Unless otherwise noted, statistics are computed over a random sample of n = 100 problem instances.
Qualitative findings. Despite the high textual similarity between the two variants, the RL. model exhibits stronger logical coherence across intermediate steps, with fewer broken or skipped chains of inference. For example, in Fig. [66 (case 2). the pre-RL model correctly infers formula but omits the subsequent multiplication by 2, an error that is notably less frequent after RL. In addition, the Post-RL model more often displays reflective behaviors (e.g.. "let me dou- ble check") that are rarely observed in pre-RL outputs as shown in the math example of Fig. [
Table 9: Summary of comparative metrics on Struc- ture Perception (n = 100). Positive values indicate post-RL improvements.
Metric
Change (Post-Pre)
Reasoning Step Count Prompting Words Logical Connectives
+2.7 steps
+14%
+23%
Answer Accuracy
+20.56%
Quantitative protocol. We assess three dimensions of Shain-of-thouge (CoT) quality: (i) Reasoning Step Count-the the number of explicitly delimited reasoning steps per sample, obtained via automatic counting with GPT-4o [95]: (ii) Lexical Usage of two categories of expressions: Prompting words
30

---

(e.g. "oh I see", "let me think step by step", "let me double check") and Logical connectives (e.g., "so", "therefore". "first", "but", "moreover">" and (iii) Answer Accuracy as reported for the accuracy rate of Structure Perception task.
Results and interpretation. Post-RL training increases the average CoT granularity (as reflected by the larger step count), the organizational scaffolding of reasoning (higher usage of prompting phrases and discourse connectives), and the task effectiveness (higher final-answer accuracy). Taken together, these observations indicate that reinforcement learning with Reason-RFT enhances both the coherence and utility of CoT: reduces fragile or truncated chains, encourages reflective self-checks, and translates these behaviors into substantial accuracy gains.
G
Visualization
In this section, we present additional visualization results on general visual reasoning and three specific task reasoning, see Fig. 15 Fig. 23 Reason-RFT demonstrates superior performance over CoT-SFT terms of logical consistency, reasoning quality, and correctness. CoT-SFT's flaws stem from incorrect assumptions and misinterpretations, highlighting the importance of accurate problem interpretation and reasoning in visual reasoning tasks.
H Limitations and Societal Impact
Limitations While Reason-RFT has demonstrated strong performance in visual reasoning tasks, there are still areas to address. Future work will explore its application across a range of computer vision models, scaling to larger architectures (c.g., 32B/72B), and integrating large-scale Mixture of Experts (MoE) models to evaluate generalization. We will also extend the framework to complex downstream scenarios, such embodied AI and autonomous driving, testing its effectiveness in real-world applications that require sophisticated visual reasoning and real-time decision -aaoonn--aan.....
Societal Impact The advancements of Reason-RFT in visual reasoning have important societal implications. By enhancing generalization and cross-domain transferability, this framework can improve AI applications in areas like medical imaging, autonomous driving, and assistive technolo- gies for the visually impaired. It also reduces overfitting and cognitive rigidity, leading to more reliable and interpretable AI systems that foster trust in human-Al collaboration. The reconstructed benchmark dataset allows for fair evaluation, promoting research in robust AI. However, ethical considerations, such biases in training data and responsible deployment, must be addressed to prevent misuse. Overall, Reason-RFT paves the way for adaptable and trustworthy AI, benefiting industries, researchers, and society.
31

---

System Prompt Spatial Transformation Task
"Your need to complete the spatial visual reasoning task according to the following rules.
600 Task Description:
Given the image of the initial state, the image of the final state, and the attributes of the initial abjects, you should determine transformaiion that can achieve change states.
attributes of the initial objects** are provided as list of tupless the following format
**(abject id, shape', 'size', 'color, 'matertal)**
Each tugle represents an object and its properties in the initial state.
transforation should be sequence functions with a length ranging from where each function is represented as Tuncfobject id.
value)**,
### Available functions and values:
** "change objoct value)** Changes object size relative to initial size. Possible values: I'smelll, mediium large""
"change colort object value)** Changes the object to new color ralative to its initial color Possible values: [yellow,, "gray", tyyan, 'bleel, forown,, 'yroen', 'rod, 'purple"]
** "change materiallohjeet id, value)** Changes the object Possuble values: glaass metaal' 'rubber]"
material relative to initial maturial.
change rojject id, value)** -- Changes the object new shape relative to its initial shape. Possible values: l'cubee, 'spberr,, bylindur"
** "change positionlotject id, value)** Moves object
position relative iniball location.
Possible values: [front' behind', lstt, light, 'roat left, 'ront, right,, 'behind, Jeft:, behind, nighr]"
'roant moving forward along object's
direction.
"ehiodd msans moving backward along the object's initial direction.
moving the left of the abject's initial orientation.
'nght meons moving to the night of the object's inibal odientation. left
'front Lef't moving diagonally toward
initial location.
'froat right means moving diagonally toward the front and night of the initial location.
"behind Jeft'
moving diagonally "behiod night means moving diagonally toward the behind and night of the initial location.
the behind and left of
location.
#4e Output F Format
You should first thinks about the ressnnigg process intemally and then provides the user with the answer The **ressonigg proces** and ** enclosed within specific lags:
#* Ressoning process Provide chain- thought, logical explanstion of problem. This should outine step- by step reasoning, exclosed within chinke...sthinke.
Finel answer (sequence of functions only)**; Enclosed within 'snnserr... s/enswers"
Now, it's your tum!
(Question) } Output the thinking process in <think> 5/hink> and finel answer in <answer> <janswer> lags.
Figure 11: The system prompt used in Spatial Transformation task.
32

---

System Prompt for Spatial Transformation Task (Add <summary> <caption> in FORMAT)
"Your need to complete the spatial visual reasoning task accorting to following cules.
#NP Task Description:
Given image of initial state, the image the final state, the attributes of initial objects, you should determine transformation that can achieve the change states.
The attribuues of the initiaal objects** provided as a list of tuples the following format:
""oobeeet id' 'shape', Viiee, kollrr, "material Each tuple represents object and its properties in the initial state.
The transformation should sequence functions with length ranging from where each function represented ** Funclobject id.
value) **
#*B Available functions and values:
1. ** channge sizetobject id. value)** Changes the objjec to new size calative to its initial size. Possible values: I'small', medium, large")
** "chhagge colorfobject value)=s Changes object color Possthle values: Tyellow: gray', 'tyan', blue', 'browa,, !green', red'; purple")
initial color.
3. "caange, matertal/ id, value) Changes the object new material relative to its initial matertal. Possible values: T'glass', 'metall, 'rabber""
** "chhange shapef object id, value) Changes object new shape relative initial shape. Possi ble values: l'eube', sphere, tylinder']"
5. **change positionfaljest id, value)** Moves the objectt to new position relative its ini tial location. Possible values: ['front', babind', light,, 'front left, 'roont right', beeiind left, 'behinn, right")
'rrnt" means moving forwart along the object's initial direction.
'behind' means moving backward along object's initial dimctiion.
'lett' moens moving to the left of the object's initial orientation.
"ight means moving the right the object's initial orientation. froat, left' means moving diagonally toward the front and Left of the initial location. initial location.
ffront right meons moving diagonally toward front right
bebund, left meens moving diagonally toward the bekind and Left of the initial location.
"behind, right means moving diagonally toward belindd and right of the initial location.
Output Format
You should first thinks about the ressoning process Interallyy and then provides user with the answee. The "*ressoninng process** *answer** enclosed within specific bags.
** Summary process Summary how will approach the problem and explain the steps you will take mach the answer, enclosed within -summary>... setraaa.......ummaea <isummary>
Caption procees**, Provide detailed description of the image, particalariy imphosizing the aspects related the question, enclosed within scaption Vcaptionn>
** Reasoning pmoces"; Provide chain thought,, logical explanation of the problem. This should outline step-by step rasoning, enclosed within stinnk...sttinkk
Final answer (sequence of functions only)"". Enclosed within kanswer>... s/mswwers
Now, It's your tun!
(Question) Output the summary process in <summary> </summary>, caption process in <naptions stoppoos thinking process in <hink <think> and final <answer> <lunswer> tags.
Figure 12: The system prompt used in Spatial Transformation task w/ <summary> and <caption> tags format.
33

---

Visual-Counting
Structure- Percepptoo
Spatial- Transformation
In- In-Domain
Out-of Dut-af-Domainn
1900
11:8
4110
SFT
COT SFT
Reasen RFT ZERO
Roasen RFT
Xero-Shet
Figure 13: Results of all methods on Qwwn2VL-2B-Insttre ID and DS performance at different training checkpoints.
Seaual-Counting
Structure- Sructure-Pereetion
Spatial- Transformation
It-Domaiin
Out-af-Domaan
Reason ZERO
Zeroo
Activaionn
Figure 14: Results of all methods on Qwen2VL-7BB -nntrrruetttttt ID and DS performance at different training checkpoints.
34

---

Visual Counting Clevr-Math (ID)
Methods
Steps
adversarial
sub-multi
addition
subtraction
AVG
Zero-Shot
93.60
84.00
55.60
96.40
82.40
100
83.60
56.40
91.20
81.60
78.20
200
69.20
67.60
91.60
82.00
77.60
400
81.60
65.60
90.80
84.80
80.70
600
72.40
73.20
92.40
89.20
81.80
ANS-SFT
800
78.40
77.20
82.80
90.40
82.20
1200
85.60
78.00
91.60
95.60
87.70
1600
92.80
82.40
94.80
96.80
91.70
2187
95.20
92.80
97.60
99.20
96.20
100
49.20
40.00
82.00
69.20
60.10
200
65.20
55.60
88.00
76.40
71.30
400
66.00
57.20
90.00
79.60
73.20
600
67.20
59.20
87.20
82.80
74.10
CoT-SFT
800
77.60
61.60
92.40
85.20
79.20
1200
76.80
70.00
91.20
93.60
82.90
1600
80.80
66.80
91.60
92.00
82.80
2187
83.20
71.20
93.20
94.40
85.50
100
92.80
88.80
94.40
96.00
93.00
200
95.60
91.60
95.60
97.60
95.10
400
92.00
87.60
84.00
96.40
90.00
600
94.40
92.80
93.60
96.00
94.20
Reason-RFT--Z
800
96.40
96.40
96.00
98.80
96.90
1200
98.40
95.60
100.00
99.60
98.40
1600
96.40
94.80
98.80
99.60
97.40
2500
98.40
95.60
99.60
100.00
98.40
100
89.60
73.20
93.60
95.60
88,00
200
89.20
78.00
95.20
96.40
89.70
400
92.80
82.40
95.20
97.60
92.00
600
94.80
86.00
96.80
97.20
93.70
Reason-RFT
800
96.80
88.40
96.80
98.80
95.20
1200
94.80
86.00
96.40
98.80
94.00
1600
94.40
91.60
97.20
99.60
95.70
2500
98.40
92.80
96.80
99.20
96.80
Table 10: Complete experimental results of -wwn2VL-2B--nnsteeee on the Clevr-Math test set after training on Cltvr-Math. "sub-multi" donates the subtraction cobbraction-mulihhopp task.
35

---

Visual Counting Super-Clevr-Math (DS)
Methods
Steps
addition
subtraction
add-sub
sub-multi
AVG
DS-D
DS-M
Zero-Shot
10.40
54.40
0.00
63.20
32.00
42.67
0.00
100
51.20
37.60
11.60
39.20
34.90
42.67
11.60
200
38.40
55.60
8.40
15.60
29.50
36.53
8.40
400
40.80
45.20
5.60
35.20
31.70
40.40
5.60
600
41.20
61.60
8.00
35.60
36.60
46.13
8.00
ANS-SFT
800
49.20
50.40
7.20
26.00
33.20
41.87
7.20
1200
44.00
53,20
38.80
35.40
45.33
5.60
1600
48.80
53.60
6.00
26.00
33.60
42.80
6.00
2187
49.60
62.00
5.20
41.60
39.60
51.07
5.20
47.20
50.00
28.80
25.60
37.90
40.93
28.80
200
56.00
52.40
38.00
34.00
45.10
47.47
38.00
400
55.20
57.20
22.40
30.40
41.30
47.60
22,40
600
58.40
55.20
24.00
35.60
43.30
49.73
24.00
COT-SFT
800
$7.60
47.60
26.80
41.60
43.40
48.93
26.80
1200
58.00
54.40
35.60
32.40
45.10
48
35.60
1600
53.20
58.40
33.20
40.40
46.30
50.67
33.20
2187
53.60
58.80
36.80
36.80
46,50
49.73
36,80
100
46.00
65.20
6.80
58,80
44.20
56.67
6.80
200
48.80
66.00
9.20
57.60
45.40
57.47
400
42.00
71.20
8.40
50,80
43.10
54.67
8.40
600
47.20
65.20
7.60
47.60
41.90
53.33
7.60
Reason-RFTe -FF-on-RFT-Zer
800
56.40
69.20
6.80
55.20
46.90
60.27
6.80
1200
$2.00
73.60
7.20
$9.20
48.00
61.60
7.20
1600
$1.60
71.60
54.80
46.10
59.33
2500
49.60
71.20
5.20
53.20
44.80
58.00
5.20
100
59.20
57.60
38.00
41,60
49.10
52.80
38.00
200
59.60
64.40
39.20
42.00
51.30
55.33
39.20
400
61.60
64.00
39.20
37.20
50.50
54.27
39.20
600
66.80
67.20
32.00
46.00
53.00
60.00
32,00
Reason-RFT
800
66.00
65.60
34.00
39.20
51.20
56.93
34.00
1200
67.20
65.20
33.60
40.80
51.70
57.73
33.60
1600
63.60
66.00
33.20
44.80
51.90
58.13
33.20
2500
68.00
67.20
28.40
44.80
52.10
60.00
28.40
Table 11: Complete experimental results of Qwen2VL-225 Owwen2VL-2B-Instruuc the Super-Clevr-Maut test set after training on Clevr-Math, "add-sub" donates the donates the inbtraction-muliiin model has previously seen during Clevr-Math training, while "Mixed Arithmetic" Arithmetic"(DS-M (DS-MM) denotes the complicated types that the model has not encountered (i.e. questions with multi-step mixture of addition and subtraction).
task, while "sub-multi" task. "Direct Arithmetic" (DDtcc"""DDD) refers to the types of questions the
36

---

Visual Counting Clevr-Math (ID)
Methods
Steps
adversarial
sub-multi
addition
subtraction
AVG
Zero-Shot
99.60
98.40
97.60
98.80
98.60
100
69.20
54.00
81.20
69.20
68.40
200
61.20
50.00
82.40
75.60
67.30
400
69.20
63.60
89.20
77.60
74.90
600
70.40
54.00
90.40
81.20
74.00
ANS-SFT
800
80.00
74.00
91.20
89.20
83.60
1200
86.80
79.20
94.40
91.20
87.90
1600
90.40
84.40
95.20
92.00
90.50
2187
96.80
89.20
96.80
97.20
95,00
100
81.60
63.60
91.20
83.60
80.00
200
80.00
64.00
92.00
88.80
81.20
400
72.40
66.00
88.80
79.60
76.70
600
77.60
66.00
94.40
89.20
81.80
CoT-SFT
800
78.40
65.20
94.00
87.20
81.20
1200
79.60
76.80
92.40
88.00
84.20
1600
86.40
78.00
92.80
93.20
87.60
2187
87.20
78.80
93.60
89.60
87.30
100
98.00
94.40
98.80
99.60
97.70
200
99.60
93.20
99.20
100.00
98.00
400
99.60
95.20
99.60
98.80
98.30
600
98.00
98.40
100.00
99.60
99.00
Reason-RFT--Z
800
99.60
98.40
99.60
98.80
99.10
1200
100.00
98.00
99.60
99.20
99.20
1600
99.60
97.60
100.00
99.20
99.10
2500
99.60
98.40
100.00
99.60
99.40
100
88.80
79.20
95.60
94.40
89.50
200
92.00
80.00
96.40
95.20
90.90
400
94.40
84.40
96.00
95.60
92.60
600
92.80
84.00
96.40
97.60
92.70
Reason-RFT
800
92.80
85.20
96.80
96.40
92.80
1200
94.80
89.60
97.20
97.60
94.80
1600
94.80
86.40
97.60
97.20
94.00
2500
96.80
88.40
99.20
98.00
95.60
Table 12: Complete experimental results of Wwen2V--77- Instruct on the Clevr-Math test set after training on Clevr-Math. "sub-multi" donates the subtraction bubrraction-mullihopp task.
37

---

Visual Counting Super-Clevr-Mathh (DS)
Methods
Steps
addition
subtraction
add-sub
sub-multi
AVG
DS-D
DS-M
Zero-Shot
46.80
75.20
4.80
41.60
42.10
54.53
4.80
100
57.60
41.20
5.60
46.40
37.70
48. 48.40
5.60
200
42.00
38.80
8.00
33.60
30.60
38.13
8.00
400
37.20
46.40
5.20
31.60
30.10
38.40
5.20
600
32.00
44.80
12.40
19,20
27.10
32.00
12.40
ANS-SFT
800
38.80
38.00
6.80
37.20
30.20
38.00
6.80
1200
42.00
42.80
12.80
32.00
32.40
38.93
1600
36.40
48.40
11.20
17.20
28.30
34.00
11.20
2187
39.60
58.80
8.00
29.20
33.90
42.53
8.00
60.00
63.60
41.60
52.30
55.07
44.00
200
67.60
66.40
48.00
46.80
57.20
60.27
48.00
400
55.20
60.40
19.60
42.00
44,30
52.53
19.60
600
64.80
61.20
35.20
43.20
51.10
56.40
35.20
COT-SFT
800
60.00
53.60
37.60
42.40
48.40
52.00
37.60
1200
51.20
56.00
35.20
39.60
45.50
48
35.20
1600
53.20
56.40
34.40
35,20
44.80
48.27
34.40
2187
51.60
51.60
33.60
32.80
42,40
45.33
33.60
100
$8.80
82.80
24.00
62.40
57.00
68.00
24.00
200
56.00
83.20
50.00
52.00
63.07
400
62.40
79.60
22.80
37.60
50.60
59.87
22.80
600
61.20
85.20
17.20
49.20
53.20
65.20
17.20
Reason-RFTe -FF-zn-
800
52.80
86.80
20.40
52.00
53.00
63.87
20.40
1200
53.60
83.20
19.20
46.80
50.70
61.20
19.20
1600
61.20
84.80
18.40
43.20
$1.90
63.07
18.40
2500
59.20
86.40
21.20
45.20
53.00
63.60
21.20
100
$3.60
$6.80
33.20
39.60
45.80
50.00
33.20
200
$2.00
61.20
31.60
44.00
47.20
52.40
31.60
400
56.00
59.60
30.80
45.20
47.90
53.60
30,80
600
56.00
64.00
31.60
50.00
50.40
56.67
31.60
Reason-RFT
800
56.00
60.00
28.00
41.60
46.40
52.53
28.00
1200
66.00
65.60
38.00
50.40
55.00
60.67
38.00
1600
64.40
59.60
32.40
48.80
51.30
57.60
32.40
2500
62.80
60.80
35.60
44.80
51.00
56.13
35.60
Table 13: Complete experimental results of QWen2VL-7B-Inssee on the Super-Clevr test set after task, while "sub-multi" donates task. "Direct Arithmetic" (eaanettic((SDD) refers to the types of questions the model
training on Clevr-Math. "add-sub" donates the the cubtraction-multinn has previously seen during Clevr-Math training. while "Mixed Arithmetic" denotes the complicated types that the model has not encountered (i.e. questions with multi-step mixture of addition and subtraction).
38

---

Structure Perception
Methods
Steps
Geometry3k (DS)
GeoMath (ID)
CHOICE
NON-CHOICE
AVG
CHOICE
NON-CHOICE
AVG
Zero-Shot
40.25
1.00
20.63
35.57
20.31
25.86
100
35.25
16.25
25.75
58.72
29.89
40.37
200
33.25
17.50
25.38
56.38
35.44
43.05
400
30.75
17.00
23.88
64.77
35.06
45.86
600
73.83
38.12
51.10
ANS-SFT
800
32.75
16.00
24.38
72.15
36.40
49.39
1200
73.83
35.44
49.39
1600
29.00
16.00
22.50
74.83
37.36
50.98
1686
28.75
16.25
22.50
74.83
37.93
51.34
100
16.50
21.50
19.00
31.54
34.10
33.17
200
7.50
23.50
15.50
32.89
35.25
34.39
400
21.50
21.25
21.38
41.61
40.04
40.61
600
43.62
36.59
39.14
CoT-SFT
800
16.50
23.50
20.00
45.97
39.27
41.70
1200
53,02
40.04
44.76
24.00
24.13
$3.69
37.93
43.66
26.75
23.75
25.25
51.34
38.31
43.05
100
32.25
17.75
25.00
41.61
31.23
35.00
200
33,00
18.50
25.75
48.99
35.06
40.12
400
41.50
23.50
32.50
52.68
34.87
41.34
600
37.00
22.75
29.88
60.74
37.55
45.98
Reason-RIT-Zeen
800
42.25
25.00
33.63
62.42
40.42
48.42
1200
43.00
23.75
33.38
61.07
39.66
47.44
1600
42.75
22.25
32.50
63.09
38.31
47.32
1610
43.25
21.75
32.50
63.09
38.89
47.68
100
37.50
23,25
30.38
50.34
41.00
44.39
200
33.50
29.25
31.38
$6.71
40.04
46.10
400
38.25
28.75
33,50
56.38
39.27
45.49
600
40.50
27.25
33.88
61.41
41.19
48.54
Reason-RFT
800
41.25
29.50
35.38
58.05
41.19
47.32
1200
40.25
31.00
35.63
61.74
42.34
49.39
1600
38.00
29.25
33.63
62.08
43.10
50.00
1610
36.75
29.50
33.13
60.74
42.34
49.03
Table Complete experimental results of QWen2VL-2BB-nstrun on the Structure Perception task after training on GeoMath.
39

---

Structure Perception
Methods
Steps
Geometry3k (DS)
GeoMath (ID)
CHOICE
NON-CHOICE
AVG
CHOICE
NON-CHOICE
AVG
Zero-Shot
45.25
23.00
34.13
61.07
38.12
46.46
100
38.50
18.25
28.38
64.77
34.87
45.74
200
32.50
22.75
27.63
69.46
35.25
47.68
400
72.48
40.42
52.07
600
32.25
18.00
25.13
73.49
39.27
51.71
ANS-SFT
800
75,50
37.93
51.58
1200
32.50
18.50
25.50
75.84
37.74
51.59
1600
32.50
18.25
25.38
75.84
37.36
51.34
1686
18.25
38.75
28.50
38.59
42.72
41.22
100
6.50
32.00
19.25
38.26
43.10
41.34
200
27.00
34.50
30.75
56.71
44.64
49.03
400
52.68
44.06
47.19
600
35,50
36.25
35.88
63.09
43.49
50.61
CoT-SFT
800
63.42
42.91
50.36
1200
29.50
37.50
33.50
64.09
44.06
51.34
29.25
36.75
33.00
61.74
44.06
50.49
58.50
41.75
50.13
56.71
45.98
49.88
100
59.00
44.25
$1.63
63.42
45.21
51.83
200
62.00
43.00
$2.50
70.47
45.40
54.51
400
70.13
46.74
55.24
600
64.75
45.25
$5.00
70.47
56.95
800
66.11
46.17
53.42
1200
69.00
43.25
56.13
71.14
45.59
54.88
1600
66.25
43.25
54.75
69.80
46.55
55.00
1610
46.75
37.50
42.13
67.79
45.79
53.79
100
53.00
37.00
45.00
72.82
46.93
56.34
200
52.75
37.25
45.00
71.14
46.55
55.49
400
51.50
37.00
44.25
73.49
48.28
57.44
600
56.75
37.25
47.00
77.52
46.17
57.56
Reason-RFT
800
59.00
40.00
49.50
79.87
48.08
59.63
1200
56,00
39.50
47.75
74.50
49.62
58.66
1600
59.00
39.50
49.25
78.52
48.28
59.27
1610
59.00
39.50
49.25
78.52
48.28
59.27
Table 15: Complete experimental results of Qwen2VL--BB-Instreee on the Structure Perception task after training on GeoMath.
40

---

AVG
29.76
4.67
10.87
25.45
31.72
37.64
47.21
49,94
50.33
11.65
14.78
17.64
33.10
37.57
43.58
42.86
18.25
20.00
20.69
25.66
30.25
32.41
32.38
43.96
49.21
53.46
57.17
61.79
62.88
64.08
31,77
5.95
10.07
31.20
31.73
42.55
52.50
53.98
53.60
13.41
16.32
15.97
36.23
39.26
51.35
49.33
25.70
28.50
25.60
27.88
33,90
40.27
40.34
49.85
54,16
57.22
59,91
63.52
65.66 65.66
66.80 3080
TRANCE-R (DS-R)
Table 16: Complete experimental results of QwwnQwwen...... ---2VL-BB-BBA 2B Instruct on the Spatial Transformation lask after training on TRANCE.
Level-3
31.13
5.59
10.17
30.53
35.46
42.95
51.23
54.47
54.33
12.84
13.77
17.91
37.60
40.78
45.92
47.15
21.09
22.82
25
31.50
33.87
37.27
37.44
48.61
53.68
59.16
61.40
64.98
67.60
68.32
Level-2
31.74
4.73
12.17
26.54
35.62
38.98
49.28
52.78
53.75
13.38
17.71
20.55
35.81
41.23
45.87
44.77
18.12
18.97
21.19
27.75
32.85
34.01
33.45
46.08
$2.27
56.35
60.60
65.17
65.60
66.25
Level-1
24.38
2.39
11.08
13.53
24.06
26.09
35.84
38.51
39.63
6.97
11.23
16.13
22.74
29.00
31.19
30.20
8.08
9.72
10.54
15.52
20.39
18.08
18.28
31.29
36.74
41.10
46.76
53.47
52.67
54.95
AVG
28.67
4.60
10.25
25.96
31.41
38.17
47.58
49.58
49.24
12.64
15.11
17.65
33.50
38.43
44.20
43.19
18.04
19.94
20.90
25.71
30.83
32.58
32.85
44,97
68.63
53.72
57.62
61.54
62.68
64,05
Level-4
30.22
6.08
9.07
32.58
29.38
42.65
53.10
54.27
53.55
14,69
15.82
15.96
35.29
40.13
52.10
50.30
25.05
27.00
25.30
27.38
34,49
39.69
40.57
50.39
$4,67
57.94
58.45
64.71
66.01
67.34
Spatial Transformation
TRANCE-L (DS-L)
Level-3
30.73
5.57
10.01
31.26
34,26
43.05
53.83
55.23
54.59
11.82
14.22
17.73
37.54
40.99
50.06
49.16
21.16
23.19
25.73
30,93
35.84
37.85
37.94
50.62
54,85
61.27
63.00
66.56
68.51
69.64
Level-2
30.56
4.73
11.33
26.76
35 35.98
42.27
49.00
52.61
52.82
13.74
17.51
21.19
38.29
43.95
45.97
44.41
17.42
20.08
21.47
29.12
33.03
34.11
34.57
0..02
49.28 29.28
55.94
61.64
62.22
63.60
65.72
Level-1
23.16
2.02
10.60
13.25
26.03
24.70
34.38
36.22
36.00
10.32
12.90
15.73
22.88
28.65
28.67
28.87
8.53
9.49
11.10
15 15.40
19.96
18.67
18.33
31.84
35.72
39.74
47.40
52.68
52.58
53.49
AVG
42.55
3.78
15.70
30.37
40.48
54.80
72.33
78.19
77.39
16.39
22.17
41.66
53.53
64.54
64.37
18.33
20.01
21.77
27.79
37.32
43.80
44.47
55.31
59.47
65.99
65.62
73.28
73.99
74.61
Level-4
39.08
5.68
13.12
33.95
33.65
47.55
71.60
70.22
20.36
16.46
16.02
37.84
45.85
62.28
59.70
26.22
28.73
26.25
27.85
34.51
40.50
42.11
53.35
59.46
64.70
63.15
69.16
72.45
73.12
64
TRANCE (ID)
Level-3
40.87
4.08
14.67
33.98
40.75
53.01
68.34
78.00
76.70
15.99
14.53
19.94
41.77
54.38
65.32
62.78
20.69
22.97
27.01
32.77
37.78
44.07
44.53
58.91
63.67
68.47
66.88
74.05
75.77
76.68
Level-2
42. 42.96
3.27
19.12
29.56
42.58
56.55
75.68
80.30
79.93
14,90
19.12
26.74
43.19
61.06
69.15
67.52
17.96
18.76
21.47
32.08
40.20
46.89
45.01
55.47
59.77
66.97
68.13
73.42
74.85
74.52
Level-1
47.28
2.10
15.90
23.97
44.95
62.10
80.70
82.85
82.70
6.99
15.45
25.98
43.85
52.82
61.40
67.47
8.44
9.59
12.35
18.47
36.78
43,72
46.21
53.52
54.97
63.80
64.33
76.47
72.88
74.10
Steps
100
200
400
80D
1600
3200
fimal
100
200
400
800
1600
3200
final
100
200
400
800
1600
3200
final
100
200
400
800
1600
3200
final
Reason-RRR -RF--ero
Reason- RFT
Method
GPT-40 Scot-Shot
ANS-SFT
COT-SPT
41

---

AVG
29.76
12,78
27.58
41.90
42.32
47.03
50.93
54.62
54.83
26.93
32.59
40.34
44.34
46.22
48.54
47.80
24.98
27.19
32.90
41.96
49,62
54,93
56.15
50,14
$1.90
$4.59
53.79
55.77
56.58
58.61
Level-4
31.77
11.18
26.48
43.02
45.25
52.88
55.92
60.98
60.38
30.36
37.54
45.90
52.83
58.31
61.02
61.48
29. 88
30.81
32.79
41.57
49.08
60.21
58.85
55.42
60.79
64.88
64.88
66.67
68.89
69.69
TRANCE-R (DS-R)
Table 17: Complete experimental results of QwenQWwee.... Dw-nn2V-7B-BBA 7B BEWVL---B-nnniii Instruct on the Spatial Transformation lask after training on TRANCE.
Level-3
31.13
10.55
27.74
45.97
47.04
50.62
56.45
61.67
61.44
30.43
37.06
44.62
49.60
52.24
54.83
54.69
27.54
30.57
37.63
51.64
60.02
61.81
57.30
58.25
60.82
60.82
66.02
63.90
63.33
46
Level-2
31.74
16.08
29.71
43.43
44.88
50.40
53.57
55,06
54.84
28.43
34.55
38.96
45.02
45.25
47.85
45.32
27.29
29.27
39.47
45,89
56.46
56.93
58.40
52.56
54.21
55.72
54.39
55.39
56.34
56.34
Level-1
24.38
13.30
26.38
35.17
32.
34.22
37.77
40.78
42.64
18.49
21.19
31.87
29.91
29.08
30.46
29.69
15.21
18.10
21.72
33.90
41.28
42.56
45.53
35.26
34.33
36.95
35.06
35.01
37.18
45.45
AVG
28.67
12.72
29.00
42.39
42.61
47.41
51.45
54,26
54.29
27.59
34,32
42.74
44.77
46.30
49.15
47.90
25.27
28.24
32.09
41.70
50.64
54.92
57.20
$1.21
52.61
54.34
55.90
55.96
56.66
59.36
Level-4
30.22
10.85
45,52
46.23
52.95
56.05
59.90
60.05
29.33
38.86
46.10
52.88
58.94
62.54
61.58
30.19
31.20
33.44
39.55
50.60
60.10
62.26
54,99
60.05
63.01
63.69
66.76
68.02
68.05
26.8
Spatial Transformation
TRANCE-L (DS-L)
Level-3
30.73
11.50
26.00
45.61
47.69
51.99
57.93
61.02
61.84
29.18
36.81
47.73
50.01
51.01
56.49
54,42
28.
31.15
35.43
45.65
52.96
58.97
61.69
60.21
60.20
62.25
64.10
66.38
65.24
64.66
Level-2
30.56
16.80
31.12
45.14
44.13
50.62
52.97
56.02
55.58
29.94
36.28
44.61
46.01
46.43
47.63
47.54
30.22
39.28
46.15
55.03
58.05
58.22
52.54
53.94
56.00
56.86
53.44
55.73
61.04
26.
Level-1
23.16
11.71
32.07
33.29
32.40
34.08
38.85
39.67
21.89
25.31
32.53
30.19
28.82
29.93
28.07
15.88
20.39
20.20
35 35.44
43.95
42.54
46.61
37.09
36.26
36.11
38.93
37.25
37.63
43.68
40.
AVG
42.55
13.53
34.09
53.27
58.16
68.91
73.67
82.01
82.19
26.38
43.52
51.83
59.74
71.29
80.79
81.31
29.93
36.52
35.78
49.56
58.54
66.51
67.67
63.58
67.55
72.20
73.11
76.43
78.92
79.97
Level-4
39.08
10.48
28.35
45.15
50.77
62.10
66.25
78.17
78.45
30.00
42.16
52.42
59.93
69.93
78.42
78.77
31.27
34.77
35.35
46.06
55.36
66.22
67.30
62.78
66.00
69.14
73.26
73.26
76.61
78.26
TRANCE (ID)
Level-3
40.87
10.96
30.67
49.4 49.43
57.17
63.97
73.80
82.83
82.50
25.97
46.02
58.20
62.20
72.82
79.99
80.54
33.22
36.80
41.70
50.43
57.30
68.79
69.45
68.53
69.53
73.62
75.58
76 76.59
80.13
81.97
Level-2
42.96
16.42
37.05
53.33
59.35
70.67
76.12
83.23
84.10
28.98
44.08
51.32
61.97
71.14
80.62
79.43
31.62
39.45
40.78
51.55
61.90
68.53
68.30
62.95
66.98
71.31
71.04
76.70
79.22
79.15
Level-1
47.28
16.25
40 30
65. 18
65.33
78.90
78.50
83,80
83.70
20.58
41.80
45.39
54.87
71.27
84.13
86.50
23.59
35.06
25.28
50.18
59.60
62.50
65.63
60.07
67.69
74,72
72.57
79.18
79.70
80.48
Steps
100
200
400
80D
1600
3200
fimal
100
200
400
800
1600
3200
final
100
200
400
800
1600
3200
final
100
200
400
800
1600
3200
final
Reason- -RF--ero
Reason- RFT
Method
GPT-40 Scot-Shot
ANS-SFT
COT-SPT
42

---

How many times larger the total number of eggs in the second group compared to the first group?
Question:
Reason-RFT:
<think>First, let's count the number of egg trays in the first group. There are trays in the first group. Each tray contains 10 eggs, so the total number of eggs in the first group is trays 10 eggs/tray
COT-SFT:
thinkk-First, let's count the number of egg trays the first group. There are trays in the first group. Each tray holds 12 eggs, so the total number of eggs in the first group is 3 trays 12 eggs/tray 36 eggs. Next, let's count the number of egg trays in the second group. There are 10 trays in the second group. Each tray holds 12 eggs, so the total number of eggs the second group 10 trays* 12 eggs/tray 120 eggs. Now, let's find the ratio of the total number of eggs in the second group to the first group: 120 eggs/ 36 eggs 10. Therefore, the second group has 10 times more eggs than the first group. </think>>
eggs. Next, let's count the number of egg trays in the second group. There are 10 trays in the second group. Each tray contains 10 eggs, so the total number of eggs the second group is 10 trays 10 eggs/tray = 100 eggs. Now, let's compare the total number of eggs in the second group the first group. The second group has 100 eggs, and the first group has 20 eggs. To find how many times larger the second group is compared to the first group, divide the total number of eggs in the second group by the total number of eggs the first group: 100 eggs/ 20 eggs = 5, 5/<thiik>>
Ganswer>10<answe
<answar>5</answe>
How many more purple circles are there than green squares?
Question:
Reason-RFT:
Athink=First, let's count number of purple circles. I see 1, 2, 3, 4, 5 purple circles. Next, let's count the number of green squares. see 1, 2, 3, 4, 5 green squares. Now, find out how many more purple circles there are than green squares, we subtract the number of green squares from the number of purple circles: (purple circles) 5 (green squares) 0. Oh, see, there are no more purple circles than green squares, squares.<(thnk>>
COT-SFT:
<think>First, let's count the number of green green squares in the image. Next, let's count the number of purple circles. There are 5 purple circles in the image. To find the difference, subtract the number green squares from the number purple circles: (purple circles) 4 (green squares) 1. </think>
squares. There
</nwerr11<answer>
<answers0//anwerr>
Figure 15: Visualization examples of Reason-RFT and CoT-SFT on general visual reasoning.
43

---

Question: Basod on image, determine which number is directly opposite to number on the die depicted. Use reasoning to arive at the correct answer from the choices provided. Choices: A. 1
B.
C. D.
COT-SFT:
Reason-RFT
<thnk>>et think about the properties of standard six- sided die. On a six- six-sided die, each number from to 6 is opposite to another of the numbers opposite
<hink>>etss analyze the image. The numbers shown are 2, 3, and To find the number opposite we need to consider the properties a standard six-sided die. On die, opposite numbers add to Since is opposite to 3, we can conclude that the number opposite to is 3.</think>
number. The sides is always 7. Number 4 opposite to number 2 because 4 + 6, and the sum of and is 7. T Therefore, the number directly opposite to number is </thik>>
<answer>C </answe>>
<answer>B </answer>
What is the total surface area of this rectangular prism dimensions are units by units by 5 units?
Question:
COT-SFT:
Reason-RFT:
<hink>>oo find the total surface area of a rectangular prism, need to calculate the sum of the areas of all faces. The formula for the surface area is 2 (length* width width" height + lengthh. Let's calculate it step- step 1. Calculate the area of the face with dimensions units by units: 3*
<think>To find total surface area a rectangular prism, we need to calculate the sum of the areas of all six faces. The formula for the total surface area is 2 (length* wiinhhh"wiith width height height length). Let's calculate it step by step. First, calculate the area of the face with dimensions 3 units and 4 units: 3
square units. 2.
12 square units. Next, calculate the area of the face with dimensions 4 units and units:
Calculate the area of the face with dimensions 4
units by units: 4 = 20 square wnits. 3. Calculate the area of the face with dimensions units by units: 5 = 15 square units. Now, add these areas together: 12 + 20 15 = 47 square units.
square units. Then, calculate the area of the face with dimensions units and 3 units: 5 * 3 15 square units. Now, add these areas together: 12 47 square units.
15
Therefore, the total surface area of the reclangular prism is 47 square units, </think>
Finally, multiply by to account for both pairs of each face: 2 * 94 square </hhnk>
4nnwwrr47</answeer
<answer=04</answer
Figure 16: Visualization examples of Reason-RFT and CoT-SFT on general visual reasoning.
44

---

R
R
What the transformation has been applied to the letter on the left to obtain the letter on the right, and which option correctly describes this transformation? Choices: (A) slide (B) flip (C) tum
Question:
COT-SFT:
Reason-RFT:
<hhnk><et me think about transformation applied to the letter on the left. The letter on the left grid of squares, and the letter on the right is also grid of squares. The orientation and shape of the letter remain the same, but the position of the letter has changed. Oh, see, this is slide transformation because the letter has boen moved horizontally, </think<>
<think>Lett analyze the image. The letter on the left is original image, and the letter on right is transformed image. The transformation involves flipping the letter horizontally. This means the letter has been mirrored along vertical axis. Therefore, the correct option describing this transformation is flip </think>
<answer>B </answer>
<answer> </answer>
Kilometers walked
Day
Kilometers
Josiah kept track how many kilometers he walked during the past days. Can you determine which number of kilometers he walked most frequently?
Question:
Monday
4
Tuesday
Wadnesday
7
Thursday
4
Saturdey 3
Sunday
7
COT-SFT:
Reason-RFT:
<hinnk-First, list the kilometers Josiah walked day: 4, 3, 7, 4, 7, 3, Now, let's count how many times each number appears: 4 appears 3 times, 3 appears 2 times, and 7 appears times. Oh, see, the number that appears the most frequently is the mode. Since appears 3 times, which is more than any other number, the mode is </think>>
<think>First, let's list the kilometers Josiah walked each day: 4, 4, 3, 7. Now, let's count how many times each distance appears: 4 km appears times. km appears times. km appears 3 times. Oh, see, the highest frequency is 3, which corresponds to km, </think>
<answer>7 </answer>>>>>>>>>>>
<aanwe>><<answwr>>>>>>
Figure 17: Visualization examples of Reason-RFT and CoT-SFT on general visual reasoning.
45

---

How many main structural elements can be identified in the suspension hridge, and what are their roles in supporting the bridge?
Question:
Reason-RFT:
<hink<Let me think about the main structural elements visible in the image. First, notice the two large tow ers supporting the bridge. These towers are crucial for providing the necessary height and strength to maintain the bridge's structure, Next, see the main spen of the bridge, which is the horizontal part connecting the towers. This span is supported by the tow ers and is essential for carying the weight of the bridge and any traffic it carries. Finally, there are additional support structures visible, such as the curved steel cables, which help distribute the we weight evenly and maintain the bridge's stability. So, the main structural elements are the towers, the main span, and the curver udved steel cables. Counting these elements, I see two towers, one main span, and multiple curved steel cables. Therefore, the main structural elements are three: the towers, the main span, and the curved steel cables, </tinnk>
COT-SFT:
<think>Let me think about the main structural elements visible in the image. First, I see the two large white towers, which are the main supports for the suspension bridge. These towers are tall and robust, providing the necessary height and strength to maintain the tension in the cables. So, there are two main structural elements: the towers, Oh, see, the towers are crucial for supporting the weight of the bridge and ensuring its stability. Without them, the bridge would not be able to span the valley effectively. </think>
canswer>-<lanswer
<unswer>3<answwr>>
Question:
How many distinct colors of lights used the Christmas tree decoration?
COT-SFT:
<hink>>ee et think about the colors described in the image. The Christmas tree is decorated with lights that described as being red, blue, and white. need to count these distinct colors. First, identify the color red, which mentioned directly. Next. see the color blue, which also mentioned. Finally, I observe the color white, which described as shining brightly. So, there three distinct colors of lights used in the decoration: red, blue, and white, white.</thnnk>>
REFF
<think>Let et me think. The image shows a Christmas tree decorated with lights. The description mentions that the lights are of different colors. I need to identify these colors. The description specifically mentions red and white lights. So, there distinct colors of lights used the decoration. decoration,<think>>
<answer>3<//nsswee
<answer>2<ansswer
Figure 18: Visualization examples of Reason- RFT and CoT-SFT on general visual reasoning.
46

---

Question: Subtractt all spheres. How many objects are left?
COT-SFT:
Reason-RFIE RFT:
Sthink- Let's s start by identifying all objocts image. can see various shapes: spheres, cubes, and cylinders. First, let's count total number of objects the image:1. 1. Red sphere 2. Green sphere nother green sphere 4. Pupple cube Brown cube 6. Yellow cube 7. Brown cylinder Blue transparent cube. (Miss one object) Now, let's identify and count spheres: Red sphere Green sphere Aother green sphere. There spheres total. Next, we subtract number spheres from total number of objects: Total objects 8, Spheres 3. Objects left after subtracting spheres Total objects Spheres 88
<think>Lett' start by identifying the objects in the image. see total 9 objects. question asks us to subtract all spheres.
There are spheres the image: a red sphere, green sphere, green sphere. So, we need to subtraci these spheres from the total of objects. (total objects) 3 (spheres) Therefore, after subtracting spheres, there are 6 objects left. </think>
<anser>SG//nsswer>>>>>>
Oh, see, after removing all the spheres, left with objects.<<thinl </think>
<answerö6</nswwr>>>
Question Add 7 small cars. How many cars are left?
COT-SFT:
Reason-RFT:
<think>Let's start by identifying the number of cars in the image. see blue SUV, red airplane, purple dirt bike, and blue small car. That makes total of cars initially. Now, according to the question, we need to add small cars to the existing ones. So, let's calculate the total number of cars after adding these small cars. Initial number of cars 4. Number of small cars to add 7. Total number of cars Initial number of cars + Number of small cars to add. Total number of cars 4 7 = 11. Therefore, after adding small cars, there are 11 cars in total. </think
<think>Let's start by identifying the cars image. see two cars: blue SUV and blue minivan. question asks add small cars the existing cars. So, we begin cars add more.
2 (existing cars) + (additional cars) cars total. Therefore, after adding small cars, there are 9 cars in total. </tink>>
<answrr-@<answer>>>
<answer>-1 </answer>
Figure 19: Visualization examples of Reason-RFT and CoT-SFT Visual Counting task.
47

---

Question Given the hyperbola shown in the image, can we determine the equation of the hyperbola by analyzing the relationship between the slope of the asymptote and the distance from the center to the vertex? Choices: A. Yes No
COT-SFI:
Reason-RFT:
<thnnk>Let's think about the properties
<think>Let's analyze the image. The hyperbola shown with two asymptotes intersecting at the center. To find equation the hyperbola, we need considor the relationship between the slope the asymptote and the distance from the center to the vertex. The equation of hyperbola with its center at the origin and aligned with the
hyperbola. The slope the asymptnte a hyperbola given by the equation M. Vifrac (a]((b) W),
where a W) and M b ()) are the coefficients from
the equation of the hyperbola MV Wiaa (x 22 (a)
Wfrac (y (2) (b 22 = W. The distance from the center to the vertex is W. o W. which is related to the semi major and semi minor axes. However, to find the equation of the hyperbola,
coordinate axes is given by WM Wfrac (x "2) (a*2)-
Wfrac (2) 22 = 1 W), where W W) and W b 1) are the distances from the center to the vertices and asymptotes, respectively. The center to the vertices is given, which suggests that W a b2 W) is given. The slope of the asymptotes is given by W lpm (brra((baa) W. In this case, the slope is which implies that WI Wfrac (b) (a] W. Theerefore, the equation of the hyperbola can be determined by these relationships. </thinks>
need more information than just the slope of the asymptote and the distance from the center the vertex. We also need the values of W and WC W. which directly provided the image. Therefore, information given insufficient to determine the equation of the hyperbola. </think>>
<answer>B <lanswer> Canswr>BB/answer>>>>>>>
<answer> <lanswer>
Question: What is the circumference of the circle?
COT-SFI:
Reason-REIT
<hink>To find the circumference of thecircle, need to know the diameter thecirole. The diameter the longest straight line that can drawn across circle, passing through center: Once have diameter, circle,
<think>First, let's determine the radius of the circle.
The problem states that the radius is 7, The formula for the circumference of circle is C = where r is the radius. Substituting the given radius, r= 7, into the formula, we have =2 (7). Calculating this gives C = 14 Therefore, the circunference of the circle 14 </thin>>>
can use the formula for the circumference which C z where is the diameter. Oh, I see, the diameter of the circle is given as So, can calculate the 7. Using the value of и as
circunference as C approximately 3.14159, I can calculate the circumference as 7 # 3.14159 = 21.99633. Rounding this to one decimal place, the circumference is approximately 22, </think>
<answer>14 </answer>
<answer=2244nnsui
Figure 20: Visualization examples of Reason-RFT and CoT-SFT on Structure Perception task.
48

---

P
R
Question: What is the length side QP in the rhombus?
COT-SFT:
Reoson-RFT:
<think>Lets think about this step-by-step. The thombus LMPQ is type of quadrlateral where all sides equal in length. We are given that side LM is equal,
<think>Let's analyze the rhombus LMPQ. We know that all sides of rhombus are equal length. The side QP is one these equal sides. Since problem states that the side MP 10, we can confirm that this is indeed length of side QP in the rhombus LMPQ. </think>
8 units long. Since all sides of thombus the length of side QP must also be equal to the length of side LM. Therefore, QP M 8 units, </think>
cimswer>10<<answer
<answer&</answer>
Consider right triangle ACB, where angle is 90 degrees, angle A is 37 degrees, and the units. What the approximate length of BC, expressed as k in terms of the given angles and length? (sin37.0 approximate 0.6, cos37.0 . approximate 0.8, tan37.0 approximate 0.75) Choices: A: 2.4 B: 3.0
Question:
length of AC
Jyeoo
C: 3.2 D: 5.0
COT-SFT:
<think<Leer" think through this step-by-step. We know that angle A 37 degrees, and we have the tangent function, write the
which the ratio of the opposite side (A the adjacent side (). Given that tan(37") 0.75, gives AB 5.333. Now, in right triangle, square
equation as AB =AC/ tan(3?"). Substituting we need to the hy hopotene BC using the Pythagorean theorem, which states of the hypotenuse is equal to the sum the squares of the other tvo sides. So, BC Isqri(A "2+ C2).. Substituting the known values, BC = 1squt(5.33 '2+ 22. Calculating this gives BC = 7.34. Therefore, the approximate length of BC is 7,34, which is not an exact match with the options. However, the closest approximation from the choices given is 3.0, which is not correct based on our calculations. Let's re-evaluate the choices and the reasoning provided, </think>
given value, = / 0.75. Calculating
<answer>Basee on the reasoning and calculations, the closest approximation from the choices given is 3.0, which is not correct based on our calculations
Resson-FF::
<think>Let's start by finding the length of BC using the tangent of angle A. We know that tanl angle A) opposite / adjacent C. Given that angle A is 37 degrees, we can write: tan(37*) = =CC/4. We also know that tan(37") is approximately 0.75. So, BC 4 0.75 =3, Therefore, the length of BC approximately 3.0. </0.n//hink>>
</asserrB</answer
Figure 21: Visualization examples of Reason-RFT and CoT-SFT Structure Perception task.
49

---

first image shows the central of initiall state, the second image shows the view of final state, and the attributes of the initial objects are listed below:
Question:
(0, cube, medium, gray, rubber) (1, spbere, medium, cyan, metal)
(2, cube, small, gray, metal) (3, cube, small, brown, metal)
(4, cylinder, large, yellow, metal) (5, sphere, small, yellow, rubber)
What is transformation process can achieve the change
from initial final state?
Initial central-view Image
Final left-view image
COT-SFT:
<hinks>Lets analyze the
states of objects the images. We have attributes of the objects, and we need
to determine transfomattoons that lead the final state.
1. **Object ges; Initially, it's medium gray rubber cube. the final state, appeurs to be spteere, So, need change shape. have moved the right increased size.
Transformation: change shapo(0, sphere)
2. **Obeect Initially, It's medium cyan metal sphere. In the finel
TTransoomation:
change position(], right)", "change sizef large)"
3. #*Object 2**; Initially, it's small gray metal cabe. the final state, it appears unchanged.
4. Object 3** Initially, it's small brown metal cube. the final state, appears unchanged.
5. ** **Object 6**; Initially, it's large yellow metal cylinder. the final state, appears unchanged.
6. **Ojectt5** Initially, it's small yellow rubber sphero. In the final stale, appears unchanged.
Now, let's compile the transformations needed achieve final from the initial 'change, shape(0,, sphere) "change positioni right)
"change sizef large)
These transfornations match the original answer movided.s/hhink>> </think>
<enswerschange, shape(0), sphere), change position[1, right), change size(1, large)
Reason-RFT:
<thin>>Losk analyze the initial and final states to delermine the transformations needod,
**Obeect (**; Initially medium gray nubber cube. the final state, appears medium gray rubber sphere.
-rranfoomation: change Shape(0, sphere)
2. **bbecct 1 Initially a medium eyan metal spbere. In the final state, it appears to be a large cyan metal sphere.
Transformation: change sizell, large)"
3. **Ojject 2 Initially small gray metal cube. In the final state, appears to be small gray metal cube.
-rransfomation: None needed as it remains unchanged.
4. **Ojject 3**; Initially a small brown metal cube. In
state, it appears be large yellow metal cube. Transformatin:: change co yallow) and change size( large)
5. Objoct 4** Initially a large yellow motal cylindor, In the final state, it appears be a large yellow motal cylinder.
Transformation: None needed as it remains unchanged.
6. **bjjecl 5** Initially a small yellow rubber sphore.
state, it appears be small yallow rubber sphere.
Transfomation: None needed as it remains unchanged.
By analyzing the changes, we can determine the sequence of transformations needed to achieve the final state from the initial state. Therefore, the transformations
"hange colort3, yellow)"
"change, size(1, large)
"change size(3, large)
'change, shape(0,, sphere)"
These transformations match the original answer provided. </think>
sanswerschange color(3, yellow), change size(1, large), change, sizef3, large), change shape(0, spbero)-
Figure 22: Visualization examples Reason-RFT and CoT-SFT on Spatial Transformation task.
50

---

The first image shows the central view initial state, the second image shows the right view of final state, and the altributes of the initial objects are listed below;
Question:
sphere, small, red, metal)
sphere, large, brown,
(2, cube, large, blue, metail)
(3, sphere, small, brown, metal)
(4, cylinder, small, blue, glass)
What the transformation process that can achieve change from initial state final stato?
Initial central-view image
Final right-view image
COT-SFT:
<thnnk>Lets analyze the initial and final of objects the images. We have the initial attributas of the objects, and to determine the transforathons that occurred.
1. *"Object0** 0**; Initially a small red metal sphere. In the final image, it appears to have moved to the front Left. So, we apply change position(O, front, left)"
2. +*Object 1**; Initially a large brown metal sphere. In the final image, it seems to have moved behind the other objects. So,
apply change, position(1, behind)'.
3. **Object 2 Initially a large blue metal cube. the final image, it appears have changed shape sphere, So, we apply
cbange shape(2, sphere)
4. +*Object 3+x Initially a small brown metel sphere. In the final image. il appears to have changed color to yellow. So, apply
"changee color(3, yellow).
5. **bjecct 4**; Initially small glass cylinder: here doesn't seem to any change attributes or position in final image.
Now, let's compile the transformations in the order they were identified:
"change color(3, yellow)"
"cbange, position(O, front, left)
"change, shape(2, sphere)
"change posilion(], 1, behind)'
These transfomations match the original answer provided..phinne
<answer>change color(3, yellow), change position(O, front lett), change shape(2, sphere), change position(1, s/answwrr>
Reason-REF
<think>Let's ana the inilial and funal slates to deterning the trasformations nooded.
1. 1.*Objoct3 3 (sphere, small, brown, motal):**
the initial state, is small brown metal sphere.
In the final state, it appears to be a small yellow cylinder.
Therefore, the transformations needed are: Change its color from brown to yellow: "change colort3, yellow)"
Change
shape from sphere cylinder: change shapel3, cylinder)
2. **** *Object I (sphore, large, brown, motal):**
initial state, is large brown metal sphere.
final
moved the right.
Therefore, the transformation needed Is: Change its position the right: "change position(1, right)"
3. Objectt 2 (cube, large, blue, metal);**
initial state, is a large blue motal cube.
final
appears be large blue metal sphere.
Thorefore, the transfomatiinn noeded is: Change its shape from cube to sphere: "change, shapol2, 2 sphere)
applying these transformations,
achiave the change from initial state final The transfomations
change colort3, yellow)"
"change, shape(3, cylinder)"
"change position[), right)
'change 2hape/2, sphere)
These trarsoomations match the original answer provided..co anwided.</thinkes </think>
<unswerschange 3olort1 yellow), change shape(3, cylinder), change position(1, right), change sbape(2, sphare) </answer>
Figure 23: Visualization examples Reason-REF and CoT-SFT on Spatial Transformation task.
51


Reviews:
[{"review_id": "r7Eic4bOw7", "reviewer": "Reviewer_ogBh", "summary": "This paper introduces Reason-RFT, a two-stage reinforcement fine-tuning (RFT) framework aimed at improving the generalization capabilities of visual reasoning models. The approach combines SFT with CoT data in Stage 1, followed by reinforcement learning via GRPO in Stage 2. The authors construct a visual reasoning benchmark covering visual counting, structure perception, and spatial transformation tasks to evaluate the framework. They report that Reason-RFT enhances reasoning ability, improves robustness under distribution shifts, and achieves data-efficient learning.", "questions": "1. Novelty of the Training Paradigm: How does Reason-RFT fundamentally differ from other SFT + RL frameworks? What specific innovations make this approach unique beyond using GRPO?\n\n2. SFT Data Generation: Which model was used to generate the SFT-CoT data? How diverse are the reasoning chains in terms of trajectory length and problem complexity?\n\n3. Reward Assignment: How does the system know which reward type (discrete, mathematical, function-based) to apply to each question? Is this predefined in the dataset annotations?\n\n4. Model Selection: Why was Qwen2VL used instead of Qwen2.5VL? Would results change significantly with the more recent model?", "limitations": "See above", "rating": 4, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n\n1. The combination of SFT with CoT and GRPO-based RL is well-motivated, aiming to activate and then further refine reasoning capabilities.\n\n2. The differentiated reward structures for discrete, mathematical, and function-based tasks reflect an attempt to tailor the training to task specifics.\n\nWeaknesses:\n\n1. Lack of Novelty in Training Paradigm: The core idea of combining SFT (especially with CoT) and reinforcement learning is not fundamentally new. Similar SFT + RL pipelines have been widely explored in both language and vision-language models. The distinction of Reason-RFT over existing methods is not made sufficiently clear.\n\n2. Insufficient Detail on SFT Data: The paper does not specify which base model generated the SFT data, nor does it elaborate on the diversity of the reasoning trajectories (e.g., variation in trajectory length, problem types, complexity). This raises concerns about the breadth of the reasoning coverage during SFT.\n\n3. Reward Application Ambiguity: It is not clearly explained how the system determines which reward type (discrete, mathematical, or function-based) to apply during training for each question. \n\n4. Model Choice Unjustified: The authors use Qwen2VL for experiments, but do not explain why Qwen2.5VL was not used. Given that Qwen2VL’s zero-shot performance is already competitive, this choice needs clarification.\n\n5. Marginal Gains over RL Baseline: The performance gap between the RL-only and the combined RL+SFT models is surprisingly small in some tasks (except spatial transformation). This raises questions about the actual contribution of the SFT stage, whether it mainly improves the spatial transformation task or if it offers limited value elsewhere.\n\n6. Limited Novelty in Reward Structure: Although the reward designs are carefully specified, they largely follow standard practices (exact match for discrete, tolerance-based for numeric, sequence overlap for functions).", "was_revised": true, "final_justification": "While the authors provided more details and clarifications to some of my questions. The question about the core contributions still remain and I lean towards maintaining the original score.", "rebuttal": "We sincerely thank the reviewer for the constructive comments and for acknowledging our **well-motivated methodology** and **effective reward design**. Below, we address your specific concerns in detail.\n\n---\n\n> **_`W1 (Q1)`: \"The novelty of our work.\"_**\n\nA: Thank you for raising this point. We would like to clarify that our work is not a direct extension or simple combination of existing methods. Instead, it introduces **a novel and unified post-training methodology** specifically tailored to the core challenges of visual reasoning, such as **cognitive rigidity** and **generalization to domain shift** in dynamic visual reasoning. \n\nIn contrast to previous works that either ***(i)*** focus exclusively on text-based reasoning (`[1]`); ***(ii)*** adopt only-SFT (`[2,3]`) or only-RL (`[4,5,6]`) strategies for post-training in isolation; or ***(iii)*** involve a few concurrent works (`[7,8]`) that combine SFT and RL but remain restricted to static visual tasks with narrowly defined reward types—our approach is **the first** to introduce a **generalizable training framework** that systematically integrates SFT and RFT in a **tightly coupled manner**, effectively addressing **both static** (e.g., structural perception) and **dynamic** (e.g., temporal visual counting, spatial transformation) visual reasoning under a unified reward formulation.\n\nThe novelty of our approach lies in the following aspects:\n\n- **An adaptive training sample allocation strategy** that dynamically balances intermediate reasoning supervision with reward-guided exploration. This strategy enables more effective coupling between the SFT and RFT stages, supporting a unified learning process that aligns model behavior with visual reasoning objectives. Unlike prior methods that treat SFT and RL as disjoint stages, our design ensures their interaction is coherent and purpose-driven.\n\n- **A unified visual reasoning reward formulation** that generalizes across diverse task formats. Beyond replicating standard reward templates, we introduce a principled reward design that captures reasoning correctness and completeness in a task-agnostic manner:\n  - **Structured-format reward** (including summary, caption, thinking and answer tags) to enhance the quality and coherence of reasoning chains.\n  - **Discrete-value reward** for tasks with discrete-value outputs (e.g., classification and counting).\n  - **Tolerance-based reward** for tasks requiring continuous-value outputs (e.g., geometric estimation, visual grounding).\n  - **Function-matching reward** for tasks with sequence-like outputs (e.g., action planning or tool calling).\n\n  These reward formulations are designed to be task-agnostic and applicable, moving **BEYOND** handcrafted, task-specific designs commonly used in prior work.\n\n- **A comprehensive visual reasoning evaluation protocol**, covering both in-domain and domain-shifted scenarios, and spanning synthetic and real-world visual environments. The evaluation rigorously benchmarks multiple dimensions of reasoning ability—including dynamic reasoning, geometric perception, and mathematical arithmetic—enabling us to assess generalization and robustness in a systematic manner.\n\n> [1] *DeepSeek-R1*, ArXiv'25.\n\n> [2] *LLaVA-CoT*, ArXiv'24.\n\n> [3] *LlamaV-o1*, ACL'25.\n\n> [4] *Visual-RFT*, ICCV'25.\n\n> [5] *VLM-R1*, ArXiv'25.\n\n> [6] *R1-Zero’s “Aha Moment”*, ArXiv'25.\n\n> [7] *R1-Onevision*, ICCV'25.\n\n> [8] *Vision-R1*, ArXiv'25.\n\n---\n\n> **_`W2-1 (Q2)`: \"CoT data construction.\"_**\n\nA: Thank you for the question. Our CoT data is built through a combination of **automated generation** and **manual verification**, with careful attention to both **diversity** and **correctness**.\n\n- **Diversity**: We use reasoning-guided templates (e.g., *\"Let's think step by step\"*, *\"Break down your reasoning\"*) to prompt models like **GPT-4o** and **Gemini-Pro**, aided by temperature sampling (`T=0.7`, `top-k=50`, `top-p=0.9`) and include manually crafted exemplars as few-shot demonstrations to enhance generation quality and depth.\n\n- **Correctness**: Outputs are filtered via both human review and scripts. We remove brief or inconsistent responses, and manually validate `10%` of each task subtype. Subtypes with issues are corrected or extended to ensure overall reliability.\n\nA full description of this pipeline will be included in the appendix of the revised version.\n\n---\n\n> **_`W2-2 (Q2)`: \"Trajectory length and problem complexity.\"_**\n\nA: We determine the **trajectory length** of each subtask by combining `50%` manually constructed representative samples and `50%` model-generated average reasoning length. During the automated filtering stage of CoT samples, we exclude those with lengths less than `60%` or greater than `140%`of the target trajectory length.\n\nStatistical analysis shows the average trajectory lengths for the three tasks are:\n- Visual Counting:  `μ ≈ 70, σ ≈ 30`,\n- Structure Perception: `μ ≈ 180, σ ≈ 80`,\n- Spatial Transformation:  `μ ≈ 400, σ ≈ 120`\n\nFor **problem complexity**, we define problem complexity based on the number of reasoning steps in the CoT process. We sampled 100 instances per task and used **GPT-4o** to evaluate the number of reasoning steps. Results indicate that CoT-SFT samples range from `3` to `10` steps, with an average of `5` steps.\n\nWe will include the statistical details of CoT-SFT construction in the appendix of the revised version.\n\n---\n\n> **_`W3 (Q3)`: \"Reward assignment.\"_**\n\nA: Thank you for your question. Reward types are  **predefined** during data construction based on each question’s ground truth format. Each instance is annotated with its expected answer type (e.g., discrete label, numerical value, or structured output), and the corresponding reward function (e.g., discrete-valued, mathematical, or functional-based) is selected to ensure alignment between output format and training feedback.\n\n---\n\n> **_`W4 (Q4)`: \"Experiments on Qwen2.5-VL.\"_**\n\nA: Thanks for your suggestion. To further validate the effectiveness of our method, we conducted experiments on **Qwen2.5-VL-3B-Instruct**. The results are summarized below:\n\n|Model|Setting|T1-1D|T1-DSD|T1-DSM|AVG|T2-ID|T2-DS|AVG|T3-ID|T3-DSL|T3-DSR|AVG|\n|-|-|-|-|-|-|-|-|-|-|-|-|-|\n|qwen2.5vl-3b|ZERO|75.9|50.9|4.4|43.7|36.8|37.4|37.1|8.6|8.3|8.3|8.4|\n||ANS|97.4|51.5|6.0|51.6|53.0|31.8|42.4|**91.1**|47.0|46.8|61.6|\n||COT|89.2|59.5|49.2|66.0|56.1|49.4|52.7|81.6|46.1|44.2|57.3|\n||RFT-ZERO|**99.0**|58.9|10.8|56.2|54.8|54.5|54.6|68.5|49.5|48.0|55.3|\n||Reason-RFT|98.8|**68.7**|**54.8**|**74.1**|**59.0**|**56.6**|**57.8**|86.7|**55.2**|**54.4**|**65.4**|\n\n\nThe results demonstrate that Reason-RFT consistently improves model performance even when applied to the more capable Qwen2.5-VL base models. Key Observations are as follow. Taking Qwen2.5VL-3B as example:\n\n- **(1) Clear Improvement in Visual Reasoning:**  \n  - Reason-RFT outperforms:\n    - ANS by `+22.5` (T1), `+15.4` (T2), `+3.8` (T3)  \n    - COT by `+8.1` (T1), `+5.1` (T2), `+8.1` (T3)  \n    - RFT-ZERO by `+17.9` (T1), `+3.2` (T2), `+10.1` (T3)\n\n- **(2) Strong Generalization under Domain Shift:**  \n  Specifically, on the T1-DSM subset `+44.0` over RFT-ZERO\n\n- **(3) Data Efficiency:**  \n  Similar to Qwen2VL, the Reason-RFT training paradigm on Qwen2.5VL demonstrates high data efficiency:  \n  achieving `~70%` of the final performance of Reason-RFT-ZERO using only `<5%` of the data.\n\nWe will add detailed results and analysis for Qwen2.5-VL in Appendix C as a new subsection: Additional Results on Alternative Backbones.\n\n---\n\n> **_`W5`: \"Stage1 CoT-SFT’s benefit beyond spatial transformation.\"_**\n\nA: Thanks for your question. While Stage 1 CoT-SFT brings the most evident gains on visual transformation tasks due to its structured format, its benefits extend across reasoning scenarios:\n\n| | VisualCount. ( DS-H ) | | SpatialTrans. ( ID+DS )| |\n|:---|:---:|:---:|:---:|:---:|\n||2B|7B|2B|7B|\n|Zero-Shot|0.00|4.80|4.35|13.01|\n|only-RL|5.20|21.20|33.74|56.68|\n|Reason-RFT|28.40|35.60|67.58|65.98|\n- **Format Alignment:**\nIn tasks with compositional or structured output formats, such as spatial transformations, CoT-SFT helps models quickly adapt to expected answer format, avoiding inefficient RL exploration and accelerates convergence.\n- **Domain Knowledge Injection:**\nOn domain-shifted tasks (e.g., VC-DS), where zero-shot performance is poor, CoT-SFT injects domain-specific reasoning traces. Compared to RL-only, Reason-RFT improves by +23.2 (2B) and +14.4 (7B).\n- **Data Efficiency:**\nCoT-SFT serves as an effective policy prior. In Fig. 3 in the main text, Reason-RFT reaches 70% of full-RFT performance using only 3% of its training data.\n- **Generalization Retention:**\nCoT-SFT acts as a regularizer to retain broad reasoning ability (Tab. 6 in the main text), mitigating overfitting seen in RL-only.\n \n> **_`W6`: \"The novelty of Reward Design.\"_**\n\nA: Thank you for the question. We would like to clarify the novelty of reward design as follow:\n\n- We propose a **structured format reward** (`Summary + Caption + Thinking + Answer`) to assess reasoning chain quality, with ablation studies confirming its effectiveness across different paradigms.  \n- We design **fine-grained function-based rewards** that go beyond sequence-level overlap, incorporating item-level exact, partial, and function-only matches. These are further evaluated under different credit assignment strategies (e.g., **exact match only**, **partial credit**, **penalized partial**). Such nuanced reward designs are rarely explored in previous work. \n\nMoveover，we emphasize that our primary contribution in this part is **NOT** the design of intricate and complex reward functions, but the introduction of a **general and practical reward formulation** that supports **broad applicability** across diverse visual reasoning tasks with discrete-value, continuous-value, or agent-based structured outputs.\n\n\n---\nWe hope our response addresses your concern and we look forward to further discussion with you."}, {"review_id": "1KZ2xNkT1J", "reviewer": "Reviewer_rmS6", "summary": "This paper studies the reasoning ability of vision-language models (VLMs) by training them to generate chain-of-thought (CoT) reasoning through supervised learning and reinforcement learning. The authors propose a framework called Reason-RFT, which uses a two-stage training process on vision tasks such as counting, structural perception, and spatial transformation.\nIn the first stage (supervised fine-tuning, or SFT), they create <image, question, chain-of-thought, answer> annotations and train the VLM to produce token-level predictions, meaning it generates both the CoT and the final answer given an image and question.\nIn the second stage, they apply reinforcement learning with a GRPO loss. This involves generating multiple CoT traces, assigning each a reward score, and using these rewards to further refine the model’s parameters.\nExperiments on tasks such as counting, perception, and spatial transformation demonstrate that combining supervised learning and reinforcement learning yields improvements and enhanced generalization.", "questions": "1. How is the initial chain-of-thought (CoT) data for the first-stage supervised fine-tuning (SFT) generated? How do you ensure the reliability and correctness of these first-round CoT annotations?\n2. After reinforcement learning, I would like to see more analysis of the quality of the CoT. Is there any quantitative comparison of CoT outputs before and after GRPO tuning?", "limitations": "Yes, the author discusses limitations in the appendix.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "Strengths:\n\n1. This paper investigates how to enhance the reasoning process (chain-of-thought, or CoT) through a two-stage approach that combines supervised learning and reinforcement learning. In the first stage, supervised learning is achieved by instruction tuning the VLM. In the second stage, reinforcement learning is applied to the VLM using the GRPO loss.\n\n2. Experiments on VQA tasks covering counting, perception, and spatial reasoning show that the two-stage training pipeline improves performance compared to existing models and offers greater transparency.\n\nWeakness:\n\n1. The two-stage training pipeline is quite similar to the training approach used in DeepSeek-R1. It is essentially an extension of training VLMs in the same manner as LLMs. The main differences lie in the chain-of-thought (CoT) generation in the first stage and the new reward function for VQA tasks, which limits its theoretical contribution.\n\n2. The process for generating new CoT annotations used in the first-stage training (Section 3.1) is not explicitly described. Explaining how the initial CoT data for SFT tuning is created is important, as it affects the ability to reproduce the work.\n\n3. Do the open-source VLM models in Table 1 follow the same zero-shot setting? To ensure a fair comparison, these open-source VLMs should be trained using the same dataset.\n\n4. The Reason-RFT does not generalize well to more generic VQA tasks, such as MMMU, RealWorldQA, and AI2D, as shown in Table 6, across different backbones. The improvements are unstable.\n\n5. Relevant works line NMN [1], IPRM [2] are missing in discussing neuro-symoblic methods (line 79)\n\n[1] Neural Module Networks，CVPR2016\n\n[2] Learning to Reason Iteratively and Parallelly for Complex Visual Reasoning Scenarios, NeurIPS 2024", "was_revised": true, "final_justification": "I have read the author's response, and they have addressed my concern, I would keep my original ratings.", "rebuttal": "We sincerely thank the reviewer for the thoughtful and encouraging feedback. We are especially grateful for the recognition of our **two-stage training framework**, as well as the **improved performance and transparency on visual reasoning**. We deeply value your comments and suggestions, and address them in detail below. We will carefully incorporate the relevant clarifications and improvements into the revised manuscript.\n\n---\n\n> **_`W1 (Q1)`: \"The novelty of our work.\"_**\n\nA: Thank you for raising this point. We would like to clarify that our work is not a direct extension or simple combination of existing methods. Instead, it introduces **a novel and unified post-training methodology** specifically tailored to the core challenges of visual reasoning, such as **cognitive rigidity** and **generalization to domain shift** in dynamic visual reasoning. \n\nIn contrast to previous works that either ***(i)*** focus exclusively on text-based reasoning (`[1]`); ***(ii)*** adopt only-SFT (`[2,3]`) or only-RL (`[4,5,6]`) strategies for post-training in isolation; or ***(iii)*** involve a few concurrent works (`[7,8]`) that combine SFT and RL but remain restricted to static visual tasks with narrowly defined reward types—our approach is **the first** to introduce a **generalizable training framework** that systematically integrates supervised fine-tuning (SFT) and reward-based fine-tuning (RFT) in a **tightly coupled manner**, effectively addressing **both static** (e.g., structural perception) and **dynamic** (e.g., temporal visual counting, spatial transformation) visual reasoning under a unified reward formulation.\n\nThe novelty of our approach lies in the following aspects:\n\n- **An adaptive training sample allocation strategy** that dynamically balances intermediate reasoning supervision with reward-guided exploration. This strategy enables more effective coupling between the SFT and RFT stages, supporting a unified learning process that aligns model behavior with visual reasoning objectives. Unlike prior methods that treat SFT and RL as disjoint stages, our design ensures their interaction is coherent and purpose-driven.\n\n- **A unified visual reasoning reward formulation** that generalizes across diverse task formats. Beyond replicating standard reward templates, we introduce a principled reward design that captures reasoning correctness and completeness in a task-agnostic manner:\n  - **Structured-format reward** (including summary, caption, thinking and answer tags) to enhance the quality and coherence of reasoning chains.\n  - **Discrete-value reward** for tasks with discrete-value outputs (e.g., classification and counting).\n  - **Tolerance-based reward** for tasks requiring continuous-value outputs (e.g., geometric estimation, visual grounding).\n  - **Function-matching reward** for tasks with sequence-like outputs (e.g., action planning or tool calling).\n\n  These reward formulations are designed to be task-agnostic and applicable, moving **BEYOND** handcrafted, task-specific designs commonly used in prior work.\n\n- **A comprehensive visual reasoning evaluation protocol**, covering both in-domain and domain-shifted scenarios, and spanning synthetic and real-world visual environments. The evaluation rigorously benchmarks multiple dimensions of reasoning ability—including dynamic reasoning, geometric perception, and mathematical arithmetic—enabling us to assess generalization and robustness in a systematic manner.\n\n> [1] *DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning*, ArXiv'25.\n\n> [2] *LLaVA-CoT: Let Vision Language Models Reason Step-by-Step*, ArXiv'24.\n\n> [3] *LlamaV-o1: Rethinking Step-By-Step Visual Reasoning in LLMs*, ACL'25.\n\n> [4] *Visual-RFT: Visual Reinforcement Fine-Tuning*, ICCV'25.\n\n> [5] *VLM-R1: A Stable and Generalizable R1-style Large Vision-Language Model*, ArXiv'25.\n\n> [6] *R1-Zero’s “Aha Moment” in Visual Reasoning on a 2B Non-SFT Model*, ArXiv'25.\n\n> [7] *R1-Onevision: Advancing Generalized Multimodal Reasoning through Cross-Modal Formalization*, ICCV'25.\n\n> [8] *Vision-R1: Incentivizing Reasoning Capability in Multimodal Large Language Models*, ArXiv'25.\n\n---\n\n> **_`W2`: \"CoT data construction.\"_**\n\nA: Thanks for your question. The CoT data is constructed through a combination of automated generation and manual verification, with both **diversity** and **correctness** rigorously ensured. Specifically:\n\n- **Diversity assurance**: We begin by leveraging a set of carefully designed reasoning-guided templates (e.g., \"Let's think step by step\", \"Break down your reasoning\", etc.) to prompt automatic generation using models like **GPT-4o** and **Gemini-Pro**. To increase stylistic diversity, we apply temperature sampling (`temperature=0.7`, `top-k=50`, `top-p=0.9`) and include manually crafted exemplars as few-shot demonstrations to enhance generation quality and depth.\n\n- **Correctness assurance**: All generated outputs undergo filtering via both manual review and automated scripts. Low-quality samples—such as overly brief reasoning steps or answer inconsistencies with ground truth—are discarded. For each task subtype, we randomly sample `10%` of the data for human validation. Subtypes with deficiencies are further corrected and supplemented, yielding a reliable and comprehensive dataset.\n\nWe will include a detailed description of the CoT data construction process in the appendix of the revised version.\n\n---\n\n> **_`W3`: \"Zero-shot setting consistency in Table 1 models.\"_** \n\nA: Thanks for your suggestion. The open-source models reported in Table 1 are evaluated under the same zero-shot setting to ensure fair comparison. This includes identical evaluation datasets, question prompts, and generation parameters (e.g., `temperature=0.0` for greedy sampling).\n\n---\n\n> **_`W4`: \"Generalization performance on generic benchmarks.\"_**\n\nA: We thank the reviewer for pointing out the generalization performance in Table 6. Our primary goal is **NOT** to achieve the best results on general VQA benchmarks, but to explore an important challenge: **how to enhance domain-specific complex reasoning without severely degrading other capabilities.** \n\n|Method|MMMU|RealWorldQA|MathVision|AI2D|\n|------|----|------------|----------|----|\n|Zero-Shot (7B)|54.10|67.19|16.30|83.00|\n|+ANS-SFT|42.66 (-11.44)|48.10 (-19.09)|9.12 (-7.18)|78.30 (-4.70)|\n|+CoT-SFT|44.67 (-9.43)|36.46 (-30.73)|15.30 (-1.00)|73.25 (-9.75)|\n|+Reason-RFT-Zero|46.44 (-7.66)|45.10 (-22.09)|10.86 (-5.44)|75.28 (-7.72)|\n|+Reason-RFT|50.04 **(-4.06)** |61.31 **(-5.88)**|17.60 **(+1.30)**|81.70 **(-1.30)**|\n\nAs shown in the table above, taking Qwen2VL-7B as example, while Reason-RFT does not significantly improve general VQA scores, it maintains a **more stable general performance** compared to other training paradigms (e.g., ANS-SFT, CoT-SFT, Reason-RFT-Zero), which often cause larger performance drops. For example, on RealWorldQA benchmark, ANS-SFT leads to a `-19.09` drop, CoT-SFT `-30.73`, and Reason-RFT-Zero `-22.09`, while Reason-RFT reduces the gap to only `-5.88`, showing a much more stable performance behavior. Moreover, in some cases, Reason-RFT even brings slight gains (e.g., on MathVision, `+1.30`). This indicates its potential in **balancing multiple abilities during training**.\n\nWe believe this is a valuable insight for future work on developing more balanced and robust multimodal models. Further clarification will be included in the appendix of the revised version.\n\n---\n\n> **_`W5`: \"Add missing citations to prior neuro-symbolic methods.\"_**\n\nA: Thank you for your suggestion. We will include the missing citations (e.g., NMN `[9]`, IPRM `[10]`) in the revised version under the neuro-symbolic methods section of the related work.\n\n> [9] *Neural Module Networks*，CVPR'16.\n\n> [10] *Learning to Reason Iteratively and Parallelly for Complex Visual Reasoning Scenarios*, NeurIPS'24.\n\n---\n\n> **_`Q1`: \"CoT data construction.\"_**\n\nA: Please refer to `W2`.\n\n---\n\n> **_`Q2`: \"Comparison of CoT quality before and after RL.\"_**\n\nA: Thank you for your question. To assess the improvement in CoT quality after RL, we compared the reasoning outputs of the Qwen2VL-3B model trained with Reason-RFT (Stage 2, post-RL) and the one trained with only CoT-SFT (Stage 1, pre-RL) on the Structure Perception task as example. The findings are as follows:\n\n**Qualitative Analysis**:\n1. Despite high textual similarity, the post-RL model exhibits stronger logical coherence between reasoning steps, with fewer broken reasoning chains. For example, in Fig. 16 (case 2) of the main text, the pre-RL model correctly infers a formula but omits the multiplication by 2 in the subsequent step—an issue less common after RL.\n2. The post-RL model demonstrates reflective reasoning behaviors (e.g., *\"let me double check\"*), which we did not observe in pre-RL outputs (e.g., Fig. 1).\n\n**Quantitative Analysis**:\n1. **Reasoning Step Count**: We sampled 100 examples and used GPT-4o to count reasoning steps. The post-RL model produced, on average, `2.7` more steps than the pre-RL model.\n2. **Lexical Usage**: We categorized two types of expressions:\n   - *Prompting words*: *\"oh I see\"*, *\"let me think step by step\"*, *\"let me double check\"*\n   - *Logical connectives*: *\"so\"*, *\"therefore\"*, *\"first\"*, *\"but\"*, *\"moreover\"*\n\n   After RL, the average frequency of prompting words and logical connectives increased by `14%` and `23%`, respectively.\n3. **Answer Accuracy**: As shown in the Structure Perception subplot of Fig. 13 in the appendix, the final answer accuracy improved by an average of `+20.56%` after Stage 2 training.\n\nThese results collectively confirm that RL enhances both the coherence and effectiveness of CoT reasoning.\n\n\n---\nWe hope our response addresses your concern and we look forward to further discussion with you."}, {"review_id": "TC5zuNnRYK", "reviewer": "Reviewer_krtY", "summary": "The authors proposed a two-stage post-training strategy for vision-language models. The post-training framework comprises two stages: 1. SFT with high-quality CoT data, 2. RLVR with GRPO. The paper provides a comprehensive evaluation of various post-training strategies on three visual reasoning tasks including visual counting, structural perception, and spatial transformation. The experimental study analyses the roles of CoT-SFT and GRPO and demonstrate that the proposed two-stage post-training improves the model performance consistently.", "questions": "1. Why both ANS-SFT and CoT-SFT leads to worse results than the zero-shot performance of Qwen2VL-7b in visual counting (ID and DS-D)?\n2. In spatial transformation, why ANS-SFT shows very strong results and pure RL gives the worst performance? \n3. Related to the above two questions: it seems that some tasks benefit more from SFT and some of them benefit more from RL. Do you have more analysis with other visual reasoning tasks and any insights how are the visual tasks coupled with the post-training strategies?\n4. How much does the ratio of data size for SFT and RL affect the performance?", "limitations": "Yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 3, "originality": 2, "strengths_and_weaknesses": "Strengths:\n\n1. a solid work that studies the roles of SFT and GRPO in the post-training of vision-language models\n2. the designed evaluation tasks/datasets are interesting and valuable for the future research\n3. the experiments provide insightful analysis and findings\n\nWeaknesses:\n\n1. the choices of base models are limited to QwenVL family. The analysis would be more comprehensive and impactful if the authors can provide the analysis with other base models when considering the interesting findings with Qwen-series models and RLVR in [1]\n\n[1] Reinforcement learning with random rewards actually works with Qwen 2.5, https://www.interconnects.ai/p/reinforcement-learning-with-random?utm_campaign=post&utm_medium=web", "was_revised": true, "final_justification": "I'd like to thank the authors for providing response to my questions. My concerns have been well addressed. I keep my rating unchanged.", "rebuttal": "We sincerely thank the reviewer for the constructive and encouraging comments. We are grateful for your recognition of our work as a solid study on the roles of SFT and GRPO in VLM post-training, the value of our proposed evaluation tasks and datasets for future research, and the insightful experimental analysis. Your feedback is highly appreciated, and we address your specific concerns in detail below. We will carefully reflect your suggestions in the revised manuscript.\n\n---\n\n> **_`W1`: \"Limited base models. Can the authors provide analysis with other base models beyond Qwen-series?\"_**\n\nA: Thanks for your suggestion. To validate the effectiveness of our proposed method, we conducted additional experiments on **InternVL3-2B-Instruct** using the Visual Counting task. The results are shown below:\n\n| Model        | Setting           | T1-1D | T1-DSD | T1-DSM | AVG    |\n|--------------|-------------------|-------|--------|--------|--------|\n| internvl3-2b | ZERO              | 79.30 | 51.20  | 5.10   | 45.20  |\n| internvl3-2b | ANS               | 96.80 | 52.00  | 6.50   | 51.77  |\n| internvl3-2b | COT               | 88.90 | 60.10  | 50.20  | 66.40  |\n| internvl3-2b | Reason-RFT-ZERO   | 98.90 | 59.40  | 12.30  | 56.87  |\n| internvl3-2b | Reason-RFT        | **99.10** | **69.80** | **55.90** | **74.93** |\n\nThe results demonstrate that Reason-RFT consistently enhances performance on the InternVL backbone. Key observations:\n\n- **(1) Substantial Gains in Visual Reasoning:**  \n  Compared to ANS, COT, and Reason-RFT-ZERO, our Reason-RFT approach leads to notable improvements:  \n  - InternVL3-2B:  \n    - +23.16 over ANS  \n    - +8.53 over COT  \n    - +18.06 over Reason-RFT-ZERO  \n\n- **(2) Robustness under Domain Shift (T1-DSM):**  \n  The largest gains occur in domain-shifted settings:  \n  - InternVL3-2B: +43.60 over Reason-RFT-ZERO on T1-DSM\n\n- **(3) Data Efficiency:**  \n  Similar to Qwen2VL, Reason-RFT shows high data efficiency on InternVL3:  \n  achieving **~70% of the final performance of Reason-RFT-ZERO using only <5% of the data**.\n\nWe will include the full results and analysis for InternVL in the revised version, under **Appendix C: More Experimental Results on Different Backbones**.\n\nAdditionally, based on your reference to the interesting findings in Qwen-series models and RLVR in [1], we further investigated the effect of random reward in the RL training stage. We conducted experiments using both **Qwen2.5VL-3B** and **InternVL3-2B**, applying discrete random rewards (0/1) and continuous random values in [0,1]. Results are as follows:\n\n**Qwen2.5VL-3B:**\n\n| Setting               | Task1  | Task2  | Task3  |\n|-----------------------|--------|--------|--------|\n| zero-shot             | 43.74  | 37.10  | 8.38   |\n| original reward       | 56.42 (+12.68) | 54.63 (+17.53) | 55.33 (+46.95) |\n| random reward (0/1)   | 36.29 (-7.45)  | 32.36 (-4.74)  | 6.57 (-1.81)   |\n| random reward (0~1)   | 34.07 (-9.67)  | 30.32 (-6.78)  | 8.32 (-0.06)   |\n\n**InternVL3-2B:**\n| Setting               | Task1  | Task2  | Task3  |\n|-----------------------|--------|--------|--------|\n| zero-shot             | 45.20  | 30.85  | 7.90   |\n| original reward       | 58.75 (+13.55) | 53.92 (+23.07) | 54.68 (+46.78) |\n| random reward (0/1)   | 37.48 (-7.72)  | 31.66 (+0.81)  | 6.42 (-1.48)   |\n| random reward (0~1)   | 33.95 (-11.25) | 29.84 (-1.01)  | 7.82 (-0.08)   |\n\nBased on these results, we conclude that random reward schemes do not enhance model performance. In fact, they degrade the accuracy for both Qwen2.5VL and InternVL3. For example, with random reward (0~1), performance drops by **-9.67** and **-11.25** on Task1 respectively.\n\n---\n\n> **_`Q1`: \"Why does SFT lead to worse results than the zero-shot performance of Qwen2VL-7B in visual counting (ID and DS-D)?\"_**\n\nA: Thanks for your question—this is indeed an interesting phenomenon. SFT tends to guide models toward memorizing domain-specific response patterns, as demonstrated in [1]. For strong base models like Qwen2VL-7B, which already perform well on visual counting tasks in the zero-shot setting (`98.60`), further SFT may not enhance reasoning ability but instead induce *cognitive rigidity*, where the model aligns too rigidly with training examples, leading to diminished generalization capability on unseen test cases. This rigidity can degrade performance on ID and DS-D test sets. \n\nThis also explains why Reason-RFT employs only a small number of samples (1.6k) during the Stage 1 CoT-SFT phase—sufficient to inject new task knowledge and activate novel reasoning patterns without cognitive rigidity.\n\n> [1] *SFT Memorizes, RL Generalizes: A Comparative Study of Foundation Model Post-training*, ICML'25.\n\n---\n\n> **_`Q2`: \"Why does ANS-SFT outperform, while pure RL underperforms on spatial transformation?\"_**\n \nA: Thanks for your question. Spatial transformation requires specific spatial reasoning and output patterns. When the base model lacks this ability (as evidenced by the low zero-shot scores of the 2B and 7B models—`4.35` and `13.1`, respectively), pure RL tends to explore ineffective regions of the reasoning space and output formats during the early stages of training. Additionally, due to the lack of specialized knowledge in pretraining, pure RL struggles to overcome the knowledge barrier—especially for smaller models like 2B—resulting in poor learning signals and ineffective reasoning. In contrast, ANS-SFT provides direct supervision to quickly learn the desired answering format, achieving readily achievable performance. Therefore, a two-stage SFT+RL (Reason-RFT) process is necessary to ensure stronger and more stable performance.\n\n---\n\n> **_`Q3`: \"How are different visual reasoning tasks coupled with SFT and RL strategies?\"_**\n\nA: Thank you for the insightful follow-up. Indeed, we observe a clear task-specific coupling between visual reasoning tasks and post-training strategies:\n- **SFT** is more effective for tasks where format learning or domain-specific answer templates dominate performance—such as spatial transformation or counting tasks with structured outputs. In these cases, direct supervision helps the model quickly acquire the correct response pattern, especially when the base model lacks such capabilities in the zero-shot setting.\n- **RL,** on the other hand, is more beneficial for tasks requiring multi-step reasoning—such as latent action reasoning or transformation sequence planning—where the model already shows basic task understanding. In such cases, RL fine-tuning helps reinforce coherent reasoning trajectories. However, for tasks the model has not yet grasped, pure RL often fails to provide effective learning signals, limiting its usefulness when used alone.\n\nWhile **neither SFT nor RL alone is sufficient**, our Reason-RFT framework demonstrates that carefully coupling the two—via CoT-based cognitive activation and reward-guided refinement—yields robust improvements. We also find that tuning the data ratio and reward design plays a critical role in maintaining adaptability without overfitting. Additional task-wise analyses and ablations will be incorporated in the appendix of the revised version.\n\n---\n\n> **_`Q4`: \"How much does the ratio of data size for SFT and RL affect the performance?\"_**\n \nA: Thanks for your question. We found that for visual reasoning tasks, the optimal range of CoT samples used for SFT activation lies between `0.8k` and `2k`. Increasing the number of CoT samples beyond 2k brings marginal gains. Interestingly, when the proportion of CoT samples exceeds approximately `40%` of the RL training data, the model performance begins to decline. Once it surpasses `80%`, the performance tends to degenerate toward that of CoT-SFT alone (i.e., cognitive rigidity). \n\nThis is a particularly interesting phenomenon. We will include a performance trend plot and further discussion on this effect in the appendix of the revised version.\n\n\n---\nWe hope our response addresses your concern and we look forward to further discussion with you."}, {"review_id": "D9l5bqNCyq", "reviewer": "Reviewer_nCpk", "summary": "This paper presents a two-stage framework for improving the visual reasoning capabilities of existing VLMs. Specifically, the authors propose to first fine-tune a model using SFT on a dataset with CoT reasoning traces, and then apply RL (i.e. GRPO) to further improve performance. The paper includes a systematic evaluation and analysis of the framework.", "questions": "How was the CoT data for the SFT stage generated? Was it generated using another VLM (i.e. GPT), was it human generated or a combination of both?\n\nWhat sampling mechanism and resolution was used in Table 1 for the baselines and the proposed methods? Was it greedy decoding or sampling with temperature? Was there any difference?\n\nHow many runs were used to compute the numbers presented in table1? Can the authors also report std?", "limitations": "yes", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths:\n\n* Comprehensive evaluation across multiple visual reasoning tasks.\n\n* Analysis of different training dynamics .\n\nWeaknesses:\n\n* While I appreciate the thorough evaluation across three visual reasoning categories, the paper overstates its main technical contribution. The use of SFT on CoT data followed by GRPO-based RL has already been explored in several reasoning works after  DeepSeek-R1. As a result, I believe the claim of Reason-RFT being a “novel training paradigm” in line 24 is a bit overstated. \n\n* The separation between in-domain (ID) and domain-shift (DS) data feels somewhat artificial, especially considering that the base model may have encountered similar samples during pretraining. Moreover, in realistic scenarios, practitioners would likely perform SFT + GRPO over a broader, more diverse dataset rather than separate curated subsets.\n\n* The paper’s presentation lacks clarity. I would recommend that the authors restructure it as an analysis paper , rather than presenting it as a  “novel training paradigm”.  For instance, the  paper should focus  more on how the SFT visual CoT traces were generated or how the reasoning quality influences downstream performance.   In fact, as it stands, it remains unclear how the SFT data was generated and most importantly whether  the SFT with CoT data is actually necessary. For instance, Table 1 shows that Reason-RFT-Zero achieves better average performance on 2 out of 3 tasks.\n\n* While I appreciate the introduced benchmark, there are already a plethora of well-established visual reasoning benchmarks with public leaderboards. I would recommend additionally evaluating the proposed method on at least one of these to facilitate comparison with existing approaches and models.", "was_revised": true, "final_justification": "Please see the response I wrote to the authors.", "rebuttal": "We sincerely thank the reviewer for the thoughtful and encouraging feedback. We will address your suggestions in detail below and incorporate the relevant improvements into the revised manuscript.\n\n---\n\n> **_`W1 (Q1)`: \"The novelty and contributons of our work.\"_**\n\nA: We would like to clarify that our work is not a simple combination of existing methods. Instead, we introduces **a novel and unified post-training methodology** specifically tailored to the core challenges of visual reasoning, such as **cognitive rigidity** and **generalization to domain shift** in dynamic visual reasoning. \n\nIn contrast to previous works that either ***(i)*** focus exclusively on text-based reasoning (`[1]`); ***(ii)*** adopt only-SFT (`[2,3]`) or only-RL (`[4,5,6]`) strategies for post-training in isolation; or ***(iii)*** involve a few concurrent works (`[7,8]`) that combine SFT and RL but remain restricted to static visual tasks with narrowly defined reward types—our approach is **the first** to introduce a **generalizable training framework** that systematically integrates SFT and RFT in a **tightly coupled manner**, effectively addressing **both static** (e.g., structural perception) and **dynamic** (e.g., temporal counting, spatial transformation) visual reasoning under a unified reward formulation.\n\nThe novelty of our approach lies in the following aspects:\n\n- **An adaptive training sample allocation strategy** that dynamically balances intermediate reasoning supervision with reward-guided exploration. This strategy enables more effective coupling between the SFT and RFT stages, supporting a unified learning process that aligns model behavior. Unlike prior methods that treat SFT and RL as disjoint stages, our design ensures their interaction is coherent and purpose-driven.\n\n- **A unified visual reasoning reward formulation** that generalizes across diverse task formats. We introduce a principled reward design that captures reasoning correctness and completeness in a task-agnostic manner:\n  - **Structured-format reward** (including summary, caption, thinking and answer tags) to enhance the quality and coherence of reasoning chains.\n  - **Discrete-value reward** for tasks with discrete-value outputs (e.g., classification and counting).\n  - **Tolerance-based reward** for tasks requiring continuous-value outputs (e.g., geometric estimation, visual grounding).\n  - **Function-matching reward** for tasks with sequence-like outputs (e.g., action planning or tool calling).\n\n  These reward formulations are designed to be task-agnostic and applicable, moving **BEYOND** handcrafted, task-specific designs commonly used in prior work.\n\n- **A comprehensive visual reasoning evaluation protocol**, covering both in-domain and domain-shifted scenarios, and spanning synthetic and real-world visual environments. The evaluation rigorously benchmarks multiple dimensions of reasoning ability—including dynamic reasoning, geometric perception, and mathematical arithmetic—enabling us to assess generalization and robustness in a systematic manner.\n\n> [1] *DeepSeek-R1*, ArXiv'25.\n\n> [2] *LLaVA-CoT*, ArXiv'24.\n\n> [3] *LlamaV-o1*, ACL'25.\n\n> [4] *Visual-RFT*, ICCV'25.\n\n> [5] *VLM-R1*, ArXiv'25.\n\n> [6] *R1-Zero’s “Aha Moment”*, ArXiv'25.\n\n> [7] *R1-Onevision*, ICCV'25.\n\n> [8] *Vision-R1*, ArXiv'25.\n\n---\n\n> **_`W2`: \"Query about the separation between ID and DS data.\"_**\n\nA: We clarify that the ID/DS split in our benchmark is **not artificial**, but based on a principled distinction: whether object categories, scene compositions, and camera viewpoints were seen during training.\n\n- **ID data**: aligns with training distributions in terms of object identity, layout, and viewpoint.\n- **DS data**: introduces systematic shifts—e.g., novel 3D assets, unseen viewpoints, or altered scene configurations.\n\nThis separation reflects **practical generalization challenges** in embodied visual reasoning, where agents often face unfamiliar and dynamic environments. For example, in our Visual Transformation benchmark, the DS set simulates real-world deployment where agents encounter new perspectives or object arrangements.\n\nThe necessity of this split is further **empirically validated**. For example, in the Visual Counting zero-shot setting, Qwen2-VL-2B's accuracy drops sharply from ID (82.40) to DS-D (42.67) to DS-M (0.00), highlighting the difficulty and importance of DS evaluation in diagnosing robustness and transferability.\n\n---\n\n> **_`W3-1 (Q1)`: \"CoT data construction.\"_**\n\nA: The CoT data is built through a combination of automated generation and manual verification to ensure both **diversity** and **correctness**:\n\n- **Diversity**: We use reasoning-guided prompts (e.g., “Let’s think step by step”) with **GPT-4o** and **Gemini-Pro**, applying temperature sampling (`temperature=0.7`, `top-k=50`, `top-p=0.9`) and adding hand-crafted exemplars to enhance variety and depth.\n\n- **Correctness**: Outputs are filtered through **manual review** and **automated scripts**. Low-quality samples (e.g., incomplete reasoning or incorrect answers) are removed. For each task subtype, `10%` of the samples are randomly selected for human validation and refinement if needed.\n\n---\n\n> **_`W3-2`: \"Impact of Reason-RFT on downstream reasoning quality.\"_**\n\nA: We compare the Qwen2.5VL-3B model trained with Reason-RFT (Stage 2, with RL) and the one trained only with CoT-SFT (Stage 1, without RL) on the **Structure Perception** task. Key findings are as follows:\n\n**Qualitative Analysis:**\n1. The RL-trained model shows stronger logical coherence and fewer broken reasoning chains.\n   - E.g., in Fig. 16 (case 2), the pre-RL model omits a crucial “×2” step, which is rare post-RL.\n2. Post-RL responses include reflective phrases (e.g., *“let me double check”*) not seen in pre-RL outputs (Fig. 1).\n\n**Quantitative Analysis:**\n1. **Reasoning steps**: On 100 samples, the RL-trained model generates `+2.7` more steps on average (per GPT-4o analysis).\n2. **Lexical patterns**: Post-RL responses include `+14%` more prompting cues and `+23%` more logical connectives.\n3. **Answer accuracy**: As shown in Fig. 13 (Appendix), final accuracy improves by `+20.6%` after RL.\n\nOverall, Reason-RFT significantly enhances reasoning fluency, structure, and downstream performance—not just correctness.\n\n---\n\n> **_`W3-3`: \"Is CoT-SFT necessary, given Reason-RFT-Zero's strong performance?\"_**\n\nA: While CoT-SFT brings the largest gains on structurally formatted tasks like Visual Transformation, its benefits are broader and foundational. It supports Reason-RFT in the following ways:\n\n| |VC(DS-H)||ST(ID+DS)||\n|-|--:|--:|--:|--:|\n| |2B|7B|2B|7B|\n|Zero-Shot|0.00|4.80|4.35|13.01|\n|only-RL|5.20|21.20|33.74|56.68|\n|Reason-RFT|**28.40**|**35.60**|**67.58**|**65.98**|\n\n- **Format Alignment**: CoT-SFT teaches structured outputs (e.g., spatial transformation), reducing RL exploration cost and improving convergence.\n\n- **Knowledge Injection**: In domain-shifted tasks like visual counting, CoT-SFT boosts generalization. Compared to RL-only, Reason-RFT improves by `+23.2` (2B) and `+14.4` (7B) on VC (DS-H).\n\n- **Sample Efficiency**: CoT-SFT acts as a policy prior. As shown in Fig. 3, Reason-RFT reaches 70% of Reason-RFT-Zero’s performance using only 3% of its training data.\n\nIn short, CoT-SFT is not just a warm-up—it is **crucial** for format learning, generalization, and efficient RL adaptation.\n\n---\n\n> **_`Q1`: \"CoT data construction.\"_**\n\nA: Plase refer to `W3-1`.\n\n---\n\n> **_`Q2`: \"Decoding strategy and temperature settings.\"_**\n\nA: We used **greedy decoding** (`temperature = 0.0`) for all methods reported in the main text. To evaluate robustness under different decoding strategies, we further tested `temperature = 0.5` and `0.7` using Qwen2.5-VL-3B-Instruct on three representative tasks. \n\n|Setting|Greedy|T=0.5|T=0.7|AVG|Greedy|T=0.5|T=0.7|AVG|Greedy|T=0.5|T=0.7|AVG|\n|--|--|--|--|--|--|--|--|--|--|--|--|--|\n|      |TASK1||||TASK2||||TASK3|||||\n|ZERO-SHOT|43.7|40.4|40.7|41.6|37.1|36.1|36.8|36.7|8.4|8.3|9.0|8.6|\n|ANS-SFT|51.6|48.7|48.2|49.5|42.4|40.4|40.1|41.0|61.6|61.0|60.8|61.1|\n|COT-SFT|66.0|62.9|60.6|63.2|52.7|51.4|51.6|51.9|57.3|56.5|56.1|56.6|\n|RFT-ZERO|56.2|53.2|52.9|54.1|54.6|54.1|54.3|54.4|55.3|55.2|55.4|55.3|\n|RFT|**74.1**|**70.1**|**68.8**|**71.0**|**57.8**|**56.8**|**56.1**|**56.9**|**65.4**|**64.7**|**64.3**|**64.8**|\n\n\nOverall, **greedy decoding yields the best absolute scores** due to its determinism. However, **relative performance rankings remain consistent** across temperatures, and Reason-RFT consistently outperforms all baselines.\n\nThese results confirm that our method is **robust and effective under varying decoding strategies**, and the main conclusions of our paper remain unchanged.\n\n---\n\n> **_`Q3`: \"Report standard deviation and clarify number of runs.\"_**\n\nA: Thank you for your question. To ensure reproducibility, we ran each method **three times** with different seeds on **Qwen2VL-2B**. Results (mean ± std) are shown below:\n\n|Method|T1-ID|DSD|DSM|AVG|T2-ID|DS|AVG|T3-ID|DSL|DSR|AVG|\n|:--|:--|:--|:--|:--|:--|:--|:--|:--|:--|:--|:--|\n|ANS-SFT|95.9 ±0.3|50.3 ±1.2|5.6 ±0.4|50.6 ±0.6|50.6 ±1.0|23.0 ±0.8|36.8 ±0.9|77.6 ±0.5|48.8 ±1.1|49.2 ±1.7|58.5 ±1.1|\n|CoT-SFT|85.2 ±0.8|49.6 ±1.2|35.2 ±1.8|56.7 ±1.3|43.8 ±0.9|24.9 ±0.7|34.4 ±0.8|64.4 ±0.1|43.5 ±0.6|43.1 ±0.4|50.4 ±0.4|\n|Reason-RFT-Zero|98.1 ±0.6|58.2 ±1.1|6.3 ±1.0|54.2 ±0.9|47.5 ±0.2|32.8 ±0.5|40.1 ±0.4|42.6 ±1.0|33.8 ±1.1|32.0 ±1.6|36.1 ±1.3|\n|Reason-RFT|96.4 ±0.6|58.6 ±1.2|31.7 ±2.9|62.3 ±1.6|49.5 ±0.5|34.1 ±1.3|41.8 ±0.9|74.4 ±0.4|64.0 ±1.4|64.0 ±0.1|67.5 ±0.6|\n\nAcross the 12 metrics above, all standard deviations are under **2.0**, with most under **1.0**, indicating high stability. Due to space constraints, we omit the Qwen2VL-7B results here but will include them in the revised version’s appendix.\n\n---\nWe hope our response addresses your concern and we look forward to further discussion with you."}, {"review_id": "juyIBbUYfG", "reviewer": "Reviewer_yjNn", "summary": "The paper introduces Reason-RFT, a two-stage post-training pipeline that aims to endow Vision–Language Models (VLMs) with stronger and more transferable visual-reasoning skills.\n\nStage 1 – SFT activation. The authors first perform SFT with CoT–annotated date so that the model learns step-by-step solutions with correct format.\n\nStage 2 – RL. They then continue training with GRPO and two rewards are leveraged: format reward and accuracy reward.\n\nTo evaluate the models, authors reconstructed a benchmark covering visual counting, structural perception, and spatial transformation.\n\nEmpirically, Reason-RFT, built on Qwen2-VL-2B/7B, (i) reaches or surpasses SOTA open and proprietary models on all three tasks, (ii) retains the lead under significant domain shift, and (iii) achieves > 90 % of full-data SFT performance using < 5 % of the data.", "questions": "* Have you tried more recent and advanced VLMs, such as Qwen-2.5-VL?\n\n* Have you tried RL without the format reward?\n\n* Do you observe any \"thinking\" words like \"wait\" or \"let me verify that again\"? From the examples you provided, I don’t see a clear difference between the CoT content and what's inside the <think></think> tags. Does this format really make a difference here?", "limitations": "Yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "## Strengths\n\n* Empirical results indicate that the proposed Reason-RFT is effective on visual tasks like counting, perception, etc.\n* Evaluations and ablation studies are detailed and comprehensive.\n* The code and data release could be useful for future research.\n\n## Weaknesses\n\n* Two core parts of the Reason-RFT is directly from existing work. The findings are expected and the novelty is limited.\n* The designed tasks are relatively easy, which limits the contribution of this work and the performance of RL. In Fig.5a, the avg response  length is ~70, which is far shorter than modern reasoning models (e.g., 8k, 16k, 32k generation lengths).\n* Lack of evaluations on more challenging benchmarks, such as MMMU, MMMU-Pro, MathVista, MathVision, etc. These are more crucial benchmarks for reflecting the reasoning abilities of VLMs.", "was_revised": true, "final_justification": "With the detailed and new results provided by the authors in the rebuttal, I have a better understanding of their motivation and evaluations of the models. With the assumption that all the new content will be included in the revised version of this paper, I'd like to increase the rating by one point.", "rebuttal": "We sincerely thank the reviewer for the encouraging and constructive feedback. We appreciate your recognition of our strong empirical results, thorough evaluations and ablations, and the potential impact of our code and dataset release. Below, we address your specific concerns in detail.\n\n> **_`W1`: \"The novelty of our work.\"_**\n\nA: Thanks. We would like to clarify that our work is not a direct extension or simple combination of existing methods. Instead, it introduces **a novel and unified post-training methodology** specifically tailored to the core challenges of visual reasoning, such as **cognitive rigidity** and **generalization to domain shift** in dynamic visual reasoning. \n\nIn contrast to previous works that either ***(i)*** focus exclusively on text-based reasoning (`[1]`); ***(ii)*** adopt only-SFT (`[2,3]`) or only-RL (`[4,5,6]`) strategies for post-training in isolation; or ***(iii)*** involve a few concurrent works (`[7,8]`) that combine SFT and RL but remain restricted to static visual tasks with narrowly defined reward types—our approach is **the first** to introduce a **generalizable training framework** that systematically integrates SFT and RFT in a **tightly coupled manner**, effectively addressing **both static** (e.g., structural perception) and **dynamic** (e.g., temporal visual counting, spatial transformation) visual reasoning under a unified reward formulation.\n\nThe novelty of our approach lies in the following aspects:\n\n- **An adaptive training sample allocation strategy** that dynamically balances intermediate reasoning supervision and reward-guided exploration. This approach strengthens the coupling between SFT and RFT, enabling a unified learning process aligned with visual reasoning goals—unlike prior methods that treat SFT and RL as separate stages.\n\n- **A unified visual reasoning reward formulation** that generalizes across diverse task formats. Beyond replicating standard reward templates, we introduce a principled reward design that captures reasoning correctness and completeness in a task-agnostic manner:\n  - **Structured-format reward** (including summary, caption, thinking and answer tags) to enhance the quality and coherence of reasoning chains.\n  - **Discrete-value reward** for tasks with discrete-value outputs (e.g., classification and counting).\n  - **Tolerance-based reward** for tasks requiring continuous-value outputs (e.g., geometric estimation, visual grounding).\n  - **Function-matching reward** for tasks with sequence-like outputs (e.g., action planning or tool calling).\n\n  These reward formulations are designed to be task-agnostic and applicable, moving **BEYOND** handcrafted, task-specific designs commonly used in prior work.\n\n- **A comprehensive visual reasoning evaluation protocol** that spans in-domain and domain-shifted settings across both synthetic and real-world environments. It systematically benchmarks key reasoning skills—such as dynamic reasoning, geometric perception, and arithmetic—to assess generalization and robustness.\n\n> [1] *DeepSeek-R1*, ArXiv'25.\n\n> [2] *LLaVA-CoT*, ArXiv'24.\n\n> [3] *LlamaV-o1*, ACL'25.\n\n> [4] *Visual-RFT*, ICCV'25.\n\n> [5] *VLM-R1*, ArXiv'25.\n\n> [6] *R1-Zero’s “Aha Moment”*, ArXiv'25.\n\n> [7] *R1-Onevision*, ICCV'25.\n\n> [8] *Vision-R1*, ArXiv'25.\n\n---\n\n> **_`W2`: \"About the CoT length.\"_**\n\nA: Thanks. First, we clarify that our tasks are far from trivial. Even top proprietary models like **GPT-4o** and **Gemini 1.5 Pro** scored below `50%` across all tasks. For example, GPT-4o achieved only `35.9%` on the Spatial Transformation task, revealing a key limitation in current VLMs. In contrast, our **Reason-RFT 2B model** reached `67.6%`, demonstrating the effectiveness of our approach.\n\nAs for CoT response length, we note that visual reasoning fundamentally differs from traditional text reasoning (e.g., math problems). Our tasks rely on perceptual inference and visual comparison, not verbose textual elaboration. For instance, in Spatial Transformation, models must recognize visual state changes and infer spatial relations—this rarely requires long-form output. Thus, an average output length of `~70` tokens is both sufficient and appropriate, and longer responses do not necessarily imply stronger reasoning.\n\n---\n\n> **_`W3`: \"More experiments on more benchmarks.\"_**\n\nA: Thanks. Our primary goal is **NOT** to achieve the best results on general VQA benchmarks, but to explore an important challenge: **how to enhance domain-specific complex reasoning without severely degrading general capabilities.** \n\n|Method|MMMU|RealWorldQA|MathVision|AI2D|\n|-|-|-|-|-|\n|Zero-Shot (7B)|54.1|67.2|16.3|83.0|\n|+ANS-SFT|42.7 (-11.4)|48.1 (-19.1)|9.1 (-7.2)|78.3 (-4.7)|\n|+CoT-SFT|44.7 (-9.4)|36.5 (-30.7)|15.3 (-1.0)|73.3 (-9.8)|\n|+Reason-RFT-Zero|46.4 (-7.7)|45.1 (-22.1)|10.9 (-5.4)|75.3 (-7.7)|\n|+Reason-RFT|50.0 **(-4.1)** |61.3 **(-5.9)**|17.6 **(+1.3)**|81.7 **(-1.3)**|\n\nReason-RFT achieves **more stable performance** than other methods, with smaller drops across general benchmarks. For instance, on RealWorldQA, it reduces the accuracy drop to only –5.9 (vs. –19.1 for ANS-SFT and –30.7 for CoT-SFT). On MathVision, it even improves over the zero-shot baseline (+1.3). These results demonstrate Reason-RFT’s potential in **balancing specialized reasoning and general capability**.\n\n\nFurther details will be included in Appendix C of the revised version.\n\n---\n\n> **_`Q1`: \"More experiments on Qwen2.5-VL.\"_**\n\nA: Thanks. To further validate the effectiveness of our method, we conducted experiments on **Qwen2.5-VL-3B-Instruct**. The results are summarized below:\n\n|Model|Setting|T1-1D|T1-DSD|T1-DSM|AVG|T2-ID|T2-DS|AVG|T3-ID|T3-DSL|T3-DSR|AVG|\n|-|-|-|-|-|-|-|-|-|-|-|-|-|\n|qwen2.5vl-3b|ZERO|75.9|50.9|4.4|43.7|36.8|37.4|37.1|8.6|8.3|8.3|8.4|\n||ANS|97.4|51.5|6.0|51.6|53.0|31.8|42.4|**91.1**|47.0|46.8|61.6|\n||COT|89.2|59.5|49.2|66.0|56.1|49.4|52.7|81.6|46.1|44.2|57.3|\n||RFT-ZERO|**99.0**|58.9|10.8|56.2|54.8|54.5|54.6|68.5|49.5|48.0|55.3|\n||Reason-RFT|98.8|**68.7**|**54.8**|**74.1**|**59.0**|**56.6**|**57.8**|86.7|**55.2**|**54.4**|**65.4**|\n\n\n- **(1) Clear Improvement in Visual Reasoning:**  \n  - Reason-RFT outperforms:\n    - ANS by `+22.5` (T1), `+15.4` (T2), `+3.8` (T3)  \n    - COT by `+8.1` (T1), `+5.1` (T2), `+8.1` (T3)  \n    - RFT-ZERO by `+17.9` (T1), `+3.2` (T2), `+10.1` (T3)\n\n- **(2) Strong Generalization under Domain Shift:**  \n  Specifically, on the T1-DSM subset `+44.0` over RFT-ZERO\n\n- **(3) Data Efficiency:**  \n  Similar to Qwen2VL, the Reason-RFT training paradigm on Qwen2.5VL demonstrates high data efficiency:  \n  achieving `~70%` of the final performance of Reason-RFT-ZERO using only `<5%` of the data.\n\nWe will add detailed results and analysis for Qwen2.5-VL in Appendix C as a new subsection: Additional Results on Alternative Backbones.\n\n---\n\n> **_`Q2`: \"Ablation experiments on RL without the format reward.\"_**\n\nA: Thanks for the suggestion. To assess the impact of the format reward during RL, we conducted additional experiments on Qwen2.5VL-3B and Qwen2VL-2B across three tasks, comparing RFT training with and without the format reward. The results are summarized below:\n\n|Method|T1-1D|T1-DSD|T1-DSM|T1-AVG|T2-ID|T2-DS|T2-AVG|T3-ID|T3-DSL|T3-DSR|T3-AVG|\n|:--|:--|:--|:--|:--|:--|:--|:--|:--|:--|:--|:--|\n|**3B RFT w/ format**|98.8|68.7|54.8|74.1|59.0|56.6|57.8|86.7|55.2|54.4|65.4|\n|**3B RFT w/o format**|97.4|67.9|54.4|73.2 **(–0.9)**|58.7|54.6|56.6 **(–1.2)**|84.4|54.6|54.0|64.3 **(–1.1)**|\n\n|Method|T1-1D|T1-DSD|T1-DSM|T1-AVG|T2-ID|T2-DS|T2-AVG|T3-ID|T3-DSL|T3-DSR|T3-AVG|\n|:--|:--|:--|:--|:--|:--|:--|:--|:--|:--|:--|:--|\n|**2B RFT w/ format**|96.8|60.0|28.4|61.7|49.0|33.1|41.1|74.6|64.0|64.1|67.6|\n|**2B RFT w/o format**|96.6|59.3|27.5|61.1 **(–0.6)**|48.2|32.1|40.2 **(–0.9)**|72.7|63.8|63.9|66.8 **(–0.8)**|\n\nWe draw two main conclusions from these results:\n\n1. **Performance drop without format reward**: Removing the format reward consistently reduced accuracy across all tasks and models, highlighting its importance in guiding RL training.\n\n2. **Model size affects convergence**: For Qwen2.5VL-3B, convergence was stable without the format reward due to its strong instruction-following ability. However, Qwen2VL-2B showed a ≈200-step delay, indicating that smaller models rely more on explicit format supervision for structured prompts like `<think>`.\n\nWe will include these additional results and discussions in a new subsection of Appendix C: *More Experiment Results on Format Reward*.\n\n---\n\n> **_`Q3`: \"More analysis of CoT.\"_**\n\nA: Thank you. In our pipeline, content within <think> represents the model’s CoT reasoning. To evaluate its improvement post-RL, we compare the Reason-RFT model (Stage 2) with the CoT-SFT baseline (Stage 1) on the Structure Perception task:\n\n**Qualitative Analysis**:\n1. The post-RL model shows **stronger logical coherence**, with fewer reasoning breaks. For instance, Fig. 16 (case 2) highlights a missing calculation step in pre-RL output that is corrected after RL.\n\n2. We observe **reflective reasoning patterns** (e.g., *\"let me double check\"*) emerging post-RL, absent in pre-RL outputs (e.g., Fig. 1).\n\n**Quantitative Analysis**:\n1. **Step Count**: GPT-4o analysis on 100 samples shows post-RL responses have `2.7` more reasoning steps on average.\n\n2. **Lexical Markers**: We categorized two types of expressions:\n   - *Prompting words*: *\"oh I see\"*, *\"let me think step by step\"*, *\"let me double check\"*\n   - *Logical connectives*: *\"so\"*, *\"therefore\"*, *\"first\"*, *\"but\"*, *\"moreover\"*\n   After RL, Use of prompting phrases and logical connectives increased by `14%` and `23%`, respectively.\n3. **Answer Accuracy**: As shown in the Structure Perception subplot of Fig. 13, answer accuracy improves by `+20.6%` post-RL.\n\nThese results confirm that RL substantially enhances the quality and depth of CoT reasoning.\n\n---\nWe hope our response addresses your concern and we look forward to further discussion with you."}]

Author rebuttal:

Meta-review: This paper studies how to improve visual reasoning ability of vision language models (VLMs). The motivation is that existing CoT SFT approaches suffers from overfitting and poor generalization. This work proposed Reason-RFT, a two-stage framework combining SFT and GRPO-based reinforcement learning. This work further introduced a new benchmark to evaluate visual counting, structural perception, and spatial transformation understanding ability. Experiments validated that Reason-RFT achieves robustness under domain shifts with strong data efficiency for multimodal reasoning.

The main strengths are that (1) the two-stage framework design is well-motivated and effective, (2) the empirical results are strong in robustness, transparency, and insightful, (3) the evaluation benchmark is valuable, (4) the data efficiency and robustness are significant. The major weaknesses are that (1) combining CoT SFT with RL is not fundamentally novel, (2) the clarity of CoT data generation process is not good, (3) the evaluation on generalization can be broader, (4) the impact of SFT stage is not significant.

After rebuttal and discussion, the novelty issue is not fully resolved but reviewers acknowledge the contributions of the method. For CoT data construction, authors provided detailed pipeline clarifications, and reviewers widely acknowledged the explaination. For evaluation, authors further provided experiments with Qwen2.5-VL and InternVL3. For impact of SFT stage, additional ablations addressed the concern.

In summary, this paper proposed a solid method implementation with SFT+RL for VLMs and shows comprehensive empirical analyses, but the idea of combining SFT with RL does not have strong novelty. Considering both the strengths and weaknesses, AC recommend accepting the paper and encourage authors to polish the paper according to the comprehensive discussion and comments.
Decision: Accept (poster)