Open Vision Reasoner: Transferring Linguistic Cognitive Behavior for Visual Reasoning
Authors: Yana Wei
Liang Zhao
Jianjian Sun
Kangheng Lin
jisheng yin
Jingcheng Hu
Yinmin Zhang
En Yu
Haoran Lv
Zejia Weng
Jia Wang
Qi Han
Zheng Ge
Xiangyu Zhang
Daxin Jiang
Vishal M. Patel
Venue: NeurIPS 2025 poster ()
Keywords: Multimodal LLM
Visual Reasoning
Cognitive Behavior Transfer
Primary area: applications

Abstract: The remarkable reasoning capability of large language models (LLMs) stems from cognitive behaviors that emerge through reinforcement with verifiable rewards. This work investigates how to transfer this principle to Multimodal LLMs (MLLMs) to unlock advanced visual reasoning. We introduce a two-stage paradigm built on Qwen2.5-VL-7B: a massive linguistic cold-start fine-tuning, 
followed by multimodal reinforcement learning (RL) spanning nearly 1,000 steps—surpassing all previous open-source efforts in scale.
This pioneering work reveals three fundamental insights: 1) Behavior transfer emerges surprisingly early in cold start due to linguistic mental imagery. 2) Cold start broadly memorizes visual behaviors, while RL critically discerns and scales up effective patterns. 3) Transfer strategically favors high-utility behaviors such as visual reflection. Our resulting model, Open-Vision-Reasoner (OVR), achieves state-of-the-art performance on a suite of reasoning benchmarks, including 95.3% on MATH500, 51.8% on MathVision and 54.6% on MathVerse. We release our model, data, and training dynamics to catalyze the development of more capable, behavior-aligned multimodal reasoners.

Open Vision Reasoner: Transferring Linguistic Cognitive Behavior for Visual Reasoning
Yana Wei!.", Liang Zhao? Jianjiann Sun
Kangheng Lin Jisheng Yin
Jingcheng Hu Yinmin Zhang", En Yu②, Haoran Lv Zejia Weng", Jía Wang", Qi Han Zheng Ge?, Xiangyu Zhang', Daxin Jiang', Vishal M. Patel"
JJhns Hopkins University Sttppunn BBPT *UCAS Open
'THU HUUT
gittuu....
Abstract
The remarkable reasoning capability of large language models (LLMs) stems from cognitive behaviors that emerge through reinforcement with verifiable rewards. This work investigates how to transfer this principle to Multimodal LLMs (MLLMs) to unlock advanced visual reasoning. We introduce two-stage paradigm built Qwen2.5-VL-7B: a massive linguistic cold-start fine-tuning, followed by multi- modal reinforcement learning (RL) spanning nearly 1,000 surps--ur all previous open-source efforts in scale. This pioneering work reveals three funda- mental insights: 1) Behavior transfer emerges surprisingly early in cold start due to linguistic mental imagery. 2) Cold start broadly memorizes visual behaviors, while RL critically discerns and scales up effective patterns. 3) Transfer strategi- cally favors high-utility behaviors such as visual reflection. Our resulting model, Open- Vision-Reasoner (OVR), achieves state-of-the-art performance on a suite of reasoning benchmarks, including 95.3% on MATH500, 51.8% on Math Vision and 54.6% on Math Verse. We release our model, data, and training dynamics catalyze the development of more capable, behavior-alignel multimodal reasoners.
Dosiso-lS-viicc pco-22k
MMMBrak
ReVisuul R1 OVR.78
Cloude-3.7 7.Saat
OparVLTlinker
200
Accuracy (%)
20
AIME 2024
AIME 2025
Matis500
MashVVisa
Vision
Versc
Figure 1: Performance comparison with state-of-the-an art models on both textual (AIME 2024, AIME 2025 77, MATH500 (88) and multimodal (Math Vista [64], Math Vision [00), Math Verse [33 math reasoning benchmarks. Open Vision Reasoner (OVR) demonstrates superior results among
open-source models and performs competitively with commercial counterparts.
"Core contribution, 'Corresponding authors: zhaoliang022sstppfnncoom vpatel36@jhu.edu
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Introduction
"The eye sees only what the mind is prepared to comprehend." " Robertson Davies
Shifting Reinforcement Learning from Human Feedback (RLHF) 74 to Reinforcement Learning from Verifiable Reward (RLVR) [1] has endowed LLMs [4 72 with unexpectedly powerful rea- soning across mathematics, code, and general problem-solving. At its core, verifiable
correctness is determined by objective, often rule-based criteria- is inherently less susceptible to "reward hacking" 88 [24] than a learned reward model. This robustness proves instrumental in large-scale RI., enabling the internalization and activation of what recent studies 29 126 135 term
cognitive behaviors- ppttterss like backtracking and subgoal decomposition that are empirically crucial for advanced reasoning.
The multimodal domain, inherently grounded in verifiable visual facts [25] uniquely suited for this paradigm. Yet, early multimodal RL efforts paradoxically adopted RLHF, relying on learned reward models to approximate objective correctness [98] 138 139]. Inspired by the success of RLVR in language models, recent efforts have started exploring rule- mlee rewards in the multimodal setting. Perception-EE RI [125] incorporates supervisions such as IoU and Euclidean distance to enhance the perceptual alignment of MLLMs, while works such as R1-One Vision [166 and VLAA-Thinking [8] construct behavior-rich visual reasoning trajectories through complex pipelines including iterative distillation and synthesizing. Recently, Re Visual-RR [14] adopts effective language-only cold start as a foundation for visual reasoning.
Despite this encouraging progress, these approaches still leave foundational question unanswered: How can linguistic cognitive behaviors transfer to MLLMs for advanced visual reasoning? To address this, we build upon the "RL with cold start" paradigm [4 by conducting large-scale training on Qwen2.5-Ve VL-L- [5), establishing it as a powerful testbed to systematically analyze how such behaviors emerge and scale in the multimodal domain.
To this end, we introduce a robust two-stage methodology designed to first instill linguistic cognitive patterns and then activate them for visual reasoning. Our process begins with a large-scale cold start, fine-tuning V on over 2 million examples to build a strong foundation. This is followed by prolonged reinforcement learning phase under the Open-Reasoner leveraging over 0.3 million mixed-modality examples. To the best of our knowledge, this represents the largest open-source RL practice on this model. The resulting model, Open- Open-Vision- -eaiooe (OVR), validates our approach by achieving strong performance across both language and multimodal benchmarks. As shown in Fig. it achieves 63.5% AIME2024 and 95.3% % on MATH500 for math reasoning, as well as 51.8 % on Math Vision and 54.6% % on Math Verse for visual reasoning.
[40] framework,
To further trace the transfer and evolution of cognitive patterns throughout training, we develop a in-depth visual cognitive behavior analysis. Three central insights are worth highlighting: (1) Behavior transfer emerges remarkably early in cold start, driven by linguistic patterns encoding mental imagery 26 45) as illustrated in Fig. [T] (2) Cold start broadly memorizes diverse visual cognitive behaviors, while RL critically discerns and scales up effective patterns. (3) Transfer follows a strategic path, favoring behaviors with high utility such as visual reflection. These findings deepen the understanding on visual intelligence scaffolded by linguistic reasoning [24]
We further examine how this paradigm impacts foundational capability of MLLMs visual percep- tion. While linguistic cold start introduces perceptual degradation, our study shows that multimodal RL can effectively recover this loss. However, we also observe the limited scalability of RL when focused solely on perceptual tasks, as reward signals increase without corresponding growth in reasoning complexity (e.g., token length). This limitation motivates a more deliberate integration of diverse, primitive visual cognitive behaviors. Such efforts represent a promising direction toward unlocking the potential of more advanced RL frameworks-mmlli-uum or even agentic RL built upon visual manipulation and imagination.
In summary, this paper advances the field through the following three key contributions:
We construct two-stage training pipeline consisting of a linguistic cold start followed by large- scale multimodal RI., enabling effective transfer of cognitive behaviors in MI.LMs.
Our Open Vision Reasoner, the largest open-source RI practice on Qwen2.5-VL.-7B, achieves superior performance on both linguistic and multimodal reasoning benchmarks.

---

Table 1: Visual Cognitive Behaviors and Linguistic Counterparts. We define four key visual cognitive behaviors, providing formal definitions, illustrative examples, and their corresponding linguistic counterparts.
Visual Behavior
Example
Definition
Linguistic
Counterpart
Visual Reflection
"Let me see the image again."
The model explicitly revisits the image after identifying potential mistake or inconsistency its reasoning. indicating an elfort to correct course.
Backtracking
Divide-and-Comee Conquer
"Lat's first look the numbers on the left."
The model breaks down complex visual problem into sub-components or regions, each addressed sequentially to reach final
Suhgoul Setting
Visual Verification
"I will now verify this against the image."
The model confirms that intermedistle conclusions visually grounded by cross-referencinn with the image before proceeding.
Verification
Goal-driven Visual Tracing
"To this answer, need to find an object that...
The model starts from desired visual conclusion and reasons backwards to identify relevant image evidence that supports
Backward Chaining
Think: Okay, need figure, which movie these emajis remind of. Let by looking each emoji individually see recal anyy movies feature elements. First, the first emoji sun, a palm tree, and on island That combination usuilly makes me think of tropical islond-themed movies. Maybe something like "Desert Island Discs" or "Cast Away'? Wait, "Cost Away has
scene..
Then the second emoji volleyball, As I mentioned, "Cast Away" is strong candidate here. But are there other movies with both tropical setting and a volleyball?
Which
Alteratively, maybe "Adventitious" another movie with beach and velleyball. but the first emaji deserted island.
these emojis remind
Away' has a. So narrawing it down, it's likely either "Cast Avay' or "The Beach." me check the key elements again. The first emoji has a sun, palm tree, and islend. The second is volleyboll. "Cast Away" has.. "The Beach" has volleyboll used scene, but the main focus survival...
small ialand with polm
sun,
more
you of?
A: Cast Away.
I'll "Cast Away" because the volleyball central the trapical setting key..
Figure 2: Multiple Cognitive Behaviors in a Single Response. This case shows triggered visual- specific cognitive behaviors like visual divide-and-conquer, reflection, goal-driven visual tracing, along with the linguistic behavior backtracking.
We conduct an in-depth analysis of visual cognitive behaviors in OVR and provide valuable insights into their transfer and evolution across training stages.
2 Cognitive Behavior Preliminaries
Recent studies have highlighted that the emergence of robust reasoning in LLMs is closely tied to the acquisition of certain cognitive behaviors 29 These behaviors reflect structured internal reasoning patterns akin to human groblem-solving Key examples include: (1) Backtracking
revising previously chosen strategy upon identifying inconsistencies (e.g., "This approach won't work because..." "), (2) Verification checking intermediate steps or partial results, (3) Subgoal Setting - breaking down complex problems into manageable components (e.g.. "First, we need to..."), and (4) Backward Chaining reasoning from the desired outcome back to required inputs (c.g., "To get 75, we need a number divisible by..."). The four patterns form a kind of textual inner monologue that emerges naturally in language- based reasoning.
Based on this, we investigate the transfer of cognitive behaviors from language to vision. We define the visual extensions of the aforementioned behaviors- -visull reflection, divide-and-conquee visual verification, and goal-driven visual tracing. Their formal definitions, examples, and corresponding linguistic counterparts are provided in Table [ while Fig. Pipresens a multimodal example encom- passing both linguistic and visual cognitive behaviors. the following sections, we present simple yet effective MILM training pipeline comprising linguistic cold start followed by multimodal RL (Section 3 and systematically analyze the transfer and scaling of these visual cognitive behaviors (Section 5.2
3

---

Open Vision Reasoner
In this section, we introduce build from Qwen2.5-VL-7B [42] from perspectives of training pipeline (Section 3.1. RL algorithm
(OVR), strong multimodal reasoning model
(Section) 3.2 and data construction (Section 3.3
3.1 Training Pipeline
To facilitate efficient cognitive development and cross-modal generalization, we employ the popular "RL with a cold start" paradigm 34 with two sequential training stages:
Stage 1: Linguistic Cold Start. The LLM module is supervised fine-tuned on language-only reasoning datasets distilled from DeepSeek-RI [34]. establishing core cognitive hehaviors such as backtracking and subgoal decomposition within purely linguistic setting.
Stage 2: Multimodal RL. We apply reinforcement learning with on both text and multimodal tasks using verifiable match rewards. This promotes reasoning generalization and aligns previously learned cognitive patterns with visual contexts, enabling effective cross-modal transfer.
40 setting
3.2 RL Algorithm
For the RL stage of our training pipeline, we adopt lightweight Proximal Policy Optimization (PPO) [82] with Generalized Advantage Estimation (GAE) [8I], following the policy and reward design used in Open-Reusoner-Zero [40]. We detail the RL for multimodal tasks below:
Proximal Policy Optimization For each input, consisting of an image / and a textual prompt q. the policy network Te generates responses .o......o... Each response o; is a trajectory (6)
(5g ag ..... I -4 of length T. The state (8) includes (and potentially encoded
(i) features) and previously generated tokens; oz is the token generated at step t. A reward r? is computed at each timestep of trajectory TÜ.
We use GAE to balance bias and variance in advantage estimation. The advantage A for state-action pair (s) (6)) in trajectory Ti is:
T-t-1
A
Σ (72))detts
where 8, Tr + <Vo(5 - Volse).
(I)
Vo is the value function, 7, À discount and GAE factors, and V (8T) for terminal states. ma updated by maximizing (0) using experiences (8e, de; At) sampled under an older policy Fold
(0)
Braa min Alip (,,
Here, (0) [s)) and is clipping parameter (e.g.. 0.2). ET..... denotes the empirical average [5)
over samples from Told- Ve is trained by minimizing Jvalue on samples from Told, using the
empirical discounted returns R. ---111 **
Tvalue (6) EEnus (Vo(s)) R&)2!
Reward Function. We adopt the minimalist rule-based reward design, which evaluates only the cor- rectness of model outputs while ignoring formatting or stylistic preferences. Specifically, we extract the predicted answer encapsulated within \boxed{) in the model's output and compare it against the reference answer. binary reward assigned-1 for exact matches, and otherwise enaabiing a clear, scalable and unhackable reward signal for reinforcement learning.

---

Table 2: Comparison on Language Reasoning and General Benchmarks. Best results are bold and the second-best are underlined for open-souree models. indicatess metrics reproduced by ourselves.
Model
AIME 2024
AIME 2025
MATH500
GPQA Diamond
MMLU
MMLU-Pro
Open-source Models
Qwen2.5 7B
42
6.7
6.7
77.6
32.8¹
72.6'
57.51
Qwen2.5-VL-P2
42
6.71
6,71
67.4
31.8¹
69.6
$1.71
Open
Zero- 7B
DeepSeck-RI-Disill-Oee
39,21
92.8
QwQQ33B- Preview
BonQ=32B-Previeww
96
50.0
33.5
90.6
54.5
Skywork-RIVal V-38BB
94.0
61.6
ReVisual-R1
14
53.3
43.3
89.2
47.5
Clove-source Models
Gemini-2.0-Flash (00
33.4
36.7
69.0
35.4
OpenAI-ol- miiii
90.0
85.2
80.3
Claude 3.7 Sonnet
20.0
13.3
80.4
61.1
80.0
Doubao-1 1-vii-sioiiii------22
26.7
20.0
85.2
56.
OVR-7B
63.5
52.1
95.3
49.8
77.2
67.9
3.3 Dataset Construction
To support cognitive transfer, we carefully curate datasets specifically tailored to each training stage, encompassing both language-only and multimodal domains.
Data Collection. We firstly broadly collect prompt-answer pairs to develop both language and multimodal reasoning skills across mathematical, scientific, and logical domains. For language- only scenarios, we utilize public benchmarks including AIME (up to 2023), MATH [38], Numina- Math [52], Tulu3 MATH 48 and OpenR 1aannRI-Math-2200 22, and other open-source datasets. We also synthesize general logical problems via programmatic generation to further enrich reasoning diversity. Multimodal scenarios incorporate datasets covering geometry problem solving (Geometry3k [61],
GeoQA 19) Geos (112), D. visual discrimination (IconQA 622, Pixmo [21] ChartQA (67). visual
puzzles (Puzzle [99 AlgoPuzzIe (30), STEM (TQA 446), ScienceQA 663, K12 from
68
and multimodal math (AtomThiik [09 in-house curated math).
Data Curation. To refine data quality, we employ multi-step curation process, Firstly, we employ pre-trained model to automatically filter out samples with high training loss, which typically indicate noise excessive complexity. Secondly, rule-based and model-assisted methods then identify and remove undesirable patterns [52]. Thirdly, we apply reweighting to balance coverage, down-weighting overrepresented categories while emphasizing rare but valuable instances. To the end, we distill responses from DeepSeek-RR R1 (34) to construct approximately 2 million cold-start data. To ensure the unhackability and stability during RL, we further exclude problems incompatible with our reward functions (e.g., proof-style questions) and apply difficulty- based heuristic filtering, removing both overly trivial and infeasible samples ensure cell-calibrated learning. This leaves around 300k multimodal RL data. Further details refer to the appendix.
4 Experiments
In this section, we first elaborate our implementation Open- present superior performance across textual benchmarks (Section 4.2) and multimodal scenerios (Section
(OVR). Then, we
4.1 Implementation Details
Our model based on Qwen2.5-VL-7B [5] and employs a two-stage training strategy. In the first stage of cold start, we independently fine-tune the LLM module for 5 epochs with a batch size of 640, a sequence length of 64k, and learning rate of 10-4 leveraging the default Qwen2.5 configuration 422. During the subsequent stage of reinforcement learning. following Open-Reasoner- Zero 40 we utilize PPO and configure GAE with ? = and = 1 to fully capture long-term dependencies crucial for reasoning tasks, enabling stable training. This RL phase proceeds for 900 iterations, during which we adopt a curriculum for the sequence length: begins 24k for the first 300 iterations, increases to 32k through iteration 700, and expands to 48k thereafter, with latest
5

---

Table 3: Evaluation Results on Visual Reasoning Benchmarks. Best results are bold and the second-bes best are underlined for open- models. Indicates results reproduced by ourselves.
Math
VathVerse
DynaMath worst
LogieVista MMMU-Pro
CharXiv desc.
Model
vision-only
striet loose
SPT Methods
LLaVA-One Visionnn 51
62.6
17.6
17.6
9.0
17.7
32.0
24.1
23.6
48.7
IitenLLM-XCoopposr2.. [33
16.2
8,2
34.7
1ntemVLL3-SB [137
70.5
28.6
33.9
23.0
37.5
43.6
37.6
73.6
InternLL2.5888 15
64.5
17.0
22.8
9.4
23.5
36.0
34,3
32.9
68.6
[16
Qwen2-VL-78 [0T
61.6
19.2
25.4
11.0
22.3
33.3
30.5
34.6
58.0
Dwe.2.55VLL.771
69.2'
25.51
41.1
21.8
31.21
53.1
47.9
36.41
67.3
QvQ-72B-Preview
15
34,9
48.2
30.7
39.0
58.2
Kimi 168 9X
66.0
21.8
34.1
18.0
32.3
42.7
Close-source Models
Gemini-2.0-Flaate (00
70.4
47.8
43.6
42.1
47.4
52.3
OpenAI-GPT-40
34.5
64.4
Claude 3.7 Sonnet
66.8
41.3
52.0
39.7
58.2
49.3
GPT-4o mini 31
55.11
27.31
30.0
31.6
48.8
41.41
37.6
34,10'
74.92!
dombao- 1.5-vision pm-
78.6
51.5
64.7
44.9
65.7
Methods
YLAA-Thinker- Qwen2222
g
59.6
19.8
33.9
15.2
30.5
36,0
68.0
26.4
48.2
22.4
41.5
48.5
RI-Onevision-78
[1
40.0
OpenVL.Thinker:-1 7B 23
65.3
23.0
38.1
16.8
35.2
44.5
MM-FFurka-QQeen-2
23.0
MMRI Ma 76
30.7
42.8
17.4
31.9
46.8
Thinkl VB
102
71.6
24.6
42.9
16.5
41.8
42.7
RI-VL-7B
X-REASONER 58
69.0
29.6
43.0
VL-Rethinker- 99
73.7
28.4
46.4
17.8
36.3
42.7
41.7
ReVisual-R1 44
27.5
52.3
WeThink [1:5]
70.9
27.2
44.7
24.4
48.0
53.0
Skywork-R/V-3BB
42.1
40.4
50.6
OVR-7B
72.1
51.8
54.6
33.5
44.6
64.8
54.8
50.2
44.5
73.6
models continuously undergoing this refinement process. We adhere to strict on-policy updates for the policy model and undertake multiple optimization steps for the critic model. Please note that our final model is uniform average of several representative intermediate checkpoints, ensuring balanced and robust performance across various benchmarks. Additional details can found appendix.
4.2 Enhanced Language Reasoning and General Capabilities
Our model is initially evaluated on a variety of language benchmarks, which cover mathematical reasoning and general problem-solving skills. Specifically, include AIME 2024, AIME 2025 (7].
MATH500 (38], GPQA Diamond [79], MMLU (36), and MMLU-Pro [03] We compare Open- Vision-Reasonn (OVR) with strong LLM baselines, including Qwen2.5-7B Distill-Qwen-77
113] DeepSeek-RI-
and Opent-Raasone--Zer--7 40
OVR demonstrates exceptional language reasoning capabilities. On the challenging AIME 2024 and 2025 benchmarks, dramatically surpasses other 7B open-source models by an average of 10% %, achieving performance comparable leading 32B models. This superiority extends to general reasoning tasks, with significant gains of +4.6% % on MMLU and +10.4 % on MMLU-Pro over darameter-matched competitors. These results highlight the effectiveness of our curated, high-quality cold-start training data.
4.3 Superior Visual Reasoning Abilities
To evaluate whether the introduced cognitive behavior transfer leads cross-modal benefits, we further assess the model on suite of multimodal reasoning benchmarks. These tasks involve image- grounded mathematical reasoning, general multimodal reasoning, and chart understanding. Specifi-
cally, we include Math Visia (64), MathVision [100), Math Verse [133], DynaMath [140), WeMath [77]
Logic Vista [TQ). MMMU-Pro [129), and CharXiv [105) for evaluation. We compare our model against strong MLLM baselines, including SFT-based methods such as LLaVA- One Vision (50) and

---

[a) Coldstart Training Dynamics
(b) RL Training Dynamics
SE7
Rowed
w w w M
Respunse Length
Rewaed
48k adues
40
64
28
500 -50
Training Steps
Training Steps
Training Dynamics. (a) The cold-start stage shows step-wise loss decrease. (b) In the RL stage, reward (purple, left axis) and average response length (orange, right axis) grow steadily, with sharp surges after each sequence length expansion.
Figure 3:
Qwen2.5-V6 V.. [5 well as recent rule-based RI methods like Open VL.Thinker 22 MM-Eureka 68 and Re Vesiuull-Ri [14].
As shown in Table 3, our model sets a new breakthrough for 7B models in visual reasoning. It is the first post-trained Qwwe2.55-VL-7B-bbal
model surpass the 50% performance while also achieving state-of-the-eee results among 7B models on DynaMath and Math Verse. This strong overall performance is further underscored by a substantial gain on MMMU-Pr (+7.2% over prior SOTA methods). These results indicate that reasoning capabilities acquired through language training can effectively transfer to multimodal tasks, leading to notable improvements in visual reasoning.
Vision,
5 Discussion
5.1 Analysis of Training Dynamics
In this section, present a comprehensive overview of the training dynamics illustrated in Fig. 3 and provide a detailed analysis of how text and multi modal reasoning metrics evolve throughout the process shown in Fig.
During the initial cold-start phase (Fig. (a)), the model's loss rapidly descends to below 0.5. Subse- quently, across multiple training epochs, the loss exhibits a step-wise, gradual decrease. In parallel, we observe corresponding surge in performance across all benchmarks (Fig. 4 which first ascend sharply before transitioning to a phase of slower, more incremental improvement toward their peak. noteworthy observation is that the aggressive training strategy detailed in Section 4. employing large batch size in concert with high learning pnte-sroves to be essential. This approach is critical for breaking the model's inherent constraints, thereby successfully imbuing it with new cognitive paradigms and sculpting more favorable landscape for reinforcement learning. It is prerequisite that enables our model, which originates from an instruction -unond base, to ultimately achieve text performance that is comparable to, even surpasses models initialized from base 42 or math-specific checkpoints 114].
( Furthermore, Fig. (b) reveals how the model's reward and average token length in the RL phase steadily advance from an initial to exceed 12k. Owing to the stability of the training configuration inherited from Open- Sheen-RRasoner-Zer 40). OVR is successfully trained on diverse corpus of over 20 multi-modal and language-only datasets without encountering any training collapse or performance degradation. Critically, whenever the token length begins platcau even decline, strategically switch to a longer context length, which invariably catalyzes the next wave of rapid reward growth. Fig. captures the coincident yet unsurprising convergent growth trajectory shared by all eight reasoning benchmarks, spanning both text and multi-modal domains, as they progressively ascend towards their zenith amidst fluctuations.
5.2 Multimodal Cognitive Behavior Analysis
Recent studies have highlighted the emergence of cognitive behaviors in MILMS during visual reasoning tasks packs-phenomnna often dubbed "visual aha moments" [3] In this work, move
7

---

24ME'2
AIME25
GPQA
Math500
Callbee
Colnutt
Goateer
Pedomssst Soor
Pedemssae Scare
Tritning Stops
Triining Strpe
Traliin Strys
Traiiin Stops
Vath
MathVision
Varhe
WeMath
srlat
32.-
Narlh
Dec Menr
Deg Seaae
0
300
Tridning Stags
Tridadng Stapa
Training Staps
Tranningg Shaps
Figure 4: Performance Evolution on Reasoning Benchmarks. OVR demonstrates sustained and convergent growth across both linguistic and multi-modal benchmarks throughout the cold start (left) and RL (right) training phases.
(a) Emergence Trend of Visual Reflection
(b) Emergence and Transfer Rates of Cognitive Behaviors
Crlld Soun
emergance fresuency (%)
1Cd Stat
Emargenes Rate (10)
02
12
12
Bacataccaua
Verlfaztin
Schaooi
Badorand
Trining Steps
Cognitive Behavior Category
Figure 5: Multimodal Cognitive Behavior Analysis. (a) Emergence of visual reflection across the cold start and RL training steps. (b) Emergence and transfer rates of four visual cognitive behaviors across base models and training stages. Numerical values denote the language-to toviision transfer rates for each behavior.
beyond plain observations and systematically investigate how these behaviors are transferred from their linguistic counterparts. Our analysis centers on the four pivotal visual cognitive behaviors introduced in Section - which are drawn from foundational research on cognitive patterns [29]. To
quantify this process, we employ GPT-4o (31 to analyze the emergence of each behavior within the inference traces of our OVR model.
Visual behaviors emerge remarkably early from cold start. Following Section 5.1 we tracked
the dynamics of visual reflection, a significant behavior mentioned in previous studys [47 [07],
throughout OVR's training. As depicted in Fig. [5] this vision-specific behavior emerges in significant quantities from the very beginning of the cold-start phase and fluctuates throughout subsequent training steps. Strikingly, we observed that even in linguistic problems, DeepSeek- R1 ''s responses (34] frequently exhibited signs of mental imagery [26 45] as shown in Fig. to construct internal visualizations to support mathematical reasoning. often articulated through phrases such as "let me visualize... or "let me see the image." Once this linguistic scaffolding was introduced into our MI..M, these mental images were rapidly grounded in actual visual input, enabling their rapid and effective generalization within OVR.
Fig. Ia].). The model appeared
Cold-start learns broadly, large-scale RL discerns critically. We further investigate how cognitive behaviors scale during large-scale RL. As shown in Fig. 5ja), after an initial, rapid instillation of patterns during the aggressive cold-start phase, their prevalence is first suppressed then amplified to unprecedented levels during multimodal RI. This counter-intuitive dynamic suggests a clear division of labor: the cold-start phase learns broadly, indiscriminately memorizing all available patterns. In
8

---

contrast, RL discerns critically, acting as a strategic filter for the crucial tokens [7] and scaling up pivotal behaviors. This process of RL- discarding the dross to select the essence significant for achieving superior generalization.
Visual transfer of cognitive behaviors strategic. To analyze the transition from linguistic to visual cognition, we track the emergence and transfer rates (detailed in Appendix C.1 of four core cognitive behaviors across both language and vision modalities. As shown Fig. b), the emergence of backtracking and verification steadily increases across training stages, underscoring their growing importance. Among these, the transfer rate backtracking shows consistent growth-
2.5% to 17.% verification exhibits near-zero transfer throughout both the cold-start and RL phases. This indicates that transfer is a strategic process, for which we posit two potential explanations: (1) Backtracking transfers more readily due DeepSeek-R1 -lleekk--sss 34 inherent "mental imagination" capabilities, while verification, lacking a direct linguistic precursor, is more difficult for the MILM to intermaiize. (2) Mirroring how humans naturally and instinctively process visual information [107], backtracking is a more fundamental component of complex visual reasoning, making its amplification higher priority during the strategic RL phase. We will investigate these hypotheses in greater depth in our future work.
5.3 Beyond Behavior: Visual Perception Analysis and Future Work
Beyond behavioral dynamics, we extend our discussion to a essential capability of MLLMs: visual perception under the cold start plus large-scale RL paradigm. In particular, we investigate two key areas of ppreeett-prreppuuall hallucination and scaling cmpertties-througl a dedicated study on our OVR model.
Cold start impairs perception, while RL enhances. We evaluated both stages of OVR, along with the base model Qwen2.5-VL-7B, on a comprehensive set of multimodal benchmarks targeting visual perception and recognition (MMBench [591. BLINK (28), MMStar [12] HallusionBench [32).
POPE [53], RoallOA [088], MME (27). MMVet (27)). As shown in Table 4 performance steadily improves across tasks such as MMBench, underscoring the effectiveness of our training paradigm. The cold-start model shows declines on several tasks, notably increased hallucinations [56] 118. likely due to token distribution shifts from large-scale linguistic data [53]. However, the regained performance on benchmarks such as MMBench and BLINK demonstrate that long-term multimodal RL can effectively mitigate these issues by discerning perceptual capabilities that critically for multimodal tasks. Looking ahead, degradation from cold start can be mitigated either by incorporating the linguistic data into model pretraining [33 89 by introducing more multimodal supervision during the cold start to establish a stronger visual foundation.
The current unscalability of RL for perception policy. Throughout the multimodal RL, we observed a strong correlation between the reward and the average response length in Fig. 3 which is finding consistent with prior practices 34 40 This reinforces response length as an effective reward proxy, indicative of scaling property tied to reasoning depth and computational resources. However, when focusing on specific discriminative perceptual tasks like OCR and counting, we observe a clear divergence. As shown in Fig. 6 while the reward can be effectively increased, the average response length remains largely stagnant.
This unscalable training dynamic on such challenging tasks hints at more fundamental issue: the absence of certain core visual cognitive behaviors. Address- ing this fundamental capability gap paramount for achieving robust multimodal scaling. Emerging re- search offers promising avenues, such as multi-turn RL with agentic tool-use (e.g., OpenAl-o3 (73) and the integration of intrinsic imagining through mental images 1117 122]. These approaches hold the po- tential bridge current limitations and unlock more scalable multimodal reasoning.
Training Dynamics
Reaort Reeporse Lengh
Resporte Loigh
Reaard
Troizing Steps
Figure 6: Training Dynamics perception tasks including OCR and counting.

---

Table Model Performance on Perception-centric Benchmarks.
MMBench
Model
BLINK
MMStar
HallusionBench
POPE
RealWorldQA
MME
MMVet
en
en
Qwen2.5-VL
85.3!
84.6
53.7!
62.1
19.0'
86.1
69.3
1659.7'
63.9
+Cold Start
86.1
82.1
51.5
62.4
55.0
82.5
63.1
1549.8
61.8
+Multimodal RI.
86.6
84.2
54.1
62.7
53.6
83.2
65.5
1559.1
63.6
6
Related Work
Recent breakthroughs like OpenAl's ol 72 have highlighted the power of RI. in unlocking and scaling reasoning capabilities 6
37 80
within LLMss. -1--eekk-R-Zeer [34] showed that rea-
soning capabilities can emerge purely through large-scale RL, leading to complex behaviors like
self-verification and reflection. Open-source efforts like Open-Reasoner-Zern (40) further demon-
strates that even minimalist RL approaches, such as vanilla PPO (2) with GAE [8]] and simple
rule-based rewards, can drive scaling in response length and benchmark performance open-source
models 42 114.
MLLMS
[01 130 have rapidly progressed from basic image captioning [57 [06] to more
128 134]. Early efforts primarily relied on supervised fine-tuning with Chain-of-TToouhht (CoTT) datasets [II] while some explored explicit reflection [107 and
challenging reasoning tasks 123
self-correction 35 mechanisms to emulate human-like reasoning patterns. More recently, methods
such as PerPO 139 and MDPO [98] adopt RL-based post-training approaches like DPO [78), where alignment learned from paired positive/negative responses. These approaches generally follow the RL from Human Feedback (RLHF) [74 126 or RL from AI Feedback (RLAIF) [49] paradigms, where signals from learned reward models or preference labels are utilized for optimization.
Inspired by the success of RLVR 34 in language models, MLLM research has shifted toward rule-based RL like GRPO 84 into the multimodal domain. This has led to two major lines of efforts: (1) designing task-specific reward objectives { 125], and (2) constructing multimodal "thinking"
85]. Additionally, recent powerful MILMS adopt a language-only cold start [14] 89). using verbal reasoning sequences as a foundation for subsequent multimodal learning. These approaches encourages human-like behaviors 29 or so-called "visual aha moments" in the model responses.
datasets that embed cognitive behaviors within CoT sequences 22
Despite these advances in MLLM, fine-grained understanding of the underlying reasoning mech- anisms remains less explored. In contrast, recent study [29] centered on LLMs posit that effective reasoning is causally linked to the model's acquisition of certain cognitive behaviors, such as verifi- cation, backtracking, subgoal setting, and backward chaining. The test-time studies have observed that invoking these patterns improve performance 77. Entropy-basee analysis further reveals that regions associated with cognitive tokens are critical for diverse and high-qualit reasoning [17]. The multimodal work like synthesizing long-form multimodal CoT data.
Thoughtss
54 attempts to explicitly instill such patterns by
7 Conclusion and Limitations
We propose a two-stage framework to investigate cognitive behavior in the multimodal domain. Combining a linguistic cold start with large-scale multimodal RL enables effective modal transfer and scaling of cognitive patterns. The resulting model, Open Vision Reasoner, the largest open-source RL practice built Qwen2.5-VL-7B, achieves strong performance across linguistic and visual benchmarks. Beyond performance, we systematically analyze visual cognitive behaviors and reveal how they emerge and evolve through training.
While the linguistic cold start provides a strong foundation for reasoning, a trade-off with perception remains. Although multimodal RL partly restores perceptual capability, further work is needed to uncover visual reasoning patterns that better balance or even jointly enhance both aspects. Beyond the observed correlations between behaviors and reasoning, establishing causal insights will further clarify the underlying mechanisms of multimodal reasoning. We hope our findings inspire future research cognitively aligned multimodal agents and open up new possibilities for scaling vision-language reasoning through behavior- centered learning.
10

---

References
[1] D. AL. Doubao 1.5 vision pro 32k, 2025. URL http:::/ww...llccrggg/ee// com/dos//
docs/
[2] L. B. Allal, L. Tunstall, A. Lozhkov, E. Bakouch, G. Penedo, and G. M. B. Hynek Kydlicek.
Open rl: Evaluating Ilms uncontaminated math competitions, Feb. 2025. URL https /huggingface wob..loog//op.e.r///aattaee
[3] Anthropic. Claude.
http:////mmnnnne///ttootttoooeee claude 2023.
[4] J. Bai, S. Bai, Chu, Cui, Dang, Deng, Fan, W. Ge, Y. Han, F. Huang, et al. Qwen technical report. arXiv preprint arXiv:2309. 16609, 2023.
[5] S. Bai. K. Chen, X. Liu, J. Wang, W. Ge, S. Song, K. Dang. P. Wang. Wang, J. Tang, et al. Qwen2. 5-vl technical report. arXiv preprint vrXiv::502.1392: 2025.
[6] M. Balunovic, J. Dekoninck, I. Petrov, N. Jovanovic, and M. Vechev. Matharena: Evaluating Ilms on uncontaminated math competitions, february 2025. URL https://matharena ai.
[7] M. Balunović, J. Dekoninck, and M. V. Ivo Petrov, Nikola Jovanović. Matharena: Evaluating Ilms on uncontaminated math competitions, Feb. 2025. URL. htttps://matharene.
[8] H. Chen, H. Tu, F. Wang, H. Liu, X. Tang, X. Du, Y. Zhou, and C. Xie. Sft or rl? early investigation into training rl rl-like reasoning large vision- lssonnlanguage models. arXiv preprint arXiv:2504. 1468, 2025.
[9] J. Chen, J. Tang. J. Qin, X. Liang L. Liu, E. Xing, and L. Lin. Geoqa: A geometric question answering benchmark towards multimodal numerical reasoning. arXiv preprint arXiv:2105. 14517, 2021.
[10] K. Chen, Z. Zhang, W. Zeng, R. Zhang, F. Zhu, and R. Zhao. Shikra: Unleashing multimodal Ilm's referential dialogue magic. arXiv preprint arXiv:2306. 15155 2023.
[11] L. Chen, J. Li, X. Dong, P. Zhang, Y. Zang. Z. Chen, H. Duan, J. Wang, Y. Qiao, D. Lin, et al. Are we on the right way for evaluating large vision-language models? arXiv preprint 2024.
arXiv:2403.
[12] L. Chen, Li, X. Dong, P. Zhang, Y. Zang. Z. Chen, H. Duan, Wang, Y. Qiao, D. Lin, et al. Are we on the right way for evaluating large vision-language models? arXiv preprint arXiv:2403.. 20330, 2024.
[13] L. Chen, L. Li, H. Zhao. Y. Song, and Vinci. R1-v: Reinforcing super generalization ability in vision-language models with less than $3. http::///gichuub.co/Deeeee http:/DDeee-Aen///RR11 2025. Accessed: 2025-02-02.
Agent/R1
[14] S. Chen, Y. Guo, 7. Su, Y. Li, Y. Wu, Chen, Chen, W. Wang, X. Qu, and Y. Cheng. Advancing multimodal reasoning: From optimized cold start to staged reinforcement learning, 2025. URL https://arxiv... http:/aaxvvvo..rreee/// abs///6///2266 25207042077
[15] Z. Chen, W. Wang, Y. Cao,, Y. Liu, 7. Gao, E. Cui, J. Zhu, S. Ye, H. Tian, 7. Liu, et al. Expanding performance boundaries of open-source multimodal models with model, data, and test-time scaling. arXiv preprint arXiv:2412.0557// 2024.
[16] Z. Chen, W. Wang, H. Tian, S. Ye, Z. Gao, E. Cui, W. Tong, K. Hu, J. Luo, Z. Ma, et al. How far are we to gpt-4v? closing the gap to commercial multimodal models with open-source suites. arXiv preprint 4404 16821, 2024.
[17] D. Cheng, S. Huang, X. Zhu, B. Dai, W. X. Zhao, Z. Zhang, and F. Wei. Reasoning with exploration: An entropy perspective, 2025. URL https: :oattps:/aaxee //wrxxiaavvvv///aa org/ ab////bbb///...////// 14758
[18] Cheng, S. Huang, X. Zhu, Dai, W. X. Zhao, 7. Zhang, and F. Wei. Reasoning with exploration: An entropy perspective. arXiv preprint arXiv:2506.14758 2025.

---

[19] Y. K. Chia, T. Y. Han, D. Ghosal, L. Bing, and S. Poria. Puzzlevqa: Diagnosing multi- modal reasoning challenges of language models with abstract visual patterns. arXiv preprint arXiv:2403. 13315, 2024.
[20] T. Chu, Y. Zhai, J. Yang, S. Tong, S. Xie, Schuurmans, Q. Le, S. Levine, and Ma. Sft memorizes, r1 generalizes: A comparative study of foundation model post-training. arXiv preprint arxiv:2501.17161 2025.
[21] M. Deitke, C. Clark, S. Lee, R. Tripathi, Y. Yang, J. S. Park, M. Salehi, N. Muennighoff, Lo, Soldaini, et al. Molmo and pixmo: Open weights and open data for state-of-the-art 2024.
multimodal models. arXiv preprint
[22] Y. Deng, H. Bansal, F. Yin, N. Peng, W. Wang, and K.-W. Chang. Openvithinker: An early exploration to complex vision-language reasoning via iterative self-improvement. arXiv preprint arXiv:2503.17352 2025.
[23] R. Dong, C. Han, Y. Peng, Z. Qi, Z. Ge, J. Yang, L. Zhao, J. Sun, H. Zhou, H. Wei, et al. Dream- I1m: Synergistic multimodal comprehension and creation. arXiv preprint arXiv:2309.1199) 2023.
[24] Y. Du, Z. Liu, Y. Li, X. Zhao, Huo, B. Wang, W. Chen, Z. Liu, Z. Wang, and J.-R. Wen. Virgo: A preliminary exploration on reproducing ol-like mllm, 2025. URL https //arxiv.oo org/ abs abs/ 2501.
[25] A. Dubey, A. Jauhri, A. Pandey, A. Kadian, A. Al-Dahle, A. Letman, A. Mathur, A. Schelten, A. Yang, A. Fan, et al. The llama herd of models. arXiv preprint arXiv:2407.2178. 2024.
[26] M. Ford. Mental models: towards cognitive science of language, inference, and conscious- ness, 1985.
[27] C. Fu, P. Chen, Y. Shen, Y. Qin, M. Zhang, X. Lin, J. Yang, X. Zheng, K. Li, X. Sun, Y. Wu, and R. Ji. Mme: comprehensive evaluation benchmark for multimodal large language models, 2024. URL/https://
org/ 23066 13394
[28] X. Fu, Y. Hu, Li, Y. Feng, H. Wang, X. Lin, D. Roth, N. A. Smith, W.-C Ma. and R. Krishna. Blink: Multimodal large language models can but not perceive. In European Conference on Computer Vision, pages 148-166. Springer, 2024.
[29] K. Gandhi, A. Chakravarthy, Singh, N. Lile, and N. D. Goodman. Cognitive behaviors that enable self-improving reasoners, or, four habits of highly effective stars. arXiv preprint vrXiv:2503.01300 2025.
[30] D. Ghosal, V. T. Y. Han, C. Y. Ken, and S. Poria. Are language models puzzle prodi- gies? algorithmic puzzles unveil serious challenges in multimodal reasoning. arXiv preprint ExXiv:2403.03844 2024.
[31] GPT-4o. Hello got-4o. 2024. URL htctg:://opemaa..oo/iioooet httpm//nndeehhlll-ttt---000 gpt-40/
[32] T. Guan, F. Liu, X. Wu, R. Xian, Z. Li, X. Liu, X. Wang, L. Chen, F. Huang, Y. Yacoob, et al. Hallusionbench: an advanced diagnostic suite for entangled language hallucination and visual illusion in large vision- language models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 44355-14385 2024.
[33] D. Guo, F. WU, F. Zhu, F. Leng, G. Shi, H. Chen, H. Fan, J. Wang, J. Jiang, J. Wang, et al. Seed1. 5-vl technical report. arXiv preprint arXiv:2505.07062. 2025.
[34] D. Guo, D. Yang, H. Zhang, J. Song. R. Zhang, R. Xu, Q Zhu, S. Ma, P. Wang, X. Bi, et al. Deepseek-rl: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501. /2948, 2025.
[35] J. He, Lin, Q Wang, Y. Fung, and Ji. Self-correction more than refinement: A learning framework for visual and language reasoning tasks. arXiv preprint arXiv:2410.04055 2024.
12

---

[36] D. Hendrycks, C. Burns, S. Basart, A. Zou, M. Mazeika, D. Song, and J. Steinhardt. Mea- suring massive multitask language understanding. International Conference on Learning Representations.
[37] Hendrycks, C. Burns, S. Kadavath, A. Arora. S. Basart, E. Tang, D. Song, and J. Stein- hardt. Measuring mathematical problem solving with the math dataset. arXiv preprint vrXivv22133.00 038444 2021.
[38] D. Hendrycks, C. Burns, S. Kadavath, A. Arora, S. Basart, E. Tang, D. Song, and J. Steinhardt. Measuring mathematical problem solving with the math dataset. Proceedings of the Neural Information Processing Systems Track on Datasets and Benchmarks I, NeurlPS Datasets and Benchmarks 2021, 2021.
[39] J. Hu, X. Wu, Z. Zhu, Xianyu, W. Wang. D. Zhang, and Y. Cao. Openrlhf: An easy-to-use, scalable and nigh-performance rihf framework. arXiv preprint arXiv:2405. 2024.
[40] J. Hu, Y. Zhang, Q. Han, D. Jiang, X. Zhang, and H.-Y. Shum. Open-reasonner-zero An open source approach to scaling up reinforcement learning on the base model. arXiv preprint MxXiv:2503.24290 2025.
[41] W. Huang, B. Jia, Zhai, Cao, Z. Ye, Zhao, Z. Xu, Y. Hu, and Lin. Vision-rl: Incentivizing reasoning capability in multimodal large language models. arXiv preprint arXiv:2003006749 2025.
[42] B. Hui, J. Yang, Z. Cui, J. Yang, D. Liu, L. Zhang. T. Liu, J. Zhang, B. Yu, K. Lu, et al. Qwen2. 5-coder technical report. arXiv preprint arXiv:2409.111 12166, 2024.
[43] A. Hurst, A. Lerer, A. P. Goucher, A. Perelman, A. Ramesh, A. Clark, A. Ostrow, A. Welihinda, Hayes, A. Radford, et al. Gpt-4o system card. arXiv preprint arXiv:2410.21226, 2024.
[44] N. Jain, K. Han, A. Gu, W.-D. Li, E. Yan, T. Zhang, S. Wang, A. Solar-Lezama, K. Sen, and I. Stoica. Livecodebench: Holistic and contamination free evaluation of large language models for code. arXiv preprint arXiv:2403.07974, 2024.
[45] P. N. Johnson-Laird. Mental models cognitive science. Cognitive science, 4(1):11-115, 1980.
[46] Kembhavi, M. Seo, Schwenk, Choi, A. Farhadi, and H. Hajishirzi. Are you smarter than sixth grader? textbook question answering for multimodal machine comprehension. In Proceedings of the IEEE Conference on Computer Vision and Pattern recognition, pages 4999-5007, 2017.
[47] A. Kumar, V. Zhuang, R. Agarwal, Y. Su, J. D. Co-Reyes, A. Singh, Baumli, S. Iqbal, C. Bishop, R. Roelofs, L. M. Zhang, K. McKinney, D. Shrivastava, C. Paduraru, G. Tucker, D. Precup, Behbahani, and A. Faust, Training language models to self-correct via reinforce- ment learning, 2024. URL https:://// tttpss://axx: //axxvv//.iiv./e/aa org/ abs/ 2409.440220 12917
org/abs//e
[48] N. Lambert, J. Morrison, V. Pyatkin, S. Huang. H. Ivison, F. Brahman, L. Miranda, A. Liu, N. Dziri, S. Lyu, Y. Gu, S. Malik, V. Graf, J. D. Hwang, J. Yang. R. L. Bras, O. Tafjord, Wilhelm, L. Soldaini, N. A. Smith, Y. Wang, P. Dasigi, and H. Hajishirzi. Tulu 3: Pushing frontiers in open language model post-training, 2025. 15124
org.
[49] H. Lee, S. Phatale, H. Mansoor, T. Mesnard, J. Ferret, K. Lu, C. Bishop, E. Hall, V. Carbune, A. Rastogi, and S. Prakash. Rlaif vs. rlhf: Scaling reinforcement lcarning from human feedback with ai feedback, 2024. URL //aariv///.iii/a/ org/ abs abs/2309. 2209. 00267
[50] Li, Zhang, D. Guo, R. Zhang, Li, Zhang, K. Zhang, Li, 7. Liu, and Li. Llava-onevision: Easy visual task transfer. arXiv preprint arXiv:2408.0332 2024.
[51] Li, Zhang, D. Guo, R. Zhang, Li, Zhang, K. Zhang, Li, Z. Liu, and Li. Llava-onevision: Easy visual task transfer. arXiv preprint arXiv:2408.0333 2024.
13

---

[52] J. LI, E. Beeching, L. Tunstall, B. Lipkin, R. Soletskyi, S. C. Huang, K. Rasul. L. Yu, A. Jiang, Z. Shen, Z. Qin, B. Dong, L. Zhou, Y. Fleureau, G. Lample, and
S. Polu. Numinamath. [[tttp: htttps://huuggingface co ho/.ea///O////aaaaaaaaaattttCaaa (https:
numina aumina_dataset.
2024.
[53] Y. Li, Y. Du, K. Zhou, J. Wang, W. X. Zhao, and J.-R. Wen. Evaluating object hallucination in large vision- language models. arXiv preprint arXiv:2305.13355 2023.
[54] Y.HH. Liao, S. Elflein, L. He, L. Leal-Taixé, Y. Choi, S. Fidler, and D. Acuna. Longper- ceptualthought:: Distilling system-2 reasoning for system-1 perception. arXiv preprint arXiv:2504. 2e0255044.535,,, 2025.
[55] A. Liu, B. Feng, B. Xue, B. Wang, B. Wu, Lu, C. Zhao, C. Deng, C. Zhang, C. Ruan, et al. Deepseck-v3 technical report. arXiv preprint ar arXiv:2412.19437, 2024.
[56] Liu, Z. Xu, Q. Wei, J. Wu, J. Zou, X. E. Wang. Zhou, and S. Liu. More thinking, less seeing? assessing amplified hallucination in multimodal reasoning models. arXiv preprint arXiv:2505. 21523, 2025.
[57] H. Liu, C. Li, Q. Wu, and J. Lee. Visual instruction tuning. Advances in neural information processing systems, 36:34892-34916, 2023.
[58] Q. Liu, S. Zhang. G. Qin, T. Ossowski, Y. Gu, Y. Jin, S. Kiblawi, S. Preston, M. Wei, P. Vozila, et al. X-reasoner: Towards generalizable reasoning across modalities and domains. arXiv 2025.
preprint arXiv:2505.0390
[59] Y. Liu, H. Duan, Y. Zhang, B. Li, S. Zhang, W. Zhao, Y. Yuan, Wang, C. He, Z. Liu, et al. Mmbench: Is your multi-modal model an all-around player? In European conference computer vision, pages 216-233, Springer, 2024.
[60] I. Loshchilov and F. Hutter. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101, 2017.
[61] P. Lu, R. Gong, S. Jiang, L. Qiu, S. Huang, X. Liang, and S.-C. Zhu. Inter-gps: Interpretable geometry problem solving with formal language and symbolic reasoning. arXiv preprint arXiv:2105.04166 2021.
[62] P. Lu, L. Qiu, J. Chen, T. Xia, Y. Zhao, W. Zhang, Z. Yu, X. Liang, and S.-C. Zhu. Iconqa: A new benchmark for abstract diagram understanding and visual language reasoning. arXiv preprint arXiv:21 10. 13214, 2021.
[63] P. Lu, S. Mishra, T. Xia, L. Qiu, K.-W. Chang, S.-C. Zhu, O. Tafjord, P. Clark, and A. Kalyan. Learn to explain: Multimodal reasoning via thought chains for science question answering. Advances in Neural Information Processing Systems, 35:2507-22 2521, 2022.
[64] P. Lu, H. Bansal, T. Xia, J. Liu, C. Li, H. Hajishirzi, H. Cheng, K.-W. Chang, M. Galley, and J. Gao. Mathvista: Evaluating mathematical reasoning of foundation models in visual contexts. arXiv preprint 1r.0v22310.02255, 2023.
[65] M. Luo, S. Tan, J. Wong, X. Shi, W. Tang, M. Roongta, C. Cai, J. Luo, T. Zhang, E. Li. A. Popa, and I. Stoica. Deepscaler: Surpassing ol prreviw with a 1.5b model by scaling rl, 2025. Notion Blog.
[66] C. Lyu, S. Gao, Y. Gu, W. Zhang, J. Gao, K. Liu, Z. Wang, S. Li, Q. Zhao, H. Huang, W. Cao, J. Liu, H. Liu, J. Liu, S. Zhang, D. Lin, and K. Chen. Exploring the limit of outcome reward for learning mathematical reasoning. 2025. URL https:/ts: https://arxx org/ 225/ 06781
[67] A. Masry, D. X. Long. J. Q. Tan, S. Joty, and E. Hoque. Chartqa: A benchmark for question answering about charts with visual and logical reasoning. arXiv preprint 22Xiv:2203.10244, 2022.
[68] F. Meng, L. Du, Z. Liu, Z. Zhou, Q. Lu, D. Fu, B. Shi, W. Wang, J. He, K. Zhang, et al. Mm-eureka: Exploring visual aha moment with rule-based large-scale reinforcement learning. arXiv preprint 2rXi:2550307766 2025.
14

---

[69] Y. Min, Z. Chen, J. Jiang. J. Chen, J. Deng, Y. Hu, Y. Tang, J. Wang. X. Cheng, H. Song, W. X. Zhao, Z. Liu, Z. Wang, and J.-R. Wen. Imitate, explore, and self-improve: reproduction report on slow-thinking reasoning systems. arXiv preprint arXiv:2412.09413, 2024.
[70] N. Muennighoff, Z. Yang. W. Shi, X. L. Li, L. Fei-Fei, H. Hajishirzi, L. Zettlemoyer, P. Liang, E. Candès, and T. Hashimoto. s1: Simple test-time scaling, 2025. URL https://arxit: //arxiv org/ aba/ab/1..... 19393
[71] OpenAl. GPT-4o Mini: Advancing Cost-Efficieet httpr//eeemneiiieeeeeeetttt////e /ontenaiccomm/dex/// 05-16.
intriiiiiiiii-tttllnnnneeeeeeeee 2025. Accessed: 2025-
[72] OpenAL. Learning to reason with llms.
https://openai..omm/ff
learning- to reason torrasoon----r-------- with-wiihhhh 11ms/ 2025.
[73] OpenAL. Introducing openai o3 and o4-mini. introducing-On -aae-ann----an-------------- and-o4-mini// 2025.
https://openai.com wtw/cocom.ii./////e
[74] L. Ouyang, Wu, X. Jiang, D. Almeida, Wainwright, P. Mishkin, C. Zhang, Agarwal, K. Slama, A. Ray, et al. Training language models to follow instructions with human feedback. Advances in neural information processing systems, 35:27730-27744, 2022.
[75] Y. Peng, X. Wang, Y. Wei, J. Pei, W. Qiu, A. Jian, Y. Hao, J. Pan, T. Xie, L. Ge, et al. Skywork rlv: Pioneering multimodal reasoning with chain-of-thought arXiv preprint S55i::2504.055999 2025.
[76] Y. Peng, G. Zhang, M. Zhang, Z. You, J. Liu, Q. Zhu, K. Yang, X. Xu, X. Geng, and X. Yang. Lmm-rl: Empowering 3b Imms with strong reasoning abilities through two-stage rule-based 2025.
rl. arXiv preprint arXhv:2503.0755
[77] Qiao, Q. Tan, G. Dong, M. Wu, Sun, Song. Z. GongQue, S. Lei, Z. Wei, M. Zhang, et al. We-math: Does your large multimodal model achieve human-like mathematical reasoning? arXiv preprint arXiv:2407.01288 2024.
[78] R. Rafailov, A. Sharma, E. Mitchell, C. D. Manning, S. Ermon, and C. Finn. Direct preference optimization: Your language model is secretly a reward model. Advances in Neural Information Processing Systems, 36:53728-53741, 2023.
[79] D. Rein, B. L. Hou, A. C. Stickland, J. Petty, R. Y. Pang. J. Dirani, J. Michael, and S. R. Bow- man. Gpqa: A graduate-level google-proof q&a benchmark. arXiv preprint arXiv:2311.1200 2023.
[80] Rein, B. I.. Hou, A. C. Stickland, Petty, Pang. J. Dirani, Michael, and R. Bowman. Gpqa: A graduate-level google-prooff q&a benchmark. In First Conference on Language Modeling, 2024.
[81] J. Schulman, P. Moritz, S. Levine, M. Jordan, and P. Abbeel. High-dimensional continuous control using generalized advantage estimation. arXiv preprint arXiv:1506.022 02438, 2015.
[82] J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov. Proximal policy optimization algorithms. arXiv preprint 1707:07770003477 2017.
[83] Z. Shao, P. Wang, Q. Zhu, R. Xu, Song, X. Bi, H. Zhang. M. Zhang, Y. Li, Y. Wu, al. Deepseckmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03.8 03300, 2024.
[84] Z. Shao, P. Wang, Q. Zhu, R. Xu, Song. X. Bi, H. Zhang, M. Zhang, Y. Li, Y. Wu, et al. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint 2402:220003300000 2024.
[85] H. Shen, P. Liu, J. Li, C. Fang. Y. Ma, J. Liao, Q. Shen, Z. Zhang, K. Zhao, Q Zhang et al. Vlm-r1: A stable and generalizable rt-style large lasion-language model. arXiv preprint ArXiv:2504.07611 2025.
15

---

[86] H. Shen, T. Wu, Q. Han, Y. Hsieh, J. Wang, Y. Zhang, Y. Cheng, Z. Hao, Y. Ni. X. Wang, et al. Phyx: Does your model have the" wits" for physical reasoning? arXiv preprint 2025.
arXiv:2505.1500
[87] J. Skalse, N. H. R. Howe, D. Krasheninnikov, and D. Krueger. Defining and characterizing reward hacking, 2025. URL httlhttp:////axii..oo wwwp://.rggbbbbbbaaabeooo wbwp////..... 13085
[88] R. Sutton. The bitter lesson. Incomplete Ideas (blog), 13(1):38, 2019.
[89] C. Team, Yue, Lin, Y. Song, W. Wang. S. Ren. S. Gu, Li, P. Li, L. Zhao, L. Li, Bao,
H. Tian, H. Zhang, G. Wang, D. Zhu, Cici, C. He, B. Ye, B. Shen, Z. Zhang, Z. Jiang, Z. Zheng,
Z. Song, Z. Luo, Yu, Y. Wang. Tian, Y. Tu, Y. Yan, Y. Huang, X. Wang, X. Xu, X. Song,
X. Zhang, X. Yong, X. Zhang, X. Deng. W. Yang, W. Ma. W. Lv, W. Zhuang, W. Liu, S. Deng,
S. Liu, Chen, S. Yu, Liu, S. Wang, Ma, Q Wang, Wang, N. Chen, M. Zhu, K. Zhou,
K. Zhou, K. Fang, J. Shi, J. Dong. J. Xiao, J. Xu, H. Liu, H. Xu, H. Qu, H. Zhao, H. Lv,
G. Wang. D. Zhang, D. Zhang, D. Zhang, C. Ma, C. Liu, C. Cai, and B. Xia. Mimo-vl technical report, 2025. URL attps://arxiv wwggriii./../oeeeoo. abs/2506.....2 03569
[90] Team, R. Anil, S. Borgeaud, J.-B. Alayrac, J. Yu, R. Soricut, J. Schalkwyk, A. M. Dai, A. Hauth, K. Millican, et al. Gemini: family of highly capable multimodal models. arXiv preprint arXiv:2312. 1805, 2023.
[91] K. Team. Kimi k1.5: Scaling reinforcement learning with 1lms. 2025.
[92] K. Team, A. Du B. Gao, B. Xing, C. Jiang, C. Chen, C. Li, C. Xiao, C. Du, C. Liao, et al. Kimi k1. 5: Scaling reinforcement learning with llms. arXiv preprint arXiv:2501.12599 2025.
[93] K. Team, A. Du, B. Yin, B. Xing, B. Qu, B. Wang, C. Chen, C. Zhang, C. Du, C. Wei, et al. Kimi-vl technical report. arXiv preprint arXiv:2504004991 2025.
[94] O. Team. Open Thoughts. hutps://open-thoughts.aa ai, Jan. 2025.
[95] Q. Team. Qvq: To the world with wisdom, 2024.
[96] Q. Team. Qwq-32b: Embracing the power of reinforcement learning. URL.: htpps:/qwennmm github. iofblog/qwq-32b, 2025.
[97] L. S. Vygotsky. Thought and language, volume 29, MIT press, 2012.
[98] F. Wang, W. Zhou, J. Y. Huang, N. Xu, S. Zhang, H. Poon, and M. Chen. mdpo: Con- ditional preference optimization for multimodal large language models. arXiv preprint arXiv:2406. 11839, 2024.
[99] H. Wang, C. Qu, Z. Huang, W. Chu, F. Lin, and W. Chen. Vl-rethinker: Incentiviz- ing self-reflection of vision-language models with reinforcement learning. arXiv preprint arXiv:2504. 2588832700.08837, 2025.
[100] K. Wang. J. Pan, W. Shi, Z. Lu, H. Ren, A. Zhou, M. Zhan, and H. Li. Measuring multimodal mathematical reasoning with math-vishan dataset. Advances in Neural Information Processing 2024.
Systems, 37:95095-955
[101] P. Wang, S. Bai, S. Tan, S. Wang, Z. Fan, J. Bai, K. Chen, X. Liu, J. Wang, W. Ge, et al. Qwen2-vl: Enhancing languael model's perception of the world at any resolution. arXiv preprint arXiv:2409. 12191, 2024.
[102] X. Wang, Z. Yang, C. Feng, H. Lu, L. Li, C.-C. Lin, K. Lin, F. Huang, and L. Wang. Sota with less: Mcts- Mcts-guided sample selection for data-efficient visual reasoning self eel--imprveeeett arXiv 2025.
preprint vrXiv:2504.055
[103] Y. Wang, X. Ma, G. Zhang, Y. Ni, A. Chandra, S. Guo, W. Ren, A. Arulraj, X. He, Z. Jiang, T. Li, M. Ku, K. Wang, A. Zhuang, R. Fan, X. Yue, and W. Chen. Mmlu-pro: A more robust and challenging multi-task language understanding benchmark. In Advances Neural Information Processing Systems, NeurIPS 2024, 2024.
16

---

[104] Z. Wang, M. Xia, L. He, H. Chen, Y. Liu, R. Zhu, K. Liang. X. Wu, H. Liu, S. Malladi, A. Chevalier, S. Arora, and D. Chen. Charxiv: Charting gaps realistic chart understanding in multimodal 1lms, 2024. Att.sttss://axii org/ abs/ abs/2406. 18521
[105] Z. Wang, M. Xia, L. He, H. Chen, Y. Liu, R. Zhu, K. Liang, X. Wu, H. Liu, S. Malladi, et al. Charxiv: Charting gaps in realistic chart understanding in multimodal llms. Advances in Neural Information Processing Systems, 37:
2024.
[106] H. Wei. L. Kong, J. Chen, L. Zhao, Z. Ge. J. Yang. J. Sun, C. Han, and X. Zhang, Vary: Scaling up the vision vocabulary for large vision- Saiiuag model. In European Conference on Computer Vision, pages 408-424. Springer, 2024.
[107] Y. Wei, L. Zhao, K. Lin, E. Yu, Y. Peng, R. Dong. J. Sun, H. Wei, Z. Ge, X. Zhang, et al. Perception in reflection. arXiv preprint 2500: 2025.
[108] X.AL. Grok-2 beta release. https: x.a.aaai//.. wtt.aaaibblog/ooooo /ttps://z.a/bbloggrrokk 2024. Accessed on: 2024-07-02.
[109] K. Xiang, Z. Liu, Z. Jiang. Y. Nie, R. Huang, H. Fan, H. Li, W. Huang, Y. Zeng. J. Han, et al. Atomthink: A slow thinking framework for multimodal mathematical reasoning. arXiv preprint arXiv:2411. 1930, 2024.
[110] Y. Xiao, E. Sun, T. Liu, and W. Wang. Logicvista: Multimodal Ilm logical reasoning benchmark in visual contexts, 2024. MtLLttts:///riivv.o
/tw.b///220020.. 049733
[111] G. Xu, P. Jin, L. Hao, Y. Song, L. Sun, and L. Yuan. Llava-ol: Let vision language models reason step-by-step. arXiv preprint arX1v:2411.10440, 2024.
[112] L. Xu, Y. Zhao, J. Wang, Y. Wang, B. Pi, C. Wang, M. Zhang, J. Gu, X. Li, X. Zhu, et al. Geosense: Evaluating identification and application of geometric principles in multimodal reasoning. arXiv preprint arXiv:2504. /2597, 2025.
[113] A. Yang, B. Yang, B. Zhang, Hui, B. Zheng, Yu, C. D. Liu, Huang, Wei, et al. Qwen2. 5 technical report. arXiv preprint arXiv:2412.15111 2024.
[114] A. Yang, B. Zhang, B. Hui, B. Gao, B. Yu, C. Li, D. Liu, J. Tu, Zhou, Lin, K. Lu, M. Xue, R. Lin, T. Liu, X. Ren, and Z. Zhang. Qwen2.5-math technical report: Toward mathematical expert model via self-improvement. arXiv preprint arXiv:2409. 12122, 2024.
[115] J. Yang, F. Ma, Z. Wang, D. Yin, K. Rong, F. Rao, and R. Zhang. Wethink: Toward general- purpose vision- language reasoning via reinforcement learning, 2025. URL /attps://arxt org/ htsp//bb/550.... 07905
[116] Y. Yang, X. He, H. Pan, X. Jiang, Y. Deng, X. Yang, H. Lu, D. Yin, F. Rao, M. Zhu, et al. R1- onevision: Advancing generalized multimodal reasoning through cross-modal formalization. arXiv preprint arXiv:2503. 10615, 2025.
[117] Z. Yang, X. Yu, D. Chen, M. Shen, and C. Gan. Machine mental imagery: Empower multimodal reasoning with latent visual tokens. arXiv preprint arXiv:2506. /7218, 2025.
[118] Z. Yao, Y. Liu, Y. Chen, J. Chen, J. Fang, L. Hou, Li, and T.-S. Chua. Are reasoning models more prone to hallucination? arXiv preprint arXiv:2505.23666, 2025.
[119] Q. Ye, H. Xu, G. Xu, J. Ye, M. Yan, Y. Zhou, J. Wang, A. Hu, Shi, Y. Shi, et al. mplug- owl: Modularization empowers large language models with multimodality. arXiv preprint arXiv:2304. 200.23444118888 14178,11181 2023.
[120] E. Yeo, Y. Tong, M. Niu, G. Neubig, and X. Yue. Demystifying long chain-of-tte of-thought reasoning 1lss. arXiv preprint arXiv:2502.03337. 2025.
[121] B. Yin, Q. Wang, P. Zhang, J. Zhang, K. Wang, Z. Wang. J. Zhang, K. Chandrasegaran, H. Liu, Krishna, Xie, M. Li, J. Wu, and Fei-Fei. Spatial mental modeling from limited views, 2025. U chttps:///axttt org/ abs/ /2ob/ 21458
17

---

[122] B. Yin, Q. Wang, P. Zhang, J. Zhang, K. Wang, Z. Wang, J. Zhang, K. Chandrasegaran, H. Liu. R. Krishna, et al. Spatial mental modeling from limited views. arXiv preprint 2025.
arXiv:2506.21455
[123] E. Yu, L. Zhao, Y. Wei, J. Yang, D. Wu, L. Kong, H. Wei, T. Wang, Z. Ge, X. Zhang, et al. Merlin: Empowering multimodal umms with foresight minds. In European Conference on Computer Vision, pages 243.44.. Springer, 2024.
[124] E. Yu, K. Lin, L. Zhao, Y. Wei, Z. Zhu, H. Wei, Sun, Z. Ge, X. Zhang, J. Wang, al. Unhackable temporal rewarding for scalable video milms. arXiv preprint 2025.
[125] E. Yu, K. Lin, L. Zhao, J. Yin, Y. Wei, Y. Peng, H. Wei, J. Sun, C. Han, Z. Ge, et al. Perception-rl: Pioneering perception policy with reinforcement learning. arXiv preprini SSKiv:2504.07954 2025.
[126] T. Yu, Y. Yao. H. Zhang, T. He, Y. Han, G. Cui, J. Hu, Z. Liu, H.-T. Zheng, M. Sun, et al. Rlhf-v: Towards trustworthy milms via behavior alignment from fine-grained correctional human feedback. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pages 13807-13816, 2024.
[127] W. Yu, Z. Yang, L. Li, J. Wang, K. Lin, Z. Liu, X. Wang, and L. Wang. Mm-vet: Evaluating large multimodal models for integrated capabilities. arXiv preprint arXiv:2308.02444 02490, 2023.
[128] X. Yue, Y. Ni, K. Zhang, T. Zheng, R. Liu, G. Zhang, S. Stevens, D. Jiang, W. Ren, Y. Sun, et al. Mmmu: A massive multi-discipline multimodal understanding and reasoning benchmark for expert agi. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9556-9567 2024.
[129] X. Yue, Zheng, Y. Ni, Y. Wang, K. Zhang, Tong, Y. Sun, B. Yu, G. Zhang, H. Sun, Y. Su, W. Chen, and G. Neubig. Mmmu-pre pro: A more robust multi-discipline multimodal understanding benchmark, 2024. URL htthstps:: : //arxivv. wwexiivvorg//.. /209 02813
[130] J. Zhang, R. Dong, H. Wang, X. Ning, H. Geng, P. Li, X. He, Y. Bai, Malik, S. Gupta, and H. Zhang. Alphaone: Reasoning models thinking slow and fast at test time. arXiv preprint DxXiv:5505224865 2025.
[131] J. Zhang, J. Huang, H. Yao, S. Liu, X. Zhang, S. Lu, and D. Tao. R1-vI: Learning to reason with multimodal large language models via step-wise group relative policy optimization. arXiv preprint arxiv:2503. 12937, 2025.
[132] P. Zhang, X. Dong, Y. Zang, Y. Cao, R. Qian, L. Chen, Q. Guo, H. Duan, B. Wang, L. Ouyang, et al. Internim- Deeemlm-xcoopose--2..5 A versatile large vision language model supporting long- contextual input and output. arXiv preprint 200.v:2407.03320. 2024.
[133] Zhang, D. Jiang, Zhang, Lin, Z. Guo, P. Qiu, A. Zhou, P. Lu, K.-W. Chang, Gao, and H. Li. Mathverse: Does your multi-modal lim truly see the diagrams in visual math problems?, 2024. https:::
org/ abs/ 2403. 14624
[134] L. Zhao, E. Yu, Z. Ge, J. Yang, H. Wei, H. Zhou, J. Sun, Y. Peng. R. Dong, C. Han, et al. Chatspot: Bootstrapping multimodal llms via precise referring instruction tuning. arXiv preprint arXiv:2307.09474 2023.
[135] R. Zhao, A. Meterez, S. Kakade, C. Pehlevan, S. Jelassi, and E. Malach. Echo chamber: RI post-training amplifies behaviors learned in pretraining. arXiv preprint arX1v:2504.07912. 2025.
[136) D. Zhu, J. Chen, X. Shen, X. Li, and M. Elhoseiny. Minigpt-4: Enhancing vision-language understanding with advanced large language models. arXiv preprint arXiv:2304. 10592, 2023.
[137] J. Zhu, W. Wang, Z. Chen, Z. Liu, S. Ye, L. Gu, H. Tian, Y. Duan, W. Su, J. Shao, et al. Internv13: Exploring advanced training and test-time recipes for open-source multimodal models. arXiv preprint 2501.2500410479 2025.
18

---

[138] K. Zhu, L. Zhao, Z. Ge, and X. Zhang. Self-supervised visual preference alignment. In Proceedings of the 32nd ACM International Conference on Multimedia, pages 291-300, 2024.
[139] Z. Zhu, L. Zhao, K. Lin, Yang, E. Yu, C. Liu, H. Wei, J. Sun, Z. Ge, and X. Zhang. Perpo: Perceptual preference optimization via discriminative rewarding. arXiv preprint arXiv:2502.04371, 2025.
[140] C. Zou, X. Guo, R. Yang, Zhang, B. Hu, and H. Zhang. Dynamath: A dynamic visual benchmark for evaluating mathematical reasoning robustness of vision language models, 2024.
19

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
The abstract and introduction accurately reflect the paper's contributions and scope. We assert that this paper pioneers the exploration of cognitive behavior transfer from language to vision. Furthermore, we establish several key insights about the transfer. Driven by these findings, we developed our Open- state-of-the-art (SoTA) results across multiple visual and language tasks.
Justification:
which achieves new
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings. It is fine to include aspirational goals as motivation long it is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer:
[Yes]
Justification:
The discussion of limitation is presented in Section
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or speech-to-text system might not be used reliably provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairesss.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged
the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
20

---

Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
Justification: the paper does not include theoretical.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification: All reproduce information including model architecture, algorithms and implementation details are provided in this paper.
Guidelines:
The answer NA means that the paper does not include experiments. If the paper includes experiments, a No answer to this question will not perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to model. In general. releasing code and data often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
specific model and empirical evaluation, may
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (c.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should possible for other researchers to have some path to reproducing verifying the results.
21

---

5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, described in supplemental material?
Answer: [Yes]
We plan publicly release all the source code and dataset upon paper acceptance.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://mips.co
for more details.
While encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https for more details.
7 //nips. htcp//blcc//uudeeeeeeeeeeeeeeeee
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: We present the experimental setting in section
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [No]
Justification:
We do not report error bars in this work.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi-
dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
22

---

The method for calculating the error bars should be explained (closed form formula, call a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful figures symmetric error bars that would yield results that are out of range (e.g. negative rates).
show in tables or
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification:
We report that all experiments are conducted on NVIDIA A100 Tensor Core
GPU.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics atttps://neurre http://wwwwp.pccoool/o//ccccctca
Answer: [Yes]
Justification: This work aligns with NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive socictal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: This work does not have potential negative societal impacts.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
23

---

Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), faireess considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: There is high risk for misuse of our models and datasets.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g.. code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
All code and data following licenses properly. And all related works cited.
Justification:
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
24

---

If assets are released, the license, copyright information, and terms of use in the thcode. has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
package should be provided. For popular datasets, paperswi
batasets.papersstt
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. If this information is available online, the authors encouraged to reach out the asset's s creators.
13. New assets
Question: Are assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: We introduce new model and data in this paper. While do not publicly release them at the submission stage to preserve anonymity, we will further release the full code, models, and accompanying documentation publicly.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moodel as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: This paper does not involve crowdsourcing experiments and research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: This paper does not involve crowdsourcing experiments and research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
25

---

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if' applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, originality of the research, declaration is not required.
Question:
Answer: [Yes]
Justification:
We use gpt-4o-mini in our evaluation.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy htttep//heerrracccc///oofeeerrrr for what should or should not be described.
26

---

The appendix includes extended details on data curation (Appendix A implementation (Appendix cognitive behavior evaluation (Appendix ( and additional case studies (Appendix D)
A Cold-Start Data Curation Details
critical component of our initial policy development is the curation of the cold-start SFT dataset. This stage serves the foundation for subsequent learning, particularly in shaping the model's 's ability to exhibit structured reasoning and cognitive behavior. To this end, we adopt multi-stage curation pipeline consisting of data collection, filtering, cleaning, and strategic reweighting.
As mentioned in Section 3.3
Data Acquisition. We begin by assembling broad set of prompl-response pairs that span diverse reasoning domains. These include math, science, and general logical reasoning tasks such as puzzles, deductive tasks, and constraint satisfaction problems. Our sources include mix of public datasets as illustrated Section
Automated Filtering. To improve the signal-to-noise ratio, filter the collected data using lightweight pretrained LLM as a proxy for quality estimation. Each instance passed through this model, and those with abnormally high training loss are flagged noisy, ambiguous, misaligned. We further apply rule-based and model-assiste pattern detectors to identify and climinate undesirable data characteristics.
Difficulty Stratification. We explicitly incorporate samples from AMC, AIME, Olympiads, and AoPS forums ensure difficulty levels. We then stratify the collected samples based on their source and inherent problem complexity to balance coverage across casy, intermediate, and challenging reasoning scenarios.
Reweighting and Balance. To address imbalances across domains and formats, we apply a reweighting scheme based coverage and reasoning relevance. Over-represented formats down-weighted, while rare but cognitively rich categories are given higher sampling probabilities. This ensures uniform distribution of reasoning challenges and minimizes overfitting to dominant patterns.
B
More Implementation Details
Model and Optimization Setup Our model is based on the Qwen2.5-VL [5] During RL, both the policy and critic networks initialized from the cold-start model. The value head is initialized from uniform distribution U( ((-v5, V5, v5) without bias. The policy and critic networks do not share weights during training. We use the AdamW optimizer with 3 = [0.9, 0.95] and no weight decay. Learning rates are set
10 for the policy and 5 10-6 for the critie. We use constant learning rates with a linear warm-up of 50 steps, and employ sample packing for improved throughput. No KL regularization or entropy bonus is used, demonstrating that vanilla PPO remains stable under our setup.
PPO Training Dynamies Each PPO update is based on 512 unique prompts, each generating 16 sampled responses using temperature and top-p sampling (both set to 1.0). To ensure training stability, we enforce striet on-policy updates for the policy: each prompt generation corresponds a single optimization step. In contrast, the critic performs 4 optimization steps per PPO update. We apply batch-level advantage normalization to stabilize training further.
C Details for Coginitive Behavior Evaluation
In this section, we detail definitions of metrics cognitive behavior analysis (Section 5.2) and show the prompt for evaluation.
27

---

C.1 Behavior Transfer Rate
To quantify how well lanuugg--aquuiee cognitive behaviors generalize the visual modality, we de- fine the Behavior Transfer Rate (BTR) for each behavior type introduced in Fig. 5 BTR is calculated as the ratio between the emergence rate of visual behaviors and that of their linguistic counterparts. Formally, we compute the Cognitive Behavior Emergence Rate in the visual modality (CBR,) and in the language modality (CBR)), both evaluated on the multimodal benchmark Math Vision (mini). The BTR is then defined as:
BTR CBR,
CBR₁
This metric reflects the cross-modal transfer efficiency of cognitive behaviors, with higher values indicating stronger behavioral generalization from language to vision.
C.2
Evaluation Prompt
We design prompts for the LLM-based evaluation. Fig. cognitive behavior Backtrucking as an example.
showcases the prompt template for the
Prompt example:
Problem Statement: "<QUESTIONS"
Model's Reasoning: "<REASONING CONTENT>"
Evaluate the textual reasoning for Backtracking g/ /Revisoon. This is when the model, in its text, realizes chosen path, calculation, or assumption is incorrect unproductive, and explicitly states it's trying different approach, redoing calculations with different numbers/operations significantly altering its line of textual reasoning.
Example: "That calculation led to dead end. Let's try subtracting Y from X instead." or "My previous assumption about the sequence was wrong, Starting over with..."
Count distinct instances such explicit textual backtracking or major revision. Provide the count <count> x <</ountt> tags. If none, use <count>o</counts
Figure 7: Prompt design for evaluating Backtracking.
D More Case Studies
We present examples of OVR's reasoning process during complex multimodal tasks. In Fig. 10 the model begins by perceiving the image and applying visual tride-and-conquee interpret each emoji individually. It then engages in subgoal setting, breaking down the task to explore multiple candidate answers based on each emoji. Throughout the process, the model exhibits backtracking, revising earlier hypotheses by cross-verifying them with elements in the image. The reasoning process concludes with final, well-justified prediction.
28

---

Question: Based on the paper, which one takes longer to train, Rt. phose or reward model?
Ground Truth: RL phase takes longer.
OVR: s/thinks Okay, so need figure out whether the R1. phase or the reward model takes longer to train bosed on the information provided in First, the paper mentions Seeda.5 model. It talis about past post-truining framework using bybrid relnforrement learning
Let through the relevant parts
RLVF (Reward Tearning with Verifier Readhuck). The phase is mentioned as requiring z4k GPU bours, while the reward model inference uses 60% GPU hours. Wait, but that's for the "reward model inference replicas each configured with reward model during inference. right? Because model trainod longer? paralleliam." So during training,
with [Reinforcement Learning with Human Fordback)
Wait, let me check again. The paper says: "The Rl, pluse of Seedh 5-VL custs 6ok GPU bours. The reward mudel trained using the same framework as Seed1. 5-V pretraining phase, requiring 24k GPU hours." Hmmm, that seems conflicting. Wait, no: "pectraining phose" is seporate. Lot me this sentence again. "The phase Seods
Walt, the
6ok CPU hours, The reward model is trained using the
Rf, phase takes hours, and reward model training (which is part of the post- training framework) uses 24k hours? That would mean the RL phase is longer than the reward model tralning. But maybe I'm misinterpaeting.
framework as Soed1.5 5-VI, pretraining phase, requiring 24k GPU hours."
Alternatively, porhaps "We use the same training framework as Seeda.5-VL VL pretraining phase for optimication techniques as in the pretraining phuse for efficient actor and critie updates, and vLL. [81] for efficient progresaive generation of rollouts. Specifically, the RL and
typo fonsusion fusion
description.
again
wording:
reward/rwfereeee RIVP training employs 3-D parallelism [61, 108, 126, 141): rollont generation and reward reference model infervnce use replicas configured
tensor parallelism using same frameworrk S-L. pretraining plase. requiting zak GPU hours."
phase 5eedi.5VV custs 6ok GPU hours.
Wait, so here it snya that during Rf. training (which includes both and RIVF?), takes 6ak hours, Then soporately, the reward model trained using the framework pretraining (which might be different) but takes 24k hours. But if than post-
peetraining took
there's mix-up bere...
let
again:
Original test from user input:
"We optimization techniques the pretraining phuse for efficient actor and critic upditess... The RL, phase Seedh.5- VL 6ok GPU
The reward model trained
framework Seedi,p pretraining phase requiring
24k
So during post-training phases:
RL Phase: sok
Reward Model Training 24k
Thus, Rt. Phase is lonper than Reward Model Training.
Therefore, answer would be that the RL Plase takes longer to train than the Reward Model. </thinks
provided information
paper:
RL Phase: Requires ""6ok GPU hours""
Reward Model Training: Requires 24k GPU hours during post-training phases.
Conclusion: The RL Phase takes longer to train compared Reward Model Training(ook vs. 24k UPU hours).
Answer: Phase
Figure 8: Case Study OCR-based Document Understanding. Red highlights some of the cognitive behaviors.
29

---

Question: Jorge's teacher asks him plot all the ordered paizs (x, DJ8 of positive integers for which w& is the width and 18 the longth vertangle with What should his groph look like?
Ground-Iruth:
<Imagels: <imaga0
ximageããe:
<imagedo:
<imaggdd
N0 C
10
A
RR
D
F.
OVR:
ethinks Okay, so Jorge needs to plot area 12, Both w and I are positive integers. Let me think about how appronch this,
pains (ve, where w the
the length rectangle with
First, the of restangle calculated by multiphynng its widtt and length. have to positive integers because they are dimensions a rectongle, and you can't have fraction of unit in this context, right? The problem says positive integers, we're dealing with whole numbers.
have the equation
Both and
Alright, need
pairs pasitiive integers
when multipliid together, they equal Let list
factors of ta. The factors of 12 are 1, a, 3.4 6, and 12. But since both and have to be positive integers, each factor pair would be (112), (2.6), (9:4), and then their reverses. Walt, but since w width and is length, does the order matter? in the ordered pair (w, does matter
switch and For example, (2.6) different from (6.2)? Because the rectangle has longth 2. rotatod. But orderod pair (w. would be
same as roctangle different. However, terms of the graph, since we'ne plotting (w, D), the points would be distinct even they are just swaps. But wait, the area is the same, but the coordinates are different. Himm...
length 6, that's
Therefore, we need to consider all possible ordered pains (w, 1) where and lare positive integens, and w 12. Tet's list sach pains.
Starting with w -- 1 wim
(1,12)
W. ==33"" ".."" 44" 12, (3.4) w 4:44 3- 12, so (4:3)
6:
"1
So ordered pairs (1.18), (2.6), (8.4). (4-3), (6.2), (12,1),
a distinet ordered pair.
So let's visualize this. The x-axis is (width) from to 12, and the y-axis is (length) from 1 to 12. Each of these points should be plotted.
Looking at options given:
(A) images: shows points that seem to go from (1.12) down to (12.1), but maybe in different arrangement.
Walt, but let me cheek the images. (imaget):
plot with points arranged from left to right along the w-axis increusing, and decreasingg So like (1,12),
(3.4). (4.33. (6.2), (12.1). But maybe
points there.
(imagen): points increasing in both dircetions? Not sure.
(imagea): Points decressing in as w incresses, but maybe with gage??
(images): Points all along the w-axis, is constant? That can't be right becausel I would have be for w-a, etc.
(Iangegg): Poluts along the
constant? Simillarly that's not possible.
Wait, but the correct plat should have points at (1,19). (9.6), (3.4), (4.3), (6.2). (12,1), So when plotted on graph with on the x- axis and on the y-axis, these points would form a diagonal line from (1122 (12.1), but only at those specific integer points...
Let's check options again. Asouming imaget other arrangement, image4 all on w-axis, image5 all on l-axis. But since the problem images and images can be eliminated because they have multiple points single determined (exoept when w-12,
six points in decrensing order, imagea might have some oer arman goment, imageg mayhe have some
""pssitive integgessi" and there exactly six pairs, the correct graph should have points, So
possible since each w, vine versa). image4 points along w-axis (I-17), but that's correct. Similarly, images would have points along I-axis (wos I2), which is also not correct. So between imaget, imagez, images.
imaget points (1.12), (2.6), (5.4), (4.3), (6.2), (12,1), then images correct. If images has politts different order, maybe incrensing, but since w is on the s-axis and on y-axis, increusing w would correspond to decressing 1, the points whould form decreasing trend...
Therefore, the correct
images, which option </think>
The problem requires plotting all urdered pairs (w, D) where and are positive integers such that the area (w "I 12). The factor pains of are (1. 12), (2. 6), 4), (4. (6, 2), and ((12, These pains form six distinet points on groph with on the x-axis and lon y-snis. The currect options, "images"" (Option A) correctly displays six points this pattern.
should show all six points arranged in decrensing trend from 12) to Among the
Answer:
Figure 9: Case Study on Math Reasoning, Red highlights some of the cognitive behaviors.
30

---

Question: Which movie do these emojis remind you of?
Ground Truth: Independeace Day
cthinks Okay, let's try figure out which movie these emojis remlnnd me First, I'll list out all the emojis again make sure I have them right. The first row has UPO, a city at night with buildings, and a sun. The second row includes the Statue of Liberty, a sun again, a bridge (probably the Golden Gate), and another sun-like emoji. Hmm, let's break them down extraterrestrial life. city at night could a setting
OVR:
one. The UPO is classic alien spacecraft, crime story or maybe thriller. The sun emojis might
that might hint at movie involving
story that involves weather changes, mayhe a sun that's cither a hero or a villain. The Statue of Liberty a specifie symabl, often representing freedom or the United States. The bridge, especially it'ss the Golden Gate, is famous landmarks in San Prancisco. Putting this together, maybe movie that has UPO, involves city night, and elements related the America, sun
weather changes.
both the US, so parlags the movic is emogis might indicate a weather theme, like a storm or heat wave. Wait, but the son could also be recurring element in story, like in "Twister" where weather is big part, but that's disaster movie. Alternatively. "The Wizard of Oz' has big but that's more of classic fairy tale.
Also, Statue of Liberty and Golden Gate Bridge
tornado and
Museum' where the statue comes to but don't think that's Another possibility is 'Independence Day' which features massive alien spaceship attacking W Washington D.C. The city nightt could be the espital, and the UPO matehes. The son emojis might be part of the movic's imagery, like the allens the U.S. government. The in movie, the attack starts the Midwest and moves towards W Washington D. C. but maybe other parts of the U.S. are involved.
Wait, the Statue Liberty
clue. There's muvie called "Night
osing sunlight or something Also, "Iudependence Day" so maybe attack happeas there?
lot action with the aliens
Liberty is in New
The Golden Gate Bridge is significant location the movie 'San Francisco 1971" which involves killer, but that's different again, could be related to the movie 'The Day After" which is about mucleur attack, but the sun might not fit.
setting. Altematively, "The Blob"
movie where giant blob invados
Wait, the emajis
Another angle: the combination of UPO city at night, and the sun. Mayhe "The Fith Element' has a flying saucer and a futuristic city, but the son lon't main element, The Stotue of Liberty la prominent in "The Happoning" where virus takes over, but that's more about
Walt, the sun emojis might be part of the movie's tille or key clement. For example, "Sunset" or "Sunrise" but those are too generic. Alternatively, "The Solar Storm" but that's not a mavic I know. Let me think movies that have all these elements. "Invasion the Budy Smatchers' has alien-like plants taking peuple, but emojis don't quite match. and boy, but not city at nicht. Wait, maybe "Trunffrmmrr" has sunken city in the movies, like in Transformers: Age of Extinction" there's a sinking city, but the emogis might part the visual officts. The Statue liberty isn't main part that mosie.
a spaceship but not city night. "E.T. the Extra Tervestrial' a flying saucer
Alternatively, "The Matrix" has digital cities and machines, but not UFOs.
Wait, "The Blob" has a giant blub that engallis town, but again, not sure.
Wait, the sun emofit could be rolated to the movie "Twister" which involves aevere thunderstorms and giant tornado. The sun night dun't
might be part the storm system, but UPO and
Wait, 'Independence Day" is big one. The movie about an allen spaceship attacking Earth. specifically the U.S. The attack storts the Midwestt and moves towards the Fast Corst. The city at night could be a reference to the attack on W Washington D.C. something.
The UFO the main alien ship. The emujis might be part of the movie's imagery, maybe the aliens heat symbol, the attack targeting the U.S. that makes sense. Golden Gate Bridge could
Statue of Liberty
part the attackss path.
Let me check the plot of ""Inependence Day" again. The movie features buge alien mothership that lands the Midwost, and the U.S. military trins destroy The atteck depicted with aliens using beat-ray weapons, which might
Washington D.C., which a major city. The Liberty is symbol the U.S. su it's part of the national pride theme. The gulden gate bridge is key location in San Francisco, which ii night be on the attack's path.
look like sunbarsts. The
night could
the
seems fit. All the emojis relate
movie 'Independence Day," Thee sun emojis might represent the heat rays, Statue Liberty U.S. symbol, and
the alien mothership,
night
could Washington D.C.
Goldon Gate Bridge ls part of the attack's movement towards the Fast Coast.
Another possibilitty "The Day After" which about nuclear but the emojis don't 't fit. "2012" about comet, but again, not matching.
So, potting it all together, the emofis are likely from "Independence Day." </thinks
The emajis collectively remind me the movic "Independence Day" (1996). Here's how each emajji aligns with the film:
I.UOO. Represents the massive allen mathership that attacks Earth.
Ciyyty at Night: Refers to the attack un Washington, D.C., a major U.S. city.
3.Sun Fmojis: Symboliaa aliena" heat- weapons, which resemblle bright, radiant barrts. 4.Statue I Liberty: A U.S. leon, emphssizing the movic's forms on defending American soil. G.Golden Gate Bridge: A key geographic reference The combination of alien thrests, T.S. landmarka,, and dramatic imagery matchos the movic's is plot about an extratermestrall invasion.
attack's progression along the Coast.
Answer:
Independence Day
Figure 10: Case Study on Emoji Quiz. Red highlights some of the cognitive behaviors. The model first performs visual decomposition to interpret each emoji individually. It then engages in subgoal setting to explore multiple candidate answers, accompanied by backtracking to revise earlier hypotheses. The reasoning concludes with final, justified prediction.
31

---

Question: number points equidistant
two parallel tangents
(A) (8) (C) 3. (D) (R) (.) infinitely many. (20th American High School Mathematics Examinstion, 1969)
DeepSeek-Rat R1: <think> |nThe problem is from the 20th American High School Mathematics Examination in 1969. It states: 'Thee number of points equidistant from cirele and two parallel tangents infinitely many-An/nkirs, need find points that equidistant from circle and from it aguin.(nnnn"" ("It says: equidistant from a circle and two parallel tangents.)" " think means equidistant from the circle itself and the pair tangents.
circle is!" with options ranging need to understand what is being asked. We have cirele and two parallel tangents to this cirele.
two tangents. The phrasing bit ambiguous, Let
usmally means shortest is the perpendicular distance./pi Bnlt says \'Irom circle and two parallel tangents." \" It might mean equidistant from the cizcle the tangents lines, the circle curve./p nurve.jn/nnPrchap
cireumference,
line
and from each of the tangents, but that doesn't make means that the point is equidistant from the circle and from the two tangents considered together, but that seems messy-In's mnnnncheee interpretation: maybe would prohably be overconstrained. nn recall that in some contexts, l'oquidistant from curve and line)" means that the shortest distance problem intends for the distance to the circle to be equal to the distance to the pair of tangents, but since the tangents are parallel, the distance the pair might the distance the closer
that the distance to the circle equal to the distance to each tangent, but that
equals
line-n here
circle
Perhapps
something, intubat doesn't sound jnght,/nnmLLLet visualize this. Let me sketch a circle. Assume it's circle with center O. Draw two parallel tangents to this circle. Since they are parallel. tangents horizontal, say
on opposite sides the circle. example, the circle centered the origin, y = and y -r, but if the cirele has radius r, and center at (o.o), then the tangents would be and - -12 No. Noljn center at (0.0), radius then the horizontal, So, 1 have circle x + y²2 = r"2, the borizontal tangents are y and y= -r. Is that correct? When y r, x r'2 the point (o.r)
lines
for vertical tangunts, but for horizontal,
want parallel tangents, say both
r"2 implies x=o, the two parallel fangents/(nnnBil in this case, they are symmetric.knn (Noow, need points such that the distance from P to the circle equal the distance from l'equidistant from circle and two parallel tangents
the circle, and the tangent is horizontal, Similarly for y=-r. So yes, yer and ymu
each
langents? that might be what mesns TheeThe problem says
Figure 11: Case Study DeepSeek-R1. Red highlights the mental imagery mentioned in Sec- tion 5.2
32


Reviews:
[{"review_id": "3GznGyCNG3", "reviewer": "Reviewer_WErP", "summary": "This paper investigates the transference of cognitive behaviors from language-only training for vision language models. The authors propose that reasoning skills learned from language-only training can significantly improve a model's performance on visual reasoning tasks. They introduce a three-stage training pipeline to facilitate and study this transfer. The resulting model, named Open-Vision-Reasoner (OVR), is a 7B parameter model based on Qwen2.5-VL.", "questions": "See weaknesses", "limitations": "None", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths:\n1. The central hypothesis—that core, abstract reasoning skills are modality-agnostic and can be transferred from language to vision—is compelling and aligns with theories of human cognition. The initial experiment showing that a language-only \"cold start\" boosts visual reasoning performance (Figure 1) is a strong and surprising finding that provides excellent motivation for the work.\n2. The proposed three-stage training pipeline is logical and well-structured. It allows for a systematic investigation of the central hypothesis and enables a clear ablative analysis (Table 5) to demonstrate the contribution of each stage. This methodical approach adds credibility to the findings.\n3. The paper demonstrates impressive results across a wide range of challenging language and visual reasoning benchmarks. OVR significantly outperforms other open-source 7B models on tasks like MathVision (+15.1%) and LogicVista (+6.5%), providing concrete evidence that the proposed training strategy is effective.\n4. The authors go beyond simply reporting benchmark scores. The analysis of both linguistic (Table 4) and visual-specific (Table 3) cognitive behaviors provides valuable insight into the model's internal changes. The attempt to quantify the \"Behavior Transfer Rate\" (BTR) is a novel contribution towards understanding the mechanisms of cross-modal generalization.\n\nWeaknesses:\n1. The paper provides no validation for the reliability or accuracy of GPT-4o-mini as a classifier for these nuanced behaviors. How often does it produce false positives or negatives? Without this, all quantitative claims about behavior frequency (e.g., in Tables 3 and 4) are built on an unverified foundation.\n2. The analysis cannot distinguish between a model that has genuinely learned a new cognitive process (e.g., verification) and a model that has simply learned to verbalize its outputs in a style that mimics verification, because such patterns were present in its training data (distilled from DeepSeek-R1). The model may be a more sophisticated \"stochastic parrot\" of reasoning steps rather than an actual reasoner. This is a critical distinction that the current analysis fails to address.\n3. The proposed three-stage training pipeline (SFT -> Language RL -> Multimodal RL) is more of a training recipe than a novel method. And such pipeline is well known in the community. The individual components are standard: PPO with GAE is a well-established algorithm, and a minimalist binary reward function for correctness is a common choice in reasoning tasks. While the application to study cognitive transfer is interesting, the work does not introduce a new algorithm, architecture, or fundamental learning principle, which limits its contribution for a top-tier venue.\n4. The paper's own results (Table 5) show a clear and significant trade-off between reasoning and perception. As the model's reasoning score on MME-R increases, its perception score on MME-P and MMVet consistently decreases from the baseline. The authors acknowledge this but frame it as a temporary issue to be addressed in future work. However, for a paper claiming to create a \"general-purpose multimodal reasoning\" system, sacrificing a fundamental capability like visual perception is a major flaw.", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for the detailed and insightful feedback. We appreciate the reviewer's recognition of our central hypothesis as \"compelling\", our methodology as \"logical and well-structured\", our results as \"impressive\", and our behavioral analysis as providing \"valuable insight\".  Below, we carefully address each of your concerns and clarify potential misunderstandings.\n\n# Q1: Evaluation Reliability\n\nWe conducted a **human evaluation** to validate the reliability of GPT-4o-mini as the behavior classifier, focusing on *visual re-inspection*. \n- The results show **high alignment** between the model and human predictions. \n- Moreover, the human annotations further **support the effectiveness of our behavior transfer**.\n\n**1. Setup:** We randomly sampled 100 queries from the MathVision-mini benchmark and collected model outputs from both Qwen2.5-VL-7B and OVR-7B (200 responses in total). These anonymized samples (image, query, response) were shuffled and evaluated on a dedicated annotation platform by five professional annotators with expertise in CV and NLP. Annotators identified whether a given response exhibited visual re-inspection behavior, based on formal definitions and detailed annotation criteria (formatted as prompts in Appendix S-Figure 1).\n\n**2. Metrics:** \n- We first compute the behavior classification accuray of GPT-4o-mini against aggregated human annotations. \n- Additionally, we compared the behavior frequencies of the two models, using GPT and human annotations respectively.\n\n**3. Results:** \n\n- GPT Classification Accuracy: Across all annotated cases, GPT-4o-mini achieved **an accuracy of 92.42%**, showing strong alignment with human judgments. The model produced **only 2 false positives and 13 false negatives**, showing a conservative tendency in behavior detection.  This indicates that GPT-4o-mini reliably captures the overall trend of behavior emergence, supporting its use as a practical and scalable proxy for behavior classification in our study.\n\n- Behavior Frequency Comparison: The table shows the behavior frequency of the two models, as measured by GPT and human annotations respectively. Both sources consistently indicate that **OVR demonstrates a substantially higher behavior frequency than the baseline**, further validating that the behavior was effectively transferred.\n| Model           | GPT (%) | Human (%) |\n|----------------|---------|-----------|\n| Qwen2.5-VL-7B  | 0.0     | 2.0       |\n| OVR-7B        | **10.1** | **19.2**   |\n\n# Q2: Distinguishing Genuine Behavior from Pattern Mimicry\n\nWe appreciate the opportunity to address the possible **misunderstanding**. We emphasize that the defined *visual-specific cognitive behaviors* has been evaluated in the main paper to handle this concern, which are **not present in the training data**. The analysis on emergence of these behaviors demonstrates the **genuinely learned** cognitive process, rather than superficial mimicry. We further clarify the definition and findings as follows:\n\n**1. Visual-specific behaviors defined to capture genuine transfer:**\nTo assess whether the model genuinely learns and transfers cognitive behaviors, we examine visual-specific behaviors grounded in visual inputs (defined in Sec. 5.4 and detailed in Appendix C.1). While these behaviors are conceptually related to their linguistic counterparts (explained in S-Table 1), they are **fundamentally different** in modality, making them well-suited for verifying true cross-modal transfer rather than superficial mimicry.\n\n**2. Zero-shot emergence confirms internalization and transfer:**\nThese behaviors are entirely **absent from all training data**. Even During the multimodal RL, the model only receives ground truth answers without any intermediate reasoning steps or behavioral scaffolding. Nevertheless, as shown quantitatively in Table 3 and qualitatively in Figure 2, these behaviors consistently emerge. This strongly suggests that the model has **internalized and strategically transferred abstract reasoning strategies to the visual domain**, rather than mimicking surface-level text artifacts. We consider this one of the most valuable findings of our work.\n\nWe will further clarify this in the revision to avoid any ambiguity.\n\n# Q3: Contribution Clarification\n\nWe acknowledge that *“RL with a cold start”* is a well-established paradigm in LLM training. In our work, this pipeline is not positioned as the core novelty, but rather **serves as a tool** to systematically investigate the cross-modal transfer of cognitive behaviors and to build a strong multimodal reasoning model (lines 37–44).\n\nWe further **clarify our distinct contributions** as follows:\n\n1. **Cross-modal cognitive behavior transfer as the central focus:** Our work uniquely centers on **the unexplored behavior transfer from language to vision**, revealing the internal mechanisms and generalization patterns behind advanced reasoning capabilies. This enables a scalable and interpretable pathway for building stronger multimodal reasoning systems. \n2. **In-depth behavioral analysis:** We provide the **first quantitative study** of behavior emergence across training stages, including the surprising zero-shot emergence of visual-specific cognitive behaviors (see Q2 response), transfer patterns (Sec. 6), and the correlation between behavior and performance (see Q3 response to Reviewer SEpa). This sheds light on how abstract behaviors generalize across modalities and benefit model's reasoning capabilities.\n3. **Superior reasoning across modalities:** OVR achieves **exceptional performance on both text and multimodal reasoning**, as well as on general-purpose benchmarks, demonstrating the value of modality-agnostic cognitive behaviors for strong reasoning capabilities. Notably, it establishes a powerful and reproducible baseline for the open-source community, with full data, model and implementation details to be released publicly.\n\n# Q4: On the Reasoning-Perception Trade-off\nWe would like to emphasize that the **primary goal** of this paper is to **enhance multimodal reasoning**. Within this context, the **partial** decline in perception metrics is **acceptable and solvable**, and therefore should not be considered a “major flaw”. We address this concern through four key aspects: \n\n**1. No \"consistent degradation\" in perception**\n  - Contrary to the reviewer's comments, MMVet scores in Table 5 **actually increase** from 57.3 to 60.8.\n  - We also evaluate on more perception-centric benchmarks, and the results show that OVR maintains or even **improves** performance across these tasks. \n| Model            | BLINK  | HallusionBench | MMbench |\n|------------------|--------|----------------|---------|\n| Qwen2.5-VL-7B    | 53.71 | 49.43         | 85.84   |\n| OVR-7B           | **54.13** | **54.90**         | **86.42**   |\n\n**2. Acceptable trade-off for the reasoning-focused OVR:**\nGiven our framework’s **emphasis** on language and multimodal **reasoning**, some marginal drop on perception metrics is an acceptable trade-off. While MME-P shows a decline of only 7.3%, reasoning metrics improve more substantially. For instance, AIME24 increases from **6.67 → 58.8** and MathVision from **25.5 → 50.0**, demonstrating a strong overall capability gain.\n\n**3. Solution through Scaling:**\nAs discussed in line 280-282, we further scale up multimodal RL, leading to a gradual recovery in perception metric **MME-P 1547→1600** and a increase in reasoning metrics like MME-R 713.6 → 725. This suggests that perception degradation is not fundemental, but **solvable** through continued training. \n\n**4. A well-studied and explainable phenomenon in the community**\n - Perception-reasoning trade-off is **a well-recognized phenomenon** in MLLM reasoning [1]. The observed decline in perception performance often arises from reduced reliance on fine-grained visual details, as models develop longer and more abstract reasoning chains that favor linguistic priors. \n - We plan to address this in future work via **prolonged multimodal RL and self-distillation**, aiming to jointly reinforce reasoning and perception abilities.\n\n[1] More Thinking, Less Seeing? Assessing Amplified Hallucination in Multimodal Reasoning Models"}, {"review_id": "nkisJX8a8t", "reviewer": "Reviewer_9Ror", "summary": "It proposes a 3-stage training pipeline for multi-modal reasoning, where cold start training for linguistic reasoning happens first, followed by RL for linguistic reasoning and finally multi-modal reasoning.", "questions": "- According to this paper, most transfer happens after the cold start training – is it because of the relative size of the data, the training method or something else? \n- The paper describes MMVet as a perception-centric benchmark, but it also includes many questions e.g. in the math split that are reasoning-heavy. As for perception-centric benchmark, What about performance on CV Bench and BLINK?\n- Are the findings generalizable to other models, other data sizes (e.g. larger data for stage2 or stage3 RL)?", "limitations": "Yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths:\n- Comprehensive evaluations\n- Strong performance compared to the baselines\n- Interesting research angle: it investigates an interesting question, even though the results are not sufficient to support the general claims made in this paper\n\nWeaknesses:\n- A big part of this submission seems to be a study contribution, but it lacks comparison to other training pipelines e.g. (1) Language cold start + multi-modal RL, (2) language cold start + multi-modal cold start + multi-modal RL; and additional settings that could be interesting to compare to e.g. (3) multi-modal cold-start + multi-modal RL; (4) language cold-start only with more training data; (5) multi-modal cold-start only with more training data\n  - The reviewer is curious what the emergence of cognitive behaviors look like for these settings above? \n- According to this paper, most transfer happens after the cold start training – is it because of the relative size of the data, the training method or something else? \n- The paper describes MMVet as a perception-centric benchmark, but it also includes many questions e.g. in the math split that are reasoning-heavy. As for perception-centric benchmark, What about performance on CV Bench and BLINK?\n- Lack of generalization: Are the findings generalizable to other models, other data sizes (e.g. larger data for stage2 or stage3 RL)? \n\nMinor: In the reward design, using exact matches for reward=1 seems a bit strict? Have the authors considered alternative designs?", "was_revised": true, "final_justification": "Thank you authors for updating the results with more ablations and benchmarks!\nThey address most of my questions/concerns, so I'm updating my rating to 4. \n\nHowever, the following questions remain:\n- The authors mentioned that multi-modal cold start data hurts performance due to their lower quality: can the authors provide a more in-depth analysis on what exactly leads to the low quality? Also, it's mentioned multi-modal and language only cold start data have conflicting formats -- would the performance improve in this mixed setting if the formats are unified or if these data are used in separate stages? \n- The authors mentioned the results hold on a larger Qwen VL model, what about models with a different language (e.g. llama) and vision backbone? \n\nIt'd be great if the authors could address these in the final version.", "rebuttal": "We thank the reviewer for the constructive feedback and recognition of our paper’s strong performance and interesting research angle. We carefully address each of your concerns below.\n# Q1: More Ablations\nWe conducted comprehensive ablations along three dimensions: (1) different cold-start data strategies, (2) RL training schedule, and (3) enhanced cold start with self-distilled multimodal data. \n\nThe results in the table are summarized:\n- Adding open-source multimodal cold-start data or applying mixed RL stages **does not outperform** our proposed training pipeline.\n- Introducing **self-distilled data from OVR** into cold start leads to improved performance.\n\n||AIME2024|MATH500|MathVista|MathVision|\n|-|-|-|-|-|\n|Baseline|6.67|67.4|69.2|25.5|\n|**Part I: Cold start**|||||\n|Lang cold start|54.4|93.1|69.7|46.2|\n|MM cold start|23.1|75.2|71.6|33.5|\n|Lang cold start + MM cold start|51.0|90.3|70.4|40.1|\n|**Part II: RL on language cold start**|||||\n|MM RL|55.0|92.9|72.0|46.5|\n|Lang RL|57.9|93.6|71.6|47.5|\n|Mixed RL|58.0|94.4|72.4|49.0|\n|**Part III: Cold start with extra data**|||||\n| Lang cold start + self-distilled mm data|59.1|93.9|72.7|50.5|\n|**Current OVR**|||||\n|Lang cold start + Lang RL + MM RL|58.8|94.1|72.9|50.0|\n\nThe ablation details are shown below:\n\n**1. Ablations on Cold Start** (Part I)\n  - Settings: We experiment with training on multimodal cold-start data either independently or in combination with language cold-start data. The multimodal cold-start datasets contain publicly available sources such as R1-OneVision [1] and Vision-R1 [2], which are constructed by either **synthesizing image-text pairs** using image descriptions and language reasoning models or **rendering** language cold-start data into multimodal form.\n  - Results & Analysis: \n    - Training with multimodal cold-start data alone leads to lower performance on both language and multimodal reasoning tasks, particularly in language reasoning (e.g., AIME2024: 23.1 vs. 54.4). This reveals the **lack of high-quality multimodal cold-start datasets**. The much better performance on MathVision also verifies that language reasoning skills are fundemental for advanced multimodal reasoning, and demonstrates the robustness of our data curation pipeline.\n    - Adding multimodal cold-start data on top of language cold-start training reduces performance. This arises from conflicting reasoning patterns: multimodal cold-start data has obvious differences in style, length and structure compared to the language-only version. Nevertheless, this challenge can be addressed through large-scale RL and iterative self-distillation.\n  - Discussion: **Constructing effective multimodal cold-start data remains a significant challenge.** Unlike the language domain, the multimodal community lacks high-quality, open-source reasoning teachers like DeepSeek-R1, and most closed-source models hide explicit reasoning traces. **Our method strategically avoids this limitation** by leveraging a strong language cold-start phase followed by multimodal RL, resulting in a robust and scalable multimodal reasoner.\n\n**2. Ablations for RL stages** (Part II)\n  - Settings: Based on the language cold start , we ablate three RL strategies: (1) language-only RL, (2) multimodal-only RL, and (3) a one-stage mixed RL that combines both.\n  - Results & Reasons: **The two-stage RL setup consistently yields the best performance** across reasoning benchmarks. From a **training efficiency perspective**, separating the stages is also more practical: language RL involves longer and more complex responses, requiring longer training time; multimodal RL, while not trivial, requires shorter response length and exhibits less scaling complexity (as discussed in line 298-303), resulting in faster convergence. Notably, recent works such as *NemoTron-1.1* [3] have adopted similar staged training strategies, achieving promising results.\n  - Discussion: While mixed RL is a valid alternative, it introduces greater training cost without clear advantages. **Further scaling with prolonged RL** has the potential to amplify the differences between mixed and separate RL stages and leads to stronger results, though it would require significantly more computation resources.\n\n**3. Enhancing Cold Start with Self-Distilled Multimodal Data** (Part III)\n  - Setting: To address the reviewer’s suggestion on “(5) multimodal cold start with more data,” we **augment the cold start with self-distilled multimodal data generated by our own OVR model**. As discussed in Point 1, the community currently lacks high-quality multimodal cold-start data and is challenged in constructing. Thus, we leverage the model’s own capabilities to distill high-quality reasoning traces, and train with a combination of language-only cold-start data and self-distilled multimodal samples.\n  - Reults & Analysis: Incorporating self-distilled data during cold start leads to **better performance** in both language and multimodal reasoning tasks, faster convergence, and lower training loss. These demonstrate the effectiveness of self-distillation [4] in bootstrapping multimodal reasoning. Moreover, they point to a promising direction: in the absence of strong open-source multimodal teacher models, **iterative RL and self-distillation** can serve as a viable and scalable strategy for progressively enhancing model capabilities.\n\n**4. Cognitive Behaviors on ablations:**\n  Given that behaviors primarily emerge during cold start, we analyze their development across the three cold start variants. Consistent with final performance metrics, the multimodal cold start does not yield an increase in either textual or visual cognitive behaviors. We attribute this to the following factors:\n  - **Multimodal cold-start data still suffers from limited quality**. The synthetic nature fails to introduce new reasoning patterns, which even hurts performances.\n  -  **Visual behaviors tend to emerge in the early steps** of cold start and plateau thereafter (visual re-inspection: 0.3 → 2.2 → 2.0 from iter 0 → iter 200 → iter 700 in OVR). Since multimodal cold start data is derived from language-only sources, the underlying patterns remain unchanged. As a result, simply adding more data, regardless of modality, does not further enhance the emergence of cognitive behaviors.\n\n[1] R1-Onevision: Advancing Generalized Multimodal Reasoning through Cross-Modal Formalization \\\n[2] Vision-R1: Incentivizing Reasoning Capability in Multimodal Large Language Models\\\n[3] AceReason-Nemotron 1.1: Advancing Math and Code Reasoning through SFT and RL Synergy\\\n[4] DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning\n\n\n# Q2: Cause of Transfer after Cold Start\nOur analysis suggests that the transfer is primarily driven by the nature of cold start data, rather than data volume.\n\n**1. Transfer originates from linguistic “mental imagery”:** The linguistic data distilled from DeepSeek-R1 often include internal visualizations (somehow reveals its mental imagery) to support mathematical reasoning, which are often articulated through phrases such as “let me visualize/sketch...”. Once this linguistic scaffolding was introduced into MLLM, these patterns were rapidly grounded in actual visual input for handling multimodal reasoning tasks.\n\n**2. Emergence happens early and is not driven by data size:** We track the emergence of visual behaviors across different iterations within the cold start. Specifically, we observe that a significant portion of visual behavior occurs at the early step and then fluctuates rather than steadily increasing (0.3 → 2.2 → 2.0 from iter 0 → iter 200 → iter 700). This suggests that the effect is not proportional to data size, but instead reflects early inductive bias amplification.\n\n# Q3: More Perception-centric Evaluations\nWe evaluate on more perception benchmarks. OVR achieves consistent improvements than the baseline in the table.\n|Model|BLINK|HallusionBench|MMbench|\n|-|-|-|-|\n|Baseline|53.71|49.43|85.84|\n|OVR-7B|54.13|54.90|86.42|\n\n# Q4: Generalization across Models and Data Sizes\nOur extended experiments support the core finding—cognitive behaviors learned in language can be transferred to vision for advanced visual reasoning—**generalizes well** across both **larger model scales** and **more data with broader task coverage**.\n\n**1. Model Scaling:** Our training on an in-house MLLM built upon Qwen2.5-**32B** confirms strong scalability, boosting AIME 2024 performance from **13 → 73** and MathVision from **35 → 60**. \n\n**2. Task & Data Scaling:** Rather than simply increasing data volume, we also emphasized **task diversity and data quality**. By adding just ~10k high-purity STEM samples to the RL training, we achieve a **+2% increase on MathVision**. \n\nThese results indicate that the benefits of behavior transfer not only generalize but also amplify with larger model capacity and curated data, reinforcing the robustness and scalability of our approach.\n\n# Q5: Reward Design\nUsing exact match as the reward function is well-suited to the nature of the studied tasks and offers robustness to reward hacking.\n\nFirst, the tasks we focus on includes multimodal math, logic, and VQA tasks, which typically have **objectively correct answers**. Exact match provides the most direct and reliable correctness signal.\n\nSecond, from a broader methodological perspective, exact match is a widely adopted reward shaping in reasoning tasks. It offers an unambiguous reward signal, thereby effectively **mitigating reward hacking**. This choice is consistent with mainstream methods such as Open-Reasoner-Zero [5].\n\nWhile we agree that reward design is an important area for future exploration, it is not the focus of this work. In future studies, we plan to investigate hybrid reward strategies that integrate model-based feedback and rule-based signals to further enhance robustness.\n\n[5] Open-Reasoner-Zero: An Open Source Approach to Scaling Up Reinforcement Learning on the Base Model"}, {"review_id": "ukPrEQhfD5", "reviewer": "Reviewer_SEpa", "summary": "This paper presents an interesting observation that the cognition behaviors trained from text-only cold start can transfer to vision reasoning tasks. Based on the finding, the authors design a three-stage training pipeline on cross-modality cognitive behavior training. The resulting model demonstrates strong reasoning ability across modalities. An in-depth analysis is also conducted to improve the understanding of multi-modal reasoning.", "questions": "1. What will be the performance if we train the model only with stage 2/3 or stage 2+3. Are all the 3 stages necessary?\n2. In table 2, many results on the section of \"RL-based methods\" are missing. What's the average gain of the model on vision-reasoning tasks and text-reasoning tasks, respectively?\n3. What's the relationship between cognition behaviors and reasoning? Does more cognition necessarily lead to better reasoning performance?", "limitations": "yes", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strength:\n1. This paper is clearly written and well-motivated\n2. The observation about cognitive behavior transfer is interesting\n3. The resulting model demonstrates strong performance on reasoning problems in both modality\n4. An in-depth analysis is provided about the cognition behavior transfer\n\nWeakness:\n1. The training framework is not well-explained. In Section 4.1, the authors propose a three-stage training, including  language-only cold start, language-only RL and multimodal RL. It's unclear why each stage is necessary, as we know that both \"language-only cold start\" and \"language-only RL\" incentivize the cognitive behaviors in reasoning. \n2. Lack of ablations/baselines. Build upon point 1, more ablation studies are needed to understand the training framework. For example, how much is the average gain on vision reasoning tasks, if we compare (state 1 + stage 2 + stage 3) with (stage 2 + stage 3) or stage 2/3 only training. Also, it'll be interested to see from empirical results whether stage 3 could enhance LLM reasoning or not.\n3. Lack of discussion and analysis. This paper contains a lot of discussion on cognition behaviors but lack discussion on the relationship between cognition behaviors and reasoning.", "was_revised": true, "final_justification": "My concerns have been addressed by the authors' response and I believe this paper will be valuable to the community. Therefore, I'm overall positive to this paper.", "rebuttal": "Thank you very much for the constructive and encouraging feedback. We appreciate your recognition of the clarity of our writing, the significance of our cognitive behavior transfer findings, and the value of our analysis. We carefully address each of your concerns as follows.\n# Q1: Necessity of the Three-stage Training & Supporting Ablations\nWe conduct detailed ablations on RL training stages. As shown below and in Table 3 and Table 5, the results highlight the critical role of each component. Key observations include:\n- **Our two-stage RL training pipeline outperforms single-stage or mixed-stage RL settings**, achieving the best average results on both language and multimodal reasoning tasks.\n- **Stage 3 further enhances both language and multimodal reasoning**, while also promoting the emergence of more visual cognitive behaviors as shown in Table 3.\n\n|                              | AIME2024 | MATH500 | MathVision | MathVerse |\n|------------------------------|----------|---------|-------------|------------|\n| Language Cold Start          | 54.4     | 93.1    | 69.7        | 46.2       |\n| +Language RL                 | 57.9     | 93.6    | 71.6        | 47.5       |\n| +Multimodal RL               | 55.0     | 92.9    | 72.0        | 46.5       |\n| +Mixed RL                    | 58.0     | **94.4**| 72.4        | 49.0       |\n| +Language RL & Multimodal RL   | **58.8** | 94.1    | **72.9**    | **50.0**   |\n\n**1. The experiments results:**\n - Our proposed training pipeline, **incorporating both Stage 2 and Stage 3, outperforms the mixed RL setting**, yielding improvements on vision reasoning tasks such as MathVision (+0.5) and MathVerse (+1.0).\n - **Each stage contributes positively** across multiple reasoning benchmarks, validating the necessity of our training approach.\n - Performing Stage 3 after Stage 2 not only enhances visual reasoning and elicits more visual cognitive behaviors (as shown in Table 3), but also further **boosts language reasoning**, with gains on AIME2024 (+0.9) and MATH500 (+0.5).\n\n**2. Discussion on Two-stage RL vs Mixed RL:**\n - Beyond performance, separating the stages is also more practical **from a training efficiency perspective**: language RL involves longer and more complex responses, requiring longer training time; multimodal RL, while not trivial, requires shorter response length and exhibits less scaling complexity (as discussed in line 298-303), resulting in faster convergence. Notably, recent works such as *NemoTron-1.1* [1] have adopted similar staged training strategies, achieving promising results.\n - Discussion: While **mixed RL** is a valid alternative, it **introduces greater training cost without clear advantages**. Further scaling with prolonged RL has the potential to amplify the differences between mixed and separate RL stages and lead to stronger results, though it would require significantly more compute resources.\n\n[1] AceReason-Nemotron 1.1: Advancing Math and Code Reasoning through SFT and RL Synergy\n\n# Q2: Comparison to RL-Based Methods\nWe complement evaluations for representative RL-based baselines in the table. Our method achieves **average gains of 6.82** on vision reasoning tasks** over these RL-based baselines and **25.95 on text reasoning tasks** over the base model, demonstrating its effectiveness across modalities.\n\n| Model                   | MathVista | MathVision | MathVerse | DynaMath | WeMath | LogicVista | MMMU-Pro | CharXiv-reas | CharXiv-desc | Avg   |\n|------------------------|-----------|------------|-----------|----------|--------|------------|----------|---------------|---------------|--------|\n| VLAA-Thinker-Qwen2.5-7B| 68.0      | 26.4       | 48.2      | 22.4     | 41.5   | 48.5       | 33.20    | 36.80         | 63.92         | 43.21 |\n| OpenVLThinker-7B       | 70.2      | 25.3       | 47.9      | 19.36    | 36.86  | 46.09      | 36.97    | 38.30         | 63.45         | 42.71 |\n| MM-Eureka-7B           | 73.0      | 26.9       | 50.3      | 24.95    | 38.1   | 48.99      | 38.42    | 39.60         | 65.30         | 45.06 |\n| OVR-7B                 | 72.9      | 50.0       | 52.9      | 32.7     | 38.2   | 55.0       | 37.8     | 43.4          | 75.6          | **51.88** |\n\n# Q3: Relationship between Behaviors and Reasoning Performance\nWe analyze both stage-wise and intra-stage correlations between cognitive behavior frequency and reasoning performance to address this question. The key conclusions includes:\n- Cognitive behavioris a **necessary but not sufficient condition** for strong reasoning abilities. \n- Higher behavior frequency **does not necessarily** imply better reasoning, as performance also depends on the task-alignment and **effectiveness of the emerging behaviors**.\n\nThe detailed observations and analysis are as below:\n\n**1. Strong stage-wise correlation**\n  - Observation: Across training stages, both the visual re-inspection behavior and reasoning performance increase progressively. The behavior frequency grows from 0.0 → 2.0 → 2.4, while MathVision performance improves from 25.5 → 46.2 → 47.5.\n  - Analysis: This stage-wise trend suggests a strong **positive correlation** between cognitive behavior frequency and reasoning capabilities. However, correlation does not imply direct causation. We futher analyze intra-stage performances as follows.\n\n**2. Intra-stage divergence highlights behavior effectiveness**\n  - Observation: \n    - During cold start, behavior frequency increases rapidly at first and then plateaus with fluctuations (e.g., visual re-inspection behavior: 0.3 → 2.2  → 2.0). \n    - In Multimodal RL, we observe an initial drop in behavior frequency, followed by a gradual recovery (2.4 → 0.5 → 2.5), yet reasoning performance continues to climb throughout. \n  - Analysis: This divergence reveals that a greater number of behaviors do not necessarily lead to better reasoning. In early Cold Start, the model generates a wide range of exploratory behaviors, regardless of whether they are redundant or effective. As training progresses, we observe that behavior frequency plateaus in late cold start and drops at the start of RL, yet performance continues to improve. This indicates that gains are driven more by **behavior effectiveness** than by behavior quantity.\n  - Further discussion: In the RL stage, the training initially reshapes the model's behaviors to maximize rewards and suppresses ineffective reasoning paths, leading to a temporary drop in behavior frequency. However, the recovery shows RL critically discerns and scales up effective patterns (the crucial tokens analyzed in [2]). This aligns with the insight that *SFT memorizes while RL generalizes* [3].\n\nWe will add the correlation analysis in the revision.\n\n[2] Reasoning with exploration: An entropy perspective\\\n[3] SFT Memorizes, RL Generalizes: A Comparative Study of Foundation Model Post-training"}, {"review_id": "673xIrP1Vn", "reviewer": "Reviewer_bA1c", "summary": "In this paper, the authors observe that reasoning capabilities can be transferred from language-only to vision-language models, leading to significant gains on visual reasoning tasks. This leads the authors to propose a 3-stage training pipeline for VLMs, encompassing 1) fine-tuning the LM module on language-only reasoning traces, 2) PPO-based RL to refine the LM outputs, and 3) PPO-based tuning of the full model on a curated multimodal reasoning set to improve the cross-domain transfer of reasoning capabilities. This pipeline is used to tune an existing foundational model, Qwen2.5-VL, into a model for visual reasoning dubbed OVR-7b. Empirically, the authors show that this model achieves consistently better results on a range of visual reasoning benchmarks compared to other models of the same size. Analyzing the reasoning traces with GPT-4o-mini, the authors also show that steps (1)-(3) increase the occurrence of “cognitive behaviors” (e.g., re-inspection or tracing) in the reasoning traces of the model. Finally, the authors also present some ablations on different secondary questions, e.g., if the additional training is detrimental for the language/vision capabilities (unrelated from reasoning) or at what step of the procedure the transfer starts happening.", "questions": "1. How do you justify the superior performances of OVR-7b on the language-only benchmarks in Table 1? For instance, why do you think it is the case the OVR-7b improves over the standard Qwen2.5-7b model on MMLU accuracy by 5.4?\n2. The ablation on the emergence of visual-specific cognitive behaviors is only performed on MathVision; would it be possible to extend it to the other visual reasoning datasets that are considered in Table 2 as well?\n3. Related to the ablation of steps (1)-(3), could you also provide an ablation not just on the presence of cognitive behaviors in the reasoning steps, but also on the actual performance on the different multimodal reasoning tasks? What I am missing right now is a table to quantify the progression from Qwen2.5-VL-7B to OVR-7b in terms of performances on the benchmarks for the different steps of your proposed pipeline.\n4. Could you reproduce/collect the results on all the datasets in Table 2 for Qwen2.5-VL-7B? This is the base model from which OVR-7b is obtained, and I believe it to be important to have a full picture of its performance.\n5. I think this missing reference could be quite relevant: VL-Rethinker: Incentivizing Self-Reflection of Vision-Language Models with Reinforcement Learning, Wang et al, 2025. Could you please comment on the differences between their and your work?", "limitations": "Yes.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 3, "originality": 2, "strengths_and_weaknesses": "**Strengths**\n\nThe paper is overall well-written, and the narration is easy to follow. The methodology proposed is sufficiently motivated and I am not aware of other works that proposed the same methods before. The authors present empirical evidence on the validity of their method. These includes benchmarking on standard language-only reasoning (mostly mathematical, AIME/MATH) and general (MMLU) benchmarks, as well a wide range of visual reasoning benchmarks (again, mostly mathematical-based reasoning). Furthermore, the paper also includes a quantification of the improvement of cognitive behaviors (through an automatic analysis of the reasoning traces), that appear to be consistently growing thanks to the 3-stage tuning procedure proposed in the paper.  \n\n\n**Weaknesses**\n\nThe main weakness that I can identify in this paper is the limited novelty factor of the proposed methodology. The three-stage training pipeline proposed to study the transfer of cognitive behaviors from language to multimodal inference is not characterized by significantly novel ideas compared to the techniques already present in the literature. Possibly, step (3) of the proposed pipeline (i.e., the multimodal RL for cross-modal adaptation) is the only original step in the method (or at least, I am not familiar with any other works doing it). On another note, I believe that the observation that improvements on mathematical and coding reasoning (resulting from RL on language-only data) can also directly benefit multimodal reasoning tasks is interesting but somehow expected.", "was_revised": true, "final_justification": "The study of cross-modal cognitive behavior transfer is principled and supported by an extensive empirical evaluation.\nThe authors provided additional experimental results during rebuttal that increased my confidence in the validity of their evaluation and claims.\nOverall, the quality of the submission is high, but some intrinsic factors (e.g., to what extent an increase in the occurrence of cognitive abilities can effectively improve reasoning performance, also mentioned by reviewer SEpa, and the overall methodological novelty compared to previous works) prevent me from recommending a full acceptance of the paper. \nIf there were an option between \"borderline accept\" and \"accept\" (as it was, for example, the \"weak accept\" score last year), I would have increased my rating to that.", "rebuttal": "Thank you very much for the thoughtful and constructive review. We are encourged by the recognition of our paper’s clarity, comprehensive evaluation, and the value of our empirical study. We carefully address each of your concerns as below:\n\n# Q1: On the Superior Performance on Language Benchmarks\nThe strong language reasoning performance of OVR-7B can be attributed to the **behavior-rich nature of the language data** and the **structure of our training pipeline**:\\\n**1. Language Cold Start** exposes the model to diverse and high-quality language reasoning traces rich in cognitive behaviors, laying a solid foundation for general reasoning.\\\n**2. Language RL** serves as a cognitive scaffold, reinforcing these behaviors and aligning them exact-match reward signals.\\\n**3. Multimodal RL** does not degrade language reasoning abilities. As shown in Lines 283–287, the model even slightly improves on language benchmarks after Stage 3, suggesting that multimodal adaptation complements, rather than interferes with, the reasoning skills acquired during earlier stages.\n\n# Q2: Extending Behavior Analysis on MathVerse\nWe have extended the same visual cognitive behavior analysis to MathVerse-MINI, following the methodology used in the main paper. The results show **a consistent trend in the emergence and amplification of cognitive behaviors** across training stages, supporting the generality of our findings across different tasks.\n|                     | Re-inspection | Decomposition | Cross-checking | Tracing |\n|---------------------|---------------|----------------|----------------|---------|\n| Baseline            | 0.00          | 4.33           | 0.00           | 0.00    |\n| +Cold Start         | 20.33         | 4.00           | 2.00           | 0.33    |\n| +Language RL        | 25.33         | 4.66           | 2.33           | 1.33    |\n| +Multimodal RL      | **29.33**         | **5.00**           | **2.67**           | **1.33**    |\n\n# Q3: Ablation on Multimodal Reasoning Performance Across Stages\n**Table 5** presents the ablation results across training stages. To further address your concern, we have now **extended the evaluation** to include more benchmarks. These results further validate the contribution of each stage.\n|                 | MathVerse | DynaMath | WeMath | LogicVista |\n|-----------------|-----------|----------|--------|------------|\n| Baseline        | 41.1      | 21.8     | 31.2   | 47.9       |\n| +Cold start     | 52.28     | 31.54    | 33.71  | 52.57      |\n| +Language RL    | 49.62     | 31.94    | 36.1   | 52.35      |\n| +Multimodal RL  | **52.9**      | **32.7**     | **38.2**   | **55.0**       |\n\n# Q4: Qwen2.5-VL-7B Performance in Table 2\nThe performance of Qwen2.5-VL-7B is reported in Row 6 of Table 2. We have now added its evaluation on MMMU-Pro (38.3). OVR-7B achieves a **7.11 average improvement** over the base model (44.45 --> 51.66) across all multimodal benchmarks in Table 2.\n\n# Q5: Difference with VL-Rethinker\nWhile both VL-Rethinker and our method leverage RL to enhance reasoning in MLLMs, our OVR is distinct in motivation, analytical focus, performance, and methodology:\n\n**1. Cross-modal cognitive behavior transfer as the central goal:** VL-Rethinker focuses on reflective fine-tuning within a vision-language setting to enhance multimodal reasoning. In contrast, our work uniquely centers on **how cognitive behaviors acquired from language-only training can transfer to visual reasoning**, revealing the **internal mechanisms and generalization patterns** behind advanced reasoning capabilies. This enables a scalable and interpretable pathway for building stronger multimodal reasoning systems.\n\n**2. In-depth behavioral analysis:** We provide the **first quantitative analysis** of cognitive behavior emergence and cross-modal transfer, systematically tracking behavior shifts across stages. Notably, we discover that visual-specific cognitive behaviors can emerge in a zero-shot manner, even though they do not appear in the training data. This angle is not investigated in prior works.\n\n**3. Strong reasoning across modalities:** Unlike prior approaches that focus solely on visual tasks, OVR also achieves **SOTA performance** across a suite of **language reasoning benchmarks**, demonstrating the value of modality-agnostic cognitive behaviors for strong reasoning capabilities.\n\n# Q6: Addressing Mentioned Weaknesses & Clarifying Contributions\nWe acknowledge that *“RL with a cold start”* pipeline is a well-established paradigm in LLM training. In our work, this pipeline is not positioned as the core novelty, but rather **serves as a tool** to systematically investigate the cross-modal transfer of cognitive behaviors and to build a strong multimodal reasoning model (lines 37–44).\n\nWe further clarify our distinct contributions as follows:\n\n**1. Cross-modal cognitive behavior transfer as the central focus:** Our work uniquely centers on the **unexplored behavior transfer from language to vision**, revealing the internal mechanisms and generalization patterns behind advanced reasoning capabilies. This enables a scalable and interpretable pathway for building stronger multimodal reasoning systems. \n\n**2. In-depth behavioral analysis:** We provide the **first quantitative study** of behavior emergence across training stages, including the surprising zero-shot emergence of visual-specific cognitive behaviors, transfer patterns in Sec. 6, and the correlation between behavior and reasoning performance (see Q3 response to Reviewer SEpa). This sheds light on how abstract behaviors generalize across modalities and benefit model's reasoning capabilities.\n\n**3. Superior reasoning across modalities:** OVR achieves **exceptional performance on both text and multimodal reasoning**, as well as on general-purpose benchmarks, demonstrating the value of modality-agnostic cognitive behaviors for strong reasoning capabilities. Notably, it establishes a powerful and reproducible baseline for the open-source community, with full data, model and implementation details to be released publicly."}]

Author rebuttal:

Meta-review: This paper explores how cognitive behaviors learned from language-only training can transfer to vision-language models. The authors posit that reasoning skills acquired in the language domain can meaningfully improve performance on visual reasoning tasks. To investigate this, they design a three-stage training pipeline and present Open-Vision-Reasoner (OVR), a 7B-parameter model based on Qwen2.5-VL.

Overall, reviewers are generally positive about the work. Two points, however, deserve attention. First, a potential trade-off exists between reasoning and perception. The authors provide additional experiments indicating that perceptual performance is minimal and can largely be neglected. Second, the analysis of correlation versus causation between cognitive behaviors and reasoning remains limited; the authors plan to expand this in the final version.

I suggest the authors further discuss these aspects, particularly the limitations. Overall, I think the paper meets the NeurIPS standard, and addressing these points in the final version would strengthen it further.
Decision: Accept (poster)