SoTA with Less: MCTS-Guided Sample Selection for Data-Efficient Visual Reasoning Self-Improvement
Authors: Xiyao Wang
Zhengyuan Yang
Chao Feng
Hongjin Lu
Linjie Li
Chung-Ching Lin
Kevin Lin
Furong Huang
Lijuan Wang
Venue: NeurIPS 2025 spotlight ()
Keywords: vision language model; reinforcement finetuning; vlm reasoning; data selection
Primary area: applications

Abstract: We introduce ThinkLite-VL, a family of visual reasoning models that achieve state-of-the-art (SoTA) performance using an order of magnitude fewer training samples, relying purely on reinforcement fine-tuning (RFT) self-improvement without any knowledge distillation. Our central insight is that sample difficulty critically influences RFT effectiveness: appropriately challenging examples can drive substantial reasoning improvements, even in low-data regimes.   However, quantifying sample difficulty in a reliable and scalable manner remains non-trivial. To address this, we repurpose Monte Carlo Tree Search (MCTS) to measure sample difficulty via the number of reasoning iterations a vision-language model (VLM) requires to solve each instance. This MCTS-based selection procedure identifies samples that induce deeper reasoning while remaining solvable, allowing us to filter a high-quality subset from 70k open-source examples spanning math, natural image understanding, and chart comprehension.  Using this approach, we select just 11k challenging samples for RFT on Qwen2.5-VL-7B-Instruct and 7.5k samples for Qwen2.5-VL-72B-Instruct. The resulting models, ThinkLite-VL-7B and ThinkLite-VL-72B, significantly outperform their respective base models across eight visual reasoning benchmarks.  In particular, ThinkLite-VL-7B improves the average performance of Qwen2.5-VL-7B-Instruct by 7\% and surpasses all existing 7B-level models, as well as much larger models such as GPT-4o, O1 and Qwen2.5-VL-72B, achieving a new SoTA score of 75.1 on MathVista.  ThinkLite-VL-72B further advances the SoTA frontier, achieving an accuracy of 79.7 on MathVista and an average benchmark improvement of 4.42 over the open-source SOTA.  These results demonstrate that MCTS-guided difficulty filtering provides a scalable and effective path toward data-efficient self-improvement in multimodal reasoning.

SoTA with Less: MCTS-Guided Sample Selection for Data-Efficient Visual Reasoning Seff-Improvement
Xiyao Wang .22 Zhengyuan Yang", Chao Fenga.4 Hongjin Lu'
Linjie Li", Chung-Ching Lin Kevin Lin?, Furong Huang 4 Lijuan Wang?
University of Maryland, College Park
"Mirosooft
University of Michigan xywanggund. edu
*Cornell University
*Equal advise
80.0
ThinkLite-VL-722 (Ours)
*
Thinklite-VL-728 (Ours)
77.5
75.0
Think (Ours) Qwe3225-ML-328
Qwen2255VL-72B
75.0
ThinkLite-VL-78 (Ours)
Accuracy
72.5
@ MiniCPM-o 2.6
WW--UREEKA-Dee--
Vision-t
IntermL22 5-388
IntemVV2..5778
72.5
AMM-EURREK-OOWe--E
Visign-RLL
70.0
OpenViTninker
70.0
OpenVLThinker
67.5
Qwen2.5-VL-78
LLVVA-VVTTB
67.5
65.0
65.0
62.5
9P1-40
A ML berry-7
30
60
Reasoning VLMs
80
62.5
10k
32k
64k
Mulberry-7AA
128k
256k
Model Size (B)
Reasoning training
Figure 1: Recent "Reasoning VLMs" studies finetune "Base VLMs" with extra reasoning training data to improve visual reasoning. This paper presents data-efficient self-improving method for better training reasoning VLMs. (Left) Comparison
VLMs with different parameter sizes on Vista. Our model ThinkLite-VL-- VL-7B achieves the state-of-the-art (SoTA) accuracy of 75.1, surpassing 1---n2.--L-722--ime Innstuuct, GPT-4o, 01, and other 7B-level reasoning VLMs. ThinkLite- VL-72B further pushes this boundary to 79.7. (Right) Comparison of the reasoning training data size used by 7B-level and 72B-level reasoning models. Our model achieves SoTA performance using only 11k data (7B) and 7.5k data (72B), and without any additional knowledge distillation.
Abstract
We introduce ThinkLite-Vi a family of visual reasoning models that achieve state-of-the-an (SoTA) performance using order of magnitude fewer training samples, relying purely on reinforcement fine-tuning (RFT) self-improvement without any knowledge distillation. Our central insight is that sample difficulty critically influences RFT effectiveness: appropriately challenging examples can drive substantial reasoning improvements, even in low-data regimes. However, quantifying sample difficulty in a reliable and scalable manner remains non-trivial. To address this, we repurpose Monte Carlo Tree Search (MCTS) measure sample difficulty via the number of reasoning iterations vision-language model (VLM) requires solve each instance. This MCTS-basel selection procedure identifies samples that induce deeper reasoning while remaining solvable, allowing us to filter high-quan high-quality subset from 70k open-source examples spanning math, natural image understanding, and chart comprehension. Using this approach, we select just 1 l1k challenging samples for RFT on Qwen2.5- -.ven..5-VL-BB- Instrutttt and 7.5k samples for VL-22---VL-2BBBB
The resulting models, ThinkLite- VL-7B and ThinkLite- VL-72B, significantly outperform their respective base models across eight visual reasoning benchmarks. In particular, Think ite-VL--B improves the average
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

performance of Qwen2.5-VL-7B- ----r--nsttuutt by 7% and surpasses all existing 7B-level models, as well as much larger models such as GPT-4o, O1 and Owen2.5-VL-72BB achieving new SoTA score of 75.1 on Math Vista. ThinkL.tt- advances the SoTA frontier, achieving an accuracy of 79.7 on Mash and an average benchmark improvement of 4.42 over the open-source SOTA. These results demonstrate that MCTS -giide difficulty filtering provides a scalable and effective path toward data-efficient data, and model are available http:///thuu..omm/iiwwaaaa Shwwwang/hhnKLit--VL
further
in multimodal reasoning. Our code,
Introduction
Large language models (LLMs) have demonstrated strong capabilities in solving complex reasoning mathematics and coding- by leveraging chain-of-thoughh prompting and reflection mechanisms [24] 36 Recent work [16] highlights the critical role of reinforcement fine-tuning (RFT)
in further enhancing reasoning performance. Remarkably, these improvements can be achieved purely via RFT, even without post-training supervised fine-tuning (SFT).
However, despite the success of RFT in LLMs, its impact on vision- Saiiunlanguage models (VLMs) has been less pronounced. A likely cause is the inherent modality gap: VLMs are pretrained on text-heavy objectives, while post-training tasks demand multimodal reasoning. Recent have addressed this by incorporating knowledge distillation and super- vised format alignment before RFT. While ef- fective, these pipelines are cumbersome, and fundamentally limit the capacity for models to improve via self-training alone.
Mathverse
MathVista
MathVision
efforts 22 12
55 84
MMStta
00
10
MMMU
In this work, demonstrate that high-quality and appropriately challenging training samples alone are sufficient to enable self-improvement in VLMs via Rit-without any knowledge dis- tillation. When the training data matches the base model's capability level, RFT can explore informative rollouts by itself and substantially elevate multimodal reasoning ability. Based on this insight, we introduce ThinkLite-VVL ily of data- data-efficient reasoning VLMs trained via RFT on small subset of difficulty-curated examples.
MMBench
AI2D
MMVet
Qwen2.5 211.555VV--BBBB -B-istuttt
MAK-FUREKA-PP SAIKAA--Wnn-- 7B
OpenVLThiniee
ThinkL.tt-VVL-78 (Curs)
Figure 2: Performance comparison on 8 visual benchmarks. Our model significantly outperforms Qwen2.5-VL-7B and other reasoning models.
Vite-VL,, fam-
The key to ThinkLite-VEL VL's performance lies in effective sample selection. We propose to repurpose Monte Carlo Tree Search (MCTS) a classic inference-ting time search algorithm to estimate the difficulty of each training instance. Specifically, reasoning iterations a VLM requires to solve a task. This search- bascc signal tightly correlates with sample difficulty and naturally identifies examples that promote deeper reasoning during training.
define difficulty the number of MCTS
Our pipeline begins with 70k open-source samples spanning three core domains: mathematical reasoning, natural image understanding, and chart interpretation. For each example, we simulate an MCTS-based inference trace using the base VLM, and rank samples by the number of reasoning steps required to reach correct solution. From this pool, extract two difficulty-filtered subsets: 11k samples for Qwen2.5-VL VL--B--- and 7.5k samples for Qwen2.5-VL- 1--...5-VL-72---siii apply RFT directly these subsets-aa no supervised fine-tuning distillation required.
We then
We evaluate our resulting models, ThinkLite- VL-7B and ThinkLite- VL--2-. on eight established VLM benchmarks. After RFT, ThinkLite- Liee-i------BBB improves the average performance of Qwen2.5-VL. 7B-Instruct from 59.69% % to 64. 18%, and outperforms a comparable baseline trained on randomly selected 11 k samples (60.89%) Similarly, Thinkl WhinkLiiteV ite-VLL-72B raises the average accuracy of Qwen2.5- VI.-72B-Instruct from 65% to 72.67%, exceeding the baseline trained on randomly selected 7.5k samples 69.91%.

---

Furthermore, compared with the most recent 7B-level reasoning VLMs. ThinkLite MhinkLite-VL-7BB consis- tently demonstrates substantial performance advantages as shown Figure 2 ThinkLite- ThinkLite-VL-7B also outperforms much larger models includdng GPT-4o, Qwen2.5-VI--72B, and ol-on the Math Vista benchmark, achieving new SoTA score of 75.1 (Figure T. Thinkl -te.Litt-- the frontier, attaining a SoTA accuracy of 79.7% % on MathVista.
VI.-72B further advances
Our key contributions are:
(I) Difficulty as a learning signal. We identify sample difficulty as a critical yet underutilized signal for enabling effective self-improvement in VLMs via RFT, and show the importance of scaling compute for identifying the appropriately challenging training sample.
(2) -CTS-guided filtering. We propose novel use of Monte Carlo Tree Search to estimate sample difficulty by measuring model reasoning iteration count, Across diverse online and offline baselines, MCTTS-guide filtering delivers superior performance, benefiting from the explicit tree search.
(3) Data-efficient RFT pipeline. We introduce ThinkLite-VL. a data-efficient visual reasoning framework that achieves SoTA performance using only 11k (7B) and 7.5k (72B) training samples, without any knowledge distillation.
(4) Strong empirical gains. We demonstrate that ThinkLite- VL-7B and ThinkLite VL-72B outper- form strong baselines and existing SoTA models across eight VLM benchmarks. Notably, ThinkLite- VL-7B improves the average performance of its base model by 7%, and achieves a new SoTA score of 75.1% Wista- iista-surpassing larger models such GPT-4o, 01 and Qwen2.5-V1.-7BB. ihinkl.tteVLL-722 further advances this with a Math Vista score of 79.7%.
(5) Open-source release. We will release the full ThinkLite- VL model family, including both ThinkLite-VL-7B and ThinkLite- VL--2B, and MCTS-filtered training sets for both Qwen2.5-VL-7B and Qwen2.5-VL-72B to support future research in multimodal reasoning.
2 Related work
Large language model reasoning. Simulating human-like thinking processes through intermediate reasoning steps has significantly improved the performance of large language models (LLMs) on tasks that require reasoning
[24]. One family of methods focuses on explicitly controlling the structure or format of the model's outputs, such as by applying Chain-of- Thought (CoT) prompting (78) and Self-Consistency [77]. Related lines of work include more elaborate reasoning strategies like Tree of Thoughts 87 Graph of Thoughts 4 Additionally, some approaches involve supervised fine-
tuning (SFT) on curated datasets with reasoning annotations (52 8g Researchers have also explored
process reward models (PRMs) that encourage systematic thought processes [35] 66 70 28 98 47].
Others incorporate search techniques, including Monte Carlo Tree Search (MCTS) or beam search, to refine or verify reasoning paths [80] 81
19 73.. Recently, large-scale RL with outcome-based reward functions has been leveraged [6 to elicit powerful reasoning capabilities in LLMs. Unlike
5
prior uses of MCTS at inference time 80 15), we employ MCTS during training to assess sample difficulty and curate gigh-impact training subset for RFT. We focus on how to use large-scale RL to enhance the reasoning ability of VLMs.
Vision language model reasoning. Vision language models [ 68
10
64
35
can perform vision tasks using language given visual input through vision encoders like [57
94
65
These models demonstrate comprehensive multimodal capabilities across various scenarios
93
39
9T 49
18
92
20 32 and exhibit reasoning capabilities to some extent 43
76
%
69
Inspired
by the success of reasoning in L.Mss, researchers have sought to improve the reasoning capabilities
of VLMs. For instance, CoT prompting is applied to VLMs 97 51 46)
21 and some papers using SFT for knowledge distillation to improve reasoning abilities. Some prior works have also explored improving VLM performance
100
create multimodal datasets [86] 83 61,
99 [2 22
17
63
More recently, RL training has emerged as promising approach to further strengthen the reasoning capabilities of VLMs [12 22 50, 82). While recent works explore SFT and RL avoiding costly knowledge distillation remains challenge. In contrast, ThinkLiter VL eliminates the need for SFT or distillation entirely and achieves SoTA performance using just 11k (7B) and 7.5k (72B) samples-an order of magnitude less than prior work. Specifically, we propose a novel approach using MCTS to filter for high-quality training instances based on the difficulty level. We
through self- ref-immprvvement strategies 102 72 75 B3
[1] 22 for VLM reasoning, efficiently utilizing training data and
3

---

Marth casconing
Category
JQA Category
Data source
Data size
Open-ended
Geometry3K
3001
Char Inferstanding
Math Reasoning
Multi-choice
GcoQA
5010
Multi-choice
Geos
66
ThinkLite-VL 70K
Open-ended
FigureQA
10000
Natural Image Understanding
Multi-choice
ScienceQA
10332
Open-ended
OK-VQA
9009
Open-ended
IconQA
10000
Chart Understanding
Open-ended
TabMWP
22579
Figure 3: Data statistic of ThinkLite--LLa training dataset. We find that converting answers to open-ended format critical reliably assessing question difficulty and effective model training.
then directly apply RL training to enhance reasoning on this curated data, demonstrating strong performance without requiring any SFT stage.
Data filtration. Data filtration aims to identify and retain high-quality. diverse, and task-relevant
data while discarding noisy redundant information optimize training efficiency and generalization performance. It is important for the pretraining phase 14 29 [79 [58 [2 95 67 (56] and instruction
103 [9] of both LLMs and VLMs. In this paper, specifically focus on filtering training instances to curate data optimally for efficient downstream RL training to improve the reasoning capabilities of VLMs. A concurrent work, MM-Eureka (50), also investigates
tuning phase 34 33
38
the impact of data filtration on RFT. While MM-Eureka (50) filters samples based on zero-shot accuracy, our
method provides more expressive and fine fine-graine grained estimate of sample difficulty. capturing both solved and musooled--uu-infommme cases. Importantly, our findings reveal that samples requiring extended reasoning- even when not solved by the model-car can be highly beneficial during RFT.
To knowledge, ThinkLite-VIL is the first framework to combine search-based sample difficulty estimation with reinforcement fin-tuning-achiees data-efficient self- seff-improvement for visual reasoning at both 7B and 72B scale, without any SFT distillation.
Training Recipe
In this section, we will introduce the complete training pipeline of ThinkLite-ViL VL. First, in Section 3.1
we describe how we collect our training data that we later sample hard problems from. Then, in Section [.2] we detail how we employ base model combined with Monte Carlo Tree Search (MCTS) for data filtering to select prompts that are challenging for the base model. Finally, in Section 3.3 we explain how we use these filtered data to train ThinkLite- VL. We note that the proposed data filtering method, introduced in Section [3.2] is the core technical contribution of ThinkL VI.. Specifically. ThinkLite- VL. highlights the importance of difficulty- aware training sample selection in self-improving training, and effectively repurposes MCTS for sample difficulty prediction.
3.1
Data Collection
We collect a total of 70k datas from widely used open-source training datasets as our initial training set, covering three category: multimodel mathematical reasoning (Geometry3K [42), GeoQA (6]. Geos (60), natural image understanding (FigureQA (25). ScienceQA 43). OK- -KA 48)), and
chart understanding (IconQA [45) TabMWP 44). ). For FigureQA and IconQA, due to the large size of their original training sets, we only randomly sample 10k data points from each as our training set. The overall data distribution is shown in Figure Each training sample is organized into the following format: (Image, id, Prompt, Answer).
Furthermore, to prevent the VIM from obtaining correct answers by merely guessing from multiple- choice options, we reformulated IconQA, FigureQA, GeometryšK, TabMWP, and OK-VQA from a multiple-choice format to open-ended format. This modification compels the VLM to derive the

---

correct answer through reasoning rather than selection, thereby increasing the difficulty of the tasks and enhancing the reliability of the data filtering process described in the subsequent section.
3.2 MCTS-based Sample Selection
In our work, the collected data primarily originates from com- monly used pretraining datasets for existing VI.Ms, which makes the model susceptible to overfitting certain sam- ples. Inspired by recent successes of data filtration in LLM SFT (53 888 and conventional reinforcement learning [59 74), we propose a MCTS- sample selection mechanism. This approach leverages the VLM's own iterative reasoning process, using the number of iterations required to reach the correct answer metric to assess the difficulty of each data sample. Consequently, we can selectively filter for those samples that more challenging for the model during RL training, rather than using the entire dataset.
10:50
32.4%
Ite 30-40
575
6.2%
20-30
24.0%
Bar 5-30
Figure 4: Data difficulty distribu- tion of our 11k training set after 7B MCTS-based data filtration. Un- solved refers to data that VLM can- not solve after 50 MCTS iterations. new reasoning step, transitions the state to 1,+1, which is formed by concatenating &, with By leveraging VLM itself as policy model, TT, we sample candidate steps from the probability distribution (a(a I, st), where x denotes the task's input prompt and I represents the input image. The MCTS process starts from the root node, Su. representing the beginning of a sentence. It then iteratively proceeds through three key phases- detail in the subsequent paragraphs. In contrast to previous studies, during the data filtering stage with MCTS, we prioritize computational efficiency and comprehensive exploration of the solution space, with our focus centered on self-rewarding setting. Consequently, throughout the MCTS process, we do not employ any pretrained separately trained process reward models, thereby simplifying and accelerating the procedure. The prompt used for MCTS is shown in Appendix A Table
Specifically, we define the state at step i, denoted as st. to represent the prefix of the reasoning chain. The introduction of
expansion and simulation-wite
are described in
Selection. In our MCTS procedure, the selection process is only determined by the visitation frequency, denoted as (st). of the current state 8t- At node 8,, the subsequent node is selected
vN(s.)
according to the following formula: 8t41 = arg max, Cpnct T+N(5
Expansion. Given current step 8t, the VLM generates is distinet actions based the prompt and image through temperature decoding. Each of these actions is then combined with the current step to form & candidates next steps. The diversity among these actions is regulated by temperature parameter, which set to 0.5 in our experiments, with i configured as 3.
Simulation. After selecting node, we directly utilize the policy my generate several reasoning steps until a final answer is produced or a preset reasoning step limit is reached. Subsequently, we employ the corresponding LLM (in experiments, the -0wen2.--VL--B--Instte VL-72B-Instruct are used, with Qwen2.5-7B-Instrul serving as the critic model) to compare the generated final answer with the ground truth answer, thereby determining the correctness of the response. If the answer is correct, the MCTS process is terminated and the current iteration number K is recorded; the answer is incorrect, the visit count N of the selected node is updated and the next iteration commences. Appendix A Table 7alle77illon iillusstraes the prompt employed for the critic model.
and Qwen2.5-
Data filtration. We apply this MCTS procedure to the entire collection of 70k data samples and record the iteration number K required to solve each problem, using Qwen2.5. 1..5--L--BB VL-7B-Inntruct and Qwen2.5-VL-72B - Insttrutt as the policy model. In this process, K served as a metric for assessing the difficulty of each sample: a higher K indicates that the VLM requires more extensive exploration to arrive at the correct answer, thereby reflecting a greater level of challenge. Ultimately, we select all samples with K greater than 5, as well as those that remained unsolved after 50 iterations, resulting in a final training set of 1k samples with 7B model and 7.5k samples with 72B model. The data difficulty distribution of 1 Ik training set of 7B model is shown in Figure 4 as example.
5

---

Table 1: Visual reasoning training data comparison between ThinkLite-VE VL-7B and other 7B-level VLM reasoning models. ALL these reasoning models have distilled knowledge from larger models or closed-source models except for MM-Fureka- Baareka-Qwen-7B.
Furrea-Qwenn--e performs accuracy- based data filtering before training and uses more data (15k) than ours. Here the data size refers to the amount of extra visual reasoning data used boost the base model for reasoning, via SFT or RFT.
Reasoning Models
Knowledge Distillation (KD)
RFT
Data size
LLaVA-Cot-IIB 83
GPT-4o
100k
Mulberry-7B 86
GPT-4o, Qwen2-VL-72B VL-72B
X
260k
Vision-RI R1-7B 122
Deepseek-RR RI
200k 10k
12
PeepSeck-RI-Disiil---
59.2k
MM-EURREKA-Qwen-77
50
15k
ThinkLite-VL-72
11k
3.3 Visual Reasoning Training
Unlike previous VLM reasoning studies, which heavily depend on large-scale Chain-of-Thought (CoT) data generated by external models and employ SFT for knowledge distillation to enhance reasoning capabilities (as shown in Table T we demonstrate that directly performing reinforcement fine-tuning (RFT) with small amount of high-quality training data can significantly enhance the reasoning ability of VLMs, without the need for extensive external data generation.
After conducting MCTS- based sample selection and obtaining filtered set of high-quality training data (11k for 7B and 7.5k for 72B), then perform RL fine-tuning on the Qwen2.5-VL. models using these selected data. Specifically, we employ Group Relative Policy Optimization (GRPO) loss function proposed by (62 for training, with the objective defined as follows:
(a 1001 Ind min
molont g. qrzc)) A.e. clip (o 19 (1,0))
Moloue g.ore)e)) 1 1 +c Arg (o, 1 (.01,)))
B (TT
lI (pp)
We provide the training prompt template during RFT in Appendix A Table
4 Experiments
4.1 Benchmark Evaluation
We systematically evaluate ThinkLite- VL on several commonly used multimodal benchmark datasets and perform comprehensive comparisons with existing reasoning models. Through these experiments, we demonstrate the effectiveness and advantages of our model in multimodal reasoning tasks.
Baseline VLMs. We compare method with both 7B level and 72B level models follows:
For 7b-level VLMs, we use Qwen2.5- VL-7B- as the base model and perform RFT on the 1 11k high-quality data obtained through MCTS based filtration, resulting in our reasoning model, named ThinkLite-VL-77 We conduct training using Easy-RI [TON] code base and set GRPO rollout number 32. Our main baselines are as follows: (1) Swen2.55VLL-7B In-stuutttt 3 serving as our base model; (2) Think Lite- VL--Random 11,, trained using RFT on a randomly sampled subset of 11 instances from the full 70k dataset. Besides, we report the performance of several recent general and reasoning VLMs for comparison. including general opensourced models L..aV--Onevviiinn-- and Intern LL2.5-88 101, the SFT-based reasoning models LLa VA-Cot- IB [83 and Mulberry- 7B [86), as well as the RFT-based reasoning models Vision-RI [22]. MM-Eureka MMM-Eureaa-Qwen-77 [50]. and Open WLThinker-B [12].
[30]
For 72B-level VLMs, we use Qwen2.5- -IL-r2B-Insttrut as the base model. We perform RFT on the 7.5k high-quality data obtained by Owen2.5-VL-722. -L-72B-Instruct through MCTS-based filtration and The 72B-level baselines include: (1) our base model 3 (2) two opensourced general VL.Ms
get 72B reasoning model ThinkILit-VVL--1
[30] and Intern VI.2.5-78B 10): (3) one opensouced reasoning model QvQ-72B (7I;; (4) ThinkLite.
Swen225-viL-72B-nnnstt
VL-Random77. trained using RFT on 7.5k randomly selected samples from the full 70k dataset.

---

Table Comparison of different VLMs on 8 widely used visual benchmarks. Our model achieves SoTA performance at both 7B level and 72B level on 6 benchmarks and reaches a SoTA performance of 79.7 on Vista among VLMs. On average, our model improves performance by 7.5% and Oween2.5-VI-72B-Innsitt
and 6.5% compared with our base models Qwven..5-VL-7B-Instii We do not evaluate Mulberry-7B on Math Vision because Mulberry-7B uses Math Vision as training dataset. We evaluate all models with same code using vLLM [27] inference. For reasoning models, we use thinking templates provided in their codebase to generate thoughts and get the final answer.
Math Math Visiio mini
Math VarhV Verse
MMBench
Math
Testmint
MMMU
MMStar
MM-
A1D
mini
Models
Data size
Avg.
Proprictary Models
OpenAI-GPT-44
63.8
36.8
50.2
69.1
64.7
83.4
69.1
84.6
65.21
OpenAl-ol
73.9
58.2
57.0
77.6
7B-level General and Reasoning Vision-Lanm Vision-Language Models
LLaVA-Onevision-75
63.2
17.4
26.2
48.8
61.7
80.8
57.5
81.4
54.63
1.2258V1...588
64.4
22.0
39.5
54.9
62.8
82.7
68.8
83.3
59.80
Beeen..5-VL-B--nsttrue
67.8
23.6
44.5
50.6
61.7
80.7
66.0
82.6
59.69
LLaVA-Cot-11B
100k
54.8
16.3
33.9
46.2
57.6
75.0
60.3
78.7
52.85
Mulberry-7B
260k
63.1
39.6
55.0
61.3
79.2
63.7
80.1
Vision-R1-7B
210k
73.5
30.7
51.9
50.5
60.2
78.9
65.6
80.4
61.46
0p
59.2k
70.2
29.6
47.9
51.9
63.2
81.3
66.9
82.7
61,71
MM-EUREKA--Qwen-T2
15k
73.0
31.9
50.3
52.3
64.1
79.3
64.9
81.4
62.15
Our 7B-Jevel Reasoning Model
ThinkLite-
11k
71.9
26.1
47.3
51.7
62.7
81.1
80.9
60.89
ThinkLite- VL-7B
11k
75.1
32.9
55.5
81.4
67.8
83.6
64.18
(Ours- Random selection)
+3.2
+6.8
+4.8
+3.8
+2.3
+2.3
+2.7
+3.29
A (Ours Open SoTA)
+1.6
+0.2
+0.5
+0.9
-1.3
-1.0
+2.03
72B-leve General Reasoning Vision EiionnLLamee Language Models
O InterVL22.578BB Swen2.5-VL-722-BEE QvQ-72B
67.5
29.3
39.1
56.8
66.1
85.9
63.7
85.6
61.75
72.3
34.9
51.7
68.7
68.9
87.2
72.3
87.9
67.99
74.8
35.2
53.3
63.4
68.4
87.4
76.3
87.2
68.25
71.4
32.7
48.6
70.3
67.2
86.3
75.9
86.6
67.37
Our 72B-level Reasoning Model
Thinkl Jite-Vi -.-
7.5k
76.4
37.1
57.5
65.8
71.3
87.6
76.7
86.9
69.91
Thinkl .ite--Vee..a VI.-7BB A (Ours Random sclection)
7.5k
79.7
43.8
64.3
68.3
72.0
88.2
77.3
87.7
72.67
+3.3
+6.7
+6.8
+2.5
+0.7
+0.6
+0.6
+0.8
+3.06
A (Ours- Open 72B SoTA)
+4.9
+8.6
+11.0
-2.0
+3.1
+0.8
+1.0
-0.2
+4.42
We also include proprietary models performance references which include OpenAI-GPT-4o and OpenAI-ol. For all models, we use 80G A100 GPUs for model training and evaluation.
Benchmarks. We select eight widely used VLM benchmarks for evaluation, namely Math
41
Math [69], Math Verse (96). MMMU 993, MMStar (8] MMBench [40,, MMVet [T], and A12D (26]. Among them, Vista, Math Vison, and Math Verse widely used in VLM research to evaluate mathematical reasoning capabilities. while MMVet also includes significant number of mathematical reasoning tasks. In contrast, MMMU, MMStar, MMBench, and AI2D primarily utilized to assess VLM's visual perception reasoning and scientific reasoning abilities.
SoTA performance over both 7B and 72B models. As shown in Table 2 Moinkite-VL--77 and MhinkLite-VL-7220 show a significant improvement in average performance across the eight benchmarks compared to the base model with the average performance increasing from 59.69 to 63.89 and 68.25 72.67, respectively. ThinkLite- VL-7B also outperforms reasoning models that primarily achieve performance enhance- ment through extensive knowledge distillation (such as LLaVA-CoT-I R1-7B, and Open VLTThinke--7BB with the closest average performance to GPT-4o. Compared to MM-EUREKA A-eeen-77 7B, which does not involve SFT knowledge distillation but adopts a larger RL training dataset, our model consistently outperforms across all benchmarks. highlighting the
and (wwen.55VLL-72 VL-72B-IInsrruc
Mulberry-7B, Vision-
7

---

Table 3: Comparison with models trained on data sampled using different selection strategies,
ThinkLite-V VI. achieves significantly better performance, highlighting the effectiveness and superiority of our proposed MCTS- bassed sample selection method.
Math Testmini
MathVision
MathVerse
MMBench
MMMU
MMStar
MM-Vet
A12D
mini
Models
Data size
Avg.
ThinkLite- VL-BBB
11k
75.1
32,9
52.1
55.5
65.0
81.4
67.8
83.6
64.18
ThinkLite- Baaiee------L Unnnooleed
5.6k
73.6
26.9
49.4
52.1
62.7
81.1
67.0
83.5
62.04
ThinkLite- MonkLite-VLLIteree VL-lterOOnly
5.4k
73.5
27.5
50.2
52.5
64.2
80.9
66.9
83.3
62.38
ThinkLite-VL -L---nddo-llllll Ik
11k
71.9
26.1
47.3
51.7
62.7
81.1
65.5
80.9
60.89
ThinkLite- VL- VL-SelfCConsisency
23k
74.6
30.9
50.1
53.8
64.1
81.3
67.1
83.3
63.15
ThinkLite-VL VL-Fullset
70k
74.3
29.9
52.2
53.1
63.7
81.6
67.2
83.0
63.13
importance of high-quality data filtering before training, and the effectiveness of the proposed MCTS-based filtering. For more discussion between offline and online data filtration, please refer to achieves best performance among all 7B-scale models on six out of eight benchmarks, with only marginal gaps behind on MMBench and MM-Vet. In addition, MhinkLite-VL-721 outperforms all existing open-source vision- vision-language models across six benchmarks. Notably, ThinkLite-VL-77 attains SoTA accuracy of 75.1 on Math Vista, exceeding both GPT-4o and ol. LhinkLite-VL-7222 further advances the frontier, reaching 79.7 on Math Vista and 64.3 on Math Verse, establishing new SoTA on both benchmarks.
Section 4.3 Analyzing individual benchmarks, ThinkLLite-ViL
Effectiveness of MCTS-based sample selection. Compared to training on an equal number of randomly selected samples from the full 70K dataset BaankLiit-VL-BB-RRnddd Ik and ThinkLite- 1-72B-Raandmny .5kk), ThinkLite-VL- across eight benchmarks, with average performance improvements of 5.4% the 7B scale and 4.4% at the 72B scale. These results further show the importance of MCTS- based sample selection.
and ThinkLite-VL-77 demonstrate a clear advantage
4.2 Importance of MCTS-based Sample Selection
We conduct ablation studies to demonstrate the importance of MCTS- sample selection. We compare five different training settings of ThinkLite- VL: (1) ThinkLite-VLLUnsolleee VL-Unsolved: Trained using only the 5.6k samples that could not be solved by MCTS, representing the most difficult subset. (2) ThinkLite- SiinkLiteeVLL-ter5500 VL--ter5Ort::: required more than 5 iterations. This set, combined with the unsolved samples, forms the full 1 training set used in Thinkl VI.. (3) Thinkl -itt-ee---andoo 1k: Trained on a randomly sampled 11k subset from the full 70k dataset, matching the size of the ThinkLite-V VL training set. (4) ThinkLite- VL-SeliConsistency: Trained on 23k samples selected based on self-consistency difficulty measure. Specifically, for each prompt, we perform 50 rollouts using answer accuracy using Swen2.5-7B-IInsruue
Trained on the subset data that VLM is able to solve via MCTS, but
and compute Samples with accuracy lower than 0.2 are selected for RFT. (5) -L-Fulsett Trained on the complete 70k dataset without any filtering. We report the evaluation results of all five settings across the eight VLM benchmarks, as shown in Table
We observe that ThinkL VL.-B, trained using 1k samples via MCTS-guided sample selection, achieves the highest average performance among all settings. It outperforms not only the random sampling baseline but also models trained the full dataset and www.eeooninnnbbsseeeeee filtering, despite using significantly fewer training samples. This highlights the effectiveness of our difficulty- aware data selection strategy. Further analysis reveals that models trained on subsets derived solely from unsolved samples or samples requiring more than five iterations also show decent performance, suggesting that hard and medium-difficulty samples contribute meaningfully to reasoning ability. However, neither subset alone is sufficient. The combination of both unsolved and medium--ifficuul samples yields the strongest and most effective training signal. Additional analyses are in Appendix
4.3 Comparison with Online Data Selection
In this section, we compare our offline data-selection strategy with online alternative and evaluate their impact on model performance. We adopt an online baseline based on self-consistency filtering: during training we keep only those samples whose rollout accuracy is greater than 0 but below 0.9,
8

---

Table 4: Comparison between ThinkLite- VL and model trained with offiine and online self-
consistency based sample selection. Our method demonstrates significant advantages.
Math Vista
Math Visionn
Math Verse mmm
MMBench
restiim
MMMU
MMStar
MM-Ver
A12D
min
Model Size
Training type
Selection method
Avg.
MCTS (Ours)
75.1
32,9
52.1
55,5
65.0
81.4
67.8
83.6
64.18
Offline
7B
SeliConsistency
74.6
30.9
50.1
53.8
64.1
81.3
67.1
83.3
63.15
Online
SeliConsistency
74.2
26.9
50.1
50.6
64.8
82.0
67.1
83.0
62.34
MCTS (Ours)
79.7
43.8
64.3
68.3
72.0
88.2
77.3
87.7
72.67
Offfine
72B
SeliConsistency
77.3
39.1
62.0
66.3
71.6
87.7
77.0
87.1
71.01
Online
SelfConsistency
76.9
71.7
87.5
77.1
87.4
70.12
drawing additional samples until the training batch is full. Table comppree this online variant with our MCTS-based offline selector and a plain offline self-consistency baseline. Similar to the findings in other RL studies [90), the online filter offers negligible improvement except converges faster. The decisive factor is still the ability to identify examples that are truly challenging for the current model, task at which MCTS selector excels due its explicit tree search.
4.4 Data Difficulty Analysis between 7B and 72B Models
We analyze the 1 and 7.5k sample sets selected by 7B and 72B models, to examine how models of different capacity agree on the sample difficulty. We find that there an overlap of 5.4k samples, where 3.6k of them are instances that neither model is able to solve within 50 MCTS iterations. The real divergence lies in the mid-difficulty stratum. We observe that for this subset, the two models often behave asymmetrically; problems easily solved by the 7B model may require many more iterations for the 72B model, and vice versa, exposing distinct reasoning heuristics across models.
We validate this model-specific preference through cross sansle training: the 1 lkk samples selected by the 7B model are used RFT the 72B model, and vice versa. Table (Jbloss that the gains both settings were markedly smaller than when each model trains on its own curated set. These results suggest that a sample set tailored to one model transfers poorly to another, even in strong-to-W sorong-to-weal teak setting. Instead, it is more effective to scale extra compute to find appropriately difficult samples that best fit the model itself, as the approach proposed in ThinkLite- VL.
Table 5: Comparison between the 7B and 72B models which trained on each other's selected samples, the resulting performance improvements drops significantly.
mimn Vision
Mishiista
Math Verse
MMBench
testmini
MMMU
MMStar
MM-Ver
A12D
mim
Models
Data size
Avg.
7.5k-72B
70.2
49,2
81.1
82.9
61.24
VLin-Litt-VL--B
11k-7B
75.1
32.9
52.1
55.5
65.0
81.4
67.8
83.6
64.18
11k-7B
76.4
38.5
58.4
67.2
70.2
87.3
76.6
87.4
70.24
ThinkLite-VLL-222
7,5k-72B
79.7
43.8
64.3
68.3
72.0
88.2
77.3
87.7
72.67
5 Conclusion
We have introduced an effective self- tef-imprrvemeet approach to enhance the reasoning capabilities of VLMs, eliminating the need for external supervision or knowledge distillation. Our key insight highlights the critical importance of selecting appropriately challenging examples for RFT. We find that when training data quality is sufficiently high, even a small dataset can substantially enhance visual reasoning performance without knowledge distillation. Building on this insight, we propose a novel data selection technique, MCTS- based sample selection, which identifies and retains challenging samples by quantifying the number of MCTS reasoning iterations. Starting from 70k

---

initial samples, we obtain high-quality subset comprising 11k and 7.5k challenging samples for 7B-level and 72B-level models, respectively. These curated datasets are then used to fine-tune the Qwen2.5-VI VL--BB-Insttuu and --eee2..-VL-72B--11
via RFT, resulting in the reasoning VLMs named SonnLLite-VI.--1 and ThinkL.tee V Our models demonstrate significant improvements across multiple visual reasoning benchmarks, and notably achieves new So'A accuracy of 79.7 7. on Math Vista and 64.3 on Verse. We hope that our findings on the difficulty-based selection of RFT training data can provide insights for training more effective reasoning VLMs.
References
[1] Gpt-4v(ision) system card. 2023.
[2] Jean-Baptiste Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech, lain Barr, Yana Hasson, Karel Lenc, Arthur Mensch, Katie Millican, Malcolm Reynolds, Roman Ring, Eliza Rutherford, Serkan Cabi, Tengda Han, Zhitao Gong, Sina Samangooci, Marianne Monteiro, Jacob Menick. Sebastian Borgeaud, Andy Brock, Aida Nematzadeh, Sahand Sharifzadeh, Mikolaj Binkowski, Ricardo Barreira, Oriol Vinyals, Andrew Zisserman, and Karen Simonyan. Flamingo: visual language model for few-shot learning. ArXiv, hiss2204.141999 2022.
[3] Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, et al. Qwen2. 5-vl technical report. arXiv preprint TrXi::2502.13923, 2025.
[4] Maciej Besta, Nils Blach, Ales Kubicek, Robert Gerstenberger, Michal Podstawski, Lukas Gianinazzi, Joanna Gajda, Tomasz Lehmann, Hubert Niewiadomski, Piotr Nyczyk, et al. Graph of thoughts: Solving elaborate problems with large language models. Proceedings of the AAAI Conference Artificial Intelligence, volume 38, pages 17682-17690, 2024.
[5] Guoxin Chen, Minpeng Liao, Chengxi Li, and Kai Fan. Alphamath almost zero: process supervision without process. arXiv preprint Stti::2000003555. 2024.
[6] Jiaqi Chen, Jianheng Tang, Jinghui Qin, Xiaodan Liang, Lingbo Liu, Eric P. Xing, and Liang Lin. Geoqa: A geometric question answering benchmark towards multimodal numerical reasoning, 2022.
[7] Lichang Chen, Shiyang Li, Jun Yan, Hai Wang, Kalpa Gunaratna, Vikas Yadav, Zheng Tang, Vijay Srinivasan, Tianyi Zhou, Heng Huang, and Hongxia Jin. Alpagasus: Training better alpaca with fewer data. In The Twelfth International Conference on Learning Representations, 2024.
[8] Lin Chen, Jinsong Li, Xiaoyi Dong, Pan Zhang, Yuhang Zang. Zehui Chen, Haodong Duan, Jiaqi Wang, Yu Qiao, Dahua Lin, et al. Are we on the right way for evaluating large vision- language models? arXiv preprint arXiv:2403.20330 2024.
[9] Ruibo Chen, Yihan Wu, Lichang Chen, Guodong Liu, Qi He, Tianyi Xiong, Chenxi Liu, Junfeng Guo, and Heng Huang. Your vision- vision-language model itself a strong filter: Towards high-quality instruction tuning with data selection. ArXiv, abs/2402. 1255/442..2501, 2024.
[10] Zhe Chen, Weiyun Wang, Yue Cao, Yangzhou Liu. Zhangwei Gao, Erfei Cui, Jinguo Zhu, Shen- glong Ye, Hao Tian, Zhaoyang Liu, al. Expanding performance boundaries of open-source multimodal models with model, data, and test-time scaling. arXiv preprint arXiv:24/2.05271, 2024.
[11] Zhenfang Chen, Qinhong Zhou, Yikang Shen, Yining Hong, Hao Zhang, and Chuang Gan. See, think, confirm: Interactive prompting between vision and language models for knowledge- based visual reasoning. arXiv preprint arXiv:2301.0522 2023.
[12] Yihe Deng, Hritik Bansal, Fan Yin, Nanyun Peng, Wei Wang, and Kai- Wei Chang. Open- vithinker: An early exploration to complex vision-language reasoning via iterative self- improvement, 2025. 2025.
10

---

[13] Yihe Deng, Pan Lu, Fan Yin, Ziniu Hu, Sheng Shen, Quanquan Gu, James Y Zou. Kai- Wei Chang, and Wei Wang. Enhancing large vision language models with self-training on image comprehension. Advances in Neural Information Processing Systems, 37:131 37:131369-111997 2024.
[14] Leo Gao, Stella Biderman, Sid Black, Laurence Golding, Travis Hoppe, Charles Foster, Jason Phang. Horace He, Anish Thite, Noa Nabeshima, Shawn Presser, and Connor Leahy. The pile: An 800gb dataset of diverse text for language modeling. ArXiv, abs/2101 1000277 2020.
[15] Zitian Gao, Boye Niu, Xuzheng He, Haotian Xu, Hongzhang Liu, Aiwei Liu, Xuming Hu, and Lijie Wen. Interpretable contrastive monte carlo tree search reasoning. arXiv preprint urXiv:2410.01707, 2024.
[16] Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song. Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-rl: Incentivizing reasoning capability in Ilms via reinforcement learning. arXiv preprint arXiv:2501. 12948, 2025.
[17] Jarvis Guo, Tuney Zheng, Yuelin Bai, Bo Li, Yubo Wang, King Zhu, Yizhi Li, Graham Neubig, Wenhu Chen, and Xiang Yue. Mammoth-vl: Eliciting multimodal reasoning with instruction tuning at scale. arXiv preprint arXiv:2412.05237, 2024.
[18] Danna Gurari, Qing Li, Abigale J Stangl, Anhong Guo, Chi Lin, Kristen Grauman, Jiebo Luo, and Jeffrey P Bigham. Vizwiz grand challenge: Answering visual questions from blind people. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 3608-3617, 2018.
[19] Shibo Hao, Yi Gu, Haodi Ma, Joshua Jiahua Hong, Zhen Wang, Daisy Zhe Wang, and Zhiting Hu. Reasoning with language model is planning with world model. arXiv preprint arXiv:2305. 14992, 2023.
[20] Yunzhuo Hao, Jiawei Gu, Huichen Will Wang, Linjie Li, Zhengyuan Yang, Lijuan Wang, and Yu Cheng. Can milms reason in multimodality? emma: An enhanced multimodal reasoning benchmark. arXiv preprint arXiv:2501 055444, 2025.
[21] Yushi Hu, Weijia Shi, Xingyu Fu, Dan Roth, Mari Ostendorf, Luke Zettlemoyer, Noah A Smith, and Ranjay Krishna. Visual sketchpad: Sketching as a visual chain of thought for multimodal language models. arXiv preprini arXiv:2406.09400 2024.
[22] Wenxuan Huang, Bohan Jia, Zijie Zhai, Shaosheng Cao, Zheyu Ye, Fei Zhao, Zhe Xu, Yao Hu, and Shaohui Lin. Vision-rl: Incentivizing reasoning capability in multimodal large language models, 2025.
[23] Aaron Hurst, Adam Lerer, Adam Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, et al. Gpt-4o system card. arXiv preprint arXiv:2410.21277 2024.
[24] Aaron Jaech, Adam Kalai, Adam Lerer, Adam Richardson, Ahmed El-Kishky, Aiden Low, Alec Helyar, Aleksander Madry, Alex Beutel, Alex Cameey, et al. Openai ol system card. arXiv preprint arXiv:2412.16720, 2024.
[25] Samira Ebrahimi Kahou, Vincent Michalski, Adam Atkinson, Akos Kadar, Adam Trischler, and Yoshua Bengio. Figureqa: An annotated figure dataset for visual reasoning, 2018.
[26] Aniruddha Kembhavi, Mike Salvato, Eric Kolve, Minjoon Seo, Hannaneh Hajishirzi, and Ali Farhadi. A diagram is worth a dozen images, 2016
[27] Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph F. Gonzalez, Hao Zhang, and lon Stoica. Efficient memory management for large language model serving with pagedattention. In Proceedings of the ACM SIGOPS 29th Symposium on Operating Systems Principles, 2023.
[28] Xin Lai, Zhuotao Tian, Yukang Chen, Senqiao Yang, Xiangru Peng, and Jiaya Jia. Step- dpo: Step-wiss preference optimization for long-chain reasoning of llms. arXiv preprint arXiv:2406. 18629, 2024.
11

---

[29] Katherine Lee. Daphne Ippolito. Andrew Nystrom, Chiyuan Zhang, Douglas Eck, Chris Callison- Burchh, and Nicholas Carlini. Deduplicating training data makes language models better. In Annual Meeting of the Association for Computational Linguistics, 2021.
[30] Bo Li, Yuanhan Zhang, Dong Guo, Renrui Zhang, Feng Li, Hao Zhang, Kaichen Zhang, Peiyuan Zhang, Yanwei Li, Ziwei Liu, et al. Llava-onevision: Easy visual task transfer. arXiv preprint arXiv:2408.03333 03326, 2024.
[31] Chunyuan Li, Zhe Gan, Zhengyuan Yang, Jianwei Yang, Linjie Li, Lijuan Wang, Jianfeng Gao, et al. Multimodal foundation models: From specialists to general-purpose assistants. Foundations and Trends@ Computer Graphics and Vision, 16(1-2):1-28) 2024.
[32] Ming Li, Ruiyi Zhang, Jian Chen, Jiuxiang Gu, Yufan Zhou, Franck Dernoncourt, Wanrong Zhu, Tianyi Zhou, and Tong Sun. Towards visual text grounding of multimodal large language model, 2025.
[33] Ming Li, Yong Zhang, Shwai He, Zhitao Li, Hongyu Zhao, Jianzong Wang. Ning Cheng, and Tianyi Zhou, Superfittrrng: Weak-to-strong data filtering for fast instruction-tuning. ArXiv, abs/2402.00530, 2024.
[34] Ming Li, Yong Zhang, Zhitao Li, Jiuhai Chen, Lichang Chen, Ning Cheng, Jianzong Wang, Tianyi Zhou, and Jing Xiao. From quantity to quality: Boosting I1m performance with self- guided data selection for instruction tuning. In North American Chapter of the Association for Computational Linguistics, 2023.
[35] Hunter Lightman, Vineet Kosaraju, Yuri Burda, Harrison Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. Let's verify step by step. In The Twelfth International Conference on Learning Representations, 2023.
[36] Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, et al. Deepseek-v3 technical report. arXiv preprint arXiv:2412. 19437, 2024.
[37] Haotian Liu, Chunyuan Li. Yuheng Li, and Yong Jac Lee. Improved baselines with visual instruction tuning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 26296-26306, 2024.
[38] Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. Advances in neural information processing systems, 36:34892-34916, 2023.
[39] Yuan Liu, Haodong Duan, Yuanhan Zhang, Bo Li, Songyang Zhang, Wangbo Zhao, Yike Yuan, Jiaqi Wang, Conghui He, Ziwei Liu, et al. Mmbench: Is your multi-modas model an all-around player? In European conference computer vision, pages 216-235 Springer, 2024.
[40] Yuan Liu, Haodong Duan, Yuanhan Zhang, Bo Li, Songyang Zhang, Wangbo Zhao Yike Yuan, Jiaqi Wang, Conghui He, Ziwei Liu, et al. Mmbench: Is your multi-modal model an all-around player? In European conference on computer vision, pages 216-233. Springer, 2024.
[41] Pan Lu, Hritik Bansal, Tony Xia, Jiacheng Liu, Chunyuan Li, Hannaneh Hajishirzi, Hao Cheng, Kai-Wei Chang, Michel Galley, and Jianfeng Gao. Mathvista: Evaluating mathematical reasoning of foundation models in visual contexts. In International Conference on Learning Representations (ICLR), 2024.
[42] Pan Lu, Ran Gong, Shibiao Jiang, Liang Qiu, Siyuan Huang, Xiaodan Liang, and Song-Chur Zhu. Inter-gps: Interpretable geometry problem solving with formal language and symbolic reasoning. 2021.
[43] Pan Lu, Swaroop Mishra, Tony Xia, Liang Qiu, Kai- Wei Chang, Song-Chun Zhu, Oyvind Tafjord, Peter Clark, and Ashwin Kalyan. Learn to explain: Multimodal reasoning via thought chains for science question answering. The 36th Conference on Neural Information Processing Systems (NeurIPS), 2022.
12

---

[44] Pan Lu, Liang Qiu, Kai-Wei Chang, Ying Nian Wu, Song-Chun Zhu, Tanmay Rajpurohit, Peter Clark, and Ashwin Kalyan. Dynamic prompt learning via policy gradient for remi-structured mathematical reasoning, 2023.
[45] Pan Lu, Liang Qiu, Jiaqi Chen, Tony Xia, Yizhou Zhao, Wei Zhang, Zhou Yu, Xiaodan Liang, and Song-Chur Zhu. Iconqa: A new benchmark for abstract diagram understanding and visual language reasoning. 2022.
[46] Bozhi Luan, Hao Feng, Hong Chen, Yonghui Wang, Wengang Zhou, and Houqiang Li. Textcot: Zoom in for enhanced multimodal text-rich image understanding. arXiv preprint SiKiv:2444.097990 2024.
[47] Liangchen Luo, Yinxiao Liu, Rosanne Liu, Samrat Phatale, Harsh Lara, Yunxuan Li, Lei Shu, Yun Zhu, Lei Meng, Jiao Sun, et al. Improve mathematical reasoning in language models by automated process supervision. arXiv preprint arXiv:2406. 00w.2260..669,, 2, 2024.
[48] Kenneth Marino, Mohammad Rastegari, Ali Farhadi, and Roozbeh Mottaghi. Ok-vqa: A visual question answering benchmark requiring external knowledge, 2019.
[49] Ahmed Masry, Do Xuan Long, Jia Qing Tan, Shafiq Joty, and Enamul Hoque. Chartqa: A benchmark for question answering about charts with visual and logical reasoning. arXiv preprint arXiv:2203. 10244, 2022.
[50] Fanqing Meng, Lingxiao Du, Zongkai Liu, Zhixiang Zhou, Quanfeng Lu, Daocheng Fu, Botian Shi, Wenhai Wang, Junjun He, Kaipeng Zhang, et al. Mm-eureka: Exploring visual aha moment with rule-based large-scale reinforcement learning. arXiv preprint 0rxiv:2503.073655 2025.
[51] Chancharik Mitra, Brandon Huang, Trevor Darrell, and Roei Herzig. Compositional chain-of- thought prompting for large multimodal models. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pages 14420- 1443-,1411,,1 2024.
[52] Niklas Muennighoff, Zitong Yang. Weijia Shi, Xiang Lisa Li, Li Fei-Fei, Hannanch Hajishirzi, Luke Zettlemoyer, Percy Liang, Emmanuel Candès, and Tatsunori Hashimoto. sl: Simple test-time scaling. arXiv preprint arXiv:2501.1939) 2025.
[53] Niklas Muennighoff, Zitong Yang. Weijia Shi, Xiang Lisa Li, Li Fei-Fei, Hannanch Hajishirzi, Luke Zettlemoyer, Percy Liang, Emmanuel Candès, and Tatsunori Hashimoto. sl: Simple test-time scaling, 2025.
[54] Guilherme Penedo, Hynek Kydlícek, Loubna Ben Allal, Anton Lozhkov, Margaret Mitchell, Colin Raffel, Leandro von Werra, and Thomas Wolf. The fineweb datasets: Decanting the web for the finest text data at scale. ArXiv, abs/2406. 17557, 2024.
[55] Yingzhe Peng, Gongrui Zhang, Miaosen Zhang, Zhiyuan You, Jie Liu, Qipeng Zhu, Kai Yang. Xingzhong Xu, Xin Geng, and Xu Yang. Lmm-rl: Empowering 3b Imms with strong reasoning abilities through two-stage rule-based rl. arXiv preprint vrXiv:2503.07550 2025.
[56] Filip Radenovic, Abhimanyu Dubey, Abhishek Kadian, Todor Mihaylov, Simon Vandenhende, Yash J. Patel, Yi Wen, Vignesh Ramanathan, and Dhruv Kumar Mahajan. Filtering, distillation, and hard negatives for vision-language pre-training. 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 6967-6977, 2023.
[57] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. International conference on machine learning, pages 8748-8763. PmLR, 2021.
[58] Laura Ruis, Maximilian Mozes, Juhan Bae, Siddhartha Rao Kamalakara, Dwarak Talupuru, Acyr F. Locatelli, Robert Kirk, Tim Rocktaschel, Edward Grefenstette, and Max Bartolo. Procedural knowledge in pretraining drives reasoning in large language models. ArXiv, 1bs/2411.12580 2024.
13

---

[59] Tom Schaul, John Quan, Ioannis Antonoglou, and David Silver. Prioritized experience replay. 2016.
[60] Minjoon Seo, Hannaneh Hajishirzi, Ali Farhadi, Oren Etzioni, and Clint Malcolm. Solving geometry problems: Combining text and diagram interpretation. In Lluís Märquez, Chris Callison-Burcl and Jian Su, editors, Proceedings of the 2015 Conference Empirical Methods in Natural Language Processing, pages 1466-1476, Lisbon, Portugal, September 2015. Association for Computational Linguistics.
[61] Hao Shao, Shengju Qian, Han Xiao, Guanglu Song, Zhuofan Zong, Letian Wang, Yu Liu, and Hongsheng Li. Visual cot: Advancing multi-modal language models with a comprehensive dataset and benchmark for chain-of-thouge reasoning. Advances in Neural Information Processing Systems, 37:8612 8642, 2024.
[62] Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, K. Li, Y. Wu, and Daya Guo. Deepseekmath: Pushing the limits of mathematical reasoning in open language models, 2024.
[63] Omkar Thawakar, Dinura Dissanayake, Ketan More, Ritesh Thawkar, Ahmed Heakl, Noor Ahsan, Yuhao Li, Mohammed Zumri, Jean Lahoud, Rao Muhammad Anwer, et al. Llamav-ol: Rethinking step-by-step visual reasoning in llms. arXiv preprint arXiv:2501.066 161166, 2025.
[64] Peter Tong, Ellis Brown. Penghao Wu, Sanghyun Woo, Adithya Jairam Vedagiri IYER, Sai Charitha Akula, Shusheng Yang. Jihan Yang, Manoj Middepogu, Ziteng Wang, et al. Cambrian-1: A fully open, vision-centric exploration of multimodal llms. Advances Neural Information Processing Systems, 37:87310 10 17-575,6, 2024.
[65] Michael Tschannen, Alexey Gritsenko, Xiao Wang, Muhammad Ferjad Nacem, Ibrahim Alabdulmohsin, Nikhil Parthasarathy, Talfan Evans, Lucas Beyer, Ye Xia, Basil Mustafa, et al. Siglip 2: Multilingual vision-language encoders with improved semantic understanding, localization, and dense features. arXiv preprint arXiv:2502. 14786 2025.
[66] Jonathan Uesato, Nate Kushman, Ramana Kumar, Francis Song, Noah Siegel, Lisa Wang, Antonia Creswell, Geoffrey Irving, and Irina Higgins. Solving math word problems with pnocess-and outcome-based feedback. arXiv preprint arXiv:2211.1427. 2022.
[67] Alex Wang, Kevin Lin, David Junhao Zhang, Stan Weixian Lei, and Mike Zheng Shou. Too large; data reduction for vision-languaa pre-training. 2023 IEEF/CVF International Conference on Computer Vision (ICCV), pages 3124-3134, 2023.
[68] Jianfeng Wang, Zhengyuan Yang, Xiaowei Hu, Linjie Li, Kevin Lin, Zhe Gan, Zicheng Liu, Ce Liu, and Lijuan Wang. Git: A generative image-to-text transformer for vision and language. arXiv preprint arXiv:2205. 14100, 2022.
[69] Ke Wang, Junting Pan, Weikang Shi. Zimu Lu, Houxing Ren, Aojun Zhou, Mingjie Zhan, and Hongsheng Li. Measuring multimodal mathematical reasoning with math-vision dataset. In The Thirty-eight Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2024.
[70] Peiyi Wang, Lei Li, Zhihong Shao, RX Xu, Damai Dai, Yifei Li, Deli Chen, Yu Wu, and Zhifang Sui. Math-shepherd: Verify and reinforce lims step-by-step without human annotations. arXiv preprint arXiv:2312.0880 2023.
[71] Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, al. Qwen2-vl: Enhancing vision-language model's perception of the world at any resolution. arXiv preprint arXiv:2409.122 12191, 2024.
[72] Xiyao Wang, Jiuhai Chen, Zhaoyang Wang, Yuhang Zhou, Yiyang Zhou, Huaxiu Yao, Tianyi Zhou, Tom Goldstein, Parminder Bhatia, Furong Huang, et al. Enhancing visual-language modality alignment in large vision language models via self-improvemenn, arXiv preprint arXiv:2405.1597 2024.
14

---

[73] Xiyao Wang, Linfeng Song, Ye Tian, Dian Yu. Baolin Peng, Haitao Mi, Furong Huang, and Dong Yu. Towards self-improvement of llms via mets: Leveraging stepwise knowledge with curriculum preference learning. arXiv preprint ar arXiv:2410.0000
2024.
[74] Xiyao Wang, Wichayaporn Wongkamjan, Ruonan Jia, and Furong Huang. Live in the moment: Learning dynamics model adapted to evolving policy. In International Conference on Machine Learning, pages 36470-36493, PMLR, 2023.
[75] Xiyao Wang, Zhengyuan Yang. Linjie Li, Hongjin Lu, Yuancheng Xu, Chung-Ching Lin, Kevin Lin, Furong Huang, and Lijuan Wang. Scaling inference-time search with vision value model for improved visual comprehension. arXiv preprint wrXiv:24/2.03700 2024.
[76] Xiyao Wang, Yuhang Zhou, Xiaoyu Liu, Hongjin Lu, Yuancheng Xu, Feihong He, Jaehong Yoon, Taixi Lu, Gedas Bertasius, Mohit Bansal, et al. Mementos: A comprehensive bench- mark for multimodal large language model reasoning over image sequences. arXiv preprini urXiv:2401.10529 2024.
[77] Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoe Le, Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou Self-consistency improves chain of thought reasoning in language models. arXiv preprint arXiv:2203. 2022.
[78] Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, al. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 15:24424-248873 2022.
[79] Sang Michael Xie, Shibani Santurkar, Tengyu Ma, and Percy Liang. Data selection for language models via importance resampling. ArXiv, abs/2302.03169, 2023.
[80] Yuxi Xie, Anirudh Goyal, Wenyue Zheng, Min-Yen Kan, Timothy P Lillicrap, Kenji Kawaguchi, and Michael Shieh. Monte carlo tree search boosts reasoning via iterative preference learning. arXiv preprint vrXiv:2405.0045.1 2024.
[81] Huajian Xin, ZZ Ren, Junxiao Song, Zhihong Shao, Wanjia Zhao, Haocheng Wang, Bo Liu, Liyue Zhang. Xuan Lu, Qiushi Du, et al. geepseck-prover-e 5: Harnessing proof as- sistant feedback for reinforcement learning and monte-carlo tree search. arXiv preprint arXiv:2408.08 080.0000.811522 52, 2024.
[82] Tianyi Xiong, Xiyao Wang, Dong Guo, Qinghao Ye, Haoqi Fan, Quanquan Gu, Heng Huang, and Chunyuan Li. Llava-critie: Learning to evaluate multimodal models. arXiv preprint arXiv:2410.02712. 2024.
[83] Guowei Xu, Peng Jin, Hao Li, Yibing Song, Lichao Sun, and Li Yuan. Llava-cot: Let vision language models reason step-by-step. 2025.
[84] Yi Yang, Xiaoxuan He, Hongkun Pan, Xiyan Jiang, Yan Deng. Xingtao Yang, Haoyu Lu, Dacheng Yin, Fengyun Rao, Minfeng Zhu, Bo Zhang, and Wei Chen. RI-onevision: Ad- vancing generalized multimodal reasoning through cross-modal formalization. arXiv preprint arXiv:2503. 10615, 2025.
[85] Zhengyuan Yang, Linjie Li, Kevin Lin, Jianfeng Wang, Chung-Ching Lin, Zicheng Liu, and Lijuan Wang. The dawn of Imms: Preliminary explorations with gpt-4v (ision). arXiv preprint mxXiv:239..144211 91):1, 2023.
[86] Huanjin Yao, Jiaxing Huang, Wenhao Wu, Jingyi Zhang, Yibo Wang, Shunyu Liu, Yingjie Wang, Yuxin Song, Haocheng Feng, Li Shen, and Dacheng Tao. Mulberry: Empowering milm with ol-like reasoning and reflection via collective monte carlo tree search, 2024.
[87] Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Tom Griffiths, Yuan Cao, and Karthik Narasimhan. Tree of thoughts: Deliberate problem solving with large language models. Advances in neural information processing systems, 16:11809-1182 2023.
[88] Yixin Ye, Zhen Huang, Yang Xiao, Ethan Chern, Shijie Xia, and Pengfei Liu. Limo: Less more for reasoning. 2025.
15

---

[89] Qifan Yu, Zhebei Shen, Zhongqi Yue, Yang Wu, Wenqiao Zhang, Yunfei Li, Juncheng Li, Siliang Tang, and Yueting Zhuang. Mastering collaborative multi-modal data selection: A focus on informativeness, uniqueness, and representativeness. ArXiv, abs/2412.06295, 2024.
[90] Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo. Yu Yue, Tiantian Fan, Gaohong Liu, Lingjun Liu, Xin Liu, et al. Dapo: An open-source Ilm reinforcement learning system at scale. arXiv preprint arXiv:2503. 14476, 2025.
[91] Weihao Yu, Zhengyuan Yang, Linjie Li, Jianfeng Wang, Kevin Lin, Zicheng Liu, Xinchao Wang, and Lijuan Wang. Mm-vet: Evaluating large multimodal models for integrated capabili- ties, 2024.
[92] Weihao Yu, Zhengyuan Yang, Lingfeng Ren, Linjie Li, Jianfeng Wang, Kevin Lin, Chung- Ching Lin, Zicheng Liu, Lijuan Wang, and Xinchao Wang. Mm-vet v2: A challenging benchmark to evaluate large multimodal models for integrated capabilities, arXiv preprint arXiv:2408. N,00:2,408.00755, 2024.
[93] Xiang Yue, Yuansheng Ni, Kai Zhang, Tianyu Zheng, Ruoqi Liu, Ge Zhang, Samuel Stevens, Dongfu Jiang, Weiming Ren, Yuxuan Sun, Cong Wei, Botao Yu, Ruibin Yuan, Renliang Sun, Ming Yin. Boyuan Zheng, Zhenzhu Yang, Yibo Liu, Wenhao Huang, Huan Sun, Yu Su, and Wenhu Chen. Mmmu: A massive distildi multimodal understanding and reasoning benchmark for expert agi. In Proceedings of CVPR, 2024.
[94] Xiaohua Zhai, Basil Mustafa, Alexander Kolesnikov, and Lucas Beyer. Sigmoid loss for language image pre-training. In Proceedings of the IEEE/CVF international conference on computer vision, pages 11975-11988 1996, 2023.
[95] Lei Zhang, Fangxun Shu, Tianyang Liu, Sucheng Ren, Hao Jiang, and Cihang Xie. Fil- ter&align: Leveraging human knowledge to curate image-text data. 2023.
[96] Renrui Zhang, Dongzhi Jiang, Yichi Zhang, Haokun Lin, Ziyu Guo, Pengshuo Qiu, Aojun Zhou, Pan Lu, Kai-Wei Chang, Peng Gao, et Mathverse: Does your multi-modal IIm truly see the diagrams in visual math problems? arXiv preprint arXiv:24033 14624, 2024.
[97] Ruohong Zhang, Bowen Zhang, Yanghao Li, Haotian Zhang, Zhiqing Sun, Zhe Gan, Yinfei Yang, Ruoming Pang, and Yiming Yang. Improve vision language model oh-inuof-thought reasoning. arXiv preprint arXiv:2410. /6/98, 2024.
[98] Zhenru Zhang, Chujie Zheng, Yangzhen Wu, Beichen Zhang, Runji Lin, Bowen Yu, Dayiheng Liu, Jingren Zhou, and Junyang Lin. The lessons of developing process reward models in mathematical reasoning. arXiv preprint arXiv:2501.07300 2025.
[99] Zhuosheng Zhang, Aston Zhang. Mu Li, Hai Zhao, George Karypis, and Alex Smola. Mul- timodal chain-of-thoughe reasoning in language models. arXiv preprint arXiv:2302.00923, 2023.
[100] Ge Zheng, Bin Yang, Jiajin Tang, Hong-Yu Yu Zhou, and Sibei Yang. Ddcot: Duty-distinet Mfain-of-thoughh prompting for multimodal reasoning in language models. Advances in Neural Information Processing Systems, 111118111111,1111111111111111191 2023.
[101] Yaowei Zheng, Junting Lu, Shenzhi Wang, Zhangchi Feng, Dongdong Kuang, and Yuwen Xiong. Easyrl: An efficient, scalable, multi-modality rl training framework. httpp:// github. com hiyouga/ EEasyR1 2025.
[102] Yiyang Zhou, Zhiyuan Fan, Dongjie Cheng, Sihan Yang, Zhaorun Chen, Chenhang Cui, Xiyao Wang, Yun Li, Linjun Zhang, and Huaxiu Yao. Calibrated self-rewarding vision language models. arXiv preprint arXiv:2405. 14622, 2024.
[103] Deyao Zhu, Jun Chen, Xiaoqian Shen, Xiang Li, and Mohamed Elhoseiny. Minigpt-4: Enhancing vision-language understanding with advanced large language models. arXiv preprint arXiv:2304 2304,0 2023.
16

---

NeurIPS Paper Checklist
The checklist is designed encourage best practices for responsible machine learning research, addressing issues of reproducibility, transparency, research ethics, and societal impact. Do not remove the checklist: The papers not including the checklist will be desk rejected. The checklist should follow the references and follow the (optional) supplemental material. The checklist does NOT count towards the page limit.
Please read the checklist guidelines carefully for information on how to answer these questions. For each question in the checklist:
You should answer [Yes], [No]. or [NA].
[NA] means either that the question Not Applicable for that particular paper or the relevant information is Not Available.
Please provide short (1-2 sentence) justification right after your answer (even for NA).
The checklist answers are an integral part of your paper submission. They are visible to the reviewers, area chairs, senior area chairs, and ethics reviewers. You will be asked to also include (after eventual revisions) with the final version of your paper, and its final version will published with the paper.
The reviewers of your paper will be asked use the checklist as one of the factors their evaluation. While "[Yes] is generally preferable to "[No]" ", it is perfectly acceptable to answer "[No] providedd proper justification is given (e.g.. "error bars are not reported because it would be too computationally expensive" or "we were unable to find the license for the dataset we used"). In general, answering binary way, we
"[No] or "[NA]" is not grounds for rejection. While the questions are phrased acknowledge that the true answer is often more nuanced, please just use your best judgment and write justification to elaborate. All supporting evidence can appear either in the main paper or the supplemental material, provided in appendix. If you answer [Yes] to a question, in the justification please point to the section(s) where related material for the question can be found.
IMPORTANT, please:
Delete this instruction block, but keep the section heading "NeurIPS Paper Checklist",
Keep the checklist subsection headings, questions/answers and guidelines below.
Do not modify the questions and only use the provided macros for your answers.
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes].
Justification: See Abstract and Introduction sections.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results be expected to generalize other settings.
is fine include aspirational goals as motivation long is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: Section -
17

---

Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, c.g., if the approach was only tested few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA].
Justification: [NA]
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions the paper (regardless whether the code and data are provided or not)?
Answer: [Yes].
Justification:
See contributions in introduction and Section 4
Guidelines:
The answer NA means that the paper does not include experiments.
18

---

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. w
For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily new algorithm, the paper should make it clear how to reproduce that algorithm.
(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g.. large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes].
Justification:
We provide all training data and code.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (nttps://nips.co
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
/ /ttppp.cc/pbllc/guudeeee http://oeeuuiiissooooooooooo
for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
19

---

Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code permitted.
6. Experimental setting/details
Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Question:
Answer: [Yes].
Justification: Section 4
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented the core the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes].
Justification: Section
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results accompanied by error bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
OK to report 1-sigma error bars, but one should state The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that out of range (e.g. negative error rates).
If error bars reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes].
Justification: Section
Guidelines:
The answer NA means that the paper does include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
20

---

The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics
Answer: [Yes].
Justification:
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: We discuss both potential positive societal impacts and negative societal impacts in the conclusion section.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has socictal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: The paper poses high misuse risks.
Guidelines:
21

---

The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes].
The paper properly credits the creators of existing assets (e.g.. VLLM, Qwen) with citations, explicitly stating their licenses and confirming compliance with their terms of use.
Justification:
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes].
Justification: This paper includes all training code in supplementary materials. Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moddl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
22

---

Answer: [NA].
Justification:
The paper does not involve crowdsourcing or research with human subjects.
Guidelines:
The answer NA means that the paper does involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Ethies, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals equivalent for research with human subjects
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Question:
Answer: [NA].
Justification: The paper does not involve research with human subjects or crowdsourcing.
Guidelines:
The answer NA means that the paper does involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing. editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, originality of the research, declaration is not required.
Answer: [Yes].
Justification: We used the Qwen2.5-VL series to filter and select training samples. This LLM-based selection process critical step in constructing our training dataset and constitutes an essential, non-standard component of our core methodology.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non sta-stad components.
Please refer to LLM policy httppp////errrpoccccc/oooeeereee for what should or should not be described.
23

---

A Prompts used in experiments
A.1 Prompt for MCTS
The prompt used for MCTS is shown in Table 6
Table 6: Prompt used for VLM during MCTS procedure. We provide two examples of in-context learning to force VLM follow the reasoning format.
MCTS Prompt Template:
Answer the question **step by step** and provide the final answer at the end, each step should end with **<end>## and put your final answer within Below are two examples: Question: BoatsRUs built 7 canoes in January of this year and then each subsequent calendar month they built twice the number of canoes they had built the previous month. How many total canoes were built by BoatsRUs by the end of May of this year?
### Step1: To find the result of the total number of canoes built by BoatsRÜs by the end of May, need to find the number canoes built each month from January to May and then add them up. <end>
### Step2: To find the number of canoes built in each month, I need to use the formula for the number of canoes built in a given month, which is the number of canoes built in the previous month times 2. <end>
### Step3: So, the number of canoes built in January is the number of canoes built in February is 7 times 2, which is 14, the number of canoes built in March is 14 times 2. which is 28, the number canoes built April is 28 times 2, which is 56, and the number of canoes built in May is 56 times 2, which is 112. <end>
### Step4: Now, can add up these numbers get the total number of canoes built by BoatsRUs by the end of May: 7 plus 14 plus 28 plus 56 plus 1 112, which is 217. <end>
### Final Answer: The answer is: 217
Question: Find the number of blue circles in the figure.
### Step 1: To find the result the number of blue eircles, need interpret the figure. The figure is Venn diagram with two labeled sets: One set labeled "blue" contains all the shapes that are blue color. The other set labeled "circle" contains all the shapes that are circular in shape. The overlapping region of the Venn diagram contains shapes that are both blue and circular. <end>
### Step 2: The overlapping region contains shapes that meet both criteria: Blue color and Circle shape. From the diagram: There **one blue circle** in the overlapping region. <end>
#### Final Answer:
The answer is:
Remember to answer the question **step by step**! Here is your question: Question: [QUESTION]
A.2 Prompt for Critic Model
The prompt used for critic model during MCTS is shown in Table
A.3 Prompt for RFT
The prompt used for RFT is shown Table
B More experiments
B.1 Reward curves of VLM with different training data
We compare the reward curves during RFT of ThinkLite-VL--ter5500 and ThinkLite-VE VL, shown in Figure S
Although ThinkLite-Vi VL-

---

Table 7: Critic prompt for MCTS simulation results evaluation.
Critic Prompt Template:
Please help me judge the correctness of the generated answer and the corresponding rationale.
Question: {
Ground truth answer: {}
Generated rationale and answer: {}
Your output should only be one sentence: the generated answer is true or false.
Table Prompt template used for reinforcement learning fine-tuning.
Prompt Template:
You FIRST think about the reasoning process as an internal monologue and then provide the final answer. The reasoning process MUST BE enclosed within <think> </think> tags, The final answer MUST BE put in
Randomi I Ik and ThinkLite-VL--FR achieve higher rewards during training, their actual benchmark performances inferior ThinkLite-VL This observation suggests that incorporating a large number of easy samples into training rapidly improves rewards but fails to enhance the model's reasoning ability. Moreover, ThinkLite-VL exhibits notably lower rewards compared ThinkLite- VL-Iter5Only, indicating that the unsolved data identified by our MCTS- sample selection strategy indeed pose significant challenges to the VLM. By progressively learning to solve these chal- lenging problems during training- even if not all are solved completely- the reasoning capabilities of VLMs can be substantially improved.
0.8
ThinkLite-VL-Randdmt
ThinkLite-VL-Fulllett
0.4
ThinkLite-VL-HNE VL-tter55ntt
ThinkLite-VL-HEE --tt-----lrrrr
0.2
Stea
120
Figure 5: Comparison of reward curves of 7B models trained with different data during RFT. IterS+Unsolved lk dataset presents the most challenging learning setting for VLM, highlighting the difficulty of the samples selected by MCTS based sample selection.
B.2 Ablation Study of Data Difficulty
In this section, we investigate how training data difficulty affects model performance. We present the average performance of models trained using different difficulty data in Table 9 Notably, the model
25

---

trained with the Iter5+Unsolved subset achieves the highest average score of 63.89, outperforming all other settings. When expanding the difficulty threshold (e.g.. Iterl0, Iter20, Iter30, and Iter40), the model performance consistently declines, suggesting that medium-difficulty samples are important for improving model reasoning ability. As the difficulty of the training data decreases, the model's performance also declines. This trend suggests that the inclusion of an excessive number of easy samples may weaken the training signal during RFT and ultimately hurt the model's reasoning ability.
Table 9: ThinkLitt-VLL-77 performance under different training data difficulty settings.
InerSvusoovved achieves the best performance.
Difficulty level
Data size
Avg. score
Fullset
70k
63.13
Unsoovveee
18k
63,29
IterS+Unsolved
11k
63.89
Iterl0+Unsolved
8k
62.65
Iter20+Unsolved
6.8k
62.61
Iter30+Unsolved
6. lk
62.39
Iter40+Unsolved
5.8k
62.26
Unsolved
5.6k
62.04
C
Case Studies
In this section, we present samples varying difficulty levels selected by the MCTS- based sample selection method across different datasets, shown in Tables 13 through [4] The difficulty levels are determined based on the number of reasoning iterations required by the VLM to arrive at the correct answer during the MCTS process, providing reference examples for understanding how the method distinguishes between easy and challenging samples.
D Limitations
While sample selection has effectively enhanced the reasoning capabilities of vision-language models (VLMs), the overall training efficiency remains a key limitation, both in data filtering and rein- forcement learning stages. Common approaches such as self-consistenny-beee proposed MCTS-based strategy require substantial time for sample filtering. Additionally, the GRPO training process incurs significant computational overhead due the large number of rollout samples needed for target value estimation. These efficiency challenges can potentially be mitigated through parallelized sample generation and processing, which we leave an avenue for future work.
selection and our
E Societal impacts
ThinkLite-V VL can positively impact society by enabling more data-cfficient and accessible develop- ment of advanced visual reasoning in vision-language models (VLMs). However, negative societal risks include the potential misuse of its MCTS-based sample selection insights for crafting sophis- ticated disinformation. Responsible development must also guard against over-reliance on these enhanced, yet still fallible, reasoning systems.
26

---

Example 3: Different difficulty samples from FigureQA
IterO
Question: Is Medium Blue less than Dark Orchid? Ground Truth Answer:
Yes.
Iter29
Question: Does Dodger Blue intersect Dark Slate? Ground Truth Answer: Yes.
Unsolved
Does Violet Red have the maximum area under the curve? Ground Truth Answer: No.
Question:
Table 10: Example of samples with different difficulties decided by MCTS-based sample selection from FigureQA.
27

---

Example 4: Different difficulty samples from ScienceQA
Pair 1
Pair 2
S
N
N
4.2
1.75
Iterr)
Think about the magnetic force between the magnets in each pair. Which of the following statements is true? Choices: (A) The magnitude the magnetic force is greater in Pair 2. (B) The magnitude of the magnetic force is greater in Pair 1. (C) The magnitude of the magnetic force the same in both pairs. Ground Truth Answer: A.
Question:
Solvent Volum::
Solvent untume:
Solution A
Solution
Iter13
Question: Which solution has higher concentration of purple particles? Choices: (A) neither; their concentrations the same (B) Solution A (C) Solution B Ground Truth Answer: B.
Unsolved
Question: What is the direction of this push? Choices: (A) away from the hockey stick (B) toward the hockey stick
Ground Truth Answer:
Table 11:
Example of samples with different difficulties decided by MCTS-based sample selection from ScienceQA.
28

---

Example 5: Different difficulty samples from OK-VQA
IterO
Question: What food group is pictured here? Ground Truth Answer: fruit.
Iter20
Question: What is the length of the surfboard the man in the black shorts at the back of the line of people is holding?
Ground Truth Answer: 7 feet.
Unsolved
What is this guy's profession? Ground Truth Answer: security.
Question:
Table 12: Example of samples with different difficulties decided by MCTTS-base sample selection from OK-VQA.
29

---

Example 6: Different difficulty samples from IconQA
IterO
Question: How many flowers are there? Ground Truth Answer: 56.
Iter10
Question: How many dots there? Ground Truth Answer: 40.
Unsolved
Question: How many stars are there?
Ground Truth Answer: 19.
Table 13: Example of samples with different difficulties decided by MCTS-based sample selection from IconQA.
30

---

Example 7: Different difficulty samples from TabMWP
red confetti
$11 per pound
gold confetti
$12 per pound
rainbow confetti
$10 per pound
silver confetti
per pound
green confetti
$12 per pound
IterO
Question: Adriana wants buy pounds silver confetti. How much will she spend? Ground Truth Answer: 36.
Spinning wheel through Number spun
Frequency
Iter22
Question: A game show viewer monitors how often a wheel numbered through stops at each number. How many people there in all? Ground Truth Answer: 29.
Ties per rack
Stem
Leaf
3
2
4
046888
5
14
6
58
7
56799
Unsolved
Question: The employee at the department store counted the number of ties on each tie rack. How many racks have at least 30 ties but fewer than 70 ties?
Ground Truth Answer: 15.
Table 14: Example of samples with different difficulties decided by MCTS-based sample selection from TabMWP.
31

---

Example 1: Different difficulty samples from Geometry3K
2x + 3
4y
8y 36
5x
IterO
Find so that the quadrilateral is a parallelogram. Ground Truth Answer:
Question:
3x
33
4w
3
15.4
17.9
83
2y+ 5
38
3z
3
11.1
20
P
Iter16
Use parallelogram M P R to find y. Ground Truth Answer: 6.45.
Question:
8
17
21
Unsolved
Question: Find the area of the parallelogram. Round to the nearest tenth necessary. Ground Truth Answer: 315.
Table 15: Example of samples with different difficulties decided by MCTS-based sample selection from GeoQA.
32

---

Example 2: Different difficulty samples from Geos
IterO
Question: What is the area of the following square, if the length of BD is * /2? Choices:
(A) 1 (B) 2 (C) (D) 4 (E)
Ground Truth Answer: D.
Iter7
Question: Given the circle at the right with diameter AB, find x. Choices: (A) 30 degrees (B)
45 degrees (C) 60 degrees (D) 90 degrees (E) None
Ground Truth Answer:
Unsolved
Question: In the diagram the right, lines I and g are parallel, and lines and b are parallel.
x = 75. What is the value of + z? Choices: (A) 75 (B) 105 (C) 150 (D) 180 (E) None Ground Truth Answer: D.
Table 16: Example of samples with different difficulties decided by MCTS-based sample selection from Geos.
33


Reviews:
[{"review_id": "Rwnbk4sRvg", "reviewer": "Reviewer_gSkw", "summary": "This paper proposes a data selection method for data-efficient reinforcement learning training of visual large language models. Specifically, the authors filter training data based on offline difficulty measured by the number of MCTS iterations the model needs to solve questions correctly. Experimental results demonstrate that training models on their selected datasets achieves leading performance across multiple benchmarks on both 7B and 72B models while using significantly less training data. Ablation studies attempt to show that MCTS-based sample selection is crucial, leading to superior performance over other offline and online data selection methods.", "questions": "Please refer to the concerns raised in the weakness section.", "limitations": "Yes, the authors discuss their limitations in Appendix D.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths:\n\n- The paper is clearly written with good logical flow, clear figures, and well-designed experimental result tables that effectively communicate the methodology and findings.\n- Extensive experimental validation across 8 benchmarks demonstrates that models trained on significantly less data (11k for 7B, 7.5k for 72B) achieve leading performance, showing impressive training efficiency and effectiveness of the proposed approach.\n\nWeaknesses:\n\n- The experimental setup for important baselines—particularly the self-consistency offline and online data selection methods—lacks clarity, raising concerns about the validity of experimental results and analysis in sections 4.2 and 4.3. Since MCTS sampling includes step-by-step reasoning before final answers, self-consistency baselines should also incorporate chain-of-thought reasoning rather than direct answers. This critical setting is not clearly specified, potentially leading to unfair comparisons.\n- Several technical details about the self-consistency baselines are unclear or potentially incorrect: For offline self-consistency (line 282), samples with accuracy lower than 0.2 across 50 rollouts are selected—does \"accuracy\" refer to \"mean accuracy@50\"? Following standard practices in methods like DAPO, samples with 0 < mean accuracy@k < 1 are typically selected, suggesting the threshold should be \"higher than or equal to 0.02\" rather than \"lower than 0.2.\" The rationale for this threshold selection is not provided. Similarly, for online self-consistency (line 297), the number of rollouts per sample is missing, and the upper bound of 0.9 rather than 1.0 lacks justification.\n- The paper lacks discussion of inference costs for MCTS-based data sampling compared to self-consistency methods, which is crucial for evaluating practical applicability.\n- Additionally, to fairly compare offline versus online data selection, the authors should evaluate MCTS-based sampling in an online setting. Without this comparison, the conclusion that \"online filtering offers negligible improvement except faster convergence\" cannot be generalized across all data selection methods.\n- The motivation for applying this method specifically to visual large language models is unclear. The MCTS-based data sampling approach appears equally applicable to text-only model training, yet the paper lacks discussion of VLM-specific design considerations or challenges. It would be valuable to understand whether this method would achieve similar performance improvements when applied to text LLM training compared to other data selection methods, which would help establish the method's broader applicability and the specific value proposition for multimodal models.", "was_revised": true, "final_justification": "The author's response has well addressed my concerns, the paper is clearly structured and gives several insightful findings. Therefore, I have changed my score to 4.", "rebuttal": "We thank Reviewer gSkw for the thoughtful and constructive feedback. We appreciate that Reviewer gSkw found our experiments to be well-designed, our results strong, and our proposed approach both efficient and effective in training. We address all raised concerns in detail below.\n\n\n\n---\n\n> W1: The experimental setup for important baselines—particularly the self-consistency offline and online data selection methods—lacks clarity, raising concerns about the validity of experimental results and analysis in sections 4.2 and 4.3. Since MCTS sampling includes step-by-step reasoning before final answers, self-consistency baselines should also incorporate chain-of-thought reasoning rather than direct answers. This critical setting is not clearly specified, potentially leading to unfair comparisons.\n\n- **We clarify that both MCTS and self-consistency baselines use the same long-CoT prompting format.** For the self-consistency baselines, we use the exact same prompt as MCTS to elicit long chain-of-thought reasoning before answer generation. This ensures a fair comparison, with both methods generating step-level reasoning traces. The prompt is provided in **Appendix A1, Table 6**.\n\n\n\n---\n\n> W2: Several technical details about the self-consistency baselines are unclear or potentially incorrect: For offline self-consistency (line 282), samples with accuracy lower than 0.2 across 50 rollouts are selected—does \"accuracy\" refer to \"mean accuracy@50\"? Following standard practices in methods like DAPO, samples with 0 < mean accuracy@k < 1 are typically selected, suggesting the threshold should be \"higher than or equal to 0.02\" rather than \"lower than 0.2.\" The rationale for this threshold selection is not provided. Similarly, for online self-consistency (line 297), the number of rollouts per sample is missing, and the upper bound of 0.9 rather than 1.0 lacks justification.\n\n- **Self-consistency accuracy is defined as the proportion of correct answers among 50 long-CoT rollouts per sample.** Specifically, we compute accuracy as n/50, where n is the number of correct final answers.\n\n- **In the offline setting, we select samples with self-consistency accuracy < 0.2 as hard samples.** \n\n- **In the online setting, we select samples with accuracy in (0, 0.9) to exclude easy samples.**  We also report in the table below the experimental results based on Qwen-2.5-VL-7B when relaxing the upper bound to 1. The results show that setting the upper bound to 1 leads to inferior performance compared to 0.9, which aligns with our earlier observation in MCTS-based offline selection—including too many easy samples may undermine reasoning improvement.\n\n|            | MathVista| MathVision | MathVerse | MMMU    | MMStar |MMBench | MM-Vet| AI2D |Avg.|\n| :----: |:----:|:----:|:----:|:----:|:----:|:----:|:----:|:----:|:----:|\n|SelfConsistency (0-0.9) paper| 74.2| 26.9| 50.1| 50.6| 64.8| 82.0| 67.1| 83.0| 62.34|\n|SelfConsistency (0-1)| 73.9| 26.1| 49.3| 50.9| 64.1| 82.2| 67.0| 82.7| 62.03|\n\n\n\n---\n\n> W3: The paper lacks discussion of inference costs for MCTS-based data sampling compared to self-consistency methods, which is crucial for evaluating practical applicability.\n\n- **We report the GPU hours required to run MCTS with 50 iterations and Self-Consistency with 50 rollouts over the full set of 70k samples, as shown in the table below.** We use 8xA100 GPUs for inference.\n\n|            | GPU hours|\n| :----: |:----:|\n|MCTS-5 interation| 8.94|\n|MCTS-50 interation| 29.38|\n|Self-Consistency-50| 27.49|\n\n- **While MCTS typically expands a much larger number of nodes compared to 50 rollout steps in Self-Consistency, in our setting, the search process is terminated early once a correct answer is found by the VLM via MCTS, and we record the actual number of iterations used.** This early-stopping mechanism significantly reduces the computational cost of MCTS, making it more efficient and feasible for large-scale sample evaluation.\n\n- Furthermore, in our study, we executed full MCTS (i.e., until a correct answer is found) across all samples solely for the purpose of analyzing the difficulty distribution shown in Figure 4. However, such exhaustive search is not required in practical applications.\nBy pre-defining an MCTS iteration threshold K for sample selection (see Lines 201–208 in the paper), we can significantly reduce the computational cost. For example, in our experiments, we selected only samples with more than 5 MCTS iterations for RL training (i.e., K=5, as described in Lines 205–207). Thus, in deployment, **we can simply run MCTS with a maximum of 5 iterations, while applying the aforementioned early-stopping mechanism (i.e., terminate once a correct answer is found).** As shown in the table, **this approach reduces the total MCTS cost from 29.38 GPU hours to just 8.94 GPU hours**, which is substantially lower than the cost of Self-Consistency-50, making data selection both efficient and scalable.\n\n\n\n---\n\n\n> W4: Additionally, to fairly compare offline versus online data selection, the authors should evaluate MCTS-based sampling in an online setting. Without this comparison, the conclusion that \"online filtering offers negligible improvement except faster convergence\" cannot be generalized across all data selection methods.\n\n- **We conduct both offline and online evaluations for MCTS-based and self-consistency-based selection.** The results using Qwen-2.5-VL-7B are summarized below:\n\n|            | MathVista| MathVision | MathVerse | MMMU    | MMStar |MMBench | MM-Vet| AI2D |Avg.|\n| :----: |:----:|:----:|:----:|:----:|:----:|:----:|:----:|:----:|:----:|\n|MCTS (Offline)| 75.1| 32.9| 52.1| 55.5| 65.0| 81.4| 67.8| 83.6| 64.18|\n|SelfConsistency (Offline)| 74.6| 30.9| 50.1| 53.8| 64.1| 81.3| 67.1| 83.3| 63.15|\n|MCTS (Online)| 74.7| 30.7| 51.7| 56.0| 65.2| 81.7| 67.8| 83.8| 63.95|\n|SelfConsistency (Online)| 74.2| 26.9| 50.1| 50.6| 64.8| 82.0| 67.1| 83.0| 62.34|\n\n- **MCTS outperforms self-consistency in both offline and online settings, demonstrating the robustness of our selection strategy.** Besides, offline MCTS performs best, suggesting that the offline vs. online discrepancy is an important future direction, which we plan to explore in follow-up work.\n\n\n\n---\n\n\n> W5: The motivation for applying this method specifically to visual large language models is unclear. The MCTS-based data sampling approach appears equally applicable to text-only model training, yet the paper lacks discussion of VLM-specific design considerations or challenges. It would be valuable to understand whether this method would achieve similar performance improvements when applied to text LLM training compared to other data selection methods, which would help establish the method's broader applicability and the specific value proposition for multimodal models.\n\n* **We focus on VLMs because multimodal reasoning remains a critical open challenge, where high-quality training data and scalable self-improvement methods are still scarce.**  Unlike LLMs, which have benefited from extensive CoT-style annotations and benchmark coverage, VLMs face a **modality gap**: visual inputs are more ambiguous, grounding is harder, and reasoning requires integrating visual perception with structured logic.\n\n* **Our method addresses these VLM-specific challenges with three concrete contributions:**\n\n  * **Visual grounding under open-ended formats**: We convert multimodal QA tasks from multiple choice to open-ended form to support more faithful difficulty estimation via MCTS, creating **a curated dataset with high-quality visual reasoning samples**.\n  * **Difficulty estimation from visual reasoning trajectories**: Our approach leverages model-internal iteration dynamics (e.g., how many steps to solve a visual question), which is less meaningful in text-only settings due to more predictable language priors.\n  * **Efficient, fully self-improving visual reasoning**: Our framework demonstrates SoTA gains on 8 VLM benchmarks *without* any external distillation, directly addressing the data and supervision bottlenecks in the VLM space.\n\n* **We agree that our MCTS-based difficulty estimation is broadly applicable and plan to extend it to LLMs.**  That said, applying it effectively in the text-only domain will require new considerations (e.g., disentangling linguistic variability from genuine difficulty), which we are excited to explore in follow-up work.\n\n\n---\n\nWe hope these clarifications address your concerns. Thank you again for your valuable comments."}, {"review_id": "Vw77elJXlL", "reviewer": "Reviewer_VM7j", "summary": "This paper introduces ThinkLite-VL, a methodology for enhancing the visual reasoning capabilities of Vision-Language Models (VLMs) through Reinforcement Fine-Tuning (RFT) using a significantly reduced number of training samples. The core contribution is a novel sample selection strategy that repurposes Monte Carlo Tree Search (MCTS) to quantify sample difficulty. This MCTS-based procedure measures difficulty by the number of reasoning iterations a base VLM (Qwen2.5-VL series) requires to correctly solve an instance. From an initial pool of 70k open-source examples, the authors filter high-quality subsets of 11k (for a 7B model) and 7.5k (for a 72B model) samples. These are then used for RFT, reportedly enabling ThinkLite-VL models to achieve state-of-the-art (SoTA) performance on several visual reasoning benchmarks, particularly MathVista, without explicit knowledge distillation during the RFT phase. The authors frame this as a data-efficient *self-improvement* process, though this improvement is critically guided by the external MCTS filtering stage.", "questions": "- Q1. The appendix mentions that MCTS filtering requires *substantial time*. To assess the method's overall efficiency, could you please provide a concrete breakdown of the computational cost (e.g., total GPU hours) for the MCTS filtering of the initial 70k samples, and compare this to the cost of the subsequent RFT phase? \n\n- Q2. How sensitive is the data selection process and final model performance to this constant and other choices like the expansion temperature and the specific critic model/prompt? \n\n- Q3. The training and test sets are largely distinct, but to rigorously test generalization, have the authors considered a stricter out-of-distribution experiment? For example, holding out all datasets from one category (e.g., *Chart Understanding*) during MCTS filtering and RFT, and then evaluating performance on benchmarks from that unseen category (e.g., TabMWP, IconQA if they have separate test splits not used elsewhere). \n\n- Q4. Do you have recommendations on when the provided curated datasets can be effectively used versus when a full, new MCTS filtering run is necessary?", "limitations": "Yes", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "$$\\textbf{Strengths}$$\n\n- The use of MCTS iterations, derived from a base model's reasoning process, to estimate sample difficulty for curating RFT training data is an intriguing approach. \n\n- The paper demonstrates strong performance improvements using remarkably few samples (11k for 7B, 7.5k for 72B) for the RFT stage, a significant reduction compared to many existing methods that require larger datasets for similar fine-tuning efforts. \n\n- ThinkLite-VL models achieve compelling results on eight VLM benchmarks, notably setting new SoTA scores on MathVista (75.1% for 7B, 79.7% for 72B), even outperforming some much larger models and proprietary systems. \n\n- The success of the RFT phase without requiring knowledge distillation from larger teacher models is a notable advantage, potentially simplifying the training pipeline for reasoning enhancement. \n\n- The paper not only finds that MCTS-derived difficulty is model-specific but also provides strong empirical justification in its appendix (Table 9) for its data selection strategy. It demonstrates that a specific mix of medium-difficulty (*Iter5+*) and very hard (*Unsolved*) samples yields optimal performance, outperforming both narrower and broader data subsets. \n\n---\n\n$$\\textbf{Weaknesses}$$\n\n- The paper's central claim of efficiency (*SOTA with Less*) pertains only to the number of RFT samples. The appendix reiterates that the MCTS filtering stage requires *substantial time* but provides no quantification of this cost. Knowing the GPU hours will help assess the method's true end-to-end computational efficiency. \n\n- The appendix reveals specific prompts and fixed parameters without exploring their sensitivity. The paper does not investigate how performance is affected by key MCTS hyperparameters.\n\n- The released datasets have limited *off-the-shelf* utility for different VLM architectures. Widespread adoption would require users to re-run the unquantified, potentially expensive MCTS filtering process for their specific models, hindering scalability.", "was_revised": true, "final_justification": "The authors' response effectively addresses the major criticisms and questions raised in the initial review. The previously \"unquantified\" computational cost is now good and shows that the MCTS filtering stage is significantly less expensive than the RFT phase, strengthening the paper's efficiency claims. The additional OOD results provide strong evidence for the generalizability of the method. While a full hyperparameter sensitivity study is still deferred, the provided insights are a reasonable first step. The authors' clarification on the reusability of the dataset and the rationale for model-specific filtering provides a clear path for future users.", "rebuttal": "We thank Reviewer VM7j for the thoughtful and constructive feedback. We appreciate Reviewer VM7j finds our proposed method novel, experiment results strong and extensive. We address each of your concerns in detail below.\n\n\n---\n\n> W1&Q1: The appendix mentions that MCTS filtering requires substantial time. To assess the method's overall efficiency, could you please provide a concrete breakdown of the computational cost (e.g., total GPU hours) for the MCTS filtering of the initial 70k samples, and compare this to the cost of the subsequent RFT phase?\n\n- **MCTS-based data selection is significantly more efficient than the RFT phase.** We report the total GPU hours required for MCTS filtering (at both 5 and 50 iterations) and for RFT with 300 training steps. We use 8xA100 GPUs for both inference and training:\n\n|            | GPU hours|\n| :----: |:----:|\n|MCTS-5 interation| 8.94|\n|MCTS-50 interation| 29.38|\n|RFT-300 step| 52.82|\n\n- **MCTS with early stopping and a small iteration cap yields practical scalability.** In our study, full MCTS runs (until a correct answer is found) were used solely to construct the difficulty distribution in Figure 4. For real deployment, we use a fixed iteration threshold K=5 (see Lines 205–207) and terminate early if the correct answer is found, greatly reducing cost without sacrificing selection quality. Moreover, our implementation of MCTS only requires searching, without the need for value function backpropagation. This significantly reduces computational complexity and makes the overall MCTS process much more lightweight and efficient.\n\n---\n\n> W2&Q2: The appendix reveals specific prompts and fixed parameters without exploring their sensitivity. The paper does not investigate how performance is affected by key MCTS hyperparameters. How sensitive is the data selection process and final model performance to this constant and other choices like the expansion temperature and the specific critic model/prompt?\n\n**Yes, MCTS hyperparameters, especially temperature and prompt format, materially affect sample difficulty estimation and downstream performance.** While we defer a full ablation to future work due to space and compute constraints, we summarize key insights below.\n\n* **Higher expansion temperatures improve exploration and yield better difficulty signals.** Raising the temperature during expansion increases reasoning diversity, allowing MCTS to discover more plausible solution paths and making the difficulty measure more discriminative. We will include this ablation study in the final version.\n\n* **Prompt design is critical for stable step-level reasoning in MCTS.** Since MCTS operates over reasoning trajectories, prompts that enforce step-by-step structure (e.g., consistent `<end>` separators, clear final answer format) are essential. Poorly structured prompts degrade both trace quality and difficulty estimation. We provide our CoT-style prompt template in **Appendix A1 (Table 6)**.\n\n* **Critic model choice has a smaller but non-negligible effect.** We use Qwen2.5-7B-Instruct for evaluating correctness, and found that using the same model as the rollout policy ensures more consistent reward signals. We plan to benchmark against alternative critics in future work.\n\n* **We plan to include a comprehensive hyperparameter sensitivity study in follow-up work.** While our current results are robust across tested seeds and configurations, we agree that systematic ablations would further solidify the findings. We will release code and configurations to support such extensions.\n\n\n---\n\n\n> W3: The released datasets have limited off-the-shelf utility for different VLM architectures. Widespread adoption would require users to re-run the unquantified, potentially expensive MCTS filtering process for their specific models, hindering scalability.\n\n* **We respectfully disagree — MCTS filtering is lightweight in practice, and re-executing it for different models is a necessary and principled design.** As detailed in our response to W3, filtering 70k samples with 5 MCTS iterations and early stopping requires only 8.94 GPU hours, which is ~3× faster than 50 rollouts self-consistency (27.49 GPU hours) and negligible compared to the RL training cost.\n\n* **The released ThinkLite-70K dataset remains broadly valuable and reusable.** It contains high-quality, standardized visual reasoning samples curated from diverse domains. This dataset can support evaluation, probing, or future model training even without rerunning MCTS — making it a useful resource independent of model-specific filtering.\n\n* **Reselecting samples per model is not a scalability flaw, but a feature of model-adaptive curriculum design.** Because difficulty is inherently model-dependent, applying MCTS to each target architecture ensures training samples are matched to the model’s current reasoning frontier — a key reason behind our strong results. This design parallels adaptive sampling methods in curriculum learning and RL.\n\n\n\n---\n\n> Q3: The training and test sets are largely distinct, but to rigorously test generalization, have the authors considered a stricter out-of-distribution experiment?\n\n- **To rigorously evaluate the generalization ability of our model, we select three additional OOD benchmarks: EMMA, ZeroBench, and MMVU.**\n  - EMMA is a multimodal scientific reasoning benchmark, where all option answer candidates are images.\n  - ZeroBench focuses on natural image reasoning; we carefully verified that none of its samples appear in our 70k training dataset, ensuring a clean test for generalization.\n  - MMVU is a video reasoning benchmark that represents a fully out-of-distribution (OOD) evaluation setting.\n\n|            | EMMA mini|ZeroBench sub|MMVU val|\n| :----: |:----:|:----:|:----:|\n|Qwen-2.5-VL-7B| 24.8|13.7|45.7|\n|ThinkLite-VL-7B| 29.8|18.9|49.8|\n\n- **Our model shows consistent gains across all OOD benchmarks.**\nThese results underscore the robustness and generalization capabilities of our method, even when applied to domains not seen during data selection or training.\n\n---\n\n\n> Q4: Do you have recommendations on when the provided curated datasets can be effectively used versus when a full, new MCTS filtering run is necessary?\n\n- **Here we propose a practical strategy for applying our method to a new base model: one can first perform an initial evaluation on a randomly sampled subset from the 70k dataset.** If the model's accuracy is comparable to that of Qwen-2.5-VL, it is reasonable to directly reuse the curated 11k hard samples for fine-tuning. However, if there is a substantial performance gap, it becomes necessary to re-run MCTS-guided selection to identify hard examples tailored to the new model's capabilities.\n\nWe hope these clarifications address your concerns and highlight both the efficiency and extensibility of our approach. Thank you again for your valuable comments."}, {"review_id": "AzY90cWt6Z", "reviewer": "Reviewer_JjLL", "summary": "This paper presents ThinkLite-VL, which utilizes MCTS to sample data and then fine-tune through RFT. These results show that MCTS based difficult filtering can provide a scalable and effctive path thwards data-efficient self-improvement.", "questions": "1.\tHow does Figure 3 show open-ended format is critical. The caption seems not to be relevant.\n\n2.\tIt seems that the proposed technique can be generally used in cross modal training. Have the authors tried this?\n\n3.\tWould the model forget old simple questions after only learning the difficult ones?", "limitations": "This part is relatively sound and the authors mention a meaningful limitation in the overall training efficiency. The reviewer does not penalize the corresponding limitation", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "## Strengths:\n1.\tThis paper focuses the problem of sample difficulty, which might be the core problem of RFT. \n2.\tThe proposed model seems to be generally effective across lots of benchmarks.\n## Weaknesses:\n1.\tAlthough sampling data is an important topic in general LLM training, this whole paradigm looks heuristic and empirical. A concrete limitation is that this framework cannot work if MCTS cannot find the correct answer after searching, where the MCTS search may not be meaningful there. This pipeline seems to be naïve and not novel. Although the proposed data sampling might be beneficial, it is only one choice, and there could be many possibilities uncovered.\n2.\tAbout the contributions. In the paper, (3) Data-efficient and (4) Strong empirical gains seem to be a duplicate. The fifth contribution open-source release isn’t clear. The authors do not mention whether they would release the training code or only the trained model. Also, authors would better submit some code to ensure or online demos for the reviewer to assess this contribution.\n3.\tWriting issues: \n(1)  Figure 1 and Figure 2 seem to be repetitive. \n(2)  Sampled data size through MCTS should be listed in Figure 3.", "was_revised": true, "final_justification": "The reviewer is convinced by the rebuttal.", "rebuttal": "We thank Reviewer JjLL for the insightful feedback and constructive suggestions. We address each concern in detail below.\n\n\n\n---\n\n> W1: Although sampling data is an important topic in general LLM training, this whole paradigm looks heuristic and empirical. A concrete limitation is that this framework cannot work if MCTS cannot find the correct answer after searching, where the MCTS search may not be meaningful there. This pipeline seems to be naïve and not novel. Although the proposed data sampling might be beneficial, it is only one choice, and there could be many possibilities uncovered.\n\n**We believe there may be a misunderstanding regarding our use of MCTS.** Our goal in to quantify the sample difficulty based on the number of iterations it takes for the VLM to reach the correct answer through MCTS. This difficulty metric is then used to guide training sample selection.\n\n- Specifically, we set an iteration limit for MCTS. **If the model fails to find the correct answer within the allowed iterations, the sample is marked as “unsolved” and included in the training set as a high-difficulty sample—not discarded or considered as “not working”.**\n\n- **Compared to the Self-Consistency approach, MCTS systematically explores a broader solution space, enabling the identification of more informative and challenging samples.** Our experiments demonstrate that MCTS-guided selection leads to superior model performance, highlighting its effectiveness.\n\n\n\n---\n\n> W2: About the contributions. In the paper, (3) Data-efficient and (4) Strong empirical gains seem to be a duplicate. The fifth contribution open-source release isn’t clear. The authors do not mention whether they would release the training code or only the trained model.\n\n- **Data-efficiency refers to achieving strong performance using a small, carefully selected subset (7.5k, 11k) of reasoning data—far fewer than existing works.**\n\n- **Strong empirical gains emphasize our model's state-of-the-art results across diverse VLM reasoning benchmarks.**\n\n- **Regarding open-source commitment, we will release the full training code, training dataset, and the ThinkLite-VL model weight to support transparency, reproducibility, and future research.**\n\n\n\n\n---\n\n> W3: Writing issues: (1) Figure 1 and Figure 2 seem to be repetitive. (2) Sampled data size through MCTS should be listed in Figure 3.\n\n\n- (1) Figures 1 and 2 serve distinct purposes. Figure 1 focuses on MathVista, showcasing state-of-the-art performance with fewer training samples, which demonstrates accuracy vs model size. Figure 2 highlights generalization **across multiple benchmarks**, emphasizing ThinkLite-VL’s consistent advantage over open-source baselines.\n\n- (2) Figure 4 presents the difficulty distribution of the final 11k hard samples. We will add sampled data size as suggested by the reviewer. \n\n\n\n---\n\n\n> Q1: How does Figure 3 show open-ended format is critical. The caption seems not to be relevant.\n\n* **We mention the open-ended format in the caption to highlight a key preprocessing decision that enables our difficulty-aware filtering.**\n  Many source datasets originally use multiple-choice questions, which can obscure true reasoning ability (e.g., via guessing).\n  By converting them to open-ended form, we ensure that both MCTS and reward evaluation operate on free-form reasoning — which better aligns with our goal of selecting samples based on reasoning complexity rather than answer selection. We have mentioned this in **Section3.1 Line156-160**.\n\n\n---\n\n> Q2: It seems that the proposed technique can be generally used in cross-modal training. Have the authors tried this?\n\n- **We use InternVL-2.5-8B as the base model and apply MCTS-guided selection to identify 19k challenging samples from ThinkLite-70k dataset for Reinforcement Fine-Tuning (RFT).** The MCTS sampling threshold is set to 5 iterations, consistent with our main setup. The training results are summarized in the table below. \n- **We observe that our approach substantially improves the reasoning performance of InternVL, demonstrating the generalizability of our method across different model architectures.**\n\n|            | MathVista| MathVision | MathVerse | MMMU    | MMStar |MMBench | MM-Vet| AI2D |Avg.|\n| :----: |:----:|:----:|:----:|:----:|:----:|:----:|:----:|:----:|:----:|\n|InternVL-2.5-8B| 64.4| 22.0| 39.5| 54.9| 62.8| 84.6| 68.8| 84.5| 60.19|\n|InternVL-2.5-8B-ThinkLite| 69.3| 24.5| 42.9| 56.2| 64.1| 84.9| 69.5| 84.7| 62.01|\n\n\n\n---\n\n> Q3: Would the model forget old simple questions after only learning the difficult ones?\n\n- **Our experimental results indicate that the model does not forget how to solve simple questions after RFT.** We evaluate ThinkLite-VL on the remaining set of easy questions and find that the model still performed well. This suggests that RFT enhances reasoning ability without compromising performance on simpler tasks.\n\n\n\n---\n\n\nWe hope these clarifications address your concerns. Thank you again for your valuable comments."}, {"review_id": "u7E3t56b95", "reviewer": "Reviewer_QP8R", "summary": "### Summary and Contribution\n\n- Sample difficulty is critical for effective VLM self-improvement—appropriately challenging examples that require deeper reasoning but remain solvable can drive substantial improvements even with limited data.\n- The authors repurpose Monte Carlo Tree Search (MCTS) to measure sample difficulty by counting how many reasoning iterations a base VLM needs to solve each training example. This allows them to rank and filter the most optimally challenging samples.\n- Starting with 70k open-source samples across math, natural images, and charts, they select only 11k samples (7B model) and 7.5k samples (72B model) based on MCTS difficulty scores, then apply pure RFT without knowledge distillation.\n- ThinkLite-VL-7B achieves 75.1% on MathVista (new state-of-the-art), outperforming much larger models like GPT-4o and O1. ThinkLite-VL-72B reaches 79.7% on MathVista.", "questions": "- Why do you think the performance when trained on all 70K worse than in the subset of dataset? \n- Can you provide a fair comparison by re-running the self-consistency baseline using exactly 11K samples (same as your method) instead of 23K? Additionally, can you clarify whether your self-consistency baseline uses chain-of-thought reasoning or direct answers, and provide results for both variants?\n- How much of your performance gain comes from MCTS-guided sample selection versus the RFT training approach? Can you compare RFT vs. supervised fine-tuning on the same MCTS-selected samples to isolate these effects?\n- The improvement over self-consistency (64.18% vs 63.15%) appears modest. Can you provide confidence intervals or statistical significance tests to establish whether these improvements are meaningful?\n- While you claim \"no SFT,\" you use Instruct models that have already undergone supervised fine-tuning. Can you test your approach on truly base models (non-Instruct) to better isolate the RFT contribution and validate your claims about self-improvement?\n- Why does your online data selection baseline select samples with accuracy 0 to 0.9 rather than prioritizing the hardest problems (lowest accuracy)? This seems counterintuitive given your difficulty-focused approach—can you explain this choice and show results with harder problem selection?", "limitations": "No, their conclusion section currently does not contain any discussion of limitations. I encourage the authors to appropriately discuss the limitations, based on some of the weaknesses I highlighted above.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "### Strengths\n\n- While model performance typically scales with data size, not all researchers have access to massive datasets and compute resources. This paper demonstrates that strategic sample selection can achieve state-of-the-art results with orders of magnitude fewer training examples, making high-performance VLM training more accessible.\n- The use of MCTS to quantify sample difficulty through reasoning iteration counts is creative and theoretically motivated, providing a principled alternative to random sampling or simple heuristics.\n- The authors include valuable ablations comparing their approach against self-consistency methods, online data selection, random sampling, and analysis of unsolved problems, demonstrating the benefits of their MCTS-guided approach.\n- Achieving new state-of-the-art on MathVista while using dramatically fewer training samples is impressive and practically significant.\n\n### Weaknesses\n\nWhile I do acknowledge multiple ablations that authors presented above, they are still lacking some key ablations. I discuss them in detail below. \n\n- Please correct me if I am wrong, but the self-consistency baseline appears to use direct answers rather than chain-of-thought reasoning. A fair comparison should include self-consistency with CoT prompting to isolate the benefits of MCTS-guided selection.\n- The self-consistency baseline in Table 3 uses 23K samples while the proposed method uses 11K. This makes direct comparison difficult—the authors should compare methods using identical sample counts (11K) to isolate the effect of selection strategy. They could either do random sampling from that 23K, or sort with increasing order of solve-rate and select first 11K. This is important, as they themselves report, training on FullSet performs worse than on carefully selected samples. \n- The authors claim RFT is sufficient, but don't compare against simple supervised fine-tuning on the same MCTS-selected samples. The performance gains might stem from good data curation rather than the RL training approach.\n- The improvement over self-consistency (64.18% vs 63.15%) appears modest. The authors should provide confidence intervals or significance tests to establish whether improvements are meaningful.\n- While claiming \"no SFT,\" the authors use Instruct models that have already undergone supervised fine-tuning on similar data. Testing on truly base models would better isolate the RFT contribution.\n- The online data selection baseline selects samples with accuracy 0 to 0.9, but doesn't justify why harder problems (lower accuracy) weren't prioritized, which seems counterintuitive given their difficulty-focused approach.\n\nMinor Comments:\n- The paper claims both 7B and 72B models achieve \"new SOTA\" on MathVista, but if 72B outperforms 7B, only one can be truly state-of-the-art. This should be clarified as \"SOTA within model class\" or similar.\n- The related work states they focus on \"large-scale RL,\" but the core contribution is data efficiency with small-scale, carefully selected samples. This framing mismatch should be resolved.\n- Lack of discussion of their limitations in conclusion section.", "was_revised": true, "final_justification": "I have raised my scores for quality, clarity, and significance based on the rebuttal, as the authors have addressed most of my concerns. I believe the paper will be clearer after the authors incorporate the suggested clarifications and explicitly discuss the limitations, as noted in my original review.\n\nI have maintained my overall score, which was already more positive than negative. While the core method is not novel—difficulty-aware training with reinforcement learning has been explored in prior work—I acknowledge that this paper is submitted under the \"Application\" track. Within that context, I believe the authors present a well-executed application of established methods to a relevant and challenging domain.", "rebuttal": "We thank Reviewer QP8R for the insightful feedback. We appreciate that Reviewer QP8R found our proposed method creative and theoretically motivated, experiment results strong and extensive. Below, we address each of your concerns in detail.\n\n\n\n---\n\n> Q1: Why do you think the performance when trained on all 70K worse than in the subset of dataset?\n\n**The 70K dataset contains many easy samples that add little learning signal.** Many examples in the full set are too easy and already solvable by the model, providing limited benefit and potentially introducing spurious patterns. By contrast, MCTS-selected samples are consistently challenging, helping the model improve reasoning rather than memorization.\n\n\n\n---\n\n> W1&Q2: Please correct me if I am wrong, but the self-consistency baseline appears to use direct answers rather than chain-of-thought reasoning. A fair comparison should include self-consistency with CoT prompting to isolate the benefits of MCTS-guided selection.\n\n**Self-consistency baseline uses the same CoT prompt as MCTS.** We confirm that the self-consistency baseline uses identical prompting to MCTS, encouraging long-form, step-by-step chain-of-thought reasoning rather than direct answer generation. The exact prompt is provided in **Appendix A.1 Table 6**, ensuring a fair comparison between the two methods.\n\n\n\n---\n\n> W2&Q2: The self-consistency baseline in Table 3 uses 23K samples while the proposed method uses 11K. This makes direct comparison difficult—the authors should compare methods using identical sample counts (11K) to isolate the effect of selection strategy. They could either do random sampling from that 23K, or sort with increasing order of solve-rate and select first 11K. This is important, as they themselves report, training on FullSet performs worse than on carefully selected samples.\n\n**Following your suggestion, we randomly sample 11k samples from the 23k dataset and performed Reinforcement Fine-Tuning (RFT).** The experimental results are presented in the table below. Compared to using the 23k set, the performance notably declined, indicating that self-consistency alone is insufficient to reliably identify the most challenging samples. This performance gap further underscores the effectiveness of our proposed MCTS-guided selection method in curating truly informative training data.\n\n|            | MathVista| MathVision | MathVerse | MMMU    | MMStar |MMBench | MM-Vet| AI2D |Avg.|\n| :----: |:----:|:----:|:----:|:----:|:----:|:----:|:----:|:----:|:----:|\n|SelfConsistency-11k| 73.5| 27.8| 50.6| 52.4| 63.9| 81.4| 66.9| 83.2| 62.46|\n|SelfConsistency-23k| 74.6 |30.9 |50.1| 53.8| 64.1| 81.3| 67.1| 83.3| 63.15|\n|ThinkLite-VL-7B| 75.1| 32.9| 52.1| 55.5| 65.0| 81.4| 67.8| 83.6| 64.18|\n\n\n\n---\n\n> W3&Q3: The authors claim RFT is sufficient, but don't compare against simple supervised fine-tuning on the same MCTS-selected samples. The performance gains might stem from good data curation rather than the RL training approach.\n\n**In the experiment summarized in the table below, we perform three epochs of full-parameter Supervised Fine-Tuning (SFT) on the selected 11k hard samples, using a learning rate of 1e-5.** Compared to the RFT results, we observe a substantial drop in performance. This degradation highlights a key distinction between the two training paradigms: during RFT, the model is required to actively engage in a thinking process to derive the final answer, thereby learning to solve problems autonomously. In contrast, SFT merely encourages the model to memorize the final answers without internalizing the reasoning process, leading to poorer generalization. These findings underscore the effectiveness and necessity of RFT for enhancing the model’s reasoning capabilities.\n\n|            | MathVista| MathVision | MathVerse | MMMU    | MMStar |MMBench | MM-Vet| AI2D |Avg.|\n| :----: |:----:|:----:|:----:|:----:|:----:|:----:|:----:|:----:|:----:|\n|Qwen2.5-VL-7B-SFT-11k| 69.2| 24.2| 47.4| 50.1| 61.6| 80.6| 65.3| 82.9| 60。16|\n|ThinkLite-VL-7B      | 75.1| 32.9| 52.1| 55.5| 65.0| 81.4| 67.8| 83.6| 64.18|\n\n\n\n---\n\n> W4&Q4: The improvement over self-consistency (64.18% vs 63.15%) appears modest. The authors should provide confidence intervals or significance tests to establish whether improvements are meaningful.\n\nWe conduct a paired t-test on the evaluation results of models obtained via MCTS (Ours) and Self-Consistency at the 7B-level offline setting, as reported in Table 4 across multiple benchmarks. **The test yields a t-value of 4.28 and a p-value of 0.0027. This statistically significant difference (p < 0.01) demonstrates that the performance gains achieved by MCTS are not due to random variation**, but instead reflect a consistent and systematic improvement in overall reasoning capability.\n\n\n\n---\n\n\n> W5&Q5: While claiming \"no SFT,\" the authors use Instruct models that have already undergone supervised fine-tuning on similar data. Testing on truly base models would better isolate the RFT contribution.\n\n- **Since Qwen2.5-VL does not provide a base version model, we instead conduct experiments using InternVL-2.5-8B as the base model.** Specifically, we apply MCTS-guided selection to extract 19k challenging samples from a 70k dataset for Reinforcement Fine-Tuning (RFT), using the same iteration threshold of 5 as in our main setup. The training results are shown in the table below. \n- **We observe that our method significantly improves the reasoning performance of InternVL**, demonstrating strong generalization across different model architectures. Moreover, these results highlight that RFT applied to a base model can still lead to substantial gains in reasoning ability.\n\n|            | MathVista| MathVision | MathVerse | MMMU    | MMStar |MMBench | MM-Vet| AI2D |Avg.|\n| :----: |:----:|:----:|:----:|:----:|:----:|:----:|:----:|:----:|:----:|\n|InternVL-2.5-8B| 64.4| 22.0| 39.5| 54.9| 62.8| 84.6| 68.8| 84.5| 60.19|\n|InternVL-2.5-8B-ThinkLite| 69.3| 24.5| 42.9| 56.2| 64.1| 84.9| 69.5| 84.7| 62.01|\n\n\n\n---\n\n\n> W6&Q6: The online data selection baseline selects samples with accuracy 0 to 0.9, but doesn't justify why harder problems (lower accuracy) weren't prioritized, which seems counterintuitive given their difficulty-focused approach.\n\n* **Our approach focuses on *model-centric* difficulty filtering — not curriculum learning.**\n  We assess difficulty based on the model’s own rollout behavior (via MCTS or self-consistency), and select samples that challenge the current model state. While curriculum learning imposes an ordering over data from easy to hard, our method does not enforce such a progression and is compatible with both offline and online setting.\n\n* **Exploring how to combine model-driven difficulty filtering with explicit curriculum strategies is a promising future direction.**\n  For example, one could dynamically anneal the acceptance threshold based on model improvement — but such mechanisms are outside the scope of this paper.\n\n\n---\n\n> Lack of discussion of their limitations in conclusion section.\n\n\n- We have summarized some potential limitations of our work in Appendix D, and we will include and discuss the additional weakness you mentioned in the updated version of the paper.\n\n\n\n---\n\nWe hope these clarifications address your concerns. Thank you again for your valuable comments."}]

Author rebuttal:

Meta-review: The paper proposes an approach for data selection for RFT training, based on the number of MCTS-iterations it takes the base model to solve a given training sample. The paper shows that selecting sufficiently difficult problems in this way (i.e., problems that require sufficiently many MCTS steps to solve, or are unsolvable by the base model) leads to significantly improved data efficiency during RFT, compared to other data selection methods.

All reviewers are positive about this submission, highlighting the novelty of the proposed approach and the strong empirical results for open-source models. The proposed method is simple, proven to lead to strong results, and works across multiple base models.

The reviews mentioned multiple important points that needed clarification, e.g. with regards to the computational cost of the proposed selection approach, but the authors addressed all of these during the rebuttal. One note: I would ask the authors to update their manuscript with these discussions — specifically, with the discussion about computational cost, the points about why they address VLM-specific problems, and the evaluations of their approach on other base models (InternVL). All of these strengthen the paper.

In summary, the paper is novel, easy to follow, and empirically effective. All reviewers are in favor of acceptance, and I recommend the paper as a spotlight presentation.
Decision: Accept (spotlight)