VisualPRM400K: An Effective Dataset for Training Multimodal Process Reward Models
Authors: Weiyun Wang
Zhangwei Gao
Lianjie Chen
Zhe Chen
Jinguo Zhu
Xiangyu Zhao
Yangzhou Liu
Yue Cao
Shenglong Ye
Xizhou Zhu
Lewei Lu
Haodong Duan
Yu Qiao
Jifeng Dai
Wenhai Wang
Venue: ICLR 2026 Poster (2026)
Keywords: Multimodal Large Language Models
Multimodal Process Reward Model
Primary area: applications to computer vision, audio, language, and other modalities

Abstract: We construct VisualPRM400K, a dataset comprising about 400K multimodal process supervision data. Building upon this dataset, we develop VisualPRM, an advanced multimodal Process Reward Model (PRM) capable of estimating the value score of each step during the reasoning process.
Under the Best-of-N evaluation setting, our model improves the reasoning performance of three types of MLLMs and four different model scales. 
Even when applied to the highly capable InternVL2.5-78B, it achieves a 5.9-point improvement across seven multimodal reasoning benchmarks.
Experimental results show that the PRM model trained on our VisualPRM400K exhibits superior performance compared to Outcome Reward Models and Self-Consistency during BoN evaluation.
To further facilitate the development of multimodal PRMs, we construct VisualProcessBench, a benchmark designed to measure the abilities of PRMs and MLLMs to detect incorrect steps in multimodal reasoning tasks.
We hope that our work can inspire more future research and contribute to the development of MLLMs. Our model, data, and benchmark will be released.

Published as a conference paper at ICLR 2026
VISUALPRM400K: AN EFFECTIVE DATASET FOR TRAINING MULTIMODAL PROCESS REWARD MODELS
Weiyun Wang Zhangwei G Lianjie Chen Zhe Chen Jinguo Zhu?, Xiangyu Zhao Yangzhou Liu Yue Cao Shenglong Ye², Xizhou Zhu Lewei Lu", Haodong Duan Yu Qiao², Jifeng Dai Wenhai Wang 8s Fuuan University, *Shanghai AI Laboratory. *Shanghai Jiaotong University, "Tsinghua University.
6 Nanjjng University, Thhe Chinese University of Hong Kong, "SenseTime Research
ABSTRACT
We construct VisualPRM400K, dataset comprising about 400K multimodal pro- cess supervision data. Building upon this dataset, we develop VisualPRM, an advanced multimodal Process Reward Model (PRM) capable of estimating the value score of each step during the reasoning process. Under the Best-of-N eval- uation setting, model improves the reasoning performance of three types of MLLMs and four different model scales. Even when applied to the highly ca- pable Intern VL2.5-78B, achieves 5.9-point improvement across seven mul- timodal reasoning benchmarks. Experimental results show that the PRM model trained on our VisualPRM400K exhibits superior performance compared to Out- come Reward Models and Self-Consistency during BoN evaluation. To further fa- cilitate the development of multimodal PRMs, we construct VisualProcessBench, a benchmark designed to measure the abilities of PRMs and MILMS to detect incorrect steps in multimodal reasoning tasks. We hope that our work can inspire more future research and contribute to the development of MLLMs. Our model, data, and benchmark will be released.
INTRODUCTION
With the remarkable success of Large Language Models (LLMs) (Touvron al., 2023a;b; Dubey et al., 2024; Bai et al., 2023a; Team, 2023; Cai et al., 2024; Brown et al., 2020; Achiam et al., 2023: Anthropic, 2024) in natural language processing, Multimodal Large Language Models (MLLMs) (Wang et al., 2023c; Li et al., 2023; Liu et al., 2023a; 2024a; Bai et al., 2023b; Wang et al., 2024e;d; Chen et al., 2023; 2024c;b; OpenAl, 2024; Reid al., 2024; Yao et al., 2024) have also achieved significant advancements across various vision-languse strong performance in perception and recogmition, a large gap remains in reasoning capabilities be- tween open-source and proprietary models. A series of studies have explored methods enhance reasoning abilities, focusing on the perspectives al., 2025; Toshniwal et al., 2025; Li et al., 2024b; Liu et al., 2024b), offline preference optimiza- tion (Pang et al., 2024; Wang et al., 2024c; Lai et al., 2024), and online reinforcement learning (Shao al., 2024; Guo et al., 2025; Ahmadian et al., 2024; Hu, 2025). Additionally, another line of search Snell et al. (2024); Dong et al. (2024); Zhang et al. (2025); Wang et al. (2023a) investigates utilizing Test-Time Scaling (TTS) to enhance the reasoning abilities of LLMs. This approach re- quires the policy model to generate multiple response candidates and select the best one, based on the quality estimation of critic model, thereby improving the response quality at the cost of higher inference time. However, TTS for MILMs remains largely unexplored.
tasks. Despite their
data collection and construction (Muennighoff
This work investigates the application of TTS for MLLMs, focusing on the Best-of-N (BoN) eval- uation strategies. The challenges of adapting TTS for MLLMs involves: (I) Lack of effective critic models. In BoN evaluation, a critic model is required to estimate the quality of each response can- didate. However, models, leading to marginal improvements compared to models without TTS. This limitation stems from the lack of sufficient critic data in their training corpus. (2) Lack of evaluation benchmarks
shown in Figure 1, existing open-source MLLMs struggle to
critic

---

Published as a conference paper at ICLR 2026
for multinodal critic models. The effectiveness of TTS heavily depends the performance of the critic model. However, directly evaluating critics under BoN settings poses two key issues. First, the evaluation cost of BoN is expensive. Although the focus is on the performance of critic models, the policy model is required to generate N reasoning processes, with the majority of computational costs arising from the policy model. Second, BoN performance is also affected by the poliey model, making it difficult to compare different critic models when paired with varying policy models.
To solve these challenges, first introduce Visu- alPRM400K, a dataset comprising approximately 400K multimodal process supervision data. Each sample includes an image, a question, a step-by- step solution, and correctness annotations for each step. Specifically, we collect question prompts from MMPR v1.1 (Wang et al., 2024c) and then gen- erate process correctness using automatic data pipeline (Wang et al., 2023a). This pipeline sam- ples multiple continuations starting from certain step and computes the expected accuracy of that step as the average accuracy of its continuations.
23
Oreud Pufaruse
45
$4.449
412
313
VENCTAN22 O1AR11255T1 InceWE1248 1OO09125-00 Irann122 5-28 500
May Maki
The overall Best-of-8 evaluation results across seven multimodal reasoning benchmarks with different critic models. Our VisualPRM greatly enhances the overall performance, while VL2.5-L struggles to be an effective critic model.
Figure I:
To facilitate the evaluation of multimodal critic models,
introduce VisualProcessBench, benchmark for evaluating PRMs and MLLMs in detecting erroneous steps in multimodal reasoning tasks. This benchmark includes 2,866 samples with 26,950
step-wise correctness labels. Each sample includes multimodal reason- ing question, step-by-ster solution, and correctness labels for each step. To ensure annotation accuracy, we employ human experts with at least university degree to manually assess the cor- rectness of each step. Unlike prior benchmarks (Zheng et al., 2024; Lightman et al., 2023), which require identifying only the first erroneous step, VisualProcessBence challenges models to detect all errors within a given solution. This adjustment aligns with recent advancements in model reflec- tion abilities, helping to reduce false negatives in evaluations. Evaluation results reveal that existing open-source MLLMs struggle accurately assess step-wiss wise correctness, highlighting the need for improved multimodal critic models.
Building upon the dataset and benchmark, we develop VisualPRM, advanced multimodal Pro- cess Reward Model (PRM) with 8B parameters, to serve as the critic model in BoN evaluation. Each training sample is formulated as multi-turn chat. The first turn includes the image, the ques- tion, and the first solution step, while each subsequent turn presents a new step. The model is trained predict the correctness of the given step at each turn. Experimental results demonstrate that VisualPRM enhances MLLM reasoning across different model families and scales. Specifi- cally. VisualPRM improves the overall reasoning performance of MiniCPM-V2, Qwen VL2..5 and VL..55LL by 8.0, 3.7, 8.4, and 5.9 points, respectively, across seven multimodal reasoning benchmarks (Yue et al., 2024; Lu et al., 2023; Wang et al., 2024b; Zhang et al., 2024b; Zou et al., 2024; Qiao et al., 2024; Xiao et al., 2024). Additionally, we compare PRMs with Outcome Reward Models and Self-Consistnncy in BoN evaluation, finding that PRMs consistently outperform both approaches.
In summary, our main contributions are as follows:
(1) We introduce VisualPRM400K, dataset comprising approximately 400K multimodal process supervision data. Building upon this dataset, we develop VisualPRM, an advanced multimodal PRM to serve the critic model in the BoN evaluation.
(2) We construct Visual,rocessBench,, benchmark designed to measure the abilitics of PRMs and MLLMs to identify erroneous steps in multimodal reasoning tasks. This benchmark comprises 2,866 samples with a total of 26,950 annotttdd step-wise correctness labels.
(3) Through extensive experiments, we demonstrate that PRMs serve effective critic models for test-time scaling of MILMs. Specifically, VisualPRM enhances the overall multimodal reason- ing performance of MiniCPM-V2.6, QwenVVL2.5-7B. Intern VL,,.5-8B, and Intern VI.2.5-78 by 8.0, 3.7, 8.4, and 5.9 points, respectively, across seven multimodal reasoning benchmarks. Further-
2

---

Published as a conference paper at ICLR 2026
Data Example from VisuIPRMH00K
Quesiin::
augls
Solatfen:
To fad baggl Siq pivan op quaditnnes wirt au peopery that sun kasteer sagle sfay cauddlavedd in 3a0ncre
Sisipp (wcmb.))
Stp 1: Witt dhe equatie for aus anglec EGs +5 sriiii sfiiii (Dc 7y/ee alear 390h
(2x+ 5)"
Noske (wenk 5071)
Shpp Siepl sepatee
R x'
6e 3301g
(2x + 7)
far
Step
lagle Slykki glves Stw-
Pand
Data Example from VisuaProcessBench
Question:
the tiperc At8 pastli U.D. ticeo ungle the the inglt oupp Chelas Boar Cash Doe
Sabatfen:
(Goood)
Goveery
regles oegporaiing
'I moudd wigt moninn hample "ange
Sup- (Grodd
Relarkead Berwee Augles laggi eagle BOG lank
50
Sap-1 (Ixomoot
Sass DO bauns lank W langle BDO lagge otr
(Grood
Fadle (aagle PV (igght
the GRP fyom onggeee hoooee ooad RG
Skp- (bnoment
Tharite, of rangl unge utr p9rianel.
Sup (Doonwnt
Fad Aarear The croaa optin R.A.
Data examples in VisualPRM400K and Bisua,,rocessBench. For VisualPRM400K, we generate the data using an automatic data pipeline. The key idea is to estimate the expected accuracy mn of the given step 8şi based on Monte Carlo sampling and consider the step correct if me; >> 0. During the training process of VisualPRM, the data is formulated as multi-turn conversations and the model is required to predict the correctness of each step conditioned on the image, question, and previous steps. For VisualProcessBenchh we collect questions from existing multimodal reasoning benchmarks and generate the solutions using leading MLLMs. Based these questions and solu- tions, we employ a team of human experts with at least university degree to manually annotate the correctness of each step in the solutions.
Figure 2:
more, our results show that PRMs consistently outperform both ORMs and SC in BoN evaluation. Additionally, experiments on VisualProcessBench reveal that existing open-source MLLMs struggle to accurately assess the correctness of each step.
RELATED WORK
Multimodal Large Language Models. wide range of efforts has been made to advance the de- velopment of MLLMs, including improvements in model architecture, data construction, and train- ing algorithms. From an architectural perspective, several studies (Liu et al., 2023a; 2024a; Yao et al., 2024; Chen et al., 2024b;c; Wang et al., 2024e;d; Liu et al., 2023b; Wang et al., 2024f; Bai et al., 2025; Yao et al., 2024) employ connectors to align visual embeddings from Vision Founda- tion Models (VFMs) (Chen et al., 2023; Zhai et al., 2023) with the latent space of LLMs (Bai al., 2023a; Touvron et al., 2023a;b; Team, 2023), achieving promising performance. Another series of works (Alayrac al., 2022; Dubey et al., 2024; Tian et al., 2024; Wang al., 2023b) extends pre-trained LLMs with additional layers to fuse visual features, reducing the number of required visual tokens while introducing extra training cost. In terms of data construction, recent studies have made significant progress (Schuhmann et al., 2022; Zhu et al., 2024; Laurençon et al., 2024; Li al., 2024b; Liu al., 2024b; Wang al., 2024c; Zhao al., 2025; Dong al., 2025; Cao et al.. 2025). For example, OmniCorpus (Li et al., 2024b) offers noisy but large-scale multimodal corpus for pre-training, while MMInstruct (Liu et al., 2024b) provides an open-source, high-quality instruction-tuning dataset. Additionally, MMPR (Wang et al., 2024c) constructs preference dataset focusing on multimodal reasoning abilities. Regarding training algorithms, the VL2.5 se- ries (Chen et al., 2024b; Wang et al., 2024c) proposes square loss and Mix Preference Optimization to enhance MLLM capabilities. Despite these advancements, existing works primarily focus on the training process of MLLMs, leaving Test-Time Scaling (TTS) for MLLMs largely underexplored. In this work, we investigate TTS applications for MLLMs, specifically focusing on the Best-of-N evaluation to improve multimodal reasoning performance.
3

---

Published as a conference paper at ICLR 2026
Question: in langle
Stop-lt: find (in langle ...
Stop-l: White equation
Step-4: Sabaitue (el) lack
Step
Final maaser
Queation & Solution
005
0JI
ExpertiedAAceuacy Accuracy
2
Correct (+)
(orrect
Incorrect (-)
Inceerect (-)
Valuc--asse FRM
D up
(o)
(a)
(+)
Advantage-hmee
Figure 3: Different modeling methods for PRMs. PRMs are developed to estimate the quality of each step in a given solution. For value-based PRMs, the quality of certain step is determined by its expected accuracy me, where step considered correct if mc; 0. For advantage-bar based PRMs, the quality of a certain step is determined by the improvement of mc; over mc-11. where step considered good if me; - m > O. During the training stage, the output space of PRMs is discretized into specific tokens, while during the inference stage, we compute the step score as the weighted sum of the generation probability for these discretized tokens.
Process Reward Models. Reward models play a crucial role in Reinforcement Learning (RL) (Schulman et al., 2017; Shao et al., 2024; Ahmadian et al., 2024; Hu, 2025) and TTS (Snell et al., 2024; Wang et al., 2023a; Dong et al., 2024; Luo et al., 2024). Outcome Reward Models (ORMs) (McAleese et al., 2024; Zhang al., 2024a; Wang al., 2024a) directly assign an overall score to the given response. In contrast, Process Reward Models (PRMs) first estimate the quality of each step in the given response and then aggregate them into final score. PRMS00K (Lightman et al., 2023) is the first open-source process supervision dataset, entirely annotated by human ex- perts. To reduce annotation costs, MathShepherd (Wang et al., 2023a) and OmegaPRM (Luo al., 2024) introduce a Monte Carlo sampling- based data pipeline to automatically estimate the quality of each step. Despite these advancements in natural language processing. multimodal PRMs remain largely underexplored. In this work, we introduce VisualPRM400K, the first multimodal process supervision dataset, and develop VisualPRM, a multimodal PRM trained on this dataset.
Benchmarks for Reward Models. The evaluation of reward models (RMs) is crucial research topic. A series of benchmarks (Lambert et al., 2024; Li et al., 2024a; Liu et al., 2024c) have been proposed to assess the effectiveness of RMs, typically formulated as a binary preference judgment task. Building on this, subsequent work (Zhou et al., 2024) extends the evaluation settings and in- cludes both pairwise and Best-of-N evaluations, providing a more comprehensive evaluation of RM performance. With the rapid advancement of PRMs, a series of benchmarks (Zheng et al., 2024; Song et al., 2025) have been introduced to evaluate their step-wise judgment capabilities. Despite these developments, there remains a lack of a multimodal process benchmark. To bridge this gap and support the development of multimodal PRMs, we introduce VisualProcessBench, benchmark designed to evaluate the ability of PRMs and MLLMs to detect erroneous steps in multimodal rea- soning tasks.
3 METHOD
During Best-of-N (BoN) evaluation, critic model is required to estimate the quality of each re- sponse candidate, In this work, we formulate the critic model as a Process Reward Model (PRM). To develop multimodal PRM, we first construct VisualPRRM40O,, dataset comprising about 400K multimodal process supervision data, as detailed in Section 3.1. We then describe our PRM's mod- eling approach in Section 3.2. Furthermore, to support the evaluation of critic models, establish Visual to measure the abilities of critic models to detect erroneous steps in multimodal reasoning, as introduced in Section 3.3.
3.1 VISUALPRM400K
Definition. As shown in Figure 2, each data sample in our VisualPRM400K consists of an image I E I, a question q E Q. a step-by-step solution s 81, } E S, and the expected accuracy annotation mc = {mcy, mC1, ... mc2) mc; E Rzo for each step, where n is the number of steps of a certain solution and mc; denotes the expected accuracy of step s,, The image sets I and question sets Q are collected from MMPR vl.1 (Wang et al., 2024c), while the step-by-step solutions S are sampled using Intern series models (Chen et al., 2024b; Wang al., 2024c).
4

---

Published as a conference paper at ICLR 2026
Process Supervision Generation. Given an image I, question q. and a solution {su, $1,-.. }. we annotate the correctness of each step s; using an automatic data pipeline. The key idea is to estimate the expected accuracy of given steps sci based Monte Carlo sampling. Specifically, the model is required to complete the solution as follows:
$>$ M q, 8gi):
where رب ; is the completion of ggi. Besides, the expected aceuracy of s; is defined as:
num(correct completions)
mc; num(sampled completions)
(2)
Notably, to reduce the data construction costs, we set the max number of steps 12 and evenly merge the steps if the number of current steps exceeds the threshold.
Statisties. During the construction process, we sample solutions for each image-question pair and split each of them into at most 12 steps. For each step, we sample 16 continuations and compute m; according to these continuations. The resulting dataset comprises approximately 400K samples and 2 million steps with process supervision. Each response averages 126.9 words and 5.6 steps, while each step averages 22.6 words. Among these steps, about 10% are incorrect steps. Despite the im- balanced distribution of correct and incorrect steps, our PRM demonstrates promising performance, as shown in Section
3.2 VISUALPRM
Overview. During the training process, we formulate the process supervision problem as a multi- turn chat task so that we can effectively leverage the generation ability of MLLMs, The image I, question g. and the first step 8p of the solution to this question are included in the first turn and new step is presented in each subsequent turn. The model is required to predict the quality of the given step each turn follows:
y4 M(n 8g9.
where denotes the quality of i-th step.
For value-based PRMs, the quality of certain step is determined by its expected accuracy me; which is similar to the definition of the value function in reinforcement learning. Following Math- Shepherd (Wang et al., 2023a; Dong al., 2024), we require the model to predict the correctness E [ of the given step, rather than the exact score of mcg The i-th step is considered correct if me; O. We also try to set threshold to reduce false positive steps, but find that such a threshold negatively impacts the PRM performance, as shown in Section B. Notably, unlike previous works (Wang et al., 2023a; Lightman et al., 2023; Dong et al., 2024), which choose to supervise only up to the first incorrect step, we always supervise all steps.
For advantage-basee PRMs, the quality of certain step is determined by the improvement of me; over mc;-1, which is analogous to the definition of the advantage function in reinforcement learning. Similar to aalue-based PRMs, the quality space is discretized into predefined values {+, meaning that the i-th step 8; results in a superior, comparable, inferior situation.
During inference stage, we first compute the scores of each step and then merge them to obtain the response score. Specifically, the score for each step is defined the weighted sum of the generation probability for the discretized scores. For value-based PRMss the weights for {+, -} are [1, 0}. For advantage-based PRMs, the weights for {+, -, - are {1, -11. Without further explanation, we average the scores of each step as the response score.
3.3 VISUAL.PROCSSSBNCH
Definition. Each sample our benchmark consists a multimodal reasoning question, a step-by- step solution, and correctness annotations for each step. Considering that recent models begin to demonstrate reflection abilities to rectify their own reasoning process, the evaluation setting used in previous works (Zheng et al., 2024; Lightman et al., 2023), which only requires the model to find the first erroncous step, may lead to a false negative estimation. Therefore, our benchmark requires the model to identify all erroneous steps in the given solution instead of only the first erroneous step.
5

---

Published as a conference paper at ICLR 2026
Data Source. Our benchmark focuses multimodal reasoning tasks, collecting images and ques- tions from existing representative multimodal reasoning benchmarks, including MMMU (Yue et al., 2024), MathVision (Wang et al., 2024b), Math Verse (Zhang et al., 2024b), DynaMath (Zou et al., 2024), and WeMath (Qiao et al., 2024). Given these questions, we generate step-by-step solu- tions using leading MILLMs, including GPT-4o (OpenAI, 2024), Claude-3.5-Sonnet (Anthropic, 2024), Gemini-2.0-Flas. (Team et al., 2023), QvO-72B-Preview (Team, 2024), and Intern VL2.5- 78B (Chen et al., 2024b). The solutions are sampled from different MLLMs to ensure their diversity.
Step Correctness Annotation. We employ a team of human experts with at least a university degree to manually annotate the correctness of each step the solutions. Specifically, 13 people worked for 3 days, resulting in workload of 39 person-days. The cost per person-day approxi- mately 37 dollars. During the annotation process, annotators are provided with the image, question, ground truth answer, and each step of the solution. Their task is to assign each step in the solution label of positive, negative, trated in Figure 2. A positive label indicates that the step is correct, while negative label signi- fies an incorrect step. The neural label is assigned to steps that do not involve any reasoning process or provide no additional information. To ensure the annotation quality, annotators are permitted to skip questions they do not understand. During the annotation process, our dataset is divided into 10 splits, each containing approximately 300 sam- ples. For each split, the authors of this paper manually review about 10% of the samples. Splits with erroncous annotations are sent back for re- re-annotation. See Section for more data examples.
Table 1: Statisties of VisualProcessBechh.
Statistics
Number
Total Samples
2866
-MMU
267
Math Vision
712
-Malt Verse
1026
DynaMath
570
WeMath
291
Source Solutions
2866
-GPT-4
870
-Cladde3 S.mnnt
865
1>QQ-72B-Previeww IIneemV....5..00
neutral, illus-
Total Steps
26950
Correcl Steps
Incorrect Steps
7691
-eural Steps
2674
Query Word Length Quartile
(22, 24, 50) (137. 193, 552) (13, 31, 67) 9.4
Response Word Length Quartile
Step Word Length Quartile
Number of Steps per Solution
Statistics. As shown in Table I, our benchmark comprises 2866 samples. To enhance the diversity of our evaluation samples, we gather questions and solutions from a wide range of benchmarks and models while carefully regulating their distribution. The statistics of step distribution presented in Section D.
macro F1 scores compare model performance, aiming to mitigate the impact of the imbalanced distribution between correct and incorrect steps. Specifically, we first compute the F1 scores separately for correct and incorrect steps and then take their average obtain the overall score.
Metrics. In this work,
4 EXPERIMENTS
In this section, we first employ VisualPRM to evaluate various MLLMs using BoN evaluation strate- gies in Section 4.1, demonstrating that PRMs can significantly enhance the reasoning abilities of MLLMs. Next, we evaluate our VisualPRM and other leading MLLMs on VisualProcessBencch in Section 4.2. Finally, the ablation studies are presented in Section 4.3 and Section B.
4.1 RESULTS WITH BEST-OF-N EVALUATION
Benchmarks. We evaluate the reasoning abilities of MLLMs across seven benchmarks, including MMMU (Yue et al., 2024), Math Vista (Lu et al., 2023), Vishon (Wang et al., 2024b), Math- Verse (Zhang et al., 2024b), DynaMath (Zou al., 2024), WeMath (Qiao et al., 2024), and Log- ic Visitta (Xiao et al., 2024). The evaluation samples include subject-based mathematical, and logical reasoning problems. We report the worst-case accuracy for DynaMath and the overall accuracy for the remaining benchmarks. For Math Verse, we report the performance the Vision-Only split.
Settings, Without further explanation. we use VisualPRM as the critic model for BoN evaluation and set N by default. The policy model is required to generate N distinct step-by-step Chain- of ooouhttt (CoT) reasoning processes with a temperature 0.7. The response with the highest score is then selected to determine the correctness.

---

Published as a conference paper at ICLR 2026
Table Results on seven multimodal reasoning benchmarks. MMMU is multidisciplinary rea- soning benchmark. MathVista, Vista, Math Vision, MathVerse Verse, DynaMath, and WeMath are mathematics benchmarks. For Math Verse, report the performance on Vision-Only (VO) split. Logic Vista is logical reasoning benchmark. Part of the results are collected from the OpenCompass leaderboard. The overall score the average score of the above benchmarks.
Model
MMMU
MathVista
MathVision
Math Verse-VO
DynaMath
WeMath
Logic
Overall
Proprictary Models
GPT-4o
70.7
60.0
31.2
40.6
34.5
52.8
47.9
Gemini-2.0 Flasaahh
69.9
70.4
43.6
47.8
42.1
47.4
52.1
53.4
Claude-3.5-Sonnet
65.3
35.6
46.3
35.7
60.4
Open- Models
V2.66PM-V26688
49.8
60.8
23.4
18.9
9.8
16.4
27.5
29.5
+VisualPRM
56.8
24.7
35.8
11.2
37.4
+7.0
+4.9
+1.3
+16.9
+1.4
+14.6
+9.8
+8.0
Twen2.5.VVL.-77
55.0
67.8
25.4
41.1
21.0
35.2
44.1
41.4
+VisualPRM
58.6
70.3
31.3
44,3
23.0
39.8
48.3
45.1
+5.9
LL2.5
64.5
17.0
22.8
9.4
36.0
32.8
+VisuaIPMM
60.2
68.5
25.7
35.8
18.0
36.5
43.8
41.2
+4.0
+4.0
+8.7
+13.0
+8.6
+13.0
+7.8
+8.4
InternVL..5 260
60.7
68.2
23.4
24.0
11.4
30.9
39.6
36.9
PVM
63.9
73.1
29.6
39.1
23.2
40,8
51.0
45.8
+3.2
+4.9
+6.2
+15.1
+11.8
+9.9
+11.4
VLtemVL2...33 LL2.5-88
PRM
69.0
73.9
35.2
46.7
30.5
46.2
53.7
50.7
+5.1
+2.0
+3.0
+9.8
+10.5
+7.9
+5.8
+6.3
InternVL.2.5-88B
70.0
72.3
32.2
39.2
19.2
39.8
49.0
46.0
+VisualPRM
35.9
+0.7
+2.8
+3.7
+7.9
+12.1
+9.3
+4.9
+5.9
Results. As shown in Table 2, VisualPRM greatly enhances the reasoning abilities of MLLMs across different model scales and families. Specifically. for models with fewer than 10 billion parameters, the overall performance of Intern VI.2.5-8B, MiniCPM-V-SB. and Qwen2.5-VI-77 improves by 8.4, 8.0, and 3.7 points, respectively, demonstrating the effectiveness of test-time scaling across different model families. For larger models, Intern VL2.5-26B, achieve substantial performance gains over their counterparts without TTS, further validating the scalability and effectiveness of TTS across different model sizes.
and Intern VL2.5-727 also
4.2 RESULTS ON VISUALPROCESSBENCH
Settings. For the evaluation of PRMs, step is considered correct if the probability of outputting "+" exceeds that outputting "_" by a certain threshold. For the evaluation of MLLMs, the model prompted to analyze each step and determine its correctness, classifying it either correct or incorrect. When computing the F1 score, we exclude steps labeled as neural by human annotators in Section 3.3.
Results. As shown in Table 3, most existing MLLMs struggle to accurately assess the correctness of each step. Specifically, the overall F1 score for random guessing 50.0, while most open-source MILMS achieve scores close to this baseline, highlighting their limitations as critic models. We manually check the judgments of these open-source MLLMs and observe that these models tend to provide positive analysis and label most steps as correct. For example, Intern VL2.5-8B achieves an Fl score of 76.8 for positive steps, while its F1 score for negative steps is only 19.2, indicating that Intern VL2.5- 1.2.5-88 5-B rarely identifies steps as incorrect. Furthermore, compared to proprietary models, our VisualPRM demonstrates competitive performance, achieving an overall score of 62.0- outperforming GPT-4o and GPT-4o-Mini, and performing on par with Gemini-2.0-Fin F..Fashh. Notably, our model, with only 8 billion parameters, more efficient than these proprietary counterparts.
4.3 ABLATION STUDIES
Effects of BoN. The training data for ORM are nearly identical to those used for PRM,

---

Published as a conference paper at ICLR 2026
Table 3: Results on VisualProcessBench. We report the macro Fl of the correct and incorrect
steps. The overall score is the micro average of the score from different data sources.
Model
MMMU
Math Vision
Math Vorse
DynoMath
WeMath
Overall
Random Guessing
50.0
50.0
50.0
50.0
50.0
50.0
Proprietary Modele
4oni-o-Mini
GPT-4o
$6.3
60.2
59.7
59.0
63.3
60.3
Gemini-2.0 Gemini-2.0-Flash 2.0-Flash
58.5
60.
62.8
66.7
$8.7
62.3
Open-source Models
MiniCPM-V2.6-88
44,9
50.9
58.9
46.7
50.4
LLVA-OV-78
42.2
LL-VA-OV-72B
46.1
48.4
53.0
57.0
57.3
$2.3
Qwen2.5-VL-2
53.1
$1.8
47.8
51.3
54.2
$1.0
@wen2.5 VL-72B
59.2
59.0
59.7
62.9
62.3
60.5
IntemVL2.55 2.55-885
47.1
45.5
47.8
50.3
50.8
48.0
InterVII 2.5. 5-26R
48.8
47.4
49.2
50.4
51.4
49.2
InterV 222.5-888
51.7
53.7
52.6
VisualPRM (ours)
except that all steps are concatenated into sin- gle step, and step-wise correctness annotations converted into a single correctness label for the outcome. Here, we increase the number of re- sponse candidates sampled from InternVL2.5-80 and select the final response using Cof--Consittency (SC) (Wang et al., 2022), Outcome Reward Model (ORM), and PRM. shown in Figure 4, increas- ing the number of response candidates N improves the reasoning performance of Intern VL2.5-8B and MiniCPM-V2.6 when using SC, ORM, or PRM, with PRM yielding the most significant improve- ments. Specifically, when using InterV the policy model, PRM outperforms SC and ORM by 2.4 and 1.5 points, respectively, under the Best-of-8 evaluation setting. Moreover, this perfor- mance gap widens as N increases, reaching 3.1 and 4.3 points when N is set to 128. Notably, when using ORM the critic model, although performance improves during Best-of-8 evaluation, further increasing N does not lead to consistent gains for Intern VL2.5-L2.5-8BB For example, the Best-of-128 performance inferior to the Best-of-64 64 performance.
a/ =aALRCD00-00
Bot
(a)
(b)
Figure Overall Best-of-N results across seven multimodal reasoning benchmarks with different policy and critic models.
Effects of PRM modeling methods. Here, we compare the value- based PRM and the advantage- based PRM introduced in Section 3.2, along with different methods for aggregating step scores into final score, including averaging. as well as selecting the maximum or minimum value. The results are presented in Table We find that value-based PRMs outperform advantage- based PRMs in both BoN evaluation settings and VL-ProcessBench. We attribute this to the inherent noise in our training data, which is generated through automatic data pipeline, making it challenging to accurately determine whether given step contributes higher lower expected accuracy. We also compare two training strategies: supervising all steps (i.e., w/o early stop) versus supervising only up to the first incorrect step (i.e., w. early stop) dur- ing training. Experimental results show that the former yields better performance. Regard- ing different score aggregation methods, we find that selecting the maximum value results in poorer performance compared to averaging or taking the minimum value. Analyzing the generated scores reveals that most responses contain high-scored step, close to 1, at the beginning of the solution. This phenomenon likely arises because
Table 4: Comparison of different critic models and score aggregation methods.
Critic Model
BoN
VL-PrrcessBBenhh
Pass@1 Random Guessing
32.8
33.0
50.0
InternVL.2.5689 Intem 78B
34,2
$2.6
Advantage PRM
+Min
36.8
$5.0
+Max
36.9
$5.0
+Average
37.4
$5.0
Value (w. early stop)
+Min
40.3
61.6
+Max
37.0
61.6
+Averag
40.6
61.6
Value twlo early stop)
+Min
40.4
62.0
+Max
35.9
62.0
+Average
8

---

Published as a conference paper at ICLR 2026
most erroneous steps appear in the middle of the solution. Our statistics of VisualProcessBench presented in Section further demonstrate this conclusion. Furthermore, averaging performs better than selecting the maximum value, likely because the latter relies single step's score, while av- eraging accounts for multiple steps and can be considered as an ensemble approach, which benefits the step quality estimation.
Juddeee---Judger. Existing MLLMs can be prompted to serve as a critic model. However, as shown in Table the InternVL2.5 series struggle improve BoN performance, resulting in only marginal improvements. Upon analyzing the generated scores, we find that these models assign similar scores most solutions. Consistent with our observations in Section 4.2, the Intern VL2.5 series tend to generate positive judgments for most steps, which hinders their ability to effectively distinguish and select the truly superior response. In addition to their effectiveness as critic models for MILMMs, their inference latency also limits efficiency. Specifically, MLLMs generate judgments for each step an autoregressive manner, which is time-consuming. computes scores for all steps in a single forward pass by using "+" as placeholder for model responses and interpreting its generation probability as the step score.
contrast, our VisualPRM
Results on text-only performance. To assess the effectiveness of VisualPRM on text-only inputs, we evaluate the Qwen2.5 series (Yang et al., 2024) and Intern series (Chen et al., 2024b) on three text reasoning benchmarks under BoN eval- uation settings: GSM8K (Cobbe et al., 2021), MATH-500 (Hendrycks et al., 2021), and GPQA- Diamond (Rein al., 2024). We report accu- racy as the evaluation metric for these benchmarks. As shown in Table our model enhances the text reasoning abilities of both the Qwen2.5 series and the I1tern series. Specifically, Qwen2.5- 7B achieves improvements of 6.1 and 5.0 points on MATH-500 and GPQA- Diamond, respectively. Similarly, Qwen2.5-72B demonstrates gains of 2.1 and 6.6 points on these benchmarks. For the Intern Intern series, IntemVVL2.5-8B,, internVL2.5. 38B, and VL2.5-7L2..578B achieve improvements of 9.4 and 5.0, 4.6 and 8.1, and 7.4 and 3.5 points, respectively, on MATH-500 and GPQA-Diamond. These results demonstrate the effectiveness of our VisualPRM in text-only scenarios.
Table 5: Results on text benchmarks.
Model
GSM8K
MATH-500
GPQA
Large Language Models
Owen2 5-78 +VisualRRM
91.6
75.5
36.4
94,5
81.6
41.4
+2.9
+6.1
+5.0
Owen2 5. 5.32B
95.9
83.1
49.5
+VisualRMM
96.1
85.4
53.5
+0.2
+2.3
+4.0
Qwen2 5-72 +VisualRMM
95.8
83.1
49.0
96.5
85.2
58.6
Multimodal Large Language Models
InternVL2 3-8B
81.9
56.8
29.3
+VisualPRM
82.9
66.2
34,3
+1.1
+9,4
+5.0
MntemVL22.5333
94.6
75.4
44.4
VVisaaPRMM
95.6
80.0
52.5
+1.0
InternVL.255 VL2
93.6
70.4
47.5
+VisualRMM
94.5
77.8
51.0
CONCLUSION
In this work, we construct VisualPRM400K, a dataset comprising about 400K multimodal process supervision data. Building upon this dataset, develop VisualPRM, an advanced multimodal Pro- cess Reward Model (PRM) capable of estimating the value score of each step during the reasoning process. With the Best-of-N (BoN) evaluation strategies, our model improves the reasoning abilities of existing Multimodal Large Language Models (MLLMs) across different model scales and fami- lies. Experimental results show that our model exhibits superior performance compared Outcome Reward Models and Seff-Consistency during BoN evaluation. To further facilitate the development of multimodal critic models, we construct Bisua,, a benchmark designed measure the abilities of PRMs and MILMMs to detect incorrect steps in multimodal reasoning tasks. Evalu- ation results show that existing open-source MI.LMs perform badly on this task. We hope that our work can inspire more research and contribute to the development of MLLMs.
Limitations. The main contributions of this study lie in the construction of the training dataset Vis0alPRM400K and the benchmark VisualProcessDench. Although we introduce VisualPRM as model baseline, our exploration of training and modeling strategies for multimodal PRMs is limited. We believe that further investigation into these strategies can advance the development of multi- modal PRMs, while we also emphasize that our proposed training dataset and benchmark provide a valuable foundation for future research on multimodal PRMs.

---

Published as a conference paper at ICLR 2026
ETHICS STATEMENT.
This study adheres to the outlined ICLR ethical guidelines and follows the principles of responsible research. Our dataset and benchmark are constructed from publicly available datasets, which mainly focus multimodal reasoning tasks. We confirm that no personally identifiable, sensitive, harm- ful data are utilized. Although human annotations are required during benchmark construction, they solely used to assess the correctness of each reasoning step and thus unrelated to any ethical concerns. Furthermore, we have carefully considered the potential societal implications of our work, including possible misuse, and conclude that contributions primarily serve advance scientific knowledge without introducing any foresecable risks.
REPRODUCIBILIIY STATEMENT.
We adhere to the reproducibility guidelines in the ICLR 2026 author guidelines. We will open- source our code, scripts, and data for training and evaluation to ensure the reproducibility of our experiments. Additionally, the training data construction pipeline will be open-sourced ensure the reproducibility of data construction. We also provide the key dyper-parameee in Appendix A.
ACKNOWLEDGMENTS
The work is supported by the National Key R&D Program of China (NO. 20222D016100)), the Youth PhD Student Research Project under the National Natural Science Foundation (No. 623B2050), and the National Natural Science Foundation of China (Grant No. 62376134).
REFERENCES
Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Ale- man, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774 2023.
Arash Ahmadian, Chris Cremer, Matthias Gallé, Marzieh Fadaee, Julia Kreutzer, Olivier Pietquin, Ahmet Üstün, and Sara Hooker. Back to basics: Revisiting reinforce style optimization for learn- ing from human feedback in llms. arXiv preprint arXiv:2402. 14740, 2024. 1,4
Cean-Baptiste Alayrac. Jeff Donahue, Pauline Luc, Antoine Miech, Iain Barr, Yana Hasson, Karel Lenc, Arthur Mensch, Katherine Millican, Malcolm Reynolds, et al. Flamingo: a visual language model for few-shot learning. NIPS, 35:23716-23736. 2022.
Anthropic.
Opus, sonnet, haiku. https:// URL https: https::/wwwwcche anthropic com/ delbe990.ccaaccccae55c33333338 htt.a//aacc.3..1oo.o....6666///2 Card Claude pdf.
The claude 3 model family:
www www. aww.ropicc.cccomm com, 2024.
6,6
Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, al. Qwen technical report. arXiv preprint arXiv:2309.166600 16609, 2023a. 1,
Jinze Bai, Shuai Bai, Shusheng Yang, Shijie Wang, Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, and Jingren Zhou. Qwen-vl: A frontier large vasion-language model with versatile abilities. arXiv preprint ar0iv:2308. 12966, 2023b.
Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, et al. Qwen2. 5-vl technical report. arXiv preprint arXiv:2502. 2025. 3
Ali Furkan Biten, Ruben Tito, Andres Mafla, Lluis Gomez, Marçal Rusinol, Ernest Valveny, CV Jawahar, and Dimosthenis Karatzas. Scene text visual question answering. ICCV, pp- 4291-4301, 2019. 21
10

---

Published as a conference paper at ICLR 2026
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. NIPS, 2020.
Zheng Cai, Maosong Cao, Haojiong Chen, Kai Chen, Keyu Chen, Xin Chen, Xun Chen, Zehui Chen, Zhi Chen, Pei Chu, et al. Internlm2 technical report. arXiv preprint arXiv:2403.174 172997 2024.
Jie Cao and Jing Xiao. An augmented benchmark dataset for geometric question answering through dual parallel text encoding. In COLING, pp. 1511-15200 2022. 21
Qi Cao, Ruiyi Wang, Ruiyi Zhang, Sai Ashish Somayajula, and Pengtao Xie. Dreamprm: Domain- reweighted process reward model for multimodal reasoning. arXiv preprint wrXiv:2505.22241, 2025. 3
Shuaichen Chang, David Palzer, Jialin Li, Eric Fosler-Lussier, and Ningchuan Xiao. Mapqa: A dataset for question answering choropleth maps. arXiv preprint arXiv:2211.0855 2022. 21
Qiguang Chen, Libo Qin, Jin Zhang, Zhi Chen, Xiao Xu, and Wanxiang Che. M3cot: A novel benchmark for multi-domain multi-step multi-modal chain-of-thought. arXiv preprint 24..22405.16775 2024a. 21
Zhe Chen, Jiannan Wu, Wenhai Wang. Weijie Su, Guo Chen, Sen Xing, Zhong Muyan, Qinglong Zhang, Xizhou Zhu, Lewei Lu, et al. Internvl: Scaling up vision foundation models and aligning for generic lingussin tasks. arXiv preprint ar 14238, 2023. 1,3
Zhe Chen, Weiyun Wang, Yue Cao, Yangzhou Liu, Zhangwei Gao, Erfei Cui, Jinguo Zhu, Shen- glong Ye, Hao Tian, Zhaoyang Liu, et al. Expanding performance boundaries of open-source multimodal models with model, data, and test-time scaling. arXiv preprint arXiv:2412.05271, 2024b. 1, 1. 4, 6, 9, 17, 18
Zhe Chen, Weiyun Wang, Hao Tian, Shenglong Ye, Zhangwei Gao, Erfei Cui, Wenwen Tong, Kongzhi Hu, Jiapeng Luo, Zheng Ma, et al. How far are we to gpt-4v? closing the gap to com- mercial multimodal models with open-source suites. arXiv preprint arXiv:2404. 16821, 2024c. 1,
Christopher Clark and Matt Gardner. Simple and effective multi-paragraph reading comprehension. In ACL, pp. 845-855, 2018. 21
Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, et al. Training verifiers to solve math word problems. arXiv preprint 2iiv:2110.114688 2021. 9
Guanting Dong, Chenghao Zhang, Mengjie Deng, Yutao Zhu, Zhicheng Dou, and Ji-Rong Wen. Progressive multimodal reasoning via active retrieval. In ACI. pp. 3579-3602, 2025.
Hanze Dong, Wei Xiong, Bo Pang, Haoxiang Wang, Han Zhao, Yingbo Zhou, Nan Jiang, Doyen Sahoo, Caiming Xiong, and Tong Zhang. Rlhf workflow: From reward modeling to online rlhf. arXiv preprint
2024. 5,
Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, al. The llama 3 herd of models. arXiv preprint arXiv:2407.21777 2024. 1,
Jiahui Gao, Renjie Pi, Jipeng Zhang, Jiacheng Ye, Wanjun Zhong, Yufei Wang, Lanqing Hong, Jianhua Han, Hang Xu, Zhenguo Li, et al. G-llava: Solving geometric problem with multi-modal large language model. arXiv preprint arXiv:2312. 13370, 2023.
Yash Goyal, Tejas Khot, Douglas Summers-Stare Dhruv Batra, and Devi Parikh. Making the v in vqa matter: Elevating the role of image understanding in visual question answering. In CVPR, pp. 6904-6913, 2017. 21
11

---

Published as a conference paper at ICLR 2026
Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song. Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-rl: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12990 2025.
Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. Measuring mathematical problem solving with the math dataset. arXiv preprint arXivv2 2021. 9
Jian Hu. Reinforce++: A simple and efficient approach for aligning large language models. arXiv preprint arXiv:2501.03262 2025. 1,4
Zheng Huang, Kai Chen, Jianhua He, Xiang Bai, Dimosthenis Karatzas, Shijian Lu, and CV Jawa- har. Icdar2019 competition on scanned receipt oer and information extraction. In 2019 Interna- fional Conference on Document Analysis and Recognition (ICDAR), pp. 1516-1520. IEEE, 2019. 21
Drew A Hudson and Christopher D Manning. Gqa: A new dataset for real-world visual reasoning and compositional question answering. In CVPR, pp. 6700-6709, 2019. 21
Kushal Kafle, Brian Price, Scott Cohen, and Christopher Kanan. Dvqa: Understanding data visual- izations via question answering. In CVPR, pp. 5648-5656, 2018. 21
Mchran Kazemi, Hamidreza Alvari, Ankit Anand, Jialin Wu, Xi Chen, and Radu Soricut. Ge- omverse: A systematic evaluation of large models for geometric reasoning. arXiv preprint 12244 2023. 21
Aniruddha Kembhavi, Mike Salvato, Eric Kolve, Minjoon Sco, Hannanch Hajishirzi, and Ali Farhadi. diagram is worth a dozen images. In ECCV, pp. 235-251, 2016. 21
Xin Lai, Zhuotao Tian, Yukang Chen, Senqiao Yang, Xiangru Peng, and Jiaya Jia. Step-dpo: Step- wise preference optimization for long-chain reasoning of llms. arXiv preprint arXiv:2406. 18629, 2024. 1
Nathan Lambert, Valentina Pyatkin, Jacob Morrison, LJ Miranda, Bill Yuchen Lin, Khyathi Chandu, Nouha Dziri, Sachin Kumar, Tom Zick, Yejin Choi, et al. Rewardbench: Evaluating reward models for language modeling. arXiv preprint arXiv:2403.1378/ 2024.
Hugo Laurençon, Lucile Saulnier, Léo Tronchon, Stas Bekman, Amanpreet Singh, Anton Lozhkov, Thomas Wang, Siddharth Karamcheti, Alexander Rush, Douwe Kiela, et al. Obelies: An open web-scale filtered dataset of interleaved image-text documents. NIPS, 36, 2024. 3
Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. In ICML, pp. 19730-19742. PMLR, 2023.
Lei Li, Yuancheng Wei, Zhihui Xie, Xuqing Yang, Yifan Song, Peiyi Wang, Chenxin An, Tianyu Liu, Sujian Li, Bill Yuchen Lin, et al. Vlrewardbench: challenging benchmark for vision- language generative reward models. arXiv preprint arXiv:2411.174.. 2024a. 4
Qingyun Li, Zhe Chen, Weiyun Wang, Wenhai Wang, Shenglong Ye, Zhenjiang Jin, Guanzhou Chen, Yinan He, Zhangwei Gao, Erfei Cui, et al. Omnicorpus: An unified multimodal corpus of 10 billion-level images interleaved with text. arXiv preprint (exii:2446.08411 2024b. 1,
Hunter Lightman, Vincet Kosaraju, Yuri Burda, Harrison Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. Let's verify step by step. In The Twelfth International Conference Learning Representations, 2023. 2, 4,
Adam Dahlgren Lindström and Savitha Sam Abraham. Clevr-math: A dataset for compositional language, visual and mathematical reasoning. arXiv preprint arXiv:2208. 05358, 2022. 21
Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. NIPS, 36, 2023a. 3,3
12

---

Published as a conference paper at ICLR 2026
Haotian Liu, Chunyuan Li, Yuheng Li, Bo Li, Yuanhan Zhang, Sheng Shen, and Yong Jae Lee. Llava-next: Improved reasoning, oct, and world knowledge, January 2024a. URL https://
1lava-vl. alava-v1.github 1o
1,3
Yangzhou Liu, Yue Cao, Zhangwei Gao, Weiyun Wang. Zhe Chen, Wenhai Wang, Hao Tian, Lewei Lu, Xizhou Zhu, Tong Lu, et al. Mminstruct: A high-quality multi-modal instruction tuning dataset with extensive diversity. arXiv preprint urXiv:2407. 15838, 2024b. 1,3
Yantao Liu, Zijun Yao, Rui Min, Yixin Cao, Lei Hou, and Juanzi Li. Rm-bench: Benchmarking reward models of language models with subtlety and style. arXiv preprint arXiv:2410.66184, 2024c. 4
Zhaoyang Liu, Yinan He, Wenhai Wang, Weiyun Wang, Yi Wang, Shoufa Chen, Qinglong Zhang, Zeqiang Lai, Yang Yang, Qingyun Li, Jiashuo Yu, et al. Interngpt: Solving vision-centric tasks by interacting with chatgpt beyond language. arXiv preprint (rxxv:23350056633 2023b. 3
Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. arXiv preprint
arXiv:1711.05 2017. 17
Pan Lu, Ran Gong, Shibiao Jiang, Liang Qiu, Siyuan Huang, Xiaodan Liang, and Song-Chun Zhu. Inter-gps gps: Interpretable geometry problem solving with formal language and symbolic reasoning. arXiv preprini 1005:22005.04655 2021a. 21
Pan Lu, Liang Qiu, Jiaqi Chen, Tony Xia, Yizhou Zhao, Wei Zhang, Zhou Yu, Xiaodan Liang, and Song-Chun Zhu. Iconqa: A new benchmark for abstract diagram understanding and visual language reasoning. arXiv preprint arXiv:2110. 13214, 2021b. 21
Pan Lu, Swaroop Mishra, Tanglin Xia, Liang Qiu, Kai-Wei Chang, Song-Chun Zhu, Oyvind Tafjord, Peter Clark, and Ashwin Kalyan. Learn to explain: Multimodal reasoning via thought chains for science question answering. NIPS, 35:2507-222 2521,, 2022. 21
Pan Lu, Hritik Bansal, Tony Xia, Jiacheng Liu, Chunyuan Li, Hannaneh Hajishirzi, Hao Cheng, Kai- Wei Chang, Michel Galley, and Jianfeng Gao. Mathvista: Evaluating mathematical reasoning of foundation models in visual contexts. arXiv preprint arXiv:2310.02255, 2023. 2,
Liangchen Luo, Yinxiao Liu, Rosanne Liu, Samrat Phatale, Harsh Lara, Yunxuan Li, Lei Shu, Yun Zhu, Lei Meng, Jiao Sun, et al. Improve mathematical reasoning in language models by automated process supervision. arXiv preprint arXiv:2406. 06592, 2, 2024.
Kenneth Marino, Mohammad Rastegari, Ali Farhadi, and Roozbeh Mottaghi. Ok-vqa: A visual question answering benchmark requiring external knowledge. CVPR, pp. 3195-3204, 2019. 21
Ahmed Masry, Xuan Long Do, Jia Qing Tan, Shafiq Joty, and Enamul Hoque. Chartqa: A bench- mark for question answering about charts with visual and logical reasoning. ACL, pp. 2263- 2279, 2022. 21
Minesh Mathew, Viraj Bagal, Rubèn Tito, Dimosthenis Karatzas, Ernest Valveny, and CV Jawahar. Infographievqa. In WACV, pp. 1697-1706, 2022. 21
Nat McAleese, Rai Michael Pokorny, Juan Felipe Ceron Uribe, Evgenia Nitishinskaya, Maja Tre- bacz, and Jan Leike. Llm critics help catch 11m bugs. arXiv preprint arXiv:2407.00215, 2024.
Anand Mishra, Shashank Shekhar, Aject Kumar Singh, and Anirban Chakraborty. Ocr-vqa: Visual question answering by reading text in images. In ICDAR, pp. 947-952, 2019. 21
Niklas Muennighoff, Zitong Yang, Weijia Shi, Xiang Lisa Li, Li Fei-Fei, Hannanch Hajishirzi, Luke Zettlemoyer, Perey Liang, Emmanuel Candès, and Tatsunori Hashimoto. sl: Simple test-time scaling. arXiv preprint XXiv:2501.1939) 2025.
OpenAl. Got-4o system
card. https:/ /openai.n.....
2024. 1, 6
13

---

Published as a conference paper at ICLR 2026
Richard Yuanzhe Pang. Weizhe Yuan, Kyunghyun Cho, He He, Sainbayar Sukhbaatar, and Jason Weston. Iterative reasoning preference optimization. arXiv preprint arXiv:2404.19733 2024.
Runqi Qiao, Qiuna Tan, Guanting Dong, Minhui Wu, Chong Sun, Xiaoshuai Song, Zhuoma GongQue, Shanglin Lei, Zhe Wei, Miaoxuan Zhang, et al. We-math: Does your large multi- modal model achieve human-like mathematical reasoning? arXiv preprint D.Xiv:4407001280 2024. 2, 6
Machel Reid, Nikolay Savinov, Denis Teplyashin, Dmitry Lepikhin, Timothy Lillicrap, Jean- baptiste Alayrac, Radu Soricut, Angeliki Lazaridou, Orhan Firat, Julian Schrittwieser, et al. Gem- ini 1.5: Unlocking multimodal understanding across millions of tokens of context. arXiv preprint arXiv:2403.05533 035500 2024. 1
David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Di- rani, Julian Michael, and Samuel Bowman. Gpqa: graduate-level google-pree srooee-rroofooo q&a bench- mark. In First Conference on Language Modeling, 2024. 9
Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Wortsman, al. Laion-5b: An open large-scale dataset for training next generation image-text models. NIPS, 35:25278-25294, 2022. 3
John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv preprint arXiv: 1707.06347, 2017. 4
Minjoon Sco, Hannanch Hajishirzi, Ali Farhadi, Oren Etzioni, and Clint Malcolm. Solving geometry problems: Combining text and diagram interpretation. In Proceedings of the 2015 conference on empirical methods in natural language processing, pp. 1466-1444440 1476, 2015. 21
Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Y Wu, et al. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300. 2024. 1, 4, 17
Amanpreet Singh, Vivek Natarajan, Meet Shah, Jiang, Xinlei Chen, Dhruv Batra, Devi Parikh, and Marcus Rohrbach. Towards vqa models that can read. In CVPR, pp. 8317-8326, 2019. 21
Charlie Snell, Jaehoon Lee, Kelvin Xu, and Aviral Kumar. Scaling Ilm test-time compute optimally can be more effective than scaling model parameters. arXiv preprint arXiv:2408.03314, 2024. 1,
4
Mingyang Song, Zhaochen Su, Xiaoye Qu, Jiawei Zhou, and Yu Cheng. Prmbench: A fine-grained and challenging benchmark for process-level reward models. arXiv preprint arXiv:2501.03124, 2025. 4
Gemini Team, Rohan Anil, Sebastian Borgeaud, Yonghui Wu, Jean- Captiaae Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, et al. Gemini: family of highly capable multimodal models. arXiv preprint arXiv:2312.11805 2023.
InternLM Team. Internlm: A multilingual language model with progressively enhanced capabilities.
httpps:/giithie com/ Int.ent//eei////mmm Int InternIM, 2023. 1,3
Qwen Team. Qvq: To see the world with wisdom, December 2024. URL https:://wwnll github. hwt.i///b//lgooee qvee qvp-72b------ bebb-pevveew... 6
Changyao Tian, Xizhou Zhu, Yuwen Xiong. Weiyun Wang, Zhe Chen, Wenhai Wang, Yuntao Chen, Lewei Lu, Tong Lu, Jie Zhou, et al. Mm-interleaved: Interleaved image-text generative modeling via multi-modal feature synchronizer. arXiv preprint arXiv:2401. 10208, 2024.
Shubham Toshniwal, Ivan Moshkov, Sean Narenthiran, Daria Gitman, Fei Jia, and Igor Gitman. Openmathinstruct-I:: 1.8 million math instruction tuning dataset. Advances in Neural Informa- fion Processing Systems, 37:34737-34774 2025.
14

---

Published as a conference paper at ICLR 2026
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models. arXiv preprint arXtw:2302.13971, 2023a, 1,
Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Niko- lay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. Llama 2: Open founda- tion and fine-tuned chat models. arXiv preprint arXiv:2307.09288 2023b. 1,3
Haoxiang Wang. Wei Xiong, Tengyang Xie, Han Zhao, and Tong Zhang. Interpretable preferences via multi-objective reward modeling and -fxture-of-expprts. In EMNLP, 2024a. 4
Ke Wang, Junting Pan, Weikang Shi, Zimu Lu, Mingjie Zhan, and Hongsheng Li. Mcasuring multimodal mathematical reasoning with math-vision dataset. arXiv preprint arXiv:2402.1480 2024b. 6,6
Peiyi Wang. Lei Li, Zhihong Shao, RX Xu, Damai Dai, Yifei Li, Deli Chen, Yu Wu, and Zhifang Sui. Math-shepherd: Verify and reinforce Ilms step-by-step without human annotations. arXiv preprint arXiv:2312.089933 2023a. 1, 2,2,4,5 4,5
Weihan Wang, Qingsong Lv, Wenmeng Yu, Wenyi Hong. Ji Qi, Yan Wang. Junhui Ji, Zhuoyi Yang, Lei Zhao, Xixuan Song, et al. Cogvim: Visual expert for pretrained language models. arXiv preprint arXiv:231 1123311.000 1.03079, 2023b.
Weiyun Wang, Zhe Chen, Wenhai Wang, Yue Cao, Yangzhou Liu, Zhangwei Gao, Jinguo Zhu, Xizhou Zhu, Lewei Lu, Yu Qiao, et al. Enhancing the reasoning ability of multimodal large language models via mixed preference optimization. arXiv preprint S.dV::2411.10442. 2024c. 1, 2,3.4, 4, 17
Weiyun Wang, Yiming Ren, Haowen Luo, Tiantong Li, Chenxiang Yan, Zhe Chen, Wenhai Wang, Qingyun Li, Lewei Lu, Xizhou Zhu, et al. The all-seeing project v2: Towards general relation comprehension of the open world. arXiv preprint arXiv:2402. 19474, 2024d. 1,3
Weiyun Wang, Min Shi, Qingyun Li, Wenhai Wang, Zhenhang Huang, Linjie Xing. Zhe Chen, Hao Li, Xizhou Zhu, Zhiguo Cao, et al. The all-seeing project: Towards panoptic visual recognition and understanding of the open world. In ICLR, 2024e. 1,3
Weiyun Wang, Shuibo Zhang, Yiming Ren, Yuchen Duan, Tiantong Li, Shuo Liu, Mengkang Hu, Zhe Chen, Kaipeng Zhang, Lewei Lu, et al. Needle in a multimodal haystack. arXiv preprint MXXi:24006.07230. 2024f.
Wenhai Wang, Zhe Chen, Xiaokang Chen, Jiannan Wu, Xizhou Zhu, Gang Zeng, Ping Luo, Tong Lu, Jie Zhou, Yu Qiao, et al. Visionllm: Large language model is also open-ended decoder for vision-centric tasks. NIPS, 36, 2023c.
Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoe Le, Ed Chi, Sharan Narang, Aakanksha Chowdh- ery. and Denny Zhou, Self-consistency improves chain of thought reasoning in language models. arXiv preprint arXiv::203: /171, 2022. 8
Yijia Xiao, Edward Sun, Tianyu Liu, and Wei Wang. Logievista: Multimodal Ilm logical reasoning
benchmark in visual contexts. arXiv preprint arXiv:2407.04973, 2024. 2, 6
An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, et al. Qwen2. technical report. arXiv preprint 11X1::2412.15115. 2024.
Yuan Yao, Tianyu Yu, Ao Zhang, Chongyi Wang. Junbo Cui, Hongji Zhu, Tianchi Cai, Haoyu Li, Weilin Zhao, Zhihui He, et al. Minicpm-v: A gpt-4v level milm on your phone. arXiv preprint arXiv:2408.00 01800, 2024. 3, 18
Xiang Yue, Yuansheng Ni, Kai Zhang, Tianyu Zheng, Ruoqi Liu, Ge Zhang, Samuel Stevens, Dongfu Jiang, Weiming Ren, Yuxuan Sun, Cong Wei, Botao Yu, Ruibin Yuan, Renliang Sun, Ming Yin, Boyuan Zheng, Zhenzhu Yang, Yibo Liu, Wenhao Huang, Huan Sun, Yu Su, and Wenhu Chen. Mmmu: A massive multi-discipline multimodal understanding and reasoning benchmark for expert agi. In CVPR, 2024. 6
15

---

Published as a conference paper at ICLR 2026
Xiaohua Zhai, Basil Mustafa, Alexander Kolesnikov, and Lucas Beyer. Sigmoid loss for language image pre-training. In ICCV, pp. 11975-11988 2023. 3
Lunjun Zhang, Arian Hosseini, Hritik Bansal, Mehran Kazemi, Aviral Kumar, and Rishabh Agarwal. Generative verifiers: Reward modeling as next-token prediction. arXiv preprint arXiv:2408. 15240,, 2024a. 4
Renrui Zhang. Dongzhi Jiang, Yichi Zhang, Haokun Lin, Ziyu Guo, Pengshuo Qiu, Aojun Zhou, Pan Lu, Sai-Wei Chang, Peng Gao, Mathverse: Does your multi-modal IIm truly see the diagrams in visual math problems? arXiv preprint arXiv:2403. 14624, 2024b. 2, 6
Zhenru Zhang, Chujie Zheng, Yangzhen Wu, Beichen Zhang, Runji Lin, Bowen Yu, Dayiheng Liu, Jingren Zhou, and Junyang Lin. The lessons of developing process reward models in mathematical reasoning. arXiv preprint arXiv:2501.07301, 2025. 1, 17
Xiangyu Zhao, Shengyuan Ding, Zicheng Zhang, Haian Huang, Maosong Cao, Weiyun Wang, Ji- aqi Wang, Xinyu Fang, Wenhai Wang, Guangtao Zhai, et al. Omnialign-v: Towards enhanced alignment of milms with human preference. arXiv preprint arXiv:2502. 18411, 20255,
Chujie Zheng, Zhennu Zhang, Beichen Zhang, Runji Lin, Keming Lu, Bowen Yu, Dayiheng Liu, Jingren Zhou, and Junyang Lin. Processbench: Identifying process emrors in mathematical rea- soning. arXiv preprint arXiv:2412.06552 2024. 2, 4,5
Enyu Zhou, Guodong Zheng, Binghai Wang, Zhiheng Xi, Shihan Dou, Rong Bao, Wei Shen, Limao Xiong, Jessica Fan, Yurong Mou, et al. Rmb: Comprehensively benchmarking reward models in Ilm alignment. arXiv preprint vxXv:2440.00989) 2024. 4
Wanrong Zhu, Jack Hessel, Anas Awadalla, Samir Yitzhak Gadre, Jesse Dodge, Alex Fang, Young- jae Yu, Ludwig Schmidt, William Yang Wang, and Yejin Choi. Multimodal c4: An open, billion- scale corpus of images interleaved with text. NIPS, 36, 2024. 3
Chengke Zou, Xingang Guo, Rui Yang, Junyu Zhang, Bin Hu, and Huan Zhang. Dynamath: A dynamic visual benchmark for evaluating mathematical reasoning robustness of vision language models. arXiv preprint 20x:::441100082 2024. 2, 6
16

---

Published as a conference paper at ICLR 2026
Table 6: Performance after Reinforcement Learning. The model trained with the additional
process reward introduced by VisualPRM achieves better performance than the vanilla GRPO.
Model
MathVista
VathVision
MathaVerse
DynaMath
WeMath
LisicVista
Overall
Intern 2iterVL22.5888
64.5
17.0
22.8
9.4
23.5
36.0
32.8
+GRPO
55.9
67.2
25.7
28.0
13.6
27.5
39.1
36.7
+Process Reward
59.0
69.2
27.0
32.1
16.8
29.1
42.7
39.4
A TRAINING HYPER-PARAMETERS
During the training process VisualPRM, the data-packing strategy (Chen et al., 2024b) enabled during training. We employ the AdamW optimizer (Loshchilov & Hutter, 2017) with the 81 of 0.9, the 32 of 0.999, and the weight decay of 0.05. The learning rate initialized as 1e-5. The training phases include linear warmup that lasts until the first 5% of training steps. The warmup is followed by a cosine decay strategy with a minimum learning rate of 0. We set the training epoch to 1.
B MORE ABLATION STUDIES
B.1 EFFECTIVENESS OF VISUALPRM ON REINFORCEMENT LEARNING
In this section, we fine-tune Intern VL2.5-8B using GRPO (Shao et al., 2024), a widely used re- inforcement learning (RL) algorithm that relies correctness- based outcome rewards. Building upon GRPO, we integrate VisualPRM to provide additional process-level rewards. Specifically, for each correct response, individual reasoning steps are further rewarded according the step scores predicted by VisualPRM. The training dataset consists of approximately 30K multimodal questions, randomly downsampled from MMPR v1.1 (Wang et al., 2024c). During training, set the learning rate to 1e-6. Each training iteration processes 512 queries, and the model parameters are updated with eight PPO steps (i.e., 32 queries per step). For each query, we sample 16 responses. For evalu- ation, we use the same benchmarks discussed in Section 4.1. The experimental results, summarized in Table 6, show that GRPO significantly enhances the reasoning capabilities of VL2.55 improving its overall performance from 32.8 to 36.7. Furthermore, by incorporating the process reward introduced by our proposed VisualPR.M the overall performance is further boosted to 39.4, demonstrating the effectiveness of the process reward provided by VisualPRM.
B.2 EFFECTS TRAINING HYPER-PARAMETERS
When training our value-based Process Reward Model (PRM) using VisualPRM400KK, we define step as correct if its expected accuracy exceeds 0. In this section, we analyze the impact varying expected accuracy thresholds for determining step correctness. As shown in Table 7, increasing the threshold results a decline in both Best-of-8 evaluation performance and VisualProcessBenchh scores. These results are consistent with the observation in 1ren2.5-Math-PPE (Zhang et al., 2025). Therefore, we suggest setting the threshold to 0 during training.
B.3 EFFECTS OF GENERATION
In this section, we analyze the impact of generation temperature on the Best-of-8 evaluation. As shown in Table 7, as the temperature increases from 0.3 to 1.3, the overall performance of InternVL.2.5-88 first improves and then declines. We attribute this phenomenon to the trade-off between response diversity and accuracy. When the temperature is low (e.g., set 0.3), the gener- ated responses lack diversity, limiting the model's performance upper bound. Conversely, when the 1.3), the responses become more random, reducing the accuracy of
temperature is high (e.g., individual responses and lowering the model's 's overall performance ceiling.
B.4 EFFECTS OF BEST-OF-N EVALUATION
In this section, we present the Best-of-N evaluation results as N increases, as shown in Table 8 and Table 9. Our results indicate that as N increases, VisualPRM consistently enhances the reasoning
17

---

Published as a conference paper at ICLR 2026
Table 7: Ablation studies about the effects of expected accuracy threshold and generationo temperaure.
Model
MMMU VathVista MathVision MatliVerse-OO DynaMath WeMath LogicVista
VisualPrroeexBench
Threshold
Threshold-0.00
59.3
68.5
25.7
35.8
18.0
36.5
43.8
41.1
62.0
Threshold=0.625
66.8
35.0
40.4
61.0
1nrehhod=001125
$8.0
67.9
27.6
35.4
17.4
35.3
41.6
40.5
60.7
Temperature
T3mperature=0.3
59.7
69.4
26.0
32.6
17.6
35.5
42.7
40.5
Temperature-0.7
Tampecature=1.0 00 61.7
67.2
27.3
35,8
16.6
34.2
43.2
40.9
1emperatur==1.3
57.9
66.1
25.0
32.1
16.8
31.9
40.5
38.6
Table 8: Overall Best-of-N results of Intern VL2.5-8B across seven multimodal reasoning benchmarks with different critic models.
Model
BoN |MMMU
Vision
MathVerse-VO DynaMath WeMath
LogicVista Overall
56.2
17.0
32.8
58.0
65.9
23.4
30.5
18.4
32.7
43.0
38.8
Self Consistency
32
16
60.4
58.6
65.8
28.0
26.1
32.6
32.1
20.8
19.4
34.1
33.0
43.4
41.0
39.8
64
59.7
66.7
26.6
33.2
20.6
35.8
43.4
40.9
128
60.6
67.4
25.7
32.0
22.6
34.7
43.2
40.9
56.2
64.5
17.0
22.8
9.4
23.5
36.0
32.8
60.2
67.0
25.3
32.5
16.4
35.0
41.8
39.7
VisuaIRMM
27.0
39.1
39,3
32
58.6
67.9
26.3
33.6
17.4
34.4
42.1
40.0
64
59.4
66.8
28.6
33.9
17.8
34.1
42.3
40.4
28.3
56.2
17.0
23.5
32.8
60.2
68.5
25.7
35.8
18.0
36.5
41.2
VisualPRM
16
60.2
69.9
27.3
36.4
19.0
38.8
42.5
42.0
32
60.3
64
61.4
69.6
30.6
38.2
18.8
40.2
45.4
43.5
128
61.7
70.8
30.3
39.3
19.4
40.9
45.4
44.0
abilities of Intern V1.2.5-8B (Chen et al., 2024b) and MiniCPM-V2.6 et al., 2024). Specifically, as increases from 8 to 128, the overall performance of Intern improves from 41.2 to 44.0. while MiniCPM-V2. -2.6 improves from 37.5 40.4, demonstrating the scalability of Test-Time Scaling for MLLMs.
B.5 EFFECTS ROLLOUTS FROM DIFFERENT MODELS
Here provide additional analysis of the data pipeline. Specifically, incorporate additional PRM training data constructed from rollouts sampled using Qwen2.5-VL-7B and HiMo-VL-7B, and we report the BoN performance of PRMs trained this expanded dataset. We construct about 74K additional samples (44K from Qwen2.5-VL. and 30K from MiMo-VL-7B). The construction process is identical to that of VisualPRM4000S with the only difference being the policy model used for sampling. As shown in Table 10, experimental results confirm that incorporating more diverse rollouts from additional models further improves VisualPRM performance. Notably, "444K rollouts from Dwen2.5-VIL indicates that the training dataset is constructed by adding an additional 44K samples on top of VisualPRMA0OK.
B.6 EFFECTS OF MAXIMUM NUMBER OF STEPS
Here, we analyze how the maximum number of steps data construction influences model perfor- mance. To reduce annotation cost, reused the annotations from the max-step 12 configuration to build datasets with max-step 1/3/6/9. We then trained separate VisualPRM models on these subsets. As shown in Table II. increasing the maximum step count does improve performance, but the gains exhibit clear diminishing returns- for instance, increasing the max steps from 9 to 12 yields only very marginal improvemett.
18

---

Published as a conference paper at ICLR 2026
Table Overall Best-of-N results of MiniCPM-V2266 across seven multimodal reasoning bench- marks with different critic models.
Model
BoN
MMMU
Vision
MatlVeerse-OO
DynuMath
WeMath
LigicVista
Overall
49.8
23.4
18.9
16.4
27.5
29.5
51.8
58.9
21.7
31.5
10.0
22.6
35.6
33.2
Self Consistency
16
51.7
60.2
21.7
31.5
11.6
25.7
35.3
34.0
32
52.2
24.3
33.1
11.4
24.3
36.0
34.5
64
51.7
61.0
23.4
34,8
12.8
25.8
35,3
35.0
49.8
60.8
23.4
18.9
9.8
16.4
27.5
29.5
56.7
66.0
22.0
33.5
10.2
24.1
38.9
35.8
VisualORM
16
36.4
65.3
24.0
32.1
10.4
27.3
36.5
36,0
32
64
58.2
67.3
22.7
35.5
11.0
30.1
37.6
37.5
49.8
60.8
23.4
18.9
9.8
27.5
29.5
56.8
37.5
VisuaIPMM
16
58.8
68.6
24.0
37.3
12.4
32.7
39.8
39.1
39,9
64
58.6
09.4
25.3
39.7
12.2
38.2
36.9
40.0
59.3
69.4
25.3
39.1
14.4
37.0
40.4
Table 10: Ablation studies on the effects of rollouts from different models. The overall Best-of-8 performance is calculated as the average Best-of-8 score across MMMU, Math Vista, Math Math Vision, Math Verse-Ve DynaMath, WeMath, and Logic Vista.
Dataset
Overall Best-of-8 Performance
VisualProcessBeencl Acc.
VisualRMMH0KK
41.2
62.0
+4K rollouts from Qwen2.5-VL
40.9
61.5
+30K rollouts from MiMo-VL
41.8
62.7
+74K rollouts from Qwen-2,5-VL and MiMo-VL
41.9
63.1
C MORE EXPERIMENTAL RESULTS
C.1 EVALUATION RESULTS BEYOND REASONING BENCHMARKS
As shown in Table 12, we additionally report the model's BoN performance on MMBench, MMStar (perception ability), ChartQA and A12D (chart understanding), and OCRBench (OCR capability), using the same experimental setup as in Table 2. The results show that VisualPRM-based BoN con- sistently yields performance improvements, with substantially larger gains than Self-Consistency. These findings provide further evidence of VisualPRM's 's potential beyond pure reasoning tasks.
C.2 LATENCY AND THROUGHPUT
Table 13 reports the average per-response latency, which measures the overall latency of generating eight response candidates under the Best-of-8 setting and producing the final response. Since Visu- alPRM computes step-level scores in parallel (i.e., using "+" placeholder and deriving its score from the output probability, requiring only one forward pass to obtain scores for all steps), does not introduce significant extra latency compared with Self- Seff-CConistenc.. The dominant latency in the overall inference pipeline still comes from the model's autoregressive generation process.
Notably, use Imdeploy to sample response candidates from the policy model and transform- ers to evaluate step-level process rewards with VisualPRM. Compared with Self-Consistency, the VisualPRRM-bus based Best-of-N evaluation adds only the latency of scoring responses. Intuitively, the additional latency introduced by VisualPRM roughly comparable to the latency required for the policy model (of the same size) to generate the last token of the response.
19

---

Published as a conference paper at ICLR 2026
Table 11: Ablation studies on the effects of maximum number of steps. The overall Best-of-8 performance is calculated as the average Best-of-8 score across MMMU, Math Vista, Math Vision, Math Verse-VO. DynaMath, WeMath, and Logic Vista.
Settings
Overall Best-of-8 Performance
VisuadProceeseBench Acc.
stepse1
39.7
58.7
max stepse3
40.3
60.0
stops=6
40.7
61.5
max steps=9
41.2
61.7
sleps=12
Table 12: Overall Best-of-8 results on perception-related benchmarks.
MMBench
ChariQA
OCRBench
MiniCPM-V2.663B
78.0
$7.5
82.4
82.1
85.2
77.0
Self Consistency
78.3
59.2
84.0
83.3
86.4
78.2
w. VisualPRM
80.3
62.1
85.0
84.3
87.0
79.7
Qwen2.5. Qw.-2,5.VLL-BB VL-7B
82.6
63.9
87.3
83.9
86.4
80.8
Self-Consistency
87.7
VisuaIPRM
83.7
64.8
87.9
85.0
86.9
81.7
Intern VL2.5-8B
Self Consisieenyy
84.4
64.5
85.6
84,9
82.5
80.4
VisuaIPRM
Intern VL2.55L2....88
87.4
69.5
88.3
89.1
85.4
83.9
Self.Consistency
87.2
70.3
89.9
89.5
86.0
84.6
W. VisualPRM
87.5
72.0
89.7
89.7
90.3
85.8
5
Mativisoo Mattiree
DynaMaah WeeMlled
80
4-
SweomeetSSgggssco (4g)
Steps (%)
60
40
20
Step Incls
Step Index
(a)
(b)
Figure 5: Step Distribution of VisualProcessBench. The X-axis represents the step index. (a) The Y-axis indicates the proportion of steps at each index relative to the total number of steps, reflecting the distribution of step positions in solutions. (b) The Y-axis represents the error rate of steps at each index, showing the likelihood of errors occurring at different step positions.
D MORE STATISTICS FOR VISUALPROCESS
The statistics for step distribution of VisualProcessBencch is presented in Figure 5. We observe that most solutions consist of fewer than 15 steps. Among these solutions with fewer than 15 steps, most solutions contain about 7 or 13 steps. For the correctness of each step, we observe that the error rate lower in the first three steps and then increases as the step index grows. We attribute this to the fact that problems requiring rise in step error rates. Notably, starting from step 15, the error rate drops sharply. This is because the number of steps in this range is relatively small, resulting in significant statistical fluctuations.
reasoning steps tend to be more challenging, leading
gradual
20

---

Published as a conference paper at ICLR 2026
Table 13: The overall Best-of-8 performance and average per-response latency of Selr- Consistency and VisualPRM.
Model
Overall Best-of-8
Lafency per response (s)
IntemWL2..-8BBSC
38.8
7.51
Inlem Vis
41.2
7.76
47.0
20.00
Vntem - VisualPRM
50.7
20.26
Intern Intem=LL..5-788-50
49.2
27.94
Intern 1nlermVL2.5-7888 W...2......7-.aaaaPPP. Vis
51.9
28.33
OCR
Question:
of
Tacts Ska Bnd
Selution:
(
To desernine sed da novior dilke
AP (pemMA122)
1.terty thet price soubte
"6. Racker 55.00
Tatto
burkuling
Arblip- (w2)).062))
Al the chere
seriice chaqee RM (8.33
TETTAL
IM 6-
AiSlipi (WCr0.5)
ACcondicc unsuen po the eacomen:
"Thal Sakeo
Thk uckk (RTTrikk
Cask pidd BM 103
T.Suleuu De soall adlo Duan sumoerr
Change serleed IMM 191 io IMv sn
108
(a)
Document
Quadion:
Selution::
0425)
Edkmmn tacilie menienad dicanee, thos
18755
(oo-k 125)
(b)
E MORE DATA EXAMPLES IN VISUALPRM400K
In this section, we provide more data examples of VisualPRMM000 in Figure 6 from different do- mains, including general visual question answering (VQA) (Goyal et al., 2017; Hudson & Manning, 2019; Marino et al., 2019; Lu al., 2021b), science (Kembhavi et al., 2016; Chen et al., 2024a; Lu et al., 2022), chart (Masry et al., 2022; Kafle et al., 2018; Chang et al., 2022), mathematics (Lind- ström & Abraham, 2022; Seo al., 2015; Cao & Xiao, 2022: Lu et al., 2021a; Kazemi et al., 2023; Gao et al., 2023), OCR (Mathew et al., 2022; Singh et al., 2019; Biten et al., 2019; Huang et al., 2019; Mishra al., 2019), and document (Clark & Gardner, 2018).
F MORE DATA EXAMPLES IN VISUALPROCESSBENCC
In this section, provide data examples in VisualProcessBench from different data sources. Specifically, we randomly choose three examples from our benchmark and visualize them Fig- 7. Additionally, Figure 8, we provide an example where the model initially generates an incorrect answer and then autonomously corrects it.
G LARGE LANGUAGE MODEL. USAGE.
In this submission, we utilize LLMs (e.g., GPT-5) to help us polish paper writing.
21

---

Published as a conference paper at ICLR 2026
Mathematics
Situs didkkey qlums fine rasy plansan lett
Sebutien:
opher
Arttipd (apmb AiSky (wundd))
qaotiee alo de offer ramoring Bond noceed witnect aphor ofgphrr
Seience
Quevtio::
vith wlst
Nomoa Fii
Plass
hood spiien centicad bulte.
Solutiem:
type oernpend gpen mabyve fourpeet
12bernal hawe Tha o Supeiaa gananily shnw crinly dirzhead promur
sile, ndi-dstedd arcoos
Siige (wen2.)
of
typically shona
Bod prostay spates grend
abmust
Siigl (uunt.))
3.8gi This type tor
ilantly searrlly medits
Baciiill beawiing oten wuilding
du higge owent (17))
4.lg hau gjice scmes sclent the giecel stroll foct naher the typ be prow datihaaoo bons
Ael Stged (wenld))
Upes curniing Di aurus D. the fooppian aboos dun hacted and the teat orthe Sei ar beching greand whili dee ack dinlacty natti.
Ta chandleratt of nummul fout, visco thate aciar and loving irprzvice en goond.
He Sty (win)u))
Fail werer Narnal
General VQA
Question:
Does puare iyneery?
To litee pieare syneartyy &i rinc cangl
Litesss the object the cuan thi cuic it rorieed reuase andern Beng stycct
2.ta at ekpcck slips. The diject appaas to hi rpmeusical, staaie ki grouatica areada the daat
caldd tf ats
UTasppeeent are endernslllllll chole apeiiin reins of avdf to cerut workid
Ae Sippp (ecedu))
kimzzi crtutt
Chart
Semerit oo Mdnitaes Endrames Load Park
beyac fowrgee polie??
5019)
dete unto funcwe parcetage Derescataaal dddicee heleew pany aldera Gosige
polioy ve nou book at carookeding valacre the labi.
20 Sigit (4210.5123)
Libeeicy the percetin
U'mnees wao trenk Demounss thery beker teadin fevigg pelloy.
buth Tasiga pello" has the parceagge is 41%
Weiiey (werdh 171)
7. Mever' the percoe oi Reyell cars who thsk the Rey lave Parry hete havll forige palloy 10%.
bnthe
3200mmb butwoen
Doewent
deidiing percentnge Demovrmta
Figure 6: More data examples from VisualPRM400K.
22

---

Published as a conference paper at ICLR 2026
Data Example from MMMU
Quatien:
Salation:
(laos)
detiations:
(Cammo)
""Spucch Tii rolo bervous domona
rSSlapptt bood ans lhei aleaess popestire toos siel obscti
(Canwo
Data Example from MMMU
Questio:
Chice of dcenddyy
Eraijr Caliie sikerr
Soletion:
To antomn the key moledy enceyo byoo the sigarano on onklens and the taad ooner Horc be trcaldioves
Stgul (Clewc))
"Myy Siguseett The vately stmeet
spaaner wilk slep (9 whelh crrope edler ragarhing
*Muadewadin tt The owiidy sersina uu De auciiand viice lading soca thikey of Eriiee The srggeaa
Migal (Camwo) (Crmad)
e* Cowl Thooedady revolves esoud Suther foring *Ht winert
Coneat maavet
(b)
Data Example from MMMU
Quntiia:
sep'ss
Salation:
(Camno))
undeoaad
acrecais owarder came, voach capically rdlatmma banaze
lodasee eseployneen the whhe woth incrasses leading to beflatia
0
NAIRE)
aneppioemut approiinuctt
sapponibustty
(Commoy)
"Megactoo preoseeiiee sepptt growt
seppl goowt typindly yr wiigge intk
mimoryy skoy rhilgs
ehange
syreent
Stgtp- (Comet)
"phiilpp exserica" lou ras, higher ae
aftect sataal tncmptaynunt, commnaa aane inftaiion "Macette he peeath the now poer where connery willoe the ling will tave went ancepp mont mate the
beel
anwealleee nava raland
Us Sigu- (beeemo))
(912 heln aigen enlitee vac Thes wilty point eane.
us Hupid Olann)
Giver daaccaage
Step (ecanw)))
seneen fee Ru C
witta inneeeed sapply grodd can
Feal Tralimm
Example MathViisio
Question:
opov
Magerrenee
Satutioa:
easietto hame foo
descrde bocwiiiing look
Uncared
Raveer
(d)
23

---

Published as a conference paper at ICLR 2026
Data Example from Math
Quetton
Itet Naue anaver qaoeon and pronic the oornet cake lates cat Dus
Quutie of dolaving tie pieuus dow placee of propher can quateat facdaa Whic piece descet bakee?
Cheice AAJ di (CC (750 (808
Solatien:
Toudberthe this pootoole ofter quekin drawnack
prophe:
Thi
vid
"Oppis
pering
mupile zume mocminn from kongae qasdmite tecaerppning opwands
HOppas decnaaii fhom left aeaiion with quatrate feaciis speriay
Siigp (Grood
The ga beciau aana partoos.
skig- (Coment)
"ntured Anower
Example MathVision
Quration
Salution:
daget
ansighoning
wrge withihinngg
Stg-d Diaan)
Skppyyyaup rawsnngg
Roy- (Aesned))
casear caseeraaaaae fut with hoade paintiag secsti ane
pown daelut slde.
6f Megu (Aeemmnc
2 Whes
shreghinet pood rik wir new was enninng tovarh the ring will by now madk are sligned sigeddee with postion in staaahoad tai.
et Sky (Aeenad))
Frow the thninngge applar than only the pawe 5u ant du lunt sc kuglgaa with the dutta saaiglooedd Bas Faalaaowar
of Steu (Peaned))
Data Example from MathVerse
Queetion:
doom dee plase diecly
daal
Sebution:
Flan tnep atoy dverter cren
Keauting::
*Uhdloaae Preliix
boonday fnoequater two poont
Bikkng dapp foooala colo daye
1ft Sittidooyy 3 11 2-x otte 1-50 oo (000 -Tee fine
skpeceiee eneccee fore fly meee -ro sunwinne 1el1 apa 2 20000 Vaankkeb do YE
daald ragie traber
24

---

Published as a conference paper at ICLR 2026
Data Exumple from MathVerse
Quetion
Acceding te
plaso drenty uawer droppssoo li cannt M,S,
Seloloo::
Ad7? cur
menn Bosile 2hd7
Westip (Camon
Da cada
the degececinggi A propaty tha sagie whaniaally as are thccctte of ontic tetze mugl sulended auazy p480-0
Neyul (Cement
"ieent presu
ettendadlyy thon watcr
(Camee)
HUdderared afforata
50
Ais
(Cement
dngrec ingle
(oomon
(h)
Data Example from Math Verse
Quation:
Accmsing cpoaiee stans
pronad de
Salution:
Stpp-l (ooool
(Connd)
The fnachin mari custmiic aquate..
and duseess
Whe a12 -2j--
18 gid)
cannna 10 lur day aich aoddar that
Rov (4
Stege (iwoooor
herwor
Theeenen opaanse right, AiL and 1/ wlurc nott ovemau but ouch have andicc tad aband
Data Example from DynaMath
Quadien:
sagle
Satutioa:
Sillw
polctlag
protectae
engle
Fiallaower
Example DynaMath
Questio:
Solao number
onacaguzi how eay beaus
ragg numberns?
Caner
New fuuror
Salation:
Heht ls
16
Lermctedy you find rage machos.
libery
Recoot
Makura
a1 (hansetoilaacea.... Raage Misinee Mi eneetwttnng -91
thrmngg calocbadd atroniing
from masimmm
thewe
Stgu decamed
Tharkee the mage of basc wllo camborteme boo citccis io.
25

---

Published as a conference paper at ICLR 2026
Data Example from DynaMath
The ralr chat bake spuscas affict crageioo crageee socrad (B a
Ohandy
Looknng mude diat, cas
ax cagaanoo 14, plotid on aodid ambyying valace Caepayy Capposs bi lightdt Canggry dows soor high wortt Crapoisc foawer calass
Magol dtcemed
Thocker. oukgery luhere ih hee vounnd hightr mader diet.
Stepo disooned
The anovar (Dt
Data Example from WeMath
Qucstion:
Cinck tai comnnfrrneee oo o app etge nuncs vai oveanne a/525 an what radex hore cade A. 8.9 C: Comnu aovmmedd No cerrec mnverr
Selution:
(Cement
conmenci "Macka mecel toveee
(Coment
The semel firite enseviecnco of aciille
N0cm
fft Co Zigida
cicunderace
cadia..
(Cemet
1fta (628) Mooeet2 90 Hoeelnnd
ralka satt..
poh Fival jan4
(m)
Data Example from WcMath
Quesiion:
puellcogaa
Salatien:
Som
aupprrrry gynniry
given
(Cowwet
New, acriaater the paoilsignas AbBT.
-AD DE
Stp- (Aoower))
wxt cas wakorine de
-AD
10 CD- Sux
AN Stp-l (ivoroed
Thankira Ei pottraarie
ADh
<3 Sumnn tecm
Teal
(n)
Data Example from WeMath
Queeton:
duss inteddigoss of in the bakweeod wook ploud Dodk tahopgo itai Cieen tat bokkk
Sebutian:
(haond)))
adde scalie stuaill
concrned hosis dillaniy oader "CCrus (uamaateeii)
Willogg (Comet
*Hbutt (ucoallent)
Figure 7: More data examples from VisualProcessBench.
26

---

Published as a conference paper at ICLR 2026
Example ViuuaPPrcestBench
Queition:
Archoonla fpxas stanke lusn bosodaa stangle the (a)) 189
coreect wwwe
Strp (hoon))
tample vinagl engle degieim
senaake crinnge tnd
antions 8S, 1481 and
(Cennd))
Tina Edeeldinca fur as kosadas vbaaga, rides oqua leugh and the angles opplec opposla foose olls
ceall trange Alti; diganos, angt- angle degrecs, wark instinng Claaao 25
25"
130
35
:
B
birp- Ueceerc
and
Coos angle gigl "iddegees aegle is dngien, Ales aught sdow 28 2000
Stepd (Ceenddf
sptioss, Opeie alnorbeed looi
rometting.
Halk
Londing ynu daagle
an which degies, feeets con the oun gia Inc. thee angle hervece tien be 100 35 122".
aocerr cpiicc (analt1
Data example with model reflection from Besu Red highlights the incorrect answer, orange highlights the reflection words, and green highlights the correct answer.
Figure 8:
27


Reviews:
[{"review_id": "lhzlKx8IVL", "reviewer": "Reviewer_Ppnx", "summary": "The paper introduces a new dataset to train multimodal Process Reward Models, a new benchmark to evaluate MM PRMs and an 8B-parameter PRM that consistently, across different model series and sizes, improves performance over MM reasoning benchmarks. The PRM is shown to outperform other TTS algorithms like Outcome Reward Models, Self-Consistency and MLLMs as critic models.", "questions": "1.\tCan the authors quantify FP and FN rates for a sub-sample of the VisualPRM400k dataset and explain if they have any filtering steps to identify and remove such demonstrations?\n2.\tCan the authors provide accuracy-vs-latency plots at multiple N comparing different BoN techniques?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1.\tTTS and using PRMs as reward functions in RL are under explored in multimodal modeling. The work here pledges to open source a large 400k sample dataset to train MM-PRMs, a benchmark to evaluate MM-PRMs and a trained MM-PRM. These contributions can be valuable for the community and foster further research.\n2.\tEffectiveness of PRMs used for TTS in improving MM reasoning across multiple model series and sizes are clearly demonstrated along with improvements compared to other TTS methods.\n3.\tDesign of the PRM benchmark  is sound with the main standouts being, considering all the erroneous reasoning steps as opposed to stopping at the first occurrence, using macro F1 scores to account for class imbalance and multiple math and reasoning datasets.", "weaknesses": "1.\tPRMs are well studied in the language modeling space. This work repurposes those studies and algorithms to the multimodal space with limited algorithmic novelty.\n2.\tAn automated Monte Carlo sampling-based pipeline is used to generate the PRM400k dataset. There are no discussions on the quality of this dataset and alignment with human judgement. How the authors ensure incorrect demonstrations are filtered out and how this could affect the trained PRM’s abilities as a critic are not discussed. 10% of the reasoning steps are negative demonstrations. Effect of PRM modeling with more balanced positive and negative demonstrations, using weaker models and models from other series to introduce diverse thinking styles is not explored.\n3.\tThe task categories in VisualProcessBench are mostly centered around math and logic. Extensibility of this methodology for other vision applications like chart, table, GUI reasoning among others will be helpful.\n4.\tWhile PRM is shown to outperform other Best-of-N strategies, a discussion about latency and throughput tradeoffs compared to other light-weight strategies can strengthen the claim.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "TtRB4g9b31", "reviewer": "Reviewer_h3nV", "summary": "The paper introduces VisualPRM400K, a ~400K-sample multimodal process-supervision dataset with step-wise expected-accuracy labels, and VisualProcessBench, a 2,866-sample benchmark with 26,950 human step-correctness annotations, to enable and evaluate process reward models (PRMs) as critics for Best-of-N test-time scaling in MLLMs. Trained as an 8B PRM that scores each reasoning step in a single forward pass, VisualPRM achieved great improvement across seven multimodal reasoning benchmarks.", "questions": "See the `Weaknesses` part.", "limitations": "", "rating": 2, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. The paper is generally well-written and easy to follow, with a clear description of the method.\n2. The paper provides intuitive visual demonstrations to help better understand the paper.", "weaknesses": "1. Label quality & construction pipeline clarity. The Monte-Carlo step-correctness (`Eq. (2)`) relies on continuations sampled from an unspecified model $M$; this risks systematic bias/noise if $M$ shares failure modes with the policy models later evaluated. In addition, merging solutions to a maximum of 12 steps may distort error localization and the temporal dynamics of “first error vs. downstream errors.” Please quantify label noise (e.g., step-level inter-rater agreement on a subset), report sensitivity to the number of sampled continuations, and analyze the effect of step-merging on PRM accuracy.\n\n2. Fairness of comparisons. Please clarify whether the reported gains in `Tab.2` are measured only against each policy model’s base (“Pass@1”) or also against strong critic baselines. To more comprehensively validate the effectiveness of the proposed approach, include outcome-based reward (ORM) and additional PRM baselines under identical Best-of-N settings (same candidate pool, N, decoding, and compute). Also expand `Tab.3` with more PRM variants and report matched-compute results.\n\n3. Generalization beyond the current suite. Beyond the seven benchmarks used (six math, one multidisciplinary), consider evaluating on more general-purpose multimodal benchmarks and on broader text-only reasoning benchmarks to substantiate cross-domain robustness.\n\n4. Limited technical novelty; strengthen the case for multimodality. The paper’s primary contributions appear to be `VISUALPRM400K` and `VisualProcessBench`, while methodological novelty is modest. To demonstrate that the *multimodal* aspect is indispensable (rather than a text-dominant signal), please add modality ablations and analyses showing performance drops when visual evidence is removed or corrupted. Such results would clarify the unique value of multimodal supervision/assessment relative to single-modal PRMs.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "iltAOwffwT", "reviewer": "Reviewer_edvQ", "summary": "The paper introduces VisualPRM400K, a ~400K multimodal process-supervision dataset built via Monte-Carlo (MC) completion to estimate step-wise correctness; an 8B VisualPRM trained on it; and VisualProcessBench, a PRM/MLLM benchmark with 2,866 samples and 26,950 human-annotated step labels. With Best-of-N (BoN) test-time scaling, VisualPRM substantially improves multiple MLLMs (e.g., +8.4 for InternVL2.5-8B; +5.9 for InternVL2.5-78B) and outperforms outcome reward models and self-consistency as the critic.", "questions": "On process reward generation: You estimate step values via MC sampling. Have you tried other strategies such as MCTS rollouts or LLM-as-a-Judge?\n\nOn PRM capacity: Why 8B? Have tried other PRM size (e.g., 1B/2B/9B/14B)? \n\nRelated work coverage (multimodal PRMs): The Related Work section focuses primarily on text-only PRMs. It should discuss recent multimodal PRM papers such as DreamPRM [1], AR-MCTS [2].\n\n[1] DreamPRM: Domain-Reweighted Process Reward Model for Multimodal Reasoning (NeurIPS 2025)   \n[2] Progressive Multimodal Reasoning via Active Retrieval (ACL 2025)", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 4, "strengths": "Timely, practical contribution: A large multimodal process dataset plus a purpose-built benchmark for PRMs addresses a clear gap and enables systematic progress on multimodal TTS. \n\nSolid empirical evidence: Consistent BoN gains across model scales; clear comparisons vs. ORM and self-consistency; ablations on value- vs advantage-based PRMs and score aggregation. \n\nClear PRM formulation: Step-wise discretized targets, single-pass scoring efficiency, and supervising all steps rather than stopping at first error are well motivated and validated.", "weaknesses": "Potential generator-bias in labels: Process rewards are derived from continuations sampled with InternVL-2.5 models. This may bias the PRM toward InternVL-style reasoning and limit transfer to other families (e.g., GPT-5, Qwen-VL). \n\nMC estimation only: The paper uses MC to estimate expected accuracy per step but does not compare with alternative credit assignment/judging strategies (e.g., MCTS, LLM-as-a-Judge).\n\nModel size choice and scaling law: VisualPRM is fixed at 8B; the paper lacks justification for this size and a scaling curve (e.g., 1B/3B/8B/14B) to reveal accuracy–latency–cost trade-offs.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "o76IY2ChPC", "reviewer": "Reviewer_DRVz", "summary": "This paper presents three significant contributions to the field of multimodal reasoning. First, it introduces VisualPRM400K, a large-scale dataset of approximately 400,000 samples with step-by-step process supervision, created using an automated data pipeline based on Monte Carlo sampling. Second, leveraging this dataset, the authors train VisualPRM, an 8B parameter Process Reward Model (PRM) designed to evaluate each step of a multimodal reasoning chain. Third, to facilitate the evaluation of such critic models, the paper proposes VisualProcessBench, a new human-annotated benchmark for identifying all incorrect steps within a reasoning process. The authors demonstrate that using VisualPRM as a critic in a Best-of-N (BoN) inference setting consistently improves the performance of various Multimodal Large Language Models (MLLMs) across seven reasoning benchmarks, outperforming Outcome Reward Models (ORMs) and Self-Consistency.", "questions": "1. Could you provide a more direct comparison of your final BoN results with other state-of-the-art methods on the same benchmarks? How does the performance of, for example, \"InternVL2.5-78B + VisualPRM\" compare to models that have been fine-tuned with advanced RL or self-improvement techniques? This would help contextualize the significance of the improvements you've achieved.\n2. Could you elaborate on the inference latency and computational cost of using VisualPRM in a BoN setting? For instance, what is the wall-clock time required to evaluate a single instance with N=8 compared to a single pass from the base model? A cost-benefit analysis would be extremely valuable for readers to understand the practical implications of your method.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. Significant and High-Quality Data Contribution: The primary strength of this work lies in the creation and release of two valuable resources: the large-scale VisualPRM400K training dataset and the high-quality, human-annotated VisualProcessBench. Constructing such resources is a laborious but crucial endeavor for the community. VisualPRM400K is, to my knowledge, the first large-scale dataset for training multimodal PRMs, and VisualProcessBench provides a much-needed, fine-grained benchmark for evaluating them. These resources will undoubtedly catalyze future research in multimodal reward modeling and reasoning.\n2. Thorough and Rigorous Experimentation: The authors have conducted an extensive set of experiments to validate their contributions. They demonstrate the effectiveness of VisualPRM across multiple model families (MiniCPM, Qwen, InternVL) and scales (from 8B to 78B). The ablation studies are comprehensive, comparing PRM with ORM and Self-Consistency, and analyzing the impact of various hyperparameters. The evaluation on VisualProcessBench, which shows that VisualPRM is competitive with powerful proprietary models like Gemini-2.0-Flash, further solidifies the quality of the trained reward model.\n3. Well-Written and Clearly Presented: The paper is exceptionally well-organized and clearly written. The motivation, methodology, and results are presented in a logical and easy-to-follow manner, making the paper's contributions accessible and understandable.", "weaknesses": "1. Performance Gains Comparison: The central application of VisualPRM is to improve MLLM reasoning via Best-of-N (BoN) inference. While the reported gains are consistent (e.g., +5.9 points for InternVL2.5-78B), the final performance often falls short of what has been achieved by other contemporary methods that focus on improving the policy model itself through advanced training techniques.\n2. In contrast, the approach in this paper is purely an inference-time strategy. While it successfully lifts the performance of existing models, it does not fundamentally enhance the models' intrinsic reasoning capabilities. The resulting performance, while improved, does not appear to push the state-of-the-art boundary as significantly as these training-focused methods.\n3. High and Under-discussed Inference Cost: The Best-of-N strategy is notoriously expensive. Using BoN with N=8, as is the default in this paper, multiplies the inference cost (both latency and compute) by at least a factor of 8, plus the overhead of running the VisualPRM critic. This makes the method impractical for many real-world applications. While the authors demonstrate performance scaling up to N=128, they do not provide a thorough discussion on the cost-performance trade-off. A more complete analysis would be necessary to assess the practical viability of this approach. The reported performance gains, while notable, may not be sufficient to justify such a substantial increase in inference cost for many use cases.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)