VisualPRM: An Effective Process Reward Model for Multimodal Reasoning
Authors: name: Weiyun Wang; name: Zhangwei Gao; name: Lianjie Chen; name: Zhe Chen; name: Jinguo Zhu; name: Xiangyu Zhao; name: Yangzhou Liu; name: Yue Cao; name: Shenglong Ye; name: Xizhou Zhu; name: Lewei Lu; name: Haodong Duan; name: Yu Qiao; name: Jifeng Dai; name: Wenhai Wang
Published: 2025-03-13 12:03:37

Abstract: We introduce VisualPRM, an advanced multimodal Process Reward Model (PRM)
with 8B parameters, which improves the reasoning abilities of existing
Multimodal Large Language Models (MLLMs) across different model scales and
families with Best-of-N (BoN) evaluation strategies. Specifically, our model
improves the reasoning performance of three types of MLLMs and four different
model scales. Even when applied to the highly capable InternVL2.5-78B, it
achieves a 5.9-point improvement across seven multimodal reasoning benchmarks.
Experimental results show that our model exhibits superior performance compared
to Outcome Reward Models and Self-Consistency during BoN evaluation. To
facilitate the training of multimodal PRMs, we construct a multimodal process
supervision dataset VisualPRM400K using an automated data pipeline. For the
evaluation of multimodal PRMs, we propose VisualProcessBench, a benchmark with
human-annotated step-wise correctness labels, to measure the abilities of PRMs
to detect erroneous steps in multimodal reasoning tasks. We hope that our work
can inspire more future research and contribute to the development of MLLMs.
Our model, data, and benchmark are released in
https://internvl.github.io/blog/2025-03-13-VisualPRM/.