MPBench: A Comprehensive Multimodal Reasoning Benchmark for Process
  Errors Identification
Authors: name: Zhaopan Xu, name: Pengfei Zhou, name: Jiaxin Ai, name: Wangbo Zhao, name: Kai Wang, name: Xiaojiang Peng, name: Wenqi Shao, name: Hongxun Yao, name: Kaipeng Zhang

Reasoning is an essential capacity for large language models (LLMs) to
address complex tasks, where the identification of process errors is vital for
improving this ability. Recently, process-level reward models (PRMs) were
proposed to provide step-wise rewards that facilitate reinforcement learning
and data production during training and guide LLMs toward correct steps during
inference, thereby improving reasoning accuracy. However, existing benchmarks
of PRMs are text-based and focus on error detection, neglecting other scenarios
like reasoning search. To address this gap, we introduce MPBench, a
comprehensive, multi-task, multimodal benchmark designed to systematically
assess the effectiveness of PRMs in diverse scenarios. MPBench employs three
evaluation paradigms, each targeting a specific role of PRMs in the reasoning
process: (1) Step Correctness, which assesses the correctness of each
intermediate reasoning step; (2) Answer Aggregation, which aggregates multiple
solutions and selects the best one; and (3) Reasoning Process Search, which
guides the search for optimal reasoning steps during inference. Through these
paradigms, MPBench makes comprehensive evaluations and provides insights into
the development of multimodal PRMs.