ProcessBench: Identifying Process Errors in Mathematical Reasoning
Authors: name: Chujie Zheng; name: Zhenru Zhang; name: Beichen Zhang; name: Runji Lin; name: Keming Lu; name: Bowen Yu; name: Dayiheng Liu; name: Jingren Zhou; name: Junyang Lin
Published: 2024-12-09 15:11:40

Abstract: As language models regularly make mistakes when solving math problems,
automated identification of errors in the reasoning process becomes
increasingly significant for their scalable oversight. In this paper, we
introduce ProcessBench for measuring the ability to identify erroneous steps in
mathematical reasoning. It consists of 3,400 test cases, primarily focused on
competition- and Olympiad-level math problems. Each test case contains a
step-by-step solution with error location annotated by human experts. Models
are required to identify the earliest step that contains an error, or conclude
that all steps are correct. We conduct extensive evaluation on ProcessBench,
involving two types of models: process reward models (PRMs) and critic models,
where for the latter we prompt general language models to critique each
solution step by step. We draw two main observations: (1) Existing PRMs
typically fail to generalize to more challenging math problems beyond GSM8K and
MATH. They underperform both critic models (i.e., prompted general language
models) and our own trained PRM that is straightforwardly fine-tuned on the
PRM800K dataset. (2) The best open-source model, QwQ-32B-Preview, has
demonstrated the critique capability competitive with the proprietary model
GPT-4o, despite that it still lags behind the reasoning-specialized o1-mini. We
hope ProcessBench can foster future research in reasoning process assessment,
paving the way toward scalable oversight of language models.