SCOPE : Compress Mathematical Reasoning Steps for Efficient Automated Process Annotation (2025)

Abstract: Process Reward Models (PRMs) have demonstrated promising results in mathematical reasoning, but existing process annotation approaches, whether through human annotations or Monte Carlo simulations, remain computationally expensive. In this paper, we introduce Step COmpression for Process Estimation (SCOPE), a novel compression-based approach that significantly reduces annotation costs. We first translate natural language reasoning steps into code and normalize them through Abstract Syntax Tree, then merge equivalent steps to construct a prefix tree. Unlike simulation-based methods that waste numerous samples on estimation, SCOPE leverages a compression-based prefix tree where each root-to-leaf path serves as a training sample, reducing the complexity from O(NMK) to O(N) We construct a large-scale dataset containing 509K samples with only 5% of the computational resources required by previous methods. Empirical results demonstrate that PRMs trained on our dataset consistently outperform existing automated annotation approaches on both Best-of-N strategy and ProcessBench.

## introduction as large language models (llms) advance in complex reasoning tasks @xcite @xcite , designing effective reward models has become increasingly crucial. process reward models (prms) @xcite evaluate the reasoning process step-by-step, providing more fine-grained supervision than outcome reward models (orms) that only assess final outputs @xcite @xcite . recent studies consistently demonstrate prms' superior performance across complex reasoning tasks simulation-based methods require numerous completions solely for q-value estimation, with these completions (gray nodes) being discarded without contributing to training. our compression-based approach eliminates such data waste by merging equivalent steps from all sampled solutions into a prefix tree, where every root-to-leaf path becomes a valuable training instance. @xcite . but prms face a significant challenge: they require intensive human effort for data annotation, as every reasoning step needs a label @xcite . to alleviate this limitation, recent studies explore simulation-based methods. for instance, mathshepherd @xcite uses monte carlo estimation to automate the data annotation. as shown in figure 1 , it samples n solutions for a math problem, and then for each step in the solution, it simulates m potential completions and calculates the q-value based on the proportion of completions that lead to the correct answer. while math-shepherd eliminates human annotation requirements, it incurs high computational complexity of o(n m k), where k denotes the average step count of solutions. recent studies have shown that math-shepherd requires 38.8× more flops than orm training @xcite . although omegaprm @xcite reduces complexity to o(n m log k) through a divide-and-conquer strategy, the efficiency gains remain limited due to typically short lengths (@xmath0. we argue that these simulation-based approaches are inherently inefficient as they generate numerous completions solely for q-value estimation (gray area in figure 1 ), resulting in wasted data. in this paper, we introduce step compression for process estimation (scope), a novel automatic prm label annotation strategy that achieves o(n ) complexity while maintaining annotation quality. unlike prior simulation-based methods @xcite , scope introduces a novel compression-based approach: first samples numerous solutions for each problem, then merges equivalent solution steps to construct a prefix tree (trie), as shown in figure 1 . for each node in the tree, its q-value is calculated as the proportion of solutions passing through it that reach the correct answer. each path from root to leaf in the tree represents a training sample with step-bystep labels. compared to simulation-based methods, scope not only achieves o(n ) complexity through step compression, but also fully utilizes all sampled solutions by incorporating them directly into training data through the prefix tree structure. the key challenge of scope lies in identifying step equivalence. naive exact string matching is too restrictive and results in limited compression. edit distance and sentence embeddings often fail to capture the subtle distinctions in mathematical reasoning @xcite . to address this challenge, we propose a code-based step compression through a three-stage process: (1) translate natural language reasoning steps into executable python code using a code llm, (2) normalize the code through abstract syntax tree (ast) @xcite (e.g., variable renaming), and (3) merge steps with identical normalized code using a trie. this approach enables precise identification of mathematically equivalent steps while being robust to surface-level variations. although code translation and ast add computation, the overall complexity remains o(n ), ensuring substantially lower computational costs for large-scale prm datasets. based on scope, we construct a prm training dataset containing 196k samples with 1.4m labels, exceeding math-shepherd's scale while requiring only 5% of its computational resources. empirical evaluation demonstrates the effectiveness of our approach, with prms trained on our dataset consistently outperforming other automated annotation approaches in both the best-of-n strategy and the processbench @xcite evaluation. our main contributions are: ## method in this section, we present scope, a novel approach for automatic prms dataset annotation: (1) first, we motivate and explain our code translation strategy, which converts reasoning steps into executable code through a code llm. (2) then, we perform ast normalization to standardize code syntax for accurate equivalence matching. (3) next, we apply step compression by merging normalized steps into a prefix tree. (4) finally, we describe our prms training details. ## experiments all experiments are conducted on a server equipped with 8 nvidia a100-80gb gpus and 512gb of system ram. we utilize pytorch @xcite as the implementation framework, sglang @xcite for sampling and deepspeed (aminabadi et al., 2022) for distributed training. ## conclusion this paper introduces scope, a novel automatic prms dataset annotation method that significantly reduces computational costs while maintaining label quality. by translating natural language reasoning steps into code and merging equivalent steps through ast normalization, our approach achieves o(n ) complexity compared to previous o(n m k) methods. using only 5% of the computational resources, we construct a largescale dataset containing 196k samples, and prms trained on our dataset consistently outperform existing approaches on both best-of-n strategy and processbench, demonstrating scope's effectiveness as a scalable solution for prm training.