ContextPRM: Leveraging Contextual Coherence for multi-domain Test-Time Scaling
Authors: Haotian Zhang
Liu Liu
Baosheng Yu
Jiayan Qiu
Likang Xiao
Yanwei Ren
Quan Chen
Xianglong Liu
Venue: ICLR 2026 Poster (2026)
Keywords: Process Reward Models
Primary area: foundation or frontier models, including LLMs
TL;DR: We shift the training of Process Reward Models from verifying domain-specific correctness to modeling domain-agnostic contextual coherence, achieving state-of-the-art multi-domain generalization.

Abstract: Process reward models (PRMs) have demonstrated significant efficacy in enhancing the mathematical reasoning capabilities of large language models (LLMs) by leveraging test-time scaling (TTS). However, while most PRMs exhibit substantial gains in mathematical domains, the scarcity of domain-specific training data and knowledge-based learning patterns limits their generalization ability when faced with other domains. To address this limitation, we shift the learning objective from verifying domain-specific knowledge to modeling domain-agnostic logical flow. Centering on \textit{contextual coherence} between chain-of-thought (CoT) steps, our approach is realized through a novel data annotation and training framework, which enhances the model's generalization capabilities across diverse domains.  For instance, our resulting model, \textbf{ContextPRM}, achieves a notable 6.5\% average accuracy improvement over the majority voting baseline via weighted majority voting across nine non-mathematical domains in MMLU-Pro, including law, history, and philosophy, significantly surpassing the 2.2\% improvement from VersaPRM and 0.5\% gains from other mathematics-focused PRMs, demonstrating consistent performance across both mathematical and non-mathematical domains.

Published as a conference paper at ICLR 2026
CONTEXTPRM: LEVERAGING CONTEXTUAL COHER- ENCE FOR MULTI-DOMAIN TEST- TIME SCALING
Haotian Zhang!. Liu Liu 1.2
Baosheng Yu
Jiayan Qiu
Likang Xiao² Yanwei Ren 1.2
Quan Chen'
Xianglong Liu
School of Artificial Intelligence, Beihang University
Haagghoou International Innovation Institute, Beihang University
3 "School of Computer Science, Beihang University *Kuaishou Technology
University of Leicester
"Nanyang Technological University
ABSTRACT
Process reward models (PRMs) have demonstrated significant efficacy enhane- ing the mathematical reasoning capabilities of large language models (LLMs) by leveraging test-time scaling (TTS). However, while most PRMs exhibit substantial gains in mathematical domains, the scarcity of domain-specific training data and knowledge-basse baaeeg learning patterns limits their generalization ability when faced with other domains. To address this limitation, verifying domain-specific knowledge to modeling domain-agnostic logical flow. Centering on contextual coherence between chain-of-thought (CoT) steps, our ap- proach is realized through a novel data annotation and training framework, which enhances the model's 's generalization capabilities across diverse domains. For in- stance, our resulting model, ContextPRM, achieves notable 6.5% average accu- racy improvement over the majority voting baseline via weighted majority voting across nine non-mathematiial domains in MMLU-Pro including law, history, and philosophy, significantly surpassing the 2.2% improvement from VersaPRM and 0.5% gains from other mathematics-focusee PRMs, demonstrating consistent per- formance across both mathematical and matthemattiall domains.
shift the learning objective from
Performance improvement over Majority Voting Baseline
Non-Matt-Adjacent Domains
Majerby Vaiing
0:00
2.02
0.00
0.00
0.00
0.00
0.00
0.00
022
2.02
0.87
3.28
3.00
36
VersSPM
4:62
9.21
1.33
4.09
0.67
4.8
1.66
2.01
Content Onedomsin
6:14
9.21
2.00
9.22
2.00
4.14
3.00
5:37
16
128
Number generated CoT
Weckayy
Heeeo
Plychology
Hostery
Majoriy Visiin
ConneztRMM
Hith-S
Figure I: Key results demonstrating ContextPRM's effectiveness. (Left)) Training on single non- math domain surpasses the prior multi-domain SOTA, VersaPRM. (Right): In non-math-adjacent domains, ContextPRM consistently outperforms all baselines under WMV sampling, showcasing superior generalization.
"Correppondigg author: 1in1::::iiibbiiii iibhiubbhuua
Code is available at http:////gtttiioo//////aaaa/2/20

---

Published as a conference paper at ICLR 2026
INTRODUCTION
Recent advancements in large language models (LLMs) have fostered significant interest in enhanc- ing their reasoning abilities (Wei et al. 2022 Kojima et al. 2022 Madaan al. 2023 Chowdhery
et al. 2023). Among the strategies explored, test-time scaling (TTS) has emerged as a promi- nent approach, leveraging additional computational resources during inference improve perfor- mance (Cobbe et al. 2001 Yao et al.; 22023a Jaech et al., 2024; Snell et al. 2025). The effective- ness of TTS methods, however, relies heavily the quality of the underlying verifiers that guide the reasoning process (Lightman et al. 2023). Process Reward Models (PRMs) represent a key class of such verifiers, and have demonstrated strong performance in reranking generated solutions through step-by-step verification (Uesato et al. 2022, Wan et al. 2024). Despite their promise, PRMs face several inherent limitations that restrict their broader applicability.
While several studies demonstrate the strong performance of PRMs on mathematical reasoning 2024), most research remains con- centrated within the math domain, focusing on synthetic data generation (Lightman et al., 2023 Wang et al. 2024a), training methodologies (Xiong et al.
tasks (Wang et al. 2024a] Zhang et al.] 2025 Yang et al.]
2024| Zhao et al., 2025). and evaluation
benchmarks (Zeng al. 2024; 2023 Song et al. 2025 Their application to non non-mathematical reasoning, however, is still insufficietlly explored. VersaPRM (Zeng et al. 2025) highlights this limitation, showing that mathematical PRMs transfer poorly non-mathematical tasks. It further proposes an efficient and automated method for generating and annotating multi-domain training data, which partially improves PRM performance outside the mathematical domain. However, critical gap remains in developing a training methodology that enables genuine multi-domain gener- alization, which requires unifying heterogeneous reasoning styles across disciplines, from the formal and symbol-dense CoT's of the sciences to the nuanced argumentation typical of the humanities.
To address this limitation, we introduce Contex(PRM, a novel approach that models the contex- tual coherence between steps in a reasoning chain, rather than evaluating them solely for isolated, domain-specific correctness. By shifting the learning objective toward domain-agnostic logi- cal flow through a novel data annotation standard and context-aware training methodology. Con- texIPRM explicitly learns to evaluate the logical transitions between consecutive reasoning steps. This design substantially improves cross-domain generalization, establishing new otate-of-the-art multi-domane test-time scaling. Our main contributions are threefold:
To better leverage training data and reinforce coherence and logical consistency in chains of thought (CoTs), we introduce the Context-Awar Training Method for PRMs, which combines new data annotation standard with a context-aware approach to evaluate logical transitions be- tween reasoning steps. с demonstrating that our context-focused method significantly enhances the cross- cross-domain generalizability of PRMs, particularly in domains beyond mathematics.
We present extensive empirical evidence showing that our context-centered approach substan- tially improves the cross-domain generalizability of PRMs, achieving notable gains in domains beyond mathematics and demonstrating its versatility and robustness.
Our resulting model, ContextPRM, achieves a new state-of-the-ar for multi-domain process reward modeling, delivering consistent performance improvements across both mathematical and non non-mathematical domains, and substantially outperforming prior work. Specifically, in domains non-adjacent to mathematics, ContextPRM achieves 6.5% average accuracy im- provement over the majority voting baseline using Weighted Majority Voting, significantly surpassing the 2.2% gain from the previous state-of-the-ar VersaPRM.
2 RELATED WORK
2.1 PROCESS REWARD MODELS
PRMs outperform Outcome Reward Models (ORMs) in enhancing LLM chain-of-thought reason-
ing, particularly for mathematical tasks (Li et al. 2023 Zhang al. 2024b a Wu et al. 2025). Re- cent approaches reduce reliance on costly human annotations through automated supervision. data
synthesis. weak and preference-bast based signals (Wang et al., 2024a Lightman al. 2023 Zeng
jet al., (202).. while frameworks for PRM training and evaluation improve scalability. robustness,
and fine-grained assessment Yuan et al. 2025 Zhang al. 2025 Xiong al. 2024 Zheng al.
2

---

Published as a conference paper at ICLR 2026
2024].
However, these methods primarily focus mathematical reasoning (Sprague et al. 2025
and early efforts to generate multi-domain PRM data often lack optimized training strategies ZZng et al. )025). In contrast, approach tailors training multi-domain data by emphasizing domain- agnostic logical flow, enabling robust cross-domain generalization.
2.2 TRAINING METHOD FOR PRMS
As step verifiers for inference LLMs, PRMs are required to assign stepwise reward scores to each reasoning step within a COT. Conventional training approaches formulate this as binary classifi- cation task, where the PRMs Iearns to predict correctness labels {0, 1] for individual steps. Re- cent studies have attempted to enhance PRM's s performance with various training method improve- ments. (Zhang et al. 2024a; Zhao et al., 2025 She et al. 2025 Khalifa et al. 2025) first trained the model to generate intermediate reasoning before producing digit rewards, trading increased test time cost for improved accuracy. (Pala et al. 2025) adds additional labels to train PRMs to detect error patterns before reward assignment. However, these methods remain largely constrained to knowledge-intensive mathematical domains. In contrast, our approach introduces a context-aware training methodology tailored for multi-domain data, which explicitly teaches the PRM to evaluate the logical transitions between consecutive reasoning steps.
2.3 TEST-TIME SCALING
Reranking-baad based inference methods, commonly used in rest-time scaling for thain-of-thought (CoT) reasoning, select the final answer from set of sampled CoT solutions using aggregation strategies. Let ac denote the answer extracted from a CoT solution C, and Cn = (C1, C,....,CC CC be N CoT's sampled independently from generator for given question. Aggregation functions such as min-, mean-, and maxx-ggregation-
the stepwise scores assigned by the PRM into a single chain-of-thought (CoT) score, which serves as the foundation for downstream reranking (Lightman)
et al.
2023 Wang et al. 2024a Sun et al.) 2024). Building on this, Majority Voting (MV) (Wang al.
2023) selects the most frequently occurring answer across sampled CoTs, while Best-of-N
(BoN) identifies the answer derived from the CoT with the highest aggregated score. Weighted Ma- jority Voting (WMV) (Uesato al. 2022) further refines MV by weighting each answer's frequency according to its Co'I' score, thereby integrating both occurrence and quality signals. Collectively, these methods enhance inference robustness and reliability by exploiting multiple reasoning traces during test-time scaling, ultimately leading to more consistent and higher-quality outputs. This lim- itation motivates our development of a context-aware approach that explicitly evaluates the logical transitions within CoTs,, thereby providing richer signal for both training and inference.
METHOD
In this section, describe the methodology for context-aware process reward model (Con- textPRM). We begin by highlighting the key differences between our training paradigm and con- ventional PRMs, focusing on how our approach models reasoning coherence. We then present the context-coherrnce-basee CoT annotation pipeline, which captures contextual dependencies within reasoning steps, and detail how ContexiPRM is applied during test-time computation.
3.1 CONTEXT-AWARE PRMS
Our context-aware PRMs extend conventional PRMs by explicitly considering the relationships be- tween consecutive reasoming steps. While traditional PRMs evaluate the correctness of each step in isolation, context-aware PRMs assess reasoning coherence across the contextual reasoning steps, enabling more accurate and nuanced reward assignment. In the following, we first review con- ventional PRMs establish the baseline, and then introduce our context-aware training approach, which leverages contextual dependencies to enhance multi-step reasoning evaluation.
3.1.1 CONVENTIONAL PRMS
Recent works on PRMs learn to assign a scalar reward to each step a reasoning process. The model is trained as binary classifier distinguish between correct and incorrect reasoning steps
3

---

Published as a conference paper at ICLR 2026
Context-Coherence bosed COT Annotation
Context Extraction
Context-Basee Training Method
Physici
Malth
LLM Verifier
P
wnarl
Correctreess
P2
Q
Binary- clossification PEFT Training
ContextRRM
Understonding
Lagic
P2
Low
Balogy
Relevcnos
Figure 2:
An overview of the ContextPRM training pipeline. Given a multi-domain question Q with multiple CoTs, each CoT is decomposed into sequence of step pairs (P.,), where i indexes the i-th COT and indexes the j-th step within it. Each pair is annotated for logical coherence using our proposed standard, i.e, correctness, understanding. logic, and relevance. During the context-based training phase, each annotated context pair is combined with the original question Q to construct training samples for binary classification PEFT, ultimately producing the final ContexiPRM model.
based the preceding context (Wang et al. 2024a Zhang et al. 2025). Formally, for given question Q and k-step Chain-of- Chain-of-Thought (CoT) denoted as C' = S the training data is structured concatenated input text T; = Q ① S + ... @ S, where ( denotes concatenation. Each step S; is labeled with a binary value 4 E {0, 10, where I denotes correct steps and denotes incorrect ones.
sequence of cumulative prefixes. For each step S., the model M receives the
This classification is typically achieved by computing cross-entropy loss over the logits of prede- fined positive ('+') and negative ("-") tokens. The loss computed only at the position of the final ken of step S, which we denote by pt- Let the model with parameters 0 output logits = -M(T;;0 The loss function for the entire CoT defined as:
LPRM (0)
Σ CrossEntropyy i=1
(o (t ( .44;;
(t- where p.p. represents the logits for the negative token t_ and positive token 44 at position pi.
However, under this training paradigm, as the context length grows with each step, the model strug- gles to diagnose the true root cause of current step failures. Furthermore, as the context length grows with each step, it becomes increasingly difficult for the model to capture the contextual logic that is most relevant to the correctness of the current step S, often failing distinguish between isolated factual errors and deeper contextual fallacies.
3.1.2 CONTEXT- TRAINING
To enhance the utilization of information within training samples and to reinforce the model's un- derstanding of coherence and logical consistency in CoTs, we propose the Context-Aware Training Method for PRM. This approach optimizes the model's 's context-awareness by constructing a com- posite contextualized representation for each reasoning step. Specifically, we transform each step S; into a contextualized representation P; achieved by a formatting function F.
Formally, for each step S, in the CoT C, we define its new context Š (ŠI.... . where S, = Q if i = 1, and S, = S >> Then, a concat function applied for each step and its context, which takes the context and the current step input and structures them into new sequence by inserting special tokens that explicitly mark the beginning of the context and the step. The resulting contextualized step is denoted as: P; = F(S.. S.). This formatting explicitly provides the model with the preceding step direct context.
This method constructs & distinct and independent training samples from a single k-step CoT. For each step E [k), the input to the model, T, is formed by directly concatenating the question Q with the contextualized representation of the current step: Т, = Q c P, The supervision signal is derived from parallel set of coherence labels, 6 E {0, 1), where denotes coherent transition.
4

---

Published as a conference paper at ICLR 2026
Table Comparison of First Incorrect Step Identification between Original and Context-Correctnes Based Labeling.
Origin Label
Total
Same
Earlier Wrong
Later Wrong
Modification rate
Correct
36935
27823
9112
24.67%
Incorrect
47163
18931
26904
1328 1328
57.04%
Total
84098
46754
36016
42.82% %
The loss computed at the final token position of the contextualized step Š;, denoted by Pt. The ContextPRM loss is then calculated by summing the losses for each of these k samples:
(t_.++) CooteexPRM (0) = CrossEntropy i.p.
(2)
McooexpPRM (T;; 0) are produced from the corresponding contextualized input Tü. This method compels the model focus not the isolated correctness of step, but the logical validity of the transition between steps.
where the logits o;
A critical component of our methodology is the tight coupling of our training method with its corre- sponding supervision signal. As our approach aims to shift the learning objective towards contextual coherence, reusing traditional annotation methods and correctness labels (l,) would create funda- mental mismatch between the training objective and the annotation logic. Therefore, to ensure that the training and data labels properly aligned, introduce a new annotation standard (detailed in Section 3.2) that is specifically designed to match our context-aware training paradigm, thereby maximizing the effectiveness of our method.
3.2
BASED COT ANNOTATION
During the data annotation phase, we proposed an annotation standard to accurately assess each context pair of COT. This standard is designed not only identify factually incorrect steps but also to discern those that are correct in isolation yet are logically flawed or unfounded within the broader context of the reasoning chain. Inspired by the 3-level annotation method in previous works (Light-) man et al. 2023 Zeng et al. 2025). we categorize each step in context pair based on the following criteria:
Context- Cohhernnc Based CoT Annotation Criteria
The step is correct, verifiable, contextually appropriate, and contributes insight- fully the solution path.
Good:
Okay: The step is correct and verifiable but is either redundant makes only minimal
progress toward the solution.
Bad: The step is categorized bad if it exhibits one or more of the following flaws.
Incorrect:
The step contains factual or calculation error.
Misinterpretation: The step is based on a misunderstanding of the problem's premise or the goal.
Logical Fallacy:
The step contains structural flaw in its reasoning, such sequitur or contradiction.
Misdirection: The step introduces information or line of reasoning that is irrelevant to the solution path.
We apply this labeling standard each context pair within CoT sequentially until the first 'Bad' step identified. All subsequent context pairs following this initial error are automatically labeled incorrect. To quantify the impact of our refined annotation standard, conducted a comparative analysis against the original labels using gpt - co-mmiini 2024--022222 -0 07-18, focusing on the identifi- cation of the first incorrect step.

---

Published as a conference paper at ICLR 2026
The quantitative results of this comparative analysis are presented in Table The table is generated by re-annotating the dataset with our context-coherence criteria and comparing the position of the first identified 'Bad' step against that from the original labels. The CoTs are grouped by their status provided by original labels. We then categorize the comparison into three outcomes: Same, the position of the first emror; Earlier Wrong. where method identifies the first errоr at an earlier position; and Later Wrong. The distinction between these outcomes is critical. The Earlier Wrong category represents the most significant modification, as it provides a fundamentally different and more precise supervision signal by preventing the model from learning from steps built upon a flawed premise. Consequently, we define the Modification rate as the proportion of CoTs in this category to measure the extent to which our method meaningfully refines the training data.
where both methods agree
Notably, our method identifies errors in 24.67% of CoTss previously considered entirely correct, highlighting a stricter standard for logical consistency. For CoT's already containing known errors, our approach locates the initial mistake earlier in majority of cases (57.04%) The substantial overall modification rate of 42.82% validates that our method provides a significantly different and more precise assessment.
EXPERIMENTS
4.1 EXPERIMENTAL SETUP
2025). ) we fine-tuned the open- source base model Llama-PRM800K, which is full-finetuned from Llama-3.1-80 1..----Innsttct with PRM800K dataset. We trained our model with LoRA (Hu et al., 2022) applied to all linear lay- ers with (r 16, 32), for 3 epochs with learning rate of le-4 and total batch size of 32, utilizing our proposed context-based loss function in 3.1.2 To ensure the fairesss of comparison, only updated the label of the training data of VersaPRM with annotation method as train- ing data, as shown in Table [ The same hyperparameter configuration is maintained for all other LoRA-tued models used in our ablation studies. We use 8xRTX 5090 GPUs for model training.
Model Training: Following the settings in VersaPRM ZZeng et al.
Evaluation Method:
To assess multi-domain performance, we employ the MMLU-Pro-CoT-Evve
(Unlabeled) test set released by VersaPRM, which comprises 2063 questions uniformly distributed across the domains of MMLU-Pro (Wang et al. 2024b), with each question accompanied by 128 candidate CoTs generated by Llama-3.1 In-sruutt to measure the performance of the PRM when integrated with test-time scaling methods such as Weighted Marordd-Maotttyyoooneee (BoN) sampling. We use Lun-Agggeattinn for both methods. To facilitate the comparison of multi-domain capabilities, the test set is further partitioned into a math-adjacent split and non- math-adjacent split. The math-adjacent split comprises all questions from the chemistry, computer science, engineering, and physics domains. Conversely, the non-math-adjacees split includes ques- tions from biology, health, psychology, business, economics, law, history, philosophy, and other.
(WMV) and Best-of-
4.2
MAIN RESULTS
In main experiments, we compared ContextPRM with several strong open-source, mathematics- focused PRMs, including Qwen2.5-Math Maanh2--MMahh-PRMMaaa PRM(Zhnng et al. 2025). Seuth-Shepherd((aaa et al., 2024a). RLHFlow- Desroe--Deepseeixiionng et al. 2024 the state- the multi-domain PRM Ver- saPRM al.) (025), and our base model,
The evaluation results are shown with three scopes: all domains average, mathematies-specift domains, and non-math-adjacent do- mains. Performance is measured using both Weighted Majority Voting (WMV) and Best-of-N (BoN) sampling methods with Majority Voting as general baseline. As shown in Figure 3 the top and bot- tom rows illustrate the performance under WMV and BoN sampling, respectively. More detailed results are available in Appendix B
Under WMV sampling: ContexIPRM achieves state-of-the-art results across all domains com- bined and non-math-adjacent domains. Overall, ContextPRM gains 5.4 accuracy improvement for all-domain questions. In nan-aaab-adjacent domains, ContextPRM improves 6.5% accuracy over the Majority Voting baseline, demonstrating superior multi-domain adaptability. Concurrently, Contex1PRM maintains highly competitive performance within the mathematics domain.

---

Published as a conference paper at ICLR 2026
Non-MathhAdjacent Domains
Inference Accuracy 1%) Interence Accuracy (%A
62.5
-5-
57.
55.0
05
62.5
57.
55.0
50
Number generated OuT solutions
Number of generaled CoT solutions Scale)
Number
Coll solutions flgg
Majority Voting
ContextPaM (Durs)
HLHHioo-Deeppeeek
VersaPRM
Weighted Majonity Voting
Liama PRMBOOK
Math Shepherd
QwenZ.5 Math
of N
Figure 3: Comparison of WMV (top) and BoN (bottom) under Min-Aggregation using ContextPRM (Ours) against VersaPRM and other open-souree math PRMs on MMLU L-PPU Cei----ro-------E- (Unlabeled) ContextPRM outperforms other models overall results, and the performance gap becomes largest in domains not adjacent to Math, demonstrating its superior generalization.
With BoN sampling: ContextPRM continues to deliver state-of-the-art performance, securing a 6.3% accuracy improvement over the Majority Voting baseline non-math-adjacent domains, further validating its strong domain generalization capabilities.
Compared to its base model stantially elevated after LoRA fine-tuning with our context-based training method (Section 1.2 which attests to the efficacy of proposed approach.
Notation:
the performance of ContextPRM is sub-
4.3 ABLATION STUDIES ON CONTEXT METHOSS
In this section, we conduct ablation studies to investigate the individual contributions of our pro- posed context-based training method 3.1.2 and context-based labeling method in 3.2 il- lustrated in Figure 4 trained two ablation models. The first employs our context-based training method but utilizes the original, unmodified training data la- bels, while "Context-label Only" is trained using the data with updated context- based labels but adheres to the standard PRM training method. For comparison, our full Contex1PRM model incorpo- rates both methods, while the VersaPRM baseline employs neither. summary of the configurations is provided in Table 2 Detailed results are available in Appendix C
Only" only
The results reveal strong synergistic effect between our proposed methods. In the math domain, models trained with either ablation method perform nearly identically to VersaPRM, whereas our full ContextPRM shows a 2.2% performance decrease, suggesting a trade-off for generalization. This trade-off is validated in the non-math-adjacent domains, where the ablation models offer only a marginal improvement over VersaPRM (0.5%), but the complete ContextPRM, integrating both methods, achieves substantial 4.3% increase in accuracy.
Context-label Only. When using the standard training method with our improved labels, the model's performance is largely comparable to the VersaPRM baseline, exhibiting parity in the math domain and a very slight improvement of 0.84% non-math-adjacent domains, which is intentional compared with the modification rate when annotation method applied, as shown in Table This observation highlights that our annotation standard is primarily designed to rectify logical inconsis- tencies, rather than to artificially boost training performance by increasing the quality of labels. The result confirms that our main contribution is more context-coherence awared supervision signal.
Context-train Only. Similarly, the comparison between the "Context-train Only" model and Ver- saPRM affirms that our training method also contributes to performance gains multi-domain test- time scaling. However, notable mismatch exists between the training objective and the supervision

---

Published as a conference paper at ICLR 2026
Table 2: Configuration of models used in the ablation study.
Model
Context-based Training
Context-baseed Labeling
VersaPRM (Baseline)
X
X
Context-train Only
XX
Context-lahel Only
X
ContextPRM (Ours)
All
Math
Non-Math-Adjacent Domains
Interence Accurocy (86 Inference Accurocy (96)
05
62.5
50.0
55.0
60-
52.5
16
32
64
120
16
32
64
128
32
64
128
65
62.5
60
50.0
55.0
-0-
52.5
16
32
54
120
16
32
64
128
2
64
128
Number of ganerated CoT solutions (log scale)
Number of generated CoT solutions Bogg scale)
Number of generated CoT solutions Dog scale)
Majority voting
Best
Contect-train Only
VersaPRM
Weighted Majority voting
ContextPRM (Ours)
Context label Only
Figure 4: Ablation study results on the contributions of context- training and context-based labeling. The combined approach (ContextPRM) consistently yields the best performance in non- math-adjacent domains.
signal. Despite this configuration that forces the model to infer contextual relationships from a signal that lacks this explicit information, the model still consistently outperforms the baseline, achieving gains 0.67% in the math domain and 1.07% in non-math- mon-maat-adjacent domains. This result robustly demonstrates the inherent superiority of our basee training framework, as it elicits better performance even when supplied with a sub-optimal, non-contextual supervision signal.
Full Integration (ContextPRMD), The results reveal strong synergistic effect between pro- posed methods. In the math domain, models trained with either ablation method perform nearly identically VersaPRM. In contrast, the full ContexiPRM shows 2.2% performance decrease, suggesting trade-off for generalization. This trade-off is further validated in the non-math-adjacent domains where ContextPRM. integrating both methods, achieves a substantial 4.3% increase in curacy. The trade-off is acceptable as shown in Figure 3 where ContexIPRM still significantly outperforms its base model performance the combined 'All' domains. This outcome validates the success of shifting the training objective toward context coherence during the PEFT stage and demonstrates the overall effectiveness of our integrated approach.
mathematics and also maintains state-of-the --thh-rrt
4.4 ABLATION STUDIES DOMAIN GENERALIZATION
In this section, we investigate the inter-domain relationships and assess whether our method retains its multi-domain capabilities when trained on limited, single-domain dataset. To achieve this, we partitioned the training set by category and applied our integrated context-based annotation and training methodology each subset individually.
As depicted in Figure 5 our method demonstrates remarkable generalization even when fine-tuned on data from certain single domains. Notably, models trained exclusively domains such "Law". 'Psychology'. or 'Philosophy' still achieve strong performance in multi-domain evaluations. This result underscores the versatility of approach, suggesting that by focusing domain-agnostic
8

---

Published as a conference paper at ICLR 2026
Performance Improvement over Majority Voting Baseline
Majorit
3.0
0,0
0.0
6/
0.0
6/0
0.0
5
00
0.0
00
0.0
-10
Math
1.3
5.0
0.7
0.6
1.4
33
Chemstry
Computer
Enginecring
2.0
Calume wite Normalino bAccurary 1963
Physies.
5.0
1.3
3.5
1.1
Tha ning Dota Domain
Biology
3.3
0.7
4.7
377
4.6
0.7
0.4
3.3
0.0
3.7
4.7
33
1.8
21
2.8
Health
29
3.3
2.0
4.0
3.3
7.0
3.1
0.7
2.3
3.3
13
4.7
3.0
6.0
1.7
2.4
29
Phychology
3.6
4.0
2.7
333
5.3
5.0
6.2
-0.7
2.0
5.4
2.0
4.1
1.0
5/4
5.1
3.4
38
Business
299
5.3
3.3
2.7
2.7
4.3
0.7
3.3
4.7
0.0
3.5
3,9
4.0
3.1
255
277
33
Economics.
5.0
2,0
2.7
0.0
5.0
0.0
-0,
2.0
3.5
-3.0
2.0
5.1
2.2
2.3
2.4
4.0
-1.3
0.7
7.0
2.1
1.3
13
7.6
3.7
33
2.4
9.3
1.0
0.3
Willsophy
ContextPRM (Curs)
10.4
Math
Chemistyy
Computer Science
Engineering
Physica
Biolay
Health
Psycholoay
Buniness
Economicc
Lave
Hatery
Pelteephy
Non Math Adjacent
Other
Math Adjacent
All Except
Evaluation Domain
Figure 5: Performance improvement the Majority Voting baseline. Each row represents model fine-tuned on a single training domain, while each column represents an evaluation domain. The results highlight the strong cross-domain generalization of models trained with our method.
logical coherence, the model can learn transferable reasoning skills that are effective even beyond the scope of its specific training data. A closer analysis of the heatmap reveals two key insights:
Training on non-math-adjacent domains yields better multi-domain generalization. The corresponding models trained these domains (e.g.. 'Psychology'. 'Law', 'Philosophy') generally exhibit stronger overall test performance. For our base model trained extensively on mathematics-heary PRM-800K dataset, these non-math datasets present greater novelty and learn- ing value. They enrich the model's understanding of diverse logical patterns, leading to more signif- icant fine-tuning effects in a sample-efficient setting.
The effectiveness of fine-tuning is correlated with the logical intensity of the training domain. Models trained on knowledge-intensivee domains such 'History" (1.2% avg. improvement) and 'Physics' (1.7% avg. improvement) show limited generalization. In contrast, models trained on logic-intensive domains like 'Philosophy' (3.4% avg. improvement), 'Psychology' (3.6% avg. im- provement), and 'Health" (2.9% avg. improvement) perform substantially better across the board. This disparity suggests that our method's success deeply rooted in its ability to leverage the un- derlying logical structure of CoT data, rather than relying domain-specine factual knowledge. Notably, despite being trained on data from a single domain, several of these models still surpass the full-dataset performance of VersaPRM (2.7% avg. improvement) in con-math-adjacen domains, demonstrating the data-efficiency of our approach.
4.5 ERROR TYPE ANALYSIS
To validate that ContextPRM''s gains stem from resolving logical inconsistencies, we classified er- ror types in model failure cases. Analysis reveals that math-intensive domains suffer predominantly from factual "Incorrectness", while non-mathematical domains exhibit a high density of logical errors. By isolating the Fixed Set (cases where VersaPRM failed but ContextPRM succeeded), Figure 6 shows ContextPRM primarily rectifies logical errors (e.g., fallacies, misinterpretationns) especially in humanities. Furthermore, a correlation analysis (Figure / between ContexuPRM'ss performance improvement and the domain's logical error ratio shows strong positive correlation

---

Published as a conference paper at ICLR 2026
(r 0.80). This confirms ContextPRM achieves multi-domain generalization specifically by hancing contextual coherence and correcting logical flaws, rather than refining factual accuracy.
Fixed Error Type Percentage Distribution by Category
15.2
55.0
25.0
143
Enee Classifiation
Logical
204
A4.4
50
40.6
6.1
18.
20.3
14.3
11.3
143
1841
143
143
143
Trrmn
AB
Chemitry
Enpivwering
Prysis
Poyttelayy
Foraaiia
Hiteyry
Otter
Category
Figure 6: Error type distribution in the Fixed Set. ContextPRM predominantly corrects logical errors.
Correlation between Logical Error Ratio and Performance improvement (Pearson Correlation: 0.80)
Performane informance Improvveenttt improvement (Accuracy %0
100-
110% Incomenti
Porformance
80
Ratio Logical
12
Chemany
Computer Sclence
Enginsering
Physica
Bidaps
Hentth
Mycholof
Businent
Frasnica
Histery
Phinwpphy
Offer
Evaluation Domain
Figure 7: Correlation (r 0.80) between logical error ratio and ContexiPRM'ss performance im- provement.
5 CONCLUSION
In this work, we addressed the critical challenge limited cross-domain generalization Process Reward Models (PRMs). Existing approaches remain constrained by domain-specific correctness signals and thus struggle to accommodate the diverse reasoning styles found across disciplines, from the formal symbolic derivations of the sciences the nuanced argumentation of the human- ities. To overcome this gap, we shifted the learning objective from verifying isolated correctness modeling domain-agnose contextual coherence. To realize our motivation, we proposed syn- awwrr
ergistic framework combining new data annotation standard with a corresponding training methodology. Our resulting model, ContextPRM, establishes a new -tate-of-the-art in multi-domain test-time scaling, achieving significant and consistent accuracy improvements across non-mathematical domains where prior models have struggled. Extensive experiments, including ablation and single-domain studies, validate that approach successfully imbues the model with robust generalization capabilities by teaching the universal principles of logical flow. Our find- ings suggest that focusing the structure of reasoning, rather than its content alone, is promising direction for building more general and robust process reward models for multi-domain test-time eest-time scaling.
10

---

Alama-PRM8000 Published as a conference paper at ICLR 2026
ACKNOWLEDGMENTS
This work was supported by the National Natural Science Foundation of China (Grant No. KZ37132301, KZ37145801 62441617), Strategic scientific and technological innovation coopera- tion Project 2025YFE0209100), CCF-Kuaishou Fund for Exploring Large Models (Grant N KuaiShou20250122, and the Research Funding of Hangzhou International Innovation Institute of Beihang University (Grant No.2024KO095),
STATEMENTS
ETHICS STATEMENT
The authors of this paper have read and adhered to the ICLR Code of Ethics. Our research focuses on developing Process Reward Models (PRMs) to enhance the logical coherence and reliability of Large Language Models (LLMss) which we helieve positive contribution. All datasets used based on the public, non-sensitive MMLU-Pro benchmark, and all base models used in accordance with their licenses. While we acknowledge the dual-use risk inherent in advancing LLM capabilities, work is specifically aimed at improving the factuality and logical consistency of model outputs, thereby mitigating risks associated with sophisticated misinformation.
REPRODUCIBILITY STATEMENT
All details required reproduce our results are provided this paper and the forthcoming sup- plementary materials. Our context-aware training methodology and data annotation standards are described in Section 3 The experimental setup, including hyperparameters and evaluation proto- cols, is detailed in Section 4 All datasets and baseline models used for comparison are publicly available on HuggingFace, as detailed in Table The source code our data processing, training, and evaluation pipeline, training data and model checkpoints will be made publicly available upon publication to facilitate further research.
Table 3: Publicly available assets used in our experiments. HuggingFace Path
Asset Type
Dutasets
Training Data
Evaluation Data
Baseline Models
Math-Shepherd
RLHFlow-Deepseek
VersaPRM
Qwen2.5-Math-PRR
Oven//guen225-MMahh-PP----
THE USE LARGE LANGUAGE MODELS (LLMs)
The authors of this paper have read the The Use of Large Language Models (LLMss policies. The authors this paper have confirmed that LLMs were not used for research ideation or for paper writing, All conceptual work, experimental design, data analysis, and manuscript composition were conducted by the authors.
REFERENCES
Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, al. Palm: Scaling language modeling with pathways. Journal of Machine Learning Research, 24(240): 1-113, 2023.
11

---

Published as a conference paper at ICLR 2026
Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, et al. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168, 2021.
Edward J Hu, yelong shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang. and Weizhu Chen. LoRA: Low-rank adaptation of large language models. In International Con- ference on Learning Representations, 2022. URL
http://wwwwee..oooooeeeeeeeeeooe forum?
id-nZevKeeFYf9
Aaron Jaech, Adam Kalai, Adam Lerer, Adam Richardson, Ahmed El-Kishky, Aiden Low, Alec Helyar, Aleksander Madry, Alex Beutel, Alex Carney, et al. Openai ol system card. arXiv preprint arXiv:2412.1677 2024.
Muhammad Khalifa, Rishabh Agarwal, Lajanugen Logeswaran, Jaekyeom Kim, Hao Peng, Moon- tae Lee, Honglak Lee, and Lu Wang. Process reward models that think. arXiv preprint arXiv:2504.1682 16828. 2025.
168228
Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. Large language models are zero-shot reasoners. Advances in neural information processing systems, 35:22199-222135 2022.
Yifei Li, Zeqi Lin, Shizhuo Zhang, Qiang Fu, Bei Chen, Jian-Guang Lou, and Weizhu Chen. Making language models better reasoners with step-aware verifier. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 5315- 5333, 2023.
Hunter Lightman, Vineet Kosaraju, Yuri Burda, Harrison Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. Let's verify step by step. In The Twelfth International Conference on Learning Representations, 2023.
Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, et al. Self-refine: Iterative refinement with self-feedback. Advances Neural Information Processing Systems, 36:46534 46594,, 2023.
Tej Deep Pala, Panshul Sharma, Amir Zadeh, Chuan Li, and Soujanya Poria. Error typing for smarter rewards: Improving process reward models with error-aware hierarchical supervision, 2025, URL
https::/4e wwwpvaaxivvv/.a org/ 255p:/5..//////.. 19706
Shuaijie She, Junxiao Liu, Yifeng Liu, Jiajun Chen, Xin Huang, and Shujian Huang. R-prm: Reasoning-driven process reward modeling, 2025. /RtLtattt:///rxittt org. abs//503.
21295
Charlie Victor Snell, Jaehoon Lee, Kelvin Xu, and Aviral Kumar. Scaling LLM test-time com- pute optimally can be more effective than scaling parameters for reasoning. In The Thirteenth International Conference on Learning Representations, 2025.
Mingyang Song, Zhaochen Su, Xiaoye Qu, Jiawei Zhou, and Yu Cheng. Prmbench: A fine- fine-grained and challenging benchmark for process-level reward models. arXiv preprint 12X1 2025. URL
wttp://wwwww///////.....o// 03124
Zayne Rea Sprague, Fangcong Yin, Juan Diego Rodriguez, Dongwei Jiang, Manya Wadhwa, Prasann Singhal, Xinyu Zhao, Xi Ye, Kyle Mahowald, and Greg Durrett. To cot or to cot? chain-of-thought helps mainly on math and symbolic reasoning. In The Thirteenth International Conference Learning Representations, 2025. URL /tRlhttps: pettpp:://opereviii
net forumm?
@d-w6nlcSSKK.
Zhiqing Sun, Longhui Yu, Yikang Shen, Weiyang Liu, Yiming Yang, Sean Welleck, and Chuang Gan. Easy-to-hard generalization: Scalable alignment beyond human supervision. Advances in Neural Information Processing Systems, 37:51 11:51186 111-51168, 2024.
Jonathan Uesato, Nate Kushman, Ramana Kumar, Francis Song, Noah Siegel, Lisa Wang, Antonia Creswell, Geoffrey Irving, and Irina Higgins. Solving math word problems with process-and outcome-based feedback. arXiv preprint arXiv:2211. 14275, 2022.
12

---

Published as a conference paper at ICLR 2026
Ziyu Wan, Xidong Feng. Muning Wen, Stephen Marcus McAleer, Ying Wen, Weinan Zhang, and Jun Wang. Alphazero-like teee-search can guide large language model decoding and training. In Forty-first International Conference Machine Learning. 2024.
Peiyi Wang, Lei Li, Zhihong Shao, Runxin Xu, Damai Dai, Yifei Li, Deli Chen, Yu Wu, and Zhifang Sui. Math-shepherd: Verify and reinforce Ilms step-by-step without human annotations. In ACL (1), pp. 9426-9439, 2024a. URL hhttps://doll org/ 10w13/ 510
acl--lngg
Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc V Le. Ed Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. Self-consistency improves chain of thought reasoning in language models. In The Eleventh International Conference on Learning Representations, 2023. URL
Yubo Wang, Xueguang Ma, Ge Zhang, Yuansheng Ni, Abhranil Chandra, Shiguang Guo, Weiming Ren, Aaran Arulraj, Xuan He, Ziyan Jiang, et al. Mmlu-pro: A more robust and challenging multi- task language understanding benchmark. Advances in Neural Information Processing Systems, 37:95266 95290, 2024b.
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoe V Le, Denny Zhou, et al. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824-248371 2022.
Yangzhen Wu, Zhiqing Sun, Shanda Li, Sean Welleck, and Yiming Yang. Inference scaling laws: An empirical analysis compute-optimal inference for LLM problem-solving. The Thirteenth International Conference Learning Representations, 2025. httpps:::///
net peet/Eorum21dd=/N44P7722
Wei Xiong, Hanze Dong. Chenlu Ye, Ziqi Wang, Han Zhong, Heng Ji, Nan Jiang, and Tong Zhang- Iterative preference learning from human feedback: Bridging theory and practice for rlhf under kl-constraint. ICML, 2024.
An Yang, Beichen Zhang, Binyuan Hui, Bofei Gao, Bowen Yu, Chengpeng Li, Dayiheng Liu, Jian- hong Tu, Jingren Zhou, Junyang Lin, et al. Qwen2. 5-math technical report: Toward mathematical expert model via self-improvement. arXiv preprint arXiv:2409. 00t.:2499.11122... 2024.
Shunyu Yao. Dian Yu, Jeffrey Zhao, Izhak Shafran, Tom Griffiths, Yuan Cao, and Karthik Narasimhan. Tree of thoughts: Deliberate problem solving with large language models. Ad- vances neural information processing systems, 36: 1809-11822. 2023a.
Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du Izhak Shafran, Karthik Narasimhan, and Yuan Cao. React: Synergizing reasoning and acting in language models. In International Conference on Learning Representations (ICLR), 2023b.
Lifan Yuan, Wendi Li, Huayu Chen, Ganqu Cui, Ning Ding, Kaiyan Zhang, Bowen Zhou, Zhiyuan Liu, and Hao Peng. Free process rewards without process labels. Forty-second International Conference on Machine Learning, 2025. URL
net forum? iid=
8ThnPFhCmE]
Thomas Zeng, Shuibai Zhang, Shutong Wu, Christian Classen, Daewon Chae, Ethan Ewer, Minjae Lee. Heeju Kim, Wonjun Kang, Jackson Kunde, Ying Fan, Jungtack Kim, Hyung I1 Koo, Kannan Ramchandran, Dimitris Papailiopoulos, and Kangwook Lee. VersaPRM: Multi-domain process reward model via synthetic reasoning data. In Forty-second International Conference on Machine Learning. 2025.
Lunjun Zhang, Arian Hosseini, Hritik Bansal, Mehran Kazemi, Aviral Kumar, and Rishabh Agar- wal. Generative verifiers: Reward modeling as next-token prediction. In The 4th Workshop on Mathematical Reasoning and Al at NeurlPS 2024a. URL httttp::/ccperrvvieee net./
Yunxiang Zhang, Muhammad Khalifa, Lajanugen Logeswaran, Jaekyeom Kim, Moontae Lee, Honglak Lee, and Lu Wang. Small language models need strong verifiers to self-correct rea- soning. In ACL (Findings), pp. 15637-15653, 2024b. URL https: //wi. orwp:o..rg111011.. v1/2024. findi inddngg-aae 924
13

---

Published as a conference paper at ICLR 2026
Zhenru Zhang, Chujie Zheng, Yangzhen Wu, Beichen Zhang, Runji Lin, Bowen Yu, Dayiheng Liu, Jingren Zhou, and Junyang Lin. The lessons developing process reward models in mathematical reasoning. arXiv preprint arXiv:2501.07301, 2025.
Jian Zhao, Runze Liu, Kaiyan Zhang, Zhimu Zhou, Junqi Gao, Dong Li, Jiafei Lyu, Zhouyi Qian, Biqing Qi. Xiu Li, and Bowen Zhou. Genprm: Scaling test-time compute of process reward models via generative reasoning. arXiv preprint prXiv:250400881,, 2025.
Chujie Zheng, Zhenru Zhang, Beichen Zhang, Runji Lin, Keming Lu, Bowen Yu, Dayiheng Liu, Jingren Zhou, and Junyang Lin. Processbench: Identifying process errors in mathematical rea- soning. arXiv preprint wrXiv:2412.06553 2024.
Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, et al. Judging llm-as-a-judge with mt-bench and chatbot arena. Advances in neural information processing systems, 36:46595-46623, 2023.
14

---

Published as a conference paper at ICLR 2026
A ERROR TYPE ANALYSIS
In this section, we conduct granular analysis of the specific error types manifested in the failure cases of both Contex(PRM and the VersaPRM baseline across each subdomain of the set. Our objective is to empirically validate that the performance gains achieved by ContextPRM stem from its enhanced capacity identify logical inconsistencies. Furthermore, we provide correlation analysis to explicitly demonstrate the relationship between the model's ability to detect logical errors and the overall improvement reasoning accuracy.
A.1 ERROR TYPE ANALYSIS BY QUESTION CATEGORY
As illustrated in Figure 9 and Figure I0 we conducted a comprehensive eтror type analysis by extracting all failure instances for both ContextPRM and VersaPRM from the complete test set. Ad- ditionally, we measured the Fixed Set, which contains instances where the baseline failed but Con- textPRM successfully predicted the correct outcome to investigate the nature of the improvements. To ensure consistency, we employed gpt-4o 40--ino-mii-- 1eniiii--22-0000 8 to classify the error types of failure cases, strictly adhering to the same classification taxonomy defined in Section 3.2 The visu- alized data reveals distinct trends:
Dominance of Factual Errors in Math-Adjacent Domains. In mathematically intensive domains such as Mathematics and Physics, the proportion of the Incorrect error type (fac- tual calculation or definition errors) is markedly higher compared to humanities disciplines. While solving problems these domains necessitates rigorous logic, the correctness of Chain-of- Chain-of-Thought (CoT) is predominantly governed by the self-consistency and absolute accuracy of high-frequency definitions, formulas, and symbolic manipulations, which are sparse in non-mathematical domains, leading to a different error distribution.
Reduction of Logical Errors in ContextPRM. Within the failure cases of VersaPRM, the proportion of logical errors ((Logical Fallacy, Misinterpretation, and Misdirection) is significantly higher than that observed in Contex1PRMM. This discrepancy indicates that ContextPRM possesses a superior capability to detect and penalize logical flaws. Conse- quently, this specific capability directly contributes to the reduction in the total number of errors, validating the effectiveness of our context-aware training objective in filtering out logically incoherent reasoning paths.
ContextPRM Type Percentage Distribution by Category
80
Error Classification
Nisdiection-
2.0
4.8
21
25.0
144
143
4.2
5.6
19e
Mishnerr retation-
11.0
2.4
211
85
14
43
15.0
116
2:4
16.7
67
83
169
17.9
22.0
Iscal Erross
701
42
48
28
43
71
90
Marh
Chementry
Computter Science
Engineering
Physica
Bisioyy
Pyydeengg
Businesi
Esmonica
Law
Ofter
Catagory
Figure 8:
Distribution of error types in failure cases for ContextPRM across different domains. The model demonstrates a lower prevalence of purely logical errors compared to the baseline.
ContextPRM Improves Reasoning Primarily by Correcting Logical Errors. The anal- ysis on Fixed Set (Figure 10) demonstrates that the majority of errors rectified by Con- textPRM belong to logical categories, rather than simple factual inaccuracies (Incorrect). Notably, this trend is even more pronounced in humanities and non-math-adjacem domains, where logical errors occur more frequently than calculation errors. This confirms that Con- textPRM enhances reasoning capabilities primarily by identifying and correcting flaws in
15

---

Published as a conference paper at ICLR 2026
VersaPRM Error Type Percentage Distribution by Category
12.0
73.0
15.1
73.2
36.0
73.5
23.7
803
Error Classificatio
Logical Failay-
Nidirection
Typ
Meerterratattonn
145
5/2
4.8
12.0
31.0
111
13A
11.3
38.
Total Emooss
771
40
25
49
54
54
82
73
77
5e
Chemistryy
Computeer Schermc
Engineering
Plynic
Blstagy
Heatth
Payctaloay
Brsemess
Econoricca
Law
Hattory
Philenaphy
Other
Category
Figure 9: Distribution of error types in failure cases for VersaPRM (Baseline). A higher proportion of logical fallacies and inconsistencies is observed compared to ContextPRM.
logical flow and contextual coherence, effectively mitigating the non-sequiturs and incon- sistencies that often plague traditional reward models in these complex, qualitative do- mains.
Fixed Error Type Percentage Distribution by Category
Incomect-
22:4
21.3
31.3
28.6
56.6
14.3
14.3
20.8
-10
60
Error Classification
Logical Fallzyy
20
24.4
00
40.0
1311
42.9
11.1
3000
4699
20.6
50
Modieection
28.0
0.2
31.3
3.0
23.0
33.3
24.3
34
1:4
25.0
1433
214
25.8
15.4
26.4
Misnnerpmetttionn
18.9
222
23.3
14.3
233
143
143
14.3
2111
AB
Meeth
Chemintry
Computer Scienca
Enginntering
Phynics
Biology
Health
Prytboingy
Business
Ecmamiic
Hestory
Phlossphy
Category
Figure 10: Distribution of error types in the Fixed Set (samples where VersaPRM failed but Con- textPRM succeeded). The heatmap highlights that ContexIPRM predominantly corrects logical rors.
A.2 CORRELATION ANALYSIS
To provide quantitative assessment of whether the efficacy of ContextPRM is indeed driven by its handling of logical errors. we conducted a correlation analysis between two key variables across different domains: the Performance Improvement achieved by ContextPRM over the baseline, and the Ratio of Logical Errors Majorrity Voting Baseline of each question category (calculated as 100% Percentage of Incorrectness Errors).
As depicted in Figure we observe a strong positive correlation between the prevalence logical emoor in domain and the magnitude of the performance gain yielded by ContextPRM. Domains with a higher density of logical pitfalls, such as Philosophy and Law, exhibit the most substantial performance boosts. This strong statistical relationship provides compelling empirical evidence that CoMtextPRM realizes its improvements specifically by correcting logical flaws and enhancing contextual coherence, rather than merely refining factual accuracy.
16

---

Published as a conference paper at ICLR 2026
Correlation between Logical Error Ratio and Performance Improvement (Pearson Correlation: 0.80)
Performance Improvement (Accuracy %)
100
Retio of Logical Lyrors (1007s incorrect)
Performance Improvement
Ratio Logical (%)
60
20
A8
Meth
Chemiety
Computer Sciancn
Engineering kng
Physlka
Biology
Health
Paychology
Susiness
Econamic
Law
History
Phllsophy
Other
Evaluation
Figure 11: Correlation analysis between the ratio of logical errors in a domain and the performance improvement achieved by ContextPRM. The strong positive correlation (r 0.80) confirms that our method most effective domains where reasoning failures are primarily driven by logical inconsistencies rather than factual inaccuracies.
B DETAILED RESULTS FOR MAIN EXPERIMENT
this section, we provide a comprehensive, domain-wise breakdown of our main experimental results. These figures supplement the aggregated results presented in Section 4.2 and offer more granular view of model performance across the diverse disciplines within the MMLU-Pro-CoT Eval benchmark. For each of the two primary test-time scaling methods, Weighted Majority Voting (WMV) and Best-of-N (BoN) sampling, we present both a line plot illustrating performance as function of the number of candidate CoTs and heatmap visualizing the performance improvement over the majority voting baseline.
B. DETAILED RESULTS FOR WMV SAMPLING
Figure 12 displays the inference accuracy of all evaluated models under the WMV sampling method across each individual domain. The horizontal axis represents the number of generated CoT solu- tions on a logarithmic scale, while the vertical axis shows the resulting accuracy. Figure [3 provides complementary view, showing the column-wise normalized accuracy improvement of each model over the standard majority voting baseline.
17

---

Published as a conference paper at ICLR 2026
Sleve
0
wating
ContedFRM Lama--TINDOO
VenaRRM
Opeer S-latt
Waighted Magrity toring
Path Chapherd
Figure 12: Detailed domain-by-domain performance comparison under WMV sampling. Con-
textPRM consistently demonstrates strong performance, particularly in non-math-adjacent domains.
Performance Improvement ever Majority Voting Baseline
Hicerty Vodinng
5.0
00
0.0
Owen2.5 Math-PFM
18
6.7
3.3
27
2.7
3.7
31
4.6
-2
0.3
6:
13
-34
3.5
277
13
0.8
14
Llno PRMSSOO.
088
03
13
2.0
2.3
18
06
6.
0.3
2.5
133
-14
18
90
0.4
98
Komstodd
03
43
83
40.
ContestRMM (0urd)
54
9.
31
10.4
3.3
A2
Heat
Chemotor
Engroeetigg
Fhysacs
Masts Saferee
Bastogy
Phytletay
Busiees
Hostar
Weinsash
4sch Anjices
Exosst.ta
Compuit
Figure 13: Heatmap of performance improvement over the majority voting baseline with WMV sampling. ContextPRM (bottom row) achieves the most significant and consistent improvements across nearly all evaluation domains, especially in non non-mathematical subjects.
18

---

Published as a conference paper at ICLR 2026
B.2 DETAILED RESULTS FOR BON SAMPLING
Similarly, this subsection details the results for the Best-of-N (BoN) sampling method. Figure [4 illustrates the inference accuracy for each model the number of candidate solutions increases. Figure presents the corresponding heatmap of performance improvements over the majority vot- ing baseline. using a purle-green color scale visualize the results. These figures confirm that the strong cross-domain generalization of ContextPRM observed under WMV sampling is robust and holds under the BoN evaluation scheme as well.
Danday
Bast of
Mast Chopherd
Figure 14: Detailed domain-by dommaan performance comparison under BoN sampling. The trend of ContextPRM's superior performance in non-math-adjacen domains continues to hold.
Performance improvement over Majority Voting Baseline
Mooffyy veting
0.0
0.6
50
0.0
6.0
0.0
0.6
6.0
00
0.0
0.0
86
0.0
0.6
60
00
Oper2.5-Math-P FRN
-15
-3.2
..1
-03
-.5
-10.0
-5.7
-33
-4.
-33
4.7
-2.
11.2
136
Stephard
-99
4.7
-12.0
-3.5
-2.1
-30
0.7
344
9.1
3.3
0.7
4.4
-A
42
.13
-3A
-17
:03
-.77
-25
0.1
25
24
29
CoRtosPRI 10ust
41
33
.99
35
166
0.
6.2
67
59
39
11.9
53
Chemmtr
Engineeiing
Physoo
Marty Adjemm
Pysteebay
Agprred
Excopt Hamh
Compuder
Exalustion Donah
Figure 15: Heatmap of performance improvement over the majority voting baseline with BoN sam- pling. ContextPRM (bottom row) again shows substantial gains, particularly in logic-intensive do- mains such as 'Philosophy' and 'History', reinforcing its strong generalization capabilities.
19

---

Published as a conference paper at ICLR 2026
C DETAILED RESULTS FOR ABLATION ON CONTEXT METHODS
To further dissect the contributions of our proposed components, this section delves into the specifics of the ablation studies referenced in Section 4.3 We present granular visualization of the perfor- mance of models trained with only our Context-Based Training or our Annotation. The following figures illuminate the interplay between these two components under both WMV and BoN sampling schemes, offering detailed view of their synergistic impact.
Based
C.I DETAILED RESULTS FOR WMV SAMPLING
Here, we present the results under the WMV sampling scheme. The line plots in Figure 16 track inference accuracy across domains, while the heatmap in Figure 17 quantifies the improvements over the majority voting baseline. Together, these figures offer a detailed view into the performance trade-offs observed when applying our methods in isolation compared their full integration.
20.501 bay
vns sokiic
Namber memna soutter
kumber bout legiccun sur www.eeeddddd...o.
Magrity og
Contoctttt 1Dura
Only
Only
Figure 16: Detailed domainn performance of ablation models under WMV sampling.
Performance Improvement Majority vottigg Baseline
69
60
6.
Only
1:3
8.3
8.0
4A
Contact label Only
4.5
8.7
2.3
13
5.3
5.8
8.3
4.6
0.7
2.2
54
2.0
41
2.3
1.7
741
35
Contex/PM (Durs)
5.4
3.3
0.7
4.3
14
5,3
67
1.4
7.2
8.
Heats
Chermaoy
Compentr Stience
Binwayy
Heratt
Lawe
Habeyy
Nos Adarmmn
Secsot Mamts
Evaluation Domain
Figure 17: Heatmap of performance improvement for ablation models with WMV sampling.
20

---

Published as a conference paper at ICLR 2026
C.2 DETAILED RESULTS FOR BON SAMPLING
To confirm the robustness of our findings, this subsection provides the corresponding evaluation under the BoN sampling method. The accuracy trends are plotted in Figure[T8] with the performance gains visualized the heatmap in Figure [9 These results reinforce the conclusions drawn from the WMV analysis, showcasing the consistent synergistic relationship between our proposed training and annotation methods.
Pachetop
Eoseciicc
of
10
Worphtad Macurty Valing
textr9gt Donr
Cundonn
Duntcell Only
Figure 18: Detailed domain-by-domain performance of ablation models under BoN sampling.
Performance improvement Majority Voting Baseline
0.6
30
0.0
0.
53
Ofly
2.7
abee Only
3.8
3.6
6.5
34
Oar (Dus)
411
3.4
.99
33
116
44
62
0,7
>9
3.0
92
29
11.3
Mett
Chemetry
Computerr Silecce
Health
Buseees
Inserr
Bom steb, Adjeeme
Diaor
Al Smipps..e Mat
Figure 19: Heatmap of performance improvement for ablation models with BoN sampling.
D QUALITATIVE ANALYSIS ON SELECTED EXAMPLES
In this section, we present qualitative analysis selected examples provide deeper insights into the performance of ContexIPRM compared to the VersaPRM baseline. We analyze two key scenar- ios presented in the following subsections:
Success Cases (Subsection D.I). where ContextPRM correctly identifies the first logical error in reasoning chain that VersaPRM fails to detect. These examples highlight the nuanced understanding of our model.
Failure Cases (Subsection D.2). where ContextPRM identifies an enror later than its actual oceur- rence according to the ground truth. These examples are chosen illustrate specific limitations and areas for future improvement.
Each case is presented in consistent format for clarity. We provide the full context, including the question, options, and the correct answer. For each Mhain-of-Thhughhe (CoT) solution, the ground- truth first erroneous step is highlighted in bold red for immediate identification. Below the reasoning steps, we display the sequence of rewards assigned by both ContextPRM and VersaPRM, followed by Explanations that analyzes the models' performance that specific case.
21

---

Published as a conference paper at ICLR 2026
D.1 SUCCESS CASES: CONTEXTPRM CORRECTLY IDENTIFIES ERRORS MISSED BY VERSAPRM
Question ID: 7251 mmlu economics Category: Economics
Question:
Which of the following criticisms standard ("plain vanilla") GARCH models can over- come by EGARCH models?
i) Estimated coefficient values from GARCH models may be negative
ii) GARCH models cannot account for leverage effects
iii) The responsiveness of future volatility to positive and negative shocks is symmetric under
GARCH formulation
iv) GARCH models cannot allow for feedback from the volatility to the returns
Options: A. (iii) and (iv) only B. (i), (iii), and (iv) only C. (i) and (iii) only D. (ii, (iii,, and (iv) only E. (i), (ii), and (iii) only F, (ii) and (iv) only G. (ii) and (iii) only H. (i), (ii), (iii),
and (iv) I. (i) and (iv) only (i) and (ii) only
Answer: E
CoT 1:
Let's break down the criticisms of standard GARCH models and how EGARCH models address them:
2. i) Estimated coefficient values from GARCH models may be negative: This can occur due the mis-specification of the model. However, it's not direct criticism of GARCH models that EGARCH models address.
3. ii) GARCH models cannot account for leverage effects: Leverage effects refer to the phe- nomenon where large negative returns are associated with higher volatility. EGARCH models can address this issue by incorporating an asymmetric effect of returns on volatil- ity.
4. iii) The responsiveness of future volatility positive and negative shocks is symmetric under GARCH formulation: This is criticism GARCH models. EGARCH models address this issue by allowing for asymmetric effects of positive and negative shocks on volatility.
iv) GARCH models cannot allow for a feedback from the volatility to the returns: This is not criticism of GARCH models, but rather characteristic of the model. GARCH models can allow for feedback from volatility to returns.
Therefore, the correct answer one that addresses criticisms (ii) and (iii), which are the leverage effects and symmetry of responsiveness.
7. The answer is (G).
ContextPRM rewards: 1.0000, 0.0173, 0.9961, 1.0000, 0.0090, 0.6758, 1.0000
VersaPRM rewards: 1.0000, 0.9336, 0.9961, 0.9922, 0.7656, 0,8203, 0.8984 Explanation:
The VersaPRM fails to identify that the student incorrectly states EGARCH
models do not overcome the criticism of potentially negative GARCH coefficient values.
Question ID: 6191-mmluu... health Category: Health
Question:
Which of the following vitamins is required for fatty acid b-oxidation?
Options: A. Vitamin C B. Vitamin E C. Vitamin B12 D. Thiamin E. Riboflavin F, Folic Acid G. Vitamin D Vitamin Biotin
Answer: E
22

---

Published as a conference paper at ICLR 2026
CoT 1:
Fatty acid b-oxidation a process by which fatty acids are broken down into acetyl-CoA units for energy production.
2. Biotin is a crucial cofactor for enzymes that catalyze the carboxylation reactions involved in fatty acid b-oxidation.
3. Therefore, biotin is essential for the process of fatty acid b-oxidation.
4. The answer is (I).
ContextPRM rewards: 1.0000, 0.0107, 1.0000, 0.3418
VersaPRM rewards: 1.0000, 0.9844, 1.0000, 1.0000 Explanation: The VersaPRM fails detect the incorrect assertion that biotin is a crucial cofactor for fatty acid b-oxidation.
CoT
1. Fatty acid beta-oxidation is a process that involves the breakdown of fatty acids to pro- duce energy.
2. Biotin is a coenzyme that plays role in fatty acid metabolism and is specifically required for fatty acid beta-oxidation.
The answer is (I).
ContextPRM rewards: 1.0000, 0.0025, 0.1099
VersaPRM rewards: 1.0000, 0.9922, 1.0000
Explanation: The VersaPRM fails to detect the factual error stating Biotin is required for fatty acid beta-oxidation.
Question ID: 3393 mmmu millobi Category: Biology
Question:
Which of the following characteristics is predicted for an early-successional plant commu- nity?
Options: A. High detrital biomass B. High number of predatory species C. High presence of fully matured plant species D. High frequency of R-selected species E. High rates of soil nutrient depletion F. High diversity of plant species G. High ratios of primary production to standing-crop biomass H. High frequency of K-selected species I. Low rates of photosyn- thesis J. High niche divergence among co-occurring species
Answer:
CoT
1. Early-successional plant communities are typically characterized by a high abundance of R-selected species.
2. These species adapted to rapidly colonize and exploit resources after distur- bances such as fire or clear-cutting, but they typically have low levels reproduc- tion and high levels of mortality.
3. R-selected species are able to quickly grow and produce large amounts of seeds, which allows them to quickly colonize a new area.
These species are usually annual or biennial plants, and they have high reproductive rate.
5. K-selected species, on the other hand, are adapted to slowly colonize and exploit re- sources, and they typically have high levels of reproduction and low levels of mortality.
23

---

Published as a conference paper at ICLR 2026
6. They usually perennial plants that have high level of competition for resources.
7. In suryyssucesssioall plant communities, soil nutrient levels typically high due to the rapid decomposition of organic matter.
Therefore, we can expect a high frequency of R-selected species in early-successional plant communities.
9. The answer is (D).
ContextPRM rewards: 1.0000, 0. 0.1069, 1 1.0000, 1.0000, 0.9922, 1 1.0000, 0.9961, 1.0000, 1.0000
VersaPRM rewards: 1.0000, 0.9805, 0.9805, 1.0000, 0.9531, 0.9922, 0.9062, 1.0000, 0.9961
Explanation: The VersaPRM fails to detect the factual error regarding R-selected species'
reproductive rates Step
CoT
1. Early-successional plant communities are characterized by a low number of plant species.
They also have a high abundance of fast-growing and highly reproductive species, known as R-selected species.
3. These species have traits that allow them to quickly colonize and exploit available re- sources.
4. R-selected species are typically small, produce many seeds, and have short lifespan.
5. In contrast, K-selected species are larger, produce fewer seeds, and have longer lifespan.
6. The answer is (D).
ContextPRM rewards: 0.0240, 1. 1.0000, 1.0000, 1.0000, 1.0000, 1.0000
VersaPRM rewards: 0.8203, 1. 1.0000, 1.0000, 1. 1.0000, 1.0000, 1.0000 Explanation: The VersaPRM fails to identify that the initial step, though factually correct, misdirects the reasoning from the question's core concept.
Question ID: 10995 1t1.5mmmmuu.pphooopppp. Category: Philosophy
Question:
Shapiro claims that surveys have been confirmed by longitudinal studies that indicate that Options: A. the use of illegal drugs is usually temporary phase in adolescence. B. most individuals experiment with illegal drugs but do not continue use. C. excessive use of legal drugs is the norm. D. individual's illegal drug inevitably diminishes over time. E. those who begin using illegal drugs typically stop before addiction sets in. F. longitudinal studies have shown no consistent patterns in illegal drug use. G. illegal drug users typically transition to legal substances over time. H. moderate and/or controlled use of illegal drugs the norm. those who begin using illegal drugs typically do not stop until they become addicted. J. excessive use of illegal drugs is the norm.
Answer:
CoT
1. The statement that surveys have been confirmed by longitudinal studies is that most individuals experiment with illegal drugs but do not continue use.
This choice is (B).
3. The answer is (B).
24

---

Published as a conference paper at ICLR 2026
ContextPRM rewards: 0.0415, 0.9922, 1.0000
VersaPRM rewards: 0.8750, 0.9531, 0.9844
Explanation: The VersaPRM fails detect that the student misidentifies the correct claim confirmed by longitudinal studies, selecting option B instead of H.
CoT
1. Shapiro claims that surveys have been confirmed by longitudinal studies that indi- cate that most individuals experiment with illegal drugs but do not continue use.
This matches option (B), which states that "most individuals experiment with illegal drugs but do not continue use".
3. The answer is (B).
ContextPRM rewards: 0.1143, 0.9844, 0.9961
VersaPRM rewards: 0.8281, 0.9727, 0.9727
Explanation: The VersaPRM fails to detect that the student's initial premise misrepresents Shapiro's s claim.
Question ID; 5860 1110000mluuohth
Category: Other
Question:
All of the following statements about gerrymandering are true EXCEPT that Options: A. is practice that is universally disallowed in all democratic nations B. is process that is only possible within a multi-party system C. major political parties in the United States D. can lead to an increase in voter turnout E, can divide like-minded voters into several districts to reduce their influence E. is a strategy used to increase the number of districts with majority of minority voters G. has been allowed by Supreme Court in some instances H. increases the control voters have over their elected officials I. favors incumbent candidates over challengers
strategy utilized by both
Answer:
CoT 1:
1. Gerrymandering is practice where electoral districts manipulated favor polit- ical party or group over another, often by dividing or packing voters into districts.
Statement D, can lead an increase in voter turnout", is not true. Gerrymander- ing often has the opposite effect, as it disenfranchise or dilute the voting power of certain groups, leading to decreased voter turnout.
The answer (D).
ContextPRM rewards: 1.0000, 0.4766, 0.0079
VersaPRM rewards: 1.0000, 0.9414, 0.8164
Explanation: The VersaPRM fails to identify the student's misinterpretation of the problem- solving strategy, specifically the lack of comprehensive evaluation of all options in 'true EXCEPT* question.
D.2 FAILUUEE CASES: CONTEXTPRM IDENTIFIES ERRORS LATER THAN GROUND TRUTH
Question ID: 1968_mmlu Jaw Category: Law
25

---

Published as a conference paper at ICLR 2026
Question:
One Sunday afternoon, a sports fan went to a professional football game. While standing in line at a concession stand, he got into an argument with three men ahead of him in the line. The three men were friends attending the game together. One of the men punched the sports fan in the mouth. A fight ensued. During the fracas, one of the other three men hit the sports fan over the head with bottle, which caused him to suffer a serious concussion. The sports fan does not know for certain which of the three men struck him with the bottle. If the sports fan asserts a claim against the man who punched him in the mouth to recover damages for the head injury, will the sports fan prevail?
Options: A. No, because the sports fan should not have argued with the three men in the first place. B. No, because the sports fan does not have legal basis to sue the who punched him the mouth. C. No, because the man who punched him in the mouth did not actually strike the sports fan with the bottle. D. No, because the sports fan could have avoided the fight. E. Yes, because the man who punched him in the mouth was the instigator who struck the first blow that started the fracas. F. Yes, because the sports fan was defending himself from an unprovoked attack. G. Yes, because the man who punched him in the mouth and the other men were acting in concert. H. No, because the sports fan cannot offer proof as to which of the men struck the sports fan over the head with the bottle. I. Yes, because the three men were acting as a group and are therefore jointly responsible. J. Yes, because all the men were involved in the fight and are therefore liable.
Answer:
CoT
1. The question is asking whether the sports fan can recover damages from the man who punched him in the mouth.
2. Since the sports fan does not know for certain which of the three men struck him with the bottle, he cannot prove that the man who punched him in the mouth was the one who actually caused the head injury.
3. Therefore, the sports fan cannot recover damages from the man who punched him in the mouth.
The answer is (H).
ContextPRM rewards: 1.0000, 0.9766, 0.4922, 0.1143
VersaPRM rewards: 1.0000, 0.9844, 0.8906, 0.9297
Explanation: While VersaPRM failed to identify any errors, and ContextPR.M incorrectly identified Step 2 as the first error, both PRMs critically missed the fundamental legal mis- interpretation in Step 1. Step I incorrectly assumes that direct causation by the specific defendant is the sole basis for liability, failing to account for principles like 'acting con- cert' or joint and several liability. This suggests shared weakness in both PRMs regarding their understanding and application of complex legal doctrines in tort law.
CoT 2:
1. The man who punched the sports fan in the mouth may be liable for the head injury caused by the bottle, but the sports fan cannot prevail because he cannot prove which of the three men actually struck him with the bottle.
2. This is example of the requirement of proximate cause in tort law, which holds that defendant can only be liable for injuries that are a direct result of their actions.
3. The answer is (H).
ContextPRM rewards: 0.8828, 0.2559, 1.0000
VersaPRM rewards: 0.9062, 0.9062, 0.9336
Explanation: VersaPRM failed to identify any errors in the provided solution. Both Con-
TextPRM and VersaPRM critically missed the fundamental error in Step 0, which incorrectly
26

---

Published as a conference paper at ICLR 2026
concludes that the sports fan cannot prevail due inability to identify the specific perpe- trator of the bottle attack, thereby misinterpreting the legal principle of "acting in concert' or joint liability. ContextPRM incorrectly identified Step the first error. This suggests shared weakness in both PRMs regarding their understanding and application of complex legal doctrines, specifically joint liability and causation tort law.
Question ID: 1986 mmlu psychologyy Category: Psychology
Question:
person who received Level rating on the Rancho Los Amigos Scale of Cognitive Functioning:
Options: A. is unresponsive to all stimuli but shows signs of basic physiological functioning such breathing and heartbeat. B. is highly responsive and coherent, but cannot remember or integrate past and recent events. C. is functioning at an intellectual level that average for his/her age, education, and demographic background. D. is responsive stimuli but cannot remember or integrate past and recent events, and exhibits impaired judgment. E. is confused and incoherent, may exhibit bizarre behavior, and is unable care for him/herself. F. is functioning at an intellectual level that is superior for his/her age, education, and de- mographic background. G. is nonresponsive visual auditory stimuli and seems to in a state of deep sleep. H. is alert and oriented and can remember and integrate remote and recent events but may have some impairment in judgment, planning, and abstract reasoning. Answer: E
CoT
1. Rancho Los Amigos Scale of Cognitive Functioning categorizes cognitive functioning into eight levels.
Level 1: No response coma
3. Level 2: Generalized response awake but unable respond
4. Level 3: Confused-agitated - confused and incoherent, may exhibit bizarre behavior, and is unable to care for him/herself
5. Level 4: Confused responsive to stimuli but unable to remember or integrate past and recent events
6. Level 5: No problem functioning at an intellectual level that is average for his/her age, education, and demographic background
7. Level 6: Automatic functioning at intellectual level that is superior for his/her age, education, and demographic background
8. Level 7: Improvement continues to improve in functioning, but still impaired in some areas
9. Level 8: Normal fully recovered and functioning at pre-injury level.
10. Therefore, the description in choice best matches person who received Level rating the Rancho Los Amigos Scale of Cognitive Functioning.
11. The answer is (D).
ContextPRM rewards: 1.0000, 0.9688, 0.9062, 0.9922, 0. 0895 0.8359, 0.7266, 0.8359, 1.0000, 0.9961, 1.0000
VersaPRM rewards: 1.0000, 0.9922, 0.9766. 0.9844, 0.9062, 0.8242, 0.8633, 0.9688, 0.9961, 0.8047, 0.9805
Explanation: VersaPRM failed identify any emror in the solution. Both ContextPRM and VersaPRM critically missed the first error Step 3, which incorrectly describes Level 3 of the Rancho Los Amigos Scale with characteristies that actually belong to Level 4. This indicates shared weakness in their precise domain knowledge regarding the Rancho Los Amigos Scale.
27

---

Published as a conference paper at ICLR 2026
Question ID: 2618_6m1m 1as18mmmluuppp mmllu.ssycholoyy Category: Psychology
Question:
The primary advantage of the forced-choice distribution of rated attributes in performance appraisal is that
Options: A. allows for more flexibility in rating employees B. reduces the bias in ratings as it is based on direct observation C. less time-consuming, since does not require in-depth analysis D. is more accurate as it focuses measurable attributes E. is more objective, since does not deal with the strengths and weaknesses of the employee F. enhances the reliability of ratings G. eliminates statistical concerns about ipsative measurement H. is easier and faster to use, since only two choices are available I. eliminates the need for continuous feedback and follow-up J. is more comprehensive it covers all aspects of performance Answer: F
CoT
1. Forced-choice distribution eliminates statistical concerns about ipsative measure- ment.
2. Ipsative measurement refers to the comparison of an individual's performance on single task or set of tasks across multiple assessments.
3. Forced-choice distribution eliminates this problem by comparing each rating choice against the other two options.
4. The answer is (G).
ContextPRM rewards: 0.9922, 0414 0.0255, 1.0000
VersaPRM rewards: 0.9961, 0.9805, 0.7812, 0.8672
Explanation: While VersaPRM failed to detect any errors, and ContextPRM incorrectly identified Step the first error, both PRMs critically missed the fundamental factual error in Step O. The ground truth states that forced-choice methods often generate ipsative data, which inherently has statistical concerns, rather than eliminating them. This indicates shared weakness in both PRMs' ability evaluate the factual correctness of initial premises within this domain.
CoT 2:
Forced-choice rating involves evaluating two choices, "above average" and "below average", for each attribute.
2. This method eliminates the halo effect and leniency, providing a more objective evalua- tion.
3. The answer is (G).
ContextPRM rewards: 0.9883, 0.2451, 0.7617
VersaPRM rewards: 0.9414, 0.9258, 0.8320
Explanation: While VersaPRM failed to detect any errors, and ContextPRM incorrectly identified Step as the first error, neither PRM recognized the fundamental misinterpretation of cchcce rating' in Step O. The ground truth clearly states that Step 0's description of the method is an oversimplification and misrepresentation, as it typically involves selecting statements from groups rather than simply evaluating 'above average" and 'below average". This suggests a shared weakness in their ability to identify foundational conceptual errors related to specific performance appraisal methodologies.
28


Reviews:
[{"review_id": "2NqLK6Wa3d", "reviewer": "Reviewer_wDYj", "summary": "The paper suggests context-coherence-based process reward modeling. Unlike past works that aim to evaluate the isolated correctness of a specific step, this work aims to assess its logical connection with the previous step. In some sense, it sounds a bit like a NLI task, aiming to evaluate whether two connected segments are entailments or contradictions.\n\nThe paper creates a labeled dataset via gpt-4o-mini and trains Llama-3.1-8B-PRM800K on the dataset. The performance of the trained model is showcased by a labeled subset of MMLU-Pro. Based on the performance gains achieved when the model is used as a external verifier. \n\nThe trained model outperform baslines highlighting the effectivenss of the proposed method.", "questions": "See weaknesses.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "(1) The paper proposes an NLI-like training schema for PRMs", "weaknesses": "(1) I might have missed something, but I am not yet persuaded that this is a big difference from what other PRMs have been doing. Other PRM models also receive the S-1 steps (saying step S is the one of interest). While they may have not been explicility trained to look at logical connection of previous steps with step S I think that would have been kept into account. Accordingly in Figure 3 we can see that the performance gains is more pronounced in non-math domains. Which I think is not surprising as the authors trained on additional non-math data.\n\n(2) I think the paper would have been more interesting if their was more focus on math. Math relies heavily on logical coherence of steps, which I think is a more suitable domain for the proposed method.\n\n(3) A qualitative study on whether the trained model effectively detects different types of errors, compared to correctness-only PRMs, might have helped persuading me about (1), that this model is different from past models \n\n(4) Not a major issue, but I would have preferred if the training have been done to different models as well. Is the PRM800k necessary for this to work? Does it work in smaller/bigger models?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "iDFph6gb8V", "reviewer": "Reviewer_jNz1", "summary": "Standard Process Reward Models (PRMs) excel at mathematical reasoning but fail to generalize to non-mathematical domains due to a reliance on domain-specific knowledge. This paper introduces ContextPRM, a novel approach that addresses this limitation by shifting the learning objective from verifying isolated correctness to modeling domain-agnostic logical flow. Through a new data annotation standard and a context-aware training method focused on the logical coherence between reasoning steps, ContextPRM achieves state-of-the-art multi-domain generalization. It demonstrates a 6.5% average accuracy improvement over the majority voting baseline across nine non-mathematical MMLU-Pro domains, significantly surpassing the 2.2% gain from the previous best model, VersaPRM.", "questions": "1. Why does ContextPRM underperform VersaPRM in the math domain? Could you further explain it?\n\n1. In Section 3.1, what is the formatting function $\\mathcal{F}$, just simple concatenation? How is it defined? What is the definition of the new context $\\tilde{\\mathcal{S}}$ and the contextualized step $\\mathcal{P}$? How does the coherence label $c_i$ come? Could you further explain them?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The paper clearly identifies a significant limitation of current Process Reward Models (PRMs): while effective in mathematical domains, they fail to generalize to non-mathematical domains like law, history, or philosophy. \n\n1. Instead of adding more domain-specific knowledge, the paper proposes a novel shift in the learning objective: moving from verifying \"domain-specific correctness\" to modeling \"domain-agnostic logical flow\", which is an intuitive and reasonable idea.\n\n1. In these domains, ContextPRM improves average accuracy by 6.5% over the majority voting baseline, outperforming the previous SOTA (VersaPRM), which achieved a 2.2% gain.", "weaknesses": "1. The model's strong cross-domain generalization comes at a cost. The ablation study reveals that the full ContextPRM model shows a 2.2% performance decrease in the math domain compared to the baseline, suggesting a trade-off was made to achieve broader applicability. \n\n1. The qualitative \"Failure Cases\" reveal that the method still fails when complex, specialized knowledge is required. In examples from law (tort liability) and psychology (Rancho Los Amigos Scale), both ContextPRM and the baseline model failed to identify the fundamental initial error. This suggests that focusing on \"logical flow\" cannot fully compensate for a lack of deep, domain-specific understanding.\n\n1. Overall, the contribution is marginal. To my understanding, this paper just extends the definition of \"bad\" steps during annotation, and introduces a simple concatenation of steps as a new \"contextualized representation\", which does not really tackle the challenge of PRMs.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "v8C4xy2p1o", "reviewer": "Reviewer_iymL", "summary": "This paper addresses the poor cross-domain generalization of Process Reward Models (PRMs), which typically excel in mathematics but fail in other domains. The authors posit this is because traditional PRMs learn domain-specific correctness rather than domain-agnostic logical flow . They propose ContextPRM, a novel framework that shifts the learning objective to modeling the contextual coherence between reasoning steps. This is achieved through a synergistic combination of a new data annotation standard that labels the logical validity of step-to-step transitions and a corresponding context-aware training methodology that forces the model to evaluate these transitions in isolation . Experiments on the MMLU-Pro benchmark show that ContextPRM achieves state-of-the-art performance, particularly in non-mathematical domains.", "questions": "The context-aware training method intentionally restricts the model's input to adjacent step pairs $(S_{i-1}, S_i)$ to evaluate the \"logical transition\". This design seems to optimize for local logical coherence. However, a reasoning chain can fail because a step $S_i$ relies on a flawed premise established in a non-adjacent prior step (e.g., $S_{i-2}$), even if the local transition from $S_{i-1}$ to $S_i$ is perfectly logical.Does this input limitation mean ContextPRM is primarily constrained to evaluating local coherence? How does the model learn to distinguish a \"locally coherent but globally flawed\" step from a \"globally correct\" one, especially when its annotation strategy marks all steps after the first error as \"incorrect\"? Was a more detailed analysis conducted on the model's ability to trace errors back to their non-adjacent roots?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The paper's primary strength lies in its novel and intuitive reformulation of the PRM learning objective. Shifting the focus from \"is this step factually correct?\" to the logical coherence is a novel approach to decouple reasoning ability from domain-specific knowledge, directly targeting the identified problem of cross-domain generalization.\n\nThis conceptual contribution is backed by robust empirical results. The model achieves substantial gains over strong baselines, including the prior SOTA (VersaPRM), in a wide array of non-mathematical domains . The claims are well-supported by extensive experiments.\n\nFurthermore, the ablation studies are comprehensive. The finding that neither the context-based training nor the context-based labeling alone provides significant benefit, but their combination yields a dramatic performance increase which provides strong evidence for the authors' core claim of a necessary synergistic effect . The single-domain training analysis, which shows that training on \"logic-intensive\" domains yields better generalization than \"knowledge-intensive\" domains, is a compelling piece of evidence supporting the method's mechanism .", "weaknesses": "The ablation study reveals the full ContextPRM performs 2.2% worse than the baseline in math, suggesting that the method's trade-off for generalization is a notable loss in a domain that relies on strict, verifiable correctness rather than just contextual flow.\n\nThe method's practicality is dependent on a new, complex, and potentially costly data annotation pipeline . The entire approach relies on re-labeling the training data using an LLM verifier (gpt-4o-mini). This introduces a dependency on the quality of the verifier and reduces the method's simplicity.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)