Variation in Verification: Understanding Verification Dynamics in Large Language Models
Authors: Yefan Zhou
Austin Xu
Yilun Zhou
Janvijay Singh
Jiang Gui
Shafiq Joty
Venue: ICLR 2026 Poster (2026)
Keywords: generative verification
large language model
test-time scaling
Primary area: foundation or frontier models, including LLMs
TL;DR: We study the factors influence LLM-based generative verification, and apply findings to verifier-based test-time scaling.

Abstract: Recent advances have shown that scaling test-time computation enables large language models (LLMs) to solve increasingly complex problems across diverse domains. One effective paradigm for test-time scaling (TTS) involves LLM generators producing multiple solution candidates, with LLM verifiers assessing the correctness of these candidates without reference answers. In this paper, we study generative verifiers, which perform verification by generating chain-of-thought (CoT) reasoning followed by a binary verdict. We systematically analyze verification dynamics across three dimensions -- problem difficulty, generator capability, and verifier generation capability -- through empirical studies on 12 benchmarks across mathematical reasoning, knowledge, and natural language reasoning tasks using 14 open-source models (2B to 72B parameter range) and GPT-4o. Our experiments reveal three key findings about verification effectiveness: (1) Easy problems allow verifiers to more reliably certify correct responses; (2) Weak generators produce errors that are easier to detect than strong generators; (3) Verification ability is generally correlated with the verifier's own problem-solving capability, but this relationship varies with problem difficulty. These findings reveal opportunities for optimizing basic verification strategies in TTS applications. First, given the same verifier, some weak generators can nearly match stronger ones in post-verification TTS performance (e.g., the Gemma2-9B to Gemma2-27B performance gap shrinks by 75.7%). Second, we identify cases where strong verifiers offer limited advantages over weak ones, as both fail to provide meaningful verification gains, suggesting that verifier scaling alone cannot overcome fundamental verification challenges.

Published as a conference paper at ICLR 2026
VARIATION IN VERIFICATION: UNDERSTANDING VERI- FICATION DYNAMICS IN LARGE LANGUAGE MODELS
Yefan Zhou 1 Austin Xu' Yilun Zhou', ! Janvijay Singh ::3 Jiang Gui?, Shafiq Joty'
Salesforce AT Research Dartmouth College
University of Illinois Airaaa-Champpijn
ABSTRACT
Recent advances have shown that scaling test-time computation enables large language models (LLMs) to solve increasingly complex problems across diverse domains. One effective paradigm for test-time scaling (TTS) involves LLM gen- crators producing multiple solution candidates, with LLM verifiers assessing the correctness of these candidates without reference answers. In this paper, study generative verifiers, which perform verification by generating chain-of-thoure (CoT) reasoning followed by binary verdict. We systematically analyze verifi- cation dynamics across three dimensions problem difficulty, generator capability, and verifier generation capability through empirical studies on 12 benchmarks across mathematical reasoning, knowledge, and natural language reasoning tasks using 14 open-souree source models (2B 72B parameter range) and GPT-4o. Our exper- iments reveal three key findings about verification effectiveness: (1) Easy problems allow verifiers to more reliably certify correct responses: (2) Weak generators pro- duce errors that are easier to detect than strong generators; (3) Verification ability is generally correlated with the verifier's 's own problem-solving capability, but this relationship varies with problem difficulty. These findings reveal opportunities for optimizing basic verification strategies in TTS applications. First, given the same verifier, some weak generators can nearly match stronger ones in post- cost-verification TTS performance (c.g., the Gemma2-9B to Gemma2-221 performance gap shrinks by 75.7%). Second, identify cases where strong verifiers offer limited advan- tages over weak ones, as both fail to provide meaningful verification gains, suggest- ing that verifier scaling alone cannot overcome fundamental verification challenges.
Code
Project
INTRODUCTION
Large language models (LLMs) have advanced rapidly in solving reasoning tasks such as mathematies and code generation, yet their outputs remain unreliable, often containing subtle or obvious mistakes (Ke et al. 2025 Lightman et al. (002)). LLM based verification (Angelopoulos
al. 2025 Huang et al.) 2024 Mao et al. 2024 Pan et al. 2025 Pandit et al., 2025) has emerged as a central mechanism to identily such errors in scalable manner. Recent work has increasingly
focused on generative verifiers (Liu et al., 2025d| Mahan et al. 2024 Zhang et al. 2025), which frame verification as next-token prediction: the model typically generates a chain-of thought (CoT) reasoning trace and then outputs a binary verdict token. This approach has been shown to outperform earlier discriminative verifiers or scalar reward models (RMs. Lightman et al.) (023), as it better leverages the inherent cext-generation capabilities of LLMs. One valuable downstream application of automatic verification is test-time scaling (TTS), where additional inference-time compute is allocated to improve generation performance. A popular paradigm of TTS is the use of verifier model to evaluate candidate responses, filter errors, and identify correct solutions. This approach underlies techniques such as rejection sampling (Brown et al. 2024). re-ranking (Zhou et al. 2025 weighted majority voting (Wang et al. 2024a 2023), and level generation (Snell et al. 2025).
Current practice in LLM verification often deploys strong, typically closed-source frontier models as verifiers. This practice rests the assumption that verification quality scales with verifier's capa- bility to solve the same problem (i.e., its generation capability), a correlation demonstrated in recent
Work done during an internship at Salesforce AI Research Work done Salesforce AT Research

---

Published as a conference paper at ICLR 2026
Generating responses
CoT Verdict
Rasporae
Incenee Resporse
True Postlive (T9)
Negatiin (Td)
Hard
Robot genresion capubd by
Prablems w/ diverse difficuity
Gonerators various
Verifiers w/ variouss ability
Verification Evaluatton
RQ: How is verification impacted by these factors?
Verificttion
Ver Rate
Veriication Accuracy
101
Ours stratfflytttttfil
Ed
Problem D Difficutyy
Generator Capablity
Verifier Generation Capability
(b)
Figure I: Overview of our study on verification dynamics. (a) We consider generative verification: an LLM generator produces a solution problem, and an LLM verifier conditions on the problem and solution to generate verification CoT followed by binary verdict ["Correat""IIncoree" We design controlled experiments that vary problem difficulty, generator generation capability, and verifier generation capability, investigating how each of these factors influences verification performance. (b) Our analysis reveals three patterns: problem difficulty governs recognition of correct responses (true positives); generator generation capability determines error detectability (true negatives); and verifier generation capability correlates with performance in a difficulty difiiullty-dpeedeent manner, revealing non-linear regimes left uncovered in prior work. The three plots were generated by aggregating benchmark data across three domains and averaging performance metrics over 15 models.
work (Chen et al. 2025c Krumdick et al. 2025 Tan et al. 2025). However, this practice may be sub- optimal given that verifying a solution is often easier than generating one from scratch, a phenomenon referred to as "verification asymmetry" (Wei) 2025). This asymmetry appears in several fields. In convex optimization, dual certificates enable efficient validation of optimality of a proposed solution, while in factorization, verifying correctness is trivial compared to finding the prime factors. Thus, it is worth investigating verification as a distinct capability rather than merely a byproduct generation.
Despite extensive research generation dynamics and the factors influencing generation qual- ity (Allen-Zhu Li.
2025 Chen et al. 2024 Ye et al. 2025). the dynamics of verification remain largely unexplored. In particular, little is known about how problem characteristics, properties of generated responses, and model capabilities interact to determine verification effectiveness. Without understanding verification dynamics, one risk misallocating computational resources by default- ing to expensive frontier models when simpler alternatives might suffice. This gap in understanding motivates our central research question: what factors influence verification success?
In this paper, we present a systematic study of generative verification across three dimensions-probbee difficulty, generator capability, and verifier generation capability-shown in Figure | We quantify verification performance by measuring the probability of the verifier recognizing both correct and in- correct generated solutions in controlled experimental settings. We focus on verifiable problems with objective ground-truth -ruth answers in mathematical reasoming, knowledge question- question-answering (QA), and natural language (NL.) reasoning domains. This allows us to objectively measure verifier and gener- ator performance, while simulating the reference-free evaluation settings where verifiers are typically deployed in practice, e.g.. in TTS. While our experiments use these domains as a testbed, we believe the insights should extend to any domain where correctness can be reliably defined and checked.
Main Findings. While prior work showed that verifier generation capability correlates with verifica- tion performance (Chen et al. 2025c; Krumdick et al. 2025 Tan et al. 2025). we reveal that two additional factors, problem difficulty and generator capability, also critically influence verification success, as illustrated in Figure [ Our analysis reveals:
Problem difficulty primarily governs the recognition of correct solutions: verifiers are more likely recognize correct solutions easy problems than on difficult ones.
2

---

Published as a conference paper at ICLR 2026
Generator capability influences error detection: errors made by weak generators easier to detect than those made by strong generators.
Verifier generation capability correlates with verification performance in manner dependent on problem difficulty: saturated (or uncorrelated) for easy problems, linear for medium problems, and threshold-limited for hard problems.
Our empirical analysis includes 2,347 math problems from eight datasets, 1,196 knowledge QA problems, and 901 NL reasoning problems. evaluated across 14 open-source models and GPT-4o.
Application to TTS. We demonstrate the practical implications of our findings for TTS. First, given the same verifier, the TTS performance of weak generator can nearly match the performance of a strong generator. For instance, Gemma2-98 achieves comparable performance to Gemma2-27B when both use the same verifier, GPT-4o. Second, we identify regimes where a strong verifier (e.g.. GPT-40) offers no additional benefit and can be replaced by weaker verifier (e.g.. Qwen2.5-7B), with both providing limited gains. This occurs with strong generators or with problems at either extreme of the difficulty spectrum.
RELATED WORK
Automatic Evaluation. The deployment of LLMs as evaluators has emerged as a central mechanism
for scalable assessment, with efforts focusing training specialized small evaluators through fine-
tuning (Singh et al., 2026 Wang et al. 2025 Whitehouse et al. 2025| Xu et al. 2025 2026 Zhang
et al. 2025). Beyond ffeerence-based verifiers
(Chen al. [025a Liu et (002b)) verification
approaches include self-verification (Chen et al.
2023 Huang et al. 2023) 2024 Kumar et al. 2024
Shinn al. 2023), , where models reflect on or critique their own outputs, and multi-agent verifi-
cation (Li et al. [2023: Lifshitz et al. 2025 Zhuge et al. 2024). where multiple agents collaborate in debate-style or hierarchical setups to improve reliability. Prior work identifies several factors influencing evaluation performance, with evaluator generation capability being particularly important. Krumdick et al. (2025) find that evaluator performance changes significantly based on whether the evaluator is capable of answering the question not. Tan et al. (2025) demonstrate the correlation between pairwise judging ability and generation ability on the same set of problems. Chen et al.) (2025b) observe linear relationships between evaluation improvements and peasoning-requieee sample proportions in fine-tuned evaluators. [Chen et al. (2025c) show a strong positive correlation between generation capability and evaluation accuracy. Our work extends these findings by identifying unexplored factors and showing that the relationship between evaluator generation capability and evaluation quality is more nuanced than previously understood. [u al. (0225) concurrent work with similar research objective, finding that cross-family verification is particularly effective, com- plementing our analysis of how problem difficulty and model capability shape verification dynamics.
Verification for Test-Time Scaling, Early studies explore how to effectively apply verification meth- ods to improve TTS performance. (nell al. (2025) show RMs improve various TTS approaches, including Best-of-N and beam search, while Liu et al. (2025a) find that compute-optimal strategies vary with policy models and problem difficulty. Recent work explores alternatives to discriminative RMs: ZZhag et al. (2025) show trained generative verifiers outperform RMs in Best-of-N, and Zhou al. | (2025)'s 's JETTS benchmark demonstrates generative evaluators match outcome RMs in reranking. While verification benefits from increased model size and test-time compute, recent work addresses how to reduce these computational costs. Saad-FFlcon et al. (2025) propose a framework aggregate weak verifiers to approach strong ones; Angellppulss al. (2025) balance weak/strong evaluators for efficiency; [Stroebl et al. (2024) analyze fundamental limits of resampling with imperfect verifiers; and (Singhi et al. (2025) propose strategies to balance solving- verification trade-offs. Our work studies the factors driving verification and explores their implications for TTS.
solving-verification
3
EXPERIMENTAL SETUP
3.1 PRELIMINARIES
Problem and Response Space. Let x denote a problem with ground-truth answer y (z). A model response r to r consists of a CoT solution and final answer a(r), and we consider the response correct if a(r) = y (z). As discussed in Section I our study uses verifiable problems with objective answers, allowing us to rigorously evaluate verifier outputs against ground-truth while simulating reference- free evaluation settings.
Generator and Verifier. A generator G maps a problem x to a distribution over responses, denoted ~ G(. A verifier V takes a problem-response pair (zr, r) and outputs judgment of correctness.
3

---

Published as a conference paper at ICLR 2026
In the binary case, Viz, r) E 1). where indicates acceptance and 0 indicates rejection. More generally, a generative verifier produces a verification CoT explaining its reasoning, followed by an explicit verdict such "Correct" or "Incoreect. The prompt templates are provided in Appendix
Generation Capability. We measure the generation capability of a model using its pass rate. For generator G and problem 2, define PG (c( Pr(a(r) y (x)
G(-[x)) as the pass rate on a single problem, i.e., the probability that G solves I correctly on one sampled attempt. We define (c( [o] EEED (G) as the pass rate aggregated over dataset D, which we use as the overall measure of a model's generation capability. Since the verifiers we study are generic L.MS (e.g., GPT- 4o), we measure verifier's generation capability using the same metric by prompting it as generator.
We estimate these pass rates empirically as (a; K) and (D; K) by sampling responses per model-problem pair. Since K is fixed at 64 throughout, we often omit K for simplicity, writing (x(()) and pa((D) We use temperature 0.7 and top-p 1.0 as default sampling hyperparameters, and adopt recommended settings when available (e.g., temperature 0.7 and top-p 0.8 for the non- non-thinkinn mode of Qwen3). Ground-truth correctness is established with Mat Ey (Kydliček) (0055), supplemented by LLM-as-a-jude- grading to reduce false negatives (details in Appendix BBI1. These 64 responses per problem-model pair are used to estimate generation capability and problem difficulty,
Problem Difficulty. We define the difficulty of problem as the average pass rate across set of diverse generators g, d(x²) = [g] EGeg (x)).. This score reflects how broadly solvable a problem is: if most generators succeed, d(x)) is high (casy problem), while if few succeed, d(x)) is low (hard problem). It provides a model-agnostic way to partition problems by difficulty, extending prior work (Snell et al. 2025), which measured difficulty relative to a single generator.
Verification Metrics and Evaluation. We evaluate verifiers using true positive rate (TPR), the probability of the verifier accepting a correct response: TPR E[V rz) a(r) = y (=)], and true negative rate (TNR), the probability of rejecting an incorrect response: TNR = E - Viz, r) a(r)= o y² (=)] We also report balanced accuracy, Accaal (TPR + TNR). which accounts for class imbalance. For verification evaluation, we subsample 8 responses from each 64-sample pool, balanced with correct and incorrect when possible. For very hard problems with fewer than correct responses, we keep all correct ones and sample incorrect ones to reach 8 total (and vice versa for easy problems). Each verifier evaluates responses from all 15 models over the full test set using greedy decoding, unless a controlled subset is specified.
Perifiaation-Augmenter Test-time Scaling. We consider the TTS setting of sampling multiple responses from the generator and filtering with verifier before evaluation. For each problem D, we sample K responses from the generator using a fixed temperature, with K = 64 in our experiments. Without verification, TTS performance is measured as (D; K') (or (D)))), the empirical pass rate defined above. With verification, the verifier V evaluates each candidate, and only responses deemed "Correct" are retained for evaluation. The performance of verification-augmennee TTS is measured as
(D,
Σ (a(r)) y² (x)) (z,
(1)
D
RED K'
where K" K V(x, r;). This metric represents the conditional pass rate, i.e., the fraction of correct responses among those retained by the verifier. A corner case arises when the verifier rejects all responses (K' = 0); in this case, we set the metric to the generator's pass rate Pa (D), so evaluation reverts to selecting from the original K responses in the non-verified setting. We define the verification gain from verifier V as the difference relative to the performance without verification, (D). which quantifies how much gain can be attributed verification. Note that our formulation of TTS differs from the common setting where a single "best" response (e.g.. by majority vote) is selected and then evaluated. Instead, we report the empirical pass rate of the verifier- pool, which can be interpreted as the expected accuracy of uniformly sampling one response from that pool. This expectation-biat based view captures the average quality of verifier-retained responses without tying performance to a specific selection strategy.
Apv = PG, v (D))
If the verifier generates an invalid output (e.g., g.. due to the CoT running out of max generation length), we treat it as an uninformative verdict of "Correct" and "Incorrect" each with probability of 50%. Computationally, we set (z,r)) 0.5 in this case, and also in Equation
4

---

Published as a conference paper at ICLR 2026
3.2 TASKS AND MODELS
Mathematical Reasoning. We collect a total of 2,347 problems from the test sets of eight mathe- matical reasoning benchmarks: GSM8K (Cobbe et al., (02T), MATH500 (Hendrycks et al. (02T).
OlympiadBench (He et al. ).24), AIME24/25 (Li et al. 2004)) AMC23 (Li et al. 2024), Minerva- Math (Lewkowycz et al. 2022). and BBEH Multi-step Arithmetic (Kazemi al. 2025). We use the entire test sets of these benchmarks, except for GSM8K, from which we subsample 600 of 1,319 problems to balance difficulty distribution and reduce the proportion of easy problems.
Knowledge. We use a subset of MMLU-Pro (aang et al., 2024b) as our knowledge category. We randomly subsample 10% from each of its 14 disciplines, yielding 1,196 problems. MMLU-Pro consists of college-level multiple-choice questions spanning STEM, humanities, and social sciences.
Natural Language Reasoning. We collect 901 multiple-choice problems from three benchmarks. (1) ReClor (validation set. Yu et al. 2020), multiple-choice benchmark requiring logical analysis of short passages. (2) FOLIO (Han et al. 2022), first-order logic reasoning benchmark in natural language. (3) GPQA Diamond (Rein et al., (2024). a dataset that consists of graduate-level multiple- choice science questions, requiring multi-step reasoning.
Models. We use 14 open-source models from four families: (1) Qwen2.5 at 3B, 7B, and 72B (Team
2024); Qwen3 4B, 8B, and 32B (Yang et al. 2025); (2) Llama-3.2 at 3B, Llama-3.1 at 8B, and
3lama-333 at 70B (Grattafiori et al. 2024); (3) Gemma-2 at 2B, 9B, and 27B (Team et al.
2024); (4) and one closed-source model GPT-4o (Hurst et al. 2024). All
Ministral 8B and Sistral-Smalll
models are instruction-turee versions by default. Each model is used both as a generator and a verifier. We use abbreviated model names in figures for space efficiency; see Appendix [.2 for mappings.
4 EXPERIMENTAL RESULTS
Our experiments focus on how problem difficulty and generator and verifier generation capability influence verification performance. We present the three research questions and main findings below.
How does problem difficulty affect verification? (Section 4.1 TPR increases steadily with decreasing problem difficulty, meaning verifiers better recognize correct responses easier problems. However, TNR shows no predictable relationship with problem difficulty. This indicates that problem difficulty primarily influences correctness recognition.
RQI:
RQ2: How does the generator's generation capability influence verification? (Section 4.2 As generators become stronger, TNR decreases substantially while TPR increases only slightly. This reveals that generator capability primarily determines error detectability: stronger generators produce errors that are harder for verifiers to identify.
RQ3: How does verifier generation capability impact verification? (Section 4.3) Verifier
generation capability and verification performance are generally positively correlated. However, the form of correlation depends heavily problem difficulty: linear correlation occurs in medium- difficulty problems, while nonlinear patterns appear in other difficulty levels.
Throughout this section, use the oracle problem difficulty measure defined in Section 3.1 which relies on ground labels, to avoid measurement error. In Appendix [D.]] we validate all key findings using a practical, label-free difficulty estimator and show that the trends remain consistent.
4.1 How DOES PROBLEM DIFFICULTY AFFECT VERIFICATION?
To examine how problem difficulty influences verification, we partition problems into four equal-sized quartiles by their difficulty score d(z), termed "hardest", "hard", "eaay",, and "easiest".
Problem difficulty primarily influences the verifier's ability to recognize correct responses. Our analysis is conducted at two levels of granularity: response level and problem level. Both analyses reveal that problem difficulty mainly shapes the verifier's sensitivity to correct responses, while not consistently affecting its ability to identify incorrect responses.
At the response level, we compute the TPR and TNR of all responses within each difficulty quartile. As shown in Figure 2 TPR increases steadily as problems become easier, while TNR shows no clear trend. This pattern is consistent across model families and domains. At the problem level, we pool responses from all generators for each problem and compute a single TPR. and TNR per problem. The distribution of these metrics within each quartile is reported in Figures 9 and I0 of Appendix [C.I. We observe that easier problems yield higher and more stable TPR, while harder problems exhibit lower and more variable TPR. In contrast, TNR distributions show no consistent
5

---

Published as a conference paper at ICLR 2026
Qwen2.5
Qwens
Limaa GPT-4o
Small (2-48)
(70B4)
Gemma2
Mistral
Medium (7-32B)
1.0
1.0
Theo Yornive
0.5
0.5
0.0
0.0
0.0
Handest
Hard
Easy
Emiest
Handest
Hand
Iasy
basket
Hardert
Head
Eiey
bastest
Problam Difficulty
Problem Dificulty
Profiens Diffsully
(a) TPR (Mathematics)
(b) TPR (Knowledge)
(c) TPR (NL Reasoning)
1.04
1.04
1.0
1its Negitre Kare
0.0
0.0
0.0
Handet
Bardd
Eliiest
Hardew
Hadest
Hladd
Easies
Pubblam Defiiclty
Prbblema Defficatyy
Problems Diffcolty
(d) TNR (Mathematics)
(e) TNR (Knowledge)
(1) TNN (NL Reasoning)
Figure 2: Problem difficulty primarily affects TPR of verification. Each curve shows verifier performance across four difficulty groups, with the r-axis indicating problem difficulty and the y-axis reporting TPR (a-c) and TNR (d-f). Colors denote model families, and line styles indicate model size.
correlation with problem difficulty, Appendix C.2 we show that the main verification dynamics about TPR identified generalize to reasoning models, while extended reasoning provides benefits and alters TNR behavior. We additionally confirm that this trend persist for a large open-source model (Qwen3-235B) in Appendix [D.3] and across different verification prompts in Appendix D.4
To understand this pattern, case studies in Figure 32 show that verifiers tend to generate their own reference solutions for comparison during verification. As the problem difficulty increases, these verifier-generated answers become increasingly incorrect, producing false negatives (FNs) that reduce TPR. We support this explanation with a large-scale analysis. We use LLL--as-uudge to detect whether verification CoT contains solving mistakes, with experimental details provided in Appendix B.3 As we show Figure 3 as problem difficulty increases, the verifier increasingly makes mistakes generating the reference answer, with 39.1% of verification FNs containing reference answer mistake in the hard set of problems. This shows that faulty reference generation is large driving factor of FNs.
50
Palse Negatives (PN)
(% of corruct samples)
FN with Solving Mistakes (% of correct somples)
Purcentage
22.1%
20
18.1%
8.7%
2,0%
8.4% 0.2%
Easy
Problem Difficuity
Figure Interpreting RQ1 finding by analyzing solving mistakes in false-negative verifications. Blue: percentage of FNs among sampled correct responses. Orange: percentage of those FNs whose verification CoTs contain solving mistakes. Error bars: 95% Cls.
4.2 HOw DOES GENERATOR CAPABILITY INFLUENCE VERIFICATION?
We study how generator capability affects verifier performance by having each verifier evaluate responses from each generator. Generators of different capabilities may produce extreme response distributions, e.g., weak generators may produce no correct response hard problems within 64 samples. To ensure fair comparison, we compute TPR on problem subsets where all generators produce at least one correct response. Analogously, TNR is computed problems where all generators produce at least one incorrect response. Details are provided Appendix B.4
As shown in Figures 4a 4c TPR remains uniformly high across nearly all settings and increases further with stronger generators. The heatmap is dominated by red colors, with values mostly above 0.7, indicating that most verifiers are already reliable recognizing correct responses. As generator capability improves, TPR approaches 1.0. This suggests that generator strength influences recognition of correct responses in relatively mild way.
Generator capability correlates with error detection in verification. Figures 4d to moving from weaker generators the left stronger ones the right, the heatmap shifts generally from

---

Published as a conference paper at ICLR 2026
0.1
0.2
0,3
0.4
0.5
0.5
0,7
9.8
0.9
1.0
(a) TPR (Mathematics)
bb) TPR (Knowledge)
TPR (NL Reasoning)
134
038
011
(d) TNR (Mathematies)
(c) TNR (Knowledge)
(1) TNR (NL Reasoning)
Figure 4: Generator capability influences verifier performance of identifying incorrect responses. Heatmaps show (a-c) TPR and (d-f) TNR when pairing 15 verifier models (rows) with 15 generator models (columns). Rows and columns are ordered by models' generation capability computed on all problems of each domain. Values indicate mean performance over the evaluation subset.
red to blue, indicating a substantial decrease in TNR. For example, in the Mathematics domain, for the Qwen2.5-72B verifier, TNR drops from 0.68 on solutions generated by Llama-3. 3.1-BB to 0.17 on those by Qwen3-32B. The overall pattern is consistent across three domains and nearly all verifiers. These results show that generator capability strongly modulates the detection of incorrect responses. In Appendix C.2 we show that this finding generalizes to reasoning models. We further verify that this trend remains robust when scaling a large open-source model (Qwen3-235B; Appendix D.3) and when varying the verification prompt (Appendix D.4.
We interpret this phenomenon through case studies in Figure Strong generators produce internally consistent reasoning chains where early mistakes (e.g., missed cases) propagate coherently, yielding well-structured but incorrect solutions that cause the verifier false positives. Weak generators produce surface-level errors such as self-contradictions, facilitating verifier rejection. We support this explanation with large-scale analysis. We use an independent L.LM-as au--uee determine whether generator's 's CoT contains surface-level errors, with experimental details provided in Appendix [B.3 The results Figure 5 show clear trend: as generator capability increases, the frequency of surface-level errors decreases. This is consistent with Yin et al. (2025). who find that error patterns of stronger models shift from basic computational mistakes to more sophisticated reasoning failures.
4.3 How DOES VERIFIER GENERATION CAPABILITY IMPACT VERIFICATION?
We measure verifier generation capability and evaluate verification performance using balanced accuracy (Accиa) on the entire test set. Each verifier evaluated on responses from all generators, and we report results both averaged across all problems and stratified by problem difficulty. To characterize the relationship between generation capability and verification performance, we employ locally weighted regression (Cleveland 1979) with bandwidth of 0.6 fit nonparametric curves. We compare R² values between nonparametric and linear fits to assess linearity. We also report the Pearson correlation coefficient (Benesty et al. 2009) another measure of linear correlation.
Figures ba to 6d show a strong overall correlation between verifier generation capability and verifica- tion accuracy, with NI. reasoning showing less linearity than other domains. This result is consistent
7

---

Published as a conference paper at ICLR 2026
Suffwee Level Ermo Rdc
Surface-Loo Error Ratte
-07
0.3
0.
82
21.0
Goop
0229
014-18
Mrvig
L1-78
Q2-28
Model
Geteestorr Capubility
(a) Surface-level Error Rate vs. Name
(b) Surface-level Error Rate vs. Capability
Figure 5: Interpreting RQ2 finding by analyzing surface-level errors in generators' solutions. (a) The z-axis lists generator models sorted by generation capability; the y-axis shows the percentage of responses containing surface-level errors, (b) The æ-axis shows the capability values of the same generators; the y-axis is the same in (a). Error bars indicate 95% CIs.
Qwen2.5
Llama3
Mistral
2-4B
24-32B
Qwen3
Gemma2
GPT
7-9B
70B+
0.0
Linerr R0-4 Norperm Ri-O 919
Liber 80-0.26, R7-0 550
Liver R1-0,500 Noopann 81-0
Taleree Acoracy
8.7-
0.7
Untercal
Varlic' Getentie Cauatity
Vadfler' Geerrtee Coueviliy
Verifee's Gmerrno Copoliiity
(a) All data (Mathematics)
(b) All data (Knowledge)
(c) All data (NL Reasoning)
0.-
6--
Cimur Macuana
Midaar Narpaaa
Limas Re-L. Nequuaniieaas Suy: Kivi Norpacu 20
Rohooe Axcuney
Earc Libar AeosAd Nirpatan 1e
Bebrca Aereeey
Zup: Libar Smouoo Narpera to
65-
2.6
6.
0U4
0.6
0,2
0.6
1.0
Varifee''s Genauise Capubiiity
Verider's Genemtiee Capebilky
Vantder's Gesemine Capabilky
(d) Stratified view (Mathematics)
(e) Stratified view (Knowledge) (1) Stratified view (NL Reasoning)
Figure 6: Distinct correlation forms between verification performance and generation capability. Solid lines represent nonparametric fits to the data; indicates the Pearson correlation coefficient. (a- c) Averaged across all problems, verifier generation capability exhibits strong linear correlation with balanced accuracy. (d-f) When stratified by problem difficulty, distinct correlation patterns emerge.
with prior work showing that evaluator accuracy tends to track the evaluator's 's task performance, with the relationship appearing nearly linear. While this global trend validates findings in prior work (Chen jet al. 2025c Tan et al. 2025), a closer inspection of the trend reveals highly non-linear regimes.
Verifier generation capability influences verification accuracy differently based on problem difficulty. Stratified analysis reveals regime-dependent correlation with phase-transition behavior. We partition problems into 10 equhl-width bins by difficulty d(x:) and analyze three representative intervals: hard [0.1, 0.3), medium [0.4, 0.5), and easy [0.8, 0.9) in Figures 6el 6f and 23a For hard problems (blue), verification accuracy shows minimal improvement with increasing capability. Mathematics plateaus around 0.65 accuracy after initial gains, while other domains remain flat throughout. Notably, verifiers achieve helow-random accuracy on hard NI Reasoning problems, which we analyze in Appendix C33 Medium problems (yellow) exhibit steady accuracy increases with capability, indicating strong linear relationships. This is confirmed by linear and nomparametric fits that yield nearly identical Rt2 values, with > 0.9. Easy problems exhibit a threshold effect at the r-axis around 0.9: below this threshold, the relationship is linear; above it, small capability improvements yield large verification gains. Hard and easy regimes show nonlinearity with nonparametric R² exceeding linear R² by 0.1-0.2 and r < 0.85. The exception is NL Reasoning on hard problems, where both fits yield near-zero R², indicating meaningful aapabilit--acuural
8

---

Published as a conference paper at ICLR 2026
relationship. Appendix C.4 provides additional results (Figure 14.. including analysis of why different patterns are observed in varying difficulty intervals.
APPLICATION TO TEST-TIME SCALING (TTS)
Our analysis in Section is conducted with verification itself as the end goal. However, our findings have direct implications for TTS. We analyze two research questions in TTS settings that naturally arise out of our previous findings, and present our results below:
RQ4: Given fixed verifier, can a weak generator match a stronger generator in TTS? (Sec- tion 5.1) Weak generators can nearly match stronger generators' post- cest-verification performance. Verification gains peak at meek-mm generators by achieving high error detection rate (TNR) while maintaining a moderately high correctness recognition rate (TPR).
RQ5: Can weak verifiers match the gains strong verifiers in TTS? (Section 5.2) The verification gain gap between weak and strong verifiers narrows at both low and high problem difficulty extremes, and when using strong generators.
The following sections present results the Mathematics domain, with complete results across all three domains in Appendices C.3 and C.6 In Appendix D.2 we show that these findings can guide generator and verifier selection TTS and lead to efficiency improvements.
5.1 CAN WEAK GENERATORS MATCH STRONGER GENERATORS IN TTS?
We evaluate TTS with a fixed verifier (GPT-40) by varying generator capability and reporting pass rates before and after verification, along with the verification gain Apv.
Before Verification
After Verification
TNR
TPR
Verification Gain
1.0
0.
1.0
Pass Rate
0.8
Gamia aprecachos
Verification Gain
0.3
0.5
0,5
TVR TNR
0.6
Gama22278 garf.
02
0.4
Liuin3:2 38 appeeches
Liana3, 1-88 in TTS pent.
0.2
Generator Models
Generator Models
(a) Pass rate
(b) Verification gain and TPR/INR
Figure 7: TTS performance before and after verification when varying generator strength. Results are reported problems with difficulty in the range [0.7, 0.8) from the Mathematics domain, including 181 problems.
(a) Pass rate before (blue) and after (orange) adding a fixed verifier (GPT-40). The generators in the z-axis are ordered from weaker (left) to stronger (right) by generation capability measured on the problem subset. (b) Bar chart shows the verification gain Ajv (left y-axis) for each generator. Lines show the verifier's TNR and TPR (right y-axis).
Verification gain peaks for weak-medium generators, enabling them to approach stronger models verrfiiifiia,inn. As shown in Figure 7a weak generators start with much lower pass rates but improve dramatically after verification, reaching levels comparable to larger models. For example, Gemma2-9B starts from significantly lower baseline but, after verification, achieves pass rate nearly matching Gemma2-27B. The performance gap shrinks from 10.3% to 2.5%, closing 75.7% of the original difference. Figure ebeexplins this phenomenon: as generator strength increases (left to right), TNR decreases sharply while TPR rises only modestly, consistent with RQ2 findings. Consequently, verification gain (gray bars) peaks weak-medium generators, which achieve high TNR for effective eтror filtering while maintaining moderate TPR to preserve correct responses. For the strongest generators, errors become harder identify, causing TNR decline and limiting gains. In Appendix C.5) we show the findings derived from the Mathematics domain generalize well to two other domains in Figure T5] We provide additional evidence confirming the generalizability: first, verification gains peak for weak-medium generators across a broad range of problem difficulties (d(z) 2 0.3) in all domains (Figures to I8]; second, performance gaps of most weak and strong model pairs can be reduced by verification when evaluated on the entire domain datasets (Figure 19) mostly achieving 30-50% reduction. These results suggest that weak generators, when paired with strong verifier, can serve as cost-effective alternative to strong generators. One might question, however, whether such a pairing is truly efficient given the cost of a strong verifier (GPT-40). In Appendix [D.5] we address this with case study showing that pairing a weak generator with a strong verifier can match the performance of strong generator alone while reducing token consumption.

---

Published as a conference paper at ICLR 2026
Generatons
Verifers
Weak (L.1m33.2235 233
Strong (Qwm22.5728))
Strong (GPTT40)
Week (Qwmn2.5-78)
Medium (Lbema3)
0.25
1.0
Veiifeatian Crain Cap
0.20
True Negative Rate
9.8
0.15
0.8
Pesitie
9.6
0.10
0.6
0.4
0.00
0.4
-0.05
pilorg
aen
past pa en
ean
pian
p
pien
boie Baan
Problem Difficulty
Problem Difficulty
Amolem
Difficuly
(a) Verification Gain Gap
(b) TPR
(c) TNR
Figure 8: Weak vs. strong verifiers under varying problem difficulty and generator strength.
The z-axis shows problem difficulty, ordered from hardest to casiest, measured relative to each generator. (a) Verification gain gap between strong and weak verifiers when applied weak, medium, and strong generators. (b) TPR increases as problems become easier; shadow band indicates TPR gap between two verifiers. (c) As generators strengthen, TNR decreases overall and the TNR gap (shadow band) narrows. Results are from the Mathematics domain.
5.2 CAN WEAK VERIFIERS MATCH THE GAINS OF STRONG VERIFIERS IN TTS?
We analyze the verification gain gap between a strong verifier (GPT-4o) and weaker one (Qwen2.5-
7B) across problem difficulty ranges and generator strengths. The verification gain Apv is defined in Section 3 and the gap between verifier ApV - Apv... Our goal is to identify when this gap narrows, such regimes suggest weak verifiers can substitute for strong ones.
The gap narrows on the extremes of problem difficulty As shown in Figure 8a the verification gain gap shrinks as problems become easier, which corresponds to the rising TPR for both weak and strong verifiers seen in Figure 8b) This aligns with our RQI findings that easier problems improve TPR for all verifiers. Even weak verifiers reliably recognize correct responses easy problems, leaving little room for strong ones to provide additional benefit. At the opposite extreme, the gap also narrows on the hardest problems. As discussed in RQ3 and shown in Figures 21d to 21 increasing verifier generation capability (or scaling up to larger models) fails to improve verification accuracy on hard problems, resulting in only marginal performance differences between weak and strong verifiers.
The gap narrows as generators become stronger. Figure shows that increasing generator capability reduces the difference between weak and strong verifiers. This is consistent with RQ2 (Section 4.2), where we observed that the verifier's TNR decreases as the generator capability increases. As both weak and strong verifiers experience lower TNR, the gap between them also shrinks, shown as the narrowing shaded band between solid and dashed curves in Figure (c).
The results here are obtained from the Mathematics domain, and we show the findings generalize to two other domains in Appendix C.6. In regimes of very easy/hard problems or when evaluating strong generators' responses, weak verifiers provide gains to TTS performance comparable to strong verifiers. However, these convergence regimes coincide with minimal verification benefit overall. Figures 21a to 21c shows verification gains drop to 0. or below for both verifiers on easy and hard problems, verification on strong generators yields peak gains of only 0.1, precisely where the gap narrows. Thus, while weak and strong verifiers converge in these regimes, this convergence occurs where both provide minimal practical value. This reveals that scaling verifiers from 7B models to GPT-4o fails to overcome fundamental verification challenges, with GPT-4o providing limited improvement over small open-source models in the identified regimes.
6 CONCLUSION
We study LLM verification across problem difficulty, generator capability. and verifier generation capability, revealing that verification success depends on their interactions. We find that problem diffi- culty primarily shapes correct solution recognition, generator capability influences error detectability, and verifier generation capability correlates with verification in problem deepeddtty patterns. We examine the implications of these findings for verification deployment in TTS, identifying both opportunities and limitations. Stronger generators may not be necessary, as weaker generators can approach the post-verification performance of stronger ones when paired with a fixed verifier. This suggests potential for strategic model pairing that could reduce computational costs in verifier-based TTS methods. Our results also identify regimes where investing in larger verifiers yields no benefit, such when evaluating responses from strong generators or problems difficulty extremes.
10

---

Published as a conference paper at ICLR 2026
ETHICS STATEMENT
This work exclusively evaluates large language models on publicly available academic benchmarks containing mathematical reasoning, knowledge, and natural language problems with objective ground- truth answers. All experiments involve automated evaluation of model outputs without human subject participation. The datasets used are established research benchmarks designed for educational problem-solving tasks. Our study aims to understand verification dynamics to improve the compu- tational efficiency of LLM systems, posing no ethical concerns regarding privacy, harmful content generation, or potential misuse.
REPRODUCIBILIIY STATEMENT
We provide comprehensive details to ensure reproducibility of our findings. All experiments use publicly available datasets and open-source/commerciaa I.LMs. We specify the model names, versions, dataset sources, and inference hyperparameters in Section 3.2 Complete prompt templates for both generation and verification tasks are provided in Appendix A The mathematical formulations of all metrics, along with estimation procedures and aggregation methods, are formally defined and clearly described in Section 3.1 and Appendix B
REFERENCES
Zeyuan Allen-Zhu and Yuanzhi Li. Physics of language models: Part 3.2, knowledge manipulation. In The Thirteenth International Conference Learning Representations, 2025.
Anastasios N Angelopoulos, Jacob Eisenstein, Jonathan Berant, Alekh Agarwal, and Adam Fisch. Cost-optimal active ai model evaluation. arXiv preprint arXiv:2506.07949, 2025.
Jacob Benesty, Jingdong Chen, Yiteng Huang, and Israel Cohen. Pearson correlation coefficient. In Noise reduction in speech processing, pp. 1-4. Springer, Berlin, Heidelberg, 2009.
Bradley Brown, Jordan Juravsky, Ryan Ehrlich, Ronald Clark, Quoc V Le, Christopher Ré, and Azalia Mirhoseini. Large language monkeys: Scaling inference compute with repeated sampling. arXiv preprini arXiv:2407.21787. 2024.
Ding Chen, Qingehen Yu, Pengyuan Wang, Wentao Zhang, Bo Tang, Feiyu Xiong, Xinchi Li, Minchuan Yang, and Zhiyu Li. xverify: Efficient answer verifier for reasoning model evaluations. arXiv preprint (tti:25044.088) 2025a.
Nuo Chen, Zhiyuan Hu, Qingyun Zou, Jiaying Wu, Qian Wang, Bryan Hooi, and Bingsheng He. Judgelrm: Large reasoning models as judge. arXiv preprint arXiv:2504.0000 2025b.
Qiguang Chen, Libo Qin, Jiaqi WANG, Jingxuan Zhou, and Wanxiang Che. Unlocking the capabilities of thought: A reasoning boundary framework to quantify and optimize chain-of-thoughh In The TThirty-eight Annual Conference Neural Information Processing Systems, 2024.
Wei-Lin Chen, Zhepei Wei, Xinyu Zhu, Shi Feng. and Yu Meng. Do Ilm evaluators prefer themselves for a reason? arXiv preprint wwrXiv:2504008844 2025c.
Xinyun Chen, Maxwell Lin, Nathanael Schärli, and Denny Zhou. Teaching large language models to self-debug, arXiv preprint 2304::200000128 2023.
Xiusi Chen, Gaotang Li, Ziqi Wang, Bowen Jin, Cheng Qian, Yu Wang, Hongru Wang, Yu Zhang, Denghui Zhang, Tong Zhang, et al. Rm-rl: Reward modeling as reasoning. arXiv preprint arXiv:2505.02387, 2025d.
W.S. Cleveland. Robust locally weighted regression and smoothing scatterplots. Journal of the American Statistical Association, 74(368):829 836, 1979. doi: 10. 10.1080/01621459, 191..9.9.9991.1111133.
Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Rejichiro Nakano, Christopher Hesse, and John Schulman. Training verifiers to solve math word problems. arXiv preprint 2rX1v:2110.14163 2021.
11

---

Published as a conference paper at ICLR 2026
Leo Gao, Jonathan Tow, Baber Abbasi, Stella Biderman, Sid Black, Anthony DiPofi, Charles Foster, Laurence Golding, Jeffrey Hsu, Alain Le Noac' h, Haonan Li, Kyle McDonell, Niklas Muennighoff, Chris Ociepa, Jason Phang, Laria Reynolds, Hailey Schoelkopf, Aviya Skowron, Lintang Sutawika, Eric Tang, Anish Thite, Ben Wang, Kevin Wang, and Andy Zou. The language model evaluation harness, 07 2024. httt:ppss//eenccoooo////eeeerree htt.a/d....rrreerrorr///2226611
Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. The llama 3 herd of models. arXiv preprint arXiv:2407.21788. 2024.
Simeng Han, Hailey Schoelkopf, Yilun Zhao, Zhenting Qi, Martin Riddell, Wenfei Zhou, James Coady, David Peng, Yujie Qiao, Luke Benson, al. Folio: Natural language reasoning with first-order logic. arXiv preprint arXiv:2209.0083 2022.
Chaoqun He, Renjie Luo, Yuzhuo Bai, Shengding Hu, Zhen Leng Thai, Junhao Shen, Jinyi Hu, Xu Han, Yujie Huang, Yuxiang Zhang, al. Olympiadbench: A challenging benchmark for promoting agi with olympiad-level bilingual multimodal scientific problems. arXiv preprint 2rXiv:2402.14008 2024.
Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. Measuring mathematical problem solving with the math dataset. NeurlPS, 2021.
Jiaxin Huang, Shixiang Gu, Le Hou, Yuexin Wu, Xuezhi Wang, Hongkun Yu, and Jiawei Han. Large language models can self-improve. In Houda Bouamor, Juan Pino, and Kalika Bali (eds.), Proceed- ings of the 2023 Conference on Empirical Methods in Natural Language Processing, pp. 1051- 1068, Singapore, December 2023. Association for Computational Linguistics. doi: 10.18653/v1/ 2023.emnlp main.67. URL hatpps:/aclaattooooo...ooo hrw.hhooooooo....o/ooooooo.ooo/o
emnlp manlp-main. 67/
Jie Huang, Xinyun Chen, Swaroop Mishra, Huaixiu Steven Zheng, Adams Wei Yu, Xinying Song, and Denny Zhou. Large language models cannot self-correct reasoning yet. In The Twelfth International Conference Learning Representations, 2024.
Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Os- trow, Akila Welihinda, Alan Hayes, Alec Radford, et al. Gpt-4o system card, arXiv preprint 2rXiv:2410.21222 2024.
Mehran Kazemi, Bahare Fatemi, Hritik Bansal, John Palowitch, Chrysovalantis Anastasiou, San- ket Vaibhav Mehta, Lalit K Jain, Virginia Aglietti, Disha Jindal, Peter Chen, et al. Big-bench extra hard. arXiv preprini 2502 2025.
Zixuan Ke, Fangkai Jiao, Yifei Ming, Xuan-Phi Nguyen, Austin Xu, Do Xuan Long, Minzhi Li, Chengwei Qin, Peifeng Wang, Silvio Savarese, et al. A survey of frontiers in Ilm reasoning: Inference scaling, learning to reason, and agentic systems. arXiv preprint arXiv:2504.09037, 2025.
Michael Krumdick, Charles Lovering, Varshini Reddy, Seth Ebner, and Chris Tanner, No free labels: Limitations of llm-as-a-judge without human grounding. arXiv preprint arXiv:2503.0500 2025.
Aviral Kumar, Vincent Zhuang, Rishabh Agarwal, Su, John Co-Reyes, Avi Singh, Kate Baumli, Shariq lgbal, Colton Bishop, Rebecca Roelofs, al. Training language models to self-correct via reinforcement learning. arXiv preprint arXiv:2409.122 12917, 2024.
H. Kydlížek. Math-verify: Math verification library. 2025. URL https::/ com/ hngggingfaace/mttt--errr
Sungjae Lee, Hyejin Park, Jaechang Kim, and Jungseul Ok. Semantic exploration with adaptive gating for efficient problem solving with language models. arXiv preprint arXiv:2501.05755 05755,, 2025.
Aitor Lewkowycz, Anders Andreassen, David Dohan, Ethan Dyer, Henryk Michalewski, Vinay Ra- masesh, Ambrose Slone, Cem Anil, Imanol Schlag, Theo Gutman-Solo, et al. Solving quantitative reasoning problems with language models. Advances in neural information processing systems, 35:3843-3857, 2022.
12

---

Published as a conference paper at ICLR 2026
Jia Li, Edward Beeching, Lewis Tunstall, Ben Lipkin, Roman Soletskyi, Shengyi Huang, Kashif Rasul, Longhui Yu, Albert Q. Jiang, Ziju Shen, et al. Numinamath: The largest public dataset in ai4maths with 860k pairs of competition math problems and solutions. Hugging Face repository. 2024. Available at
/
co datasets s/att//ATT-- AT-MO/ /ww.inamaataaaaomm
Ruosen Li, Teerth Patel, and Xinya Du. Prd: Peer rank and discussion improve large language model based evaluations. arXiv preprint arXiv:3307.02762, 2023.
Shalev Lifshitz, Sheila A. Mcllraith, and Yilun Du Multi-agent verification: Scaling test-time compute with multiple verifiers. In Second Conference on Language Modeling, 2025.
Hunter Lightman, Vineet Kosaraju, Yuri Burda, Harrison Edwards. Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. Let's verify step by step. In The Twelfth International Conference on Learning Representations, 2023.
Runze Liu, Junqi Gao, Jian Zhao, Kaiyan Zhang, Xiu Li, Biqing Qi, Wanli Ouyang, and Bowen Zhou. Can 1b IIm surpass 405b Ilm? rethinking compute-optimal test-time scaling. arXiv preprint arXiv:2502.06703, 2025a.
Shudong Liu, Hongwei Liu, Junnan Liu, Linchen Xiao, Songyang Gao, Chenggi Lyu, Yuzhe Gu, Wenwei Zhang, Derek F Wong. Songyang Zhang, et al. Compassverifier: A unified and robust verifier for Ilms evaluation and outcome reward. arXiv preprint arXiv:2508.03666 2025b.
Zichen Liu, Changyu Chen, Wenjun Li, Penghui Qi, Tianyu Pang, Chao Du, Wee Sun Lee, and Min Lin. Understanding rl rl-zero-like training: A critical perspective. In Conference on Language Modeling (COLM), 2025e.
Zijun Liu, Peiyi Wang, Runxin Xu, Shirong Ma. Chong Ruan, Peng Li, Yang Liu, and Yu Wu. Inference-time scaling for generalist reward modeling. arXiv preprint arXiv:2504.024999 2025d.
Jack Lu, Ryan Techan, Jinran Jin, and Mengye Ren. When does verification pay off? a closer look at Ilms as solution verifiers. arXiv preprint arXiv:2512.02300 2025.
Dakota Mahan. Duy Van Phung, Rafael Rafailov, Chase Blagden, Nathan Lile, Louis Castricato, Jan-Philipp Fränken, Chelsea Finn, and Alon Albalak. Generative reward models. arXiv preprint arXiv:2410. 12832, 2024.
Yujun Mao, Yoon Kim, and Yilun Zhou Champ: competition-level dataset for fine-grained analyses of Ilms' mathematical reasoning capabilities. In Findings of the Association for Computational Linguistics ACL 2024, pp. 13256-13274, 2024.
Melissa Z Pan, Mert Cemri, Lakshya A Agrawal, Shuyi Yang, Bhavya Chopra, Rishabh Tiwari, Kurt Keutzer, Aditya Parameswaran, Kannan Ramchandran, Dan Klein, Joseph E. Gonzalez, Matei Zaharia, and Ion Stoica. Why do multiagent systems fail? In ICLR 2025 Workshop on Building Trust in Language Models and Applications, 2025.
Shrey Pandit, Austin Xu, Xuan-Phi Nguyen, Yifei Ming, Caiming Xiong, and Shafiq Joty. Hard2verify: A step-level verification benchmark for open-ended frontier math, 2025.
David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Dirani, Julian Michael, and Samuel Bowman. Gpqa: A graduate-level google-proot q&a benchmark. In First Conference on Language Modeling, 2024.
Jon Saad-Falcon, E Kelly Buchanan, Mayee F Chen, Tzu-Heng Huang, Brendan McLaaughin, Tanvir Bhathal, Shang Zhu, Ben Athiwaratkun, Frederic Sala, Scott Linderman, et al. Shrinking the geneeraiion-veriiicatiin gap with weak verifiers. arXiv preprint arXiv:2506.18208 2025.
Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. Reflexion: Language agents with verbal reinforcement learning. Advances in Neural Information Processing Systems, 36:8634-8652. 2023.
Janvijay Singh, Austin Xu, Yilun Zhou, Yefan Zhou, Dilek Hakkani- and Shafiq Joty. On the shelf life of finetuned LLM-judges: Future proofing. backward compatibility, and question generalization. In The Fourteenth International Conference Learning Representations, 2026.
13

---

Published as a conference paper at ICLR 2026
Nishad Singhi, Hritik Bansal, Arian Hosseini, Aditya Grover, Kai- Wei Chang, Mareus Rohrbach, and Anna Rohrbach. When to solve, when to verify: Compute-optimal problem solving and generative verification for Ilm reasoning. arXiv preprint wrXiv:2504.01003 2025.
Charlie Victor Snell, Jachoon Lee, Kelvin Xu, and Aviral Kumar. Scaling LLM test-time com- pute optimally can be more effective than scaling parameters for reasoning. The Thirteenth International Conference Learning Representations, 2025.
Benedikt Stroebl, Sayash Kapoor, and Arvind Narayanan. Inference scaling flaws: The limits of I1m resampling with imperfect verifiers. arXiv preprint arXiv:2411.17500 2024.
Shaoning Sun, Jiachen Yu, Zongqi Wang, Xuewei Yang, Tianle Gu, and Yujiu Yang. S2j: Bridg- ing the gap between solving and judging ability in generative reward models. arXiv preprint arXiv:2509.22099 2025.
Sijun Tan, Siyuan Zhuang, Kyle Montgomery, William Yuan Tang, Alejandro Cuadron, Chenguang Wang, Raluca Popa, and Ion Stoica. Judgebench: A benchmark for evaluating LLM-based judges. In The Thirteenth International Conference on Learning Representations, 2025.
Gemma Team, Morgane Riviere, Shreya Pathak, Pier Giuseppe Sessa, Cassidy Hardin, Surya Bhupatiraju, Léonard Hussenot, Thomas Mesnard, Bobak Shahriari, Alexandre Ramé, et al. Gemma 2: Improving open language models at practical size. arXiv preprint 2xXi:2448.000113 2024.
Qwen Team. Qwen2 technical report. arXiv preprint arXiv:2407. 10671, 2024.
Han Wang, Archiki Prasad, Elias Stengel-Eskin, and Mohit Bansal. Soft self-consistency improves language model agents. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), 2024a.
PeiFeng Wang, Austin Xu, Yilun Zhou, Caiming Xiong, and Shafiq Joty. Direct judgement preference optimization. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, November 2025.
Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc V Le, Ed H. Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. Self-consistency improves chain of thought reasoning in language models. In The Eleventh International Conference on Learning Representations, 2023.
Yubo Wang, Xueguang Ma, Ge Zhang, Yuansheng Ni, Abhranil Chandra, Shiguang Guo, Weiming Ren, Aaran Aruiraj, Xuan He, Ziyan Jiang, et al. Mmlu-pro: A more robust and challenging multi- task language understanding benchmark. Advances in Neural Information Processing Systems, 37: 95266-95299 2024b.
Jason Wei. The asymmetry of verification and verifier's law. http:: www. jasonwei. net/
bl o www.teeeriiiiiiiooooo aant-oooo--o.oi.aaaaaaaaaaaoo--- ai
2025.
Chenxi Whitehouse, Tianlu Wang, Ping Yu, Xian Li, Jason Weston, Ilia Kulikov, and Swarnadeep Saha. J1: Incentivizing thinking in llm-as-a-judge via reinforcement learning. arXiv preprint 2.ti::2505./03200 2025.
Austin Xu, Yilun Zhou, Xuan-Phi Nguyen, Caiming Xiong, and Shafiq Joty. J4r: Learning to judge with equivalent initial state group relative policy optimization. arXiv preprint 133..4600...336, 2025.
Austin Xu, Xuan-Phi Nguyen, Yilun Zhou, Chien-Sheng Sheng Wu, Caiming Xiong, and Shafiq Joty. Foundational automatic evaluators: Scaling multi-task generative evaluator training for reasoning- centric domains. The Fourteenth International Conference on Learning Representations, 2026.
An Yang, Baosong Yang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Zhou, Chengpeng Li. Chengyuan Li, Dayiheng Liu, Fei Huang, et al. Qwen2 technical report. arXiv preprint arXiv:2407.1667 2024.
14

---

Published as a conference paper at ICLR 2026
An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao. Chengen Huang, Chenxu Lv, et al. Qwen3 technical report. arXiv preprint arXiv:2505.09388 2025.
Tian Ye, Zicheng Xu, Yuanzhi Li, and Zeyuan Allen-Zhu. Physies of language models: Part 2.1, grade-school math and the hidden reasoning process. In The Thirteenth International Conference on Learning Representations, 2025.
Zhangyue Yin, YuHong Sun, Xuanjing Huang, Xipeng Qiu, and Hui Zhao. Error classification of large language models on math word problems: A dynamically adaptive framework. In Findings of the Association for Computational Linguistics: EMNLP 2025, 2025.
Weihao Yu, Zihang Jiang, Yanfei Dong, and Jiashi Feng. Reclor: A reading comprehension dataset requiring logical reasoning. arXiv preprint arXivv2002..44326, 2020.
Lunjun Zhang, Arian Hosseini, Hritik Bansal, Mchran Kazemi, Aviral Kumar, and Rishabh Agar- wal. Generative verifiers: Reward modeling as next-token prediction. In The 4th Workshop on Mathematical Reasoning and Al NeurlPS'24, 24, 2024.
Lunjun Zhang, Arian Hosseini, Hritik Bansal, Mehran Kazemi, Aviral Kumar, and Rishabh Agarwal. Generative verifiers: Reward modeling as next-token prediction. In The Thirteenth International Conference on Learning Representations, 2025.
Yilun Zhou, Austin Xu, PeiFeng Wang, Caiming Xiong, and Shafiq Joty. Evaluating judges as evaluators: The JETTS benchmark of LLM-as-judges test-time scaling evaluators. In Forty- second International Conference on Machine Learning, 2025.
Mingchen Zhuge, Changsheng Zhao, Dylan Ashley, Wenyi Wang, Dmitrii Khizbullin, Yunyang Xiong, Zechun Liu, Ernie Chang, Raghuraman Krishnamoorthi, Yuandong Tian, al. Agent-as-a-judge: Evaluate agents with agents. arXiv preprint arXiv:2410. 10934, 2024.
15

---

Published as a conference paper at ICLR 2026
APPENDIX
A Prompt Templates
16
B Additional Preliminaries and Setup
22
B. I Details of Math Problems Correctness Check
22
B.2 Model Naming Conventions
23
Detailed Experimental Setup of Large-scale Interpretability Analysis
B.4 Detailed Experimental Sctup of RQ2
23
Extended Analysis and Additional Results
24
More Details Problem Difficulty Analysis
Analysis on Reasoning Model
27
Explanation Below-Randon Verification Performance on Hard Problems
29
Additional Results of Verifier Generation Capability
29
Additional Results of Generator Analysis in Test- Time Scaling
30
C.6
Additional Results of Verifier Analysis in Test- Time Scaling
35
D
Extended Studies and Ablations
36
D.I Estimation of Problem Difficulty
36
D.2 Efficiency and Performance Analysis of Model Choices in TTS
37
D.2.1 RQ4.1: How to Choose Generators Given a Fixed Verifier?
38
D.2.2
RQ5.1: How Choose Verifier Given Fixed Generator?
40
D.3
Ablation Study on Large Models
D.4
Ablation Study Verilication Prompts
42
D.5
Comparison Analysis of Strong Model as Generator or Verifier
43
Case Studies
43
THE USE OF LARGE LANGUAGE MODELS (LLMS)
LLMs were used exclusively as writing assistance tools in preparing this manuscript. Specifically, we employed LLMs for grammar checking. An LLM--ased image generation tool was used to create the robot caricature in Figure T All research ideation, experimental design, analysis, and scientific conclusions are entirely the work of the authors. The LLMs played no role in the conception of research questions, methodology development, or interpretation of results. Authors take full responsibility for all content in this paper, including any text refined with LLM assistance.
A PROMPT TEMPLATES
Response Generation Prompt. Here, we provide the prompts to generate model responses to questions from three domains. For each model, we use its default system prompt as specified in the model documentation.
16

---

Published as a conference paper at ICLR 2026
Mathematical datascts
#+d Usez Prompt
(problem)
Please step by step, put your final within \boxed ()
Knowledge: MMLU-Pro
User Prompt
The folloxing are multiple choice questions (with answers) about |catagoryl- Think atop by step and then output the answer in the rormat \"The anawer (XI where X the correct letter choice.
Question: (quastion)
optiona:
(options:
Answer: set's think step by atep.
Natural Language Reasoning: ReClor
#&d Usez Prompt
Answer the following multiple choice question. The last line of respense should the fowiiwnn format: "Anawer: $1ETTER' twoth quotes) where $1ETTER is ona of Think step step before answering.
(context]
(question)
(choices)
Output your anzwez strictly follswing this format: Reascning: kstep-by-step reasoning>
Answer: syour chofce>
Natural Language Reasoning: FOLIO
#+# Uses Prompt (question)
Based on the above, can the following be inferred? (conclusion)
Think step-by-step betore outputting your answer, which must be "Yes", "No", or "Uncertain"
Output your strictly following this format: Beascning: Your reasoning here or Uncertain.
Natural Language Reasoning: GPQA Diamond
#4# User Frompt
Answer the following multiple choice question. The last line of should the follewing format: "Answer: $1ETTER' (without quotes) where $LETTER is one of ABCD. Think step step before answering.
Question: [question]
Opticns:
(choices:
Output your answer strictly following this format: Reascning: <step-by-sstep ressoning?
Answer: <your choice>
17

---

Published as a conference paper at ICLR 2026
Verification Evaluation Prompt. Below, present the prompt template used to evaluate verification performance.
System Prompt
Flease act impertial judge and evalusce the correctness the zesponse provided by an Al assistent to the user prompt displayed below. You will be given the assistanz's response.
evaluating assistant's response, identify any nistakes insccurace information. Be objective as possible. Avoid any bisses, such as order of responses, longth, stylistic elements formatting.
Before providing an your final vardict, think through the judging precess and output your thoughts exp innaaaaiion
providing your explenstion, you must output only your final vercict with label:
following
The zeaponse is correct: [[Correct]]
2. The response is incorrect: [[Tneorreat]]
Use the following template: Explanation:
detailed though: process explanation. Verdict: [[Correct,] or [[Incorract)..
#4# Usez Prompt
<|0aer Prompt [question]
<|Thh Start Assistant's Answer|> [reaponsa]
<[The End of Aesistant's a Answer 1>
Verification Evaluation Prompt (Solve-then-Verify), Below, present a variant of the prompt template used to evaluate verification performance. This template is adapted from the idea of Chen al. (2025d); Sun et al. (2025) that prompts the model solve the problem first and use its own solution as a reference for verification.
44# System Prompt Flease
impartial judge and eveluate correctness response provided by an Al assistant to the user prompt displayed below. You will be given the assistan:/s respense.
First, you MJST solve the question yourself and pat your final answer following the fornat zequested <lJser Frompt Frovide your solution with finel answer before proceeding to the evaluation. When evaluating the candidate's response, you MUST refer your own solution.
as oojective as possible. avold any blases, such as arder of responsea, length, or stylistie elements like formatting.
After providing your explenstion, you output only of the following choices your final vercict with label:
1. the sesponae is correct: [[Correct]] The reaponse is incorrect: [[Tncorrect]]
Use the following template: solution:
reasening and final answer the problem.
Expianation: Your detailed though: process as explanaticn. Verdict: [[Correct:] ez [[incorrect]].
44# Usez Frompt
<|User Prempt (question)
<|The Start of Assistant's Answer|> (responsa]
</The End of Rasistant's Answer|>
Verification Evaluation Prompt (Step-by-Step). Below, we adopt a variant of the verification
template from /hang al. 2024). This template only has user prompt.
18

---

Published as a conference paper at ICLR 2026
A4# Use: Prompt
teacher. Grade Soluzion, verifying cocrestness step by s:ep. A= the end of the Solution verification, when you give your final grade, write is in the Torm "Verification: Is the answer correct (Yes/Hlo2 x", wheze is either ( (boxed([[e)] Question: (quastion] Solutions [response]
Fallback Correction Check Prompt. When Math Math-Verife Fy returns unparsable incorrect results, we employ LLM-as-judge as fallback mechanism for correctness verification. Below, we provide the prompt template used for this secondary verification step:
#t# User Frompt
nath problem, its answer, model's generated answer, detarmine 1
the model's s generaled anawer is correct.
VALICATION CR-TERIA:
Toent! fy final answer, which vaually put inside Wboxed [answer] or
**anaweive
2. The answer most be mathematically equivalent to the correct ansver complete with clear final 4. The answes must not just contain similar numbers it must reach the correct conclusion
anawe:
5. If the generated enswer contains multiple different final answers or is ambi about which is the final answer, mark it as "Falae'
IMPORTANT: Juat having the same numbera as the ground truth is NOT aufficient the must actually the designated fornat.
problem correctly provide
rinal
Respond with 'True! If the answer is correct and complete and "False' if is incorrect or incomplete,
Directly provide your judgement "True" "Falze' without any other descripsion.
Problem: (problem)
Correct Answer: (groun_
Model's Generated Answer: |model_ readosee)
Your judgement:
Verification Analysis Prompt for Detecting the Solving Behavior. Below present the prompt template used to analyze if verification CoTs contain solving behavior. This prompt asks the verifier solve the problem first and then
System Prompt
Please act impartial analyzer and determine whether the AT assiecant, when evaluating a candidate response, generates its own solution (full or pastial) as rešerence either explicitly implicitly.
<*Yay Indicators
asaistant GENErAtRD The assistant determines what the correct answe: should be (even implicitly)
solutior [[Yea]]
assistant works through probler logic independent le.g., *she premises actually x", "what be infecred is y*)
The assistant constructs its ewn intespresation of what follows frem the problem
The assistant makes claims ahout what the "correct reasoning" would be
*ey Indicators that the assistant ONLY INSPECTED the candidate [[Ne]]:- The assistant only pointa out errors the catels reasoning flow wishout determining the correct answer
The assistant only identifies missing steps or unsupported leaps in the candidate's chain
The assistant on' verifiea conatatency within the candidate's own reasoning chain "X actually g"
purely on candidare failed juatify
i.irportant:<> Even implicit partial sclutions count generating solution. assistant reveals what believes to be true/false about the problem, it has generated a solution.
+x3ample 1:**
19

---

Published as a conference paper at ICLR 2026
<]User Prompt [examplei__qeetionn:
<IThhe of Candidate Response|> texamplei_reeppnse:
<(The End of Candidate Responsel>
<[The Start Asst stant'a Rvalus:ton|s litampplee_evauetiiin
<IThe End of Assistant's Evaluation >
A expectee Analyata:aa [examplei_analyeii:
Now analyze the following case. Look carefully for ANY statement where the assistent determines is actually aastatant generated Its own solution.
valid, inferable these indieate the
Use the fol lowing template:
Explanations Your reasoning for why the assistant did or did not generate i:s own solution. Quote specific párases that reveal solution generation. Verdict: [[Xes]] or ( [[No]]
A4d Usez Prompt
<|User Prompt [question]
<|ithe Start of Candidate kesponse|> (responss)
<|The End of candidate Responsel>
<[The Start Assistant's Eveluscion/> (evaluation)
<[The End of Assistant's Evaluation/>
Verification Analysis Prompt for Detecting the Reasoning Mistakes. Below, we present the prompt template used to analyze if verification CoTs contain any basic reasoning mistakes.
### System Prompt
to datermine mistake in it' reasoning when eva huating the candi response.
impartial checker.
evaluates
You the provided geld_enswer whether it is full explanation just final answer as the reference for whaz is correct.
You MUST ignore the evaluator's s final verdict entirely. Do NoT read it, do NoT interpret it to influance your judgmant any way.
it, Not
the evaluator's reasoning contradieus the answer, or the evaluator asserts incorrest mathematicel logical statements, then the evaluator's reasoning incorrest.
If the evaluator's reasoning is consisient with the saswer and containa no cla'ma ma, then the avalussor's reasontng ng correct.
After reviewing the evalustion, output:
Explanation: brief explanstion of whether the evaleator's a reasoning is correct or incorrect.
Verdiet: [[Correct]] oz [[Incorrect]]
44# Uses Prompt <|User Prompt (question)
</Correct
Froblem/>
[correct_ansvec]
<IThe Start of Candidate Response|> (xesponse)
<|The End of datd Reaponse|>
20

---

Published as a conference paper at ICLR 2026
<IThe Start of Assistant's Evaluacion/> levaluation on) <IThe End of Rasistant's Bvaluation
Generator Response Analysis Prompt for Identifying Surface-level Error
44d Syatem prompt checking sell col2-contradictory ecro:
You impartial analyzer. You will given question candidate response. The candieaae reaponse 15 known to be incorrect.
Your ONLY task is to determine whether the candidate
internelly consistent.
Consistency Definitions 4/4
Definition "Consistent";
The reasoning does not contracict itself.
The steps follow coherent internal logic.
The respoase may contain wrong assumptiens, missing but it is still consistent if i: doea not conflict with itself.
wrong math,
refinition of "Not Consisten:":
The candidate response contains explicit or inpliatt contradiction.
It states mutually incompatible claims.
Its ressoning steps cannot simulteneously be true based on its own premises.
### Strict Prohibitions ###
IMPORTAT::
NOT check cosrectness.
DO NOT compare the true
ONLY evaluste whether the reasoning contradicts itself.
Now analyze the fo lowing case. Respond using the format below:
Explanation: Briefly explain why the candidate response is internally consistent or inconsistent.
Verdiet: [[Yes]] (consistenti [[No]] faos consisteat)
#+# System prompt for checking baste math ervor
You are impartial analyzar. You will given a quest lon and candidata responge. The candidate reaponse is known in advance to be incorsect.
Your ONLY task is to determine whether the candidate respoase contains basic calculation mistakes basic machematioal knowledge
counts as basic errer ### A reaponse contains pasic math error it i5 includes:
Incorsect arithmetic (wrong addition, subscaction muitiplication, division). simplification algebraic ranipulation.
incorsect fundamental moth facts
(e.g., parity miatakea, mfaiden:iffing primes, incorvect factorial values).
Incorsect counting in basic combinatozics (e.g., aumber of simple ou.comes, pai:s).
Incor=act use
probahility rules (e.g., wrong multipl i
rule!.
#+ What does NOT count as basic math erroz if:
response does NO: centain basic mach All arithmetic steps are numerically correct, nathenatioal facts cules are applied correctly,
correctly,
3asic counting/probbbiili even the overall reasoning ia wrong, conclusions incorrect. even impostant steps are missing.
### Strict Prohibitions ###
IMPORTANE;
DO NOT eveluate correctness of the final answer.
candidate response DO NOT judge consistency high-level reasoning.
NOT compare
aпswer.
ONLY check for basic arithmetis or elementary math knowledge mistakes.
Use the following format:
21

---

limits (e.g., mid sentence, Published as a conference paper at ICLR 2026
asplianation Brief explain whether the candicate resporse containa basic ca calculation math-knowd knowlecke errora. Verdiet: [[Yes]] (contaans basic nath ezroz) or :Noo] (does not conzein basic mach exror)
#+# System prompt for checking incompleteneas error
You impartial analyzer, You will given question and candidale sesponse. The candidate response is known in advance to be incorrect.
Your ONLY task la to determine whether the dane response actual ly provides a final the question.
#4 What counta as providing a final answer (Verdiet: ([Yea])) À candidate zesponse PROVIDES final
if: it clearly stetes a specific finel value, equation, or expression that answers the question,
even if the value is wrong.
anawe: may given any format, Thaide Whoxed;
In LaTeX nath, e.g., (315)
As plain text, e.g., "The probebility 2/3."
in bold or marked as +*Final Answer:** followed by a concrete number or expression,
#4# SPECIAL CASE: Token Limit Cutofr 4#d zesponse clearly mid-word,
mid equation, or stops abruptlyl, then:
You
this the model ATTEPTTIN provide final In this casa, output Verdict: ([Yes]).
#+# Khat counts as NOT providing final answer (Verdiet: [[No]]) ## candidate response
provide
if:
It only gives high-level discussion, explanation, or strategy withous stazing a concrete resuit.
says the result as "you can row compute* without doing it. partial
is hard compute, suggests using calculstor, laaves the
(e.g., up expression never clearly clains it as the finel answer). only restates the problem, gives definitions, explicit cutcoma.
trails off
evaluates approaches without
commitaing
### Strict Prohibitions ### IMPORTANT:
answer. DO NOT compare the candidate response te the true aпswer.
NOT evaluate correctness
NOT judge consistency high-level reasoning.
ONT.Y check for providing final answer.
fol lowing format:
Explanation:
Briefly explein
candidate response provides answer.
Verdict: [[xes]] (provides final answer) oz [[No]] ] (does not provide a finel answer)
B ADDITIONAL PRELIMINARIES AND SETUP
B. DETAILS OF MATH PROBLEMS CORRECTNESS CHECK
Here we detail the evaluation procedure for establishing response correctness, including fallback methods. Ground-truth correctness determined using Math-Verify (Kydlíček) 2025). Math-Veri fy fails to parse an answer returns incorrect, we recheck with other maathngaaching verifiers from open-source repositories 1m-eval (Gao et al. 2024). Dr GRPO (Liu et al. 2025c), and Q h-Math (Yang al. 2024). We further apply GPT-4.I-min and Qwen2.5-72B to con- duct reference- based evaluation and check the equivalence the model prediction and ground-truth answers. The prompt template for L.-M- verification provided in Appendix
22

---

Published as a conference paper at ICLR 2026
B.2 MODEL NAMING CONVENTIONS
Throughout this paper, we use abbreviated model names in figures and tables to improve readability and space efficiency. Table I provides the complete mapping between abbreviations and full model names, All models referenced are instruction-tune -unetttedd versions unless otherwise specified.
Table 1: Mapping between abbreviated model names used in figures and their full names. All models are instruction- tunee versions.
Abbreviation
Full Model Name
Abbreviation
Full Model Name
G2-2B
Gemma2-28
Q3-4B
Qwen3-4B
G2-9B
Gemma2-9B
Q3-8B
Qwen3-8B
G2-27B
Gemma2-27B
Q3-32B
Qwen3-32B
L3-3B (L-3B)
Hamma3.2-3B
M-8B
Ministral-8B
L3-8B (L-8B)
Llama3. 1-8B
M-24B
Mistral-Small-24B
L3-70B (L-70B)
Llama3.3-70B
gpt-4o
GPT-4o
Q2.5-3B
Qwen2.5-3B
Q2.5-7B
Qwen2.5-7B
Q2.5-72B
Qwen2.5-72B
B.3 DETAILED EXPERIMENTAL SETUP OF LARGE-SCALE INTERPRETABILITY ANALYSIS
This section details the experimental setup for the large-scale interpretability analysis supporting the findings in Section 4.1 (RQ1) and Section 4.2 (RQ2).
For RQI, we apply GPT-4.1-mini as LLM-as-judge with two prompts (in Appendix A): (i) to determine whether the verifier's CoT shows evidence of generating its solution claim; (ii) if so, to detect whether that generated content contains mathematical or reasoning mistakes. A verification satisfied. We this
CoT is labeled containing solving mistake only if both conditions analysis on 367,920 GPT-4o verification responses, the same ones for computing TPR. Because the number of correct responses varies across difficulty bins (harder problems yield fewer correct responses), we sample 5,000 correct responses per difficulty bin. For each bin, we compute the percentage of FNs and the percentage of FNs that contain solving mistakes. We perform 10,000 bootstrap resamples and report the mean and 95% confidence intervals (CIs).
For RQ2, we examine three categories of surface-level errors: (i) self-contradictions. (ii) basic arithmetic or factual mistakes, and (iii) incomplete responses lacking a final answer. The three prompts for detecting each error are included in Appendix A and we use GPPT-4.1minn as the judge. We apply this analysis to 98, 782 incorrect generator responses, the same ones used for TNR. For each problem, we randomly sample one incorrect response from each of the 15 generators and compute the proportion of these responses that contain surface-level errors. We perform 10,000 bootstrap resamples and report the mean and 95% Cls.
B.4 DETAILED EXPERIMENTAL SETUP OF RQ2
Due to significant differences in generator capability, when measuring TPR, for some very difficult problems, none of the 64 responses sampled from weak model are correct. To ensure fair evaluation unaffected by intrinsic problem difficulty, we exclude these problems and keep only those where every generator produces least one correct response. We apply analogous filtering for TNR. keeping only problems where each generator produces at least one incorrect response. Beyond filtering problems, we also carefully balance how many responses we evaluate from each generator. As described in Section 3 we subsample 8 responses from each generator's 64-sample pool for verification evaluation, aiming for correct and incorrect when possible. However, across these 8-response subsets, stronger generators may have produced more correct responses than weaker ones. This would bias our metrics by creating different denominators per generator. To address this, randomly select one correct response per problem from each generator's 's 8 8-response pool when computing TPR (and analogously for TNR). We repeat this evaluation with random selections eight times and report the mean.
23

---

Published as a conference paper at ICLR 2026
C EXTENDED ANALYSIS AND ADDITIONAL RESULTS
C.I MORE DETAILS IN PROBLEM DIFFICULTY ANALYSIS
In Section 4.1 show that problem difficulty primarily influences the verifier's ability to recognize correct responses. As discussed in the main paper, our analysis is conducted at two levels of granularity: response level and problem level. Figure[]shows results at the response level. Figures g and 10 show results at the problem level, summarizing the distribution of TPR and TNR across difficulty quartiles. Together, these results confirm our main finding that problem difficulty strongly correlates with TPR but has systematic effect on TNR.
24

---

Published as a conference paper at ICLR 2026
Handest 25%
Hand 25-50%
Easy 50-75%
Easiest 15-100%
Llamu-3 2:3B
Gemmu2-2 20
Qwen2 5:3B
Qwea344
Elamal 1-BB
1.0-
0.0
Camman'
Qrees2 5
CwemS-aB
Liemaši 3-00
Germat 2718
1.0
0.5
5-725
Qwent-128
Miristal-IB
Mistra--mmil-245
GPT-4
1.0
0.5
0.0
Mathematics
Llamo
Genma2
Qwee2 3B
Qwest
Llame3.1
1.0
0.5
Cummma2488
Gommad-278
1.0
F
0.5
0.0
1.0
Peiiee
0.5
0.0
Knowledge
1.0
0.5
Gommae2.0 3B
Queed 5.78
Cwass-48 IB
Elmaa 3-70B
Genmat 278
The Preine
0.5
0.0
Qwas2.5-12B
Owea3-3322 32
Miristrl-FB
Mistral SmulB-24B
GPT-4a
1.04
0.5
0.0
(c) NL Reasoning
Figure 9: Problem difficulty correlates with verification TPR per-problem level across three domains. Each boxplot shows the distribution of per-problem TPR for 15 verifier models, grouped by difficulty quartiles. TPR exhibits strong positive correlation with problem easiness: easier problems consistently yield higher and less variable TPR.
25

---

Published as a conference paper at ICLR 2026
Handest 25%
Hand 25-50%s
Easy 50-75%
Easiest 15-100%
Llame33 2-3B
Gemma2 20
Qwen2 5:3B
Qwen3-4B
Elamal 14BB
1.0
Sma Neae
0.5
Camman
Quee2
CweenS-ae
Liemasi 3-700
Germat 27B
1.0
0.5
The
5-726
Qwest-128
Miristal-IB
Misira--mmil-245
GPT-4a
1.0
0.5
Mathematics
Llamo
Geama:
Qwesd
Llam331
1.0
0.5
1-20m
Gommad-218
1.0
0.5
Quem2
Qwamt-1221
Miristal-A8t
Mintral-mmell-44R
CIPT-4a
0.5
0.0
Knowledge
1.0
0.5
The
0.0
Commma2-998
Quena 5-711
Liamati, 3-700l
Germat -218
Tox Negiive
0.5
0.0
Owen2 0-722 5-728
Owes3-328
Mirista-BBB
Mistal Small-248
GPT-4a
0.5
0.0
(c) NL Reasoning
Figure 10: Problem difficulty shows systematic correlation with verification TNR per- problem level across three domains. Each boxplot shows the distribution of per-problem metrics for 15 verifier models, grouped by difficulty quartiles. TNR doesn't 't show obvious correlation with problem difficulty, exhibiting inconsistent trends across models.
26

---

Published as a conference paper at ICLR 2026
C.2 ANALYSIS ON REASONING MODEL
Our main analysis focuses on instruction-tunel models, which represent the typical setting for verification systems in current practice, including recent judge models (Tan et al. 2025 Wang al., 2025) and verifier work (Liu al., 2025b; Zhang a (025). We prioritize models without extensive CoT reasoning because verification often demands low-latency solutions, particularly for reinforcement learning training and TTS applications where rapid evaluation is critical. However, a recent trend involves training long-reas reasoning evaluators (Chen al. 2025d; Whitehouse al. 2025) that generate extended CoT before making verification decisions. To examine whether our findings generalize to this emerging paradigm,
include two reasoning models (Qwen3-80) -heen---B-hiining and Qwen3-32B-Thhhe T P and analyze how they perform across our research questions. These models generate longer reasoning traces before producing binary verdicts, representing the state-of-the in reasoning-enhancee verification.
In Figures II and 12 we evaluate the conclusion of RQ1 (Section 4.1 We observe that reasoning models exhibit the same TPR pattern as instruction-tuned models: easier problems consistently yield higher TPR across all three domains. This indicates that the fundamental relationship between problem difficulty and correctness recognition persists with extended reasoning. However, reasoning models exhibit a notable difference in TNR behavior. Unlike instruction-tune models, where TNR showed no systematic relationship with problem difficulty, both reasoning models demonstrate improved TNR as problems become easier across all three domains. This pattern suggests that, with extended reasoning, error detection becomes easier when problems become easier.
In Figure 13 we evaluate the findings RQ2 (Section (42) on reasoning models and find that they maintain the patterns observed in instruction tundd models. TPR remains consistently high with mild increases as generator strength increases, while TNR decreases more significantly (goes from red to white) with stronger generators. This indicates that the fundamental challenge of detecting errors from capable generators persists despite enhanced reasoning capabilities.
These findings demonstrate that reasoning models offer some advantages for error detection easier problems while preserving the core verification dynamics we identified. Problem difficulty continues govern correctness recognition, and generator capability primarily influences error detectability across different verification paradigms.
Owen3-8B-Thinkee
Qwen3-12B-
1.0
The Poeiive Kine
Rote
Praiive
0.5-
0.5
0.5
Uhe
0.0
0.0
Handett
Hbrd
Eagy
Easiest
Hardest
Hard
Elsyy
basiest
Harbest
Hard
Easy
Emies
Problan Difficulty
Prublum Diffouly
Problam Diffeell
(a) TPR (Mathematics)
(b) TPR (Knowledge)
(c) TPR (NL Reasoning)
1.04
Thu Negave Kare
Tue Negatre
0.5
0.5
0.5
0.0 Handat
0.0
0.0
Hand
Easy
Emiss
Hadden
Easy Problom Diffloity
Hand
Easiat
Hardol
Haad
Easy
Emici
Problem Diffieiity
Problem
Diffienity
(d) TNR (Mathematics)
(e) TNR (Knowledge)
(I) TNR (NL Reasoning)
Figure 11: Verification performance of reasoning models across problem difficulty at the per- response level. TPR (a-c) and TNR (d-f) for Qwen3-8B- ----Tinniig and Qwen3-32B -oon--32-TThinknn across difficulty quartiles in three domains. Both reasoning models show increasing TPR and TNR as problem difficulty decreases.
We use the suggested sampling hyperparameter (temperature 0.6, top-p 0.95).
27

---

Published as a conference paper at ICLR 2026
Hardest 25%
Hard 25-50%
Easy 50-75%
Fasiest 75-100%%
True Positive Rate (TPR)
Qwen5-8B-Thinde
Owen5-8B-Thinkee
Qwen3-8B- T-inkiiinng
True Posiivve Rate
1.0
1.0
Truc Posilive Rate
1.0
Truc Positive
0.5
0.0
0.0
0.0
(a) Mathematics
(b) Knowledge
(c) NL Reasoning
Qwan3-32B-Thine Thinnkig
Qwen3-32B-Thimn
Twen3-32B-Thinkeee
True Posiivve Rate
1.0
Posltive Rate
1.0
Tue Povitive Rate
1.0
0.5
0.0
0.0
0.0
(d) Mathematics
(c) Knowledge
(f) NL Reasoning
True Negative Rate (TNR)
Qwen3-8B-Thinne -iB--hiiniingg
Qwen3.8BB Bwee3--B-Thinning
Drven3-8B-Thinkeee
Truo Nogative Rate
1.0
1.0
True Negative Rate
1.0
Truc Nogative
0.5
0.5
0.5
(g) Mathematics
(h) Knowledge
(i NL Reasoning
Owen3-32B-Thiine 32n-Thinking
Qwen3-32B-Thhiee 22D-hhiking
Qwen3-32D- Brmm3-32D-Thinking
True Negaivv Rate
1.0
True Nogative Rate
1.0
T
True Negative Rate
1.0
0.5
0.5
0.5
0.0
0.0-
(i)
Mathematics
(k) Knowledge
(I) NL Reasoning
Figure 12: Verification metrics for reasoning models across difficulty quartiles at the per- problem level. Each boxplot shows distribution of per-problem TPR and TNR for Qwen3-8B- across difficulty quartiles in three domains. Both TPR and TNR distributions shift higher and become less variable as problems become casier.
Thinking and
(a) TPR (Mathematics)
(b) TPR (Knowledge)
(c) TPR (NL Reasoning)
(d) TNR (Mathematics)
(e) TNR (Knowledge)
(1) TNR (NL Reasoning)
Figure 13: Reasoning models as verifiers paired with generators of varying capability. TPR (a-c) and TNR (d-f) for Qwen3-8B-Thhe -h-nnking and Qwen3-32B-Thinkkin verifiers when evaluating responses from 15 generator models across three domains. Generators are ordered left-to-right by increasing generation capability, measured separately for each domain. Red indicates higher performance, blue indicates lower performance.
28

---

Published as a conference paper at ICLR 2026
C.3 EXPLANATION OF BELOW-RAN HELLO-RANDOM VERIFICATION PERFORMANCE ON HARD PROBLEMS
In Figure 6f we observe that verifiers achieve balanced accuracy below the random baseline of 0.5 on hard problems from the NL Reasoning domain, a result that needs explanation. This phenomenon can occur in reference-free evaluation when verifiers employ "soove-and-mmtch" verification strategy, where they attempt to solve the problem independently and then compare their answer with the generator's 's response.
For NL Reasoning tasks with 3-way or 4-way multiple choice formats, this mechanism can produce below-random performance when verifiers consistently fail to solve hard problems correctly. In such cases, the verifier never correctly identifies true positive responses (TPR 0) because it always produces wrong answers that don't match correct generator responses. However, it can still identify some true negatives when both the generator and verifier happen to select the same wrong answer. For three-way choices, the TNR = 0.5. With TPR near zero and TNR remaining positive, the balanced accuracy falls below 0.5.
This phenomenon is specific to tasks with limited answer spaces. The affected problems are those in the hard set with d(z) < 0.3, where even strong models achieve very low pass rates. It occurs in NL Reasoning because this domain includes three-way multiple-choice questions from datasets like FOLIO. It does not occur in Mathematics, where responses open-ended strings, in Knowledge domains with 10-way multiple choice, where the large answer space dilutes the effect.
C.4 ADDITIONAL RESULTS OF VERIFIER GENERATION CAPABILITY
providing correlation analysis between verifier generation capability and verification accuracy across the entire problem difficulty range in Figure [4] The results confirm our finding from the main paper that the correlation form varies with problem difficulty: medium problems show strong positive linear relationships, while hard and easy problems exhibit non-linear trends. Below, we provide clearer analysis of these patterns.
Here we present additional results for RQ3 from Section 4.3
Threshold-limited pattern on difficult problems. The threshold- regime appears in difficult problems, for example, in Figure 14 (a) in the difficulty range [0.2, 0.3). In this regime, increasing the verifier's generation capability does not translate into noticeably better verification, which is observed as the saturation pattern. As an example, Qwen3-32B (largest rectangle) has a generation capability of about 0.6, yet its verification performance is nearly identical to Qwen2.5-72B (circle) with a lower generation capability of around 0.4. Both models plateau around verification score of 0.7.
The underlying reason is that on extremely difficult problems, even the verifiers with the best generation capability not that good (S 0.6 pass rate). As a result, pushing generation ability doesn't 't yield many gains, as the models still lack the intrinsic ability to "handle" these problems. This implies that reliable verification requires the verifier to reach a sufficiently high level of generation capability. When models fall below this requirement on difficult problems, verification performance plateaus.
Transition to a linear pattern on medium problems. When moving to medium-difficuul problems, such those Figure 14 (a) range [0.4, 0.5), the relationship becomes more linear. In this regime, Qwen3-32B reaches a higher generation capability (around 0.8), and its verification performance increases accordingly to 0.75. This improvement allows Qwen3-32B to outperform another model with generation capability of around 0.5 and verification around 0.7, indicating a linear pattern.
In this intermediate regime, we see the most linear correlation. These problems are of average difficulty, and models typically demonstrate a baseline level of ability in solving such problems, as measured by generation capability. As such, we find that this regime matches well with regimes analyzed in prior work (Chen et al. [202cc] Tan al.) 2025). Therefore, this region exhibits the most intuitive behavior: Model verification ability tracks directly with its ability to understand (i.c., solve) the problem. It's important to note that these problems are solvable (i.e., not extremely difficult).
Saturated pattern on easy problems. In Figure 14 a) for the difficulty range (0.9, 1.0), Qwen2.5-7B (medium circle) already reaches nearly 1.0 in generation capability, the same the stronger Qwen3- 32B. Their difference on the r-axis is therefore essentially zero, yet their verification performance differs by about 0.1. This is the saturated pattern.
29

---

Published as a conference paper at ICLR 2026
The underlying reason is that differences in verification performance come from factors other than the ability to solve the problem itself. One such factor is whether the verifier can detect more subtle or advanced mistakes made by strong generators. As illustrated in the case studies in Figure 33 a model may be able to solve a problem correctly but still fail to recognize certain non-obvious or high-level erros in another model's reasoning. Thus, perfect generation capability is not sufficient for achieving strong verification performance on casy problems.
These findings highlight the need for regime-aware verifier strategies. On hard problems, strong verifiers are unnecessary as performance plateaus regardless of capability. On medium problems, selecting models with better generation capability consistently yields better verification. On easy problems, selecting higher-capabile strong models with similar capabilities show vastly different verification performance. Thus, optimal selection of strong verifiers requires supplementary benchmarking or alternative evaluation metrics.
models works well among weak-to-meding verifiers, but
C.5 ADDITIONAL. RESULTS GENERATOR ANALYSIS TEST-TIME SCALING
This subsection provides complementary results for the generator analysis presented in Section 5.1 demonstrating the generalizability of our findings across domains and problem difficulties. Figure 15 extends the analysis from the main paper to Knowledge and NL Reasoning domains. Our central finding from RQ4 holds consistently. As Figures [5a to 15c shows, verification gains peak at weak-medium generator strength, enabling these generators to substantially close performance gaps with stronger models. The underlying mechanism driving this phenomenon, identified in RQ2, remains consistent across domains. Figures [5d to [[ffshows that, as generator strength increases. TNR decreases sharply while TPR rises only modestly. For the strongest generators, the collapsed TNR limits verification gains as errors become increasingly difficult to detect. This brings high verification gains at weak-medium generator levels. In the main paper. we show results on problems with difficulty range d(x) E [0.7, 0.8) in Figure 7 Here, Figures 16 to I8 report results across the entire difficulty range for three domains, respectively. Figure shows the percentage of performance gap closed by verification for all sestongg generator pairs, computed on problems within each domain.
30

---

Published as a conference paper at ICLR 2026
Qwen2.5
A
Llama3
Mistral GPT
2-4B
24-32B
Qwen
Gemma2
*
7-9B
70B+
Dhutficallyy
Difficdt (B192)
Demioryyiogs (02.83)
Diffciiy (8304))
Diftcdly (B677)
Detiooby (6))
Difielty
00
64
34
08
02
14
of
6o Li 04
MM
06
62
An
as
0a
04
04
09
Cquiiiyy
(a) Mathematics Defticlly
(11A2)
oage:
Birtichly
02-
Difficelyyrgg (6..6.))
Difficley (1677)
Demieryyope (6.788)
Difficlt (1
neee (89.1
op
13
14 00
08
10
04
13
13
04
04
Verifer Geneztre Capaiily
Wethee Ceceration Copaaday
Verifas Guuenaie Cayiiiity
Velfler Geuratio Cipiiiity
(b) Knowledge
Dullisbyy maan (....))
Difficlly rean: (142)
Deliedky meac (b2.3)
Dilledy mag (83044
Bilikuly mane: (1482)
@
05-
96
Vestaan Gearrian Cepraaa Caporday
Vattlan Gaeatoe Capabiity
Vartia Grartiin Cipuslay
Varifeus Gonestie Capaiityy
Verfarr Couraton Capasiity
(c) NL Reasoning
Figure 14: Correlation between verification performance and generation capability across problem difficulty ranges on three domains. Balanced accuracy as a function of verifier generation capability for difficulty ranges from (0.0,0.1) to (0.9,1.0). Performance exhibits three distinct regimes: plateaus on hard problems, strong positive correlation on medium problems, and high variance with saturated capability on easy problems. Marker shapes indicate model family; sizes represent model scale.
31

---

Published as a conference paper at ICLR 2026
Before Verification
After Verification
TNR
TPR
Verification Gain
Pone Kase
0.81
9.8
Pass Nats
8.8
0.0
Upnrao216 115 peat
Gummnd 2TB ITS pat
8.6
Laral---0 apocches
0.
0.4
Clamad 1-88 an 17S peif
0.4
Gcttrratr Medels
Generatee Medels
Gonneratce Medels
(a) Pass rate (Mathematics)
(b) Pass rate (Knowledge)
(c) Pass rate (NL Reasoning)
Vonifuation Coin
Voriication Grin
Voniecaioo
199 TNN
Generntor Models
Genernoo Models
Generator Models
(d) Verif. metrics (Mathematies)
(c) Verif. metries (Knowledge)
(f) Verif. metrics (NL Reasoning)
Figure 15: TTS performance before and after verification when sweeping generator strength. (a-
c) Pass rate before (blue) and after (orange) adding a fixed verifier (GPT-4o)), across generators ordered from weaker (left) to stronger (right) by generation capability. (d-f) Bar chart shows the verification gain Apv (left y-axis) for each generator. Lines show the verifier's TNR and TPR on the same datasets (right y-axis). Results are reported on problems with difficulty in the range [0.7. 0.8) for three domains. Problem counts across domains: 181 (Mathematics), 154 (Knowledge), 97 (NL Reasoning).
Zrobicx Uidiveity Ruad (UAA))
Priien Deftecde Dhensttr//hheeeee PA0.
Prostes Dofted Madd (011.
Proter toeete M112
1.0
2 4.3
68
2 144
42-
6.0
Madii
PriMunn WiTTaaky Ruii E (-3-3)
Jridaa Duffadky Pai Rao B2A3
Prolian DuTaady Dais Rail C (0504
Trodlar Duflaaty Ran 9304
10.
1.0
Thane
R.
62
0.2
60
Gascrator
Qaatzer (0.50
hridees
3.0
Grenner Madde
Gendree Modie
Trohien Diffeshy. Raki R.A-7)
Triden Dfffeste Bac RA60
Prodien Duffcdyy Bac 10.70
Trotin Dfffndt BaadBad 125S
0.3
60
Molds
Ted D205 2oe Rrnt (1.10))
1.0
E
8as
Granzar Muids
Gaaceoo Modch
GUREEZ Maldi
Figure 16: Verification-augmeeee TTS performance across the full range of problem difficulties, shown here for the Mathematics domain. Each pair of figure corresponds to a different difficulty interval (measured by pass rate d(z)), with the left panel showing pass rates before (blue) and after (orange) verification, and the right panel showing verification gain Aoy (bars) alongside the verifier's ' TNR (green) and TPR (purple). Compared to Figure 7 which focused only on problems with d(z) E [0.7, 0.8). this includes the entire difficulty range.
32

---

Published as a conference paper at ICLR 2026
Pobice Delleey 77a Jeso (MLAZ
1.3
1.0
0.4
0.4
63
03-
Cirennee Medix
Goarrerr Maddi
Graeene Madek
Maddi
Prldaa Dufaady Rai 00:04
Prodlas Duffailly Rass M466
9.2
62
0.2
80
9.0
Neiten Ieffie ByctPon F-4--15
Priden Peffie Ritt F-40-9
Prider Wiw Par (0.00
Poiiee Diffeer Deertae BAAN
1.0
1.0
Fran Buan
Than B314
Gmaoddrr Madds
Ouasdir Mabds
Ouenzer Malds
Qaxtra Malds
ProMon 34fTeaky B.A00)
Priden Diffadyy B36
Puddar Duffaily Dafra 2A1A
1.-
Paa
201
Pho
022
60
Figure Verification-augmeneed TTS performance across the full range of problem difficulties, shown here for the Knowledge domain. Each pair of figure corresponds to a different difficulty interval (measured by pass rate d(x)), with the left panel showing pass rates before (blue) and after (orange) verification, and the right panel showing verification gain Apy (bars) alongside the verifier's TNR (green) and TPR (purple). Compared to Figure 7 which focused only on problems with d(z) [0.7, 0.8). this includes the entire difficulty range.
33

---

Published as a conference paper at ICLR 2026
Pridae by Kayg PIOG
Prokaes (ifieiilllly /eesslilllieeeeeee W1A7
2
0.5
R
204
0.5
Port
64
Tebicn Difieetty Pur (1220)
Deftre P.203
Poster Doacell (0.50
1.0
Thans Bate
0.5
2 64
Prilian XcflaaayyyPP (Pan Rubil d BAL-5
Proden Diflealy Rra)) BAU-9
Prldan Dufaady Dan Rubi 00.504
Prolier DJf fualliy Rani 90504
100
1.0
02-
Than Run
Graates Madill
Grenner Moddl
Gearre Modil
Profien Srffenhy
Proden Deffee Riat RUS0
1.5
Ditfeald Bast (00).)
Fass
80
Figure 18: Verification--aummenee TTS performance across the full range of problem difficulties, shown here for the NL reasoning domain. Each pair of figure corresponds to different difficulty interval (measured by pass rate d(z)). with the left panel showing pass rates before (blue) and after (orange) verification, and the right panel showing verification gain Apy (bars) alongside the verifier's TNR (green) and TPR (purple).
Percentage Gap Closed
100%%
-75%
-00%
25%
55%
100%
Cumnrrts Oppakiyy
000
by Gummtta Cupebiit
Hanked Gemoniuu Capabil
Octer Rantod By Urreatt Capetily
(a) Mathematics
(b) Knowledge
NL Reasoning
Figure 19: Percentage of TTS performance gap between weak and strong generators closed by verification. Each heatmap shows the fraction of the performance gap between a weaker generator (I-axis) and stronger generator (g-axis) that is closed by verification with a fixed verifier GPT-4o. Green cells indicate a larger gap closure, meaning the weaker model approaches the stronger one after verification. A value greater than 100% means that the originally weaker model performs better with verifier augmentation. Purple cells indicate negative values where verification increases the gap.
34

---

Published as a conference paper at ICLR 2026
C.6 ADDITIONAL RESULTS OF VERIFIER ANALYSIS TEST- TIME SCALING
Here we provide complementary results to the verifier analysis in Section 5.2 Figure 20 presents two other domains' results. Figure 21 presents additional metries in the same setup for complete analysis, including balanced accuracy and verification gains. We can see that our findings from Mathematics generalize to other domains. Figures 20a 20c shows that weak verifiers can approximate strong verifier performance in ITS, at the extremes of problem difficulty or responses generated by strong generators. While on these regimes, we show that both verifiers provide limited verification gain in Figures[2Tajoof Notably, in the NL Reasoning domain (Figure(0i) on the strongest generator, both fall below 0.5, and the weak verifier's TNR exceeds that of the strong verifier. Despite this TNR inversion, the overall verification performance gap (shown balanced accuracy in Figure remains narrow, with the strong verifier maintaining a slight advantage due to its superior TPR.
21
Generntors
Verifiers
Weak (L.mma3.2-3B) Medium (Llama3.1188) 1-8BB)
Strong (Qwen2.5-72B)
Strong (GPT-40)
Weak (Qwen2.S-7B)
0.175
0.155
Vertiaaiion Gain Gap
0.20
Verivattoo Gai Gap
0.20
Veii,aue Cain
0.125
0.15
0.10
0.075
0.10
0.05
0.025
0.050
0.00
0.05
0,000
(2660
(itee))
-0.05
(963)
Rian
Problem Diticclly
Preblem Diffically
Problem Difficully
(a) Verif. Gain Gap (Mathematies)
(b) Verif. Gain Gap (Knowledge)
(c) Verif. Gain Gap (NL Reasoning)
1.0
1.0
1.0
Time Postiv Rate
Posaive
0.8
True Positive
088
0.6
3.6
0.4-
0.4-
Risag
Rost
Ristd
8440
men)
Ruen
polt
man
Rise
Roto
paart
Plase
Problem Difficuty
Problem Dimenty
Problem Diffectty
(d) TPR (Mathematics)
(c) TPR (Knowledge)
(f) TPR (NL Reasoning)
0.8-
True Negntiv Rate
Thee Negative
0.6
Thue Negstive
0.6
0.6
0.4
0.4
0.2
R240
Roan
Problem Difficui
Proble Diflicaly
Problm Diflicag
(g) TNR (Mathematics)
(h) TNR (Knowledge)
TNR (NL Reasoning)
Figure 20: Analyzing verification gain gaps and TPR/TNR between verifiers under varying problem difficulty and generator strength. The r-axis shows problem difficulty measured relative to each generator. Shaded regions visualize the difference in metrics between verifiers for each generator. (a-c) Verification gain gap between strong and weak verifiers. (d-f) TPR increases as problems become easier for all generator-verifier combinations. (g-i) TNR decreases generators become stronger, with TNR gap narrowing.
35

---

Published as a conference paper at ICLR 2026
Generators
Verifters
(L.ama3.2-1BB
Strong (Qwen2.5-72B)
Strong (GPT-40)
Weak (Qwen2.5-7B)
Medium (Llama3.1-88)
Veriftatton Cati
0.3
Verifeation Cain
0.25
0.25
0.20
Veriiicaion
0.20
0.2
0.15
0.1
0.05
0.05
0.00
0.00
(ersan
Boep
plon
pous
Problem Difficuly
Problem Difficulty
Problem Difficulty
(a) Verif. Gain (Mathematics)
(b) Verif. Gain (Knowledge)
(c) Verif. Gain (NL Reasoning)
Balanced Accuncy
Balanced Accurcy
Balunced Ascurcy
0.8
0.8
0.7
0.6
0.6
0.5
0.
Rereb
(siep
Bleg
Rag
Krea
Razan
(Peb
Rie
min
miea
Rleg
levess
lawe
(923)
Rieh
Rieo
jeie
Poolem Dufically
Poolem Diffically
Auelem Difficully
(d) AcCbal (Mathematics)
(e) AcChal Knowweddge)
Accia (NI. Reasoning)
Figure 21: Analyzing verification gains and Accaal between verifiers under varying problem diffi- culty and generator strength. The z-axis shows problem difficulty measured relative to each genera- tor. Shaded regions visualize the difference in metrics between verifiers for each generator. (a-c) Verifi- cation gain for both strong (solid lines) and weak (dashed lines) verifiers across three generators: weak, medium, and strong. (d-f) Balanced accuracy for the same verifier-generatoo combinations. The accu- racy gap (shaded regions) between verifiers is smallest on the hard problems (left side of the z-axis).
D EXTENDED STUDIES AND ABLATIONS
D.1 ESTIMATION OF PROBLEM DIFFICUL.TY
We adopt the uncertainty- based estimator of problem difficulty from Lee et al. (2025). The idea is that a problem is more difficult when model produces highly diverse or inconsistent answers across repeated stochastic samples. For a given problem, we sample a generator model K times and collect the final answers: (a(r1), ((r1)))((2,))))))) a(r2), a(rk)) Let the set of unique answers be (ui,-.., uM }. where M < K. Let n, denote the number of times answer uj appears among the K samples. The corresponding empirical probabilities are:
Pj K
j= M.
(2)
We compute the Shannon entropy of the empirical answer distribution:
M
= Σ
Pj log Pj
(3)
3=1
The maximum possible entropy occurs when all K samples produce distinct answers, giving Hmax log K. We define the normalized difficulty score as
H
difficulty(x)
(4)
log K
This value lies in [o, 1). taking 0 when all samples agree (easy problems) and when all samples differ (hard problems), and serves as incertainty-basee estimate of the intrinsic difficulty of each problem.
For implementation, draw 8 solutions from Qwen2.5-3B for each question and compute the uncertainty- based difficulty estimator defined above. We use the resulting estimated difficulty values to reproduce the main results of RQ1 (Section 4.11) and RQ3 (Section 4.3). As shown in Figures 22 [3] the findings remain consistent when using this estimated difficulty measure, demonstrating that our conclusions hold under practical disticulty-ssiimation setting.
36

---

Published as a conference paper at ICLR 2026
Qwen2.55
Qwen3 Mistral
Lluma3
Small (2-4B)
Large (70B+))
Gemman
GPT-4o
Medium (7-328)
1.0
1.0
Posiive
True Negative Rate
0.5
0.5
0.0
Handest
Hard
Easy
Enolest
Handest
Fasy Probiem Difficulty
Hard
Enslest
Problem Difficulty
(a) TPR (Mathematics)
(b) TNR (Mathematics)
Figure 22: Ablation study of RQ1 with estimated problem difficulty. The experimental setup except that replace the oracle difficulty with our
follows that of Section 4.1 and Figure 2 uncertainty-based difficulty estimation.
Qwen2.5 Qwen3
Llama3
Mistral GPT
2-4B
24-32B
Gemma2
7-9B
70B+
y- X-0000
Matlar Liner X0-0 Nogoonn Ri-q ae
Balincdd Aucoucedd..ccoee
Linar Neperan X--075
0.24
0.6
0.0
0.2
0.4
0.5
Vertilet's Generaaion Cupability
(a) Stratified view (Mathematics)
Figure 23: Ablation study of RQ3 with estimated problem difficulty. The experimental setup follows that of Section 4.3 and Figure [6] except that we replace the oracle difficulty with uncertainty-based difficulty estimation.
D.2 EFFICIENCY AND PERFORMANCE ANALYSIS OF MODEL CHOICES IN TTS
This subsection examines how to make cost-effective model choices using our findings from RQ4 and RQ5 (Section 5). From RQ4, we observe that when weaker generators achieve high verification performance, they also get larger TTS verification gains, and can approach the performance of stronger generators. From RQ5, we find that in certain regimes, smaller verifiers can obtain verification performance comparable to larger ones, with similar verification gains. These results motivate us to verification metric guide generator and verifier selection in TTS.
In these experiments, we follow the efficiency metric and difficulty-stratified stratified analysis used Liu et al. (2025ap. For each subset stratified by problem difficulty, split the problem set into 50% validation set and a 50% test set. Verification metrics are computed on the validation set. Specifically, for each problem, the verifier, and compute balanced accuracy (the average of TPR and TNR). We use this metric select the generator or verifier among the candidates, and then evaluate their efficiency and TTS accuracy on the test set.
sample 8 balanced correct and incorrect responses from the generator, apply
For the efficiency metric, we use inference FLOPs, computed as 2N D Differnnce- where N represents the model parameters and Dinterence is the total number of tokens generated during inference. For open-source models, we approximate N using the parameter counts indicated their model names (e.g., we use 2 x 10 parameters for Gemma2-2B), For the proprietary model GPT-4o, we estimate the number of active parameters during inference as 100B, since the largest open-source model in our study is 72B and GPT-4o is widely understood to operate at larger effective scale. This estimate is used exclusively for relative efficiency comparison and does not affect our experimental findings. We report FLOPs averaged over all problems. When results are reported without verification, only the generator FLOPs are included; when results are reported with verification, the FLOPs both the generator and the verifier are summed. We use TTS sample size of 64 for every model combination
37

---

Published as a conference paper at ICLR 2026
and problem. We compute the total FLOPs used per problem and report the average FLOPs over the problem set.
For the accuracy metric, use the pass rate defined in Section 3.1 which measures the expected accuracy obtained by uniformly sampling one response from either the set model-generated samples from the verifier-retained samples. To complement the pass rate, also report majority-vote accuracy.
We consider the following two subproblems, named RQ4.1 and RQ5.1.
D.2.1 RQ4.1: How TO CHOOSE GENERATORS GIVEN A FIXED VERIFIER?
We compare three generator models (Gemma2-2B, 9B, and 27B) under fixed verifier, and we restrict the comparison to models within the same family and version to avoid confounding effects from differences in pre-training quality. Across three difficulty ranges: easy [0.9, 1.0), medium [0.7, 0.8). and hard [0.3, 0.4), we select the generator with the highest verification performance measured on the validation set. We vary the verifier between Qwen3-32B and GPT-4o.
Results of RQ4.1 The results are presented in Figure[44]pai rate, Qwen3-32B as verifier), Figure 25 (pass rate, GPT-4o verifier), Figure 26 (majority vote, Qwen3-32B verifier), and Figure 27 (majority vote, GPT-4o as verifier). Each figure contains three rows: (1) TTS performance vs. efficiency without verification the test set, (2) the same analysis with verification, and (3) balanced accuracy on the validation set. The model with the highest balanced accuracy is selected as the cost-efficient choice and highlighted in rows 2 and
Without verification, the largest model is usually the preferred choice due to its large accuracy advantage. With verification, there are cases where smaller models become compute-efficient choices because they achieve comparable TTS performance while requiring fewer FLOPs. This is consistent with our earlier observation Section 5 RQ4. Notably, the generator selected by the highest balanced accuracy on the validation set (red circle) consistently matches the compute-efficient choice: either identifies a smaller model that achieves accuracy close to the larger one or preserves the largest model when it retains clear performance advantage. This shows that verification-base metrics provide effective guidance for generator selection.
38

---

Published as a conference paper at ICLR 2026
27B
Compute-efficien-hhiie
(a) Easy problem, Verifi metrics
Medium problem, Verifi. metrics
Hard problem, Verifi. metries
Balancdd Astumay
Dalanced Dalmocd Accuraay
Rilenial Awrzayy
Balawed Baluned Accuazy
Ralegual Aurenc
0.8
(d) Easy problem, wio Verification
Medium problem, wio Verification
(1) Hard problem, wio Verification
Took Accuniy
0.30
0,00
0.2
0.3
0.5
04
0.6
08
1.0
0.25
0.50
0.75
1m
125
1.90
(g) Easy problem, wi Verification
(b) Medium problem, wi Verification
(
Hard problem, w/ Verification
Tusk Accrssy
0.5
0.15
0.0
3/0
0.9
1.
2.0
2.4
2.6
2.0
FLOPs Problem (1e15)
PLOPS Problem (1e!5)
FLOPS Problem (le15)
Figure 24: Using a verification metric to select compute-efficient generators across different problem difficulties. Qwen3-32B is used as the verifier, and generator candidates are Gemma2- 2B, 9B, and 27B. (a-c) Balanced accuracy the validation varies across difficulty ranges and identifies different preferred generators: the 2B model on easy problems, 9B on medium problems, and 27B hard problems. (d-f) TTS accuracy
compute the test set without verification. (g-i) TTS accuracy versus compute with verification. The generators selected by balanced accuracy correspond the compute-efficient choices: on easier problems, smaller generators achieve accuracy close to larger models while requiring fewer FLOPs, whereas on hard problems, the largest generator retains performance advantage.
2B
9B
27B
Compute-efficien choice
(a) Easy problem, Verifi. metrics
(b) Medium problem, Verifi. metries
(6) Hard problem, Verifi. metrics
Dalanced Accumiy
1.0
1.0
Hulanced
Dalanced
0.6
28
27B
2B
27B
28
278
(d) Easy problem,
(e) problem, Verification
problem, Verification
1.0
Task Asguncy
0.5
0.15
0.0
0.0
0.3
0.4
0.5
0.2
0.4
0.6
0.25
0.50
0.75
3.00
1.25
1.50
1.75
Easy problem, Verification
Medium problem, w/ Verification
(i) Hard problem, w/ Verification
1.0
Taok Accuncy
0.39
25
FLOP per Problem (1e15)
FLOPs per Problem (1e15)
FLOP per Problem (1e15)
Figure 25: Using a verification metric to select compute-efficient generators across different problem difficulties. GPT-4o is used the verifier; task accuracy is measured by pass rate. The experimental setups and observations are consistent with Figure 24
39

---

Published as a conference paper at ICLR 2026
Compute-effcieet choice
problem, Verifi. metrics
(b) Medium poblem, Verifi. metries
Hard problem, Verifi. metrics
Bahnedd Acurnyy
Roley AlAieayy
Baluced Aconacy
Relavee Aunnay
Balanced Acconcy
Bukinal Amiuns
0.4
0.5
301
(d) Easy problem, wio Verification
(c) Medium problemm, w/o Venfication
Hard problem, w/o Verification
04
Tork Acuurnty
03
0.6
0.2
0.25
0.50
0.15
3.00
1.25
3.50
1.75
(g) Easy problem, w/ Verification
(h) Medium problem, w/ Verification
(i) Hard problem, w/ Verification
1.0
Tusk Accurao
0.5
0.0
0.0
0.9
1.0
3.3
12
1.3
18
2.11
3.2
2.4
266
2.0
5.0
FLOPs Problem (lo15)
FLOPS Problem (1e15)
PLOPs Problem (1e13)
Figure 26: Using a verification metric to select compute-efficient generators across different problem difficulties. Qwen3-32B used as the verifier; task accuracy measured by majority vote. The experimental setups and observations are consistent with Figure 24
2B
9B
Compute-efficiet choice
(a) Easy problem, Verifi. metrics
(b) Medium problem. Verifi, metrics
(c) Hard problem, Verifi. metrics
1.3
1.3
Dulanced Ascaneyy
Relen Aiioeary
Rilaned Aunray
Bahancdd Accuncyy
Bobis Aroonyy
Beluncd
28
2B
2711
203
2713
(d) Easy problem, w/o Verification
(c) Medium problem, wro Verification
(1) Hard problem, w/o Verification
Task TackAscura Ascursyy
0.5
0.5
0.15
0.0
011
0.1
0.6
055
866
08.4
0.6
1.0
0.35
0.50
0.15
3.00
1.35
1.50
3.75
(g) Easy problem, Verification
Medium problem, Verification
Hard problem, Verification
1.0
Tunk Tirk Acouney
0.5
0.0
FLOPs per Problem (1e15)
FLOPs per Problem (1e15)
FLOPs per Problem (3e15)
Figure 27: Using a verification metrie to select compute-efficient generators across different problem difficulties. GPT-4o is used as the verifier; task accuracy is measured by majority vote. The experimental setups and observations are consistent with Figure 24
D.2.2 RQ5.1: How TO CHOOSE VERIFIER GIVEN FIXED GENERATOR?
We evaluate whether verification metrics can guide the choice between two verifiers (GPT-40 and Qwen2.5-7B) under different levels of problem difficulty. Following the procedure in Appendix D.2.1
40

---

Published as a conference paper at ICLR 2026
compute the verification metric the validation set and use it to select the verifier. TTS accuracy and computation cost are then reported on the test set.
Results of RQ5.1 The results are presented in Figure 28 pass rate) and Figure 29 (majority vote). For each figure, the first two subplots report TTS accuracy versus compute when using a small or large verifier. The final subplot reports the verification metrics.
On the hardest problem, GPT-4o yields only a small balanced accuracy improvement over Qwen2.3- 7B. The minimal metric difference suggests the smaller verifier. Evaluated on the test set, the accuracy difference between the two verifiers is small. It confirms that the stronger verifier offers limited additional benefit in this regime. On hard problems, GPT-4o shows clear gap of balanced accuracy over the small model. This difference points to choosing GPT-4o. On the test set, this leads to substantially improved TTS accuracy despite higher compute cost.
These results show that verification accuracy provides actionable guidance. When compute is limited, one can compare the balanced accuracy of the large and small verifiers: if the gap is small, the smaller verifier is the more cost-effictt cost-efficient choice; if the gap is large, the stronger verifier yields more substantial verification gains in TTS.
Veri Peraiiicatinn Balanced Accuracy
Hardest Problems: Task Accuracy by Pass Rate
(c) Hard Problemss: Task Task Accuracy by Pass Rate
0.80
014
3.0
Vedifle:: Qwa125.77
Vecifle: Qwen2.5
Vedifle: 5we225
Belanod Acourecy
0.75
Verffer: CPP)-l
0.5
wi Vertfization
Verifir: GP 1-4
Venifer CiPi--o
w/o Varifizaiion
0.70
Tack Accurey
Tink Accurey
0.6
A-14.9%
0.65
0.-
0.60
vestivations sto
veificeis
0.55
0.
0.50
Hardert
Hard
Problem Difficuity
FLOPs per Problem (le15)
FLOPS per Problem (le15)
Figure 28: Verffiation-mettii-uuuddd verifier selection across problem difficulty; task accuracy measured by pass rate. (a) Balanced accuracy on the validation set: when the metric gap between GPT-4o and Qwen2.5-7B is small (hardest problems), the smaller verifier is selected; when the gap is large (hard problems), the stronger verifier is selected. (b-c) Corresponding TTS accuracy on the test set: consistent with the metric-based choice, GPT-4o yields only small improvement on the hardest problems but provides substantially larger gains on the hard problems.
(6) Verification Metric Balanced Accuracy
Hardest Problems:
(c) Hard Problems: Task Accuracy by Majority Vote
Task Accuracy by Majority Vote
0.80
1,000
Verifar Qwant
Balancod Assuracy
0.75
Varifia: GPT-4
Verifia: GPT-4a
0.975
0.70
Accurccy
0.3
wis Veificetion
Task Accuracy
0.559
0.65
0.2
0.925
5.5% vattliction
verticater pap
0.60
4-3,
0.900
Verillen Qwenl
0.55
A
0.875
VeriGe: GPT-4o Web Veritlattio
Hundect
Hard
Problem Difficulty
FLOPss per Problem (le15)
FLOPs per Problem (1e15)
Figure 29: Veriffcation-metric-nnn verifier selection across problem difficulty; task accuracy is measured by majority vote. The experimental setups and observations are the same as Figure 28
D.3 ABLATION STUDY ON LARGE MODELS
We conduct ablation study using a large LLM with more than 100B parameters, Qwen3-235B (Qwen/Qwen3-2351) 2-3--m3--23--BBBBBB- A22B-Instruut-2500 temperature 0.7, top-p 0.8, and top-k 20. We repeat the experiments for RQI (Section 4.1 and RQ2
We use the recommended sampling hyperparameters:
41

---

Published as a conference paper at ICLR 2026
(Section 4.2) the Mathematics domain and report the results in Figure 30 The results show that our main findings remain consistent when using a larger model.
1.0
True Positive Rate
0.5
0.0
Hardest
Easy Problem Difficulty
Hard
Easiest
(a) RQ1 Trend of TPR.
Perforaance
0.1
Verifier
Qweri-335B
Qwerd 235B
0.0
031
AM
G2-38
L3.38
L2.0
M-4B
G2-48
62:27
023.18
Ma00
02.4-
L2-300
aPT-4e
Q23-720
Qf 40
Q3-88
Generator
(b) RQ2 Trend of TNR
Figure 30: Ablation study of our main findings with the large model Qwen3-235B, The experi- ments follow the same setup as Sections 4.1 and 4.2 and conducted on the Mathematics domain. (a) The trend that TPR increases as problem difficulty decreases is consistent with our RQ1 finding. (b) The trend that TNR decreases the generator strength increases is consistent with our RQ2
finding.
D.4 ABLATION STUDY ON VERIFICATION PROMPTS
We conduct an ablation study to evaluate the robustness of our findings under different verification prompts. All prompt templates used in this study are listed in Appendix A In addition to the original "Verification Evaluation Prompt," prompt from Zhang et al. (2024), and a "solve-then "solve-then-veriff" prompt that encourages the verifier to first solve the problem before evaluating the candidate response, strategy shown to be effective in prior work [Chen et al. (025d). These two prompts differ substantially from our original design.
consider two alternatives: concise "step-by-step" verification
The experiments follow the same setup Sections 4.1 and 4.2 and are conducted on the Mathematies domain. For each prompt, we apply Qwen2.5-72BB as the verifier to evaluate responses produced by all 15 generator models. The results in Figure 31 show that main findings regarding TPR and TNR dynamics remain consistent across all prompts, indicating that the observed verification dynamics robust prompt variations.
42

---

Published as a conference paper at ICLR 2026
1.0
1.0
Trie Positive Rate
Positive Rate
0.5
0.5
0.0
0.0
Hardest
Hard
Easy
Easiest
Hardest
Easy Problem Difficulty
Hard
Easiest
Problem Difficully
(a) RQI Trend of TPP (Solve Moole-thenn--e Verity)
(b) RQI Trend of RPPR (Step-by bteepp
Perfirmsncc Rate
0.0
0.1
02
03
04
05
066
0.7
0.8
0,9
10
Vosiee
Oesertitt
(c) RQ2 Trend TNN (Solve-then- Verify)
(d) RQ2 Trend TNR (Step-by-Step
Figure 31: Ablation study of findings under different verification prompts. (a, c) Using woole-theen-verify prompt for verification. (b, d) Using "step-by-step" prompt for verification. The trend that TPR increases difficulty decreases (RQ1) and TNR decreases the generator strength increases (RQ2) still holds under both prompts.
Table 2: Comparison of TTS performance and token cost per problem when using strong model as the verifier versus as the generator. Token count is reported on average per prompt.
Generator
Verifier
Pass Rate
Avg. GPT--40 Tokens
Avg. Qwen3-4B Tokens
Qwen3-4B
0.938
551
GPT-4o
0.952
482
Qwen3-4B
GPT-4o
0.954
193
551
D.5 COMPARISON ANALYSIS OF STRONG MODEL AS GENERATOR OR VERIFIER
We examine whether it is practical to strong model as verifier instead of using it solely generator. We compare two setups: (i) using GPT-4o as the generator, and (ii) using GPT-4o as the verifier together with a smaller generator, Qwen3-4B.
We use the same problem set as in Section 5.1 Mathematics problems with difficulty in the range 0..7,0.8), consisting of 181 problems. The TTS sampling size is 64. Task accuracy is measured using the pass rate, and efficiency is measured using the average tokens per prompt.
Here the findings. Both setups achieve the same performance, but using GPT-4o verifier consumes 193 tokens on average per prompt, whereas GPT-4o as generator uses 483 tokens per prompt average. This represents 2.5x reduction in token usage (and thus cost) per prompt. Thus, for practitioners who host small models but care about maximizing performance per dollar, leveraging the relatively short-form nature of verification with API models represents massive savings.
E CASE STUDIES
domain,
difficulty: Maihematics,
Hardest (d(x))
Generator: Qwen2,5.31
Verifier: GPT-4o
Question: Bob and Aliec each have bag that contains one ball of each of the colors, bluc,
red, and violet. Alicc
randomly selects one hall from her bag and purs into Bob's hag. Bob then randomly selects one hall from his hag and puts it into
Alice's bag. What
probability that after this process the contents of the bags the same?
bug, bag will the other colors. After Bob selects a ball and places it into Alice's is hag, the two hags will have the same contents if and only Boh
Ground- Truth Solution: After puts
each
43

---

Published as a conference paper at ICLR 2026
picked one the two balls in his bag that are the same color, Bocause there are balls in the bag when Bob makes his selection, the probability of seleeting one the same colored pair 2/6
Generator Output (Currect): To determine the probability that the contents of both Bob's and Alice's 's bags are the same after the process descrihed, Initially, each bag contains one hall of each of the colors: blue, green, oringe, red, and vinlet.
analyze each possible step and its impact on contents the bags.
#ae Step-by-Siep Step Analysis Details omitted |
#*e Identifying the Conditions for Equality Details omitted
#ae Calculating Probubility Details omitted | The probability then calculated as follows:
Probability Number lavorable outcomes
Total number of outcomes
3
Thus, the prubahility that after this process the contents of the bags the same
Verifier Output (False negative): Explanation: The problem involves two stops: Alice transferring ball Bob's bag and then Bob transferring ball back Alice's bag. Initially, both bags have one ball of each color. After Alice transfers ball, Bob's bag will have two balls of one color and one each of the other four colons, For the bags to identical afler Bob's 's trunsfer. Bob must return the only scenario where the hags will end up identical, restores the original
color hall that Alice initially transferred. This
distribation of colors both bags.
The assistant's 's response correctly identifies that there are possible outcomes for Bob's selection (since he has balls to choose from) and that there are favorable outcomes (retuming one the two balls the color that Alice transferred). However, the assistant incorrectly concludes that the probability 1/3. The probability should be 1/5, Bob must specifically relumn the ball the color that Alice transferre, which only specilic ball the in bag. [Verilier generates wrong answer 1/5 and compare with generator's correct answer 1/3]
Verdiet: [IInconrec]]
Prohlem domain, dataset and difficulty: Natural Langunge Reasoning, FOLIO, Hardest (d(xx) 0.122)
Generator: Llama3. 1-8B
Verifier: Qwen2.5.722
Question: Some mammals have teath. Platypuses have no teeth. Platypuses are mammals. Humans have teeth.
Based the above, the following be inferred? Platypuses reptiles
Ground-Trun Truth Solution: Uncertain
The statement that some mannals have teeth and humnns have leeth suggests that huving teeth is churacteristic of mammals, hut it's not exclusive to them. 2. Platypuses indeed mammals, stated in the information provided. 3. The fact that platypuses have no teeth does not contradict the fact that they are mammals, as having no teeth is also a characteristic
Generator Outpul (Correct): Reusoning: 1.
mammals, mentioned point The only group animals that not mammals and have been ruled out as category for platypuses are not the mammals, but rather the reptiles.
However, concluding that platypuses reptiles because they have leeth not sound inference. because having teeth does not preclude an unimal from heing a mammal, as shown point 3. Moreover, it contrudicts the given staterent that platypuses are indeed mammals.
5. Therefore. platypuses being reptiles cannot be inferred based the given information.
Answer: Uncertain
Verifier Outpui (False negutive): Explanation: The assistant's 's response is mostly correct in its reasoning but contains a minor inacouracy in the final conclusion. The assistant correctly identifies that the fact that platypuses have no teeth does not make them reptiles, as some mammals also lack teeth. However, the assistant's final answer of "Uncertain' is not appropriate. Given the information provided, it is clear that platypuses are mammals,
there no evidence to suggest they are reptiles. Therefore, the correct answer should because statement "Platypuses reptiles" can be definitively refuted hased given information, [Although verifier recognizes, that generator's 's reasonnng process correct, rejects answer.]
mismatch own (incorrect)
Verdiet: [[Incomret]]
Figure 32: Case study: How problem difficulty impacts TPR (RQ1). False negative verification on high-difficulty problems demonstrates why TPR decreases as problems become harder. When verifiers cannot correctly solve challenging problems, they produce incorrect reference solutions and reject generators' correct answers based on these flawed references.
Problem domain, dataset and difficulty: Mathematics, MATII-500, Easy (d(x)) 0.571)
Generator: Llama-3.1 3.1-8B
Verifier: Qwen2.5-722 7-B
Question: Paul and Jesse each choose a numher at raddom from the first six primes. What is the probability that the sum of the numbers they choose even?
Ground-Trun Truth Solution: [ Derails omitted Therefore, the probability that the sum of the numbers Paul and Jesse choose IS even is
13
44

---

Published as a conference paper at ICLR 2026
Generator Output (Incorrect); To find the probability that sum of the numbers Paul Jesse choose is even, we need first understand what numbers make
The sum numbers is both
are either even odd.
Since the first primes are 2, 3, 7,
13, we have only one even prime, which
Now, let's s consider the possible scenarios:
1. Both Paul and Jesse choose the number 2. In this case, the sum is even.
2. of or Jesse chooses nomber 2, and other chooses an odd prime. case, the sum is odd.
3. Both Paul and Jesse choose odd primes. this case. the is (Generatoor correctly identilies two for for odd sum.]
[Probobdiiyy calculations omiited onvitted]
Now, let's scenario (both
probahilities of
scenarios choose 2, and chooses and other an odd prime)
odd prime) get total probability an even stim:
Probability 1/36+ + 10/36 25/36 Probability 36/36 Probability = The probability that the of the numbers Paul and Jesse choose
is 100%. This is because impossible for the numbers two odd numbers. [Generator makes self contradiction: Farlier stated and claims odd sum impossible]
he odd, they will always chonse either
scenario 2 gives odd sum. count scensrio
Verifier Output (True negative); The assistant's response contains logical identifies the scenarios where the of the numbers chosen by Paul and Josse is even. However. the assistant incorrectly adds the probabilities of sccnarios, including sccuario where calculation. [Verifier catches the solf-contraiiction generator.]
the final calculation. The assistant correctly
odd, which should not be included in the final probability
correct calculation should only odd primes: 25
probahilities the scenarios where
even: Roth choose 58
The probability of the sum being even is the sum of thewe Lwo probabilities:
25
26
13
36
36
36
18
13
Therefore, the final answer should be
[Verifier solve the problem correctl]]
18
Verdict: [[Incomec]]]
Same problem and verifier above with strong generator:
Generator: GPT-4a
Verifier: Qwen2.5-72B
Generator Output (Incorrect): To find the probubility that the sum the numbers chosen by Paul and Jesse from the first six prime
numbers even, proceed follows:
***isst first prime =umbers"*;
six prime numbers 3, 7, and 13.
2. *Identiff even and odd primes**:
Even prime:
Odd primes: 5, 11, 13
3. **nnerrtaadd when the even**:
even numbers even both numbers are odd.
Since there only one even prime (2), for sam he even, hoth numbers must he odd, [Fatal error: Tgnores both choosing
(even=cvenmeven) is also possible.
4. *CCalculate the total number of ways to choose numbers**; + | Calculation details omitted omitted
5. Callculat the number of ways to gel an even sum**; | Calculation details omitted omitted
6. *Calculate the probability**: + Calculation details omitted omitted
25
Thus, the probability that the sum of the numbers they choose is even is
[Final answer wrong becouse Tollowing steps
consistently based fotal error made above.]
Verifier Output (False positive): Explanation: The assistant's 's response logically breakdown of the steps:
mathematically accurate. Here's 1. "*Identifying the first six primes** **; The assistant comectly identifies the first six prime numbers 5, 7, and 13.
2. **Condition
"ven"*; The assistant correctly states that number odd). |Verilier convinced by generator and
even, both numbers must odd
only even prime is choosing
the sum odd
ignores the emror here.]
3. **Total number possible outcomes** + The assistant correctly calculates the total number of outcomes as 6 36.
4. *Number of outcomes where both numbers are editte; The ussistant correctly identifies that there are 5 odd primes and culculates
the number of outcomes where both numbers are odd 5 25.
5. **Probability calculation**. * The assistant correctly calculates the probability as the number favorable outcomes (25) divided
the total number of outcomes (36), resulting in 50
The assistant's 's reasening and calculations are correct Uhrogbout response. lAlthough verifier demonstrated correct problem- solving when evalunting Llama3. incorrect reasoning.]
example, doesn't I apply the same mechanism GPT-4o, instead accepting GPT-4o's
Verdict: [[Correat]]
45

---

Published as a conference paper at ICLR 2026
Figure 33: Case study: How generator capability influences TNR (RQ2). We compare verifier behavior on the same problem when evaluating weak (Llama3,1-8B) versus strong (GPT-4o) genera- tors. The weak generator correctly lists all the cases but then contradicts itself in the final calculation, producing an inconsistent solution. The verifier catches this apparent mistake and rejects The strong generator makes single early-stage emror but maintains internal consistency throughout its reasoning, generating a well-structured and self-contained chain that appears correct. Despite the solution containing actual error, the verifier fails to detect it and accepts the solution.
46


Reviews:
[{"review_id": "79aav8SRIS", "reviewer": "Reviewer_PDQk", "summary": "The paper presents an experimental study of generative verifiers across 15 models of diverse sizes on 12 benchmarks.\nIn their extensive evaluation, they make the following observations:\n1. Problem difficulty has an inverse correlation with the True Positive Rate (TPR) of the verifier, i.e., harder problems have a lower TPR of the verifier, while this is not the case for the True Negative Rate (TNR).\n2. The capability of the LLM in answer generation correlates positively with the error detection capability in verification, and errors made by stronger LLMs are harder to detect than those made by weaker LLMs.\n3. Verifier capability is positively correlated with verification performance, but it is non-linear for very easy and very hard problems.\n\nThe paper further makes the following observations when applying these findings to Test-Time scaling (TTS): \n\n4. A weak generator can benefit much more from a strong verifier.\n\n5. Test time scaling helps especially for medium difficulty problems, and less or not for very easy or very hard problems.", "questions": "1. Plot 1(b) and corresponding observation: Is the hidden variable in this plot not the problems that a generator cannot solve, and in this sense, related to the problem difficulty?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- overall solid experimental evaluation and setup (some limitations see below).\n- Overall very extensive study w.r.t. datasets and models.\n- A very well-structured formulation of 5 research questions and answering them experimentally.", "weaknesses": "## Major\n1. Some of the findings are maybe unsurprising or it is difficult to benefit from the the findings in practice. E.g. Using a very strong verifier (GPT 4o) on weaker answer models seems of limited practical use.\nOr that verifiers do not provide much benefit for very easy or very difficult problems: while interesting, the paper does not examine how to reliably identify very hard and very difficult problems, but is able to group by difficulty based on ground truth, in practice one is likely confronted with easier and more difficult problems without knowing which ones are easier and which ones are more difficult.\n2. RQ4: Is it relevant to verify a weak model with a much stronger verifier? Given the cost of verification (see e.g. Singhi et al., 25), would GPT 4o (w/o verification) already be as good or better? Or just use multiple GPT 4o answerers.\n2. Related to the previous point, I miss the point GPT 4o in the x-axis in Figures 5 (a), (b).\n\n\n## Minor\n4. Most References are given as arXiv versions. If they are published, the published version should be cited.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "aK9NyhfsDN", "reviewer": "Reviewer_TAxB", "summary": "This work presents an empirical study on the verification dynamics and its implications to TTS. The analysis is centered around problem difficulty, generator and verifier capability. The main findings of this work: verifiers works better on easy problems, weak generators, and further, verification performance is affected by verifier generation capability with a different trend dependent on problem difficulty. The TTS applications inspired by these findings focus on matching the gap between weak and strong generators, and verifiers respectively, which provides guidance of effective TTS design.", "questions": "Q1. in Figure 2, are verifiers and generators the same? somehow i missed this information.\n\nQ2. there seems a typo in the caption of Figure 5: why are (a-c) and (d-f) mentioned?\n\nQ3. in the final paragraph, line 482, \"as weaker generators can approach the post-verification performance of stronger ones when paired with a fixed verifier.\", isn't there a specific condition, e.g. decently strong fixed verifier like gpt-4o, otherwise the weak verifier cannot close the gap between weak and strong generators?\n\nQ4. in the paragraph of line 371, \"Verifier generation capability influences verification accuracy differently based on problem difficulty\", how does the difficulty defined here? my understanding is that the questions are split based on the generation capability (i.e. x-axis), but why are there overlaps between the three difficulty groups?\n\nQ5. Figure 9 (d-f) shows a different trend of reasoning LLMs from instruct models, does the main discussion in RQ1 still hold? \n\nQ6. most evaluated models are less than 100B, why? do authors think the same trend will still hold for larger models such as deepseek V3 or Qwen-235B?", "limitations": "", "rating": 2, "confidence": 2, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "The results are comprehensive, including the wide range of evaluated models (instruct models and reasoning models as discussed in appendix) and discussed aspects from problem difficulty to generator/verifier capability.", "weaknesses": "a). one main theme of the analysis is problem difficulty, but in reality, we do not know the difficulty labels beforehand, then how do we know if a question is on the difficulty extreme where verifier engineering is shown to be less effective? so its practicality on TTS applications is a bit limited in my opinion.\n\nb). efficiency analysis is lacking: is there a compute-optimal TTS approach given a task and a list of candidate generators/verifiers. i imagine something like [1] can be very helpful here.\n\nc). authors should improve task accuracy analysis, besides the verification performances, especially in the TTS applications part. so that the gap between oracle accuracy and verified accuracy can be compared and see how promising each combination is. (is this reported as pass rate in Figure 5a? not entirely follow the metric in the figure) Further, how to choose the best verifier/generator configuration given a task?\n\nd). last but not the least, this work focus on generative verifiers from a LLM-as-a-judge perspective, without fine-tuning; but prior work [2] mentioned that vanilla LLM doesn't perform well on verification. i think the authors should compare the effectiveness of the proposed approach (or the converged optimal selection as discussed above) with the SOTA verifiers beyond LLM-judge, e.g. the top ones on rewardbench.\n\n[1] https://arxiv.org/abs/2502.06703\n[2] https://arxiv.org/abs/2408.15240", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "exgClUAAKd", "reviewer": "Reviewer_QrzN", "summary": "This paper presents a empirical study on the dynamics of LLM-based verification for test-time scaling. The authors investigate how problem difficulty, generator capability, and verifier generation capability influence verification performance across mathematical reasoning, knowledge, and natural language reasoning tasks. Key findings include that easy problems allow verifiers to certify correct responses more reliably, errors from weaker generators are easier to detect, and verifier capability correlates with performance in a difficulty-dependent manner that varies by problem difficulty.", "questions": "1. How might your findings apply to other verifiable domains, such as coding?\n2. The paper highlights distinct non-linear correlation patterns between verifier generation capability and verification performance (e.g., saturated, linear, threshold-limited). Could the authors offer a more in-depth theoretical or empirical analysis of why these different regimes emerge across varying problem difficulties?\n3. Did you explore different verification prompting strategies? How might alternative verification frameworks affect the observed dynamics?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "1. The paper conducts a thorough investigation across three crucial dimensions (problem difficulty, generator capability, verifier capability) and evaluates performance on a wide array of 12 benchmarks using 14 diverse LLMs.\n2. The paper details its experimental setup, metrics, model and prompts, ensuring high reproducibility.\n3. The findings provide concrete guidance for optimizing TTS applications, such as identifying scenarios where weaker generators can achieve performance comparable to stronger ones.", "weaknesses": "1.  Despite conducting extensive analytical experiments, many conclusions remain relatively intuitive rather than providing deeper theoretical insights. For instance, the findings that \"easy problems allow verifiers to more reliably certify correct responses\" and \"weak generators produce errors that are easier to detect than strong generators\" align with common intuition but lack mechanistic explanations about why these patterns emerge.\n2. The use of pass rate as the sole difficulty metric (d(x)) may oversimplify the complex nature of problem difficulty, potentially missing important nuances in what makes problems challenging for verification.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "HJUp7Ho4k0", "reviewer": "Reviewer_1wQz", "summary": "This paper investigates the factors that influence verification performance when large language models (LLMs) are used to assess the correctness of solutions generated by other LLMs. The authors conduct a systematic empirical study across three dimensions: problem difficulty, generator capability, and verifier generation capability. Using 12 benchmarks spanning mathematical reasoning, knowledge QA, and natural language reasoning, they evaluate 14 open-source models and GPT-4o in both generator and verifier roles. The study reveals three key findings: (1) verifiers more reliably certify correct responses on easier problems, (2) weak generators produce errors that are easier to detect than those from strong generators, and (3) verification ability correlates with verifier generation capability in a difficulty-dependent manner. The authors demonstrate practical implications for test-time scaling, showing that weak generators can nearly match strong generators when paired with the same verifier, and identifying regimes where strong verifiers provide limited advantages over weak ones.", "questions": "1. How sensitive are the main findings to the choice of prompt templates for generation and verification? Did you experiment with alternative prompting strategies, and if so, how did they affect the observed patterns?\n2. In Section 4.1, you show that problem difficulty primarily affects TPR. However, for reasoning models in Appendix C.2, TNR also increases with easier problems. Can you provide more insight into why extended reasoning changes this relationship?\n3. The definition of problem difficulty as average pass rate across generators may be influenced by the specific set of models used. How would the findings change if you used a different set of generators or defined difficulty based on human performance?\n4. For the TTS experiments in Section 5, you use K=64 samples per problem. How sensitive are the verification gains to this choice? Is there a point of diminishing returns where additional samples provide minimal benefit?\n5. The paper identifies regimes where strong verifiers provide limited additional benefit (Section 5.2). Are there alternative verification strategies (e.g., multi-verifier consensus, uncertainty-aware verification) that might overcome these limitations?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The paper addresses an important and timely problem, as LLM-based verification is increasingly central to test-time scaling methods and the efficient deployment of language models in production systems.\n- The experimental design is comprehensive and well-controlled, systematically varying problem difficulty, generator capability, and verifier capability across 12 benchmarks and 15 models to isolate the effect of each factor.\n- The finding that generator capability primarily influences error detectability (TNR) rather than correct response recognition (TPR) is novel and provides actionable insights for model selection in verification pipelines.\n- The case studies in Figures 20 and 21 effectively illustrate the mechanisms behind the main findings, showing how problem difficulty affects TPR through reference solution quality and how generator capability affects TNR through solution consistency.\n- The practical implications for test-time scaling are clearly demonstrated, showing substantial performance gap reductions between weak and strong generators when using the same verifier (Section 5.1 and Figure 17).\n- The paper identifies regimes where strong verifiers provide minimal additional benefit over weak verifiers, which has important cost-efficiency implications for practitioners deploying verification systems at scale.\n- The difficulty-dependent correlation analysis in Section 4.3 reveals nuanced patterns (saturated, linear, threshold-limited) that prior work assuming simple linear relationships overlooked.", "weaknesses": "- The paper lacks theoretical analysis or mechanistic explanations for the observed patterns, relying entirely on empirical observations without providing deeper understanding of why these verification dynamics emerge.\n- The definition of problem difficulty as average pass rate across generators is model-dependent and may not capture intrinsic problem characteristics, potentially confounding difficulty effects with model-specific biases.\n- The experimental scope is limited to verifiable problems with objective ground-truth answers, which excludes important domains like open-ended generation, creative writing, and subjective evaluation tasks where verification is also needed.\n- The paper does not address the computational costs of verification, which is a critical practical consideration given that verification gains may not justify the additional compute in all scenarios.\n- The analysis of reasoning models (Appendix C.2) is relegated to the appendix despite showing different TNR behavior, suggesting that the main findings may not fully generalize to this increasingly important class of models.\n- The selection of specific prompt templates, sampling hyperparameters, and verification evaluation procedures is not thoroughly ablated, leaving uncertainty about the robustness of findings to these design choices.\n- The paper identifies regimes where verification provides limited gains but does not propose actionable strategies for overcoming these fundamental limitations beyond avoiding those regimes.\n- Some experimental details are missing or unclear, such as the exact procedure for handling invalid verifier outputs, the rationale for the specific 4 correct/4 incorrect sampling strategy, and how the fallback LLM-based correctness checking affects results.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)