Title: Rethinking Optimal Verification Granularity for Compute-Efficient Test-Time Scaling
Venue: NeurIPS 2025
Accepted: yes
Keywords: Large Language Models

Test-Time Scaling

Efficient Machine Learning
Abstract: Test-time scaling (TTS) has proven effective in enhancing the reasoning capabilities of large language models (LLMs). Verification plays a key role in TTS, simultaneously influencing (1) reasoning performance and (2) compute efficiency, due to the quality and computational cost of verification. In this work, we challenge the conventional paradigms of verification, and make the first attempt toward systematically investigating the impact of verification granularity—that is, how frequently the verifier is invoked during generation, 
beyond verifying only the final output or individual generation steps.
To this end, we introduce Variable Granularity Search (VG-Search), a unified algorithm that generalizes beam search and Best-of-N sampling via a tunable granularity parameter $g$. Extensive experiments with VG-Search under varying compute budgets, generator-verifier configurations, and task attributes reveal that dynamically selecting $g$ can improve the compute efficiency and scaling behavior. Building on these findings, we propose adaptive VG-Search strategies that achieve accuracy gains of up to 3.1\% over Beam Search and 3.6\% over Best-of-N, while reducing FLOPs by over 52\%. We will open-source the code to support future research.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors raise a new line of tuning during test-time compute inference for large language models: the frequency and strength of a *verifier*, which assigns a score to a reasoning step performed by an LLM (often from a pretrained reward model.) The authors demonstrate that existing methods of verification in TTC lead to a number of inefficiencies, most salient of which is a computational inefficiency where verification steps at too fine a granularity lead to wasted FLOPs with little information gain (since consecutive reasoning steps often highly correlate with correctness). To address this, the authors systematically analyze how varying verification granularities within their search technique, VG-Search, can impact computational efficiency and accuracy. The authors show that a granularity in-between every-step and once at the end often leads to the best performance in trading off compute available to the LLM itself v. the verifier. The authors also introduce adaptive VG-search, where compute or accuracy constraints can guide the search for the optimal verification granularity, and show that these found granularity values effectively trade off less compute or higher accuracy in comparison to existing (naive) methods.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\n* I find that the introduction of the new axis of verification granularity is valuable as a standalone contribution and important in test-time compute optimization. This parameter is something that the community should focus on and bring into existing TTC optimization methods to further analyze and improve performance.\n* The paper is broadly quite well written and easy to follow. The contribution is straightforward, but I view this as a strength: an intuitive insight into further optimizing TTC can bring about expected and desired tradeoffs in compute and accuracy.\n* The experimental results do tell a good story that backs up the claims of the authors. I found especially interesting that there was not always a consistently best granularity in Figure 3 when fixing the strength of the generator which strongly motivates the methods developed in section 5.\n* Sections 3-4 (and most of 2) are very detailed and clear from a technical standpoint without becoming pedantic or unnecessarily rigorous.\n\nWeaknesses:\n* There are some details that still lead to confusion on my end that I'd like to have clarified (see questions section.)\n* I felt the toy example in line 74 is not the demonstrably best choice. With infinite monkeys, there is almost certainly a need for a very fine granularity when it comes to verification since they’ll probably get the next step wrong - perhaps a different example would more strongly motivate the need for a tradeoff.\n* Section 2.3 is not particularly insightful and could probably be condensed to just a few lines. it’s pretty easy to follow how a reduction of verifier calls can scale down test time compute.\n* The choice of generator and verifier pairings in section 4 don't make a ton of sense to me and I'd like to see more pairings. At the very least, seeing performance by the most performant generator paired with the most performant verifier would be valuable here.\n* The biggest issue I have is with section 5, which I think is much too short given its significance with respect to the rest of the paper and lacks many clarifying details. More analysis here would really strengthen the paper. Namely: how long do the AM-g and CM-g methods take to converge? What is the computational overhead of finding the optimal g* before actually deploying that on the test/validation set? I assume that the reported numbers are all after g* is actually found, but finding this g* always practical? Is it a fair comparison if these numbers don’t include that burn-in cost to find the optimal g*? And if the reported numbers don't report the computational overhead required during the g* search process, there should be a thorough discussion and analysis on when and how to use g* during actual execution. When do we reuse g* during live inference? When do we compute a new g*? Is it for each finetuned model? What if a finetuning dataset has subsets that may require different g*?\n\nMinor:\n* Lines 100-102: Variable names B1 and B2 are somewhat obscuring naming choices\n* The lead takeaway in line 168 isn't the most insightful. I'd instead lead with the takeaway regarding how different granularities are optimal across different pairings/baselines and characterize why this might be the case.\n\nOverall, I think the new line of insight presented by the authors is something that the NeurIPS and broader community should pay attention to. With that being said, I think the work can be greatly strengthened with additional clarity and analysis and my score will reflect that.\n\nQUESTIONS:\n* Can you explain why rejected intermediate steps in beam search get expanded on in Fig. 2? I thought that rejected steps shouldn't be further expanded on by beam search.\n* The usage of the verifier is not fully clear to me (as mentioned in the intro and then again on line 67.) What threshold do you use to decide whether to accept or reject a reasoning step?\n* Can you provide more intuition in line 116 regarding the choice of the simpler algorithmic design: is it because some solutions just start out as incorrect and you want to catch these early?\n* Line 256: How does epsilon get selected? \n* See questions regarding section 5. Do the reported numbers don't report the computational overhead required during the g* search process?\n\nLIMITATIONS:\nThe limitations are briefly spelled out in the appendix. One of the limitations discussed brings about the same (similar) point I made in my comments on Section 5.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-25"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis work studies LLM test-time scaling for reasoning tasks by repeated sampling and verification.\nThe authors argue that dense verification in standard beam search can be wasteful,\nand propose a generalized variant that allows verification at variable granularity:\nfor each branch, generate $g \\ge 1$ steps before verification and pruning.\nEmpirical results confirm that higher accuracy and efficiency can be achieved by selecting the granularity properly.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths:**\n\n* The proposed method is sensible, simple yet effective.\n* The empirical results are promising.\n* The overall writing is clear.\n* There are some good explanations, intuitions and insights, for example:\n    * The authors spot redundant verification in beam search, as shown in Figure 1;\n    * Lines 168 - 184 explain why stronger generators can benefit more from sparse verification;\n    * Lines 202 - 213 explains why dense verification at each step is likely sub-optimal.\n\n\n**Weaknesses:**\n\nThere is room for improvement in experiments, for example:\n\n* Empirical evaluation is done with limited model scale, up to 7B. Nonetheless, the rationale of the proposed method seems to hold regardless of model scale.\n* Each experimental result is obtained with a single trial, rather than averaged over multiple independent trials. As a result, some curves are a bit messy (e.g., Fig 1(d) and Fig 3), and small differences (e.g., 1% in accuracy) might be partially due to randomness of experiments.\n* A related issue is that, for Question 7 in the checklist regarding statistical significance, the authors say \"Our main results do not contain randomness as we fixed the seed\". This is obviously not a valid argument (as the choice of seed itself can be random); otherwise, any empirical study can refrain from running multiple trials and reporting statistical significance, by the same argument.\n\n\nNonetheless, it is understandable that computational costs can be high for additional experiments with larger model scales or larger number of trials.\n\nQUESTIONS:\nNone\n\nLIMITATIONS:\nYes, the authors discussed in Appendix A.4 that the hyperparameter g (granularity) is chosen once (e.g., by optimizing performance on a validation set) and then fixed throughout at inference time.\nFinding a way of adaptively setting this hyperparameter may require further work.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper makes the observation that the two most prominent Test-Time-Scaling methods, Beam Search, and Best-Of-N, can be unified into a single framework, where the “granularity” (or frequency of verification steps) is varied. Beam Search invokes the verifier at each step (granularity 1), and Best-Of-N invokes the verifier at the end of each generation. The paper very reasonably questions whether a verification frequency between the two might yield more cost effective TTS, and introduces VG-Search, which parametrizes the granularity. The main findings are that stronger generators, easier tasks, and larger sample budgets tolerate higher granularity.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n- The paper addresses a novel and significant aspect of verifier-guided test time scaling, which is the frequency of verification steps (granularity.) It frames existing methods (Beam-Search and Best-Of-N) as two extremes of a unified perspective, and introduces VG-Search to interpolate between the methods.\n- VG-Search gives a practical way to improve the accuracy-compute trade-off, with up to a 3.6% accuracy gain or a 52% reduction in FLOPs.\n\nWeaknesses:\n- The \"atomic unit\" of generation and verification is based on newline characters, whereas it may be useful to consider more semantically meaningful notions of a \"step\" (this is acknowledged by the authors.)\n- Beyond varying granularity by task difficulty, it would be interesting to consider varying granularity dynamically during execution of a task or within a generation process.\n\nQUESTIONS:\n- Could you give more detail on how a dynamic adjustment of granularity might be implemented? For instance, could the verifier's confidence score on a partial solution be used as signal to determine he length of the next generation segment before the next verification?\n- Your experiments are focused on mathematical reasoning. How do you expect your findings on optimal verification granularity to generalize to other domains, such as creative writing, code generation, or long-form question answering?\n- The analysis assumes an accurate Process Reward Model (PRM). How would the best granularity change if the verifier itself is noisy or poorly calibrated?\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes Variable Granularity Search (VG-Search), a verifier-guided decoding algorithm that introduces a tunable granularity parameter g to control how often a reward model is queried.\nIt groups candidate sequences into segments, queries the verifier only at segment boundaries, and enables budget-aware tuning of verification frequency.\nExperiments on MATH-500, AIME, and a validation subset shows that the optimal g depends on compute budget and generator-verifier strength; an adaptive strategy gains up to 3.1 pp accuracy over beam search and 3.6 pp over Best-of-N while cutting FLOPs by more than 52%.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n1. Proven scaling and effective extended framework. VG-Search generalizes two popular test time scaling methods with a single hyperparameter and supplies an explicit FLOPs model, enabling great trade-off analysis across budgets and model pairs.\n2. Complete and comprehensive evaluation. The study sweeps g, branch factor, verifier size, task difficulty, and sample count, and validates both compute savings and accuracy gains with clear ablations and an adaptive policy that outperforms strong baselines on multiple datasets.\n3. Detailed design analysis via experiments. The profiling further reveals that strong generators prefer sparse verification whereas weaker ones need dense checks, and that investing saved compute in a larger verifier is more beneficial than raising the branch factor.\n\nWeaknesses:\n1. This paper just names its checker “Skywork-o1 PRM” and states it keeps the last score, but it never tells its model size, training data, or prompt, making the speed and bias unclear.\n2. No comparison with alternative verification paradigms. Although the related-work section recognises outcome-level verifiers, step-wise PRMs, generative PRMs and multi-verifier ensembles, the experiments test just a single discriminative PRM. Nevertheless, it's still positive to observe the test-time scaling on it.\n\nQUESTIONS:\nQ1: How would VG-Search handle tasks that supply only outcome-level natural-language feedback rather than deterministic unit tests?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-03"}