Title: On the Scaling Flaws of Verifier-Guided Beam Search
Primary area: applications to computer vision, audio, language, and other modalities
Keywords: inference-time compute scaling, beam search, verifiers, ovm, prm

Abstract: Large language models (LLMs) struggle with multi-step reasoning, for which inference-time scaling—via sequential or parallel scaling—has emerged as a promising strategy. While sequential scaling (exemplified by O1 and DeepSeek-R1) has recently achieved substantial success, it remains unclear why parallel scaling, such as verifier-guided beam search, has lagged behind. In this paper, we argue that its strength is, paradoxically, also its limitation: verifiers can boost performance under limited sample sizes by elevating promising reasoning paths, yet the same mechanism can also hide or cut off the valid paths that lead to correct answers. Empirically, we uncover a systematic issue--scaling flaws--in verifier-guided beam search, across models, benchmarks (GSM8K, MATH, AIME25), and verifier types (outcome value models, process reward models). Specifically, the search outperforms repeated sampling at small sample sizes but its advantage diminishes—and ultimately reverses—as the sample size grows. We attribute this to verifier failures: imperfect verifiers misrank candidates and can erroneously prune all valid paths, with these effects exacerbated on more challenging scenarios. To mitigate verifier failures, we explore reducing reliance on verifiers and conduct preliminary investigations using two simple methods.
Overall, our findings expose fundamental limitations of verifier-guided beam search and  explain why this line has struggled to realize its potential.

TL;DR: scaling flaws hinder the effectiveness of verifier-guided beam search