Title: Uncertainty-Aware Search and Value Models in LLMs
Primary area: applications to computer vision, audio, language, and other modalities
Keywords: uncertainty, search, verifier

Abstract: Value model-guided search is effective in steering the generation but suffers from verifier failures: imperfect verifiers may mistakenly prune all the valid paths. This limitation could arise from reliability degradation in value models in unseen reasoning paths. To address this, we propose an uncertainty-aware search framework that includes two key components: (1) uncertainty-aware value models that incorporate uncertainty into predictions, and (2) an uncertainty-aware selection process using the proposed efficient Group Thompson Sampling algorithm. Experiments on two In-Distribution (ID) settings (GSM8K and MATH) and three Out-Of-Distribution (OOD) settings (e.g. AIME25 and Minerva Math) show that our method mitigates verifier failures. This work establishes the first systematic integration of uncertainty quantification in LLM search paradigms.

TL;DR: we introduce uncertainty qualification to mitigate the verifier failures