Title: A Bias–Variance Tradeoff Perspective for Improving Test-Time Scaling
Primary area: foundation or frontier models, including LLMs
Keywords: Test-time scaling, probabilistic inference

Abstract: Parallel Test-time scaling (PTTS) improves the mathematical reasoning performance of large language models (LLMs) by aggregating multiple candidate solutions at inference time.  Existing work improves TTS either by introducing an external verifier to guide candidate aggregation or refining the generator to produce better-quality candidates. However, an intrinsic and principled perspective for systematically comparing the pros and cons of these methods is still lacking. To bridge this gap, we propose a general probabilistic inference framework that recasts existing methods through the lens of the bias–variance tradeoff. This perspective reveals that verifier-based methods correspond to importance sampling (IS) estimators, which exhibit unbiasedness but suffer from high variance, whereas generator-based methods correspond to variational inference (VI) estimators, which achieve lower variance but introduce systematic bias. To mitigate this tradeoff, we propose TSMC-TTS, which employs twisted sequential Monte Carlo (TSMC) as the inference algorithm to effectively reduce variance while preserving unbiasedness. The generator and verifier are jointly optimized based on a theoretically derived upper bound to achieve monotonic variance reduction without compromising unbiasedness. Rigorous theoretical analysis and comprehensive experimental results demonstrate the efficacy of our proposed method.