Scaling Test-time Compute for LLM Agents
Authors: name: King Zhu; name: Hanhao Li; name: Siwei Wu; name: Tianshun Xing; name: Dehua Ma; name: Xiangru Tang; name: Minghao Liu; name: Jian Yang; name: Jiaheng Liu; name: Yuchen Eleanor Jiang; name: Changwang Zhang; name: Chenghua Lin; name: Jun Wang; name: Ge Zhang; name: Wangchunshu Zhou
Published: 2025-06-15 17:59:47

Abstract: Scaling test time compute has shown remarkable success in improving the
reasoning abilities of large language models (LLMs). In this work, we conduct
the first systematic exploration of applying test-time scaling methods to
language agents and investigate the extent to which it improves their
effectiveness. Specifically, we explore different test-time scaling strategies,
including: (1) parallel sampling algorithms; (2) sequential revision
strategies; (3) verifiers and merging methods; (4)strategies for diversifying
rollouts.We carefully analyze and ablate the impact of different design
strategies on applying test-time scaling on language agents, and have follow
findings: 1. Scaling test time compute could improve the performance of agents.
2. Knowing when to reflect is important for agents. 3. Among different
verification and result merging approaches, the list-wise method performs best.
4. Increasing diversified rollouts exerts a positive effect on the agent's task
performance.