Scaling Test-time Compute for LLM Agents
Authors: name: King Zhu, name: Hanhao Li, name: Siwei Wu, name: Tianshun Xing, name: Dehua Ma, name: Xiangru Tang, name: Minghao Liu, name: Jian Yang, name: Jiaheng Liu, name: Yuchen Eleanor Jiang, name: Changwang Zhang, name: Chenghua Lin, name: Jun Wang, name: Ge Zhang, name: Wangchunshu Zhou

Scaling test time compute has shown remarkable success in improving the
reasoning abilities of large language models (LLMs). In this work, we conduct
the first systematic exploration of applying test-time scaling methods to
language agents and investigate the extent to which it improves their
effectiveness. Specifically, we explore different test-time scaling strategies,
including: (1) parallel sampling algorithms; (2) sequential revision
strategies; (3) verifiers and merging methods; (4)strategies for diversifying
rollouts.We carefully analyze and ablate the impact of different design
strategies on applying test-time scaling on language agents, and have follow
findings: 1. Scaling test time compute could improve the performance of agents.
2. Knowing when to reflect is important for agents. 3. Among different
verification and result merging approaches, the list-wise method performs best.
4. Increasing diversified rollouts exerts a positive effect on the agent's task
performance.