Strategist: Learning Strategic Skills by LLMs via Bi-Level Tree Search
Authors: name: Jonathan Light, name: Min Cai, name: Weiqin Chen, name: Guanzhi Wang, name: Xiusi Chen, name: Wei Cheng, name: Yisong Yue, name: Ziniu Hu

In this paper, we propose a new method Strategist that utilizes LLMs to
acquire new skills for playing multi-agent games through a self-improvement
process. Our method gathers quality feedback through self-play simulations with
Monte Carlo tree search and LLM-based reflection, which can then be used to
learn high-level strategic skills such as how to evaluate states that guide the
low-level execution.We showcase how our method can be used in both action
planning and dialogue generation in the context of games, achieving good
performance on both tasks. Specifically, we demonstrate that our method can
help train agents with better performance than both traditional reinforcement
learning-based approaches and other LLM-based skill learning approaches in
games including the Game of Pure Strategy (GOPS) and The Resistance: Avalon.