Think in Games: Learning to Reason in Games via Reinforcement Learning
  with Large Language Models
Authors: name: Yi Liao, name: Yu Gu, name: Yuan Sui, name: Zining Zhu, name: Yifan Lu, name: Guohua Tang, name: Zhongqian Sun, name: Wei Yang

Large language models (LLMs) excel at complex reasoning tasks such as
mathematics and coding, yet they frequently struggle with simple interactive
tasks that young children perform effortlessly. This discrepancy highlights a
critical gap between declarative knowledge (knowing about something) and
procedural knowledge (knowing how to do something). Although traditional
reinforcement learning (RL) agents can acquire procedural knowledge through
environmental interaction, they often operate as black boxes and require
substantial training data. In contrast, LLMs possess extensive world knowledge
and reasoning capabilities, but are unable to effectively convert this static
knowledge into dynamic decision-making in interactive settings. To address this
challenge, we propose Think in Games (TiG), a novel framework that empowers
LLMs to develop procedural understanding through direct interaction with game
environments, while retaining their inherent reasoning and explanatory
abilities. Specifically, TiG reformulates RL-based decision-making as a
language modeling task: LLMs generate language-guided policies, which are
refined iteratively through online reinforcement learning based on
environmental feedback. Our experimental results show that TiG successfully
bridges the gap between declarative and procedural knowledge, achieving
competitive performance with dramatically lower data and computational demands
compared to conventional RL methods. Moreover, TiG provides step-by-step
natural language explanations for its decisions, greatly improving transparency
and interpretability in complex interactive tasks.