Agents Play Thousands of 3D Video Games
Authors: name: Zhongwen Xu, name: Xianliang Wang, name: Siyi Li, name: Tao Yu, name: Liang Wang, name: Qiang Fu, name: Wei Yang

We present PORTAL, a novel framework for developing artificial intelligence
agents capable of playing thousands of 3D video games through language-guided
policy generation. By transforming decision-making problems into language
modeling tasks, our approach leverages large language models (LLMs) to generate
behavior trees represented in domain-specific language (DSL). This method
eliminates the computational burden associated with traditional reinforcement
learning approaches while preserving strategic depth and rapid adaptability.
Our framework introduces a hybrid policy structure that combines rule-based
nodes with neural network components, enabling both high-level strategic
reasoning and precise low-level control. A dual-feedback mechanism
incorporating quantitative game metrics and vision-language model analysis
facilitates iterative policy improvement at both tactical and strategic levels.
The resulting policies are instantaneously deployable, human-interpretable, and
capable of generalizing across diverse gaming environments. Experimental
results demonstrate PORTAL's effectiveness across thousands of first-person
shooter (FPS) games, showcasing significant improvements in development
efficiency, policy generalization, and behavior diversity compared to
traditional approaches. PORTAL represents a significant advancement in game AI
development, offering a practical solution for creating sophisticated agents
that can operate across thousands of commercial video games with minimal
development overhead. Experiment results on the 3D video games are best viewed
on https://zhongwen.one/projects/portal .