GEM: A Gym for Agentic LLMs
Authors: name: Zichen Liu; name: Anya Sims; name: Keyu Duan; name: Changyu Chen; name: Simon Yu; name: Xiangxin Zhou; name: Haotian Xu; name: Shaopan Xiong; name: Bo Liu; name: Chenmien Tan; name: Chuen Yang Beh; name: Weixun Wang; name: Hao Zhu; name: Weiyan Shi; name: Diyi Yang; name: Michael Shieh; name: Yee Whye Teh; name: Wee Sun Lee; name: Min Lin
Published: 2025-10-01 15:55:57

Abstract: The training paradigm for large language models (LLMs) is moving from static
datasets to experience-based learning, where agents acquire skills via
interacting with complex environments. To facilitate this transition we
introduce GEM (General Experience Maker), an open-source environment simulator
designed for the age of LLMs. Analogous to OpenAI-Gym for traditional
reinforcement learning (RL), GEM provides a standardized framework for the
environment-agent interface, including asynchronous vectorized execution for
high throughput, and flexible wrappers for easy extensibility. GEM also
features a diverse suite of environments, robust integrated tools, and
single-file example scripts demonstrating using GEM with five popular RL
training frameworks. Along with this, we also provide a set of baselines across
24 environments using REINFORCE with Return Batch Normalization (ReBN), which
-- unlike GRPO -- is compatible with the full RL setting of dense per-turn
rewards and offers better credit assignment. We further conduct apple-to-apple
benchmarking of PPO, GRPO and REINFORCE in both single- and multi-turn settings
using GEM to shed light on the algorithmic designs. Lastly, GEM also functions
as a convenient evaluation toolkit besides a training environment. We hope this
framework can help accelerate future agentic LLM research.