The Landscape of Agentic Reinforcement Learning for LLMs: A Survey
Authors: name: Guibin Zhang; name: Hejia Geng; name: Xiaohang Yu; name: Zhenfei Yin; name: Zaibin Zhang; name: Zelin Tan; name: Heng Zhou; name: Zhongzhi Li; name: Xiangyuan Xue; name: Yijiang Li; name: Yifan Zhou; name: Yang Chen; name: Chen Zhang; name: Yutao Fan; name: Zihu Wang; name: Songtao Huang; name: Yue Liao; name: Hongru Wang; name: Mengyue Yang; name: Heng Ji; name: Michael Littman; name: Jun Wang; name: Shuicheng Yan; name: Philip Torr; name: Lei Bai
Published: 2025-09-02 17:46:26

Abstract: The emergence of agentic reinforcement learning (Agentic RL) marks a paradigm
shift from conventional reinforcement learning applied to large language models
(LLM RL), reframing LLMs from passive sequence generators into autonomous,
decision-making agents embedded in complex, dynamic worlds. This survey
formalizes this conceptual shift by contrasting the degenerate single-step
Markov Decision Processes (MDPs) of LLM-RL with the temporally extended,
partially observable Markov decision processes (POMDPs) that define Agentic RL.
Building on this foundation, we propose a comprehensive twofold taxonomy: one
organized around core agentic capabilities, including planning, tool use,
memory, reasoning, self-improvement, and perception, and the other around their
applications across diverse task domains. Central to our thesis is that
reinforcement learning serves as the critical mechanism for transforming these
capabilities from static, heuristic modules into adaptive, robust agentic
behavior. To support and accelerate future research, we consolidate the
landscape of open-source environments, benchmarks, and frameworks into a
practical compendium. By synthesizing over five hundred recent works, this
survey charts the contours of this rapidly evolving field and highlights the
opportunities and challenges that will shape the development of scalable,
general-purpose AI agents.