Group Sequence Policy Optimization
Authors: name: Chujie Zheng, name: Shixuan Liu, name: Mingze Li, name: Xiong-Hui Chen, name: Bowen Yu, name: Chang Gao, name: Kai Dang, name: Yuqiong Liu, name: Rui Men, name: An Yang, name: Jingren Zhou, name: Junyang Lin
Organization: Qwen

This paper introduces Group Sequence Policy Optimization (GSPO), our stable,
efficient, and performant reinforcement learning algorithm for training large
language models. Unlike previous algorithms that adopt token-level importance
ratios, GSPO defines the importance ratio based on sequence likelihood and
performs sequence-level clipping, rewarding, and optimization. We demonstrate
that GSPO achieves superior training efficiency and performance compared to the
GRPO algorithm, notably stabilizes Mixture-of-Experts (MoE) RL training, and
has the potential for simplifying the design of RL infrastructure. These merits
of GSPO have contributed to the remarkable improvements in the latest Qwen3
models.