Title: iGRPO: Self‑Feedback–Driven LLM Reasoning
Primary area: foundation or frontier models, including LLMs
Keywords: Reinforcement Learning, Reasoning, Large Language Models

Abstract: Large Language Models (LLMs) have demonstrated considerable promise in solving complex mathematical problems, yet they still fall short of producing fully accurate and consistent solutions. Reinforcement Learning (RL) has emerged as a powerful framework for aligning these models with task-specific rewards, thereby improving their overall quality and reliability. In particular, Group Relative Policy Optimization (GRPO) has emerged as an efficient, value-function-free alternative to Proximal Policy Optimization (PPO). In this paper, we introduce Iterative GRPO (iGRPO), a two-stage extension of GRPO that boosts performance by incorporating a self-feedback mechanism. Specifically, iGRPO first samples multiple candidate responses (self-feedback stage) and selects the highest-reward completion as feedback for the second stage (self-reflection stage). By conditioning the model on this top-performing draft, iGRPO effectively guides the policy to generate refined solutions surpassing the best first-stage candidates. Through systematic comparisons with GRPO across different base models (e.g., Nemotron-H-8B-Base-8K, DeepSeek-R1 Distilled), we extensively validate the effectiveness of iGRPO. In addition, further training the OpenReasoning-Nemotron-7B model with iGRPO algorithm results in new state-of-the-art benchmarks of 85.62\% and 79.64\% on AIME24 and AIME25, respectively. These results underscores the potential of iterative, self-feedback–driven RL approaches for advancing the reasoning performance.

TL;DR: Self-feedback driven reasoning for LLMs.