Title: Multi-Agent Evolve: LLM Self-Improve through Multi-Agent Co-evolution
Primary area: reinforcement learning
Keywords: large language models, self-improvement, reinforcement learning

Abstract: Reinforcement Learning (RL) has demonstrated significant potential in enhancing the reasoning capabilities of large language models (LLMs). However, the success of RL for LLM heavily relies on human-curated datasets and verifiable rewards, which raises concerns about its scalability and generality.
Motivated by the self-play paradigm, with proven success in developing agents for games and Go, recent research attempts to enhance LLM reasoning capabilities without human-annotated data. However, their methods primarily depend on a grounded environment to provide feedback, such as a Python interpreter or a game engine. Extending such methods to generic domains remains challenging because of the lack of domain-agnostic, non-hackable rewards.
To address these challenges, we propose Multi-Agent Evolve, a framework that enables LLMs to self-evolve in solving diverse tasks including mathematics, reasoning, and general knowledge Q\&A.
The core design of Multi-Agent Evolve leverages a triplet of interacting agents (Proposer, Solver, Judge) instantiated from a single LLM and applies reinforcement learning to optimize their behaviors based on self-rewarding techniques. The Proposer generates questions, the Solver attempts solutions, and the Judge evaluates both while co-evolving through reinforcement learning with self-rewarding techniques.  
Experiments on Qwen2.5-3B-Instruct demonstrate that Multi-Agent Evolve achieves an average improvement of \% across multiple benchmarks. These results highlight Multi-Agent Evolve as a scalable and data-efficient method for enhancing the general reasoning abilities of LLMs with minimal reliance on human-curated supervision.