GEPA: Reflective Prompt Evolution Can Outperform Reinforcement Learning
Authors: name: Lakshya A Agrawal; name: Shangyin Tan; name: Dilara Soylu; name: Noah Ziems; name: Rishi Khare; name: Krista Opsahl-Ong; name: Arnav Singhvi; name: Herumb Shandilya; name: Michael J Ryan; name: Meng Jiang; name: Christopher Potts; name: Koushik Sen; name: Alexandros G. Dimakis; name: Ion Stoica; name: Dan Klein; name: Matei Zaharia; name: Omar Khattab
Published: 2025-07-25 17:42:32

Abstract: Large language models (LLMs) are increasingly adapted to downstream tasks via
reinforcement learning (RL) methods like Group Relative Policy Optimization
(GRPO), which often require thousands of rollouts to learn new tasks. We argue
that the interpretable nature of language can often provide a much richer
learning medium for LLMs, compared with policy gradients derived from sparse,
scalar rewards. To test this, we introduce GEPA (Genetic-Pareto), a prompt
optimizer that thoroughly incorporates natural language reflection to learn
high-level rules from trial and error. Given any AI system containing one or
more LLM prompts, GEPA samples system-level trajectories (e.g., reasoning, tool
calls, and tool outputs) and reflects on them in natural language to diagnose
problems, propose and test prompt updates, and combine complementary lessons
from the Pareto frontier of its own attempts. As a result of GEPA's design, it
can often turn even just a few rollouts into a large quality gain. Across four
tasks, GEPA outperforms GRPO by 10% on average and by up to 20%, while using up
to 35x fewer rollouts. GEPA also outperforms the leading prompt optimizer,
MIPROv2, by over 10% across two LLMs, and demonstrates promising results as an
inference-time search strategy for code optimization.