Title: Lessons Learned: A Multi-Agent Framework for Code LLMs to Learn and Improve
Venue: NeurIPS 2025
Accepted: yes
Keywords: Multi Agent

Large Language Model

Code Optimization
Abstract: Recent studies show that LLMs possess different skills and specialize in different tasks. In fact, we observe that their varied performance occur in several levels of granularity. For example, in the code optimization task, code LLMs excel at different optimization categories and no one dominates others. This observation prompts the question of how one leverages multiple LLM agents to solve a coding problem without knowing their complementary strengths a priori. We argue that a team of agents can learn from each other's successes and failures so as to improve their own performance. Thus, a lesson is the knowledge produced by an agent and passed on to other agents in the collective solution process. We propose a lesson-based collaboration framework, design the lesson solicitation--banking--selection mechanism, and demonstrate that a team of small LLMs with lessons learned can outperform a much larger LLM and other multi-LLM collaboration methods.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper presents LessonL, a multi-agent collaboration framework where multiple LLM agents iteratively improve their code optimization and generation capabilities by learning from each other's successes and failures. The core mechanism involves three components: (1) lesson solicitation, where agents generate explanatory lessons from their solution attempts; (2) lesson banking and selection, where lessons are stored and filtered based on effectiveness and relevance; and (3) effectiveness adjustment, which dynamically updates lesson utility scores. The authors evaluate their approach on code optimization (ParEval, PolyBench) and generation (HumanEval, MBPP) benchmarks, demonstrating that small model ensembles can outperform larger individual models under similar resource constraints.\n\nSTRENGTHS AND WEAKNESSES:\n### Strengths\n\n- The framework achieves substantial and consistent improvements across different metrics and benchmarks, with some cases showing dramatic performance gains.\n- The paper provides extensive experiments across multiple benchmarks for both code optimization and generation, with proper statistical analysis including error bars and multiple runs across different programming modes.\n- This paper reveals that different LLMs have their own areas of expertise in code optimization tasks, and their capabilities can be enhanced by learning from lessons (accumulating experience).\n\n### Weaknesses\n\n1. The core contribution seems limited as multi-agent collaboration with experience sharing has been extensively explored in existing collaborative learning frameworks[1], with the main difference being application to coding tasks.\n2. The lesson selection mechanism (splitting selections half-and-half between high-speedup and high-relevance) appears arbitrary. The effectiveness adjustment using simple linear interpolation with epsilon parameter lacks sophistication and theoretical grounding. Could you explain more about this selection mechanism qualitatively and quantitively?\n3.  The introduction motivates the work by highlighting that different LLMs have complementary strengths in different coding categories (e.g., Qwen7B excels at \"geometry\" while Deepseek7B excels at \"scan\"). However, the proposed framework has LLMs learn from the same LLMs than leveraging different LLMs' strengths. This creates a logical inconsistency. Could you provide more supportive experiments to support this claim?\n\n[1] Li J, Lai Y, Li W, et al. Agent hospital: A simulacrum of hospital with evolvable medical agents[J]. arXiv preprint arXiv:2405.02957, 2024.\n\nQUESTIONS:\n1. Can the lessons accumulated in this way be used to train LLMs and make them more powerful?\n2. How does performance scale with increasing numbers of agents (n > 3) and interaction rounds (T > 4)? \n3. Could you provide statistics on the average number of LLM calls per method across different benchmarks? LessonL requires additional lesson solicitation steps beyond the main solution generation, and with multiple rounds and agents, the total LLM invocations could be significantly higher than baseline methods. A detailed breakdown showing  for each method would provide better understanding of the true computational overhead and help validate the cost-effectiveness claims in Figure 4.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-06-29"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper introduces LessonL, a novel multi-agent framework for collaborative problem-solving in code optimization and generation. It leverages the complementary strengths of multiple LLM agents, enabling them to learn from each other's successes and failures. The framework features a lesson solicitation–banking–selection mechanism, facilitating iterative improvement of solutions. Empirical evaluations demonstrate that LessonL outperforms single-agent and multi-agent baselines, achieving state-of-the-art performance on benchmarks like ParEval and HumanEval. Ablation studies confirm the effectiveness of key components, and cost analysis shows LessonL's efficiency. Case studies highlight insightful optimizations, such as divide-and-conquer strategies and precomputation techniques.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\n- A novel multi-agent framework for Code LLMs that enables collaborative learning and knowledge sharing between agents.\n\n- Comprehensive experimental evaluation across multiple LLMs and agent architectures, demonstrating consistent performance improvements.\n\n- Clear and detailed demonstration of the framework's effectiveness through ablation studies and case studies that highlight specific optimization strategies.\n\n\nWeaknesses:\n\n- The paper lacks sufficient justification for selecting the specific three open-source LLMs used in the experiments.\n\n- The framework's generalizability from code optimization to code generation tasks appears limited, with relatively weaker performance in the latter domain. This limitation warrants further investigation.\n\nQUESTIONS:\nCould you provide some failed examples generated by LessonL and discuss their potential mitigation strategies? \n\nThis would help better understand the framework's limitations and areas for improvement.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces LessonL, a multi-agent framework that enables code LLMs to collaboratively solve coding problems through shared learning. The key innovation is a \"lesson\" mechanism—concise knowledge pieces that explain why solutions succeed or fail—allowing agents to learn from each other's experiences. Unlike existing multi-agent approaches that assign fixed roles or use simple voting, LessonL enables dynamic knowledge transfer among agents.\nThe framework operates through four iterative stages: (1) Lesson Solicitation from solution outcomes, (2) Lesson Banking in a central repository, (3) Lesson Selection based on speedup and semantic relevance, and (4) Solution Generation using selected lessons. The authors demonstrate that teams of small models (7B-14B parameters) can outperform larger models like GPT-4o on both code optimization and generation tasks across six benchmarks.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\nThe lesson-based collaboration mechanism draws natural inspiration from human peer learning. This represents a significant departure from fixed role-based (planner, coder, debugger) or voting-based multi-agent systems.\n\nThe paper evaluates on 6 benchmarks with thorough ablation studies validating each component. The cost-effectiveness analysis considering both monetary and computational costs is particularly valuable. Results demonstrate statistical rigor with multiple runs and standard deviations reported.\n\nThe paper aims to address code optimization—an important but understudied area compared to code generation. The interpretable lessons produced as byproducts have educational value.\n\nAchieves state-of-the-art results, with impressive speedups and the surprising finding that small model teams can outperform GPT-4o.\n\nWeaknesses\nScalability: The framework is only tested with 3 agents, leaving unclear how performance and coordination complexity scale with more agents. Evaluation is limited to 5 rounds, so long-term behavior is unknown. There's no comparison with recent test-time scaling methods (e.g., DeepSeek-R1-Distill/Qwen 3, o1-style reasoning) beyond basic CoT and Reflexion.\n\nLesson Quality and Generalization: The paper lacks mechanisms to verify lesson correctness beyond empirical performance, creating risk of error propagation through incorrect lessons. There's no discussion of handling conflicting or contradictory lessons. The paper doesn't evaluate cross-benchmark transfer—whether lessons from one benchmark help on another.\n\nAnalysis: Missing analysis of which types of problems benefit most from this approach.\n\nQUESTIONS:\nHow does performance scale with 5+ agents? Is there a point of diminishing returns or negative interference?\nHow do you handle potentially incorrect lessons? Have you observed cases where misleading lessons degraded performance across multiple agents?\nHow does LessonL compare with recent test-time compute scaling approaches like those used in DeepSeek-R1(-Distill)\n\nLIMITATIONS:\nna", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-04"}

comments: SUMMARY:
This work proposed to develop a team of agents that can learn from each other's successes and failures so as to improve their own performance. Lessons are stored in the memory and passed on to other agents in the collective solution process.

This is an interesting work. The idea is straightforward and effective in terms of self-learning speed and performance improvement, when compared with other prompting / memory / reflection based work.

STRENGTHS AND WEAKNESSES:
The paper is well written. I have no problem grasping the core idea of implementation and experiments in short period of time. However, the differentiation of this work with several previous works are not very clear. For example, for coding agents, there seem to be a lot of works last year developing multi-agent frameworks, such as https://arxiv.org/abs/2408.07060. They also have self-reflection or LLM judge steps to for better retrial. 

1. Can the authors discuss the differentiations of this current against previous multi-agent frameworks with memory? 
2. Can the authors clarify more about the lessons bank? How is this memory module designed and its implementation differentiation from RAG-based memory?

QUESTIONS:
See above

LIMITATIONS:
See above
TITLE: Official Review
CLARITY: 4
IMPACT: 2
ORIGINALITY: 2
DATE: 2025-07-06