RPG: A Repository Planning Graph for Unified and Scalable Codebase Generation
Authors: Jane Luo
Xin Zhang
Steven Liu
Jie Wu
Jianfeng Liu
Yiming Huang
Yangyu Huang
Chengyu Yin
Ying Xin
Yuefeng Zhan
Hao Sun
Qi Chen
Scarlett Li
Mao Yang
Venue: ICLR 2026 Poster (2026)
Keywords: planning
repository generation
agent
code generation
Primary area: other topics in machine learning (i.e., none of the above)
TL;DR: To address ambiguity and verbosity in natural language–based repository generation, we propose the Repository Planning Graph (RPG), and further build ZeroRepo and the RepoCraft benchmark.

Abstract: Large language models excel at generating individual functions or single files of code, yet generating complete repositories from scratch remains a fundamental challenge. This capability is key to building coherent software systems from high-level specifications and realizing the full potential of automated code generation. The process requires planning at two levels: deciding what features and modules to build (proposal stage) and defining their implementation details (implementation stage). Current approaches rely on natural language planning, which often produces unclear specifications, misaligned components, and brittle designs due to its inherent ambiguity and lack of structure. To address these limitations, we introduce the Repository Planning Graph (RPG), a structured representation that encodes capabilities, file structures, data flows, and functions in a unified graph. By replacing free-form natural language with an explicit blueprint, RPG enables consistent long-horizon planning for repository generation. Building on RPG, we develop ZeroRepo, a graph-driven framework that operates in three stages: proposal-level planning, implementation-level construction, and graph-guided code generation with test validation To evaluate, we construct RepoCraft, a benchmark of six real-world projects with 1,052 tasks. On RepoCraft, ZeroRepo produces nearly 36K Code Lines and 445K Code Tokens, on average 3.9× larger than the strongest baseline (Claude Code), and 68× larger than others. It also achieves 81.5% coverage and 69.7% test accuracy, improving over Claude Code by 27.3 and 35.8 points. Further analysis shows that RPG models complex dependencies, enables more sophisticated planning through near-linear scaling, and improves agent understanding of repositories, thus accelerating localization. Our data and code are available at https://github.com/microsoft/RPG-ZeroRepo.

Published as a conference paper at ICLR 2026
RPG: A REPOSITORY PLANNING GRAPH FOR UNIFIED AND SCALABLE CODEBASE GENERATION
Jane Luo Xin Zhang 1i Steven Liu 1-1 Jie Wu Jianfeng Liu' Yiming Huang? Yangyu Huang' Chengyu Yin Ying Xin
Yuefeng Zhan' Hao Sun' Qi Chen
Scarlett Li' Mao Yang'
Microsoft, Tiinghua University SUniversity California, San Diego
'Beijing Jiaotong University
ABSTRACT
Large language models excel at generating individual functions or single files of code, yet generating complete repositories from scratch remains a fundamental challenge. This capability is key to building coherent software systems from high- level specifications and realizing the full potential of automated code generation. The process requires planning at two levels: deciding what features and modules to build (proposal stage) and defining their implementation details (implementa- tion stage). Current approaches rely natural language planning, which often produces unclear specifications, misaligned components, and brittle designs due its inherent ambiguity and lack of structure. To address these limitations, we introduce the Repository Planning Graph (RPG), a structured representation that encodes capabilities, file structures, data flows, and functions By replacing free-form natural language with an explicit blueprint, RPG enables consistent Long-horizon planning for repository generation. Building on RPG, we develop ZeroRepo, a graph-driven framework that operates in three stages: proposal-level planning, implementattion-lvell construction, and graph-guides code generation with test validation To evaluate, we construct RepoCraft, a benchmark six real-world projects with 1,052 tasks. On RepoCraft, ZeroRepo produces nearly 36K Code Lines and 445K Code Tokens, average 3.9x larger than the strongest baseline (Claude Code), and 68x larger than other baselines. It achieves 81.5% coverage and 69.7% test accuracy, improving over Claude Code by 27.3 and 35.8 points. Further analysis shows that RPG models complex dependencies, en- ables more sophisticated planning through near-linear scaling, and improves agent understanding of repositories, thus accelerating localization. Our data and code are available httpp:: /ttpttgttbtuut///// www.tttw...oom..ae mi wwtp:/microooot..oooooooo PRpooossstttRtt----- RRt./PPG-G---- ZeroRepo
unified graph.
INTRODUCTION
Recent large language models (LLMss) have shown strong performance on function-level and file-level code generation, reliably producing functions and files from natural language descriptions (Zhu et al.) 2024 Wang et al. 2025 Liu et al. 2025 Zeng et al., 2025). However, scaling this capability from functions and files to generate large-scus software repositories from scratch remains a fundamental challenge. The core difficulty is bridging the gap between high-level user intent and the repository's 's intricate network of files, classes, and dependencies navigating this gap necessitates process of progressive planning, which naturally decomposes into two complementary phases: proposal-level planning, which determines what to huild by defining the functional scope and key capabilities, and implementationn-evee planning, which determines how to build it by specifying the file structure, interfaces, dependencics, and data flows.
et al.] (2025); Li (2025). Successfully
Equal contribution.
soorespnniing author.
This work is done during their internships at Microsoft.

---

Published as a conference paper at ICLR 2026
Prior work has explored this challenge through three paradigms. Distributed planning frameworks (e.g., MetaGPT (Hong et al.), ChatDev (Qian et al. 2024) assign specialized roles such as manager, architect, and engineer negotiate between requirements and implementations. systems (e.g., Paper2Code (Seo et al., 2025), AutoP2C (Lin et al. (025)) follow fixed pipelines that first build architectural skeletons before filling in details. Iterative terminal agents (e.g.. Open- Hands (Wang et al.). Claude Code (Amthropic) 2025b), Gemini CLI (Google) 2025)) externalize intermediate plans, often in markdown, and refine them step by step. Despite their differences, these approaches share a dependency: natural language as the intermediate medium for planning.
warkllow-based
While natural language remains flexible and numan-readabee medium, it can often be less efficient for surge-ccale repository generation. Its inherent ambiguity may blur distinctions between intent and constraints (Wang al. 2024), its lack of explicit hierarchy makes dependency tracking particularly difficult (Besta et al. 2024). and static plans may gradually degrade over long horizons without adaptive adjustment (Sun et al. 2023). When extended to automatic repository generation, these limi- tations can more easily lead to unstable proposal-level planning, where functionalities are sometimes incomplete, overlapping, or unevenly scoped (Zhu et al. 2025), and fragmented implementation-leve planning, where plans drift across iterations, introducing inconsistencies in dependencies, data flows, and modular boundaries (Almorsi et al. 2024; Ashrafi et al., 2025).
To address these limitations, introduce the Repository Planning Graph (RPG), a persistent and evolvable representation that unifies proposal and implementation planning for repository generation. RPG encodes functional goals and designs in single graph: nodes capture hierarchical capabilities with files, classes, and functions, while edges specify semantic relations and data flows. By replacing free-form language with a structured medium, RPG provides compact, interpretable basis for consistent long-horizon planning. Building on this representation, we develop ZeroRepo, a graph- driven framework for controllable repository generation. Given a user specification, ZeroRepo proceeds in three stages: (1) Proposal-Level Construction, which organizes and refines requirements into a functional graph via a large-scale feature tree; (2) Implementation-l.co
Construction, which expands this graph into the full RPG by encoding file skeletons, interfaces, and flows; and (3) Graph-Guided Code Generation, which traverses the RPG in topological order with test-driven development, guided localization, and iterative editing.
To evaluate agents' planning ability in repository generation, we construct RepoCraft, a benchmark of six projects with 1,052 tasks. On RepoCraft, ZeroRepo attains 81.5% functional coverage and a 69.7% pass rate, exceeding the strongest baseline (Claude Code) by 27.3 and 35.8 points, while producing repositories with 36K Lines of Code and 445K Code Tokens, about 3.9x larger than Claude Code and 68x larger than other baselines. Further analysis shows that Repository Planning Graph (RPG) captures complex dependencies, including inter-module data flows and function-level relations. It enables near-linear scaling of functionality and code size, supporting complex planning and providing a foundation for large-scale repositories and long-horizon development. As global representation, RPG enhances agents' repository understanding and accelerates localization.
Our Contributions are list below:
1. We introduce the Repository Planning Graph (RPG), a unified representation integrating proposal- and implementation-level planning, encoding functionality, file structures, data flows, and function designs.
2. We develop ZeroRepo, graph-driven framework that constructs RPG through proposal- and implementation-leve planning, and generates code with test validation.
3. To evaluate agent planning ability in repository generation, we build RepoCraft, bench- mark of 6 projects with 1,052 tasks assessing coverage, accuracy, and code scale.
4. On RepoCraft, ZeroRepo achieves strong improvements over basclines, reaching 81.5% functional coverage and nearly 69.7% test accuracy, while producing repositories 3.9x larger than the strongest baseline. Further analysis shows that RPG captures complex dependencies, enables more sophisticated planning through near-linear scaling, and enhances agents' repository understanding, thereby accelerating localization.
2 RELATED WORK
LLM-based Code Generation SOTA models (e.g., GPT-4o (OpenAl, 2024), Claude 4 (An
thropic 2025a), DeepSeek-R1 (Guo et al. 2025) excel at SWE tasks such code completion test
2

---

Published as a conference paper at ICLR 2026
generation (Zafar et al. 2022 Dakhel et al. 2024). refactoring (Gautam et al. 2025 and program re-
pair (Jimenez et a..). Instruction-tume variants (c.g.. Qwen- (Hui et al. 2024). EpiCoder (Wang
et al. (025g) further improve reliability. These advances deliver strong function- performance, forming the basis for broader Software Fngineering(SWE) progress.
Agents for Repository-Level Generation Agent frameworks embed LLMs and can automate SWE tasks: multi-agent systems (Qian et al. 2024] Hong et al.) assign roles; workflows (Seo) jet al. 2025; Lin et al. 2025) run pipelines; industrial tools (OpenAT, 2025| |Antrropic] 2025b) enable more advanced automation. Yet most rely on ephemeral language plans that lack persistent
structure, producing fragmented implementations; ZeroRepo instead uses a graph-guide abstraction for structured planning and execution.
dky York Aeder-aggeerdd
B.I File Skelstun Encodlnng
Legtna pepeeel Apeiieetegeree
Oriin
Napplng
acc/data. load
sec/alges scc/oval
cats jed
Prepaca-Leeel Conseraction
Erchwliing Sisder
Bost Nodia
prepromsspy
Reatune
Barkow
Victor
beratiwlyy Upytas Flae
to Sodiriaz
Trankee
Agic
Query
system
Jieyy
Inserpy
Design
staualas au po:
Implemettations
B.2 Dato Flow and Fusctions Eacoding
B.I
& Puuctten
Book
Lod
Desige
Bax
def faccusaf, y1-
Aodia
C. Gropp-Gralteed Codk Gaveraine
cef predict 20:
Guade-Gasdedd Search
Nedes
fad slas buodwak
Eepacttory Viee
Design
Updare Boss Charen
Eependency Explccs
Dala Flow
larriteyy Updaic Farntion
Punctiono
Topolcaical Ordaring
<Tas Eetvinaber:
Lesf
del Junt Sew (1927.p1
Nides
retur riid centeett
The ZeroRepo pipeline for repository generation. (A) Proposal-Level Construction maps query to a functionality graph. (B) Implementation- Construction refines via (B1) ) File Structure Encoding into file fugmeenteetedd graph and (B2) Pata-Flow/Funetion Encoding into the Repository Planning Graph (RPG). (C) Graph-Guided Code Generation traverses RPG to generate the repository.
Figure I:
3 REPOSITORY PLANNING GRAPH CONSTRUCTION
To address the ambiguity of natural language plans, we propose the Repository Planning Graph (RPG), a structured representation that encodes repository functionality and implementation logic as nodes and edges. Building on RPG, we develop ZeroRepo, a framework for repository generation from scratch. This section first introduces the structure of RPG (33.)) and then explains how ZeroRepo constructs it through proposal-level planning (33.2) and implementation-leee refinement The overall pipeline is shown in Figure A-)..
3.1
REPOSITORY PLANNING GRAPH STRUCTURE
As shown in Figure 2 RPG provides unified representation for repository planning by encod- ing functionality and implementation in a structured graph that is both explicit and machine- interpretable, rather than unstable or ambiguous natural language. Its nodes carry dual semantics and serve complementary roles. At the functional level, they represent progressively refined ca- pabilities: high-level modules (e.g., algorit hms)) decompose into mid-level components and ultimately into leaf nodes corresponding to concrete algorithms. At the structural level, this hi- erarchy closely mirrors repository organization: root nodes align with file regions or directories,
3

---

Published as a conference paper at ICLR 2026
intermediate nodes with files, and leaf nodes with specific functions or classes, thereby unifying functional decomposition with the code structure.
Pantall
Maciine Learninng Library
Beyond hicrarchical nodes, edges in RPG cap- ture explicit, grounded, and execution dependen- cies across levels. Inter-module edges (black arrows in Figure 2 encode data flows, e.g., outputs from Data Loading feeding into ML Algori hmms and then Evaluation. Intra- module edges (gray dashed arrows) capture file- level orderings; for instance, load data py precedes p preprocess.... py, with outputs passed to preprocessing. Collectively, these edges impose topological order aligning functional decomposi- tion with code organization, ensuring coherence between global semantics and local implementation.
Taake Fostin
Tupa
Eage Kiiass TTape
Jali MIEIAO
Rara Jurt
Figure 2: Repository Planning Graph: nodes encode repository capabilities, edges capture hi- erarchy and flows
3.2 PROPOSAL-LEVEL CONSTRUCTION
At the proposal level, the goal is to translate high-level specifications into a functionality graph. This involves two steps: selecting repository-aligned subtree via an explore-exploit search over ZeroRepo's 's knowledge base, and refactoring it into the graph (algorithms detailed in Appendix B.J.
A Global Tree as Knowledge Base LLMs alone provide unstable and biased capability enumer- ation, often with incomplete coverage (Valmeekam al. 2023 Armony et al.. 2025). To stabilize planning, we use the EpiCoder Feature Tree (Wang et al. 2025), an ontology of 1.5M software capa- bilities, as ZeroRepo's 's knowledge base. Its breadth and hierarchy act as a structured prior, mitigating randomness and bias and improving coverage. For efficient retrieval, each node is embedded in vector space, with its hierarchical path stored as metadata in a vector index. This preserves semantic similarity and structural context, enabling precise, scalable grounding. Statistics are in Appendix B.2
Explort-Exploit Subtree Selection Using the Feature Tree as structured knowledge base, the first step is to construct repo-aligned subtree tailored to the user's goal. Exhaustive enumeration infeasible at the 1.5M scale, so ZeroRepo incrementally expands the subtree via an explore -xpploit strategy. (I) Exploitation ensures precision: we retrieve top-k feature paths most aligned with the user goal and augment them with keywords suggested by LLM queries.
(2) Exploration ensures diversity: we deliberately expand into unvisited regions of the ontology to capture less obvious but relevant functionalitics. Candidates from both strategies are filtered by the LLM and integrated into the evolving subtree, yielding a balanced and comprehensive foundation for downstream planning.
Refactoring by Goal Alignment The repo-aligned subtree, though capturing relevant functionali- ties, still inherits the generic organization of the global ontology. To align it with the user's 's repository goal, we refactor feature placements into modular functionality graph. The LLM partitions func- tionalities into cohesive modules following software engineering principles of cohesion and coupling. For instance, in a machine learning library, metrics such as silhouette score are reorganized under an evaluation module rather than within clustering algorithms. The resulting graph establishes clear functional boundaries, encoding proposal-level planning directly into the representation.
3.3 INPLEMENTATION-LEVER CONSTRUCTION
After proposal-level construction establishes the multi-level functional plan, the graph is further enriched with implementation details, culminating in the complete Repository Planning Graph (RPG) at this stage. The process includes encoding the repository's file structure, modeling inter-module data flows and intra-module orderings, and specifying concrete functions and interfaces.
3.3.1 FILE STRUCTURE ENCODING
While proposal-level planning defines modular functionalities, it remains abstract and detached from implementation. To bridge this gap, the graph is extended with folder and file layouts, instantiating a repository skeleton that maps modules into executable structures, yielding ale-anemented graph.
4

---

Published as a conference paper at ICLR 2026
Folder-Level Encoding Proposal-level planning partitions functionalities into modular subgraphs, yet this abstraction does not define the repository's structure. We enrich root nodes with folder- level specifications, assigning each subgraph directory namespace (e.g., algos/, eval/)) This encoding couples semantic modularity with structural separation, ensuring descendants inherit a consistent namespace and the repository skeleton aligns with high-level capability decomposition.
File-Level Encoding Once folder regions are encoded at root nodes, the graph is enriched by assigning files to intermediate nodes, specifying how module functionalities map to executable files. For example, preprocessing utilities consolidate into preprocess py, while models like linear regression and variants group into 1 linear mode ls py. Embedding file structure preserves semantic cohesion, reduces cross-file coupling, and yields file-augmented graph anchoring design.
3.3.2 DATA FLOW AND FUNCTIONS ENCODING
After obtaining the file-aummented graph, this stage finalizes the full Repository Planning Graph (RPG) by assigning executable roles to leaf nodes. To ensure coherence across modules and functions, we first incorporate inter- and intra-module data flows as input-output constraints, then abstract shared structures design anchors, and finally refine leaf nodes into concrete functions classes.
Data-Flow Encoding To ground interface design in execution, the graph is augmented with data- flow edges capturing inter- and intra-module relations. Globally (Figure(2). typed input-output flows connect subgraph roots; for example, a data-loading module may provide an array of training data to algorithms. Locally, files within a module are ordered to ensure coherent, dependency-aware implementation. These flows impose hierarchical order constraining and organizing interface design.
To improve scalability and maintainability, recurring input output patterns across modules are abstracted into common data structures or base classes, serving anchors that enforce consistency and reduce redundancy. For example, algorithms can be unified under a BaseEst mator class standardize interaction with preprocessing and evaluation modules.
Abstracting Global Interfaces
Adaptive Interface Design Within each file-level subgraph, leaf features cluster into executable interfaces by semantic relatedness. Independent features become standalone functions, while in- terdependent ones form shared classes with methods. For example, in Figure load_json and load csv are grouped into a Dat aLoader class, while elastic is implemented as an Elast i icNet t class. This adaptive mapping balances granularity and cohesion, yielding Repository Planning Graph (RPG) that preserves modularity and semantic consistency at repository scale.
TRAPH-GUIDED CODE GENERATION
As shown in Figure given a user query and the completed RPG, ZeroRepo generates repositories by traversing the graph in topological order, ensuring dependencies precede dependents. At each leaf node, test-driven development (TDD) is applied: tests are derived from the specification, then the corresponding functions or classes are implemented and validated; failing cases trigger revisions until passing the iteration limit is reached. Only functions that pass all tests are committed, enabling incremental expansion while preserving stability. Further details are in Appendix D
Graph-Guided Localization and Editing To handle implementation and debugging requests, we adopt two-stage workflow: first localizing the target in the RPG, then editing the associated code. Localization leverages the graph's global structure and three tools: (1) RPG-Guided search, which uses functionality-based fuzzy matching to identify candidate functions: (2) repository code view, retrieving full interface bodies for inspection or modification; and (3) dependency exploration, tracing edges to reveal related modules and interactions. Once localized, the agent revises or generates the corresponding code to complete the requested implementation or repair.
Graph-Guided Test Validation To ensure correctness and catch emrors early, validation proceeds in stages aligned with the graph. Each function or class is first checked in isolation using unit tests from its docstring. Validated components trigger regression tests when modified, while subgraphs undergo integration tests to verify data flows and contracts across modules. A lightweight majority-vote diagnosis separates genuine implementation errors from environment test issues, automatically handling the latter and returning the former for repair via the localization-editnng workflow.
5

---

Published as a conference paper at ICLR 2026
Table Overview of the six reference repositories and their paraphrased counterparts (Para. Name) in RepoCraft. #F. Cate. denotes functional categories, #Files the total source files, LOC the effective lines of code, and Task Counts the evaluation tasks for measuring code accuracy.
Real Repo
Para. Name
#F. Cate.
#Files
LOC
Code Tokens
Task Counts
scikit-learn
MLKit-Py
47
185
65.972
592,187
236
pandas
TableKit
81
217
106,447
943,873
175
sympy
SymbolicMath
40
699
218,924
2,339,881
192
statsmodels
StatModeler
88
271
83,325
893,824
234
requests
HttpEasy
22
17
2,793
22,297
50
django
py WebEnginn
42
681
109,457
917,622
165
5
EXPERIMENT SETUP
5.1 REPOCRAFT BENCHMARK
A key challenge in repository-lett generation is the lack of benchmarks for end-to-end reasoning and planning. Prior work mainly targets incremental development (editing, refactoring, or bug fixing
in existing codebases (Jimenez et al. Gautam al. 2025 Zhang al. 2025 Li et al. 2025 Huang
et al. 2024)) or repository generation with detailed skeletons and specifications that limit autonomous planning (Zhao et al. 2025 Starace al., 2025). RepoCraft fills this gap by requiring agents
build complete repositories from high-level natural language descriptions and evaluating them against
real-world projects on scale, functionality, and correctness, with final statistics in Table [
5.1.1 REFERENCE REPOSITORY SELECTION
RepoCraft grounds evaluation in six widely used Python projects: scikit-learn, pandas, sympy, statsmodels, requests, and django. They are chosen as strong references for embodying high-quality engineering with active community development, modular design, and comprehensive tests. Covering domains from scientific computing to web frameworks, they ensure breadth and realism. To mitigate pretraining leakage, we paraphrase their names and descriptions before providing them to agents.
5.1.2 METRICS
RepoCraft evaluates generated repositories along three dimensions (detailed in Appendix E.3.I
Functionality We evaluate both Coverage and Novelty. Coverage measures the proportion of func- tional categories defined in official documentation that are represented in the generated repository; category is counted as covered Novelty measures the proportion of generated functionalities not in the reference taxonomy, capturing the system's ability to propose coherent and meaningful new capabilities beyond the specification.
least one generated functionality corresponds to it (Appendix (E44).
Accuracy Correctness at task level via (1) Pass Rate: fraction of tests passed, and (2) Voting Rate: fraction confirmed by majority-vote checks.
Code-Level Statistics We sreport repository scale indicators, including file count, normalized Lines of Code (LOC), and token count, measured after excluding non-core code such tests and examples.
5.1.3
FUNCTIONAL TASK CONSTRUCTION AND EVALUATION
Project
Filter Files
Parse Test Tree
Collect Tasks
SSuppl
sideamy
setup-py
tests/
Sed und_cpame
examples/
G
RGADME
test.vvm.py py
onectass onecleseve
Senerata Taclo
Temnestu SVa alpatton..
build/
Malcefile
test. sparsngy
prixt
Figure 3: Pipeline for Evaluation Task Collection. It comprises test file filtering, hicrarchical parsing into test trees, sampling and filtering. and final task generation.
To assess models' planning ability on constructed repositories, we evaluate whether they (i) implement the intended algorithms and (ii) realize them correctly. Simple measures of repository size coverage

---

Published as a conference paper at ICLR 2026
insufficient for this purpose, so RepoCraft introduces task- evaluations that capture both functional fidelity and implementation accuracy (see Appendix E.2 for details).
To enable such fine-grained evaluation, RepoCraft derives tasks from reference repositories. As shown in Figure 3 we collect available test functions and classes, organize them hierarchically following each project's modular structure, and apply stratified sampling ensure representative coverage. Trivial or non-algorithmic tests are filtered out, resulting in a diverse and computationally meaningful set of 1,052 tasks that closely mirror practical software evaluation.
Each task includes a natural language description of the target algorithm, a ground-truth test, and auxiliary materials. Evaluation proceeds in three steps: (1) Localization, mapping requirements to candidate functions or classes in the generated repository; (2) Semantic Validation, applying majority-vote checks over two rounds to confirm fidelity to the specification; and (3) Execution Testing. adapting and running the ground-truth test to verify interface correctness under realistic inputs and outputs. This design mirrors real-world development while reducing sensitivity to spurious model enrors. We use o3 3imnn the base model for automated evaluation.
5.2 BASELINES
We compare three paradigms: (1) Multi-agent frameworks (MetaGPT (Hong et al.). ChatDev (Qian itt al. 2024)) assigning specialized roles for end-to-end development; (2) Workflow-basee system
2Code (Seo et al. 2025)) with a fixed three-stage pipeline; (3) Terminal agents (Codex
CLI (OpenAl, 2025). Claude Code CLI (Anthropic) 2025b), Gemini CLI (Google 2025), Open- Hands (Wang et a performing natural language editing, debugging, and multi-file reasoning. For comparability, MetaGPT, ChatDev, and Paper2Code are run with two backbones: 03-mini (OpenAl 2025) and Wwen3-Coder-480E ----4880-------- ABBB------nsnuuc (Qwen3-Coder) Team 2025). Codex CLI, Claude Code CLI, and Gemini CLI are evaluated with their official strongest model. We enable Terminal Agents to retrieve real-world knowledge via web search. To ensure fairness, all runs extend to 30 iterations, with agents prompted at each step to propose or implement functionality.
5.3 IMPLEMENTATION DETAILS
We run 30 iterations for feature selection Proposal-Leve Graph Construction and use infly/inf- retriever-vl Yang al. 2025 for node embeddings. In the Code Generation Stage, each function allows up to 8 debugging iterations with 20 localization attempts. For test failures, we use 5-round majority voting for attribution and allow up to 20 remediation attempts for or environment errors.
MAIN RESULTS
Table 2: Performance of agent frameworks and model backbones RepoCraft. "Nov." denotes nov-
elty rate; the number in parentheses is Novel/Total, where Novel the count of novel functionalities and Total the number of planned ones. Gold Projects evaluation pipeline, and per-repository results are reported in Appendix F2
confidence ablation for the automatic
Agent
Model
Cov, (%)
Nov. (%) (Novel/Total) 1
Pass. / Vot. (%)
Files t
LOC 1
Tokens 1
03-mini
16.6
225.3
MetaGPT
Qwen3-Coder
17.1
0.0 (0.0/32.7)
3.2/9.4
8.5
326.5
3369.3
CharDev
03-mini
18.3
9.2 (3.0/32.8 8)
2.6/ 10.5
410.3
4458
Qwen3-Coder
22.1
(1.5/3833
6.//116
6.3
540.7
5422.2
OpenHands
03-mini
22.0
0.3 (0.1666))55)
5.19
9.8
292.2
2712.8
Qwen3-Coder
21.7
0.0 (0.00.3333)7)
5.8/112
458.0
4778.3
Paper2Code
63-mini Qwen3-Coder
21.7
5.2 (2.4//00
6.0/15.8
7.2
$47.7
5920.8
30.2
5.5 (4.0/73.8)
1.9
1365.2
14,555.0
Codex CL.I
o3 pro gemini 2.5 pro claude 4 sonnet
28.4
0.0 (0.0/48.5)
11.0/20.0
5.3
611.5
6248.5
Gemini
42,0
(0.871
1484.8
14,922.2
Claude Code CLI
54.2
6.7 (41.6/6210)
33.9/52.5
33.3
10,586.7
105,236.2
Gold Projects
Human Develogers
81.0792.0
345
97,819.7
951.614
03-mini Qwen3-Coder Coderr
81.5
136 (1515/11144))
69.7/75.0
271..5
23,977.3
260,761.2
ZeroRepo
75.1
9.2 (108./1113.3)
/
389.0
36,941.0
445,511.8
RPG enables richer functionality and larger repositories. ZeroRepo shows that RPG-guided planning yields repositories of greater scale, diversity, and novelty than existing approaches. On RepoCraft, it achieves 81.5% coverage with o3 10-mnnn, ni, 27.3% absolute improvement the
7

---

Published as a conference paper at ICLR 2026
configs
Inputs (list...)
general
scripts
Skeleton
time_ series
Time-indesed arra
Prediction
array
evaluation
Module UML
Temporal features
model_ base.py
Processed data
Output teatures
src
data_ lifecylle
Reduced features
Models
visualization
clustering
Interpretation
data_lifecycke
Clustor
Model
objects
explainability
Clustering
>
exprimental
Classes
explainability
feature_end feature_enigeerine enigeering
general
models
Kin-eans
time series
DESCAN
DBSCANDetector
visualization
FeutureSet
Structure:
Code Dependency
evaluation
Dectalontrectlaisttee
Autumndfclestuuesssettor
RaseModel
README.md
AdaRosstElassifer
DieClassVMDeetettor
setup.py
SVMClassifier
RudgeRagressor
SeleckBestieetures
inherit invole
RANSACRegressoe
ChiquuacePetturSSllttrr
Figure 4: Dependencies in the repository generated by Qwen3-Codee Creerr on MLKit including (1) the repository skeleton folder/file level. (2) inter-module data flows, and (3) interfaces dependencies.
strongest baseline (Claude Code). Beyond covering required functionality, ZeroRepo demonstrates in- novation, reaching novelty rates of 11-13% with over 100 new functionalities. whereas most baselines add fewer than 10. In size, ZeroRepo with @wen3-Codes generates 36K L.OC and 445K tokens, corresponding to 3.9x Claude Code and about 68x other baselines. Among methods, ZeroRepo is closest to deval-deeeloped Gold Projects, underscoring that RPG is key structured representation for building repositories that are larger, more diverse, and closer to real-world complexity.
RPG enhances reasoning consistency and structural fidelity. Beyond scale, ReroRepo delivers higher correctness and stability. To ensure reliability, we first validate the automatic localization and validation pipeline on human-developed Gold Projects, where achieves 81.0% pass rate and 92.0% voting agreement, establishing the ceiling under our test harness. Under the same protocol, ZeroRepo attains 69. 99.7% pass rate with o3-mini, an absolute improvement of 35.8% compared to the Claude Code. These results indicate that RPG serves as structured reasoning representation that enforces modular boundaries and functional contracts, thereby supporting coherent planning and yielding repositories that more faithfully realize intended specifications.
RPG induces complex data flows and dependencies. To illustrate the capacity of RPG-guided planning for generating complex repositories, visualize ZeroRepo with Qwen3-Coder the MLKit-Py task. At the file level, RPG organizes a coherent folder hierarchy; at the module level, inter-module flows define execution pipelines from data_ fata_lifecyyll through cluster ng and models to evaluate io;; and at the function level, inheritance and invocation edges capture intri- cate class interactions. These results show that RPG induces layered dependencies and coordinated execution, enabling repositories with both structural complexity and internal coherence.
7 ANALYSIS
7.1 ANALYSIS OF THE RPG'S SCALABILITY
RPG enables near-linear growth of repository functionalities. A key question in repository- level generation is whether functionalities can continue to expand with iterative planning over time and scale in practice, or whether growth instead stagnates. To examine this, we com- pute the number of planned features at each iteration on RepoCraft, averaging results across 30 rounds for strong baselines (Claude Code, Gemini CLI, Codex CLD) and for ZeroRepo.
8

---

Published as a conference paper at ICLR 2026
Comparison
As shown in Figure 5 ZeroRepo exhibits near-linear growth, surpassing I,100 leaf features with o3-mini, while natural- -lntuura----gggeee--ee based baselines show only limited scalability: Claude Code grows steadily but with diminish- ing rates, Gemini CLI increases slowly before converging by round 30, and Codex ceases features after 4-5 iterations,
Festure Counts
664
These results demonstrate that RPG provides persistent, extensible substrate that refines high-level goals into richer functionalities while sustaining structural consistency, ul- timately offering stronger and more reliable foundation for modeling complex repositories.
RPG ensures near-linear growth in repository size. Functional scalability matters only when realized in code. We measure this by tracking repository growth in lines of code (LOC) across iterations. As shown in Figure [ ZeroRepo sustains near-linear expansion, surpassing 30K LOC within 30 iterations. In contrast, natural-language baselines stagnate: Claude Code and Gemini CLI plateau at 3-4K LOC, while Codex stays below 1K. This gap flects a core weakness: natural language planning accumu- lates inconsistencies, producing fragmented specifications that fail to converge into coherent code. By contrast, RPG provides a structured, extensible representation where new functionalities are anchored in explicit modules, interfaces, and data flows. This grounding ensures expansions mate- rialize as code, enabling repositories to grow in size and integrity. The results highlight RPG's capacity to sustain scaling in codebase size and structure, establishing it as foundation for long-horizon code generation.
Figure 5: Feature comparison of Ze- roRepo (o3-min) mim against strong base- lines across iterations.
Code Growth by Reration
Caaw
Cloak
Figure 6: Scaling behavior of LOC across iterations on MLKit-Py.
7.2
ANALYSIS OF RPG's STABILITY AND INNOVATION POTENTIAL
RPG supports comprehensive and extensible func- tionality. A key challenge in repository generation satisfying user requirements while coherently extending beyond them. As shown in Table ZeroRepo scales coverage from 70.2% (Iter 5) nearly 96% (Iter 30), significantly surpassing baselines that plateau below 60% (Table 2) Simultaneously, it maintains 8% nov-
Table Coverage and Novelty of the Con- structed RPG over Iterations on MLKit-Py
Iteration
Cov. (%) t
Nov. (%)
5
70.2
4.6 (15.3/336, I)
10
80,9
5.4 (29.011/54..0)
city with over 100 additional features, whereas baselines
15
83.0
4.9 (39.0/796.0)
contribute fewer than 50. These results suggest RPG
20
85.1
5.2 (51.0/981.0 O))
25
87.2
7.0 (73.5/1043.0)
functions as a persistent planning substrate, enabling broad coverage and principled growth beyond reference
30 (ours)
95.7
7.9 (99.4/1258.0)
implementations (validated in Appendix F.3)
7.3 ANALYSIS OF GRAPH-GUIDED LOCALIZATION
Graph guidance accelerates agent localization. We evaluate the impact of RPG guidance by comparing lo- calization steps with and without graph support 4Ta
Table 4: Ablation results for Graph-
Guided Localization on ML.KitPy using D3.mini. Steps (mean + SD). wo/-
Across Integration Testing (Int. Test.), Incremental De- velopment (Iner. Dev.), and Debugging (Debug.). graph- guided search reduces effort by 30-50%. This shows that RPG equips agents with a principled navigation mechanism, enabling faster dependency tracing, more accurate bug localization, and smoother module integra- tion, thereby improving efficiency. Compared to natural language, RPG offers global structural representation of the repository, enabling agents to localize targets from umnttionlity-wiie wide perspective and accelerating the development cycle.
Graph" denotes ZeroRepo without Graph.
Category
InfTest
IneDey
Debug
ZeroRepo
6.2 + 2.1
6.8 1.8
+
wlo Graph 13.3
11.1
10.8 2.6
8.5 + 2.9

---

Published as a conference paper at ICLR 2026
7.4 ANALYSIS OF AUTOMATED EVALUATION
Automated evaluation is reliable and aligned with human judgments. tomated evaluators preserve the human ranking and relative gaps between ZeroRepo and Claude- Code across all metrics. In particular, 03-mini attains strong Pearson correlations with human scores (0.893 for coverage and 0.958 for novelty), indicating close agreement at the instance level. These results suggest that the automated judge captures the same semantic distinctions as human experts, supporting its use as a robust and scalable proxy for manual assessment in experiments. Detailed results are provided in Appendix E.5
Table 5: Comprehensive evaluation. Top: per- formance metrics. Bottom: Pearson agreement with humans.
that au-
Method
Metric
DeepSeck
GPT-
e3-anini
Homan
Profermance
Endluttion
Cov, (1)
53.4 1.1
58.7 1.2
52.9 0.8
50,5
Clande-Code
Nov. (53
0.6
8.9
6.3 0.2
4,0
(PVV)
36.7/52.1
35.1/49.3
36.2/52.0
0.8
ZeroRepo
1.3
0.8
(PN)
61.9/79.2
66.1/70.3
69.8/76.3
62/073300
Peorson Connisteccy vd Haman
0.8912
Peusson
0.8901
0,9587
ABLATION ON EPICODER FEATURE TREE
Covarge beft and Rewalyy (rghtt over Baracoss
LOC (rafti and Tolona otgati over herations
30000
Freefim Jul FFT)
Zacehoao beo 681
Zerfaaa yetn CmT
Rowlyy 11674
Figure 7: Ablation on the EpiCoder Feature Tree (EFT): coverage (left axis) and novelty (right axis) over iterations.
Figure 8: Ablation on the EpiCoder Feature Tree (EFT): LOC (left axis) and generated tokens (right axis) over iterations.
To isolate the RPG structure, we ablate the EpiCoder Feature Tree (EFT) on MLKit-Py (o3-mii mmmmminn.. We compare Claude-Codt (unstructured), ZeroRepo (w/o EFT), and ZeroRepo (with EFT), extending the EFT-free run to 40 iterations to assess convergence (Table 6 Figures
RPG structure drives scalability. Comparing Ze- roRepo (w/o EFT) against Staude-Code isolates the structural impact. While the unstructured baseline sat- urates at Iteration 10 (59.6% coverage), the RPG agent sustains growth, reaching 25,202 LOC and 87.2% cover- age by Iteration 30. This 7x volume increase suggests repository-scaal planning is intrinsic to the graph struc- ture, mitigating context limitations that stall unstructured models.
Table 6: Iteration-30 30 ablation on MLKit- Py (o3-mini).
Sartig
(9)
Nov. (96)
Files
Tokens
Clande-Code
59.6
0.0 (0/163)
51 3,559 37,086
ZemoRepo (wis KR)
87.2
6.4(62/74)
191 25,201
271
ZemiRepo (with KB)
95.7
7.9 99/1288)
266 31,596 351,554
EpiCoder Feature Tree as an Accelerator. Ablation confirms that while EFT enhances efficiency, not a prerequisite for capability. Removing EFT introduces temporal delay rather than performance ceiling: ZeroRepo (w/o EFT) reaches 97.9% coverage by Iteration 40, matching the EFFTennhnced version at Iteration 30. Both configurations exhibit robust linear growth in LOC (y) over iterations (x)). Strong linear fits for EFT-enhanced (y ss 983x + 2992, R² = 0.97) and EFT-free (y z 800x + 989, R² 0.98) settings indicate the RPG structure ensures stable planning, with EFT optimizing the initial context (intercept) and generation velocity (slope).
CONCLUSION
In this paper, we presented the Repository Planning Graph (RPG), structured representation unifying proposal- and implemennatioo-llvel planning for repository generation. Building on RPG, we devel- oped ZeroRepo. graph-driven framework that achieves state-of-the-art coverage, correctness, and scalability on the RepoCraft benchmark. Our analyses show that RPG captures complex dependen- cies. supports increasingly sophisticated planning through near-linear scaling, and enhances agents' repository understanding, thereby improving localization. These findings highlight the promise of graph-based representations foundation for horizoorioon, large-scale repository generation.
10

---

Published as a conference paper at ICLR 2026
REFERENCES
Amr Almorsi, Mohanned Ahmed, and Walid Gomaa. Guided code generation with Ims:: A multi- agent framework for complex code tasks. In 2024 12th International Japan-Africa Conference on Electronics, Communications and Computations (JAC-ECC), pp. 215-218. IEEE, 2024.
Anthropic. Introducing Claude httpss//www.abbbooocc.ccomm
2025a
Anthoopic. Claude code: Agentic coding command -ine tool, April 2025b. URL ColLhttp::/// www anthropic. com cwwudel
Ma' ayan Armony, Albert Meroño-Pchuula,, and Gerard Canal. How far are Ilms from symbolic planners? nip-based perspective. arXiv preprint arXi::2508.01300 2025.
Nazmus Ashrafi, Salah Bouktif, and Mohammed Mediani. Enhancing 1lm code generation: A systematic evaluation of multi-agent collaboration and runtime debugging for improved accuracy, reliability, and latency. arXiv preprint arXiv:2505.021 033,, 2025.
Maciej Besta, Nils Blach, Ales Kubicek, Robert Gerstenberger, Michal Podstawski, Lukas Gianinazzi, Joanna Gajda, Tomasz Lehmann, Hubert Niewiadomski, Piotr Nyczyk, et al. Graph of thoughts: Solving elaborate problems with large language models. In Proceedings of the AAAI conference on artificial intelligence, volume 38, pp. 17682-17690, 2024.
Arghavan Moradi Dakhel, Amin Nikanjam, Vahid Majdinasab, Foutse Khomh, and Michel C Desmarais. Effective test generation using pre-trained large language models and mutation testing. Information and Software Technology, 17 171:107468, 2024.
Dhruv Gautam, Spandan Garg, Jinu Jang, Neel Sundaresan, and Roshanak Zilouchian Moghad- dam. Refactorbench: Evaluating stateful reasoning in language agents through code. In Proceedings of the International Conference Learning Representations (ICLR) 2025, 2025. URL http://////pp.eeeeoooaaaaaoaaaoa 7b44ee77453e
-bbst Coneeaatooettttttttooooooeeeeeee
html
Google. Gemini cli: Open-source ai agent for developer terminals, June 2025. Accessed: 2025-07-22.
Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Peiyi Wang, Qihao Zhu, Runxin Xu, Ruoyu Zhang, Shirong Ma, Xiao Bi, et al. Deepseek-rl incentivizes reasoning in Ilms through reinforce- ment learning. Nature, 645:633-63 638, 2025. doi: 0011088//411366-055-009222
Sirui Hong, Mingehen Zhuge, Jonathan Chen, Xiawu Zheng, Yuheng Cheng, Jinlin Wang, Ceyao Zhang, Zili Wang, Steven Ka Shing Yau, Zijuan Lin, et al. Metagpt: Meta programming for multi-agent collaborative framework.
Yiming Huang, Jianwen Luo, Yan Yu, Yitong Zhang, Fangyu Lei, Yifan Wei, Shizhu Hc, Lifu Huang, Xiao Liu, Jun Zhao, and Kang Liu. Da-code: Agent data science code generation benchmark for large language models. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing (EMNLP 2024), pp. 18487-13521 Miami, Florida, USA, 2024. 2024 emn emm-pmain...........mmmmmmmmm...
URL htt.hpp://accaanoollog....o
Binyuan Hui, Jian Yang, Zeyu Cui, Jiaxi Yang, Dayiheng Liu, Lei Zhang, Tianyu Liu, Jiajun Zhang, Bowen Yu, Keming Lu, et al. Qwen2. 5-coder technical report. arXiv preprint wrXiv:2409..2/12 2024.
Carlos E Jimenez, John Yang, Alexander Wettig, Shunyu Yao, Kexin Pei, Ofir Press, and Karthik R Narasimhan. Swe-bench: Can language models resolve real-world github issues?
Haiyang Li. Mrg-bench: Evaluating and exploring the requirements of context for repository-level code generation. arXiv preprini ar0iv:2508. 250.009998,,9 2025.
Wei Li, Xin Zhang, Zhongxin Guo, Shaoguang Mao, Wen Luo, Guangyue Peng. Yangyu Huang, Houfeng Wang, and Scarlett Li. Fea- beach: A benchmark for evaluating repository- code generation for feature implementation. In Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL 2025), 2025. URL https: /t//aaaanttt http//aaaaatoooo...oooooooo acl ong.ong..o.. 8397
11

---

Published as a conference paper at ICLR 2026
Zijie Lin, Yiqing Shen, Qilin Cai, He Sun, Jinrui Zhou, and Mingjun Xiao. Autop2c: An Ilm-based agent framework for code repository generation from multimodal content in academic papers. arXiv preprint arXiv:2504.20115, 2025.
Yifei Liu, Li Lyna Zhang. Yi Zhu, Bingcheng Dong, Xudong Zhou, Ning Shang, Fan Yang, and Mao Yang. rstar-coder: Scaling competitive code reasoning with a large-scale verified dataset. arXiv preprint arXiv:2505.21297 2025.
OpenAl. Gpt-4o. https://openai: wttp://wc.iciiiiieeeeeeeee 40
2024.
OpenAl. Codex cli: Local-first terminal ai coding agent, April 2025. URL https://openaa
com/ com/ Lntroducing-codexx
OpenAL. Openai 03-mini reasoning model. System Card and official release (OpenAD,, 2025. URL / com/ htt///nddxx///ee pandde//ppeaa-o3-miin. Released January 31, 2025. Accessed: 2025-07-28.
Chen Qian, Wei Liu, Hongzhang Liu, Nuo Chen, Yufan Dang, Jiahao Li, Cheng Yang, Weize Chen, Yusheng Su, Xin Cong, et al. Chatdev: Communicative agents for software development. In Proceedings of the 62nd Annuul Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 15174-15186, 2024.
Minju Seo, Jinheon Baek, Seongyun Lee, and Sung Ju Hwang. Paper2code: Automating code generation from scientific papers in machine learning. arXiv preprint 2224::2500..1992 2025.
Giulio Starace, Oliver Jaffe, Dane Sherburn, James Aung, Jun Shern Chan, Leon Maksin, Rachel Dias, Evan Mays, Benjamin Kinsella, Wyatt Thompson, et al. Paperbench: Evaluating ai's ability to replicate ai research. arXiv preprint arXiv:2504.00 01848, 2025.
Haotian Sun, Yuchen Zhuang, Lingkai Kong. Bo Dai, and Chao Zhang. Adaplanner: Adaptive planning from feedback with language models. Advances in neural information processing systems, -2822022-2245, 2023.
Hongyuan Tao, Ying Zhang, Zhenhao Tang, Hongen Peng, Xukun Zhu, Bingchang Liu, Yingguang Yang, Ziyin Zhang, Zhaogui Xu, Haipeng Zhang, et al. Code graph model (egm): A graph- integrated large language model for repository-level software engineering tasks. arXiv preprint 169002 2025.
Qwen Team. Qwen3 technical report, 2025. URL htl.:tttt..///artt...rr/////////
Karthik Valmeekam, Sarath Sreedharan, Matthew Marquez, Alberto Olmo, and Subbarao Kambham- pati. On the planning abilities of large language models (a critical investigation with a proposed benchmark). arXiv preprint arXiv:2302.06776. 2023.
Evan Wang, Federico Cassano, Catherine Wu, Yunfeng Bai, Will Song, Vaskar Nath, Ziwen Han, Sean Hendryx, Summer Yue, and Hugh Zhang. Planning in natural language improves Ilm search for code generation. arXiv preprint arXiv:2409.03333. 2024.
Xingyao Wang. Boxuan Li, Yufan Song, Frank F Xu, Xiangru Tang, Mingehen Zhuge, Jiayi Pan, Yueqi Song, Bowen Li, Jaskirat Singh, et al. Openhands: An open platform for ai software develop- ers generalist agents. In The Thirteenth International Conference on Learning Representations.
Yaoxiang Wang, Haoling Li. Xin Zhang, Jie Wu, Xiao Liu, Wenxiang Hu, Zhongxin Guo, Yangyu Huang, Ying Xin, Yujiu Yang, Jinsong Su, Qi Chen, and Scarlett Li. Epicoder: Encompassing diversity and complexity in code generation. volume 267, pp. 63569-63606. PMIR, 2025. URL mlr.rrr......................... wwt.:mmw......ee..mm.mmmmmmmee. www.s/vesssvv.. wtt.5///waggbbi.ic hhmml
URL https://proceeiing wwwpeeeiiinnn........m..m
Junhan Yang. Jiahe Wan, Yichen Yao, Wei Chu, Yinghui Xu, and Yuan Qi. inf-retriever-v11, 2025.
2025.
Muhammad Nouman Zafar, Wasif Afzal, and Eduard Enoiu. Evaluating system-level test generation for industrial software: A comparison between manual, combinatorial and model-based testing. In Proceedings of the 3rd ACM/IEEE international conference on automation of software test, pp. 148-159 2022.
12

---

Published as a conference paper at ICLR 2026
Huaye Zeng, Dongfu Jiang, Haozhe Wang, Ping Nie, Xiaotong Chen, and Wenhu Chen. Acecoder: Acing coder rl via automated test-case synthesis. arXiv preprint arXiv:2502.017/8, 2025.
Linghao Zhang, Shilin He, Chaoyun Zhang, Yu Kang, Bowen Li, Chengxing Xie, Junhao Wang, Maoquan Wang, Yufan Huang, Shengyu Fu, Elsie Nallipogu, Qingwei Lin, Yingnong Dang, Saravan Rajmohan, and Dongmei Zhang. Swe-bench goes live! In Proceedings of the Thirty-Ninth Anmual Conference on Neural Information Processing Systems, 2025. URL https: //rxiv
org/ /log/abs/2e 250/2225552233
Wenting Zhao, Nan Jiang, Celine Lee, Justin T. Chiu, Claire Cardie, Matthias Gallé, and Alexander M. Rush. CommitO: Library generation from scratch. Proceedings of the International Conference Learning Representations (ICLR) 2025, 2025. Utthttss::/ http://///peeeeeee.eeee.e newppdwwweee/
1d Md-Maa
Qihao Zhu, Daya Guo, Zhihong Shao, Dejian Yang, Peiyi Wang, Runxin Xu, Y Wu, Yukun Li, Huazuo Gao, Shirong Ma, et al. Deepseek-coder-v2: Breaking the barrier of closed-source models in code intelligence. arXiv preprint 1ttxxi::240.1193311 2024.
Yueheng Zhu, Chao Liu, Xuan He, Xiaoxue Ren, Zhongxin Liu, Ruwei Pan, and Hongyu Zhang, Adacoder: An adaptive planning and multi-agent framework function-level code generation. 2025.
arXiv preprint
13

---

Published as a conference paper at ICLR 2026
A THE USE OF LARGE LANGUAGE MODEL
In this paper, we employ large language model (LLM) for proofreading and icon creation.
B APPENDIX OF PROPOSAL-LEVEL GRAPH CONSTRUCTION
The construction of the RPG is central to our framework, as transforms high-level repository goals into a structured and persistent representation. The process starts with carefully designed prompting strategies for selecting repository-relevant functionalities from the global feature ontology, followed by iterative refinement ensure both semantic coverage and modular coherence.
B.1 ALGORIIHMM OF FUNCTIONALITY GRAPH CONSTRUCTION
Algorithm Feature Sampling with Diversity-Aware Rejection Sampling
Require: Root node R; frequency library F;; temperature t: per-tree sample size S; overlap threshold
P; maximum number of retries max
1: function BASESAMPLE(A F, S)
2: selected_set <
3: for 1 to 5 do
4: C GET SEET_CHILDRENAR
5: if C = / then
6:
break
7:
end
8:
Ja F[i] for
9:
P: < J:/ Ei/Ce Jj for i
10:
Q: 4 TEMPTRANSFORMER t) for all i C
cur_node <- SAMPLE (41 .2)...)) selctee..ettADDDeuu... Rt + cur_node
12:
13:
move root downward for next step
14:
end for
15: return selected_see set
16:
end function
17:
function REJECTSAMPLE(R, F,t, S, p. Tmax)
best T 0; best +- +00
19: T* <- 0)
for
T max do
b retry up to T Tmax times sample candidate tree I compute overlap with seen nodes
21:
Teand +- BASSSSAMPLE F, ovl + OVERLAP(T cands Swen
22:
23:
of bestovl then
24:
best ovl ovl; best T Toand
b update best candidate so far
25:
end if
26:
ovl S & then
1 -Tennd
accept immediately if overlap S threshold
28:
break
end if end for
30:
if T
then
32:
T < best T
> fallback: choose least-overlap candidate
end
34:
return 7
>> return the final selected tree
35:
end function
Rejection Sampling Algorithm We extend the base sampling strategy introduced in EPI-
CODER (Wang et al. 2025) by incorporating diversity-aware rejection mechanism, shown
14

---

Published as a conference paper at ICLR 2026
in Algorithm At each step, candidate tree is accepted only if its overlap with previously sampled nodes is below a specified threshold: otherwise, the tree with the minimal overlap is returned. This encourages broader feature space exploration under limited number of sampling iterations.
Repository-Aligned Subtree Selection Algorithm 2 outlines the procedure for constructing repository-specific feature subtree from global feature tree. The algorithm iteratively selects candi- date features based combination of exploitation (retrieving top-scoring nodes) and exploration (sampling unvisited nodes). At each iteration, an L.M agent filters and ranks candidates, proposes missing but relevant features, and performs batch-level self-checks ensure internal consistency. Accepted candidates are incorporated into the current subtree, and the process continues until a fixed iteration budget is reached. The resulting subtree captures features most relevant the target repository while balancing coverage and quality.
Algorithm 2 Repository-Speccfee Subtree Selection
Require: Global Feature Tree T; Repo description R; iteration budget K; top-k; termination
threshold т; batch size В; LLM subtree T
Ensure: Repository-speciie
1: Initialize current repo tree T + 0; missing features Cmissing +
0; visited set V < 0
2: for k 1........... K do
itterate with given budget select promising nodes (exploit) sample unexplored nodes (explore)
3: Exxploit - RETRIEVETOPK R. top-k:
4: Eexplore f- SAMPLEUNVISITED(T7
5: Candidate selection via LLM
6:
Cexploit + LLL..SELBEEEEPPOOTCAADDDDEESSSS Eexploit T', filter exploit candidates
7: Cesplare f- J.M...ELEECEEEPP.ORRCAAADDDDDEEE EEeppoore T,, filter explore candidates
8:
Cuissing <-
Cmissiigg WLL.LLL.L.PLLOO.L
PROPOSPOIIIOINNNTT R) generate novel candidates not in tree A merge all candidate sources
9:
Craw f Cexploit U Cexplore U Cmissing
10:
// Batch self-check (filter useful paths within each batch)
11:
for all batch B C Craw with B < B do
process in small batches accept only consistent/relevant paths expand repo-specific tree mark all evaluated paths as visited
12:
K" < LLM JLM.SSEFFCHECKTT: BB) T INSERTPATHS(T", V- VUB
13:
14:
end for
16: end for
return T
return final subtree
Repository Subtree Reorganization into the funeciionaityy graph The algorithm operates in three stages to refactor subtree. In the first stage, an LLM agent iteratively extracts meaningful features from the input, organizing them into subgraphs until sufficient coverage of leaf nodes is reached. In the second stage, the agent reorganizes subgraphs by merging semantically related components or moving branches across groups to improve structure. Finally, each subgraph is refined to ensure naming consistency and hicrarchical coherence, yielding clean, interpretable functionality graph.
B.2
DETAILED CONSTRUCTION PROCESS
Level
#Elements
Examples
functionality, data structures, data processing text processing, process monitoring, create flowehart heap allocation, dayjs, affine transformation
2
1,527
3
21,739
4
113,348
update record by ID. automated versioning. M5 Model Tree add vertices, angular velocity, find minimum by group, mark outlier data min with inclusion, multiple with keyword
5
613,311
6
33,801
7
781
validate against thesaurus, swipe event detection
Table 7: Statistics of the global feature across hierarchical levels, with representative examples from each level.
15

---

Published as a conference paper at ICLR 2026
2000
PyWeeEnngine
EttpEaay
1200
03-mini
1600
qwen3_comm coder
1000
Leaf Count
1400
Leaf Count
Leaf Count
1200
80D
1000
400
600
800
600
400
200
200
200
30
30
30
Iteration
Iteration
Iteration
MuKIt-Py
Tablekit
SymbolicMatth
1800
800
1200
1600
1400
Leaf Count
1000
Leaf Count
1200
Leaf Count
1000
400
600
800
400
600
200
200
0
10
20
30
0
10
20
30
0
10
20
30
Iteration
Iteration
Iteration
Figure 9: Evolution of Feature Tree Leaf Counts over Iterations Across Repositories, Highlighting the Differences Between qwen3 coder and 03-mini
Global Feature Tree The global feature tree consists of more than one million nodes across seven hierarchical levels (Table 77. reflecting a
Top-Lavel Feature Catagories Leaf Nada Count
den enccuuing on
Lavala Cataggey
broad and diverse functional knowledge base.
Nevertheless, the distribution of features across Level-1 categories is highly skewed (Figure 10) In particular, the data processing branch domi- nates the tree, while many other categories contain only a small number of nodes, resulting a pro- nounced long-tail distribution. Such bias is inher- ent to real-world software ecosystems, where data processing utilities are disproportionately preva- lent compared to specialized functionalities. As consequence, constructing repository-specife RPG requires large-scale filtering and reorganiza- tion in order to extract the most relevant features and mitigate the imbalance of the global distribution.
Figure 10: Distribution of feature counts under Level-1 categories in the global feature tree.
Model-Specific Growth Patterns Beyond the two traces in Fig. Concretely, qwen3-coder exhibits the most open expansion, with an approximately linear increase in leaf counts per itera- tion-maximizing coverage early but with higher risk of admitting loosely related features. 03-mini follows with moderately aggressive trajectory, striking balance between breadth and relevance. Together, these curves delineate different points on the recail-precision spectrum of subtree selection strategies that can be matched to repository needs.
From Global Repository-Speedfic Distributions The comparison between the global feature tree (Fig I0) and the final repository-speciin profiles (Figs 12) highlights the transformative effect of -udde- reorganization. While the global tree dominated by generic categories such as data process i and user interaction the restructured graphs consistently downweight these high-frequency but less discriminative categories and elevate domain-relevant branches to the foreground. This shift effectively counteracts the inherent long-tail bias of the global ontology, redistributing feature density toward categories that better capture repository semantics. As a result, the constructed graphs are not only semantically sharper but also more functionally coherent with respect to the target domain. Between models, qwen3-coder favors broad coverage with slower convergence and higher variance, whereas 03-mint achieves more balanced trade-off between
16

---

Published as a conference paper at ICLR 2026
compusation cawar
agerthr
Campen
monny
MO WoOOO entuncement
ersunble matheds
Leaf Count scale)
Leaf Count scain) Categories
PyWeblEngine
dil pocpitt
agerther
Canegory
cawar
Dr Categories
SymbolicMath- Festure Categories
compurtees CONDE
den
conmuuttioon apcrtai
Catregoryy
mebebi oon aiurceeeess
cat process
flgg
Figure 11: Final distribution of feature counts across subtrees for all repositories under 03-mini. The figure shows how features are reorganized after the iterative construction process, reflecting the model's preference in balancing breadth and precision.
generality and specificity. Together, these contrasting tendencies illustrate complementary strategies along the recall-precision spectrum, offering flexibility in matching feature selection to downstream repository needs.
reveal how repository-specific func- tionalities are consolidated into coherent modular organizations. Compared to the more diffuse subtree distributions, the resulting graphs exhibit markedly skewed allocation of functionalities across subgraphs: small number of core subgraphs absorb the majority of features, while peripheral subgraphs remain lightweight. This reflects natural modularization process, where dominant clusters correspond to central repository capabilities and minor clusters capture auxiliary or specialized func- tions. Between models, the partitioning strategies diverge: gwenš-coder produces larger number of medium-sizze subgraphs, favoring breadth and parallel coverage; whereas 03-mini yields a more balanced distribution, with several subgraphs of comparable size anchoring distinet semantic roles. These differences indicate that model-driven reorganization not only mitigates the global ontology's 's long-tail bias but also shapes the granularity of modular decomposition, thereby influencing how functional responsibilities are distributed within the generated graph.
Final Graph Structures
The final RPGs (Figure 13a 13b
B.3
PROMPT TEMPLATE
Parts of Prompt Templates for Exploit-Exxpoon Strategy in Subtree Selection
Prompt for Exploitation Paths
You are a GitHub project assistant responsible for expanding a repository's feature tree through path-based modifications ensure alignment with the project's goals.
In each response, you will be given:
An Exploit Feature Tree: A curated subset of high-relevance feature paths.
The Current Repository Feature Tree.
When returning selected paths, always use ""path/to/feature" format with / the separator.
17

---

Published as a conference paper at ICLR 2026
Ing
conputtiion
requett coniiee rntor
ageetir
Caleggry
paace
rnteest constructor
connectoe menagement
cimwraboe by roductio
Leaf Count (byg scale)
Last Count (leg scale) Categories
PyWebEnngne- Categories
Agrtter
zoe terector
Canmsgory
Gasaee speat
4M0 po0.004
di atauue
SymbolicHath Feeture Categories
Feature Categories
dit poccsskn
Cattepory
und
optimbattor
Agrrttrr
diovia retrrnatioo
Count (leg
Count (leg scate)
Figure 12: Final distribution of feature counts across subtrees for all repositories under qweni-coder The figure shows how features are reorganized after the iterative construction process, reflecting the model's preference in balancing breadth and precision.
Objective (excerpt)
Expand the Repository Feature Tree so it: 1. Aligns with the repository's purpose and scope.
2. Achieves broad coverage across functional areas.
Ensures essential capabilities are represented.
4. Identify and fill critical gaps.
Selection Principles (excerpt)
Select exclusively from the Exploit Feature Tree.
Include all non-duplicated, useful paths.
Maintain structural balance by covering underrepresented modules.
Exclusions (excerpt)
Skip generic infra (e.g., logging, configuration) and abstract goals (e.g., "optimize CPU usage").
Response Format
Respond only with Thought and Action. <think>
Reason about relevance and gaps in the Exploit Tree. </think> <action>>
"ll selected feature feature.paths"
"path/to/feature"...
</action>>
Prompt for Exploration Paths
You are GitHub project assistant responsible for expanding repository's feature tree through path-based modifications to ensure alignment with the project's goals.
18

---

Published as a conference paper at ICLR 2026
PyWab
StatModdter
Higguay
lede MURtsPy
TableiCe
SymbolliHMtth
WVb
unsentt
30
1O Hocks
(a) 03 -mini
Pyllebttngine
StotModeler
Hilptary
Eclever
lapis
HTTCC
#Lait ticdal
MUX0-Py
TTablekitt
SymbollcMath
Festing
wanccorpcom
spli
#LAY
Ewwen--coder
Figure 13: Leaf node counts distribution across feature subgraphs in each repository RPG, reorganized by different models.
In each response, you will given:
A Sampled Feature Tree (Exploration Tree).
The Current Repository Feature Tree.
When returning selected paths, always use ""pathttoffeatrre" format with 5/" as the separator.
Objective (excerpt)
Improve and expand the Repository Feature Tree that it: Aligns with the repository's purpose and usage scenarios.
Achieves comprehensive coverage of core and supporting areas.
Selection Principles (excerpt)
Select only from the Exploration Tree.
Include actionable, aemaintrelevant features.
Skip paths already present in the current Repository Tree.
Slight over-inclusion acceptable.
Exclusions (excerpt)
19

---

Published as a conference paper at ICLR 2026
Do not select generic infra (e.g., logging, config) large-scale features (e.g., cloud integra- tions).
Response Format
Respond only with single <think> and <actions block. <think>
Explain how each Exploration Tree path was evaluated and why it was included or excluded.
</think>
<action>>
"all_selected feature.paths":
""path/toffeature...
</action>
Parts for Prompt Template for Retrieving Missing Features
Instruction You are GitHub project assistant tasked with designing a functionally complete,
production-grat gradde repository.
Your goal is to identify and recommend missing functional capabilities algorithms that the project should include, based on its real-world purpose, scope, and domain expectations. Focus intended functionality not the existing Feature Tree, which may be incomplete. Objective (excerpt)
Identify groups of functionally concrete features that: 1. Align with the repository's 's domain and purpose.
Are entirely missing or only superficially represented.
3. Are specific and implementable (e.g., functions, classes, modules, algorithms).
Inclusion Criteria (excerpt)
Must be code-level operations (computation, transformation, algorithm, evaluation).
Realistically implementable within the repository's 's scope.
Both standard and advanced algorithms are allowed.
Exclusion Criteria (excerpt)
Do not include abstract intentions (e.g., "improve accuracy"), generic infra (e.g., logging, connectors), placeholders, or duplicates.
Naming Rules (excerpt)
Use 3-5 lowercase words, separated by spaces,
Each leaf node must describe a concrete algorithm or behavior.
Avoid vague terms, camelCase, snake_case.
Structure Guidelines (excerpt)
Organize into logical hierarchies (up to 4-5 levels).
Reflect computational architecture, not documentation taxonomy.
Response Format Respond with ONLY a <think> and <act faction> block: <think>
Reason about functional domains, workflows, and algorithms that are missing from the current Feature Tree but expected in real-world use.
</think>
<action>>
features": "missing.features" "root node": { "child node 1": "lleaf feature 1", "leaf feature 2" ]. child node 2": "leaf feature 3". "leaf feature 4" }
</action>>
C APPENDIX OF IMPLEMENTATION-EE GRAPH CONSTRUCTION
This section illustrates how the RPG is enriched with file organization and function design to form concrete code structures.
20

---

Published as a conference paper at ICLR 2026
C.I PROMPT TEMPLATE FOR IMPLEMENTATION GRAPH CONSTRUCTION
We provide the prompt templates that guide the transformation from graph subtrees into modular code skeletons.
Prompt
You are system architect tasked with designing the inter-subtree data flow for Python software repository. Your goal to define how data moves between functional modules (subtrees) including who produces it, who consumes it, and how it is transformed and express this structured, directed graph.
## Data Flow
### Format
[{ "frrom": "<source subtree name>", "to": " "<target subtree name>", "data_id"; "<unique name description of the data>>, "data type": "<type structure of the data>", "transfor- "<summary of what happens to the data, anything?"" }....
mation":
### Validity & Structural Constraints
2. Full Connectivity Required
Every subtree listed in [trees_names} must appear at least one edge.
No subtree should be isolated or unused.
Acyclic Structure
The data flow must form a Directed Acyclic Graph (DAG):
Field Guidelines
'data_i'': Use unique, descriptive names to identify each data exchange.
"datattpe": Use precise and interpretable types to describe the structure, format, or abstract role of the data being passed.
"transformation:: Describe how the data modified, filtered, enriched, or combined. unchanged, say ""none":
## Output Format <solution>
{{
"from"" : "to": "..." "data id": "..., "data type": "...", "transformation": "...""
</solution>>
Parts of Prompt Templates for Raw Skeleton Mapping
You are a repository architect responsible designing the initial project structure software repository in its early development stage. Your task is to design clean, modular file system skeleton that organizes the repository into appropriate top-level folders based on these subtrees.
## Requirements
1. The folder structure must clearly separate each functional subtree and reflect logical domain boundaries.
2. Folder names must be concise, meaningful, and follow Python conventions (e.g.. 'snake 'snake_case"), Names should feel natural and developer-friendly
3.
Folder names do not need to match subtree names exactly.
Treat subtree names as functional labels.
Rename folders needed for clarity and convention, while preserving the correct mapping.
When assigning a subtree to a folder, include the exact subtree name in the mapping (c.g..
"'ml models": M"Machine Learning"]")
You may choose flat structure (all folders at root level) or a nested structure (e.g., under 'sre'), depending what best supports clarity, organization, and practical use.
5. Include commonly used auxiliary folders as appropriate.
6. The proposed structure should balance clarity, scalability, and maintainability. Avoid
21

---

Published as a conference paper at ICLR 2026
unnecessary complexity excessive nesting.
## Output Format
Return single JSON-style nested object representing the repository folder structure:
"'folder name": ("Subtree Name"]' means this folder assigned a specific subtree. The name in the list must match exactly what appears in the given list of subtrees.
"folder.name": [ means the folder exists but does not correspond to a specific subtree (e.g., utility support folders).
"fllle name.ext": null" indicates the presence of a file. File content is not required.
Prompt for Mapping Feature Paths to Python Skeleton Files
You a repository architect tasked with incrementally assigning all remaining leaf-level features from a functional subtree into the repository's file structure. This is an iterative process, You are not expected assign all features once each round should make clear, meaningful progress. Your ultimate goal is a production- -grde file organization that evolves cleanly and logically over time.
## Context
In each iteration, you will receive:
A list of unassigned leaf features (each is full path like ""a/bc"').
A designated functional folder under which all new paths must begin.
A partial skeleton showing the current structure (existing assignments are hidden).
Assign the remaining features to "py" file paths that:
Begin with the designated folder.
Group semantically related features together.
Reflect how real developers would modularize logic in production Python codebase.
Prefer organizing major functional categories into subfolders when appropriate.
### File & Folder Structure
Group features by functionality into logically meaningful modules that reflect real-world development practice.
Avoid bundling many unrelated features into single file
If folder contains 10 more files, introduce subfolders based semantic structure (e.g.. 'format',, 'elient/', 'csv/") to keep directories manageable.
### Naming & Organization Guidelines
### Examples
## Output Format
You must structure your response in clearly separated blocks, each wrapped with the appropriate tags:
<think>
Explain how you grouped the features into logically coherent modules with clean file and folder structure.
Describe how your choices improve clarity, minimize clutter, and reflect good design princi- ples.
</think>
<solution>
"<<path to filel.py>": ""feature",, "feature2" J, "<path to file2.py>": ("feature"")
</solution>>
22

---

Published as a conference paper at ICLR 2026
Prompt for Converting Subgraphs into Base Classes
You are expert software engineer tasked with designing reusable abstractions and shared data structures for Python codebase.
## Base Class Output Format
You must return your design as set of code blocks grouped by target subtree and file: ## General
### path/to/file.pp
"python
## <Subtree Name>
### path/to/file.py "* "python
</solution>>
## Design Strategy
Abstractions must follow system structure and dataflow analysis, not mechanical repetition.
Shared Data Structures: define for nodes with high out-degree (outputs consumed widely). Good candidates are feature batches, inference results, or training containers. Create a global type only when field names, data types, and usage context are stable and consistent.
Functional Base Classes: define for nodes with high in-degree (consuming many inputs).
Use when multiple modules share roles (e.g-, cleaning, predicting), follow common lifecycles ('runO', 'build(),, 'validate()),, rely on similar hooks. Principles:
Avoid speculative abstractions.
Prefer fewer, well-justified classes (typically 1-3 globally).
Capture structural commonality that aids extensibility and coordination.
## Output Formate
Wrap your entire output in two blocks:
<think>
</think>
<solution>>
## SubtreeA
### path/to/file.py
python
</solution>>
Prompt for Mapping Feature Paths to Interfaces
You are designing modular interfaces for large-scale Python system. You are given repository context: overview, tree structure, skeleton, data flow, base classes, upstream interfaces, target subtree, and target file.
### Objective
For each feature, define exactly one interface (function or class).
Provide imports, signature, and detailed docstring (purpose, args, returns, assumptions).
23

---

Published as a conference paper at ICLR 2026
No implementation: "pass'.
One interface per block.
### Design Guidelines
Function: simple, atomic, stateless.
Class:
stateful, multiple methods, inherits base class, or extensible.
Prefer fewer, well-justified abstractions.
Group only tightly related features.
Use global abstractions sparingly.
### Output Format
Use two blocks:
<think>
reasoning
</think>>
<solution>>
design_itte ffor feeturefeetturss=ffettrrr/e
"python
# One interface (function or class) with docstring and pass
</solution>>
C.2 CASE OF BUILT SKELETON AND DESIGNED INTERFACES
We present the skeleton generated by 03-mini, together with the mapping between the generated skeleton and the nodes of machine learning algorithms. In addition, we illustrate one two designed base classes as well as concrete functions or classes.
Repository Skeleton
al liggorttthgg hmaa/ advanced/
enhancements/
general__chancemente/
init__py
haw.raaeeeeaanneeettt...... s.
optimization_end wtw.mmttea_eeeiggg.eeeee
regressior_emhancemmntssss
init__py
extendes__echhsiques
axtended_mmthedds/
_init_
valiettioo_anddcchateeriii
new_models/
_init__Py -PY
new_modei_techniquuee
new
init__py
baseliaes.py
supplemsital_elgoritte
iiit__-y
tonal lillttttttttttt s___mthhodaappp
btr_oo__eedddddepeeeeeeeeee percept ron.
lttpooo__ddd_fttrra_aaaa importane__mee met hothods.pp
diverse__algo:iiimmt__eemmme
essemblle_vvalueiiinneddddd
www.aaadddeeeall_eeeeeeeeeeeeeee det.ection.ppy
meta_.gtaaiaatiionmmciddd
tokenizattiormmeccodd...
init__Py
engemble_tree/
24

---

Published as a conference paper at ICLR 2026
boosting_baeging/
boosting/
_ini__.py
boce:iigg_edvaness_eeee
acooring__lgprithmm...
hooos//ng_paramstrrrttoo
stacking_voting/
primary.py
secondary.py
init__Py
bagging-FV
decision_Lrees/
_ini__-py
gisddient_boottiinntteeeee
1d3.py
post_prrnning-py
randon_ forest.__
regression_tree.py
init_.py
regression/
lineer_modele:
iiit__pP
lasso.py
puitiple_limerr.e
polynomial.py
simple_linear.py
iiit init_.py
stastis_set_.egresssio...
cidge_classifier.py
supersiised/
claasification/
logistic/
_init_.y
cast.py
optimisattton.pp
sigmoid.py
init__PP
decision_tres.u
knearest.py
naive_bayes.py
pipport_vector
Lnit
unauperviced/
cluatering/
init__.y -py
advanced
kmeans.py
suppl teaetaaemttttttertaaina
dieensionsllly_eeeeici.oo
init
extended_dr.pp
keinel_pea.py
pct.py py
Init init_ppy
jata_conversioon
iiit init__.p
api_requests.py
featura_ancodingg.
resture_extraatiio..p
[ormat_
Sammst_oonversion.pp
sql_queries.py
data_Lianstorm/
Eilser_advanced.pr
Hilter_basi.pp
joln_operations..p
ccaling_edvanced,
scaling_besic.py
sorting-py
splitting.py
numerics/
_iitt_
basic_etaLlstieepp
tlock_multiplicate
decompocitions.gy
25

---

Published as a conference paper at ICLR 2026
dot_proddctee.p
inversions.py
fatrix_factoriiatiion.y
matrik_ressrrngeeestt...e
regressiion_statistt
spacse_lu.py
pregrocessing/
_init__py
cev_ic.py
data_cleaning.py
dimonsienolltty..nnallla
json_io.py
log.transformations
nolee__ugmenialiion.pp
init_.py
dats_processing/
analysts_pipel11
analytical/
_init_pp
aggreepattooaalgorichms..pp
dela_pee:turbatioo..yy
jata_guery-Py
join_operatians.py
cample_parcilionn.py
sessonai_analysis.py
eipeline_ueilltties/
_init_.py
jata_strremingg.F
learning_aetup-Py
model_valldation.
pe:formence_mmtrricsuu
init_.py
cleaning__roppratioon
advanced/
init
dup baate_handling
imputatior_methode
outlier_detection.
preperation/
_iit___p
data_splitting..P
impatacier_labelllnn
validation.Py
ipit_.py
sype_conversion.py
integrattion_narge/
_init_
aggceqation_relzzie
rerge_operations.ps
merge_search.ep
integretior__soragee
-init_
opi_operattoons.yy
conversion_auth..y
diesionary_coofige.p
export_integration.pp
io_logging.py
menipulation/
Liii
data_manípulatioon
shuffling.py
string_pivet/
pivoting.py
stringg.perrations..y
__it__ppy
Lesture
File_io.py
Lexi_entanceemetts.pp
utilities/
inyt__By
netadaLa.py
metrics.py
parallel.py
26

---

Published as a conference paper at ICLR 2026
ppacse_sLoragee
text_processinng.
Va. validation/
inspection/
svervieM-py
sorting-Py
stetistics.pt
data_innegriityPP
-ni .PY
extendad_evall
diagnostica/
_init_.py
model_quality.pp
statistical_diate
temporal_enalysis.py
.p
exglainabil
preciotiveeasseeseeene
robustness.py
mach_utils/
algorithms/
core_tecaniques/
init__Py
fon.py
natrix_operations.c
orrimiiatiion_add...
otr.roootooodddoociioooooooooo seleccion.py
mettis:ical_methodde init.
auxiliary/
init
late_manipulation..yy
gemeetric_cperstiinss..
gradieont_nn_iimgiin
rath_computations..2
ml_utilities.py
opt imiiatioonn
outlier_yalidation
sandor__perations.pp
sensor_ond_likelllen
text_proceastng.pp
time_processing.py
deta_ppreproeesaingg
_iit___py
Model_pprrsistence.pp
sampling.py
text_tools.py
performance/
init .y
drf drl*t__etcction.pp
statistical_testss
systen_monitoring..t
time_seriie__mmalyiii..
-intt_.Pp
advancee_ennalysis.pp
data_resampling..yy
evaluatiion_met:icss.
hyperparanater__unnngg
bodel_export_and_.cr
mnlin__lesrrnn___uppo
pipeline_creator.pr
simulation/
init__.p
hypesparameter_tunnee ppy
nearest_neighhorr_cem searchhhrrerrrppy
randor__ampling.py
simulation.py
time_interpolation..g
statiatiica__nnalyylle//
-ini _init_.py
deseripcive_atata.
inferontiall_aethddbp
ruliivarriae__anlyyyl...p
probetilistic__cdeels
aurvival_enalysis.pp
27

---

Published as a conference paper at ICLR 2026
lete__serie___mdeeeeee .py
varlance_mett caannccmeetrios
init_..p
compute/ ml ml__ettheda/
_1ntt__Pp
clustering_metthode
swn_valldation.py
opt iotimiaaion/
__iit_.Py
training_oontrro
stat stat_innfeeece//
-iiit
hypothesis_testt__ddie
advanced.d
hypothesis_tests_ basceessssssss.ppy
rodel_evallatton metrics-py
parameter_estiimtiion..
staiartice__esstt_mtiiii
ime_sseriee/
distriburien/
iiii__
bayestan_like
Distributtooo_eesimatiiii
..ffereatial..yyohhhsss...
infeernttial_voriincee.pp
init init__py
forecast_plots.py
foreaatting_alggrrtthm-.-.
init_.py
model_ model_eval/
deployment/
Liit
deployment_ops-.y
teocing_recources.pp
eva evaluation/
diagnestica/
init__py .py
advancce__diagootticccu
pe:tormance_metriion
init
aedittioni__aallyll..pp
ecros_dieplay.pp
regressiion_daanoostie
training_ogs.pp
management/
_ini
counterfactual.pp
Testure__xplenatioos
hyperparametes.jy
persistennceoops.pp
e:pelling_inttagratloon.
unce:tainty.py
visualization/
-nit__Py
dashbcard.pp
visuel_reperts.gy
init__Py
peline/
daLa_cleaning/
dimersiionsllty.py
encoding.py
filtering-Ey
imputation.py
knn_methoda..pt
merging-Py
deployment/
init_..p
export.py
intogration_reatinng
interactive ive..r.tee.py
nonitoring.py
enline.pp
28

---

Published as a conference paper at ICLR 2026
ptandard_ppooss.yy
yyand_anallysts.ppr
evaluationn
_p
cross_validation,
ooLooptimieatioo..yy
netrics_plote.py
nisc_evaluation.pp
monitcring.py
Lesture_.ngiineecinng/
_init__py
tasie_transformatiioo
extract:oon.py
interaction.py
selection.py
synthesia.py
erchestration/
_init_
con:iguratioo.pyy
startug.py
transformezs.py
workflom.py
preprocessing/
initt_..
advanced_Parraing.ry
imputation.py
inpat_validation
training/
Lintt
training_edfustmmettt
eroiniigg.trrategies..p
tuning/
_init_.pp
calibration.py
evolution.py
gaussian.py
reta.py
parzen.py
1yy PP
ul/
interactivity/
domain_commander
gene:al__ormezde.pp
help_aupport-py
navigattion_actionss
visualization/
iinit init__ppy
dashboard.gy
atandard_charrts.py
Init__py .py
init init__Fy
main.py setup.py
SubGraph-to-Skeleton
Machine Learning Algorithna
AdvancedExtensioon [-> ero/allporittmm//dvveeeedd
Xiscellaneous [ dir: arc/ httprr/ithaa///aaeeeeeeeee wtwpe//ee//pppem/allaal.ma..aala algorrithms
Otheralgoricims file: szc/elgorithms adwancedd/dd/aaaaaa suppnccdd//ppplleee sugplmmettal.alpcctthe
fisaificatioo
fea.ure
fila: ara/clgorithms/
advanced/ wtt.__sww_oppiaieeeieennneeeeee
regression_ande
file: src/algorithms/
advanced/ huppne/////ppppleeeeeeeeee h emental mmantal_algorrthhee
advanced htw.ae/eeeeenntttieeeeeiooaaaaao
erc/algo:ithms
/advaazed/
file:
svm
files arc/al aoraehm//ddvvee
py, file: sac/:lllssttmm/vvaeeedssssu.ssse
file:
ere/
Beranced_tokaiizzee htt.:/accccccoooooooeeeeceeeeeae file: sec/algorithms/advame
aupplemertal hwwperttallllggotttt///aaaaae www.eaaltttoooaarore.ooooooooooa lacorall__detee donection.pe
29

---

Published as a conference paper at ICLR 2026
file: stc/algoriiimm http:////w....//////////////aaaa
tokenization__men hotprattooonmeth........... les ara/al
Extendeddeecaniques
Extendedkethods [-> file: are/ htwpr//thhn//iaanncc///ttttetee
ion_and and model ledel_ files are/
NewModels src/algorithms//atioo
rew_model_techhiiqee file: ttt//algo...lim//daaaee....
http:eeeeeeeccoeeeeeeeeeeeeeeeee Baselines [+> file: http:////........../////////a/aa
Eahancement Bcbancceenn--nnnGGeeeen diz: ac0/A//poriaamm////vaaeee////eee
GeneralSnihancementts [-> file: wtt.a//pcc.iiie////////////aaaaa
file: sze/
regression_an'te wwt.onnnnnnnmmettsappommmmm file: http:///hera////ooo///////aaaao anhancomentss/
anterraaeenannmmmeets//aaee http:////o.tteeeaaaaaaaaaatteaee files sro/
rise_generaLeenhhtcemmetsssss file:
Regrassion (-> dir:
LinearModals |-> dir: arc/a http:////rrrtttt///e/reeestiiio [
Multipletinner: [ file: http////pprtii//////sss///////es
mitiple__iness
files htt/:///tiii////sss////////ooooa
.pyl FolynomislRegressin
file: htt.:///.....ha///gg/s//////aaaa
polyncmial.pyl
basec [-> file: ott.a//ee.......a..ao..ooaoaaaa.
file: attpl/oooottti//rreeeeei/////nno
SimpleLinear ( file:
.pyl
ocher [-> dir: acc/allarttemm//eeeesioo
Ridgefegresstmmlaassll
file: arc//l/pprttmmm//erreeinnn
clägg_classifier.ppr
FlastioNet [-> file: are/a o httprrtieeeettteessssooooo
Unsupervisedleerniige (-> dir: stc/elyp::illmm/mmuupeeree
Dimentae Dimenssionallttyeedcttiioo [-> dir: sxc/alecrtthmss/usspeevviee
demencionllityrrddeett.
VernelMethen [-> file: http:////gatttt/t/////////////// rey_utdtctioon
OtherDR (-> file:
extended_dr.pry
PCR file: http://............/////a///aaaa
cluttering [-> dir:
xHeans file: htte://cttt/mmmmmgeeer//aaaaaaaa
Otherclusterimn [-> fi atdpa//agrrrrrie//////ssaaaaaaee
supplemental_
AdvancedClusterringg [-> file: arc/al http://///...ttt////////////////
Rasesaooo eevancee_ccusteringe
Easeablesiinn
EnseablehndIreetellloo [-> dir: sece/allooiiittt/eeemsleee.eeer
Boostinglagging [>> dir:
Stacki inVotting wtr__ottirrg/ecoddayy.... /tt///ecooddyyypy.oe/ file:
file: sre/al wttp:allggtttteeeeeeeeeeee
stacking_voting///
tree/
bogetiing_boggin//tteeeeeee Seschmm/ensemiin /ewbmbenmmmmeeeeeeemee tree /boost
Bagging [-> file: sre/algorithmm
bagg ng/ http///ggggg......./.....
GradientBoosting > file:
bagginggg//
gradiens_boostinng.ee
Boosting [-> file: att/algorishmm///nneeeo censemble_rree//bbtttiiin...ano.
boost.ing_llgort
tile: http:///oo.t.tttt/eemmmm.e.ee.
adg_nnee fealures. file: aze/
algorithms/ htt.://mseeeeebeeeeeeeeeeee wetee//ecciiioatteeeee drccissoi.reee/geeeeent.... /tt.es//eeeeenn..oosaa...eeeee.
Lree/boosting http://oossitooppaaeii//oooooooo
bestiing_parmmmter_tuuu
Decisionirees [=> dir: htappo/schhss/eemmmmeeee.e.
ID3 [+> file:
RegressionTree file: att.
regression_tree.
Gradient:ccotiingizee [-> file: htt.://r......ia...a.aaaaaaa.eea
FostPruning file: http:///g........o.m.aoa.aaaaaaa
poat_pruring.py) RandemPorest
Landoo_foores..pt
Sapervisedlearrinn [+> diz: aet.ppecchh//eeerrraaaaaaaaaaaas
30

---

Published as a conference paper at ICLR 2026
Claseicication [ die: http////.........oeee////////aie
cRegressatnn [=> file: arc/algori pte//algorttmmm//gaaerttnae sed/class float flcation/
rile:
Lagistiic/ogtimiizsiiooo file: asc/alppccithmm//ueevvieede
Suppert [-> files htt.a///a..a..//////////////////
decision_tree.py Iile: Bddaaggtttaa/sspprrrr////saaaaaa
suppor__vecte..py
NaiveBayea [=> file: http://eeeiiiieeee///////aaaaaaa ]
file: art.gs/ssr//rooiiaaaaaaaaaaiiair
Base Classes Example
# Seneral
#4# (egeeetaallbaeee_coooeeeeete
11 "'yython
from abe import ABC, abstractmathod
clasa BaseComponent (ABC):
6**
Abstract base cláas for core pipeline componenta.
Provides standard lifecycle: -initializa)): Prepare the componant. proccss Process irput data. finalize() Clean up zesources.
Concrete subclasses must implement process
def Lnint____n (self) +> Kone:
ieltiaiitdali
initialize None:
Set up the component before processing starts.
selr.initfallzed True
Rebstrectmethed
process isels, datal:
Process the input data.
Arga:
Iaput
predefined
Returns:
Ousput data after processing.
pasa
(see))
Tear down clean up componant proceasing.
ielf.initialiied False
class Bst Cetimatorremponee :DaseConponent))
nn*
Abstract base class for estimator components (e.g., madels).
Defines the contract for model training and prediction.
Rebstrectmethed def fit (self, training_datal Kone:
Train the estimator using the provided training data.
Arga:
training_date: batch conforming shared TrainingBatch
pass
31

---

Published as a conference paper at ICLR 2026
@abstraclmethhd
def predict fsellf, input_data)):
Senerate predictions based input data.
Arga:
datut_data: Data in Tormat apecicied by the pipeline.
Returna:
Predictions coczespending to the input Seatures. .
pass
def process iself, dalal:
estimator, proceas defaults prediction. return seldipredict (data)
Designed Interfaces
#
Fecture Patho:
regu arizationn,, ficting", httpeeaaa/////eeoooo///////////a Xegression with ynomt mode regression", regression', "Ragresaion/ Regreasion with requlariaation"
LineacModels/ #4# szc/ http://ph.....eeeeeeeee/e/eeee/a from scc.genssa..aaee_cmmreee. class heppmm/lllgsssoorossttttttttotoo
kstimatorcomppeent
def Linit_ (self, degres: int, regularizatien lombaaa float=0.0) >> Kone: pasa
def (self, X: list (flost), y: 1ist [floaa]] None:
Fit the polynomial regresaton model to the provided data.
constructs the polynomial features based on the specified dagree and applies
optional reqularization requlariiation_lmbee la provided 0).
Args:
(list [float]]: list of feature values.
(1ist [float]]: 1: 11st of target va lues correaponding to the features.
Returns: Nose
Raises:
ValueError: the degrae supported input lists ampty miamatched.
pass
def predict isels, (flcaa)) [ist(S]
Generate predict tons using the fitted polynonial regression mode...
Trensforms input Seatures
polynemial features and computes
the output via the fitted model coefficients. Applies regularization adjustments is the model was fitted with zegularization term.
Args:
(floattt)
feature
prediction.
Returns: list [flott] A list of predicted valuea.
pass
32

---

Published as a conference paper at ICLR 2026
C.3 PATTERNS IN IMPLEMENTATION- GRAPH CONSTRUCTION
The mapping from RPGs to code structures exhibits a strong isomorphic relationship: each subgraph corresponds coherent code region, with files, classes, and functions serving structural anchors. Table illustttss this correspondence for the case of 03-mini during sk laarn generation, where algorithmic subgraphs (c.g., ML Algorithms. Data Processing, ML Pipeline) map to a larger number of files and functions, while auxiliary subgraphs (c.g.. Diagnostics. Visualization) remain compact yet feature-dense. This pattern reflects the semantic granularity of different subgraphs: core com- putational domains require broader structural scaffolding. whereas specialized domains concentrate more features per unit. Extending to the crosstrepy view Table g we observe that both models preserve this structural isomorphism but with distinct emphases: 03-mini tends to distribute features more evenly across units, while qwen3-coder consistently produces the highest feature densities, especially at the class level. Together, these results demonstrate that the graph-to-code translation process not only preserves the hierarchical semantics of the RPG but also manifests in distinct structural footprints that vary with model choice.
Table 8: Per-repository structural statistics across 03-mini and qwen-coder. "Count" = number of entities (Files/ClassesFunnciions repository; "Avg Feat." = mean number of features per entity (features per file/class/functiion..
03-mini
qwen-codee
Repo
Files
Classes
Functions
Files
Classes
Functions
Count
Avg. Feat.
Count
Avg. Feat.
Count
Avg. Feat.
Count
Avg,
Count
Avg. Fear.
Count
Avg. Feat.
TableKir
475
3.64
252
2.28
1092
1.05
271
6.69
496
1.62
587
1.50
MLKityPy
266
4.74
321
1.64
708
1.04
566
2.44
815
1.30
281
1.13
StatModeler
4.71
2.47
726
330
1.07
SymbolicMath
126
4,70
95
2.17
370
1.04
89
8.98
71
1.73
786
0.86
Py Webhinngiee
440
3.89
576
1.74
689
1.02
482
3.52
890
1,26
501
1,13
HappEacy
104
4.17
2.43
235
1.03
178
1.52
1.06
Average
271.7
4.31
240
2,12
636.7
1.03
319.3
4.90
514
1.44
488.7
1,12
Table 9: Structural distribution of files, classes, functions, and feature densities corresponding to each subgraph in the feature graph of 03-mini during sklearn synthesis. Here, "Files/Classes/uume tions" denote the number of code units mapped from each subgraph; "File/Class/Function Features" the total extracted features; and "Avg Features/..." iidicates the average number of features per unit type.
Subgraph
Files
Classes
Punctions
File Feutures
Class Features
Funztion Featares
Avg Feat//ile
Avg Feat./Class
Aug Peat./Func
ML Algoritmms
58
171
67
323
256
67
5.57
1.50
1.00
Muih Uiiiites
47
26
162
143
40
103
3,04
1.54
1.01
169
59
Pipeline
149
Core Operations
30
15
124
33
91
4,13
2.20
1.20
MI. Computation
19
39
34
88
$4
34
4.63
1.38
1.00
MI. Pvallation
17
12
51
77
25
52
4.53
208
1.02
ML Dingnostics
200
Visualization
D APPENDIX OF GRAPH-GUIDED REPOSITORY GENERATION
D.1 DETAILS ON LOCALIZATION
To facilitate the localization stage in graph-guided repository generation, we designed graph-guided toolset that allows agents to systematically explore and map design-level fea- tures onto conerete code artifacts. The tools support both fine-grained inspection of files and interfaces, well as feature-driven exploration across the repository. Specifically. ure.mmmmaaaaaaaaeeeeeeeeeeeeee _mp and get.interface.com code structures and retrieval of their implementations, while expand www. www.lllfflodoe laaafnoddeiii -inf and wtw.ehhhiiittfff..b....... wtw.fffnnnininaai....oooo ionality allow navigation the RPG and fuzzy semantic search. Finally, the Terminate command ensures that the localization process produces ranked and standardized output. Together, these tools provide a structured workflow that balances automation with fiexibility, ensuring both accuracy and interpretability in the localization process.
enable inspection of
33

---

Published as a conference paper at ICLR 2026
Localization Tools
#+# Interiace Inspection Tcols
htwp__ffl__ittraeeeeeeo ceatterfaee_faturee_eeee feature_map (file_patth
Inspects aingle Python file to list the interface structures (functions, classes, methodst it contains, along with the feature mappings they aupport. +Usege+: Useful for quickly understanding which interfaces exist in given file and the festure tags assoriated with then.
Axxenples:
viey_ to_fill interfice attarfoce_fattrr__na att/a/pprttm///caaatiirrr///
(end(varbatin]
veet_interface_coteer (target__eeee
Aatrieves the full implementation code a specific fanction, class, or method, given its ly ifiei (file path entity name).
.Uszgev: Applied when particular interface has been locased and its source code detail.
EExamplet:
(beginiverbatinn!
get_interface_comeee (I' src/ccre/data lesderr py: doad_
gat ger_interface_come conttctttcooctttttoocoo (['
(end(verbat
#4 Peature- Explozation Tools 'expand Led__eeee_____eeeeeeeeeeeeeeee (feetureee
Given feature path from the implemented feature this tool expands and lists all asseciated interfaces (functions or classes) in a atructural summary. analyzing
functional node design mapa to repository interfaces.
Applied
Examplet:
(begin(verbate
info (' wlgoritth/Sspperiieee Learning/hhasssiccciii Algerithms/Naive /Navve Bayes')
\and/varbatinn
fantrch_Interfaee Interface_ by ionctiooaalllli (keywords)
Perforas Euzay semantic search for interfaces based on given keywords and returns the top-5 most zelevant interface implementations.
Uszge+: Useful when the exact file or interfece name is unknewn, but functionality- related keywords available. Examples:
\begin(verbatim)
search_ searoh_interrace, oow.aaooooootttttoooollooooaoo l'opptmize", rinitialize')
vend/verbating
### Terminstion Tool
"Terminate (resuult)
Terminates the localization exploration and returas the final zanked 1'st of located interfaces.
zesult follow specified ssoN-style fornat, including file path and isterface type (function, class, or method).
AUsggev: Invoked after cempleting exploration to deliver the finel interface localization results.
Examplee:
\begin
Derminate
["file_patà"; ttop__file_fullpatie "incerface": "method: classl. functioni"),
["Eile_path:; =oop__til__om
"interface": "function: function2");
["file_patà"; ttop__flle_iullatta "interface": "class: Class3"),
Vand(varbatimi
D.2 TOOLS FOR CODING
To support systematic repository modification, we introduce suite of editing tools that operate at multiple levels of code granularity.
Editing Coding Tools
### edit_wholl_cllaa_tie (ftt
elass_namet
34

---

Published as a conference paper at ICLR 2026
Use when: eatire class (including a11 methods and its decstring) requires editing or replacement.
Oulput muat: Provide the full class definition, with all methods and docstring,
#+# cait_method_ef_cle inw.eeeannaaannnnneeaeiaee file (file___fft inaide a claas _lass_name, method_ edited or replaced.
Use a single
Output muat:
Return the full 'class ClassName: block conteining only the target method.
Exclude unrelated
Do not output the method slone; it must appear within its class bleck.
### fust__fuctiinn. incooonnnn_______ (flll
function namee
whens A top-level function requi rea aciting or repl aceeentet.
Output muat: Provide the full Tunetion, including signature, logic, and docstring.
edit_imports andisssandddde annignatttett ([ileelleeleeeeeeeee
Use when: Inport statemeats or top-level assigamenta in the file need to be added or corsected.
Output must:
Contain only import statementa and tog-level assignments (no functions or classes), Follow strict impert order: (1) standard Library, (2) third party packages, (31
locsl modules.
Inportant:
Do not renove existing imports unless they are demonstrably incorrect (e.g., invalid path, type, ron-existert moduls).
Retain imports even if they appear unused, to preserve runcine dependenctes.
#t# Terminate()
invocations are recessary. important: Do not call premacurely; ensure the issue is fully resolved before tormination.
required are complete
D.3 TRAPH-GUIDER LOCALIZATION AND EDITING LOGS
To illustrate how the proposed system operates in practice, we include representative logs that demon- strate the end-to-end process: (i) locating relevant interfaces in the repository through RPG-guider exploration, and (ii) applying code editing operations to implement the required functionality. These logs capture the step-by-step decision process, tool invocations, and final termination results.
Task Example Trajectory
Task:
This 1saue 1s to implement the AdvancedDi fferent waw.reennttaatarattattotaooo class in sro/ ammbollcc/rrhannee//salluu wtwpulusssummeiooo/eeoooo wttm//dffeeeetaalla sll.ayyp............. Contributore should encapsulate the nezessary transformation logie and ensure the methods integrate wall the rest the symbolin slgabra franework,
arver handl ing input validation. expected in order gracefully manage undefined behaviors.
Ageni: <thinks
will begin exploring file *sre/ cifferential, .py" since the http://mmmblleeeeeeeeeeeeeeeeeee emented here. inspect its structure and idea of what is already provided, and any hints or TODOs might be presant.
summatiion class should be use view saterffae_eeatrreeeeeer festure_ map tool on that
</thinks <solution> ...
view view_fill__ie fill__inaarfcaaaaaae http://wwwwa........o.o////////. sfeera/...............oooeeeee.. py")
</salutian>
Eav: "'python
fron typing import Any, Dict
#4# Features: Calculua Advanced Bummet.on/D.ifereettiall Asymptotie Methods/Rdvanced Transformation rtthods/ssymptten
series expension, Calculus Advanced Summation/ bifferential ot aethods/Advanced Trans ton Methods/dii fethod//iifeeeet..e late special furctions, Calculus Advanced Lurnetien/Diffeeenn Advanced "rensformationn iiontaaioon wethods/simplas inequality expressions class fferentiann arorofforrr::
Asymptolic Methods/
35

---

Published as a conference paper at ICLR 2026
def reymppooli__.errieeeeepaaaeeeeee teell,,, expression: Any, expension_order: int). Aay;
def sifferentiate_specii. function (self, expressien: Aay, function_names stz, additional_ Dict (stz, Anyl =oye)) -> Any:
def simplity ineuualiyy inequality_eRpr: Any, tolerance: float=le: 05) -> Any:
Agent:
<think>
I'll check the ti "are/symbol " / oo noe it is part of the Enhanced Symbolic Algorithna simplification area, and it might contaln fuactions or metheds relevent to simplifying inequelities. s/think>
<solution>
view_fll
</solutions
# "Ppython
typing import Any
fron sre.expr_t "rrrrrraarrrrm.. smmp.. oatiofoooooo...ooo wattor.ttcctitioo techniqqus import
apply_comnstalinn aetee_sssciaaiiee_eeee...o
order_tezma
from src.synbolic from are.symbolic cemrbbllic.coo core oxpresss...mmiiiiat.io...o...
expressier.mnippaaaiiiocc
import collect_like_teet
import
canonicalize. wew.eaaaleeeeppesiiiooooooo
**# reatures: hatpliifccaaicnllgeeeeee..
Simplification/aant techniques/simpll.
expressions def simplity_exprest wwtplyy_eexeseeeoooooo (expr: Any]
Anys
...
Agent: <think>
will now search for interfaces related "differentiate special function" or advanced symbol fffrenntiaatto since our rethod di aate_spccial special_fuuetioon on miight rely those routines. This might zeveal supporting Lanctiona logic that be reused.
</think> <aol oon
...
search_ search_interface interfaceeeecc by funett tfaarnntttt special function', aadvanced symbolic cifferentiationn]]
</solution>
Rov: Keyword "di fferent fate special. function" matched interfaces: claas
BulcoattiDiifferenietttoo Mo warlaleeee istorr
advencedDyopeessionnnllyy
Keyword "advanced symbolic diCferentiation! matched interfaces: class
aucomatisDifferemiiitte.. class iatgeraddttionnpeesitiiss class
maltsvarrableotteeerriicoo
#4 File: hat.llaassdddttterrr.eiiiaaaaaaa
from typing import Any, List, Callable
**# Festures: Differenttiatiinn/eeeesessslllle
partisl derivatives, s...frrmtt.atiooo/sssctllleeeeee rechni ques ccom/co partial derivatives, Differentiation t lotp gradient computatior, awt.feeettaaion//pppat zed/ Techniques//accoiaal matrix, Differentia ist htwpaa////ppeeaaeeeeeeeeeeeeeeee htt.:///edd//einnoooos////llllll symbolic hesslan
computation, sifferentiation/Spee wtt.ii///peeeeieeeee....... clasa Yullivariiableelleeeeet.... htt.://eewaa...eeeee.eeaeaaaataa
differentistion.
def validate_partian wwwpptaalddeiiittiieeeeeeeeeeeee (sel,, function: Callable(...
veriables: List
[str], computed: Anyl >> bool:
def compute_partial derreeeeaattilldderiivivieeeeee function: Any, variable: str) Any:
def compute_aymbo www.yyybollooeee e_graddentt (salf, function. one Any, variables: [st(st >> Any:
36

---

Published as a conference paper at ICLR 2026
def computee_aaaobikaaaaatiittttlll Functiona: biet [Anyl, variablee: [ist[ste]) Any
def compuuuesymbbiicc..esssiiaeeeeee function: Any, variables: [ist(s -> Any:
def sifferentiste itw.semttittemmmutiiiiiiiiia (sels, function: Any, variebles: List [atr]] Any:
Agent: <think>
Mext, 1'll search for interfacea that mention "saymptotic series expanaton" because that funetionality is zequired in DereeeDDieeennillleeann bttpooicc_eeiee_exppeeee
This might zeveal candidate functions that can be to generate series expansions.
inserfaces related "simliiyy inequality" to existing oogi for irequality simplification that we can levarage.
any
Let's perform both searches. </thinks
search_incerfaes byceerffco_by_funeee fu asymptotic series expension" bearch_interface bouneeee___eettttteeeeeeeee Tunntiooaall oy (['ainp) ify inequality")
</solution>
Keyword "asymptotic expansion" interiaces: adhencedapressiinnnnlyyyy class waltaweriaallaltffeeett.t. class Numericall-tograate
# Pile:
from typing import Any, List, Callable
#*# Features: oifforentiaation/SSeee /ttpeeiiiiiee//e.h.iiaaaaaaaaaaa
partial derivatives,
fffrrentt' at
ques/corpute partial derivat ives, ferferent
gradient computatios, silferentiation/ losioo///pealleeed Setanigues//arcbiiee matrix, Differentist "efferentirriin/peecialll computation, htt.::/ceio.eeees.eeeeeeeeeeeeee class Mal tllariiablll fferentiator:
ssSfeeetssstoo/pppiilaaeeeeeieie htw.s//ggeeeeeiigge/eeeeeeeeeeee /sybbolic hessian
differentistion
<thinks
We have gathered several candidate supporting interiaces needed for our implementation ierennddiffeeennlllleeeeeee
cur analysis indicates that the following components are very likely to be relevant cependencies:
These ace our top interface level code units that are moat likely to serve as essentiel dependeacies for implementing the class advace@Difereniiilleennnooo with its thres key methods.
</thinks ksolution>
Tecainate
("Eile_peth";
"interface"; "functien:
["rill_path*; "sro/aymbol httmmm//ll////aanne///eeoo iplificat ( "interface";
function: simplify simpllly_espresseirr".
("Eile_peth";
"claas: http://p.aassoom//eeeetaaaattttt
oat.r//iccannmyllle.eeeeessoas "interface":
("Eile_peth":
"interface": "class:
Autoattausifferetiitss"""
("Etle_path"; "sre/arith_algeee www.aagaagaaaaatoonnnnnno inering//nniieerr.....m aybro_aymolloaap"" "interface":
"claas: SymboliideeuattonSoovvee..
1)
Coding
Agen:: <thinks
We need implement the AdvancedDi iirreniiineitreeeiiililoeeeee
class in the fill arc/aymbolic/ caaacaddcalccllse satulltassmmattttoooooo summation/S fferent fal..p.. The class should provide three methods saymptotic_sedt wtw__eeeeeeeepeeiieeee expansion, we will
expanslor_order integer and that the expression is provided. Our implemestation will simulate an asymptoric expansion by returning new symbolic expression
positive
typa (e.g
37

---

Published as a conference paper at ICLR 2026
"asymptol including the original expression and the specified order. Simi arly, aifferentiate fate__petaa f_ should validate that the provided function name is amozg a set of allowed special Functions [for example, ezf, gamma, beta, bessel, hypergecnetriet
simulate specialized differentiation by returning new symbolic expression that indicates that the transformation has been app Finally, stmpli tygiutylllla dictionaryl and that tolerance is positive float reasonably bounded (ue can enforce tolerance C and maybe below to avoid very high tolerance) and then simplitication by deep-copying
check that
input atructured (1.0
irput expression with additional. metadata fields. We also include proper ercor handling and input validatien. </think>
<solution>
adit_whole, sacoleells___io fn_ ("aaeearo//aaaea symboo le/enhanced/ca aaa sus.ssuumaaiinoo /ttp:uuttto//////ffeennnnnnnnnn
"'python
Lyping import Any, Dict import copy
class AdvancedDi ff ffrentt ial Transformar:
Provides advanced differential transformation methods for asynototic series expansion,
diffecentiating special functions, simplifying inequality expressions within symbolic mathematics.
This clasa encapsulates transformstion techniques that are applied to symbol te expreseions
the context of differentisl and asymptotic methods, enabling enhanced symbolic anslyses and computations.
Nethods:
asymptotie_sern ser____see__sssaiiaeee__e_ea expanssaionn texprassion: Any, expansion_orate oo_order: int) Any: Expand the provided expresaion into its saymptetic series representation to the given order.
di seterrentitt__specteee satte_sppssll_tttttoooooo function (expression: Any, Sunction_nane iamem str, additioral_parae paraama Dict [str, Anyl None] Anys Differentiate symbolic expression that containg special function using advanced rules.
immpliyy_inequalle
segquallty_eppppe Any, tolerance: flest 1e-5) Any: Simplify symbolic inequality expression using transformation techniques raduce comp exity.
</solutions
Code Fatch Diff:
+++ arc/symbol wttooooo///////eeeoeeeooooo le/ennancedd/e leulllllllaeee httpmm/ttto//ffffennntt.oooo
8B -1,4 11,5
from typing import Any, Dict +import copy
claas **B
88 - 6 +19,7 88
Sanolify_jaequall (inequelity expp:: Any, tolerance: flost le 5) Anys
Simplify symbolic inequality expression using transformation techniques to reduce complaxity.
('erf', 'gamna', "beta', "bessel', "bypergeonetzie")
def
expression: expanaion_order int) Any
n**
88 -26, 15 +28,23 23 88
Args:
expressicn (Anyl: The symbolic expression to be expanded.
expansion_order (int): order up which expansion should computed.
Returns: Anys symbolic expression representing asymptotic series expansion.
sdge Cases:
38

---

Published as a conference paper at ICLR 2026
expansion_ordee is non posilive, the behavior undefLned. The "unction expeats val symbolic expression input.
Raisea:
Valueberer: expansion_ordes
positive integer.
typerrsor: If the expression is not in valid symbolin format (e.g.r
ctettonary).
pess
(erpaaiioornaaaiiinoooorree expansion_orde: D:
Agent: <solutions
Terminate
s/solution>
D.4 TESTING FRAMEWORK
As illustrated in Algorithm the testing framework combines unit tests, regression tests, and integration tests into unified process. Unit tests validate each new patch at fine-graine level, regression tests re-execute existing nodes ensure backward compatibility, and integration tests verify that multiple patched components interact correctly when combined.
Algorithm 3 Patch- Oriented Testing with Unit, Regression, and Integration Stages
Require: Patch set P: repo skeleton R: dependency code D; existing unit nodes Nu: existing
integration nodes Nú; task description θ
1: function TESTPATCHES(P. R. D,
2: Tunit f- [[; Tinte
3: Tiraj < funit: : f}. inte::::: : 0}}
4: P' P FINDDEPPATCHES(()
Extend patch with dependency patches
5:
for patch E do
6:
Rold FINDEXSSINGUUXTNODUUU...
7:
if Rold # 1 and
p) then > Regression test: reuse existing node if signature/logic unchanged
8:
7nww 5 -olld
9:
else
Onew traj t- CREATROORUUAAEEUUNNNOODEE D, O nold)
11:
Taragl [unii/[ppkee/]]] +- traj
12:
end
13:
R. R.INSERFFILL(mvwu.ttttt..ff
14:
res + The.EEXECCUTTEEST]
15:
APPENDPPE
16:
end for
17:
for patch group G clustered by integration-node do
18:
Nold +- FINDEXSTINGGNNEGGAATTONNOOO
G)
if nold # 0 and ALLEQUAL(mata 9) then
20:
nnew f nold
> Regression integration test: reuse existing node
else
22:
Onews traj f- CREETEINTEEGRATTONNOODEG
23:
Tarag praglinte) & traj
24:
R.INSRRTFLLE file, test.cost end
25:
R.ee:. EXECUTEXEE)))))))( 27: APPENN-PP)N
26:
28:
end for
29: return Tumit U Tintey Trraj
30:
end function
As illustrated in Algorithm 4 the testing pipeline proceeds in sequence stages: branch planning, test generation, execution, judgment, and repair. First, candidate test branch is cre- ated for the given code unit(s). Then, test code is generated and wrapped into a TestNode or Integrationnest which is executed inside controlled Docker environment. The execu-
39

---

Published as a conference paper at ICLR 2026
tion results are judged by LLM; if failures are detected, the framework automatically generates fix queries and iteratively repairs the test until a validated version is obtained.
Algorithm 4 End-to-End Test Generation, Execution, and Repair
Require: Repo skeleton R; tested unit(s) U; source code C; optional prior node nold' maximum
retries Tmax
function BUNTESSIINPPPEELNECR.. U, C,nold)
Main entry point for testing workflow
2: -|- Step 1: Plan test branches
3: branch t GENER Roaid Tmax)
4: I/- Step 2: Generate candidate test code
5:
test_code. t- TEENERATETEST
C, U, Mota)
6:
I- Step 3: Build a TestNode
7:
if represents integration of multiple units then INTEGRATIIONTESTNODEEE test.code,
8:
9:
else
10:
< UNITTESTNODE(U, C, test_coode
11:
end if
12:
Step 4: Execute test code in Docker
13:
result + P.EXXEUUTETEST)
output wissultsssoout || result.stderr
15:
-| Step 5: LLM judge outcome
result contains errors then
17:
[err_type, reviews) +- LLMJUDGE(C, test_code, output, branch) err_ E (test_codee envi envi ronment] then
18:
19:
query <- GENEEATEEFOXQUERY(( lest_code, oulput, branch, reviews) n f FIXTESTANDENN((uuryy U, C, output, n)
20:
21:
end if end
22:
23:
1/- Step 7: Return final validated test node
24: return n
end function
D.5 STATISTICS OF THREE STAGE
Table 10 demonstrates that graph-guided localization provides reasonable efficiency across reposito- ries, with Incremental Development generally easier to localize than Integration Testing Debugging. In terms of models, o3-miní achieves higher localization efficiency but with larger variance, whereas qwen3 -coder shows more stable yet overall lower efficiency. These results suggest that while graph guidance is effective, model capacity and stability jointly influence localization performance.
Table 10: Localization results across six open- repositories under three task categories: Inte- gration Testing, Incremental Development, and Debugging Each entry reports the mean performance with standard deviation std) of the corresponding model-task pair.
Model
Task
TableKit
MLKit-Py
Htplasy
PyWeblingine
StatModeler
SymbolicMath
Integration Testing
13.33=#2.22
8.7514.32
10.94:3.44
6.65=1.98
9.24::3.65
7.88413.30
63-mini
Incremental Development
12.30=:5.19
9.83:44.00
11.00:5.09
12.51=6.67
12.62=6.15
9,93--5.1
Debugging
11.59::5.74
8.2414.40
9.1525.55
10.28:8.50
13.02=7.01
8.90:6621
Integration Testing
6.16.12..37
6.62.12.13 12
7.89 1.242
5.93 2.06
9.24
7.88.13.30
qwen3-coder
Incremental Development Debugging
6.98-1.92
6.49441.79
6.75=22211
6.01#2.11
6.25=1182
6.62=2.47
5.94+2.19
6.42+11.94
As shown in Table I1 03-mini achieves relatively high code success rates across repositories, often exceeding 75% and in some cases approaching 90%, whereas qwen3-coder lags behind with rates around 50-55%. In contrast, the corresponding test coverage remains moderate, typically within the 60-70% range. Figure [ furtther illustrates that coverage fluctuates and tends to decline as code length increases: shorter implementations reach high class-level coverage, but both funetion-level
40

---

Published as a conference paper at ICLR 2026
and overall coverage drop significantly with greater complexity. These results suggest that while current models are increasingly effective at generating functional code, their ability to produce comprehensive and high-quality test cases remains limited, highlighting test generation as a key bottleneck for practical deployment.
Table 11: Average success rate and test coverage (%) for six repositories across two models.
Model
TableKit
MLKii-Py
HupEasy
PyWebEngine
StatModkler
SymbolicMali
Coverage
Coverage
Coverage
Coverage
Coverage
Coverage
03-mini
81.8%
65.0% 48.0%
82.8% $2.0%
61.0% 46.0%
88.9% 50.0%
64.0% 45.0%%
74.7% $3.0%8
60.0%
71.0% 54.0%
62.0% 48.0%
84.8% 51.0%
59.05
qwen3-coder
55.0%
47.0%
46.0%
All
88
Class
Function
Success Coverage vs Code Length
Type Class
Function
Average Coverage
0.6
0.4
0.2
100
120
140
160
Code Length (Lines, bin=20)
Figure 14: Test coverage of 03-mini on MLKit-Py during generation. The figure shows how the coverage of generated test functions varies as code length increases.
E DETAILS ABOUT REPOCRAFT BENCHMARK
In this section, we describe the construction of the REPOCRAFT benchmark, covering four key aspects: the choice of repositories, the preparation of test data, the evaluation methodology, and the configuration of agent systems.
E. REPOSITORIES SELECTION
For the benchmark, curated six representative open-source repositories: scikit-learn, pandas, Django, statsmodels, SymPy, and requests. These projects span diverse functional domains including machine learning, data analysis, web frameworks, statistical modeling, symbolic computation, and HTTP communication, thereby ensuring broad coverage of typical software development tasks. To prevent models from simply memorizing or retrieving solutions from training data, we deliberately anonymized the repositories by modifying their names and descriptions. Furthermore, the task instructions prohibit directly reusing the original implementations, requiring models to generate solutions guided only by feature specifications. This setup enforces a fairer evaluation, focusing on the models' capacity for feature- -gruuddd reasoning and code generation rather than exploitation of prior exposure.
E.2 EVALUATION TASKS COLLECTION
To construct a diverse and reliable evaluation set, we developed an automated pipeline that extends and systematizes the collection of test functions from the official repositories. Our design leverages
41

---

Published as a conference paper at ICLR 2026
Table 12:
Overview of the six benchmark repositories in REPOCRAFT. Each repository is anonymized by renaming to prevent direct memorization or retrieval by models. We list both the anonymized names and their original counterparts, together with category, purpose, and scope.
Original
Anonymized
Category
Purpose
Scope
Name
Name
scikit-
MLKit-Py
Machine
Provides efficient tools for data mining and analy- sis, supporting classical su- pervised and unsupervised learning algorithms.
Focuses model training, evaluation, and selection for standard ML tasks; ex- cludes deep learning and dis- tributed systems.
learn
Learning
Framework
sympy
SymbolicMath
Symbolic Computa- tion Library
Enables symbolic mathemat- ics including algebraic ma- nipulation, calculus, simpli- fication, and equation solv- ing.
Pure Python implementation of symbolic and algebraic computation, lightweight and extensible, without external dependencies.
pandas
TableKit
Data Analy- sis Library
Provides flexible data struc- tures (e.g.. DataFrame, Se- ries) for manipulating and analyzing tabular data.
Supports efficient single- machine structured data processing; excludes dis- tributed data frameworks.
django
PyWchEngine
Web Frame- work
High-level framework for rapid development with ORM, routing, templating, and admin support.
Offers an all-in-one toolkit for building web applica- tions on small-scale/single- server deployments.
requests
HttpEasy
HTTP
Simple, human- human-friendly brary for sending HTTP requests and handling re- sponses.
requests, re- sponses, cookies, sessions, headers; excludes advanced networking and asyne fea- tures.
Covers
Client brary
Li-
statsmodels
StatModeler
Statistical Modeling
Provides econometric and statistical modeling tools, in- cluding regression, time se- ries, and hypothesis testing.
Focuses classical statisti- analysis and diagnostics; excludes modern machine learning and deep learning.
Library
the fact that mature open-source projects typically include comprehensive test suites with robust inputs and ground-truth outputs, ranging from unit-level checks to integration-level workflows. These tests provide a principled source of evaluation data, ensuring that generated repositories are assessed on both algorithmic diversity and functional correctness.
Test Function Harvesting. For each repository, we first gathered all available test functions and classes. These serve as the raw pool evaluation candidates, capturing the behaviors developers themselves deemed important to verify.
Hierarchical Categorization. Next, we organized the collected tests into a hierarchical taxonomy. At the top level, categories follow the natural modular structure used by human developers (e.g.. metrics, theerrmodell decomposi tion.. Within each category, we grouped related test classes and functions by algorithmic target. For example:
"matrics":
Leat_regresssonn;
Functions";
" "reg_targetas
"tet___decc__ee__ttrtttt"""
"test ceaet__hhec__raeg_cee #tcer__chek_nnggrrggess..caeepe
"regreastoo_metriced
"ees__esgresionnmetiies
"pinbell looss:
"test_mess mean pinball
42

---

Published as a conference paper at ICLR 2026
"test_dummy "test_dummy.cuaniiil quantiaanneeeee... pttpleepppaaareeeeeeeee cuning", "test__pinbll
relation ceno_rreltoottitthllee _ithttt__aa""
This taxonomy mirrors repository semantics: higher levels correspond to broad functional modules, while deeper levels capture fine- graigraind algorithmic tests.
Sampling and Filtering. To ensure balanced coverage, applied the sampling algorithm( (Alg T to draw representative subsets of test categories. Each sampled test was then refined into a task description that models could follow during generation. Finally, we filtered out cases irrelevant to core algorithmic behavior (e.g., string formatting checks, version consistency tests), retaining only tests that probe substantive computational functionality.
Example Task Instance. To illustrate the outcome of the pipeline, consider the following task specification extracted from the Django repository:
"category": "gis_migrations",
"filo*; "tests/gia_
"module": *olasa Operat TonTesta",
"cap": "epatial_index",
"fncttions:
"cest_alter_fiied__ee spaatddadd_ppaalll____l pet.epppaaall_ddeeee
sectt_alter_fiel__eee rew_eeemeeeeeee selddd_eemee___ssiiie spapev_____aal__deeee"
Sceat_alter_fielll http://lllalll...t....ttttlllll
"test_alter_tiell" siatter_fieldduitthspaallii lr
"task_query":
testing algorithm that applies migration operations models, ensuring that spstial indexes on spatial fields are proparly creatad, enabled disabled,
dictated by migration opecifications
"id": "djenga-0109"
Each task is represented by (i) its repository category and file location, (ii) the associated test class and functions, and (iii) natural-language query summarizing the algorithm under test.
Given such a task, the benchmark provides the algorithm description, its corresponding in- put-output ground truth, and the test method. Evaluation is then conducted along two dimensions: (1) Algorithm Presence whether the generated repository contains an implementation that matches the target algorithm, and (2) Algorithm Correctness whether the adapted tests pass against the generated implementation, reflecting functional accuracy. This dual perspective allows us to measure both coverage of algorithmic functionality and the reliability of generated implementations.
E.3 AGENT PIPELINE
The evaluation employs a three-stage agent pipeline to connect task descriptions with generated repositories and derive executable judgments of success.
Stage 1: Localization. Given a task and its algorithmic description, the agent first explores the generated repository locate candidate functions classes that may implement the target algorithm. This step uses the exploration tools detailed in Appendix D.I and produces a set of potentially relevant code anchors.
Stage 2: Majority- Vote Validation. To verify whether the localized candidates truly correspond to the target algorithm, we employ unjority-voting mechanism with a large language model (LLM). Each candidate is evaluated five times; the majority outcome is taken the decision. If the validation fails, the pipeline triggers a re-localization attempt. The localization-valiin validation loop is retried up to three times; if all attempts fail, the repository is judged to lack an implementation of the algorithm.
43

---

Published as a conference paper at ICLR 2026
Stage 3: Test Adaptation and Execution. For validated candidates, the agent then adapts the task's 's reference test code. Concretely, the provided ground-trun truth test (including inputs, outputs, and checking methods) is rewritten to match the naming and structural conventions of the localized function or class. The adapted test executed, and its outcome determines whether the generated implementation is functionally correct.
This pipeline ensures that evaluation captures both coverage (whether an algorithm is present in the generated repository) and correctness (whether its implementation passes the adapted tests).
E.3.1 METRICS
To comprehensively evaluate the generated repositories, we adopt multi-dimennional set of metrics that capture four complementary aspects: functionality alignment, novelty, execution accuracy, and code scale. The motivation to move beyond a single success/failure judgment and instead characterize (i) whether the right algorithms are generated, (ii) whether introduced, (iii) whether these implementations actually work, and (iv) at what level of scale and complexity they realized. Together, these metrics provide a holistic view of the strengths and limitations of different models.
functionalities
Functionality Coverage. The first question is whether a model can reproduce the expected range of functionalities in target repository. We extract feature descriptions from both ground-truth reposi- tories and generated repositories, and define a reference set of categories C {ci...... CK) based official documentation and developer guidelines. Generated functionalities are obtained either from structured intermediate outputs (for agent- based methods) or directly from raw code (for baseline models). To align generated features with reference categories, we perform K-Means clustering with C as fixed centroids, plus additional centroid COOD for out-of-distribution features. Each generated feature 9i mapped to (g:) E {coOD}; with assignments further reduce semantic drift. Coverage is then defined the fraction of reference categories that are "hit" by at least one generated feature:
{01.-.- .001....9.00 9N
refined by an LLM-as-Judge
K
Coverage
Bo E EG. f(g:) cz).
This metric quantifies how well the generated repository aligns with the intended functionality footprint.
Functionality Novelty. Coverage alone cannot distinguish between model that simply memorizes existing categories and one that proposes extensions. To capture creativity and diversity, we measure the proportion of generated functionalities that fall outside the reference taxonomy. Specifically, novelty is the fraction of generated nodes assigned to the out-of-distribution centroid cooD:
1
Novelty
1 (S(x)) = =coon]
High novelty indicates tendency introduce capabilities, though such capabilities may or may not be useful. This metric therefore best interpreted jointly with accuracy (below).
Functionality Accuracy. Even if a repository covers the right categories, the implementations must be correct. We therefore evaluate repository-specifie tasks by checking whether generated code passes adapted test cases. Two complementary statistics are reported:
Voting Rate - the fraction of tasks where the -valiitttoo pipeline confirms that an implementation of the target algorithm is present. This measures algorithm presence.
Success Rate- the fraction of tasks where the adapted tests execute successfully. This measures algorithm correctness.
Together, these metrics disentangle whether errors stem from missing functionality versus incorrect implementation.
44

---

Published as a conference paper at ICLR 2026
Code-Level Statistics. Finally, report statistics the scale and complexity of generated codebases. This helps distinguish minimal solutions from more realistic, full-fledged repositories. We compute these metries over filtered Python source files, excluding directories unrelated to core functionality (e.g., tests, exampl es, benchmare rk). The reported quantities are:
File Count: number of valid source files, reflecting modular spread;
Normalized LOC: effective lines of code after removing comments, docstrings, and blank lines, capturing implementation size;
Code Token Count: number of tokens in normalized code, measured with a standard tokenizer, reflecting lexical complexity.
By jointly considering these four dimensions (coverage, novelty, accuracy in terms of presence and correctness, and scalc), we obtain a nuanced evaluation of generated repositories. This design ensures that models are rewarded not only for producing functional code, but also for producing diverse. accurate, and realistically sized repositories.
E,4 GROUND- TRUTH TAXONOMY FOR COVERAGE AND NOVELTY CALCULATION
"Supervised learning":
"Linear Models": 1l,
"Kernel ridge regression": [] "Support Vector Kachines"; "SvC": [], "SVR": [],
"OneClassSVX"; []
}
"Nearest Neighbors"; [),
"Gaussian Processes"; [],
"Cross decomposition". [],
"Naive Bayes": [],
"pecis sion Trees": [],
"knsembles": [],
"Multiclass and multioutput algorithms": (),
"Feature selection"; [], "Semi- "Speissupe learning": [],
"Isotonic regression": [],
"Probability calibration": [1,
"Noural netwerk models (supervised)"; []
"Unsupervised sed learning":
"Gaussian mixture models": [],
"Manifold Learning": [],
"Clustering"; [],
"Bielustering": Il,
"matrix facterization problems": [],
"Covariance estimation": [],
"Novelty and Outlier Detection": [],
"Density Estimation"; [],
"Neural network models (unsupervised)): []
"Model selection and ovaluation":
"Cross-validationn" [],
"Tuning the hyper-parameters of an estimator"; [I,
"Validation curves":
"Classification Metrics": [].
"Regressicn Metrics": [],
"Clustering Metrics": []
"Inppection":
45

---

Published as a conference paper at ICLR 2026
"Permutation feature importance": []
"Dataset transformations": {
"Feature extraction": {
"Feature hashing": [], "Text feature extraction": (],
"Image feature extraction";
"Preprocessing data":
"Scal "Scaling and Normalization
[],
"Di briscretizati and Binari zation": [],
"Pol ynomial and Non-linear reature Engineering": [],
"Categorical Feature Encoding": II,
"Missing Value Imputation" [],
"Kernel and Matrix Centering": []
"Unsupervised dimensionality reduction"; [],
"Random Projection"; [],
"Kernel Approximation": [],
"Pairwise metrics, Affinities and Kernels"; []
}
"Dataset loading utilities"; thr
"Model persistence"; []
"requests":
"Core Request Features"; {
"HTTP Methed Support": [],
"URL and Query Handling"; (),
"Request Bedy Construction": [].
"Custom Headers": []
)
"Response Handling":
"Response Body Access": [I
"JSON Processing"; [],
"Response Metadata"; [],
"Cookic Handling"; []
"Session Management": {
"Session Persistence"; [],
"Session Customization"; [1
"Advanced Configuration":
"Timeouts and Retries":
"Redi Control"; [],
"saming and Chunking": [].
"Authentication Support"; [].
"Event Hocks": []
"Security and Transport"; {
"SSL Verification"; [],
"Client Cortificates"; [],
"Transport Control"; [],
"Proxy Support":
"Compl iance and Encoding": "Encoding Handling"; [],
"Standards Compliance"; 1I,
"Blocking Behavior";
46

---

Published as a conference paper at ICLR 2026
E.5 ANALYSIS OF AUTOMATED EVALUATION
Our main results rely on an automated, model-based evaluation pipeline rather than purely manual annotation. To justify this design, we examine whether these automatic judges produce assessments that are consistent with human raters and stable across different backbone models.
E.5.1 FUNCTIONALITY EVALUATION
Table 13:
Per-repo Coverage (Cov) and Novelty (Nov) mean + std across responses for each model and method, with human annotation as reference (all values in %).
DeepSeek-V3. V3.1
03-mini
GPT-5
Human Cov
Repo
Cov
Nov
Cov
Nov
Cov
Nov
Nov
Method: Claude-Code
TableKit
56.1
1.6 0.5
55.0
1.7
57.9
3.7
0.9
54.5
2.1
StatModeler
30.7 2.0
2.9
1.5 1.3
41.5 +
0.8
2.2
2.6
40.1
2.5
Myth
64.6
0.1
4.7
2.8 0.4
68.8 0.0
1.7
0.4
40.1
2.1
HupEasy
5.2
0.0
5..0
0.0
5.05
0.0
52.5
0.0
MLKit-Py
11.812
2. 0.
1
5.0
1.5
51.2
8.8
PyWebEngine
65.4 3.4
16.3
1.33:33 3.5
7.00
11.0
1.
10.0
64.4
8.4
Method: Repo-333-33
TableKit
73.2 4-22:3..3 3.8
0.5 +
0.4
6.3
9.9
74.2
+ 0.0
1.3 0.4
72.8
0.8
StatModeler
71.2 2.6
8.1
28.8
128
80.8
0.2
15.8 0.4
76.9
29.3
SymbolicMath
61.1 + 4.8 0,0
6.2
0.1
14.6229
0.1
64.5
0.8
HupEasy
1.1
96.3
3.2
97.7 3.2
0.5
100.0
4.4
MLKit-Py
88.7
2.5
17.3
8..0
1..1
94.8
+ 3.4
13.9
0.4
94.2
14.9
Py PyWebEngine
68.1 1.2
13.2 3.2
80.5 1.1
22.7
78.0 2.1
8.4 =
1.8
74.2
11.1
We first ask whether automatic annotators capture functionality-level behavior in way that consis- tent with human judgments. Table 13 reports per- repo coverage (Cov) and novelty (Nov) means and standard deviations for each method under differ- ent automatic annotators, with human judgments reference. Across repositories and methods, the automatic annotators recover the same qualitative patterns humans: ZeroRepo variants attain consistently higher coverage and novelty than Claude- Code, and the magnitude of these gaps broadly similar to that observed under human annotation. Table further summarizes Pearson correlations between model-based and human scores aggregated over all repo-method pairs. Coverage correlations range from 0.71 to 0.88, and novelty correlations from 0.44 to 0.78, indicating moderate strong alignment with human assessments. Taken together, these results suggest that our automated evaluation is reasonably robust, agrees well with human annotators both per-repo and aggregate levels, and remains stable across different backbone models used as judges.
Table 14: Pearson Consistency Between Models
and Humans on Coverage and Novelty Metrics
Model
Coverage Pearson
Novelty Pcarson
DeepSeek- gpt-4o gpt-5
0.780121
0.890096
0.893207
0.958715
0.814894
0.869036
E.5.2 LOCALIZATION AND MAJORITY- VOTE STABILITY
Beyond end-to-end coverage and novelty, we further examine the reliability of the key intermediate stages in our automated pipeline: repository-levat localization and majority-vote (MV) validation. To this end, randomly sample 200 evaluation tasks for each of two systems (ZeroRepo and the Claude- Code baseline) and have human annotators re-label the pipeline's decisions (Table 15). Existence Rate indicates how often the generated repository actually contains correct implementation of
47

---

Published as a conference paper at ICLR 2026
Table 15: Manual validation of localization and majority voting (MV) our automated pipeline on 200 sampled tasks for each model. Existence Rate measures whether the required algorithm is present in the generated repository; Localization Accuracy measures correctness of the identified implementation given existence; MV metrics evaluate the reliability of the majority step.
Model
Exist. Rate
Loc. Ace.
MV Ace.
MV Prec.
MV Rec.
MV F1
Claude-Code 50%
89.00%
85.50%
77.95%
99.00%
87.22%
ZeroRepo 73%
84.00%
79.00%
78.26%
98.63%
87.27%
the target algorithm, Localization Accuracy measures whether the pipeline correctly identifies that implementation when it exists, and the MV metrics (accuracy, precision, recall, F1) quantify the reliability of the majority-vote step. We observe high localization accuracy for both systems (84-89% and consistently strong MV F1 (about 87% for both ZeroRepo and Claude-Code Code), with MV recall above 98%. These results suggest that, conditioned on solution being present, our pipeline can reliably locate the corresponding code and that the MV stage is both stable across models and unlikely to discard correct implementations, providing additional evidence for the robustness of our automated evaluation.
E.5.3 ACCURACY EVALUATION STABILITY
Table 16: Accuracy (Pass/ / Vote Rate) for Claude Code and ZeroRepo(a3-mini on 100 sampled tasks, evaluated by different model and human judges.
Method
DeepSeck-V3. V3.1
GPT-5
63-mini (Ours)
Human
Claude Code
38.7/
3
36,2 3.25/5200 0.71
33.0/
Zemoepp (o3-mini)
61.9/ 79.2
65.1 1
68.95 + 7979 4.90
62.0 73,0
We further assess the stability of our accuracy (Pass / Vote Rate) evaluation pipeline under different judges. We re-run the accuracy evaluation on 100 randomly sampled tasks using DeepSeek- V3.1,
GPT-5, our o3-mini- ni-b-sedd evaluator, and human annotators (Table 16). Across all judges, Ze- roRepo(o3 -mmmni) attains substantially higher pass and vote rates than Claude Code, and the gaps are of similar magnitude for model-based and human evaluations (e.g., humans report 62.0% / 73.0% for ZeroRepo versus 33.0% / 50.0% for Claude Code). This consistent ranking and stable difference across judges indicate that the accuracy evaluation component of our pipeline is robust to the choice of judging model.
E.5.4 TEST CODE QUALITY
Table 17: Manual audit of M-aappeeed test code for Claude Code and ZeroRepo. We randomly sample 100 tests per method from the main experiments and check their correctness and error types.
Model
#Tests
Correct (%)
Import/Name
Sig. Mismatch
Data Mismatch
Harderded
Claude
100
ZeroRepo
100
4
Finally, we examine the quality of the LLM-addpted test code used in our experiments. We manually audit 100 tests for Claude Code and 100 tests for ZeroRepo, checking whether each test correctly encodes the intended behavior and categorizing any errors (Table T7}. We find that 91% of tests for Claude Code and 90% for ZeroRepo are correct. The remaining cases (9 for Claude Code, 10 for ZeroRepo) are dominated by minor, localized issues such as import or name errors, signature mismatches, and occasional data-structure mismatches; that trivially forces a pass. None of these errors indicate systematic flaws in the test-adaptation procedure. This analysis suggests that the vast majority of adapted tests reliable, and that the pass/fail rates reported in the main paper are driven by genuine code quality rather than artifacts of incorrect tests.
observe only single "hardcoded" test
48

---

Published as a conference paper at ICLR 2026
F EXPERIMENT RESULTS
F.1 BASELINE CONFIGURATIONS
To ensure fair comparison, we evaluate three representative systems for repository synthesis: MetaGPT, ChatDev, and Paper2Code, together with several single-agent LLM baselines. All methods are run with their official or default configurations.
MetaGPT. MetaGPT is multi-agent framework that simulates software company by assigning roles such as Product Manager, Architect, Project Manager, Engineer, and Tester. The agents collaborate following predefined Standard Operating Procedures to complete planning, design, implementation, and debugging.
ChatDev. ChatDev also follows a company-style organization, where agents take charge of require- ment analysis, coding, testing, and review. It uses chat-based interaction mechanism coordinate between stages. We run ChatDev with its default settings.
Paper2Code. Paper2Code is a fixed workflow system designed to convert machine learning papers into executable repositories. It follows a three-stage pipeline of planning, analysis, and generation, which we execute sequentially using the default setup.
Vibe-Coding Agent (OpenHands, Codex, Claude Code, Gemini CLI). For comparison with standalone LLM systems, we configure each model with maximum of 30 iterations. The first round is initialized with the repository description. In each subsequent round, the model receives fixed self-reflection prompt:
Please check whether the current repository still has any features that could be enhanced or further improvements,
any misaing functionnlity that needs addad. If there or if you consider the task complete, pleaze reply with "yes" only. LE there are still potensial enhancements improvements to be made, please centinue warking them, do not reply with "yes" just because you concerned about complexity.
F.2 DETAILED EXPERIMENT RESULTS
We report the results different methods on six repositories. For each repository, the methods are evaluated under the settings enable direct comparison.
Table 18: Performance on the MLKKitPy "Nov." denotes the novelty rate; the number parentheses Novel/Total, where Novel is the number of novel functionalities and Total is the total number of planned functionalitics.
Agent
Model
Cov. (%) 1
Nov. (5) (Novel/Total) 1
Pass. Vot. (%)
Files
LOC 1
Tokens
6.3/7.3
3.0
928.0
MetaGPT
Qwen3-Coder
19.2
0.0 (0.0/23.0)
9.9/ 12.0
8.0
170.0
1718
ChatDev
63-mini
Qwen3-Coder
12.8
8.5
14.3 (2/14)
0.0 (0/49)
10.5 11.5
6.3/7.3
6
7
280
163
3100
2064
OpenHands
03-mini
31.9
0.0 (0/39)
11.57
14
272
2499
Qwen3 Coder
(0/48)
11.0/14 14.0
Paper2Cude
03-mini
25.5
0.0 (0/41)
17.8/ 19.9
5
564
6346
Qwen3-Coder
31.9
(0/118)
18.8/24.6
12
1710
20821
Codex CLI
o3 pro gemini 2.5 pro
0.0 (0/59)
11.0/ 16.9
14
829
8344
Gemini CLI
59.6
0.0 (0/141)
0.0/33.5
19
2316
24782
Claude Code
claude sonnel
(0/163)
Gold Projects
Human Developers
85,1/98.3
185
65972
592187
63-mini Qwen3-Coder
97.9
4.7 (58)
73.5/
266
31596
351554
ZeroRepo
15.0 (176/1233)
63.6/74..
642
60553
741634
49

---

Published as a conference paper at ICLR 2026
Table 19: Performance on the HttpEasy repo "Nov." denotes the novelty rate; the number in parentheses is Novel/Total, where Novel is the number of novel functionalities and Total is the total number planned functionalities.
Agent
Model
(%) 7
(56 (Novel/Total) ?
Pass. Vot. (%)
Files
LOC 1
Tokens
MetaGPT
03-mini
22.7
0.0 (0/12)
5.0/ 15.0
I
167
1802
Qwen3-Coder
31.8
0.0 (0/17)
20.0/ 25.0
4
175
2023
18.2
15.0
2055
ChaiDev
Qwen3-Coder
40.9
3.5 (1/31)
20.0/ 30.0
2
323
3151
OpenFands
03-mini
22.7
0.0 (0/5)
20.5 28.2
3
72
669
Owen3-Coder
31.8
0.0 (.22)))
20.07 30.0
2
214
1960
Paper2Cude
03-mini
27.3
0.0 (0/18)
0.0/24.2
5
192
1856
Qwen3 Coder
(1/39)
0.0/
Codex CLI
v3 pro
45.5
0.0 (0/19)
14.0/ 28.0
1
197
1879
Gemini CL.I
gemini 2.5 pm
59.1
3.1 (1/33)
40.07 56.0
1
420
5407
Claude Code CLI
claude sonnet
50.0
0.0 (0/2))
36.0/42.4
2
436
4931
Gold Projects
Human Developers
72.37 87.2
17
2793
22297
ZeroRepo
o3-mini
100.0
2,05 (7/433)
64.0/ 72.0
109
6192
61922
Qwen3-Coder
95.5
0.3 (2/854)
54.0/ 64.0
245
15559
165051
Table 20: Performance on the Enginn repo "Nov." denotes the novelty rate; the number in parentheses is Novel/Total, where Novel is the number of novel functionalities and Total is the total number of planned functionalities.
Agent
Model
Cov. (%)
Nov. (58) (Novel//iota)) 1
Pass. / Yot. (%6
Files
LOC 1
Tokens
MetaGPT
63-mini
27.1
0.0 (0/52)
0.0/13.5
421
3733
Qwen3 Codder
18.8
0.0 (0/52)
0.0/9.2
238
1928
ChatDev
63-mini
25.0
0.0 (0/40)
0.0/144
8
372
3185
Qwen3-Coder
12.1
679
5950
03-mini
31.3
(1/55)
0.0/14.
18
304
2628
OpenHands
Qwen3-Coder
25.0
0.0 (0/52)
0.0/19.1
427
3996
Paper2Code
03-mini
27.1
0.0 (Q/46)
0.0/
11
619
6342
Qwen3-Coder
43.8
0.0 (0/103)
0.0/
10
1761
16076
o3 pro gemini 2.5 pro claude sonnet
(0/88)
769
Gemini CLI
45.8
0.3 (1/318)
7.6/48.1
45
2975
27655
Claude
(669/2165)
33.9
34302
Gold Projects
Human Developers
81.6 86.5
681
109457
917622
ZeroRepo
03 mini Qwen3-Cuder
79.2
38.2 (86))168))
74.1 84.4
430
27647
275782
(244/1561)
F.3 EXAMPLES OF COVERAGE CALCULATION AND NOVELTY ASSESSMENT
In this subsection, provide examples of how coverage and novelty are computed from the constructed RPG, illustrating category alignment for coverage and out-of-distribution detection for novelty.
Analysis of Coverage Examples. These examples demonstrate that our coverage metric provides a reasonable allocation of generated functionalities to reference categories. Core areas such as regression, classification, clustering, and preprocessing utilities (e.g., normalization, imputation) are distributed into their respective modules without overlap misplacement. This validates the soundness of our metric design for assessing functional com- pleteness. Moreover, the RPG ensures that functionalities are not only well aligned with reference categories but also diversified across them, highlighting its effectiveness planning substrate for repository-level generation.
consistently captured, while supporting
"SVR":
"NuSVR"
1,
"Gaussian mixture models": [
50

---

Published as a conference paper at ICLR 2026
Table 21: Performance on the TableKit repo. "Nov." denotes the novelty rate; the number in parentheses is Novel/Total, where Novel is the number of novel functionalities and Total is the total number planned functionalities.
Agent
Model
(%)
(%) (Novel//Toal) f
Pass. Vot. (%) 1
Files
LOCT
Tokens
MetaGPT
03-mini
13.2
0.0 (0/21)
0.0/11.5
186
1814
Qwen3-Coder
6.6
0.0 (0/17)
0.0
133
1453
ChaiDev
Qwen3-Coder
19.7
0.0 (0/54)
0.0
918
9168
OpenFands
03-mini
Qwen3-Cuder
11.8
11.8
0.0 (0/3)
0.0 (0/26)
0.0/12.1
18/11..
193
174
1914
1753
Paper2Cude
03-mini
17.1
9.4 (5/53)
0
7
529
5325
Qwen3 Coder
(0/61)
20.4
Codex CLI
a3 pro
11.8
0.0 (O/2)
21.1 30.9
2
552
6299
Gemini CLI
gemini 2.5 pro
44.7
0.0 (0/117))
28.6 48.5
1249
12242
Claude Code CLI
claude 4 sonnet
52.6
0.0 (0/19))
53.1
8509
83834
Gold Projects
Human Developers
90.6 94.0
217
106447
943873
(011
ZeroRepo
Qwen3-Coder
65.8
13.9 (178/1500)
48.0 64.8
347
32387
389886
Table Performance on the StatModeler repo "Nov." denotes the novelty rate; the number in parentheses is Novel/Total, where Novel is the number of novel functionalities and Total is the total number planned functionalities.
Agent
Model
Cov. (%) 7
Nov. (%) (Novel/Tolal) 1
Pass. Vot. (%)
Files
LOC
Tokens
03-mini
5.6/6.1
6
228
2330
MetaGPT
Qwen3 Coder
5.7
0.0 (0/10)
0.0 0.88.8
13
437
5435
ChatDev
63-mini
21.J (8/38)
1.1/9.5
9
726
9644
Qwen3-Coder
11.4
0.0 (0/18)
3.2/7.7
6
320
3797
03-mini
13.6
0.0 (0/32)
7.9/9.0
9
335
3338
OpenHlnds
Qwen3-Coder
(0/27)
9.5/128
Paper2Code
03-mini
12.5
21.6 (8/29)
0.0/ 10.7
9
8133
9793
Qwen3-Coder
30.0 (12/50)
3.2/14.0
8
1179
13519
Codex CLI
o3 pro
20.5
0.0 (0/23
8.279.9
9
709
8473
Gemini CLI
gemini 2.5 pro
23.7
0.0 (0/55)
13.5/
6
736
8063
Claude Code CLI
clande sonnet
34.1
0.0 (0/191)
18.4/27.8
28
4043
46182
Gold Projects
Human Developers
87.2//6.2
271
83325
893824
ZeroRepo
03-mini Qwen3-Coder
77.3
15.6 (143/1021
76.4/81.1
220
24141
294292
77.3
8.2 (83/1113)
66.2/73.9
436
47370
598058
"gmm expectation teximization "dp gaussian mixture"
Scaling and Normalization":
"quantile scaling",
"scale to [0, 11",
"z-score scaling", "IQR scaling"
"Missing Value Imputation": [
"mean imputation",
"matrix completion imputation",
"impute using K-nearest neighbors",
"impute with global median"
"impute mi missing data",
"Knsembles":
"light gradient boosting", "Eisttradien "bagging classification trees", "random forest", "LightGBM",
51

---

Published as a conference paper at ICLR 2026
"CatBoost", "XGBoost"
"Clustering Metrics": "density peak clustering", "gap statistic", "silhouette score calculation", "inertia calculation"
"Naive Bayes":
"multinomial haeve bayes", "bernoulli naive bayes", "gaussian naive bayes"
"Linear Models":
"ridge regression",
"lasso regression", "huber regression", "ransac regression"
"sv"":
"soft margin svx",
"hard margin SVM",
"SVM with precomputed kernel"
"Proxy Support"; "rotate proxy list", "auto detect system proxy",
"custom dns resalver integration"
"HTTP Method Support": "send POST request",
"GET request with cookies",
"send DELETE request",
"PUT with JSON payload"
,,
"URL and Query Handling": "encode path segments", "parse query string", "normalize request url"
"Redirect Control";
"auto follow redi rects", "limit redi chain"
}, "Authentication Support": "send basic auth", "include cauth2 bearer token", "refresh auth token"
"Timcouts and Retries"; [ "set request timeout", "apply exponential backoff", "custom retry hock"
"SON Processing";
"auto deserialize isen", "validate jsen schema",
52

---

Published as a conference paper at ICLR 2026
"serialize dict to JSON"
"SSL verification";
"ssl hostname verification", "load custom certificates"
"Streaming and Chunking"; "process chunked response", "resume file download support"
Analysis Novelty Examples The novelty cases illustrate two key observations. First, novelty captures meaningful extensions rather than random noise: in MLKit-Py, such as Prophet forecasting... STL decomposition, and genetic programming feature synthesis, while in StatModele: new capabilities include vector autoregression and Cox proportional hazards models. Second, the new functionalities proposed by the RPG remain reasonable within the target domain: they extend statistical modeling, optimization, robustness analysis in ways that align with real-world software evolution. Together, these examples confirm that the RPG supports not only stable replication of reference repositories but also the introduction coherent and domain-consistent innovations.
coherent additions
nnw_featuree
"vector autoregression model", "forecasting with Prophet", "genetic progranming feature syathesis", "multi-offective bayesian optimization", "online Tearning", "apriori asscciation mining", "Cox propertional hazards model",
"STL decomposition",
"temporal drift detection", "fuzz testing", "interactive dashboards", "NoSQL queries", "pareto optimization", "deoograpaic parity test", "secure argument parsing",
"new features":
"vector autoregression model", "forecasting with Prophet",
"genetic programming feature syathesis" "multi-objectic" bayesian optimization", "online learning",
"apriori asscciation mining", "Cox proportional hazards model",
"STL decomposition", "temporal drift detection", "fuzz testing",
"interactive dashboards", "NoSQL queries",
53

---

Published as a conference paper at ICLR 2026
"pareto optimization", "demographic parity test", "secure argument parsing",
F.4 EXAMPLES OF LOCALIZATION BEHAVIOR
TableKit
MLKit-Py
view interface, Hettone map
siew Interlace, fhattrn, JIND
Tbtal Call Court
3000
search interface by functionally by get inteinttrattteoo content
Court
2500
ogann, Jest rode, info
search, Interface, Juuccionallt
2000
eownd jeaf rode jnts
Temminate
2000
get Lnnemfannnn content
Taminate
1500
1000
1000
Step
Step
HatpEasy
PyWebEngine
interfoo, foatane functionally
4000 -
www_fiejrtteffac maa
Total Call Count
Total Call Count
search, functinaltyy
Taminate
get content
get content
ouaand isaf,
ogean, sall
400
2000
200
0
20
20
Step Index
Step Index
SymbolicMath
StatModeler
Total Call Count
Total Call Count
2000
get_imerface
get,
1000
1500
1000
2
20
11
13
2
20
11
Step Index
Step Index
Figure 15: Aggregated function call frequency distribution across localization steps all repositories using o3-mini.
Graph guidance structures localization into systematic search. Figure 15 shows that with graph guidance. localization behavior follows a structured CCG pattern (Coarse Search Content Inspection + Global Graph Exploration). The agent begins by traversing the RPG at a coarse level to identify high-level candidates, then inspects content-rich nodes for detailed signals, and finally explores semantically related structures across the graph. Termination calls rise as the search converges. This progression indicates that the RPG reshapes the agent's behavior into a systematic, relation-aware search process, replacing ad hoc repetitive probing.
54


Reviews:
[{"review_id": "cDLcagdPaS", "reviewer": "Reviewer_1t6n", "summary": "This paper tackles the challenging problem of scaling large language models (LLMs) from generating single files to complete, multi-file software repositories. The authors argue that current approaches, which often rely on natural language for high-level planning, are brittle and fail to scale due to ambiguity and a lack of structure.\nTo address this, the paper introduces three main contributions:\n- Repository Planning Graph (RPG): A novel graph-based representation that serves as a structured blueprint for repository generation.\n- ZeroRepo Framework: A graph-driven agent framework that builds and utilizes the RPG.\n- RepoCraft Benchmark: A new, challenging benchmark.", "questions": "What is the practical cost of running ZeroRepo? Could the authors provide an estimate of the total tokens, API calls, or wall-clock time required to generate one repository compared to the Claude Code baseline over its 30 iterations?", "limitations": "", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- Moving from function-level to repository-level generation is arguably the most critical open problem in the area. This paper tackles it directly.\n\n- The main results table (Table 2) shows a massive performance gap between the RPG-based ZeroRepo and all baselines (multi-agent, workflow, and terminal-based) on all key metrics: coverage, accuracy, and scale.\n\n\n- The paper shows a clear benefit for the graph-guided localization, with the RPG speeding up localization tasks by 30-50% compared to a \"w/o Graph\" baseline.", "weaknesses": "- Dependency on External Knowledge Base: The \"Proposal-Level Construction\" stage seems heavily dependent on the \"EpiCoder Feature Tree,\" a pre-existing 1.5M-node ontology of software capabilities. It is unclear how much of the system's strong performance is due to the novel RPG framework versus this massive, highly-structured knowledge base.\n\n- Evaluation of \"Novelty\": The paper introduces \"Novelty\" as a metric and provides qualitative examples of new features (e.g., \"Prophet forecasting\" ). However, the paper does not seem to evaluate the functional correctness of these novel features. The main accuracy metrics (Pass/Voting Rate) are based on the 1,052 tasks from the reference projects. It's unclear if the \"novel\" features are correct, functional implementations or just plausible-sounding, well-structured stubs.\n\n- Potential for Benchmark Leakage: The RepoCraft benchmark is based on extremely well-known projects (pandas, scikit-learn, etc.).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "A3487tVug5", "reviewer": "Reviewer_rT1s", "summary": "The paper introduces the Repository Planning Graph, a structured graph representation that integrates proposal and implementation planning for repository-level code generation. Specifically, proposal-level planning decides what functionalities to include, and implementation-level planning decides how to realize them. RPG encoding functionalities, data flows, file structures, and class/function dependencies into a coherent graph. Based on this graph, the authors develop ZeroRepo, a framework that constructs and traverses the RPG to generate repositories from natural language specifications. In addition, the authors introduce a benchmark RepoCraft to evaluate the ZeroRepo. RepoCraft covers six real-world software projects with 1052 functional tasks. Experiment demonstrates ZeroRepo outperforms other baselines in coverage, test accuracy, and code scale.", "questions": "1. How is the accuracy for automatic localization and major voting? Providing a manual verification on a subset would strengthen the evaluation.\n\n2. How would the cost compare to RPG+RepoCraft and traditional natural language based code repository generators?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The method integrates functional and structural dependencies, providing graph abstraction as an intermediate for natural language description and code repository implementation. The RPG is both explicit and fully machine interpretable.\n\n2. The paper also provides a graph-guided code repository generation method. The method traverses the RPG in topological order, applying test-driven development to ensure incremental expansion while preserving stability.\n\n3. The paper also provides a valuable testbed covering six projects and 1052 tasks, assessing coverage, accuracy, and code scale.\n\n4. The experiment demonstrates significant improvement compared with SOTA methods. The additional scaling analysis shows near-linear growth in functionality and repository size.", "weaknesses": "1. The paper only includes a single ablation study without a graph, while the effectiveness of other components remains unclear (e.g., exploration strategy). As there are 3 generation level, it is important to conduct ablation study. For example, can we remove stage A?\n\n2. RepoCraft relies on automatic localization and major-voting validation. This may introduce evaluation errors from LLMs.\n\n3. Efficiency analysis of computational overhead from RPG constructions is not discussed.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "SGRdT3lFkb", "reviewer": "Reviewer_PvAb", "summary": "The paper introduces the Repository Planning Graph (RPG), a structured representation that unifies proposal-level capability planning with implementation-level file, class, and function dependencies for repository generation. Building on RPG, the ZeroRepo agent conducts planning, graph-guided code synthesis, and test validation. The authors also release RepoCraft, a benchmark derived from six paraphrased real-world repositories (1,052 tasks). ZeroRepo substantially outperforms multi-agent, workflow, and terminal baselines on RepoCraft, reaching 81.5% coverage, 69.7% pass rate, and dramatically larger codebases, with analyses showing near-linear scaling and efficiency benefits from graph-guided localization.", "questions": "1. Was the EpiCoder ontology filtered or deduplicated to ensure it does not contain paraphrases or derivatives of the six target repositories used in RepoCraft? Did all baseline methods have equal access to this knowledge base?\n2. Beyond the Gold Project validation on human-developed repositories, can the authors provide evidence (e.g., correlation with manual evaluation) that the LLM-based coverage/novelty pipeline produces reliable judgments consistently across different types of generated code?\n3. Which component—the feature tree initialization, data-flow encoding, or graph-guided generation—contributes most to the observed performance gains? An ablation removing RPG structure while maintaining the same planning budget would clarify this.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- RPG provides a clear structured alternative to natural-language planning, coupling capability decomposition with file-level structure and data-flow constraints, with the design well illustrated in Figure 2. This representation offers an explicit, interpretable blueprint that can support consistent long-horizon planning in repository generation.\n- RepoCraft offers a comprehensive benchmark with six substantial real-world repositories, paraphrased specifications, and 1,052 evaluation tasks derived systematically from test files, filling a gap for repository-scale assessment of agent planning capabilities.\n- ZeroRepo achieves substantial gains over strong baselines, demonstrating +27.3% coverage and +35.8% pass rate improvements compared to Claude Code (Table 2) while generating significantly larger repositories (36K LOC, 445K tokens vs. 10.6K LOC, 105K tokens),.", "weaknesses": "- ZeroRepo relies heavily on the 1.5M-node EpiCoder feature ontology as its knowledge base for proposal-level planning. However, it is unclear whether all competing baselines had equal access to comparable structured priors.\n- The coverage and novelty metrics depend critically on K-means clustering with LLM adjudication, and correctness evaluation relies on LLM-adapted test cases. Beyond the Gold Project sanity check (81.0% pass rate on human-developed repositories), there is limited independent evidence that these automated evaluation pipelines produce stable and reproducible judgments, particularly for edge cases where generated functionality diverges significantly from reference implementations.\n- Key architectural decisions—including reliance on the feature tree for initial planning, data-flow encoding at multiple hierarchy levels, test-driven code generation with majority-vote diagnosis—lack comprehensive ablation studies. The paper only provides an ablation for graph-guided localization (Table 4); without component-level comparisons, it is difficult to attribute the reported performance gains specifically to the RPG representation versus confounding factors like the planning budget, LLM backbone choice, or architectural engineering.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "SXKyUlGU9O", "reviewer": "Reviewer_Gi5H", "summary": "This paper presents RPG (Repository Planning Graph), a prompting framework for generating software repositories with LLMs. RPG generates sketches of a repository in graphs, where nodes represent functions and edges represent dependencies and information flow. The actual code is then generated from this sketch graph.\n\nExperiments with two backbone models (o3-mini and Qwen3-Coder) show that RPG outperforms some other frameworks such as MetaGPT and Paper2Code on six repositories.", "questions": "- According to Figure 1 and 2, edges in the RPG indicate hierarchy and information flow. Does this imply that graphs in RPG are always acyclic? If so, what's the advantage of the proposed RPG over graphs automatically constructed with program analysis tools, such as file structure tree, import dependency, and control flow? If not, can the authors provide an example of cyclic graph?\n\n- Line 238: there are only three subplots (A, B, C) in Figure 1, and no D. I suppose this is a typo.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "The problem (generating an entire repository from nothing) is interesting. Experiments show improvement compared with multiple other frameworks.", "weaknesses": "- In essence, RPG is a prompting framework and does not involve training better models, limiting its novelty and contribution. Experiments only involve two API-based models, raising questions about RPG's applicability in many real-world applications that require deploying models locally.\n\n- The problem setting - generating an entire repository from scratch - is unrealistic. The most common application of LLMs in software engineering is resolving an existing issue or implementing new features in an existing repository.\n\n- Line 106-107: I'm confused by the structure here. I can understand that \"multi-agent systems assign roles, while workflows follow stages\". But then the authors state \"Industrial systems automate SWE tasks\". How is this parallel to the previous two? Aren't multi-agent systems and workflows supposed to automate SWE tasks?\n\n- Figure 1 is too crammed and looks messy.\n\n- References are not formatted professionally. For example, Line 522, 527, 587 are all arxiv preprints, yet three different formats are used. The authors also cited many papers' preprint versions when there are peer-reviewed versions. For example, Epicoder (Line 598) is published at ICML, while DeepSeek-R1 (Line 528) is published at Nature.\n\n- Details are lacking in Section 3.2. What models are used for graph node embedding? What models are used for feature path retrieval?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)