DOI: 10.5281/zenodo.17338423
title: Software Self-Extension with SelfEvolve: an Agentic Architecture for Runtime Code Generation

description: Replication Package: Software Self-Extension with SelfEvolve

Submitted to: SEAMS 2026 (International Symposium on Software Engineering for Adaptive and Self-Managing Systems)

Paper Title: Software Self-Extension with SelfEvolve: an Agentic Architecture for Runtime Code Generation

Overview

This replication package contains datasets, empirical results, and supplementary materials for evaluating SelfEvolve—an architecture enabling runtime self-extension through autonomous code generation, codebase integration, and cross-session composition.

Complete implementation and baseline comparison code: https://anonymous.4open.science/r/self-evolving-systems-FEC4

Contents

1. Evaluation Datasets (11 Tasks)

Curated benchmark for evaluating runtime codebase integration capabilities across three task categories:

Integration Tasks (4): Multi-file codebase navigation and integration



Patient Risk Analyzer (Healthcare): 752 LOC, 12 files, 20 classes

Student GPA Calculator (Education): 783 LOC, 10 files, 10 classes

Salary Analyzer (HR Analytics): 616 LOC, 13 files, 12 classes

Inventory Low Stock Alert (E-commerce): 577 LOC, 8 files, 7 classes


Compositional Tasks (3): Cross-session capability building



Matrix Eigenvalue Composition (Linear Algebra)

Portfolio Risk Assessment (Finance)

IoT Sensor Pipeline (IoT Systems)


Data Processing Tasks (4): External dataset manipulation



Book Recommender (100 books, CSV)

Friend Suggester (100 users, JSON)

Movie API (100 movies, CSV)

Performance Tracker (50 reviews, CSV)


2. Empirical Results

TDD Pipeline Ablation Study:



With TDD: 51/55 runs (92.7%), avg 2.2 iterations

Without TDD: 40/55 runs (72.7%), avg 4.7 iterations

Statistical significance: Wilcoxon signed-rank test W=55, p<0.001, r=0.98


Baseline Framework Comparison (165 experimental runs):



SelfEvolve: 51/55 (92.7%)

AutoGen: 17/55 (30.9%)

MetaGPT: 15/55 (27.3%)

AgentCoder: 0/55 (0.0%)

Statistical significance: Wilcoxon + Bonferroni correction (α=0.0167), all p_adj < 0.012


3. Supplementary Materials

Appendix:



Qualitative architectural comparison: Evidence for 11 capabilities with direct quotations from papers

Quantitative empirical evaluation: Complete methodology, failure analysis, statistical tests


Complete Codebase (anonymous.4open.science):



SelfEvolve implementation with all components

Baseline configurations for fair comparison

All 165 evaluation results with detailed logs

Statistical analysis scripts


Technical Specifications

Requirements: Python 3.10+, minimal dependencies

Statistical Methodology:



Problem-level pairing (N=11 paired observations)

Wilcoxon signed-rank test (non-parametric)

Bonferroni correction for multiple comparisons (α=0.05/3=0.0167)


License

Datasets, Code, and Appendix: CC BY 4.0

Reproducibility: All results can be reproduced using provided datasets and codebase. See documentation in repository.

Keywords: Self-evolution, Runtime Code Generation, LLM Agents, Multi-Agent Systems, Test-Driven Development, Codebase Integration, Cross-Session Composition