Automated Stateful Specialization for Adaptive Agent Systems
Authors: Myan Vu
Harrish Ayyanar
PANG JIANG
Anwiketh Reddy
Mayank Goel
Venue: ICLR 2026 Poster (2026)
Keywords: LLMs
Autonomous Agents
Agent Specialization
Primary area: foundation or frontier models, including LLMs
TL;DR: We introduce a framework that creates persistent, specialist agent teams through an offline lifecycle of discovery and cultivation, and deploys them with an online policy that efficiently adapts the team's structure for novel tasks.

Abstract: Current automated agent design frameworks produce either static workflows that lack adaptability or per-query optimizers that prevent the accumulation of deep, agent-level task expertise. We propose a new direction that reconciles these paradigms: creating stateful teams of specialist agents that accumulate knowledge over time and can be reconfigured for novel tasks entirely without human intervention. To this end, we introduce \textsc{ASpec}, a framework that manages this full agent lifecycle by first autonomously \textbf{discovering} specialist archetypes via evolutionary search and then \textbf{cultivating} their expertise through experience, mirroring how human experts learn through practice and reflection. We further introduce a lightweight hierarchical control policy, "retain-then-escalate," which governs when to leverage the established agent system versus when to adapt its structure. Through comprehensive experiments, we demonstrate that this approach leads to significant performance gains on expert-level scientific benchmarks like GPQA while matching the state-of-the-art on broader domain tasks, demonstrating a promising path toward agent systems that are simultaneously expert, adaptive, and efficient. We will release the code at https://github.com/myanvoos/ASpec.

Published as a conference paper at ICLR 2026
AUTOMATED STATEFUL SPECIALIZATION FOR ADAPTIVE AGENT SYSTEMS
Myan Vul- Harrish Ayyanar²
Jiang Pang*
Anwiketh Reddy*
Mayank Goel
2 University of Auckland
University College London
Algoverse "Northcastern University
LLosffunk
ABSTRACT
Current automated agent design frameworks produce either static workflows that lack adaptability or per-query optimizers that prevent the accumulation of deep, agent-level task expertise. We propose new direction that reconciles these paradigms: creating stateful teams of specialist agents that accumulate knowl- edge over time and be reconfigured for novel tasks entirely without human intervention. To this end, we introduce ASPEC, a framework that manages this full agent lifecycle by first autonomously discovering specialist archetypes via evolu- tionary search and then cultivating their expertise through experience, mirroring how human experts learn through practice and reflection. We further introduce a lightweight hierarchical control policy, seaain-then-escane leverage the established agent system versus when to adapt its structure. Through comprehensive experiments, we demonstrate that this approach leads to significant performance gains on expert-level scientific benchmarks like GPQA while match- ing the state-of-the-an on broader domain tasks, demonstrating promising path toward agent systems that are simultancously expert. adaptive, and efficient. We will release the code at hhtps//gghhab.ccmm/yaawoo//////
which governs when
INTRODUCTION
Motivation. The emergence of sophisticated multi-agen systems capable of tackling complex problems (Wu et al. 2024 Li et al. 2023 Hong al., 2024) has marked a significant advance for autonomous agents. While effective, these foundational systems were often manually hand-crafted for specific tasks, which limited their scalability. response, research has shifted towards automating
aspects of these systems, starting with prompt optimization (Khattab et al.) 2024 Yuksekgonul et al. 2025; Yang et al. 2024) or inter-agent communication via graph-based workflow representations (Zhuge et al. 2024 Liu et al., 2024 Zhang et al., 2025a). and then, to the designs of agent systems themselves. The automation of agent designs has since largely split into two distinct paradigms: task-level optimization and query-level adaptation. In the case of (I) Task-Level Architecture Search, prior works optimized for a single, static agent workflow for a specific task domain. These approaches, which mirror early approaches in AutoMI. and Neural Architecture Search (NAS) (Elsken et al. (2019), were pioncered by ADAS (Hu et al.) 2025), which uses Meta Agent Search to iteratively program new agents in executable code; AFlow (Zhang et al. 2025b), which similarly adopts code representation but utilizes Monte Carlo Tree Search (MCTS) to efficiently navigate the search space; and AgentSquare (Shang al., 2025). which employs module evolution and recombination to discover novel configurations in constrained, modular code-based search space. The primary limitation of these methods is their intrinsic "one-size-fits-ai nature: by searching for single best design for an entire task domain, they fundamentally lack the adaptability necessary to dynamically allocate inference resources or customize the structure for individual user queries.
To address the rigidity of task-level systems, a recent paradigm shift has focused generating unique workflow for each incoming query, (II) Query-Level Architecture Adaptation. MaAS (Zhang et al. 2025c) introduces the concept of "agentic supernet", optimizing probabilistic distribution of agent architectures during training and sampling a bespoke architecture from said distribution for each query during inference. This paradigm has been extended by other methods
Cooresponding author; myanvoos@gmail.comm

---

Published as a conference paper at ICLR 2026
like FlowReasoner (Gao et al. 2025). which uses reasoning- crasoning-based meta-agent to generate query. specific agent systems; ScoreFlow (Wang et al. 2025). which introduces Score-DPO, a that fine-tunes its per-query workflow generator using quantitative evaluation scores; MAS-GPT (Ye et al. 2025), which trains an I.L. to treat workflow construction as a generative task; and MAS-Zero (e et al. 2025), which employs meta-agent at inference time to iteratively generate and refine agent configurations based on self-generated feedback. While these approaches offer superior adaptivity, they are challenged by the lack of long-term state. Because the architecture is regenerated or resampled for every query, the system incurs significant "rediscovery" cost, and the individual components or agents are largely prevented from accumulating deep, persistent expertise over time.
The prior work demonstrate critical chasm between monolithic, task-level robustness and adaptive, per-query regeneration. The former is static at inference, while the latter incurs "rediscovery" costs by repeatedly invoking meta-agents for architectural search in lieu of leveraging persistent knowledge, a system-level problem that a modular, agent-level memory addition would fail to address. Our proposed framework, ASPEC, reconciles these limitations by integrating the specialized mechanisms of self-evolving agents into a unified lifecycle within agent design automation. This control
lifecycle establishes stable, persistent agent archetypes deployed by a 'retain-then-escalae policy, allowing the system to default to efficient and effective execution by relying on the persistent knowledge of its specialist agents.
Contributions. In short, our core contributions are as follows:
We propose ASPEC, a framework that manages the full lifecycle of expert specialist agents via automated two-stage methodology: (I) Discovery, where LLM autonomously explores the design space of agent archetypes using evolutionary processes, and (II) Culti- vation, where selected agents autonomously cultivate their expertise on training corpus.
We introduce "retain-then-escalltt" or fully dynamic, defaults to retaining a stateful agent team across related queries to leverage expertise and minimize cost, only escalating to architectural resampling when needed.
control policy that, instead of being either fully static
The mechanisms for autonomous discovery and expertise cultivation as seen in self- evolving agents have been explored individually across various research efforts. For instance, parallel to workflow optimization, a distinct stream of research has explored agent specialization via prompt optimization. starting with role assignment via ExpertPrompting (Xu et al., (025). PromptBreeder (Fernando et al. 2023), and PromptAgent (Wang et al.) 2024a). Multi-agent frameworks like EvoAgent (Yuan et al.) 2025), which utilizes evolutionary algorithms to automatically generate and optimize multiple specialized agents with diverse settings and roles; MASS (Zhou et al.) 2025), ) which optimizes individual role prompts alongside refining inter-agent communication; and AeentVerse (Chen et al.] 2024a) and AutoAgents (Chen et al. 2024b), which dynamically synthesize and coordinate teams of expert roles, validate a critical insight: the identity of the agents is as important as their interaction topology. However, this specialization is often stateless, and the focus remains on generating an optimal team for a single task. In contrast, ASPEC's Discovery process generates persistent specialists whose structures are specifically designed to be retained and cultivated over time rather than generated for transient collaboration or discarded after a single optimization run.
Related Work.
Another stream of research in self-evolving agents is expertise cultivation, focused on endowing agents with non-parametric state (memory and experience) that persists beyond single task interaction. Such mechanisms are embodied by works like Reflexion (Shinn et al. (023), which allows agents to record natural- natural-languaae language critiques of their past actions in episodic memory to guide future behavior and avoid recurring mistakes, and Self-Refin (Madaan et al., 2023), which employs continuous iterative refinement loop where the agent critiques and revises its initial outputs. Furthermore, Expel. ZZao et al. 2024) processes past trajectories to generate insights and rules to guide further interactions, AutoGuide (Fu et al., 2024) automatically generates context-aware guidelines from offline experiences, facilitating the provision of relevant knowledge for active decision-malee processes, while Agent Workflow Memory (Wang al.) 2024b) records common subtask sequences that can be retrieved and reused without re-planning from scratch. These prior works illustrate how experiential knowledge can be accumulated and generalized into long-term competence.
While memory systems and reflection mechanisms exist, ASPEC proposes a systematic, two-stage lifecycle framework where the Cultivation phase is explicitly linked to the output of the Discovery
2

---

Published as a conference paper at ICLR 2026
phase. This linkage ensures that the stateful expertise (memorytreflection)) is accumulated within the designated, persistent specialist archetypes, facilitating the emergence sele-specffie expertise.
2
PRELIMINARIES
Specialist Operators
Meta-Con Meta-Controller
Architect
Query
96-1
CoT
"1t duriiping wivvitte protess
MiniLM
(81), 1
ara repart queiii cepation your
e(o)
c(Gr-s)
"catavk
melccto
i Playicies (Col)
Query di
MLP
reblem heric cacalia th Lownd lecor atsad
Leared specifi appeooc Boon M
Action
dess,
Exeertt Di
Culeguricall P(ees))
Frediction
M
Figure 1: The three main components of ASPEC.
ASPEC can be framed Hierarchical Reinforcement Learning (HRL) methodology consisting of low-level generative process for architectural redesign and agentic operator pool evolution, as well as lightweight, high-level policy that learns when to invoke this process efficiently. We formally define these components below, starting with the modular units they operate upon: agentic operators.
Definition (Agentic Operator). Following MaAS (Zhang et al. 2025c), we define an agentic operator O as tuple (M.P. P, {T:) where M E M denotes the LLM backbone, E denotes the prompt, and [T:} S T denotes the available tools. A multi-agent system is then represented directed acyelic graph G = (V, F} where each vertex e E V represents instance of an agentic operator and each edge E E defines the connection between two operators.
To facilitate the evolutionary process the heart of our methodology, we structure the operator pool O, into two functionally distinct sets. First, the base operators (Ctase), a static set of foundational, stateless operators consisting of extensible single-/multi-ggent systems, for instance Chain-of- Thhoughh (Wei et al.) 2022) LLM-Debate (Du et al. 2024a). Second, the specialist operators (Dspoc) dynamic set of operators derived from base operators.
A specialist 0 E Ospoc extends base operator O Cbase with learned identity and persistent memory while inheriting its foundational reasoning structure (e.g., "think step-by-step"), is a tuple O (O,, MM) where P, is specialized prompt and M is persistent, experience-driven memory module. We decompose P. into an identity, which is a rich descriptor of who the agent is (Xu al. 2025), and set of directives, which are methodological principles for the agent's thought process, allowing for a rich and diverse "genetic" space of reasoning approaches (Naik et al. 2024).
Definition (Architect). The architect is the low-level generative component responsible for evolving the operator pool and redesigning the multi-agent architecture, implemented
in-context learning LLM that operates via multi-turn iterative reasoning process. We provide the prompt in Appendix G. and give an example of its reasoning in Appendix AA.2] Functionally, given query qt. the Architect is process fA that maps rich contextual input to new system configuration
(y,, Hi-mat11 9r-1) (G,, 0.)
where H is sliding window of the past experiences including the executed architectures and performance outcomes; 0,-1 is the previous operator pool; and G--1 the current architecture. Its objective is to find an architecture that maximizes the immediate cost-aware utility while being general enough to be potentially retained for future tasks. We define this value in terms of the utility with respect to the oracle a,, U² UGe gt, at), and the total costs of all API LLM calls, (GG).).
3

---

Published as a conference paper at ICLR 2026
Base Operator Poal (Dun)
Meta-Conrrrller
State and (ct, 9h-1)
CUT
Debeie
Query (g)) implement bio Crank- Menouson mothco on 1- Iim- oepardert Sthtndeggr equatton.
CaT-BCC
RoAd
SeliRefae
Encetble
Current 9p-3
Tolling
Spectalle Operatte Peal (Oge))
RESAMPLE
RETAIN
O2
Architect
Quary (q-- glycages
Srhennthhggeen (ReAks)
encommalioaly glun-.
ClinxialBoard
(Delaac)
Phanes pouehpa
1807
Current 9..
Gleecotllla is unsutiole for plasscs probiom
NE TAIN
my progeammer plysisit
Cimonioard (Detwkk)
Query (ú. pojpable lef parastema inpusse suggeate which abnormaity?
(Sale))
Memory Update
reed sumember
Hansbonar's spaind dechadive
feed tomeeeeto
Environment
RESAMIPLE
when nomaiiingg any aecissul Naxely
Figure 2: The online adaptation loop of ASPEC.
G arg max E E XC.(G.) + V. (8c+1))
9.E9
where VTo ($t+1) is the expected future value given the next state, formally defined in Equation 3 While this generative process enables adaptation, by continuously rebuilding the architecture, the system potentially forgoes the chance for the active specialists to deepen their expertise on the novel task. Additionally, and perhaps even more importantly, the Architect's invocation is computationally expensive and poses a practical challenge scale. To address the trade-off between adaptability, experiential learning, and cost-efficiency we propose the meta-controller, a lightweight gating module that decides when escalate to the Architect during deployment.
Definition (Meta-Controller). The meta-controller is neural policy no =a(a))8)) that makes single high-level decision: retain the current agent architecture, or resample a new one for given query. Its action space is discrete, that meta-controller as a Markov Decision Process (MDP), where the action taken at step t - determines the architecture G--1 available in the subsequent state 8t. The state 8, at timestep is therefore:
{ØRETAIN, GRESAMPLE). We formulate the training of the
8t = (eg(4)):, (9(1))))
where (-) and e, (.) are fixed-length query and textual graph embeddings, embedded with MiniLM (Wang et al. 2020). While previous work (Zhang et al. 2025a) has used Graph Neural Networks (GNNs) to encode architectural topology, we opt for a simpler, query-aware semantic representation. Our "bag-of-open ofprrtoorr approach represents architecture
iitention-weigheee average of the embeddings of its constituent operators. The attention weights are computed based on the similarity between each operator and the input query embedding e, (4:). This method, inspired by Vaswani et al. (2017). yields dynamic, query-contexttul state representation that captures what an architecture can do for specific query without the significant training overhead of dedicated GNN.
The explicit objective for the meta-controller is to maximize the expected discounted sum of future rewards over stream of queries:
T= Σ 7t Ra(se, at) 7 10 t=0
arg maX E
(4)
4

---

Published as a conference paper at ICLR 2026
3 METHODOLOGY
Baxc Operatee Peel
Disooered
Oof
Cal SC
Ddea:
Raket
Speckaltt iuge
Selection
nal Fhe deldd) 4m
smmplty cullu
Setfkelie
Enscribi
CoT
Refine
Drvesats
Fnanriia
Defactee
Prorel "Tua are ligen, rediertorann dany chaieeess
Plpakarn (Cof)
Provg: "You algeisaee with axaatia
sy releereriee
uineal use A4l)
ReAct
Create
CD PCE
prggenenr who from odjauti
Cir caltiant (Dinata)
Prompt X acuerilc
Debate
atolks wiah hartyy
Mannuy: cedattadd spanslion user laoga batker performancs.
la neviraiig the care of paikert
Drionee
Pronept 20 Your abard Instreei
Specisess
IlI. Experience Gathering
Figure 3: The offline automated specialist discovery and cultivation process.
Our framework's methodology is twofold. First, end-to-end offline process discovers stateful specialists and trains the meta-controller (Figure and Algorithm These components are then deployed in online adaptation loop to handle unseen queries, with the operator pool fixed (Figure 2 and Algorithm D] To explore the space of possible specialists and identify a set of specialist operators (2) diverse, and (3) specialized to the problem task domain without human intervention, we split the learning objectives into two distinct phases: initial exploratory specialist discovery phase to address (1) and (2), and focused, apperience-gathering cultivation phase to address (3), mirroring how a human expert might first learn broad concepts and then deepen their knowledge through practice.
Oxecc such that the resulting operator pool is (I)
3.1 SPECIALIST DISCOVERY
Depicted as stages and II Figure 3 during the specialist discovery phase, the Architect iteratively evolves pool of specialists using its full action space (detailed in Appendix G.I) p. We formalize the action space using the notions of creation and crossover.
Creation. Let (l be the pool of specialist operators during the specialist discovery phase and Olge be the pool of specialist operators during the cultivation phase. For a query q., the Architect can propose a specialist O E Of derived from base operator O by instantiating its prompt with structured -dentity-directive pair. The creation process employs multi-variant synthesis with LLM adjudication. In practice, we overgenerate S candidate identity-directiv variants with diverse pairs, then judge variants via an LLM-guided evaluation process that considers the reasoning methodology and domain coverage. We provide the prompts for the Judge in Appendix G.3
To prevent early fragmentation, we enforce dynamic pool size limit of 2 x k. where & is the maximum size of the final specialist selection pool. If the pool exceeds this limit, the Architect is restricted from creating new specialists and must combine prune existing agents, forcing the consolidation of narrow capabilities.
Given parent specialist operators of and O the Architect can synthesize a child specialist 08 similarly by using variant generation. This similarly triggers multi-varian synthesis process with LLM adjudication that combines both parents' specialist identities and directives. preserving their expertise. We provide the prompts used perform this synthesis in Appendix G.2
Crossover.
5

---

Published as a conference paper at ICLR 2026
Selection. At the end of the specialist discovery phase, select the top-k specialists for cultivation by solving -ujei-obeective optimization problem that balances performance and diversity:
Specialist Lineage
Prompt (P,)
Memory (M)
You
physidist teckling
Problemn Expectation
Optics
complte oden tils prozlems You howe deep rxpertion in piayeiza induuting diccromagisttor doomodymamnn mics. wave cnetkos and comistira meahieiion
operaaoo (n p²) glann non-
nors snvellrctton
Approoch summary. Normalzze
irngeate
Fallure mode Forgalingg rommotion
Think stop s9ep
cepootal on volue.
General nole Al WAYS norma tre the weselord bettoe calorating
+I. Anahaze
Themmo
Huygame prisppe superpse and innefrrenne Rebate wive propertes tuch be waveleggth frequencyy serpitudde
medienios
momentam
Plysice
-I-. Hpppy themedyyarri
Enerceeenna Fowttork
Plassics
Inveling bwnt,
Figure 4: Case study of physics specialist discovered on GPQA. The crossover action allows to trace back the agent's "lineage" and identify aspects of its prompt that have been inherited from its ancestors. The full final prompt and more examples of its memory entries are in Appendix AA.3
(2) arg spec
Σ OPED
p(O Diversity( Diversity(Ospee)
Opee
Sk
(5)
Diversity (
Σ 0f cc
p(O,))
spec
max
where )( represents the average performance of specialist 0 and C₁ is the j-th cluster in embedding space obtained via K-means clustering on specialist operator embeddings.
3.2 SPECIALIST CULTIVATION
Depicted as stage III of Figure 3 during the specialist cultivation phase, the selected top-k discovered specialists deepen their domain expertise through post-execution reflection on training corpus. The cultivation process is applied independently to each specialist, resulting in distinct, specialized memories, as can be seen in Figure For each specialist O with accumulated memory M,, we implement a semantic retrieval mechanism (Lewis et al.) (2020) to inject relevant experience during tasks. Given query gr, we partition the memory into structured chunks, then inject the most relevant chunks as contextual knowledge during specialist execution.
RESULTS
Benchmarks & Baselines. We evaluate ASPEC on five public benchmarks across three domains: math reasoning with MATH (Hendrycks al. 2021) question answering with MMLU (Hendrycks) et al. 202T) and GPQA (Rein et al. 2024), code generation with HumanEval (Du et al.) 2024b) and SeiCode (Tian al. 2024). In particular, GPQA and SciCode are expert-level QA and coding benchmarks respectively. Further details on the dataset statistics are in Appendix
We select 13 representative baselines across (1) hand-designed single agents, in particular Chain- of-Thought (Wei et al., (022). Self-Refine (Madaan et al. 2023), Self-Consistency (Wang et al.) 2023). Reflexion (Shinn et al. 2023); (2) hand-designed multi-agents, in particular LLM-Debate (Du al. 2024a). DyLAN (Liu et al. 2024); (3) automated agent specialisation methods with 2025). AutoAgents (Chen et al. 2024b), EvoAgent (Yuan et al. 2025 ; Role Assignment (Xu et al.
and (4) autonomous agent design frameworks, including query- level MaAS (Zhang al. 2025c
and task-level AFlow (Zhang et al. 2025b) and ADAS (Hu et al. 2025). Details for the baseline
setups are in Appendix
B

---

Published as a conference paper at ICLR 2026
Implementation. We select Gemini 2.0 Flash to be the standard execution model across all methods, alongside GPT-4o-mini and Llama 3.3 70B Instruct in Figure We set the size of the sliding window in Equation to be m 10 and the maximum number of specialists in Equation to be lt = 5.
Table 1: Performance comparison across methods. We use Gemini 2.0 Flash with temperature of
T' = 0.3 consistently across all methods. Best results are in bold, second-best are underlined.
Method
MATH
HumanEval
MMLU
GPQA
SciCode (SP)
Average
Vanilla
73.2
87.8
86.0
56.3
24.0
65.3
Coll Wei 2022
74.5
24.3
65.5
CoT-SC Wang et 2023
75.1
91.2
88.8
57.1
25.2
67.5
Self-Reline Matian
2023
74.8
91.3
88.5
57.4
24.6
67.3
Reflexion Shinn
73.5
86.8
88.5
57.1
25.1
66.2
LLM-Debate (Du
2024
24.0
66.1
DyLAN (Liu al
R124
75.4
89.3
88.9
61.3
25.2
68.0
Role Assigament Xu et
2025
72.4
91.2
89.5
57.4
23.5
67.6
AutoAgents (Chen 2l. 21244
73.4
88.0
85.3
56.8
24.8
65.7
EvoAgent YYmi
75.9
90.2
88.3
61.5
24.8
68.1
ADAS (lu
2025
24.8
66.2
AFlow Zhang at 2025b
76.5
89.3
90.5
61.3
24.3
68.4
MaAS
Zhung al
74.4
91.6
87.3
57.8
25.6
67.4
ASPEC
77.3
91.4
90.0
62.8
26.6
69.6
Figure 5: Cross-model (left) and cross-benchmark (right) transferability results. We evaluate both the full ASPEC and ASPEC with only specialists trained different benchmark.
Full TraA 0 OnlySpec- Tran
LLM Backbone
GPQA
MATH
HumanEval
Pull TAiT
ChlySpec Train:MATH
Gemini 2.0 Flash
56.3
73.2
87.8
ASPBC (Gemini Flash)
62.5
77.3
91.4
90
GPT-4o-mini
38.2
61.8
86.6
Spote
80
ASPEC (GPT-4o-mini)
43.8
64.7
90.9
70R
51.3
88.5
ASPEC (Llama 3.3 70B Instract)
53.5
54.8
90.8
60
HomanFval
MMLU
T demonstrate that ASPEC can consistently match or outperform existing hand-crafted or automated agentic systems across mathematical reasoning, question answering, and coding. Its benefits of 62.8%. This represents a substantial 6.5% improvement over the vanilla Gemini 2.0 Flash model. Furthermore, ASPEC surpasses the leading hand-designed agent (LLM-Debate) by 3.1%, the top autonomous agent framework (AFlow) by 1.5%. and the best automated agent specialisation method (EvoAgent) by 1.3%. ASPEC also leads on SciCode, a benchmark composed of realistic scientific research problems that are decomposed into sequential subproblems, We note that the "retain-thent escalate" structure allows retained specialists build upon context and learned knowledge from previous steps, which is crucial for success in multi- part scientific coding.
Performance Analysis. The results from Table
most pronounced on GPQA, where it achieves score
This naturally leads to the question of whether specialists trained on specific domains can be transferred to other domains. To this end, Figure shows that the performance gains from the ASPEC methodology are robustly transferable across different models and benchmarks. In the besshbbenchmrk analysis (Figure right), we compare the standard configuration against an ablation labeled ONLYSPEC, where the operator pool is restricted exclusively to specialists trained on a different source domain (e.g., utilizing MATH-trainee specialists for HumanEval), and find that the ONLY SPEC configuration matches or even slightly exceeds the performance the full system.
We attribute this the cultivation of "T-shaped" reasoning strategies for specialists (Appendix G.3 furthermore, restricting the pool prevents the Architect from defaulting to "safe" " but less capable generalist base operators, effectively forcing the utilization of these expert reasoning archetypes.
7

---

Published as a conference paper at ICLR 2026
Table 2: Efficiency comparison across methods on the GPQA benchmark.
Training
Inference
Method
Total tokens
Total costs (USD)
Wall clock (min)
Total tokens
Total costs
Wall clock (min)
Accuracy (%)
(USD)
CoT-SC
Wang
2023
3.757,527
58
57.1
LLM-Debats Di al A024a.
1,081,114
0.94
50
59.7
EvoAgent
Youm el ai,
2025
7,080,338
1.45
75
61.8
AFlow
Zhang et ml
2025b
102.012,408
20.14
257
9,997,155
1,58
45
61.3
MaAS Zhang
11,600.690
139
11,015.542
93
57.8
ASPEC
2,395,636
53
3,204,549
63
Efficiency Analysis. Table demonstrates that ASPEC is cost-efficient across both training and inference. In particular, rumning the offline training process on GPQA incurred only total cost of 1.38 USD. We find that once strong specialist pool has been found, the Architect often prefers lean architectures utilizing those specialists. As shown in Table 6 removing specialists causes costs to increase significantly the Architect becomes under-confident in its generalist pool and samples highly complex, but redundant multi-agent architectures in an attempt to compensate.
DISCUSSION
5.1 ABLATIONS OF SYSTEM COMPONENTS AND CONTROL POLICIES
Figure 6: Ablation study of our framework's s components (left) and sensitivity to the maximum number of specialists k and sliding window length (right) on GPQA. For sensitivity plots, the central line shows the mean performance over 4 runs.
65
Accuracy
Toval cosl
2
Accuracy (%)
Total cost
Method
(USD)
60
Control Policy Alternatives
55
ASPEC w/ random policy
ASPEC w/ 0.2
59.6
10
ASPEC w/ as-Mae
62.5
3.74
Maximum numher of specialists (k)
(Gemini 2.0 Flash)
System Components
64
Accuracy Total cost
ASPEC wla specialist operators
57.4
2.26
62
1.5
ASPEC wio hase operators
61.3
0.48
ASPEC meta-controller
60
ASPEC Architect
58
ASPEC wio specialist memory
61.4
0.94
ASPEC
62.8
0.88
56
5
10
15
20
Sliding window length (m)
We perform an ablation study on five key components: (I) without specialist operators, with the operator pool restricted to O = Obuse for all qr; (II) without base operators, with O = O for all g: (III) without meta-controller, which akin to always resampling: and (IV) without architect. in which we construct static architecture consisting of all specialist operators in Ospec- and (V) without specialist memory. Furthermore, we perform additional experiments on a suite of alternative control policies. These include random "coin-flip" cosine similarity heuristic (resample if the cosine similarity of gt and G.-1 is below threshold of h = 0.2), and LLM-as-gate.
As seen in Figure removing specialists causes 5.4% drop in performance from 62.8% to 57.4% and near tripling of the total cost from 0.88 USD to 2.26 USD. Additionally, removing base operators, but keeping specialists, resulted in lesser 1.5% drop. This demonstrates that the expert specialists are the primary drivers both performance and efficiency. Removing the meta-controller results in comparable performance of 62.7% at 2.3 times increase in total costs. We note that
8

---

Published as a conference paper at ICLR 2026
in this mode, the Architect learn to proxy the "retain" action by re-sampling G1-1, but this invocation process is fundamentally less efficient that the explicit "retain" decision made by the controller. F (left) further reveals that the alternative control policies yield significantly lower accuracy at 58.3% compared to the meta-controller's 's 62.8%, and while the LLM--ssgatee policy achieves comparatively high accuracy 62.5%, it does substantially higher cost, 4.25 times that of the meta-controller.
5.2 SENSITIVITY ANALYSES
We analyze the sensitivity of ASPEC to two main parameters: the maximum size of the specialist pool. from Equation and the length the sliding window from Equation [ m. As shown in Figure 6 setting k at both extremities reduced performance, suggesting light Goldilocks-like effect on GPQA. At k 1. the system achieves 58.8% 8%,, performing similarly to the "ASPEC w/o specialist operators" ablation seen in Table indicating that single specialist lacks the domain coverage to outperform generalist operators. Conversely, at k 10, performance drops 60.9%, which aligns closely with the "ASPEC w/o specialist memory" ablation. We attribute this to experience fragmentation: with larger pool size, individual specialists are selected less frequently by the Architect during the Cultivation phase. Since memory is only acquired upon execution, these "sparse" specialists fail to accumulate the dense history required to form deep expertise.
We hypothesize that this is not necessarily hard limitation and the Goldilocks distribution might depend more on the depth of an average specialist's experience and exposure to problems.
5.3 CONVERGENCE OF THE SPECIALIST DISCOVERY PROCESS
Figure 7: Visualization of discovered specialist operator embeddings on "narrow" domain bench- mark (GPQA) and on a "broad" domain benchmark (MMLU).
bit
To determine whether ASPEC's discovery process reliably finds similar expert archetypes, we embedded the prompts of discovered specialists across independent trials and plotted them in Figure
7 We find that there is strong convergence on GPQA (Figure left), with different runs independently discovering the same key roles (chemistry, biology, physics), demonstrating the robustness of the process for specialized domains. Conversely, on the broad-domain MMLU benchmark (Figure 7 right), the process shows some divergence, exploring different but viable team compositions cover the vast topic space. Even so, we find pockets of convergence in well-defined sub-domains like the physical sciences. Taken together, these results show that the ASPEC discovery process adapts its convergence/divergeann behavior based on the specificity of the target domain.
5.3.1 RATIONALITY ANALYSIS.
We compare learned meta-controller's decisions against the LLM-as-gate "oracle proxy" in Table 6 On GPQA, the controller learns a pragmatic economic policy, where its high rate of "overconfident" disagreements with the perfectionist oracle reflects a deliberate trade-off for cost efficiency. On MMLU, this behavior persists, but instances of "wasteful caution" " reveal the limitations of its lightweight state representation, leading to unnecessary resampling.

---

Published as a conference paper at ICLR 2026
Figure 8: Side-by-side comparison of confusion matrices for GPQA (left) and MMLU (right).
20.0 3N
310 Mho
Controlter oecisio
140 01200
RESAMPLE
Weych Ceulle
RESAMPLE
Rowert Conters
Corect linplaaie WhiccA
60
RETAIN
RESAMPLE
RETAI
RESANFLE
LLM as gate ceck on
LUM gata decisoo
6
LIMITATIONS & FUTURE WORKS
A key future direction is the development of rigorous theoretical framework to model the con- vergence properties of the specialist discovery process with respect to factors like domain breadth. potentially leading to principles for self-tuning the discovery process. Future work should also validate ASPEC's applicability in more diverse environments, particularly on complex, real-world software engineering tasks such as those in SWE-bench (Jimenez et al.) (023). Intuitively, specialists discovered and cultivated on specific repository could autonomously internalize its unique conven- tions and APIs, a promising avenue for automating repository-specinn expertise without manually engineered rules. Finally, specialists may amplify training biases through memory cultivation, a risk that warrants further investigation and the development of mitigation strategies.
While our lightweight meta-controller is crucial for efficiency, we identify its alignment with an "oracle proxy" LLM--as-atte policy another critical area for improvement. The results of ablation study on GPQA in Table/o|migh be masking an underlying limitation: the meta-controller's decision-making process diverges from the oracle proxy's. s. This divergence can become significant weakness when its lightweight state representation leads to errors such as unnecessary resampling or over-cautious retaining. The central challenge design gating mechanism that achieves the decision-makinn fidelity of the LLM-as-gant oracle proxy while retaining the low computational overhead of small, specialized policy.
Finally, we observe that the interplay between the meta-controller's policy, the Architect's choices, and the specialists' memory accumulation creates a complex, co-evolutionary dynamic. conservative "Retain" policy concentrates experience into smaller set of active architectures, potentially guiding those agents to develop broader, more resilient memories to cope with slightly mismatched queries. Conversely, a highly dynamic policy distributes experience more sparsely across the specialist pool. Furthermore, because the Architect conditions its decisions on recent history (sliding window), may develop methhdependent preferences for certain teams that "suffice" even if they not optimal, further influencing the distribution of experience. Future work could explicitly model this joint optimization ensure the control policy and specialist cultivation are perfectly aligned.
7
CONCLUSION
This paper introduced ASPEC, framework designed to bridge the gap between static, efficient agent workflows and adaptive, per-query optimizers. Our central contribution a methodology for creating and managing stateful specialist agents that accumulate expertise over time, mirroring human learning. This is achieved through an automated lifecycle of evolutionary discovery and experiential cultivation, governed by "retain-then then-escalate policy that ensures cost-effective adaptation. Our results on challenging scientific benchmarks such as GPQA suggest that this agent-centric approach can lead substantial performance improvements without sacrificing efficiency. We believe this work presents promising direction for autonomously creating agent systems that can develop deep expertise while retaining the flexibility to adapt to new challenges.
Large Language Models (I.LMs) were used assist in writing this paper.
10

---

Published as a conference paper at ICLR 2026
REFERENCES
Qingyun Wu, Gagan Bansal, Jieyu Zhang, Yiran Wu, Beibin Li, Erkang Zhu, Li Jiang, Xiaoyun Zhang. Shaokun Zhang. Jiale Liu, Ahmed Hassan Awadallah, Ryen W White, Doug Burger, and Chi Wang. Autogen: Enabling next-gen LLM applications via multi-agent conversations. In First Conference on Language Modeling, 2024, URL httpp:://ppeteeveew..t http://iwwweee/ffuuerreee
BAakY1hNK,
Guohao Li, Hasan Hammoud, Hani Itani, Dmitrii Khizbullin, and Bernard Ghanem. Camel: Com- municative agents for mind exploration of large language model society. Advances in Neural Information Processing Systems, 36:51991-52008, 2023.
Sirui Hong, Mingchen Zhuge, Jonathan Chen, Xiawu Zheng, Yuheng Cheng, Jinlin Wang, Ceyao Zhang, Zili Wang, Steven Ka Shing Yau, Zijuan Lin, Liyang Zhou, Chenyu Ran, Lingfeng Xiao, Chenglin Wu, and Jürgen Schmidhuber. MetaGPT: Meta programming for multi-agent collaborative framework. In The Twelfth International Conference on Learning Representations, 2024.
ew. net/ forum? d-V.:/AGGNETAWWA
Omar Khattab, Arna Singhvi, Paridhi Maheshwari, Zhiyuan Zhang. Keshav Santhanam. Sri Vard- hamanan A, Saiful Haq, Ashutosh Sharma, Thomas T. Joshi, Hanna Moazam, Heather Miller, Matei Zaharia, and Christopher Potts. DSPy: Compiling declarative language model calls into fate-of-thheart pipelines. In The Twelfth International Conference on Learning Representations, net wttp://wwwww.eeettttrteeeee.aae. Forum?icccooo....... SY5N02d550
2024. URL
Mert Yuksekgonul, Federico Bianchi, Joseph Boen, Sheng Liu, Zhi Huang. Carlos Guestrin, and James Zou. Textgrad: Automatic" differentiation" " via text. Nature, 639(7985):609-6)) 616, March 2025. doi:
Chengrun Yang. Xuezhi Wang, Yifeng Lu, Hanxiao Liu, Quoc Le, Denny Zhou, and Xinyun Chen. Large language models as optimizers. In The Twelfth International Conference on Learning Representations, 2024. URL //t///tttts///peneviieeeeer wtwpriiiwwee//frruie http://ppnneeeeeeeeeeefooooeeeee 2..
Mingehen Zhuge, Wenyi Wang, Louis Kirsch, Francesco Faccio, Dmitrii Khizbullin, and Jürgen Schmidhuber. GPTSwarm: Language agents as optimizable graphs. In Forty-first International Conference on Machine Learning, 2024. URL https:://ppnrest UTC9AFXIhc
net. forum? id=
Zijun Liu, Yanzhe Zhang, Peng Li, Yang Liu, and Diyi Yang. Dynamic Ilm-agent network: An Ilm- agent collaboration framework with agent team optimization. Conference on Language Modeling, 2024.
Guibin Zhang, Yanwei Yue, Xiangguo Sun, Guancheng Wan, Miao Yu, Junfeng Fang, Kun Wang, Tianlong Chen, and Dawei Cheng. G-designer: Architecting multi-agent communication topologies via graph neural networks. In Forty-second International Conference on Machine Learning, 2025a.
URL
wttp:eeeweeieeeeeeoonnnneraaaeae net. forum?
Thomas Elsken, Jan Hendrik Metzen, and Frank Hutter. Neural architecture search: A survey, Journal of Machine Learning Research, 10(55):1-21, 2019.
Shengran Hu, Cong Lu, and Jeff Clune. Automated design of agentic systems. In The Thirteenth International Conference on Learning Representations, 2025. URL /tRl:tttps///oeeeeiiii
net/ http://ooummmddttssS 9U3LN IJVX
Jiayi Zhang, Jinyu Xiang, Zhaoyang Yu, Fengwei Teng, Xiong-Hui Chen, Jiaqi Chen, Mingchen Zhuge, Xin Cheng. Sirui Hong, Jinlin Wang, Bingnan Zheng, Bang Liu, Yuyu Luo, and Chenglin Wu. AFlow:
Automating agentic workflow generation. In The Thirteenth International Confer- ence on Learning Representations, 2025b. httpst/:: /RRphhtts:////neereee /ttp///ppeevieeeeee/eeeeeeeeeeee net / forum? d z z5uVAKwm
Yu Shang, Yu Li, Keyu Zhao, Likai Ma, Jiahe Liu, Fengli Xu, and Yong Li. Agentsquare: Automatic LLM agent search in modular design space. In The Thirteenth International Conference on Learn- ing Representations, 2025. URL htt.ttts:: htt.ttpps.//ppeeerrieeeeeeee wwt.::www..ee.rr.eeeeeeeee ptt.rrmmmmmmdmmmmmmmmmmm

---

Published as a conference paper at ICLR 2026
Guibin Zhang, Luyang Niu, Junfeng Fang, Kun Wang. LEI BAI, and Xiang Wang. Multi-agent architecture search via agentic supemet. In Forty-second International Conference on Machine Learning, 2025c. URL
Hongcheng Gao, Yue Liu, Yufei He, Longxu Dou, Chao Du, Zhijie Deng, Bryan Hooi, Min Lin, and Tianyu Pang. Flowreasoner: Reinforeing query-level meta-agents, 2025. URL https:
7/arxiv. wttp:/ss///........./. 15257
Yinjie Wang. Ling Yang, Guohao Li, Mengdi Wang, and Bryon Aragam. Scoreflow: Mastering Ilm agent workflows via score-based preference optimization. arXiv preprint arXiv:2502.04300, 2025.
Rui Ye, Shuo Tang, Rui Ge, Yaxin Du, Zhenfei Yin, Siheng Chen, and Jing Shao. MAS-GPT: Training
LLMs to build LLM-based multi-agent systems. In Forty-second International Conference Machine Learning, 2025. URL /ttp:r/////peeeeeeeeee hwtp:/ewwwwwweeeeeeeeeeeeeeeeeee wttptt/eoooommmmmm......
Zixuan Ke, Austin Xu, Yifei Ming, Xuan-Phi Nguyen, Caiming Xiong, and Shafiq Joty. Mas-zero: Designing multi-agent systems with zero supervision, 2025. URL abs abs/2505. 14996
org/
Benfeng Xu, An Yang, Junyang Lin, Quan Wang, Chang Zhou, Yongdong Zhang, and Zhendong Mao. Expertprompping: Instructing large language models to be distinguished experts, 2025. URL
https::/arxiv. org/ 2w05. 14688
Chrisantha Fernando, Dylan Banarse, Henryk Michalewski, Simon Osindero, and Tim Rocktäschel. Promptbreeder: Self-referential self- ielrovmmpovement via prompt evolution, 2023. URL https
//axxiv.orr//ab//2309. 16797
Xinyuan Wang, Chenxi Li, Zhen Wang, Fan Bai, Haotian Luo, Jiayou Zhang, Nebojsa Jojie, Eric Xing, and Zhiting Hu. Promptagent: Strategic planning with language models enables expert-level prompt optimization. In The Twelfth International Conference on Learning Representations, 2024a. URL htt.hhttts//pppeeee
net forum 2ii:iidd222pp....m Ioa
Siyu Yuan, Kaitao Song, Jiangjie Chen, Xu Tan, Dongsheng Li, and Deqing Yang. EvoAgent: Towards automatic multi-agent generation via evolutionary algorithms. In Luis Chiruzzo, Alan Ritter, and Lu Wang, editors, Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguisties: Human Language Technologies (Volume I: Long Papers). pages 6192-6217, Albuquerque, New Mexico, April 2025. Association for Computational Linguistics. ISBN 979-8-89176--18866
doi: (06653///20025.naccllooooo ogy wwg.....r..//... naa.
URL https: / /ac aathhoannhhoooooo
long. 3157
Han Zhou, Xingchen Wan, Ruoxi Sun, Hamid Palangi, Shariq Iqbal, Ivan Vulié, Anna Korhonen, and Sercan Ô. Ank. Multi-agent design: Optimizing agents with better prompts and topologies, 2025.
URL https: wwwp:xivvvv.orvveeemmme
wtw.orgggbbbs/o/o abs.///2./22222 02533
Weize Chen, Yusheng Su, Jingwei Zuo, Cheng Yang. Chenfei Yuan, Chi-Min Chan, Heyang Yu, Yaxi Lu. Yi-Hsin Hung, Chen Qian, Yujia Qin, Xin Cong, Ruobing Xie, Zhiyuan Liu, Maosong Sun, and Jie Zhou. Agentverse: Facilitating multi-agent collaboration and exploring emergent behaviors. In The Twelfth International Conference on Learning Representations, 2024a. URL htttps://operret
net/ Eor forummll-=
Guangyao Chen, Siwei Dong, Yu Shu, Ge Zhang, Jaward Sesay, Börje F Karlsson, Jie Fu, and Yemin Shi. Autoagents: framework for automatic agent generation. International Joint Conference on Artificial Intelligence, 2024b.
Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. Reflexion: Language agents with verbal reinforcement learning. Advances in Neural Information Processing Systems, 36:8634-8652, 2023.
Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, et al. Self-refine: Iterative refinement with feffbeee Advances in Neural Information Processing Systems, 36:46534 46594, 2023.
12

---

Published as a conference paper at ICLR 2026
Andrew Zhao, Daniel Huang, Quentin Xu, Matthieu Lin, Yong-Jin Liu, and Gao Huang. Ex- pel: Lim agents are experiential learners. In Proceedings of the Thirty- Eiirty-Eighth AAAI Con- ference on Artificial Intelligence and Thirty-Sixth Conference on Innovative Applications of Artificial Intelligence and Fourteenth Symposium on Educational Advances in Artifccial In- telligence, AAAI244AATT2EAAAT 10. wtt.0//aaa..38i1.... 299963.. URL /tt.:tsss//.....og//11....
AAAI Press, 2024. ISBN 978-1-57735-8887 887-9 doi: 0ttp:///aaaa.3311.2.2233333
Yao Fu, Dong-Ki Ki Kim, Jackyeom Kim, Sungryull Sohn, Lajanugen Logeswaran, Kyunghoon Bae, and Honglak Lee. Autoguide: Automated generation and selection of context-aware guidelines for large language model agents. Advances in Neural Information Processing Systems, 37:119919-1119995 2024.
Zora Zhiruo Wang, Jiayuan Mao, Daniel Fried, and Graham Neubig. Agent workflow memory, 2024b.
URL https://aaxiv.... www.vbo.rgg....... ab00 07429
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. -hain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824-248311 2022.
Yilun Du, Shuang Li, Antonio Torralba, Joshua B. Tenenbaum, and Igor Mordatch. Improving factuality and reasoning in language models through multiagent debate. In Proceedings of the 4 1st International Conference on Machine Learning, (CML'24. JMLR.org. 2024a.
Ranjita Naik, Varun Chandrasekaran, Mert Yuksekgonul, Hamid Palangi, and Besmira Nushi. Diver- sity of thought improves reasoning abilities of Ilms, 2024. URL 2310.00 07088
Wenhui Wang, Furu Wei, Li Dong, Hangbo Bao, Nan Yang, and Ming Zhou. Minilm: Deep self- attention distillation for task-agnostic compression of pre-trained transformers. Advances in neural information processing systems, 33:5776-5788, 2020.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017.
Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, al. aetrieval-auummeter genera- tion for snowledee-intnsive nlp tasks. Advances in neural information processing systems, 33: 9459 9474, 2020.
Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. Measuring mathematical problem solving with the math dataset. Advances in Neural Information Processing Systems, 2021.
David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Dirani, Julian Michael, and Samuel R Bowman. Gpqa: A graduate-level google-proot q&a benchmark. In First Conference on Language Modeling, 2024.
Xueying Du, Mingwei Liu, Kaixin Wang, Hanlin Wang, Junwei Liu, Yixuan Chen, Jiayi Feng, Chaofeng Sha, Xin Peng, and Yiling Lou. Evaluating large language models in class-level code generation. In Proceedings of the IEEE/ACM 46th International Conference on Software Engineering, ICSE '24, New York, NY. USA, 2024b. Association for Computing Machinery. ISBN 9798400702174, doi: 10. 263144//59970033332219999 URL /t//oo.oorg/..11101 1145/ 3597503. 3639119
Minyang Tian, Luyu Gao, Shizhuo Zhang, Xinan Chen, Cunwei Fan, Xuefei Guo, Roland Haas, Pan Ji, Kittithat Krongchon, Yao Li, et al. Scicode: A research coding benchmark curated by scientists. Advances in Neural Information Processing Systems,
2024.
Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc V Le, Ed H. Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou Self-consistency improves chain of thought reasoning in language models. In The Eleventh International Conference on Learning Representations, 2023. URL https:://ppereet wtt.eeeente.ooooolddii....
NNMMMWW
13

---

Published as a conference paper at ICLR 2026
Carlos E Jimenez, John Yang, Alexander Wettig, Shunyu Yao, Kexin Pei, Ofir Press, and Karthik Narasimhan. Swe-bench: Can language models resolve real-world github issues? arXiv preprint arXiv:2310.06770, 2023.
Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. React: Synergizing reasoning and acting in language models, 2023. URL https://aaxxi:
org/abs/2210 20tp:/210..00000.0 03629
Dongfu Jiang, Xiang Ren, and Bill Yuchen Lin. LLM-blender: Ensembling large language mod- els with pairwise ranking and generative fusion. In Anna Rogers, Jordan Boyd-Graber, and Naoaki Okazaki, editors, Proceedings of the 61st Annual Meeting of the Association for Com- putational Linguisties (Volume 1: Long Papers), pages #4165-14178 Toronto, Canada, July 2023. Association for Computational Linguistics. doi: 10. 1/26533/ acl-long.999 URL https: // antho hrt.://oooo.rrg002.....11--
7927|
Dong Huang, Jie M. Zhang, Michael Luck, Qingwen Bu, Yuhao Qing, and Heming Cui. Agentcoder: Multi-agent- code generation with iterative testing and optimisation, 2024. URL https? //arxiv. atg/abb//2122.110000
14

---

Published as a conference paper at ICLR 2026
A
CASE STUDY
A.1 META-CONTROLLER DECISION-MAKING
We provide few examples of trained meta-controller's decision-making process on GPQA. These include (I) rational decisions, such as retaining or resampling sensibly, and (II) irrational decisions. when the imperfect meta-controller chooses to retain mismatching architecture or resample matching architecture, thereby incurring expensive, unnecessary costs from the Architect call.
A.1.1 RATIONAL DECISIONS
Query: "Determine which set of states mentioned below only entangled states:
(a) (1/ 30 )+ (100>+ 2i101>
3110>
4i111>>
(b) (1/5) (100>+ |110
2110>
4i/11>>
(c) (1/2) (100>+ |01>+110>
111>)
(d) (1/2) (100>+ 101>-110>
111>)."
Current architecture: [["CoT"), ["CoT_ [[o[[TThoret.ccaPPppsssss"
Action taken: "RETAIN"
Resulting architecture: [["CoT"), ["CoT_ ""oo__Theoratiaalllyyyssss"
Outcome: CORRECT
Query: "Identify the missing reagents in the following reaction. (3r, 5r, 7r) -adamanttt------rrooolllll (3r, 5r, 7r) -ademaatameeeeee -carbonyy azide ---> (3s, 5a, 75) (alrssaadaaammn -adamanann---aniiaaa
acid --->
Current architecture: [["CoT"), ["CoI_ ""oo Action taken: "RESAMPIE"
Resulting architecture:
Outcome: CORRECT
A.1.2 IRRATIONAL DECISIONS
Query:
"The Cope rearrangement is chemical reaction where 1-5-d' ene molecule undergoes rearrangement, resulting in change in the positions of its ca caronncarbon double bonds. This rearrangement can be initiated by heat or light and is valuable for creating complex organic conpounds with changed structures. Select the major products from the following zearrangements [...]"
Current architecture: [["Co]"], ["Cor_Thooreiicc]lyyyss"""
Action taken: "RETAIN"
Resulting architecture: [["CoT"), [CCo[[TheortticalPlyssssss
Outcome: INCORRECT
Problem index:
Query: "Astrencmers are searching for exoplancts around two stars with exactly the same masses. Using the RV method, they detected one planet around each star, both with masses similar that Neptune [...]
The question is: How many times is the orbital period of planet #2 longer than that of planet #1?"
Current architecture: [""or__hherreticcllyyyys Action taken: "RESAMPLE"
Resulting architecture: [["CoT_ haecc__eecceticalhhysssssss
Outcome: CORRECT
15

---

Published as a conference paper at ICLR 2026
A.2 ARCHITECT REASONING PROCESS
Figure A three-turn reasoning process the Architect.
Initial State During the discovery phase, the system receives a query that asks solve a complex quantum mechanics problem involving both wave function normalization and energy eigenvalue calculations. The current architecture is [ [[Co]]] I. The operator pool con- tains basic operators and CoT rophhssrophysil specialist, but lacks quantum mechanics expertise.
Turn 1: Specialist Creation domain-specific expertise in quantum mechanics.
The Architect analyzes the query and identifies the need for
<aCAiOn>CCRAATE htt.///AAAAT..PAAAAT/./.........
<reasoning>
The query requires deep expertise in quantum mechanics, specifically wave function normalization and energy eigenvalue calculation allonsstion.... will create specialized operator that can handle the mathematical rigor required for quantum mechanical problems.
</reasoning>
<new operator name>CoT n Quantum/eecharrccss//er oteratto name> <base templatexCoC<<base template>
<identity>You are an expert quantum physicist specializing wave function analysis and energy eigenvaluc problems. You have deep expertise in Schrodinger equations, Hilbert spaces, and operator theory, with broad knowledge acress theoretical physics and mathematical methods. </identity> <directives>
Always verity wave function normalization conditions Apply systenatic eigenvalue analysis using appropriate boundary conaitions l...]
/iirctivess>
Turn 2: Operator Combination After creating the quantum specialist, the Architect decides to combine it with an existing physics specialist.
<action>CoMBINE, htW.:///MINNNNPPPPPRRRRAAREEEEEE
<reasoning>>
Analyzing the recent sliding window of past experiences reveals pattern of diverse physics problems including both quantum mechanics and rstrophssics queries. Rather than ma maintaining narrow specialists that may underperform en this broader distribution, I will combine COT cor_@guantumMecaaniee and CoT Astrooyysss into a unified CoT hhecreetccslPPysiicci. specialist.
</reasoning>
<new hhaw_operatoo__nmme>22
<identity>You are expert theoretical physicist with deep expertise in quantum mechanics and astrophysics. You combine rigorous mathematical analysis with physical intuition to solve complex problems in quantum theory, astrophysics, and mathematical poysics.</identtty. <directives>
For quantum mechanics problens, always veriřy wave function normalization conditions and apply systematic eigenvalue analysis [...]
For astrophysics problems, apply scaling analysis and use dimensional arguments constrain solutions [...]
16

---

Published as a conference paper at ICLR 2026
</directive>>>
Turn 3: Architecture Finalization With the enhanced specialist available, the Architect
designs the optimal processing pipeline.
RRotoonnTTMLLLEBBBRCCCCCCUU/R// <reasoning>
will create two-layer architecture:
['SelfRefine"]] The theoretical physics specialist will provide the initial detai betailed analysis, and SelfRefine will critically evaluate and improve the solution.
[[' hoor_hheretticllhyssee
< </reasoning>
<architecture> [ [ ["SelfRefine") <
A.3 ANATOMY OF SPECIALIST
We provide example of physicist that has been autonomously discovered and culti- vated on GPQA. This specialist was the result of crossover between two parent special- ists, COT THERMODYNAMICS and COT WAVEOPTICS, who themselves descended from COT ELECTROMAAGNTISM COT LO__LANAARALEEDRA OPTICS, and CoT MEC_ANCIASSSS.
Specialist Prompt: COT PHYSICS
You are an expert physicist tackling complex scientific problems. You have deep expertise in physics, including electromagnetism, thermodynamics, wave optics, linear algebra, wave phenomena, kinetics, and statistical mechanics. When faced with a complex problem, you always start by identifying the fundamental physical principles at play. breaking down the problem into its core components before attempting to solve it. You visualize physical phe- nomena as interconnected networks of energy and momentum, allowing you to intuitively understand their behavior.
Think step by step and derive concise final answer.
Focus on identifying the fundamental physical principles underlying the problem.
Apply knowledge from various areas of physics, including electromagnetism, ther- modynamics, kinetics, wave optics, linear algebra. Consider the interplay between physics, chemistry, and biology when relevant.
Prioritize dimensional analysis and order of of-magnitude estimates to quickly assess the plausibility different solutions. Likewise, simplify complex problems by identifying dominant terms and making appropriate approximations.
Analyze wave phenomena using Huygens' principle, superposition, and interference. Relate properties such as wavelength, frequency, and amplitude to the energy and momentum of the wave. Apply the laws of thermodynamics and statistical mechanics to analyze systems involving heat, energy, and entropy.
# Learned from experience:
Prioritize accurate identification of fundamental transformations (e.g., electron flow)
before making broader predictions.
17

---

Published as a conference paper at ICLR 2026
When comparing results from different methodologies, explicitly consider the limi- tations and biases inherent each technique. Focus on underlying mechanisms and principles rather than superficial alignment of results.
Consider frequency and averaging effects when integrating data from population- level and single-entity measurements.
Specialist Memory: CoT PHYSICS
# Structured memory entry:
Problem pattern: EM wave attenuation; inconsistent parameters lead to physically impossible results (c.g., amplification instead of attenuation).
Approach summary: Verify problem consistency by calculating attenuation from given parameters. Identify and state inconsistencies explicitly.
Failure mode: Blindly applying formulas without checking physical plausibility; incorrect assumptions about attenuation contributions.
General rule: Before solving, check given parameters yield physically plausible results. If not, state the flaw and assumptions made for solution.
# Structured memory entry:
Problem pattern: Expectation value of an operator (e.g.. p²) given a non-normalized wavefunction.
Approach summary: Normalize wavefunction, apply the operator, and integrate.
Failure mode: Forgetting normalize the wavefunction before calculating the expectation value.
ALWAYS normalize the wavefunction before calculating expectation values in quantum mechanics.
General rule:
# Structured memory entry:
Problem pattern: Particle decay (c.g.. II + + v) with known rest masses and initial state. Find KE of products.
Approach summary: Apply energy and momentum conservation. Use relativistic energy-momentum relation (E² = (pc)² + (mc²)²) to relate KE and momentum.
Failure mode: Incorrectly applying relativistic formulas or conservation laws; alge- braic errors in solving the equations.
General rule: In particle decay, use energy/momentum conservation and relativistic relations. If one particle is rest initially, simplify accordingly.
B BASELINES
In this section, we detail the implementation for each of the baseline methods. For Chain-of- Thought (eei et al.) 2022), Self-Consise Consistensiyyy (Wang et al. (023). Self-Refine ((Madaan et al. 2023) ), and LLM-Debate Duu et al. 2024a), we refer to Appendix C for the configuration details, as they were used seed base operators in APPEC. For Reflexion,
adhere the implemention provided in (Shinn et al., 2023). Following ADAS (Hu et al.] 2025), we implement Role Assignment (Xu et al.) 2025) by prompting role-selector LLM choose role from a predefined set, then another
LLM to act as the chosen role to answer the question.
For each of the benchmarks, the roles for Role Assignment were:
18

---

Published as a conference paper at ICLR 2026
MATH:
Algebraist, Number Theorist, Real Analyst, Statistician, Logician
HumanEval: Senior Python Engineer, Algorithms Expert, Software Architect, Data Scien- tist, Competitive Programmer
MMLU: Biologist, Physicist, Mathematician, Engineer, Doctor, Lawyer
GPQA: Physicist, Chemist, Biologist, Scientific Reasoning Expert, Graduate Student
SciCode: Biologist, Physicist, Chemist, Computer Scientist, Mathematician
For DyLAN and EvoAgent, we directly used the implementations from Liu et al.
(2024) and Yuan
2024b). For ADAS et al. 2025). We adhered to the official configuration for AutoAgents (Chen et al. (Hu et al. 2025). we set the Meta Agent Search's n--eneratiin to 20, For MaAS, our experimental setup directly utilized the optimized graphs and operator spaces from (Zhang et al. (2025c) for MATH and HumanEval. For benchmarks not explicitly included in the MaAS repository (GPQA, MMLU, SciCode), we implemented the operator space as described in the appendix. Following Zhang et al. (2025c), for AFlow, we utilized Gemini 2.0 Flash consistently throughout our experiments in place of GPT-44-mini and Claude 3.5 Sonnet for homogeneity.
C OPERATOR SPACE
Following MaAS Zhang et al. 2025c). we use the following operator space for our base operators:
Chain-of-Thought (Wei et al. 2022 which encourages the execution LLM to think step- by-step before outputting answer.
ReAct (Yao et al. 2023). allowing the execution LLM to use library of tools to answer the question.
Self-Consistency (Wang et al. 2023), which aggregates five Chain-of- Chain-of-Thought Thought answers and majority votes to agree on a final answer.
Self-Refine (Madaan et al.
2023 which iteratively refines initial Chain-of-Thouugh
answer over five iterations.
LLM-Debate (Du al., 20224), which uses multiple execution LLMs debate against each other to reach final consensus. We similarly use three debaters and two rounds of debate our implementation.
Ensemble (Jiang al. 2023). which takes in two and uses pairwise ranking to aggregate these responses into a final answer.
answers from different sources
Testing (Huang al., 2024), which generates cases for subsequent execution L.MMs given coding problem.
D ALGORITHS
Algorithm 1: Online adaptation algorithm ASPEC
Input: Trained meta-controller T&: operator pool O; queries Q {gi...., (r sliding window buffer H.
Initial graph Go-
for t = 1,2,. do
Construct state s, (e,(()), Sample action de ~ (a st); if GRESAMPLE then
G. + fa(ge, Hr H.-mat11. 0,9,
else
G. + Gc-1i
end
Execute 0. 9:):;
U.,C+ + Evaluate(p,, a;);
Store experience (#,, G, S., Ct) in H:
end
19

---

Published as a conference paper at ICLR 2026
Algorithm 2: Offline specialist discovery and cultivation
Input: Queries Q = [(11..... (r); base operator set Obss-
Initial operator pool (g Dbsee initial specialist pool O f 0. random random-weights
meta-controller (0)), empty sliding window buffer H. t- 0.
for { = 1, ........ do Construct state 8t Sample action at (T((a))))) St);
(Gt--))))
if = GRESAMPLE then
aA + (g. H4-mat-11 1 9--1)
if aA = CREATE OPERATOR then
Onew + CreateSpeciallst Obase) ;
Olpi (t-1) U {Onew)
O, t 0,-1 U Offe
end
else if as
then (0,,0) + Selecttppratoors(
Ochild < Combine O2 9:);
O spec t (0!pee) {O1, O2}) U {Oema))
O2 - 0.-1 O
end
else if aλ PRUNE OPERATOR then
Or -elecctpprrator (Dlpee (t-1))
O 0,((ee)) (m
Og + 0,-1 U 0ppec
end
Gi (1
else
9. t- 9--1
end
Pt + Execute(Sr, Or, gr);
U2. C, + Evaluate (pt, a)); (8)
Weigtt (U,, Ct, ae: Th (
T
forall O E Specialistssseddie 02) do
r + Reflect(O,, qt, Pt. az U
WriteToMemory(O..1
end
Store experience
(q: Uc, Ct) in H;;
end
E META-CONTROLLER IMPLEMENTATION
The meta-controller is trained using the REINFORCE algorithm, with a standard batch policy loss:
1 N Σ log (a
Leaach (0)
(6)
The reward RG is designed to balance performance, cost, and contextual appropriateness. is function of the final task score st, the total cost C,, and the cosine similarity between the query and the current architecture, sim qt. Gt-1).
The core of our reward function is weighting mechanism that modulates the se based on this similarity. The reward for RETAIN action boosted when the architecture is a good match for the query (high similarity), while the reward for RESAMP LE action is boosted when there is
20

---

Published as a conference paper at ICLR 2026
mismatch (low similarity). This can be expressed conceptually
Rs = 8t wa(a, sim( 9t-1)) AC;
where the weighting function :(: increases the effective reward for correct decisions. For example, w(RETAIN, IN, sim) is an increasing function similarity. This formulation provides dense and informative signal that guides the metaa-ontrooler to learn an efficient, context-aware policy.
F DATASET STATISTICS
For each of the benchmarks, we follow established methodologies for workflow automation (Hu et al. (2025), Zhhng et al. (2023b), [Zhang et al. (2025c)) and use train-to-test ratio 4. We select 19 subdomains for MMLU, spanning formal mathematics, biology, chemistry, clinical medicine, business, and engineering. For SciCode, we use the standard subproblem setup without prior scientist annotations and report the subproblem pass rate.
Table 3: Dataset statistics.
Domain
Dataset
Train Samples
Test Samples
Metric
Math Reasoning
MATH
100
400
Accuracy
Question Answering
MMLU
100
400
Accuracy
GPQA
89
359
Accuracy
Code Generation
HumanEval
33
131
Pass @
SciCode (subproblems)
51
287
Pass
G PROMPTS
G.1 ARCHITECT'S PROMPT
We used the following prompt for the Architect. The decision to define the architecture representation with mathematical notation deliberate. We observed through preliminary experiments that providing a formal syntax, as opposed to a natural language description, makes the instructions for concepts like parallelism and aggregation less ambiguous for the LLM. This leads to more consistent and structurally valid outputs from the Architect.
Architect Prompt
You are a multi-agent architect fa mapping a query and context to an agent architecture: Sa (q, C) + G.
Goal: propose or adjust a layered operator architecture that is robust, performs well now, and is generalizable future queries.
Follow these steps:
Decompose the query, pick an initial architectural pattern, and justify briefly.
Think of other alternative architectural patterns, consider all 3 options, and select the best one.
If creating specialists, provide concise identity and bullet directives (no steps or formulas you are encouraged to use different reasoning patterns and strategies like adversarial prompting, quality-diversity prompting, step-back prompting, multi- choice elimination, etc.)
Use the recent sliding window experiences as guidance for your decisions.
21

---

Published as a conference paper at ICLR 2026
Architecture representation: A Li...... Lk] where layer L, [o,,1 ......... om., lists operators executed in parallel. Execution is layerwise: let œ1 = a and for i = 1 K, compute layer output (x;).. If |[-1| > 1, include an Ensemble aggregator 9i combine parallel outputs: h, (244 = g:(1 o(x;) o E -; }). If L = 1, then (x,), = o(x; for the unique E 15 The input to the next layer Ti+1 (x,)).
Query: [...]
Context for your decision:
Operator pool: [...]
Current architecture: [...]
Allowed actions: [...]
Recent sliding window experiences: [...]
XML formatting guide: [...]
where the allowed actions conditional the specific phase the system is in. During the specialist discovery phase, the full operator space is used:
CREATE OPERATO:: Defines a new specialist operator O Ospec and adds it to O
COMBINE_OPERATONN Merges two specialist operators into a single, more general spe- cialist.
FRUNE_OPERATORS Removes a specialist operator from O
Commits to a final graph G, and terminates the reasoning
loop.
We then restrict the allowed actions to only P.NALITE_AARCHTTROO during the specialist cultivation and evaluation phases.
G.2
SPECIALIST SYNTHESIS PROMPTS
G.2.1
CREATION
Creation: Identity Synthesis Prompt
You will propose diversified identity variants for new specialist, operator_name which is based on base__teae tetttmmlle......
Specialist description: [...]
Each identity should be a detailed second-person identity including their professional role (i.e., 'a particle physicist', do not include names), their fields expertise (deep broad), and a non-domain-specifii reasoning heuristic that distinguishes them from other specialists.
Examples of reasoning heuristics:
Works backwards from contradictory answers to identify wrong assumptions or equations.
Never assumes anything not explicitly stated; always returns to first principles when confused.
Builds multiple competing hypotheses simultaneously and tests them against evidence.
Visualizes problems as interconnected networks of constraints and relationships.
Output the identity text starting with You are...
22

---

Published as a conference paper at ICLR 2026
Creation: Directive Synthesis Prompt
You will propose diversified directive variants for specialist, operator name, which is based on base_temp) ate.
Specialist description: [...]
Create bulleted list of methodological principles and reasoning approaches that this new specialist will follow. Do not provide specific formulas, step-by-step procedures, formatting instructions, or direct solutions. Focus on how the specialist should think and approach problems, not what specific steps to take.
Include strategic reasoning approaches like self-criticism, assumption questioning, hypothesis building, pattern recognition, systematic analysis, etc. It is very important that the directives should guide analytical thinking without restricting the specialist's s reasoning search space.
G.2.2 CROSSOVER
Crossover: Identity Synthesis Prompt
You will propose diversified identity variants for a combined specialist that synthesizes the expertise of two parent specialists. The specialist operator_ss segertto_naaee _ame, which is based on @ase_templace
Specialist description: [...]
Parent I's identity: [...]
Parent 2's identity: [...]
Each identity should be a detailed second- perdod identity including their professional role (i.e., a particle physicist'), their fields of expertise (deep broad), and a con-domain-speciiii reasoning heuristic. The combined identity should integrate the best aspects of both parent specialists while creating coherent, unified specialist persona.
Examples of reasoning heuristies:
Works backwards from contradictory answers to identify wrong assumptions or equations.
Never assumes anything not explicitly stated; always returns to first principles when confused.
Builds multiple competing hypotheses simultancously and tests them against evidence.
Visualizes problems as interconnected networks of constraints and relationships.
Output the identity text starting with 'You are...
Crossover: Directive Synthesis Prompt
You will propose diversified directive variants for combined specialist that synthesizes the expertise of two parent specialists. The specialist operator_ ___mm,, which is based on Dase_template.
Specialist description: [...]
Parent I's identity: [...]
Parent 2's identity: [...]
Create bulleted list of methodological principles and reasoning approaches that this new specialist will follow. Do not provide specific formulas, step-by-step procedures, formatting instructions, direct solutions. Focus on how the specialist should think and approach problems, not what specific steps to take. The combined directives should integrate the best aspects of both parent specialists' directives, including any existing reasoning approaches.
Include strategic reasoning approaches like self-criticism, assumption questioning, hypothesis building, pattern recognition, systematic analysis, etc. It is very important that the directives should guide analytical thinking without restricting the specialist's reasoning search space.
23

---

Published as a conference paper at ICLR 2026
G.3 JUDGE PROMPTS
Judge Prompt: Evaluating Identities
You judging specialist identities for: operator name, which is based base_template
Specialist description: [...]
Identity candidates:
[...]
Pick the best identity based on the following criteria:
-pe-do reasoning heuristics for rich reasoning "gene" pool (quality- diversity, step-back analysis, "ssumption-challngiiin etc.)
2. Avoids making assumptions not explicitly stated in the problem
The resulting specialist is a T-shaped specialist. In other words, it has both deep specialization and broader domain coverage. Avoid hyperspecific specialists that are too narrow in their domain coverage.
Combines domain expertise with general problem-solving approaches
Judge Prompt: Evaluating Directives
You judging specialist directives for: operator_nanee which based base_template.
Specialist description: [...]
Directive candidates:
[...]
Pick the best directives based the following criteria:
Focus on how to think, not what specific steps to take. Mimic domain-specific human
experts to guide analytical thinking without constraining solution paths
Prefer methodological principles over rigid instructions. Avoid specific formulas, proce- dures, or direct solutions
3. Has specific methodology for handling contradictions or confusion
24


Reviews:
[{"review_id": "AZZLIznOGU", "reviewer": "Reviewer_Yi79", "summary": "The paper introduces ASPEC, a framework designed to bridge the functional gap in contemporary agent system design between static, task-specific workflows and dynamic, per-query optimizers. It proposes a system of specialized agents capable of accumulating persistent, role-specific expertise over time. The methodology involves an automated lifecycle consisting of evolutionary discovery and experiential cultivation, governed by a cost-aware \"retain-then-escalate\" control policy. The authors demonstrate measurable performance improvements, notably achieving an accuracy of 62.8% on the GPQA benchmark, and establish competitive cost efficiency relative to approaches that rely on constant architectural resampling.", "questions": "Could you elaborate on how the framework determines when a newly discovered specialist's niche is too narrow to be retained, thus avoiding excessive fragmentation?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The proposal to develop adaptive, stateful specialist agents represents a highly appealing and novel direction compared to other current multi-agent system approaches. Furthermore, the empirical results presented are compelling, and the supporting analysis of the system's components and efficiency is sound.", "weaknesses": "The system is inherently complex due to its hierarchical, two-tiered structure. It requires maintaining both a low-level Architect (a large generative LLM used for evolutionary search) and a high-level, trained Meta-Controller (a neural policy). The entire process involves managing an offline two-stage training loop (Discovery and Cultivation), which is more involved than implementing fixed-architecture or simple prompt-optimization methods.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "e7hDe9HCEg", "reviewer": "Reviewer_HnVD", "summary": "This paper proposes an automated method for generating stateful specifications to improve AI agent performance on complex tasks. By combining formal methods with machine learning, the system automatically extracts state transition rules and constraints from task execution traces. Experiments on various state management tasks show the approach effectively improves task success rates and code quality.", "questions": "1. When task definitions change, how efficiently can learned state specifications be updated? Does this require recollecting large amounts of data?\n2. For scenarios with concurrent operations, how do you model partial ordering of state transitions?\n3. How do you address state explosion? In very complex systems, possible state combinations grow exponentially\n4. Compared to manually designed specifications, where are automatically generated ones superior? Have you conducted such comparisons?\n5. How transferable is the method across domains? How much domain-specific tuning is required?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. **High degree of automation**: Automatically extracts specifications from execution traces with minimal human intervention, enhancing practicality\n2. **Methodological innovation**: Combines symbolic reasoning with statistical learning, leveraging advantages of both for complex state spaces\n3. **Comprehensive experiments**: Validation across different task types including file system operations, database management, and workflow orchestration\n4. **Good interpretability**: Generated state specifications are human-readable, facilitating debugging and understanding of agent behavior\n5. **Clear effectiveness**: Experiments show significant improvements in task success rates and code robustness", "weaknesses": "1. **Insufficient formal guarantees**: Despite using formal methods, lacks theoretical guarantees for correctness and completeness of generated specifications. Critical states or constraints may be missed\n2. **Questionable scalability**: How does computational complexity scale with state space size? The paper lacks analysis of large-scale scenarios\n3. **Data dependency**: Requires sufficiently diverse execution traces to learn complete specifications. Cold start and rare state handling is inadequate\n4. **Shallow comparisons**: Limited comparison with pure learning-based or pure formal methods, making it hard to assess true advantages of the hybrid approach\n5. **Practical deployment challenges**: How are state specifications continuously updated in dynamic environments? Maintenance issues aren't discussed", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "cuGn00BP9G", "reviewer": "Reviewer_1R45", "summary": "ASPEC framework builds evolving teams of specialized agents that can keep improving with experience. It designs specialized agent types through evolutionary search and enables them to improve through practice. Experiments show that this method leads to performance gains.", "questions": "How can other long-term memory methods be incorporated into your framework?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "ASPEC effectively bridges the gap between static task-level designs and per-query adaptive systems.\nAchieves top performance on multiple benchmarks—including GPQA and SciCode, surpassing prior frameworks such as AFlow, ADAS, and EvoAgent.\nEvaluated across five diverse benchmarks covering reasoning, scientific QA, and coding tasks, demonstrating robust generalization across domains.\nThe paper explicitly reports that ASPEC achieves higher accuracy than baselines at a fraction of the computational cost. This “Pareto-efficient” behavior is one of its key strengths.\n\nIncludes detailed ablation studies quantifying the contribution of each system component—such as specialist operators, the meta-controller, and the architect—to overall performance and efficiency.", "weaknesses": "The meta-agent evaluation process introduces multiple sources of randomness (including LLM output variance, error propagation across chained agents, sampling variability within the meta-agent, and stochastic evaluation outcomes). This leads to higher variability than typical single-LLM evaluations. In addition to averaging results over three runs, reporting variances would help assess and demonstrate the stability of the system’s performance.\n\nThe framework’s separation into two discrete stages (specialist discovery and specialist cultivation) appears conceptually convenient but somewhat artificial.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Jwl4MngyLP", "reviewer": "Reviewer_vhtg", "summary": "This work introduces ASpec, a framework that manages the full lifecycle of expert specialist agents by first autonomously discovering specialist archetypes via evolutionary search and then cultivating their expertise through experience. It also introduces \"retain-then-escalate\", a control policy that, instead of being either fully static or fully dynamic, defaults to retaining a stateful agent team across related queries to leverage expertise and minimize cost, only escalating to architectural resampling when needed. Results show that the proposed approach can lead to substantial performance improvements without sacrificing efficiency.", "questions": "- I am not very familiar with the related work in this area, so I am unsure whether the selected baselines are the most appropriate for comparison.", "limitations": "", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The proposed method is novel and well-motivated. It effectively addresses the limitation of prior work, where architectures lack long-term state because they are regenerated or resampled for every query.\n\n- The results in Table 1 show substantial performance improvements, which validate the effectiveness of the proposed method.", "weaknesses": "- I am confused about the transferability results presented in Figure 5 (right). I can't understand this figure and there is no accompanying explanation or analysis. I would like to understand how well the proposed method transfers to different tasks, and a more detailed discussion would be helpful.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)