AgentSquare: Automatic LLM Agent Search in Modular Design Space

## Abstract

Abstract Recent advancements in Large Language Models (LLMs) have led to a rapid growth of agentic systems capable of handling a wide range of complex tasks. However, current research largely relies on manual, task-specific design, limiting their adaptability to novel tasks. In this paper, we introduce a new research problem: Mo dularized L LM A gent S earch (MoLAS). We propose a modular design space that abstracts existing LLM agent designs into four fundamental modules with uniform IO interface: Planning , Reasoning , Tool Use , and Memory . Building on this design space, we present a novel LLM agent search framework called AgentSquare, which introduces two core mechanisms, i.e. , module evolution and recombination , to efficiently search for optimized LLM agents. To further accelerate the process, we design a performance predictor that uses in-context surrogate models to skip unpromising agent designs. Extensive experiments across six benchmarks, covering the diverse scenarios of web, embodied, tool use and game applications, show that AgentSquare substantially outperforms hand-crafted agents, achieving an average performance gain of 17.2% against best-known human designs. Moreover, AgentSquare can generate interpretable design insights, enabling a deeper understanding of agentic architecture and its impact on task performance. We believe that the modular design space and AgentSquare search framework offer a platform for fully exploiting the potential of prior successful designs and consolidate the collective efforts of research community.
Code repo is available at https://github.com/tsinghua-fib-lab/AgentSquare .

## 1 Introduction

The past few years have witnessed remarkable progress in the development of Large Language Models (LLMs) Achiam et al. (); Touvron et al. (), giving rise to the proliferation of numerous agentic systems Weng (); Shen et al. (). For example, “chain-of-thought” prompting has unlocked the general-purpose reasoning capabilities of LLMs Wei et al. (), and memory mechanisms have been proven effective in simulating human behavioiur Park et al. (). These emerging LLM agents have demonstrated astonishing abilities to transform a wide range of tasks, including solving mathematical problems Romera-Paredes et al. (), navigating the web Nakano et al. (), providing financial advice Ding et al. () and informing medical decisions Li et al. (). Therefore, the design of agentic systems plays a crucial role in harnessing the power of LLMs for various downstream applications.

However, current research predominantly relies on manually designed agentic systems tailored for specific tasks, which often depend heavily on expert insight and intensive human labor. Furthermore, these task-specific agent designs frequently struggle to adapt to novel tasks. A few recent works have explored using LLMs to rewrite and optimize the prompts of existing agents Fernando et al. (); Yang et al. (). However, these rewriting approaches are limited in their ability to recombine the strengths of agentic modules discovered by different researches and located in separate codebases. Another line of research focuses on optimizing the configuration of multi-agent systems Chen et al. (); Yuan et al. (); Li et al. (); ; Wang et al. (). These efforts are orthogonal to the optimization of single-agent systems, as they focus more on the role-playing and interaction patterns among multiple agents, rather than the design of agentic modules.

Figure: Figure 1: AgentSquare is a modular framework for designing and optimizing LLM agents. We first propose a modular design space of LLM agents and extract 4 types of standardized modules including planning, reasoning, tooluse, and memory. Based on this, we design a novel LLM agent search framework to automatically discover good-performing agents.
Refer to caption: x2.png

This paper addresses a novel research problem — Modularized LLM Agent Search (MoLAS). The goal is to automatically optimize LLM agent designs by leveraging the experience of published or evaluated modules. Therefore, the core of our work is a modular design space for LLM agents, comprising 4 categories of modules: *Planning*, *Reasoning*, *Tool Use*, and *Memory*. This design space is abstracted from a thorough literature review of existing agentic systems (details provided in Section 2). It is important to note that our goal is not to propose the most comprehensive, one-size-fits-all LLM agent design space, but rather to demonstrate that our modular design space enables researchers and intelligent search algorithms to fully exploit the potential of prior successful designs. Thus, our key consideration is providing a standardized IO interface for these modules, facilitating easy recombination of modules from different agentic systems and hence enabling efficient search for novel agents. Our design space is also highly extensible, allowing new agentic systems to be integrated as plug-in modules. Therefore, it provides a platform to consolidate the collective efforts of the research community on LLM agents. The overview of this work is illustrated in Figure 1.

Building on this modular design space, we propose a novel LLM agent search framework called AgentSquare. Specifically, AgentSquare optimizes LLM agents through the mechanisms of *module evolution* and *recombination*. The *module evolution* mechanism leverages an evolutionary meta-prompt to explore new modules through prompt-level optimization, which jointly models task descriptions, existing modules, and the performance of evaluated modules. Besides, the *module recombination* mechanism performs module-level optimization by leveraging the reasoning power of LLMs to strategically search for promising module combinations. To reduce the expensive evaluation costs of LLM agents, we further introduce a performance predictor that implements an in-context surrogate model for newly proposed LLM agents, enabling us to skip unpromising candidates and significantly accelerate the search process.

We conduct comprehensive evaluations on six widely adopted benchmarks, covering diverse use cases in web, embodied, tool use and game scenarios. Our experiments show AgentSqaure can discover novel LLM agents that outperform hand-crafted agents across all six benchmarks, scoring an average performance gain of 17.2% compared to the best known human designs. Besides, AgentSqaure also surpasses other search algorithms in terms of having a steeper optimization trajectory. More importantly, case studies reveal that AgentSquare can provide human interpretable design insights for newly discovered, good-performing agents.

The key contributions of this work are as follows:

- •
We propose a novel modular design space for LLM agents, enabling researchers to easily build on previous successful designs and accumulate new discoveries as a community.
- •
We design the AgentSquare framework that efficiently searches for novel and good-performing LLM agents via the novel mechanism of *module evolution*, *module recombination*, and *performance predictor*.
- •
Experiments across six diverse tasks show that our method discovers novel LLM agents that outperform all known human designs. Besides, AgentSqaure can generate human interpretable design insights for these novel agents.

## 2 A Modular Design Space of LLM Agents

Figure: Figure 2: Illustration of the modular agent design space and agentic workflow (left) and the standardized IO interface of four types of modules (right).
Refer to caption: x3.png

A modular design space for LLM agents facilitates the reuse of prior successful designs and supports the exploration of new architectures.
At the core of such modularization is the standardization of input-output interfaces, which ensures both extensibility and seamless integration with existing designs.
Many experts in the field have proposed building LLM agentic systems with key modular components from engineering (Weng, ) and cognitive perspectives (Sumers et al., ; Shao et al., ).
However, these proposals remain largely conceptual, lacking implementable solutions to unify existing LLM agents.
Besides, current LLM workflow program frameworks (*e.g.*, LangChain and AutoGPT) only provide operation-level components, which cannot support module-level search that best exploits the potential of prior successful designs.

To address these problems, we perform a comprehensive literature review of publications from NeurIPS, ICML, and ICLR over the past three years.
The review focuses on papers with the keywords “LLM”, “Agent”, or “Large Language Model” in their titles while excluding works related to multi-agent systems or agents that require additional training.
Note that our aim is not to propose the most comprehensive, one-for-all LLM agent design space, but to offer a standardized framework that enables the recombination of existing agents and facilitates the discovery of new ones.
As a result, we sort out 16 popular LLM agents and abstract a modular design space with 1050 possible combinations, which can be easily extended when new modules are discovered.
Below, we describe the agentic workflow and the function of each modules in our design space.

Workflow overview.
The proposed agent workflow operates through an iterative process with the interconnection of the above four modules, as shown in Figure 2.
Upon receiving a task $d$, the agent starts with the *planning* module, decomposing it into $n$ sub-tasks$\{s_{1},s_{2},\dots,s_{n}\}$.
Next, these sub-tasks are passed to the *reasoning* module sequentially.
Taking the sub-task $s_{i}$ description as input, the *reasoning* module explores to prompt LLMs to give the result.
When reasoning encounters limitations in internal knowledge of LLMs, the *tool use* module is activated to select an appropriate tool from the pre-defined tool pool $\tau$, supporting problem-solving.
Besides, the reasoning process also accesses the *memory* module which reads and writes necessary observations and experiences from a memory database $mem$ to help reasoning.
The reasoning result of each sub-task will be transformed into actions, guiding the agent to interact with the external environment.
After all sub-tasks are finished or the reasoning process gets stacked, the agent will activate the *planning* module to adjust the plan with the received feedback.
The agent conducts such a trial-and-error loop until the task $d$ is completed or the set maximum trial number is reached.

Planning.
The planning module is responsible for decomposing the targeted task into smaller sub-tasks.
Given a task description $d$ and optional feedback information $f$, the planning module $P$ strategically decomposes the targeted task into a sub-task sequence $\{s_{1},s_{2},\dots,s_{n}\}=P(d,f)$.
Such decomposition is critical for handling very complex tasks with long-term characteristics, especially for agents in open-world environments such as MineCraft (Wang et al., ; ).

Reasoning.
LLMs have exhibited remarkable reasoning abilities under advanced prompting approaches such as CoT (Wei et al., ), ToT (Yao et al., ), and SoT (Shang et al., ), shaping the foundation of the intelligence of LLM agents.
The reasoning module $R$ is invoked to solve the sub-tasks sequentially after planning, which takes each sub-task $s_{i}$ and optional feedback information $f_{i}$ as input and outputs a solution $r_{i}=R(s_{i},f_{i})$.

Tool use.
The ability of using external tools (Shen et al., ; Schick et al., ) overcomes the limitations of the LLM’s internal knowledge during the reasoning process.
Formally, given certain problem $p_{ij}$ derived from the reasoning process of sub-task $s_{i}$ and a pre-defined tool pool $\tau$, the tooluse module $T$ selects the best-matched tool $t_{ij}$ to address the problem, denoted as $t_{ij}=T(p_{ij},\tau)$, where $t_{ij}\in\tau$.

Memory.
Memory plays a critical role by storing past thoughts, actions, and observations of agents (Park et al., ; Shinn et al., ).
During the reasoning process, these internal logs are dynamically written to and retrieved from the memory database $mem$, controlled by the memory module $M$.
The writing process can be expressed as $mem=M_{write}(o,mem)$, where $o$ denotes the current observations. The retrieval process is $m=M_{retrieve}(o,mem)$, where $m$ denotes the retrieved knowledge relevant to the current situation.

## 3 AgentSquare Framework

Figure: Figure 3: Overview of AgentSquare search framework. AgentSquare optimizes LLM agents through the mechanisms of module evolution and recombination. We further introduce a performance predictor that implements an in- context surrogate model for efficient evaluation of novel agents.
Refer to caption: x4.png

### 3.1 Problem formulation of MoLAS

In the proposed modular design space, an LLM agent $A$ can be instantiated with the combination of a planning module $P$, a reasoning module $R$, a tooluse module $T$ and a memory module $M$, denoted as $A=(P,R,T,M)$.
Given the task description $d$ and the set of all possible modules with standardized IO interface $\{\mathbb{P},\mathbb{R},\mathbb{T},\mathbb{M}\}$.
We formulate an optimization problem for searching LLM agent architectures within the modular design space.
The objective is to identify the optimal module combination in a solution space defined by a Cartesian product of four design dimensions to maximize agent performance.
Let the performance evaluation function of the task be ${Eval}_{d}(\cdot)$, the optimization problem of MoLAS is defined as follows:

$$ $\operatorname*{arg\,max}\limits_{P\in\mathbb{P},R\in\mathbb{R},T\in\mathbb{T}, M\in\mathbb{M}}{Eval}_{d}(P,R,T,M).$ (1) $$

### 3.2 AgentSquare search algorithm

Solving the optimization problem of MoLAS features three key challenges: (1) The search space, defined as the Cartesian product of four orthogonal modules, is vast and hard to explore;
(2) the module sets encompass any code with standard IO interfaces, making the module selection an open-ended problem; (3) the high costs of agent evaluation during the search process constrain the overall search scale.
To tackle these issues, we introduce AgentSquare, an automatic search framework to optimize LLM agents within the modular design space.
Facing the vast search space of MoLAS, we propose *module recombination* operation utilizing LLMs to strategically reason to identify more promising module combinations.
Such operation broadens the coverage of child samples, overcoming the limitations of prompt rewrite methods that explore only a restricted space.
However, only searching in the existing module combinations also narrows the search space, thus we propose *module evolution* operation which employs an evolutionary meta-prompt to search new modules through code-level optimization.
This operation, combined with module recombination, enables the search of any module combination in the open-ended solution space.
Finally, to mitigate the high costs of frequent evaluations of searched agents, we design a *performance predictor* as an in-context surrogate model for evaluating searched agents, significantly accelerating the search process and reducing real-valued costs.

The overall framework of AgentSquare is illustrated in Figure 3 and the algorithm is presented in Algorithm 1.
Next, we detail the key components of the AgentSquare search process.

Figure: Algorithm 1 Algorithm of AgentSquare

### 3.3 Initialization

Insights from existing AutoML studies indicate that a well-chosen initialization enhances warm-up and improves search efficiency by avoiding unpromising populations (So et al., ; Yuan et al., ).
AgentSquare starts by initializing a global experience pool $\mathbb{E}=\{(P,R,T,M,v)|P_{0}\in\mathbb{P},R_{0}\in\mathbb{R},T_{0}\in\mathbb
{T},M_{0}\in\mathbb{M}\}$ to seed agents that are well-designed (as mentioned in Section 2) along with their real-valued performance $v$.
The module pools $\{\mathbb{P},\mathbb{R},\mathbb{T},\mathbb{M}\}$ are set to the standardized modules extracted from these seed agents.

### 3.4 Module Recombination

Given the vast solution space of MoLAS, relying solely on prompt rewriting leads to a limited exploration confined to the neighbor of the initial state.
To expand the exploration space, we propose leveraging LLMs as a *self-adaptive proposer*, which iteratively reason to identify promising module combinations with accumulated experience beyond the original agent configuration.
Denote the initial agent of the recombination phase as $A_{r}^{0}=(P_{0},R_{0},T_{0},M_{0})$, where $P_{0}\in\mathbb{P},R_{0}\in\mathbb{R},T_{0}\in\mathbb{T},M_{0}\in\mathbb{M}$.
The module combination proposer LLM $\pi_{\theta}$ incorporates targeted task description $d$, existing module pools $\{\mathbb{P},\mathbb{R},\mathbb{T},\mathbb{M}\}$ and the performance experience of searched module combinations $\mathbb{E}$ to propose promising new agents $A_{r}$:

$$ $A_{r}=\pi_{\theta}((P_{0},R_{0},T_{0},M_{0}),d,N,\mathbb{P},\mathbb{R},\mathbb {T},\mathbb{M},\mathbb{E}).$ (2) $$

The meta-prompt used for module recombination is presented in Figure A.15.
Based on the initial agent configuration $A_{r}^{0}$, the LLM proposes $N$ offspring $\{A_{r}^{1},A_{r}^{2},...,A_{r}^{N}\}$ by replacing certain modules of $A_{r}^{0}$ with alternatives from the module pool.
For instance, a possible solution could be $(P_{0},R^{{}^{\prime}},T_{0},M_{0})$, where $R^{{}^{\prime}}\in\mathbb{R}$ is a different reasoning module selected from the module pool.
Then, the created $N$ new agents are evaluated with a performance predictor $\pi_{p}$ (detail in Seciton 3.6) and the best one goes to the next episode as initialization.

### 3.5 Module Evolution

As mentioned above, the solution space for each module type is open-ended, allowing any code with a standardized I/O interface.
Consequently, searching only with module recombination narrows the solution space and limits the upper bound of agent performance.
To address this problem, we design a *module evolution* operation with an evolutionary meta-prompt to search for new modules through program-level optimization.
Specifically, we introduce a module-programming LLM $\pi_{\xi}$ to evolve new modules by jointly modeling task descriptions, existing modules, and the performance of previously evaluated modules.
Meta-prompt of module evolution is presented in Figure A.17.
Leveraging LLMs as module programmer has several appealing advantages.
Modern LLMs are typically trained on extensive code datasets, enabling advanced code comprehension and generation capabilities.
Additionally, integrating existing module codes as in-context examples, along with past experiences, can elicit the reflective reasoning abilities of LLMs to identify promising optimization directions.
Denote the initial agent in the module evolution stage as $A_{e}^{0}=(P^{{}^{\prime}}_{0},R^{{}^{\prime}}_{0},T^{{}^{\prime}}_{0},M^{{}^{
\prime}}_{0})$, the module programmer LLM produces a population of child agents by evolving current modules of $A_{e}^{0}$.
Formally the module evolution operation is denoted as follows:

$$ $A_{e}=\pi_{\xi}((P^{{}^{\prime}}_{0},R^{{}^{\prime}}_{0},T^{{}^{\prime}}_{0},M ^{{}^{\prime}}_{0}),d,N,\mathbb{P},\mathbb{R},\mathbb{T},\mathbb{M},\mathbb{E}).$ (3) $$

The created new modules are appended to the standardized module pools $\{\mathbb{P},\mathbb{R},\mathbb{T},\mathbb{M}\}$ and each module is used to individually mutate the initial agent, resulting in $N$ child agents $\{A_{e}^{1},A_{e}^{2},...,A_{e}^{N}\}$.
For example, $(P^{*},R_{0},T_{0},M_{0})$ represents a solution where the planning module is mutated into a new variant $P^{*}$.
These child agents are then evaluated and updated to the historical experience pool $\mathbb{E}$.
The best-performing one is selected as the initial agent for the subsequent recombination phase.

### 3.6 Performance Predictor

The last challenge in automatic agent search is the high API cost incurred during the evaluation of each candidate agent.
Many agent tasks require multiple steps and involve substantial input and output tokens, leading to prohibitive evaluation costs.
For instance, evaluating a simple CoT agent based on GPT-4o in ALFWorld (Shridhar et al., ) requires around $60, making the agent search economically unsustainable at scale.
To tackle this issue, we propose incorporating an additional LLM $\pi_{p}$ as a performance predictor to serve as an in-context surrogate model for novel agent evaluation, enabling the exclusion of unpromising candidates and significantly accelerating the search process.
Compared to real environment evaluation, such an in-context surrogate model requires significantly fewer tokens, making it more cost-efficient and support larger-scale search.
Similar approaches have been effectively applied in neural architecture search (NAS), where LLMs are leveraged to evaluate the performance of generated network architectures  (Jawahar et al., ; Chen et al., ).
The meta prompt of the performance predictor is shown in Figure A.16.
Here, given a newly searched agent $A^{\prime}$, the performance predictor $\pi_{p}$ thoroughly consider task descriptions $d$, module profiles and in-context performance examples of previously tested agents $\mathbb{E}$ to score novel agents.

$$ $v^{\prime}=\pi_{p}(A^{\prime},d,\mathbb{P},\mathbb{R},\mathbb{T},\mathbb{M}, \mathbb{E}),$ (4) $$

where $v^{\prime}$ is the predicted performance of the evaluated agent.
Empirical results demonstrate that the predicted performance of agents closely matches their actual performance, verifying the effectiveness of the proposed performance predictor, which is detailed in Section 4.3.

## 4 Experiments

### 4.1 Experimental Setup

Task setup.
We conduct experiments on six representative tasks covering four domains: embodied, game, web and tool applications, which are widely adopted by existing LLM agent benchmarks (Ma et al., ; Xi et al., ), more details are presented in Appendix A.1.

Baselines.
We compare AgentSquare with three types of baselines including hand-crafted agents, module-level search, and prompt-level search methods. More details are presented in Appendix A.1.

AgentSquare setup.
We implement AgentSquare and conduct experiments using both GPT-3.5-turbo-0125 and GPT-4o (Achiam et al., ). To ensure a fair comparison, we use the same number of few-shot examples across all methods. The initial agent is set as a random module combination, and the search process terminates after 5 consecutive iterations without performance improvement.

**Table 1: Performance comparison of searched agents from AgentSquare and (1) existing human-designed agents (2) module search baselines (3) prompt search baselines based on GPT-4o on six tasks across different domains.**
|  |  | Web | Embodied | Tool | Game |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- |
| Baseline Type | Method | Webshop | ALFWorld | SciWorld | M3Tool | TravelPlanner | PDDL |
| Hand-crafted Agents | CoT | 0.485 | 0.405 | 0.697 | 0.448 | 0.487 | 0.542 |
| Cot-SC | 0.512 | 0.426 | 0.656 | 0.461 | 0.413 | 0.495 |  |
| Self-refine | 0.461 | 0.567 | 0.654 | 0.442 | 0.000 | 0.514 |  |
| ToT | 0.501 | 0.437 | 0.741 | 0.453 | 0.380 | 0.476 |  |
| Step Back | 0.468 | 0.279 | 0.220 | 0.434 | 0.000 | 0.486 |  |
| TP | 0.398 | 0.404 | 0.576 | 0.387 | 0.430 | 0.518 |  |
| HuggingGPT | 0.519 | 0.481 | 0.680 | 0.354 | 0.510 | 0.584 |  |
| Voyager | 0.366 | 0.425 | 0.776 | 0.247 | 0.523 | 0.412 |  |
| Generative Agents | 0.499 | 0.477 | 0.663 | 0.402 | 0.480 | 0.553 |  |
| DEPS | 0.481 | 0.459 | 0.740 | 0.278 | 0.540 | 0.591 |  |
| OPENAGI | 0.506 | 0.510 | 0.718 | 0.322 | 0.533 | 0.616 |  |
| Dilu | 0.451 | 0.433 | 0.682 | 0.475 | 0.360 | 0.463 |  |
| Module Search | Random | 0.533 | 0.620 | 0.704 | 0.438 | 0.563 | 0.660 |
| Bayesian | 0.549 | 0.634 | 0.749 | 0.502 | 0.537 | 0.650 |  |
| Prompt Search | OPRO | 0.505 | 0.380 | 0.569 | 0.309 | 0.523 | 0.589 |
|  | AgentSquare | 0.607 | 0.695 | 0.781 | 0.524 | 0.583 | 0.669 |

### 4.2 Experimental Results

Main results.
We conduct extensive experiments to compare our method against three types of baselines on six tasks and present results based on GPT-4o in Table 1 and results on GPT-3.5 in Table A.3.
Additionally, we evaluate the agents’ API costs and provide a performance-cost comparison in Figure A.7 to Figure A.12.
From these results, we have the following observations:

- •
AgentSquare can effectively discover better agents compared with human-designed agents.
On the six representative agent tasks, the best agent searched by AgentSquare consistently outperforms human-designed agents in terms of performance.
Specifically, as shown in Table 1 and Table A.3, compared with the best human-designed agent, AgentSquare achieves an average 14.1% performance improvement on Webshop, 26.1% improvement on ALFWorld, 20.5% improvement on SciWorld, 30.6% improvement on M3Tool, 6.0% improvement on Travelplanner, 6.0% improvement on PDDL.
Simultaneously, the best agent from AgentSquare is commonly cost-efficient, which strikes the best performance-cost trade-off among all compared agents as seen in Figure A.7 -Figure A.12.
- •
AgentSquare provides a more efficient searching approach for LLM agent optimization.
To further demonstrate the effectiveness of the search pipeline of AgentSquare, we also compare with two types of agent searching methods including module-level and prompt-level search.
Compared with the best agent crafted from these searching methods, AgentSquare achieves an average 8.4% performance improvement on Webshop, 8.1% improvement on ALFWorld, 11.0% improvement on SciWorld, 12.8% improvement on M3Tool, 2.5% improvement on Travelplanner, 1.4% improvement on PDDL.

Search trajectory in AgentSquare.
We present the search trajectory under 15 iterations using AgentSquare and other searching methods on ALFWorld and Webhop tasks in Figure 4.
Results on other tasks are presented in Figure A.13 and Figure A.14.
AgentSquare demonstrates a steady convergence trajectory, where more advanced agents are continually emerging during search.
In contrast, module-level searching methods including random and Bayesian search lack a clear and insightful search direction.
Prompt-level search methods such as OPRO are constrained by a limited modification space, leading to minimal performance improvements.
As a result, they all encounter performance bottlenecks during the search process, resulting in sub-optimal agent architectures.
Besides, we find that simple module-level search methods such as random recombination greatly outperforms prompt-level search, indicating the importance of searching in the modular design space.

Figure: Figure 4: AgentSquare search trajectory on Alfworld and Webshop.
Refer to caption: x5.png

**Table 2: Ablation study of AgentSquare on GPT-4o on six tasks across different domains.**
| Method | Webshop | ALFWorld | SciWorld | M3Tool | TravelPlanner | PDDL |
| --- | --- | --- | --- | --- | --- | --- |
| AgentSquare (full) | 0.607 | 0.695 | 0.781 | 0.524 | 0.583 | 0.669 |
| w/o module evolution | 0.564 | 0.649 | 0.736 | 0.502 | 0.577 | 0.614 |
| w/o module recombination | 0.560 | 0.616 | 0.710 | 0.481 | 0.280 | 0.669 |

### 4.3 Ablation Study of AgentSquare

Effectiveness of module evolution and recombination.
There are two key operations in the searching framework of AgentSquare: module evolution which creates new modules and module recombination which strategically recombines existing ones.
To verify the effectiveness of each design, we tested three variants of AgentSquare: the full model, a version without module evolution, and a version without module recombination.
The results based on GPT-4o and GPT-3.5 are presented in Table 2 and Table A.5, respectively.
It can be seen that dropping each design results in a noticeable performance decline and the module recombination has a larger impact.
Module recombination significantly expands the search space, reducing the risk of falling into a local optima.
Meanwhile, module evolution facilitates the discovery of more advanced modules tailored to specific tasks.
These two operations collaborate well ensuring the effectiveness of the search process in AgentSquare.

Figure: Figure 5: Validation of the effectiveness of the performance predictor (correlation between the actual and predicted performance) on each task.
Refer to caption: x6.png

Effectiveness of performance predictor.
In this part, we verify the effectiveness of this design empirically.
Figure 5 illustrates the predicted performance of given agents versus their actual tested performance based on both GPT-3.5 and GPT-4o on all six tasks.
It can be found that the predicted performance closely aligns with the actual performance, demonstrating the effectiveness of the introduced performance predictor.
For instance, the evaluation cost of the predictor is only about 0.025% of the cost of a full evaluation based on GPT-4o in ALFWorld, demonstrating its remarkable cost-efficiency.

### 4.4 Discovered Best Agents from AgentSquare

In this section, we provide some illustrations of the searched best agents, especially some discovered promising modules.
Table A.4 summarizes the searched best agent from AgentSquare and the best hand-crafted agents on all tasks.
We can observe that AgentSquare can adaptively identify promising agents with both previously existing and newly programmed modules tailored to the given task.
For instance, the discovered best agent for ALFWorld combines an existing well-designed memory module from Generative Agents with newly created planning (named TD) and reasoning modules (named SF-ToT).
By comparison, the best hand-crafted agent Self-refine focuses only on reasoning module design while overlooking other functional modules, leading to suboptimal performance.
Moreover, we illustrate two new modules and the human interpretable design insights discovered on ALFWorld in Figure 6.
More illustrations are listed in the Figure A.18 to Figure A.23.

Figure: Figure 6: New module discovered through AgentSquare search on ALFWorld.
Refer to caption: x7.png

## 5 Related Work

### 5.1 LLM-based Autonomous Agents

LLM-based autonomous agents are an advanced AI system using a core LLM to manage external functional modules and interact with environments.
In the early stage, LLM-based agents possess very simple architectures, which just directly query LLMs to conduct decision-making and take action (Huang et al., ; Ahn et al., ).
More recently, advancements have equipped agents with some other LLM-centric functional modules including planning (Hao et al., ; Huang et al., ; Zeng et al., ), reasoning (Wei et al., ; Yao et al., ; Shang et al., ), using tools (Shen et al., ; Schick et al., ), and monitoring memory (Wang et al., ; Park et al., ), greatly enhancing the capabilities of LLM agents.
Along with the improvement of the single agent, there’s another line of work trying to build more advanced multi-agent systems by strategically organizing individual agents for both simulation (Li et al., ; Chen et al., ) and targeted task solving (Qian et al., ; Xiao et al., ; Chen et al., ; Li et al., ).
The emergence of more and more sophisticated agent produces remarkable performance improvement, however, their architectures and codebases differ greatly with each other.
The lack of a unified design space and consistent terminologies across individual works makes it hard to compare different agents, understand their evolution routes, and guide new agent design directions.

### 5.2 Automatic Design of LLM-based Agents

LLM-based agent system, as the most advanced AI system, has not yet formed a unified design space and an automatic design approach.
Existing efforts on automatic building of LLM agents focus on three aspects: engineering-oriented open resources, conceptual framework of LLM agent design, and automatic design of partial components of LLM agents.
Firstly, engineering-oriented open resources like LangChain(^*^**https://github.com/langchain-ai/langchain) and BabyAGI(^†^††https://github.com/yoheinakajima/babyagi) has provided convenient ways to build an LLM-centric agentic system, however, they still need human participation to organize different modules and can’t support the optimization of the designed agent.
Secondly, there’s has been some conceptual framework trying to provide a unified design principle of LLM agents, such as CoALA (Sumers et al., ).
It points out four basic modules of language agents from the perspective of cognitive science, including long-term memory, external grounding, internal actions and decision-making.
However, it’s still a vision of how LLM agents should be in the future, without providing a practical design framework.
Thirdly, there have been some separate works to automate parts of LLM agent systems, such as prompt optimization (Do et al., ; Ma et al., ) and workflow search (Li et al., ; Hu et al., ).
Nevertheless, these works over-simplify the structure of agentic systems to pure prompts or codes, unable to provide a clear design space of LLM agents.
Besides, another common shortcoming of existing works is that all of them are proposed without much consideration of existing designed agent architectures, thus hard to be representative and universal.
Differently, our proposed modularized LLM agent design space can well cover existing efforts of LLM agent design, and provide a practical approach for automatic agent architecture optimization.

## 6 Conclusion

In this work, we introduce a novel modular design space for LLM agents, allowing researchers to build upon successful prior designs and collectively accumulate new insights.
Based on this, we propose a novel research problem, Modularized LLM Agent Search (MoLAS), which aims to automatically optimize LLM agent designs by leveraging the knowledge gained from previously published or evaluated modules.
To address the challenge of vast search spaces, we present AgentSquare, an automatic search framework to optimize LLM agents through module evolution and recombination, enabling the exploration of the entire solution space.
We further introduce a performance predictor as an in-context surrogate model for evaluating novel LLM agents to accelerate the search process.
AgentSquare consistently discovers novel agents that surpass all existing hand-crafted ones, while also offering interpretable design insights that deepen our understanding of these new agents.
Overall, our work offers a transition from studying individual LLM agent designs to studying LLM agents within a modular design space, further consolidating the collective efforts of research community.

## Appendix A Appendix

### A.1 Experimental Setup

Task setup.
We evaluate AgentSquare and compared methods on six representative tasks covering four key domains which are widely adopted by existing LLM agent benchmarks (Ma et al., ; Xi et al., ):

- •
Embodied: ALFWorld (Shridhar et al., ) with text-based household tasks where agents navigate and interact with objects using text commands, ScienceWorld (Wang et al., ) with interactive science tasks requiring agents to navigate rooms and perform experiments, testing scientific commonsense;
- •
Game: PDDL (Ma et al., ) including many strategic games where agents use PDDL expressions to complete tasks;
- •
Web: WebShop (Yao et al., ) focusing on online shopping tasks where agents browse and purchase products based on user instructions;
- •
Tool: TravelPlanner (Xie et al., ) with many travel planning tasks where agents use tools and data to create detailed plans, (6)M3ToolEval (Wang et al., ) including complex tasks requiring multi-turn interactions with multiple tools.

Baselines.
We compare AgentSquare with three types of baselines:

- •
Hand-crafted agents. We compare with 12 hand-crafted agents including CoT (Wei et al., ), CoT-SC (Wang et al., ), Self-refine (Madaan et al., ), ToT (Yao et al., ), Step back (Zheng et al., ), Thought propagation (Yu et al., ), HuggingGPT (Shen et al., ), Voyager (Wang et al., ), Generative Agents (Park et al., ), DEPS (Wang et al., ), OPENAGI (Ge et al., )and Dilu (Wen et al., ).
- •
Module search methods. We compare with two module-level agent optimization methods including the random combination of existing modules and Bayesian (Zhou et al., ) module combination optimization inspired by Bayesian optimization in NAS (White et al., ).
- •
Prompt search methods.
We select OPRO (Yang et al., ) as a representative prompt-level optimization approach, which leverages LLMs as optimizers by generating and refining instructions through iterative prompts.

AgentSquare setup.
We implement AgentSquare and conduct experiments using both GPT-3.5-turbo-0125 and GPT-4o (Achiam et al., ). To ensure a fair comparison, we use the same number of few-shot examples across all methods. The initial agent is set as a random module combination, and the search process terminates after 5 consecutive iterations without performance improvement.

**Table A.3: Performance comparison of searched agents from AgentSquare and (1) existing human-designed agents (2) module search baselines (3) prompt search baselines based on GPT-3.5 on six tasks across different domains.**
|  |  | Web | Embodied | Tool | Game |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- |
| Method Type | Method | Webshop | ALFWorld | SciWorld | M3Tool | TravelPlanner | PDDL |
| Hand-crafted Agents | CoT | 0.504 | 0.369 | 0.142 | 0.172 | 0.080 | 0.151 |
| CoT-SC | 0.527 | 0.381 | 0.105 | 0.181 | 0.167 | 0.178 |  |
| Self-refine | 0.439 | 0.388 | 0.222 | 0.098 | 0.000 | 0.109 |  |
| ToT | 0.510 | 0.381 | 0.143 | 0.189 | 0.163 | 0.147 |  |
| Step Back | 0.478 | 0.375 | 0.027 | 0.128 | 0.120 | 0.137 |  |
| TP | 0.429 | 0.299 | 0.168 | 0.139 | 0.063 | 0.122 |  |
| HuggingGPT | 0.518 | 0.502 | 0.270 | 0.012 | 0.470 | 0.212 |  |
| Voyager | 0.427 | 0.369 | 0.301 | 0.008 | 0.480 | 0.149 |  |
| Generative Agents | 0.539 | 0.388 | 0.153 | 0.144 | 0.060 | 0.123 |  |
| DEPS | 0.555 | 0.474 | 0.308 | 0.017 | 0.500 | 0.186 |  |
| OPENAGI | 0.507 | 0.448 | 0.257 | 0.008 | 0.430 | 0.178 |  |
| Dilu | 0.418 | 0.291 | 0.000 | 0.131 | 0.137 | 0.054 |  |
| Module Search | Random | 0.562 | 0.569 | 0.367 | 0.235 | 0.473 | 0.216 |
| Bayesian | 0.581 | 0.611 | 0.269 | 0.217 | 0.497 | 0.210 |  |
| Prompt Search | OPRO | 0.507 | 0.376 | 0.032 | 0.193 | 0.513 | 0.179 |
|  | AgentSquare | 0.617 | 0.651 | 0.432 | 0.285 | 0.520 | 0.219 |

**Table A.4: Comparison between the searched best agent from AgentSquare and the best human-designed agent on all tasks.**
| Task | Planning | Reasoning | Tooluse | Memory | Best Hand-crafted Agents |
| --- | --- | --- | --- | --- | --- |
| Webshop | IO | HTSS | / | Dilu | HuggingGPT |
| ALFWorld | TD | SF-ToT | / | Generative Agents | Self-refine |
| SciWorld | Voyager | CoT | / | Hier | Voyager |
| M3Tool | / | CoT-SC | ToolBF | / | Toolbench |
| TravelPlanner | DEPS | CoT | TH | / | DEPS |
| PDDL | IR | CASRC | / | Generative Agents | OPENAGI |

**Table A.5: Ablation study of AgentSquare on GPT-3.5 on six tasks across different domains.**
| Method | Webshop | ALFWorld | SciWorld | M3Tool | TravelPlanner | PDDL |
| --- | --- | --- | --- | --- | --- | --- |
| AgentSquare (full) | 0.617 | 0.651 | 0.432 | 0.285 | 0.520 | 0.219 |
| w/o module evolution | 0.595 | 0.623 | 0.288 | 0.236 | 0.483 | 0.202 |
| w/o module recombination | 0.578 | 0.546 | 0.310 | 0.258 | 0.267 | 0.173 |

Figure: Figure A.7: Performance versus API costs visualization on ALFWorld task.
Refer to caption: x8.png

Figure: Figure A.8: Performance versus API costs visualization on Webshop.
Refer to caption: x9.png

Figure: Figure A.9: Performance versus API costs visualization on Sciworld.
Refer to caption: x10.png

Figure: Figure A.10: Performance versus API costs visualization on M3tool.
Refer to caption: x11.png

Figure: Figure A.11: Performance versus API costs visualization on Travelplanner.
Refer to caption: x12.png

Figure: Figure A.12: Performance versus API costs visualization on PDDL.
Refer to caption: x13.png

Figure: Figure A.13: AgentSquare search trajectory on M3tool and PDDL (more hand-crafted agents, specific module combinations when surpassing best hand-crafted and the final evolved agent, other search baselines).
Refer to caption: x14.png

Figure: Figure A.14: AgentSquare search trajectory on Sciworld and Travelplanner (more hand-crafted agents, specific module combinations when surpassing best hand-crafted and the final evolved agent, other search baselines).
Refer to caption: x15.png

Figure: Figure A.15: Meta-prompt of module-level AgentSquare search.
Refer to caption: x16.png

Figure: Figure A.16: Meta-prompt of performance predictor.
Refer to caption: x17.png

Figure: Figure A.17: Meta-prompt of program-level AgentSquare search.
Refer to caption: x18.png

Figure: Figure A.18: New module discovered through AgentSquare search on Webshop.
Refer to caption: x19.png

Figure: Figure A.19: New module discovered through AgentSquare search on Sciworld.
Refer to caption: x20.png

Figure: Figure A.20: New module discovered through AgentSquare search on M3tool.
Refer to caption: x21.png

Figure: Figure A.21: New module discovered through AgentSquare search on Travelplanner.
Refer to caption: x22.png

Figure: Figure A.22: New module discovered through AgentSquare search on Pddl.
Refer to caption: x23.png

Figure: Figure A.23: New module discovered through AgentSquare search on Pddl.
Refer to caption: x24.png

## Peer Reviews

confidence: 3
contribution: 3
initial_rating: 5
presentation: 3
questions: The authors write that: “In contrast, module-level searching methods including random and Bayesian search lack a clear and insightful search direction." Where do we see this? When looking at the figures, it seems that random search is a pretty competitive baseline?
soundness: 3
strengths: - Modularity and Reusability: The modular allows the reuse and recombination of components, which aligns well with LLM advancements in modularization and scalability. 

- Effective Search Mechanism: The combination of module evolution, recombination, and performance prediction seems to be a robust optimization strategy. The proposed performance predictor effectively reduces evaluation costs, addressing practical limitations in real-world deployments of LLM agents.

- Comprehensive Evaluation: Benchmarks across domains such as web applications, embodied AI, and gaming provide evidence of AgentSquare’s efficacy and generalizability.

- Interpretable Insights: The framework’s ability to provide human-interpretable design insights is a useful addition, potentially helping in  aiding the design and tuning of future LLM-based agents.
summary: The paper introduces AgentSquare, a framework designed to automatically optimize LLM agent architectures within a modular design space. It proposes a novel approach, termed Modularized LLM Agent Search (MoLAS), by defining a modular architecture divided into Planning, Reasoning, Tool Use, and Memory modules. AgentSquare employs module evolution and recombination mechanisms, along with a performance predictor, to explore and identify optimal combinations within the design space efficiently. The framework is evaluated on six benchmarks, showing a 17.2% performance improvement over existing hand-crafted agent designs.
weaknesses: My main issue with the paper is that while the modular approach is beneficial in the short-term, it may limit flexibility by enforcing predefined components. Extending the modular design to allow more dynamic, task-specific modules could enhance its applicability.

Additionally, the framework’s reliance on LLM-driven suggestions for module evolution and recombination could inherit biases or inefficiencies from the LLM models themselves, potentially limiting the quality of novel configurations.

Minor comment: 
- Many citations miss parenthesis around them. Try using \citep instead.

---

confidence: 4
contribution: 3
initial_rating: 6
presentation: 3
questions: 1.Are there any specific criteria for selecting the 16 LLM agents when constructing module pools? Besides, human labor involved in standardizing these modules might become prohibitive if one hopes to leverage a larger number of existing agents. Are there any measures to tackle this problem? 

2.Could you elaborate a bit more on the evaluation of candidate solutions? Specifically, it is currently unclear when solutions are evaluated by the performance predictor and when they are tested in the real task environment. 

Thank you!
soundness: 3
strengths: 1.The paper is well-organized and uses language that is easy to understand. 

2.The paper is well motivated, addressing an interesting research question. It innovatively consolidates existing (and potentially upcoming) LLM agent designs into a unified framework, and effectively leverages their successful experience for better design. 

3.The experiments are conducted with both quantitative evaluations (including task performances, API costs and search trajectories), and qualitative analyses such as the specific module combinations of optimized agents and insights drawn from the newly discovered modules. 
 
4.The authors have made their code repository available on GitHub, which ensures reproducibility and promotes future work.
summary: This paper introduces a modular design space for LLM agents named “Modularized LLM Agent Search (MoLAS)” and proposes an evolutionary framework named “AgentSquare” for optimizing and recombining the various modules (covering planning, reasoning, tool use and memory) within this design space. It further introduces a performance prediction model for ruling out unpromising candidate solutions, hence rendering the search process more efficient. Through comprehensive evaluations it is demonstrated that their autonomous, modular design of LLM agents significantly outperforms manual designs and other search algorithms, and offers interpretable insights that complement human knowledge. This paper thus contributes to more standardized and scalable development of agentic systems that exploit prior successful experience, minimizing the reliance on human intervention.
weaknesses: 1.There are some inconsistencies within the paper that might cause confusion. For example, Equation (2) and (3) indicate that both module recombination and evolution take past experience as input, which is not the case in Figure 3. Besides, the random initialization (as mentioned in experimental setup) seems to contradict the arguments made at the beginning of Section 3.3. 

2. It seems more fair to also take into account the API cost incurred by search when you compare the performance-cost trade-offs of AgentSquare and manual design. However, the paper did not mention this information. 

3.The experimental results appear to be obtained from a single run. The authors are suggested to carry out repeated experiments, and report averaged results and standard deviations to demonstrate the robustness of their approach.

---

confidence: 3
contribution: 3
initial_rating: 5
presentation: 2
questions: 1. What is the evaluation and metric function to optimize the searching process ?
2. How does it scales if extending to multi-agent problem?
3. Is it able to solve manipulation tasks ?
soundness: 2
strengths: This method shows it's originality in using evolutionary and recombined mechanisms to automatically explore optimal combinations, effectively consolidating prior research efforts. It also provides empirical testing across diverse benchmarks, showing improvement over handcrafted and prior methods. The research extends to its broad applications and the potential to unify efforts within the LLM agent community, reducing reliance on task-specific human design part and enabling a more systematic exploration of agent architectures.
summary: This paper proposed a framework for automatically selecting and optimizing for off the shelf LLM-based methods from existing literature. It proposes a new concept:MoLAS to enhance agent performance through modular evolution and recombination. The design includes four fundamental modules: Planning, Reasoning, Tool Use, and Memory and uses an in-context performance predictor to efficiently evaluate designs, outperforming manually crafted agents across six benchmarks by an average of 17.2%.  In sum up, AgentSquare provides a platform for researchers to reuse and extend prior successful LLM agent designs.
weaknesses: The paper lacks clarity regarding the definition of certain components of the method, particularly the performance evaluation function mentioned in Section 3.1. It is unclear whether this refers to the API cost introduced later or another evaluation metric. Additionally, the method shows limited novelty, as it primarily focuses on leveraging LLMs to recombine and select existing components rather than introducing a fundamentally new application or capability for LLMs. Although the proposed approach does achieve better performance than existing handcrafted methods, it does not address or eliminate the core limitations of prior works, given that it relies heavily on reusing previous designs.

---

confidence: 5
contribution: 3
initial_rating: 1
presentation: 2
questions: See main review.
soundness: 3
strengths: See main review.
summary: I am disheartened to report I believe this paper involves a form of plagiarism and likely involves intentional academic misconduct (or at least suspect behavior). For full transparency, and because I am an author of the plagiarized work and thus have a stake beyond a normal reviewer, I am making my review public and non-anonymous. 

We posted a paper called “Automatic Design of Agentic Systems” (ADAS) on August 15th to arXiv and shared it on X and LinkedIn. It was submitted for review at ICLR 2025. 

This review is of AgentSquare: Automatic LLM agent search in modular design space. It was posted to arXiv on Oct 8 2024. 

This AgentSquare paper clearly takes many components from the ADAS paper, including the main ideas (slightly modified), repackages them (after adding a bit of new work), and presents the paper as a completely new set of ideas, all without acknowledging the significant degree to which it has lifted many key elements from our paper. It does cite ADAS, but only as an afterthought in the last paragraph before the conclusion, rather than writing throughout the paper (including in the motivation) that the ADAS paper is an extremely relevant piece of prior work and from which they copied (and slightly adapted) core pieces: the main ideas, unique writing choices, data visualizations, code, and prompts (which have significant chunks directly copied without modification). 

It is hard to believe this is an honest mistake. Instead, the evidence causes me to conclude it is an attempt to get an ICLR publication by hoping the reviewers are not aware of how similar this paper is to a previous arXiv paper. Given how similar this new piece of work is, I would expect (and feel scholarship demands) this new paper to mention ADAS early and often, including discussing how it inspired their work and how the works differ, and to acknowledge that this new paper uses many pieces of the ADAS paper. I would also expect AgentSquare to compare to ADAS, since a central claim is that it is an improvement over the approach ADAS took. Instead, the paper is written in such a way that a busy reviewer/reader might not know such a similar work exists, and thus mistakenly attribute credit for the big ideas, results, and novelty to this new work.

If the authors are somehow so unaware of what is appropriate and required in academic writing that this is all an honest mistake, then in my view they should let the community know that, apologize, and entirely rewrite the paper to properly inform readers. 

Note: this is almost certainly not a case of the work being done concurrently, because the paper copied so many pieces from ADAS (including prompts used in their experiments). 

To be clear, there are some nice new innovations in the paper (if the data are to be believed, see below). Properly written (and with more careful comparisons to ADAS and ablations of what makes this paper different and if they help), this paper could have been accepted as a nice innovation on top of ADAS. Instead, I believe it should be rejected as plagiarism. There is not enough time and rounds of review in this conference's review process for reviewers to consider a full rewrite (including back and forth) at this stage. 

Here is evidence of untoward behavior:

1. The prompts from ADAS are largely copied verbatim and then modified in a few places. The amount of overlap makes it certain they copied from our work, yet they did not acknowledge doing so. They should report using a prompt from another paper. But more importantly, it shows how similar the work is to, and was inspired by, ADAS, and thus that ADAS should have been mentioned throughout (as prior work, something to be compared to, something they built on and were inspired by, etc.). You can see the similarities here: https://drive.google.com/file/d/1vHPW2EXvx7LjFv-kDhQHyb_VGHTnndD8/view?usp=sharing

2. ADAS did something rare: it said it was (A) recognizing and naming “a new research area” named Automatic Design of Agentic Systems, and (B) introducing a new search algorithm within that area named Meta Agent Search. AgentSquare clones that template, saying “we introduce a new research problem: Modularized LLM Agent Search (MoLAS)” and “building on this design space, we present a novel LLM agent search framework called AgentSquare.” It is uncommon for papers to name a new research area explicitly, let alone also introduce an instance within it. 

3. Moreover, MoLAS is not a new research area. It is a subset of ADAS. It is a more constrained subset of ADAS, at least the way it is initially presented, which is that MoLAS is constrained to combine pre-existing, human-designed agent modules. However, in a contradiction that is confusing and harms novelty, later the paper also says the modules can be changed, removing this constraint, meaning that MoLAS is thus just ADAS with a new name?

4. Even if ADAS is not identical, it clearly is a very related prior work, yet it is not mentioned in the introduction, nor anywhere in the paper until the second-to-last paragraph. Yet other, much less related work is mentioned (line 053), suggesting an intentional choice to deceive the reader/reviewer by hiding work that hurts the novelty claims of this paper. Also, some of the claims in the introduction say effectively that things like ADAS do not exist or are rare (line 047): clearly ADAS should be mentioned in that context, but isn’t. 

5. The plot style of ADAS Fig. 3a is very similar to Fig. 4, providing further evidence of the central role the ADAS paper played in inspiring and catalyzing the AgentSquare paper. 

6. One (somewhat, see * below) new thing in AgentSquare vs. ADAS is an explicit surrogate model that predicts an agent’s performance, saving compute vs. running a real evaluation. However, I am very skeptical of this data. Even with much training and years of research, getting surrogate models to be highly predictive (e.g. in Neural Architecture Search) is very difficult. I have a hard time believing an LLM is even good zero shot, yet these data say it is nearly PERFECT zero shot. That raises an important question: why are there so few (5) points shown per run (e.g. versus the many more shown in many more per run in Fig. 4)? How did you select these 5? Can you provide code that replicates this experiment, so the community can run it independently and verify it? I do not mean the overall search, which is expensive, but just the zero-shot predictions and the evaluations for all agents produced per experiment. I have never asked for such a thing in review before, but extraordinary claims require extraordinary verification, in my mind, especially given the cloud of suspicion raised by all the other issues in this review.  If the surrogate model is indeed this good, that is a major discovery and worthy of publication in some form, and I recommend you share it in a properly written paper. 

The net effect of all of this evidence forced me to conclude the paper is at best deeply flawed due to many innocent mistakes, and at worst intentionally dishonest, with the latter seeming far more likely. 

If there were no plagiarism issues, I would also point out:

- There is no ablation of memory, one of the key new things they claim they added, and thus no evidence it helps

-  The paper should cite https://arxiv.org/abs/2206.08896

- *Surrogate models arguably implicitly exist within ADAS since it asks for proposals the model thinks will be high-performing, meaning it could use its predictions to guide proposals. However, we know from things like Chain of Thought and Reflexion that one can coax better performance by asking an LLM in the right way and/or asking it to reflect post-hoc on a creation. 

- The paper claims that ADAS’ search in the space of code is an over-simplification. While it is true that describing the search space is simple, searching in it is MUCH more complex. I think it is much more appropriate to say that MoLAS is simplifying search by constraining it to a simple, small search space vs. ADAS, the latter of which allows any possible agent. 

- The paper claims that ADAS is proposed without consideration of existing agent architectures. I disagree, since one can easily inject them as seeds in the search archive (and we did consider that). But we prefer that to be a choice: it likely speeds up search, but also biases it. 

It saddens me to make these accusations. However, I deeply believe in the integrity of the scientific process, including peer review. It relies on trust, and I think we need to stay vigilant and hold people accountable so we can maintain that trust as much as possible. 

Jeff Clune

Professor, Computer Science
University of British Columbia
JeffClune.com

Canada CIFAR AI Chair & Faculty Member
Vector Institute
weaknesses: See main review.

Meta-review: Wow, working through the reviewer discussion for this paper was a trip. As someone who has seen multiple cases of plagiarism accusations, both well-founded and unfounded, it was really interesting to see how this played out. As it turns out, I am happy with the outcome. It seems clear to me that the research described in this paper was independently conceived and executed, but also that the original manuscript was downplaying the role of the ADAS paper (which was published to arXiv earlier) to the point of seemingly trying to obscure it. The new version of the AgentSquare paper is much improved in that regard.

The AgentSquare paper is also a good paper overall - an interesting new approach to an important problem with good results. Clear accept.

Decision: Accept (Poster)