A Dynamic LLM-Powered Agent Network for Task-Oriented Agent Collaboration
Authors: Zijun Liu
Yanzhe Zhang
Peng Li
Yang Liu
Diyi Yang
Venue: COLM (2024)
Keywords: large language model, LLM-powered agent, multi-agent collaboration, team optimization
TL;DR: An LLM-agent collaboration framework that dynamically selects agents in a task-oriented, principled way.

Abstract: Recent studies show that collaborating multiple large language model (LLM) powered agents is a promising way for task solving. However, current approaches are constrained by using a fixed number of agents and static communication structures. In this work, we propose automatically selecting a team of agents from candidates to collaborate in a dynamic communication structure toward different tasks and domains. Specifically, we build a framework named Dynamic LLM-Powered Agent Network ($\textbf{DyLAN}$) for LLM-powered agent collaboration, operating a two-stage paradigm: (1) Team Optimization and (2) Task Solving. During the first stage, we utilize an agent selection algorithm, based on an unsupervised metric called Agent Importance Score, enabling the selection of best agents according to their contributions in a preliminary trial, oriented to the given task. Then, in the second stage, the selected agents collaborate dynamically according to the query. Empirically, we demonstrate that DyLAN outperforms strong baselines in code generation, decision-making, general reasoning, and arithmetic reasoning tasks with moderate computational cost. On specific subjects in MMLU, selecting a team of agents in the team optimization stage improves accuracy by up to 25.0% in DyLAN.

Published as a conference paper at COLM 2024
A Dynamic LLM-Powered Agent Network
for Task-Oriented Agent Collaboration
Zijun Liu', Yanzhe Zhang?, Peng Li3, Yang Liu Diyi Yang²
l Dept. of Comp. Sci. & Tech., Institute for AI, Tsinghua University, Beijing, China Georgia Institute of Technology, Georgia, USA
lnsttitute for AI Industry Research (AIR), Tsinghua University, Beijing, China
+Jiangsu Collaborative Innovation Center for Language Competence, Jiangsu, China Stanford University, California, USA
liuzi Suzijun20@mails, tsinghua. edu. diyiyestanford.cem
Abstract
Recent studies show that collaborating multiple large language model (LLM) powered agents a promising way for task solving. However, cur- rent approaches constrained by using fixed number of agents and static communication structures. In this work, we propose automatically selecting team of agents from candidates to collaborate in dynamic communication structure toward different tasks and domains. Specifically, we build a framework named Dynamic LLM-Poweered Agent Network (DyLAN) for LLM-powered agent collaboration, operating a two-stage paradigm: (1) Team Optimization and (2) Task Solving. During the first stage, we utilize an agent selection algorithm, based on an unsupervised metric called Agent Importance Score, enabling the selection of best agents according to their contributions in preliminary trial, oriented to the given task. Then, in the second stage, the selected agents collaborate dynami- cally according to the query. Empirically, we demonstrate that LYN outperforms strong baselines in code generation, decision-making general reasoning, and arithmetic reasoning tasks with moderate computational cost. On specific subjects in MMLU, selecting a team of agents in the team optimization stage improves accuracy by up to 25.0% in DyLAN
Query
Query
2+11 2=2 E
Agent Selection
Answer
T-FFN
T-FFN
Team Optimization
Task Solving
Figure 1: DyLAN adopts a two-stage paradigm. Agents communicate in structure of the temporal feed-forward network (T-FFN). At the "Team Optimization" stage, DyLAN performs agent selection for the most contributory agents in a primary collaboration, oriented to tasks domains. The selected agents then collaborate dynamically for an answer on the given query at the "Task Solving" stage.
Work done when the first author was a UGVR visiting student at Stanford University.
Code is available at https:: //gi www.. com SAL PNNLP/DLLA

---

Published as a conference paper at COLM 2024
1 Introduction
Large Language Model (LLM) agents (Richards & et al. 2023 Nakajima 2023
Reworkd 2023) have demonstrated promising pertormance on various tasks, ranging from reason- ing (ao al. 2023), code generation (Shinn et al. 2023 to embodied tasks such as video
gaming (Wang et al. 2023a) and autopilot systems (jin et al. 2023). Given the impracticality
of a single agent managing all these tasks efficiently, recent research has shifted towards
multi-agent collaborations, yielding significant advancements (Li et al. 2023 Du et al. 2023
Wang et al. 2023c liang et al. 2023 Shinn et al. 2023 Chen et al. 2024 Wu et al. 2023)
As an analogy to human society, how human teams function may provide valuable insights for developing more effective multi-agent collaboration systems. For instance, recent studies have demonstrated that certain effective communication structures, derived from human society, also play a positive role in multi-agent collaborations (Yin et al., 2023 Chen et al., 2024). In addition to communication structures, another notable characteristic of human teams is that they would optimize team members according to the given task. Take medical consultations as an example. Collaborations in dynamic structures are evident when the composition of the team changes within the procedure of consultation, as some doctors may become less relevant in major as the conversation goes deeper and "leave" the consultation, leading to corresponding changes in the communication structure. Team optimization is often observed in the varying initial composition of doctors for consultations with different diseases, influenced by changes in the related medical fields and the contribution of each doctor. These characteristics prompt an important question: Does a dynamically changing team of agents benefit LLM-powered agent collaborations similarly?
However, the question not well-addressed yet. While various communication structures have been studied for different tasks, such as debating for reasoning (Du et al. 2023 Liang
let al. 2023 Xiong et al.) 2023) and self-collaboration for coding (Dong et al. 2023 Qian
et al. 2023a b), these communication structures do not alter members in the agent team and remain fixed throughout the collaboration. It indicates that task-oriented dynamic
selections in agents are not thoroughly explored in current research. Furthermore, in the context of agent teams, most existing studies opt for hand-crafting agents from human
priors (Liu et al. 2023 Nakajima 2023|Hong et al., 2024Shinn al., 2023 Li et al.) 2023
or employ an LLM to generate them (Wang et al. 2023c Chen et al. 2023b Christianos
eet al. 2023). These approaches generally predefine agents without further validation of
the collaboration process. This leads to static agent teams or rebuilding teams without
principled verification (Chen et al. 2024). Challenges still remain for optimization methods.
As a first attempt towards addressing the above question, we introduce novel framework named Dynamic LLM-Powered Agent Network (DyLAN). DyLAN conceptualizes multi- agent collaboration using temporal feed-forward networks (f-FFNs). In this formulation, each communication step of the agents corresponds to a network layer, with nodes rep- resenting the agents involved at that step and edges indicating communications between agents, for incorporating dynamic agent teams agnostically, DyLAN functions in two stages to incorporate task-oriented agent collaborations (Figure D. The first stage is termed Team Optimization, where we select top contributory agents unsupervisedly among the initial team of candidates according to the task query, based on their individual contributions. message passing algorithm on the T-FFN termed agent selection in Section 3.4 inspired by the pack-propagation algorithm (Rumelhart et al. 1986) and neuron imporiance scores (Yu et al.) 2018). This algorithm measures the contribution of each agent at the first stage with an unsupervised metric named Agent Importance Score. The most contributory agents form a smaller team to collaborate at the second stage Task Solving, thereby minimizing the impact of less effective agents on the final answer. Specifically, the collaboration begins with a team of agents, and an LLM-powered ranker in the middle dynamically deactivates low-performing agents (i.e., agent team reformation), thus expanding the T-FFN, integrating dynamic communication structures into DyLAN (Section 3.3.2). Incorporating agent selection, DyLAN effectively identifies and coordinates a task-oriented team of agents in principled way. Extensive experiments demonstrate that DyLAN outperforms strong baselines in various tasks, including code generation, decision- making, general reasoning, and arithmetic reasoning. Notably, agent selection in DyLAN has
We propose a wwwarddbackward
2

---

Published as a conference paper at COLM 2024
improved accuracy by up to 25.0% in certain subjects of the MMLU dataset (Hendrycks et 2021a), underlining the significance of dynamic agent teams.
In summary, our contributions are threefold:
We introduce a novel framework named DyLAN for task-oriented agent collabo- ration in two stages with agent selection, marking a significant advancement in the study of dynamic agent teams.
DyLAN innovatively formulates agent collaborations in temporal feed-forward networks with agent feam reformation, enhancing its adaptability and reducing dependence on human preconceptions.
Empirical results demonstrate the superior accuracy, efficiency, and stability of DyLAN across various tasks, underscoring the need for dynamic agent teams.
Related Work
Team Optimization of LLM-Powered Agents The construction of agent teams is the essen- tial and initial step for LLM-powered agent collaboration. TPTU (Ruan et al. (023) and Chameleon (Lu et al.) 2023 decompose tasks to choose or create tools accordingly. Recent studies also use LLMs to generate a fixed number of role prompts for agents in response to a task query (Wang et al. 2023c Suzgun & Tauman Kalai 2024), or for each round of
discussion (Chen et al. 2024).. However, manual prompts require careful design, which is impractical for adaptation on each task or domain, and prompting LLMs with predefined or generated descriptions may not result the desired abilities of the agents without verifi- cation. Therefore, posteriorly selecting a team of agents based on their actual behaviors in the collaboration according to the task becomes essential. While team optimization for LLM agents is relatively new area, human-tean optimization has been studied for a long time. For instance, Liu et al. (2015) show that skill contribution is essential for selecting crowd workers to solve outsourced tasks efficiently. Based on peer rating, researchers have devel- oped an algorithm for managing online workers in an optimal organization (Lykourentzou et al., 2022). Drawing inspirations, we introduce an unsupervised algorithm to select a team of agents by quantifying their contributions based on peer ratings in Section 3.4
Communication Structures in LLM-Powered Agent Collaboration Collaboration between multiple LLM agents has demonstrated strong performance on various tasks in recent years and has emerged as a promising approach to enhance the capabilities of individual LLMs. To enable collaborations between multiple agents, recent studies have developed different communication structures and assigned agents in pre-defined architecture. For instance, researchers have found taking multiple LLM instances to debate for fixed number of rounds can boost their factuality and reasoning capacities (Du et al. 2023 Liang et al. 2023 Xiong et al.] 2023). To aggregate multiple LLM responses, LLM-Blender (Jiang et al. 2023
calls different LLMs in one round and uses pairwise ranking to combine the top responses.
It has also been shown effective in distributing workloads to L.MSs and concatenating their
answers, thus producing better results (Ning et al. 2024; Suzgun & Tauman Kalail 2024
Qiao et al. 2024). It is worth noting that existing studies (Hao et al. 2023 2023) Zhang et al. 2023b
have tried organizing LLM instances into linear layers, but they mainly studied supervised learning in context space and LLM evaluation, respectively, not the scenario in which we are interested. However, running LLMs in static architecture may limit the performance and generalization. On specific reasoning tasks, adopting a dynamic directed acyclic graph structure for LLMs has been shown effective (Zhang et al. 2023). Alsso, recent studies (Yin et al. 2023 Chen et al. 2024 Zhang et al. 202ga:Zhuge et al. 2024 have demonstrated that optimal communication structures vary with tasks and compositions of agents. Aligned with the findings, we propose a structure that adjusts dynamically based on selecting agents according to the tasks and the construction of the agent team in Section 3.3.2
Evaluation of the Contribution of LLM-Powered Agents It is non-trivial to evaluate the contribution of each LLM agent in multi-agent system, especially when they communicate over multiple rounds. In the single-round setting, existing methods use LLMs heavily for evaluation. To overcome the over confidence of LLMs (Xiong et al.) 2024), pairwise
3

---

Published as a conference paper at COLM 2024
Method
Single Exec.
LLM-Blender
LLM Debate
Reflexion
CAMEL
AgentVerse
DyLAN
Commmunication
Struc-
(V,E)
Multiple Roles
Manual
Manual
Generated
Man.deGen.
Early Stopping
Dynamic
Optimization
Table 1: Comparison between DyLAN and representative previous works. In the second row, nodes denote agents at different time steps (V), arrows represent edges (E), and color indicates the role of agents.
ranking based on additional LLM ranker has been introduced in LLM-Blender (Jiang et al. 2023). To rank n responses with an independent LLM a single round, they compare all (n²) pairs. For better efficiency, researchers use a k-length sliding window to choose top k responses within O(nk) pairwise comparisons (Qin et al. (023). However, these methods have not been extended to multi-round settings. Inspired by the neuron importance score (Yul al. 2018), we evaluate agents by propagating and aggregating single-round peer ratings in a propaaaaonn manner (Rumelhart et al. 1986). In this way, we then introduce an unsupervised metric called Agent Importance Score to quantify the contribution of each agent in multi-round collaborations (Section 3.4)
3
Dynamic LLM-Powered Agent Network
3.1 Overview
We introduce framework for LLM-powered agent collaboration named Dynamic LLM- Powered Agent Network (DyLAN), facilitating dynamic communication structures and automatically task-oriented agent selection in two-stage fashion (Figure 1]: an optimized agent team is constructed in the first stage "Team Optimization" through preliminary trial and then the team collaborates to solve the task in the second stage "Task Solving".
A core component of DyLAN is the temporal feed-forward networks (T-FFNs), whose nodes denote agents and edges denote the communication channels between agents (Figure left). T-FFNs serve not only the abstraction of communication structures but also the computation graph. From this perspective, as shown in Table various LLM-powered agent
collaboration systems (Jiang al. 2023 Shinn et al. 2023 Du et al. 2023 al. 2023 Chen et al. 2024) can be represented by similar network structures as T-FFNs. DyLAN IS the only framework that supports multiple agents with roles and tools, early stopping (Section 3.3.2 dynamic communication structures and team optimization simultaneously. To be specific, for team optimization, our agent selection algoríthm is performed as backward message passing algorithm on the T-FFN (Figure righht, and for task solving, agent team reformation expands the T-FFN dynamically with messages passing forward.
To make it easier to understand, we will start by explaining the formulation of T-FFNs. Then, we will move on the task solving stage, and finally, we will explain the team optimization stage, which relies on components of task solving.
3.2 Temporal Feed-Forward Networks (T-FFNs)
T-FFN is a multi-layer network, of which each layer represents a time step. Its formal definition is follows.
Definition 1 (Agents) Agents participating the collaboration are represented by
A= {=1, 22, ,Nn), where N denotes the total number of agents, and a; can (1) an LLM-powered agent possibly equipped with tools, (IT) independent tool, e.g., scripts, code interpreters.
(1)

---

Published as a conference paper at COLM 2024
3=3
Agent Selection
( Propagation
2 Aggregation
Selection
2+
2+
Σ
G = 0.65
Answer
0
0.35
3.
1:15
0.20
Agent Team Reformation
Peer rate the responses predecessors.
Aggregate the ratings from successors.
Top 2 clite agents are selected based importance scores.
T-FFN
Figure 2: The left part shows how DyLAN outputs the answer in a temporal feed-forward network (T-FFN), where nodes represent agents at specific time steps (Section 3.2). Agent team reformation functions in the middle steps, during which the low-performing agent is deactivated in subsequent time steps. The right part depicts agent selection (Section 3.4), where the contribution of each agent in a primary trial is automatically evaluated in three steps using Agent Importance Score, denoted I. Then, the top-ranked agents based on will be selected as the optimized, task-oriented team of agents.
Definition 2 (Nodes) The t-th layer of T-FFN consists of N nodes, each of which corresponding to one agent:
Vi = {vE,, ,t,N
(2)
where: = 1, T, and node ii corresponds to agent a;
Definition 3 (Edges) Edges in T-FEN refer to the communication channels between nodes, forming the communication structure between agents. Specifically, the set of edges between the nodes in layer t 1 and t is denoted
E = {(01-1,, 01,)) S V1-1 V.,
(3)
where 2,... ,,, and (0z-1,5, V1j) denotes an edge connecting nodes U and Ut,j-
Definition 4 (T-FFN) Finally, the T-FFN corresponding to the collaboration is defined T-layer network:
= OV,...... .V;;; E1, ,T--,T)
(4)
Note that we only consider T-FFNs where edges only exist in adjacent layers. However, edge can be added to arbitrary pairs of nodes, making the T-FFN capable of representing more complex communication structures.
3.3
Task Solving
Task solving involves performing inference on the T-FENs, jointly with agent team reformation, which be elaborated on in the subsequent two sections.
3.3.1 Inference
Before details, first introduce the formulation of message passing T-FFNs.
Definition (Message Passing) Given T-FEN, node Vtjj, a set of adjacent nodes U (u,,..., "kk) and the messages M = (ma..... mug} received by Dt, where is the size Uf and mug is the message sent from uk to Vtj, message passing aggregates all the messages M to produce a updated message Ah, for Dt,,, which is formally defined
May = fmp (M, (M,))
(5)
where fmp (x.)) is the aggregator function.
5

---

Published as a conference paper at COLM 2024
We refer the algorithm as forward message passing when U is the set of all adjacent nodes of Dt, from the previous time step, i.e,, U = {ux|Vug (uk, Vt,)) E E,-1,). Similarly, it is referred as backward message passing when U is the set of all adjacent nodes from the next time step: U1 [ug]Vuk, (Vtj..uz m2.) )-
Definition 6
With the above formulation, we can describe the inference process of a T-FFN g in the manner of forward message passing. During collaborations on given task, an agent at specific time step takes the responses, iee, messages, from other agents at the previous time step input and generates responses based on the task query. Based on different types of agents at Utj, we can implement fmp (-, respectively: (1) concatenating input messages along with the task query into prompt templates (refer to task instructions in Appendix D) and take the response from LLM after generation tool calling, or (II) filter the input that the tool can process, e.g, code completions and structured text.
During inference, we begin feeding the task query 9 where Q denotes the dataset. By passing responses of nodes V; at time step f - to nodes V; at f, agents can perceive responses from all nodes at the previous time step and perform collaborative behavior, which might include criticizes, advice, refinement, or quality reviews, depending on the implementation of agents. Formally, the inference process is defined
into agents at time step (V1),
finter(g. 9) = O,
(6)
where argmax { [M} and MM denotes the responses from Vr. Please refer to Algo- rithm for detailed procedure.
3.3.2 Agent Team Reformation
Given a set of agents A, agent team reformation aims to identify more contributory agents and construct a dynamic communication structure accordingly. To this end, we leverage an additional LLM instance, referred as the "LLM Ranker", " to analyze responses from the agents participate in the current time step and give out a ranking, prompted by the template of "Ranker" " in Appendix D Then, the top-ranked agents are allowed to participate in the next time step. in other words, edges will only be added for these top-ranked agents, resulting in a dynamic communication structure.
Formally, suppose the set of agents participates in time step is A:= = {ax}, and the top- ranked agents are A++1 = [ar), where k and I are the indices of the agents as defined in Equation (1), then we can obtain two nodes sets V = {Vri/aa A, and 1/+1
(Di+LiVVa E 11 and the edge set FAr11 is defined as
Eg,+1 V/d V/11-
The process progresses iteratively until the stop condition is met, and finally, we get a T-FFN A for the query input 9. We use function fIAS to denote the entire computation:
g = fiA((A,)).
(8)
To further enhance efficiency, we introduce an early-stopping mechanism. Inspired by the Byzantine Consensus theory (Castro & Liskov 1999), at least 3p + agents are needed to tolerate p faulty agents in a single round of communication. Following the theory, the inference process will be terminated when over 2/3 of agents in single layer have consistent answer. In practice, the inference process will also be terminated when the maximum time step is reached. Note that none of the consistency measures used in prior work (Wang et al., 2023b; Aggarwal et al., 2023 Yin et al., 2023) applies to multi-round multi-agent interaction since their theories are assumed to execute single LLM instance multiple times or expect all agents to reach the same answer.
3.4
Team Optimization
The goal of team optimization is to select a subset of agents from candidates based on their contributions evaluated from primary trial, such that the new team solves the task query

---

Published as a conference paper at COLM 2024
Method
Pass@1
#API Calls
Success Rate
#API
Method
Reward
Calls
Single Execution
73.2
(+0.0)
1.00
CodeT
65.8
(-7.4)
20.00
Direct Execution
50.6
(+0.0)
28.0
14.52
CodeT (Codex)
74.8
(+1.6)
20.00
ReAct
53.8
(+3.2)
30.0
8.40
Reflexion
68.3
(-4.9)
4.05
ReAct-SC
58.0
(+7.4)
36.0
25.75
LATS
81.1
(+7.9)
48.00
Reflexion (trint=4)
62.0
(+11.4)
40.0
25.40
LATS
64.5
(+13.9)
38.0
> 400
CAMEL
69.5
(-4.1)
12.03
AgentVerse
75.0
(+1.8)
22.50
BOLAA
66.0
(+15.4)
40.0
32.40
DyLAN (Owss
82.9
(+9.7)
16.85
DyLAN (Ours)
68.3
(+17.7)
42.0
24.85
Table 2: Experimental results the CG task (left) and results on the DM task (right). The number in parentheses indicates the difference relative to the single execution or direct execution. We indicate the foundation model of methods except for GPT-35-turbo. The median of three trials is reported when non-zero temperature is used.
more effectively and efficiently, Formally, given task query 4, a set of agents A, trial is performed based on the algorithm proposed in Section 3.3.2 resulting in T-ENN gig.. And team optimization formulated as
À fOptim A. 4), where A C A.
foptim is implemented as in a three-step procedure of agent selection (Figure right):
(1) Propagation: Each node rates the solutions to the task query from its predecessors, which a forward message passing process. Formally, given node Dtj and an edge (0 Ptz), the message M7111, sent from 1, to Vi, defined as the response the task query 4 from the agent a; at the previous time step. The aggregator function fmp(;, Pt,)) is implemented
as a scoring function p(s) (y--), which maps the prompt Pj. the input query 9, and all the messages M to the rating scores. Here, we use -11ii to refer the rating score on 8/-1,i from Pt.j and
1,1,, Wt 122.. Wt -;-,,; L,N ((55 (pHg.M).
(10)
(2) Aggregation; Each node aggregates the ratings it has received from its successors towards itself to quantify its own contribution independently at different time steps. The contribution of node D is the sum of its successors' contribution multiplied by their peers' ratings on the agent's response. Aggregation a backward message passing process. Formally, given a node 0/-1, and an edge (7,-1i, Dt,j), Dtj),, the message Ma, sent from Pt to 0/-1; is defined as W1-1if- And the aggregator function fmp is defined weighted sum function:
1
Σ
I.j WW Li,j
(11)
(DL-1AP%,)
where 4 denotes the contribution of (n2i-
(3) Selection: During the last step, we sum up the scores for the same agent over all time steps to derive an importance score for each agent, and extract the top-k agents that are most contributory according to these scores to form the optimized agent team. Formally, the Agent Importance Score I; for agent a; is defined as
Σ 4.
(12)
In practice, we initialize the contributions the final layer first, and step backward to perform Aggregation layer by layer (Algorithm 2). The definition guarantees that the agent importance scores add up to each layer, which benefits fair comparison. Other details, such initializing contributions in the final layer, presented in Appendix B.2
7

---

Published as a conference paper at COLM 2024
Method
Prompting
Algebra
Counting and Probability
Geometry
Intermediate Algebra
Number Theory
Pre- Algebra
Pre- Calculus
Overall
RAPI
Calls
Single Execution
43.6
29.3
21.3
158
300
48.9
31.6 (40.0)
LLM Debate
LI M-Benerr
Col
50.2
47.5
25.5
25.3
22.3
23.8
13.8
13.1
28.9
39.7
46.7
48.0
19.0
158
31.7 (103)
32.4 (1088
600
8.00
DyLAN (Ouns)
52.9
27.2
25.3
15.5
33.5
55.2
19.0
35.7 (+4.()
7.15
Single Execution
Complex
49.1
29.7
22.3
14.6
33.4
5388
1688
34.1 (40.0)
1.00
PHP
Coll
51.1
33.7
25.4
17.3
35.1
57.7
16.1
36.5 (12.4)
3,67
DyLAN (Ours)
53.7
33.3
26.1
18.
33.5
58.7
18.9
37.6 (+3.5)
6.1
Table 3: Accuracy (%) on the AR task. The number in parentheses indicates the performance difference relative to a single execution.
Method
Hum- anities
Social Science
STEM
Other
Overall
#API Calls
Task
#Agents
Tool Usage
Performance Improvement
#API Calls
Random
25.0
25.0
25.0
25.0
25.0
76.2
82.9
23.04 + 16.85 32.03 24.85
Single Exec
DM
8 144
x
53.0 3 68.3
LLM Blender
60.4
75.2
66.3
70.7
67.3 (10.9)
6.00
GR
7-4
69.5 70.5
8.30 + 4.39
LLM Debate
59.8
77.4
69.0
75.5
69.3 (+2.9)
12.00
DyLAN
(+.5()
Table 5: Demonstration of experiment Table 4: Accuracy (%) on the GR task. "Other" settings, including the number of agents stands for subjects like business, health, and misc and the performance throughout team in the MMLU dataset. We report the median of optimization. We report reward for the DM task.
three runs for experiments.
4 Experiments
4.1 Setup
Code Generation (CG) We use the HumanEval benchmark, with 164 human-labeled function-level completion codes and unit tests (Chen et al. 2021). Unit tests are used to validate the correctness of generated codes. We leverage two strong baselines CodeT (Chen et al. 2023a) and Reflexion (Shinn al., 2023) along with the single execution. For multi- agent baselines, we re-implement CAMEL (Li et al. 2023) and AgentVerse (Chen et al. 2024 under their original configurations for fair comparisons.
Decision Making (DM) We evaluate our methods in the WebShop environment, selecting 50 environments in its test set (Chen et al. 2021] following the setting of LATS (Zhou et al. 2023). WebShop requires to find the item given an instruction of the customer. It provides "reward" as an intrinsic metric for item-instruction relevance, and "success" is marked when the reward is 1.0. Besides ReAct (Yao al., 2023) and Reflexion, we re-ran multi-agent method BOLAA (Liu et al. 2023), and a single-agen method LATS as strong baselines.
General Reasoning (GR) For the general reasoning task, we use the MMLU dataset (Hendrycks et al., (2211a), which contains four categories of a vast amount of prob- lems 57 subjects. We down-sample 1/5 of the problems in the test set because of its huge quantity. We choose LLM Debate (Du et al. 2023), LLM-Blender (Jiang et al., 2023), and the single execution on LLM as baselines.
Arithmetic Reasoning (AR) We leverage the test set of MATH (Hendrycks et al., 2021b) for evaluation, which consists of 7 subareas and 5,000 questions total. To draw fair comparison and verify the robustness, we categorize methods by different prompting strategies and select strong baselines accordingly. Preliminary experiments show that collaborating agents in different domains (e.g,, algebra and geometry experts) does not make significant improvement, therefore we adopt agents with same prompts for all methods.
DyLAN Setup In Table we elaborate the setup of DyLAN. To keep in line with baseline methods, we only equipped DyLAN with code interpreters as tools in the CG task. It is worth noting that agent selection is performed for each subject in the GR task, for each web page in the DM task, and directly for CG task in the team optimization stage. Due to space limitations, please refer to Appendix B.1 for details.
8

---

Published as a conference paper at COLM 2024
70.5
71.8
70.
Method
Acc.
AR
RAPI
Acc.
GR
RAPI
(4 agent))
70/0
DYLAN
DyLAN
35.7
7.15
70.5
4.39
08.5
59:5
CAPI
z066 atr
wyb
35.0
33.8
13.00
8.20
69.9
7.05
060
08.3
DyLAN
$884
2.33
Method
Passi1
CG
#APT
Reward
DM
#API
LLM Detote agenn)
0..0
DyLAN
82.9
16.85
68.3
24.85
'Namiir Agenta
Numtbar of Agjrns
undo
80.5
19.00
67.5
54.25
76.2
17.98
66.0
48.90
Figure 3: Impact of optimized agent team size. 2~4 agents are selected from 7 candidate agents based on Agent Im- portance Score. Accuracy (left) and #API calls (right) on the GR task are visualized.
Table 6: Impact off the early-
stopping mechanism (es) and agent team reformation (atr).
4.2 Main Results
In Table 2 Table and Table 4 we report the results on each dataset respectively. The number of API calls serves as proxy for the efficiency of communication structures for agents, which cannot be clearly determined from token consumption which varies greatly depending on tasks and prompting strategies. Since "Task solving" after "Team Optimization" " is essential for testing and deployment, we mainly report the cost from the second stage. The difference can be seen in Table 5 and further discussions in Appendix C.1
DyLAN improves overall performance on different tasks with a reasonable computational cost. From Table we find DyLAN realizes an 10.2% improvement to LLM Debate in terms of accuracy, with 10.6% lower #APT calls (L3 vs. L4), suggesting it is a better trade-off between efficiency and effectiveness. Similar trends can be observed as DyLAN has better performance with only 36.6% API calls of LLM Debate (L5 vs. L4 in Table 4 and 35.1% of LATS on CG and <6% on DM (L8 vs. L5 (left), L7 vs. L5 (right) in Table 2). We argue it can be attributed to the feed-forward structure and early-stopping mechanism, which allows different solutions be delivered simultaneously and confirmed rapidly. In contrast, for methods in sequential architecture like PHP and Reflexion (Table Z) incorrect intermediates might easily influence the final output due to the single thread of reasoning or code generation and review. Also, ReAct on DM tasks exhibits similar failures due to misoperations in the middle. In contrast, Reflexion and LATS access the environment at certain states for multiple times to for reflection, limiting generalizabilities In case, any feedback from predecessors could be rated by successor nodes, making it easier to rectify potential invalid actions. Moreover, we that DyLAN dynamically adjust the cost based on the difficulty of tasks. For instance, most questions in the MMLU dataset are less challenging than MATH, DyLAN has 2.76 fewer API calls on the query from the former. However, compared to other tasks, DyLAN introduces relatively lower improvements in AR tasks, which might be due to the high knowledge dependency of the MATH dateset.
DyLAN benefits from the team optimization. Moreover, we found that a dynamic team of task-oriented agents could enhance DyLAN. For different subjects in GR tasks, agent compositions are adjusted correspondingly to improve up to 25.0% in accuracy, as shown in Table 7 As denoted in Table a dynamically selected team of agents could result in signifi- cant performance improvement, especially for DM tasks, where agents might have great interference from others. The overall performance can significantly improved (up to 6.7%) with lower computational costs on each tasks after agent selection. Moreover, it also suggests that Agent Importance Scores can effectively capture and reflect the actual contributions of agents wide range of tasks. We further verify this claim in Appendix C.6.
4.3 Ablation Studies
Impact of Optimized Agent Team Size Fewer proper agents in a team could have better performance. As shown in Figure DyLAN with an optimized team of agents outper- form both DyLAN before team ophimizatin and LLM Debate with 4 agents, suggesting the effectiveness of our proposed agent selection. The efficiency is also significantly improved

---

Published as a conference paper at COLM 2024
Subject
Optimized Composition
Performance Improvement
#Code Writers
#Code Reviewers
Passi1
#API
Calls
college
Economist, Lawyer, Programmer, Mathematician,
mathematics
25.0 :0.0 65.0
6
76.2
23.04
Lawyer, Psychologist, Fconomist, Programmer
4
4
82.9
16.85
management
:7.2 90.5
81.1
14.64
4
2
77.4
12.50
high school
Historian, Programmer, Psychologist, Mathematician
78.0
11.73
statistics
9.3 65.1 - 74.4
2
75.6
9.60
elinical
Ductor, Mathematician,
knowledge Programmer. Psychologist
5.7 + 75.5
Table 8: Different compositions of agents on the CG task of an optimized team of agents. Agent teams are optimized by the Agent Importance Score from the first line.
public
Historian, Psychologist,
:.55545 39.1
relations
Lawyer, Mathematician
Table 7: The optimized composition of agents and perfor- mance improvement on different subjects the GR task.
by 52.9% and 67.8%, respectively. Probably because the imbalance of agents' expertise and opinions interfere with each other before optimization, especially on GR, where few candidates are relevant to subject.
Robustness Agent Importance Score The Agent Importance Score is robust over the imbal- ance of agent roles. On GR tasks, the candidates are imbalanced in terms of expertise. For most queries, there are less than 2 candidates that are related according to their role prompts. In Table we found agent selection is capable for selecting related agents, e.g,, "Mathemati- cian" " for college mathematics", that matches human priors. However, if candidates are all vastly different from the task domain, e.g., "public relation", where the improvement is less significant. We also tested DyLAN on CG tasks with different amount of code writers and code reviewers after the "Team Optimization" stage. It is worth noting that a single run of "Team Optimization" could provides reusable Agent Importance Scores for multiple trials of agent selection. In Table we exhibit the results under imbalanced teams of agents. We verified the imbalance of code writers and reviewers after optimization won't 't cause great performance drops. Though, reviewers affect the performance slightly greater than writers (I2,33 vs. 14,5), indicating the necessity of the amount of reviewers for code refinement.
Impact of Early-Stopping and Agent Team Reformation As shown in Table 6 early- stopping mechanism boosts efficiency to great extent by minimizing #API calls by 45.0%, 66.2%, 11.3%, and 54.2% on AR, GR, CG, and DM tasks respectively, while providing slight performance improvement. Agent team reformation, however, is critical to enhance the correctness of the final answer. We conjecture it is because agents are filtered for temporary mistakes in LLMs, such as hallucinations, etc. Additionally, answer comparison is more challenging for open-ended tasks like CG or DM tasks. We use the BLEU score with a 0.9 threshold for consistency checks. This makes early stopping less effective since agents may generate codes in different formats, leading to fewer opportunities to stop early.
Stability of DyLAN with Different Backbone Models There is also a notable difference in CG tasks when the backbone model changes (Table Reflexion and CodeT's performances heavily related to the backbone model (L4 vs. and L6 L9). Instead, DyLAN shows steady, consistent high performance (L7 vs. L10) under different backbone models with almost the same amount of API calls.
Conclusion and Future Work
This work introduces a framework named Dynamic L.LM-Powered Agent Network (Dy- LAN) for collaboration of dynamic agent teams on complicated tasks. DyLAN functions in a two-stage paradigm, enabling agents to interact in a dynamic structure with agent team reformation. In "Team Optimization" stage, the agent selection algorithm based on an unsupervised metric termed Agent Importance Score, selects top contributory agents principled way for collaboration on "Task Solving" Overall, DyLAN reveals improvement on diverse tasks with relatively less computational cost compared to baselines. In the future, plan to explore the effectiveness of DyLAN built on open-source foundation models.
10

---

Published as a conference paper at COLM 2024
Acknowledgments
We thank Yilun Du for the helpful assistance on code implementation. We sincerely thank William Held, Ruibo Liu, Dr. Yue Zhuge, Noah Shinn and Kangfu Zheng for their valuable feedback on the project.
Ethics Statement
LLM-powered agent systems are widely used in practical applications. DyLAN could also effortlessly cover practical software development, virtual room chat, video games, and so on (Hong et al. 2024 Nascimento et al.) 2023 Zhou et al. 2023 Zhu et al. 2023 Chan et al., 2024). In these open-world environments, agents may operate as planners, actors, etc. DyLAN only requires people to give rough instructions on the constitute of agents and could automatically optimize better team of agents construct an efficient multi-agent system. These systems could benefit from DyLAN to reduce human labor on designing agents and have a better performance on their target tasks.
Also, the overall architecture of DyLAN (Figure reflects the optimal collaboration or- ganization of human online workers (Lykourentzou et al. 2022), and reveals significant performance in agent collaborations. Theretore, simulating human collaboration by LLM- powered agent collaborations under DyLAN might also be possible. Optimizing human collaboration by searching and simulating LLM agents will hopefully be more convenient and effective. We also acknowledge the potential risk in pretrained language models used in the paper, e.g., GPT-3.5 and GPT-4, which may cause improper responses. Furthermore, creating agents by hand and LLM generations might prompt L.MM-ooeeeed agents to act or response misaligned with principles of society, which may possibly happen during agent collaborations. However, we think team optimization process potentially alleviate this situation.
References
Pranjal Aggarwal, Aman Madaan, Yiming Yang, and Mausam. Let's sample step by step: Adaptive-consistency for efficient reasoning and coding with LLMs. In Houda Bouamor, Juan Pino, and Kalika Bali (eds.), Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pp. 12375-12399 Singapore, December 2023. Association for Computational Linguistics. doi: 10. 00.18653/1112033emmppp-./
URL https
//aclanthology.o .orpllooyooooooooooooooooo org/2023 wwn.p-nanaiii mopo-mmii.... 761
Miguel Castro and Barbara Liskov. Practical byzantine fault tolerance. In Proceedings of the Third Symposium on Operating Systems Design and Implementation, OSDI '99, pp. 173-186, USA, 1999. USENIX Association. ISBN 1880446391,
Chi-Min Chan, Weize Chen, Yusheng Su, Jianxuan Yu, Wei Xue, Shanghang Zhang, Jie Fu, and Zhiyuan Liu. Chateval: Towards better LLM-based evaluators through multi-agent debate. In The Twelfth International Conference on Learning Representations, 2024. URL http:://opeeerreeeeeeeeeeererrrr
Bei Chen, Fengji Zhang, Anh Nguyen, Daoguang Zan, Zeqi Lin, Jian-Guang Lou, and Weizhu Chen. Codet: Code generation with generated tests. In Proceedings of The Eleventh International Conference on Learning Representations, 2023a. URL http::: httpps://openreeveew. net/ forrmm?dd-ktrw@aCme
Guangyao Chen, Siwei Dong, Yu Shu, Ge Zhang, Sesay Jaward, Karlsson Börje, Jie Fu, and Yemin Shi. Autoagents: The automatic agents generation framework. arvio preprint XrX::::309.11 102888788..... 2023b.
Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, Alex Ray, Raul Puri, Gretchen Krueger, Michael Petrov, Heidy Khlaaf, Girish Sastry, Pamela Mishkin, Brooke Chan, Scott Gray, Nick Ryder, Mikhail Pavlov, Alethea Power, Lukasz Kaiser,
11

---

Published as a conference paper at COLM 2024
Mohammad Bavarian, Clemens Winter, Philippe Tillet, Felipe Petroski Such, Dave Cum- mings, Matthias Plappert, Fotios Chantzis, Elizabeth Barnes, Ariel Herbert-Voss, Voss, William Hebgen Guss, Alex Nichol, Alex Paino, Nikolas Tezak, Jie Tang, Igor Babuschkin, Suchir Balaji, Shantanu Jain, William Saunders, Christopher Hesse, Andrew N. Carr, Jan Leike, Josh Achiam, Vedant Misra, Evan Morikawa, Alec Radford, Matthew Knight, Miles Brundage, Mira Murati, Katie Mayer, Peter Welinder, Bob McGrew, Dario Amodei, Sam McCandlish, llya Sutskever, and Wojciech Zaremba. Evaluating large language models trained on code. Xrviv preprint ar arXio:2107.03374 2021.
Weize Chen, Yusheng Su, Jingwei Zuo, Cheng Yang, Chenfei Yuan, Chi-Min Chan, Heyang Yu, Yaxi Lu, Yi-Hsin Hung, Chen Qian, Yujia Qin, Xin Cong, Ruobing Xie, Zhiyuan Liu, Maosong Sun, and Jie Zhou. Agentverse: Facilitating multi-agent collaboration and exploring emergent behaviors. In The Twelffh International Conference on Learning Representations, 2024. URL httptt:://ppeeeeiiiiiie www.:nnt/formmmid-etooooooo
Filippos Christianos, Georgios Papoudakis, Matthieu Zimmer, Thomas Coste, Zhihao Wu, Jingxuan Chen, Khyati Khandelwal, James Doran, Xidong Feng, Jiacheng Liu, Zheng Xiong, Yicheng Luo, Jianye Hao, Kun Shao, Haitham Bou-Ammar, and Jun Wang. Pangu- Agent: Tine-lunable Generalist Agent with Structured Reasoning. arXiv preprint arXiv:2312.14878 2023.
Yihong Dong, Xue Jiang, Zhi Jin, and Ge Li. Self-collaboration Code Generation via ChatGPT. Xiv preprint ar xxXt::2304.0750) 2023.
Yilun Du, Shuang Li, Antonio Torralba, Joshua B. Tenenbaum, and Igor Mordatch. Im- proving factuality and reasoning in language models through multiagent debate. arXiv preprint ar Xix:2330551533 2023.
Rui Hao, Linmei Hu, Weijian Qi, Qingliu Wu, Yirui Zhang, and Liqiang Nie. Chatllm network: More brains, more intelligence. arXiv preprint Xiv:22344.12998, 2023.
Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. Measuring massive multitask language understanding. In Proceedings of the International Conference on Learning Representations (ICLR), 2021a.
Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. Measuring mathematical problem solving with the math dataset. In Proceedings of Thirty-fifth Conference on Neural Information Processing Systems, 2021b.
Ralf Herbrich, Tom Minka, and Thore Graepel. Trueskill A bayesian skill rating system. In B. Schölkopf, J. Platt, and T. Hoffman (eds.), Proceedings of Advances in Neural Information Processing Systems, volume 19. MIT Press, 2006.
Sirui Hong, Mingchen Zhuge, Jonathan Chen, Xiawu Zheng, Yuheng Cheng, Jinlin Wang, Ceyao Zhang, Zili Wang, Steven Ka Shing Yau, Zijuan Lin, Liyang Zhou, Chenyu Ran, Lingfeng Xiao, Chenglin Wu, and Jürgen Schmidhuber. MetaGPT: Liyi Meta programming for multi-agent collaborative framework. In The Twelfth International Conference on Learning Represenlations, 2024. http:p/ss.ggeeeeeeeeeeee////oooa www.:net/foormiiidivaaaatata
Dongfu Jiang, Xiang Ren, and Bill Yuchen Lin. LLM-blender: Ensembling large language models with pairwise ranking and generative fusion. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 14165 14178, Toronto, Canada, July 2023. Association for Computational Linguistics. URL https://aclantholoe org. 2023. acl long. 792
Ye Jin, Xiaoxi Shen, Huiling Peng, Xiaoan Liu, Jingli Qin, Jiayang Li, Jintao Xie, Peizhong Gao, Guyue Zhou, and Jiangtao Gong. Surrealdriver: Designing generative driver agent simulation framework in urban contexts based on large language model. ar arviv preprinit arXit:2309.13199, 2023.
12

---

Published as a conference paper at COLM 2024
Guohao Li, Hasan Abed Al Kader Hammoud, Hani Itani, Dmitrii Khizbullin, and Bernard Ghanem. CAMEL:
Communicative agents for "mind" exploration of large language model society. In Proceedings of Thirty-seoenth Conference Neural Information Processing Systems, 2023. URL httpps://openreviev.. fttp://tt//oooomm//d11.y/2.2
Tian Liang, Zhiwei He, Wenxiang Jiao, Xing Wang, Yan Wang, Rui Wang, Yujiu Yang, Zhaopeng Tu, and Shuming Shi. Encouraging divergent thinking in large language models through multi-agent debate. ar preprint ar arXi::2305.19118, 2023.
Qing Liu, Tie Luo, Ruiming Tang, and Stéphane Bressan. An efficient and truthful pricing mechanism for team formation in crowdsourcing markets. In 2015 IEEE International Conference on Communications (ICC), Pp. 567-572, 2015.
Xiao Liu, Hao Yu, Hanchen Zhang, Yifan Xu, Xuanyu Lei, Hanyu Lai, Yu Gu, Hangliang Ding, Kaiwen Men, Kejuan Yang, Shudan Zhang, Xiang Deng, Aohan Zeng, Zhengxiao Du, Chenhui Zhang, Sheng Shen, Tianjun Zhang, Yu Su, Huan Sun, Minlie Huang, Yuxiao Dong, and Jie Tang. Agentbench: Evaluating LLMs as agents. In The Twelfth International Conference on Learning Representations, 2024. URL https://openrevieee www.teeuunniiiiie zAdUBBaCTQ
Zhiwei Liu, Weiran Yao, Jianguo Zhang, Le Xue, Shelby Heinecke, Rithesh Murthy, Yihao Feng, Zeyuan Chen, Juan Carlos Niebles, Devansh Arpit, Ran Xu, Phil Mui, Huan Wang, Caiming Xiong, and Silvio Savarese. Bolaa: Benchmarking and orchestrating llm-augmented autonomous agents. arXiv preprint (ttt:2300.0556)) 2023.
Pan Lu, Baolin Peng, Hao Cheng, Michel Galley, Kai-Wei Chang, Ying Nian Wu, Song- Chun Zhu, and Jianfeng Gao. Chameleon: Plug-and-play compositional reasoning with large language models. In Proceedings of Thirth-seventh Conference on Neural Information Processing Systems, 2023. /ttp::tss:///penntt wttp://wwwwee....mmmm///tttt/o/.
Scott M. Lundberg and Su-In Lee. A unified approach to interpreting model predictions. In Proceedings of the 31st International Conference on Neural Information Processing Sys- tems, NIPS'17, pp. 4768-4777 Red Hook, NY, USA, 2017. Curran Associates Inc. ISBN 9781510860964.
Ioanna Lykourentzou, Federica Lucia Vinella, Faez Ahmed, Costas Papastathis, Konstanti- nos Papangelis, Vassilis- Khan, and Judith Masthoff. Self-organization in online collaborative work settings. Collective Intelligence, 1(1), sep 2022.
Kaixin Ma, Hongming Zhang, Hongwei Wang, Xiaoman Pan, and Dong Yu. Laser: Llm agent with state-space exploration for web navigation. ar preprint ar axXio:2399008172 2023.
Yohei Nakajima. Babyagi.
2023.
Nathalia Nascimento, Paulo Alencar, and Donald Cowan. GPT-in-the-LLoop Adaptive Decision-Making for Multiagent Systems. ar preprint arXim:2308.10433, 2023.
Xuefei Ning, Zinan Lin, Zixuan Zhou, Zifu Wang, Huazhong Yang, and Yu Wang. Skeleton- of-thought: Large language models can do parallel decoding. In The Twelfth International Conference on Learning Representations, 2024. URL
mqVgBbbCC99]
OpenAI. Gpt-4 technical report. arXiv preprint arXit:2303.08777, 2023.
Matt Post. call for clarity in reporting BLEU scores. In Proceedings of the Third Conference on Machine Translation: Research Papers, pp. 186-191, Belgium, Brussels, October 2018,
Association for Computational Linguistics. URL. https:::p//. http:/wwwwwwwww...../////./ org/ orgg/nthology.
W18-6319
Chen Qian, Xin Cong, Wei Liu, Cheng Yang, Weize Chen, Yusheng Su, Yufan Dang, Jiahao Li, Juyuan Xu, Dahai Li, Zhiyuan Liu, and Maosong Sun. Communicative agents for software development. arxiv preprint ar arXio:2307.0099 2023a.
13

---

Published as a conference paper at COLM 2024
Chen Qian, Yufan Dang, Jiahao Li, Wei Liu, Weize Chen, Cheng Yang, Zhiyuan Liu, and Maosong Sun. Experiential co-learning of software-developing agents. arxiv preprint (rXXi:2312.17025, 2023b.
Shuofei Qiao, Ningyu Zhang, Runnan Fang, Yujie Luo, Wangchunshu Zhou, Yuchen Eleanor Jiang, Chengfei Lv, and Huajun Chen. AUTOACT: Automatic Agent Learning from Scratch via Self-Planning. ar Kiv preprint arXio:2401.05268, 2024.
Zhen Qin, Rolf Jagerman, Kai Hui, Honglei Zhuang, Junru Wu, Jiaming Shen, Tianqi Liu, Jialu Liu, Donald Metzler, Xuanhui Wang, and Michael Bendersky. Large language models are effective text rankers with pairwise ranking prompting. ar Xie preprint arXiv:2306.1756, 2023.
Shuo Ren, Daya Guo, Shuai Lu, Long Zhou, Shujie Liu, Duyu Tang, M. Zhou, Ambrosio Blanco, and Shuai Ma. Codebleu: method for automatic evaluation of code synthesis. Xiv preprint arxie:2009.10297, 2020.
Reworkd. Agentgpt.
2023.
Toran Bruce Richards and et al. Auto-gpt: An autonomous gpt-4 experiment. https //github. ctwp://iiiiiiiiatttrrrtttttaaass
2023.
Jingqing Ruan, YiHong Chen, Bin Zhang, Zhiwei Xu, Tianpeng Bao, du qing, shi shiwei, Hangyu Mao, Xingyu Zeng, and Rui Zhao. TPTU: Task planning and tool usage of large language model-based Al agents. In NeurlPS 2023 Foundation Models for Decision Making Workshop, 2023. URL
net www..et/oorrmiiiofrrrrtjtaalll
David E Rumelhart, Geoffrey E Hinton, and Ronald J Williams. Learning representations by back-propagating errors. nature, 323(6088) 1986.
Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik R Narasimhan, and Shunyu Yao. Reflexion: language agents with verbal reinforcement learning. In Thirth-seventh Conference on Neural Information Processing Systems, 2023. URL https:://operreeiee net/
(Formm3jdxvAEIhfcckW
Mirac Suzgun and Adam Tauman Kalai. Meta-Prompting: Enhancing Language Models with Tano-Agn Scaffolding. ar Xrv preprint ar (rXiv:2401.12954, 2024.
Guanzhi Wang, Yuqi Xie, Yunfan Jiang, Ajay Mandlekar, Chaowei Xiao, Yuke Zhu, Linxi Fan, and Anima Anandkumar. Voyager: An open-ended embodied agent with large language models. In Second Ageni Learning in Open-Endedness Workshop, 2023a. URL https://openreviies nett/forum?id=pAMNWGG..
Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc V Le, Ed H. Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. Self-consistency improves chain of thought reasoning in language models. In The Eleventh International Conference on Learning Repre- sentations, 2023b. /tt/httt:::/pperrviiii wwtp:eorommdddd1
Zhenhailong Wang, Shaoguang Mao, Wenshan Wu, Tao Ge, Furu Wei, and Heng Ji. Un- leashing cognitive synergy in large language models: A task-solving agent through multi-persona self-collaboration. arXiv preprint ar arXiv:2307.00 200:0000.,05000, 2023c.
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, brian ichter, Fei Xia, Ed H. Chi, Quoc V Le, and Denny Zhou. Chain of thought prompting elicits reasoning in large language models. In Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho (eds.), (eds)), Advances in Neural Information Processing Systems, 2022. URL https://openreview (ittforruumidd..v00Mee.....
Qingyun Wu, Gagan Bansal, Jieyu Zhang, Yiran Wu, Shaokun Zhang, Erkang Zhu, Beibin Li, Li Jiang, Xiaoyun Zhang, and Chi Wang. Autogen: Enabling next-gen Ilm applications via multi-agent conversation framework. arxiv preprint arXiv:2308.081 08155,, 2023.
14

---

Published as a conference paper at COLM 2024
Fen Xia, Yan Liu, Jue Wang, Wensheng Zhang, and Hang Li. Listwise approach to learning to rank: Theory and algorithm. In Proceedings of the 25th International Conference on Machine Learning, ICML '08, pp. 1192-1199, New York, NY, USA, 2008. Association for Computing Machinery. ISBN 9781605582054.
Kai Xiong, Xiao Ding, Yixin Cao, Ting Liu, and Bing Qin. Examining inter-consistency of large language models collaboration: An in-depth analysis via debate. In Houda Bouamor, Juan Pino, and Kalika Bali (eds.), Findings of the Association for Computational Linguistics: EMNLP 2023, pp. 7572-7590, Singapore, December 2023. Association for Computational Linguistics. doi: 10.18653/ w02.022..nniinggseeeeeeee findings--emllp550s URL https
//aclanthology.org wtt.e/..rrg2222........ wwwdiiiinnsnsell.....
Miao Xiong, Zhiyuan Hu, Xinyang Lu, YIFEI LI, Jie Fu, Junxian He, and Bryan Hooi. Can LLMs express their uncertainty? an empirical evaluation of confidence elicitation in LLMs. In The Twelfth International Conference on Learning Representations, 2024. URL
Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik R Narasimhan, and
Yuan Cao. React: Synergizing reasoning and acting in language models. In Proceedings of The Eleventh International Conference Learning Representations, 2023. URL https
boppenreview.net/oorumii VIYYUU-U
Zhangyue Yin, Qiushi Sun, Cheng Chang, Qipeng Guo, Junqi Dai, Xuanjing Huang, and Xipeng Qiu. Exchange-of-thought Enhancing large language model capabilities through cross-model communication. In Houda Bouamor, Juan Pino, and Kalika Bali (eds.), Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pp. 15135-15153, Singapore, December 2023. Association for Computational Linguistics. doi: 10.18653/v1/ http///w//222/mmme/// comnl--main 936.
http://haaaaaaaoog......o org/ 2023. eenn..mnlpp.......mmmmmm emnlp-mail
936
Ruichi Yu, Ang Li, Chun-Fu Chen, Jui-Hsin Lai, Vlad I. Morariu, Xintong Han, Mingfei Gao, Ching- Yung Lin, and Larry S. Davis. Nisp: Pruning networks using neuron importance score propagation. In 2018 TEEE/CV Conference on Computer Vision and Pattern Recognition, pp. 9194-9203, 2018.
Jintian Zhang, Xin Xu, and Shumin Deng, Exploring Collaboration Mechanisms for LLM Agents: A Social Psychology View. ar arXio preprint ar 2rXt::2310.02124, 2023a.
Xinghua Zhang, Bowen Yu, Haiyang Yu, Yangyu Lv, Tingwen Liu, Fei Huang, Hongbo Xu, and Yongbin Li. Wider and Deeper LLM Networks are Fairer LLM Evaluators. arXiv preprint ar Xiv::2300 01866, 2023b.
Yifan Zhang, Jingqin Yang, Yang Yuan, and Andrew Chi-Chih Yao. Cumulative reasoning with large language models. arXiv preprint ar (tt::238.00377 2023.
Chuanyang Zheng, Zhengying Liu, Enze Xie, Zhenguo Li, and Yu Li. Progressive-hint prompting improves reasoning in large language models. arXiv preprint arXio:204400997,, 2023.
Andy Zhou, Kai Yan, Michal Shlapentokh- Rothhommnhh-RRaan,, Haohan Wang, and Yu-Xiong Wang. Language agent tree search unifies reasoning acting and planning in language models. ar Xiu preprint ar xrXiv:2310.04406, 2023.
Xuanhe Zhou, Guoliang Li, and Zhiyuan Liu. LLM As DBA. arXio preprint arXiv:2308.05481 2023.
Xizhou Zhu, Yuntao Chen, Hao Tian, Chenxin Tao, Weijie Su, Chenyu Yang, Gao Huang, Bin Li, Lewei Lu, Xiaogang Wang, Yu Qiao, Zhaoxiang Zhang, and Jifeng Dai. Ghost in the minecraft: Generally capable agents for open-world environments via large language models with text-based knowledge and memory. arXiv preprint ar Xriv:2305..17144 2023.
Mingchen Zhuge, Wenyi Wang, Louis Kirsch, Francesco Faccio, Dmitri Khizbullin, and Jürgen Schmidhuber. Language agents as optimizable graphs. arXiv preprint XxXi::2402.16823, 2024.
15

---

Published as a conference paper at COLM 2024
A Discussion & Limitation
In experiments, we view code generation tasks as representative of open-ended generation tasks and adopt BLEU to decide whether two answers are consistent in early stopping mechanism in Section 3.3.2 In fact, the performance could be further leveraged by task- specific methods like CodeBLEU (Ren et al. 2020) or CodeT (Chen et al. 2023a
For practical usage, the agent-evaluation metrics could cooperate with human annotation to give a more precise evaluation result on individual contributions of agents, mainly when facing data scarcity problems. Furthermore, we simply incorporating agent selection on Dy- LAN with agent team reformation, as primary step towards collaboration of dynamic agent teams. It still remains to be seen how cooperate off-collaboration and in-collaboration optimization methods in a finer granularity to further improve performance and efficiency in -LW-eeweredd agent collaboration systems.
Additionally, though agent selection could differentiate top contributory agents, in extreme cases where the majority of agents are designed to contradict the task requirement, low performance might be caused, e.g., agents are prompted or trained to generate codes may face difficulties in clinical question answering. To tackle the imbalance of high- and low- performing agents, replicating agents with high Agent Importance Score instead of including low-score agents could a solution. Additionally, in extreme circumstances, we can automatically introduce agents from more capable LLMs with validation, in addition to agent selection.
B Implementation Details
B.1 Detailed Experiment Settings
Algorithm 1 The Inference Process of DyLAN on Algorithm The Team Optimization Arbitrary Query
Process foptim of Agent Importance
Score within DyLAN
Input: T-FFN g = (Vi.... VT: Er. Fr 1,T),
Query
Input:
Output T-FFN G (V, E) Output: Agent Importance Score of agents
Output: Final Answer
// E
T-1 11-1) V
// U._1 V/.
// m,,i E M, denotes the response
// m1,5 &Me denotes the response from UY E Vt-
from 11 E V/.
flag + False
for T; do
for tor 1;T do if agent team reformation time step & then
if { f,,, v1, E E} +
Mtop + (o-k(((m; ( 1 E Vi-13) E <- E((("r x), Dr lmp E Maa,,'
then
if -flag then flag + True distribute scores for 4,,1 else
f-13 13 MM, f- m-jijj (V. Dt,)) E
else
Mi 1 1 [m,
Vi,3k, (vi,i, 0141,)) E E1,41,
D(s) E E}
Mij + Fmp (m,- 1 (v, 1, vti) P.)
[w: 1,1-,,, W0 1 t
fti (i,g,MM-1)
((8)
end if
if early stopping then T
1, 1i - I 1 1, 1ijr
3 U1j)) E
break
end if
end
end
end for
end for
f (ostProces/maxCoum((M11)
16

---

Published as a conference paper at COLM 2024
Common Settings In all experiments, use gpt-35-tubo-038 for every method if
not specified. The version of GPT-4 is GPT-4-0613. 3. In Table 2 "(Codex)" denotes code-davinci-002 from OpenAI (Chen et al. 2021 OpenAI 2023 ) All experiments with non-zero temperature is repeated for three times and the median is reported. To avoid the context length issue in prior work (Du et al. 2023 Liu et al. 20244,, we set memory space for agents in LyLAN to 1 only to keep the treshest responses of predecessors. We set max tokens to 2048 for GR and ÁR tasks and 1024 for CG and DM tasks to avoid ex- ceeding the maximum context length. The construction of candidates are demonstrated in Appendix
D We set N 4 in I-FFN after team optimization because the early-stopping mechanism requires at least four agents to tolerate one different response at a specific time step (Section (.3.2) when reaching consensus over 2/3 of agents, it allows for 4 - N = =1 excetional response. We use a listwise ranker in the agent team reformation of DyLAN because of the effectiveness and efficiency, compared to ELo rating (Herbrich et al. 2006) or Sliding Window (Qin et al. 2023) we have tested in Appendix C.5 We use the same ranker to implement LLM-Blender (lang et al., 2023) in experiments. We set k 2 in the agent team reformation, because it's the minimal number for collaborations and we empirically found it brings great trade-off between effectiveness and efficiency. To avoid positional bias, for each time step t, we shuffle the responses from agents at f - when passing messages towards agents
The detailed inference algorithm is in Algorithm early-stopping mechanism, we need to determine whether the answers from the nodes in the same layer of DyLAN are consistent. For classification and decision-making problems, the answers are consistent if identical, and for open-ended generation, the consistency is determined by a threshold of BLEU score.
[]] To implement the
Experiments on Reasoning Tasks In general reasoning, we extract the answer from the response by matching the last "(X" or "(X)", where "X" represents A, B, C or D. On average, Agent team reformation functions on the third time step. They could go through at maximum T = 4 rounds of interaction. We also searched temperature in {0, 0.2, 0.8, 1.0} for the best configuration for each system. In arithmetic reasoning, we set temperature to O for the single execution and PHP, 0.2 for LLM Debate, LLM-Blender, and DyLAN with Complex CoT prompts, and 1.0 for DyLAN with simple CoT prompts in Table 3 since systems with the same prompts will give all the same responses if temperature is zero, causing degradation. Prompting templates are replicated from their original studies, including normal CoT prompts (Wei al. 2022) from the MATH dataset (Hendrycks et al., 2021b) and Complex CoT from PHP (Zheng et al. 2023). We follow the answer extraction method from the origin paper (Hendrycks et al. 2021b). We construct DyLAN with 4 agents assigned no specific roles and let agents to interact for at maximum f = 4 rounds under T-FFN formulation. We reported the classification accuracy of each category av averaged across subjects and the numbers of API calls of running DyLAN on the optimized team of agents.
Experiments on Code Generation Tasks In the code generation task, we set temperature to 0 for the single execution, Reflexion, and 0.8 for LLM Debate, LLM-Blender, CodeT, and DyLAN in T In DyLAN, we optimized four agents to write code and four agents to give code reviews from 12 candidates in Appendix The selected seleceee code writers are "Python Assistant", "Algorithm Developer", "Computer Scientist", and "Programmer", and the selected code reviewers are "Syntax Checker", "Unit Tester", "Reflector", and "Ranker". "Syntax Checker" " is pure external tools using a code interpreter for syntax checking without LLMs, and "Unit Tester" is equipped with code interpreter. The tool is triggered when LLM generated codes inside the format
An" " In DyLAN, solutions given by code writers are reviewed by code reviewers in at maximum T rounds. At time step 1 = 1,3,4, 6, code writers gives solutions and code reviewers review it at = 2,5. And agent team reformation occurs at 4. To ensure the participation of each agent, early- stopping mechanism functions at the third layer and later (f > 3). We use BLEU score in the early-stopping mechanism. We calculate BLEU by sacreBLEu (Post 2018). For answer post-processing, we store all unit tests from the unit tester (if exists in the system) and randomly select the final output from the top 5 code completions from all nodes that pass most tests.
python''n
2The signature of saCreBI.EU is "nrefs: /case:mixed /effreo besec/aassmootheep/eeerriso
17

---

Published as a conference paper at COLM 2024
Experiments on Decision Making Tasks In the decision-making task, we set temperature
to 0 for DyLAN and all baselines in Table 2 For ReAct with self-consistency (denoted by
ReAct-SC in the table) (Wang et al., 2023b), we sampled three times for each response. In
DyLAN, we optimized four agents from 8 candidates which are depicted in Appendix D
All methods are also conducted on not-35-ttubo-196 We did not select LASER (Ma et al.
2023) as a baseline, because it requires GPT-4 for better performance and it extracts all valid
actions in each page into function calls, instead of detected by agent itself, which we decide to be a different setting. We divide the pages of the WebShop environment into 3 parts: the initialization page for "searching" part, the item list page for "exploring" part, and the item details pages for "item" part. Thus, we managed to optimize teams for each part from agents in Appendix D "Search Optimizer", "Budget Analyst", "Instruction Analyst", "Decision Reflector" for "sarchinng" group, "Decision Maker", "Budget Analyst", "aroys" PProduct Explorer", "InstructionAnallsst" for "exploring" group, and "Budget Analyst", "Description Reader", "Decision Maker", "Result Estimater" for "item" group. Agents interact for at maximum = 4 steps for each action. We simply concatenate observations of previous actions on each decision. For answer post-processing, we skip invalid actions from the outputs of Vr.
B.2 Calculation of Agent Importance Score
To implement the agent selection algorithm under DyLAN, only one sentence needs to be injected into the end of the prompt of each node in T-FFN: "Along with the answer, give score ranging from 1 to to the solutions of other agents. Put all {nump] scores in the form like [[1, 5, 2, II"",, where nump denotes the number of predecessors
of the node. The prompt functions as the f(6 in Section 3.4 and we extract wai from its response at the same time when we extract the message that passes between nodes. The scores are normalized so that their sum (Σ,11 Whii)) equals 1. To avoid positional bias, responses from agents at previous time step are shuffled when rating.
In Algorithm 2 initial contributions are distributed on nodes the last layer. For reason- ing and dicision dicision-making tasks, we uniformly distribute contributions to agents that give consistent answers in the last layer. On code generation tasks, we uniformly distribute contributions in the final round with no syntax error in their answers.
C Additional Results
In this section, detailed results and additional experiments are presented.
C.1 Data Efficiency of Team Optimization
We further demonstrate the data effi- ciency of agent selection by performing it based on different amounts of data. The experiments are conducted on five subjects in the GR task (the same as Ta- 7) and the CG task. We sample the subsets with the proportions of 1% and 10% of the original dataset. Agent Ii- portance Score for agent selection is av- eraged on the subsets, and the selected team tested on the whole dataset. We raise random selection and human prior selection as baselines. The latter is sim- ulated by GPT-4 prompted by the task and agent descriptions (Appendix D). As shown in Table 9 by optimizing the team 10% of the original dataset, DyLAN has demonstrated similar performance compared to using the whole dataset, with only 0.2 loss on GR and 0.6 loss on CG. We can observe that even with only 1% of the original dataset,
Indicator
Dataset
GR
CG
NA
63.5
76.2
Random
64.8
75.6
Human Prior
66.7
78.0
1%
68.9
79.3
Agent
10%
72.2
82.3
Imp. Score
100%
73.6
82.9
Table 9: Experimental results of different indica- tors used in agent selection during team optimiza- tion in DyLAN on five subjects in the GR and CG tasks. "Dataset" denotes the proportion of dataset used in team optimization.
18

---

Published as a conference paper at COLM 2024
DyLAN could obtain significant improvement of +3.7 over random selection CG. From observation, agents augmented with tools are always selected during team optimization under different proportions of the dataset, indicating the effectivenes: of Agent Importance Score as an indicator. Please refer to Appendix C.3 for a detailed analysis of the human priors
C.2 Robustness of different foundation models in DyLAN
Besides using GPT-3.5 for DyLAN on CG tasks
Method
Pass@1
#API Calls
in Table 2 we also experiment with GPT-4 in
Table 10 Due to budget limits, we directly reuse the performance reported in the paper of base- lines, including LATS (Zhou et al. 2023), Reflex- ion (Shinn al. 2023), Meta-GPT (Hong al., 2024), and Agent Verse (Chen et al. 2024), and es- timate the cost in terms numbers API calls.
Single-Agent Methods
Single Execution
88.4
(+0.0)
1,00
LATS Reflexion
94.4
(+6.0)
>40.0
91.4
(+3.0)
7.32
Multi-Agent Methods
Meta-GPT AgentVerse DyLAN (Ours)
85.9
(+3.5)
>30.00
DyLAN is also constructed by agents which are optimized based on GPT-3.5, as demonstrated in Appendix B.1 We found that DyLAN consis- tently outperforms other multi-agent methods, indicating the effectiveness of dynamic agent team in T-FFN structure and the cross-model transferability of optimization results on agent teams. Although LATS outperforms DyLAN, it requires over 40 times GPT-4 calls per sample to conduct inference-time MCTS on GPT-4, which demonstrates poor efficiency.
89.0
(+0.6)
27.00
92.1
(+3.7)
15.94
Table 10: Experimental results on the CG task on GPT-4-061 The number in parentheses indicates the difference rela- tive to the single execution or direct exe- cution. The bold font denotes the results of our method and the best results derlined.
C.3 Human Priors and Agent Importance Scores
We further investigated how these agents selected by our unsupervised metric Agent Impor- lance Score differ from human priors (e-g., these predefined roles). To do so, we calculated agent importance scores for 7 agents
each subject of the MMLU dataset. As example, we show the subjects where the agent of "Doctor" and "Programmer" " has the highest agent
importance score among all agents in Table 11 and Table 12
Though most subjects seems to be reasonably aligned with the role of the agent based human priors (with green annotations), there are some subjects that do not match hu- man priors, e.g., High School Computer Science as the sub- ject that "Doctor" has the high- est score. exhibits the dif- ference between human pri- ors and the evaluation results of agent importance scores on agents with human-made or L.L.--geerteed prompts.
Role
Doctor
Programmer
high school computer science clinical knowledge college biology
high school physics
electrical engineering
high school government and politics college computer science
Top 10 Sub- jects
professional medicine
nutrition
college chemistry
high school US history
high school mathematics
human aging anatomy
formal logic
abstract algebra
high school biology
machine learning
high school psychology
computer security
Table 11: Subjects on which agents have the top-ranked Agent Importance Score in the experiment with DyLAN of agents on the GR task. Green annotation denotes the fields related to the role from the human perspective, which are annotated manually.
19

---

Published as a conference paper at COLM 2024
Role
Mathemtticcian
Lawyer
Historian
Economist
Psychologisi
college physies
high school microeconomics
foreegn policy
high school science
global facts public relations
US fursign pollcy
medioal genatics
aoceomettics
jurägnaaence
college comp science
podbisszey
World seligions
Ingical fallacies
busines ethies
Sub-
Top 10
jects
highschoo school matberaties
abstract algebra
econometrics
marketing
hummn siging
management
formal logic
wotelogy
high school government and pulitics
public relations
philosophy
astronomy
high school Eamppan histoey
high achood micnocommmiis
profeesional acconnting
computer security
high school US histury
philowophy moral disputes
management
international
world seligionn
high adhool statieiks
moxal dispuiuss
profosiioal ascounting
jurtaprudence
machiine Joarning
prodessional
arsuality
intersatonaal low
Reigh school Furopean hnstory
college matheratics
Table 12: Subjects on which agents have the top-ranked Agent Importance Score in the same experiment in Table [1 the human perspective.
Green annotation denotes the fields highly related to the role from
We also compare current agent selection method that is implemented with Agent Importance Score with the implementation with Human Prior Selection on few subjects in the MMLU and the HumanEval datasets. For Human Prior Selection, we setup GPT-4 mimicking human selecting the agents for collaborations based on the description of the task and role prompts of each agent. We provide prompt templates in Appendix D As shown in Table [3] the implementation with Agent Importance Score steadily outperforms Human Prior Selection. There are two major reasons: (1) Compared to posterior optimization methods, prior selection may not grasp the actual behaviors of agents, and may not understand which agents are most contributory helpful to others in the real collaboration process. Thus, in High School Statistics, Clinical Knowledge, and Public Relations subjects in the MMLU dataset, prior selection performs even worse than random selection. (2) Human Prior Selection might struggle to understand tool augmentation without peer ratings from fellow agents. From our observation, "Unit Tester" and "Syntax Checker" were not selected for code generation tasks, which may cause lower performance.
#Agents
Optimization
College Mathematics
Management
High School Statistics
Clinical Knowledge
Public Relations
Overall
Indicator
7
(before optimization)
40.0
76.2
65.1
69.8
54.5
63.5 (+0.0)
Random Selection
45.0
71.4
67.4
71.7
54.5
64.8 (+13))
4
Haman Prior Selection
60.0
80.1
65.1
69.8
54.5
66.7 (+3.2)
Agent Importance Score
65.0
90.5
74.4
75.5
59.1
73.6 (+10.1)
#Agents
Optimization Indicator
Pass@1
#API Calls
12
(before optimization)
76.2 (+0.0)
23.04
Random Selection
75.6 (-0.6)
17.73
Human Prior Selection
78.0 (+1.8)
16.37
Agent Importance Score
82.9 (+6.7)
16.85
Table 13: Detailed performance of different indicators of agent selection on five subjects in GR tasks (top) and the CG task (bottom). The five subjects in GR tasks and other settings are identical to Table The overall accuracy in the top table denotes the accuracy across the five subjects.
C.4 Stability of DyLAN on Temperature
We tested few methods on the AR (with simple CoT prompts) and the CG tasks under both low and high temperatures and repeated each experiment three times when the temperature was not zero. We exhibit the experimental results in Figure 4 From experimental results, we found that DyLAN is more stable on different hyper-parane parameters.
Experiments show that temperature greatly influences arithmetic reasoning and code gener- ation tasks. In Figure we found that most baseline methods have significant performance drops when temperature increases, but DyLAN shows strong robustness to various tem- peratures. We surprisingly found that DyLAN gets better results when temperature rises, suggesting it has benefited from diversity instead of being disturbed by low-quality answers of high-temperature agents. The agent team reformation may lead to the higher accuracy by keeping best responses when agents' replies become more diverse. In conclusion, the collaboration of different roles functions effectively and robustly in the dynamic architecture.
20

---

Published as a conference paper at COLM 2024
36
85
Temperature-0.2
Temperature-O.J Temperature-O..
Temperature-1.0
Accurary
32
Pass(a
75-
30
70
26
60
DyLAN (Ours)
Single
Debate
Blender
DyLAN (Ours)
Single
Reflexion
Figure 4: Performance of different methods under low and high temperatures on AR (left) and CG (right) tasks. DyLAN shows better robustness to different temperature and even takes ad advantage of higher temperature.
Nonetheless, higher temperature requires DyLAN to take more API calls (about +0.98 on average on AR tasks (temperature: 0.2 + 1.0)).
C,5 Different Ranking Methods
Ranking Method
Overall Accuracy
#API
We also tested different ranking methods for agent team reformation of DyLAN on the GR task. We tested listwise ranker with our own prompts, pairwise GPT ranker from original LLM-Blender (Jiang et al. 2023), Elo Score from TrueSkill (Herbrich et al. 2006) also imple- mented with pairwise ranker, and pairwise ranker with Sliding Window algorithm (Qin et al. 2023). In Table 14] we show that different ranking methods have a relatively low impact performance, probably because of strong discrimination ability of GPT-3. 5, but pairwise ranking methods always consume higher computational cost. Thus, chose listwise ranker in implementation of DyLAN.
Calls
Listwise Ranker
70.5
4.39
M-
70.1
19.27
Pairwise
Score
70.3
19.55
Sliding Window
70.3
11.40
Table 14: Overall accuracy (%) DyLAN with different ranking method in the agent team reformation on the GR task. Other settings are identical with Table 4
C.6 Does Agent Importance Score Captures Actual Contributions?
Shapley Value is a widely used supervised metric for evaluating contribution of a single agent in multi-agent system. Though it is not suitable for unsupervised Team Optimization, by viewing it as ground-truth metric for measuring individual contributions, we can use it for validating the Agent Importance Score. We implement simplified algorithm for LLM- powered agent collaboration systems. Given that the collaboration process is symmetric in the formulation of the temporal feed-forward network (Section 3.2). we could reduce the permutation set in the original formula [Lnddbegg & Lee 2017) to the combination set:
S;(R) C
Σ TEC
(Performance (i)) Performance (T))
(13)
where R is the set of agents in the system, C is the combination set of R R E R, and Performance denotes the overall performance of the system on the current task, e.g., classification accuracy or Pass@1. The metric requires ground truth and multi-pass results of the system with different subsets of agents. We use classification accuracy for classification tasks and Pass@1 for code generation tasks. However, its computation cost is still too high when the number of agents grows larger due its combinatorial complexity.
To examine Agent Importance Score an indicator of agent selection with Shapley Value, we also randomly chose three combinations of three agents out of all 7 candidates to assemble T-FFN and calculated the Shapley Value and the Agent Importance Score GR tasks. In the GR task, The roles of candidates in DyLAN match the categories of MMLU in human priors, including "Mathematician" and "Programmer" for STEM, "Lawyer" and "Historian" for
21

---

Published as a conference paper at COLM 2024
Humanities, "Economist" and "Psychologist" in Social Science, and "Doctor" for clinical questions in the "Other" category. During experiment with a T-FFN with at least one agent matches the category of the question, it is called In-Domain scenario; vice versa.
In Appendix C.6 we report the correlations between Shapley Values and Agent Importance Scores. We are curious whether Agent Importance Score is an unsupervised substitution for Shapley Value. So, we calculated two list-wise metrics for the similarity between distributions:
the KL divergence and ListMLE (Xia al. 2008), between Agent Importance Scores and Shapley Value. It indeed shows a high correlation between the distributions of the two metrics during in-domain scenarios.
In summary, we use Shapley Value as a self-evident metric for measuring individual contribu- tion, showing that Agent Importance Score emerges promising, unsupervised alternative with light computational complexity.
In- nn-Domaiin
Off-Domain
Metric
DKL
ListMLE
DKL
LisiMLE
Shapley Value
0
0.673
0
0.674
Agent Importance Score
0.229 x 10 3
0.686
0.347 x 10-3
0.693
Uniform Distribution
0.359 x 10-3
0.693
0.327 x 10-3
0.693
Table 15: Correlation between different metrics for quantifying agents' contributions in DyLAN on the GR task. We compute the KL divergence (Dkl) and the ListMLE loss (LLisMMLE) between Shapley Value and other metrics on each subject and report the average value. The In-Domain column means at least one agent in DyLAN matches the category of the subject according to Appendix C.6 and Off-Domain means none of agents matches the subject.
C.7
Case Study
In BigureBannd Figure 6 we demonstrate the cases of DyLAN on the code generation and the general reasoning tasks, respectively. First, we notice that the communication structure different between figures, exhibiting dynamic architecture of DyLAN on different queries. The former gives answer at t 4, the latter at t 2. We also notice that the answer is gradually growing better along the temporal axis. In Figure 6 the "Mathematician" agent selected for the arithmatic query and it gives correct answer while convincing other agents, agent selection method is effective. We can also observe that the distribution of Agent Importance Score is reasonable. Also, instructing LLM agents to rate scores on predecessors hints them to reflect on predecessors' responses, which might be helpful to give better answers. Last but not least, agents with different roles lead diverse conversation and make full use of each one, which benefits performance and robustness.
Last but not least, we provide qualitative analysis on a failure case in Figure 7
22

---

Published as a conference paper at COLM 2024
V
Programmmer "python
T Answer (Early Stopping)
T Query
Agent Team Reformation
pi Examples:
(p-13 while n- if am
def modolrc int, int]: ""Returm 2
modulo Examples:
Although A3 considered edge cases, but A1 and
Please follow the function and completo implementation Python function. Reply the complition a code block.
A2
Alponilan Duveloper
afficiancy
return pow[2, p)
effeciant 'nurpyy ""ythonn return
Top2: 11, 2]
It's more efficient and readable Python's pow function.
powee/2
(nocment
Fiad Anooee
(1000,
Teen
Al: A2
Syntox
Preseed
Falledt NameEroor name In. hummy not delined
Pregrammet
Domputer Sclentist python return pow
Coce
Apologize vensen consadeetng casce when P' equals python
Corerater
A18A2 The code is clear
Anower
and
command inc Python
p)
lip -- sdiee caew ecuats
mcrrptt below
consicers
"yython
importt rumpy
Python Assistant
AR: Developer to import the untamal package humpy Ore tin should sp
powert2
base 2% while n
python
numptt as
(comect)
(incorect)
pow(2. n.
@V
KG V4
3 V
Figure 5: A case of DyLAN solving code generation task. Different agents are recruited to write code and give feedback. At the time steps 2,5 code reviewers are asked to provide code reviews. The result grows better layer by layer regarding correctness, efficiency, and readability. Different directions implementation are delivered forward in implicit multiple paths. We ignore the peer rating scores in multi-round responses for computing Agent Imporlance Scores due space limits.
23

---

Published as a conference paper at COLM 2024
- message passing 1/2
Vi
cuestion (Quer)
@ V22
Asthematician
from
problem
Muthomatioon
the problem. there are studenss remaining the
Let's start by working backowere from the end of
Agent Selitton Iraa
Agent Solution Iny)
whyt Therefore, answer Agent solution connect: if--o their solution incorsect Score: Agent solution dt Soore
Agentt seens
Let's
Agent Edlution Ihal
we
of students (16)
Aguen
can you give Pi.al
solution
answer?
before half
library So the answer (2)52 stushents
to give other apants---. the Torn [1 51
Agent solution 4: This agent asknowledges not leing shie mnsewer the queetton Scave [S, 2
Programmer
solive problem,
wore part of the group that did not go That woold be i it Therefore, the answor is (C) 24 soudents.
halt the sturentts
leb,
To Progrmmer from the end -h Theretore, the anower (0) studentts. As for other apenta sclutions. Agent solution Score Agent
baolwwadd
Soore B--rrhere unoo
Score 2o-nworking backwards
hut then
Tharefone, requiinee
Agent solution NA applicabred [. 2,3,2,00
remaiing the and the pwriod that Therefone if coudents remained after half the classe eentt to the llnury, thei meses the origredd number students the dloss was double that. atudennta
Pormomier
anho this poobi need -eettttttt total
students.
Doctor
solution Suatre S--. Agent oolittion Soore 1. This colution
apploogite, but as doctor. am not qual anoove math profilema as apourstaly as possiibl.. Howerer recommend in assistance dance
incorrect and doea not correclyy work core Agent solition Soore
and socurste- they give good
solution
math nome of the
Final Answer (Farly Stopping)
probllm Thaewloon,,
Ovcrall
2.
cholccs provided.
(D)
solve the problem, -.. Therefon,, the number of studerts in the cless before half of trem went the computar- -o. the answer
half students in class the libary. Later pariod, half of the nemen ming at students go to computer the Inb there ara students remairing the claso, how many shudents originally by in the clese??
"At
wasss too wa.e
(1) Scliition by other agents Agont solation Soore
However
wa nue biis wilt
lotiioon
insonwelly asoumeed which stated Agem solution & Soore 2- 2---Howwver their methed doubled muteed of
cousents
(0)32 coudents
studients
a Agent
Agent ootution è Soore This agent was provide solution
(1 Query
Importance Scores
Figure 6: A case of DyLAN solving general reasoning task. Different agents recruited to give and refine solutions. The result incorrect at the first time step but correct at the second time step. It includes the ratings from agents for calculating Agent Importance Scores.
24

---

Published as a conference paper at COLM 2024
V
@ Finel Answer (Early Stopping)
( Query
Alporitim Daveloper
K Optimization Directions
def minnathiggnn gnd, Int):
directions NG. 11 (1. 01
pysiton
len(grid) [(0. up. right, down min_path [N*%=1]*
(1,
Reflector
You have to find minimum path length
Although write afficient algorithm, the not condensed...
kinn the grid.
while heap:
def die6. path):
val path
len(aath)
Examples:
D4 y in visited:
heaps (happsp(hap))
min_ e min(min_path, path) return
Input:
contine
grid [11223), (4.5.6]
visited adolf/ vl)
[7 =
Result: Falled (Timeout)
Output: 2,
Figure 7: failed case of DyLAN solving code generation task. Optimization directions provided by Code Reflector are wrong.
25

---

Published as a conference paper at COLM 2024
D Prompt Templates
In DyLAN, agents are assigned roles with prompts extracted from an open-source code base relative research projects (Du et al., 2023 Shinn et al. 2023 Zheng et al. 2023), and generation results of GPT-4-0613, besides manual construction. The prompt of each agent are constructed by concatenation of the role prompt and optional tool descriptions, as the system prompt, and instruction prompts. We exhibit the instruction templates of different datasets, the prompts of all agents, and their sources in 16 We annotate the task where each prompt is used in the parenthesis, and the source of each prompt template. We omit the in-context examples of AR tasks from the original dataset of MATH (Hendrycks et al. 2021b) and PHP Zheng et al. 2023), and WebShop from ReAct (Yao et al. 2023).
Prompt
Content
Source
MMLU Instruction (GR)
Here is the question: (question}
Manual
These are the solutions to the prob- lem from other agents: {responses}
Using the reasoning from other agents as additional advice with critical thinking, can you give an updated answer? Examine your solution and that other agents step by step. Notice that their answers might be all wrong. Put your answer in the form (X) at the end of your response. (X) represents choice (A), (B), (C), (D).
MATH Instruction (AR)
Follow the given examples and answer the mathematics problem.
Manual
{question}
These are the solutions the prob- lem from other agents: {responses}
Using the reasoning from other agents as additional advice with critical thinking, can you give an updated answer? Examine your solution and that other agents step by step. Notice that their answers might be all wrong.
HvmanEval Instruction (CG)
You must complete the python function give you by rectifying previous implemen- tations. Use the other information as hint. Be sure use the same indentatin specified. Furthermore, you may only write your response in code/ commmmts.. [improved impl]:
Retrieved
"python
{function signature} ..1
Please follow the template by repeat- ing the function signature and complete the new implementation in jimproved impl]. If no changes are needed, simply rewrite the implementation in the Python code block.
26

---

Published as a conference paper at COLM 2024
Prompt
Content
Source
WebShop Instruction (DM)
{history of observatons and actions}
Retrieved
These are the suggested next action from other agents: {actions from other agents)
Using the solutions from other agents additional advice with critical thinking, can you give an updated action in response to previous observations and actions? Please select the item after searching (click the product name 'BO...). And when you buy the item ('click[Buy Nowl', please make sure you have selected an item and entered it's description page. Do not search for multiple times. Based the example and previous actions and observations on the new instruction, give your next action in the format of "Action: search[...]" or "Action: click[....''
Human Prior Selection Instruction (-)
A few agents will collaborate on the same task query. Please select the optimal composition of the candidate agents based on the description the task and the agents' profiles.
Manual
Tass: {subject)
AAgents
{ Agent/Code Writer/Judge (Name): Role Prompt/Description/
We want to select k agents/code writers/code reviewers among these candidates. Please write the agent IDs as the following format: [1, 2, 3, 4]. There
could be multiple agents with the same ID.
Ranker Instruction (-)
Here is the question: (question)
Manual
These are the solutions the prob- lem from other agents: {responses}
Please choose the best 2 solutions and think step by step. Put your answer in the form like [1,2] or [3,4] at the response.
your
Mathematician (GR)
You mathematician. You good math games, arithmetic calculation, and long-term planning.
Retrieved
Programmer (GR)
You are programmer. You are good computer science, engineering, and physics. You have experience in designing and de- veloping computer software and hardware.
Retrieved
Lawyer (GR)
You are a lawyer. You are good at law, poli- tics, and history.
Retrieved
Historian (GR)
You are historian. You research and ana- lyze cultural, economic, political, and social events in the past, collect data from primary sources and use to develop theories about what happened during various periods of history.
Retrieved
27

---

Published as a conference paper at COLM 2024
Prompt
Content
Source
Economist (GR)
You are economist. You are good at eco- nomics, finance, and business. You have perience on understanding charts while in- terpreting the macroeconomic environment prevailing across world economies.
Retrieved
Psychologist (GR)
You are a psychologist. You are good at psychology, sociology, and philosophy. You give people scientific suggestions that will make them feel better.
Retrieved
Doctor (GR)
You doctor and come up with creative treatments for illnesses or diseases. You are able to recommend conventional medicines, herbal remedies and other natural alterna- tives. You also consider the patient's age, lifestyle and medical history when provid- ing your recommendations.
Retrieved
Python Assistant (CG)
You are Python writing assistant, an AI that only responds with python code, NOT ENGLISH. You will be given a function sig- nature and its docstring by the user. Write your full implementation (restate the func- tion signature).
Retrieved
Algorithm Developer (CG)
You are an algorithm developer. You are good at developing and utilizing algorithms solve problems. You must respond with python code, no free-flowing text (unless in comment). You will be given a func- tion signature and its docstring by the user. Write your full implementation following the format (restate the function signature).
Retrieved
Computer Scientist (CG)
You computer scientist. You good writing high performance code and rec- ognizing corner cases while solve real prob- lems. You must respond with python code, free-flowing text (unless in a comment). You will be given a function signature and docstring by the user. Write your full im- plementation tion following the format (restate the function signature).
Retrieved
Programmer (CG)
You are an intelligent programmer. You must complete the python function given to you by the user. And you must follow the format they present when giving your an- swer! You can only respond with comments and actual code, no free- flewffnoing text (unless in a comment).
Retrieved
Coding Artist (CG)
You are coding artist. You write Python code that not only functional but also aes- thetically pleasing and creative. Your goal is to make the code an art form while main- taining its utility. You will be given a func- tion signature and its docstring by the user. Write your full implementation following the format (restate the function signature).
Generated
28

---

Published as a conference paper at COLM 2024
Prompt
Content
Source
Software Architect (CG)
You are software architect, skilled in de- signing and structuring code for scalability, maintainability, and robustness. Your re- sponses should focus on best practices in software design. You will be given a func- tion signature and its docstring by the user. Write your full implementation following the format (restate the function signature).
Generated
Unit Tester (CG)
You are Al coding assistant that can write unique, diverse, and intuitive unit tests for functions given the signature and docstring.
Retrieved
Syntax Checker (CG)
Null
Code Reflector (CG)
You are Python writing assistant. You will be given series of function implementa- tions the same function signature. Write a few sentences to explain whether and why the implementations are wrong. These com- ments will be used hint and your goal is write your thoughts on the n-th previous implementation after [reflection n].
Generated
Debugger (CG)
are debugger, specialized finding and fixing bugs in Python code. You will be given function implementation with bug in it. Your goal is to identify the bug and provide corrected implementation. In- clude comments explain what was wrong and how it was fixed.
Generated
Quality Manager (CG)
You are a quality manager, ensuring that the code meets high standards in terms of read- ability, efficiency, and accuracy. You will be given function implementation and you need to provide code review. Comment on its correctness, efficiency, and readability, and suggest improvements if needed.
Generated
Ranker (CG)
You are a Python writing assistant. You will be given series of function implementa- tions of the same function signature. You need choose the best implementations in consideration of correctness, efficiency, and possible corner cases.
Generated
Search Optimizer
As a Search Optimizer, analyze a user's vague or broad instruction and suggest more precise and effective set of keywords filters to accurately find products that meet their specific requirements. Please fo- cus more on searching actions when giving the next action, especially on providing in- formative and accurate searching words in "search[...]"
Generated
Budget Analyst
As Budget Analyst, guide setting a realistic budget for their shop- ping needs, considering product categories, market prices, and personal financial con- straints. Analyze whether the product matches the budget one by one. Please avoid searching multiple times. Please fo- cus more on on which product to click in "click[...]' when giving the next action.
in
Generated
29

---

Published as a conference paper at COLM 2024
Prompt
Content
Source
Product Explorer
As Product Explorer, offer guidance how to effectively compare different prod- ucts based on their features, prices, and re- views. Advise on key factors to consider for making informed decisions in various prod- uet categories. You are preferred to "clickli Prev]" "click[Next eI" to go different pages, and click on the item name to browse its details.
Generated
Instruction Analyst
As an Instruction Analyst, evaluate cus- tomer's 's stated needs and preferences, and provide structured approach to ensure the selected product align with these instruc- tions. Do not keep refining searching. You are preferred to check out products that might align with instruction. Otherwise, give the most reasonable action for the next step
Generated
Description Reader
As Description Reader, detail how read and interpret product descriptions and spec- ifications to match them with a customer's ss specific needs, focusing on identifying key features, benefits, and potential drawbacks.
Generated
You are preferred click into each prod- uct and "click[Desrription" to see product details.
Decision Maker
As a Decision Maker, you are more confi- dent to purchase related products without refining searching results. If you believe product suitable choice, proceed click the product and persuade other agents to click "click[Buy Now]". If not, recom- mend the most logical next step for rapid adaptation for the next product.
Generated
Decision Reflector
As Decision Reflector, provide a frame- work for customers to critically evaluate their potential purchases, considering their initial requirements, product features, and overall value. Guide them in reflecting on whether choice truly meets their needs. Please avoid repeated searching. If you think it's good buy the product, go "click[Buy Now]". Otherwise, give the most reasonable action for the next step.
Manual
Result Estimater
As a Result Estimator, describe how to pre- dict the potential satisfaction and success of customer's purchase decision based on their needs, product choice, and market trends. Offer insights into how these choices may meet their expectations. If you have any suggestions, print them in "think[...]' Otherwise, give the most reasonable action for next step.
Generated
Table 16: Instruction and prompting templates used in different datasets and agents.
30


Reviews:
[{"review_id": "j0Ihl4uIli", "reviewer": "Reviewer_2JJf", "summary": "In this paper, it proposes a new method for dynamically conduct multi-agent collaboration. In incorporates the team operation process and task solving process, which can improve the performance of agent collaborations. The experiments have shown the effectiveness of this method.", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank the reviewer for the constructive feedback. Below, we address each of the concerns you raised.\n# 1. Error Bars in Experimental Results\nThank you for pointing out the absence of error bars in our results. We acknowledge the importance of including error bars for a comprehensive presentation. There are two main reasons for this omission:\n\n1. **Variance Related to “Temperature”:** The variance in our results is highly influenced by the hyper-parameter “temperature,” which is searched within a range (e.g., {0, 0.2, 0.8, 1.0} for GR&AR), as detailed in Appendix B.1. Different temperatures were used for different methods. Therefore, the variance mainly reflects the impact of temperature rather than providing additional insightful information about the method.\n\n2. **Monetary Constraints:** Due to budget limitations, we conducted three runs for methods with non-zero temperatures. Three samples were insufficient to calculate meaningful error bars (e.g., 1-sigma or 2-sigma bars). Consequently, we opted to report the median result, which offers a reliable representation within our resource constraints.\n# 2. Comparison with MetaGPT\n**Performance:** Please refer to Table 7 in Appendix C.3. Our findings indicate that MetaGPT (85.9) performs worse than bare GPT-4-0613 (88.4) with task instructions and requires significantly more API calls for each sample in code generation tasks. DyLAN (92.1) achieves 7.2% improvement against MetaGPT. Additionally, MetaGPT's framework is not easily adaptable to non-programming tasks.\n\n**Methodology:**\n- **Agent Filtering:** MetaGPT does not incorporate agent filtering methods. Our approach to team optimization involves selecting a team of agents from an initial pool, which is distinct from MetaGPT's fixed framework with five predefined agent roles.\n- **Related Work:** As further related work, SPP [1] introduces agent generation by LLMs but does not filter agents. The exchange-of-thought [2] employs thought-level consistency for response filtering, which is not applicable at the agent level. Currently, we are not aware of any principled agent-level team optimization methods besides DyLAN.\n# 3. Paper Formatting\nThank you for your feedback regarding the paper's formatting. We will make necessary formatting adjustments to improve our paper.\n\n[1] https://arxiv.org/abs/2307.05300.  \n[2] https://arxiv.org/abs/2312.01823.\n\n---\n\n**We appreciate your time in reading our rebuttal and look forward to further discussions.**", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "Advantage:\n1.The problem is practical and the motivation is sensible. It is similar to create a meta agent filter to construct an agent team. Some previous works like MetaGPT also study this problems as well.\nThe techniques sound great, which utilizes methods from graph learni", "reasons_to_reject": "Shortcoming & Question:\n1.It seems that there is no error bar in the results of experiments.\n2.What about the comparison with MetaGPT?\n3.The paper formatting can be improved, such as reducing the empty blocks within it.", "questions_to_authors": "see above", "extra_scores": {}}, {"review_id": "6syX97klhE", "reviewer": "Reviewer_B67Q", "summary": "In this paper, the author introduce the Dynamic LLM-Agent Network (DyLAN) framework, designed to facilitate collaboration automatically among multiple LLM agents.The author employs T-FNN to dynamically regulate interactions between different agents and Introducing Agent Importance Score to measure each agent's contribution, reducing manual intervention, which is quite innovative in this field. The whole framework includes two stages: \"Team Optimization\", which identifies and excludes underperforming agents, and \"Task solving\" performing inference based on the T-FFNs. \n\nThe author conducted experiments on DyLAN across four tasks: Code Generation, Decision Making, General Reasoning, and Arithmetic Reasoning, comparing it with several previous SOTA methods. DyLAN not only improves performance but also keeps API calls at a relatively low level. To demonstrate the effectiveness of DyLAN, the author conducted lots of experiments and discussions. However, many important findings were included in the appendix. Personally, I believe some crucial discussions such as C.2 and C.8 should be included in the main paper.", "questions": "", "limitations": "", "rating": 8, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank the reviewer for the constructive feedback. Below, we address each of the concerns you raised.\n\n# 1. Crucial Discussions in Main Paper\n\nWe appreciate your observation and agree that some crucial discussions should be included in the main paper. We will consider incorporating these discussions for a more comprehensive view of our work within the page limits.\n\n# 2. Limited Improvement on the AR Task\n\nWe have identified the following reasons for the phenomenon:\n\n- **Knowledge Dependency of MATH Dataset:** The AR task utilizes the challenging MATH dataset, which is highly knowledge-dependent. Despite this, DyLAN still significantly outperforms baselines under the same prompting strategy, with relative improvements of 10.2% and 3.0% compared to the best baseline, as shown in Table 2.\n- **Role Assignment:** Compared to other tasks, role-playing introduces relatively lower improvements in AR tasks, due to the high knowledge dependency of MATH. In comparison, Table 13 shows that selecting appropriate agents for math subjects in MMLU, which are less knowledge-dependent, results in greater performance improvement. \n\n# 3. Impact of Early Stopping and Agent Team Reformation\n\nWe explain the reason for phenomena for each task:\n\n- **GR and AR Tasks:** Early stopping functions when answers from agents are exactly the same. For multiple-choice problems in GR or mathematical problems, we use exact matching to compare answers. Early stopping is more effective in GR since problems are usually simpler than AR, allowing agents to reach consensus earlier.\n- **CG Tasks:** Answer comparison is more challenging for open-ended tasks like CG tasks. We use the BLEU score with a 0.9 threshold for consistency checks. This makes early stopping less effective since agents may generate codes in different formats, leading to fewer opportunities to stop early.\n- **DM Tasks:** DM tasks face a similar challenge due to the large action space and arguments (e.g., click[item name], search[query words]). The multi-step nature of DM tasks means that the length of actions for each sample is greatly affected by the quality of decisions. With incorrect actions, the agent may continue acting until the maximum step is reached, making both ablations significantly impactful.\n\nWe will include these discussions in the next version of our manuscript to provide a clearer understanding.\n\n---\n\n**We appreciate your time in reading our rebuttal and look forward to further discussions.**", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. Using T-FNN to dynamically regulate interactions between different agents and Introducing Agent Importance Score to measure each agent's contribution, reducing manual intervention, which is quite innovative in this field.\n\n2. The idea of introducing Byzantine Consensus theory to achieve early stopping is very interesting and proved to be effictive through ablation study.\n\n3. The author demonstrated the outstanding performance of DyLAN through various experiments and comparisons with current sota methods.", "reasons_to_reject": "I don't have any major concerns that would lead me to reject the submission of this article, although:\n1. As I said  in  summary, some crucial discussions should be included in main paper.\n2. The improvement of DyLAN on the AR task is limited compared to other tasks. What might cause this phenomenon?", "questions_to_authors": "1. In the ablation study, I noticed an interesting phenomenon: the early stop has a significant impact on API calls in arithmetic reasoning and general reasoning tasks. Whereas in the decision-making task, both methods have considerable impacts. For code generation, however, neither method seems to have much effect. What reasons might the author attribute to this phenomenon?", "extra_scores": {}}, {"review_id": "UxgQrXKZUW", "reviewer": "Reviewer_sGnL", "summary": "This paper introduces an approach called DyLAN that incorporates a two-stage paradigm: i) Team Optimization, the temporal feed-forward neural network (T-FFN) that selects most contributary agents for the given task and ii) Task solving, the T-FFN facilitate agents to communicate with each others to solve a bunch of a diverse of tasks. To further enhance the efficiency, DyLAN uses early stopping mechanism to terminate the conversation when most of agents in a single layer agree to have a consistent answer.  In the experiment section, the DyLAN is proved to outperform with baselines such as LLM-Blender, LLM Devate, ReAct, and Reflexation, etc, on code generation, decision making, and general reasoning, etc, and shows that DyLAN.", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank the reviewer for the feedback.\n# 1. Selection of Agents\n- **Selecting a Proper Team is Challenging:** We acknowledge that some agents might be redundant for specific queries. Empirically, fewer properly selected agents can outperform more with redundancy (Table 5&Fig.3). **However, identifying these agents precisely is hard.** Our goal is to identify the most contributory agents in a principled way. The Agent Importance Score is effective as an unsupervised metric, but not an oracle.\n  - **The Challenge to Identify Contributory Agents:** In Fig.5, agents do not reach the correct answer at t=1 but later. This illustrates the difficulty of agent identification brought by multi-turn communications. Also, **seemingly irrelevant agents from human perspectives may make significant contributions** (App.C.5). Empirically, Table 13 shows selecting agents based on human preference performs lower.\n  - **The Challenge to Decide the Team Size:** In Fig.6, agents refine their answers after talking to others. This case illustrates the challenge to determine the optimal team size priorly. Although Mathematician alone is sufficient for the query in Fig.6, it is not always adequate for others. In Fig.3 and Table 8, the chosen team size in our settings strikes better effectiveness and efficiency.\n- **Generalization Benefits:** Including various roles could enhance generalization. In Table 10, different subjects benefit from distinct selected teams. But if the majority are irrelevant to tasks by design (e.g., coding agents for DM tasks), the result may be negatively impacted.\n# 2. Framework Design and Scalability\n- **General Formulation:** The pattern of DyLAN is designed to be general. DyLAN treats each agent at each time step equally (Sec.3.2). In Fig.6, the agents at t=2/4 are identical to those at t=13. The pattern is consistent in AR and DM tasks.\n- **Role of Even-Numbered Layers:** In Fig.5, judges are in the even-numbered time steps because they require predecessors. The setup is of the general pattern.\n- **Generalizability and Scalability Across Tasks:** For instance, in GR tasks with 57 different subjects, DyLAN selects teams from 7 varied agents, significantly improving performance across subjects (Table 5&10). The system prompt for each agent is universal and prompt in DyLAN for various tasks is only different in task instructions (usually 1 sent.).\n\n---\n**We appreciate your time in reading our rebuttal and look forward to further discussions.**", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "1. Solid Experiments: The paper provides a robust experimental methodology, demonstrating DyLAN's effectiveness across a range of downstream tasks such as coding, general knowledge quetsion answering, math etc. It shows that cost DyLan can enhance the performance of the above mentioned tasks while keep low cost on computation with respect to reasonable times of function call. \n\n2. The idea of DyLAN is novel, it trains a dynamic temporal feedforward network to select the imporatant agents, and the early stopping seems to be effective to reduce the cost on long-term conversation.\n\n3. The ablation study is thorouh.", "reasons_to_reject": "1.  **Extended Query on the Selection and Performance of Agents in the Initial Pool**: \"Upon close examination of Figure 5 within the manuscript, there appears to be a lack of clarity regarding the roles adopted by the other agents involved in the code generation task. It is noted that the 'programmer' and 'algorithm developer' agents seem to outperform the rest, which might suggest that these two agent roles are disproportionately influential in the task's outcome. This observation raises the question of whether these two agent roles alone suffice for the task at hand. Similarly, in the mathematical reasoning tasks, the 'programmer' and 'mathematician' agents are indicated as the primary contributors, prompting the inquiry of whether including agents with roles such as 'doctor' or 'economist' might be an overcomplication in the context of mathematical tasks. If the initial agent pool is curated for specificity to different tasks, then the inclusion of seemingly irrelevant agents could be considered redundant and may detract from the efficiency of the overall system. Could the authors provide insight into the rationale behind the selection of these varied roles and how their inclusion impacts the performance and generalizability of the DyLAN framework?\"\n\n2. Upon reviewing Figures 5 and 6, it is observed that layers with an even number typically assume a critical function, evaluating the responses generated by the preceding odd-numbered layers. This pattern suggests that the effectiveness of the DyLAN framework may be contingent upon the utilization of highly curated prompts tailored to fit each agent's distinct roles and scenarios. The observed phenomenon raises concerns regarding the scalability and flexibility of the system, particularly in scenarios where the diversity of tasks and roles is substantial. It would be beneficial for the authors to address the necessity for such detailed prompt curation and discuss potential strategies to streamline this process, ensuring that the framework remains adaptable across a wide array of tasks without compromising performance.\n\nHowever, I find the paper to be of high quality overall; if the authors can provide clear and satisfactory responses to the queries raised, I would be willing to increase the points.", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper introduces a new framework, DyLAN, for dynamic selection and collaboration of LLM-powered agents tailored to specific tasks and domains. Key strengths include the introduction of an Agent Importance Score for team optimization, which significantly enhances performance over static methods, and the comprehensive empirical validation across various tasks such as code generation, decision-making, general reasoning, and arithmetic reasoning. 
The reviewers appreciated the practical motivation and technical soundness of the approach. By examining the reviewers' comments and the author responses, the AC thinks this paper warrants an acceptance with minor improvements in the camera ready version.
Decision: Accept