A Dynamic LLM-Powered Agent Network for Task-Oriented Agent Collaboration

Authors: zijun  liu
yanzhe  zhang
peng  li
yang  liu
diyi  yang
Venue: colmweb_org_COLM_2024_Conference
Published: 2024-03-23 10:24:15

Abstract: Recent studies show that collaborating multiple large language model (LLM) powered agents is a promising way for task solving. However, current approaches are constrained by using a fixed number of agents and static communication structures. In this work, we propose automatically selecting a team of agents from candidates to collaborate in a dynamic communication structure toward different tasks and domains. Specifically, we build a framework named Dynamic LLM-Powered Agent Network ($\textbf{DyLAN}$) for LLM-powered agent collaboration, operating a two-stage paradigm: (1) Team Optimization and (2) Task Solving. During the first stage, we utilize an agent selection algorithm, based on an unsupervised metric called Agent Importance Score, enabling the selection of best agents according to their contributions in a preliminary trial, oriented to the given task. Then, in the second stage, the selected agents collaborate dynamically according to the query. Empirically, we demonstrate that DyLAN outperforms strong baselines in code generation, decision-making, general reasoning, and arithmetic reasoning tasks with moderate computational cost. On specific subjects in MMLU, selecting a team of agents in the team optimization stage improves accuracy by up to 25.0% in DyLAN.

Main paper: 
 introduction: 
 Large Language Model (LLM) agents (Richards & et al., 2023; Nakajima, 2023; Reworkd, 2023) have demonstrated promising performance on various tasks, ranging from reasoning (Yao et al., 2023) , code generation (Shinn et al., 2023) to embodied tasks such as video gaming (Wang et al., 2023a) and autopilot systems (Jin et al., 2023) . Given the impracticality of a single agent managing all these tasks efficiently, recent research has shifted towards multi-agent collaborations, yielding significant advancements (Li et al., 2023; Du et al., 2023; Wang et al., 2023c; Jiang et al., 2023; Shinn et al., 2023; Chen et al., 2024; Wu et al., 2023) .As an analogy to human society, how human teams function may provide valuable insights for developing more effective multi-agent collaboration systems. For instance, recent studies have demonstrated that certain effective communication structures, derived from human society, also play a positive role in multi-agent collaborations (Yin et al., 2023; Chen et al., 2024) . In addition to communication structures, another notable characteristic of human teams is that they would optimize team members according to the given task. Take medical consultations as an example. Collaborations in dynamic structures are evident when the composition of the team changes within the procedure of consultation, as some doctors may become less relevant in major as the conversation goes deeper and "leave" the consultation, leading to corresponding changes in the communication structure. Team optimization is often observed in the varying initial composition of doctors for consultations with different diseases, influenced by changes in the related medical fields and the contribution of each doctor. These characteristics prompt an important question: Does a dynamically changing team of agents benefit LLM-powered agent collaborations similarly?However, the question is not well-addressed yet. While various communication structures have been studied for different tasks, such as debating for reasoning (Du et al., 2023; Liang et al., 2023; Xiong et al., 2023) and self-collaboration for coding (Dong et al., 2023; Qian et al., 2023a; b) , these communication structures do not alter members in the agent team and remain fixed throughout the collaboration. It indicates that task-oriented dynamic selections in agents are not thoroughly explored in current research. Furthermore, in the context of agent teams, most existing studies opt for hand-crafting agents from human priors (Liu et al., 2023; Nakajima, 2023; Hong et al., 2024; Shinn et al., 2023; Li et al., 2023) or employ an LLM to generate them (Wang et al., 2023c; Chen et al., 2023b; Christianos et al., 2023) . These approaches generally predefine agents without further validation of the collaboration process. This leads to static agent teams or rebuilding teams without principled verification (Chen et al., 2024) . Challenges still remain for optimization methods.As a first attempt towards addressing the above question, we introduce a novel framework named Dynamic LLM-Powered Agent Network (DyLAN). DyLAN conceptualizes multiagent collaboration using temporal feed-forward networks (T-FFNs) . In this formulation, each communication step of the agents corresponds to a network layer, with nodes representing the agents involved at that step and edges indicating communications between agents, for incorporating dynamic agent teams agnostically. DyLAN functions in two stages to incorporate task-oriented agent collaborations (Figure 1 ). The first stage is termed Team Optimization, where we select top contributory agents unsupervisedly among the initial team of candidates according to the task query, based on their individual contributions. We propose a forward-backward message passing algorithm on the T-FFN termed agent selection in Section 3.4, inspired by the back-propagation algorithm (Rumelhart et al., 1986) and neuron importance scores (Yu et al., 2018) . This algorithm measures the contribution of each agent at the first stage with an unsupervised metric named Agent Importance Score. The most contributory agents form a smaller team to collaborate at the second stage -Task Solving, thereby minimizing the impact of less effective agents on the final answer. Specifically, the collaboration begins with a team of agents, and an LLM-powered ranker in the middle dynamically deactivates low-performing agents (i.e., agent team reformation), thus expanding the T-FFN, integrating dynamic communication structures into DyLAN (Section 3.3.2). Incorporating agent selection, DyLAN effectively identifies and coordinates a task-oriented team of agents in a principled way. Extensive experiments demonstrate that DyLAN outperforms strong baselines in various tasks, including code generation, decisionmaking, general reasoning, and arithmetic reasoning. Notably, agent selection in DyLAN has improved accuracy by up to 25.0% in certain subjects of the MMLU dataset (Hendrycks et al., 2021a) , underlining the significance of dynamic agent teams.In summary, our contributions are threefold:• We introduce a novel framework named DyLAN for task-oriented agent collaboration in two stages with agent selection, marking a significant advancement in the study of dynamic agent teams. • DyLAN innovatively formulates agent collaborations in temporal feed-forward networks with agent team reformation, enhancing its adaptability and reducing dependence on human preconceptions. • Empirical results demonstrate the superior accuracy, efficiency, and stability of DyLAN across various tasks, underscoring the need for dynamic agent teams. 
related work: 
 Team Optimization of LLM-Powered Agents The construction of agent teams is the essential and initial step for LLM-powered agent collaboration. TPTU (Ruan et al., 2023) and Chameleon (Lu et al., 2023) decompose tasks to choose or create tools accordingly. Recent studies also use LLMs to generate a fixed number of role prompts for agents in response to a task query (Wang et al., 2023c; Suzgun & Tauman Kalai, 2024) , or for each round of discussion (Chen et al., 2024) . However, manual prompts require careful design, which is impractical for adaptation on each task or domain, and prompting LLMs with predefined or generated descriptions may not result in the desired abilities of the agents without verification. Therefore, posteriorly selecting a team of agents based on their actual behaviors in the collaboration according to the task becomes essential. While team optimization for LLM agents is a relatively new area, human-team optimization has been studied for a long time.For instance, Liu et al. (2015) show that skill contribution is essential for selecting crowd workers to solve outsourced tasks efficiently. Based on peer rating, researchers have developed an algorithm for managing online workers in an optimal organization (Lykourentzou et al., 2022) . Drawing inspirations, we introduce an unsupervised algorithm to select a team of agents by quantifying their contributions based on peer ratings in Section 3.4.Collaboration between multiple LLM agents has demonstrated strong performance on various tasks in recent years and has emerged as a promising approach to enhance the capabilities of individual LLMs.To enable collaborations between multiple agents, recent studies have developed different communication structures and assigned agents in pre-defined architecture. For instance, researchers have found taking multiple LLM instances to debate for a fixed number of rounds can boost their factuality and reasoning capacities (Du et al., 2023; Liang et al., 2023; Xiong et al., 2023) . To aggregate multiple LLM responses, LLM-Blender (Jiang et al., 2023) calls different LLMs in one round and uses pairwise ranking to combine the top responses. It has also been shown effective in distributing workloads to LLMs and concatenating their answers, thus producing better results (Ning et al., 2024; Suzgun & Tauman Kalai, 2024; Qiao et al., 2024) . It is worth noting that existing studies (Hao et al., 2023; Zhang et al., 2023b) have tried organizing LLM instances into linear layers, but they mainly studied supervised learning in context space and LLM evaluation, respectively, not the scenario in which we are interested. However, running LLMs in a static architecture may limit the performance and generalization. On specific reasoning tasks, adopting a dynamic directed acyclic graph structure for LLMs has been shown effective (Zhang et al., 2023) .Also, recent studies (Yin et al., 2023; Chen et al., 2024; Zhang et al., 2023a; Zhuge et al., 2024) have demonstrated that optimal communication structures vary with tasks and compositions of agents. Aligned with the findings, we propose a structure that adjusts dynamically based on selecting agents according to the tasks and the construction of the agent team in Section 3.3.2.It is non-trivial to evaluate the contribution of each LLM agent in a multi-agent system, especially when they communicate over multiple rounds. In the single-round setting, existing methods use LLMs heavily for evaluation. To overcome the over confidence of LLMs (Xiong et al., 2024) , pairwise ranking based on an additional LLM ranker has been introduced in LLM-Blender (Jiang et al., 2023) . To rank n responses with an independent LLM in a single round, they compare all O(n 2 ) pairs. For better efficiency, researchers use a k-length sliding window to choose top k responses within O(nk) pairwise comparisons (Qin et al., 2023) . However, these methods have not been extended to multi-round settings. Inspired by the neuron importance score (Yu et al., 2018) , we evaluate agents by propagating and aggregating single-round peer ratings in a back-propagation manner (Rumelhart et al., 1986) . In this way, we then introduce an unsupervised metric called Agent Importance Score to quantify the contribution of each agent in multi-round collaborations (Section 3.4). 
overview: 
 We introduce a framework for LLM-powered agent collaboration named Dynamic LLM-Powered Agent Network (DyLAN), facilitating dynamic communication structures and automatically task-oriented agent selection in a two-stage fashion (Figure 1 ): an optimized agent team is constructed in the first stage "Team Optimization" through a preliminary trial and then the team collaborates to solve the task in the second stage "Task Solving".A core component of DyLAN is the temporal feed-forward networks (T-FFNs), whose nodes denote agents and edges denote the communication channels between agents (Figure 2 left). T-FFNs serve not only as the abstraction of communication structures but also the computation graph. From this perspective, as shown in Table 1 , various LLM-powered agent collaboration systems (Jiang et al., 2023; Shinn et al., 2023; Du et al., 2023; Li et al., 2023; Chen et al., 2024) can be represented by similar network structures as T-FFNs. DyLAN is the only framework that supports multiple agents with roles and tools, early stopping (Section 3.3.2), dynamic communication structures and team optimization simultaneously. To be specific, for team optimization, our agent selection algorithm is performed as a backward message passing algorithm on the T-FFN (Figure 2 right), and for task solving, agent team reformation expands the T-FFN dynamically with messages passing forward.To make it easier to understand, we will start by explaining the formulation of T-FFNs. Then, we will move on to the task solving stage, and finally, we will explain the team optimization stage, which relies on components of task solving.A T-FFN is a multi-layer network, of which each layer represents a time step. Its formal definition is as follows.Definition 1 (Agents) Agents participating the collaboration are represented byEQUATIONwhere N denotes the total number of agents, and a i can be (I) an LLM-powered agent possibly equipped with tools, or (II) an independent tool, e.g., scripts, code interpreters. Peer rate on the responses of its predecessors. Aggregate the received ratings from successors.t = 3 t = 4 t = 3 t = 4Top 2 elite agents are selected based on importance scores. where the contribution of each agent in a primary trial is automatically evaluated in three steps using Agent Importance Score, denoted as I. Then, the top-ranked agents based on I will be selected as the optimized, task-oriented team of agents.Definition 2 (Nodes) The t-th layer of a T-FFN consists of N nodes, each of which corresponding to one agent:EQUATIONwhere t = 1, • • • , T, and node v t,i corresponds to agent a i .Definition 3 (Edges) Edges in a T-FFN refer to the communication channels between nodes, forming the communication structure between agents. Specifically, the set of edges between the nodes in layer t -1 and t is denoted asEQUATION)where t = 2, • • • , T, and (v t-1,i , v t,j ) denotes an edge connecting nodes v t-1,i and v t,j .Definition 4 (T-FFN) Finally, the T-FFN corresponding to the collaboration is defined as a T-layer network:EQUATIONNote that we only consider T-FFNs where edges only exist in adjacent layers. However, edge can be added to arbitrary pairs of nodes, making the T-FFN capable of representing more complex communication structures.Task solving involves performing inference on the T-FFNs, jointly with agent team reformation, which be elaborated on in the subsequent two sections.Before details, we first introduce the formulation of message passing on T-FFNs.Definition 5 (Message Passing) Given a T-FFN, a node v t,j , a set of adjacent nodes U = {u 1 , • • • , u K }, and the messages M = {m u 1 , • • • , m u K } received by v t,j , where K is the size of U , and m u k is the message sent from u k to v t,j , message passing aggregates all the messages M to produce a updated message mv t,j for v t,j , which is formally defined asEQUATIONwhere f mp (•, •) is the aggregator function.We refer the algorithm as forward message passing when U is the set of all adjacent nodes of v t,j from the previous time step, i.e., U = {u k |∀u k , (u k , v t,j ) ∈ E t-1,t }. Similarly, it is referred as backward message passing when U is the set of all adjacent nodes from the next time step:U = {u k |∀u k , (v t,j , u k ) ∈ E t,t+1 }.With the above formulation, we can describe the inference process of a T-FFN G in the manner of forward message passing. During collaborations on a given task, an agent at a specific time step takes the responses, i.e., messages, from other agents at the previous time step as input and generates responses based on the task query. Based on different types of agents at v t,j , we can implement f mp (•, v t,j ) respectively: (I) concatenating input messages along with the task query into prompt templates (refer to task instructions in Appendix D) and take the response from LLM after generation or tool calling, or (II) filter the input that the tool can process, e.g., code completions and structured text.During inference, we begin feeding the task query q ∈ Q into agents at time step 1 (V 1 ), where Q denotes the dataset. By passing responses of nodes V t-1 at time step t -1 to nodes V t at t, agents can perceive responses from all nodes at the previous time step and perform collaborative behavior, which might include criticizes, advice, refinement, or quality reviews, depending on the implementation of agents. Formally, the inference process is defined asEQUATIONwhere o = argmax{M T } and M T denotes the responses from V T . Please refer to Algorithm 1 for detailed procedure.Given a set of agents A, agent team reformation aims to identify more contributory agents and construct a dynamic communication structure accordingly. To this end, we leverage an additional LLM instance, referred as the "LLM Ranker", to analyze responses from the agents participate in the current time step and give out a ranking, prompted by the template of "Ranker" in Appendix D. Then, the top-ranked agents are allowed to participate in the next time step. In other words, edges will only be added for these top-ranked agents, resulting in a dynamic communication structure.Formally, suppose the set of agents participates in time step t is A t = {a k }, and the topranked agents are A t+1 = {a l }, where k and l are the indices of the agents as defined in Equation (1), then we can obtain two nodes sets V ′ t = {v t,k |∀a k ∈ A t } and V ′ t+1 = {v t+1,l |∀a l ∈ A t+1 }, and the edge set E t,t+1 is defined asEQUATIONThe process progresses iteratively until the stop condition is met, and finally, we get a T-FFN G A q for the query input q. We use function f IAS to denote the entire computation:EQUATIONTo further enhance efficiency, we introduce an early-stopping mechanism. Inspired by the Byzantine Consensus theory (Castro & Liskov, 1999) , at least 3p + 1 agents are needed to tolerate p faulty agents in a single round of communication. Following the theory, the inference process will be terminated when over 2/3 of agents in a single layer have a consistent answer. In practice, the inference process will also be terminated when the maximum time step is reached. Note that none of the consistency measures used in prior work (Wang et al., 2023b; Aggarwal et al., 2023; Yin et al., 2023) applies to multi-round multi-agent interaction since their theories are assumed to execute a single LLM instance multiple times or expect all agents to reach the same answer.The goal of team optimization is to select a subset of agents from candidates based on their contributions evaluated from a primary trial, such that the new team solves the task query more effectively and efficiently. Formally, given a task query q, a set of agents A, a trial is performed based on the algorithm proposed in Section 3.3.2 resulting in a T-FNN G A q . And team optimization is formulated asEQUATION)f Optim is implemented as in a three-step procedure of agent selection (Figure 2 right):(1) Propagation: Each node rates the solutions to the task query from its predecessors, which is a forward message passing process. Formally, given a node v t,j and an edge (v t-1,i , v t,j ), the message m v t-1,i sent from v t-1,i to v t,j is defined as the response to the task query q from the agent a i at the previous time step. The aggregator functionf mp (•, v t,j ) is implemented as a scoring function f (s)t,j (•, •, •), which maps the prompt p j , the input query q, and all the messages M to the rating scores. Here, we use w t-1,i,j to refer to the rating score on v t-1,i from v t,j , andEQUATION(2) Aggregation: Each node aggregates the ratings it has received from its successors towards itself to quantify its own contribution independently at different time steps. The contribution of node v t-1,i is the sum of its successors' contribution multiplied by their peers' ratings on the agent's response. Aggregation is a backward message passing process. Formally, given a node v t-1,i and an edge (v t-1,i , v t,j ), the message m v t,j sent from v t,j to v t-1,i is defined as w t-1,i,j . And the aggregator function f mp is defined as a weighted sum function:EQUATIONwhere I t,i denotes the contribution of a t,i .(3) Selection: During the last step, we sum up the scores for the same agent over all time steps to derive an importance score for each agent, and extract the top-k agents that are most contributory according to these scores to form the optimized agent team. Formally, the Agent Importance Score I i for agent a i is defined asEQUATIONIn practice, we initialize the contributions in the final layer first, and step backward to perform Aggregation layer by layer (Algorithm 2). The definition guarantees that the agent importance scores add up to 1 in each layer, which benefits fair comparison. Other details, such as initializing contributions in the final layer, are presented in Appendix B.2.  
setup: 
 Code Generation (CG) We use the HumanEval benchmark, with 164 human-labeled function-level completion codes and unit tests (Chen et al., 2021) . Unit tests are used to validate the correctness of generated codes. We leverage two strong baselines CodeT (Chen et al., 2023a) and Reflexion (Shinn et al., 2023) along with the single execution. For multiagent baselines, we re-implement CAMEL (Li et al., 2023) and AgentVerse (Chen et al., 2024) under their original configurations for fair comparisons.We evaluate our methods in the WebShop environment, selecting 50 environments in its test set (Chen et al., 2021) following the setting of LATS (Zhou et al., 2023) . WebShop requires to find the item given an instruction of the customer. It provides "reward" as an intrinsic metric for item-instruction relevance, and "success" is marked when the reward is 1.0. Besides ReAct (Yao et al., 2023) and Reflexion, we re-ran a multi-agent method BOLAA (Liu et al., 2023) , and a single-agent method LATS as strong baselines.General Reasoning (GR) For the general reasoning task, we use the MMLU dataset (Hendrycks et al., 2021a) , which contains four categories of a vast amount of problems in 57 subjects. We down-sample 1/5 of the problems in the test set because of its huge quantity. We choose LLM Debate (Du et al., 2023) , LLM-Blender (Jiang et al., 2023) , and the single execution on LLM as baselines.Arithmetic Reasoning (AR) We leverage the test set of MATH (Hendrycks et al., 2021b) for evaluation, which consists of 7 subareas and 5,000 questions in total. To draw a fair comparison and verify the robustness, we categorize methods by different prompting strategies and select strong baselines accordingly. Preliminary experiments show that collaborating agents in different domains (e.g., algebra and geometry experts) does not make significant improvement, therefore we adopt agents with same prompts for all methods. In Table 2 , Table 3 , and Table 4 , we report the results on each dataset respectively. The number of API calls serves as a proxy for the efficiency of communication structures for agents, which cannot be clearly determined from token consumption which varies greatly depending on tasks and prompting strategies. Since "Task solving" after "Team Optimization" is essential for testing and deployment, we mainly report the cost from the second stage. The difference can be seen in Table 5 , and further discussions in Appendix C.1.DyLAN improves overall performance on different tasks with a reasonable computational cost. From Table 3 , we find DyLAN realizes an 10.2% improvement to LLM Debate in terms of accuracy, with 10.6% lower #API calls (L3 vs. L4), suggesting it is a better trade-off between efficiency and effectiveness. Similar trends can be observed as DyLAN has a better performance with only 36.6% API calls of LLM Debate (L5 vs. L4 in Table 4 ), and 35.1% of LATS on CG and <6% on DM (L8 vs. L5 (left), L7 vs. L5 (right) in Table 2 ). We argue it can be attributed to the feed-forward structure and early-stopping mechanism, which allows different solutions to be delivered simultaneously and confirmed rapidly. In contrast, for methods in sequential architecture like PHP and Reflexion (Table 2 ), incorrect intermediates might easily influence the final output due to the single thread of reasoning or code generation and review. Also, ReAct on DM tasks exhibits similar failures due to misoperations in the middle. In contrast, Reflexion and LATS access the environment at certain states for multiple times to for reflection, limiting generalizabilities. In our case, any feedback from predecessors could be rated by successor nodes, making it easier to rectify potential invalid actions. Moreover, we see that DyLAN dynamically adjust the cost based on the difficulty of tasks. For instance, most questions in the MMLU dataset are less challenging than MATH, DyLAN has 2.76 fewer API calls on the query from the former. However, compared to other tasks, DyLAN introduces relatively lower improvements in AR tasks, which might be due to the high knowledge dependency of the MATH dateset.DyLAN benefits from the team optimization. Moreover, we found that a dynamic team of task-oriented agents could enhance DyLAN. For different subjects in GR tasks, agent compositions are adjusted correspondingly to improve up to 25.0% in accuracy, as shown in Table 7 . As denoted in Table 5 , a dynamically selected team of agents could result in significant performance improvement, especially for DM tasks, where agents might have great interference from others. The overall performance can be significantly improved (up to 6.7%) with lower computational costs on each tasks after agent selection. Moreover, it also suggests that Agent Importance Scores can effectively capture and reflect the actual contributions of agents on a wide range of tasks. We further verify this claim in Appendix C.6.Impact of Optimized Agent Team Size Fewer proper agents in a team could have better performance. As shown in Figure 3 , DyLAN with an optimized team of 3 agents can outperform both DyLAN before team optimization and LLM Debate with 4 agents, suggesting the effectiveness of our proposed agent selection. The efficiency is also significantly improved The Agent Importance Score is robust over the imbalance of agent roles. On GR tasks, the candidates are imbalanced in terms of expertise. For most queries, there are less than 2 candidates that are related according to their role prompts.In Table 7 , we found agent selection is capable for selecting related agents, e.g., "Mathematician" for "college mathematics", that matches human priors. However, if candidates are all vastly different from the task domain, e.g., "public relation", where the improvement is less significant. We also tested DyLAN on CG tasks with different amount of code writers and code reviewers after the "Team Optimization" stage. It is worth noting that a single run of "Team Optimization" could provides reusable Agent Importance Scores for multiple trials of agent selection. In Table 8 , we exhibit the results under imbalanced teams of agents. We verified the imbalance of code writers and reviewers after optimization won't cause great performance drops. Though, reviewers affect the performance slightly greater than writers (L2,3 vs. L4,5), indicating the necessity of the amount of reviewers for code refinement.As shown in Table 6 , earlystopping mechanism boosts efficiency to a great extent by minimizing #API calls by 45.0%, 66.2%, 11.3%, and 54.2% on AR, GR, CG, and DM tasks respectively, while providing slight performance improvement. Agent team reformation, however, is critical to enhance the correctness of the final answer. We conjecture it is because agents are filtered for temporary mistakes in LLMs, such as hallucinations, etc. Additionally, answer comparison is more challenging for open-ended tasks like CG or DM tasks. We use the BLEU score with a 0.9 threshold for consistency checks. This makes early stopping less effective since agents may generate codes in different formats, leading to fewer opportunities to stop early.There is also a notable difference in CG tasks when the backbone model changes (Table 2 ). Reflexion and CodeT's performances are heavily related to the backbone model (L4 vs. L5 and L6 vs. L9). Instead, DyLAN shows a steady, consistent high performance (L7 vs. L10) under different backbone models with almost the same amount of API calls. 
conclusion and future work: 
 This work introduces a framework named Dynamic LLM-Powered Agent Network (Dy-LAN) for collaboration of dynamic agent teams on complicated tasks. DyLAN functions in a two-stage paradigm, enabling agents to interact in a dynamic structure with agent team reformation. In "Team Optimization" stage, the agent selection algorithm based on an unsupervised metric termed Agent Importance Score, selects top contributory agents in a principled way for collaboration on "Task Solving". Overall, DyLAN reveals improvement on diverse tasks with relatively less computational cost compared to baselines. In the future, we plan to explore the effectiveness of DyLAN built on open-source foundation models. 
: 
 Figure 1 : DyLAN adopts a two-stage paradigm. Agents communicate in a structure of the temporal feed-forward network (T-FFN). At the "Team Optimization" stage, DyLAN performs agent selection for the most contributory agents in a primary collaboration, oriented to tasks or domains. The selected agents then collaborate dynamically for an answer on the given query at the "Task Solving" stage. 
 
 Appendix: In experiments, we view code generation tasks as representative of open-ended generation tasks and adopt BLEU to decide whether two answers are consistent in early stopping mechanism in Section 3.3.2. In fact, the performance could be further leveraged by taskspecific methods like CodeBLEU (Ren et al., 2020) or CodeT (Chen et al., 2023a) .For practical usage, the agent-evaluation metrics could cooperate with human annotation to give a more precise evaluation result on individual contributions of agents, mainly when facing data scarcity problems. Furthermore, we simply incorporating agent selection on Dy-LAN with agent team reformation, as a primary step towards collaboration of dynamic agent teams. It still remains to be seen how to cooperate off-collaboration and in-collaboration optimization methods in a finer granularity to further improve performance and efficiency in LLM-powered agent collaboration systems.Additionally, though agent selection could differentiate top contributory agents, in extreme cases where the majority of agents are designed to contradict the task requirement, low performance might be caused, e.g., agents are prompted or trained to generate codes may face difficulties in clinical question answering. To tackle the imbalance of high-and lowperforming agents, replicating agents with high Agent Importance Score instead of including low-score agents could be a solution. Additionally, in extreme circumstances, we can automatically introduce agents from more capable LLMs with validation, in addition to agent selection.Algorithm 1 The Inference Process of DyLAN on an Arbitrary QueryInput: T-FFN G = (V 1 , • • • , V T ; E 1 , • • • , E T-1,T ), Query q Output: Final Answer o // E = {(v t,i , v t+1,j )} T-1 t=1 , v t,i , v t+1,j ∈ V // = T t=1 V t . // m t,i ∈ M t denotes the response from v t,i ∈ V t . for t = 1; T do if agent team reformation at time step t then M top ← top-k({m t-1,j |v t-1,j ∈ V t-1 }) E ← E\{(v t ′ ,j , * ), ( * , v t ′ ,j )|m t ′ ,j ∈ M top , t ′ ≥ t -1} m t,j ← m t-1,j , ∀(v t-1,j , v t,j ) ∈ E else ∀i, ∃k, (v t,i , v t+1,k ) ∈ E t,t+1 , m t,i ← f mp ({m t-1,j |(v t-1,j , v t,i ) ∈ E t,t+1 }, v t,i ) end if if early stopping then T ← t break end if end for o ← postProcess(maxCount{M T }) Algorithm 2 The Team Optimization Process f Optim of Agent Importance Score within DyLAN Input: Output o, T-FFN G = (V, E) Output: Agent Importance Score of agents I // m t,i ∈ M t denotes the response from v t,i ∈ V t . flag ← False for t = T; 1 do if {v t,i |∃k, (v t-1,k , v t,i ) ∈ E} ̸ = ϕ then if ¬flag then flag ← True distribute scores for I t,i else M t-1 ← {m t-1,j |(v t-1,j , v t,i ) ∈ E} [w t-1,1,i , ..., w t-1,m,i ] ← f (s) t,i (p i , q, M t-1 ) I t-1,j ← I t-1,j + I t,i w t-1,j,i , ∃(v t-1,j , v t,i ) ∈ E end if end if end forCommon Settings In all experiments, we use gpt-35-tubo-0301 for every method if not specified. The version of GPT-4 is GPT-4-0613. In Table 2 , "(Codex)" denotes code-davinci-002 from OpenAI (Chen et al., 2021; OpenAI, 2023) . All experiments with non-zero temperature is repeated for three times and the median is reported. To avoid the context length issue in prior work (Du et al., 2023; Liu et al., 2024) , we set memory space for agents in DyLAN to 1 only to keep the freshest responses of predecessors. We set max tokens to 2048 for GR and AR tasks and 1024 for CG and DM tasks to avoid exceeding the maximum context length. The construction of candidates are demonstrated in Appendix D. We set N = 4 in T-FFN after team optimization because the early-stopping mechanism requires at least four agents to tolerate one different response at a specific time step (Section 3.3.2); when reaching consensus over 2/3 of agents, it allows for 4 -2 3 N = 1 excetional response. We use a listwise ranker in the agent team reformation of DyLAN because of the effectiveness and efficiency, compared to ELo rating (Herbrich et al., 2006) or Sliding Window (Qin et al., 2023) we have tested in Appendix C.5. We use the same ranker to implement LLM-Blender (Jiang et al., 2023) in experiments. We set k = 2 in the agent team reformation, because it's the minimal number for collaborations and we empirically found it brings great trade-off between effectiveness and efficiency. To avoid positional bias, for each time step t, we shuffle the responses from agents at t -1 when passing messages towards agents at t. The detailed inference algorithm is in Algorithm 1. To implement the early-stopping mechanism, we need to determine whether the answers from the nodes in the same layer of DyLAN are consistent. For classification and decision-making problems, the answers are consistent if identical, and for open-ended generation, the consistency is determined by a threshold of BLEU score.In general reasoning, we extract the answer from the response by matching the last "(X" or "(X)", where "X" represents A, B, C or D. On average, Agent team reformation functions on the third time step. They could go through at maximum T = 4 rounds of interaction. We also searched temperature in {0, 0.2, 0.8, 1.0} for the best configuration for each system. In arithmetic reasoning, we set temperature to 0 for the single execution and PHP, 0.2 for LLM Debate, LLM-Blender, and DyLAN with Complex CoT prompts, and 1.0 for DyLAN with simple CoT prompts in Table 3 , since systems with the same prompts will give all the same responses if temperature is zero, causing degradation. Prompting templates are replicated from their original studies, including normal CoT prompts (Wei et al., 2022) from the MATH dataset (Hendrycks et al., 2021b) and Complex CoT from PHP (Zheng et al., 2023) . We follow the answer extraction method from the origin paper (Hendrycks et al., 2021b) . We construct DyLAN with 4 agents assigned no specific roles and let agents to interact for at maximum T = 4 rounds under T-FFN formulation. We reported the classification accuracy of each category averaged across subjects and the numbers of API calls of running DyLAN on the optimized team of agents.In the code generation task, we set temperature to 0 for the single execution, Reflexion, and 0.8 for LLM Debate, LLM-Blender, CodeT, and DyLAN in Table 2 . In DyLAN, we optimized four agents to write code and four agents to give code reviews from 12 candidates in Appendix D. The selected code writers are "Python Assistant", "Algorithm Developer", "Computer Scientist", and "Programmer"; and the selected code reviewers are "Syntax Checker", "Unit Tester", "Reflector", and "Ranker". "Syntax Checker" is pure external tools using a code interpreter for syntax checking without LLMs, and "Unit Tester" is equipped with a code interpreter. The tool is triggered when LLM generated codes inside the format ```python\n(code)\n```. In DyLAN, solutions given by code writers are reviewed by code reviewers in at maximum T = 6 rounds. At time step t = 1, 3, 4, 6, code writers gives solutions and code reviewers review it at t = 2, 5. And agent team reformation occurs at t = 4. To ensure the participation of each agent, earlystopping mechanism functions at the third layer and later (t ≥ 3). We use BLEU score in the early-stopping mechanism. We calculate BLEU by sacreBLEU 2 (Post, 2018 ). For answer post-processing, we store all unit tests from the unit tester (if exists in the system) and randomly select the final output from the top 5 code completions from all nodes that pass most tests.In the decision-making task, we set temperature to 0 for DyLAN and all baselines in Table 2 . For ReAct with self-consistency (denoted by ReAct-SC in the table) (Wang et al., 2023b) , we sampled three times for each response. In DyLAN, we optimized four agents from 8 candidates which are depicted in Appendix D. All methods are also conducted on gpt-35-tubo-1106. We did not select LASER (Ma et al., 2023) as a baseline, because it requires GPT-4 for better performance and it extracts all valid actions in each page into function calls, instead of detected by agent itself, which we decide to be a different setting. We divide the pages of the WebShop environment into 3 parts: the initialization page for "searching" part, the item list page for "exploring" part, and the item details pages for "item" part. Thus, we managed to optimize teams for each part from agents in Appendix D: "Search Optimizer", ''Budget Analyst", "Instruction Analyst", "Decision Reflector" for "searching" group, "Decision Maker", "Budget Analyst", "Product Explorer", "InstructionAnalyst" for "exploring" group, and "Budget Analyst", "Description Reader", "Decision Maker", "Result Estimater" for "item" group. Agents interact for at maximum T = 4 steps for each action. We simply concatenate observations of previous actions on each decision. For answer post-processing, we skip invalid actions from the outputs of V T .To implement the agent selection algorithm under DyLAN, only one sentence needs to be injected into the end of the prompt of each node in T-FFN: "Along with the answer, give a score ranging from 1 to 5 to the solutions of other agents. Put all {num p } scores in the form like [[1, 5, 2, ...] ]", where num p denotes the number of predecessors of the node. The prompt functions as the f (s) t,i in Section 3.4 and we extract w t,i,j from its response at the same time when we extract the message that passes between nodes. The scores are normalized so that their sum (∑ N i=1 w t,i,j ) equals 1. To avoid positional bias, responses from agents at previous time step are shuffled when rating.In Algorithm 2, initial contributions are distributed on nodes at the last layer. For reasoning and dicision-making tasks, we uniformly distribute contributions to agents that give consistent answers in the last layer. On code generation tasks, we uniformly distribute contributions in the final round with no syntax error in their answers.In this section, detailed results and additional experiments are presented. The experiments are conducted on five subjects in the GR task (the same as Table 7 ) and the CG task. We sample the subsets with the proportions of 1% and 10% of the original dataset. Agent Importance Score for agent selection is averaged on the subsets, and the selected team is tested on the whole dataset. We raise random selection and human prior selection as baselines. The latter is simulated by GPT-4 prompted by the task and agent descriptions (Appendix D).As shown in Table 9 , by optimizing the team 10% of the original dataset, DyLAN has demonstrated similar performance compared to using the whole dataset, with only 0.2 loss on GR and 0.6 loss on CG. We can observe that even with only 1% of the original dataset, DyLAN could obtain a significant improvement of +3.7 over random selection on CG. From observation, agents augmented with tools are always selected during team optimization under different proportions of the dataset, indicating the effectiveness of Agent Importance Score as an indicator. Please refer to Appendix C.3 for a detailed analysis of the human priors. Besides using GPT-3.5 for DyLAN on CG tasks in Table 2 , we also experiment with GPT-4 in Table 10 . Due to budget limits, we directly reuse the performance reported in the paper of baselines, including LATS (Zhou et al., 2023) , Reflexion (Shinn et al., 2023) , Meta-GPT (Hong et al., 2024) , and AgentVerse (Chen et al., 2024) We further investigated how these agents selected by our unsupervised metric Agent Importance Score differ from human priors (e.g., these predefined roles). To do so, we calculated agent importance scores for 7 agents on each subject of the MMLU dataset. As an example, we show the subjects where the agent of "Doctor" and "Programmer" has the highest agent importance score among all agents in Table 11 agents on the GR task. Green annotation denotes the fields related to the role from the human perspective, which are annotated manually.Though most subjects seems to be reasonably aligned with the role of the agent based on human priors (with green annotations), there are some subjects that do not match human priors, e.g., High School Computer Science as the subject that "Doctor" has the highest score. It exhibits the difference between human priors and the evaluation results of agent importance scores on agents with human-made or LLM-generated prompts. 11 . Green annotation denotes the fields highly related to the role from the human perspective.We also compare current agent selection method that is implemented with Agent Importance Score with the implementation with Human Prior Selection on a few subjects in the MMLU and the HumanEval datasets. For Human Prior Selection, we setup GPT-4 mimicking human selecting the agents for collaborations based on the description of the task and role prompts of each agent. We provide prompt templates in Appendix D. As shown in Table 13 , the implementation with Agent Importance Score steadily outperforms Human Prior Selection.There are two major reasons: (1) Compared to posterior optimization methods, prior selection may not grasp the actual behaviors of agents, and may not understand which agents are most contributory or helpful to others in the real collaboration process. Thus, in High School Statistics, Clinical Knowledge, and Public Relations subjects in the MMLU dataset, prior selection performs even worse than random selection. (2) Human Prior Selection might struggle to understand tool augmentation without peer ratings from fellow agents. From our observation, "Unit Tester" and "Syntax Checker" were not selected for code generation tasks, which may cause lower performance. We tested a few methods on the AR (with simple CoT prompts) and the CG tasks under both low and high temperatures and repeated each experiment three times when the temperature was not zero. We exhibit the experimental results in Figure 4 . From experimental results, we found that DyLAN is more stable on different hyper-parameters.Experiments show that temperature greatly influences arithmetic reasoning and code generation tasks. In Figure 4 , we found that most baseline methods have significant performance drops when temperature increases, but DyLAN shows strong robustness to various temperatures. We surprisingly found that DyLAN gets better results when temperature rises, suggesting it has benefited from diversity instead of being disturbed by low-quality answers of high-temperature agents. The agent team reformation may lead to the higher accuracy by keeping best responses when agents' replies become more diverse. In conclusion, the collaboration of different roles functions effectively and robustly in the dynamic architecture. Nonetheless, higher temperature requires DyLAN to take more API calls (about +0.98 on average on AR tasks (temperature: 0.2 → 1.0)). 4 .We also tested different ranking methods for agent team reformation of DyLAN on the GR task. We tested listwise ranker with our own prompts, pairwise GPT ranker from original LLM-Blender (Jiang et al., 2023) , Elo Score from TrueSkill (Herbrich et al., 2006 ) also implemented with pairwise ranker, and pairwise ranker with Sliding Window algorithm (Qin et al., 2023) . In Table 14 , we show that different ranking methods have a relatively low impact on performance, probably because of strong discrimination ability of GPT-3.5, but pairwise ranking methods always consume higher computational cost. Thus, we chose a listwise ranker in our implementation of DyLAN.Shapley Value is a widely used supervised metric for evaluating contribution of a single agent in a multi-agent system. Though it is not suitable for unsupervised Team Optimization, by viewing it as a ground-truth metric for measuring individual contributions, we can use it for validating the Agent Importance Score. We implement a simplified algorithm for LLMpowered agent collaboration systems. Given that the collaboration process is symmetric in the formulation of the temporal feed-forward network (Section 3.2), we could reduce the permutation set in the original formula (Lundberg & Lee, 2017) to the combination set:EQUATION)where R is the set of agents in the system, C is the combination set of R\{i}, i ∈ R, and Performance denotes the overall performance of the system on the current task, e.g., classification accuracy or Pass@1. The metric requires ground truth and multi-pass results of the system with different subsets of agents. We use classification accuracy for classification tasks and Pass@1 for code generation tasks. However, its computation cost is still too high when the number of agents grows larger due to its combinatorial complexity.To examine Agent Importance Score as an indicator of agent selection with Shapley Value, we also randomly chose three combinations of three agents out of all 7 candidates to assemble a T-FFN and calculated the Shapley Value and the Agent Importance Score on GR tasks. In the GR task, The roles of candidates in DyLAN match the categories of MMLU in human priors, including "Mathematician" and "Programmer" for STEM, "Lawyer" and "Historian" for Humanities, "Economist" and "Psychologist" in Social Science, and "Doctor" for clinical questions in the "Other" category. During experiment with a T-FFN with at least one agent matches the category of the question, it is called a In-Domain scenario; vice versa.In Appendix C.6, we report the correlations between Shapley Values and Agent Importance Scores. We are curious whether Agent Importance Score is an unsupervised substitution for Shapley Value. So, we calculated two list-wise metrics for the similarity between distributions: the KL divergence and ListMLE (Xia et al., 2008) , between Agent Importance Scores and Shapley Value. It indeed shows a high correlation between the distributions of the two metrics during in-domain scenarios.In summary, we use Shapley Value as a self-evident metric for measuring individual contribution, showing that Agent Importance Score emerges as a promising, unsupervised alternative with light computational complexity. In Figure 5 and Figure 6 , we demonstrate the cases of DyLAN on the code generation and the general reasoning tasks, respectively. First, we notice that the communication structure is different between figures, exhibiting dynamic architecture of DyLAN on different queries.The former gives answer at t = 4, the latter at t = 2. We also notice that the answer is gradually growing better along the temporal axis. In Figure 6 , the "Mathematician" agent is selected for the arithmatic query and it gives a correct answer while convincing other agents, agent selection method is effective. We can also observe that the distribution of Agent Importance Score is reasonable. Also, instructing LLM agents to rate scores on predecessors hints them to reflect on predecessors' responses, which might be helpful to give better answers. Last but not least, agents with different roles lead a diverse conversation and make full use of each one, which benefits performance and robustness.Last but not least, we provide qualitative analysis on a failure case in Figure 7 . In DyLAN, agents are assigned roles with prompts extracted from an open-source code base 3 , relative research projects (Du et al., 2023; Shinn et al., 2023; Zheng et al., 2023) , and generation results of GPT-4-0613, besides manual construction. The prompt of each agent are constructed by concatenation of the role prompt and optional tool descriptions, as the system prompt, and instruction prompts. We exhibit the instruction templates of different datasets, the prompts of all agents, and their sources in Table 16 . We annotate the task where each prompt is used in the parenthesis, and the source of each prompt template. We omit the in-context examples of AR tasks from the original dataset of MATH (Hendrycks et al., 2021b) and PHP Zheng et al. (2023) , and WebShop from ReAct (Yao et al., 2023) . The signature of sacreBLEU is "nrefs:1|case:mixed|eff:no|tok:13a|smooth:exp|version:2.3.1". 


Decision: This paper introduces a new framework, DyLAN, for dynamic selection and collaboration of LLM-powered agents tailored to specific tasks and domains. Key strengths include the introduction of an Agent Importance Score for team optimization, which significantly enhances performance over static methods, and the comprehensive empirical validation across various tasks such as code generation, decision-making, general reasoning, and arithmetic reasoning. 
The reviewers appreciated the practical motivation and technical soundness of the approach. By examining the reviewers' comments and the author responses, the AC thinks this paper warrants an acceptance with minor improvements in the camera ready version.

Reviews: review: paper_summary: paper_summary: In this paper, it proposes a new method for dynamically conduct multi-agent collaboration. In incorporates the team operation process and task solving process, which can improve the performance of agent collaborations. The experiments have shown the effectiveness of this method.
strength_weakness: strength_weakness: Shortcoming & Question:
1.It seems that there is no error bar in the results of experiments.
2.What about the comparison with MetaGPT?
3.The paper formatting can be improved, such as reducing the empty blocks within it.
questions: questions: see above
paper_summary: paper_summary: In this paper, the author introduce the Dynamic LLM-Agent Network (DyLAN) framework, designed to facilitate collaboration automatically among multiple LLM agents.The author employs T-FNN to dynamically regulate interactions between different agents and Introducing Agent Importance Score to measure each agent's contribution, reducing manual intervention, which is quite innovative in this field. The whole framework includes two stages: "Team Optimization", which identifies and excludes underperforming agents, and "Task solving" performing inference based on the T-FFNs. 

The author conducted experiments on DyLAN across four tasks: Code Generation, Decision Making, General Reasoning, and Arithmetic Reasoning, comparing it with several previous SOTA methods. DyLAN not only improves performance but also keeps API calls at a relatively low level. To demonstrate the effectiveness of DyLAN, the author conducted lots of experiments and discussions. However, many important findings were included in the appendix. Personally, I believe some crucial discussions such as C.2 and C.8 should be included in the main paper.
strength_weakness: strength_weakness: I don't have any major concerns that would lead me to reject the submission of this article, although:
1. As I said  in  summary, some crucial discussions should be included in main paper.
2. The improvement of DyLAN on the AR task is limited compared to other tasks. What might cause this phenomenon?
questions: questions: 1. In the ablation study, I noticed an interesting phenomenon: the early stop has a significant impact on API calls in arithmetic reasoning and general reasoning tasks. Whereas in the decision-making task, both methods have considerable impacts. For code generation, however, neither method seems to have much effect. What reasons might the author attribute to this phenomenon?
paper_summary: paper_summary: This paper introduces an approach called DyLAN that incorporates a two-stage paradigm: i) Team Optimization, the temporal feed-forward neural network (T-FFN) that selects most contributary agents for the given task and ii) Task solving, the T-FFN facilitate agents to communicate with each others to solve a bunch of a diverse of tasks. To further enhance the efficiency, DyLAN uses early stopping mechanism to terminate the conversation when most of agents in a single layer agree to have a consistent answer.  In the experiment section, the DyLAN is proved to outperform with baselines such as LLM-Blender, LLM Devate, ReAct, and Reflexation, etc, on code generation, decision making, and general reasoning, etc, and shows that DyLAN.
strength_weakness: strength_weakness: 1.  **Extended Query on the Selection and Performance of Agents in the Initial Pool**: "Upon close examination of Figure 5 within the manuscript, there appears to be a lack of clarity regarding the roles adopted by the other agents involved in the code generation task. It is noted that the 'programmer' and 'algorithm developer' agents seem to outperform the rest, which might suggest that these two agent roles are disproportionately influential in the task's outcome. This observation raises the question of whether these two agent roles alone suffice for the task at hand. Similarly, in the mathematical reasoning tasks, the 'programmer' and 'mathematician' agents are indicated as the primary contributors, prompting the inquiry of whether including agents with roles such as 'doctor' or 'economist' might be an overcomplication in the context of mathematical tasks. If the initial agent pool is curated for specificity to different tasks, then the inclusion of seemingly irrelevant agents could be considered redundant and may detract from the efficiency of the overall system. Could the authors provide insight into the rationale behind the selection of these varied roles and how their inclusion impacts the performance and generalizability of the DyLAN framework?"

2. Upon reviewing Figures 5 and 6, it is observed that layers with an even number typically assume a critical function, evaluating the responses generated by the preceding odd-numbered layers. This pattern suggests that the effectiveness of the DyLAN framework may be contingent upon the utilization of highly curated prompts tailored to fit each agent's distinct roles and scenarios. The observed phenomenon raises concerns regarding the scalability and flexibility of the system, particularly in scenarios where the diversity of tasks and roles is substantial. It would be beneficial for the authors to address the necessity for such detailed prompt curation and discuss potential strategies to streamline this process, ensuring that the framework remains adaptable across a wide array of tasks without compromising performance.

However, I find the paper to be of high quality overall; if the authors can provide clear and satisfactory responses to the queries raised, I would be willing to increase the points.

Comments: title: Response
comment: Thank you for the response. I keep the score (6) unchanged.