SYMPHONY: Synergistic Multi-agent Planning with Heterogeneous Language Model Assembly
Authors: Wei Zhu
Zhiwen Tang
Kun Yue
Venue: NeurIPS 2025 poster ()
Keywords: MCTS; Multi-Agent; Sequential Decision Making; LLM
Primary area: applications
TL;DR: We propose a multi-agent frameowork with heterogeneous LLM pool to support better decision making

Abstract: Recent advancements have increasingly focused on leveraging large language models (LLMs) to construct autonomous agents for complex problem-solving tasks. However, existing approaches predominantly employ a single-agent framework to generate search branches and estimate rewards during Monte Carlo Tree Search (MCTS) planning. This single-agent paradigm inherently limits exploration capabilities, often resulting in insufficient diversity among generated branches and suboptimal planning performance.
To overcome these limitations, we propose  $\textbf{SY}$nergistic $\textbf{M}$ulti-agent $\textbf{P}$lanning with $\textbf{H}$eter$\textbf{O}$geneous la$\textbf{N}$gauge model assembl$\textbf{Y}$ ($\textbf{SYMPHONY}$),  a novel multi-agent planning framework that integrates a pool of heterogeneous language model-based agents. 
By leveraging diverse reasoning patterns across agents, SYMPHONY enhances rollout diversity and facilitates more effective exploration.
Empirical results across multiple benchmark tasks show that SYMPHONY achieves strong performance even when instantiated with open-source LLMs deployable on consumer-grade hardware. When enhanced with cloud-based LLMs accessible via API, SYMPHONY demonstrates further improvements, outperforming existing state-of-the-art baselines and underscoring the effectiveness of heterogeneous multi-agent coordination in planning tasks.

SYMPHONY: Synergistic Multi-agent Planning with Heterogeneous Language Model Assembly
Wei Zhu Zhiwen Tang" Kun Yue
School of Information Science and Engineering, Yunnan University, Kunming, China Yunnan Key Laboratory of Intelligent Systems and Computing, Kunming, China zhuwei @stu. @stu. ynu. edu. en, {zhiwen tang, kyue)&ynu edu. cn
Abstract
Recent advancements have increasingly focused on leveraging large language mod- els (LLMs) construct autonomous agents for complex problem-solving tasks. However, existing approaches predominantly employ a single-agent framework to generate search branches and estimate rewards during Monte Carlo Tree Search (MCTS) planning. This single-agent paradigm inherently limits exploration ca- pabilities, often resulting in insufficient diversity among generated branches and suboptimal planning performance. To overcome these limitations, we propose Synegisttic Multi-agent Planning with HeterOgeneous laNgauge model assemblY (SYMPHONY() F.. novel multi-agent planning framework that integrates pool heterogeneous language model-based agents. By leveraging diverse reasoning patterns across agents, SYMPHONY enhances rollout diversity and facilitates more effective exploration. Empirical results across multiple benchmark tasks show that SYMPHONY achieves strong performance even when instantiated with open-source LLMs deployable on consumer grade hardware. When enhanced with cloud-based LLMs accessible via API, SYMPHONY demonstrates further improvements, outperforming existing state-of-the-an baselines and underscoring the effectiveness of heterogeneous multi-agent coordination in planning tasks.
Introduction
The advent of large language models (LLMs) has significantly advanced the development of au- tonomous agents capable performing complex tasks across various domains, including question answering, code generation, and web navigation. These LLM-based agents leverage the extensive knowledge and reasoning capabilities inherent in LLMs make decisions and plan actions. A prevalent approach in this context is the integration of Monte Carlo Tree Search (MCTS) [10] with ILMs, wherein the L.M guides the exploration potential action sequences achieve specific goals This combination has shown promise in enhancing the decision-man mmaking processes of autonomous agents.
14 46 28 12].
Despite the recent progress in integrating LLMs with planning algorithms, existing methods [34] [
predominantly adopt single-model paradigm in which one LLM queried multiple times with identical or slightly perturbed prompts to simulate diverse action branches during MCTS. The underlying assumption that the stochasticity or sampling variance of the model is sufficient to generate rollouts that explore wide range of potential solutions. However, in practice, this approach suffers from a critical limitation: the outputs tend exhibit high similarity across calls, often reflecting the same dominant reasoning pattern learned by the model [14] 46 12]. As result, the generated rollouts lack meaningful diversity, leading to narrow and redundant search trajectories.
4K 29
2 44
Cooresponding author
Code available at hhttps://g/i http://gtttuuu..o//////oo www.:/wEEE/HHEEETT--. http:///hhhh...oo///mmeeeem////e hub/SYMPHONY
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

This deficiency severely constrains the agent's 's exploration capability within the solution space. When the search tree is populated with highly similar branches, the planner becomes susceptible to local optima, and its ability to discover novel or unexpected solutions is greatly diminished. In particularly challenging tasks that require compositional reasoning or multi-step tool use, the agent may fail to identify the correct solution path altogether. Even in cases where the solution is eventually found, the process may involve excessive sampling and token consumption, incurring significant computational overhead. These inefficiencies highlight fundamental mismatch between the need for diverse exploration in planning and the limited variability achievable by repeatedly sampling from a single, monolithic LLM.
To address the above limitations, we propose SYnergistic Multi-agent Planning with Heterogene LaNguage Model Assembly (SYMPHONY). The framework integrates multiple language models into a unified planning system that enhances multi-step reasoning through diversity-awar search, adaptive coordination, and reflective adaptation.
central innovation of SYMPHONY is its heterogeneous agent pool, composed of LLMs with diverse pretraining sources and reasoning styles. Instead of relying on a single agent, SYMPHONY assigns different agents to generate candidate actions each search node, thereby introducing structural diversity into the search tree. This diversity increases the likelihood of generating complementary reasoning paths, reduces model- speccffc biases, and improves performance on complex, multi-hop hop tasks, Empirical results show that expanding model diversity leads to more unique branches per node and consistent gains in task accuracy.
In addition to model heterogeneity, SYMPHONY incorporates several complementary components that further enhance planning performance. A UCB-based scheduling strategy dynamically allocates agents based on historical effectiveness, improving coordination across agents. An entropy-modulatte confidence scoring (EMCS) mechanism calibrates value estimates using agent-level uncertainty, yielding more stable evaluations. Finally, a pool-wise memory sharing mechanism enables agents to learn from past failures through natural language reflections, which shared across the agent pool and incorporated into future prompts. These components together support efficient, adaptive, and robust search behavior.
We evaluate SYMPHONY across three distinct environments that represent key capabilities of L.LM- based agents: multi-hop reasoning (HotpotQA), sequential decision making (WebShop), and code generation (MBPP). Experimental results show that SYMPHONY consistently outperforms strong baselines. In addition improved performance, SY SYMPHONY achieves higher planning efficiency, requiring fewer MCTS node expansions to reach correct solutions. Notably, the framework delivers competitive or superior results even when built upon cost-effective models, demonstrating practical value without relying on high-cost large-scale deployments.
2 Related Work
2.1 LLM-based Planning and Reasoning
Early work on LLM-based reasoning focused on improving consistency and correctness through guided inference. Brown et al. [5] introduced in-context learning with exemplars, while the Chain-
of-Thought paradigm [34 19 encouraged models to generate step-by-step rationales during prediction. Later studies proposed more structured prompting techniques, such as meta-prompting [45 and meta-constrrint-guidee inference [30]. to scaffold the reasoning process with predefined formats global constraints.
To enhance reasoning adaptability, several methods incorporate dynamic feedback. Yao et al. 43 interleaves environment interactions with reasoning steps, Shinn al. [29] enables self-correction through natural language reflections, and code-based approaches Qiu et al. (27). Chen et al. [ iteratively refine outputs based on execution results. Xu et al. (37 further improves performance by prompting LLMs to rearticulate and revise their own reasoning chains.
As tasks grew more complex, researchers began to move beyond linear inference and explore tree-structured reasoning. Tang et al. (31] and Zhang et al. [44] introduced early mechanisms for maintaining and refining multiple hypotheses in dialogue and QA tasks. Wang et al. (32 aggregates
diverse reasoning paths through sampling and majority voting, while Pan et al. 25 dynamically
adjusts decoding strategy between heuristic and deliberative modes.

---

A more principled formulation of structured search appears in Tree of Thoughts 44 which organizes reasoning steps into a decision tree with intermediate backtracking. Building on this, Monte Carlo Tree Search (MCTS) has been applied to guide reasoning more systematically: Hao et al. [14] treat the
I.LM as a world model within a reward-driven search process, and Zhou et al. [46] further integrate
reasoning, planning, and reflection within an MCTS- framework using learned value estimates and external feedback. Shi et al. [28] employ memory- aummor-- single-agent MCTS to enhance decision-making in text-based games.
While prior work focuses on structured reasoning with single model, our approach introduces model-level heterogeneity to enhance diversity and robustness in planning.
2.2 Multi-Agent Collaboration with Language Models
Multi-agent frameworks leverage multiple LLMs or specialized modules to improve reasoning diversity, adaptability, and robustness. Early approaches adopt static task division, assigning agents predefined roles and communication protocols. For instance, ChatDev [26] simulates software development by dividing planning, coding, and testing among fixed-role agents, while MetaGPT [15] enforces similar pipelines using hand-crafted coordination logic. AutoAgents M automates agent instantiation but still operates under rigid, rule-based interaction patterns. Although effective in structured environments, these systems struggle with dynamic tasks due to their limited flexibility.
More recent work shifts toward dynamic coordination, enabling emergent collaboration and context- aware adaptation. Verse (8] adopts blackboard architecture where agents communicate freely through shared language-based memory, CAMEL (20) introduces turn-based agent dialogue for zero-shot task-solving, while AutoGen [35] allows agents to negotiate roles and delegate subtasks on the fly. In complex QA settings, WebGPT [33] decomposes queries into search, summarization, and synthesis subtasks, and MAd [21 employs adversarial debate between LLMs to expose reasoning flaws. MASTER [12] integrates multi-agent behavior into MCTS by adapting the UCT formula using reward signals. AgentCoder [I6] further demonstrates the utility of functional specialization in code generation, coordinating programmer, test designer, and test executor agents within a feedback loop to ensure correctness and completeness.
Unlike existing frameworks that rely on uniform agents and costly coordination, our method enables lightweight, heterogeneous collaboration through principled search and memory sharing.
3 SYMPHONY
3.1 Background and Overview
A Markov Decision Process (MDP) 4 provides a principled framework for modeling sequential decision-making, defined by the tuple (S, A, T, 7), where S is the state space, A is the action space, T : S x A + P(S) defines the transition dynamics, R S x A + R is the reward function, and 7 E [0, 1] is the discount factor. At each timestep, the agent observes a state s E S. selects an action a E A. transitions to new state s' ~ P( s, a), and receives a reward R(s, a). The objective to learn a policy that maximizes the expected cumulative discounted return.
LLMs can be naturally integrated into this framework to support high-level reasoning and decision- making. Specifically, an LLM can serve policy by generating actions conditioned on language- based state representations, as a value function by estimating expected returns from textual trajectories, or world model by predicting future states and rewards through learned knowledge. Unlike traditional reinforcement learning agents that rely on explicit environment modeling and manually designed reward signals, LLM--bseed agents leverage pretraining on large corpora to internalize commonsense, domain knowledge, and structured reasoning. This allows them to operate effectively in complex, open-ended environments with minimal task-specific engineering.
Monte Carlo Tree Search (MCTS) [0 is a sample-based planning algorithm that incrementally builds search tree by balancing exploration and exploitation. It has been widely used in sequential docision-making problems and is well-suited for integration with LLM-based agents, as it allows structured reasoning guided by model-generated priors.
Formally, given the MDP setup, MCTS constructs a partial search tree rooted at the initial state so, iteratively performing four steps: selection, which traverses the tree using upper confidence
3

---

bound to choose promising actions; expansion, which adds new child nodes for unexplored actions; simulation (or rollout), which estimates future rewards using policy; and backpropagation, which updates statistics along the visited path. A detailed description of MCTS can be found in Appendix C
In this work, we adapt MCTS by incorporating L.Mss to guide both the selection and rollout phases, replacing uniform or heuristic strategies with model-informed priors that focus exploration on semantically meaningful regions. Building on this foundation, we introduce SYMPHONY, synergistic multi-agent planning framework designed to enhance both the efficiency and robustness of SYMPHONY extends classical MCTS through several key innovations: a heterogeneous ensemble of LLM agents with diverse inductive priors, UCB-driven adaptive agent scheduling strategy, a pool-wise memory sharing protocol enabling decentralized reflective adaptation, and an entropy-awart utility modulation mechanism for fefiidnnc-caaibratee evaluation. These components collectively promote diverse trajectory generation, context-aware coordination, coherent information propagation and reliable value estimation. The theoretical analysis and complete pseudocode of SYMPHÓNY can be found Appendix A
LLM-based decision-makee
Current Observation:
Riw (loas))
R($m)) Z(ar) (1 - -(ae))
Action: click(< Prev]
This attempt fniled...
Microphone
Action: elick[Ruy New
Ideal for
Studio...
Price: $32.99
Action: clcc((Deerription)
Task still pending...
R
R.
r2
RA
+3
Visit Roward R
Agent, N; R4
Agenty N₂ R2
Sa f
x
Agentt,- No R.,
Hetergeneous Agent Pool
8
8
st
X
x
5
st
SA .r
Update
Memory Module
WehSShp
Wireless Bluetooth DECT Headphones and UsB-C...
Instraction:
musec cancell cnnycvol wh missophont and price lurves Gan. 60.00 dullar Senrch..
The the in...
Price: $331.99
Figure 1: SYMPHONY System Overview.
3.2 Heterogeneous Agent Pool
The heterogeneous agent pool in SYMPHONY is designed to enhance rollout diversity by incor- porating multiple language models with varied inductive biases and reasoning behaviors. Unlike traditional MCTS approuches that rely on repeated queries single language model, SYMPHONY maintains a collection of distinct language models, each serving as an independent agent that contribute complementary perspectives during search. Formally, the agent pool represented as {M , ................... M (ks)
(A))
J. where M, is the 7t agent based on language model after the htth
memory update.
These agents may be instantiated from either open- moerce models that are deployable on consumer- grade hardware, or large-scale cloud-based models accessible only via remote API. Different agents exhibit complementary strengths in reasoning depth, factual precision, abstraction ability, and stylistic preferences, which collectively enhance the system's 's capacity to explore diverse trajectories in the search space.
SYMPHONY employs uniform outptt interface for agents pool. More specifically, the input to the agent pool at the tth step is Po h hz-1), where o (expansion, evaluation, reflection) is the function indicator of language models. Po is the corresponding prompt template, and he is the interaction history /4-1 (8o, do, ', 8 (4-1). This design choice facilitates modularity. New models can be added or removed without altering the core planning algorithm. ensures
compatibility with future advances in LLMs and facilitates efficient reuse of available computational

---

resources under different deployment settings, Prompts for each stage can be found in case studies (Appendix D
3.3 Agent Scheduling
To operationalize the functional heterogeneity of the agent pool, SYMPHONY implements an adaptive dispatch mechanism grounded in the Upper Confidence Bound (UCB) principle, formulating agent selection at each MCTS rollout step as structured multi-armed bandit problem. Rather than relying on static sampling heuristics or fixed priority weights, the framework dynamically calibrates agent choice based on performance statistics, enabling context-sensitive allocation of reasoning capacity.
Formally, for each agent M E MM(), the scheduler maintains a cumulative utility estimate Q(M, ( reflecting empirical rollout effectiveness, Let S MM(e) denotes the of nodes generated by agent M S M!D) = {st+1 T(tt, for agent (((h) as N(u)) Similarly, the cumulative average score for agent M, defined as Q(M!")) Enes R(ss)/IS Mool- The selection priority search node 8, is governed by the (()))
1))))) We record the total invocation count
canonical UCB expression:
N.M In total
UCB(M,(") = ((M ta.
(I)
N( +1
Here denotes exploration-exploiian trade-off hyperparameter, N: = N(M
represents the total number of scheduling decisions made thus far, and the denominator smoothing term ensures initialization- historical returns low invocation frequency, thereby enabling simultaneous exploitation of high- confidence models and exploration of underutilized reasoning modes.
optimism. This formulation favors agents that exhibit either superior
Crucially, this scheduling mechanism is not isolated module but is tightly interwoven with the recursive structure of MCTS, encompassing action generation and reflective evaluation. Following UCT-guided node traversal, frontier node is expanded by dispatching M(()) E (M()) selected via Equation which queried using the expansion prompt:
a, = M² (P Pexpamsion (Se, h1-1))
where h encodes the accumulated interaction trace. The returned actions populate the search frontier with semantically diverse and structurally varied hypotheses.
The agent scheduling mechanism is also used in creating pool-wise reflection memory and node evaluation with EMCS, which will be detailed in the subsequent subsections.
We further establish, from theoretical perspective, that sampling agents from the ensemble with non-zero probabilities leads strictly lower expected error than deterministically selecting single agent. The detailed proof is provided in Appendix
3.4 Pool-wise Memory Sharing
To support continual adaptation without parameter updates, SYMPHONY introduces a pool-wise memory sharing mechanism based on decentralized reflection with natural language. Rather than relying on explicit retraining, agents update their behavior by integrating peer- ger-geeeeatee reflections into prompt-level memory.
When trajectory terminates unsuccessfully, Trail (so, ......, ST), UCB-selected agent M generates structured reflection R. summarizing the failure. This reflection is broadcast to the entire agent pool and treated shared memory block. As reflections accumulate from different agents and episodes, they form diverse collective memory that enhances generalization and coordination.
5

---

To manage memory constraints and maintain efficiency, each agent retains fixed-size buffer updated via a FIFO policy. Reflections are incorporated through prompt-level memory updates:
M Update(MM()) R²), R =M, M (Prettcctoon hz-1))
(3)
This update mechanism enables behavioral adjustment without modifying model parameters, support- ing lightweight and scalable adaptation across heterogeneous agents.
Kntropy-Modulatee Node Evaluation
To improve value estimation during search, SYMPHONY introduces an entropy- modulated node evaluation strategy that adjusts utility scores based on agent confidence. Upon expanding new node Sr, scheduled agent M² E (M performs an internal evaluation, producing value estimate Z (s²) E [0, 1] and a confidence score C((ss) E (0, 1):
(s:), C(st) M4 (St, ht-1))
To integrate these outputs, SYMPHONY employs Entropy-Modulated Confidence Scoring (EMCS), which penalizes uncertain predictions by down-weighting value estimates using the entropy of Berouuli distribution. Here, the confidence score C(ss)) is interpreted as the success probability of Bernoulii variable: the entropy is maximal at C(s)) 0.5, indicating maximum uncertainty, and approaches zero as C(ss) - 0 or (ss,) + 1, reflecting high confidence.
R(s:) ((s) (1 E(sz))
where E(st) = -C(s)) In C(s,)
In(1 (1-
C(s)))
This formulation preserves confident evaluations while suppressing uncertain ones, ensuring that nodes with ambiguous outcomes have reduced influence. Compared to fixed heuristics, EMCS offers uncertainty-aware real-time modulation with minimal overhead, leading to more stable and reliable planning behavior within the MCTS loop.
Experiments
evaluate our approach across three representative tasks spanning reasoning, decision-making, and code generation. Specifically, conduct experiments on: (1) multi-hop question answering using HotpotQA [40] to assess reasoning capabilities; (2) goal-directed interaction on WebShop [41] to evaluate decision-making and planning; and (3) code generation MBPP 3 test the model's ability to reason and produce executable solutions.
4.1 Experiment Settings
SYMPHONY supports flexible agent composition and is compatible with range of language models under different computational constraints. We evaluate two deployment configurations: SYMPHONY-S. designed for consumer- grade hardware, and SYMPHONY-L, which leverages large-scale foundation models via cloud-based APIs.
SYMPHONY-S comprises open-source models that can be executed locally, including 1ren2.5-7B-Instrucee [39], Hiistral-7B-Instrutt-ea [18), and 1-mma-3.1-8BIInttii [13]. This
configuration supports efficient inference with minimal deployment cost. In contrast, SYMPHONY-L comprises high-performance models: GPT-4 [T) Qwen-Max (2024-09-19) [38], and DeepSeek-V (2025-03-24) 22
which operate through API endpoints within inference-as-a-servee infrastructures.
All experiments are carried out under a unified protocol aligned with previous work [29 46 [2]. To
ensure comparability, we apply consistent prompt formats and fixed hyperparameter settings across both configurations, including decoding temperature, planning depth, rollout budget, and number of demonstrations. To mitigate L.M stochasticity, each experiment is repeated 3 times the same data set, and the mean accuracy reported. The detailed hyper-paramett settings are described in Appendix D

---

Table 1: HotpotQA.
Table 2: WebShop.
Table 3: MBPP
Method
Exact Match f
Method
Score
SR t
Method
Pass
(Python)
(Rust) T
CoT
0.34
0.60)
0.29
CaT-SC
ReAct 45
0.39
0.38
II 41
0.62
0.29
GPT-4(CoT)
GPT-4 29
0.683
0,800
0,710
Reflexion
29
0.51
ReAct 43
0.54
0.32
GPT-4(RcAct) 43
ToT
0.55
Reflexion (29
0,64
0.35
Reflexion
29
0.771
0.754
RAP
14
Fine-tuning [1
0.68
0.45
RAP 14
0.714
LATS
46
0.71
AgentKit [36
0.70
LATS 46
Beam Retrieval
44
0.73
LATS [46
0.76
0.38
MetaGPT [5
0.877
MASTER
IP]
0.76
MASTER 12
0,80
MASTER 12
AgentVerse
0.890
0.910
SYMPHONY-S SYMPHONY-L
0.59
Human Expert [41
0.82
0.60
AgentCoder 1k
0.918
0.79
SYMPHONY-S
0.82
0.56
SYMPHONY-S SYMPHONY-L.
0.927
0,946
SYMPHONY-L
0.88
0.72
0.965
0.974
Note: Metrics are normalized the [0.1] range; A dash (-) marks those not reported the publication.
4.2 Reasoning:HotpatQQA
Setup. HA 440 is large-scale benchmark for multi-hop question answering, constructed from Wikipedia and containing approximately 113,000 question-answer pairs. In line with prior work [33 29, 46 [2], employ oracle feedback setting, where the environment immediately indicates whether a selected answer is correct. This setup is designed to isolate and evaluate the agent's 's decision making capabilities during interaction, rather than its ability to generate final answers. Evaluation on this dataset is based on the exact match (EM) metric.
We compare SY SYMPHONY against representative baselines from four categories: (1) Linear reasoning methods such as CoT [4 and COT-SC [33]: (2) Feedback-driven approaches including ReAct (43]
and Reflexion [29] ; (3) Structured reasoning methods such as ToT [42], RAP [14, LATS [46], and
Beam Retrieval [44]: and (4) the multi-agent framework MASTER [12), which builds multi-agent
from the same LLM. Baseline results are taken from Gan al. 12 where GPT-4 is used uniformly
across all methods.
Results. SYMPHONY demonstrates strong performance across all baseline categories. The lightweight SYMPHONY-S outperforms both linear reasoning and feedback-driver baselines, and performs comparably to structured search methods like RAP. The stronger SYMPHONY-L surpasses all structured baselines, including MASTER, achieving state-of-the-- performance on HotpotQA. These improvements reflect SYMPHONY's ability to combine model heterogeneity with coordinated compositional reasoning.
4.3 Sequential Decision Making:WehShop
Setup. WebShop 41 is simulated e-commerce platform featuring over 1.18 million products and 12,000 natural language queries. Agents must navigate the website using browser-like operations (e.g., search, click, select) identify items that satisfy user constraints. Performance is measured by average score, which reflects partial attribute satisfaction, and success rate (SR), which reflects full constraint satisfaction.
We compare SYMPHONY against a comprehensive set of baselines reflecting five categories: (1) Task-native methods including imitation learning (II.), IL+LL, and Human Expert [41; (2) Supervised models such as a fine-tuned LLM MM: (3) Feedback-driven reasoning, including ReAct, Reflexion; (4) Structured search methods including LATS and AgentKit; and (5) the multi-agent framework: MASTER [[2]. All baselines were reproduced under consistent settings by Gan et al. [2] using GPT-4, ensuring fair comparison.
Results. SYMPHONY outperforms all baseline categories. Compared task-native and super- vised approaches, it achieves higher task completion while requiring no domain-specing training. Against feedback-driven and structured search methods, exhibits stronger planning efficiency and generalization. Finally, SYMPHONY-L surpasses the multi-agent MASTER, establishing new performance benchmark. These results underscore SYMPHONY's adaptability across different task-specific execution environments.
7

---

1.0
1.0
1.0
100
Diversity Type
performance
Branch Diversity Proportion (%%
L-Unique
2-Unique
0.8
0.8
0.8
3-Unique
Performance
+ Unique
0.6
0.6
0.66
0.4
0.4
-0
0.2
0.2
0.2
Llama
Owea Mistral
Owen- Li--Llam Liama
Ow Owee--Mise Mistral
Llamo Misral
All Three
0.0
Liama
Oween
Quea- Llama
Mistral
Owen- Ower-aHlse Histral
Llama -lam--Aii Metral
All Three
0.0
Liama
Owen
Owen over-Li Llama Qwen Mistral Liams Mistral
Mistral
All Three
0.0
(b) WebShop
(o)MBPP
Figure 2: Branch Diversity vs. Task Performance. Bars and left y-axis shows the branch diversity,
while lineplot and right y-axis shows the task performance.
4.4 Programming:MBPP
Setup. The Mostly Basic Programming Problems (MBPP) 3 involves multi-step code generation tasks that require condition decomposition, procedural planning, and implementation. Each task provides a description natural language and test suite. Success defined by passing tests. We follow (29) to evaluate both Python and Rust versions of the datasets using the MultiPL-E compiler suite 6]-
Baselines span three major categories: (1) Single-agent methods, including GPT-4 and Reflexion 29
representing the performance ceiling of basic prompting and reactive reasoning: (2) Multi-agent frameworks, such as MetaGPT [5]. AgentVerse [8) and AgentCoder [17), which explore different collaboration strategies; and (3) Search-based approaches, including RAP [4,, LATS [46. and
MASTER [1], which emphasize structured optimization. All baseline results are drawn from or reproduced by Gan et [12] under consistent backbone and data settings.
Results. SYMPHONY achieves strong performance across all baseline categories. Compared to single-agent methods, it demonstrates superior reasoning depth and planning efficiency. Against multi-agent frameworks, SYMPHONY provides more effective solution search via the introduction of heterogeneous agent pool. Compared to search-based approaches, it attains of-te-of-the-rr art performance in cross-language settings, including Rust, a programming language usually ignored by previous works. These results confirm SYMPHONY'S robustness, generality, and computational efficiency in code generation.
4.5 Diversity Analysis
Branch diversity plays crucial role in effective search. To assess its impact, we evaluate how different agent pool configurations affect both task performance and branch diversity across all three tasks using SYMPHONY-S. The expansion width is fixed at 4, and each node's 's candidate branches are categorized by output umiqueness: (a) 4Uniqque: all branches distinct, (b) 3-Unique, (c) 2-Unique, and (d) 1-Unique: all branches identical. Higher frequencies of 3-Unique and 4- 4-Unique indicate more diversified and informative exploration.
As shown in Figure 2 increasing agent heterogeneity, from single-agent to pairwise and full-trio configurations (e.g., Owenn-Mistral+LLamma) leads substantial rise in 4-Uqiq expansions. On MBPP, for example, this proportion exceeds 80% under the full ensemble, compared to under 20% the single-agent setting. This increase in structural diversity strongly correlates with improved accuracy, with SYMPHONY outperforming single-agent baselines by over 30% on MBPP and showing similar trends on HotpotQA and WebShop. These findings highlight the critical role of model-level diversity in enhancing search coverage and reasoning robustness.

---

Table 5: Ablation Study.
Method
HolpoiQA(EM)
WebShop<SROf
MBPP
SYMPHONY-S
0.59
0.56
0.927
w/o Agent Scheduling
0.51
0.48
0.906
w/o Memory Sharing
0.45
0.46
0.871
w/o EMCS
0.51
0.49
0.892
We also experimented with alternative diversity-promoting strategies such adversarial prompting and temperature scaling, but found their effect to be marginal. Detailed comparisons are included in Appendix
4.6 Efficiency and Cost Analysis
Table Comparison of the search tree size on HotpotQA.
TM
33%
COCe Seniia Locter (USD)
Method
K
HotpotQA
#Nodes
676
33.97
RAP
10
0.44
31.53
TOT
50
0.49
84.05
1,32
1.10
LATS
50
0.61
66.65
SYMPHONY-S SYMPHONY-I.
16.39
Figure Comparison of model invocation frequency and final performance on HotpotQA.
10
0.79
9.47
To evaluate SYMPHONY's practicality, we analyze two key aspects: the size of the search tree and the cost of model inference- both crucial to real-world deployment.
Compared methods like LATS, which use a large trajectory budget (K 50) and wider expansion (n = 5) on HotpotQA and WebShop, SYMPHONY achieves comparable or better results with much smaller values (K 10, 4), indicating more compact search process.
We further assess efficiency by measuring average node expansions MCTS on HotpotQA. As shown in Table SYMPHONY consistently requires fewer expansions and even outperforms LATS with a fraction of its search budget, reflecting strong sample efficiency.
In terms of cost, SYMPHONY-L, reduces reliance expensive models by using a heterogeneous agent pool. As shown in Figure 3 GPT-4 is used in only 40% of calls, yet SYMPHONY-L. still outperforms GPT-4-only baselines. Token-level cost details are provided in Appendix F
Together, these results show that SYMPHONY achieves efficient and cost-effective planning through smaller search trees and more economical model usage.
4.7 Ablation Study and Hyperparameter Tuning
To evaluate the impact of SYMPHONY'S core components, we perform a series of ablation studies by selectively disabling key modules, including UCB-based agent scheduling, pool-wise memory sharing, and EMCS scoring. As presented Table [ removing any of these components leads to consistent performance degradation across tasks. These results underscore the effectiveness of dynamic agent scheduling, collaborative memory sharing, and uncertainty-aware scoring enhancing overall system performance.
We further conduct hyperparameter tuning for the UCB exploration coefficient a used in agent scheduling, well as the MCTS parameters and K, which jointly determine the search strategy and computational efficiency. Detailed analyses and results are provided Appendix and Ap- pendix H An extended analysis of architectural robustness under varying agent compositions and noise perturbations also included Appendix F offering deeper insights into SYMPHONY's stability and adaptability. Case studics are included Appendix L

---

Conclusion and Future Work
We present SYMPHONY, a multi-agent planning framework that combines MCTS with diverse pool of language models. By leveraging model heterogeneity and incorporating adaptive scheduling, entropy-modulated confidence scoring, and memory sharing. SYMPHONY improves both search diversity and planning effectiveness. Experiments across multiple benchmarks show consistent gains accuracy and efficiency. Importantly, SYMPHONY performs well even with models that run on consumer-grade hardware, making it practical and scalable solution.
Future research will focus on extending SYMPHONY to unstructured noisy environments, reducing reliance on manually tuned hyperparameters, and integrating fairness and robustness considerations into the planning process. We also plan to explore more efficient memory architectures support scalable, continual adaptation.
Acknowledgements
This work is supported by the Joint Key Project of National Natural Science Foundation of China (U23A20298), Yunnan Fundamental Research Project Yunnan Key Laboratory of Intelligent Systems and Computing (ISC24Y03), and Professional Degree Graduate Practice Innovation Project of Yunnan University (2C-252514097).
Open Project Program of
References
[1] Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08771 2023.
[2] Anthropic. Introducing claude 3.5 sonnet, 2024. URL https: //wp:: wwthaopic..comm/aaaeee
Claude-3-8e s-ntomett
[3] Jacob Austin, Augustus Odena, Maxwell Nye, Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang, Carrie Cai, Michael Terry, Quoc Le, et al. Program synthesis with large language models. arXiv preprint urXiv:2 07732, 2021.
[4] Richard Bellman. Dynamic programming. science, 153(3731):34-77, 1966.
[5] Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et Language models are few-shot learners. Advances in neural information processing systems, 33:1877-19011 2020.
[6] Federico Cassano, John Gouwar, Daniel Nguyen, Sydney Nguyen, Luna Phipps-Costin, Donald Pinckney, Ming-Ho Ho Yee, Yangtian Zi, Carolyn Jane Anderson, Molly Q Feldman, et al. Multipl- A scalable and extensible approach to benchmarking neural code generation. arXiv preprint arXiv:2208.0888 08222, 2022.
[7] Guangyao Chen, Siwei Dong, Yu Shu, Ge Zhang, Jaward Sesay, Börje F Karlsson, Jie Fu, and Yemin Shi. Autoagents: A framework for automatic agent generation. arXiv preprint arXiv:2309.1720 2023.
[8] Weize Chen, Yusheng Su, Jingwei Zuo, Cheng Yang, Chenfei Yuan, Chen Qian, Chi-Min Chan, Yujia Qin, Yaxi Lu, Ruobing Xie, et al. Agentverse: Facilitating multi-agent collaboration and exploring emergent behaviors in agents. arXiv preprint arXiv:2308. 10848, 2(4):6, 2023.
[9] Xinyun Chen, Maxwell Lin, Nathanael Schärli, and Denny Zhou. Teaching large language models to self-debug. arXiv preprint arXiv:2304.00 230.20528.. 2023.
[10] Rémi Coulom. Efficient selectivity and backup operators in monte-carlo tree search. In International conference computers and games, pages 72-83. Springer, 2006.
[11] Hiroki Furuta, Kuang-Huei Lee, Ofir Nachum, Yutaka Matsuo, Aleksandra Faust, Shixi- ang Shane Gu, and Izzeddin Gur. Multimodal web navigation with instruction-finetuned
10

---

foundation models. In The Twelfth International Conference on Learning Representa- tions, ICLR 2024, Vienna, Austria, May 7-11, 2024. OpenReview.net, 2024. URL https //openreview net/forumm.ddeefPBBii..
[12] Bingzheng Gan, Yufan Zhao, Tianyi Zhang, Jing Huang, Li Yusu, Shu Xian Teo, Changwang Zhang, and Wei Shi. MASTER: A multi-agent system with LLM specialized MCTS. In Luis Chiruzzo, Alan Ritter, and Lu Wang, editors, Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguisties: Human Language Technologies (Volume 1: Long Papers), pages 9409-9426, Albuquerque, New Mexieo, April 2025. Association for Computational Linguistics. ISBN 979-8-89176-1894 URL wwg/////// 2025 maaaclllong /76/
htttps:/acclatthlllee
[13] Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ah- mad AI-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, Amy Yang, Angela Fan, Anirudh Goyal, Anthony Hartshorn, Aobo Yang. Archi Mitra, Archie Sravankumar, Artem Korenev, Arthur Hinsvark, Arun Rao, Aston Zhang, Aurelien Rodriguez, Austen Gregerson, Ava Spataru, Baptiste Roziere, Bethany Biron, Binh Tang, Bobbie Chern, Charlotte Caucheteux, Chaya Nayak, Chloe Bi, Chris Marra, Chris McConnell. Christian Keller, Christophe Touret, Chunyang Wu, Corinne Wong, Cristian Canton Ferrer, Cyrus Nikolaidis, Damien Allonsius, Daniel Song. Danielle Pintz, Danny Livshits. Danny Wyatt, David Esiobu, Dhruv Choudhary, Dhruv Mahajan, Diego Garcia-Olano, Diego Perino, Dieuwke Hupkes, Egor Lakomkin, Ehab AlBadawy, Elina Lobanova, Emily Dinan, Eric Michael Smith, Filip Radenovie, Francisco Guzmán, Frank Zhang, Gabriel Synnaeve, Gabrielle Lee, Georgia Lewis Anderson, Govind Thattai, Graeme Nail, Gregoire Mialon, Guan Pang. Guillem Cucurell, Hailey Nguyen, Hannah Korevaar, Hu Xu, Hugo Touvron, Iliyan Zarov, Imanol Arrieta Ibarra, Isabel Kloumann, Ishan Misra, Ivan Evtimov, Jack Zhang, Jade Copet, Jaewon Lee, Jan Geffert, Jana Vranes, Jason Park, Jay Mahadeokar, Jeet Shah, Jelmer van der Linde, Jennifer Billock, Jenny Hong, Jenya Lee, Jeremy Fu, Jianfeng Chi. Jianyu Huang, Jiawen Liu, Jie Wang. Jiecao Yu, Joanna Bitton, Joe Spisak, Jongsoo Park, Joseph Rocca, Joshua Johnstun, Joshua Saxe. Junteng Jia, Kalyan Va- suden Alwala, Karthik Prasad, Kartikeya Upasani, Kate Plawiak, Ke Li, Kenneth Heafield, Kevin Stone, Khalid El-Arini, Krithika Iyer, Kshitiz Malik, Kuenley Chiu, Kunal Bhalla, Kushal Lakhotia, Lauren Rantala- Yeary, Laurens van der Maaten, Lawrence Chen, Liang Tan, Liz Jenkins, Louis Martin, Lovish Madaan, Lubo Malo, Lukas Blecher, Lukas Landzaat, Luke de Oliveira, Madeline Muzzi, Mahesh Pasupuleti, Mannat Singh, Manohar Paluri, Marcin Kardas, Maria Tsimpoukelli, Mathew Oldham, Mathieu Rita, Maya Pavlova, Melanie Kam- badur, Mike Lewis, Min Si, Mitesh Kumar Singh, Mona Hassan, Naman Goyal, Narjes Torabi, Nikolay Bashlykov, Nikolay Bogoychev, Niladri Chatterji, Ning Zhang. Olivier Duchenne, Onur Çelebi, Patrick Alrassy, Pengchuan Zhang, Pengwei Li, Petar Vasic, Peter Weng. Prajiwal Bhargava, Pratik Dubal, Praveen Krishnan, Punit Singh Koura, Puxin Xu, Qing He, Qingxiao Dong, Ragavan Srinivasan, Raj Ganapathy, Ramon Calderer, Ricardo Silveira Cabral, Robert Stojnic, Roberta Raileanu, Rohan Maheswari, Rohit Girdhar, Rohit Patel, Romain Sauvestre, Ronnie Polidoro, Roshan Sumbaly, Ross Taylor, Ruan Silva, Rui Hou, Rui Wang. Saghar Hos- seini, Sahana Chennabasappa, Sanjay Singh, Scan Bell, Scohyun Sonia Kim, Sergey Edunov, Shaoliang Nie, Sharan Narang, Sharath Raparthy, Sheng Shen, Shengye Wan. Shruti Bhosale, Shun Zhang. Simon Vandenhende, Soumya Batra, Spencer Whitman, Sten Sootla, Stephane Collot, Suchin Gururangan, Sydney Borodinsky, Tamar Herman, Tara Fowler, Tarek Sheasha, Thomas Georgiou, Thomas Scialom, Tobias Speckbacher, Todor Mihaylov, Tong Xiao. Ujjwal Karn, Vedanuj Goswami, Vibhor Gupta, Vignesh Ramanathan, Viktor Kerkez, Vincent Gonguet, Virginie Do, Vish Vogeti, Vitor Albiero, Vladan Petrovic, Weiwei Chu, Wenhan Xiong, Wenyin Fu, Whitney Meers, Xavier Martinet, Xiaodong Wang. Xiaofang Wang, Xiaoqing Ellen Tan, Xide Xia, Xinfeng Xie, Xuchao Jia, Xuewei Wang. Yaelle Goldschlag, Yashesh Gaur, Yasmine Babaci, Yi Wen, Yiwen Song, Yuchen Zhang, Yue Li, Yuning Mao, Zacharie Delpierre Coudert, Zheng Yan, Zhengxing Chen, Zoe Papakipos, Aaditya Singh. Aayushi Srivastava, Abha Jain, Adam Kelsey, Adam Shajnfeld, Adithya Gangidi, Adolfo Victoria, Ahuva Goldstand, Ajay Menon, Ajay Sharma, Alex Boesenberg, Alexei Baevski, Allie Feinstein, Amanda Kallet, Amit Sangani, Amos Teo, Anam Yunus, Andrei Lupu, Andres Alvarado, Andrew Caples, Andrew Gu, Andrew Ho, Andrew Poulton, Andrew Ryan, Ankit Ramchandani, Annie Dong, Annie Franco, Anuj Goyal, Aparajita Saraf, Arkabandhu Chowdhury. Ashley Gabriel, Ashwin Bharambe, Assaf Fisenman, Azadeh Yazdan, Beau James, Ben Maurer, Benjamin Leonhardi, Bernie Huang, Beth Loyd, Beto De Paola, Bhargavi Paranjape, Bing Liu, Bo Wu, Boyu Ni, Braden Hancock,
11

---

Bram Wasti, Brandon Spence, Brani Stojkovic, Brian Gamido, Britt Montalvo, Carl Parker, Carly Burton, Catalina Mejia, Ce Liu, Changhan Wang, Changkyu Kim. Chao Zhou, Chester Hu, Ching-Hsiang Chu, Chris Cai, Chris Tindal, Christoph Feichtenhofer, Cynthia Gao, Damon Civin, Dana Beaty, Daniel Kreymer. Daniel Li, David Adkins, David Xu, Davide Testuggine, Delia David, Devi Parikh, Diana Liskovich, Didem Foss, Dingkang Wang. Duc Le, Dustin Holland, Edward Dowling, Eissa Jamil, Elaine Montgomery, Elconora Presani, Emily Hahn, Emily Wood, Eric-Tuan Le, Erik Brinkman, Esteban Arcaute, Evan Dunbar, Evan Smothers, Fei Sun, Felix Kreuk, Feng Tian, Filippos Kokkinos, Firat Ozgenel, Francesco Caggioni, Frank Kanayet, Frank Seide, Gabriela Medina Florez, Gabriella Schwarz, Gada Badeer, Georgia Swee, Gil Halpem, Grant Herman, Grigory Sizov, Guangyi, Zhang, Guna Lakshminarayanan, Hakan Inan, Hamid Shojanazeri, Han Zou, Hannah Wang, Hanwen Zha, Haroun Habeeb, Harrison Rudolph, Helen Suk, Henry Aspegren, Hunter Goldman, Hongyuan Zhan, Ibrahim Damlaj, Igor Molyhog, Igor Tufanov, Ilias Leontiadis, Irina-Flena Veliche, Itai Gat, Jake Weissman, James Geboski, James Kohli, Janice Lam, Japhet Asher, Jean-Baptiste Gaya, Jeff Mareus, Jeff Tang, Jennifer Chan, Jenny Zhen, Jeremy Reizenstein, Jeremy Teboul, Jessica Zhong, Jian Jin, Jingyi Yang. Joe Cummings, Jon Carvill, Jon Shepard, Jonathan McPhie, Jonathan Torres, Josh Ginsburg, Junjie Wang, Kai Wu, Kam Hou U, Karan Saxena, Kartikay Khandelwal, Katayoun Zand, Kathy Matosich, Kaushik Veeraraghavan, Kelly Michelena, Keqian Li, Kiran Jagadeesh, Kun Huang, Kunal Chawla, Kyle Huang, Lailin Chen, Lakshya Garg. Lavender A, Leandro Silva, Lee Bell, Lei Zhang. Liangpeng Guo, Licheng Yu, Liron Moshkovich, Luca Wehrstedt, Madian Khabsa, Manav Avalani, Manish Bhatt, Martynas Mankus, Matan Hasson, Matthew Lennie, Matthias Reso, Maxim Groshev, Maxim Naumov, Maya Lathi, Meghan Keneally, Miao Liu, Michael L.. Seltzer, Michal Valko, Michelle Restrepo, Mihir Patel, Mik Vyatskov, Mikayel Samvelyan, Mike Clark, Mike Macey, Mike Wang, Miquel Jubert Hermoso, Mo Metanat, Mohammad Rastegari, Munish Bansal, Nandhini Santhanam, Natascha Parks, Natasha White, Navyata Bawa, Nayan Singhal, Nick Egebo. Nicolas Usunier, Nikhil Mehta, Nikolay Pavlovich Laptev, Ning Dong, Norman Cheng, Oleg Chernoguz, Olivia Hart, Omkar Salpekar, Ozlem Kalinli, Parkin Kent, Parth Parekh, Paul Saab, Pavan Balaji. Pedro Rittner, Philip Bontrager, Pierre Roux, Piotr Dollar, Polina Zvyagina, Prashant Ratanchandani, Pritish Yuvraj. Qian Liang, Rachad Alao, Rachel Rodriguez, Rafi Ayub, Raghotham Murthy, Raghu Nayani, Rahul Mitra, Rangaprabhu Parthasarathy, Raymond Li, Rebekkah Hogan, Robin Battey, Rocky Wang. Russ Howes, Ruty Rinott, Sachin Mehta, Sachin Siby, Sai Jayesh Bondu, Samyak Datta, Sara Chugh, Sara Hunt, Sargun Dhillon, Sasha Sidorov, Satadru Pan, Saurabh Mahajan, Saurabh Verma, Seiji Yamamoto. Sharadh Ramaswamy, Shaun Lindsay, Shaun Lindsay, Sheng Feng, Shenghao Lin, Shengxin Cindy Zha, Shishir Patil, Shiva Shankar, Shuqiang Zhang, Shuqiang Zhang, Sinong Wang. Sneha Agarwal, Soji Sajuyigbe, Soumith Chintala, Stephanie Max, Stephen Chen, Steve Kehoe, Steve Satterfield, Sudarshan Govindaprasad, Sumit Gupta, Summer Deng, Sungmin Cho, Sunny Virk, Suraj Subramanian, Sy Choudhury, Sydney Goldman, Tal Remez, Tamar Glaser, Tamara Best, Thilo Koehler. Thomas Robinson, Tianhe Li. Tianjun Zhang, Tim Matthews, Timothy Chou, Tzook Shaked, Varun Vontimitta, Victoria Ajayi. Victoria Montanez, Vijai Mohan, Vinay Satish Kumar, Vishal Mangla. Vlad Ionescu, Vlad Poenaru, Vlad Tiberiu Mihailescu, Vladimir Ivanov, Wei Li, Wenchen Wang, Wenwen Jiang, Wes Bouaziz, Will Con- stable, Xiaocheng Tang, Xiaojian Wu, Xiaolan Wang. Xilun Wu, Xinbo Gao, Yaniv Kleinman, Yanjun Chen, Ye Hu, Ye Jia, Ye Qi Yenda Li, Yilin Zhang, Ying Zhang. Yossi Adi, Youngjin Nam, Yu, Wang, Yu Zhao, Yuchen Hao, Yundi Qian, Yunlu Li, Yuzi He, Zach Rait, Zachary Detito, Zef Rosnbrick, Zhaoduo Wen, Zhenyu Yang, Zhiwei Zhao, and Zhiyu Ma. The llama herd of models, 2024. URL httpt:tt/ss/xii..to org/ abs/ 2407 21783
[14) Shibo Hao, Yi Gu, Haodi Ma, Joshua Jiahua Hong, Zhen Wang, Daisy Zhe Wang, and Zhit- ing Hu. Reasoning with language model is planning with world model. arXiv preprint a30iv:2305. 14992, 2023.
[15] Sirui Hong. Xiawu Zheng, Jonathan Chen, Yuheng Cheng, Jinlin Wang. Ceyao Zhang, Zili Wang, Steven Ka Shing Yau, Zijuan Lin, Liyang Zhou, et al. Metagpt: Meta programming for multi-agent collaborative framework. arXiv preprint arXiv:2308 00352, 3(4):6, 2023.
[16] Dong Huang, Qingwen Bu, Jie M. Zhang, Michael Luck, and Heming Cui. Agentcoder: Multi- agent-based code generation with iterative testing and optimisation. CoRR, abs/2312.13010 2023. doi: 10.48550/ARXXI.2212...000 13010. URI.tttss: //doi. org/ 48550/arivv 2312
13010
12

---

[17] Dong Huang, Jie M Zhang, Michael Luck, Qingwen Bu, Yuhao Qing, and Heming Cui. Agentcoder: seatti-ggett-bbsseee based code generation with iterative testing and optimisation. arXiv preprini arXiv:2312. 13010, 2023.
[18] Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, Lélio Renard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas Wang, Timothée Lacroix, and William El Sayed. Mistral 7b, 2023. URL Intpps: http///axxvv.ggabbb////e/a
[19] Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. Large language models are zero-shot reasoners. Advances in neural information processing systems, 15:22199-22213, 2022.
[20] Guohao Li, Hasan Hammoud, Hani Itani, Dmitrii Khizbullin, and Bernard Ghanem. Camel: Communicative agents for" mind" exploration of large language model society. Advances in Neural Information Processing Systems, 36:51991-52008 2023.
[21] Yunxuan Li, Yibing Du, Jiageng Zhang, Le Hou, Peter Grabowski, Yeqing Li, and Eugene Ic. Improving agett debate with sparse communication topology. arXiv preprint arXiv:2406.11776, 2024.
[22] Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, et al. Deepseek-v3 technical report. arXiv preprint 1934472412.99437. 2024.
[23] Rejichiro Nakano, Jacob Hilton, Suchir Balaji, Jeff Wu, Long Ouyang, Christina Kim, Christo- pher Hesse, Shantanu Jain, Vineet Kosaraju, William Saunders, et al. Webgpt: Browser-assisted with human feedback. arXiv preprint arXiv:2112.099: 02.0932, 2021.
[24] Xuefei Ning. Ziman Lin, Zixuan Zhou, Zifu Wang, Huazhong Yang, and Yu Wang. Skeleton- of of-thought: Prompting ilms for efficient parallel generation. arXiv preprint arX1v:2307.15337. 2023.
[25] Jiabao Pan, Yan Zhang, Chen Zhang, Zuozhu Liu, Hongwei Wang, and Haizhou Li. Dynathink: Fast or slow? a dynamic decision-making framework for large language models. arXiv preprint 2rXiv:2407.01000 2024.
[26] Chen Qian, Xin Cong, Cheng Yang, Weize Chen, Yusheng Su, Juyuan Xu, Zhiyuan Liu, and Maosong Sun. Communicative agents for software development. arXiv preprint arXiv:2307.07924 6(3), 2023.
[27] Linlu Qiu, Liwei Jiang, Ximing Lu, Melanie Sclar, Valentina Pyatkin, Chandra Bhagavatula, Bailin Wang, Yoon Kim, Yejin Choi, Nouha Dziri, al. Phenomenal yet puzzling: Testing inductive reasoning capabilities of language models with hypothesis refinement. arXiv preprint arXiv=2310.08559, 2023.
[28] Zijing Shi, Meng Fang, and Ling Chen. Monte carlo planning with large language model for text-based game agents. arXiv preprint arXiv:2504. 16855, 2025.
[29] Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. Reflexion: Language agents with verbal reinforcement learning. Advances in Neural Information Processing Systems, 36:8634-8652, 2023.
[30] Mirac Suzgun and Adam Tauman Kalai. Meta-prompting: Enhancing language models with task-agnostic scaffolding. arXiv preprint arXiv:2401. 224.:2400..1944, 2024.
[31] Zhiwen Tang, Hrishikesh Kulkarni, and Grace Hui Yang. High-quality diversification for task-oriented dialogue systems. arXiv preprint arXiv:2 106.:20906.089,,, 2021.
[32] Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoe Le, Ed Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. Self-consistency improves chain of thought reasoning in language models. arXiv preprint at Xr0:v: 2022.
13

---

[33] Xuezhi Wang. Jason Wei, Dale Schuurmans, Quoc V. Le, Ed H. Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. Self-consistnney improves chain of thought reasoning in language models. In The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023. OpenReview.net, 2023. URL htttt:://tppereeeessss (orum?1d-1PLLNIMME.
net/
[34] Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoe V Le, Denny Zhou, et al. Chain-of Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824-2438 249327, 2022.
[35] Qingyun Wu, Gagan Bansal, Jieyu Zhang, Yiran Wu, Beibin Li, Erkang Zhu, Li Jiang, Xiaoyun Zhang. Shaokun Zhang, Jiale Liu, et al. Autogen: Enabling next-gen Ilm applications via multi-agent conversation. arXiv preprint urXiv:2308.08155. 2023.
[36] Yue Wu, Yewen Fan, So Yeon Min, Shrimai Prabhumoye, Stephen McAleer, Yonatan Bisk, Ruslan Salakhutdinov, Yuanzhi Li, and Tom Mitchell. Agentkit: Structured Ilm reasoning with dynamic graphs. arXiv preprint arXiv:2404. 1483, 2024.
[37] Xiaohan Xu, Chongyang Tao, Tao Shen, Can Xu, Hongbo Xu, Guodong Long, Jian-Guang Lou, and Shuai Ma. Re-reading improves reasoning in large language models. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 15549-15575, 2024.
[38] An Yang, Baosong Yang. Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, et al. Qwen2. 5 technical report. arXiv preprint 1XXi::4422.55155 2024.
[39] An Yang. Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, et al. Qwen2. 5 technical report. arXiv preprint arXiv:2412.1511 2024.
[40] Zhilin Yang. Peng Qi, Saizheng Zhang, Yoshua Bengio, William W Cohen, Ruslan Salakhut- dinov, and Christopher D Manning. Hotpotqa: A dataset for diverse, explainable multi-hop question answering. arXiv preprint arXiv:1809.09600 2018.
[41] Shunyu Yao, Howard Chen, John Yang, and Karthik Narasimhan. Webshop: Towards scalable real-world web interaction with grounded language agents. Advances in Neural Information Processing Systems, 35:20744-2013 20757, 2022.
[42] Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Tom Griffiths. Yuan Cao. and Karthik Narasimhan. Tree of thoughts: Deliberate problem solving with large language models. Ad- vances in neural information processing systems, 36: 16:11180-11822 2023.
[43] Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. React: Synergizing reasoning and acting language models. International Conference on Learning Representations (ICLR), 2023.
[44] Jiahao Zhang, Haiyang Zhang, Dongmei Zhang, Yong Liu, and Shen Huang. End-to-end beam retrieval for multi-hop question answering. arXiv preprint arXiv:2308 X1X0:2308.08873, 2023.
[45] Yifan Zhang, Yang Yuan, and Andrew Chi-Chih Yao. Meta prompting for ai systems. arXiv preprint arXiv:2311.1 1..2211111142. 2023.
[46] Andy Zhou, Kai Yan, Michal Sonpenttokh-Rothmaae Haohan Wang, and Yu-Xiong Wang. Language agent tree search unifies reasoning, acting, and planning in language models. In Proceedings of the stst International Conference on Machine Learning, ICML'24. JMLR.org. 2024.
14

---

A Algorithmic Details of SYMPHONY
The Pseudocode for SYMPHONY can found Algorithm
B Theoretical Analysis
Theorem 1 (Strict Improvement of Agent Pool Sampling). Consider agent pool that samples multiple agents with non-zero probabilities. If satisfies (i) correct coverage, meaning that at each step least one agent outputs the correct action, and (ii) non-triviality, meaning that no single agent is correct on all steps, then the ensemble achieves a strictly lower expected error than any single deterministic agent.
Proof. Let {M1,.... M ) denote the agents, and let az be the ground truth action step Let Cie E {0, 1) be the indicator whether agent M; makes an error at time t, i.e.,
e {
if Mc(ss) + at
otherwise
Then, the total error of agent M, is Ej ie. sampling each M, with probability pi 0,
Σ -=1 Cja, while the expected total error of the ensemble,
m ps Crr
E Enns]
By the assumption of correct coverage, for every t, there exists least one i such that ii,t 0, and since p; > 0, we have:
Cie <
Now consider any model Mj. If there exists any & such that Cj,t but Em P: GiSt 1, then:
Cja = E(Fens] < F3
CLA
By the assumption of non-triviality, such must exist, the inequality is strict.
C Monte Carlo Tree Search (MCTS)
Monte Carlo Tree Search (MCTS) [0] is planning algorithm that balances exploration (trying under-sampled actions) and exploitation (preferring high-reward actions) through iterative tree search. When integrated with LLM-based agents, MCTS leverages the language model's prior knowledge to guide efficient exploration in sequential decision making.
We continue to use the notation introduced in $3 -namlly, the state space S, action set A, reward function R, policy æ, and empirical estimates (s),, visit counts (s),, (p)), as well as the rollout policy Trollout and exploration constant c. MCTS incrementally grows partial search tree rooted at So by repeating the following four steps until a budget (e.g. number of rollouts) is exhausted:
Selection. Starting from the root, descend the tree by choosing each visited node the action:
In N
UCT(s) arg max ((s) + v N(s) SES
(6)
Here Q(s) is the current average return for 8, (ss) the visit count of s.and p the parent node of s, and 0 balances exploration vs. exploitation.
15

---

Algorithm 1 SYMPHONY (80. M, n, D, K,
Require:
Initial agent pool M = {M...... Mm number of generated actions n, depth limit D, number of roll outs K, exploration weight a
1: Initialize action space A, interaction history H
2: Initialize cumulative utility estimate (MM) = 0, selection count N; for each agent M2 e M.
total scheduling decisions Noot!! 0
3: Initialize the search tree with root node so
4: for k <- 0..... K do
5:
for < ...., 6: if is not terminal then for i 6 I..... do
do
7:
8:
Select modulated score node for expansion
Selection via Eq. Agent Scheduling via Eq.
9:
M < arg maxM, EM UCB(M
Update Mr. Na <- Na + Neotal f- Niotat +
11:
Sample (4) M (Pe Pexpansion (s)
(8)
Expansion
(1)
12:
Get of from environment, S 841
O2
13:
M <- arg maxMME EM (CB(MM)
Z(s)"), C(s ~ .1(h) (P Povvttuaiinn (9) --C(s (1) ln E(C(((+1)
(1))))
1+1
(1 (($ In(1 ((+1)
16:
R(s Z(5 11 ((C($441))
(1)
Entropy-Moodulted -Modulated Evaluation
17:
S M = {5441 ~ -T(s,, M, (st, hx-1))}
18:
Update M No + N N, N +1, +1, Q(M)) + Eneesng R(st)
19:
M (1 11:
( (a( E H
20:
Add it to children end for
22:
R + SIMULATE(Ss, k, D d)
Simulation
indicates success then return
24:
end if
26:
end if
27:
8; is terminal or -- then
28:
Get from environment
29:
o not success then
30:
Tfail & (so, ag. a.....,877
31:
R + MM (Prettection CaaiecttoonTfaa (Taii))
Scheduled agent generates reflection
for each agent M, (*) M do
32:
Update agent memory M ( Update( R) end for
33:
Memory Sharing
34:
35:
Backpropagate reward R up the visited path
Update Q and per Eq.
36:
end if
37:
end
end for 39: end for
procedure SIMULATE(SS
41: R 0
for i to d do
43:
if s terminal then break end
Select agent (R) sim and Sample action observe Env.step(a)
45:
r R(s), R Rr
47:
end for
48:
return R
49: end procedure
16

---

Expansion. Upon reaching a leaf node with untried actions, expand by adding one (or more) child node(s) corresponding an unexplored action E A.
Simulation(Rollout).
From the new node execute trajectory of length I under the lightweight policy Wrollouts accumulating the discounted sum to estimate the value of st
T
Rsim
Σ
R(s, at), dt
Trollout (- st)
r=0
Backpropagation.
Propagate Raim up the path (so; an), (T), updating each edge a,a
encountered:
(()) - ((+) + 1,0(() - Q()) + Rsim Q(s)
(8)
N(s)
Under the standard assumptions that every action is eventually explored infinitely often-i.e (s) (s. do for all a-the UCT update guarantees that Q(() (+) almost surely.
In we demonstrate how replacing the uniform heuristic components in Selection and Simulation
with M-M-eerieed priors and rollout policies can dramatically improve sample efficiency by guiding
search toward semantically promising regions of the tree.
D Hyperparameter Settings
To ensure the reproducibility of our results, we detail the hyperparameter settings that led to the best performance across all tasks. Unless otherwise specified, the following parameters shared across all experiments: the number of rollouts per node is set to
4: the exploration constant in UCT is to c 2, following the configuration in LATS [46]: the UCB scheduling parameter is a 20; the temperature for action-sampling agents is set to 0.2 to better follow the input instructions, while the evaluation agents use temperature of ensure deterministic value estimation. Under the S SYMPHONY-S setting, since it involves three models, the system can be comfortably run on three 24GB RTX 4090 GPUs, with sufficient memory headroom.
HotpotQA: We use K = 10 candidate actions per step and adopt 3 few-shot examples.
WebShop: We also set 10, but use single few-shot example tailored to the task format.
MBPP:
We follow the setup in LATS and employ = 8 with a zero-shot prompting strategy.
These settings were selected based on empirical validation and strike a balance between performance and computational efficiency.
E Token Cost Comparison
potential limitation tree-structured reasoning is the increased token consumption it incurs. we systematically evaluate the computational cost of SYMPHONY in comparison to previous methods following the evaluation protocol used in the comparison between LATS [46) and Specifically,
42
14
46 12]
measure the average number of tokens consumed per question the AotpotQA dataset. Token usage data for ToT 42 and RAP [4] obtained from the reproduction results reported by LATS.
MASTER [2]
As shown in the table 6) even though the proposed method shares the same theoretical sample complexity, in practice, our method achieves the best task performance while incurring the lowest token cost, effectively addressing the computational overhead typically associated with tree-based reasoning.
F Analysis of Architectural Robustness
To demonstrate the robustness of our framework, we conduct complementary extension experiments along two axes: (1) detailed analysis of node expansion quality within heterogeneous ensembles, and
17

---

Table Comparison of average token consumption per question and task performance on HotpotQA. SYMPHONY achieves the highest task accuracy while incurring the lowest token cost, effectively mitigating the computational overhead of tree-structured reasoning.
Method
Token Consumption +
Performance
ToT
210,215
0.49
RAP 14
176,500
0.54
LAT'S 46
173,290
0.63
MASTER 12
10,937
0.76
SYMPHONY-L
7,906
0.79
(2) an evaluation of the framework's modular reliability in both single-agent and reasoning-capable model settings.
F1 Expansion Quality in Heterogeneous Ensembles
Explicitly labeling individual expansions as beneficial or detrimental challenging in the absence of task-specific heuristics. Therefore, adopt the overall task success rate practical proxy, where beneficial expansions are those that contribute to successful task completion.
To further validate this, we conducted an additional experiment WebShop, combining GPT-4 with two weaker models, Llama-3.1 1--n---..--B----- -SS--untt and Distral-7B-Instruutvv.... and compared the results against the single-agent baseline SYMPHONY (GPT-4). As shown in Table 7 despite the lower individual performance of the collaborating models, the heterogeneous ensemble outperformed standalone GPT-4, confirming SYMPHONY'S robustness model heterogeneity and the effectiveness of its coordination mechanisms.
We also performed paired-sample t-test to assess the statistical significance of the observed improve- ments. The ensemble variant exhibited a statistically significant gain in the Score metric compared to the GPT-4-only baseline (two-tailed = 0.0106; one-tailed 0,0053), indicating that SYMPHONY consistently produces higher-quality trajectories characterized by more informative intermediate actions and stronger partial progress. This further confirms that heterogeneous model pool can facilitate beneficial node expansions.
Table Comparison of beneficial node expansions on WebShop. Metrics report the average Score and Success Rate (SR). Mistral refers to In0, and Llama refers to Llama-3.1-88 Instruct.
Method
Score
SR
SYMPHONY(GPT-4)
0.80
0.60
(TMPHHONY(GTT44+MssttttLtL
0.83
0.62
F.2 Single-Agent and Reasoning Model Reliability
To validate that components such as pool-wise memory sharing and entropy-modulated node eval- uation can be effectively applied in single-agent settings, and to examine how SYMPHONY in- teracts with modern reasoning-cappe ccapble models that internally perform operations such as chain-of- thought(CoT), reflection, and backtracking, we included both strong single-agent baseline and reasoning-capable models in our experiments.
As shown in Table under identical experimental settings, SYMPHONY(GPT-4) with GPT-4 serving as the sole agent responsible for both node expansion and evaluation, significantly outperforms standalone GPT-4 as well as prior single-agent baselines. This confirms the effectiveness of the other components within our framework. SYMPHONY(Claude)) which integrates Claude-3.5-Sonnee 20240620 2 single agent within framework, also surpasses both the standalone Claude model
18

---

and (GPT-4) confirming that our framework is fully compatible with state-of-the- reasoning models and does not interfere with their internal inference processes.
These results indicate that the SYMPHONY framework not only adapts effectively to single-agent configurations but also supports powerful reasoning models, further demonstrating the robustness of our architecture.
Table Performance of single- modell and reasoning models adapted within our framework across three tasks.
HotpotQA (EM)
WebShop (Score)
WebShop (SR)
MBPP-Python (pass@) I)
MBPP-Rust
Method
(pass@) I)
Claude-3.5
0.51
0.71
0.41
0.894
0.903
SYMPHONY(Claiil
0.76
0.82
0.61
0.947
0.951
SYMPHONY (GPT-4)
0.76
0.80
0.60
0.912
0.924
SYMPHONY-S
0.59
0.82
0.56
0.927
0,946
G Model Selection Strategy
To evaluate the impact of the exploration coefficient in the dynamic agent pool the search performance of the SYMPHONY framework, we investigate how varying the a parameter in the UCB- based scheduling formula influences the trade-off between exploration and exploitation. Specifically, we conduct experiments using SYMPHONY-S across three benchmark datasets.
The experimental results demonstrate that when is small favoring the exploitation of high- performance agents the system's 's performance gradually converges toward that of single dominant agent. In this setting, the scheduler exhibits strong bias toward eurly-rewarding agents, which may have benefited from environmental noise or local optima. This leads to overexploitation, effectively system, and causes the framework to fall into
degrading the agent pool into pseudo- single-inneeeeeee suboptimal local minima. When a is set to a moderate value, the balance between exploration and exploitation becomes insufficiently responsive to the dynamic nature of multi-model collaboration. The scheduler neither fully capitalizes on the strengths of superior models nor effectively mitigates interference from weaker ones. In contrast, with larger a, the system benefits from exploration- driven diversity, showing significant gains in efficiency. This behavior suggests spontaneous avoidance of long-term over-reliance on any single model. The three-model pool is able trigger complementary decisions at critical nodes, enabling more robust collaborative planning. Based on these findings, we select 20 as the optimal configuration.
Performance of Three Tasks Under Different a Values
0.9
0.8
Task Performance
0.7
0.6
0.5
HotPotQAA
0.4
WebShop
MBPP
0.3
0.1
a
Figure 4: Comparison of model invocation frequency and final performance on HotpotQA, highlight- ing the coft-effectiveness of each method.
19

---

Table 9: Parameter study on Number of Branches.
Number of Branches
WebShop(SR)f HotpotQA(EMM)*
MBPP(pass @ 1)1
2
0.34
0.35
0.684
3
0.47
0.46
0.869
4(All datasets used)
0.59
0.56
0.927
H Search Parameter Ablation
In MCTS, the parameters y (the number of child nodes expanded at each step) and (the number of trajectories) are key determinants of the search strategy and computational efficiency. This work is driven by the goal of reducing the high simulation cost inherent in MCTS- frameworks an issue prominently seen in RAP 14 and LATS 46) Compared to RAP, LATS demonstrates better cost-effectiveness, making it a practical baseline for low-resource adaptations.
The parameter K defines the number of trajectories used to search for solutions. Larger K values improve the accuracy of value estimation but significantly increase both computational time and memory usage. LATS conducts ablation studies with K E {10, 30, 50). observing the best perfor- mance at 50 and the worst at K = 10. To evaluate our method under constrained resources, we adopt the smallest tested setting, K = 10. The parameter controls the branching factor, i.e., the number child nodes expanded per step. Higher n values allow broader exploration but incur linear increases in rollout and backpropagation costs. LATS reports experiments with {3, 5, 10), with n = 5 yielding the strongest performance. To explore the trade-off under tighter constraints, we limit our experiments to E {2, 3, 4]. We exclude 1, it reduces the search to single-path traversal, defeating the objective multi-path reasoning.
Experimental results (Table 9 ultimately confirm the effectiveness of our cost-efficient design: we achieve substantial performance gains while significantly reducing the resource demands of the MCTS- based search process. Notably, our approach refrains from scaling up parameter counts to pursue marginal improvements, as such gains would come at the expense of considerable computational overhead.
I Alternative Diversity Enhancements
To validate the necessity of our dynamically scheduled heterogeneous agent pool, we empirically eval- uated two commonly used diversity-enhancemee strategies, adversarial prompting and temperature scaling. We found that neither achieved comparable task performance.
For adversarial prompting, we inserted an explicit instruction at each node to discourage similar- ity with preceding nodes, encouraging divergent strategies through prompts such as: ''The current response must not replicate previous nodes and should demonstrate exploratory thinking" For temper- ature scaling, we varied the sampling temperature within the typical range of [o, 2), covering outputs from deterministic to highly stochastic, while keeping all other generation parameters constant. We use SYMPHONY-S, a heterogeneous agent pool composed of three locally deployable language mod- els, set the temperature to 0.2, as the baseline configuration. Based on this setup, we systematically adjust the aforementioned diversity methods and evaluate their performance across three benchmark task sets.
Results showed that adversarial prompting, which enforces diversity by design, actually degraded task performance, suggesting that forcing dissimilarity can conflict with task coherence. Temperature adjustments had minimal effect on outcome quality, with the best performance observed at lower temperatures. This indicates that diversity introduced by adjusting individual model outputs does not match the structural diversity achieved through heterogeneous agent coordination. Moreover, in complex maaiing tasks, strict adherence to input instructions is essential, making low- temperature decoding more suitable. Overall, temperature alone proved insufficient for sustaining meaningful diversity under dynamic, multi-tum conditions.
20

---

Effeet of Diversity Methods Across Tasks
1.0
Bascline
Adversarial Prompt
0.8
Temp=0.0
Performance Score
Temp-1.0
0.7
Temp-2.0
0.6
0.5
0.4
0.3
0.2
HotPotQA
WebShop
MBPP
Figure 5: Performance comparison of diversity strategies across tasks.
Limitations
Despite its effectiveness across range of planning and reasoning tasks, SYMPHONY has several limitations. First, the current framework assumes access to structured environments with reliable feedback signals-such oracle evaluation or deterministic execution maces-which may not extend to more open-ended, dynamic, or noisy real-world settings. Generalizing SYMPHONY to less predictable environments will require more robust uncertainty modeling and adaptive feedback handling.
Second, the framework relies on manually tuned hyperparameters (e.g., trajectory count, expansion width, agent composition), which may require task-specific adjustment to balance performance and cost. This reliance limits out-of-the-box generalization and highlights the need for automated, task-adaptive hyperparameter optimization strategies.
Finally, while SYMPHONY promotes diversity through multi-agent coordination, it does not explic- itly address fairness, bias mitigation, or robustness against adversarial inputs. As multi-agent LLM systems gain traction in high-impact applications, incorporating safeguards for ethical reliability and adversarial resilience will be critical in ensuring trustworthy deployment.
K Broader Impacts
This work introduces SYMPHONY, multi-agent planning framework that enhances reasoning and eecision-makingg efficiency large language model (LLM)-based systems. The proposed framework has the potential for broad positive societal impact. By improving the efficiency and robustness of complex task-solving using LLMs, SYMPHONY can benefit a range of domains including education, scientific research, healthcare, and digital services. In particular, its ability to achieve competitive performance using open open-source models and resource-aware deployment configurations makes it accessible for low-resoure settings, thereby promoting the democratization of advanced AI capabilitics.
However, this work also raises several potential risks. The integration of multiple LLM agents with autonomous planning capabilities may introduce challenges controllability, transparency, and unintended behavior propagation. Misuse in high-stakes decision- makinng scenarios- such such as legal or financial automation- could amplify existing biases or errors encoded in the underlying models. Moreover, reliance on API-accessible proprietary models (as in SYMPHONY-L) may reinforce concentration of power among small number of providers, raising concerns about accessibility, fairness, and data governance.
21

---

L
Case Study
L.1 HotpotQA
Solve question answering task with interleaving Thought, Action, Observation steps. Thought can reason about the current situation, and Action can be three types:
(1) Search[entity), which searches the exact entity Wikipedia and returns the first paragraph if it exists. If not, it will return some similar entities to search.
(2) Lookup/keywordd which returns the next sentence containing keyword in the current passage.
(3) Finish(answer]] which returns the answer and finishes the task.
Question: Human Error" is the season finale of the third season of a tv show that aired on what network?
Agentsr
Thought need to search "Human Error" season finale to find out which TV show it belongs to, then determine the network it aired on.
Action Search["Human Error" season finale]
Observation I: Could not find "Human Error" season finale. Similar: ['House season 3', 'Endgame (Star Trek: Voyager) (redirect from Star Trek: Voyager finale)', "List of House episodes (section Season I (2004-05))', 'Chonlathorn Kongyingyong', 'Omar Avila"].
Agent
Evaluation: The agent correctly identifies the need to disambiguate the episode title by first deter- mining the TV show associated with "Human Error." However, the current search query is overly narrow and fails to yield relevant results, returning loosely related pages. This suggests a need to refine the search strategy strategy-perhaps by including additional keywords like "TV show" or "House" (if known), or directly querying for episode lists that include "Human Error" as title. While the reasoning direction is sound, the execution falls short in terms information retrieval.
Value Estimate: 0.40
Confidence Score: 0.55
Agent
Thought The previous search for "Human Error" season finale did not yield the desired results, likely because the exact phrasing was not found. I should instead search for "Human Error" TV episode identify the show belongs
Action 2: Huuman Error" TV episode] Observation 2: Could not find "Human Error" TV episode. Similar: ['House season 3 (redirect from Human Error (House episode))', 'Human Error (Star Trek: Voyager)', 'Daniela Farinacci', 'Leeanna Walsman', 'List of House episodes').
Agent.,,
Evaluation: This reformulated query shows improved specificity over the previous one by targeting "TV episode" rather than "season finale," and the results include relevant candidates such as House and Star Trek: Voyager. These matches suggest that the agent now on the right track identifying the correct show. However, disambiguation is still needed to determine which of the two shows had an episode titled "Human Error" as its season finale.
Value Estimate: 0.65
Confidence Score: 0.70
Agent.., upp
Thought 3: The search for "Human Error" TV episode suggests that is episode from the show "House." should now confirm this and then find out on which network "House" aired.
Action 3: Search/Human Error (House episode)]
Observation 3: The third season of House (also called House, M.D.) aired on FOX from September 5, 2006[1] to May 29, 2007.[1] Early in the season, House temporarily regains the use of his leg due to ketamine treatment after he was shot in the season two finale.[2] Later in the season, he leaves stubborn patient in an exam room with thermometer in his rectum.[3] Because House is unwilling to apologize, the patient, who turns out to be a police detective, starts an investigation around House's Vicodin addiction.[4) David Morse joined the cast for seven episodes as the detective Michael Tritter. He was cast for the role after working with House creator David Shore CBS' Hack. [6]. Jane Adams, Erich Anderson, Omar Avila, Jurnee Smollett-Bell Geoffrey Blake, Tanner Blaze, Ben Bledsoe, Mare Blucas, Mika Boorem, Kacie Borrowman, David Bowe, Paula Cale, Helen Carey,
22

---

Greg Cipes. Monique Gabriela Curnen, Dabier, Meredith Eaton, Mary Elizabeth Ellis, Shonda Farr, Arabella Field, Colleen Flynn, Lyndsy Fonseca, Thomas Mikal Ford, Patrick Fugit, Carla Gallo, Christopher Gartin, Jason Winston George, Skyler Gisondo, Meta Golding, Meagan Good, Eve Gordon, Joel Grey, Tracy Howe, Carter Jenkins, Dustin Joiner, Heather Kafka, Krista Kalmus, Tory Kittles, Clare Kramer, Deborah Lacey, Nick Lane, John Larroquette, Brian Leckner, Sheryl Lee, Geoffrey Lewis, Braeden Lemasters, Tess Lina, Jodi Long, Donald Sage Mackey, Bailee Madison, Wendy Makkena, Stephan Markle, Dave Matthews, Shyann McClure, Michael Medico, Joel David Moore, Zeb Newman, Jenny O'Hara, Slade Pearce, Piper Perabo, Adina Porter, Kathleen Quinlan, Annie Quinn, Anne Ramsay, Mercedes Renard, Jake Richardson, Tyson Ritter, Jemny Robertson, Alan Rosenberg. Vyto Ruginis, Jessy Schram, Dustin Seavey, Alyssa Shafer, Kurtwood Smith, Tony Spiridakis, Josh Stamberg, Cassi Thomson, Cooper Thomton,, Beverly Todd, Mandy June Turpin, Raviv Ullman, Pruitt Taylor Vince, Jascha Washington, Damien Dante Wayans, Katheryn Winnick and Jamison Yang.. Season three's most-viewed episode was "One Day, One Room", which was watched by almost 27.4 million viewers.[7] An average 19.4 million viewers watched season three of House, making it the seventh most-watched show of the 2006-2007 television season.[8] Jennifer Morrison and Joel Grey submitted the episode "Informed Consent" for consideration in the categories of Outstanding Supporting Actress in a Drama Series and Outstanding Guest Actor in a Drama Series respectively at the 59th Primetime Emmy Awards.[9) Neither was nominated..
Agentxr
Evaluation: The agent's 's current action successfully confirms that "Human Error" is indeed an episode from House, and provides the key information that House aired FOX. This marks critical turning point in the reasoning trajectory, as it directly addresses both parts of the original instruction. The query demonstrates precise targeting and resolves prior ambiguity. Though the observation is verbose, the relevant evidence is clearly retrievable, supporting a confident next step toward answer extraction.
Value Estimate: 0.92
Confidence Score: 0.85
Agent.;
Thought 4: The observation confirms that "Human Error" is an episode of the TV show "House." Now, need to find out which network "House" aired.
Action 4: Search|House (TV show) network]'
Agent.,
Evaluation: The agent takes a natural follow-up step to verify the broadcasting network House, transitioning smoothly from content validation to final answer retrieval. Given that the previous observation already included the relevant network information (FOX), this action is slightly redun- dant. However, it may still reinforce answer certainty, especially in ambiguous noisy retrieval environments. The move reflects cautious reasoning but could be optimized for efficiency. Value Estimate: 0.78
Confidence Score: 0.70
Agent.
Thought 4: The search confirmed that "Human Error" is an episode from the TV show "House." Now, I need to find out which network "House" aired on to answer the question. Action 4: Finish[FOX]
Agent...
Evaluation: The agent effectively synthesizes information gathered across previous steps and delivers a final answer consistent with the original question. By selecting Finish[FOX], it correctly identifies the network on which House aired, demonstrating accurate extraction, integration of episodic details, and goal alignment. This reflects a strong end-to-end reasoning trajectory with minimal unnecessary exploration.
Value Estimate: 0.95
Confidence Score: 0.92
L.2 WebShop
Solve shopping tasks by alternating between Observation and Action steps. The observations provided by the environment help infer the current situation. Actions are divided into two types: (I) search(entity]: Search for the exact entity on the product website. If it exists, return products that
23

---

match the entity's requirements.
(2) click[keywords]: Click on keywords listed inthe latest description returned to the page.
Instruction: would like a ounce bottle of bright citrus deodorant for sensitive skin, and price lower than 50.00 dollars
Agentsr
Action: search[3 ounce bright citrus deodorant sensitive skin]
Observation:
[Back to Search]
Page (Total results: 50)
[Next>]
[B078GWRCIJ] Bright Citrus Deodorant by Earth Mama Natural and Safe for Sensitive Skin, Pregnancy and Breastfeeding, Contains Organic Calendula 3-Ounce $10.99
[B078GTKVXY] Ginger Fresh Deodorant by Earth Mama Natural and Safe for Sensitive Skin, Pregnancy and Breastfeeding, Contains Organic Calendula 3-Ounce $10.99
BO88KVVJ4XN Barrel and Oak Aluminum-Frie Free Deodorant, Deodorant for Men, Essential Oil- Based Scent, 24-Hour Odor Protection, Cedar & Patchouli Blend, Gentle on Sensitive Skin (Mountain Sage, 2.7 oz, 2-Pack) $15.95
Agent
Evaluation: The search action aligns well with the user's instruction, effectively capturing all key constraints- "3 ounce," " "bright citrus," "deodorant," and "sensitive skin." The top result, Bright Citrus Deodorant by Earth Mama, is direct match in terms of scent, size, and suitability for sensitive skin, and priced well below the $50 limit. Additional results, while slightly divergent scent or packaging, also remain within the acceptable range and provide relevant alternatives. This suggests that the current node is promising for achieving the user's goal with minimal correction required in future steps.
Value Estimate: 0.92
Confidence Score: 0.88
Agent Mo
Action: think/B07GGRRCJJ The product's description aligns well with the user's specified require- ments, indicating strong match terms of both features and preferences. This suggests that the current reasoning trajectory is effective. The agent proceeds by clicking on the item to access detailed information and verify its suitability before making a final decision. Observation: [Back to Search]
Page (Total results: 50)
[Next>]
[B078GWRCIJ] Bright Citrus Deodorant by Earth Mama Natural and Safe for Sensitive Skin, Pregnancy and Breastfeeding, Contains Organic Calendula 3-Ounce $10.99
[B078GTKVXY] Ginger Fresh Deodorant by Earth Mama Natural and Safe for Sensitive Skin, Pregnancy and Breastfeeding, Contains Organic Calendula 3-Ounce $10.99
[B08KBVJ4K J4XN] Barrel and Oak Aluminum- Deodorant, Deodorant for Men, Essential Oil- Based Scent, 24-Hour Odor Protection, Cedar & Patchouli Blend, Gentle Sensitive Skin (Mountain Sage, 2.7 oz, 2-Pack) $15.95
Agent.r,
Evaluation: The agent's decision to engage reflective reasoning (think demon- strates goal-aware deliberation. It identifies that the top-listed item matches the user's constraints on scent, size, skin sensitivity, and price. The action prioritizes validation over haste by choosing to inspect the product further, which is prudent step given that detailed attributes- such as ingredients or user ciiiews-oftee influence final suitability. This intermediate reflection helps prevent premature commitment and strengthens decision robustness.
Value Estimate: 0.95
Confidence Score: 0.91
Agent.,,
Action: click[BO7SGWRCUJ
Observation:
[Back to Search]
K Prev]
24

---

scent [assorted scents][bright citrus lavender][ginger fresh][simply non-scents] size [travel (4-pack)][3 ounce (pack of 1)[[3-ounce (2-pack)] Bright Citrus Deodorant by Earth Mama Natural and Safe for Sensitive Skin, Pregnancy and Breastfeeding, Contains Organic Calendula
3-Ounce Price: $10.99
Rating: N.A.
Description]
[Features]
[Reviews]
[Buy Now]
Agent.r,
Evaluation: The agent's decision click[B078GWRCU) is both timely and effective. After iden- tifying strong candidate in the search results, this action transitions the agent into product-level inspection- an essential move for verifying detailed specifications before commitment. The ob- servation confirms multiple scent and size options, with the "bright citrus" 3-ounce variant clearly available at $10.99, well within budget. The agent now has access to rich product metadata (e.g.. description, features), which supports final validation comparison if needed. This action reflects coherent and informed progression in the planning trajectory. Value Estimate: 0.93
Confidence Score: 0.90
Agent.r
Action: click[bright citrus]
Observation: You have clicked bright citrus.
Agent
Evaluation: The action click[bright citrus] shows the agent verifying or reinforcing the selected product sprifnt--peciffcclll the scent preference. While this step may seem redundant given that the desired scent was already visible, it ensures explicit confirmation before finalizing the purchase. This cautious move reflects a preference for certainty in decision-making making, though it could slightly delay progress if overused.
Value Estimate: 0.80 Confidence Score: 0.72
Agent My
Action: click[3 ounce (pack of 1)]
Observation: You have clicked ounce (pack of I).
Agent
Evaluation: The action click[3 ounce (pack of 1)] confirms the exact size requirement specified in the instruction. This is a critical verification step, as selecting the wrong size (c.g.. multi-pack or travel set) would violate the user's constraints. While the size already visible, this explicit confirmation ensures the agent on track to meet all specified conditions before purchase. Value Estimate: 0.85
Confidence Score: 0.78
Agent.r
Action: click[Buy Now]
Agent.,
Evaluation: The action click[Buy Now] represents the final execution step, completing the task with product that satisfies all user constraints scent, size, skin sensitivity compatibility, and budget. Given the prior confirmations and detailed alignment with the instruction, this decision reflects confident, goal-aligned conclusion to the reasoning trajectory. Value Estimate: 0.95
Confidence Score: 0.90
25

---

L.3 MBPP
You are an AI Python assistant. You will be given your previous implementation of function, series of unit tests results, and your self-reflection your previous implementation. Write your full implementation (restate the function signature).
Given a list of integers nums and an integer target, find the longest contiguous subarray such that the sum of its elements is less than or equal target. multiple such subarrays exist with the same maximum length, return the one that appears first. Return the subarray itself (not just its length or indices).
Agent
Thought 1: To find the longest contiguous subarray whose sum is less than equal the target, I can use sliding window approach. I'll expand the window to the right, and when the sum exceeds the target, I'll shrink it from the left. I'll keep track of the maximum length window found and return the corresponding subarray. Action 1:
def cebarttsbarrayybbb... sum sum limit (nums: List [int], len (nums)
target: int) -> List [int]:
left, right =
max_ max_length
current_eun
result []
while right <
current_sume nums [right]
while current_sur target:
current_ sum nums [left]
left +=
if right left >= max :ength:: max max_length right left 1 result nums [left:right*:: right +=
return result
Observation:
[unit test results]:
Tests passing:
assert songest_subarray_.ite hitbbarray_withhha_liiiiiii ([1 2, 3, 4, 5] 8) [1, 2, 31
assert assert 3] assert Longett.ubrraay.iibbemm.m..aa
2, 3, 4, 5] 15) [1, 2, 3, 4, 5]
-1, 2, -2, 3, 33,, 2) == [1, -1, 2, -2
10) 0
assert assert
0)
-5) 0]
Tests failing:
assert Longet_.ubarray..tttt.uuuuu ([5, 6, 8, 9], 4) 0
output: [5]
Agent.r
Evaluation: The implementation failed the where no subarray fulfills the condition. The issue in the implementation due to the use >= instead of > in the condition to update result. Because of this, it returns subarray even when the sum is greater than the target, as it still updates the result when the current subarray length is equal to the previous longest subarray length. To overcome this eπrоr, we should change the condition only update the result when the current subarray length is strictly greater than the previous longest subarray length. This can be done by replacing >= with > in the condition.
Value Estimate: 0.85
Confidence Score: 0.90
Agent
Thought 2: To optimize the search for the longest subarray with sum < target, can pre-expand the window greedily long as the sum remains within the limit. Then, I'll proceed with the standard sliding window approach: adding elements to the right, and shrinking from the left when the sum
26

---

exceeds the target. I'll track and update the longest valid subarray found.
Action 2:
def longest_subarray.uu
sum_ limit (nums: List [int], target: int) List[int]
n len (nums)
left, right = 0,
max max_length
current_sum
result []
while current_aus + numa [right] <= target:
current_suu +- nums nums [right]
right +- 1
while right
current_ eum numa numa(rigtt [right]
while current_sum target:
current_ sum nums [left]
left +=
max_ lengtent max max_length right left + 1 result nums rleht:right++] right +-
if right left
return result
Observation:
[unit test results]:
Tests passing:
assert ommgstt.uuarrryy.ittbss_____ 10) == 0
assert assert Longess_.ubarrryyy.ttbbss.l... 0) ==
-5) 0
Tests failing: assert
2, 3, 4, 5], 8) == [1, 2, 3] output:
list index out of range
assert
2, 3, 4, 5], 15) [1, 3, 4, 5]
output:
1ist index out of range assert
1im 6, 7, 8, 9], 4) 0 # output:
list index out of range assert www.rraaawwthhhaaaae
-1, 2, -2, 3, -3], 2) [1, -1, 2, -2,
3] output:
list index out of range
Agent,,
Evaluation: The implementation failed 4 out of the 7 test cases due to IndexError. The issue stems from the while loop while current_ sum nums(right] <= target: which directly accesses nums(right) without checking if right is within the bounds of the list. This results in runtime error when right goes beyond the list length. To overcome this error, we need to add a bounds check for the right variable in the mentioned while loop. We can modify the loop condition while right len(nums) and current_ + nuih(right] <= target: This change will ensure that we only access elements within the bounds of the list, thus avoiding the Index(Error.
Value Estimate: 0.80
Confidence Score: 0.85
Agent
Thought 3: Negative numbers break the standard sliding-window pattern, so switch prefix-sum sean that works for any sign. For each start index reuse prefix sums to test only subarrays that can beat the current best length; this is O(n²) in the worst case but small and safe for the given tests, and it guarantees the earliest longest match.
Action 3:
def bongest_subarray_wiie sum limit (numa: List [int], target: int) List[inti] ***
Return the earliest longest contiguous sub-array whose sum is <= target.
If no such sub-array exists, return an empty list.
27

---

len (nums) if == 0: return
# Pre-compute prefix sums: pref [i] sum (nums [:1])
pref [0] (n 1)
for range(n) (n)
pref pref[i] nums [i]
best_len best_start
for start in range(n):
# Only try end positions that can beat current best_len for end in range (start + best_len, n): if pref cur_len end start if cur_len best_len: best_len cur our_len best_start start
pref [start] target:
return nums (best_sssst best_atart beat_len]
Observation:
[unit test results]:
Tests passing:
assert Longest..barrrry..ii..mm_tittttt 10) = 0
assert boggettt.ubarrryy...bbssumm_laad 0) 0
assert longett.ubbrrrxyy.ttb___mmmmmmm
-5)
assert Loggett_cuuarrryyyxtttsss______
2, 3, 4, 5), 8) [1, 2,
assert tonget_.ubarraa.ttttamm...iiiia_ 2,
5] 15) [1, 2, 3, 4, 5]
assert loggst_.ubarrryy.tttbummaaiitaat 6, 9 4) = 0
assert longes..ubbrrrg.tttt.mmm..../// -1,
3, 33, 2) == [1, -1, 2, -2,
3]
Tests failing:
Agent
Evaluation: The implementation employs sliding window (two-pointer) technique to efficiently identify the longest contiguous subarray whose sum does not exceed the given target. It successfully handles edge cases such empty input, all elements exceeding the target, and arrays with negative numbers- paseeengspanssngggggee all provided test cases. The algorithm maintains a moving sum and shrinks the window necessary, ensuring out-of-bound errors infinite loops. The structure is clean and robust. One limitation that if there are multiple subarrays of the same maximum length, may not always return the earliest one. However, since the task does not explicitly require this, the current approach is considered correct and effective. Value Estimate: 0.95
Confidence Score: 0.97
Reflection Example:
I appeared to have retrieved the correct information about The Oberoi Family and the location it's head office, and provided corresponding answer. However this answer does not exactly match the ground truth answer so should try a different wording, such as Delhi.
The failure in the previous trial occurred because the action to directly click "Buy Now" was executed without first ensuring that the selected item met all the specified criteria, such as being a long clip-in hair extension, natural looking, and priced under $40.00. The initial action bypassed the necessary steps of verifying these details.
In previous attempt. checked if string was a palindrome by comparing it to its reverse, but failed normalize the string (ignoring case and non -lnaanhhanierric characters). This caused errors for inputs like "A man, a plan, canal: Panama". Proper preprocessing with lowercase conversion and filtering non-alphanumeric symbols would have avoided this issue.
28

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
The abstract and introduction clearly articulate our key contributions, which are consistently supported by the empirical results presented in the paper.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification: This paper discusses the limitations in Appendix
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]

---

Theoretical assumptions and complete proofs are provided in Appendix B which rigorously demonstrate the effectiveness of the proposed method.
Justification:
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
The paper provides detailed descriptions of the experimental setup, model variants, evaluation metrics, and baseline implementations in Section 4 and Appendix D allowing faithful reproduction of the results.
Justification:
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm.
(b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g., a large language model), then there should either be way access this model for reproducing the results way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reprodueibility may be tricky in some cases, in which authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path reproducing verifying the results.
5. Open access to data and code
30

---

Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Question:
Answer: [Yes]
Justification: We have submitted the code anonymized zip file. Upon acceptance of this work, all code and data will be promptly released to public with comprehensive documentation to ensure full reproducibility.
Guidelines:
The answer NA means that paper does not include experiments requiring code. Please see the NeurIPS code and data submission guidelines (https://nips.cm cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
weellii/ggiies//CC Caweaee//ooSSSbbbiicaa SoroodeeumisssiooPPe.. www.iiissiiooiiiooioomommm..mimi for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results the new proposed method and baselines. If only subset of experiments reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: The paper provides comprehensive details about training setups, including data splits, hyperparameter, model configurations, and optimization strategies, which are detailed in Section and Appendix [
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to a level of detail that is necessary appreciate the results and make sense of them.
The full details can provided either with the code, appendix, or as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: The reported experimental results are averaged over multiple independent runs ensure robustness and reflect the stability of the methods.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by enror bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
31

---

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not show in tables figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification:
The paper specifies the model variants and computing hardware used for all experiments, which sufficiently satisfies reproducibility requirements.
Guidelines:
The answer NA means that the paper does include experiments.
The paper should indicate type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (c.g., preliminary or failed experiments that didn't 't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics Ethics/https:
Answer: [Yes]
Justification:
The research complies with the NeurIPS Code of Ethies.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (c.g.. if there special consid- eration due to laws or regulations in their jurisdiction..
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: This broader impacts of this work is discussed in Appendix K
Guidelines:
The answer NA means that there is no societal impact of the work performed.
32

---

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [Yes]
Justification: Our research is based on open-source models and datasets. We cite all original papers that produced the code package or dataset. There is no risk of misuse in work.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model. for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: All third-party party assets used in the paper are properly cited with clear references, and their licenses and terms of use have been respected according to the original publications.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
33

---

For scraped data from a particular source (e.g., website). the copyright and terms of service of that souree should be provided.
If assets are released, the license, copyright information, and terms use in the datasets.paperssiii paperswi thcode. www.com//aaaa datasets package should be provided. For popular datasets,
has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
If this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification:
We provide documentation along with the submitted code.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/mmodl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
The paper does not involve crowdsoureing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with human subjects.
34

---

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if' applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Does the paper describe the usage of LLMs if is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification: Core method development in this research does not involve LLMs as any important, original, non-standard components.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to our LLM policy hatt::///earrrppccc/ooofeeeeeee/ /c/////oeeeeeeee//222////.. for what should or should not be described.
35


Reviews:
[{"review_id": "K6BEtqYDl8", "reviewer": "Reviewer_y451", "summary": "This paper looks into a system called SYMPHONY that tries to improve planning and reasoning by using not just one, but a mix of different language models. The idea is that these models, since they're trained differently and might “think” in slightly different ways, can complement each other when solving complex tasks.\n\nInstead of sticking to a single LLM, SYMPHONY brings together a pool of models and lets them work through problems collectively. There’s a mechanism that decides which model to use at each step based on how well it’s done in the past (a kind of UCB strategy). When a plan doesn’t work, the system writes out a short natural language “reflection” of what went wrong, and shares that with all the models. It also includes a scoring method that accounts for how confident the models seem about their outputs.\n\nThe authors tested this on three very different tasks: answering multi-hop questions (HotpotQA), making decisions in a simulated shopping scenario (WebShop), and writing code (MBPP). Across the board, SYMPHONY did better than well-known baselines like ToT and Reflexion—not just in terms of getting the right answers, but also using fewer tokens.\n\nAll in all, the paper makes a good case that combining several diverse models and letting them share feedback can improve both accuracy and efficiency, without needing extra training or tuning.", "questions": "Q1. In your experience, have there been any cases where bringing in more diverse models actually made things worse? I’m curious if there’s ever a point where the added heterogeneity introduces too much noise or conflict, instead of helping the system.\n\nQ2. Can you share any real examples of what the reflections look like? Even just a snippet or two would help me understand how these reflections affect what the agents do next, and how meaningful or consistent they tend to be.\n\nQ3. Roughly how often does the reflection mechanism come into play during rollouts? I’m trying to get a feel for whether it’s a frequent part of the process or something that only kicks in when an agent fails.\n\nQ4. What size of memory buffer did you use for storing reflections? And did you happen to try out different sizes to see if that had any noticeable effect on performance?\n\nQ5. Have you noticed any particular types of scenarios where SYMPHONY doesn’t work as well? For instance, in messier tasks where the signals are noisy or the goals aren’t well-defined, does it tend to struggle there?", "limitations": "Yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths\n\n- SYMPHONY is tested on three very different tasks—HotpotQA (for multi-hop QA), WebShop (decision-making), and MBPP (writing code in Python and Rust). Across all of them, it does better than well-known methods like ToT, Reflexion, LATS, MASTER, and AgentCoder. The authors also run the tests with different random seeds, which helps make the results more trustworthy.\n    \n- The paper is easy to follow. Each part of SYMPHONY is explained with math, diagrams, and simple descriptions. The figures and tables help a lot, so even if you’re not an expert, it’s not too hard to understand how the system works.\n    \n- The way they use UCB to pick which model to call during search makes sense—it balances trying new models with sticking to the ones that work. Also, instead of retraining anything, agents just share short natural language reflections. That’s a simple but smart way to help them learn from each other.\n    \n\nWeaknesses\n\n- The paper shows strong results, but it doesn’t talk much about the theory behind how SYMPHONY works—especially when using multiple agents and random rollouts. There’s no clear explanation of whether the system is stable or always gets better with more steps.\n    \n- Reflections are a big part of this method, but the paper doesn’t show any actual examples. It’s hard to know how useful they are without seeing what they look like or how consistent they are.\n    \n- There’s not much discussion about when SYMPHONY might fail. For example, what if agents give very different answers, or if one agent shares a reflection that misleads the others? These kinds of issues aren’t really addressed, which could be important in harder or noisy tasks.", "was_revised": true, "final_justification": "I sincerely thank the authors for their detailed explanation. Most of my concerns have been addressed, and I will keep my original score.", "rebuttal": "We sincerely thank the reviewer for these thoughtful and constructive comments. Below, we provide detailed responses to each of the raised concerns, covering theoretical soundness, reflection examples and usage, diversity trade-offs, and limitations in noisy tasks.\n\n### Weakness 1: On Theoretical Understanding: Why Should SYMPHONY Work?\n\nTo theoretically analyze the benefit of multi-agent coordination in sequential decision-making, we consider a setting in which multiple large language models (LLMs) are deployed as independent agents. At each decision step, one agent is sampled from a fixed distribution and is responsible for selecting an action. We assume that each agent may make mistakes on certain inputs, but that the agent pool collectively satisfies a correct coverage property: at every step, there exists at least one agent capable of choosing the correct action.\n\nFormally, Consider a sequential decision task with $T$ steps and a set of $m$ LLMs $\\\\{M_1, M_2, \\\\dots, M_m\\\\}$. At each step $t$, let $a_t^{\\*}$ be the correct action. Assume that for every step $t$, there exists at least one model $M_i$ such that $M_i(s_t) = a_t^{\\*}$. Then, for any probability distribution $\\\\{p_1, \\\\dots, p_m\\\\}$ with $p_i > 0$ and $\\\\sum_i p_i = 1$, the ensemble model that samples $M_i \\\\sim p_i$ at each step has strictly lower expected total error than any individual model in the ensemble:\n\n$$\n\\\\mathbb{E}[E_{\\\\mathrm{ens}}] < \\\\min_{j \\\\in [m]} E_j\n$$\nLet $e_{i,t} \\\\in \\\\{0,1\\\\}$ be the indicator of whether model $M_i$ makes an error at time $t$, i.e.,\n$$\ne_{i,t} = \n\\begin{cases}\n1 & \\text{if } M_i(s_t) \\ne a_t^* \\\\\\\\\n0 & \\text{otherwise}\n\\end{cases}\n$$\nThen the total error of model $M_j$ is\n$$\nE_j = \\sum_{t=1}^T e_{j,t}\n$$\nThe ensemble model samples $M_i \\sim p_i$ at each step, and its expected total error is\n$$\n\\mathbb{E}[E_{\\mathrm{ens}}] = \\sum_{t=1}^T \\sum_{i=1}^m p_i \\cdot e_{i,t}\n$$\nBy the assumption of correct coverage, for every $t$, there exists at least one $i$ such that $e_{i,t} = 0$, and since $p_i > 0$, we have:\n$$\n\\sum_{i=1}^m p_i \\cdot e_{i,t} < 1\n$$\nNow consider any model $M_j$. If there exists any $t$ such that $e_{j,t} = 1$ but $\\sum_{i=1}^m p_i \\cdot e_{i,t} < 1$, then:\n$$\n\\sum_{t=1}^T \\sum_{i=1}^m p_i \\cdot e_{i,t} < \\sum_{t=1}^T e_{j,t}\n\\Rightarrow \\mathbb{E}[E_{\\mathrm{ens}}] < E_j\n$$\nSince such $t$ must exist unless $M_j$ is correct at all steps (which contradicts the existence of error in practice), the inequality is strict.\n\nWe now proved that a randomized ensemble, formed by sampling agents independently at each step, achieves a strictly lower expected total error than any single fixed agent. This result provides a theoretical justification for leveraging model heterogeneity and stochastic selection to enhance robustness and accuracy in sequential reasoning tasks.\n\nWe appreciate your valuable suggestion and will address it in a subsequent version of the paper.\n\n### Weakness 2 & Question 2: Missing reflection examples\n\nReflection: The failure in the previous trial occurred because the action to directly click \"Buy Now\" was executed without first ensuring that the selected item met all the specified criteria, such as being a long clip-in hair extension, natural looking, and priced under \\$40.00. The initial action bypassed the necessary steps of verifying these details.\n\nReflection: I appeared to have retrieved the correct information about The Oberoi Family and the location of it's head office, and provided a corresponding answer. However this answer does not exactly match the ground truth answer so I should try a different wording, such as Delhi.\n\nWe appreciate your valuable suggestion and will address it in a subsequent version of the paper.\n\n### Weakness 3 & Question 1: When SYMPHONY might fail?  Is it possible that bringing in more diverse models actually made things worse?\n\nThank you for your insightful question. We fully acknowledge that while introducing diversity among models can enhance exploration, it also has the potential to introduce noise or inconsistency if not carefully controlled.\n\nThis issue is directly examined in our Appendix G, where we perform targeted experiments designed to amplify disagreement and behavioral variance among agents through adversarial prompting and temperature scaling. In particular, the adversarial prompting strategy forces each agent to avoid replicating actions taken by its siblings, thereby maximizing diversity in candidate expansions. This results in a high frequency of fully unique child nodes (4-Unique expansions) at the same search layer. However, this artificially induced extreme heterogeneity leads to a noticeable decline in task performance, confirming that excessive diversity can indeed be detrimental.\n\nBy contrast, our default SYMPHONY setup employs a more natural and balanced form of model heterogeneity by selecting large language models with differing architectures and pretraining data. These models produce outputs that are diverse yet semantically coherent without explicit adversarial constraints. Our empirical results in Section 4.5 demonstrate that the diversity brought by heterogeneity improves branch uniqueness and overall task accuracy, effectively balancing exploration with consistency.\n\nFurthermore, the multi-step Monte Carlo Tree Search process helps mitigate risks associated with individual model errors. Multiple candidate actions are generated at each expansion, and the Upper Confidence Bound (UCB)-based agent scheduler prevents any single model from dominating the search. Even when misleading or noisy actions arise, our Entropy-Modulated Confidence Scoring mechanism (Section 3.5) appropriately downweights low-confidence evaluations during value estimation.\n\nTogether, this combination of structured search, adaptive agent selection, and uncertainty-aware scoring ensures that SYMPHONY maintains robustness against local noise or conflicting agent reflections, even under challenging or noisy conditions.\n\nIn summary, while we do observe performance degradation under artificially maximized diversity, our standard heterogeneous agent configuration is deliberately designed to preserve coherence while leveraging complementary strengths. This careful balance allows SYMPHONY to improve performance reliably without suffering from the adverse effects of excessive model diversity.\n\n\n### Question 3 and 4: Reflection mechanism\n\nThank you for your insightful questions. The reflection mechanism is an important component of our system, enabling continual adaptation through natural language summaries without parameter updates, as detailed in Section 3.4.\n\nTo clarify its usage frequency, reflections are triggered only when a search trajectory ends unsuccessfully. At such termination points, a UCB-selected agent generates a reflection summarizing the failure. These reflections are stored in a shared memory buffer accessible to all agents in future rollouts, allowing them to learn from diverse reasoning perspectives.\n\nSpecifically:\n\n● The reflection mechanism is not invoked at every search step but selectively activated only upon failed trajectory termination. Due to the MCTS design, which backpropagates low values and visit counts to discourage revisiting poor branches, the chance of encountering the same failure repeatedly is low. Thus, reflections act as lightweight cautionary signals for unexplored or uncertain branches rather than frequent interventions throughout the rollout.\n\n● We maintain a fixed-size FIFO memory buffer that stores only the three most recent reflections. This small buffer aligns with MCTS behavior: failures are backpropagated to reduce revisit likelihood, making repeated errors unlikely. The reflections serve primarily as brief warnings for yet-unseen failure cases.\n\n● Each reflection is a short natural language snippet held temporarily in runtime memory during a single task episode, keeping the memory footprint minimal. After task completion, reflections are discarded, ensuring resource efficiency and enabling efficient multi-agent sharing without redundant storage.\n\n● Our relatively small rollout budget further limits failure trajectory occurrences, reducing the need for a larger reflection memory.\n\nWhile we have not extensively explored varying buffer sizes, prior work such as Reflexion (Shinn et al., NeurIPS 2023)  indicates that increasing reflections can boost performance but tends to plateau, with diminishing returns beyond a certain size. We appreciate this suggestion and consider it a valuable direction for future research.\n\n### Question 5: Particular types of scenarios where SYMPHONY doesn’t work as well? For instance, in messier tasks where the signals are noisy or the goals aren’t well-defined, does it tend to struggle there?\n\nThank you for this insightful question. You are absolutely right—SYMPHONY does face challenges in scenarios where task signals are noisy, goals are ambiguous, or the problem structure is particularly complex. These conditions make perfect performance difficult. \n\nFor example, many tasks in the WebShop dataset involve such complexities. On this dataset, strong baselines like LATS achieve a success rate around 0.38, while SYMPHONY improves this to 0.72. Although we have made significant progress in addressing these challenges, the problem is not fully solved, and improving robustness in such noisy or ill-defined scenarios remains an important direction for future research.\n\nWe sincerely appreciate your careful review. We hope our clarifications have resolved your concerns, and we would be happy to engage in further discussion if needed.\n\n### References:\nShinn, N., Cassano, F., Gopinath, A., Narasimhan, K., & Yao, S. (2023). Reflexion: Language agents with verbal reinforcement learning. Advances in Neural Information Processing Systems, 36, 8634–8652."}, {"review_id": "19BZOcIqJb", "reviewer": "Reviewer_AH1d", "summary": "Existing single‐agent planning approaches lack of structural diversity constrains exploration, leading to redundant search trajectories and degraded planning quality. To address this, the paper introduce SYMPHONY (SYnergistic Multi‐agent Planning with HeterOgeneous laNguage model assemblY), a multi‐agent MCTS framework that orchestrates a diverse pool of language‐model agents. By assigning different models to propose and evaluate actions, SYMPHONY substantially increases rollout diversity and enables more robust exploration. Across three challenging benchmarks, SYMPHONY delivers outperforming existing state-of-the-art baselines.", "questions": "Do all agents serve the same function? If so, in what way are they “heterogeneous” rather than merely independent copies of the same model?\n\nHow are next states determined? Is each agent queried on a separate environment copy in parallel?\n\nWhat exactly is stored in “pool-wise memory”? Natural-language reflections, embeddings, or something else? How do agents access and use this memory during search? How does memory differ from the set of past trajectories?\n\nMissing Related Work: For example, “Monte Carlo Planning with Large Language Model for Text-Based Game Agents” (ICLR 2025)", "limitations": "See weaknesses.", "rating": 3, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths\n\nThe research problem is compelling: leveraging multiple agents to enrich planning diversity.\n\nThe paper evaluates SYMPHONY with several off-the-shelf LLM APIs for fair comparison.\n\nExperiments use benchmarks such as HotpotQA, WebShop, and MBPP cover reasoning, sequential decision-making, and code generation.\n\nWeaknesses \n\nIt’s unclear what is fundamentally new: beyond assembling multiple models, which components are original contributions?\n\nWhy is a straightforward UCB bandit strategy (balancing per-agent invocation counts and rewards) sufficient? What insights guided this choice?\n\nHow does SYMPHONY’s runtime and memory usage compare to single-agent MCTS or existing multi-agent methods?", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your careful reading and thoughtful questions. We appreciate the opportunity to clarify several aspects of SYMPHONY and highlight its conceptual and empirical contributions.\n\n### Weakness 1: what is fundamentally new?\n\nThank you for your question. The fundamental novelty of SYMPHONY lies in its heterogeneous expert pool, which is a carefully curated collection of structurally and pretraining-diverse open-source LLMs such as Qwen, Mistral, and Llama. Unlike ensembles of homogeneous or similarly configured models, this diverse pool leverages architectural differences to produce complementary reasoning behaviors. Importantly, by coordinating multiple smaller open-source models deployable on consumer-grade hardware, SYMPHONY achieves performance that approaches or even surpasses that of the much larger, closed-source GPT-4.\n\nThe other core mechanisms, namely the UCB-based expert scheduling, the shared runtime reflection memory, and the entropy-modulated node evaluation, are all designed specifically to support and enhance this heterogeneous model pool. The UCB scheduler adaptively routes search efforts among agents to balance exploration and exploitation efficiently. The reflection memory stores natural language summaries of failure cases to share cautionary insights across agents, guiding reasoning away from known pitfalls in real time. Meanwhile, the entropy-modulated evaluation selectively preserves useful candidate expansions while filtering out misleading ones without expensive rollouts.\n\nTogether, these components form a tightly integrated decision-making framework built around the heterogeneous expert pool, enabling SYMPHONY to deliver robust, high-quality performance beyond what standard homogeneous ensembles or majority voting can achieve.\n\n### Weakness 2: Why is a straightforward UCB bandit strategy (balancing per-agent invocation counts and rewards) sufficient? What insights guided this choice?\n\nThank you for the question. We chose a UCB1-style bandit mechanism for agent selection based on several key insights.\n\nFirst, agent selection in SYMPHONY naturally maps to a multi-armed bandit problem, where balancing exploration and exploitation over time is crucial. UCB offers a principled, low-overhead solution with strong theoretical regret guarantees and asymptotic fairness among agents.\n\nSecond, compared to learned routing methods that require additional training and complexity, UCB is stateless, parameter-free, and highly interpretable. This simplicity suits the recursive, high-frequency decision process in MCTS and does not interfere with convergence properties.\n\nThird, our empirical results (Table 5) demonstrate that removing UCB-based scheduling significantly degrades performance, confirming that this seemingly simple strategy is essential for effective coordination among heterogeneous agents.\n\nFinally, the reward signals，derived with EMCS, are often sparse and non-stationary due to partial observability. UCB naturally handles such variability, maintaining robustness in uncertain environments.\n\n\n### Weakness 3: SYMPHONY’s runtime and memory usage are compared to single-agent MCTS and existing multi-agent methods\n\nThank you for your question. We provide a detailed analysis of SYMPHONY’s runtime and memory usage in Section 4.6 and Appendix Table 6 to ensure a thorough and fair comparison.\n\nFirst, all methods were tuned with comparable tree parameters to enable equitable runtime evaluation. As shown in Table 4, SYMPHONY achieves higher task success rates while generating fewer tokens and expanding fewer search nodes than strong single-agent baselines such as ToT, RAP, and LATS.\n\nSecond, we compare SYMPHONY against MASTER, a recent state-of-the-art multi-agent method, using their reported metrics since their code is unavailable. Appendix Table 6 shows SYMPHONY uses fewer nodes and fewer tokens, confirming superior runtime and memory efficiency.\n\nBecause all methods are runtime frameworks based on tree-structured search, the cost can be measured transparently by the total number of expanded nodes and total model-generated tokens. From a runtime perspective, SYMPHONY uses smaller tree hyperparameters and terminates earlier under identical task budgets, leading to faster and more efficient search. From a memory perspective, the explicit count of visited nodes serves as a fair proxy.\n\nIt is true that hosting SYMPHONY requires maintaining multiple models rather than a single one. However, by leveraging multiple open-source models deployable on consumer-grade hardware, SYMPHONY outperforms GPT-4-level performance without requiring prohibitively expensive infrastructure. Meanwhile, its more compact MCTS search tree further reduces computational overhead compared to single-agent approaches.\n\nIn summary, SYMPHONY balances hosting cost with runtime and memory efficiency to deliver both more effective and more efficient task solving compared to single-agent or existing multi-agent methods.\n\n### Question 1: Heterogeneity in agent pool\n\nThank you for your question. While all agents share the same interface and perform the same high-level function within SYMPHONY, their fundamental differences lie in their reasoning styles, stemming from their architecture and pretraining methodology. We select models from different organizations, such as Qwen, Mistral, and Llama, that vary in network design, training data, tokenizer vocabularies, and alignment objectives.\n\nThese intrinsic differences lead each agent to follow distinct reasoning paths and exhibit unique inductive biases. It is this heterogeneity in reasoning patterns, not merely having multiple copies of the same model, that introduces the meaningful diversity we rely on. Our framework leverages this reasoning-mode heterogeneity to explore complementary solution subspaces, improving robustness and overall task performance.\n\n### Question 2: How are next states determined? Is each agent queried on a separate environment copy in parallel?\n\nThank you for your question. SYMPHONY maintains a single environment instance, with explicit environment states saved at each node of the MCTS tree to support branching and backtracking.\n\nAt each expansion step, a subset of agents is selected via the UCB scheduler. Each agent independently proposes a candidate action, generating multiple hypothetical child nodes. These candidates are then evaluated collectively using Entropy-Modulated Confidence Scoring (EMCS) combined with UCT, and only one child node is selected for further exploration.\n\nSince SYMPHONY operates in text-based environments, environment transitions are simulated by saving and restoring interaction histories (i.e., textual state). Each MCTS node stores its corresponding environment state snapshot, enabling efficient re-entry or rollback to any prior state without needing multiple environment copies running in parallel.\n\nThis approach ensures accurate and memory-efficient environment simulation during search, while supporting sequential agent queries rather than parallel environment instances.\n\n### Question 3: Memory Mechanism\n\nThank you for your question. The pool-wise memory stores natural-language reflections, which are concise verbal summaries generated exclusively after failed MCTS rollouts. These reflections provide diagnostic feedback to guide and improve future decision-making.\n\nEach reflection is a high-level natural language description of a failure case. For example, in the WebShop task, a reflection might be: “Filtered by color before narrowing down by category — may miss relevant items.”\n\nThe memory is implemented as a fixed-size FIFO buffer holding the three most recent reflections per MCTS episode (i.e., per task-specific search tree). This design ensures runtime efficiency and prevents the accumulation of stale or irrelevant reflections.\n\nWhen an agent is selected to propose candidate actions at a node, the current pool-wise memory—the latest reflections—is injected into the agent’s prompt. This provides contextual warnings or reminders that help the agent avoid repeating past mistakes.\n\nImportantly, this memory differs fundamentally from a set of past trajectories or embeddings. It does not store raw trajectory data nor embeddings, nor does it involve parameter updates. Instead, it acts as a lightweight, natural-language augmentation that enables on-the-fly adaptation during test time without additional training.\n\nThis mechanism allows SYMPHONY to dynamically capture failure patterns and share them across heterogeneous agents during search, enhancing robustness while maintaining low computational overhead.\n\n### Question 4: Missing Related Work\n\nThank you for bringing this relevant work to our attention. The cited paper addresses single-agent memory-augmented MCTS for text-based games, sharing the goal of enhancing decision-making through memory mechanisms. However, there are key differences between their approach and SYMPHONY.\n\nSpecifically, SYMPHONY proposes a multi-agent decision routing framework coordinating heterogeneous LLMs, while the cited work focuses on a single-agent setting. Moreover, our memory mechanism employs natural-language reflections dynamically injected at runtime to guide reasoning, in contrast to their stored trajectory-level memory used for retrospective lookback.\n\nWe note that this paper was posted on arXiv on April 23, 2025, at 16:23 UTC, just days before the NeurIPS 2025 submission deadline. According to NeurIPS guidelines, works appearing on public servers within March 1st, 2025 prior to submission are considered concurrent. Due to the timing and our extensive experimental commitments, we were unable to reference it in the original submission. We appreciate this helpful pointer and will include a thorough discussion and citation in the final revision.\n\nWe sincerely appreciate your careful review. We hope our clarifications have resolved your concerns, and we would be happy to engage in further discussion if needed."}, {"review_id": "s90XIcS6xT", "reviewer": "Reviewer_aqqm", "summary": "This paper proposes SYMPHONY, a multi agent framework with different models that utilize a variation of Monte Carlo Tree Searching planning.  The authors adopt heterogeneous models to prevent narrow rollouts. The authors evaluate SYMPHONY on HotPotQA, WebShop and MBPP, showing promising results on all three benchmarks.", "questions": "Q1: Line 1236 and afterwards, \"Refelcion\" -> \"Reflection\"?\n\nQ2: How big is the Reflection buffer for each dataset and each experiment of SYMPHONY? Are all agents sharing the same reflections?\n\nQ3: Is there any better way in agent selection instead of averaging?", "limitations": "Yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strength:\n1. Using multiple models to diversify rollouts and decision trajectories is simple yet effective.\n\n2. The manipulation on $Q$ term of UCT is clever, mainly based on Entropy-Modulated node evaluation, which is simple yet effective, making the scoring dynamic based on each agent as a scorer for itself. Such scores are reflected dynamically on difficulty of various tasks. The average of multiple agents to reflect on $Q$ term is straightforward.\n\n3.The section of efficiency and cost analysis is essential to assess such multi agent framework, and SYMPHONY shows great efficiency in terms of token consumption.\n\n4. Experiments set up are detailed and fairly complete.\n\nWeakness:\n1. It would be great if more explanation is provided about why choosing the Bernoulli Distribution under Entropy-Modulated Confidence Scoring (EMCS).", "was_revised": true, "final_justification": "During rebuttal, the authors explained why choosing the Bernoulli Distribution under Entropy-Modulated Confidence Scoring (EMCS). I value this explanation more than others.\n\nThe authors explained the settings of reflections, and pointed out future directions in agent selection.\n\nThus, I maintain my rating (4).", "rebuttal": "### Weakness 1: It would be great if more explanation is provided about why choosing the Bernoulli Distribution under Entropy-Modulated Confidence Scoring (EMCS).\n\nThank you for highlighting this important aspect. Our choice to use the Bernoulli distribution in Entropy-Modulated Confidence Scoring (EMCS) is driven by the need for a principled and efficient way to calibrate agent confidence across heterogeneous models.\n\nIn SYMPHONY, each agent assigns a scalar confidence score $c \\in (0, 1)$ to a candidate node’s value estimate. Since agents vary in architecture, training data, and scale, their raw confidence scores are not directly comparable or equally reliable.\n\nTo address this, we interpret ccc as the success probability of a Bernoulli distribution and compute its Shannon entropy:\n$$\nH(c) = -c \\ln c - (1 - c) \\ln(1 - c)\n$$\n\nThis entropy quantifies the epistemic uncertainty of the agent’s estimate:\n\n● It peaks at $c = 0.5$, reflecting maximum uncertainty.\n\n● It approaches zero as $c \\to 0$  or $c \\to 1$, indicating high confidence.\n\nWe then modulate the original confidence score by this entropy, effectively downweighting uncertain (high-entropy) scores and emphasizing confident (low-entropy) ones. This modulation requires no additional hyperparameters and integrates seamlessly into node evaluation.\n\nThe Bernoulli entropy is a natural choice because:\n\n● It is a well-established, theoretically grounded measure of uncertainty in probabilistic decision-making.\n\n● Its symmetry and strict convexity ensure consistent and smooth handling of both overconfident and underconfident scores.\n\n● It is computationally efficient, avoiding complex resampling or ensemble methods, which is crucial for real-time MCTS.\n\nEmpirically, as demonstrated in Table 5, EMCS improves node evaluation reliability amid noisy or divergent outputs from heterogeneous agents. It enables SYMPHONY to balance trust in decisive agents with caution toward uncertain ones, enhancing overall multi-agent search performance.\n\n### Question 1: Line 1236 and afterwards, \"Refelcion\" -> \"Reflection\"?\n\nThank you for catching this. The term in that section should indeed be “Evaluation” rather than “Reflection.” It refers to the process where an agent evaluates and assigns a score to a node, often providing a brief explanation or rationale.\n\nSince we do not employ a separate reasoning model for scoring, the agent performs a lightweight verbal evaluation using natural language. This approach enhances the stability and interpretability of the score by grounding it in reasoning, rather than relying on isolated scalar values. We will correct this terminology in the final version.\n\n### Question 2: How big is the Reflection buffer for each dataset and each experiment of SYMPHONY? Are all agents sharing the same reflections?\n\nThank you for this excellent question. We are happy to clarify the details regarding the reflection buffer in SYMPHONY.\n\nThe reflection buffer is scoped at the task level, meaning it is associated with each individual MCTS search tree corresponding to a single task instance—for example, one shopping session in WebShop or a single question in HotpotQA. It is not shared or accumulated across the entire dataset.\n\nDuring the search process for each task, we maintain a fixed-size FIFO buffer containing the three most recent reflections. This size balances memory efficiency with sufficient representational capacity to capture meaningful failure insights. Once the search for that task concludes, whether successfully or not, the buffer is cleared because its role is to influence decision-making only within the context of that specific task’s trajectory.\n\nAll agents share this reflection buffer. This design enables heterogeneous agents to benefit from collective verbal memory of prior failures, supporting our hypothesis that diverse models provide complementary perspectives and improve reasoning when informed by shared reflective feedback.\n\nAdditionally, the inherent MCTS backpropagation mechanism reduces the likelihood of revisiting suboptimal trajectories, so only a modest reflection buffer is necessary. Reflections act as verbal reinforcement that complements rather than replaces MCTS’s intrinsic credit assignment.\n\nWe also note that related work, such as Reflexion (Shinn et al., NeurIPS 2023) , demonstrates that increasing reflection frequency and buffer size can enhance performance up to a saturation point. Investigating the impact of buffer size and retention strategies on planning quality is an important avenue for future research.\n\nIn summary, the reflection buffer is task-specific, modest in size, shared across agents, and plays a supportive role in improving multi-agent collaboration within each discrete task.\n\n\n### Question 3: Is there any better way in agent selection instead of averaging?\n\nThank you for this insightful and forward-looking suggestion.\n\nCurrently, SYMPHONY employs a vanilla UCB strategy that uses mean reward estimates to balance exploration and exploitation among agents. This choice is driven by several factors:\n\n● Simplicity and robustness: UCB is a well-established, parameter-free, and interpretable method that integrates naturally with the recursive structure of MCTS.\n\n● Strong empirical support: Our ablation study (Table 5) shows that removing or replacing UCB, such as using round-robin or uniform sampling, leads to a notable drop in accuracy.\n\nThat said, your suggestion opens promising avenues for future research. Incorporating weighting schemes based on recency or contextual similarity could enable more adaptive agent routing, particularly in environments with concept drift or non-stationarity.\n\nWe are keen to investigate more sophisticated selection methods, including softmax-weighted UCB, Bayesian UCB, or contextual bandits, to further enhance flexibility and adaptivity in multi-agent coordination.\n\nThank you again for your valuable input, which aligns well with SYMPHONY’s vision of dynamic and effective multi-agent collaboration.\n\nWe sincerely appreciate your careful review. We hope our clarifications have resolved your concerns, and we would be happy to engage in further discussion if needed.\n\n### Reference:\nShinn, N., Cassano, F., Gopinath, A., Narasimhan, K., & Yao, S. (2023). Reflexion: Language agents with verbal reinforcement learning. Advances in Neural Information Processing Systems, 36, 8634–8652."}, {"review_id": "troEoXTYGU", "reviewer": "Reviewer_QwDw", "summary": "SYMPHONY introduces a synergistic multi-agent planning framework that integrates heterogeneous language models to enhance Monte Carlo Tree Search (MCTS) for complex planning tasks. The framework addresses single-agent limitations by employing multiple diverse language models that collaborate across all four MCTS phases: selection, expansion, simulation, and backpropagation. It offers flexible deployment configurations (SYMPHONY-S for consumer hardware, SYMPHONY-L for cloud APIs) with a modular architecture including agent manager, coordination engine, MCTS planner, and evaluation modules. Experimental results demonstrate significant performance improvements over traditional single-agent methods, establishing a new paradigm for LLM-based autonomous planning through collective intelligence.", "questions": "See  Weaknesses.", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths\n\n1. Novel Problem Formulation：\nThe paper addresses a novel and significant problem by identifying the limitations of single-agent MCTS approaches in complex planning tasks. The introduction of heterogeneous multi-agent collaboration represents a fresh perspective that hasn't been systematically explored in LLM-based planning before.\n\n2. Clear and Comprehensible Presentation：\nThe paper is well-written with clear explanations of complex concepts, making the methodology and technical details accessible to readers. The logical flow from problem identification to solution design and experimental validation enhances overall readability and understanding.\n\n3. Strong Alignment Between Motivation and Experimental Analysis：\nThe experimental results strongly support the paper's core motivation, particularly demonstrated in Figure 2 where the diversity differences between methods are significant and clearly visible. This empirical evidence directly validates the theoretical premise that multi-agent approaches provide superior exploration diversity.\n\n4. Efficient Method with Practical Deployment Considerations：\nThe proposed method demonstrates superior efficiency compared to baseline approaches while offering practical value through dual configurations. SYMPHONY-S (consumer-grade hardware) and SYMPHONY-L (cloud API) provide flexible deployment options that align with real-world application scenarios and resource constraints.\n\n\nWeaknesses\n\n1. Limited Real-World Agent Scenario Alignment：\nThe experimental datasets have a significant gap compared to real agent scenarios, with environments like WebArena being more aligned with practical use cases. However, given that the authors follow the strongly related baseline work MASTER (which also adopts the same datasets), this is more of a personal concern rather than a major flaw in the experimental design.\n\n2. Insufficient Analysis of Node Expansion Quality：\nThe experiments lack deeper analysis regarding the quality of diverse node expansions - specifically, whether beneficial versus detrimental expansions are strongly correlated with model capabilities. This becomes particularly important when there are significant capability gaps between different models in the heterogeneous ensemble.\n\n3. Missing Single Strong Model Baseline：\nAn additional baseline should be supplemented since components like Pool-wise Memory Sharing and Entropy-Modulated Node Evaluation are actually applicable to single agents. The authors should include experimental results of a single powerful model within their proposed framework for fair comparison.\n\n4. Unclear Effectiveness with Reasoning Models：\nIt remains uncertain whether the experimental conclusions hold for reasoning models, as these models already perform MCTS-like operations during long chain-of-thought processes, including reflection and backtracking. This raises questions about whether such capabilities might diminish the value of the proposed multi-agent approach.", "was_revised": true, "final_justification": "The authors have addressed my concerns in their rebuttal, and I will maintain my positive score.", "rebuttal": "Thank you for your thoughtful and constructive feedback. We respond point-by-point to each of your concerns and questions below, and we deeply appreciate the opportunity to clarify and expand on the key aspects of our work.\n\n\n### Weakness 1: Limited Real-World Agent Scenario Alignment\nThank you for raising this valuable concern. We fully agree that aligning evaluation tasks with real-world agent scenarios is crucial for practical impact, and we genuinely appreciate your suggestion of incorporating environments like WebArena. This is an excellent direction, and we’re glad you brought it to our attention.\n\nWe recognize that WebArena, with its open-ended and dynamic web environment, offers a more realistic approximation of real-world agent usage. Its task structure involving multimodal interactions, tool usage, and exploration provides a compelling platform for testing generalist agents in complex scenarios.\n\nThat said, our current experimental suite consists of HotpotQA, WebShop, MBPP-Python, and MBPP-Rust, and was carefully selected to cover diverse and challenging reasoning demands across multiple domains.\n\n● HotPotQA: Multi-hop question answering requiring fact chaining and intermediate reasoning.\n\n● WebShop: Simulates goal-driven dialogue and sparse reward navigation over structured catalog environments.\n\n● MBPP (Python & Rust): Program synthesis under strict correctness constraints, reflecting structured output reasoning and functional validation.\n\nWhile these tasks may lack the visual or interactive complexity of WebArena, they nonetheless capture essential characteristics of real-world agent behavior, such as long-horizon planning, reasoning under uncertainty, and memory across steps.\n\nWe wholeheartedly agree that extending SYMPHONY to more grounded and interactive environments is an important next step. Based on your suggestion, we plan to include WebArena or similar environments in future iterations of our framework, and we believe our method’s modularity and memory design make it a natural fit.\n\nWe sincerely appreciate this insightful feedback. It encourages us to further extend the scope of SYMPHONY, and we look forward to pursuing this direction in future work.\n\n### Weakness 2: Insufficient Analysis of Node Expansion Quality\n\nThank you for raising this important point regarding the quality of node expansions in relation to model capabilities within a heterogeneous ensemble.\nWe address this concern from two perspectives:\n\nFirst, although explicitly labeling expansions as “beneficial” or “detrimental” is challenging without task-specific heuristics, we use overall task success rate as a practical proxy. Beneficial expansions are those leading toward successful task completion. Our results show a clear correlation between model capability and expansion quality—larger, more capable models (e.g., GPT-4) tend to produce more effective expansions, reflected in higher success rates.\n\nSecond, SYMPHONY’s design inherently mitigates risks from capability imbalances. The system prevents weaker models from dominating by combining:\n\n● UCT-driven branch selection that favors high-value paths,\n\n● UCB-based agent routing that downweights agents with historically poorer performance,\n\n● Tree-structured search with backtracking that limits repeated exploitation of suboptimal nodes.\n\nThis architecture ensures robustness against noisy or poor expansions. While “one bad apple” may exist, the tree search allows strong agents to reinforce better branches and dilute negative influence.\n\nTo further support this, we conducted an additional experiment on WebShop combining GPT-4 with two weaker models (Llama-3.1-8B-Instruct and Mistral-7B-Instruct-v0.3). Despite the weaker collaborators, the ensemble outperformed standalone GPT-4, confirming SYMPHONY’s robustness to heterogeneity and the effectiveness of our coordination mechanisms.\n\n> | WebShop               | Score | SR   |\n> | :-------------------- | :---- | :--- |\n> | SYMPHONY(GPT-4) | 0.80  | 0.60 |\n> | SYMPHONY(GPT-4+Mistral+Llama)   | 0.83  | 0.62 |\n\n\n### Weakness 3: Missing Single-Strong-Model Baseline\n\nThank you for this valuable suggestion. We agree that components like Pool-wise Memory Sharing and Entropy-Modulated Node Evaluation can be applied in single-agent settings, and we have accordingly added a strong single-agent baseline for fair comparison.\n\nSpecifically, we incorporate GPT-4, a widely adopted and powerful general-purpose model, as the sole agent within the full SYMPHONY framework across all three datasets. This ensures a meaningful and equitable baseline since GPT-4 is also used by many recent methods.\n\n● The experiments run the identical framework as the multi-agent setup, but with GPT-4 as the only agent responsible for node expansion and evaluation.\n\n● Even as a single model, our framework significantly improves over raw GPT-4 and prior single-agent baselines, demonstrating the value of our pool-wise memory and entropy-modulated evaluation mechanisms.\n\n● Nonetheless, as discussed in our response to Weakness 2, the heterogeneous multi-agent SYMPHONY configuration still surpasses this enhanced single-agent baseline, confirming that model diversity yields complementary strengths unattainable by any individual model alone.\n\n> |                       | HotpotQA(EM) | WebShop(Score) | WebShop(SR) | MBPP-Python(pass\\@1) | MBPP-Rust(pass\\@1) |\n> | :-------------------- | :----------- | :------------- | :---------- | :------------------- | :----------------- |\n> | LATS | 0.71         | 0.76           | 0.38        | 0.811                | -             |\n> |MASTER | 0.76         | 0.80           |-      | 0.910                | -             |\n> | SYMPHONY(GPT-4) | 0.76         | 0.80           | 0.60        | 0.912                | 0.924              |\n> | SYMPHONY-L | 0.79         | 0.88           | 0.72        | 0.965               | 0.974              |\n\nThese findings underscore the modularity and effectiveness of our approach in both single-agent and multi-agent contexts. We will include these new results in the revision to address your suggestion.\n\n\n### Weakness 4: Unclear Effectiveness with Reasoning Models\n\nThank you for this thoughtful and insightful question. We appreciate your concern regarding how SYMPHONY interacts with modern reasoning-capable models that internally perform operations like chain-of-thought (CoT), reflection, and backtracking.\nOur detailed response is as follows:\n\n● Focus on Cost-Efficient Test-Time Scaling: The primary motivation behind SYMPHONY is to achieve competitive performance comparable to large, cloud-based models—but at significantly lower cost—by coordinating a diverse ensemble of lightweight agents at test time without heavy pretraining or fine-tuning. This approach emphasizes scalability and practicality, showing that structured multi-agent coordination and reflection mechanisms enable smaller models to collectively reach strong  performance.\n\n● Experimental Fairness and Controlled Comparison: While reasoning-tuned models naturally exhibit stronger task performance, most baselines we compare against (e.g., RAP, LATS, MASTER) do not incorporate such specialized models. To maintain fair and controlled comparisons, we intentionally excluded these powerful reasoning models from our main experiments, preventing inflated baselines that could obscure SYMPHONY’s architectural contributions.\n\n● Verifying Compatibility with Reasoning-Capable Models: To address your concern, we conducted an additional experiment integrating Claude-3.5-Sonnet-20240620, a reasoning-enhanced agent, as a single agent within our framework\n\n> |                       | HotpotQA(EM) | WebShop(Score) | WebShop(SR) | MBPP-Python(pass\\@1) | MBPP-Rust(pass\\@1) |\n> | :-------------------- | :----------- | :------------- | :---------- | :------------------- | :----------------- |\n> | Claude-3.5-Sonnet | 0.51         | 0.71           | 0.41        | 0.894                |0.903              |\n> | SYMPHONY(Claude-3.5-Sonnet) | 0.76         | 0.82           | 0.61        | 0.947                | 0.951              |\n> | SYMPHONY(GPT-4) | 0.76         | 0.80           | 0.60        | 0.912                | 0.924              |\n> | SYMPHONY-S | 0.59         | 0.82           | 0.56        | 0.927                | 0.946              |\n\n\nSYMPHONY (Claude) outperformed both the standalone Claude model and SYMPHONY (GPT-4) under identical settings, confirming that our framework is fully compatible with state-of-the-art reasoning models and does not interfere with their internal inference capabilities. Moreover, SYMPHONY-S, built on lightweight agents, also surpassed Claude-3.5-Sonnet, aligning with our core motivation of achieving strong task performance through efficient test-time coordination of smaller models.\n\nMore importantly, this demonstrates that SYMPHONY’s benefits extend beyond ensembles of weaker agents, offering potential for hybrid setups that coordinate multiple specialized reasoning agents to further improve decision quality.\n\nWe sincerely thank the reviewer for this valuable suggestion, which helped us showcase the generality and extensibility of our approach beyond the original scope.\n\nWe sincerely appreciate your careful review. We hope our clarifications have resolved your concerns, and we would be happy to engage in further discussion if needed."}]

Author rebuttal:

Meta-review: SYMPHONY is a multi-agent framework for LLM-based planning that integrates different models within MCTS. Authors argue that while single-agent search often collapses into narrow, repetitive rollouts, coordinating multiple models with distinct inductive biases can expand the search space, improve exploration, and yield more robust plans. 

There are three components—UCB-based agent scheduling, shared reflection memory, and entropy-modulated node evaluation. Approach is tested on benchmarks spanning reasoning (HotpotQA), decision-making (WebShop), and code generation (MBPP). Results show improvements over baselines, ToT, Reflexion, LATS, and MASTER, with both lightweight (open-source only) and large (API-based) configurations.

Reviewers agree that the problem is timely and well motivated, the paper is clearly presented, and the results are strong across tasks. They found the heterogeneity compelling: rather than ensembling similar models, the authors demonstrate how architectural and pretraining diversity introduces meaningful exploration patterns. 

Reviewers also appreciated the ablations, cost–efficiency analysis, and the new experiments in rebuttal, including single-strong-model baselines, reasoning-model compatibility, and case studies illustrating how weaker models help escape local failure modes. The authors also provided a theoretical argument showing why randomized heterogeneous ensembles should lower expected trajectory error, further grounding the approach.

The main weaknesses were about novelty and framing: some felt the individual components (UCB, reflections, entropy scoring) are straightforward, and the contribution lies more in their integration than in algorithmic innovation. Concerns were also raised about missing discussion of failure cases, stability, and alignment with real-world agent settings beyond curated benchmarks. Still, the rebuttal addressed these points convincingly, and the combination of conceptual clarity, practical efficiency, and empirical strength makes this a solid contribution. I recommend to accept as poster, not a spotlight, since the methodological novelty is moderate and rests more on system design than on a fundamental algorithmic advance.
Decision: Accept (poster)