Can Dependencies Induced by LLM-Agent Workflows Be Trusted?
Authors: Yu Yao
Yiliao Song
Yian Xie
Mengdan Fan
Mingyu Guo
Tongliang Liu
Venue: NeurIPS 2025 poster ()
Keywords: Multi-LLM-Agent System
Primary area: applications
TL;DR: This paper presents a dynamic framework that enables reliable execution under violated conditional independence assumptions.

Abstract: LLM-agent systems often decompose high-level objectives into subtask dependency graphs, assuming that each subtask’s output is reliable and conditionally independent of others given its parent responses. 
However, this assumption frequently breaks during execution, as ground-truth responses are inaccessible, leading to inter-agent misalignment—failures caused by inconsistencies and coordination breakdowns among agents. 
To address this, we propose SeqCV, a dynamic framework for reliable execution under violated conditional independence. 
SeqCV executes subtasks sequentially, each conditioned on all prior verified responses, and performs consistency checks immediately after agents generate short token sequences. 
At each checkpoint, a token sequence is accepted only if it represents shared knowledge consistently supported across diverse LLM models; otherwise, it is discarded, triggering recursive subtask decomposition for finer-grained reasoning. 
Despite its sequential nature, SeqCV avoids repeated corrections on the same misalignment and achieves higher effective throughput than parallel pipelines. 
Across multiple reasoning and coordination tasks, SeqCV improves accuracy by up to 30\% over existing LLM-agent systems.
Code is available at https://github.com/tmllab/2025_NeurIPS_SeqCV.

Can Dependencies Induced by LLM-Agent Workflows Be Trusted?
Yu Y Yiliao Song*, Yian Xie Mengdan Fan Mingyu Guo Tongliang Liu Liu*
Sydney AI Centre, The University of Sydney Adelaide University Monash University Peking University tongliang. liudsydney. edu.au
Abstract
LLM-agent systems often decompose high-level objectives into subtask depen- dency graphs, assuming that each subtask's 's output is reliable and conditionally independent of others given its parent responses. However, this assumption fre- quently breaks during execution, as ground-truth responses are inaccessible, leading inter-agent mixalignment--Faan failurs caused by inconsistencies and coordination breakdowns among agents [ To address this, propose SEQCV, dynamic framework for reliable execution under violated conditional independence. SEQCV executes subtasks sequentially, each conditioned on all prior verified responses, and performs consistency checks immediately after agents generate short token sequences. At each checkpoint, token sequence is accepted only if it represents shared knowledge consistently supported across diverse LLM models; otherwise, it is discarded, triggering recursive subtask decomposition for finer-graing rea- soning. Despite its sequential nature, SEQCV avoids repeated corrections on the misalignment and achieves higher effective throughput than parallel pipelines. Across multiple reasoning and coordination tasks, SEQCV improves accuracy by up 30% over existing LLM-agent systems. Code is available github. wttttuu...oommmlla///////// Nawpallaaa222__eeuueeeee NewrIII_SeeeS""
Introduction
ILMMageen systems are designed to conceptualize the powerful large language models (LLMs) to
perform a broad range of tasks 3 4. On one hand, existing research focuses on improving the
performance of LLM-agent systems across various applications. For example, some systems
tailored for specific domains such as software development [5] 6 7l scientific discovery [8] 90
and human behaviour simulation [ 1 Others focus augmenting LLM agents with external
tools [14 15 16), or on establishing benchmarks to evaluate LLM-agent performance [7
18
On the other hand, recent studies have raised two critical questions: 1) what constitutes an LLM-agent system 21 22 and which attributes most influence its performance 233 Although there no
clear agreement on these two questions (24] 25 one certainty has emerged: multi-I.I.M Mattiii----aagenn systems demonstrated superior performance than single LLM-ageet systems, highlighting the importance of orchestrating collaboration among LLM-agents optimize task execution [26] 211..
To enhance the reasoning capabilities of LLMs for complex tasks, a high-level task objective is often decomposed into a series of intermediate reasoning steps, such as chain of thought (CoT) (27). ReAct
28 , tree of thought (ToT) 22). Reflexion 30). In M-genttt systems, these steps correspond to subtasks, as exemplified by AutoGPT (31), BabyAGI [32). LangChain (33), and Llama-inde [34]. Recent studies suggest that decomposing tasks as graphs can further enhance performance, both
Equal Contributions. X Corresponding Author.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

[Detline the RL siides structure] [Uroteskdsskr
C Lecring (Tiaaa Furn)
Dey O fatwol (DQ4)
Content
Dep S (PON) ordere Gioennn
|oustesldee for deep Q Olearing dlgorittm)
The soliaa
Exeatio
490 Fouor "hp of 471--4/--000 (o
[bsttekits for Peliyy freekent Mtathors)
[Unteskdaa for
forte
action (fffwers senatie fow DON)
pranoss of supt enooh
cales
Fifferest Notations frem Q-karnlig
Gueaning
mun
O-tearning Netatites
[Combise all the lides]
(a) Task defines the conceptual skeleton (b) An intermediate subtask generates addi-
(c) Another parallel subtask produces new style instead inheriting existing lend- ing inter-agent misalignment.
and corresponding 1TEE style, which should tional content following the initial stracture. inherited by
Figure I: Inter-agent misalignment [MASFT 2.5 [1] when generating reinforcement learning teaching slides in 14T=X. Task defines the structure, while downstream parallel tasks independently create slides, leading to inconsistent styles.
single-agent systems, such as graph of thought (GoT) 35
and LangGraph [36]. and multi-agent
systems, including GPTSwarm [37]. Flow (38). AFlow [39].
AgentPrune [40), and AoT [1].
orchestration systems often assume that each subtask produces a reliable response that depends only on its parent nodes. In practice, this assumption of conditional indepen- dence often fails, leading to the problem of inter-agent misalignment. When subtasks are executed in parallel without conditioning or incompatible results. Figure iillusraate such a case in multi-agent setup for RL slide generation. The workflow first delegates the slide-structure definition to one agent, then launches parallel agents to generate algorithm-specific slides (e.g., Q-Learning and DQN). Because these subtasks run inde- pendently and the structure lacks unified notation policy, their outputs use inconsistent symbols and formatting. common mitigation strategy to introduce an additional LLM agent to review and reconcile all outputs, but this approach substantially reduces efficiency.
These traph-as-workppee
one another's intermediate outputs, agents may produce inconsistent
T2
Tj
T3
R₂
R
R
R1
R₂
R
(a) workflow: T3 T2, T1 Ta Ideal: given Rl. Rl and Ris independentt. Real: given R1, Ri and Ria are dependent.
(b) workflow; T2 T3 Ty.
(c) workflow: T3 Ti T3- Ideal: given fl. Hl and Iiss independent. Real: given R1. Rig and At are dependent.
Ideal: given Rs, Ri and figs are independent.
Real: given R1, R2 and Rss are dependent.
Figure 2: Examples of a high-level task decomposed to three subtasks {Tı, T2, Ta as graph. R, is the latent ground-trult answer while ft, response actually observed. Arrows indicate causal dependencies. where - indicate an explicit causal dependency in real execution; - indicate an ideal (theoretical) causal dependency; + indicates that the dependency can be either statistical or causal; + highlights the reason why conditional independence breaks. Intuitively, there exists at least one path where Ri and Rg remain dependent given R2-
Why does this misalignment happen? Decomposing a task into a directed acyclic graph (DAG) usually encode dependencies among subtasks. Consider the example of decomposition into three subtasks, where (eguee/e]eummrateer all possible DAGs. In theory, each subtask (node) is assumed to be conditionally independent of others given its parents. That is, with only ground-truth response R. removing its parent disconnects Rt from the rest. For example, in (a), deleting R₁ isolates Ri, from R3, implying R2 and R3 are conditionally independent given R, However, practice, only approximate responses R, are available, each causally dependent on its ground -ruth counterpart R,, Therefore, R2 and Rg share hidden dependencies through R1, violating conditional independence.
Motivated by this finding, we propose SEQCV, dynamic framework that enables reliable execution under violated conditional independence assumptions. In SEQCV, subtasks are executed sequentially, each conditioned on all prior responses and verified via consistency checks immediately after agents generate short token sequence. At each checkpoint, the generated token sequence deemed reliable if it remains semantically consistent across multiple LLM models. When inconsistency is detected,

---

the unreliable sequence is discarded, and recursive splitting mechanism is activated to decompose the subtask into smaller components. Despite its sequential execution, SEQCV avoids repeated inner-agent misalignment corrections on the same issue and achieves higher effective throughput than parallel pipelines. Across 8 creative agentic tasks, SEQCV improves accuracy by up to 30%.
Our contributions: I We inspire subsequent research in LLM-agent systems by investigating the root cause of inter-agent misalignment in graph-based orchestration. Our practical analysis identifies the violation of the conditional independence assumption as the core issue. We further highlight that this assumption cannot be satisfied in practice due the nature of LLM-agent behavior. 2 We propose SEOCV as a practical solution to mitigate failures arising from this infeasible assumption. ② Our SEQCV enables ensembling multiple LLM models by optimizing the selection of reliable outputs. We design sequential execution and swift verification pipeline, coupled with recursive splitting mechanism, to ensure tasks are decomposed into manageable subtasks aligned with LLM capabilities. Experiments standard benchmarks and real-world tasks demonstrate consistent improvements, offering a promising direction for future research on accuracy and efficiency enhancement.
2 Methodology
We present SEQCV, dynamic framework that employs recursive splitting mechanism to ensure that tasks are always decomposed into manageable units aligned with the capabilities of LLMs. This allows multiple LL.M-agents to execute subtasks and verify their outputs effectively within the system. Furthermore, SEOCV facilitates the ensembling of multiple LLM models by optimizing the selection of reliable responses. In the following, we articulate the problem formulation, conceptual overview. We then describe the execution (i.e., how responses are generated, Section 2.1). the verification (cross-model validation, Section 2.2p and the recursive splitting mechanism (Section
Problem Formulation Let O denote high-level task objective and let g = (T, E) be directed acyelic graph (DAG) of subtasks = {Ti.... Im } as its nodes with edges encoding parent- carent-child dependencies among nodes. In prior graph-as wwors-wwwowwwwwwwwoooooooooooow systems, each T, emits an approximate response R; conditioned only on its parents {R, j E Pa(i)), assuming conditional independence from all other nodes. In practice, hidden correlations among LLM outputs violate this assumption, causing inter-agent misalignment. Our goal is to execute G reliably by verifying the reliability of each subtask's 's response before this response serves as input context for downstream subtasks.
Overview As shown in Figure 3 SEQCV decomposes given high-level task into sequence of subtasks and orchestrates team of diverse LLM agents to execute these subtasks reliably by always conditioning on the complete history of verified outputs. Beginning with the first subtask, an agent from one L.M model generates a candidate response in the context of all previously accepted segments of responses, ensuring full awareness of prior decisions. This candidate response is then pecr-reviewed by agents of other LLM models. Only when a majority consensus confirms its coherence and consistency, the candidate response is appended to the global history as accepted response. consensus is not achieved, the current subtask automatically decomposed into simpler subtasks, each re-entering the same generate-verify verify--ppy loop. This process continues sequentially, recursing only when necessary, until all tasks yield reliable responses or a maximum recursion depth is reached. By combining full-trace conditioning with early, targeted splitting, SEQCV prevents hidden dependencies from triggering error cascades while maintaining end-to-end efficiency.
Advances of SEQCV As illustrated in Figure 2 latent dependencies can propagate along paths in the workflow graph that do not correspond to direct parent-child edges, since an LLM --gnerrrtteed response is only approximation of ground truth. This implies that hidden dependencies generally exist even when the observed workflow graph suggests conditional independence. By constructing a task's context encompass all previously generated responses, capture these hidden dependencies and enforce high-level consistency in terminology, style, and factual content across subtasks. This approach replaces the summary and rerun verun-after-com after-completion modules used in existing work [38] 39 to resolve conflicts and ensures that computational resources and token consumption are focused on generating extra information. In other words, SEQCV allocates tokens efficiently and wisely.
To further improve reliability, we assess the reliability of each subtask. Note that this challenging because we lack the ground truth of intermediate results. We propose that it can be approximated by
3

---

Vvssa global contONt
Cumren gpobal contest
Ouroer: guoda conteet
Dewolop gamo....
Agent
Geprant
Apent
Segient
Agen:
Segrrent
Apent:
+
Crmen-Maade
Seyyoor
Round
Varilied aogment
enquertidly KDNGOw (ead
Apert
Apart
Valdation
Segmont
Bogmeng
Segment
Segment
Segmeent
6
Gegment
Sloot bost
Subliahk2
Subrank
Validation
Final
SubMaaR
consoncue IS not 357leved
Figure 3: Overview SEQCV. high-level executed sequentially and split if necessary. During task exccution, each agent generates a segment candidate response conditioned on the verified context of all prior outputs. The generated segment is then peer-reviewed by other LLM-agents. If consensus approves, it accepted reliable response; otherwise, the subtask splits into simpler simpler subtasks), re-entering the generate-vent verer--l-veri loop. This recursion continues until all tasks are resolved or maximum depth is reached, ensuring reliability while maintaining efficiency. Algorithm Timplements segment-level generation and validation is provided and Algorithm implements dynamic splitting.
(c.g., task is split four
the extent to which its response reflects common sense across different models. We only split task when this cross-model consensus indicates that splitting necessary to resolve disagreement.
To further enhance runtime efficiency, SEQCV employs segment-level execution. In this way, validation occurs early via peer review, and if most results incorrect, the process be halted immediately, avoiding further wasted computation.
2.1 Response Generation
Segment-Basee Response Generation For subtask T, each agent E A receives the subtask context and global context prompt so far (see Subtask-Context Generation and Global-Context Construction below), and generate full response segment
$)1i,a(4))
Generate(LLM, Ty-1ji.a; I1.),
(I)
subject to maximum length 4g. During code implementation, lo is a tunable hyperparameter that controls the granularity of subtask refinement. For ease of reading. we denote (1,) simply as Si,a in the following sections. Each S;, is verified via Cross-Model Validation (see Section 2.2 A
passed Sy-1ji, will be accepted a full, verified response Ry-1jk,a lia for subtask context generation.
Subtask-Context Generation The bubtask-contexs generation means for each subtask T,, we assemble single prompt Tia that includes the entire execution trace date:
T--1ja [T:; Aijicas Raji,ar ..... Ry R
where each R,,,, a is full, verified response (from S.j.a) to address subtask Tg-
This subtask-context conditioning reduces inter-agent misalignment from any contradiction or mis- match between R. and its earlier outputs. As illustrated in Figure 2 latent correlations can propagate workflow graph paths beyond direct parent-child edges during practical execution, because the ALLM-generate response is an approximation of ground-truth. By generating subtask context T.Jiaa that encompasses all previously verified segments responses, we can: (a) capture such hidden dependencies; (b) enforce high-level consistency in terminology, style, and factual content across subtasks; and (c) mitigate the risk of a model "forgetting" context outside its own conditioning set.
Global-Context Construction With collection of generated subtask- {TTycCl) (y)s means this subtask is completed) over all possible a, majority voting will invoke to select the best π by
Sclect(LLM, I1;),
(2)

---

and each where I1,+1 [ : 11,,--. II. That is, the global context up to i 1 (as T₃ is completed) constructed by concatenating the optimal subtask-context generations. While Select(-) invokes an LLM, it only needs to output a decision of which candidate to select, rather than generate a lengthy response. This design significantly improves efficiency and reduces computational cost.
2.2 Verification and Splitting
Because conditional independence among subtask responses rarely holds in practice, SEQCV executes subtasks sequentially, verifying each response segment before incorporating it into the global context for downstream tasks. While this guarantees reliability, naive sequential execution can lead high end-to-end latency. To improve efficiency without compromising correctness, we introduce two complementary mechanisms: segment-level cross-model validation and dynamic recursive splitting of misaligned subtasks. These mechanisms jointly prune incorrect response segments at early stage, enabling the system to focus computational resources on reliable content as much as possible.
Cross-Model Validation With regard to generating the j-th segment of subtask T; every agent dt evaluates each peer candidate response {S]]41>-- S),,L by returning binary vote
Ua (o:
if Syya is coherent and consistent with Te. 1=
otherwise,
We then consider each candidate response Siks reliable for its generator as if it receives at least rn votes, where [ 1/n. In that case, agent a₂ commits RiIi,e = Sjii. After verification via the above Pross-Model Validation, the Subtask-Context Context Generation is invoked, and each agent continues segment-based response generation based on its own verified subtask context of its own, rather than relying on a single "best" response.
If Siji, fails to achieve rn votes, then agent ae treats Sylie as unreliable generation and discard it. This segment- based per-agent validation preserves the diversity of all plausible solutions while ensuring that no agent propagates an unreliable response.
Dynamic Recursive Splitting If none of the candidate responses for subtask T; pass the cross- model validation, we discard all candidate segments and invoke learned decomposition model Maplit on the global context II, This yields new directed subgraph
g' = ((Ti, ... T;-1, TT,.,...... Ti., Ti+1, ... Tm)) &,
where each T' a simpler subtask and & encodes newly added internal dependencies. Namely, 9 updated to new DAG G' by inserting small subgraph. To preserve the original ordering of nodes other than T,, every parent of T; (i.e., p E Pa(T.)) is directed to each new subtask TV (a.-)* T (1.-) is then directed to every child of T3 (i.e., c E Ch(T))).
We recompute a topological ordering of G' and resume sequential execution at the first newly inserted subtask, processing each T(,,) at depth d + I under the same segment-basee response generation, cross-model validation, and splitting. By allocating efforts to correct the misaligned region and preserving the remainder of the workflow intact, this mechanism bounds the cost of recovery and prevents global pipeline re-execution. Recursion terminates when all subtasks verify successfully or when the maximum depth dmaax is reached; in the latter case, we fill the original T; by selecting the candidate response that achieved the highest total vote count.
Experiments
We conduct thorough experiments to evaluate SEQCV by performing extensive benchmark assess- ments on six standard datasets, comparisons with state-of-the-art (SOTA) reasoning models, efficiency comparison analyses, and investigations using cross-validation modules and workflow decom- position modules. Our key findings reveal consistent performance enhancements across various tasks, with particularly notable improvements in multi-hop reasoning. By comparing with leading reasoning models, we demonstrate SBQCV's efficacy as a versatile framework. Additionally, our efficiency comparison experiments further substantiate SEQCV's 's flexibility and efficiency. Finally, evaluations of critical components such as the DAG structure and decomposition mechanism through cross- validaain modules and workflow decomposition modules verify the necessity of our design.
5

---

Algorithm Segment-level Generation and Valida- tion
Algorithm 2 Running and Dynamic Splitting DAG
glogaa context 11, depth d, agent pool Require: DAG 9 = (T, E), global
Require:
Task T,, global context II, agent pool A. vote threshold max round Jmaxe, max token per round lg
A. primary A. vote threshold r, max depth dmax split model Maplit
function SEGGENIT; II, A, Tr lns )
function RUNGRAPHIG, II,
for to Jo Jmax do
D Each agent generates a full
if dmax then
response segment for all do end Sjlt,a ~ Ceneraae(LLMM, "5-1jKa. lo)
< < MAX MoXVoro( (frgiss Dl dopth limit: force to pick
via Equatio
11
7
1111
# #success +- O per-agent context update for all A do votes - ΣsEA Verify (b, Sylner my- 11-1) majority voting votes > |AI/2 then
2. Cross
Cross-model model validation and
return 9
end
ord (OP)ORDER(S)
for T; in ord do (stafus, mj SEGÚEN(U,, 11, if status = success then
Jjiia I5g lii,t 1
1 [=": TT.]
append verified segment
#success + #success
else
(T. T
end for if suorrsssTT .A| then return (failure, B)
SPLITGRAPHI T(, b D-ppit T. into simpler subtasks
INSERTSURTAAKKSG T, (T (Ti,))
return RUNGRAPHIG, II, d +
b recurse on
end if
updated graph end if for return end function
return (success, it.)
end function
3.1 Experimental Setup
Agentic Tasks We evaluate 8 tasks designed to test creativity, multi-step reasoning, and adherence to diverse constraints. The detailed task prompts can be found in Appendix D
NeurIPS Website:
The task requires creating website with four elements: navigation menu, a paper submission section, a schedule display, and speaker profiles. The primary constraint is that the solution must use only HTML and CSS.
Create a 30-page LaTeX presentation on maximum likelihood estima- tion with motivation, problem statement, intuitive solution, detailed math equations. It must be implemented in LaTeeX format.
Lecture Slides Generation:
This game combines action and logic, requiring implementation of player movement, enemy AI, collision detection, a scoring system, and game-over mechanism. must be implemented in Python with Pygame, without sound or external images.
Pac-Tank Arcade Game:
Electrical-Cireuit Puzzle Game: This puzzle involves a drag-and-drop interface, circuit validation, level progression, visual feedback, and a win condition. The task must be implemented using vanilla JavaScript with no libraries.
This hybrid game merges snake mechanics with chess logic, requiring snake movement, chess piece mechanics, growth mechanism, strategic gameplay, and lose condition.
Snake-Chess Fusion:
The allowed setup Python with Pygame and no sound.
RPG Task Manager: This productivity tool introduces gamified task management through task CRUD operations, quest progression, character statistics, and save/load functionality. It must be implemented using HTML, CSS, and JavaScript, with no backend.
The task combines falling-block mechanics with match-3 detection, line clearing, score tracking, and level difficulty progression. It uses Python with Pygame, again without sound.
Rejeeeee Mash-up:
Family Travel Planning:
The goal is produce LaTeX-based travel plan including flight details, accommodation bookings, daily agenda, and cost estimates.
Baselines We compare the generated results against three recent baselines: Flow [8, AFlow B3 and Atom [41] as well as miniini- [42 via OpenAl interface. We implement SEQCV by using a mixture of 04-mini and o3-mini to complete each task. For each method, we run three times and select the best result. For code tasks, we manually check if they implement the necessary functionality required. For all tasks, we use GPT to evaluate which one better and identify the core weaknesses based on the prompt.

---

Table 1: Average performance seven agentic tasks. HR: Hard Requirements, ES: Execution Success, CA:
Constraint Adherence. Our method (SEQCV) substantially outperforms baselines across three metrics, demonstrating superior capability in handling complex creative and interactive tasks.
Method
HR
ES
CA
AVG
AFlow
36%
30%
40%
36%
Atom
26%
40%
30%
29%
Flow
56%
70%
40%
58%
04-mini-high
81%
100%
70%
82%
SeqCV
83%
100 %
100%
88%
Performance metrics Each method is evaluated on seven challenging agentic tasks using three key metrics. Hard Requirements (HR) measures the percentage of task-specifie requirements fully satisfied. Execution Success (ES) represents the percentage of runs completing without major bugs. Constraint Adherence (CA) quantifies the percentage of runs respecting all specified constraints. We also compare the running time for different Al agent systems.
Quantified Results
Each method is executed three times, and the best run is selected for reporting. For sede-generation tasks, manually verify whether the produced code correctly implements the required functionalities. For all tasks, GPT-4.1 based evaluation assists in assessing correctness and identifying weaknesses based the prompt. All method names masked during evaluation ensure fair comparison.
Table Jummmriiesz the average performance across seven tasks. The results highlight consistent improvements by our proposed method.
SeqCV achieves the highest total score of 88%, outperforming 04-mini-higes (82%) and Flow (58%). It attains perfect scores on both Execution Success and Constraint Adherence, indicating superior reliability and precision. Compared to bascline models. SeqCV shows gains of +6% over o4-mini- high, +30% over Flow, +52% over AFlow, and +59% over Atom. The results further reveal that SeqCV's model ensemble combining 04-mini and 03-mini outperforms the stronger individual model.
3.3 Per-Task Analysis
We provide detailed analysis for each agentic task, examining the major weaknesses of different methods. For each task, we present visual results and analyze the specific failure modes of baseline methods. We present detailed discussions of three tasks in the main text: NeurIPS Website Devel- opment, Lecture Slides Generation, and Family Travel Planning. Analyses for the remaining five tasks (Pac-Tank Arcade, Snnetrical-Circuce Puzzle, Snake-Chess Fusion, RPG Task Manager, and TetrisBBejewerr Bejeweled Mash-up) are provided in Appendix B
3.3.1 NeurIPS Website Development
The task requires creating conference website with navigation menu, paper submission section, schedule display, and speaker profiles, constrained to HTML+CSS format only. Figure shows representative outputs from different methods.
Weaknesses Analysis AFlow generates an incomplete website missing the navigation menu and schedule sections. The submission form exists but lacks validation, and the HTML structure is poorly nested. Atom produces static placeholder content without interactivity. and the minimal CSS results in an unprofessional appearance. Flow outputs well-structured HTML document but with limited styling and an unformatted schedule section. 04-mini-high achieves good layout and styling, yet omits detailed speaker biographies and required form fields. In contrast, SEQCV fulfills all requirements with comprehensive HTMI. structure, polished styling, and functional validation, though mobile responsiveness can be improved.

---

(a) AFlow
(b) Atom
(c) Flow
(d) 04-mini-high
(c) SEQCV
Figure 4: NeurlPS Website Development results. From left to right: oulputs from different methods showing varying levels of completeness and styling quality.
(a) AFlow
(b) Atom
(c) Flow
(d) o4-mini-high
(c) SEQCY
Figure 5: Lecture Slides Generation results. Sample pages showing varying levels of completeness, mathematical rigor, and pedagogical structure across different methods.
3.3.2 Lecture Slides Generation
The task requires creating a comprehensive 30-page LaTeX presentation on maximum likelihood estimation suitable for a 2-hour research-level lecture. The slides must include motivation, problem statement, intuitive solutions, detailed mathematical derivations, and be self-contained. Figure shows sample slides from different methods.
Weaknesses Analysis AFlow produces only 13 slides focusing on two distributions, lacking variety, intuition, and real-world examples. Atom creates 14 slides with shallow coverage, incomplete derivations, and placeholder text, omitting key theoretical properties and motivation. Flow delivers full 30-slide deck with strong theory but no visual aids or case studies, limiting engagement. 04-mini-high provides 31 slides with solid fundamentals but lacks depth, numbering, and advanced topics such as EM or logistic regression. SEQCV generates well-structured 30+ slide presentation with complete derivations, examples, and practical insights, though it would benefit from more visuals and intermediate mathematical steps for clarity.
3.3.3 Family Travel Planning
This task generates a NeurIPS 2025 travel itinerary with flight details, accommodation bookings, daily agenda, and cost estimates in LaTeX format. Figure shows the generated itineraries.
Weaknesses Analysis AFlow omits flight times and airline details, provides vague accommodations, and contains calculation errors. Atom outputs placeholder bookings and unrealistic pricing with minimal agenda content. Flow includes detailed activities and realistic bookings but has LaTeX compilation issues from missing packages and faulty table syntax. 04-mini-high presents coherent content but lacks detailed schedules and a complete cost breakdown including taxes and contingencies. SFQCV produces a complete, woll-formatted LaTeX document with accurate travel details, realistic budgeting, and professional presentation.
8

---

(n) AFlow
(b) Atom
(c) Flow
(d) od od-mini-high
(c) SEOCY
Figure 6: Family Travel Planning showing LaTeX-formater itineraries with flight details, accommodations, daily schedules, and budget breakdowns.
Table 3: Average running time (seconds) of SEQCV compared baselines over tasks.
Method
slides
nips-web
pac-fank
snake-chess
fask-RPG
tetris-bejew
fravel
avg
04- mini-high
47s
40s
59s
58s
54.43s
AFlow
202s
242s
170s
115s
139s
118s
160.86s
Flow
1083s
354s
330s
1539s
571s
560s
218s
665.00s
464s
476s
533s
556s
668s
533s
428s
522.57s
SEQCV (Ours)
255s
173s
160s
285s
225s
217s
205s
217.148
3.4 Misalignment Analysis
To directly measure the effect of misalignment reduction, we conducted controlled experiment isolating the impact of our sequential conditioning mechanism. We used the same work- flow produced by parallel baseline method and modified only the execution mode: the parallel execution was replaced with sequential conditioning, while all other components (such as cross- cross-verification and recursive splitting) were disabled. Thus, the only difference between the settings was whether sub- tasks were executed concurrently or sequentially conditioned on prior outputs. Any performance difference can therefore be attributed to reduced inter-agent misalignment.
Table 2: Controlled comparison iso- lating the effect of inter-agent mis- alignment. Sequential conditioning im- proves accuracy by reducing depen- dency conflicts across agents.
Configuration
Score (%)
Parallel (baseline)
52.3
Sequential only
58.4 .4
This controlled setup demonstrates that sequential conditioning mitigates inter-agent misalignment arising from dependency inconsistencies, resulting in measurable accuracy gains.
3.5 Running Time Efficiency
As shown in Table 3 SEqCVachieves an average runtime of 217s across eight representative tasks, corresponding 2.5x speed-up over Flow (665s) s) and 2.1x speed-up over Atom (523 On the six most computationally intensive tasks (slides, nips-web, pac-tank, snake-chess, task-RPG and tetris-bejew), SBoC Vreduces execution time by 51 51%-82% compared to Flow, shaving more than half off runtime. Although parallel pipelines can process multiple sub-tasks concurrently, they incur frequent misalignment corrections that erode effective throughput. By adopting carefully optimized sequential strategy, SEQCVavoide these costly corrections and delivers higher end-to-end throughput on complex tasks.
4 Related Work
LLM-agent Systems The emergence of large language models (LLMs) has revolutionised agent capabilities, reshaping their role in artificial intelligence and expanding the scope of their applications 21. With the rising adoption of LLM agent in the system, LLM-agent systems show great

---

potential in automation task execution 23 4 acTOSs variety of domains, such as such as software
development [5] 7). scientific discovery 8 9 T0 and human behaviour simulation [T 13
The capability of M---- can be further augmented with external tools and APIs 14 15 16
enabling a wide use in a diversity of scenarios and benchmarks [17
To enhance the reasoning capabilities of LLMs, complex objectives are typically decomposed into intermediate reasoning steps. including chain of thought (CoT) [27], ReAct [28). tree of thought
(ToT) [29], and Reflexion [30]. Single-agent implementations of this approach include AutoGPT B1. BabyAGI [32] LangChain [33], and Llama-index 34
Decompositing a high-level task into subtasks as a structured graph can further improve coordination and robustness of the task execution,
both in single-agent settings (e.g., Graph of Thought (GoT) [35,, LangGraph (36) and in multi-agent
settings (c.g., GPTSwarm 37.. Flow [8]. AFlow [9]. AgentPrune [40], AoT [1
LLM-agent Systems Orchestration With regards to the orchestration of agents, a branch of approach applies a conversational collaboration and numan-in-tth-loon interaction. Examples include AutoGen 2]. Reflexion [30). DSPy (45,, CAMEL 4 and Voyager [46], which enable multi- round dialogues, reflective reasoning. and declarative composition for improved coordination [47]. However, this approach can be less effective in multi-agent systems such as AG2 [ and ChatDev, where dialogue breakdowns often limit performance. Alternatively, role-based orchestration assigns predefined agent roles and structured workflows, as demonstrated by MetaGPT (3, which uses standard operating procedures (SOPs) to coordinate LLM agents on software development tasks. Instead of a predesigned pipeline, modular and dynamically executable workflows offer improved performance and flexibility. Systems such GPT-Swarm [37]. Flow [38, AutoFlow [48], and Cut-the- -rap (40) adopt DAG-based or communication
wtwpuuiiiiattooooommiieeee optimized designs to enable fault-tolerant
and reconfigurable execution. Further enhancements in adaptability are achieved through graph-based
agent communication protocols (49 and reinforcement learning- based role assignment strategies [50].
Task Verification Alongside Orchestration Alongside the orchestration, task verification is es- sential for ensuring the result quality and system robustness in LLMM-agent workflows []. Existing techniques include self-consistency voting [51] reflective refinement (30, LLM-as-a-judge eval-
uation [22, verifier agents (53), and token-level consistency checks 1 Hierarchical validation
frameworks further incorporate syntactic, semantic, and goal-specific checks (38), while arbitration schemes resolve conflicts via voting or consensus [47] Additional innovations include progressive refinement loops (54), structured debate voting-based decision making 440, dual-layer validation
pipelines [55). dynamic fallback complctions [56], and role reassignment for failing agents [57]. Systems typically terminate when retry limits are exceeded (54), semantic drift detected [I] or insufficient agent participation occurs [38]. Recovery mechanisms range from returning partial results to invoking fallback models or recursive decomposition 37). However, limited work has systematically compared the efficiency and token cost of these task verification methods.
5 Conclusion
Inter-agent misalignment is a critical factor contributing to the failure of multi-LLM-aggan systems. In this study, we investigated its underlying cause. Through practical examination of state-of-ther art LLM-agent systems that employ graph-based orchestration, we identified the violation of the conditional independence assumption as the root issue. This assumption, which requires agents to generate ground-trus -tuuh responses at each step, can NOT be satisfied in practice due to the inherent limitations of current LLMs. To address this, proposed SEQCV, framework that enhances output reliability via sequential execution and swift verification. SEQCV further introduces a recursive splitting mechanism decompose tasks into smaller, manageable subtasks aligned with the L.MMs' capabilities. Unlike existing methods, SEQCV directly concatenates verified responses without requir- ing full-context re-evaluation, significantly reducing token costs and improving system efficiency. We evaluated SEQCV on both standard benchmarks and real-world user tasks, demonstrating consistent improvements over existing otate-of-the-art approaches in both accuracy and efficiency.
10

---

Acknowledgments
TLL is partially supported by the following Australian Research Council projects: FT220100318, DP220102121, LP22000057,, LP220200949.
References
[1] Mert Cemri, Melissa Z. Pan, Shuyi Yang. Lakshya A. Agrawal, Bhavya Chopra, Rishabh Tiwari, Kurt Keutzer, Aditya Parameswaran, Dan Klein, Kannan Ramchandran, Matei Zaharia, Joseph E. Gonzalez, and lon Stoica. Why do multi-agent IIm systems fail? In Proceedings of the Building Trust Workshop at the 13th International Conference on Learning Representations (ICLR), Singapore, April 24-28, 2025. University California, Berkeley and Intenso Sao Paolo, 2025. Equal contribution by first three authors.
[2] Qingyun Wu, Gagan Bansal, Jieyu Zhang, Yiran Wu, Shaokun Zhang, Erkang Zhu, Beibin Li, Li Jiang, Xiaoyun Zhang, and Chi Wang. Autogen: Enabling next-gen LLM applications via multi-agent conversation framework. In First Conference on Language Modeling, 2024.
[3] Sirui Hong. Mingehen Zhuge, Jonathan Chen, Xiawu Zheng, Yuheng Cheng, Jinlin Wang. Ceyao Zhang, Zili Wang, Steven Ka Shing Yau, Zijuan Lin, Liyang Zhou, Chenyu Ran, Lingfeng Xiao, Chenglin Wu, and Jürgen Schmidhuber. MetaGPT: Meta programming for multi-agent collaborative framework. In The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024, 2024.
[4] Guohao Li, Hasan Hammoud, Hani Itani, Dmitrii Khizbullin, and Bernard Ghanem. CAMEL: communicative agents for "mind" exploration of large language model society. Advances in Neural Information Processing Systems 36: Annual Conference Neural Information Processing Systems 2023, NeurlPS 2023, New Orleans, LA, USA, December 10 16, 2023, 2023.
[5] Chen Qian, Wei Liu. Hongzhang Liu, Nuo Chen, Yufan Dang. Jiahao Li, Cheng Yang, Weize Chen, Yusheng Su, Xin Cong, Juyuan Xu, Dahai Li, Zhiyuan Liu, and Maosong Sun. ChatDev: Communicative agents for software development. In Lun- Wei Ku, Andre Martins, and Vivek Srikumar, editors, Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics ( (Volume 1: Long Papers), pages 15174-15186 15186, Bangkok, Thailand, August 2024. Association for Computational Linguistics.
[6] Xingyao Wang, Boxuan Li, Yufan Song, Frank F. Xu, Xiangru Tang. Mingehen Zhuge, Jiayi Pan, Yueqi Song, Bowen Li, Jaskirat Singh, Hoang H. Tran, Fuqiang Li, Ren Ma, Mingzhang Zheng, Bill Qian, Yanjun Shao, Niklas Muennighoff, Yizhe Zhang, Binyuan Hui, Junyang Lin, Robert Brennan, Hao Peng, Heng Ji, and Graham Neubig. OpenHands: An Open Platform for AI Software Developers Generalist Agents. In The Thirteenth International Conference on Learning Representations, ICLR 2025, Singapore, April 24-28, 2025, 2025.
[7] Genghan Zhang, Weixin Liang, Olivia Hsu, and Kunle Olukotun. Adaptive self-improvement LLM agentic system for ML. library development. In Workshop on Reasoning and Planning for Large Language Models, 2025.
[8] Kyle Swanson, Wesley Wu, Nash L. Bulaong, John E. Pak, and James Zou. The virtual lab: Ai agents design new sars-cov-2 nanobodies with experimental validation. bioRxiv, 2024.
[9] Chris Lu, Cong Lu, Robert Tjarko Lange, Jakob Foerster, Jeff Clune, and David Ha. The AI Scientist: Towards fully automated open-ended scientific discovery. arXiv preprint arXiv:2408.06292 2024.
[10] Michael Skarlinski, Tyler Nadolski, James Braza, Remo Storni, Mayk Caldas, Ludovico Mitchener, Michaela Hinks, Andrew White, and Sam Rodriques. Futurehouse platform: Superintelligent ai agents for scientific discovery. https://www http://wwfuuureeeeeeeeeeeee futurehouse org/ rrsearch- htt.arrcaarroocccmmmtaaaaaaaannn wtw.rinngnuuuuurruu---eeeeeee www.hisoe-paattooaommmoomo agents,
May
2025. Research Announcement, FuturcHouse.

---

[11] Joon Sung Park, Joseph C. O'Brinn, Carrie J. Cai, Meredith Ringel Morris, Percy Liang, and Michael S. Bernstein. Generative agents: Interactive simulacra of human behavior. In the 36th Annual ACM Symposium on User Interface Software and Technology (UIST' '23), UIST '23, New York, NY, USA, 2023. Association for Computing Machinery.
[12] Ziyi Yang, Zaibin Zhang, Zirui Zheng. Yuxian Jiang, Ziyue Gan, Zhiyu Wang. Zijian Ling, Jinsong Chen, Martz Ma, Bowen Dong. Prateek Gupta, Shuyue Hu, Zhenfei Yin, Guohao Li, Xu Jia, Lijun Wang, Bernard Ghanem, Huchuan Lu, Chaochao Lu, Wanli Ouyang, Yu Qiao, Philip Torr, and Jing Shao. Oasis: Open agent social interaction simulations with one million agents, 2024.
[13] Chong Zhang, Xinyi Liu, Zhongmou Zhang, Mingyu Jin, Lingyao Li, Zhenting Wang. Wenyue Hua, Dong Shu, Suiyuan Zhu, Xiaobo Jin, et al. When ai meets finance (stockagent): Large language model-based stock trading in simulated real-world environments. arXiv preprint arXiv:2407. 18957, 2024.
[14] Yujia Qin, Shihao Liang, Yining Ye, Kunlun Zhu, Lan Yan, Yaxi Lu, Yankai Lin, Xin Cong, Xiangru Tang. Bill Qian, Sihan Zhao, Runchu Tian, Ruobing Xie, Jie Zhou, Mark Gerstein, Dahai Li, Zhiyuan Liu, and Maosong Sun. Toolllm: Facilitating large language models to master 16000+ real- woaldworld apis. In The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024, 2024.
[15] Yongliang Shen, Kaitao Song, Xu Tan, Dongsheng Li, Weiming Lu, and Yueting Zhuang. Hugginggpt: solving ai tasks with chatgpt and its friends in hugging face. In Proceedings of the 37th International Conference on Neural Information Processing Systems, Red Hook, NY, USA, 2023. Curran Associates Inc.
[16] Wenqi Fan, Yujuan Ding, Liangho Ning. Shijie Wang, Hengyun Li, Dawei Yin, Tat-Seng Chua, and Qing Li. A survey on rag meeting Ilms: Towards retrieval-augmented large language models. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, KDD '24, page 6491-6501, New York, NY, USA, 2024. Association for Computing Machinery.
[17] Shishir G. Patil, Tianjun Zhang, Xin Wang, and Joseph E. Gonzalez. Gorilla: Large language model connected with massive apis. In A. Globerson, L. Mackey, D. Belgrave, A. Fan, U. Paquet, J. Tomezak, and C. Zhang, editors, Advances in Neural Information Processing Systems, volume 37, pages 126544-126565.
Curran Associates, Inc., 2024.
[18] Xiao Liu, Hao Yu, Hanchen Zhang, Yifan Xu, Xuanyu Lei, Hanyu Lai, Yu Gu, Hangliang Ding, Kaiwen Men, Kejuan Yang, Shudan Zhang, Xiang Deng, Aohan Zeng, Zhengxiao Du, Chenhui Zhang, Sheng Shen, Tianjun Zhang, Yu Su, Huan Sun, Minlie Huang, Yuxiao Dong, and Jie Tang. Agentbench: Evaluating Ilms agents. In The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024, 2024.
[19] Tianqi Xu, Linyao Chen, Dai-Jie Wu, Yanjun Chen, Zecheng Zhang, Xiang Yao, Zhiqiang Xie, Yongchao Chen, Shilong Liu, Bochen Qian, Philip Torr, Bernard Ghanem, and Guohao Li. Crab: Cross-environment agent benchmark for multimodal language model agents, 2024.
[20] Yiheng Xu, Hongjin Su, Xing Chen, Boyu Mi, Qian Liu, Weijia Shi, Binyuan Hui, Fan Zhou, Yitao Liu, and Tianbao Xic. Lemur: Harmonizing natural language and code for language agents. In The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024, 2024.
[21] Bang Liu, Xinfeng Li, Jiayi Zhang, Jinlin Wang, Tanjin He, Sirui Hong, Hongzhang Liu, Shaokun Zhang, Kaitao Song, Kunlun Zhu, et al. Advances and challenges foundation agents: From brain- inspired intelligence to evolutionary, collaborative, and safe systems. arXiv preprint 1xXiv:2504.01990 2025.
[22] Julia Wiesinger, Patrick Marlow, and Vladimir Vuskovic. Agents. https: // www.www....... com/whitepaper-ageeee agentss May 2025. Kaggle Whitepaper.
12

---

[23] Shaokun Zhang, Ming Yin, Jieyu Zhang, Jiale Liu, Zhiguang Han, Jingyang Zhang, Beibin Li, Chi Wang. Huazheng Wang, Yiran Chen, and Qingyun Wu. Which agent causes task failures and when? on automated failure attribution of Ilm multi-agent systems. In Proceedings of the 4sst International Conference on Machine Learning (ICML). PMLR, 2025. Spotlight.
[24] Shaokun Zhang. Jieyu Zhang, Jiale Liu, Linxin Song, Chi Wang, Ranjay Krishna, and Qingyun Wu. Offline training of language model agents with functions as learnable weights. In Forty-first International Conference on Machine Learning, 2024.
[25] Shaokun Zhang, Jieyu Zhang, Dujian Ding, Jiale Liu, Mirian Del Carmen Hipolito Garcia, Ankur Mallick, Daniel Madrigal, Menglin Xia, Victor Rühle, Qingyun Wu, and Chi Wang. Ecoact: Economic agent determines when to register what action. In Workshop on Reasoning and Planning for Large Language Models, 2025.
[26] Jintian Zhang, Xin Xu, Ningyu Zhang, Ruibo Liu, Bryan Hooi, and Shumin Deng. Exploring collaboration mechanisms for LLM agents: A social psychology view. Lun-Wei Ku, Andre Martins, and Vivek Srikumar, editors, Proceedings of the 62nd Annual Meeting of the Associa- tion for Computational Linguistics (Volume 1: Long Papers), pages 14544-14607, Bangkok, Thailand, August 2024. Association for Computational Linguistics.
[27] Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed H. Chi, Quoc V. Le, and Denny Zhou. Chain-of-thought prompting elicits reasoning in large language models. In Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, NeurlPS 2022, New Orleans, LA, USA, November 28- December 9, 2022, 2022.
[28] Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. ReAct: Synergizing reasoning and acting in language models. In International Conference on Learning Representations (ICLR), 2023.
[29] Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Thomas L. Griffiths, Yuan Cao, and Karthik R Narasimhan. Tree of thoughts: Deliberate problem solving with large language models. In Thirty-seventh Conference on Neural Information Processing Systems, 2023.
[30] Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. Re- flexion: Language agents with verbal reinforcement learninn In Advances in Neural Information Processing Systems (NeurIPS), 2023.
[31] Toran Bruce Richards. Auto-gpt: An autonomous gpt-4 experiment. htttps://githbb com/ Significant -rattaaatataattt-aGGrF. 2023. GitHub repository.
[32] Yohei Nakajima. Babyagi: An experimentlll prototype framework for building self-building autonomous agents. httpps://gittub.ccomm/// www.eeaaaaaaaaaaaaaaaaaaaaa 2023. GitHub repos- itory.
[33] Harrison Chase. Langchain: Build context-aware reasoning applications. https://githubb com/ bangchain--iilanggenn 2022. GitHub repository.
[34] Jerry Liu. Llamaindex: A data framework for IIm applications. httpp://giithub.com peeryiliulllamajindee 2022. GitHub repository.
[35] Maciej Besta, Nils Blach, Ales Kubicek, Robert Gerstenberger, Michat Podstawski, Lukas Gianinazzi, Joanna Gajda, Tomasz Lehmann, Hubert Niewiadomski, Piotr Nyczyk, and Torsten Hoefler. Graph of thoughts: solving elaborate problems with large language models. In Proceedings of the Thirty-Eighth AAAI Conference on Artificial Intelligence and Thirty-Sixth Conference on Innovative Applications of Artificial Intelligence and Fourteenth Symposium on Educational Advances in Artificial Intelligence, 2024.
AAAI Press,
[36] LhngChain Inc. Langgraph: A framework for building stateful, multi-actor applications with 1lms. :ttpps//Langchainn www.aiiigggi.... github.. 1o/ Langgraph/ 2025. GitHub repository.
13

---

[37] Mingchen Zhuge, Wenyi Wang, Louis Kirsch, Francesco Faccio, Dmitrii Khizbullin, and Jürgen Schmidhuber. Gptswarm: Language agents as optimizable graphs. In International Conference on Machine Learning (ICML), 2024.
[38] Boye Niu, Yiliao Song, Kai Lian, Yifan Shen, Yu Yao, Kun Zhang, and Tongliang Liu. Flow: Modularized agentic workflow automation. In The Thirteenth International Conference on Learning Representations, ICLR 2025, Singapore, April 24-28, 2025, 2025.
[39] Jiayi Zhang, Jinyu Xiang, Zhaoyang Yu, Fengwei Teng, Xiong-Hui Chen, Jiaqi Chen, Mingchen Zhuge, Xin Cheng, Sirui Hong, Jinlin Wang, Bingnan Zheng, Bang Liu, Yuyu Luo, and Chenglin Wu. AFlow: Adaptive workflow construction for llm-based agents. In The Thirteenth International Conference on Learning Representations, ICLR 2025, Singapore, April 24-28, 2025, 2025.
[40] Guibin Zhang, Yanwei Yue, Zhixun Li, Sukwon Yun, Guancheng Wan, Kun Wang, Dawei Cheng, Jeffrey Xu Yu, and Tianlong Chen. Cut the crap: An economical communication pipeline for lim-based multi-agent systems. In The Thirteenth International Conference on Learning Representations, ICLR 2025, Singapore, April 24-28, 2025, 2025.
[41] Fengwei Teng, Zhaoyang Yu. Quan Shi, Jiayi Zhang, Chenglin Wu, and Yuyu Luo. Atom of thoughts for markov Ilm test-time scaling, 2025.
[42] OpenAL, 04-mini-high. Large language model, 2025, Accessed: 2025-05-16, via OpenAl API.
[43] Guibin Zhang, Yanwei Yue, Xiangguo Sun, Guancheng Wan, Miao Yu, Junfeng Fang, Kun Wang, Tianlong Chen, and Dawei Cheng. G-designer: Architecting multi-agent communication topologies via graph neural networks. In ICLR 2025 Workshop Foundation Models in the Wild, 2025.
[44] Zhenran Xu, Senbao Shi, Baotian Hu, Jindi Yu, Dongfang Li, Min Zhang, and Yuxiang Wu. Towards reasoning in large language models via multi-agent peer review collaboration. 2023.
http::/aaxii.orr/aa///11111./11
[45] Omar Khattab, Arnav Singhvi, Paridhi Maheshwari, Zhiyuan Zhang, Keshav Santhanam, Matei Zaharia, and Christopher Potts. Dspy: Compiling declarative language model calls into state- of-the-art pipelines. In The Twelfth International Conference on Learning Representations, 2024.
[46] Guanzhi Wang, Yuqi Xie, Yunfan Jiang, Ajay Mandlekar, Chaowei Xiao, Yuke Zhu, Linxi Fan, and Anima Anandkumar. Voyager: An open-ended embodied agent with large language models. Transactions on Machine Learning Research, 2024.
[47] Zhenhailong Wang, Shaoguang Mao, Wenshan Wu, Tao Ge, Furu Wei, and Heng Ji. Unleash- ing the emergent cognitive synergy in large language models: A task-sol solving agent through multi-persons self-collaboration. In Kevin Duh, Helena Gomez, and Steven Bethard, editors, Proceedings of the 2024 Conference of the North American Chapter of the Association for Computationna Linguistics: Human Language Technologies (Volume 1: Long Papers), pages 257-279, Mexico City, Mexico, June 2024. Association for Computational Linguistics.
[48] Zelong Li, Shuyuan Xu, Kai Mei, Wenyue Hua, Balaji Rama, Om Raheja, Hao Wang, He Zhu, and Yongfeng Zhang. Autoflow: Automated workflow generation for large language model agents, 2024.
[49] Shengchao Hu, Li Shen, Ya Zhang, and Dacheng Tao. Learning multi- multi-agent communication from graph modeling perspective. In ICLR 2024 Conference, 2024.
[50] Zelai Xu, Chao Yu, Fei Fang, Yu Wang, and Yi Wu. Language agents with reinforcement learning for strategic play in the werewolf game. In Proceedings of the 41st International Conference on Machine Learning, ICML'24. JMLR.org, 2024.
[51] Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoe Le, Ed H. Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. Self-consistency improves chain of thought reasoning in language models. In The Eleventh International Conference on Learning Representations, 2023.
14

---

Association for [52] Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang. Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, Hao Zhang, Joseph E. Gonzalez, and Ion Stoica. Judging LL LLud- with MT-bench and chatbot arena. In Thirty-seventh Conference on Neural Information Processing Systems Datasets and Benchnarks Track, 2023.
[53] Jiuzhou Han, Wray Buntine, and Ehsan Shareghi. Verifiagent: unified verification agent in language model reasoning. http://aariii.ogg/////5505..... 0,0406, 2025.
[54] Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang. Shashank Gupta, Bodhisattwa Prasad Majumder, Katherine Hermann, Sean Welleck, Amir Yazdanbakhsh, and Peter Clark. Self-refine: Iterative refinement with self-feedback. In seirth-seventh Conference on Neural Information Processing Systems, 2023.
[55] Ningyuan Xi, Xiaoyu Wang, Yetao Wu, Teng Chen, Qingqing Gu, Yue Zhao, Jinxian Qu, Zhonglin Jiang. Yong Chen, and Luo Ji. Dual-layer training and decoding of large language model with simultaneously thinking and speaking, 2024.
[56] Maor Ivgi, Ori Yoran, Jonathan Berant, and Mor Geva. From loops to oops: Fallback behaviors of language models under uncertainty. http::/aarii.orggab//2000.....
2024.
[57] Jen tse Huang. Jiaxu Zhou, Tailin Jin, Xuhui Zhou, Zixi Chen, Wenxuan Wang, Youliang Yuan, Michael R. Lyu, and Maarten Sap. On the resilience of Ilm-based multi-agent collaboration with faulty agents. http:://arzin.orglla//22200 20000/.00/0200000000 2024.
[58] Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. Mcasuring mathematical problem solving with the MATH dataset. In Proceedings of the Neural Information Processing Systems Track on Datasets and Bench- marks I, NeurlPS Datasets and Benchmarks 2021, December 2021, virtual, 2021.
[59] Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reichhro Nakano, Christopher Hesse, and John Schulman. Training verifiers to solve math word problems. CoRR, abs/21 10. 14168, 2021.
[60] Qihao Zhao, Yangyu Huang, Tengchao Lv, Lei Cui, Qinzheng Sun, Shaoguang Mao, Xingxing Zhang, Ying Xin, Qiufeng Yin, Scarlett Li, and Furu Wei. MMLU-CF: A contamination-fies multi-task language understanding benchmark. CoRR, abs/2412.15199 2024.
[61] Mirac Suzgun, Nathan Scales, Nathanael Schärli, Sebastian Gehrmann, Yi Tay, Hyung Won Chung, Aakanksha Chowdhery, Quoe Le, Ed H. Chi, Denny Zhou, and Jason Wei. Chal- lenging beg-bench tasks and whether chain-of-thoughe can solve them. In Findings of the Association for Computational Linguisties: ACL 2023, Toronto, Canada, July 9-14, 2023, pages 13003-13051. Association for Computational Linguistics, 2023.
[62] Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William W. Cohen, Ruslan Salakhut- dinov, and Christopher D. Manning. Hotpotqa: A dataset for diverse, explainable multi-hop hop question answering. In Ellen Riloff, David Chiang, Julia Hockenmaier, and Jun 'ichi Tsujü, editors, Proceedings of the 2018 Conference Empirical Methods in Natural Language Pro- cessing, Brussels, Belgium, October 31 November 4, 2018, pages Computational Linguistics, 2018
[63]
Yushi Bai, Xin Lv, Jiajie Zhang, Hongchang Lyu, Jiankai Tang, Zhidian Huang, Zhengxiao Du, Xiao Liu, Aohan Zeng, Lei Hou, Yuxiao Dong, Jie Tang, and Juanzi Li. Longbench: A bilingual, multitask benchmark for long context understanding. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), ACL 2024, Bangkok, Thailand, August 11-16, 2024, pages 3119-3137. Association for Computational Linguistics, 2024.
[64] Harsh Trivedi, Niranjan Balasubramanian, Tushar Khot, and Ashish Sabharwal. Musique Multihop questions via single-hop question composition. Truns. Assoc. Comput. Linguistics, 10:539-554, 2022.
15

---

[65] Xanh Ho, Anh-Khoa Duong Nguyen, Saku Sugawara, and Akiko Aizawa. Constructing A multi-hop QA dataset for comprehensive evaluation of reasoning steps. In Proceedings of the 28th International Conference on Computational Linguistics, COLING 2020, Barcelona, Spain (Online), December 8-13, 2020, pages 6609-6625. International Committee on Computational Linguistics, 2020.
[66] Michihiro Yasunaga. Xinyun Chen, Yujia Li, Panupong Pasupat, Jure Leskovec, Percy Liang, Ed H. Chi, and Denny Zhou. Large language models as analogical reasoners. In The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024. OpenReview.net, 2024.
[67] Harsha Nori, Yin Tat Lee, Sheng Zhang, Dean Carignan, Richard Edgar, Nicolò Fusi, Nicholas King, Jonathan Larson, Yuanzhi Li, Weishung Liu, Renqian Luo, Scott Mayer McKinney, Robert Osazuwa Ness, Hoifung Poon, Tao Qin, Naoto Usuyama, Christopher M. White, and Eric Horvitz. Can generalist foundation models outcompete special-purpose tuning? case study in medicine. CoRR, abs/2311.11443 2023.
[68] Shengran Hu, Cong Lu, and Jeff Clune. Automated design of agentic systems. CoRR, abs/2408.0844 2024.
[69] Zhenni Bi, Kai Han, Chuanjian Liu, Yehui Tang, and Yunhe Wang. tf-oott. Scaling test-time compute for enhancing LLM reasoning. CoRR, abs/2412.09072 2024.
[70] OpenAl. gpt-4o-mini. Large language model, 2025. Accessed: 2025-05-16, via OpenAl API.
[71] OpenAI. apt-4.1-namp Large language model, 2025, Accessed: 2025-05-16, via OpenAl API.
16

---

APPENDIX
Contents
Limitation
18
B
More Results Agentic Tasks
18
Lecture Slides Maximum Likelihood Estimation
19
B.2
Website Development NeurlPS 2025
21
Game Development Pac-Man fuses Tank City
22
B.4
Web Game Development Puzzle Game
24
B.5
Game Development Snake Fuses Chess
26
8.6
Tool Development Task Manager RPG
28
8.
Game Development Tetris Fuses Bejeweled
29
8.8
Family Travel Plan
31
E
yal valuuaion on Other Benchmarks
32
System Design
33
Prompts for SEQCV
35
17

---

A Limitation
By design, SEQCV linearizes the task graph g = - into directed acyelic sequence subtasks. While this design suits many content generation and document authoring scenarios, cannot naturally express cyclic or recurrent dependencies among agents. In domains such as continuous control, multi step planning with feedback loops, or systems where subtasks iteratively refine one another, purely acyclic representation becomes restrictive. Although one could in principle introduce an explicit time index & convert loops into directed acyclic graph, this workaround often results in cumbersome encodings. More flexible architectures, such as OpenAl's SwarmAgent or general message passing frameworks, may be better suited tasks that require intrinsic cyclic structures.
SEQCV conditions each subtask T; on the full history of verified responses. In practice, large scale tasks quickly exhaust LLM's context window. While a hierarchical summarization step can compress earlier segments into a shorter representation, summarization inevitably introduces information loss and cannot guarantee preservation of all context necessary for downstream subtasks. As model context size limits increase, this constraint will lessen. Until then, careful trade offs between context fidelity and sequence length required, and some applications may remain infeasible.
Our method fails to provide benefits for reducing inter agent misalignment only in a specific type of wide graph. This occurs when each independent branch (1) has no incoming edges from outside the branch (that is, no parent nodes pointing into it), and (2) has no connections to nodes other branches. In such strictly isolated structure, there is little opportunity for our method to reduce inter agent misalignment, as no misalignment exists.
B More Results on Agentic Tasks
Real users expect autonomous assistants handle diverse, multi-step requests ranging from struc- tured document synthesis to interactive code generation and personalized logistics planning with demonstrating strong domain expertise and consistently high-quality output. To evaluate whether SEQCV meets these expectations, we design eight end-to-end agentic tasks spanning:
Long-ted text generation & structure (e.g., self-contained lectures, multi page itineraries).
Multi-modee code synthesis (e.g., LaTeX, HTML/CSS/IS, Python/Pygame),
Dynamic reasoning & planning (e.g., game AI, resource management, travel logistics), and
Cross-domaie adaptability (e.g., from mathematical instruction family travel planning).
These tasks are carefully crafted probe the core competencies of general-purposs agent system. The eight tasks cover wide range of domains with different output modalities:
Lecture-slide generation: 30-page LaTeX slide for two-hour lecture about maximum- likelihood estimation for research students.
Conference website design: HTML/CSS site for NeurIPS 2025.
Pac-Tank arcade game: Python/Pygame implementation merging gaming rules from both Pac-Man and Tank City.
Electrical-circuit puzzle game: Browser-based drag-and-droy interface using vanilla JavaScript.
Snake-Chess fusion:
Real-time Python/Pygame game fusing the gaming rules of Snake with simplified chess tactics.
RPG task manager: A desktop-style "RPG" mapping to-do items to quests (HTML/C- SS/IS).
Tetris Bejeeeed mash-up: Python/Pygame game fusing the gaming rule of Match-3 and line-clearing game.
Family travel planning: LaTeX- f-rmmttee itinerary for NeurIPS 2025, including flights, lodging, daily agenda, and cost estimates.
18

---

Baselines We implement SEQCV by using mixture of o4-mini and 03-mini to complete each task. We compare the generated results against three recent baselines: Flow [38] AFlow [9 and Atom
41 as well comparing to plain Q&A results from 04-mini-high [42] via OpenAl interface. In each of the following subsections, we first present the prompt for the task description, followed by the performance results of SEQCV and the baselines.
Evaluation For each method, we run three times and select the best result. For code tasks, manually check if they implement the necessary functionality required. For all tasks, we use GPT evaluate which is better and identify the core weaknesses based on the prompt. All method names are masked to ensure a fair comparison. After that, we will further manually check whether the weaknesses listed are correct.
Overall, our method significantly outperforms existing baselines. Notably, the framework enhances the model's s reasoning ability with mixture of 04-mini and 03-mini, it achieves even better perfor- mance than 04-mini-high.
B.1 Lecture Slides Maximum Likelihood Estimation
Task 1: Lecture Slides
Lecture slide:
I am a lecturer. I am teaching the machine learning course for research students. Please generate lecture slides for maximum likelihood estimation.
Note that:
The lecture duration is hours, so we need generate 30 pages.
2. The slides should include motivation, problem, intuitive solution, and detailed math equations.
3. Please make sure the lecture well self-contained.
Output Format:
ETTEX code
(n) (b) Atom
(c) Flow
(d) o4-mini-high
(c) SEQCV
Figure 7: Illustration of lecture slides.
AFlow This slide deck covers the essentials maximum likelihood estimation: it begins with a title and outline that includes motivation, problem statement, intuitive solution, mathematical formulation, examples for Bernoulli and Normal distributions, properties, applications, challenges, and conclusion. It includes log- likeliiol definitions and derivations for Bernoulli and Gaussian distributions, with brief discussion of asymptotic properties. However, the deck includes only 13 topical slides, which are far fewer than the 30 required resulting in insufficient coverage for two-hour lecture. The content is limited to just two examples and omits other important distributions such Poisson, Exponential, and Gamma. Furthermore, while mathematical formulas are included, intuitive explanations and real-world applications are sparse, making the slides less accessible to students who benefit from contextual learning.
19

---

Table 4: Summary of Major Weaknesses in Lecture Slide Implementations (MLE)
Method
Major Weakness
AFlow
Covers only 13 slides, far short of the required 30. Focuses on just two distributions (Bernoulli and Normal), omitting others like Poisson or Exponential. Lacks sufficient intuitive explanation and applied context. Provides only 14 slides with limited derivations. Contains placcholders and lacks worked examples. Does not clearly explain when or why to MLE.
Atom
Flow
While it meets the 30-slide requirement and covers a wide range of MLE topics. includes minimal discussion of applied use cases and gives only brief treatment of model limitations.
04-mini-high
Produces a complete slide count but with shallow treatment of advanced topics. Omits logistic regression and EM algorithm, and some slides are underdeveloped with limited explanation.
SEQCV
Offers the broadest distribution coverage and includes case studies and computational methods. Minor gaps include occasional skipped steps in derivations.
Atom This version opens with a "Comprehensive Guide" framing and includes a standard table of contents: Introduction, statistical foundations, intuition, mathematical foundations, generalized MLE, properties, computation, applications, and conclusion. defines MLE formally and includes a placeholder for an example graph. Nonetheless, it contains only 14 slides and remains too brief to fill a two-hour session. The presentation does not include detailed explanations and lacks any worked examples beyond placeholder. Only the basic likelihood function is shown while omitting important and profound concepts. It also does not provide a discussion of why or when MLE should be used, leaving gap in forming good motivation for learning or practical guidance.
Flow The Flow output delivers complete 30-slide deck that methodically addresses all major topics. It begins with motivation and background, followed by a formal problem statement and worked examples for Bernuuli, Gaussian, and exponential family distributions. It further discusses asymptotic properties such as consistency, normality, and efficiency, and explores Fisher informa- tion, the Cramér-Rao bound, and the invariance property. Numerical optimization techniques like Dowton-Raphson and EM algorithms are also introduced, as are extensions to models like logistic regression. However, the deck relies almost entirely on text and equations. There are figures or diagrams to enhance intuition. Additionally, while logistic regression is mentioned, other real- world case studies are largely absent, and topics like model misspecification and robust alternatives MLE are only briefly touched upon.
04-mini-high This submission produces a concise 31-slide deck that introduces MLE fundamentals including motivation, mathematical formulation, and properties. It defines the likelihood and log- likelihood functions and walks through closed-form solutions for Bernoulli, Gaussian, and Poisson distributions. The slides also touch on asymptotic properties and basic computational techniques like Newton-Raphson and MAP. That said, the deck has several shortcomings. It does not include slide numbers, which makes referencing and navigation difficult. The coverage of example distributions, while accurate, is shallow and does not extend to more advanced topics such as the EM algorithm or regression models. Additionally, many slides are sparsely populated, with only formula and minimal surrounding explanation.
Our method generates a richly detailed lecture deck that includes a well-organized struc- ture beginning with an introduction and motivation, followed by a clear problem statement, intuitive explanation, and thorough step-by-step derivations. These derivations span a wide range of distribu- tions including Normal, Bernoulli, Exponential, Poisson, and Gamma. The presentation also includes applied case studies, computational strategies like Nowton-Raphson and EM, and discussions of robustness, limitations, and practical considerations such as logistic regression and comparisons with Bayesian methods. It concludes with a Q& A slide and reading suggestions. While comprehensive, the slides would benefit from the inclusion of intuitive visual aids such as likelihood surface plots or EM iteration diagrams to help learners better grasp abstract concepts. Moreover, although the slide
SEQCV
20

---

content is extensive, some derivations skip intermediate steps, which could be confusing for students less familiar with salculus-based proofs.
Overall Among the five methods evaluated, Flow and SEQCVare are the most successful in meeting the task requirements. Flow achieves the target length with a precise 30-slide structure and provides a comprehensive theoretical treatment, yet it lacks visual illustrations and concrete, practical applica- tions. SEQC covers the broadest scope, including advanced material and practical implementation tips, and is well-suited for research- audience. However, it would benefit from clearer visual structure and improved use of diagrams. AFlow and Atom show promise with strong outlines and partial derivations but fall significantly short in slide count and content depth. 04-mini-high, although organized and conceptually accurate, is too minimal detail.
B.2
Website Development NeurlPS 2025
Task 2: NeurIPS2025 Website
I want create website for the following conference:
Conference Name: The Thirty-Ninth Annual Conference on Neural Information Processing Systems (NeurIPS 2025)
Date: Tuesday, Dec 2nd through Sunday, Dec 7th, 2025
Location: San Diego Convention Center, California, United States
Organizer: Neural Information Processing Systems Foundation
Please generate a detailed website structure and content for this conference. Ensure the content is professional, clear, and suitable for an international academic conference.
Output Format: HTML and CSS
(a) AFlow
(b) Atom
(c) Flow
(d) 04-mini-high
(e) SEQCV
Figure 8: Illustration of website Development.
The content is too short, and it does not meet the standards of the international academic conference.
AFlow
Atom The Atom implementation only displays schedules for December 2nd and 3rd, omitting any other relevant dates. Furthermore, the provided content lacks sufficient depth and detail meet the standards expected of an international academic conference.
Flow meets the basic task requirements but fails to generate specific dates for each day as specified in the prompt. Additionally, the use of the designated key above is non-functional and
Flow
21

---

Table 5: Summary of major weaknesses in schedule generation implementations
Method
Major Weakness
AFlow
Content lacks the depth and quality expected for an international aca-
demic conference.
Atom
Displays only schedules for December 2nd and 3rd, and the content lacks the depth and quality expected for international academic conference.
Flow
Although it meets basic task requirements, it fails to generate specific dates for each day, includes broken key that triggers page request, and presents insufficient content for academic standards.
04-mini-high
major weakness; all task requirements are fulfilled with appropriately formatted and complete academic scheduling content.
SEQCV
No major weakness; generates a correct and complete schedule aligned with international academic standards.
instead triggers unintended page request. The overall content is sufficiently detailed and does not align with the quality expected for international academic events.
This version successfully fulfills the outlined task requirements. The schedule presented as expected and meets the standard for academic presentation formatting.
04-mini-high
SEQCV Similar to 04-mini-high, SEQCVmeets all the specified task requirements, generating the schedule and content manner appropriate for the intended academic context.
Overall While Atom and Flow demonstrate partial task completion, both suffer from either incom- plete date coverage or inadequate content quality. In contrast, Hi-mini-high and SEQCVfully satisfy the prompt's 's demands and produce output suitable for academic presentation.
B.3 Game Development Pac-Man fuses Tank City
The generation results presented in Figure
9
22

---

Task 3: Pac-Man fuses Tank City
Develop a single-player game that fuses mechanics from Pac-Man and Tank City,
The player controls Pac-Man-Iae like character navigating maze, collecting pellets for points. Enemy tanks act as ghost-like chasers, actively pursuing the player using basic pathfinding.
Enemy tanks can shoot bullets in four directions. Unlike traditional bullets, these bullets do disappear-instead, they remain in the maze collectible pellets. The player must guide Pac-Man to eat as many of these bullets as possible to gain points and avoid tank power-ups.
If bullets left uncollected, enemy tanks can absorb them to accelerate their shooting rate, making them more dangerous. Tanks can also switch between patrol and chase modes to track the player.
The player can collect special items that temporarily allow Pac-Man to eat enemy tanks for bonus points.
The game must support:
Maze-based movement and pellet collection
Al-controlled enemy tanks with patrol and chase behaviors
Enemy tank shooting mechanics with persistent bullets
Tank interaction with bullets (absorb and power-up)
A GUI (using Pygame) showing the maze, player, enemies, pellets, and bullets
Smooth animations and clear visual feedback
Sound effects not required.
Output Format: Python
LE
AFlow
(b) Atom
Flow
(d) o4-mini-high
SEQCV
Figure 9; Illustration Pac-Man fuses Tank City.
AFlow failed to meet the core requirements of the task. The game only implemented few basic functions and lacked critical gameplay elements. The maze was not successfully generated, enemy tanks did not fire bullets, and the tanks remained stationary instead exhibiting tracking or patrol behaviors.
Atom also did not fulfill task objectives. The output consisted an empty maze without any interactive elements. There was no player character, no pellets or bullets to collect, and no tanks present in the scene. As result, the game was entirely con-functional.
Flow partially addressed the requirements. In this version, tanks were able to track the player using basic AI, but they could not shoot bullets. This left significant portion of the gameplay mechanics such persistent bullets and their interactions unimplemented.
23

---

Table 6: Summary of major weaknesses in pac-man tank city fusion game implementations.
Method
Major Weakness
AFlow
Incomplete implementation; maze generation failed, tanks remain static, and no bullet mechanics or tracking behavior is present.
Atom
Game is non con-functional; only an empty maze is rendered with no player, lanks, pellets, or bullets. Core gameplay elements
entirely missing. Enemy tanks can track the player but cannot shoot bullets, leaving out key mechanies such as persistent projectiles and bullet interactions.
Flow
04-mini-high
Players and tanks do not follow maze paths and can move freely across the screen; bullets vanish on hit instead of persisting, and patrol logic breaks at screen edges.
SEQCV
Mostly fulfills task requirements but suffers from flawed movement controls- arrow key inputs only move the player between corridor ends, preventing precise navigation.
04-mini-high included several game components but deviated substantially from the task expec- tations. When bullets hit the player, the player would die immediately, but the bullets would not remain in the maze as collectibles. The maze existed visually but did not function as a constraint for movement. Both the player and tanks could move freely across the screen without following the maze paths. Additionally, enemy patrols disappeared off the edge of the screen, and bullets lacked the special interactions specified in the task.
SEQCV came closest to satisfying the task requirements. It included the necessary components, flawed. Using the arrow keys moved the player from one
but player movement within the end of a corridor to the other without allowing accurate navigation to specific positions, making it difficult control the character effectively.
Overall, among the five methods evaluated, only SEoCVVmplemented the required features to meaningful extent. However, it still exhibited critical navigation issues within the maze, preventing precise control and detracting from the gameplay experience.
B.4 Web Game Development Puzzle Game
Task 4: Puzzle Game
Create a casual browser puzzle game where players build electrical circuits from components.
Drag-and and-drop resistors, capacitors, and switches onto a grid to complete Tech: HTML5 CSS3 + vanilla JavaScript.
Mechanics:
target circuit.
Levels: 20 puzzles of increasing difficulty.
Scoring: Based on correctness and build time.
Output Format: HTML5 + CSS3 + vanilla JavaScript
(a) AFlow
(b) Atom
(c) Flow
(d) o4-mini-high
(e) SEQCV
Figure 10: Illustration of Puzzle Game.
24

---

Table 7: Summary of major weaknesses in puzzle game implementations.
Method
Major Weakness
AFlow
Incomplete implementation with no grid, drag-and and -rop functionality, or playable levels; the game cannot be initiated.
Atom
Entirely con-functional: lacks grid, interactivity, and level structure; button presses yield response.
Flow
No major weakness; the game is playable, visually clear, and meets the task requirements.
04-mini-high
No major weakness; provides clean, functional interface with color- coded components and full gameplay.
SEQCV
No major weakness. A minor issue that that text slightly overlaps.
AFlow AFlow did not meet the task requirements. No playable game produced, and the implementation appeared incomplete and overly simplistic. The interface only listed the names the components resistors, capacitors, and switches, without any drag-and-drop functionality. Furthermore, there was no grid layout, and the game could not be initiated. Level progression was also entirely missing.
Atom Atom also failed deliver working puzzle game. The page lacked grid and the basic mechanisms required to begin gameplay. No levels were implemented, and features such as a timer or an answer validation button were absent. Additionally, pressing buttons on the screen did not trigger any response, rendering the game unplayable.
Flow Flow fulfilled the task requirements and successfully produced playable puzzle game. The game interface was visually appealing, and different colors were used to represent resistors, capacitors, and switches.
The 04-mini-high version also the task requirements and presented fully functional game. The interface was clean and attractive, and the components were color-coded to enhance visual clarity.
04-mini-high
SEQCV SeOCVproduccd playable game that satisfied the task criteria. A noteworthy enhance- ment in this version was the inclusion of a hint system, which added depth and improved gameplay. Nevertheless, the drag-and and-drop interaction lacked polish, when components were placed on the grid, they retained their original labels and overlapped.
Overall summary, AFlow and Atom failed to deliver playable games and did not fulfill project requirements. Flow, 04-mini-high, and SEQCV S to create playable games, though each had distinct shortcomings. Among these, Flow, 04-mini-high, and SEQCV out the most successful implementations in terms of functionality and gameplay experience.
25

---

B.5
Game Development Snake Fuses Chess
Task 5: Snake Chess
Develop real-time arcade game that merges Snake with simplified chess tactics.
Player:
Controls a "snake" that grows by eating apples.
Chess Pieces: Randomly spawning pawns, knights, and bishops move following their rules.
Mechanics:
If the snake's head collides with an apple ~> length+1, score+10.
2. If the snake's s head lands chess piece's square + capture it: score piece value (pawn=5, knight=10, bishop=15); snake doesn't grow.
3. Knights can shoot bullets. When the player shot - length-1. If length - game over.
4. Bullets disappear when they hit the edges.
5. Chess pieces move one step every second; they collide with the snake's body + game over.
6. Board wraps around edges.
7. No extra image files.
UI: Pygame window showing the grid, snake, apples, chess picces, current score, and "next piece spawns in Xs."
Output Format: Python
Boore: 270
AFlow
(b) Atom
(c) Flow
Next piece 4s
(d) 04-mini-h
(e) SEQCV
Figure 11: Illustration of Snake Fuses Chess,
AFlow The AFLow implementation suffers from critical omissions. Although three chess pieces appear the start of the game, no further pieces spawn, rendering the "next piece spawns in Xs"
26

---

Table 8: Summary of major weaknesses in snake-chess implementations.
Method
Major Weakness
AFlow
Fails spawn additional chess pieces and does not execute piece move- ment self-collision detection, leaving key mechanics inactive after initial setup.
Atom
Omits critical mechanies i.c., boundary and knight projectiles.
Flow
Suffers from unmanageable speed, broken bullet mechanics, and lacks dynamic chess piece behavior, severely compromising playability. No major weakness; implements all core features including wraparound, per-second movement, scoring, collisions, and UI elements correctly. Core gameplay mechanics are solid and complete, but the missing back- ground grid detracts from spatial clarity during play.
04-mini-high
SEQCV
indicator non-functional. Additionally, pawns, knights, and bishops remain static, failing to move at mee-second intervals as specified. While the apple collection mechanics work correctly, growing the snake and adding to the scor, self-collision does not lead to a game over, which deviates from the core rules of snake gameplay.
Atom Atom handles apple and piece scoring accurately, awarding points appropriately for apples and captured pieces. However, it lacks two fundamental features: edge wrapping and knight projec- tiles. When the snake crosses the game board's boundary, it disappears instead wrapping around to the opposite side. Although knights spawn and move correctly, they never shoot bullets, omitting an important gameplay hazard.
Flow The Flow variant struggles significantly with game pacing and bullet behavior. The snake's movement speed is excessively high, making the game difficult to control. Furthermore, bullets emerge from off- off-screen positions and instantly end the game upon impact, rather than subtracting single segment of the snake's length. Like AFLow, Flow fails to implement the per-second movement of chess pieces and does not spawn any additional pieces beyond the initial set.
04-mini-high This implementation meets the full range of required mechanics. It correctly handles snake growth and scoring upon apple consumption and awards appropriate points for capturing chess pieces without increasing the snake's length. Chess pieces move one-second intervals and spawn at regular timed intervals. The snake wraps around board edges, self-collision results in game over, and knights shoot bullets that reduce the snake's length by one segment. Bullets also disappear upon reaching the edge of the board. The UI includes both a score display and an accurate "next piece in Xs" timer.
SEQCV This version adheres to specified mechanics except for missing visual grid on the game board. Despite the absent background layout, the game successfully incorporates timed chess piece spawning, per-second piece movement, accurate wraparound behavior, and self-collision detection. Apples cause snake growth and score increase, and chess pieces captured correctly without length gain. Knight bullets correctly subtract one segment from the snake and vanish when they reach the screen edge.
Overall AFLow and Flow both fail implement timed chess piece behavior and do not handle self-collision correctly. Flow is further hindered by erratic game speed and flawed bullet mechanics. Atom handles scoring and piece motion but omits crucial features like edge wrapping and knight attacks. In contrast, 04-mini-high and SEQCV successssllllll deliver the intended Snake-Chess hybrid gameplay. The only deviation in SEoCVis the lack of grid background, while o4-mini-high satisfies every requirement, making these two implementations the most faithful to the design prompt.
27

---

B.6
Tool Development TTas Manager RPG
Task 6: Task Manager RPG
Build a desktop "RPG" where user's 's real-world todo items become in-game quests.
Features:
User adds a task + appears as a "quest" on the map.
2. Completing the task (marking done) triggers battle simulation against a monster.
3. Success grants XP and loot; failure reduces HP.
Shop uses gold buy potions that restore HP.
5. Audios are not needed
No extra image files
Output Format: HTML+CSS+IS
RPG ToDo Quest
Your Quests
Dus
Shop
(4) AFlow
(b) Atom
(c) Flow
Nechb
04-mini-high
SEOCV
Figure 12: Illustration Task Manager RPG.
Table 9: Summary of major weaknesses in task manager RPG implementations.
Method
Major Weakness
AFlow
Task input is non-tunctional, tasks appear after submission, and
RPG features are triggered, likely due to missing event bindings or state logic.
Atom
Shares the same failure as AFlow: the interface accepts input but fails to reflect changes create quests, indicating broken UT-state integration.
Flow
Allows task submission and rendering. but the "Complete" button has effect which battle logic and stat progression are entirely unimplemented. No major functional weakness; all core features are present. However, styling is minimal, and the UI lacks polish compared to alternatives.
04-mini-high
SEQCV
No major functional weakness; all gameplay mechanics operate correctly. Minor improvements could be made in dynamic cost modeling and transport detail.
Fllow
The AFlow implementation fails at the most basic level: when task is entered and the "Add Task" button is clicked, no quest appears the map. There is no visual console feedback.
28

---

and the underlying application state remains unchanged. This indicates that essential wiring, such as imput bindings, a task-tracking tracking array, or event listeners is likely missing. As a result, the key functionality of mapping tasks to quests and triggering RPG mechanics upon completion is completely unimplemented.
Atom Atom's 's behavior closely mirrors that of AFlow. Although the UI displays imput field and a button, clicking "Add Task" does not result in any visual change or state update. No quests are rendered, and none of the game mechanics proceed. Like AFlow, this suggests disconnect between user input and the internal quest-tracking logic which possibly due to incomplete event handling or missing state integration.
The Flow version succeeds in registering new tasks: clicking "Add Task" correctly appends the quest to the UI list. However, the quest's "Complete" button is fonncinntio No battle modal is triggered, and RPG mechanics such as XP gain, HP loss, or gold rewards do not respond to quest completion. The core gameplay loop thus partially implemented, but the combat simulation and stat updates are missing, likely due to a lack of handlers modal logic associated with quest state changes.
Flow
04-mini-high This version delivers complete and fully functional implementation of all six specified features. Tasks are correctly transformed into quests on the map; completing a task launches battle interface; outcomes appropriately modify XP, HP, and gold balances; and shop enables gold to be exchanged for potions. While the user interface is visually basic, it is coherent and operational. Every critical user interaction performs as intended.
SEQCV SEoCVmatches the functionality of o4-mini --iiii-high but adds refinements. It correctly maps user tasks to quests, triggers polished battle dialogs upon completion, and manages game state updates across XP, HP, and gold. The shop is responsive and well-integrated. Visually, the interface is more polished and consistent than 04-mini-high, with smoother transitions and more intuitive dialogs.
Overall Among the five methods evaluated, only o4-mini mini-highh and SEoCCssccessfulll implement the full RPG task manager experience. Flow partially satisfies the requirements by registering tasks quests but fails connect this the rest of the gameplay loop. Both AFlow and Atom break down at the input stage, unable to transform tasks into game objects due to likely omissions event binding state logic.
In terms of prompt adherence and effectiveness:
Feature completeness: SEQCV~ o4-mini-high > Flow > Atom 2 AFlow.
UI/U quality: SEQCV> o4-mini- high Flow Atom AFlow (not assessable).
The two fully functional solutions indicate proper use event-driven design and state management, whereas the others likely suffer from poor component integration or uninitialized logic.
B.7 Game Development- - Tetris Fuses Bejeweled
Task 7: Tetris-fuses Bejeweled
Develop game that fuses Tetris and Bejeweled mechanics. This game needs to add keyboard control function. Falling tetrominoes should lock into grid and transform into colored gems. The game must support both Tetris line-clearing and Bejeweled match-3 clearing, triggering chain reactions and bonus points. Include GUI (using framework like Pygame) that displays the game grid, current score, and next tetromino preview, along with smooth animations. No sound effects are needed.
Output Format: Python
A Fllow The Fllow implementation correctly spawns and locks tetrominoes into the grid, but omits any "game over" logic: new pieces continue appearing and stacking beyond the top boundary,
29

---

Table 10: Summary of major weaknesses in game implementation.
Method
Major Weakness
AFlow
Omits "game over" logic and never triggers Bejeweled-style match-
3 eliminations which pieces stack beyond the top and identical gems remain static.
Atom
Fails to launch entirely (crashes hangs on startup), preventing any assessment of gameplay mechanics or GUL.
FLow
Overly permissive elimination logic clears non-contiguous or unintended
clusters, producing unpredictable chain reactions.
04-mini-high
Gameplay and animations are smooth, but the score display is truncated or inconsistently updated during combos, undermining point tracking. Core mechanics work correctly.
SEQCV
violating the core Tetris constraint. Moreover, the Bejeweled match-3 mechanic never activates, adjacent gems of the same color remain static, so no eliminations or chain reactions occur.
Atom Atom's build fails launch: on startup the application either crashes or hangs indefinitely, preventing any gameplay. As result, none of the required features that tetromino locking, line clearing, gem transformation, GUI display, or keyboard controls, can evaluated.
Flow FLow implements both line-clearing and match-3 behaviors, but its elimination logic is overly permissive: it sometimes clears non-contiguous or unintended clusters, wiping out gems that aren't directly matched. This overreach causes unpredictable chain reactions and undermines player strategy.
04-mini-high Among the five variants, o4-mini-high delivers the smoothest play experience: falling, locking, line clears, and match-3 mechanics all function correctly, and keyboard inputs are responsive, with fluid animations. However, its score display truncated inconsistently updated during combos, preventing players from accurately tracking points and bonuses.
SEQCV SEQCVsuccessfully integrates both Tetris and Bejeweled rules, with correct grid locking and match-3 elimination. Visually it matches the other implementations.
Overall None the five implementations fully satisfies the original specification. Atom doesn't run, offering zero utility. AFlow lacks both game-over enforcement and gem clearance. FLow's match detection overactive, compromising game integrity. 04-mini-high and SFOCV solid mechanics and visuals yet misses robust scoring system.
30

---

B.8
Family Travel Plan
Task 8: Travel Plan
I need attend the NeurIPS 2025 conference.
Dates: Tuesday, December 2nd through Sunday, December 7th, 2025. Location: San Diego Convention Center, California, United States.
I want to bring my husband and two children with me. We will be departing from Beijing. Please help me plan:
1. Flights: Round-trip options from Beijing San Diego, considering family-friendly airlines and layovers.
2. Hotel:
Family-friente accommodations near the convention center, preferably with kitchenette adjoining rooms.
3. Daily trip plan: Sightseeing and activities suitable for children, balanced with my conference schedule.
4. Cost estimate: Total budget breakdown for flights, hotel, meals, local transportation, and activities.
Output Format: LaTeX itinerary plan
(a) AFlow
(b) Alom
(c) Flow
(d) o4-mini-high
(e) SEOCV
Figure 13: Illustration of Travel Plan.
Table Summary major weaknesses in itinerary methods
Method
Major Weakness
AFlow
Lacks specific flight schedules, concrete daily activity times, and detailed budget breakdowns.
Atom
Omits key travel details and lacks transparency in airfare and meal cost estimates.
Flow
Inflates the total budget without justification.
04-mini-high
Includes inconsistent pricing and misalign with the conference schedule, Minor issues like return flight date mismatch and oversimplified meal budgeting; lacks local transport comparison.
SEQCV
This itinerary correctly identifies the conference dates (December 2-7, 2025) and suggests family-friendly airlines, hotels with kitchenettes or adjoining rooms, a structured day-by-day activity plan, and an overall cost estimate. However, it lacks concrete details such as exact flight schedules and specific departure/return times from Beijing. The cost estimates are presented broad ranges rather than broken down by adult and child, making budgeting less actionable. Daily activities
AFlow
31

---

described generically, for instance, "family dinner" appears without reference to timing reservation logistics. Moreover, the itinerary omits supplementary but important considerations such as visa information, travel insurance, or packing recommendations.
Atom The "Atom" plan efficiently aligns the flights, hotel, daily itinerary, and budget components, and provides total figures for each category. Nevertheless, it omits key travel details, including layover cities, exact flight durations, and a transparent rationale for the total airfare of $1,900 for four passengers which an amount that appears inconsistent with typical market rates. The hotel section is limited single option without mentioning alternatives amenities such family dining options. The daily itinerary does not specify conference session times or consider the ages of the children when recommending activities. Additionally, some estimates, such as $720 allocated for meals over six days, lack justification and may appear inflated.
Flow This itinerary is branded with the NeurIPS name and includes three detailed flight options, each with carrier codes, schedules, and differentiated fares for adults and children. also provides a well-organized day-by-day schedule and proposed seven-night hotel stay. However, it incorrectly shifts the travel dates to December 1-8 rather than the specified December 2-7, and suggests a seven- seven-night stay instead of the expected five. The total budget of $11,790 is notably excessive and not well justified, raising questions about its accuracy.
04-mini-high This version includes concrete flight legs, pricing details, and two hotel recommenda- tions with citations. Despite its thorough structure, it introduces significant errors. The itinerary title mistakenly lists "December 27, 2025," which is unrelated to the actual conference. The flight dates span December 1-8, conflicting with the user's request. Currency conversions are inconsistently applied, and hotel rates are uniformly set at $300 per night without reflecting differences in suite types amenities like kitchenettes. The activity plan compresses the conference into five days but fails to align with actual session counts and lacks proper integration of thild-friendly scheduling.
SEQCV This itinerary is the most comprehensive and well-aligned with the prompt. It correctly spans December 2-7, provides three well-diffeeentiatte flight options with complete schedules and per-passenger pricing, and offers three hotel choices near the venue, each with detailed features and family suitability. The daily schedule is rich in content and time-specific, and the cost table is clear and supplemented with general travel tips. Minor shortcomings include return flight scheduled for December 8 rather than December 7, and a flat meal budget of $200 per day that does not distinguish between adult and child needs. Additionally, it lacks a comparison of local transportation options, such as the relative costs of public transit versus rideshares.
Overall All five methods address the core requirements of flight planning, lodging, daily itinerary, and budgeting, but their levels of detail and accuracy vary significantly. AFlow provides sensible range of accommodations and correctly aligns with the conference dates but lacks precision in its flight information and day planning. Atom is concise and organized but falls short in logistical specificity and cost justification. Flow stands out for its exhaustive airline data and scheduling, yet mislabels the conference, adjusts the dates incorrectly, and overestimates the budget without explanation. o4-mini- high demonstrates strong structure and citation use but misaligns dates, misprices accommodations, and misrepresents the event timeline. In contrast, SEQC Moott closely adheres to the prompt, presenting a balanced, richly detailed itinerary with thoughtful accommodations and scheduling. While there is room for minor improvement, particularly in return date aligament and cost granularity, it remains the strongest overall solution.
C Evaluation on Other Benchmarks
Datasets We also evaluate SEQCV on commonly used reasoning benchmarks. Note that these benchmarks are not primarily designed to assess agentic behavior. We include these benchmarks only for completeness. For example, the GSM8K dataset states: "These problems take berween 2 and 8 steps to solve, and solutions primarily involve performing sequence of elementary calculations using basic arithmetic operations." " Such problems can often be solved effectively by a single high- performance model with strong thain-of-thought reasoning capabilities, leaving limited room for
32

---

multi-agent advantages. Moreover, we observed that some samples in these datasets contain incorrect ground-trute answers.
The evaluation covers four categories of reasoning tasks: Mathematical reasoning: MATH 58 (617 level-5 problems from geometry, statistics, number theory, algebra, and calculus) and GSM8K [59]: Knowledge-intent intensivee reasoning: MMLU-CF [60]; Logical reasoning: BBH [61]; Multi-hop reason- ing: HotpoiQA (62) and LongBench [63] (samples from MuSiQue [64] and 2WikiMuliHHoQAA [65]
for long-context reasoning). For each benchmark dataset, we randomly sample 300 examples for evaluation. We follow the experimental setup of recent state-of-the ahr- studies
4
Baselines We compare SEQCV with two groups of baselines: (1) Classical prompting methods:
Chain-of-Thought (CoT), CoT with Self-Consistency (n 5) (27]. MultiPersona [47]. Self-Refine
(up to iterations) [54] Analogical Reasoning [66]. and MedPrompt (3 answers, 5 votes) [7]; (2)
Advanced reasoning frameworks: AFlow 339 ADAS 68), Forest of Thought (FoT) [69], Atom
(AoT) 41. and Flow [8) For FoT, we implement the Tree-of- Tree-of-Thoughts variant (n = 3) for general applicability. All results are averaged over three runs.
We use gpt-4o-mini 70 as the main backbone model for all baselines. For SEQCV, we evaluate configurations: same: three gpt-4o-mini models; different: gpt-4o-mini, gpt-4. --mini, and gpt-4. 10044.1100 1-nano (Table [2]. Following AoT, we provide each model with different types of prompts to encourage diverse reasoning processes and avoid cache effects. We use gpt-4. i-nano 771 to automatically verify whether the model predictions match the ground truth and to compute accuracy. The results are presented in the following table.
Table 12: Performance comparison across tasks (%). We compare SEQCV with classical prompting methods and advanced reasoning frameworks. Results are reported exact match accuracy for MATH, GSM8K, BBH, and MMLU-CF, and as FI scores for HotpotQA and LongBench.
Method
MATH
GSMBK
BBH
MMLU-CF
HotpoiQA
LongBench
Avg.
CoT
78.3
90.9
78.3
69.6
67.2
57.6
73.7
CoT-SC (n=5)
81.8
92.0
83.4
71.1
66.2
58.6
75.5
Self-Refine
78.7
91.7
80.0
69.7
68.3
58.2
74.4
MultiPersona
50.8
92.5
81.1
70.3
69.8
59.3
75.4
MedPrompt
50.0
93.2
82.0
70.8
70.5
60.1
76.1
Analogical Reasoning
65.4
87.2
72.5
65.8
64.7
$2.9
68.1
ADAS
76.0
90.8
75.3
68.9
64.5
56.2
71.9
AFlow
76.0
76.1
FoT (n==)
82.5
94.0*
82.4
70.6
59.1
75.9
SEQCV (same)
80.5
92.0
70.8
76.5*
66.8
77.2
SRQCC(different)
86.5
94.3
87.3
71.5
62.0
79.9
D
System Design
The system orchestrates multiple agents through an iterative three-phase cycle. First, agents execute tasks in parallel with shared memory access. Second, cross-agent validation using majority voting filters low-quality outputs. If the validation success rate falls below a threshold, the task is decom- posed into smaller subtasks that are executed recursively, with results accumulated across subtasks. Otherwise, consensus voting selects the best result for archival. Shared memory is updated after each iteration, enabling progressive refinement until convergence or the maximum number of iterations reached.
33

---

Tel Deyt
Agen Shewe
Prome Coop Val placies
Qainggr
JOAY aiplls
Che culputa
Asarie and Tasut
Figure 14: Multi-agent collaborative architecture.
nating
Tik
Figure 15: Three-phase execution cycle.
34

---

E
Prompts for SEQCV
Prompt 1: Segment Generation Prompt
You Incremental Task Developer whose sole job is continue the current task in sequential segments.
Policy
First iteration (no prior output): generate the initial part of the result for the task.
Subsequent iterations (prior output exists): generate the next part of the result by either choosing override previously generated segments seamlessly continue from where the last segment ended.
Each neu_content section should around 300 tokens.
If any changes to previously generated content are required, use override.
Ensure the result logically and syntactically coherent, needing manual editing.
Aim for quality and depth in each step; do not rush to completion.
A maximum of 10 rounds is allowed.
You may improve the justify section in future rounds.
No omissions are permitted the generated result.
Output Format
Return a single JSON object with exactly four keys:
1. justify (string): Explain what was done in this step, decisions made, and what remains. This guides future development.
2. new_content (string):
Generate the next logical segment of the overall result.
Must integrate smoothly with previous output and contain no more than 300 tokens.
3. status (string):
"cmmletee" This output, when combined with any previous content, forms full and correct implementation that meets the task requirements.
" "ongoing": The implementation is still in progress and needs additional iterations. Note: For code tasks, do not generate the main function or full framework yet if the status "ongoing".
4. mode:
"override" or "continue"
" "ontinue": Appends this segment the previous output. "override": Replaces all previously generated content with this result.
Sample response format:
{
"justify": ""
"new thew_content" "" "status": ""
"mode":
35

---

Prompt 2: Task Decompose Prompt
You are a task decomposer. Your job to split a task into a set of strictly simpler interde- pendent subtasks, forming Directed Acyclic Graph (DAG).
Guidelines for DAG-Based Decomposition
Each subtask should be indivisible, focused, and simpler than the original task.
Define subtasks such that they can be executed only after their dependencies completed. The overall structure must form a valid DAG (no cycles).
Do not include any meta-po meta-planning subtasks (e.g.. "Plan steps". "Outline", "Define framework"). The executor will handle all planning.
All subtasks must adhere strictly to the output format defined in the original objective. No extra headers, annotations, explanations are allowed.
If subtasks produce partial outputs to be combined, their results should be compati- ble and composable in a way that aligns with the final task format.
The system cannot create any files.
Output Format
Return a single JSON object with exactly one key:
1. subtasks (list): a list of all subtasks, each with its unique ID, objective, and a list of dependency IDs it depends on.
"subtasks": [
"id": 0, "objective"; "...", "depends_on"s on": []
"id": 1, "ob "objective": "...". "depends_one on": [
"id": 2,
"objective". "...", "depends_on" on": []
"id": 3,
"objective": "...", "depends_on" _n":: 0
36

---

Prompt 3: Cross-Model Validation Prompt
You are an intermediate-step validator. Your task to assess whether the accumulated previous output and the current output fragment correctly align with the overall task and are error-free enough to continue generation.
Inputs:
1. overall objective
2. current task description
3. current output fragment
Validation Criteria
Does the current fragment logically continue the task as described?
Is it free from critical errors, omissions, or contradictions?
Is it sufficient to serve as the basis for the next iteration?
Output Instructions
Return a single JSON object with exactly two keys:
justify: justify the error found
2. result (str): output "true" "false" only
Prompt 4: Cross-Moder Voting Prompt
You are validator. Your task is to rank all of the accumulated previous outputs and determine which answer is correct.
Inputs:
1. Overall objective
2. Different versions of answers
Output Format
Return a single JSON object with exactly two keys:
1. justify: clearly reason in logic why you think the answer is better
2. votes (list): list of top-2 version indices (e.g. OD, ordered from the best the worst. Use 0, 1, 2, ... to represent each version.
Sample response format:
"justify": "", "votes": []
}
NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The main claims made in the abstract and introduction accurately reflect the paper's contributions and scope.
37

---

2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification: The limitations have been discussed in the main content.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
Justification: The paper does not include theoretical results.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided not)?
Question:
Answer: [Yes]
Justification: The reproducibility is ensured by detailed instructions and the release of code.
5. Open access to data and code
Question: Does the paper provide open the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: This paper releases anonymized data and code.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: Experimental setup and detailed settings are presented in the main context and Appendix as well code.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [No]
Justification: The paper does not report error bars.
8. Experiments compute resources
Question: For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Answer: [No]
Justification: The paper does not require specific CPU or GPU resources.
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics http:p/://.oorpssccco/////ccctta cc/ppublic/Ethhicccudddlla
Answer: [Yes]
Justification: The research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative socictal impacts of the work performed?
38

---

The details of the new assets have been discussed. It does not another Answer: [NA]
Justification: The presented technique is intended for general use and, at this early stage, has explicit connection any societal impact.
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: The paper poses no such risks.
12. Licenses for existing assets
Question: Are the creators original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: All the original papers and dataset used for producing experimental results have been cited appropriately.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification:
asset.
Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: The paper does not involve crowdsourcing nor research with human subjects. 15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/teview based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: The paper does not involve crowdsoureing nor research with human subjects.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology, scientific rigorousness, or originality the research, declaration not required.
Answer: [Yes]
Justification: The LLM is used only for writing, editing, formatting purposes. The method development in this research does not involve LLMs as any important, original, or standaadd components.
39


Reviews:
[{"review_id": "iTeZ88CQyz", "reviewer": "Reviewer_C5z9", "summary": "The paper considers failures in multi-agent systems caused by inter-agent misalignments. When complex tasks are decomposed into a graph of subtasks, the system often assumes conditional independence between them. However, this assumption is frequently violated. Since LLMs produce approximate outputs without access to ground truth, hidden dependencies can emerge, leading to cascading errors. To address this, the authors propose a framework called SEQCV. It executes subtasks sequentially, conditioning each new step on the full history of previously verified outputs. SEQCV also introduces a sequential generation and cross-validation mechanism. This allows the system to selectively split complex subtasks into smaller ones. The authors demonstrate that SEQCV improves accuracy across several benchmarks. It also reduces overall execution time by catching errors early, thus avoiding expensive full-context corrections later.", "questions": "1) What are the latency measurements for the accuracies reported in Table 1? It would be useful to provide an accuracy-latency trade-off curve. Such a curve could illustrate performance across the different system configurations you tested.\n\n2) How was cross-validation performed? Specifically, how does this methodology apply to a system where agents have distinct and interdependent roles, such as the CEO, Software Engineer, and Code Reviewer in ChatDev?\n\n3) The paper uses a \"consensus of LLMs\" as a proxy for ground truth. What is the justification for this assumption? In particular, \n- How did you account for shared systemic biases given the LLMs were likely trained on similar data? \n- There is previous work showing that majority voting does not always scale [3]. In this light, can you better justify this method?\n- Were the LLM consensus results ever grounded? For example, were they validated against a benchmark created by human experts? Please specify.\n\n4) The fact that LLM outputs are approximations and that errors can cascade in a sequential process is a well-understood challenge in ML-based decision processes [1,2]. Could you clarify wether and how your paper provides a new perspective when it comes to framing the problem beyond previously identified challenges, e.g., (a) LLMs are unreliable and (b) workflows need robust error handling?\n\n[1] Sun, Chuanneng, Songjun Huang, and Dario Pompili. \"LLM-Based Multi-Agent Decision-Making: Challenges and Future Directions.\" IEEE Robotics and Automation Letters (2025).\n[2] Dai, Peng, and Daniel Weld. \"Artificial intelligence for artificial artificial intelligence.\" Proceedings of the AAAI Conference on Artificial Intelligence. Vol. 25. No. 1. 2011. \n[3] Chen, Lingjiao, Jared Quincy Davis, Boris Hanin, Peter Bailis, Ion Stoica, Matei Zaharia, and James Zou. \"Are more llm calls all you need? towards scaling laws of compound inference systems.\" arXiv preprint arXiv:2403.02419 (2024).", "limitations": "Yes.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 2, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n1) The efficiency of multi-agent LLM systems is an important and understudied research area. This paper provides an insightful study on this topic.\n2) The paper proposes a new framework, SEQCV, to improve the reliability and efficiency of multi-agent LLM systems.\n3) The paper validates SEQCV on six standard reasoning benchmarks (including MATH and HotpotQA) and show the proposed solution performs well. In addition, the paper includes a challenging, domain-specific (game development) task. Finally, I found the results provided in the Appendix being insightful as well.\n\nWeaknesses:\n1) Questionable claims and assumptions:\n- The claim that existing literature assumes subtasks in a DAG decomposition are conditionally independent is quite strong. You seem to draw inspiration from reinforcement learning and optimal control theory by suggesting that LLM-based workflows resemble partially observable Markov decision processes (POMDPs) rather than Markov decision processes (MDPs). However, this is not a novel insight. It is already widely accepted that LLM (or any neural model) outputs are approximations, not ground truths. Therefore, ML-based workflows are inherently non-Markovian.\n- You should clearly justify the differences in core assumptions. In particular, many prior works limit inter-agent input sharing not due to a belief in conditional independence, but for context management reasons. Conditioning on full dialogue history can indeed resolve non-Markovian issues, but it is memory-intensive.\n\n2) Generality: The main contribution of this paper is a practical method for managing context. By conditioning on previously verified outputs, the method helps mitigate non-Markovian effects in LLM workflows. This is a useful idea. However, it is unclear how the approach would generalize to more autonomous agentic systems. In such settings, agents choose their communication partners, and the interaction graph may not be a DAG, e.g., include cycles to implement things like self-refinement.\n\n3) Lack of ablation studies: SEQCV introduces several interesting components: Global-Context Construction, Subtask-Context Generation, Cross-Model Validation, and Dynamic Recursive Splitting. However, the paper lacks ablation studies. Without them, it is hard to assess the individual contribution of each module. This limits the overall evaluation and interpretability of the system.\n\n4) Clarity: You should introduce and define all symbols before using them in Section 2. Otherwise, the formalism is difficult to follow.", "was_revised": false, "final_justification": "", "rebuttal": "### Dear Reviewer C5z9,\n\n### We sincerely thank you for your positive support and thoughtful feedback. Your insightful comments, especially on reinforcement learning and ablation studies are very helpful in guiding our revisions to make the manuscript more accessible and relevant to a broader audience. Below, we have summarized your comments and suggestions, along with our detailed responses.\n\n**Q1. Ablation Study**\n\nThank you for much for this important questions and efforts on helping us improve our paper. Due to limited space, please kindly refer to Q1 of Reviewer asVu for more details.\n\n**Q2. This paper draws inspiration from an RL concept, i.e., POMDPs**\n\nThank you very much for the insightful question! \nWe formulate the data-generating process using probabilistic graphical models, which allow to directly read off the conditional independencies. POMDPs are a specific instance of such models, which is why our insights can be naturally interpreted through POMDPs. In our revision, we have also provided analysis from the POMDP perspective in the revised paper, e.g., treating the ground truth answer as a latent state.\n\n**Q3. Clarify how your paper provides a new perspective?**\n\nThank you so much for the opportunity to clarify our novelty. We realized the need to articulate our contributions and perspectives more explicitly. In our revised version, we have carefully emphasized the following key points. Feel free to highlight further improvements needed. We truly appreciate your time and effort.\n\n1. **Theoretical Formulation and Identification of Inter-Agent Misalignment**\nBuilding on the analysis on underlying causes of *inter-agent misalignment*, we provide theoretical formulation using probabilistic graphical models to show that inter-agent misalignment arises from violations of conditional independence and can occur across three task execution patterns:\n* $T_2 \\rightarrow T_1 \\rightarrow T_3$\n* $T_2 \\leftarrow T_1 \\rightarrow T_3$\n* $T_2 \\leftarrow T_1 \\leftarrow T_3$\n\nThis theoretical foundation was missing in prior work and offers a new lens for understanding agent system failures.\n\n2. **Key Insight and Real-World Impact**\n* We emphasize that conditioning each subtask on the full history of verified outputs is crucial for maintaining inter-agent alignment. \n* It is also necessary to sacrifice parallelism in favor of sequential execution to ensure reliability. \n* This insight has immediate real-world relevanceas DAG-based decompositions are already employed in industrial systems such as AWS’s KIRO IDE, which translates user coding requirements into executable DAGs. In such bug-sensitive domains, reliability must take priority.\n\n3. **Strong Empirical Results on Novel Tasks**\nBeyond benchmarks, we evaluate our method in creative tasks where models must combine rules from different tasks (e.g., Pac-Man + Tank City, Snake + RPG elements). These hybrid tasks cannot be solved by memorization, as they are not in the LLM's training data. On these tasks, our method achieves substantial improvements.\n\n4. **Efficiency Gains**\nDespite using sequential execution, our approach achieves a **2.5× speedup over Flow** (665s → 217s) and a **2.1× speedup over Atom** (523s → 217s), while maintaining higher quality outputs. \n5. **Additional Contributions with Potential to Inspire Future Work**\n* Leveraging differences across multiple models for agreement-based validation could be a promising direction for future research.\n* We propose that agent workflows should be split only when the task is difficult (determined via model disagreement).\n* Our segment-based validation also enables early stopping, which helps reduce unnecessary token consumption.\n\n**Q4. Prior works limit inter-agent input sharing not due to a belief in conditional independence, but for context management reasons**\n\nThank you so much for the insightful question.\n\nWe believe that prior works may not have noticed the issue that conditional independence cannot be trusted. Even they had explicitly recognized this, their system design would differ, such as only partitioning context when approaching the model's input limit. However, in their experiments, despite the context length for all tasks being far away from reaching the model's limitation, they still partition the context, which suggests an implicit and possibly unnecessary precaution.\n\n**Q5. Conditioning on full dialogue history is memory-intensive**\n\nThank you for the valuable comment. We agree that relying on full dialogue history introduces significant memory overhead. Existing methods often depend on an additional summary agent to integrate subtask outputs by consuming and aggregating the entire dialogue history (e.g., summary.py in the FLOW GitHub repository). However, such design is **not efficient** as it not only increases memory consumption but also compromises the benefits of parallelism. By introducing a mandatory sequential post-processing step, it reduces the system to a parallel stage followed by a sequential bottleneck.\n\nIn contrast, our method does not require a summary agent. Instead, we perform a single, lightweight sequential pass to integrate results, which maintains efficiency and preserves the scalability advantages of parallel agent execution.\n\nWe strongly agree that minimizing reliance on full dialogue history is an important and open challenge. We are actively investigating this direction, aiming to discover patterns of dependence and conditional independence in natural language.\n\n**Q6. Latency measurements**\nWe use runtime as the metric for measuring latency, as reported in Table 10 in Appendix B (page 26). Although our method follows a sequential design, it benefits from removing the summary agent (as discussed in A4), which allows it to outperform existing methods in generation speed. Furthermore, while AFlow shows comparable speed, this is not due to the system's efficiency but rather because it generates a significantly smaller number of tokens than us.\n\n**Q7. Can the method be extended to a system having agent roles?**\nThank you for the question. Currently, there are two major streams in agentic systems that can work in a complementary manner:\n\n1. **Coordination-based streams**, where agents select communication partners and assign tasks (e.g., selecting a Software Engineer to perform a specific subtask).\n\n2. **Content-generation-based streams**, where a group of agents collaboratively generate content to complete a task (e.g., a team of Software Engineers working together to develop a solution).\n\nOur work focuses on the second stream, which can be seamlessly embedded into coordination-based systems to support more complex inter-agent interactions. This stream already has impactful real-world applications. For instance, AWS recently introduced the KIRO IDE, which decomposes a coding task into a DAG of subtasks.\n\nMore importantly, we believe that timely highlighting the need to carefully handle the conditional independence assumption can be crucial for the future design of agentic systems, especially in high-stakes applications like coding, where safety and correctness are paramount.\n\n\n**Q8. How does the approach include cycles to implement things like self-refinement?**\n\nWe support task refinement through recursive task splitting: when different LLMs disagree on a task's output, we rerun it by decomposing it into smaller sub-tasks.\n\nOur method also allows to incorporate other refinements. For example, similar to Flow, we can add a task refinement module outside the DAG to support iterative improvements. However, it's worth noting that such refinement is computationally expensive. Flow with refinement requires roughly 5× more runtime than without refinement.\n\n**Q9. Justify assumption of using LLMs' consensus as ground truth**\nThis assumption is borrowed from conventional ensemble-based methods, which have been shown to improve performance. We assume that when multiple LLMs agree, the answer is likely to be correct.\n\n\n**Q10. There is previous work showing that majority voting does not always scale [3]. In this light, can you better justify this method?**\n\nOur method align with the claim that majority voting does not scale linearly with the number of LLMs. This insight directly informed our design: we do not use as many LLMs as possible. Instead, we use a small ensemble of three competitive LLMs (gpt-4o-mini, gpt-4.1-mini, and gpt-4.1-nano) that exhibit sufficient diversity while maintaining comparable performance. This setup balances diversity and efficiency while minimizing noise from weaker or redundant models.\n\n\n**Q11. How to reduce systemic biases if LLMs are trained on similar data?**\n\nThere is no silver bullet for this issue, not just for LLM-based agents, but even for human collaboration. We acknowledge this as an important direction for future work.\n\nAt present, we should encourage the use of models with similar performance that are trained on different datasets. Additionally, designing prompts that encourage models to produce answers via different reasoning processes may also help mitigate shared biases.\n\n**Q12. Were the LLM consensus results ever grounded?**\n\nTo the best of our knowledge, there is no annotated dataset specifically designed to evaluate the quality of LLM consensus, making direct quantification difficult.\n\nWe conducted ablation studies by removing the consensus component, which resulted in worse performance, demonstrating its effectiveness.\n\n**Q13. Notation table**\n\nWe have followed your comments and have added a notation table at the start of our method section."}, {"review_id": "aOgrF20eIv", "reviewer": "Reviewer_asVu", "summary": "The paper shows that graph‑based multi‑LLM workflows break down because subtasks’ outputs are not conditionally independent; this causes style, reasoning, or objective misalignment among agents. The authors propose SEQCV, which executes subtasks sequentially, conditioning each on all verified history. In add ition, it performs segment‑level cross‑model voting to accept or discard partial outputs, and recursively splits a failing subtask into simpler ones. Across six benchmarks, SEQCV improves accuracy by 3‑4pp and cuts runtime on complex tasks despite the sequential flow.", "questions": "1. How much do sequential execution, cross‑validation, and recursive splitting each contribute?\n2. Can independent branches of a DAG run in parallel without re‑introducing misalignment?\n3. What exact criterion defines “agreement,” and how do you guard against unanimous but wrong answers?", "limitations": "SEQCV requires an acyclic task graph and may struggle with feedback loops .", "rating": 4, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "strengths:\n1. Solid empirical gains; diverse tasks; transparent methodology and clear figures, pseudocode, and motivation.\n2. Addresses a real reliability gap in agent systems; technique is broadly applicable.\n3. Fine‑grained cross‑model verification 、 dynamic splitting is novel.\n\n\nWeaknesses:\n\n1. Accuracy gains are moderate; sequential flow may bottleneck very wide graphs.\n2. Relies on established ideas (voting, recursion) stitched together.", "was_revised": false, "final_justification": "", "rebuttal": "### Dear Reviewer asVu,\n### Thank you for your postive support and professional comments for helping improve our paper. We follow your comments and make further clarification as follows.\n\n**Q1. Accuracy gains & Ablation Study**\n\nThank you for the insightful comments and this valuable opportunity for us to strengthen our empirical study.\n\nFor benchmarks in Table 1, the average accuracy gain is 3% compared to the best state-of-the-art agent system. We believe that the benchmark tasks do not well reflect the challenges commonly encountered in the wild. For example, the GSM8K task “These problems take between 2 and 8 steps to solve, and solutions primarily involve performing a sequence of elementary calculations using basic arithmetic operations\" can be easily answered. Therefore, applying multi-agent system does not provide a significant advantage here as a single high-performance agent with extensive chain-of-thought reasoning capabilities is enough.\n\nThese observations motivated us to design a suite of more creative and novel tasks (in Appendix A). Inspired by the Grok-2 evaluation approach, we ensure that most designed tasks are not present in the training data of LLM-agents by combining multiple rule sets (e.g., Pac-Man + Tank war), thereby requiring advanced reasoning rather than memorization. \n\n---\n\nWe have followed your comments and conducted additional experiments to show quantified results for creative tasks, where our method achieves substantial improvements.\n\n**Performance Metrics**\nWe evaluate each method on seven challenging agentic tasks using three metrics:\n\n* **Hard Requirements (HR)**: % of task-specific requirements fully satisfied\n* **Execution Success (ES)**: % of runs completing without major bugs\n* **Constraint Adherence (CA)**: % of runs respecting all specified constraints\n\n**Task Specifications:**\n- NeurIPS Website: 4 requirements + 1 constraint (HTML+CSS format)\nonstraints (Python, no sound, no external images)\n- ...... (for saving spaces) \n- Tetris + Bejeweled: 5 requirements + 2 constraints (Python, no sound)     \n- Travel Plan: 4 requirements + 1 constraint (LaTeX format)\n\nTotal Score = (HR + ES + CA) / 3.\n\n**Table 1. Average Performance Across Methods**\n*This table is newly introduced to give overall context.*\n| Method       | HR  | ES   | CA   | **Total** |\n| ------------ | --- | ---- | ---- | --------- |\n| AFlow        | 36% | 30%  | 40%  | **36%**   |\n| Atom         | 26% | 40%  | 30%  | **29%**   |\n| Flow         | 56% | 70%  | 40%  | **58%**   |\n| o4-mini-high | 81% | 100% | 70%  | **82%**   |\n| **SeqCV**    | 83% | 100% | 100% | **88%**   |\n\n**Key Findings**\n\nBaseline agent systems use o4-mini. Our method, SeqCV, uses both o4-mini and o3-mini together and achieves 88% accuracy. In contrast, a high-performance single model (o4-mini-high) scores 82%, which demonstrates that our method using multiple weaker models can outperform a stronger model.\n\n---\n\nWe have also followed your comments and conducted ablation studies with each task run over five trials, and the best results were selected and counted for evaluation.\n\n**Table 2. Ablation Study: Component Contributions**\n| Configuration                                       | Score     | Δ vs. Parallel |\n| --------------------------------------------------- | --------- | -------------- |\n| **Parallel (baseline)**                             | 52.3%     | –              |\n| **+ Sequential Only**                               | 58.4%     | +6.1%         |\n| **+ SeqExec + Cross-Val**                           | 78.6%     | +26.3%         |\n| **+ SeqExec + Cross-Val + RecSplit** *(Full SeqCV)* | **88.0%** | **+35.7%**     |\n\n**Q2. Can independent branches of a DAG run in parallel without re-introducing misalignment?**\n\nIf we can fully trust the output of a branch, i.e., when the conditional independence satisfies, then it can safely run in parallel. For example, suppose we have two independent tasks, T1 and T2, both pointing to a downstream task T3 (i.e., T1 → T3 and T2 → T3). Since T1 and T2 are independent of each other, they can be executed in parallel without causing misalignment, as long as their outputs are properly synchronized before T3 begins. However, this is not a realistic assumption in the wild.\n\nIt may also worth mentioning that our method is **faster** than existing parallel methods:\n- **SeqCV**: 217s (our method)\n- **AFlow**: 161s (generates far fewer contents, see demos links in the appendix)\n- **Flow**: 665s (2.5× slower than ours)\n- **Atom**: 523s (2.1× slower than ours)\n\nThe speed-up comes from\n1. **Eliminates summary bottleneck**: Existing methods require a sequential \"summary agent\" stage after parallel execution (e.g., see summary.py in Flow repository), creating a parallel→sequential workflow\n2. **Segment-level early stopping**: We validate incrementally and can halt immediately when errors are detected\n\n**Q3. What exact criterion defines \"agreement\"?**\n\nFor each result segment generated by an LLM, we prompt multiple models to independently vote **yes** or **no** on the following questions:\n* Does the current segment logically continue the task as described?\n* Is it free from critical errors, omissions, or contradictions?\n* Is it sufficient to serve as the basis for the next iteration?\n\nIf the majority (or unanimous) vote is **yes** for all three questions, we treat the segment as agreed upon and continue generation without splitting.\n\n**Q4. How do you guard against unanimous but wrong answers?**\n\nThanks for this insightful questions. There is no silver bullet for detecting wrong outputs without human intervention.  In fact, even with human intervention, it can be challenging to evaluate correctness for novel or complex questions. This should be a fundamental limitation of current AI systems. This also holds for our system: if multiple competitive LLMs unanimously agree on a flawed output, the system will unfortunately still fail.\n\nTo mitigate this risk, one strategy is to incorporate external verification and feedback mechanisms whenever possible. For example, in code generation tasks, we can write a script to automatically compile the generated code to check for any compilation errors and further modify the generated results based on those errors.\n\n**Q5. Sequential flow may bottleneck very wide graphs**\n\nThank you for this insightful comment. we have acknowledged it to our limitations in the revised paper. It is also worth mentioning that:\n* Our method only fails to provide benefits in a very specific type of wide graph. Specifically, one where each independent branch (1) has no incoming edges from outside the branch (i.e., no parent nodes pointing into it), and (2) has no connections to nodes in other branches. In such a strictly isolated structure, there is little opportunity for our method to reduce inter-agent misalignment, as there is no misalignment.\n* In all other cases, our method offers a trade-off between maximizing reliability and improving context management, especially when branches interact or depend on shared upstream information.\n* It's important to note that \n  * context size is usually not a limitation, as current LLMs (e.g., even GPT-4o Mini) support a 128,000-token context window—equivalent to approximately 96,000 words or around 17,000 lines of code.\n  * Maximizing reliability can be the first priority in many domains. For example, in code generation, AWS recently introduced the KIRO IDE, which decomposes a coding task into a DAG of subtasks. In this bug-sensitive domain, conditioning on the full context and executing sequentially is often the safer choice.\n\n**Q6. Relies on established ideas**\n\nThank you for this important question about novelty. While individual components like voting and recursion exist in prior work, our contributions lie in the **systematic analysis of why existing methods fail** and **how to combine these techniques effectively** to address an important inter-agent misalignment problem in multi-agent systems.\n\n**Our key contributions**:\n\n1. **Systematic formulation of inter-agent misalignment**: We provide the first rigorous analysis using probabilistic graphical models to show why parallel execution violates conditional independence assumptions, leading to misalignment. This theoretical foundation was missing in prior work.\n\n2. **Discovery of hidden dependencies**: We identify that latent dependencies can propagate along paths raised by approximation of output generated by LLMs, which explains a major reason for existing inter-agent misalignment occurs.\n\n\n3. **Real-world impact**: Highlighting the need to carefully handle conditional independence assumptions is crucial for future agentic system design, especially in high-stakes applications where safety and correctness are paramount. As noted in Q5, DAG-based decomposition is already used in industry systems (AWS KIRO IDE). Our insights about conditional independence violations have immediate relevance for such systems.\n\n4. **Novel system design innovations**:\n\n- **Sequential based method**: propose an effective solution that **conditioning on all history** rather than just parent nodes, which eliminates hidden dependencies.\n   \n- **Segment-level cross-validation**: Unlike traditional voting, we validate incrementally at the segment level, enabling early error detection and preventing wasted computational run time cost.\n\n-  **Adaptive recursive splitting**: We split tasks **only when necessary** (when consensus fails), not upfront like existing methods. This adaptive approach also reduces unnecessary computational and run time cost.\n\n-  **Elimination of summary bottlenecks**: Our design removes the need for separate summary agents that create sequential bottlenecks in parallel systems. Existing methods rely on an additional \"summary\" agent to integrate subtask outputs by consuming and aggregating the entire dialogue history (e.g., `summary.py` in the FLOW GitHub repository)."}, {"review_id": "orEphS0UKw", "reviewer": "Reviewer_KXbZ", "summary": "This paper introduces SEQCV, a dynamic framework designed to address the issue of inter-agent misalignment in LLM-agent systems. The authors identify the violation of conditional independence assumptions—where subtask responses are assumed to be reliable and dependent only on parent responses—as the root cause of these misalignments. SEQCV tackles this by executing subtasks sequentially, conditioning each on all prior responses, and immediately verifying them through consistency checks across diverse LLM models. If a response is deemed unreliable, a recursive splitting mechanism breaks down the subtask into smaller components. Experiments demonstrate that SEQCV improves accuracy by up to 17.3% and reduces execution time by more than half on complex tasks compared to existing methods.", "questions": "- How is \"semantic consistency\" precisely defined and measured during the cross-model validation process, especially given that LLMs may generate varied but equally valid responses for certain subtasks?\n\n- What is the practical impact of the \"maximum recursion depth\" in the recursive splitting mechanism? Does reaching this limit imply a failure to solve the subtask, and if so, how does SEQCV handle such scenarios and their impact on the overall task objective?\n\n- Could the paper provide a more detailed breakdown of the latency implications of SEQCV's sequential execution and frequent verification steps across different task complexities, perhaps with a comparison to the latency of re-execution after misalignment in parallel approaches?", "limitations": "Yes.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "## Strengths\n\n- The paper pinpoints the violation of conditional independence assumptions in LLM-agent workflows as a core issue leading to inter-agent misalignment, which significantly impacts quality and runtime efficiency. This highlights a fundamental challenge in current LLM-agent systems.\n\n- SEQCV introduces a dynamic framework that ensures reliable execution despite violated conditional independence assumptions. It achieves this through sequential subtask execution, conditioning on prior responses, consistency checks, and a recursive splitting mechanism for unreliable outputs.\n\n- SEQCV shows substantial gains in both accuracy and efficiency across various tasks and datasets, including mathematical reasoning, knowledge-intensive reasoning, logical reasoning, and multi-hop reasoning. It improves accuracy by up to 17.3% and reduces execution time by more than half on complex tasks.\n\n## Weaknesses\n\n- The paper does not provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments. This hinders reproducibility for other researchers.\n\n- The evaluation relies on an LLM (gpt-4.1-nano) to determine whether predictions match ground truths and calculate accuracy. While this can be efficient, it introduces a potential bias or limitation if the evaluating LLM itself has errors or biases, or if its \"ground truth\" assessment isn't truly objective.\n\n- SEQCV's verification step relies on \"semantically consistent across diverse LLM models\" and \"peer-reviewed by agents of other LLM models\". While the paper mentions using \"gpt-4o-mini, gpt-4.1-mini, and gpt-4.1-nano as the core models\" for evaluation and \"three gpt-4o-mini models\" or \"gpt-4o-mini, o4-mini and o3-mini\" as backbone models, it doesn't explicitly define what constitutes \"diverse\" enough models to ensure robust cross-model validation. The effectiveness of this mechanism heavily depends on the actual diversity and complementary strengths of the chosen LLMs, which isn't thoroughly explored.\n\n- While the paper claims SEQCV \"avoids costly misalignment corrections and delivers higher effective throughput than parallel pipelines\" , the fundamental sequential nature of subtask execution and the consistency checks at each token sequence checkpoint  inherently introduce overhead. Although it aims to avoid costly misalignment corrections, the frequent verification and potential recursive splitting could still lead to higher overall latency for simple tasks where misalignment is less likely. The efficiency gains might primarily be realized on complex tasks prone to significant misalignment errors in parallel setups.", "was_revised": true, "final_justification": "I've confirmed the author's rebuttal, and I will keep my original score.", "rebuttal": "### Dear Reviewer KXbZ,\n\n### Thank you so much for your postive support and insightful comments! We follow your comments and make clarification as follows.\n\n\n**Q1. Information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments**\n\nOur experiments require minimal local computational resources, as they primarily involve calling LLM APIs. Any standard laptop with a stable internet connection is sufficient.\n\nFor reference, the hardware configuration we used is:\n\n- 13th Gen Intel(R) Core(TM) i7-13700H @ 2.40 GHz\n\n- 16.0 GB RAM (15.7 GB usable)\n\n**Q2. The evaluation relies on an LLM to determine whether predictions match ground truths and calculate accuracy**\n\nThanks for this construtive comments.  \n- We have included F1 scores and Hit rate in the appendix E. \n- The LLM-based evaluation is more suitable for reflecting accuracy because LLM-generated answers often contain formatting variations or additional context.\n\nFor example:\n- **Multiple choice problems**: The model might answer \"answer: A\", \"A.\", or \"**A**\" - all correct but requiring semantic understanding\n- **Q&A questions**: Answers can be more complex with equivalent but differently phrased responses\n\nImportantly, we use the LLM evaluator with great care. Each question–answer pair is evaluated independently. Specifically, one questions at a time via API calls to determine whether the answer is correct or not. In this way, context length and potential dependencies across questions do not interfere with the judgment. We then manually aggregate the results to compute accuracy.\n\n**Q3. The effectiveness of this mechanism heavily depends on the actual diversity and complementary strengths of the chosen LLMs, which isn't thoroughly explored**\n\nThank you for raising this important point. You are correct that diversity alone is not sufficient when selecting backbone LLMs. \n\n**Our selection principle**: We believe model choice should be task-specific. For a given task, we aim to select different models that (1) achieve similar accuracy (i.e., no significantly weaker model), and (2) are trained on different datasets to ensure diversity. \n\n**Practical implementation**: Since task-specific accuracy assessment remains an open challenge, in practice, we select models that are publicly accessible via API and demonstrate similar overall performance—for example, gpt-4o-mini, o3-mini, and o4-mini.\n\n\n**Q4. The fundamental sequential nature of subtask execution and the consistency checks at each token sequence checkpoint inherently introduce overhead**\n\nWhile sequential execution does introduce some overhead, our design includes several optimizations that actually make it faster than existing parallel methods:\n\n**Efficiency of consistency checks**: \n- The check takes approximately 2 seconds (also depending on network speed), as we only require models to output a single token (\"yes\" or \"no\"), making the validation process very lightweight. \n- More importantly, all cross-model checks are independent and run in parallel. As a result, checking one answer takes roughly the same amount of time as checking multiple answers.\n\n**System optimizations**: We have implemented several improvements for sequential execution:\n- **Segment-based validation**: Early validation allows immediate error detection and prevents wasted computation\n- **Elimination of summary bottlenecks**: Unlike existing methods that require a sequential \"summary agent\" stage after parallel execution, we directly concatenate results\n- **Adaptive splitting**: We only decompose tasks when necessary (when consensus fails), avoiding unnecessary overhead\n\n**Empirical results**: Despite the sequential nature, our method achieves faster runtime than existing parallel methods shown in Appendix B:\n- **SeqCV**: 217.14s (our method)\n- **AFlow**: 160.86s (generates fewer tokens)\n- **Flow**: 665.00s (2.5× slower than ours)\n- **Atom**: 522.57s (2.1× slower than ours)\n\nThe reason AFlow is faster is simiply because it generates significantly fewer tokens.\n\n**Q5. Could the paper provide a more detailed breakdown of the latency implications of SeqCV's sequential execution and frequent verification steps across different task complexities, perhaps with a comparison to the latency of re-execution after misalignment in parallel approaches?**\n\n- Direct comparison with parallel methods' re-execution latency after misalignment is challenging because existing parallel methods cannot identify when misalignment occurs. \n- Hard to identify when misalignment occurs is precisely our key motivation for designing a sequential approach that directly reduces inner misalignment issues rather than attempting to detect and correct them post-hoc.\n\n\nAs detailed in **Appendix B** (and noted in our response to Q4), our method runs in **217.14 seconds**, which is faster than several parallel baselines.\n\nAs explained earlier, each verification step is very lightweight, requiring approximately **2 seconds**, since it only involves generating a single token (\"yes\" or \"no\") in parallel across models.\n\n* **Runtime breakdown**:\n  In our experiments, we observe that about **24.7 seconds** are spent on verification, each taking1.5 to 2.5 seconds depending on network conditions. The remaining **~192.4 seconds** are used for sequential LLM execution.\n\n  | Component                          | Total Time (s) |\n  | ---------------------------------- | -------------- |\n  | Sequential LLM execution           | \\~192.4        |\n  | Verification | \\~24.7         |\n  | **Total Runtime (SeqCV)**          | **217.1**     |\n\n* It is worth mentioning that the main bottleneck in the verification step is network latency and API response time. The results can vary significantly depending on network conditions.\n\n\n**Q6. Potential recursive splitting could still lead to higher overall latency for simple tasks where misalignment is less likely**\n\nThank you so much for the important comments. For simple tasks, recursive splitting is **rarely triggered**, as our method only performs decomposition when necessary.\n\nWe have followed your comments and summarized the decomposition rates on benchmark datasets. Using our method with gpt-4o-mini, gpt-4.1-mini, and gpt-4.1-nano together, the results are as follows:\n\n* **GSM8K** (math reasoning): 4%\n* **MATH** (complex mathematics): 6%\n* **HotpotQA** (multi-hop reasoning): 6%\n* **LongBench** (long-context reasoning): 7%\n\n\n**Q7. How to check if output sequence is \"semantic consistent\" for cross-model validation process**\n\nWe apologize for the confusion, this was a writing issue. To avoid further misunderstanding, we have removed the term \"semantic consistent.\" The checking is performed through a structured multi-model voting process.\n\n- For each result segment generated by an LLM, we prompt multiple models to independently vote **yes** or **no** on three specific criteria (see Appendix F for the prompt):\n  - **Logical continuation**: Does the current segment logically continue the task as described?\n  - **Error-free quality**: Is it free from critical errors, omissions, or contradictions?\n  - **Sufficiency for progression**: Is it sufficient to serve as the basis for the next iteration?\n\n- **Consensus mechanism**: If the majority (or unanimous) vote is **yes** for all three questions, we treat the segment as semantically consistent and continue generation without splitting. This approach captures semantic coherence beyond simple syntactic matching.\n\n\n**Q8. What is the practical impact of the \"maximum recursion depth\" in the recursive splitting mechanism? Does reaching this limit imply a failure to solve the subtask, and if so, how does SeqCV handle such scenarios and their impact on the overall task objective?**\n\nIn our experiments, we set the maximum recursion depth to 3. This limit serves as a practical safeguard against infinite decomposition while allowing sufficient depth for most complex tasks.\n\n**Empirical frequency**: \n- **Benchmark datasets**: None of the standard benchmark tasks reached the maximum recursion depth\n- **Creative tasks (Appendix A)**: Only 2 out of 35 challenging creative tasks exceeded the limit\n\n**Failure handling**: If the maximum recursion depth is reached without consensus, our system marks the subtask as failed, then uses a single model to execute the task and ensembles the result. This approach prevents system deadlock while maintaining minimal functionality."}, {"review_id": "eYiwA9jmUR", "reviewer": "Reviewer_QP7F", "summary": "The authors delve into the root cause of inter-agent misalignment and argue that it stems from the failure of the “conditional independence” assumption in existing workflow designs. Building on this observation, they introduce SEQCV, a dynamic framework that tackles the problem through sequential execution, cross-verification, and recursive decomposition.", "questions": "na", "limitations": "yes", "rating": 2, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 2, "originality": 2, "strengths_and_weaknesses": "## Strength\n\n1. This paper is generally in a good structure and easy to follow.\n2. This paper demonstrated performance gains on several benchmarks, suggesting its effectiveness.\n\n## Weakness\n\n- Token cost would be high. Every trajectory and answer is verified, which appears to incur excessive token costs and risks exceeding the model’s context window. Even for easy query such as 1+1, the workflow method still needs to decompose and verify, which introduces unnecessary cost.\n\n- The novelty of the proposed method is unclear. Currently, it seems like another one manually-designed workflow.\n\n- The paper does not explain how tasks are decomposed.\n\n- No ablation study: subsequent experiments do not directly validate whether inter-agent misalignment has been solved, making the link between the stated problem and the experiments unclear.\n\n- The framework’s reliance on strictly sequential execution would make the workflow slow during inference.", "was_revised": false, "final_justification": "", "rebuttal": "### Dear Reviewer QP7F,\n\n### Thank you very much for your detailed review and constructive comments.\nWe have carefully followed your suggestions and added comprehensive ablation studies to the revised version.\n\nAfter receiving your feedback, we recognize that we should have placed greater emphasis on key content in the appendix and highlighted important claims more clearly. In response, we have revised the paper to include more detailed explanations and discussions to avoid further confusion.\n\n**Q1. For easy queries such as 1+1, the workflow method still needs to decompose and verify**\n\nThis will not be decomposed, as mentioned in line 111 under the advantage: \"We only split a task when this cross-model consensus indicates that splitting is necessary to resolve disagreement.\"\n\nWe have followed your comments and summarized the decomposition rates on benchmark datasets. Using our method with gpt-4o-mini, gpt-4.1-mini, and gpt-4.1-nano together, the results are shown in **response to Q6 of Reviewer QP7F**.\n\nThank you so much for helping improve our paper.\n\n**Q2. The paper does not explain how tasks are decomposed**\n\nThe Task Decompose Prompt and task decomposition template are shown in Appendix F.\n\nNote that this only happens when the answer provided by multiple LLMs on the same task is very inconsistent (50% disagreement). If it is a simple task, it will not be decomposed.\n\nTo make it clearer, we have provided generated workflows in our revised paper, see one as follows\n\n```json\n{\n  \"subtasks\": [\n    {\n      \"id\": 0,\n      \"objective\": \"Implement a Maze class that loads and stores the maze layout, including wall and path definitions, and provides collision detection.\",\n      \"depends_on\": []\n    },\n    {\n      \"id\": 1,\n      \"objective\": \"Implement a Pellet class defining pellet properties and a draw method for rendering on the maze.\",\n      \"depends_on\": []\n    },\n#### .... for (saving spaces) ....\n    {\n      \"id\": 5,\n      \"objective\": \"Implement a Tank class handling AI movement (patrol and chase), collision with Maze, shooting persistent bullets, and absorbing bullets to increase firing rate.\",  \n      \"depends_on\": [0, 2]\n    },\n    {\n      \"id\": 6,\n      \"objective\": \"Implement a Game class that initializes Pygame, loads the Maze, instantiates Pellet, Bullet, SpecialItem, Player, and Tank objects, and manages overall game state and scoring.\",\n      \"depends_on\": [0, 1, 2, 3, 4, 5]\n    },\n    {\n      \"id\": 7,\n      \"objective\": \"Implement the main game loop within the Game class to handle event processing, update all game objects, perform collision checks, render everything each frame, and enforce a consistent frame rate.\",\n      \"depends_on\": [6]\n    }\n  ]\n}\n```\n\n**Q3. The novelty is unclear. Currently, it seems like another manually-designed workflow**\n### Response to Manual Workflow Concern\nThank you so much for the insightful question. We do not use a manually designed workflow. We believe there may be some confusion. Any system requires a basic workflow definition—this is true even for the simplest setup, such as passing user input to a GPT-4o model to produce output. In that sense, all systems have some form of defined workflow to ensure functionality.\n\nAs discussed in Q2, our method does not rely on a fixed, manually designed workflow. Instead, it allows large language models (LLMs) to generate task-specific workflows only when necessary, based on disagreement detection. This dynamic and adaptive workflow generation is a key feature of our approach.\n\nTo avoid further confusion, we have revised our paper to clarify the notion of \"workflow\" and emphasize the distinction between fixed/manual workflows and dynamically generated ones.\n\n### Novelty Justification\nWe found that we should emphasize the contributions and perspectives we want to provide more explicitly. We have carefully added the following to our revised version. Please do not hesitate to let us know where further improvements are needed. Thanks for your time and effort.\n1. **Theoretical Formulation and Identification of Inter-Agent Misalignment**\n   We provide a systematic formulation of the data-generation process in multi-agent systems using probabilistic graphical models. This formulation enables a rigorous understanding of inter-agent misalignment by showing that it arises from violations of conditional-independence assumptions.\n2. **Key Insight and Real-World Impact**\n   * We emphasize that conditioning each subtask on the full history of verified outputs is crucial for inter-agent alignment.\n   * In some cases, it is necessary to sacrifice parallelism in favor of sequential execution to ensure reliable task execution.\n   * This insight is particularly important given that DAG-based decomposition is already used in industry AI agent systems. For example, AWS KIRO IDE precisely decomposes a user's coding requirement into a DAG and executes it.\n3. **An Effective Solution to Inter-Agent Misalignment Issues**\n   Our method provides a framework for mitigating inter-agent misalignment across diverse domains. The same mechanism works for mathematical reasoning (GSM8K: 96.3%), complex multi-hop tasks (HotpotQA: 83.5% F1), and novel code generation scenarios.\n4. **Strong Empirical Results on Novel Tasks**\n   We evaluate our method in compositional game environments where models must combine rules from different games (e.g., Pac-Man + Tank mechanics, Snake + RPG elements). These hybrid tasks require true reasoning rather than relying on memorization or shallow pattern matching, since these game combinations have not been designed before.\n5. **Efficiency Gains**\n   Most remarkably, our sequential approach achieves a 2.5× speedup over Flow (665 s → 217 s) and a 2.1× speedup over Atom (523 s → 217 s) while maintaining superior quality.\n6. **Other Novel Contributions That May Inspire Future Work**\n   - Investigating how to leverage different models and their differences could be important and warrant further study.\n   - An agent system should split workflows only when a task is difficult, not at the outset. In our paper, we use agreement across different models as a surrogate for task difficulty.\n   - Segment-based validation for early stopping can be useful for saving tokens.\n\n**Q4. Token cost would be high, there are risks exceeding the model's context window**\n\nThank you for the insightful comments. \nThe output token cost for **verification** can be ignored, as the **output token** is minimal, typically just a single token such as “correct” or “wrong.”\n\nThe main contributors to token cost are:\n\n1. The number of **input tokens** required for verification (i.e., providing the candidate output and relevant context), and\n2. The number of **LLMs** used to perform cross-verification.\n\nFor example, in our current setup, each verification involves three models verifying each other’s outputs, requiring six parallel verification calls. While this increases input token usage, the output remains negligible. Note that input tokens are much cheaper than output tokens in terms of cash cost (typically 1/4 to 1/5). We have carefully included a limitations section that covers this information.\n\nWe believe this trade-off is acceptable. As shown in the response to **Q1** of Reviewer asVu, introducing **cross-validation with multiple models** improves performance by **20.4%**. We believe this performance gain justifies the modest additional cost. Furthermore, this multi-model, same-task paradigm is gaining traction in the community. For example, the recent open-source project **MassGen** has already attracted 1.7k Discord users.\n\n\nFor the risk of the model's context window, we acknowledge this limitation but this should not be a major one for many real-word tasks. As current LLMs (e.g., GPT-4o mini) can have a 128,000 token context window which is approximately 96,000 words or around 17,000 lines of code.  For Gemini API, it can even allow 50,000 lines of code.\n\n\n**Q5. The framework's reliance on sequential execution would make the workflow slow during inference**\n\n**Table 10** in *Appendix B (page 26)* provides runtime comparisons across all tasks.\n\nIt is worth mentioning that:\n\n* Although SeqCV is slower than the single-agent baseline, it significantly outperforms it in generation quality **without** using the \"high\" version of the GPT model. Moreover, it is **2.5× faster than Flow** and **2.1× faster than Atom**, while achieving **better output quality**.\n* While AFlow appears faster at 160.86 s, this is because it generates sparse results (**see demo links in our appendix**).\n\nThe efficiency gains stem from our advanced system design:\n\n* We have implemented segment-based cross-validation to verify intermediate outputs before task completion, avoiding unnecessary computation.\n* We split tasks only when they are genuinely difficult.\n* Existing methods rely on an additional \"summary\" agent to integrate subtask outputs by consuming and aggregating the entire dialogue history (e.g., `summary.py` in the FLOW GitHub repository). By introducing a mandatory sequential post-processing step, they reduce the system to two stages: a parallel stage followed by a sequential stage. Our method directly concatenates the generated results, eliminating the need for a separate summary module in a subsequent stage.\n\n\n**Q6. No ablation study: subsequent experiments do not directly validate whether inter-agent misalignment has been solved**\n\nBy following your comments, we conducted control experiments using parallel and sequential runs with 4o-mini. \n\nNote that directly measuring inter-agent misalignment is challenging, as it generally requires **manual inspection of all the generated answers**. To address this, we adopt **quantitative performance** as a **proxy**, which is based on the assumption that fixing inter-agent misalignment leads to more coherent and reliable final outputs.\n\nDue to space constraints, the both results are presented in **our response to Q1 of Reviewer asVu**."}]

Author rebuttal:

Meta-review: This paper addresses a real and timely failure mode in multi-LLM agent workflows: inter-agent misalignment arising from violated conditional-independence assumptions. The paper then proposes SeqCV, a sequential, history-conditioned execution with segment-level cross-model verification and adaptive recursive splitting.  
Reviewers broadly agree the problem framing is useful, the system design is clear, and the empirical evidence is persuasive: they report consistent gains on standard reasoning benchmarks and larger creative tasks, with runtimes competitive with or better than popular parallel pipelines, despite the move to sequential conditioning.   

There were questions and concerns regarding experiments and clarity. The rebuttal seem to address some of the key issues, with ablations that isolate component contributions such as sequentialization, cross-validation, splitting. The discussion also resulted in addition of latency breakdowns, decomposition prompts, and reproducibility details.  
Remaining concerns are mostly about scope and positioning: the novelty is largely on careful integration of well known ingredients which might be deemed incremental; reliance on LLM consensus without additional verification; and token-/context-cost accounting is still deemed thin. 

After reviewing the paper and author discussion, I would discount the Reviewer QP7F's criticism as to me they largely lean on points that are either not core issues or already addressed. To list of few of these and why they are invalid concenrs:
- "the workflow method still needs to decompose and verify, which introduces unnecessary cost." The method splits only when cross-model consensus indicates disagreement. That is stated verbatim.  
- "The novelty of the proposed method is unclear. Currently, it seems like another one manually-designed workflow." To me there seems to be enough novelty and the split is learned and applied dynamically to the DAG during execution.
- "It is a manually designed workflow." The split is learned and applied dynamically to the DAG during execution. Algorithm 2 shows how the graph is updated and recursed. 
- "No ablation study" The paper explicitly lists investigations using cross-validation modules and workflow-decomposition modules to verify component necessity. While the deepest ablations are added in the discussion/rebuttal, the main text includes these component studies.
Decision: Accept (poster)