ComfyMind: Toward General-Purpose Generation via Tree-Based Planning and Reactive Feedback
Authors: Litao Guo
Xinli Xu
Luozhou Wang
Jiantao Lin
Jinsong Zhou
Zixin Zhang
Bolan Su
Ying-Cong Chen
Venue: NeurIPS 2025 poster ()
Keywords: General-Purpose Generation
Collaborative AI system
ComfyUI
Primary area: applications
TL;DR: This paper introduces ComfyMind, a novel collaborative AI system for general-purpose generative tasks leveraging ComfyUI.

Abstract: With the rapid advancement of generative models, general-purpose generation has gained increasing attention as a promising approach to unify diverse tasks across modalities within a single system. Despite this progress, existing open-source frameworks often remain fragile and struggle to support complex real-world applications due to the lack of structured workflow planning and execution-level feedback. To address these limitations, we present ComfyMind, a collaborative AI system designed to enable robust and scalable general-purpose generation, built on the ComfyUI platform. ComfyMind introduces two core innovations: Semantic Workflow Interface (SWI) that abstracts low-level node graphs into callable functional modules described in natural language, enabling high-level composition and reducing structural errors; Search Tree Planning mechanism with localized feedback execution, which models generation as a hierarchical decision process and allows adaptive correction at each stage. Together, these components improve the stability and flexibility of complex generative workflows. We evaluate ComfyMind on three public benchmarks: ComfyBench, GenEval, and Reason-Edit, which span generation, editing, and reasoning tasks. Results show that ComfyMind consistently outperforms existing open-source baselines and achieves performance comparable to GPT-Image-1. ComfyMind paves a promising path for the development of open-source general-purpose generative AI systems.

ComfyMind: Toward General-Purpose Generation via Tree-Based Planning and Reactive Feedback
Litao Guo" HKUST(GZ))
Xinli Xu' HKUST(GZ))
Luozhou Wang HKUST(GZ)
Jiantao Lin HKUST(GZZ)
Jinsong Zhou HKUST(GZ))
Zixin Zhang HKUST(GGZ)
Bolan Su Bytedance
Ying-Cong Chen HKUST(GZZ, HKUST
ext to Image G eeneratio
Reasoning Generation
Ultra-high Resolution Geveration
(1: fesoaao
whher be
Boo Febiul
Gevaate
Clai
Images Mierge
Image O Outpaint
Eaed on glvar ocfenee
praniale singet
Object Removal
Lggo Embedding
Image Editing
Style Generation
Aoael of dee ddeiico
The buciorment
Video G eeneration
Image Inpaint
12V Generation
Video Interpolation
Face Swap
Video Edit
Image Refine
Figure 1: Overview of generative and editing capabilities supported by ComfyMind.
Abstract
With the rapid advancement of generative models, general-purpose purpose generation has gained increasing attention promising approach unify diverse tasks across modalities within a single system. Despite this progress, existing open- source frameworks often remain fragile and struggle to support complex real-world applications due to the lack of structured workflow planning and execution-level feedback. To address these limitations, present ConfyMind, collaborative AI system designed to enable robust and scalable general-purposs generation, built the ComfyUl platform. ComfyMind introduces two core innovations:
Indicates Equal Contribution
I Indicates Corresponding Author
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Semantic Workflow Interface (SWI) that abstracts low-level node graphs into callable functional modules described in natural language, enabling high-level composition and reducing structural errors; Search Tree Planning mechanism with localized feedback execution, which models generation as hierarchical decision process and allows adaptive correction at each stage. Together, these components improve the stability and flexibility of complex generative workflows. We evaluate ComfyMind on three public benchmarks: ComfyBench. GenEval, and Reason- Edit, which span generation, editing, and reasoning tasks. Results show that ComfyMind consistently outperforms existing open-source baselines and achieves performance comparable to GPT- GPT-Image- ComfyMind paves a promising path for the development of open-source general-purpont generative Al systems. Project page: https: /nge:/https://giee //tpt/ggtttt.tt/////// cww/En/m/////c EnWieiiiiiiiiiiniiieneeeeneenenn C Ressearch/ComfyMinn
Introduction
The rapid development of visual generative models has demonstrated remarkable performance across multiple generative tasks, including text-to-image generation [133], image editing [466), and video generation 770). In recent years, research has gradually shifted towards end-to-end general-purpose generative models [0]33 aiming to handle these diverse tasks within a single unified model. However, existing open-source seneral-purpose generative models [2 still face series of challenges, including instability in generation quality and lack of structured planning and composition mechanisms required to handle complex, multi-stage visual workflows. These limitations affect the model's ' performance and robustness in real- world applications. In contrast, the newly released OpenAl's s GPT-Image-1
[33 has garnered widespread attention for its remarkable capabilities in unified image generations [1]. However, the closed-soure, nature of GPT- and its primary focus on image generation tasks limit its applicability and scalability across broader multimodal generation tasks.
The ComfyUl platform provides a potential path toward achieving open-source general-purpose generative approach. ComfyUl is an open-source platform designed to create and execute generative workflows, offering a mode-based interface that allows users construct visual generative workflows represented as JSON according to their needs. The platform's 's modular design offers high flexibility in constructing workflows. However, despite its flexibility, building complex workflows from scratch remains a challenge, particularly when dealing with customized or intricate task requirements, which demand substantial expertise and considerable time for trial and error. To address this, recent research [6] has begun exploring the use of large language models (LLMs) to construct customized workflows, thereby enabling general-purposs visual generation.
Building on prior work based on ComfyUl [17]. such ComfyAgent [16), an automated solution for generating workflows from natural language instructions was proposed. ComfyAgent employs a multi-pronged mechanism to convert natural language instructions into executable workflows. This approach involves pseudo-code codee translation to convert ISON structures into Python-like code, dy- namic retrieval [18] of node documentation to stan- dardize outputs, and refinement at the workflow level. While this has made significant progress, ComfyAgent also reveals two core issues in low- level workflow generation. First, it treats workflow construction as a flat, token-based decoding task, making it difficult to effectively model modularity and hierarchy. This leads to node omissions, se- mantic mismatches, and fragile compositions that are challenging to generalize across tasks. Second, the system lacks execution-leve feedback mechanism. Once the workflow is constructed, the system cannot obtain any feedback or error information during generation, hindering incremental corrections and reducing overall robustness.
Comfy gent
Ours
Multi-Agent System
Multi-Agent System
Planzing Agetz
Planning A gent
Combine Agunt
Refine Agent
Exccution Agent
Vode Docapeo
Catrintio
Cantylli Fatters
Dowciption
Moricfior: J SSN
Imugs 1ide
Evaisation Agest
ComfyUl Platform
Results
Resuits
result vertflcation step
Self-verifying execution One workflow description doc.
Large-srale dataset dec X
Figure 2: Structural comparison between ours and ComfyAgent.
To address the challenges faced by ComfyAgemt we drew inspiration from human user workflow construction paradigms. We observed that human users typically do not build complex workflows from

---

scratch but instead decompose tasks into smaller subtasks and select appropriate template workflows for each subtask based on higher-level semantics. This modular, step-by-step planning process. combined with localized feedback strategies, allows for incremental refinement and adaptation. When failure occurs, adjustments are made locally rather than globally. This hierarchical planning and reactive feedback strategy enhances the ability to solve complex problems and increases robustness.
In this work, we simulate this human-like strategy and propose a novel framework called ComfyMind. As shown in Figure our framework exhibits strong generality, supporting a wide range of image and video generation and editing tasks. The framework represents workflow generation as the semantic combination of template workflows, rather than the boken-based synthesis of node configurations. Specifically, Comfy Mind treats template workflows as atomic semantic modules, each with clearly defined function, input/output interfaces, and natural language descriptions. By reasoning over these higher-level components, ComfyMind achieves more stable and controllable task compositions. In addition, the semantic-level abstraction also enables effortless integration of new workflows, allowing ComfyMind quickly adapt to emerging
contributed models and tasks.
ComfyMind consists of two core mechanisms. The first is the Semantic Workflow Interface (SWI), which abstracts low-level node graphs into callable semantic functions annotated with structured inputs, outputs, and natural-language captions. This abstraction allows language models to operate on workflows at the semantic level, reducing exposure to platform-spp -eperm---peffic syntax and minimizing structural errors. The second mechanism is Search Tree Planning with Local Feedback Execution, which models task execution as hierarchical decision process. Each node the planning tree represents sub-task, and each edge corresponds to a selected SWI module. During execution, failures trigger localized corrections at the current tree layer, avoiding full-chain regeneration and significantly improving robustness. The comparison with previous work is shown in Figure 2
We evaluate ComfyMind on three public benchmarks: ComfyBench GenEval, and Reason-Edit. On ComfyBench, ComfyMind improves the workflow pass rate from 56.0% to 100%, and the task resolution rate from 32.5% to 83.0%. On GenEval, achieves an overall performance score of 0.90, outperforming all baseline methods including GPT-Image-1 1. On Reason-Eao it reaches GPT-score of 0. 0.906, surpassing all open-source systems and matching the performance of proprietary models.
Our contributions are summarized follows:
We introduce ComfyMini visual content creation as a modular, semantically structured planning process, and can be instantiated on node-based execution systems (c.g., ComfyUI) that support modular composition and hierarchical task planning.
pureroe- generation framework that conceptualizes
We propose unified control mechanism that integrates a Semantic Workflow Interface for high-level modular abstraction with Search Tree Planning with Local Feedback Execution, enabling semantically grounded composition, adaptive correction, and robust realization of complex multi-stage workflows.
We validate ComfyMind on three public benchmarks: ComfyBench, GenEval, and Reason- Edit, covering tasks in generation, editing, and reasoning. The results demonstrate strong performance, broad task generality, and the potential of semantic workflow composition as a foundation for general-purpose visual generation.
2
Related work
2.1 General-Purpose Visual Generation
Traditional visual generative models typically design task-specific architectures 19-21 address various generation tasks. With the rapid development of generation models, many studies have shifted towards general- purpose generation models, aiming to solve a wide range of generation tasks
using single model [4 II 25 29]. These approaches generally unify image understanding and image generation within one model, enabling more efficient and consistent performance across different tasks. Recently, OpenAl's 's closed model GPT-Image-I [13 has gained significant attention due to its remarkable capabilities in image generation, editing, and modification, achieving top performance across multiple domains.
3

---

This study aims to leverage the open-s platform ComfyUl to design a collaborative Al system. By constructing flexible workflows for each task, we strive achieve general purplse visual generation. Compared to current seneral-purpose generation methods, our approach offers distinct advantages: not only encompasses video modalities but also effectively handles the structured planning and integration of complex, multi-stage visual workflows.
2.2 Planning and Feedback Mechanisms in Collaborative AI Systems
In collaborative Al systems, planning and feedback mechanisms are essential for efficiently managing complex tasks. Early systems like AutoGPT 30) generate static goal lists with minimal intermediate feedback, often leading to inefficiencies when sub-goals fail. In contrast, more dynamic systems like AutoGen [31] allow for iterative plan adjustments through dialogue, enhancing adaptability. OpenAgents (32] provides structured execution pipelines tailored to domain-specific tasks but at the cost of flexibility. ReAct [33]
introduces a reasoning-acting loop that offers fine- lineggriined control, though it may not be ideal for more complex workflows that require extensive coordination across multiple agents. MetaGPT + models human-team roles to standardize collaboration, but it lacks
the flexibility needed for dynamic, evolving tasks.
Despite these advancements, existing systems still face challenges in localized error handling and recovery. Many systems lack mechanisms for evaluating intermediate outcomes, making them prone to failure in dynamic, non-linear workflows, such those encountered in generative tasks like CI [16]. Motivated by these limitations, we propose a hierarchical search-tree planning strategy combined with local feedback execution ensure more robust adaptation and fine- ane-grained error recovery during task execution.
3 ComfyMind
3.1 Co Platform
ComfyUl an open-source platform for designing and executing generative pipelines. User-defined workflows are encoded as JSON-based directed acyclic graphs (DAGs) and interpreted node-by-node on the server to produce images videos. This node-level abstraction significantly lowers the entry barrier for human artists by modularizing the execution process into visual, interpretable components.
CImfyUL, with the efforts of the open-source community, has integrated support for wide range of advanced models, covering various modalities from image generation to video production. Notable models include Flux 33, Flux Redux 35 ACE++ (36). Segment Anything 37 (8), Hunyuan (8), and Wan [9) among others. This broad support enables users to experiment with state-of-the-an tools for more diverse and sophisticated creative processes. In contrast, building complex workflows from scratch remains a significant challenge. Constructing workflows that meet highly customized or complex task requirements not only requires deep technical knowledge but also demands significant time and trial-and-error, creating considerable barriers for users trying to efficiently develop new workflows or address specific needs.
3.2 Semantic Workflow Interface
ComfyAgent aims to construct demand-customized ComfyUl workflows through LLMs to achieve general approach to visual generation. It has meticulously built a large-scale Node document dataset, designed an intricate collaborative Al system, employed pseudocode to replace JSON for enhanced workspace representation, and utilized RAG technology. However, it still faces issues such as syntax errors in workflows and missing key Nodes, which result in pass rate of only 56% on ComfyBench. The fundamental reason for this low pass rate lies in the vulnerability of low-level representations to LLMs and the conceptualization of workflow construction as a planar, token-based decoding task, which makes it difficult to effectively model modularity and hierarchy.
In contrast to ComfyAgent's paradigm of building entire workflows at the low-level, we adopt a human-like approach to workflow construction by decomposing generation tasks into modular subtasks, each handled independently by planning agent. Within each subtask, the planning agent selects the most appropriate atomic workflow from the workflow library as a tool. Unlike complex workflows, each atomic workflow is responsible for simple, single-step generation process, such

---

as text-to-ine -mmage generation or mask generation. In other words, we replace the single token in ComfyAgent with atomic workflows as the minimal unit in workflow construction.
Following this approach, we introduce a semantic workflow interface, which uses natural language functions instead of low-level JSON specifications construction. Each atomic workflow, encapsulating a function, is annotated with a simple natural language description outlining its purpose, required parameters, and usage. Based on this metadata, the planning agent in ComfyMind selects the most appropriate function for invocation. During the invocation, required parameters (e.g.. prompts or reference images) and optional high-level constraints are passed. The execution agent then maps the selected function to its corresponding ISON representation, injecting the parameters. Finally, LLMs perform adaptive parameter-leve tuning on the JSON to meet additional constraints. The resulting workflow is executed via the Ul platform, thus completing the generation of the individual subtask.
intermediate representation for workflow
This abstraction allows the LLM to operate entirely at the semantic level, bypassing the complexities of low-level syntactic grammar and the difficulty of effectively modeling modularity and hierarchy. By eliminating this bottleneck, ComfyMind significantly enhances the robustness of execution. The SWI also minimizes the reliance on fine-grained node documentation. While ComfyAgent' operation depends on meticulously crafted dataset containing 3,205 distinct node descriptions, ComfyMind only requires single unified document to describe the available atomic workflows. Without the need for RAG. ComfyMind can directly inject workflow metadata into the LLM's s context window, ensuring full visibility and eliminating the dependency on external lookups. Ultimately, this reduction in documentation facilitates the seamless integration of newly developed or task-specifer workflows. This design enables ComfyMind to quickly integrate emerging workflows from the broader Comfy UI community, while allowing users the flexibility to customize workflow documentation and repositories to meet specific needs.
Search Tree Planning with L Local Feedback Execution
Execution Agest
User Instruction
Best Node
Plaazing Agent
Generttd Plazning
Lubedio and
ghoos agat cvein sctiann
Evalustion. Agent
Papse Parazeetrr
Execution Agent
Foedback
Autb--djuuiigg W Woelsloww For A oditional Requimeers
Collaborative Al System
Modi
Swach Tiu Rleaing Eredark Zavodio
Comtyca Wockflow Execation
Result
Sade
Dasulo::
Workspace Updane
the
PV
Astie Style
Recunively enter the nec: paaningg aode
X
X
Figure 3: Overview of ComfyMind pipeline. Given instruction, the system first parses the task and delegates to Planning Agent. The Agent incrementally explores a semantic search tree, where each node proposes candidate workflow and receives local feedback based on execution results.
3.3 Search Tree Planning with Local Feedback Execution
atomic workflow using natural-language function calls. However, the system must still determine; how to compose multiple SWI calls into a coherent and task-completing program. To address this, as sketched in Figure 3 ComfyMind introduces mechanism we term Search Tree Planning with Local Feedback Execution, which formulates workflow construction as a search process over a semantic planning tree. In this structure, each node represents local planning agent responsible for specific sub-task, while each edge denotes an execution agent that invokes an SWI function and propagates the result. complete path from the root to leaf yields the final visual output that satisfies the user instruction.
As introduced in Section 3.2 SWI enables LLMs to invoke community-
At each planning node, the agent examines the current hierarchical workspace state- incee-incudiiingg text, images, context, and the available workflow documentation. Based this information, it generates
5

---

chain of SWI functions aimed at advancing the current task. Only the first function in the chain is executed this stage, with its arguments passed the execution agent. This transition is equivalent to following edge in the planning tree.
The execution agent translates the selected function into its canonical JSON form defined by the SWI, applies lightweight parameter adjustments based on higher-level constraints, and executes the workflow using the ComfyUI platform. Throughout the process, the underlying DAG structure is preserved to ensure syntactic correctness, After execution, a Vision-Language Model (VLM) parses and annotates the generated visual content. The resulting artifact, its semantic description, and the updated task specification collectively define the workspace for the next planning node.
If the planning agent determines that its sub-task issues a termination signal and invokes the evaluation agent to assess the final output in terms of semantic alignment and perceptual quality. If the result passes evaluation, the search ends successfully. Otherwise, a failure signal and diagnostic feedback are passed to the parent node, which records the outcome and revises its planning strategy accordingly. If no viable options remain the current level, the error signal propagates upward. Crucially, all feedback is strictly confined to the current hierarchy level, preventing global rollback and preserving valid partial results.
be completed with single operation, it
Compared to the step-by-step poserve-then--ee execution style ReAct [39 planners, our method offers complete history tracking and structured backtracking capabilities. This allows the system to roll back only to the most recent viable decision point upon failure, rather than restarting the entire process moocss--hhss avoiding redundant recomputation. At the same time, improves planning stability by preventing repeated re-planning cycles caused by the lack of stable intermediate states, which can otherwise lead to strategy oscillations and failure converge.
4
Experiments
To assess our system's generative capabilities, we conduct a three-pronged evaluation. Comfy- Bench [16] quantifies the system's ability to autonomously build workflows and ceneral-purpose generation; GenEval [40] evaluates the system's T2I generation capabilities; Reason-Edth (6] mea- sures how well complex editing instructions are executed. Experiments demonstrate that our method surpasses the strongest open-soure baselines by a substantial margin across all three benchmarks and achieves performance comparable to GPT- An ablation study further confirms the contribution of each design component. More experimental results are presented in Appendix A
4.1
Autonomous Workflow Construction
Table 1: Evaluation of Autonomous Workflow Construction on ComfyBench [16].
Vanilla
Complex
Creative
Total
Agent
%aass
&Resolve
%Pass
Resollv
Pass
Ressolve
PaPss
&Resolve
GPT-4o + Zero-shot
0.0
0.0
0.0
0.0
0.0
0.0
0.0
0.0
GPT-40 Few-shol
32.0
16.7
16.0
GPT-4o CoT 42
44.0
29.0
11.7
8.3
12.5
0.0
28.0
17.0
GPT-4o CoT-SC [33
45.0
34.0
11.7
5.0
15.0
0.0
29.0
18.5
Claude-3.5-Sonnct RAG [8]]
27.0
13.0
23.0
6.7
7.5
0.0
22.0
8.5
Llama-3.1-70B RAG
58.0
32.0
23.0
10.0
15.0
5.0
39.0
20.0
(PT--4 RAG
62.0
45.0
21.7
40.0
7.5
52.0
23.0
ul-mini RAG
32.0
16.0
21.7
8.3
12.5
7.5
25.0
12.0
ol-preview RAG
70.0
46.0
48.3
23.3
30.0
12.5
55.5
32.5
ComfyAgent [T
67.0
46.0
48.3
21.7
40.0
15.0
56.0
32.5
Ours
100.0
92.0
100.0
85.0
100.0
57.5
100.0
83.0
We assessed the autonomous workflow construction capacity of our method in ComfyBench [16]. ComfyBench comprises 200 graded difficulty generative and editing tasks that span image and video modalities. For each task, the agent must synthesize workflows that can be executed by ComfyUL. The benchmark reports (i) a pass rate, reflecting whether the workflow runable, and (ii) a resolve rate, reflecting whether the output satisfies all task requirements.
As shown T powered by SWI, our system achieves 100% pass rate across all difficulty tiers. Our methods climinate JSON-level failures that still impede the strongest baseline, ComfyAgent.

---

More importantly the proposed Search-Tree Planning with Local-Feedback Execution delivers substantial gains in task resolution: relative ComfyAgent, Resolve rate increases by 100% % 292%, and 2 on the Vanilla, Complex, and Creative subsets, respectively. Appendix further demonstrates that our system successfully addresses wide spectrum of user instructions. The strong generalization ability and output quality evidenced here point to multi-agent systems based on ComfyUl as a promising avenue toward pererol-purpose generative AI.
Text-eo-Image Generation
4.2.1 Quantitative Results
We evaluate our system's 's capability in T2I generation using GenEval [40].
GenEval measures
compositional fidelity across six dimensions, including single or two objects, count, color accuracy. spatial positioning, and attribute binding. We compare our method against three strong categories of baselines: (i) Frozen
Encoder Mapping Methods, represented by SD3; (ii) I.M/MM..M-enhanee methods, such as Janus and GoT;; and (iii) OpenAl's recently released GPTT-mmae-11
As shown Table 2. our system achieves an overall score of 0.90, benefiting from its integration of prompt optimization workflows and local feedback execution. This result surpasses all baselines by +0.16 over SD3 and +0.10 over Janus-Pro-7B. Moreover, our system exceeds GPT-Image-1 in five out of six dimensions and the overall score. These results demonstrate that our ComfyUl-based system not only offers strong generality but also is capable of consolidating the strengths of diverse open models, achieving state-of-the performance in image synthesis.
Table 2: Evaluation of T2I generation on GenEval
40
Obj.: Object. Attr: Attribution.
Method
Overall
Single Obj.
Two Obj.
Counting
Colors
Position
Attr. Binding
Frozen Text Encoder Mapping
Methods
SDv1.5 44
0.43
0.97
0.38
0.35
0.76
0.04
0.06
SDv2.1
44
0.50
0.98
0.51
0.44
0.85
0.07
0.17
SD-XL
0.55
0.98
0.74
0.39
0.85
0.15
0.23
DALLE-2 2
45
0.52
0.94
0.66
0.49
0.77
0.10
0.19
SD3 303-Medium
0.74
0.99
0.94
0.72
0.89
0.33
0.60
M.Ms/II.MM Finhaceed Methods
LlamaGen 46
0.32
0.71
0.34
0.21
0.58
0.07
0.04
Chameleon
0.39
LWM 47
0.47
0.93
0.41
0.46
0.79
0.09
0.15
SEED-X 10
0.49
0.97
0.58
0.26
0.80
0.19
0.14
Emu3- Gen
22
0.54
0.98
0.71
0.34
0.81
0.17
0.21
Janus 11
0.61
0.97
0.68
0.30
0.84
0.46
0.42
JanusFlow
22
0.63
0.97
0.59
0.45
0.83
0.53
0.42
Janus-Pro-7P
29
0.99
0.89
0.79
0.66
0.99
0.69
0.34
0.27
GPT Gma--
13
0.84
0.99
0.92
0.85
0.92
0.75
0.61
Collaborative Al Systems
ComfyAgent [6
0.32
0.69
0.30
0.33
0,50
0.04
0.04
Ours
0.90
1.00
1.00
1.00
0.97
0.62
0.80
4.2.2 Qualitative Results
Figure showcases representative and challenging cases from GenEval. Our method follows prompts, outperforming existing models on core constraints like counting, color, position and Attribution Binding. In the counting task, only our system generates exactly four keyboards with clear visual separation. For atypical color and position, we demonstrate superior image quality and instruction consistency. Regarding attribute binding, models like SD3 and Janus-Pro often entangle attributes and fail to localize them correctly. GPT-Image-1, while generally cntructtionfolllwwiii fragmented and visually incoherent compositions. In contrast, our method not only satisfies fine- grained directives but also integrates them into aesthetically coherent, contextually grounded scenes. These gualitative results corroborate the quantitative gains reported earlier.
often produces

---

A photo of Four computer keyboards
A photo frishee A bove truck
JaussPro
GPT-Image-
GPT-lmage1
photo of Red dog
A photo of Brown oven and Purple train
-PT-lmage1
GPT-Image-
Figure 4: Qualitative comparison challenging GenEval 40) cases. Under constraints such as counting, color, position and attribute binding, only our method successfully satisfies all instructions, clearly outperforming SD3, Janus-Pro. and GPT-Image-1.
4.3 Image Editing
4.3.1
Quantitative Results
We further evaluate our system's image editing capability on the Reason-Edit [6] Following setting in benchmark, we adopt the GPT-score [6] as the evaluation metric. This score quantifies both the semantic fidelity to the editing instruction and the visual consistency non-edited regions.
We compare our method against the most advanced open-source baselines, including GoT [12).
SmartEdit 66, CosXL.-Edit (6, SEED-X 10, MGIE [88] MagicBrush 449 and IP2P [4,, as well
as the strongest dlosed-sourge model, GPT- GPT-Image-1 As shown in Figure 5 our method achieves score 0.906 the highest among all open-souree souree frameworks. This result represents a substantial
improvement of +0.334 over the previous open- spen-source SOTA SmartEdit (0.572).
Moreover, method achieves performance competitive with GPT--maae-1 (0.929), narrowing the gap between open and closed models. This gain arises from our sys- tem's planning and feedback mechanism, which enables it synthesise and combine the most effective editing work- flows contributed by the ComfyUI community. Through reasoning and iterative correction, our agent can adap- tively select diverse workflows, improving the stability and precision of edits across varied scenarios. These re- sults highlight the reasoning-drive editing capability of our system, and suggest strong potential future per- formance gains through integration with more powerful workflows and models.
DIN
01380
0.20
Figure 5: Quantitative Comparison
Reason-Edit 6 benchmark.
4.3.2 Qualitative Results
We further present qualitative results to assess the semantic understanding and visual fidelity of our system under challenging editing instructions. We select two representative tasks from the Reason-Edit [6] benchmark. As shown in Figure
method consistently demonstrates the most faithful and visually coherent results across both tasks. Compared to existing open-source baselines, system not only identifies the correct semantic target (e.g., apple vs. bread vs. orange juice) but also executes edits with minimal disruption to adjacent regions.
Although GPT-Image-1 succeeds in executing editing instructions. it struggles to maintain visual consistency in non-edited regions. As illustrated Figure GPT-Image- loses details in non-edited areas (e.g., patterns the juice box, yogurt container, and jam jar in the zoom-in views), alters color tones and image style, inaccurately preserves materials (e.g., wood texture), and changes the original aspect ratio. These flaws are also mentioned in GPT-eval [4].

---

In contrast, our method accomplishes the instructions with minimal edits, effectively preserving visual details, image style, material properties, and proportions. These observations highlight our method's superior capability to perform precise and coherent edits.
Please replace food contains most vitamin with an orange.
Input Image
LnstructPix2Ptx
Magice rust.
InstruetDiffusion
SmartEeit-7g
GPI- GPI-Image
Ous
Inpu
GPT
Ours
Drars
In Detall
Please remove the object that can be used to have meals.
Input Image
InstrucPPi2PPx
MagicBrush
InstractDiffasion
SmartFoit-77
GPT-Image 1
Ours
Figure 6: Qualitative Comparison Reason-edit
6
benchmark.
4.4 Ablation Study
To isolate the contributions of our key design components, conduct ablation study the ComfyBench benchmark (Table We evaluate three variants: the full system, a version without search tree planning, and version without feedback execution. Benefiting from our Semantic Workflow Interface, all variants achieve 100% pass rate; the main differences lie the resolve rate.
Table 3: Ablation results of ComfyMind with different architectures ComfyBench.
Vanilla
Complex
Creative
Total
Agent
Paass
% Resolve
&asss
%Resolve
%Pass
RReslvv
% %aass
% Resollve
Ours
100.0
92.0
100,0
85.0
100.0
$7.5
100.0
83.0
Ours w/o Tree Planning
100.0
86.0
100,0
43.4
100.0
50.0
100.0
66.0
Ours w/o Feedback 100.0
68.0
100.0
55.0
100.0
17.5
100.0
54.5
Removing the search tree planning module results in notable drop in task resolution, particularly on complex tasks (85.0% to 43.4%), underscoring its role decomposing multi-step instructions and selecting suitable workflows. Similarly, disabling the local feedback mechanism significantly degrades performance, especially on creative tasks (57.5% to 17.5%), highlighting its importance for iterative correction and adaptive refinement. The results confirm that key components essential for achieving high success rates autonomous workflow construction.
Conclusion
In this work, propose ComfyMind, novel framework built on the ComfyUI platform that ad- dresses key challenges in general-purpose visual generative AL By conceptualizing visual content creation modular, semantically structured planning process and incorporating tree-based planning with local feedback execution, ComfyMind improves the stability and robustness multi-stage work- flows. Our framework outperforms previous open-source methods and achieves results comparable to emag-- on benchmarks ComfyBench, GenEval, and Reason-Eaa ComfyMind offers a promising path towards scalable, open-source solutions for complex generative tasks.
Discussion This work focuses on developing a collaborative system with general-pune purpose visual generation capabilities, rather than generating complete workflow JSON files. The goal is to explore

---

how such a system can plan, execute, and provide feedback for complex tasks through semantic planning. modular composition, and localized corrections. While we use ComfyUI workflow modules for execution, the workflow itself is component of the system, not the research focus. By operating higher semantic level with the SWI, our approach reduces the complexity and uncertainty of workflow generation, offering more robust and practical solution for real-world applications.
6
Acknowledgements
This work is supported by National Natural Science Foundation of China (No. 62206068), HKUST- HKUST(GZZ) Cross-Campus Collaborative Research Scheme (Project No. C036) and Guangdong Provincial Department of Science and Technology's '1+I+1' Joint Funding Program for Guangdong- Hong Kong Universities.
References
[1] Dustin Podell, Zion English. Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas Müller, Joe Penna, and Robin Rombach. Sdxl: Improving latent diffusion models for high-resolution image synthesis. arXiv preprint wxXiv:237.001552 2023.
[2] Patrick Esser, Sumith Kulal, Andreas Blattmann. Rahim Entezari, Jonas Müller, Harry Saini. Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, al. Scaling rectified flow transformers for high-resolution image synthesis. In Forty-first international conference on machine learning, 2024.
[3] Black Forest Labs. Flux, 2024.
labe/ flux
[4] Tim Brooks, Aleksander Holynski, and Alexei A Efros. Instructpix2pix: Learning to follow image editing instructions. In Proceedings of the IEEE/CVF conference on computer vision and puttern recognition, pages 18392-18402, 2023.
[5] Stability Ai. Cosxl, 2024. URL UKLEEppp///ngiiigtoooo//ssssss/.
[6] Yuzhou Huang, Liangbnn Xie, Xintao Wang, Ziyang Yuan, Xiaodong Cun, Yixiao Ge, Jiantao Zhou, Chao Dong, Rui Huang. Ruimao Zhang, et Smartedit: Exploring complex instruction prauuction-based image editing with multimodal large language models. In Proceedings of the IEEF/CVE Conference on Computer Vision and Pattern Recognition, pages 8362-8371, 2024.
[7] Yoav HaCohen, Nisan Chiprut, Benny Brazowski, Daniel Shalem, Dudu Moshe, Eitan Richardson, Eran Levin, Guy Shiran, Nir Zabari, Ori Gordon, et al. Lix-video: Realtime video latent diffusion. arXiv preprint arXiv:2501. 2024.
[8] Tencent. Hunyuanvideo, 2024. URL hattpps://gthuuu..cc//tttaaa////
Ang Wang, Baole Ai, Bin Wen, Chaojie Mao, Chen-Wer Wei Xie, Di Chen, Feiwu Yu, Haiming Zhao, Jianxiao Yang, Jianyuan Zeng, et al. Wan: Open and advanced large-scale video generative models. arXiv preprint arXiv:2503.20) 20: 14, 2025.
[10] Yuying Ge, Sijic Zhao, Jinguo Zhu, Yixiao Ge, Kun Yi, Lin Song, Chen Li, Xiaohan Ding, and Ying Shan. Seed-x: Multimodal models with unified multi-granularity comprehension and generation. arXiv preprint arXiv:2404. 14396, 2024.
[11] Chengyue Wu, Xiaokang Chen, Zhiyu Wu, Yiyang Ma. Xingchao Liu, Zizheng Pan. Wen Liu, Zhenda Xie, Xingkai Yu, Chong Ruan, et al. Janus: Decoupling visual encoding for unified multimodal understanding and generation. arXiv preprint arXiv:2410. 13848, 2024.
[12] Rongyao Fang, Chengqi Duan, Kun Wang, Linjiang Huang, Hao Li. Shilin Yan, Hao Tian, Xingyu Zeng Rui Zhao, Jifeng Dai, et al. Got: Unleashing reasoning capability of multimodal large language model for visual generation and editing. arXiv preprint arXiv:2503. /0..003.00669 10639, 2025.
[13] OpenAL. gpt-image-1 1, 2025. URL htttt::/p///ptooorrr openai. com//ocssmodees//geg image-1
[14] Zhiyuan Yan, Junyan Ye, Weijia Li, Zilong Huang, Shenghai Yuan, Xiangyang He, Kaiqing Lin, Jun He, Conghui He, and Li Yuan. Gpt-imgeval: A comprehensive benchmark for diagnosing gpt4o in image generation. arXiv preprint wxXiv:2504.0278. 2025.
[15] Rinon Gal, Adi Haviy, Yuval Alaluf. Amit H Bermano, Daniel Cohen Or, and Gal Chechik, Comfygen: Prompt-adaptive workflows for text-to-image generation. arXiv preprint 2rX0022410.01731, 2024.
10

---

[16] Xiangyuan Xue, Zeyu Lu, Di Huang. Zidong Wang, Wanli Ouyang, and Lei Bai. Comfybench: Benchmark- ing Ilm-based agents in comfyui for autonomously designing collaborative ai systems. URL https://rxii. org/ubs/2409.01 392, 2024.
[17] ComfyUl Contributors. Comfy UI: A powerful and modular stable- gui. https:///itthut com/ comtyanonyyouus/Comffy 2023. Accessed: 2025-05-14.
[18] Patrick Lewis, Ethan Perez, Aleksandra Pikus,, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, et al. -aggeentdd generation for knowledge- intensive nlp tasks. Advances in neural information processing systems, 33:9459 9474, 2020.
[19] Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models. In Proceedings of the IEEE/CVF international conference on computer vision, pages 384773847, 2023.
[20] Hu Ye, Jun Zhang, Sibo Liu, Xiao Han, and Wei Yang. Ip-adapter: Text compatible image prompt adapter for text-to-ime text-to-image diffusion models, arXiv preprint arXiv:23388.677 2023.
[21] Rinon Gal, Yuval Alaluf, Yuval Atzmon, Or Patashnik, Amit Bermano, Gal Chechik, and Daniel Cohen-Or. An image is worth one word: Personalizing text-to-image generation using textual inversion. arXiv preprint arXiv:208.0010 2022.
[22] Yiyang Ma. Xingchao Liu, Xiaokang Chen, Wen Liu, Chengyue Wu, Zhiyu Wu, Zizheng Pan, Zhenda Xie. Haowei Zhang. Liang Zhao, et al. Janusflow: Harmonizing autoregression and rectified flow for unified multimodal understanding and generation. arXiv preprini 2rXiv:2411.09975 2024.
[23] Chameleon Team. Chameleon: Mixed-moda early-fusion foundation models.
arXiv preprint
arXiv:2405.098/3 2024.
[24] Xinlong Wang. Xiaosong Zhang, Zhengxiong Luo, Quan Sun, Yufeng Cui, Jinsheng Wang. Fan Zhang, Yueze Wang. Zhen Li, Qiying Yu, al. Emu3: Next-token prediction all you need. arXiv preprint arXiv:2409.1888 18869, 2024.
[25] Yecheng Wa, Zhuoyang Zhang, Junyu Chen, Haotian Tang, Dacheng Li, Yunhao Fang, Ligeng Zhu, Enze Xie, Hongxu Yin, Li Yi, et al. Vila-u: a unified foundation model integrating visual understanding and generation. arXiv preprint (tt:22009.042299 2024.
[26] Jinheng Xie, Weijia Mao. Zechen Bai, David Junhao Zhang. Weihao Wang, Kevin Qinghong Lin, Yuchao Gu. Zhijie Chen, Zhenheng Yang. and Mike Zheng Shou. Show-o: One single transformer to unify multimodal understanding and generation. arxiv preprint arXiv:2408. 12528, 2024.
[27] Chuyang Zhao, Yuxing Song, Wenhao Wang. Haocheng Feng, Errui Ding, Yifan Sun, Xinyan Xiao, and Jingdong Wang. Monoformer: One transformer for both diffusion and autoregression. arXiv preprint arXiv:2409. 16280, 2024.
[28] Shitao Xiao, Yueze Wang, Junjie Zhou, Huaying Yuan, Xingrun Xing. Ruiran Yan, Chaofan Li, Shut- ing Wang. Tiejun Huang, and Zheng Liu. Omnigen: Unified image generation. arXiv preprint arXiv:2409. /1340, 2024.
[29] Xiaokang Chen, Zhiyu Wu, Xingchao Liu, Zizheng Pan, Wen Liu, Zhenda Xie, Xingkai Yu, and Chong Ruan. Janus-pro: Unified multimodal understanding and generation with data and model scaling, arXiv preprint ar Xiv: 17811, 2025.
[30] Toran Bruce Richards. AutoGPT, http::///ggtthh....oo////aaaaaaa
2023.
[31] Qingyun Wu, Gagan Bansal, Jieyu Zhang, Yiran Wu, Beibin Li, Erkang Zhu, Li Jiang, Xiaoyun Zhang. Shaokun Zhang, Jiale Liu, et al. Autogen: Enabling next-gen Ilm applications via multi-agent conversation. arXiv preprint arxiv:2300.. 08155, 2023.
[32] Tianbao Xie, Fan Zhou, Zhoujun Cheng, Peng Shi, Luoxuan Weng, Yitao Liu, Toh Jing Hua, Junning Zhao, Qian Liu, Che Liu, et al. Openagents: An open platform for language agents in the wild, arXiv preprint 2r1iv:3300 10634, 2023.
[33] Shunyu Yao. Jeffrey Zhao, Dian Yu, Nan Du Izhak Shafran, Karthik Narasimhan, and Yuan Cao. Re- act: Synergizing reasoning and acting in language models. In International Conference on Learning Representations (ICLR), 2023.
[34] Sirui Hong. Xiawu Zheng, Jonathan Chen, Yuheng Cheng, Jinlin Wang. Ceyao Zhang, Zili Wang. Steven Ka Shing Yau, Zijuan Lin, Liyang Zhou, et al. Metagpt: Meta programming for multi- agent collaborative framework. arxiv preprint 23l.:23080003522 3(4),6, 2023.

---

[35] Black Forest Labs. Flux. Min.I-redux-eer 2024. URL htt.:///e.//////gggnotoooooooood FLUX. 1-Reux-de
[36] Chaojie Mao, Jingfeng Zhang. Yulin Pan, Zeyinzi Jiang, Zhen Han, Yu Liu, and Jingren Zhou. Ace++: Instruction-based image creation and editing via context-aware content filling. arXiv preprint arXiv:2501.0245) 2025.
[37] Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C Berg. Wan-Yen Lo, et al. Segment anything. In 2023 IEEE/CVF International Conference on Computer Vision (ICCV), pages 3992-4003. IEEE, 2023.
[38] Nikhila Ravi, Valentin Gabeur, Yuan-Ting Hu, Ronghang Hu, Chaitanya Ryali, Tengyu Ma, Haitham Khedr, Roman Rädle, Chloe Rolland, Laura Gustafson, et al. Sam Segment anything in images and videos. arXiv preprint arXiv: 00714, 2024.
[39] Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik R Narasimhan, and Yuan Cao. React: Synergizing reasoning and acting in language models. In The Eleventh International Conference on Learning Reprexentations.
[40] Dhruba Ghosh, Hannanch Hajishirzi, and Ludwig Schmidt. Geneval: An object-focused framework for evaluating text-to-image alignment. Advances in Neural Information Processing Systems, 36:52132-52152, 2023.
[41] Tom Brown, Benjamin Mann, Nick Ryder, Melanic Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. Advances in neural information processing systems, 33:1877-1901, 2020.
[42] Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc Le, Denny Zhou, et al. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824 2483-2287722 2022.
[43] Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoe Le, Ed Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. Self-consistency improves chain of thought reasoning in language models. arXiv preprint 11X1::2003.11171, 2022.
[44] Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björ Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEF/CVF conference on computer vision and pattern recognition, pages 10684-10695, 2022.
[45] Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hicrarchical text-conditional image generation with clip latents. arXiv preprint 2rXiv:2204.06125, 1(2):3, 2022.
[46] Peize Sun, Yi Jiang, Shoufa Chen, Shilong Zhang, Bingyue Peng, Ping Luo, and Zehuan Yuan. Autore- gressive model beats diffusion: Ilama for scalable image generation. arXiv preprint 2665522406.06525 2024.
[47] Hao Liu, Wilson Yan, Matei Zaharia, and Pieter Abbcel. World model on million-length vidco and language with ringattention. arXiv e-prints, pages arX2v-2402, 2024.
[48] Tsu-Jui Fu, Wenze Hu, Xianzhi Du, William Yang Wang, Yinfei Yang, and Zhe Gan, Guiding instruction- based image editing via multimodal large language models. arXiv preprint 2rX9v 2023.
[49] Kai Zhang, Lingbo Mo. Wenhu Chen, Huan Sun, and Yu Su. Magicbrush: A manually annotated dataset for instruction-guidee guidedd image cditing. Advances Neural Information Processing Systems, 36:31428-31440 2023.
[50] Yuwei Niu, Munan Ning. Mengren Zheng, Bin Lin, Peng Jin, Jiaqi Liao, Kunpeng Ning. Bin Zhu, and Li Yuan. Wise: A world knowledge-informed informed semantic evaluation for text-to-imape generation. arXiv preprint arXh::2503.07266 2025.
[51] Junsong Chen. Jincheng Yu. Chongjian Ge. Lewei Yao, Euze Xie, Yue Wu, Zhongdao Wang, James Kwok, Ping Luo, Huchuan Lu, et al. Fast training of diffusion transformer for photorealistie text-to-image synthesis. arXiv preprint 23000233 2023.
[52] Daiqing Li, Aleks Kamko, Ehsan Akhgari, Ali Sabet. Linmiao Xu, and Suhail Doshi. Playground v2. 5: Three insights towards enhancing aesthetic quality in text-to-image generation. arXiv preprint arXiv:240..17245 2024.
12

---

[53] Siqi Kou. Jiachun Jin, Zhihong Liu, Chang Liu, Ye Ma, Jian Jia, Quan Chen, Peng Jiang, and Zhijie Deng. Orthus: Autoregressive interleaved image-text generation with modality-specifie heads. arXiv preprint urXiv:2412.00127. 2024.
[54] Zhenyu Wang, Aoxue Li, Zhenguo Li, and Xihui Liu. Genartist: Multimodal Ilm agent for unified image generation and editing. Advances Neural Information Processing Systems, 37:128374-128333 128395, 2024.
13

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: Yes. We provide detailed experimental evidence supporting these claims Sec. and
Appendix A
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be pereeived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals motivation long it is clear that these goals attained by the paper.
2. Limitations
Question: Does paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: Yes. discuss the limitations Appendix
Guidelines:
The answer NA means that paper has no limitation while answer means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust results are violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect the scope of the claims made, e.g., if the approach was only tested a few datasets with few runs. In general, empirical results often depend implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
grounds for rejection,
Theory assumptions and proofs
Question: For each thcoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
Justification: The paper presents a system-level framework without formal theoretical results or proofs.
Guidelincs:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross-referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
14

---

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification: Yes. We use publicly available benchmarks description of method in Sec. and Appendix C
40
16
6
50
and provide detailed
Guidelines:
The answer NA means that the paper does not include experiments.
the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example. if the contribution is novel architecture, describing the architecture fully might suffice, if the contribution is specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to hosted model (e.g., in the case of a large language model), releasing of model checkpoint, or other means that appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submissions provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily new algorithm, the paper should make it clear how to reproduce that algorithm.
(b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (e.g.. a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g.. with open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, which case authors are welcome to describe the particular way they provide for reprodueibility. In the case of closed-source source models, it may be that access to the model is limited in some way (c.g.. registered users), but it should be possible for other rescarchers have some path reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code. with sufficient instructions to faithfully reproduce the main experimental results, described in supplemental material?
Answer: [Yes]
Justification:
The code has been released on the GitHub page: https://gtt com/
BaVisson--Researcch/Ce Caasearrch/ComtyMin
EnVision
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines htttpp://niip.cc//puee
Saiddes/CoeSubbni
for more details.
While we encourage release of code and data, we understand that this might not possible,
so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g.. for a new open-soure benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https: : //:/://nppp cc/public/ out.dess/CooeeSttmmiessssoooooo for more details.
15

---

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from script and why.
At submission time, preserve anonymity, authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g.. data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: Yes, Please see Sec. 3 4 and Appendix A
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, supplemental material.
Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate informa- tion about the statistical significance of the experiments?
Answer: [No]
We evaluated method four publicly available datasets. Due to the high computa- tional cost of running repeated trials across all benchmarks, we did report statistical significance metrics. However, the performance trends are consistent and clearly support our main claims.
Justification:
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, overall run with given experimental conditions).
The method for caleulating the error bars should be explained (closed form formula, call to library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report I-sigma error bars, but one should state it. The authors should preferably report 96% CI. if the hypothesis of Normality of errors
a 2-sigma error than state that they not verified.
asymmetric distributions, authors should be careful not show in tables or figures symmetric error bars that would yicld results that are out of range (c.g. negative error rates). error bars are reported tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Answer: [Yes]
Justification: Yes. We provide related information for the computer resources
in Appendix C
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
16

---

The paper should provide the amount of compute required for each of the individual experimental well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn 't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics
Answer:
[Yes]
Justification: Yes. followed the NeurlPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurlPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (c.g., if there is a special consideration due to laws regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive socictal impacts and negative socictal impacts of the work performed?
Answer: [No]
Justification: This paper focuses on academic research, does not involve any private data, and has no explicit negative socictal impact.
Guidelines:
The answer NA means that there is no socictal impact of the work performed.
the authors answer NA or No, they should explain why their work has societal impact or why the paper does not address societal impact.
Examples negative societal impacts include potential malicious or unintended uses (e.g.. disinformation, generating fake profiles, surveillance). fairness considerations (e.g., deploy- ment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
there are negative societal impacts, the authors could also discuss possible mitigation strategies (c.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitor- ing misuse, mechanisms to monitor how system Icarns from feodback over time, improving the efficiency accessibility ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have high risk for misuse (e.g., pretrained language models, image generators, scraped datascts)?
Question:
Answer: [NA]
Justification:
We not release any models or datasets that pose a risk misuse.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use usal-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines restrictions to access the model or implementing safety filters.
17

---

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but encourage authors take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators original owners of assets (e.g., code, data, models), used in the paper, properly credited and the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: Yes. We credited all assets properly and respected their licenses.
Guidelines:
The answer NA means that the paper does not existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include a URL.
The name of the license (c.g., CC-BY 4.0) should be included for cach asset. scraped from particular source (e.g.. website), the copyright terms of service of that source should provided.
assets are released, license, copyright information, and terms of use the package should be provided. For popular datasets, paperswi thcode. www.coom//aaac /attttttstts has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.
For existing datasets that are re-per re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
this information not available online, the authors are encouraged to reach out to the asset's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: The paper does not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the datasct/code/model as part of their sub- missions via structured templates. This includes details about training, license, limitations,
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an amonymized URL or include anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsoureing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, well as details about compensation (if any)?
Answer: [NA]
Justification: The paper does not involve crowdsoureing nor research with human subjects
Guidelincs:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should included the main paper.
According to the NeurIPS Code of Fthics, workers involved data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals equivalent for research with human subjects
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Question:
18

---

Answer: [NA]
Justification:
The paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer means that paper does not involve crowdsourcing nor research with human subjects.
Depending on the country which research is conducted, approval (or equivalent) may be required for any human subjects research. If you ohtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors adhere to the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applica- ble), such the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe usage LLMs it an important. original. or non-standard component the core methods this research? Note that if the LLM used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the rescarch, declaration is not required.
Answer: [Yes]
Justification: We use LLMs as part of our core methodology. Its role is described in Sec. and Appendix
Guidelines:
The NA means that the method development in this research does not involve LLMs as any important, original, or non-standard components.
Please refer our LLM policy (https: http::////auurgeccc//////errrrep for what should or should not be described.
19

---

A Supplementary Experiments
A.1 Supplementary Ablation Study
A.1.1 Ablation Study on Different LLMs
Table Ablation results of Comfy Mind with different LLMs.
Vanilla
Complex
Creative
Total
Agent
Paass
&Rcsolve
&ass
&Resolve
%Pass
&Resolve
&Pass
%Resolve
ComfyMind with GPT-4o
100.0
92.0
100.0
85.0
100.0
57.5
100.0
83.0
ComfyMind with Deepseek-V?
100.0
90.0
100.0
71.7
100.0
60.0
100.0
78.5
To further demonstrate the robustness of our system, we conducted evaluations on the Comfy JomfyBench benchmark using different LLMs in ComfyMind. As shown in Table [ both Deepseek-V2 V3 and GPT-4o achieve strong performance when employed the primary LLMs. Specifically, both models reached 100% task pass and approximately 80% overall task completion rate. These results further confirm stability and reliability of our system across different underlying LLMs.
A.1.2 Ablation Study on Feedback Granularity
To verify the effectiveness of the proposed localized feedback mechanism, we conduct an ablation study on ComfyBench, investigating the impact of different feedback granularity levels on task performance.
We design three feedback granularity levels: Judgment contains only binary success/failure signals; Mudgment+Anallstt adds outcome explanations specifying reasons for success failure; and Judg- Anallssnalyiss Analysist+Dssrription (Ours) includes detailed image-grounded information such as completed progress, visual observations, and suggestions for plan correction.
Three task types (Vanilla, Complex, Creative) are evaluated, with the success rate as the core metric. The ablation results are shown in Table
Table 5: Ablation results of different feedback granularity levels.
Feedback Type
Vanilla f
Complex 1
Creative t
Total 1
Judgment
89.0
58.33
47.5
71.5
Jndgsent+Analysis
91.0
66.67
55.0
76.5
(Ours)
92.0
85.0
57.5
83.0
As shown Table 5 task performance steadily improves with increasing feedback granularity. The binary Judg- ment feedback yields the lowest overall success rate (71.5%), while incorporating analytical explanations (Judg- AnalsAna clevates the total rate to 76.5%. Our full full-granullrity feedhack (Indgment-Anmllyiss-Deerr maalyyisDDserrpptinn outperforms all variants across task types, particularly in Complex tasks (85.0%, 18.33 pp higher than Judg- ment), confirming that richer image-grounded feedback provides the planning agent with actionable insights for informed decision-making. Notably, our search tree design mitigates context dilution at each planning node, ensuring fine-grained feedback remains effective without additional computational overhead- -vaiiaatin the rationality of our localized feedback implementation.
A.2 World knowledge-Informed Semantic Synthesis
To assess our system's capabilities in complex semantic understanding, reasoning, and world knowledge integration for text-to-image generation, conduct evaluation the recent WISE [50] benchmark. This
benchmark contains three primary categories: cultural commonsense, spatiotemporal reasoning (including Space and Time subcategorics), and natural sciences (comprising Physics, Chemistry, and Biology subfields), totaling 25 specialized domains with 1,000 challenging prompts.
The evaluation metric WiScore combines Consistency, Realism, and Aesthetic Quality through weighted normalization, with a maximum score of 1. Higher WiScore indicates stronger capability in accurately depicting objects and concepts using world knowledge. As shown in Table 6 our method achieves superior score of 0.85, surpassing all models, including GPT-Image-I (0.80). Our approach significantly enhances world knowledge integration for open-source solutions, outperforming FLUX.1-dde (0.50) by 0.35 points and enabling open-source models to match GPT Imagee i's performance. The exceptional performance on WISE confirms our system's 's generalizability and high-quality output in generative tasks.
20

---

Table 6: Evaluation of World Knowledge-Informit Semantic Synthesis on WISE [50] Benchmark.
Method
Cultural
Time
Space
Biology
Physics
Chemistry
Overall
Dedicated 121 Models
FLUX.1-dev B
0,48
0.58
0.62
0,42
0.51
0.35
0,50
PLUXX.-schneel
0.39
0.44
0.50
0.31
0.44
0.26
0.40
PixArt-Alpha 51
0.45
0,50
0.48
0.49
0.56
0.34
0.47
playground-v2.5 52
0,49
0.58
0.55
0,43
0.48
0.33
0,49
SDv 44
0.34
0.35
0.32
0.28
0.29
0.21
0.32
SDv2.1
0.30
0.38
0.35
0.33
0.34
0.21
0.32
3D-XL-base-0.9
0,43
0.48
0.47
0,44
0.45
0.27
0,43
SD3-Medium 2
0.42
0.44
0.48
0.39
0.47
0.29
0.42
SD3.5-Medium
0.43
0.50
0.52
0.41
0.53
0.33
0.45
SD3.5-Large
2
0.44
0.50
0.58
0.44
0.52
0.31
0.46
Unify MI.MM Models
Emu3 24
0.34
0.45
0.48
0,41
0.45
0.27
0.39
Jamus-1.33
0.16
0.26
0.35
0.28
0.30
0,23
JamusFlow-I.33
122
0.13
0.26
0.28
0.20
0.19
0.11
0.18
Janus-Pro-1B
(29
0.20
0.28
0.45
0.24
0.32
0.16
0.26
Janux-Pro-7B
22
0.30
0.37
0.49
0.36
0.42
0.35
Orthux-7B-base
0.07
0.10
0.12
0.15
0.15
0.10
0.10
Orthus-7B-instruet
53
0.23
0.31
0.38
0.28
0.31
0.20
0,27
show-o-demo
26
0.28
0.36
0.40
0,23
0.33
0.22
0.30
show-o-demo-511
0.28
0.40
0.48
0.30
0.46
0.30
0.35
vila-u-7b-256
25
0.26
0.33
0.37
0.35
0.39
0.23
0.31
GPT-Image-1
13
0.81
0.71
0.89
0.83
0.79
0.74
0.80
Collaborative AI Systems
Ours
0.90
0.79
0.92
0.77
0.79
0.82
0.85
A.3
Quantitative Evaluation for Planning Strategy
To verify the effectiveness of the proposed planning method, we conduct quantitative comparisons with two representative planning strategies on the ComfyBench benchmark.
ReAct is reactive planning strategy that dynamically adjusts action sequences based on real-time observations without global task decomposition or stable intermediate decision mechanisms. Planning- Act is canonical multi- agent planning framework that adopts "root-node full planning recursive subtask exccution" and maintains fixed plan once plan-act cycle initiates.
We evaluate three core metrics: Resolve Rate, Repetition Rate, and Average Time per Task, which respectively reflect a system's 's problem-solving capability, stability of exploration, and efficiency. Resolve Rate measures the success rate in solving compositional tasks. Repetition Rate captures the frequency with which atomic workflows redundantly selected within single task. Time represents the average execution time per task.
Table 7: Quantitative comparison with baseline methods ComfyBench.
Method
Resolve Rate %
Repetition Rate %
Time (s) 1
ReAct
54.0
34.59
257, .43
Planning- Act
66.0
28.24
279.79
Ours
83.0
8.70
225.12
As shown in Table method achieves the best performance across all metrics. ReAct's low Resolve Rate and high Repetition Rate from its lack of global planning, leading inefficient trial-and-error. Planning-Act's rigidity (fxed plan) and redundant workflow selection (absence stable intermediate decision nodes) result suboptimal Resolve Rate and the longest execution time. In contrast, our method mitigates strategy oscillations and enhances efficiency by integrating global planning with real-time adaptability and stable intermediate decision nodes, outperforming baselines in both effectiveness and behavioral stability.
21

---

A.4 Supplementary Experiment on Inference Latency
To justify the inference latency trade-off and verify the accompanying performance gains, we conduct supple- mentary experiments on Geneval, comparing our system with baseline atomic workflows (SD3.5-L, FLUX.1 Dev) that form its foundation. The comparative results of inference time and generation performance are shown in Table
Table Inference latency and performance comparison on Geneval.
Model
Time (9)
Overall 1
Single Obj.
Two Obj.
Counting
Colors
Position 1
Attr. Binding
SD3.5-L 1.
47.06
0.71
0.98
0.89
0.73
0.83
0.34
0.47
FLUX.1 Dev
56.51
0.66
0.98
0.81
0.74
0.79
0.22
0.45
Ours
119.94
0.90
1,00
1.00
1.00
0.97
0.62
0,80
Although our system incurs additional time due to planning, evaluation, and possible regeneration, it achieves sig- nificantly higher overall scores. The performance gain justifies the roughly 2x increase in runtime, demonstrating that system pushes the boundary of what current models can accomplish.
B
Additional Results and Generation Examples
As an extended demonstration, we present ComfyMind's exemplary outputs across diverse generative tasks in Figures 7 B and 2 substantiating its versatile adaptability and superior performance in cross-domain generation scenarios.
C
Implementation Details
C.1 System Architecture
Logically, the system is divided into two independent subsystems: the ComfyUl server and automation control system. Both are deployed on the same physical server but operate within isolated runtime environments and listen on separate ports. This design ensures robust separation and scalability in task concurrency handling and resource management. The server is equipped with NVIDIA RTX A6000 GPU with 48GB of VRAM, providing sufficient computational capacity for generation tasks.
Within the system architccture, the ComfyUl platform is responsible for exccuting atomic workflows. It deployed as a local service that listens on a designated port and provides dual interaction interfaces. one hand, it exposes web interface through which users can access the frontend via a browser to edit nodes, configure parameters, and perform real-time executions. On the other hand, Comffy UI incorporates both HTTP and WebSocket protocols, supporting standardized RESTful API task submissions and result retrieval, while enabling real-time progress tracking via WebSocket. this study, latter interface primarily utilized. Technically, the client loads an adapted workflow in ISON format, submits a generation request via the HTTP API, and establishes a persistent WebSocket connection to receive real-tin time exccution updates and final results. Built atop the ComfyUl execution layer, ComfyMind functions the high-level planning and task management system, responsible for process orchestration, scheduling, and result evaluation. is developed based the ComfyAgem framework.
C.2 Compilation of Atomic Workflow Library and Descriptive Documents
To ensure adaptability across multimodal generation and editing tasks, a diverse set of atomic workflows systematically collected, curated, and organized form a foundational resource for general- general-purpose purpose content generation. These workflows were sourced through a combination of selective acquisition and custom development, aiming to balance stability and broad applicability.
First, standard workflows from the ComfyBence Bench benchmark suite were systematically tested. Suboptimal examples were discarded to ensure overall quality. Sccond, a curated selection of high-quality workflows was extracted from the official ComfyUl website and documentation to meet common generation needs. Additionally, workflows with strong user ratings and proven effectiveness Civitai and OpenArt, enhancing the practicality and robustness of the workflow library.
sourced from popular communities such
Beyond these standardized resources. functional workflows tailored complex reasoning tasks were developed by leveraging the flexibility of ComfyMind, in conjunction with LLMs and VLMs. These workflows include capabilities for reason generation, reason editing, and prompt enhancement, supplementing limitations in the standard resource base.
22

---

Text to Image G eneration
(1) Generate an image of cat sitting on windowsill looking outside
(2) Generate an image of a small village covered in snow with smoke coming from chimneys
(3) G Generate an image of a hot air balloon floating over a scenic valley at sunrise
Reasoning Image G Generation
(1) Generate an image of Most representative craft of India
(2) Generate image of A massive stone statue of mythical creature that is a prominent historical landmark in Egypt
(3) Generate an image of Marshmallow over a bonfire
Figure 7: More examples generated by ComfyMind
23

---

Image Editing
(1) Y ou are given an image icccream.jpp .ppg" Please according to the reference image, generate coffee with ice cream version of the ice cream.
(2) Based on the given reference images train.jpg'' replace the train in the image with High-speed train
(3) Based on the given reference images dinner.jgg remove the knife and fork in the image
(4) Based on the given reference images castle.jpg', replace the castle in the image with Chineses traditional temple
(5) Generate an image of a 'cherris.jpg'' convert the image into a version of advertisement, is placed on the exhibition stand with advanced lighting and color matching.
(6) Y ou are given an image 'man.jp'.. Generate another photo to show the man an elderly version of himself.
Figure More examples generated by ComfyMind
24

---

Style Transfer
(1) Based on the given reference image, generate an image describing A little girl is rowing a boat in downtown park, with dog on board. In the foreground, willow trees surround the park. In the middle ground, Ferris wheel and a carousel from the amusement park are visible. In the background, skyscraped can be seen.
(2) Based the given masked reference image 'tower mask.png generate an image: A tower glowing with lanterns during a traditional night festival, with fireworks lighting up the distant sky and ancient trees surrounding the festive square.
Image Refine
(3) Y ou are given an portrait image with wrong generated hands' abnormal hands.png. Please refine the hands to make them look more natural.
Image O utpaint
(4) Based on the given reference images 'deerjpg', outpaint the image in four directions with all 512 pixels, and the prompt is: Standing on the green grass, the deer's body is bathed in golden glow from the setting sun.
Image Merge
(5) Based on the given two reference images pyramid.jpg and PiffelTower.jjug generate merged image
FIN
F Face Swap
(6) "B asse on the given reference images pportrii.jpg' and 'face reference.jpg' swap the faces of the two images
Figure More examples generated by ComfyMind
25

---

C.3 Supplementary Description of the ComfyMind Method
C.3.1 Semantic Workflow Interface Details
To enable SWI, unified semantic description schema was developed for all atomic workflows. The system only requires single, consistent atomic workflow description document to support all reasoning tasks. This document employs hybrid format combining natural language and standardized parameter interfaces to specify each workflow's 's core functionality, input/output requirements, and optional extensions.
The core structure includes: the required workflow name, the corresponding prompt and visual input. Optional components include supplementary semantic constraints such as resolution, frame rate, or upscaling ratio. This modular, uniform documentation enables semantic-level workflow selection and parameter matching during reasoning, without requiring parsing of underlying node structures structures-greatly improving planning efficiency and ease expansion or maintenance.
C.3.2 Execution Agent Details
The execution agent in the ComfyMind system plays a crucial role in bridging high-level planning and low-level execution. Once the planning agent completes the decomposition of a subtask and outputs set of invocation instructions, the exccution agent is activated to handle parameter processing, workflow adaptation, execution, and result feedback. The overall logic adheres to the fundamental principle: "if execution succeeds, proceed the next subtask; if it fails, return to the current node for local backtracking." This ensures tight integration between planning and execution.
Before initiating the execution process, certain specialized semantic workflow interface functions may require a preprocessing step. In such cases, the execution agent invokes LLMs to perform deep reasoning and semantic expansion on the user's input prompt or initial task description. For example, if the user provides command like "light passes through a prism,, " the system leverages the LLM's knowledge to infer related phenomena such as spectral dispersion and colorful light bands, thereby optimizing the prompt and providing richer and more accurate input conditions for the subsequent generation process.
Next, the execution agent parses the invocation parameters passed down from the planning agent, loads the corresponding atomic workflow template, and fills in placeholders with information such as the prompt text and input visual data paths. This results in fully instantiated and executable workflow. To better adapt to the task's specific requirements, the execution agent further adjusts workflow parameters, such as image resolution. sampling steps, and output frame rate. This adaptive tuning is performed at the parameter level without altering the underlying DAG structure of the atomic workflow, thus preserving the correctness of node connections and ensuring execution stability.
Once adaptation is complete, the execution agent submits the workflow to the ComfyUl platform through a standardized interface and establishes real-time connection to monitor generation progress and status updates. After task exccution finishes, the system automatically parses the generated results. uses visual-languse understanding module to extract semantic descriptions, detailed information, and scene characteristics from the outputs, synchronizing them with the current workspace state. This update provides the most recent context for subsequent subtask reasoning, maintaining continuity and coherence in the reasoning chain.
If the entire process completes successfully, the execution agent returns success signal and hands off the updated workspace state to the next planning node. If any exceptions occur during execution such parameter mismatches, insufticient generation quality, or timeouts the execution agent returns a detailed failure reason and log information. This allows the planning agent to perform localized search backtracking and re- re-plannig. Through this design, the execution agent effectively ensures seamless integration from high-level semantic instructions to concrete execution, while offering strong adaptability and robust error handling, significantly enhancing the system's overall stability and reasoning flexibility.
C.3.3 Evaluation Agent Details
The Evaluation Agent is responsible for assessing the quality of generated results and their consistency with the given instructions, cnsuring that system outputs meet expected requirements. When the overall process concludes, the system invokes the Evaluation Agent to inspect the generated content against a unificd standard and relay the results back to the higher-level planning module.
In its design, the Evaluation Agent adopts the classification based assessment strategy used by ComfyAgent dividing workflows according to task types such as text-to-image, image-to-image, text-to-video, image-to-video, and video-to-video, with tailored evaluation criteria for each category. Evaluation is conducted along two main dimensions: the quality of the generation such as clarity, richness of detail, and visual naturalness and adherence to instructions, verifying whether the output accurately reflects the user's intended theme, setting, or stylistic directives. Each evaluation returns two components: binary judgment indicating whether the current task is considered complete, which guides the system on whether to proceed or backtrack; and detailed failure
26

---

analysis when evaluation is unsuccessful. explicidl identifying reasons such as lack of detail, semantic deviation, stylistic inconsistency, thereby aiding the Planning Agent in refining subsequent inference steps.
To accommodate varying tasks and application requirements, the Evaluation Agent supports dynamic thresh- old adjustment. The system can raise or lower evaluation standards depending on task complexity, quality expectations, or usage scenarios, enabling flexible balance between content quality and exccution efficiency.
D Discussion
GenArtist [54] has made notable advancements in automating complex visual generation leveraging large-scale models. Given the inherent similarity between GenArtist and our work, it is essential to clarify their differences and positioning. Our work and GenArtist share common vision: addressing the challenges posed by complex instructions via modular paradigm. A fundamental divergence in objectives exists between the two works: GenArtist focuses on image-centric tasks, whereas our research aims to develop a scalable and intelligent framework tailored for the more ceneral-purpost, ComfyUU-centric open spen-source ccosystem. This divergence in objectives translates to distinct technical abstractions. GenArtist adopts tool-centric planning paradigm, while core innovation the "Semantic Workflow enables the system to perform planning directly using community-validate high-level workflows. This difference in abstraction levels further dictates contrasting failure-hande hhauurrn-nnningggg paradigms. GenArtist employs an efficient "local repair" strategy, utilizing editing tools to rectify flawed image outputs. In contrast, our feedback- decision loop offers enhanced flexibility: based on generated feedback, it can fine-tur tune the current workflow (e.g.. prompt refinement and parameter adjustment), strategically replace it with successful execution. We argue that this adaptive planning and exccution paradigm critical for addressing the complex, multi-modal tasks targeted our work, and contributes enhanced robustness and scalability to broader open-source community.
alternative workflow upon failure, or proceed to the subsequent step upon
E
Limitation
Although ComfyMind supports modular workflow composition and automated planning, the current system lacks a user-friendly interface for manually customizing or modifying the sequence of atomic workflow invocations. Users have limited ability to adjust planning strategies, override intermediate steps, specify task-specific preferences through the UI. This may hinder broader adoption among non-technical users practitioners with speciffc needs. Enhancing interface support more flexible user-controllable planning customization is an important direction for future development.
F
System Prompt
This section outlines the important system prompt in ComfyMind, defining agent behavior, response style, and task boundaries, as demonstrated by Figures To Figures 19
27

---

Preprocessing
# #Objective:
Determine prompt optimization is needed:
If the task Do not have reference image video, the prompt must be optimized(e.g. Generate 2- second video of a river flowing through a valley with mountains in the background. The result should be high-quality video.). If the task has reference image or video, do not optimize the prompt.
# Prompt Optimization Guidelines
## For T21 (Text-to- Imaae))) well-optimized prompt follows this structure: Promet Subject + Scene + Style + Camera Language A tmosphere + Detail Enhancement
Subject: Clearly define the main subject, including characteristics, appearance, and actions. Example: "A charming 10-year-old old Chinese girl, wearing bright red dress, smiling under the sunlight."
Scene: Describe the environment, background elements, and setting. Example: "A bustling ancient Chinese market, filled with vibrant lantems and merchants selling silk and spices."
Style: Specify an artistic style or visual treatment (see Style D Dictionary below). Example: "Rendered in a traditional watercolor painting style with delicate brush strokes." "
Camera Language: Define shot type, angles, and movement (see Camera L Language Dictionary). Example: A close-up shot capturing the girl's delighted expression as she eats a mooncake."
A tmosphere: Convey the mood and emotional tone (see tmosphere Dictionary). Example: "Warm and nostalgic, evoking a sense of childhood happiness."
Detail Enhancement: A dd refined details to enrich the composition. Example: "Soft golden light filtering through the hanging lantern,, creating an ethereal glow."
## For T2V 12V (Text- to V Vieoo Image-to-Videee well- well-optimized prompt follows this structure:Prompt Subject Scene Motion Camera Language A tmosphere Style
"A black- haired Miao ethnic girl, dressed in traditional embroidered attire, adorned with silver jewelry that reflects sunlight."
Subject: Describe the main character or object with specific attributes.
black-haired
Scene: Define the background, setting, and environmental elements. elements.Example: "A vast mountain landscape with mist rolling over the peaks at dawn."
Motion: Describe movement speed, style, and effect. effect. Example: "She gracefully spins, her silver jewelty jingling softly with each movement." "
Camera Language: Specify shot type, camera angles, and motion tracking (see Camera Language Dictionary) Example: "A smooth tracking shot following her dance, shifting from low-angle close- up to a sweeping wide shot."
Figure 10: System Prompt for Preprocessing, Part 1
28

---

Preprocessing
A tmosphere: Define the mood and ambiance (see A tmosphere (atiionary).xxmmplee "Serene and majestic, evoking deep connection cultural heritage""
Style: Choose distinct visual artistic style Style (rrtoonay).Exxapllee "A cinematic style with a soft golden hue, enhancing the mystical feel of the scene."
#* Final Prompt
Finally, combine all the elements Subject, Scene, Motion, Camera Language, A Amooppere, Style) into one paragraph.
## Prompt Dictionary
1. Camera Language
Shot Types (Framing):
Close-up Shot: Captures fine details, expressions, objects in high focus. Exampee "A close-up of an old scholar's hands delicately flipping the pages ancient manuscript."
Medium Shot: Shows the subject from the waist up, pmviding knight. in battle-wom aroor standing before buming castle." "
context ContextExample "A medium shot of
Wide Shot (Long Shot); Caplures the subject fully within a vast environment environment.Examplle Example: "A lone traveler walking across an endless desert. under a blood-red sunset."
Bird's Eye View (Overhead Shot): Provides top- down perspective for dramatic effect. Example: "A bird's eye view of cyberpunk city illuminated by neon signs and holograms."
Camera Motion Bechiqque::oolyy (Push-in Shot): Gradually moves closer to intensify focus. Example: "The camera slowly pushes towards crying soldier, emphasizing his sorrow."
Pull-out (Zoom-out Shot): Moves backward to reveal a larger scene, Example: zoom out shot transitioning from a painter's brushstroke to reveal grand Renaissance artwork."
360- 5:00Degree Rotation (Orbit Shot): Encircles the subject for a dramatic effect, Exammle: "A 560-degree shot around a warrior as he stands amidst a baltlefield, flames and debris flying around him."
Tracking Shot (Follow Shot): Follows subject in motion dynamically Example: "A tracking shot following dancer through dimly lit theater, capturing each step and gesture."
2. tmosphere (Mood & Emotion)
Energetic/ Joyful/ Uplifting: Bright lighting, vibrant colors, and lively movement Example: "A lively marketplace where childron laugh and vendors showcase colorful handmade goods under warm sunlight."
Dreamlike/ Surreal Mystical: Soft focus, floating claments, ethereal lighting. Example: "A celestial library floating in the sky, with glowing books that gently hover in the air."
Figure 11: System Prompt for Preprocessing, Part 2
29

---

Preprocessing
Lonely/ Melancholic/ Quiet Muted tones, slow movement, and vast empty qpaces.Example "A lone figure sitting on swing in an abandoned park under cloudy sky."
Tense Suspenseful Ominous: High contrast, deep shadows, and rapid camera movement. movement.Example: "A flickering streetlamp illuminates dark alley as footsteps echo ominously in the distance."
Majestic/ / Grand / Awe- ineeiiipiing:: Sweeping wide shots, dramatic lighting, and grand compositions. Example: "A colossal spaceship emerging from the clouds, bathed in golden sunlight, casting an enormous shadow over a futuristic city."
3. Style (Artistic Direction)
Cyberpunk: Neon lights, dark cityscapes, high tech elements. Example: "A hacker in hooded jacket, surrounded glowing holographic data streams a futuristic Tokyo street."
Post-A A pooallppic (W asteland Style): Rugged, destroyed environments, muted colors. Example lone wanderer in tattered clothes walks through desolate wasteland, canying a rusted rifle."
Traditional Chinese Painting (Guofeng)) Ink wash, delicate linework, soft color palettes Exampe: "A scholar flowing robes sitting under an ancient pine tree, gazing at distant misty mountains.
Felt A nimation Style: Soft, handmade textures, childlike charm. Example: "A woolen puppet caaaracterooyullly baking cookies in miniature kitchen." "
Classic Art Inspired: Mimics famous artworks like Van Gogh, Rembrandt, Ukiyo-e.Example: "A modemn city paintod in the swirling brushstrokes Gogh's 'Starry Night!." "
# Oulput
If optimization is not required, only None output. optimization is required, the output only includes the optimized prompt, which is wrapped by <oplimized _proppt>ppppppp <loplimized prompt>
Figure 12: System Prompt for Preprocessing, Part 3
30

---

Planning
# Role
Y ou are advanced W orkfloww Selection A gent. Y our task is carefully plan sequential use of workflows based the current context. This sequence represents the steps required to complete the task by executing workflows order. Y need Think Step by Step.
# Instructions
Strictly follow these guidelines:
1. Carefully analyze the current input and search history.
2. Evaluate the applicability of available workflows.
3. *Must* consider previous failed workflow attempts workflows that have been recorded as incapable of completing task should not be selected again.
4. Apply logical reasoning: if workflow has been recorded as failing for follow-up up task (not the current one due to possible workflow errors), selecting similar workflow should be done with caution.
5. Achieving the desired result is the top prionty! If there suitable workflows to proceed with the task, but reordering the workflow sequence can help, the generation order requested by the user can be adjusted when necessary.
6. If no single workflow can advance the task, think deeply and creatively combine multiple workflows and execute them sequentially to complete the task.
7. Do not arbitrarily select workflows- only choose them if they can advance the task. If no suitable workflow can proceed with the task, return failure signal.
8. Based the planned workflow sequence, determine the remaining steps (remaining steps). Think step by step: if only one workflow call noeded, remaining steps If soquential workflow calls are needed, remaining steps 1, and so on.
9. ou need to be aware of the difference between instructions and prompts. Instructions are descriptions of tasks, while prompts are just descriptions of the images in the tasks. For example, a task to remove an object should have the prompt "The Object" instead of "Remove the object"
10. A Analyze whether the has additional requirements for the generated result, such video duration, resolution, frame rabe, upscaling factor, object placement, position, or any other requirements not covered by tool input.
11. Note: dditional requirements for generation results not include vague terms like high quality, seamless integration, without visible artifacts, etc.
12. If the context indicates that certain workflow cannot complete the task, it is strictly forbidden to select that workflow again. Instead, tyy more complex, multi-step workflow chain to solve the problem. For example, if direct 'Replace Object" workflow fails, do not select same workflow again. Instead, consider: Using another workflow with object replacement functionality; Or First utilizing masking workflow to generate a mask based prompt, then using an inpainting workflow with the masked image as input.
Figure 13: System Prompt for Planning, Part
31

---

Planning
13. Use past failed results to refine workflow selection and modify additional requirement parameters. For example, evaluation shows that the generated video duration tno short, recognize the user's intended duration and add to additional requirements.
Tools
have access to the following tools: (Jon.dummps(TOOL..)
## W orkflows and its Function and Input Parameters IMMPRTANTT*
Y ou have access to the following W Workflows:
(Temp_ W Workflow meta info)
#* ATTENTION
is important to distinguish between instructions and prompts. Prompts only contain generated content, not requirements. For example, "Create a video of the cityscape with the perspective changing based on the image" is not allowed in the prompt generated by Image to Video. Correct prompt should be "cityscape" For example, inpaint the red apple to green apple not allowed. Correct prompt should be "green apple".
Double check, and ensure the format of the tool imput follow the Tool and W Workflow's JSON schema.
Parameters not mentioned need be kept by default and added the additional requirements. For example,
there is no mention of changing the resolution( Uscale means the resolution is increased) and video duration (even if the frame rate is increased), but they need to be kept and added to the additional requirements.
additional requirements should only contain the additional requirements used in the current step.
If a task is given in two steps but there is no corresponding workflow for one of the steps, the tasks can be integrated and completed in single step. For example, if the user requests generating video first and then upscaling it. but there is no workflow for video upscaling the workflow library, the video can generated twice the resolution in one step to achiove the same effect.
# Output
## Output Parameters
A fter analyzing the workflow sequence, you must return aJSON object where:
tool and tool input represent the first tool that needs to be invoked.
remaining steps indicates how many more tool invocations are required to complete the task based on the planned sequence.
message provides a description of the current state and explanation of the planned workflow.
Figure 14: System Prompt for Planning, Part 2
32

---

Planning
additional requirements represents any extra user requirements for the generated result at this step, such as video duration, resolution, frame rate, upscaling factor, object placement, position (e.g., "the building should on the left"), or any other specifications not included tool input. Note: This does not include prompt-relates attributes such as "detailed" or "high quality." The result should be returned as string.
#* Output Format
The return format should include the thought process wrapped in <think> </think>, followed by the JSON object wrapped in <json> <fjson>.
{{
"tool": "Generate Using W Workflow"",,
"tool imput": <parameters for the selected tool, Must match the Jool' schema>(*DD Do not leave any placeholder),
"remaining steps": <number of steps remaining complete the task>,
"message": <The description of current state the explanation the plan>,
"additional requirements": <additional requirements this step", which may include resolution, frame rale, upsacle rate, etc. The format should be string>
}
Figure 15: System Prompt for Planning, Part
33

---

Tools
"name": "Generate Using. Workflow",
"description': "Generate Image or Video by workflow",
"parameters":
"type": "object",
"properties":
"workflow_namee" {
"type": "string,.
'description:: "Name of the workflow use"
}.
"prompt": {
"type": "dist",
description": "Text prompt for generation. The specific number and dictionary key depends on the
selected workflow requirements. Example: (%8%PPOMPP1%%% 'cat', '% "8%PPRMP22%% 'dog')"
}.
"input images": {
type": "dist",
"deccription: "Path to imput images. The specific number and dictionary key depends on the selected workflow requirements. Example: (%91/MAE1%%%: '1.png'. 96%1MAGE%%% '2.jpeg'"
}.
"input videos": {
"type": "dist",
'description": "Path to input videos. The specific number and dictionary key depends the selected
workflow requirements. Example: (%%%VID001%% '1.mp4' '% "%%VVDE02%%% '2.gif)""
}.
"required(ALL PARAMETERS ARE OPTIONAL EXCEPT WORKFLOW NAME)";
"workflow name",
"prompt",
"input images",
videos"
videos"
Figure 16: System Prompt for Tools Definition
34

---

Updating Workspace
Y helpful assistant that can update the input of the user. will give you the current input and the output of the tool. An will tell you the function of all workflows and the chain of thought the workflow selection.
Please update the input based on output, and add explanation for the new input, such as the content the new image(e.g. The intermediate steps generating the video, the masked image for inpainting, the background- masked-imagg -iadii-meee..... Elc.).
ATTENTION:
1. our output should be aJSON object. Totally follow the JSON schema of the input.
2. You should read the information of the workflow and the chain of thought, accroding the workflow's 's function guess what steps you have completed and what steps you may next complete. Then add the content of the added input parameters and them instruction. This may include the content the new promptiimaee/vide output.
3. ou should update the instructions for the workflow that you just completed. For example, the user asked you to generate an image first and then upscale it At this time, you noticed that the workflow you just ran performed the task of generating image. At this time, you should modify the instructions to: The täsk of generating image has been completed, and the next step upscale the image.
4. ou should pay attention to the timeliness of user's instructions. For example, The instruction:generate an image with resolution of XXX or a video with a duration of XXX. Such instructions are permanent. Therefore, it should continue to be passed, and at the same time remind the subsequent workflow to continue to maintain the generated resolution and video time.
5. * *MPORTANN** You MUST add information and introduction for the generated file to 'file, meta info Do not leave empty).
6. IMPORTANT* Y MUST maintain A LL Previous step ffile meta info' of the previous files(e.g. the image, the video, etc.). If the previous files are not mentioned in the 'file meta info', you should add them to the file meta info'. E.g: This image is the original input image for removing the background.
Then will give you the original input, information of the workflow and the output of the workflow.
Figure 17: System Prompt for Updating Workspace
35

---

Workflow A daptive A djjusmennt
Yuu advanced Al model tasked with customizing workflows based specific user requirements and visual inputs. our primary goal adjust pypeeparameeer without altering the workflow's structure node connections. Follow these steps to achieve this:
1. Analyze Visual Input:
Carefully examine the provided visual content to understand its key elements and context.
Identify any specific features or areas relevant to the user's request (e.g., determining where to place an object like small dog).
2. Understand User Requirements:
Clearly interpret the user's specific demands, such adding object to image or modifying certain aspects of the workflow.
Ensure you fully grasp the requirements before proceeding with any changes.
Ignore requests about generatio quality. For example: high quality, seamless integration, without visible artifacts, etc.
3. Review Workflow Structure:
Analyze the given workflow to understand its current structure and node connections.
Identify which hyperparameters control the aspects you need to modify based on the user's request (e.g. The length of the video is controled by video frames and fes)(e.g. nalyze which nodes control which prompt's layout).
Think step by step how to modify the parameters. (e.g. Video length(seconds) = video frames fpstframe rate), you should add reduce the video frames to make video frames fps( frrame rate) = the required length) Modify the layout parameters of corresponding prompt meet the requirements)
4. Modify Hyperparameters:
djjst only the necessary hyperparameters meet the user's requirements, ensuring the workflow's structure and node connections remain unchanged.
Base your modifications on the analysis the visual input and the user's specific demands.
5. Provide Clear Outpul:
Relurn the modified workflow with a clear explanation of the changes made.
Ensure the output is easy to undersland and directly addresses the user's needs.
Figure 18: System Prompt for Workflow Adaptive Adjustment, Part
36

---

Workflow A daptive A djjusmennt
Important A Atention:
Thee output must** follow the format:
1.Chain of Thought
2." "jsoon workflows"
The video duration error does not need be modified if it is within 0.5s
The number of video frames of the video generation model *WAN cannot be selected arbitraniy, must meet the condition modulo and remainder 1. For example (25, 29, ...)
Limitation:
+*DO NOT* change any prompt, and any path of the files.
Must follow the rule of JSON format. Do not add any other comment.
Example (Chain Thouggh)::
Suppose the requirement to add a small dog to an image. First, confirm the image resolution to ensure compatibility with the workflow. Next, analyze the given image to determine the most suitable location for placing the small dog, considering factors like existing objects and composition. Then, review the provided workflow to identify which hyperparameters control the insertion coordinates of the small dog. Based on your analysis, modify these hyperparameters to place the dog appropriately. Finally, reburn the updated workflow with a clear explanation of the changes made.
Next, I will provide you with the image/video, workflow and the requirements. Think step by step.
Figure 19: System Prompt for Workflow Adaptive Adjustment, Part 2
37


Reviews:
[{"review_id": "qVv7K6p7Sj", "reviewer": "Reviewer_xjBj", "summary": "This paper presents ComfyMind, a system for image generation and editing from complex text prompts by coordinating various foundation models as tools. The key idea is to utilize VLM to construct a workflow tree and call APIs from a tool library with feedback. Experiments on GenEval show promising results.", "questions": "What is the inference latency of the proposed method? Can authors make the comparison to the compared methods?", "limitations": "The paper has discussed the limitations.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "## Strenth\n- ComfyMind is an agent-based system that is training-free and easy to use.\n- The paper is clearly written, and the results look promising.\n\n## Weakness\n- **Major concern:** My major concern lies in the technical novelty. A closely related work, [GenArtist](https://openreview.net/pdf?id=Ur00BNk1v2) [1], proposes a similar approach and is not cited or discussed. To be specific, GenArtist also proposes to use VLM to decompose tasks into a tree-based tool execution flow that is executed with feedback control. GenArtist also supports various tasks like image generation and editing from language descriptions. While some details like the structure of every tree node are different, and ComfyMind supports more tasks by using more agents and more advanced foundation models, the core idea of ComfyMind is basically the same as GenArtist. Please clarify it and do a thorough comparison.\n- Typically, such agent-based image edition and generation systems are prone to the quality of the first generated image or video. The quality might not be sufficiently good, while the evaluation might still be processed as good, since the overall quality is not bad. One way to address this might be executing image generation path several times and pick the best one, but this might be time-consuming.\n- Besides the benchmarks used in this paper, please try to conduct experiments on T2I-CompBench [2] and DreamBench++ [3], and make a comparison with CompfyMind. These are good benchmarks for demonstrating the generation capability and will enrich the solidity. Note that you should try to use the same agents to make the comparison fair.\n\n\n[1] Wang, Zhenyu, Aoxue Li, Zhenguo Li, and Xihui Liu. \"Genartist: Multimodal llm as an agent for unified image generation and editing.\" Advances in Neural Information Processing Systems 37 (2024): 128374-128395. \\\n[2] Huang, Kaiyi, Kaiyue Sun, Enze Xie, Zhenguo Li, and Xihui Liu. \"T2i-compbench: A comprehensive benchmark for open-world compositional text-to-image generation.\" Advances in Neural Information Processing Systems 36 (2023): 78723-78747.\\\n[3] Peng, Yuang, Yuxin Cui, Haomiao Tang, Zekun Qi, Runpei Dong, Jing Bai, Chunrui Han, Zheng Ge, Xiangyu Zhang, and Shu-Tao Xia. \"Dreambench++: A human-aligned benchmark for personalized image generation.\" The Thirteenth International Conference on Learning Representations (2025).", "was_revised": true, "final_justification": "I have carefully read the authors' rebuttal and other reviews. I think this paper has its contributions and can be accepted, but the novelty is somewhat limited to me. Hence, I raise my original score to borderline accept, as I think this paper can be accepted.", "rebuttal": "We sincerely thank Reviewer xjBj for the thoughtful review and constructive suggestions.\n\n**> W1.**\n\n> A closely related work, GenArtist, proposes a similar approach… Please clarify it and do a thorough comparison.\n\n*Answer:*\n\nWe sincerely thank you for pointing out the important work of GenArtist. GenArtist has achieved remarkable success in automating complex visual generation using large models. Our work and GenArtist both stem from a similar vision—to tackle the challenge of complex instructions through a modular paradigm. We sincerely apologize for this oversight in citation and solemnly promise to add detailed comparative experiments in the revised version. The goals of our work and GenArtist differ: GenArtist focuses on image-related tasks, whereas our work aims to provide **a scalable and intelligent framework** for the more general-purpose, ComfyUI-centric open-source ecosystem.\n\nThis difference in goals leads to different technical abstractions. GenArtist plans with **tools**, whereas our core innovation, the **\"Semantic Workflow Interface\"**, enables our system to directly plan with community-validated high-level **workflows**. This difference in abstraction level also dictates our different paradigms for handling failure. GenArtist employs an efficient \"local repair\" strategy, using editing tools to patch flawed images. In contrast, our feedback-based decision loop is more flexible: based on the feedback, it can **fine-tune** the current workflow (e.g., by modifying the prompt and parameters), **strategically replace** it with a new one upon failure, or **continue** to the next step upon success.\n\nWe believe this adaptive planning and execution paradigm is crucial for the complex, multi-modal tasks we target, such as video generation, and brings greater robustness and scalability to the entire open-source community. Thank you again for your valuable feedback.\n\n\n**> W2.**\n\n> Agent-based systems are prone to the quality of the first generated image… Evaluation might still be rated as good despite poor initial quality… One way to address this might be executing the image generation path several times and picking the best one.\n\n*Answer:*\n\nWe sincerely thank the reviewer for this valuable suggestion and completely agree with your point: selecting a high-quality initial generation is crucial for improving the quality of the final work. We agree with and can easily implement the solution you proposed within our framework. Specifically, our system can generate a set of candidates and then either utilize the evaluation capabilities of VLM to automatically select the best starting point, or introduce human-in-the-loop interaction to allow the user to make a personal selection. Regarding the time-consumption issue you mentioned, we plan to adopt a parallel generation strategy. Since multiple candidates can be generated simultaneously, this approach would not significantly increase the overall waiting time.  We will further integrate these strategies and will discuss them in future.\n\n\n**> W3.**\n\n> Please conduct experiments on T2I-CompBench and DreamBench++ to enrich the solidity of evaluation.\n\n*Answer:*\n\nWe sincerely appreciate your suggestion regarding enhancing the robustness of our evaluation. Following your advice, we have conducted detailed experiments on T2I-CompBench and DreamBench++ while keeping the Agent component of our system unchanged.\n\nDue to time constraints, we performed experiments on the first three categories of T2I-CompBench, with the results shown below:\n\n| Method       | Color ↑   | Shape ↑ | Texture ↑ |\n| ------------ | -------- | ------ | ------ |\n| DALL-E 3     | 0.7785   | 0.6205 | 0.7036 |\n| Janus-Pro-7B | 0.6359   | 0.3528 | 0.4936 |\n| FLUX.1       | 0.7407   | 0.5718 | 0.6922 |\n| GenArtist    | 0.8482   | **0.6948** | 0.7709 |\n| Ours         | **0.8825**   | 0.6825 | **0.7866** |\n\nThe experiments demonstrate that our method outperforms other approaches on T2I-CompBench. Compared to GenArtist, our method achieves better performance in the Color and Texture categories, and competitive, closely matched performance in the Shape category. These results further validate the effectiveness of our method in this text-to-image task.\n\n\n| Method          | Concept Preservation ↑  | Prompt Following ↑  |\n| --------------- | -------------------- | ---------------- |\n| BLIP-Diffusion  | 0.547                | 0.495            |\n| Emu2            | 0.528                | 0.689            |\n| IP-Adapter-Plus | 0.833                | 0.413            |\n| GenArtist       | 0.848                | 0.603            |\n| GenArtist (more tools) |0.852\t         | 0.753            |\n| Ours            | **0.857**            | **0.771**        |\n\nAdditionally, the table above presents the performance of our method on DreamBench++. Our method outperforms other approaches, including GenArtist and its tool-extended version, in both concept preservation and prompt following. This further illustrates the effectiveness of our method in image customization tasks.\n\nWe believe this strong performance, particularly on these complex compositional and personalization benchmarks, directly stems from our system's core design. The ability to compose high-level workflows via the SWI and to adaptively select strategies allows ComfyMind to handle these challenging tasks more effectively than approaches that rely on lower-level tool planning. We will integrate these new results and corresponding analyses into the final version of our paper to substantially strengthen our evaluation section. Thank you again for this valuable guidance.\n\n\n**> Q1.**\n\n> What is the inference latency of the proposed method? Please compare with other methods.\n\n*Answer:*\n\nTo address your question regarding inference latency, we have conducted detailed experiments on the ComfyBench benchmark. On Geneval, we compared the inference latency and performance of our method with two atomic workflows (FLUX.1 and SD3.5-L) that our atomic workflow is based on. The results are shown in the table below.\n\n| Model          | **Time(Seconds) ↓** | **Overall ↑** | Single Obj. ↑ | Two Obj. ↑ | Counting ↑ | Colors ↑ | Position ↑ | Attr. Binding ↑ |\n|----------------|---------------------|---------------------|-------------|----------|----------|--------|----------|---------------|\n| SD3.5-L        | **47.06**               | 0.71                | 0.98        | 0.89     | 0.73     | 0.83   | 0.34     | 0.47          |\n| FLUX.1 Dev     | 56.51               | 0.66                | 0.98        | 0.81     | 0.74     | 0.79   | 0.22     | 0.45          |\n| Ours           | 119.94              | **0.90**                | **1.00**        | **1.00**     | **1.00**     | **0.97**   | **0.62**     | **0.80**          |\n\nAs can be seen, although our system requires additional time for planning, evaluation, and refinement, the increase in latency is not excessive, remaining approximately twice that of the atomic workflows. This reflects that our system can quickly converge to high-quality generation without getting stuck in oscillating and repeated attempts. Furthermore, our system achieves significantly higher performance than the underlying atomic workflows on Geneval, confirming that the additional inference time is worthwhile.\nIn addition, we conducted detailed experiments on ComfyBench, which covers a broader range of task types, comparing the inference latency and performance of our method with other common planning approaches (such as ReAct and Planning-Act). Here, Planning-Act refers to the approach of first conducting global planning and then recursively executing subtasks.\n\n| Method       | Time(Seconds) ↓ | Resolve Rate \\% ↑ |\n| ------------ | -------------- | -------------- |\n| ReAct        | 257.43         |  54.0 |\n| Planning-Act | 279.79         |  66.0 |\n| Ours         | **225.12**     |  **83.0** |\n\nThe experiments show that our method achieves the shortest time on ComfyBench while also attaining the highest task resolution rate, demonstrating the excellent capability of our collaborative system."}, {"review_id": "fgLpjROaH6", "reviewer": "Reviewer_j82d", "summary": "This paper introduces ComfyMind, a system that optimizes prompts through reactive feedback and leverages downstream models to support a wide range of visual generation tasks. Specifically, the proposed Semantic Workflow Interface (SWI) abstracts low-level nodes into callable modules, facilitating high-level composition across diverse tasks. Extensive experiments demonstrate the advantages and effectiveness of ComfyMind in visual generation, editing, and reasoning.", "questions": "+ Apart from the final results, an analysis of the LLM planning process should be conducted. How robust is this process? If the LLM makes incorrect decisions or generates flawed prompts, will the errors propagate to the final performance? How can this potential issue be mitigated? What is the bottleneck of the proposed pipeline — the LLM or the downstream visual generation models?\n+ In the appendix, instead of only showing the final visualizations, the entire LLM reasoning process should be presented. Failure cases should also be discussed to inform future research improvements.\n+ It is already known that prompt rewriting can significantly boost GenEval performance. In Table 2, do the baseline models also use rewritten prompts? If not, is this a fair comparison?\n+ Since ComfyMind requires LLM planning and iterative feedback, how does its efficiency compare to each of the target baselines? Does the performance gain justify the additional time overhead?", "limitations": "yes", "rating": 4, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 2, "originality": 2, "strengths_and_weaknesses": "**Main Strengths**\n+ The paper is well-written and easy to follow.\n+ It provides numerous qualitative examples and system prompts, which support the reproducibility of the proposed system.\n+ The proposed pipeline is model-agnostic, allowing newer downstream visual generation models to benefit further.\n\n**Main Weaknesses**\n+ Although the performance is promising, ComfyMind appears to be a combination of LLM agents [1, 2] and prompt rewriting techniques [4, 5, 6], which are already well-known to enhance visual generation.\n\n[1] DiffAgent: Fast and Accurate Text-to-Image API Selection with Large Language Model\n\n[2] GenArtist: Multimodal LLM as an Agent for Unified Image Generation and Editing\n\n[3] Improving Image Generation with Better Captions\n\n[4] Tailored Visions: Enhancing Text-to-Image Generation with Personalized Prompt Rewriting\n\n[5] Dynamic Prompt Optimizing for Text-to-Image Generation\n\n[6] Prompt Adaptation as a Dynamic Complement in Generative AI Systems", "was_revised": false, "final_justification": "", "rebuttal": "We sincerely thank you for your clear, detailed, and thought-provoking comments. Below, we address each of your concerns and questions point by point.\n\n**> W1.**\n\n> ComfyMind appears to be a combination of LLM agents and prompt rewriting, which are known to enhance visual generation.\n\n*Answer:*\n\nWe sincerely thank the reviewer for astutely pointing out the connection between our research and the two important research directions of LLM agents and prompt optimization. We acknowledge that these studies have laid an important foundation for our work. However, our research is not a simple combination of these two.\n\nThe originality of our work lies in providing a complete, reliable, and adaptive framework and technical pathway based on ComfyUI. The Semantic Workflow Interface addresses the frequent omissions and syntax errors in workflow construction that plague other methods, enabling agents to focus on the ability to compose and call at a high level. Based on SWI, Search Tree Planning allows for exploring more and better solutions while maintaining efficiency. Finally, Local Feedback Execution enables the system to maintain generation quality while supporting fine-grained adaptive decision-making: for instance, when the current path is ineffective, it can strategically replace the entire workflow; when the problem is complex, it will continue to compose new capabilities; and when there are only minor deviations, it will optimize its prompts or parameters. In conclusion, our work is not a simple superposition of existing technologies, but rather provides a promising approach for the open-source generative ecosystem through interlocking designs.\n\n**> Q1.**\n\n> Apart from the final results, an analysis of the LLM planning process should be conducted. How robust is this process? If the LLM makes incorrect decisions or generates flawed prompts, will the errors propagate to the final performance? How can this potential issue be mitigated? What is the bottleneck of the proposed pipeline — the LLM or the downstream visual generation models?\n\n*Answer:*\n\nTo directly address your concerns about the robustness of the LLM planning process, we conducted comprehensive comparisons against representative planning strategies — ReAct and Planning-Act — on the ComfyBench benchmark. Planning-Act is a canonical method widely adopted in multi-agent systems, which performs full planning at the root node and then recursively executes subtasks in a top-down manner. The results are as follows:\n\n| Method | Resolve Rate \\% ↑ | Repetition Rate \\% ↓ |\n|--------|-------------|----------------|\n| ReAct | 54.0 | 34.59 |\n| Planning-Act | 66.0 | 28.24 |\n| Ours | **83.0** | **8.70** |\n\nHere, Repetition Rate reflects how often the planner redundantly selects the same atomic workflow. Compared to ReAct and Planning-Act, our method achieves the lowest repetition rate, indicating fewer oscillations and better learning from past failures. The higher resolve rate also demonstrates broader exploration and stronger robustness of the planning process.\n\nWe acknowledge that LLMs can make flawed decisions or generate incorrect prompts. Our system is designed to significantly mitigate their downstream effects. Syntax errors and atomic workflow failures are typically detected early, prompting the planning agent to replan or backtrack. For other errors (e.g., low-quality generations or flawed prompts), the evaluation agent identifies the issue and triggers Local Feedback Execution. When the rollback reaches the erroneous planning node, the system re-plans accordingly. In this way, while errors may increase generation time or path length, they are effectively contained and do not propagate to the final output.\n\nAs for the main bottleneck of our pipeline, we believe it lies in the downstream visual generation models. In most failure cases, the LLM agent generates reasonable plans, but the outputs fail due to insufficient capability or instability of the generative models.\n\n**> Q2.**\n\n> In the appendix, instead of only showing the final visualizations, the entire LLM reasoning process should be presented. Failure cases should also be discussed.\n\n*Answer:*\n\nThank you for the valuable suggestion. We fully agree that providing complete processing traces and failure cases, which is critical for transparency and deeper understanding. We have already collected a set of representative examples, covering both successful reasoning steps and typical failure cases. Due to policy constraints, we are currently unable to include associated images or PDFs. However, we firmly commit to adding detailed reasoning traces and error case discussions in the appendix of the revised version.\n\n**> Q3.**\n\n> In Table 2, do the baseline models also use rewritten prompts? If not, is this a fair comparison?\n\n*Answer:*\n\nIn Table 2, most baseline models were indeed evaluated using their default inputs, without explicit prompt rewriting. To ensure a fair comparison, the Prompt Preprocessing step—an explicit external rewriting module—was disabled during our experiments, and the original prompts were directly provided to the system.\n\nHowever, within ComfyMind, prompt rewriting can occur as a dynamic internal step—either during planning (e.g., to adjust prompts dynamically when generation quality is insufficient) or through workflows explicitly designed for prompt optimization. This internal rewriting is considered part of ComfyMind’s overall adaptive capability rather than external preprocessing.\n\nWe will gladly add a more explicit ablation study in our revised manuscript, ensuring an absolutely fair comparison by enabling comparable rewriting mechanisms for baseline methods\n\n**> Q4.**\n\n> Since ComfyMind requires LLM planning and iterative feedback, how does its efficiency compare to each of the target baselines? Does the performance gain justify the additional time overhead?\n\n*Answer:*\n\nWe present two sets of comparisons to address your question about efficiency and performance trade-offs:\n\n| Model          | **Time(Seconds) ↓** | **Overall Score ↑** | Single Obj.↑| Two Obj.↑| Counting↑| Colors↑| Position↑| Attr. Binding↑|\n|----------------|---------------------|---------------------|-------------|----------|----------|--------|----------|---------------|\n| SD3.5-L        | **47.06**               | 0.71                | 0.98        | 0.89     | 0.73     | 0.83   | 0.34     | 0.47          |\n| FLUX.1 Dev     | 56.51               | 0.66                | 0.98        | 0.81     | 0.74     | 0.79   | 0.22     | 0.45          |\n| Ours           | 119.94              | **0.90**                | **1.00**        | **1.00**     | **1.00**     | **0.97**   | **0.62**     | **0.80**          |\n\n\nAlthough our system incurs additional time due to planning, evaluation, and possible regeneration, it achieves significantly higher overall scores. The performance gain justifies the roughly 2x increase in runtime, demonstrating that our system pushes the boundary of what current models can accomplish.\n\n| Method       | Time(Seconds) ↓ | Resolve Rate \\% ↑ |\n| ------------ | -------------- | -------------- |\n| ReAct        | 257.43         |  54.0 |\n| Planning-Act | 279.79         |  66.0 |\n| Ours         | **225.12**     |  **83.0** |\n\nIn broader multi-step tasks, our approach not only solves more tasks but also completes them more efficiently than other agent-based methods.\n\nTaken together, these two sets of results show that ComfyMind introduces a moderate time overhead compared to baselines but delivers substantially better performance, validating its design and effectiveness."}, {"review_id": "ln15TtSIf4", "reviewer": "Reviewer_yctv", "summary": "This paper introduces ComfyMind, a collaborative AI system built on the ComfyUI platform, designed for robust and scalable general-purpose generation. Addressing the fragility of existing open-source frameworks in complex real-world applications, ComfyMind introduces two core innovations: a Semantic Workflow Interface (SWI), which abstracts low-level node graphs into callable functional modules described in natural language, enabling high-level composition and reducing structural errors; and a Search Tree Planning mechanism with localized feedback execution, which models generation as a hierarchical decision process, allowing adaptive correction at each stage. Evaluated on three public benchmarks (ComfyBench, GenEval, and Reason-Edit) spanning generation, editing, and reasoning, ComfyMind consistently outperforms existing open-source baselines and achieves performance comparable to GPT-Image-1, paving a promising path for open-source general-purpose generative AI systems.", "questions": "1. Within the search tree planning, is MCTS (Monte Carlo Tree Search) or another specific search algorithm used to guide node selection and path exploration? If not, how does the agent effectively avoid local optima or inefficient exploration?\n\n2. The localized feedback relies on a VLM for parsing and annotating generated content. The VLM's own performance and its ability to understand complex visual details could become a bottleneck for feedback quality and overall system robustness.", "limitations": "See weakness and questions.", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 4, "significance": 4, "originality": 4, "strengths_and_weaknesses": "Strengths:\n\n1. Innovative High-Level Semantic Abstraction: The Semantic Workflow Interface (SWI) is a key highlight, abstracting ComfyUI's low-level node operations into natural-language function modules. This significantly simplifies workflow construction, reduces syntax errors, and enhances the language model's ability to understand and compose complex tasks.\n\n2. Robust Localized Feedback Mechanism: The Search Tree Planning with Local Feedback Execution is highly effective. By modeling workflow construction as a hierarchical decision process and triggering local corrections upon execution failures, it avoids global rollbacks and redundant computations, greatly enhancing system robustness in complex tasks.\n\n3. Broad Generality and Excellent Performance: ComfyMind demonstrates strong performance across three diverse benchmarks (ComfyBench, GenEval, Reason-Edit) for generation, editing, and reasoning. It not only significantly outperforms open-source baselines like ComfyAgent but also matches or even surpasses the closed-source GPT-Image-1 in some aspects, proving its powerful general-purpose generation capabilities.\n\nWeakness\n\nLocalized feedback is crucial for ComfyMind's robustness, but the paper does not quantify how this feedback is implemented. For instance, the granularity of VLM parsing and annotation, the specific format of failure signals, and how parent nodes revise their planning based on this feedback are not detailed.", "was_revised": true, "final_justification": "Overall, the responses are detailed, well-motivated, and supported by new empirical evidence. They strengthen the clarity, technical rigor, and extensibility of the work. I am satisfied that the authors have adequately addressed the concerns, and the revised version will be significantly improved.", "rebuttal": "We sincerely thank you for your detailed and insightful comments. Below, we provide point-by-point responses to your concerns and suggestions.\n\n**> W1.**\n\n> The paper does not quantify how the feedback is implemented… the granularity of VLM parsing, failure signal formats, and how parent nodes revise plans are not described.\n\n*Answer:*\n\nTo directly address your concern regarding the robustness of localized feedback, we conducted additional experiments on ComfyBench, analyzing how different levels of feedback granularity affect task success rates. \n\n| Method | Vanilla \\% ↑ | Complex \\% ↑ | Creative \\% ↑ | Total \\% ↑ |\n|--------|---------|---------|----------|-------|\n| Judgment | 89.0 | 58.33 | 47.5 | 71.5 |\n| Judgment+Analysis | 91.0 | 66.67 | 55.0 | 76.5 |\n| Judgment+Analysis+Description(Ours) | **92.0** | **85.0** | **57.5** | **83.0** |\n\nHere,\n- Judgment refers to feedback containing only a binary success/failure signal.\n- Analysis adds explanations of the outcome, specifying the reasons for success or failure.\n- Description includes more detailed image-grounded feedback, such as what has been completed, visual observations, and suggestions for correcting the current plan or direction.\n\nThese results demonstrate the effectiveness of our localized feedback design. Increasing feedback granularity provides richer information for the planning agent, improving its ability to make informed decisions. This benefit is further amplified by our search tree design, which prevents the context from becoming too long at each planning node—ensuring that even fine-grained feedback remains effective without being diluted.\n\nAs for the format of failure signals and how parent nodes revise plans:\n- Failure signals are returned to the parent node in a structured JSON format.\n- The plan revision mechanism works by injecting the feedback content into the parent node’s context. The planning agent then performs reasoning based on this context, which includes the description of past failures, previously chosen workflows, and the current input. It avoids paths known to be incorrect or unsuccessful and selects the next best atomic workflow based on this updated information.\n\n\n\n**> Q1.**\n\n> Is MCTS (Monte Carlo Tree Search) or another specific search algorithm used? If not, how does the agent avoid local optima or inefficient exploration?\n\n*Answer:*\n\nThank you for this insightful question. We do not use MCTS or any traditional search algorithm. Instead, our system is driven by LLM-based heuristic search, leveraging the strong reasoning and planning capabilities of LLMs.\n\nThese capabilities enable the model to perform informed and adaptive path selection, effectively avoiding local optima without the need for additional tree search mechanisms. When the agent encounters a dead-end or inefficient trajectory, it can proactively exit the path and re-plan. We also set an upper limit of 5 candidate paths per planning agent, which balances sufficient exploration of the solution space while preventing local optimality.\n\nAnother reason we deliberately avoid using MCTS or reinforcement learning (e.g., GRPO) is to preserve generality and ease of use. Training in specific domain may improve performance, but reduce generalization and hinder the system's ability to incorporate stronger workflows or components in the future. Our design emphasizes plug-and-play modularity and general-purpose applicability across domains.\n\n**> Q2.**\n\n> The VLM's own performance in understanding complex visual details could become a bottleneck for the feedback quality and system robustness.\n\n*Answer:*\nWe fully agree with this insightful observation. The performance of the evaluation VLM is indeed a critical factor, and its potential for misjudgment in complex scenes is a known limitation we have actively considered in our design.\n\nIn our current implementation, we already mitigate this to some extent by using structured(Judgment, Analysis, Description) and targeted queries for evaluation, which constrains the VLM's task and reduces ambiguity. However, to further enhance robustness, our modular framework is designed for a straightforward extension to a multi-granularity feedback mechanism. This involves a hierarchy of checks: a high-level VLM for overall semantic coherence, combined with more specialized models (e.g., an object detector or attribute classifier) for fine-grained verification. This approach reduces the risk of single-point failure by cross-validating feedback. We will add a detailed discussion of this enhancement strategy to the appendix of the final version.\n\nWe also concur that as foundational VLMs continue to rapidly advance, this bottleneck will naturally diminish. Our plug-and-play architecture is perfectly positioned to seamlessly benefit from these future improvements."}, {"review_id": "RBKsiZMEcL", "reviewer": "Reviewer_Hh8S", "summary": "This work proposes ComfyMind, which is a planning system built on top of ComfyUI, a node-based execution system for image/video generation. ComfyMind contains two main components: 1) a semantic interface that allows natural language inputs and 2) a planning algorithm with localized feedback that searches for stable and high-quality workflows. ComfyMind has been evaluated on open-source benchmarks, and results showed that it consistently outperforms existing open-source baselines.", "questions": "n/a", "limitations": "n/a", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "Strengths:\nClarity: The work is presented clearly, with well-defined motivations and a straightforward, easy-to-follow pipeline. Detailed comparisons with prior works further enhance understanding.\nSignificance: The proposed algorithms have been proven to be useful in three sets of experiments.\n\nWeaknesses:\nWhile the search tree planning algorithm is described at a high level, and its efficacy is justified through experimental results and ablation studies, a more detailed quantitative analysis would strengthen the work. For instance, the claim that \"This allows the system to..., ...lead to strategy oscillations and failure to converge\" could benefit from direct quantitative comparisons (e.g., efficiency, stability) between the proposed method and baselines like ReAct. Presenting experiments with more detailed ananysis, rather than the final metric, would be helpful.", "was_revised": true, "final_justification": "The additional experiments show good stability and efficiency of the algorithm. I will keep my score.", "rebuttal": "We sincerely thank you for your thorough review and constructive suggestions. In response to your comment, we have added a more direct and detailed quantitative analysis to support our views.\n\n**> W1.**\n\n> The claim that “This allows the system to… avoid strategy oscillations” could benefit from direct quantitative comparisons (e.g., efficiency, stability) between the proposed method and baselines like ReAct.\n\n*Answer:*\n\nTo address your concern regarding the need for more detailed quantitative analysis, we conducted comprehensive comparisons with representative planning strategies—ReAct and Planning-Act—on the ComfyBench benchmark. The results are presented below:\n\n| Method | Resolve Rate \\% ↑ | Repetition Rate \\% ↓ | Time (Seconds) ↓ |\n|--------|-------------|----------------|----------------|\n| ReAct | 54.0 | 34.59 | 257.43 |\n| Planning-Act | 66.0 | 28.24 | 279.79 |\n| Ours | **83.0** | **8.70** | **225.12** |\n\nWe evaluate three core metrics: **Resolve Rate**, **Repetition Rate**, and **Average Time per Task**, which respectively reflect a system’s **problem-solving capability**, **stability of exploration**, and **efficiency**.\n\n**Resolve Rate** measures the success rate in solving compositional tasks, directly demonstrating the planner’s ability to construct effective multi-step generations.\n\n**Repetition Rate** captures the frequency with which atomic workflows are redundantly selected within a single task. High repetition suggests issues such as oscillations or failure to adapt contextually. Our method significantly reduces this rate, indicating more stable and purposeful search behavior.\n\n**Time** represents the average execution time per task. Our approach benefits from broader exploration and avoids falling into repetitive local patterns, allowing it to solve problems earlier and more efficiently.\n\nFor **ReAct**, its higher repetition rate (34.59) and lower success rate (54.0) suggest wasted exploration. Although ReAct executes actions in a reactive manner based on current observations, it often lacks global planning and stable intermediate decision nodes. This leads to repeated trial-and-error behavior, resulting in inefficient progress and strategy oscillations. The relatively high average task time (257.43s) further indicates that steps is not used efficiently, and the system spends more time cycling through ineffective action sequences.\n\nFor **Planning-Act**, this is a canonical method widely adopted in multi-agent systems. It follows a structure where the agent performs full planning at the root node and then recursively executes subtasks. However, this architecture has two major drawbacks:\n1. It lacks real-time adaptability: though it can re-plan after errors, its plan fixes once a plan-act cycle starts, with no way to adjust mid-execution based on results—e.g., it can’t dynamically add an image enhancement workflow if quality is too low mid-step. This rigidity also limits exploring similar-function workflow branches, hindering optimal solutions and contributing to its lower Resolve Rate (66.0%).\n2. Due to the absence of stable intermediate decision nodes, it frequently needs to reselect workflows required for intermediate steps, leading to significant redundancy and inefficiency. These limitations are reflected in its high Repetition Rate (28.24), and the longest average Time per Task (279.79s) among all compared methods.\n\nIn contrast, **our method** achieves the best performance across all three dimensions: higher success, lower redundancy, and faster convergence.\n\nThis analysis provides concrete empirical evidence supporting our claim that the proposed planning mechanism mitigates strategy oscillations and enhances convergence efficiency, outperforming existing baselines both in effectiveness and in behavioral stability."}]

Author rebuttal:

Meta-review: The paper received four expert reviews. The authors provided a rebuttal that attempted to address the concerns raised in the reviews. The reviewers read the rebuttal and engaged with the authors.  The reviewers unanimously like the paper and recommended accept. The area chair agreed with the recommendation and decided to accept the paper. Congratulations! Please see the reviews for feedback on the paper to revise the final version of your paper and include any items promised in your rebuttal.
Decision: Accept (poster)