PerfGuard: A Performance-Aware Agent for Visual Content Generation
Authors: Zhipeng Chen
zhongrui zhang
Chao Zhang
Yifan Xu
LAN YANG
Jun Liu
Ke Li
Yi-Zhe Song
Venue: ICLR 2026 Poster (2026)
Keywords: Agent
Large Language Model
Image Generation
Image Editing
Primary area: applications to computer vision, audio, language, and other modalities
TL;DR: We propose PerfGuard, a framework that models tool performance to improve planning accuracy in AIGC.

Abstract: The advancement of Large Language Model (LLM)-powered agents has enabled automated task processing through reasoning and tool invocation capabilities. However, existing frameworks often operate under the idealized assumption that tool executions are invariably successful, relying solely on textual descriptions that fail to distinguish precise performance boundaries and cannot adapt to iterative tool updates. This gap introduces uncertainty in planning and execution, particularly in domains like visual content generation (AIGC), where nuanced tool performance significantly impacts outcomes. To address this, we propose PerfGuard, a performance-aware agent framework for visual content generation that systematically models tool performance boundaries and integrates them into task planning and scheduling. Our framework introduces three core mechanisms:  (1) Performance-Aware Selection Modeling (PASM), which replaces generic tool descriptions with a multi-dimensional scoring system based on fine-grained performance evaluations; (2) Adaptive Preference Update (APU), which dynamically optimizes tool selection by comparing theoretical rankings with actual execution rankings; and (3) Capability-Aligned Planning Optimization (CAPO), which guides the planner to generate subtasks aligned with performance-aware strategies. Experimental comparisons against state-of-the-art methods demonstrate PerfGuard’s advantages in tool selection accuracy, execution reliability, and alignment with user intent, validating its robustness and practical utility for complex AIGC tasks.

Published as a conference paper at ICLR 2026
PERFGUARD: A PERFORMANCE-AWARC AGENT FOR VISUAL CONTENT GENERATION
Zhipeng Chen 1.2 Zhongrui Zhang', Chao Zhang? Yifan Xu², Lan Yang Jun Liu Ke Lik4... and Yi-Zhe Song*
School of Artificial Intelligence, Beijing University of Posts and Telecommunications, China "Bejjig Digital Native Digital City Research Center, China
School of Computer Science and Engineering, Beihang University, China SketchX, CVSSP, University of Surrey, United Kingdom Chenxi Shuzhi (Beijing) Technology Co., Ltd, China
(zhipengchen199) 998, zhongrui.zhang, ylan, liujun, 1 likel (eee19900bbupt edu, cn, ariczhang20C9@mmall com, yifan-xu@buae edu. en, y.songesurrey ac.
ABSTRACT
The advancement of Large Language Model (LLM))powered agents has en- abled automated task processing through reasoning and tool invocation capa- bilities. However, existing frameworks often operate under the idcalized as- sumption that tool executions are invariably successful, relying solely on tex- tual descriptions that fail distinguish precise performance boundaries and can- not adapt to iterative tool updates. This gap introduces uncertainty in planning and execution, particularly in domains like visual content generation (AIGC), where nuanced tool performance significantly impacts outcomes. To address this, we propose PerfGuard, a performance-aware agent framework for visual content generation that systematically models tool performance boundaries and integrates them into task planning and scheduling. Our framework introduces three core mechanisms: (1) Performance-Awan Selection Modeling (PASM), which replaces generic tool descriptions with a multi-dimensional scoring sys- tem based fine-grained performance evaluations; (2) Adaptive Preference Up- date (APU), which dynamically optimizes tool selection by comparing theoret- ical rankings with actual execution rankings; and (3) Capability-Alignn
Plan- ning Optimization (CAPO), which guides the planner to generate subtasks aligned with performance-awar strategies. Experimental comparisons against state-of- the-art methods demonstrate PerfGuard's advantages in tool selection accuracy, execution reliability, and alignment with user intent, validating its robustness and practical utility for complex AIGC tasks. The project code is available at http://hhhwo.c.coeeeeeeeaaaa////
INTRODUCTION
In recent years, with the continuous advancement of Large Language Model (LLM) technology (Guo et al. 2025) Yang et al. 2025a Fang et al. 2025b), agent-based automated task processing
has become an important research direction across various fields (Curtarolo et al.) 2012 Gao et al. 2024b Wang et al. 2024b Agashe al. 2025). By constructing system frameworks with logical reasoning capabilities and equipping agents with the ability to invoke external tools, researchers aim to achieve the decomposition, reasoning, and autonomous execution complex tasks, thereby surpassing the limitations of traditional single tools rule-based systems. Most existing research focuses on the task planning and tool scheduling strategies of agents, emphasizing the rationality of
the planning process (Agashe al. 2025 Zhang al. 2025a Hong al. 2024a). However, these
Chao Zhang and
Ke Li are co-corresponding authors:
saschangg0099@gaall.com
like 1kket990bupt.ccu.cc

---

Published as a conference paper at ICLR 2026
Input
Create a hand- hond-psinted Bustrration a warn caturn field with green cabbages licf cowered oul red bird ranaging searlry. The pornting sheeld feel secsendl and coolegica.
Sehhakk 2: Add saved Pasio next
eight cablages
the leage to peinting style
Performance Aware Selection Modeling
hule
Bust looel Layst
Bext Tool: StepiX
Best Toot Ulneal
Fean
615
lome
677
Tools Performance
Performance
Tool Rating
Weignt
Boundary
Figure 1: PerfGuard decomposes user requests into subtasks for iterative visual content generation. By modeling tool performance boundaries via PASM, selects the most suitable tool in each round to ensure precise alignment between planning, execution, and user intent.
studies generally operate under the ideal assumption that "tool invocations are always successful," lacking systematic evaluation of the actual success rate of tool exccution. Against this backdrop, how tool selection and their actual execution outcomes impact the overall accuracy of agent plamning and decision-making remains a critical issue that has not been fully explored.
In current research, the description of tool capabilities often relies general textual descriptions, which are difficult to accurately reflect their true performance boundaries. This issue is particu- larly prominent in the field of visual content generation (AIGC). Although existing systems (such as CompAgent (Wang et al.) 2024c), GenArtist (Wang et al.) 2024b), etc.) can enhance genera- tion outcomes through task decomposition and multi-model scheduling, their descriptions of tool capabilities remain relatively coarse. These descriptions fail to clearly distinguish the specialized capabilities and applicable scenarios of differrnnt tools. Taking text-to-image generation as an ex- ample, common tool descriptions such as "capable of generating images aligned with the semantics of the input text" neither reflect the performance differences between various models nor support precise tool matching by agents in complex tasks, thereby introducing uncertainty into the planning and execution processes
To address the aforementioned challenges, this paper introduces PerfGuard, performance-awar agent framework for visual content generation. The framework aims to explore methods for model- ing tool performance boundaries and leverage their impact on task planning and scheduling mech- anisms. In response to the limitations posed by ambiguous tool capability descriptions, we pro- pose Performance-Awar tions with a multi-dimensional scoring mechanism based on fine-grained performance evaluation. Within this mechanism, the Worker dynamically selects the tool that best meets the performance requirements of the subtask generated by the Planner, thereby enhancing the accuracy and efficiency of task execution at the underlying scheduling level.
Selection Modeling (PASM), which replaces traditional textual descrip-
Acknowledging that preset performance boundaries (often derived from benchmark test results) may deviate from actual lask execution outcomes, we further introduce an Adaptive Preference Updating (APU) method. This method continuously optimizes the performance boundary matrix by comparing the theoretical ranking of candidate tools with their observed performance during real task execution. This improves the accuracy of task-tool matching and enhances the system's ' adaptability to real-world scenarios.
To better align the task planning process with tool performance, we propose a Capability-Alignes Planning Optimization (CAPO) mechanism. This enables the Planner to generate high-quality task plans under the guidance of the performance-driven selection strategy facilitated by PASM. In each planning iteration, the Planner generates multiple candidate subtask plans and improves planning accuracy by comparing their output results. Through step-by-step supervision, the Planner leams to form planning patterns consistent with the performance-awan strategy, thereby systematically enhancing the robustness of the reasoning process.
To validate the effectiveness of PerfGuard, we conducted comparative experiments with existing representative visual content generation methods. In various tasks such image generation and
2

---

Published as a conference paper at ICLR 2026
editing, PerfGuard demonstrated advantages in tool selection accuracy, task execution reliabilityy, and alignment with user intent. The results confirm the robustness and practical value our framework.
RELATIVE WORKS
Recent advances in visual content generation have significantly improved controllability and seman- tie aligmment. Models like FLUX (Labs] 2024), Stable Diffusion3 (Esser et al. 2024), and DALLLE3
(Betker et al., 2023) generate images from textual prompts, while ControlNet (Zhang et al. 2023), 12I-Adapter (Mou al.) 2024), InstanceDiffusion (Wang et al. 2024a) and VersaGen (Chen al.) 2025b) incorporate multimoda signals to better match user intent. To support fine-graines control, Layou (Feng et al., 2023), RPG (Yang et al. 2024b), GoT (Fang et al., 2025a), and T2I-RI Jiang al. 2025) leverage LLMs to decompose prompts into region-specifcc semanties. Systems like CompAgent (Wang et al. (202cc) and GenArtist (Wang et al.] (022b) coordinate generation and editing tools, while MCCD (Li et al., 2025) and T21-Copilot (Chen et al. 2025a) improve perfor- mance through model cooperation. CLOVA (Gao et al. (024a) improves the success rate visual tasks, including face swapping. by enhancing the tool's prompt pool through the introduction of self-reflection and prompt tuning. However, most approaches assume reliable tool execution and overlook how performance boundaries affect planning accuracy. PerfGuard addresses this gap by explicitly modeling tool capabilities and execution feedback.
3 PRELIMINARY
Standardized Agent System (Agashe et al. 2025 Hong et al. 2024b): Analyst, Planner, Worker, and Self-Evaluator. These roles handle task interpretation, planning, execution, and feedback respectively, enabling stepwise execu- tion with continuous refinement. Building on this architecture, PerfGuard introduces Performance- Aware Selection Modeling to optimize tool selection for the Worker, ensuring better alignment with task requirements and improved execution performance.
Standardized agent systems typically consist of four core roles
Step-aware Preference Optimization (SPO) In the visual domain, aligning image generation outputs with human aesthetic preferences has been key challenge. Inspired by Direct Preference Optimization (DPO) (Rafailov al. 2023) for aligning language model outputs with human prefer- ences, researchers proposed Diffusion-DPOO (Wallace et al. (2024) and D3PO (Yang et al. 2024a), which utilize a trained reward model to evaluate multiple random samples from a diffusion model and identify winning samples r" and losing samples æ². To further improve the aesthetic quality of each intermediate step in the diffusion process, SPO (Liang et al. 2024) introduces Step-Aware Preference Model (SPM) that evaluates and optimizes intermediate outputs every step, ensuring that candidate samples are aligned with the optimal sample. The optimization objective is defined as:
((0) = Egp (epppp/ppp(e/tteeooo00
log (a(108
log
po(zy offictt+1 Prer(x)' 2t cxnctt+1)
po(af x c,,1, Peella 24 t
log
where a denotes the sigmoid function, e represents the input condition, a is a regularization hyper- parameter, Pief refers to the reference probability from the fixed initial denoising model pe. and denotes the model parameters to be updated.
Motivated by SPO, we extend its methodology and apply the principle of stepwise intermediate output optimization to better align the Planner's 's decision-making and tool execution with optimal performance.
METHODOLOGY
Within the PerfGuard framework, we build on the standardized agent system to enable structured, stepwise planning and execution of visual generation tasks, illustrated in Fig. 2 (1) Upon receiv-
3

---

Published as a conference paper at ICLR 2026
User Input
Sehabllitt-Alignee Planning Optimization
[e
Summary
puppy witlring
random
sunglosses and
Target Semantics
nunning
Evaluation
C--1i
Planner
Self- Evaluntor
Analyst
random
Adaptive Preference Updating
C₁
Evalustion Feedback
subbask
sup1
Selt- Eata--valltttrr
Planner
Worker
3001
Wesnh
Wask
0--1
DUp2 1001
rank: 1
Porformance-Awnt Selection Modeling
best tool
A
result
rank: 2
Generation Agent
Visual Content
Ware
M2
MC
update
Rebeory
Figure 2: PerfGuard models tool performance boundaries via PASM to match the most suitable tool for each subtask, maximizing decision efficiency. It further integrates Adaptive Preference Updating to enhance real-world adaptability, and applies CAPO to align planning with performance-aware strategies.
ing multimodal inputs such as images textual instructions, the Analyst parses the information to produce a task summary r", target image semantics s", and evaluation goals g. (2) The Planner uses r", s", and tool performance profiles B to decompose the task into subtasks uee which by the Worker. Evaluation results éx from each stage are fed back to guide subsequent decisions We-1, enabling iterative refinement. (3) The Worker selects appropriate tools from the library to execute each He and generate image outputs or- (4) Each output o, is assessed by the Self-Evaluator across multiple visual dimensions measure alignment with goals g. providing feedback for contin- uous improvement. The definitions of agent roles and the tool library are provided in Appendix A
executed
4.1 PERFORMANCE- SELECTION MODELING
To rigorously define fine-grained tool performance boundaries, we propose the Performance-Aware Selection Modeling strategy. This method systematically aligns the Planner's 's subtasks with the most appropriate tools according to user-specified capability preference dimensions, thereby mitigating planning errors arising from ambiguous definitions of tool capabilities.
Tool Performance Boundaries Precise performance-awarc scheduling begins with fine-graine reeggraiedd performance boundary definition. We construct multi-dimensional scoring system to evaluate tools in the library. Specifically, we design the performance boundary dimensions of tools by referring authoritative benchmarks in image generation and editing. For image generation tools, semantic accuracy is assessed across seven dimensions including color, shape, texture, 2D spatial, 3D spatial, non-spatial semantics, and numeracy, based on T2I-compbench (Huang al. 2023). For image editing tools, effectiveness is evaluated across seven dimensions including addition, removal, re- placement, attribute alteration, motion change, style transfer, and background change, following the evaluation criteria defined in ImgEdit-Benne (Ye et al. 2025).
This mineniddimenioonll scoring framework enables flexible modeling across domains using stan- dardized metrics from large-scale datasets to ensure fairness and objectivity. It supports accurate performance profiling and evolves with new tools and benchmarks. To reduce evaluation costs, we directly adopt scores from T21-compbench and EngEdditBench the performance boundary matri- ces for generation and editing tools. A detailed description of the performance boundary dimensions and their design rationale is provided in Appendix A.7 Serformance-Driverr Selection The Worker Terker leverages predefined tool performance boundary dimensions D to select the most suitable tool for sub-task u, provided by the Plan- ner. For each u. the Worker leverages tool performance profiles to generate a preference weight Whask E Blxd where d denotes the number of performance dimensions. This vector captures the
4

---

Published as a conference paper at ICLR 2026
relative importance of each dimension according the characteristics of ug. Task suitability scores Stoots for all tools are then computed by combining Waak with the tool performance boundary matrix Mp E Rax! (where tools have similar functionalities), enabling performance-driven tool selection. Formally, the computation expressed as:
Wtask = Worker (U,, B, D)
Stoots Weas (M)
R argsort(Staols- descending)
Here, Normalize()) normalizes tool scores across all tools for each performance dimension. Stoots R. represents the weighted suitability of all tools for ue and R provides their descending ranking. denotes the information of the tool library This approach allows the system automatically select tools based on their intrinsic performance characteristics, without requiring users to define task-specific preferences.
4.2 ADAPTIVE PREFERENCE UPDATING
In practice, tool performance boundaries may originate from benchmarks or expert-like evalua- tions based on prior tool usage. These boundaries can contain inaccuracies due to differences in task-relevant dimensions subjective biases. To enhance the accuracy of tool performance bound- ary scores, we propose an Adaptive Preference Updating mechanism that iteratively adjusts the scores based actual tool usage. Specifically, during candidate tool selection, exploration-exploitaiio strategy: the top m tools with the highest weighted preference scores are selected from the library, while additional tools randomly sampled from the remaining to increase the likelihood of selecting potentially high- tools. This mechanism ensures that the tool performance boundary matrix Mp more accurately reflects actual task requirements, enabling adaptive iterative updates:
implement
Raheary top (Stoots) rand,, aand,,Stoolle (Stoots (m + 1 2])
M Normalize M, Weask 77 A)
A Raheory Ractual
m + n
Here, represents the direction coefficient, reflecting the difference between the theoretical ranking Raheory and the actual usage ranking Recuull and denotes the update step size. When tool's actual usage rank surpasses its theoretical rank, its performance boundary score is increased according to the weighted preferences and the distribution of task-specific emphasis across dimensions; other- wise, it is decreased. Ractual is derived from comparative evaluations of multiple candidate outputs conducted by multimodal large model, with the evaluation procedure detailed in Appendix [A.8] For newly added tools lacking sufficient usage experience or benchmark results, we initialize their scores using the average performance boundary scores of similar tools in the corresponding dimen- sions within the current library, ensuring that their potential not overlooked in subsequent tool usage and iterative updates.
4.3 CAPARILITY-ALIGNE PLANNING OPTIMIZATION
To further enhance the Planner's s stepwise decision decision-making and provide indirect feedback on the execution effectiveness of tools selected via Cerformance-Awar Selection Modeling in PerfGuard, we extend Step-aware Preference Optimization (SPO) Liang et al. (2024) and propose Capability- Aligned Planning Optimization (CAPO) for the Planner's autoregressive planning process.
Decision Performance Estimator To evaluate the effectiveness of the Planner's output at each step we adopt the Self-Evaluator Tisaluater as the Planner's Decision Performance Estimator. For each sub-task execution result oz, the Self-Evaluator assesses it based on the corresponding evalua- tion goals g across multiple semantic dimensions:
local TEvaluater ( (Ot, gl local) + y global Tevalumter (O,, galobal)
(4)
1=0
5

---

Published as a conference paper at ICLR 2026
estronsut robble-ctwened astersid itte-shoped balt dangles from the fishing loe
with
fishing
screme forms court scenct smooth nosdin rodling chair is placed beside the
grece mesh funce, with cary woaring oreen glouee sifting peocefully on it, The court features wiboant blue with snap white boundtry lines,
es glowiig Uike spaie Eun swin through casiric teckddeg spinal gulciics and thremering
Image Generation
cut
cought
upper
Got
GenArtiit
Tel:Copeikt
Topoopiit
Others
Ours
Others
Ours
Prompt: speer right
Proept: change child granty
the
Meuetsino
rise the backgraund the say.
Image Editing
Segar EXit
Complitin
OmnilGon
Stap D. Balt
Gendyriee
Others
Ours
Others
Ours
Figure 3: Comparison of PerfGuard's visual results across tasks. Top:
visualization for complex ext-to-image generation. Bottom: visualization for multi-round image editing.
Here, the evaluation goals consist of global semantics gāoobal and local semantics g!ocal weighted
by 7. is the number of local dimensions, and e, denotes the Planner's decision evaluation at step t.
Stepwise Planning Optimization At each step t, the Planner generates candidate sub-tasks fu, uz ...., ., ). Each sub-task produces corresponding output {or of....,.. of which is evalu- ated by the Sell-Evaluator. The sub-task with the highest evaluation score is selected as the winning sample uy, and the lowest-scoring sub-task the losing sample w/. Accordingly, the planner's optimization objective function can be changed from Eq. to:
C(0) (L, of a0",, adf- "5" (B,be-1))
log a a
(alog log pa(uf 8",8" ,,, hr-1) (ue 8" h2-1)
(o | T" s", ho-1) BB,b-1)))
(5)
log Pierluh s*, B, 1,
here, h = {(uo, eo). (4--1,6-) (111 denotes the history of sub-task executions and corre- sponding outputs evaluations up to timestep 1
CAPO enables the Planner to iteratively align sub-task decisions with feedback from the Self- Evaluator, enhancing its awareness of tool execution performance and thereby supporting more accurate and effective task planning.
To improve efficiency in trajectory data collection, a memory retrieval mechanism is integrated. Optimal sub-task sequences from previously successful tasks are stored as reusable experiences. During the generation of new candidate sub-tasks, exploration exploration-exploitation strategy is applied: among i candidates, 3&k are retrieved using CLIP Radford et al. (202T) similarity scores with the current task as the query, selecting the top-5 most similar sequences contextual guidance, while the remaining (1 - k) candidates are generated randomly by the Planner.
EXPERIMENTS
We conducted both qualitative and quantitative comparisons of PerfGuard against various image generation and editing models. The evaluation spans three benchmarks covering different task types: basic image generation (12I-CompBench (Huang et al. (023)). advanced image generation (OnelG- Bench (Chang et al. 2025).. ) and complex image editing (Complex-Fdit (Yang al. 2025b)). De-

---

Published as a conference paper at ICLR 2026
tailed experimental settings, descriptions of baseline methods, agent prompts and instructions, as well as additional results and visualizations, are provided in the supplementary material A.II
5.1 QUALITATIVE RESULTS AND ANALYSIS
We compared the proposed PerfGuard with several existing methods on text-to-image generation and image editing tasks. The visualization results reveal three key observations: i) In text-to-image generation, traditional diffusion models struggle with complex prompts involving multiple entities and detailed attributes. Their limited language understanding leads to poor semantic alignment. For example, FLUX (Labs) 2024) and SD3 (Esser et al. 2024) fail to generate a cat in space- suit. CoT-based methods like T2I-RI (Jiang et al. 2025) and GoT (Fang et al. 2025a) incorporate LLMs, but due to reliance on a single-generation tool, they still miss key elements actions, such as GoT omitting the fishing pose and several specified objects. aggettbasee methods show improve- ment in semantic parsing and tool orchestration. However, GenArtist (Wang et al.) 2024b) lacks aerformance-aware tool selection strategy, resulting in planning errors and missing elements. T21- Copilot (Chen al., 2025a) performs better through multi-module semantic decomposition, but its limited tool diversity still leads to omissions, such as spiral galaxies and green glasses. ii) In multi-round editing tasks, traditional methods like ICEdit (Zhang al. 2025c) and AnySD (Yu et al. 2025) deliver the weakest results. GenArtist, despite using multiple le tools, suffers from poor capability matching, leading to suboptimal edits. StepIX Edit (Liu et al. 2025) benefits from LLM- enhanced understanding of long instructions, but without intelligent planning and execution, it fails to capture key details-for example, the kite does not transform into a dragon. iii) Across both generation and editing tasks, PerfGuard consistently achieves the most accurate and visually aligned outputs. This demonstrates that its performance-guiee guided tool selection enhances single-step execu- tion accuracy and improves overall task planning.
5.2 QUANTITATIVE RESULTS AND ANALYSIS
To comprehensively validate the effectiveness of PerfGuard, we utilize three distinct benchmarks, namely T21- T2I-Compeench (Huang et al.] 2023), OneIG-Bench (Chang et al. 2025), and Complex- Edit (Ynn et al. 2025b), to objectively evaluate its visual reasoning performance across both image generation and editing tasks from multiple perspectives.
Basic Image Generation Comparison We compare the proposed PerfGuard with various image generation methods on basic tasks, as shown in Tab "21-ComBBench evaluates images in terms of attribute binding and object relationships. From the table: (i) Traditional models like FLUX and SD3 remain competitive, with texture, non-spatial, and complexity metrics approaching or surpassing CoT-based methods (T2I-RI, GoT). (ii) CoT-based methods rely on LLM fine-tuning, limiting them to certain tasks; simple prompts may yield overly complex interpretations and inaccurate images. (iii) agent-based methods (GenArtist, T2I-Copilot) use self-correction to regenerate low-quality outputs, improving reliability. (iv) PerfGuard adapts capabilities to match the best-suited model for different tasks, achieving optimal performance across all dimensions.
Advanced Image Generation Comparison To further assess the effectiveness of proposed method in visual reasoning, we evaluated various approaches on OnelG-Bench across diverse sce- narios and complex text prompts, as shown in Tab (i) For more complex generation tasks, FLUX and SD3 show notably lower performance on reasoning metrics, highlighting that integrating LLMS improves the ability to handle complex information. (ii) Regarding alignment accuracy, GoT and GenArtist perform worse than other methods, indicating that single large model has limited capac- ity for complex tasks. (iii) T2I-Copilot and PerfGuard (Ours), leveraging multi-agent collaboration, can plan each step of visual reasoning more precisely when handling cross-domain information, achieving optimal results in both alignment and reasoning metries. (iv) PerfGuard does not show a large margin over other methods in alignment and text metrics due toolset limitations, which cap its generation capabilities. However, its performance-awar tool selection enables smarter planning, leading to clear advantages in reasoning.
Complex Image Editing Comparison We evaluated complex editing performance on the Level- 3 subset of Complex-Edit Edit (Yang et al. (025b) to assess scalability and effectiveness, shown in Tab. 3 Our method selects the best-performing tools based on task-specific capability matching, enabling precise execution across diverse editing types. As result, it achieves the highest scores
7

---

Published as a conference paper at ICLR 2026
Table Basic Image Generation Comparison on T21-CompBench (Huang al.
2023
Attribute Binding
Object Relationship
Model
Complex f
Color
Shape
Texture
Spatial
Non-Spatial
FLUX Labs 2024
0.7407
0,5718
0.6922
0,2863
0.3127
0.3771
SD3 Esser el al. 2024
0.8132
0.5885
0.7334
0.3200
0.3140
0.3703
GoT
Fang ct al. 2025a
0.4793
0.3668
0.4327
0.2238
0.3053
0.3255
T21-RT [Tang
2025
0.8130
0.5852
0.7243
0.3378
0,3090
0.3993
GenArtist Wang ct al.
2024b
0.8482
0.6948
0.7709
0.5437
0.3346
0.4499
T21-Copilot (Chen el al.
2025a
0.8039
0.6120
0.7604
0.3228
0.3379
0.3985
Ours (PerfGuard)
0.8753
0.7366
0.8148
0.6120
0.3754
0.5007
Table 2: Advanced Image Generation Comparison EneIG-Bench
(Chang et al.
2025)
Method
Type
Alignment
Text 1
Reasoning
Style
FLUX Labss. 2024
Diffusion
0.786
0.523
0.253
0.368
SD3 EEser et al.
2024
Diffusion
0.801
0.648
0,279
0.361
GoT Fang et al.
2025a
CoT
0.767
0.504
0.290
0.369
T2I-R1 (Jiang et al.
2025
CoT
0.793
0.662
0.297
0.370
GenArtist (Wang
2024b)
Agent
0.747
0.501
0.285
0.352
T2I-Copilot (Chen et al.
2025a
Agent
0,821
0.679
0,318
0.386
Ours (PerfGuard)
Agent
0.834
0.684
0.350
0.395
Table 3: Complex Image Editing Comparison
Table 4: Ablation Study on Design: C., P., and stand for CAPO, PASM, and APU.
on Complex-Eod Edit
Yang et
2025b)
Method
IF
PQ
IP +
0f
C.
P.
A.
Color
Spatial
Complex
AnySD
4.13
7.14
9.08
6.78
X
0,8239
0.5600
0.4327
StepIX Edit
7.95
8.66
7.70
8.10
x
x
0,8466
0.5756
0.4493
GenArtist
6.14
7.24
6.19
6.52
V
0,8521
0.5919
0,4412
OmniGen
7.52
8.86
8.01
8.13
v
0.8596
0.6005
0.4738
Ours
8.95
8.56
8.84
Ours (full)
0.8753
0.6120
0.5007
in Instruction Following (IF) and Perceptual Quality (PQ). AnySD scores highest in Identity Preser- vation (IP) due to minimal edits in many Level-3 samples, which also leads to a lower IF score. Overall, our approach outperforms all baselines, demonstrating strong generalization across visual reasoning and generation tasks.
5.3 ABLATION STUDY
Ablation on Design We performed ablation experiments on the key modules of PerfGuard (Tab. [4. with the results summarized follows: i) Relying solely conventional text descrip- tions for tool capabilities often leads to misselection, forcing the Worker to perform near-exhaustive attempts, resulting in the lowest performance. ii) Even with the Capability-Aligned Planning Opti- mization mechanism, the lack of an accurate tool selection strategy hinders the Planner's task plan- ning, resulting in limited overall performance improvement. iii) Introducing the Performance-Awar Selection Modeling mechanism significantly improves some metrics, with the color dimension in- creasing by 3.42% and the texture dimension by 5.7%. iv) Further applying Adaptive Preference Updating fine-tur men-tuee preference scores for Planner- generated sub-tasks, enhancing tool selection pre- Planner-generated
cision and raising the complex dimension from 0.4412 to 0.4738. v) The performance-diee driven tool selection strategy improves tool selection accuracy in downstream tasks, enhancing sub-task execu- tion efficiency. This, turn, boosts overall task planning accuracy by the Planner, further optimizing PerfGuard's 's performance with CPAO support.
We conducted a systematic evaluation of tool invoca- tion error rates for different capability
Capability Matching Method Ablation.
strategies on the "complex_ves subset of T21- CompBench (Fig. 4). The results indicate that relying solely on textual descriptions with QWen3- 14B (Yang et al. 2025a) (orange bar) results in high error rate of 77.8%, due the presence of similar tools with differing capability focuses, which makes text-based selection unreliable. Even when assisted by the state-of-the-ar large language model GPT-4o (Fang al. 2025b) (yellow bar),
8

---

Published as a conference paper at ICLR 2026
77.8%
0.14
72.2%
0.15
60-
58.1%
Error Rate (%)
24
30.9%
20
18
Ter Description 3.44
Text Harr-a
met Besotpttin Oplana
Denchmork Speen 1-0
APU- TDWnn 1.88
28
Figure 5: Ablation in Eq. 3 When
Figure 4: Comparison of capability matching methods. Our method substantially reduces tool
? = 0.13, the error rate reaches its minimum of 14.2% at step 800.
selection errors.
User Input
Help me this image
casy festive winter scene: the
background replaced
the
w/O CAPO
wreath basket full ornaments:
and pinecones, candle glows glass, and checolate with candy cone.
orange juice
Figure 6: Visualization the ablation results for CAPO. Operations of Planner. A: Replace the background with snowy scene. B: Change the fruit bowl to festive wreath basket with ornaments and pinecones. C: Place lit candle beside the glass on the table. D: Change the drink to hot chocolate, with candy cane in place of the straw. E: Swap the fruit bowl for festive basket of pinecones and decorations. orange juice with hot chocolate and substitute the straw with festive candy cane. H: Change the background to snowy Christmas setting.
Place candle beside the glass, softly glowing. G: Replace the
the егтог rate remains high 72.2%, highlighting the limitations of LLMs in interpreting capability deseriptions alone. Incorporating an external experience module with QWen3-14B (green bar) re- duces the error rate 68.1% by storing and retrieving historical successful experiences, though the effectiveness is still constrained by retrieval reliability and differences tool capabilities. Lever- aging benchmark-inittalizee performance score matrix with QWen3-14BB (blue bar) perform task-specific capability matching significantly lowers the еттог rate to 30.5%. Further applying the Preference Updating mechanism (purple bar) optimizes the error rate to 14.2%, demonstrating that capability- aware matching combined with adaptive optimization can effectively enhance the accu- racy and robustness of tool selection.
Ablation on Update Step Size To validate the effectiveness of the Adaptive Preference Updating method, as shown in Fig. [5] we studied the impact of different r) values Eq. 3 on tool selection enorr rate using the same dataset as in Fig. Ablation experiments with y set to 0.1, 0.13, and 0.15 show that small 7) (0.1) results in slow error reduction, while a large v (0. 15) accelerates
initial convergence but causes severe oscillations in later stages. In contrast, 77 0.13 achieves a more efficient and stable decrease, reaching the optimal error rate of 14.2% at step 800. These results indicate that 7 0.13 provides balanced trade-off between convergence speed and stability, effectively optimizing tool selection under the current experimental setup.
Ablation Capability-Aligneel Planning Optimization We conducted visual ablation study on the CAPO to examine the impact of Planner training, as shown Fig.66 For fair comparison, we retained only Step1X Edit in the toolset and removed visual supervision from the Self Self-Fvaluator.

---

Published as a conference paper at ICLR 2026
Trated Secrtaaiin
35,02
19999
BAR Parliburd
Plnniing
39.73
0200
(Priiardd
53.38
4000
17.5
Selecting
18-42
66.81
2289
Evaluating
8:72
Geskrtis
9.39
T2l- Copllit
Oura
Reasoning Time(s)
Number of Tools
Figure Time consumption of the inference process for different agent methods.
Figure 8: Token consumption comparison of tool selection methods.
Results show that a trained Planner can perceive tool performance boundaries and understand how operation order affects outcomes. For instance, in Fig. 6 editing the background first reduces the success rate of later steps, as StepIX Edit may introduce inaccuracies that affect other entities like the table. This also suggests that tool limitations can inversely influence planning accuracy.
5.4 EFFICIENCY COMPARISON
Time Consumption Comparison To validate the inference efficiency of PerfGuard, we uniformly employed VL-322B (Qwen3-VL, (025) as the L.M for PerfGuard, GenArtist, and T2i- Copilot. Inference was performed on the same dataset as Tab. [T, and recorded the time con- sumption for task planning, tool selecting, and image evaluatin per round. As shown in Fig. 7 our method exhibits significantly lower time consumption in all three processes compared counter- parts. Particularly, while T21-Copilot's fixed toolset minimizes its tool selection time, GenArtist's detailed textual descriptions tool capabilities require more reasoning time when the tool quan- tity is higher. Conversely, our method, by analyzing sub-tasks and outputting capability- preference weights, achieves a tool selection time substantially lower than GenArtist.
Token Consumption Comparison To demonstrate the efficiency of our method in tool selection, we expand the problem into large-scale tool management within future agent communities. Specifi- cally, we simulate a large tool library using GPT-4o (Fang et al. 2025b), where the number of tools ranges from to 200, generating tool information with textual descriptions and multi multi-dimensional ratings. The specific details are provided in the Supplementary Material A. We the "com- plex_vel" subset from 121-CompBench for the task and compare PerfGuard's performance-driven tool selection with traditional text-based methods, with maximum token output of 8192. We com- pare total token consumption (input and output) between the two methods. Fig. 8 shows: 1) The traditional method consumes more tokens, as struggles to define tool capabilities, resulting in catastrophic growth in token consumption as the number of tools increases. without addressing se- lection correctness. 2) PerfGuard, by focusing on task-specific dimensions, is unaffected by the number of tools. 3) As dimensions increase (from d=4 to d=16), token consumption for PerfGuard mainly increases slowly the input prompts. This demonstrates PerfGuard's superior efficiency in tool management and selection for future agents.
CONCLUSION
In this work, we address key challenge in agent-based visual content generation: the lack of precise modeling of tool performance boundaries, which often leads to unreliable planning and inconsistent execution. By incorporating performance-aware mechanisms and feedback- refinement, framework improves decision reliability and strengthens alignment with definee goals. These results highlight the importance of bridging tool capability understanding with planning logic. Fu- ture efforts will focus on dynamic tool integration and expanding to multimodal tasks to further enhance adaptability and generalization.
10

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGMENTS
This work was supported in part by the Postdoctoral Fellowship Program of the China Postdoctoral Science Foundation (CPSF) under Grant No. GZC20251088.
ETHICS STATEMENT
This work complies with the ICLR Code of Ethics (nttpa://icLr.c///nnl) CadeOfRthi cs). Our study does not involve human subjects, sensitive personal data, or any form of biometric information. All datasets used publicly available and widely adopted in the research community. We have taken care to avoid generating reinforcing harmful content, stereo- lypes, or biases in both model design and evaluation. No proprictary or confidential data was used. There are no known conflicts of interest or external sponsorship that could influence the outcomes of this research. We acknowledge the importance of ethical considerations in Al research and have made efforts to ensure transparency, reproduciiiii,,, and fairness throughout the development of our framework.
REPRODUCIBILITY STATEMENT
We are committed to ensuring the reproducibility of our work. To facilitate this, we have provided comprehensive implementation details, experimental settings, and evaluation protocols in the main paper and appendix. All datasets used in our experiments are publicly available, and we include detailed data preprocessing steps in the supplementary materials. For our proposed framework and algorithms, we have submitted an anonymous source code repository tary materials, which includes scripts for training, evaluation, and visualization. The repository is available at https:: gi thub. com/Fell: (cc.iihhhhann//2Pccm P rd. We also provide abla- tion studies and hyperparameter configurations to support reproducibility. We encourage readers and reviewers to refer to the appendix and supplementary files for further details.
part of the supplemen-
REFERENCES
Saaket Agashe, Kyle Wong, Vincent Tu, Jiachen Yang. Ang Li, and Xin Eric Wang. Agent s2: A compositional generalist-speciee specialist framework for computer use agents. arXiv preprint XXXv:2554.000000 2025.
James Betker, Gabriel Goh, Li Jing, Tim Brooks, Jianfeng Wang, Linjie Li, Long Ouyang, Juntang Zhuang. Joyce Lee, Yufei Guo, al. Improving image generation with better captions. Computer Science. hltps://cdn. openai. com/papers/dull--3. pdf. 2(3):8, 2023.
Jingjing Chang, Yixiao Fang, Peng Xing, Shuhan Wu, Wei Cheng, Rui Wang, Xianfang Zeng, Gang Yu, and Hai-Bao Chen. Oneig- bench: Omni-dimensionn nuanced evaluation for image generation. arXiv preprint urXiv:2506.07977 2025.
Chieh-Yiu Chen, Min Shi, Gong Zhang, and Humphrey Shi. T2i-copilot: A training-free multi- agent text-to-image system for enhanced prompt interpretation and interactive generation. arXiv preprint arXiv:2507.2053: 2025a.
Junsong Chen, Jincheng Yu, Chongjian Ge, Lewei Yao, Enze Xie, Yue Wu, Zhongdao Wang, James Kwok, Ping Luo, Huchuan Lu, and Zhenguo Li. Pixart-a: Fast training of diffusion transformer for photorealistic text-to-image synthesis, 2023. URL https: arxiv. org/ abs/2300. 00426
Zhipeng Chen, Lan Yang, Yonggang Qi, Honggang Zhang, Kaiyue Pang, Ke Li, and Yi-Zhe Song. Versagen: Unleashing versatile visual control for text-to-image synthesis. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pp. 2394-2402, 2025b.
Stefano Curtarolo, Wahyu Setyawan, Gus LW Hart, Michal Jahnatek, Roman V Chepulskii, Richard H Taylor, Shidong Wang, Junkai Xue, Kesong Yang, Ohad Levy, al. Aflow: An
11

---

Published as a conference paper at ICLR 2026
automatic framework for high-throughput materials discovery. Computational Materials Science, 18:218-226, 2012.
Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas Müller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, et al. Scaling rectified flow transformers for high-resolution image synthesis. In Forty-first international conference on machine learning, 2024.
Rongyao Fang, Chengqi Duan, Kun Wang, Linjiang Huang, Hao Li, Shilin Yan, Hao Tian, Xingyu Zeng, Rui Zhao, Jifeng Dai, et al. Got: Unleashing reasoning capability multimodal large language model for visual generation and editing, arXiv preprint arXiv:2503. 2tt.::5503.0069,, 2025a.
Yi Fang, Bowen Jin, Jiacheng Shen, Sirui Ding, Qiaoyu Tan, and Jiawei Han. Graphgpt-o: Syn- ergistic multimodal comprehension and generation on graphs. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 19467-19476, 2025b.
Weixi Feng, Wanrong Zhu, Tsu-jui Fu, Varun Jampani, Arjun Akula, Xuehai He, Sugato Basu, Xin Eric Wang, and William Yang Wang- Layoutgpt: Compositional visual planning and gen- eration with large language models. Advances in Neural Information Processing Systems, 36: 18225-18250, 2023.
Zhi Gao, Yuntao Du, Xintong Zhang, Xiaojian Ma, Wenjuan Han, Song Cang-Chhn Zhu, and Qing Li. Clova: A closed-loop visual assistant with tool usage and update. In Proceedings of the IEEE/CVF conference on computer vision and paltern recognition, pp. 13258-13268, 2024a.
Zhi Gao, Bofei Zhang. Pengxiang Li, Xiaojian Ma, Tao Yuan, Yue Fan, Yuwei Wu, Yunde Jia, Song- Chun Zhu, and Qing Li. Multi-modal agent tuning: Building vim-driven agent for efficient tool usage. arXiv preprint arXiv:2412. 15606, 2024b.
Daya Guo, Dejian Yang, Haowei Zhang. Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-rl: Incentivizing reasoning capability in lims via reinforcement learning. arXiv preprint arXiv:2501. 12948, 2025.
Sirui Hong, Yizhang Lin, Bang Liu, Bangbang Liu, Binhao Wu, Ceyao Zhang, Chenxing Wei, Danyang Li, Jiaqi Chen, Jiayi Zhang, et al. Data interpreter: An Ilm agent for data science. arXiv preprint arXiv:2402. 18679, 2024a.
Sirui Hong, Mingchen Zhuge, Jonathan Chen, Xiawu Zheng, Yuheng Cheng, Ceyao Zhang. Jinlin Wang, Zili Wang, Steven Ka Shing Yau, Zijuan Lin, et al. Metagpt: Meta programming for multi-agent collaborative framework. International Conference on Learning Representations, ICLR, 2024b.
Kaiyi Huang, Kaiyue Sun, Enze Xie, Zhenguo Li, and Xihui Liu. T2i-compbench: A compre- hensive benchmark for open-world compositional text-to-image generation. Advances in Neural Information Processing Systems, 36:78723-7887 78747, 2023.
Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Os- trow, Akila Welihinda, Alan Hayes, Alec Radford, et al. Gpt-4o system card. arXiv preprint arXiv:2410.21276, 2024.
Dongzhi Jiang, Ziyu Guo, Renrui Zhang, Zhuofan Zong, Hao Li, Le Zhuo Shilin Yan, Pheng-Ann Heng, and Hongsheng Li. T2i-r1: Reinforcing image generation with collaborative semantic-level and loken-level cot. arXiv preprint arXiv:2505.00705 2025.
Woosuk Kwon, Zhuohan Li. Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph Gonzalez, Hao Zhang, and Ion Stoica. Efficient memory management for large language model serving with pagedattention. In Proceedings of the 29th symposium on operating systems princi- ples, pp. 611 12666666622 2023.
B.F. Labs. Flux, 2024. hwt.:httt//ggghhu..h.tt www.:oo///llcc-coomcmmmo wwwaacck-fooos--e. laai-laassffoo
12

---

Published as a conference paper at ICLR 2026
Mingcheng Li, Xiaolu Hou, Ziyang Liu, Dingkang Yang, Ziyun Qian, Jiawei Chen, Jinjie Wei, Yue Jiang. Qingyao Xu, and Lihua Zhang. Meed: Multi-agent collaboration based compositional dif- fusion for complex lext-to-image generation. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 1 13263-13272, 2025.
Zhanhao Liang, Yuhui Yuan, Shuyang Gu, Bohan Chen, Tiankai Hang. Ji Li, and Liang Zheng. Step-aware preference optimization: Aligning preference with denoising performance at each step. arXiv preprint arXiv:2406.04314 2(5):7, 2024.
Tsung- Yi Lin, Michael Maire, Serge Belongie, James Hays, Pictro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In European conference on computer vision, pp. 740-755. Springer, 2014.
Shiyu Liu, Yucheng Han, Peng Xing, Fukun Yin, Rui Wang, Wei Cheng, Jiaqi Liao, Yingming Wang, Honghao Fu, Chunrui Han, al. Steplx-edit: A practical framework for general image editing. arXiv preprint Mxiv:2504.117666 2025.
Chong Mou, Xintao Wang, Liangbin Xie, Yanze Wu, Jian Zhang, Zhongang Qi. and Ying Shan. T2i-adapter: Learning adapters to dig out more controllable ability for text-to-image diffusion models. Proceedings of the AAAI conference on artificial intelligence, volume 38, pp. 4296- 4304, 2024.
Chong Mou, Yanze Wu, Wenxu Wu, Zinan Guo, Pengze Zhang, Yufeng Cheng, Yiming Luo, Fei Ding, Shiwen Zhang, Xinghui Li, al. Dreamo: A unified framework for image customization. arXiv preprint arXiv:2504.1699 16915, 2025.
Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy V Vo, Mare Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel HAZIZA, Francisco Massa, Alaaeldin El-Nouby, et al. Dinov2: Learn- ing robust visual features without supervision. Transactions on Machine Learning Research.
Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhom,, Jonas Müller, Joe Penna, and Robin Rombach. Sdxl: Improving latent diffusion models for high-resolution image synthesis. arXiv preprint arX1v:2307.019552 2023.
Qwen3-VL. Qwen3-vl.
2025.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pp- PmLR, 2021.
Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model is secretly reward model. Advances in neural information processing systems, 011553228-374411 2023.
Bram Wallace, Meihua Dang, Rafael Rafailov, Linqi Zhou, Aaron Lou, Senthil Purushwalkam, Stefano Ermon, Caiming Xiong, Shafiq Joty, and Nikhil Naik. Diffusion model alignment using direct preference optimization. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8228-8238, 2024.
Xudong Wang, Trevor Darrell, Sai Saketh Rambhatla, Rohit Girdhar, and Ishan Misra. Instancedif- fusion: Instance-level control for image generation. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 6232-6242, 2024a.
Zhenyu Wang, Aoxue Li, Zhenguo Li, and Xihui Liu. Genartist: Multimodal Ilm as an agent for unified image generation and editing. Advances in Neural Information Processing Systems, 37: 128374-128395. 2024b.
Zhenyu Wang, Enze Xie, Aoxue Li, Zhongdao Wang, Xihui Liu, and Zhenguo Li. Divide and conquer: Language models can plan and self-correct for compositional -ext-to-image generation. arXiv preprint arXiv:2401.1 /5688, 2024c.
13

---

Published as a conference paper at ICLR 2026
Shitao Xiao, Yueze Wang, Junjie Zhou, Huaying Yuan, Xingrun Xing, Ruiran Yan, Chaofan Li, Shuting Wang. Tiejun Huang, and Zheng Liu. Omnigen: Unified image generation. Proceed- ings of the Computer Vision and Pattern Recognition Conference, pp. 15294-1330 2025.
An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al. Qwen3 technical report. arXiv preprint WrXiv:2505.09388 2025a.
Kai Yang, Jian Tao, Jiafei Lyu, Chunjiang Ge, Jiaxin Chen, Weihan Shen, Xiaolong Zhu, and Xiu Li. Using human feedback to fine-tune diffusion models without any reward model. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8941-8951 2024a.
Ling Yang, Zhaochen Yu, Chenlin Meng, Minkai Xu, Stefano Ermon, and Bin Cui. Mastering text- to-image diffusion: Recaptioning, planning, and generating with multimodal llms. In Forty-first International Conference on Machine Learning, 2024b.
Siwei Yang, Mude Hui, Bingchen Zhao, Yuyin Zhou, Nataniel Ruiz, and Cihang Xie. Complex-Edite Cot-like instruction generation for complexity-conttollbbe image editing benchmark. arXiv preprint 1311:25504.11133, 2025b.
Hu Ye, Jun Zhang, Sibo Liu. Xiao Han, and Wei Yang. Ip-adapter: Text compatible image prompt adapter for text-to-image diffusion models. arXiv preprint arXiv:2308.06722 06721,, 2023.
Yang Ye, Xianyi He, Zongjian Li, Bin Lin, Shenghai Yuan, Zhiyuan Yan, Bohan Hou, and Li Yuan. Imgedit: A unified image editing dataset and benchmark. arXiv preprint 2rX5: 2025.
Qifan Yu, Wei Chow, Zhongqi Yue, Kaihang Pan, Yang Wu, Xiaoyang Wan, Juncheng Li, Siliang Tang, Hanwang Zhang, and Yueting Zhuang. Anyedit: Mastering unified high-quality image editing for any idea. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 26125-26133 2025.
Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control text-to-image diffusion models. In Proceedings of the IEEE/CVF international conference computer vision, pp. 3836-3847 2023.
Yao Zhang, Zijian Ma, Yunpu Ma, Zhen Han, Yu Wu, and Volker Tresp. Webpilot: A versatile and autonomous multi-agent system for web task execution with strategic exploration. In Proceedings of the AAAI Conference Artificial Intelligence, volume 39, pp. 23378-23382 2025a.
Yuxuan Zhang, Yirui Yuan, Yiren Song. Haofan Wang, and Jiaming Liu. Easycontrol: Adding efficient and flexible control for diffusion transformer. arXiv preprint arXiw:2503.07027, 2025b.
Zechuan Zhang, Ji Xie, Yu Lu, Zongxin Yang. and Yi Yang. In-context edit: Enabling instructional image editing with in-context generation in large scale diffusion transformer. arXiv preprint arXiv:2504.20690. 2025c.
Haozhe Zhao, Xiaojian Ma, Liang Chen, Shuzheng Si, Rujie Wu, Kaikai An, Peiyu Yu, Minjia Zhang, Qing Li, and Baobao Chang. Ultraedit: Instruction-based fine-grained image editing at scale, 2024. httphttps: /tatpp:://ariv.... wrg.orrg///..... 22000 05282
A APPENDIX
A.1
USE OF LLMS
We LLMs for research ideation. Details described in A.2
14

---

Published as a conference paper at ICLR 2026
A.2 EXPERIMENTAL SETUP
Large Language Model Configuration PerfGuard employs vLLM (won et al. 2023) as its large language model inference engine and adopts MetaGPT (Hong et al. 2024b) as its underlying framework. For agents responsible for multimodal analysis (Analyst and Evaluator), we use GPT- 4o (2024-08-01-prevvi)) -1--vi--revvew) (Hurst et al. (024). whereas agents dedicated to visual reasoning (Planner and Worker) use QWen3-14B Yang et al. 2025a) for trajectory data collection. The collected trajectories are then used to train QWen3-8B 8B through Capability-
Planning Optimization. During later testing and inference, we replace the Planner's 's language model with QWen3-8B.
Tool Library Configuration To ensure PerfGuard possesses sufficient visual reasoning capabili- ties, we configure three types of visual reasoning models in the tool library to validate our approach: "text-to-image tools," "image editing tools." and "customized generation tools." " The "text-to-image
tools" include FLUX (Labs) 2024), SD3 (Esser et al. 2024). PixArtt-a (Chen al. 2023), ) and
SDXL (Podell et al.
2023); the "image editing tools" " include AnySD (Yu et al.
2025 UltraEdit
(Zhao al., 2024), iCEdit (Zhang al. (025c). and Step1X Edit (Liu et al. 2025 y the "cus-
tomized generation tools" include DreamO (Mou et al. 2025), EasyControl (Zhang et al. 2025b),
and IPAdapterPlus (Ye et al. 2023).
Hyperparameter Configuration For Adaptive Preference Updating (Eq. 3) we set the number of candidate tools to 3, selecting the top 2 tools by score (m = 2) and randomly selecting tool (n = 1), with a update step size vi = 0.13. For Capability-Aligned Planning Optimization (Fq. 5 the number of sampled candidate sub-tasks is k = 5, and the proportion of experience-based sub- tasks is 8 = 0.4.
i) For the image generation task, we systematically compared three categories of methods: diffusion model-based approaches (e.g.. FLUX (Labs) 2024). SD3 (Esser et al. (024)) Chain-of-Thie ofoohhh (CoT)-based approaches (e.g., GoT (Fang et al. 2025a). T2I-RI (Jiang et al., ). and agent-based approaches (e.g.. GenArtist (Wang et al. 2024b). T21-Copilot (Chen et al.) 2025ah). By contrasting these strategies, we aim to analyze how different visual reasoning mecha- nisms impact the semantic accuracy of generated images. ii) For the image editing task, we evaluated not only pure diffusion- based methods (c.g.. ICEdit (Zhang et al. 2025c). AnySD (Yu et al. 2025)) but also StepIX XX Edit (Liu et al. 2025), which integrates large language model (LLM) techniques. To ensure a fair comparison, we additionally included general-purpos models capable of both image generation and editing (e.g., GenArtist and OmniGen (Xiao et al. 2025)).
Competitors
A.3 USER STUDY
To validate our method's practical performance, we conducted user study with 15 non-experts using 20 images from the validation subset of MS-COCO (Lin et al., (014). Text descriptions were generated by GPT-4o (Fang et al., 2025b), and participants could either input their own text or use these to create matching images. We also tested our method with image-only input. The visualization results are shown in Fig. [9] 1) When using only text, existing methods often lost key details (c.g., foil and "Bubble Up" label in row 2). while our method preserved critical semantics; 2) Interestingly, with image-only input, PerfGuard still achieved accurate generation through fine- grained visual understanding, demonstrating its robust cross-modal understanding capability
We conducted a more detailed evaluation of the images generated by users, which included objective assessments using DINOV2 score (Oquab et al.) (DINO, which is used to measure the semantic sim- ilarity between the generated image and the reference image.) and CLIP score (Radford al.. 2021) (CLIP, which is used to measure the semantic similarity between the generated image and the given text.), as well as subjective evaluations from users. In the subjective evaluation, users rated the gen- crated images on scale of to 5 based on condition match (Cond., which is the user's score of how well the generated image matches the given conditions.) and aesthetic quality (Aesthetic, which is the user's score for the overall aesthetic appeal of the generated image.). After the experiment, users selected the best image generation tool. The experimental results, as shown in Tab. 55] indicate that the objective evaluation results, in terms of image-image consistency and text-image consistency, align closely with the trends presented in the comparative experiment in Tab. T and Tab. 2 In the subjective evaluation, GenArtist scored the lowest in condition match and aesthetic appeal due to its lack of accurate understanding and optimization of information. T21-Copilot, which focuses on image generation tasks, performed better in condition match and aesthetic appeal by optimizing and
15

---

Published as a conference paper at ICLR 2026
close-up, black-and-whit photograph featuring sliced, stuffed bread roll with its dark filling clearly visible on a white plate, positioned next to small white container.
grilled hot dog topped with mustard, resting on piece of foil, sits quietly on car's hood alongside bottle of green soda labeled "Bubble Up."
Reference Image
Ours
Ours
GenArtist
T21-Copilot
(only image)
(only text)
Figure 9: The visualization of the user study, where PerfGuard, GenArtist, and T2I-Copilot are used to generate output images that are most similar the reference images from the MS- COCO dataset. Among them, PerfGuard compares the results generated using only image input and only text input.
Table 5: User study of different methods.
Method
DINO
CLIP
Condition
Aesthetic
User-Pref
GenArtist
0.7440
0.3401
3.15
2.53
6.7%
T2I-Copilot
0.8134
0.3652
3.42
3.69
20.0%
Ours (Only text)
0.8467
0.3723
3.67
3.88
73.3%
Ours (Only image)
0.8716
0.3962
3.80
4.12
enriching the input information. However, these methods lack accurate understanding of the infor- mation and tool selection during the actual generation process, which is why PerfGuard achieved the best subjective and objective results. Furthermore, we asked users to choose the best tool for each image generated by the three methods and recorded the proportion of users who favored each tool (User-Pref). Among the tested samples, 73.3% of users chose PerfGuard, indicating that our method provided the best user experience across various input formats.
A.4 PERFORMANCE COMPARISON AGENTS USING THE SAME LLM
To balance multimodal analysis and inference performance while ensuring faimess in performance comparison, we replaced the LLMs of GenArtist T2I-Copilot, and PerfGuard with Qwen3-VL-333 and compared their performance on image generation tasks. The remaining experimental configura- tions and datasets were consistent with those in Tab. I The experimental results, shown in Tab, 6 in- dicate the following: 1) GenArtist and T2I-Copilot heavily rely on the performance of closed-soures source LLMs for task analysis and planning. As result, due to their lower LLM generalization ability, their performance metrics suffer significant decline. 2) our approach, the closed-source MLLM only used as image interpreter to assist the LLM in performing the analysis and planning process. Therefore, when all LLM modules are replaced with Qwen3- VL-32B, the performance degradation is minimal. 3) Even when GPT-4o is replaced with Dive33-L-3222 the trend remains consistent with that shown Tab. [ where our method outperforms GenArtist and T2I-Copilot across multiple metrics.
16

---

Published as a conference paper at ICLR 2026
Table 6: Performance comparison of agents using the same LLM.
Method
Color
Spatial
Complex
GenArtist (Qwen3-VL-
0.5670
0.2928
0.2321
T2I-Copilot Miwven3-VL-222-
0.6755
0.2257
0.2461
Ours (Owen3-VI-32B)
0.8500
0.5481
0.4538
Ours (Original Config)
0.8753
0.6120
0.5007
Table 7: Performance comparison of PerfGuard using different LLMs
Method
Color
Spatial
Complex
Degradation (Complex)
GenArtist (Qwen3- (Qwen3-VL-32B) VL-32B)
0.5670
0.2928
0.2321
48.4%
GenArtist (Original Config)
0.8482
0.5437
0.4499
T2I-Copilot (Qwen3-VL-32B
0.6755
0.2257
0.2461
38.2%
Cop-Copilot (Original Config)
0.8039
0.3228
0.3985
Ours VL-333
0,8500
0.5481
0,4538
9.4%
Ours (GPT-40))
0.8577
0.6004
0,4813
3.9% %
Ours (Original Config)
0.8753
0.6120
0.5007
A.5 ABLATION STUDY WITH DIFFERENT LLMS
We validated the generalization and robustness of the PerfGuard framework across different LLMs by replacing the MLLM configuration with either the closed-source GPT-4o or the open-source Qwen3-VL-38B. The ablation study followed the same experimental setup as the comparison ex- periments in Tab[[ with the only difference being the replacement of the MLLM. Since GPT-4o is a sourcd-source model, we could not implement the CAPO training process as originally proposed. Therefore, an experience replay mechanism was adopted to ensure that the performance of Perf- Guard based on GPT-4o closely matched the planning accuracy of CAPO. The experimental results, shown in Tab. 7 indicate that replacing all PerfGuard modules with Qwen3-VL-38B GPT-4o led to a slight performance degradation. Compared to the performance degradation rates GenArtist and T21- T21-Copliot, our method shows better adaptability different LLMs. For instance, when using Qwen3-VL-38B VL-38B in the Complex dimension of T21-CompBench. GenArtist's performance degrades by about 48.4%, T21-Copliot' 's by about 38.2%, while PerfGuard's performance degrades by only 9.4%. This demonstrates that the proposed method achieves superior performance across various LLM settings (both open-source and closed-source). further confirming its robustness and general- ization capability across different LLMs.
A.6 LIMITATIONS AND CHALLENGES
The PerfGuard method we proposed is a preliminary attempt to address the issue of accurate tool selection in agent systems. However, there are still limitations and challenges that need be solved: the Performance-Aware Selection Modeling strategy we proposed relies on existing benchmark scores to initialize the tool performance boundary matrix. However, domains outside of visual content generation, high-quality benchmarks may not always be available for efficient initialization of the capability boundary matrix. Therefore, future work will focus on expanding this method to other domains, such as visual reasoning tasks, and advancing it to the level of multi-agent capability discovery for better multi-agent and multi-tool capability matching and collaboration.
A.7 DESIGN AND SPECIHICATION OF PERFORMANCE BOUNDARIES IN PERFGUAR
We initialize tool performance boundaries by leveraging existing calti-dimensional evaluation benchmarks conducted on large-scale datasets. Specifically, we adopt the evaluation dimensions from (2i-compbench (Huang et al. (203) and ImgEdit- Bench (Ye et al., 2025) as the performance boundary dimensions for image generation and editing tools in the library, respectively. The detailed definitions are as follows:
Image generation performance boundary dimensions:
17

---

Published as a conference paper at ICLR 2026
color: indicates the accuracy of the object's color in the generated image.
shape: indicates the accuracy of the object's shape in the generated image.
texture: indicates the accuracy of the object's material or surface quality in the generated image, such "wooden", "metallic", etc.
2D-spatial: indicates the accuracy of the 2D spatial relationships between objects in the generated image, such as "on side of", "on the left", "on the top of"" "next to", etc.
3D-spatial: indicates the accuracy of the 3D spatial relationships between objects in the generated image, such as "behind", "hidden by", "in front of"" etc.
numeracy: indicates the accuracy of the number of objects the generated image.
non-spatial: indicates the accuracy non-spatial relationships between objects in the gen- erated image, such as "A is holding B", "C is looking D", "E is sitting F", etc.
Image editing performance boundary dimensions:
addition: indicates the accuracy of adding objects to the image.
removement: indicates the accuracy of removing objects from the image.
replacement: indicates the accuracy of replacing objects in the image.
attribute-alter: indicates the accuracy of modifying the attributes of objects in the image.
motion-change: indicates the accuracy modifying the actions, movements, or spatial positions of objects in the image.
style-transfer: indicates the accuracy of modifying the overall style of the image.
background-change: indicates the accuracy of modifying the background of the image.
A.8 DEFINITION OF RACTUAL IN ADAPTIVE PREFERENCE UPDATING
For the actual usage ranking Ractual in Eq 3 we employ GPT-4o directly compare the outputs of multiple candidate tools and evaluate their effectiveness in executing the given subtask. The prompt configured as follows:
Prompt Engineering of Actual Usage Ranking
task: task
Multiple output images are generated after executing the task. Please compare ONLY these output images and analyze provide their ranking from best worst. Do not include any images outside of this list in your analysis or ranking. If multiple images meet the task criteria equally well, prioritize the image that appears most natural and visually coherent.
A.9 PROMPT ENGINEERING
The agent system designed in this work consists of four roles: Analyst, Planner, Worker, and Self- Evaluator. Their prompt engineering strategies will be presented in this subsection.
Prompt Engineering Analyst
Please analyze the user's needs based the provided content and summarize their quirements.
specific image is referenced, the path to the reference image must be specified. **No assumptions are allowed about the user-provided information; the output must closely align with the user's given information.** *
**The output must be derived through precise and correct reasoning, rather than copying the user's input, **
Transform the user input into concrete visual elements for the final image, avoiding overly simple or abstract terms.
18

---

Published as a conference paper at ICLR 2026
The output task must be **recise and concise, within 20 tokens
Output the task in the format: <task>> Your summary task </task>.
2. Please provide the semantics of the final output image (i.e., what the final rendered image looks like) in textual form.
The output semantic should be described in terms of key objects in the image, their at- tributes (numeracy, categories, color, texture, etc.), spatial relationships, background, and image style, etc..
The output semantic must be *precise and concise, within 20 tokens**. Output the semantic in the format: Esemantic>-Texteee semantic information of the target image. </semantic>.
Prompt Engineering of Planner
Task: Current image semantics:
Target image semanties:
Available features:
Image generation: Create an image strietly matching the target semantics. Specify only required dimensions: quantity (use "exactly" if needed), position, attributes, material, color, style, lighting, or semantic relationships.
2. Image editing: Modify an existing image to gradually match target semantics. Adjust only necessary dimensions; do not add unrelated objects. Regeneration of the whole image is not allowed.
Instructions:
Analyze the target image semanties, task requirements, and historical operation infor- mation (if available).
Provide the next processing step to gradually meet the final task requirements through subsequent multi-round interactions.
Each operation should be concise (less 30 words) while retaining essential elements. Use precise instructions (e.g., "remove the apple on the far right"), avoiding vague expres- sions.
effective operation per round; if the task is complex and model capability allows, multiple operations can be included in one round.
Preferably output single
For generation tasks, output images should be natural and harmonious.
For editing tasks, do not regenerate images arbitrarily; only modify necessary parts.
If multiple operations can achieve the task, select the one with the highest success rate.
Specify dependencies clearly: Edepend>Nones /eoone//depenn depend> if independent, or <depend>round X</depend> based on a previous round.
Prompt Engineering of Worker
Task:
1. You have two types of tools choose from: textZimage-ennrratonn tool, image-editing tool. Please choose the appropriate tool-type based on the task requirements. Output XML
format:
<category>your select tools tools-type<ccateeoryy
2. you choose the text2image-genereee assign weights for the following preferences:
tool category, analyze the task and 2D- 10-spatiall 2D spatial relationships (e.g., on the left, next to)
color', 'shape', 'tetture' '2D-spatial', 3DDsspatial' numeracy', nponsipatial
color: object's color requirement
shape: object's shape requirement
texture: material/surface quality (e.g., wooden, metallic)
19

---

Published as a conference paper at ICLR 2026
3D- 3D-spation ial: 3D spatial relationships (e.g., behind, in front of)
nuneracy: number of objects
hon-spatial: non-spatial relationships (e.g., A is holding B)
3. If you choose the Amage--ditiig tool category, analyze the task and assign weights for:
''dditton', 'emovement', replacemennt' 'attribute-alter', mot on mmotion-chang', "style-transfer' "background-change"
addition: adding objects
removement: removing objects
repl acement: replacing objects
attribute-alter. modifying object attributes
motion-change: changing actions positions
style-transfer: modifying image style
hetgggroond-cchaagee modifying background
Notes:
Weights range from 0 to I; higher values indicate greater importance.
The sum of all weights must be
Assign very low values (even to unimportant dimensions.
Ensure weights strictly reflect task requirements.
Do not confuse preferences between the two tool types.
A.10 CONFIGURATIONS RELATED TO THE TooL SELECTION SIMULATION EXPERIMENT IN SEC.
5.4
The tool selection prompt engineering based on textual descriptions.
The tool selection prompt engineering based on textual descriptions.
You are an expert tool selection agent.
Task Description:
[task_ _desckiptipnn
Below is the complete list available tools in your tool library.
These tools provided in the attached file "(tool_file".
TOOLS BEGIN
toot
TOOLS END
Each tool includes:
A precise tool name
A description its capabilities
The type of task it is designed for
Your objective:
Carefully analyze the task and select the SINGLE most appropriate tool.
Instructions for reasoning:
1. For each tool in the library, analyze whether it is suitable for the given task.
2. For each tool, provide brief reasoning:
Why this tool is suitable (or not suitable) for the task.
3. After analyzing all tools, give your final choice of the SINGLE most appropriate tool.
4. Output the final answer ONLY
XML tag in the following form:
<tool Y TOOL_ NAMEA to
Rules:
20

---

Published as a conference paper at ICLR 2026
You must reason about all tools individually before making the final choice.
Provide clear and concise reasoning for each tool.
Do not skip any tool in the analysis.
-The XML output at the end must exactly match the chosen tool's name.
No explanation outside of the tool analysis and final XML output.
The tool selection prompt engineering based on performance-driven selection.
The tool selection prompt engineering based on performance-driven selection.
You are an expert tool selection agent.
Task Description:
(task description)
Your objective:
1. Carefully analyze the task requirements.
2. Analyze the overall task, firstly.
3. For ALL relevant dimensions (both Text2Image and Imagenditing dimensions):
Analyze the importance of this dimension for the given task.
Assign weight between 0 and based on its importance. Provide brief reasoning for the assigned weight. Text2Image dimensions: (TEXTTIMAGE__ AGEEDIEESSSONS ImageEditing dimensions: (IMAGE EDITEEDDDDDDME DIMENSION) Instructions for output:
1. First, reason about each dimension individually. For example:
Dimension: color
Importance: High
Reason: The task emphasizes vivid and harmonious colors in the wizard's clothing and mag- ical effects.
Assigned weight: 0.25
2. Repeat for all dimensions, even if the weight is O.
3. After analyzing all dimensions, give the final category and weights in **exact XML.
*o:
<preference>
<diml>weeight<<ddim>>>> <!-- Reason: explanation for weight -->
<cim2>weight<</dim>> <!-- --- Reason: explanation for weight -->
</preference>
Rules:
Weights must sum to 1.
Use all relevant dimensions for weighting: irrelevant dimensions can have weight 0.
Provide concise reasoning for each dimension in the XML comment.
Only output the XML; do not include explanations outside the XML.
A.11
MORE VISUALIZATION RESULTS
We conducted extensive visualizations of PerfGuard across various image-related tasks. Fig. 10 presents multiple examples from the image editing task, Fig. [1 showcases several cases from the text-to-image generation task, Fig. 12 illustrates a range of customized image generation results, and Fig. 13 depicts multiple instances of PerfGuard's error correction workflow during image generation.
21

---

Published as a conference paper at ICLR 2026
Modify the original image by switching the wooden platter to one with a polished marble finish, removing the middle fork, and changing the chocolate cupcake a lighter caramel color.
Original image
Replace the platter with polished marble.
Remove the middle fork.
Lighten the cupcake to caramel.
Edit the original image by changing the clothing to denim jacket texture, replacing background with concent stage, and updating guitar body to vibrant green.
Original image
Change clothing to denim texture.
Replace background with concert stage,
Update guitar color to vibrant green.
Modify the original image to make the fire hydrant appear aged with rust, change the women's shirt to soft-blue color, set scene against cityscape background.
Original image
Age the fire hydrant with rust.
Change the woman's shirt to soft-blue,
Replace the background with cityscape.
the original image by changing the chocolate cake to blue, removing the background flogs, adding flower bouquet falling confetti,
Original image
Recolor the cake to blue.
Remove the background flags,
Add a bouquet and falling confetti,
Edit the original image by redesigning fire hydrant stor pattern, replacing the ground with lush green grass where white rabbit sits, and setting the dusk.
Original image
Redesign the fire hydrant with pattern.
Replace the ground with green grass and add white rabbit.
Change time of day to dusk,
Figure 10: Visualization examples of the image editing task
22

---

Published as a conference paper at ICLR 2026
Create an image of a woman a yellow hat and dress sitting on garden bench, holding basket of red roses. The scene should have classic, romantic ambiance.
A woman in yellow hat and dress sits on stone bench in a garden, holding a basket of roses.
Add white wrought-iron arch and fountain to the background.
Change the roses in the basket to red roses,
Cover the bench with ivy.
Create detailed close-up image rustic, weathered woaden wall that showcases grain and knots. image should convey sense authenticity and being well-used, with faint water stains along the top indicating its age and the effects of the elements.
A rustic wooden wall with weathered planks, visible wood grain, knots, and warm brown tones.
Mount two vintage metal hooks on the left and hang woven strow hat below them.
Add trailing ivy growing from the right-side gap the wall.
Include subtle stain marks along the top edge of the wall.
Generate landscape painting, The should feature winding path leading through foreground flowers towards cozy red-roafed houses nestled among trees, hazy mountains in the distance. elderly person should working field nearby.
serene landscape with vibront magento flowers, trees, nestled red- roofed houses, distant hazy mountains, and blue sky.
Change season to autumn.
Add winding stone path.
elderly person working field the houses.
Generate tranquil, atmospheric night scene of modern bedroom. The composition should focus on nightstand by large window, holding deep blue backpack, soft pink toothbrush, and glowing smartphone. Outside the window, show a rainy night view.
tranquil night scene of a
room with deep blue backpack and a toothbrush on night- stand, by a large window.
Change toothbrush color pink.
glowing smart- phone the nightstand.
Replace the starry sky with rainy window view.
Figure 11: Visualization examples of the text to image generation task
23

---

Published as a conference paper at ICLR 2026
This searching for treasures on floor.
toy is exploring on the floor.
Add some schools fish above toy.
Remove the child.
Add treasures.
Generate image in style, cat wearing colorful hot by a riverbank with flowers on the shore sparkling water.
Make show sparkling effect.
Add several blooming flowers by the river.
Make river water show sparkding effect.
river.
redifurred, beret-wearing tiger that rooning.
Replace the the tiger with tiger.
Wear beret.
Figure 12: Visualization examples of the customized image generation task
24

---

Published as a conference paper at ICLR 2026
There are five pianos stage, surrounded some walnuts and fresh flowers, and stage lights shine on the pionos.
ERROR: pionas the image.
pianos.
some walnuts.
instead of five.
Increose the stage lighting effects.
Add fresh flowers
Erase
ERROR:The motor- cycle stationary and missing huge billboard.
A cyberpunk-style city street scene, with focus speeding motorcycle that leaves a long trail of light behind, set against backdrop of huge holographic billboards.
ERROR:The motor- cycle stationary and the overall picture is not harmonious.
Add light trails and billboards.
Rich and mone detailee characterization prompts: Generate a vibront, neon-drenched cyberpunk city street of night. futuristic motorcycle speeds down the street, leaving bright light trails behind it. In the bockground. massive, glowing holagraphic advertisements dominate the skyline.
ERROR:The
A cat wearing magic floating mid- ain, with twinkling constantly falling down.
is not in the air
ERROR:The cat's bady is severely distorted.
Place the cat
more detailed characterization prompts: An elegent cat floating effortlessly mid-air, donning starry wizard hat. The cat sits in cross- legged, weightless pose as the hat's brim, surrounded by soft, luminous clouds.
twinkling stars from
Figure 13: Visualization examples of multiple instances of PerfGuard's error correction workflow
during image generation
25


Reviews:
[{"review_id": "2sPe9atMy1", "reviewer": "Reviewer_t5Fu", "summary": "The paper proposes PerfGuard, a performance-aware agent framework for visual content generation. It aims to address the gap in current agent-based systems, which often operate under the assumption that tool executions are always successful. PerfGuard incorporates three core mechanisms: Performance-Aware Selection Modeling (PASM), Adaptive Preference Update (APU), and Capability-Aligned Planning Optimization (CAPO). These mechanisms model tool performance boundaries, optimize tool selection, and align task planning with execution outcomes, providing enhanced reliability in visual content generation tasks.", "questions": "see the weaknesses", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The framework introduces a novel approach to visual content generation by incorporating performance-aware mechanisms.\n\n2. The introduction of a multi-dimensional performance evaluation system for tools is a clear strength. It provides a more detailed and reliable method for tool selection, addressing the limitations of previous systems that relied on general textual descriptions.", "weaknesses": "1. The method heavily relies on the context-learning capabilities of large language models (LLMs). While this is an interesting approach, it may not offer substantial improvements over previous systems that already utilize LLMs for similar tasks.\n\n2. The paper does not sufficiently discuss existing methods, particularly in relation to visual content editing tools. For example, \"CLOVA: A Closed-Loop Visual Assistant with Tool Usage and Update\" (CVPR 2024) has already considered tool evaluation and updates. This overlap should be addressed in the discussion to strengthen the paper's contribution.\n\n3. While LLMs have shown remarkable capabilities in various domains, their use in image editing tool evaluation and updates is still in the early stages. The paper should address the potential limitations and challenges of this approach, including issues related to LLM performance, tool generalization, and robustness in diverse image editing tasks. The authors should try different LLMs to show the robustness of this method.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "3HH308E9ze", "reviewer": "Reviewer_8HZR", "summary": "This paper proposes PerfGuard, a performance-aware agent framework for visual content generation (AIGC). The core idea is to move beyond relying on vague text descriptions for tool selection. Instead, it proposes three mechanisms: (1) PASM, which models tool capabilities using a fine-grained, multi-dimensional scoring matrix ($M_p$); (2) APU, an online feedback mechanism to dynamically update this matrix based on execution outcomes; and (3) CAPO, a training strategy (inspired by DPO/SPO) to fine-tune the agent's planner to align its sub-task decomposition with the known tool capabilities.\n\nThe authors claim that this framework leads to SOTA performance on several AIGC benchmarks compared to existing agent-based methods.", "questions": "[Re: Unfair Comparison]: Given the critical dependence on GPT-4o, can the authors please provide an ablation study where the Self-Evaluator and Analyst are replaced with a strong open-source VLM (e.g., LLaVA, Qwen-VL) or a non-LLM metric (e.g., CLIP Score)? This is essential to prove that the performance gains come from the PerfGuard framework itself and not from GPT-4o.\n\n[Re: Disentanglement]: To prove the synergistic value of the framework, can the authors please provide the missing ablation: Baseline + CAPO only (i.e., the smart planner with the original naive, text-based tool selector)? This would allow us to understand the independent contribution of the planner and selector, and to check for any \"1+1>2\" effect.\n\n[Re: Cold-Start Problem]: Can the authors please elaborate on the scalability of their approach? How would one feasibly construct the $M_p$ matrix in a new domain that lacks a comprehensive, multi-dimensional benchmark like T2I-CompBench? Acknowledging this as a major limitation would be a start.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "Novel and Important Problem: The paper correctly identifies a key weakness in current agent frameworks: the \"idealized assumption\" of tool capabilities. The core idea of modeling fine-grained \"performance boundaries\" (PASM) is a significant and logical step forward for the field.\n\nComplete Framework: The authors propose a complete, end-to-end vision, including selection (PASM), online adaptation (APU), and planner alignment (CAPO). This holistic approach is commendable.\n\nClear Presentation: The paper is well-written, and the core concepts are easy to grasp.", "weaknesses": "Unfair Comparison & Critical Confounding Variable: This is the most severe flaw. According to Appendix A.2, the framework uses GPT-4o for the Analyst and Self-Evaluator roles. This Self-Evaluator (GPT-4o) provides the entire learning signal for both the CAPO training (via Eq. 4 & 5) and the APU updates (via $R_{actual}$). The Analyst (GPT-4o) is also used during inference. The baseline methods (e.g., GenArtist) are not afforded this powerful, external model. Therefore, the SOTA claims are invalid. The paper is essentially comparing a system distilled from and assisted by GPT-4o against open-source models, which is an unfair comparison.\n\nLack of Disentanglement (The \"1+1>2\" Problem): The ablation study in Table 4 fails to answer the most critical question: where does the performance gain come from? Is it the better planner (CAPO) or the better selector (PASM)? The gains might just be additive. A crucial experiment is missing: Smart Planner (CAPO) + Naive Selector (Text-based). Without this, the value of the complex PASM/APU system is unclear, as is the value of the CAPO training.\n\nPracticality and the \"Cold-Start\" Problem: The paper's claim of \"practical utility\" is undermined by its reliance on pre-existing benchmarks (T2I-CompBench, ImgEdit-Bench) to create the $M_p$ matrix (Section 4.1). This completely avoids the hardest and most expensive part of the problem: how to build this fine-grained performance matrix from scratch for a new domain or new set of tools. This significant limitation (which I know from experience is a major barrier) is not adequately discussed.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "2y3ea66zpo", "reviewer": "Reviewer_SDJK", "summary": "This paper presents PerfGuard, a performance-aware agent framework for visual content generation. The key insight is that existing agent systems assume tools always work perfectly and use vague text descriptions that don't specify what each tool actually does well. To fix this, PerfGuard introduces three mechanisms: (1) PASM replaces text descriptions with multi-dimensional performance scores, (2) APU dynamically adjusts these scores based on actual execution results, and (3) CAPO aligns the planning process with performance-aware tool selection. Experiments on three benchmarks show improvements in tool selection accuracy and output quality for both image generation and editing tasks.", "questions": "1. What criteria did you use to select tools for the library? Were any potentially useful tools excluded, and if so, why?\n\n2. You found η=0.13 works best in your experiments. Does this parameter need tuning for different task types or tool libraries? How would users know what value to use?\n\n3. Any plans to add user studies? It would be interesting to see if humans prefer PerfGuard's outputs over baselines.\n\n4. How does computational cost scale with tool library size? What's the latency for tool selection and planning compared to baseline agent systems?\n\n5. Could the performance dimensions extend to other visual tasks like video generation or 3D content creation? What modifications would that require? (Just for discussion)", "limitations": "", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "- The paper tackles an overlooked problem in agent-based visual generation. Most prior work assumes tools execute reliably, but this paper explicitly models tool performance boundaries. The combination of PASM, APU, and CAPO feels well thought out and addresses the problem systematically rather than with isolated fixes.\n\n- The experiments are thorough. The authors compare against multiple baselines including diffusion models, CoT-based methods, and other agent systems across different task types. The ablation studies clearly show what each component contributes. I also appreciate the hyperparameter analysis, which helps with reproducibility.\n\n- The paper is well organized. The framework architecture is clearly explained, with distinct agent roles and mechanisms. Figures 1 and 2 effectively illustrate the workflow. The appendix provides good implementation details.", "weaknesses": "1. The tool library only includes a specific set of popular models like FLUX, SD3, and Step1X Edit. I'm curious why certain relevant tools are excluded. For example, what about newer 2025 models or domain-specific editing tools? This raises questions about how well the framework generalizes to different tool ecosystems.\n\n2. When adding new tools, the method initializes performance scores by averaging similar tools. This seems reasonable but might miss unique capabilities of novel tools. Have the authors tried other initialization strategies, like few-shot evaluation or transfer learning? A comparison would strengthen this design choice.\n\n3. The quantitative metrics are comprehensive, but there's no user study. Since visual generation is inherently subjective, getting human feedback on the outputs would make the claims more convincing. Even a small preference study would help.\n\n4. How does PerfGuard scale when the tool library grows large? If you have hundreds of tools, does the multi-dimensional scoring and adaptive updating become computationally expensive? The paper doesn't really discuss this.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)