Rethinking Global Text Conditioning in Diffusion Transformers
Authors: Nikita Starodubcev
Daniil Pakhomov
Zongze Wu
Ilya Drobyshevskiy
Yuchen Liu
Zhonghao Wang
Yuqian Zhou
Zhe Lin
Dmitry Baranchuk
Venue: ICLR 2026 Poster (2026)
Keywords: diffusion models
image and video generation
Primary area: generative models

Abstract: Diffusion transformers typically incorporate textual information via (i) attention layers and (ii) a modulation mechanism using a pooled text embedding. Nevertheless, recent approaches discard modulation-based text conditioning and rely exclusively on attention.
In this paper, we address whether modulation-based text conditioning is necessary and whether it can provide any performance advantage. Our analysis shows that, in its conventional usage, the pooled embedding contributes little to overall performance, suggesting that attention alone is generally sufficient for faithfully propagating prompt information. However, we reveal that the pooled embedding can provide significant gains when used from a different perspective—serving as guidance and enabling controllable shifts toward more desirable properties. This approach is training-free, simple to implement, incurs negligible runtime overhead, and can be applied to various diffusion models, bringing improvements across diverse tasks, including text-to-image/video generation and image editing.

Published as a conference paper at ICLR 2026
RETHINKING GLOBAL TEXT CONDITIONING IN DIFFUSION TRANSFORMERS
Nikita Starodubcey Daniil Pakhomov2
Zongze Wu²
Ilya Drobyshevskiy Dmitry Baranchuk!
Yuchen Liu2
Zhonghao Wang²
Yuqian Zhou² Zhe Lin?
Yandex Research, Adobe Research
https://gitann wwm com /qui/quuuuu ck j kee/modu 1 at on gun-gudddaes
ABSTRACT
Diffusion transformers typically incorporate textual information via (i) attention layers and (ii) modulation mechanism using pooled text embedding. Never- theless, recent approaches discard modulation-based text conditioning and rely exclusively attention. In this paper. address whether modulation-ba based text conditioning is necessary and whether it can provide any performance ad- vantage. Our analysis shows that, in its conventional usage, the pooled embedding contributes little to overall performance, suggesting that attention alone is generally sufficient for faithfully propagating prompt information. However, we reveal that the pooled embedding can provide significant gains when used from a different seapeettvee-srvvinn as guidance and enabling controllable shifts toward more desirable properties. This approach is training- ffree, simple to implement, incurs negligible runtime overhead, and can be applied to various diffusion models, bring- ing improvements across diverse tasks, including text-to-image/videe generation and image editing.
INTRODUCTION
Since the pioneering works on diffusion models (DMs) (Ho et al., 2020 Song al. 2020), the UNet architecture (Ronneberger et al. (215)) has served as the dominant backbone for diffusion- based image generation. This trend extended to text-to-image models (Saharia et al., 2022 Nichol ct al. 2021), which employ UNet-based architectures (Rombach al. 2022) and incorporate the CLIP text encoder (Radford et al.] 2021) to condition the model on text sequences through the attention mechanism (Vaswani a (017). Later, models such as |Podell et al. (2023) began to incorporate the pooled CLIP embedding via modulation mechanisms (Karras et al.] 2017 2019). ) in addition the token-wise text embeddings. More recently, works including Labs et 2025 ;
(aa Labs (2024); Esser el al. 20 Kong et al. (2024); Cai et al. (2025) have adopted transformer-basee
architectures (Peebles Xie (023) while retaining modulation- based text conditioning. Recent models (Wan et al.. 2025| Wu et al. 2025 Agarwal et al. [225]] Xie et al. (2024) discard global text conditioning, achieving comparable text alignment by relying solely on attention. This transition raises questions about the role and necessity of global text conditioning, which we aim to explore.
We observe that, at first glance, modulation-based text conditioning appears non-contributory and attention alone is sufficient to capture textual information. However, we argue that it is premature to discard global text conditioning and that it should instead be leveraged from a different perspective. Specifically, we draw inspiration from the interpretability of the modulation mechanism (Karras et al.) 2019) and the ability of CLIP to control it (Garibi et al. (025). We suggest that the pooled text embedding can act corrector, adjusting the diffusion trajectory toward better modes.
In summary, our contributions are as follows: (1) We conduct an in-depth analysis of global text conditioning in contemporary DMs and find that it plays only minor role relative to attention-based text conditioning. (2) We show that global text conditioning can yield significant improvements when viewed from the perspective of modulation guidance. Furthermore, we enhance its effectiveness by proposing dynamic strategies. (3) We introduce techniques for integrating the pooled embedding
WWrk partially done during an internship Adobe Research

---

Published as a conference paper at ICLR 2026
into fully attention-basel models, thereby improving their performance via modulation guidance. (4) From a practical standpoint, our approach is simple to implement, incurs negligible overhead, and delivers performance gains tatte-of-the-art multi- and few-step DMs across text-to- cot-to-imageevideo and image-editing tasks.
RELATED WORK
Several post-training approaches have been proposed to improve DM quality. The first group centers on classifier-free guidance (CFG) modifications (Ho Salimans 2022). Specifically, prior works im-
prove CFG by optimizing scale factors (Fan et al. 2022). addressing off-manifold challenges (Chung) ict al. 2024). modifying the unconditional branch (Karaas et al., (2024), mitigating oversaturation at high CFG scales (Sadat al. 2024| 2025 Lin al. (0024), and introducing dynamic CFG strate- gies (Kynkäänniemi et al. 2024 Sadat et al.) 2023 Wang et al. 2024 Yehezkel et al., 2025). In contrast, method complements CFG and, importantly. can also be applied to few-step DMs (Song) iet al.
2023 Sauer et al. 2024b Yin et al. 2024a Starodubcev et al. (2225 that do not use CFG.
The second group focuses test-time optimization. A dominant line of work (Chefer al.
2023 | (Soo
et al. 2025 Yiflach et al. 2025 ei
2023 Rassin et al. 2023 Agarwal et al. 2023 Dahary et al.
2024 Marioriyad al. 2025 Binyamin al. [025 Phung al. 2024 Chen al.
2024 Kwon
ct al. 2022) relies on handcrafted loss functions, typically guided by heuristics about how attention
maps should behave, and optimizes these maps accordingly. Other methods focus on optimizing only
the initial noise rather than the full denoising trajectory (Eyring et al. (2025] Ma et al. 2025a |
al. 2024] Guo al., 2024). or on fine-tuning LoRAs to extract different concepts (Gandikota et
2024 In contrast, our approach avoids complex loss design and intensive model tuning while still
improving performance.
Finally, works most closely related to ours are attention guidance methods. These methods (Chen al. [025]] Hong et al. 20033 Ahn et al. 2025| Nguyen et al., 2024) leverage positive and negative prompts, compute attention outputs for both, and perform controlled extrapolation in the attention space-pushing the model toward positive prompts and away from negative ones. Our approach also
relies guidance in feature space but applies it through small MLP rather than through attention.
3 MODULATION LAYERS
In this section, we briefly recap the key components of modulation layers used in transformer DMs.
State-of-the-art text-to-image DMs (Labs 2024 Cai et al., 2025) typically represent images as sequences of continuous tokens, aligning them with text tokens in a unified representation. This combined sequence is processed through a series of transformer blocks (Peebles & Xie (023). which primarily consist of MI.Ps, normalization, and attention layers. To condition the model on a text prompt, two types of encoders are usually used: T5 (Raffel et al. 2020) and CLIP text encoder (Radford et al., 202T), which operate as follows:
y(, t) MLP(t, CLIP(P)). s= [T)(p)) x].
(1)
Here, y denotes global conditioning vector derived from the time step and the pooled embedding of the prompt p. whereas denotes the concatenated sequence of image tokens x and text tokens (p)... The sequence s then processed via cross-attention to incorporate text information, while the global conditioning vector is shared across the entire model and constructs modulation space that influences the modulation layers.
Mod(s, y) = (y) -s + (y)),
Here, Os and Bg are the coefficients of the modulation layer, representing scaling and shifting operations, respectively. Notably, modulation layers have proved effective in enabling semantic control and transformation in GANs (Karras et al., 2019 2020 2021). In DMs, they have been used to address image editing problems (Garibi et al. 2025 Dalva al. 2024). While these layers have shown effectiveness in semantic control tasks, their role in improving image generation quality remains unexplored.
2

---

Published as a conference paper at ICLR 2026
Desieton from letiall image (DrourSita) releive crmder tolees
Configuration
CLIP Score 1
PickScore 1
ImageReward
FLUX schnell
Initial,
short
30.1
21.6
0.
w/o CLIP,
short
29.0 (-1.)
21.3 (-0.3)
4.5 (-1.7
w/o T5, short
28.9 (-1.2)
(-.06)0.6)
1.5 (-4.)
04
Initial,
long
21.0
10.3
w/o CLIP,
long
32.8 (-0.)
(-000
(+0..)
0.5
wo T5, long
30.7 (-2.)
19.9 (-1.)
(-
HiDream-Fast
0.
Naasler
Initial,
short 30.3
21.8
7.9
w/o CLIP, short
(-0.0)
2-.00-
(++0.)
w/o Llama, short
20.2 (-101)
18.2 (-3.6)
-21.5 (-2944
Initial,
long
32.9
21.5
12.8
w/o CLIP. long
32.9 (-0.0)
21.5 (-000)
13.0 (+0.2)
w/o Llama, long
16.8 16.1)
16.0 (-4.)
20.88 (3336)
leng
WAB long
CLI
wils
Table 1: Image quality results for short and long
Figure 1: (top) Difference between images (Dream- Sim) with and without CLIP function of prompt length. (bot) For long prompts, images without CLIP generally do not differ from the initial ones.
prompts. The CLIP embedding does affect output quality on long prompts for FLUX schnell and has no effect for HiDream-Fast.
4 ANALYSIS OF THE POOLED TEXT EMBEDDING ROLE
In recent DMs, there is trend to discard the pooled text embedding and rely solely on the timestep t to produce y. i.e., MLP(t, CLIP(p)) + (t) In this setup, the text is incorporated only through the text encoder T5. However, no strict justification for this design choice has been provided. Therefore, in this section, investigate the impact of the pooled embedding on the generative performance of DMs.
Influence of the CLIP pooled embedding. First, analyze the influence of CLIP on text-to-image generation performance. To this end, we examine two contemporary models: FLUX schnell and HiDream--Fas Fast. Specifically, setting (p) + O, and comparing it to the standard case with CLIP enabled. Our key observation that the pooled CLIP embedding is partially inactive in FLUX schnell and fully inactive in HiDream-Fast.
analyze the impact of CLIP by removing the pooled embedding,
Specifically, find that the influence of CLIP in FLUX schnell is inconsistent: it is negligible for long prompts but can be impactful for short ones. To confirm this, we construct two subsets of prompts (1K each) from the MJHQ dataset (Li al. 2024): short (10 tokens) and long (77 tokens). We then evaluate the DM's performance on each subset. In Table metrics (CLIP Score, PickScore, and IewgeReward) for each subset. We observe that for long prompts, CLIP has little effect, with only a minimal impact on quality. In contrast, for short prompts, its influence is more pronounced.
T (top), we report image quality
Moreover, in Figure T we analyze the difference between images generated with and without CLIP as a function of prompt length (measured by the number tokens). We find that for longer prompts, the deviation from the initial generation becomes negligible, and the images fully resemble the initial ones, as visually confirmed in Figure (bottom).
For HiDream-Fast Fast, observe slightly different behavior: the CLIP pooled embedding exhibits no effect for either short or long prompts, as numerically confirmed in Table I (bottom).
Influence of the pooled embedding on other models. Additionally, explore the reintegration of CLIP into a DM from which it was originally absent. To this end, we consider the COSMOS model (Agarwal et al. 2025) and incorporate the CLIP pooled embedding into it as described in Section 3 In this case, we observe the same behavior as with the HiDream-Fatt model: CLIP has no influence. This result numerically confirmed Section 6 Finally, in Appendix we observe the same effect in the instruction-guided image editing task performed with the FLUX Kontext model. In Section we show that this limitation can result in insufficient editing strength for complex cases.
3

---

Published as a conference paper at ICLR 2026
5 MODULATION GUIDANCE
Our observations raise questions about the necessity of using the pooled embedding in generative tasks. However, although the pooled text embedding may seem uninformative in some cases, we propose reconsidering its role from different perspective-ons that can lead to improved generative performance in DMs.
Guidance in modulation space. We draw inspiration from the understanding that modulation layers can drive semantic changes in generated images (Karras et al., 2019). Moreover, the CLIP encoder is trained to construct a shared space between images and text, resulting in interpretable geometry. Thus, we suggest that CLIP enables interpretable modifications of the modulation space using natural language and guides the model toward modes with more desirable properties.
We propose a training- free, plug-and and-play technique reactivate CLIP and strengthen its influence during generation, drawing inspiration from Garibi (2025). Specifically, we amplify its effect by introducing guidance the modulation space.
yp, t) Vpp P+. P p-,
y(p.t) + w- (v(p+.t)- -y(p,t)))
We note that y affects only the modulation coefficients and is shared across all DM blocks, thereby incurring negligible computational overhead compared to basic generation. Moreover, this technique can be applied top of CFG guidance or with distilled DMs that do not rely on CFG.
To provide intuition behind the guidance, we first analyze it from the perspective of se- mantic changes. Prior work has focused on identifying interpretable directions in DMs, either through supervised (Gandikota et al. 2024) unsupervised approaches (Gandikota et al. 2025). In contrast, we demonstrate that such interpretable directions are already em- bedded within the model and can be accessed by shifting the modulation space. Specif- ically, in Figure 2 we consider two exam- ples: P+
Short
Original generation
Long huir
Long hair; Modern car and p = Short hair; Old car. We ob- serve that the pooled embedding can substan- tially influence the generated image, leading to both local (hair length) and global (car style) changes.
Old
Original generation
Figure 2:
The modulation guidance enables local (top) and global (bottom) changes and encourages its use to shift a DM toward modes with better properties.
Our observations suggest that modulation guidance provides an additional degree freedom in generation, beyond what CFG offers. Building this, propose using it to enhance generation quality across multiple dimensions. Specifically, we consider general changes: aesthet ics, complext ty, and specific changes: hands correcti on,, object count ing, color, position. For the latter, we focus on common criteria typically measured in T2I bench- marks (Gossh al. 2023). Notably, our technique requires only the selection of suitable prompt for each category no additional training or fine-tuning is necessary. In Appendix D we present the prompts used for each targeted aspect.
Dynamic modulation guidance. We find that constant guidance scale w is generally effective, but excessively high values can overweight the prompt and cause the model neglect textual information (Appendix C). To address this, we draw inspiration from dynamic CFG (Sadtt et al.) 2023. Kynkäänniemi et al., (024)), which has shown promising results DMs. Unlike dynamic CFG, aim to adjust across layers rather than across time steps.
We consider the simplest variant present in Figure ((). We discuss more strategies in Appendix B First, we compare the dynamic version against constant modulation guidance in terms of the aesthet- ics-pne sos-prompt fidelity trade-off. To this end, we apply both types of guidance with different scales w on 1K prompts from the MJHQ dataset (Lian al. 2023). We compute PickScore (Kirstain et al.) 2023) for aesthetics quality and CLIP score (Hessel et al.) (202T) for text correspondence. The results presented in Figure a) confirm that dynamic guidance provides better trade-off than constant
4

---

Published as a conference paper at ICLR 2026
Aestheties quality promps fidelity trade-off
21.70
dynamie mod. guidance
Anstluries qualing PickScaee
21/73
constant mod. gridance
21:20
21.68
21.03
21.02
3.5
30.4
20.7
Layer
Text relevance, CLIP score
(4)
h
Figure 3: Analysis dynamic modulation guidance. (a) Dynamic guidance offers better trade-off between aesthetic quality and prompt correspondence than constant modulation guidance. (b) We use step function, controlled by i, that skips the first few layers of the model our form of dynamic guidance. Additional variants are explored in Appendix
guidance. Our approach improves image quality without compromising prompt correspondence relative to 0 (the initial model without modulation guidance).
In our experiments, we find that dynamic modulation guidance generalizes well across tasks, sug- gesting that it can be applied new tasks without additional tuning. We also observe that more complex strategies (Appendix C) can yield better results in some cases, offering an additional degree of improvement.
What does modulation guidance actually do? We address the question of how the model is affected by the guidance in improving the gen-
Crat, mndiattonn gashine
Origiel gesoetion
Diffcrncc
erated content. To this end, we analyze the case of ha nds correct on. Specifically, in Fig- ure Ia), a), we visualize the attention map corre- sponding to the word hands for specific im- age. Interestingly, we observe that the model places greater focus the relevant region, high- lighting more distinctly. In addition, in Fig-
b, left), plot the averaged attention map
for all tokens in the corresponding prompt. We find that the model primarily shifts its attention toward more relevant lokens-such as hands and child.
To confirm this intuition, we analyze a subset of prompts focused on hands correction and split all tokens into four groups: non-content tokens, the token hands, tokens related to hands, and other important tokens. The results in Figure lb, right) confirm that the model shifts its attention toward hands and hand-related tokens.
Figure 4: After applying modulation guidance, the model focuses more on the desired features, such hands (a, b).
Integrating the pooled text embedding into CLIPP-rre models. Finally, we extend modulation guidance to models without pooled text embeddings, showing that it can improve generation quality. To this end, we fine-tune existing text-to-image/vide models Agarwal et al. (2025): Wan al. (2025) by introducing the pooled embedding. Specifically, we train small MLP on top of the pooled text embedding and add it to the timestep embedding, while keeping the rest of the network frozen. The model behaves identically to the original when the pooled embedding is set to 0. Importantly, we train on the model's synthetic data to ensure that improvements do not stem from dataset differences.

---

Published as a conference paper at ICLR 2026
Table 2: Performance of text-to-image DMs with and without modulation guidance (gray) on Aesthetics and Complexity, evaluated with human preferences and automatic metrics. Human win rates are reported with respect to the original model; green indicates statistically significant improvement, red decline. For automatic metrics, bold denotes improvement over the original model.
Side-by-Side Win Rate, % Complexity
Automatic Metries, COCO 5k
Model
Relevance
Aesthetics
Defects
PickScore t CLIP
IR
HPSv3
FLUX sehnell
35.6
10.2
11.3
Aesthetics
11.0
Complexity
69
35.9
10.8
FLUX dev
23.1
34.7
10.5
12.4
Aesthetics
44
69
52
23.2
34.5
11.0
12.8
Complexity
48
59
72
47
23.1
34.6
11.1
12.8
SD3.5 Large
23.0
35.8
10.5
11.1
Aesthetics
50
62
70
47
23.1
35.9
10.7
11.2
Complexity
49
49
60
45
23.0
35.8
11.7
11.0
HiDream
23.4
34.4
11.7
13.2
Aesthetics
49
60
80
46
23.5
34.4
12.1
13.7
Complexity
47
52
70
45
23.5
34.4
11.9
13.3
COSMOS
23.0
35.0
11.4
+ CLIP
50
35.0
11.4
Aesthetics
Complexity
11.8
We highlight two important aspects of the training process. First, we propagate the textual prompt solely through the pooled text embedding, using unconditional prompt for T5. This design forces the model to perceive textual information through the pooled embedding. Second, we employ a distillation- based training regime. Specifically, sample clean image. add noise to it, and then generate two predictions: one from the original model (without the pooled embedding) and one from the modified model (with the pooled embedding). The objective is minimize the MSE loss between these two predictions. This distillation approach is well-suited for few-step DMs, as it climinates the need for complex adversarial or distribution-matchingg losses (Yin et al. 2024a).
EXPERIMENTS
6.1 TEXT-TO-IMAGE GENERATION
Configuration. We validate approach state-of-the-art text-to-image DMs that include modulation-bassee text conditioning: FLUX schnell (Sauer et al. 2024a). FLUX (Labs) 2024). SD3.5 Large (Esser et al., 2024), and HiDream (Cai et al. (025) In addition, we consider the CLIP-free COSMOS model (Agarwal et al. 2025) and fine-tune it for 4K iterations to introduce the pooled text embedding. We train the model its own synthetic dataset of 500K samples, following the generation settings of Agarwal et al. (2025) and using prompts from Li et al. (2024).
We evaluate performance using two types of metrics: human preference and automatic evaluation. Human preference is measured via side-by-side comparisons, where annotators assess image pairs on four criteria: text relevance, aesthetics, complexity, and defects (details in Appendix []. For general changes, we use 128 prompts from PartiPrompts YYu et al. 2022), generating two images per prompt. For specific changes, we use 70 prompts from CompBench (Jia al. 2023) for cb ebject counting and 200 LLM ---eerrrete prompts for hands correct on. For automatic evaluation, we report CLIPP score (Hessel et al., 202T), ImageReward (IR) (Xu et al., 2023), PickScore (PS) (Kirstain et al.) 2023), and HPSv3 (Ma et al. 2025b), tested on 5K prompts from COC02014 (Lin al. 2014). We also use GenEval (Ghosh et al. 2023 to validate modulation guidance across multiple benchmark criteria.
Our main baselines are the original models without modulation guidance. In addition, we con- sider the Normalized Attention Guidance approach (Chen al.) (2225) and LLM-enhanced prompt modifiers (Lian et al. 2023). Finally, we include the test- cest-time optimization method Concept Slid- (Gandikota al. 2024) for the hands correct on task.

---

Published as a conference paper at ICLR 2026
FLUX, schnell
FLUX
Hi-Dres Drearr
SD3.5 Large
COSMOS
Ours, quantt
Original
a smiling banana wearing a bandana
Ours, complexity
Original
cardboard spaceship
Figure 5: Qualitative results of modulation guidance for Aest ics (top) and Comp 1e by (bottom). The hesthet ics guidance notably improves image quality, while the Comp 1 ex tity guidance can enhance the complexity of both the main object and background details.
General changes. In this case, we focus on two aspects for improvement: aesthetics and complexity. These aspects are crucial for text-to- text-to-imant image generation and are typically the targets of self- sulf-super fine-tuning techniques (Startsev al. 2025) RL-based approaches (Wallace et al., 2024), which are commonly adopted in DMs. However, we demonstrate that our simple technique achieves significant improvements without any fine-tuning. The only requirement is to select appropriate positive and negative prompts, along with suitable dynamic guidance strategy. Our choices are summarized in Table and discussed Appendix D
Table 2 reports numerical results, showing clear human preference gains for both aspects. Aesthetics guidance significantly improves both aesthetics and complexity, while comp exityy guidance mainly enhances complexity. Automatic metrics show consistent ImageReward gains across all models and HPSy3 improvements in most cases, except for SD3.5 Large with compl lexityy guidance. Importantly, we observe that introducing CL.PP into COSMOS does not improve perfor- mance and even reduces complexity; gains appear only when combined with modulation guidance, confirming that CLIP alone is ineffective. We note slight drops in text relevance for FLUX dev and in defects for COSMOS, though these are minor. Qualitative examples are shown Figure and Appendix[I
Specific changes, Next, we focus on improving ob jbject countings hands correct ion, color, and position. The first two particularly important, they have been extensively studied in prior work (Binyamin et al. 2025] Gandikota et al. (024). For object counting we the number of target objects the positive direction, while for hands correction, draw
7

---

Published as a conference paper at ICLR 2026
Table 3: Quantitative results of the modulation guidance for specific changes. The modulation
guidance yields improvements according to GenEval and human preference.
GenEval
ShS Win Rate, % Hands correction
Model
Object Counting
Color
Position
Object Counting
Original
79
25
39
41 59 (+ (8)
FLUX schnell
Ours
65 (+-9)
86
(+-7
30
(+-5)
61 (+22))
Ours, abjeet counting
Original
four balloons....
eight microwaves
seven strawberries
seven pigs
eight cows
Ours, hands
Original
Figure 6: Qualitative results of the modulation guidance for Ob ject counting (top) and Hands correction (bottom).
inspiration from Gandikota et al. (2024) in designing positive and negative prompts. Further details are provided in Table
We present the results in Table and Figure 6 Improvements are observed in several aspects of the GenEval benchmark, including object counting, color, and position. According human evaluation, our approach improves the original model by 22% in ob object count fonntingg and 18% in hands correction. We report text relevance and defects as the evaluation criteria for object counting and hands correction, respectively.
Comparison with baselines. Normalized Attention Guidance (Chen al. 2025 targets general changes, so compare it with our aesthet cs guidance using ShS evaluation. Similarly, we compare Concept Sliders (Gandikota et al. 2024) with our hands correction guidance by evaluating defects. For LLM-enhanced prompts (Lian et al. 2023), we consider general changes, hands correct i ion, and ob ob ject counting ing. Results in Appendix (Tables g and 9 show that approach outperforms Normalized Attention Guidance by 34% and Concept Sliders by 16%. without additional computational overhead. Moreover, Table sshows that modulation guidance can further improve performance when combined with I.MM-ehhanced prompts.
8

---

Published as a conference paper at ICLR 2026
Table 4: Quantative evaluation VBench. The results show an improved dynamic degree compared to the original models and baseline approach (normalized attention guidance).
Model, video
total score
motion smoothness
dynamic degree
aesthotic quality 1 overall consistency
Original
50.51
Hunyan, 13B
Medulation guidince
57.55 56
99.03
63.61
56.50
21.09
Original
62,
98, 76
75.25
57.85
19.01
ConeVid,
Norm. attent. guidance
63.58
58.39
74.22
62.08
19.61
Modulation guidance
65.43
96.45
86.59
57.65
19.02
CanVVdd 1.3B
POO
bicyele gliding through snowy field.
Figure Qualitative comparison between the original Caus Vid and Caus Vid with modulation guidance.
Rekeence
ELUX Kentod
FLAN Sertee Mad peiance
Releraes
FLU Koeked
BLUX Konten Mad palans
Insinatioss Torn peoglle intoo rohuts
Intracian Replece fie can will Boats
Figure 8: Qualitative results for text- text-guided image editing tasks. We observe that FLUX Kontext sometimes struggles with complex edits, while modulation guidance can mitigate this limitation.
6.2 TEXT-TO-VIDEO GENERATION
Configuration. We apply modulation guidance to Hunyuan 13B (Kong et al. 2024) and Caus 1.3B (Yin al. (2024b). The latter does not include a CLIP model, so we fine-tune it for 1K iterations. To evaluate performance, we use VBench (Huang et al. 2024), which covers various aspects. this experiment, we apply the same aesthet ieeticcs guidance in the text-to-image task. In addition, we compare our approach with Normalized Attention Guidance.
The results are presented in Table 4 and Figure 7 Importantly, we observe improvements in dynamic degree for both models, with particularly strong gains for CausVid This notable because Cius is distilled from WAN (Wan et al.] (005), and video models typically lose dynamics after distillation. Furthermore, we find that incorporating CLIP provides no improvement. Additional visual comparisons are provided in Appendix
Results.
6.3 INSTRUCTIIO-GUIDERR IMAGE EDITING
Finally, we address image editing using the FLUX Kontext model (Labs et al. 2025), which, as we find, can struggle with complex edits involving multiple objects. To overcome this, we apply modulation guidance, using the final prompt as the positive direction and a blank prompt as the negative. We validate our approach on the SEED-Data benchmark (Ge et al., (024) and present the results and implementation details in Appendix Representative examples are shown in Figure g
7 CONCLUSION
In this paper, we revisit the role of the pooled text embedding, showing that, despite its weak influence, can improve performance across tasks and models when used from different perspective. We present ablation studies in Appendix C where dynamic modulation guidance outperforms constant guidance, offering greater flexibility for practitioners. Limitations are discussed in Appendix H

---

Published as a conference paper at ICLR 2026
REFERENCES
Aishwarya Agarwal, Srikrishna Karanam, KJ Joseph, Apoorv Saxena, Koustava Goswami, and Balaji Vasan Srinivasan. A-star: Test-time attention segregation and retention for text-to-image synthesis. In Proceedings of the IEEE/CVF International Conference Computer Vision, pp. 2283-2293, 2023.
Niket Agarwal, Arslan Ali, Maciej Bala, Yogesh Balaji, Erik Barker, Tiffany Cai, Prithvijit Chat- topadhyay. Yongxin Chen, Yin Cui, Yifan Ding, et al. Cosmos world foundation model platform for physical ai. arXiv preprint (ixiv:2500.0357555 2025.
Donghoon Ahn, Hyoungwon Cho, Jaewon Min, Wooseok Jang, Jungwoo Kim, SeonHwa Kim, Hyun Hee Park, Kyong Hwan Jin, and Seungryong Kim. self-reetifying diffusion sampling with perturbed attenttoo guidance, 2025. URL httpp:://axxi....aa///
abs/ //////443.1377
Omri Avrahami, Or Patashnik, Ohad Fried, Egor Nemchinov, Kfir Aberman, Dani Lischinski, and Daniel Cohen-Or. Stable flow: Vital layers for training-free ffree image editing. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 7877-7888, 2025.
Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, Humen Zhong, Yuanzhi Zhu, Mingkun Yang. Zhaohai Li, Jianqiang Wan, Pengfei Wang, Wei Ding, Zheren Fu, Yiheng Xu, Jiabo Ye, Xi Zhang, Tianbao Xie, Zesen Cheng, Hang Zhang, Zhibo Yang, Haiyang Xu, and Junyang Lin. Qwen2.5-vl technical report. arXiv preprint arXiv:2502. 2025.
I Lital Binyamin, Yoad Tewel, Hilit Segev, Eran Hirsch, Royi Rassin, and Gal Chechik. Make it count: Text-to-image generation with an accurate number of objects. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 13242-13251, 2025.
Qi Cai, Jingwen Chen, Yang Chen, Yehao Li, Fuchen Long, Yingwei Pan, Zhaofan Qiu, Yiheng Zhang, Fengbin Gao, Peihan Xu, et al. Hidream-il: A high-efficient image generative foundation model with sparse diffusion transformer. arXiv preprint arXiv:2505.222
2025.
Hila Chefer, Yuval Alaluf, Yael Vinker, Lior Wolf, and Daniel Cohen-Or. Attend-and-excite: Attention-basee semantic guidance for text-to-image diffusion models. ACM transactions on Graphics (TOG), 42(4):1 2023.
Dar-Yen Chen, Hmrishav Bandyopadhyay, Kai Zou, and Yi-Zhe Song. Normalized attention guidaneee Universal negative guidance for diffusion models, 2025. URL http:tss///a.....orr/////////// 2505 21179
Minghao Chen, Iro Laina, and Andrea Vedaldi. Training-free layout control with cross-attention guidance. In Proceedings of the IEEE/CVF winter conference on applications of computer vision, pp. 5343-5353, 2024.
Hyungjin Chung. Jeongsol Kim, Geon Yeong Park, Hyelin Nam, and Jong Chul Ye. Cfg++: Manifold- constrained classifier free guidance for diffusion models. arXiv preprint arXiv:2406.0807 2024.
Omer Dahary, Or Patashnik, Kfir Aberman, and Daniel Cohen-Or. Be yourself: Bounded attention for multi-subject text-to-image generation, 2024. URL http::ss//aai...orgg///.o/
16990
Yusuf Dalva, Kavana Venkatesh, and Pinar Yanardag, Fluxspace: Disentangled semantic editing in rectified flow transformers, 2024. URL htll:ttta///xxx.....
abs 22122 2412.09611 09611
Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas Müller, Harry Saini, Yam
Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, et al. Scaling rectified flow transformers for high-resolution image synthesis. In Forty-firs international conference on machine learning. 2024.
Luca Eyring, Shyamgopal Karthik, Karsten Roth, Alexey Dosovitskiy, and Zeynep Akata. Reno: Enhancing one-step text-to-image models through reward- -baed noise optimization. Neural Information Processing Systems (NeurIPS), 2024.
10

---

Published as a conference paper at ICLR 2026
Luca Eyring. Shyamgopal Karthik, Alexey Dosovitskiy, Nataniel Ruiz, and Zeynep Akata. Noise hy- pernetworks: Amortizing test-time compute in diffusion models. arXiv preprint arXiv:2508.09900 2025.
Weichen Fan, Amber Yijia Zheng. Raymond A Yeh, and Ziwei Liu. Cfg-zero*: Improved classifier- free guidance for flow matching models. arXiv preprini arXiv:2503. 18886, 2025.
Rohit Gandikota, Joanna Materzyúska, Tingrui Zhou, Antonio Torralba, and David Bau. Concept sliders: Lora adaptors for precise control in diffusion models. In European Conference on Computer Vision, pp. 172-188. Springer, 2024.
Rohit Gandikota, Zongze Wu, Richard Zhang, David Bau, Eli Shechtman, and Nick Kolkin. Slider- space: Decomposing the visual capabilities of diffusion models. arXiv preprint arXiv:2502.01639 2025.
Daniel Garibi, Shahar Yadin, Roni Paiss, Omer Tov, Shiran Zada, Ariel Ephrat, Tomer Michaeli, Inbar Mosseri, and Tali Dekel. Tokenverse: Versatile concep personalization in token modulation space, 2025.
Yuying Ge, Sijie Zhao, Chen Li, Yixiao Ge, and Ying Shan. Seed-data-edit technical report: A hybrid dataset for instructional image editing, 2024. 04407
Dhruba Ghosh, Hannaneh Hajishirzi, and Ludwig Schmidt. Geneval: An object-focused framework for evaluating text-to-image alignment. Advances in Neural Information Processing Systems, 36: 52132-52152, 2023.
Xiefan Guo, Jinlin Liu, Miaomiao Cui, Jiankai Li, Hongyu Yang, and Di Huang. Initno: Boosting text-to-image diffusion models via initial noise optimization. 2024. URL https://arxiv org/abs abs/ 2404. 04650
Jack Hessel, Ari Holtzman, Maxwell Forbes, Ronan Le Bras, and Yejin Choi. Clipscore: A reference- free evaluation metric for image captioning. arXiv preprint arXiv:2 2021.
Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance. arXiv preprint arXiv:2207.12598 2022.
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6800-6851 2020.
Susung Hong, Gyuseong Lee, Wooseok Jang, and Seungryong Kim. Improving sample quality of diffuusin models using self-attention guidance, 2023. URL https::/aaxii..orrttt abs/
2210. 00939
Ziqi Huang, Yinan He, Jiashuo Yu, Fan Zhang, Chenyang Si, Yuming Jiang, Yuanhan Zhang, Tianxing Wu, Qingyang Jin, Nattapol Chanpaisit, Yaohui Wang, Xinyuan Chen, Limin Wang. Dahua Lin, Yu Qiao, and Ziwei Liu. VBench: Comprehensive benchmark suite for video generative models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024.
Bohan Jia, Wenxuan Huang, Yuntian Tang, Junbo Qiao, Jincheng Liao, Shaosheng Cao, Fei Zhao, Zhaopeng Feng, Zhouhong Gu, Zhenfei Yin, Lei Bai, Wanli Ouyang, Lin Chen, Fei Zhao, Zihan Wang, Yuan Xie, and Shaohui Lin. Compbench: Benchmarking complex instruction-guidee image editing, 2025. URL https: http://aaxvvvvv.. httpg//bb//25500 12200
Tero Karras, Timo Aila, Samuli Laine, and Jaakko Lehtinen. Progressive growing of gans for improved quality, stability, and variation. arXiv preprint arXiv:1710.101900 2017.
Tero Karras, Samuli Laine, and Timo Aila. A style-based generator architecture for generative adversarial networks. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 4401-4410. 2019.
Tero Karras, Miika Aittala, Janne Hellsten, Samuli Laine, Jaakko Lehtinen, and Timo Aila. Training generative adversarial networks with limited data. In Proc. NewrIPS, 2020.
11

---

Published as a conference paper at ICLR 2026
Tero Karras, Miika Aittala, Samuli Laine, Erik Härkönen, Janne Hellsten, Jaakko Lehtinen, and Timo Aila. Alias-free generative adversarial networks. In Proc. NeurIPS, 2021.
Tero Karras, Miika Aittala, Tuomas Kynkäänniemi, Jaakko Lehtinen, Timo Aila, and Samuli Laine. Guiding diffusion model with a bad version of itself. Advances in Neural Information Processing Systems, 77:52996-5302) 2024.
Yuval Kirstain, Adam Polyak, Uriel Singer, Shahbuland Matiana, Joe Penna, and Omer Levy. Pick- a-pie: An open dataset of user preferences for text-to-image generation. Advances neural information processing systems, 36:36652-36663,, 2023.
Weijie Kong, Qi Tian, Zijian Zhang, Rox Min, Zuozhuo Dai, Jin Zhou, Jiangfeng Xiong, Xin Li, Bo Wu, Jianwei Zhang, et al. Hunyuanvideo: A systematic framework for large video generative models. arXiv preprint arXiv:2412.03600 2024.
Mingi Kwon, Jaeseok Jeong, and Youngjung Uh. Diffusion models already have a semantic latent space. arXiv preprint arXiv:2210.. 10960, 2022.
Tuomas Kynkäänniemi, Miika Aittala, Tero Karras, Samuli Laine, Timo Aila, and Jaakko Lehtinen. Applying guidance in limited interval improves sample and distribution quality in diffusion models. Advances in Neural Information Processing 3 37:122458-1224485 2024.
Black Forest Labs. Flux. https: httt:s///ikhaab..cmm/aakk----
wtt.ccoo/blll... -aacck--oress: foreft--eeoo.o -atestt-abb/EE110
2024.
Black Forest Labs, Stephen Batifol, Andreas Blattmann, Frederic Boesel, Saksham Consul, Cyril Diagne, Tim Dockhorn, Jack English, Zion English, Patrick Esser, Sumith Kulal, Kyle Lacey, Yam Levi, Cheng Li, Dominik Lorenz, Jonas Müller, Dustin Podell, Robin Rombach, Harry Saini, Axel Sauer, and Luke Smith. Flux.1 kontext: Flow matching for in-context image generation and editing in latent space, 2025. URL http::tts::/aaa .tweiiivorrrgo www.//g..ss...ooo. 2506. 15742
Daiqing Li, Aleks Kamko, Ehsan Akhgari, Ali Sabet, Linmiao Xu, and Suhail Doshi. Playground v2.5: Three insights towards enhancing aesthetic quality in text-to-image generation, 2024.
Yumeng Li, Margret Keuper, Dan Zhang, and Anna Khoreva. Divide & bind your attention for improved generative semantic nursing, arXiv preprint arXiv:2307. 10864, 2023.
Long Lian, Boyi Li, Adam Yala, and Trevor Darrell. LIm-grounded diffusion: Enhancing prompt understanding of text-to-image diffusion models with large language models. arXiv preprint arXiv:2305.13655. 2023.
Shanchuan Lin, Bingchen Liu, Jiashi Li, and Xiao Yang. Common diffusion noise schedules and sample steps are flawed. In Proceedings of the IEEE/CVF winter conference on applications of computer vision, pp. 5404-5411, 2024.
Tsung-Yi Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In European conference on computer vision, pp. 740-755. Springer, 2014.
Nanye Ma. Shangyuan Tong, Haolin Jia, Hexiang Hu, Yu-Chuan Su, Mingda Zhang, Xuan Yang, Yandong Li, Tommi Jaakkola, Xuhui Jia, et al. Inference-time scaling for diffusion models beyond scaling denoising steps. arXiv preprint arXiv:2501.097.
2025a.
Yuhang Ma, Xiaoshi Wu, Keqiang Sun, and Hongsheng Li. Hpsv3: Towards wide-spectrum human preference score. arXiv preprint arXiv:2508..:777 2025b.
Arash Marioriyad, Mohammadali Banayeeanzade, Reza Abbasi, Mohammad Hossein Rohban, and Mahdieh Soleymani Baghshah. Attention overlap is responsible for the entity missing problem in text-to-image diffusion models!, 2025. URL https: /ttp///avvvv///////////ae//aaavl org/ abs /0bs/24100 2400. 2410.20972
Viet Nguyen, Anh Nguyen, Trung Dao, Khoi Nguyen, Cuong Pham, Toan Tran, and Anh Tran. Snoopi: Supercharged step diffusion distillation with proper guidance, 2024. URL RtLhttps //arxiv.orr//bb//241. 7 02687
12

---

Published as a conference paper at ICLR 2026
Alex Nichol, Prafulla Dhariwal, Aditya Ramesh, Pranav Shyam, Pamela Mishkin, Bob McGrew, Ilya Sutskever, and Mark Chen. Glide: Towards photorealistic image generation and editing with text-guided diffusion models. arXiv preprint arXi::2112.10741) 2021.
Jonas Oppenlaender. A taxonomy of prompt modifiers for toxt-to-image generation. Behaviour & Information Technology, (3(15)33733-3776 2024.
William Peebles and Saining Xie. Scalable diffusion models with transformers. Proceedings of the IEEE/CVF international conference computer vision, pp.
2023.
Quynh Phung. Songwei Ge, and Jia-Bin Huang. Grounded text-to-image synthesis with attention refo- cusing. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 7932-7942, 2024.
Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas Müller, Joe Penna, and Robin Rombach. Sdxl: Improving latent diffusion models for high-resolution image synthesis. arXiv preprint arXiv:2307.01992, 2023.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. International conference on machine learning, pp- 8748-8760
PmLR, 2021.
Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of machine learning research, 21 (140):1-66) 2020.
Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text- conditional image generation with clip latents. arXiv preprint arXiv:2204.0612. 1(2):3, 2022.
Royi Rassin, Eran Hirsch, Daniel Glickman, Shauli Ravfogel, Yoav Goldberg, and Gal Chechik. Linguistic binding in diffusion models: Enhancing attribute correspondence through attention map alignment. Advances in Neural Information Processing Systems, 36:3536-3555 -3596, 2023.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High- resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF confer- ence on computer vision and pattern recognition, pp. 10695, 2022.
Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation. In International Conference on Medical image computing and computer- assisted intervention, pp. 234-241. Springer, 2015.
Seyedmorteza Sadat, Jakob Buhmann, Derek Bradley, Otmar Hilliges, and Romann M Weber. Cads: Unleashing the diversity of diffusion models through condition-anneale sampling. arXiv preprint arXiv:2310.17344 2023.
Seyedmorteza Sadat, Otmar Hilliges, and Romann M Weber. Eliminating oversaturation and artifacts of high guidance scales in diffusion models. In The Thirteenth International Conference on Learning Representations, 2024.
Seyedmorteza Sadat, Tobias Vontobel, Farnood Salchi, and Romann M Weber. Guidance the fre- quency domain enables high-fidelity sampling at low cfg scales. arXiv preprint 271. 2025.
Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily L Denton, Kamyar Ghasemipour, Raphael Gontijo Lopes, Burcu Karagol Ayan, Tim Salimans, et al. Photorealistic text-to-image diffusion models with deep language understanding. Advances in neural information processing systems, 15:36479-36499) 2022.
Axel Sauer, Frederic Boesel, Tim Dockhorn, Andreas Blattmann, Patrick Esser, and Robin Rombach. Fast high-resolution image synthesis with latent adversarial diffusion distillation. In SIGGRAPH Asia 2024 Conference Papers, pp. 1-11, 2024a.
13

---

Published as a conference paper at ICLR 2026
Axel Sauer, Dominik Lorenz, Andreas Blattmann, and Robin Rombach. Adversarial diffusion distillation. In European Conference on Computer Vision, pp. 87-103. Springer, 2024b.
Hoigi Seo, Junseo Bang, Haechang Lee, Joohoon Lee, Byung Hyun Lee, and Se Young Chun. Geometrical properties of text token embeddings for strong semantic binding in text-to-image generation. arXiv preprint arXiv:2503.23011 2025.
Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. arXiv preprint arXiv:2011.13456. 2020.
Yang Song, Prafulla Dhariwal, Mark Chen, and Ilya Sutskever. Consistency models. 2023.
Nikita Starodubcev, Denis Kuznedelev, Artem Babenko, and Dmitry Baranchuk. Scale-wise distilla- tion of diffusion models. arXiv preprint arXiv:2503. 16397, 2025.
Valerii Startsev, Alexander Ustyuzhanin, Alexey Kirillov, Dmitry Baranchuk, and Sergey Kas- tryulin. Alchemist: Turning public text-to-image data into generative gold. arXiv preprint
wrXiv:2505.1929) 2025.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017.
Bram Wallace, Meihua Dang, Rafael Rafailov, Linqi Zhou, Aaron Lou, Senthil Purushwalkam, Stefano Ermon, Caiming Xiong, Shafiq Joty, and Nikhil Naik. Diffusion model alignment using direct preference optimization. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition, pp. 8228 8228-8238, 2024.
Team Wan, Ang Wang, Baole Ai, Bin Wen, Chaojie Mao, Chen Wei Xie, Di Chen, Feiwu Yu, Haiming Zhao, Jianxiao Yang, et al. Wan: Open and advanced large-scale video generative models. arXiv preprint arXiv:2503. 250323344 2025.
Xi Wang, Nicolas Dufour, Nefeli Andreou, Marie-Paule Cani, Victoria Fernández Abrevaya, David Picard, and Vicky Kalogeiton. Analysis of classifier-free guidance weight schedulers. arXiv preprint arXiv:2404.1388 13040, 2024.
Chenfei Wu, Jiahao Li, Jingren Zhou, Junyang Lin, Kaiyuan Gao, Kun Yan, Sheng-ming Yin, Shuai Bai, Xiao Xu, Yilei Chen, et al. Qwen-image technical report. arXiv preprint arXiv:2508. 023324, 2025.
Enze Xie, Junsong Chen, Junyu Chen, Han Cai, Haotian Tang. Yujun Lin, Zhekai Zhang, Muyang Li, Ligeng Zhu, Yao Lu, et al. Sana: Efficient high-resolution image synthesis with linear diffusion transformers. arXiv preprint arXiv:2410. 10629, 2024.
Jiazheng Xu, Xiao Liu, Yuchen Wu, Yuxuan Tong, Qinkai Li, Ming Ding, Jie Tang, and Yuxiao Dong. Imagereward: Learning and evaluating human preferences for text-to-image generation. Advances in Neural Information Processing Systems, 36:15903-15935 2023.
Shai Yehezkel, Omer Dahary, Andrey Voynov, and Daniel Cohen-Or. Navigating with annealing guidance scale in diffusion space. arXiv preprint arXiv:2506.24100 2025.
Sapir Esther Yiflach, Yuval Atzmon, and Gal Chechik. Data-driven loss functions for inference-time optimization in text-to-image generation. arXiv preprint arXiv:2509.02299; 2025.
Tianwei Yin, Michaël Gharbi, Taesung Park, Richard Zhang, Eli Shechtman, Fredo Durand, and Bill Freeman. Improved distribution matching distillation for fast image synthesis. Advances in neural information processing systems, 37:47455-47455 2024a.
Tianwei Yin, Qiang Zhang, Richard Zhang, William Freeman, Fredo Durand, Eli Shechtman, and Xun Huang. From slow bidirectional to fast causal video generators. arXiv e-prints, pp. arXiv-2411 2024b.
Jiahui Yu, Yuanzhong Xu, Jing Yu Koh, Thang Luong, Gunjan Baid, Zirui Wang, Vijay Vasudevan, Alexander Ku, Yinfei Yang, Burcu Karagol Ayan, et al. Scaling autoregressive models for content- rich text-to-image generation. arXiv preprint arXiv:2206. 10789, 2(3):5, 2022.
14

---

Published as a conference paper at ICLR 2026
Reference Image
without CLIP
with CLIP
replace the clouds thunderstorm
change the background sunset
Figure 9; We observe that the CLIP text encoder does not influence instruction-guided image editing performed with the FLUX kontext model.
Table 5: Configuration of hyperparameters for dynamic modulation guidance
Task
Positive prompt
Negative prompt
Guidance strategy
Uitra-detailed, photorealistie, cinematic
Text-to-image aesthetics
Lew-res, flat, cartoonish
Strategy Figure
Text-to-inage complexity
Ext remerlll complex, the highest quality
Very simple, details at all
Strategy in Figure 10,
Strategy Figure i i2 30, is M1 wy
Text-to-image hands correction
Natural and realistic hands
Unnatura hands
Text-to-image object count nn
Very simple, no detai Is at all
Strategy Figure 5, w =
bb)
[m] [ohjects]
Ultra-detailed, photorealist.ee cinematic
res, flat, cartoonish
Strategy Figure
Text-to-video
Strategy in Figure 5, ==
Image editing
Textual prompt
APPENDIX
A ADDITIONAL ANALYSIS FOR FLUX KONTEXT MODEL
Here, analyze the impact of the CLIP model on FLUX Kontext (Labs et al. 2025). We find that dropping the pooled embedding does not affect editing results, as visually confirmed in Figure 9 ad- dition, evaluate performance
Table 6: Editing quality for the FLUX kontext model (with and without CLIP). CLIP has no effect on the model.
Configuration
CLIP Score, Image
CL.IP Score, Text f
the
CLIP+T5
79.3
29.3
w/o CLIP
80 (+0.7)
29.3 (0)
SEED-Data benchmark (Ge et al.] 2024) with and without the pooled text embed- ding. We compute the CLIP score (Hessel et al. 2021) to measure reference preservation and prompt correspondence. The results in
the observation.
15

---

Published as a conference paper at ICLR 2026
Averaged, attention layer
Speeilee changee
Conural change-
Attendion Valuo
Layer
m
10
20
Loyer
Figure 10: Analysis on dynamic modulation guidance. To derive a dynamic guidance scale, we (a) analyze how the model allocates attention different features by computing averaged attention maps over two token groups (specific and general). Building on this, we (b) explore dynamic strategies for setting layer-specific w values.
The lack of impact in the editing may stem from the out-of-distribution nature of instructions for the CLIP model. We find that this mismatch can lead to a lack of editing strength, particularly in complex scenes with multiple objects. To address this, propose using the final prompt the CLIP input and applying modulation guidance.
B STRATEGIES FOR DYNAMIC GUIDANCE
Recent studies show that attention layers in transformer models specialize at different depths, with each layer focusing distinct levels of semantic detail (Avrahami et al.] 2025). This insight encourages us to investigate which parts of the attention stack are most appropriate for injecting guidance, depending on the desired effect. For example, if fine- fine-grained attributes such as hands are mainly shaped by mid-layer attention, then targeting guidance at those specific layers is more effective and reduces the risk of unintended modifications in other regions of the image.
Thus, we construct two prompt subsets of 1, 000 examples each: one targeting local features (e.g., hands, face, eyes) and the other targeting global features (e-g., real sm, cinemat ic, crisp). We then generate images for each subset and collect the corresponding attention maps for each target aspect. Finally, we average these maps across examples and present the results for different layers in Figure [0(a), a). We observe that the model primarily focuses local features in two layer regions: layers 10-30 and 42-58. In contrast, attention to global features remains relatively constant, with slight drop between layers 20 and 35.
Based on this analysis, we propose applying dynamic modulation guidance the layer level. We present four possible strategies in Figure b),), with strategies 3 and 4 designed to resemble the observed attention behavior for specific changes. Interestingly, in Appendix [ we find that these strategies provide better results for hands correcti on. For global changes, the step function (case 1) performs well, outperforming the constant scale. Despite introducing additional hyperpa- rameters, our dynamic guidance offers an extra degree of improvement for practitioners, which believe is important in real-world applications.
C ABLATION STUDY
Dynamic modulation guidance. First, we ablate different dynamic modulation guidance strategies. Specifically, we consider the FLUX schnell model, testing on the aesthetics, hands correction, and object counting aspects.
We consider different dynamic guidance strategies from Figure b)(b) and compare them constant value of w = 3. For dynamic strategies, we use the following parameters.
16

---

Published as a conference paper at ICLR 2026
Table 7: Ablation study dynamic modulation guidance strategies using human preference (side- by-side win rate). The results demonstrate that dynamic guidance outperforms constant guidance approach.
Configuration
Constant
Strategy
Strategy
Strategy 3
Strategy 4
Original
52
48
49
45
41
Hands correction
Ours
48 (-4)
52 (+-4)
51 (+2)
55 (+10)
59 (+18)
Original
50 50 (-0)
39
40
45
39
Object counting
Ours
61 (+22)
60 (+20)
55 (+10)
61 (+22)
Original
38
28
43
43
46
Aesthetics
Ours
62 (+24)
72 (+44)
57
(+14)
57 (+14)
54 (+8)
Original, FLUXX schnell
Constant modulation guidance
Dynamic modulation guidance
a close-up of bloody mary cocktail
Figure 11: Qualitative comparison of modulation strategies for aesthetics. Constant guidance can overweight the original prompt, leading significant divergence, whereas dynamic guidance better balances quality and prompt correspondence, allowing the use of larger w without degradation.
= 0
W = 2
W = 3
4
Constant guidance
Dynamic
guidance
a wolf on a plain background
Figure 12: We find that dynamic modulation guidance improves image content (c.g., makes the wolf's fur more detailed) while preserving prompt correspondence. In contrast, constant scales can neglect the prompt request even at small scales (w=2).
17

---

Published as a conference paper at ICLR 2026
7=0 (Strt layer)
2
Original
Imagine meticulonsty detailed. hyperrealistic portrait of an aged sage with piercing eyes and flowing, white beard
Figure 13: Influence of starting layers for complexity guidance. Different choices of i with fixed w = illustrate how earlier later starting layers balance between preserving the original image and improving complexity. In particular, 18 and 28 preserve the overall image while enhancing fine-grained details such as faces and hands.
w- 1.0
W 1.5
3.0
1F 8.0
Original
crayon draving space elevator
Figure 14: Influence of guidance strength w for aesthetics. With fixed = 5, increasing improves image quality by boosting the main object (the elevator) and background details. However, excessively large values, such as w = 8.0, can introduce artifacts.
Strategy 1. i = 5, = 3;
Strategy 2. 11 13, i2 = 30, == 3:
Strategy 3. We use two exponential functions with centers at é 20, i2 = 50, and w = 3;
Strategy 4. i 13, 30, ig 45, wi 3, w2 1.
Strategies and are designed follow the attention pattern illustrated in Figure a)
We conduct a human preference study comparing these strategies to the original model, with results presented in Table First, we observe that dynamic strategies yield higher performance gains com- pared to a constant scale for hands correction and object counting. Moreover, strategy 4 demonstrates the best performance on hands correction, which aligns with the analysis of attention behavior. For object counting, strategies 1 and 4 perform equally well. We therefore select strategy 1 for this aspect due to its simplicity.
Second, for aesthetics guidance, we observe that strategy achieves the best results, while constant guidance also performs well. However, we find that constant w can introduce artifacts. shown in Figure II] constant guidance overweight the original prompt, causing significant divergence from the source image. In contrast, dynamic guidance achieves a better balance between quality enhancement and prompt correspondence, enabling the use of higher values without introducing artifacts as shown in Figure [2
Influence of guidance strength and starting layer number. Next, analyze how the results change across different starting layers and modulation guidance strengths w. Our main dynamic strategy is the step function (strategy in Figure 3)), and ablate different choices for this strategy.
18

---

Published as a conference paper at ICLR 2026
FLUX dev, CFG
CFG=
CFG = 3.5
W =
W =3
a close-up of an old-fashioned cocktail
Figure 15: We apply modulation guidance across different CFG values and observe consistent improvements, confirming that it complementary CFG.
Specifically, in Figure 13 we evaluate different starting layers with a fixed w = 3 under complexity guidance. This setting allows balance original image preservation with complexity improvement. In particular, = 18 and = 28 fully preserve the original image while enhancing only fine-grained details such face and hands.
Then, in Figure we examine the influence of different values with a fixed starting layer i = 5 under aestheties guidance. We observe that higher w enhances the main object (e.g.. the elevator in the example) but also improves background details. However, excessively large values, such as 8, may introduce artifacts.
Modulation guidance for different CFG. Finally, we examine how modulation guidance behaves under different CFG values, demonstrating that can operate effectively on top of CFG. Using the FLUX dev model with complexity guidance, we evaluate multiple CFG values in combination with modulation guidance. The results [igure[5]shoov show that modulation guidance improves performance across different CFG values, confirming that it is complementary to CFG.
D HYPERPARAMETERS CHOICE
In Table we provide the hyperparameters configuration used in our experiments.
For general changes (aesthetics and complexity), we use positive and negative prompts, following the quality-improvin prompt modifiers commonly adopted in DMs (Oppenlaender) (024). In both cases, we employ strategy for dynamic modulation guidance with w = 3, but vary the starting layer. Specifically, for complexity, apply guidance at deeper layers to better preserve the original content while refining nigh-frequene details.
For specific changes (hands correction and object counting), adopt strategies and 4, as suggested by the ablation study. For hands correction, we use simple positive and negative prompts: Natural and real 1 i stt hands and Unnatural hands. For object counting, the positive direction
19

---

Published as a conference paper at ICLR 2026
Table Comparison with baselines for general changes. We use Normalized Attention Guidance and LLM-enhanced prompts as baselines, and conduct human evaluation on two inseeria-esthetiie and complexity-reponn the corresponding win rates.
Aesthetics
Complexity
Model
Variant
Baseline
Variant
Baseline
Variant
Baxeline: LLM-enhanced prompis
FLUX schnell
Ours
45
55 (+10)
38
62 (+24)
FLUX schnell
Ours + LLMMechanced
39
61 (+22)
26
74 (+48)
COSMOS
Ours LLM-enhanced
41
59 (+18)
35
65 (+30)
Baseline:
Normalized Attention Guidance
FLUX schnell
Ours
33
67 (+34)
(+58)
Table 9: Comparison with baselines for specific changes. We use Concept Sliders and LLM-enhanced prompts as baselines, and conduct human evaluation on two criteria: defects for hands correction and text relevance for object counting, reporting the corresponding win rates.
Defects, Hands
Text relevance, Counting
Model
Variant
Baseline
Variant
Baseline
Variant
Baseline:
LLM- ALM-enhanced prompts
FLUX schnell
Ours
26
74 (+48)
39
61 (+22)
Baseline:
Concept Sliders
FLUX schnell
Ours
42
58 (+16)
adapted per prompt but follows general structure: [n][objectts] where the main object and desired count are taken from the prompt.
For text-to-video generation, we the configuration as in aesthetics guidance for text-to-image generation. We find that this not only makes the videos more realistic but also significantly improves their dynamic degree.
For image editing, we adopt the configuration commonly used in CFG: the original prompt serves as the positive direction and blank prompt as the negative. This setup increases editing strength in cases where the base FLUX Kontext model struggles. For this setting, we use strategy 1.
E BASELINES COMPARISONS FOR TEXT-TO-IMAGE GENERATION
We compare our approach against the following baselines: Normalized Attention Guidance (Chen al.) 2025),, used for general changes: Concept Sliders (Gandikota al. 2024), applied hands correction; and LLM-enhanced prompts (Oppenlaender, 2024), which we consider for both general and specific changes.
For the LLM-enhanced baseline, we use an LLM to modify the prompt sets by adding additional beautifiers, following the same structure used construct the positive directions in modulation guidance. For the other approaches, we adopt the default configurations provided in their respective papers.
We present the results for general changes in Table 8 We observe significant improvements over Normalized Attention Guidance for both criteria (aesthetics and complexity). Importantly, our method does not incur additional overhead, unlike Normalized Attention Guidance, which requires extra passes through computationally intensive attention layers. Second, we find that our approach can be applied top of LiM-ennanced prompts and brings additional improvements. This is especially important in practice, where different modifiers are commonly applied to basic prompts (Ramesh
al. 2022
20

---

Published as a conference paper at ICLR 2026
Table 10: Comparison editing performance measured by VLM scores Editing Strength and Reference Preservation.
Editing Strength
Reference Preservation
Configuration
Material
Object
Style
Replace object
Material
Object
Style
Replace object
Flux Kontext
66 24
78 #2
68 d:
71 25
93 =0.1
92 103
77
90 :2
Flux Kontext w/o CLIP
69 (+3)
78 (0)
68 (0)
71 (0)
93 (0)
93 (+1)
79 (+2)
90 (0)
Flux Kontext using final prompt
(+3)
73 (-3)
(0)
(+2)
93 (+1)
80 (+3)
(-1)
Flux Kontext, modulation guidance
(+13)
(+3)
72 (+1)
78 (+7)
92(0)
78 (+1)
(-1)
Reference image
CFG 2.5
CFG 4.5
Edited image,
CFG 2.5 + Mod. guidance 3
CFG 8.5
CFG 10.5
Figure 16: We find that the FLUX Kontext model sometimes struggles with complex image edits, and even higher CFG values do not alleviate this issue. In contrast, modulation guidance can effectively address such cases.
We present the results for specific changes in Table First, we find that our approach outperforms the i.LM-enhanced prompt baseline both tasks (hands correction and object counting). Notably, for hands correction, the LLM-enhanced prompt approach can lead to divergence- where the model overemphasizes hands and neglects other parts of the image. In contrast, our approach localizes model attention without adversely affecting the rest of the image. Second, we find that our approach even brings improvements over the Concept Sliders approach, without requiring test-time optimization.
F INSTRUCTION-GUIDER IMAGE EDITING
Here, we present the numerical results for instruction-uuided image editing using the FLUX Kontext model (Labs et al., 2025). Specifically, we evaluate four settings: (1) the original model; (2) the model without CLIP; (3) the model using the final textual prompt instead of the editing instruction for CLIP; and (4) the model with modulation guidance. For the latter, we use the final prompt as the positive prompt and blank prompt the negative, as summarized in Table
To evaluate performance, we follow the basic setting of FLUUX Kontext and generate images using the SEED-Data benchmark (Ge et al., (2024), which provides reference images, editing instructions, and final textual prompts. Evaluation is conducted with a VLM model (Bai ct al. 2025), which is asked to assess editing strength and reference preservation 0 100 scale. For this purpose, we provide the VLM with triples consisting of the reference image, the edited image, and the corresponding instruction.
We report the results in Table 10
First, we observe that removing CLIP does not degrade performance and even yields small improvements, further supporting intuition that CLIP does not contribute
21

---

Published as a conference paper at ICLR 2026
Table 11: Performance of text-to-image DMs with and without modulation guidance (gray) on Aesthetics and Complexity, evaluated with human preferences and automatic metrics for long and short prompts. Human win rates are reported with respect to the original model; green indicates statistically significant improvement, red decline, For automatic metrics, bold denotes improvement over the original model.
Side-by-Side Win Rate, % Complexiyy
Automatic Metries, COCO 5k
Model
Relevance
Aesthetics
Defects
PickSoore 1 CLIP
IR
HPSS3
FLUX schnell, short prompts Ours, Aesthetics guidance
6.2
49
64
81
57
21.9
30.2
7.4
8.5
FLUX schmell, long prompts
21.0
33.1
10.3
10.8
Ours, Aesthetics guidance
33.3
meaningful gains. Second, we find that using the final prompt instead of the editing instruction for material and replacement criteria while
the CLIP model leads to inconsistent serccomes-improeee degrading performance on object editing. Finally, we observe that modulation guidance consistently provides improvements across all criteria in terms of editing strength.
Specifically, modulation guidance improves performance on complex editing cases, such as those involving multiple objects. As shown in Figure [6] this problem cannot be solved by simply increasing the CFG scale-only only modulation guidance provides improvements.
G ADDITIONAL EXPERIMENTS
Additionally, we report experimental results for long and short prompts separately to demonstrate that our approach works well with long prompts, whereas basic CLIP tends to influence only short prompts. We conduct a quantitative evaluation using prompts from the MJHQ dataset separated into long and short prompts. We calculate automatic metrics using 1, 000 prompts and conducted a human evaluation using 300 prompts. The results are presented in Table IT We find that our modulation guidance also has positive impact on long prompts. For instance, human evaluation shows improvements of +200 aesthetics and +46% in image complexity compared to the original model (FLUX schnell).
H LIMITATIONS
Our approach also has several limitations. First, it does not address text-to-image correspondence, meaning that it cannot improve how accurately the generated image reflects the input prompt. This limitation is inherent to the modulation guidance design, which focuses on enhancing aesthetic quality, complexity, and other visual attributes rather than semantic alignment. Second, our method introduces small number of additional hyperparameters that must be tuned to achieve optimal performance. While this tuning process relatively straightforward, it may add an extra step compared to baseline methods that do not require such configuration.
I MORE VISUAL RESULTS
We provide additional visual comparisons in Figures 17
18
9
20
21
and 23
J HUMAN EVALUATION
The evaluation conducted using Side-by-Side (ShS) comparisons, where assessors are presented with two images alongside textual prompt and asked to choose the preferred one. For each pair, three independent responses are collected, and the final decision is determined through majority voting.
The human evaluation is carried out by professional assessors who are formally hired, compensated with competitive salaries, and fully informed about potential risks. Each assessor undergoes de- tailed training and testing, including fine-grained instructions for every evaluation aspect, before participating in the main tasks.
22

---

Published as a conference paper at ICLR 2026
In human preference study, compare the models across four key criteria: relevance to the textual prompt. presence of defects, image aesthetics, and image complexity. Figures 24 [7] 25
26jillustrate the interface used for each criterion. Note that the images displayed in the figures randomly selected for demonstration purposes.
K
ADDITIONAL DISCUSSION
This work involves human evaluations conducted through side-by-side image comparisons to assess model performance across various criteria (e.g., aesthetics, complexity, and defects). All human studies were performed with informed consent, and participants were compensated fairly for their time. No personally identifiable information was collected, and all data were anonymized prior to analysis. Our research uses publicly available datasets and pre-trained models, adhering to their respective licenses and terms of use. While our method aims to improve the quality and controllability of generative models, we recognize the potential for misuse of generative technologies, including the creation of misleading or harmful content. We encourage responsible use and recommend implementing safeguards in real-world applications.
We note that in this paper large language model (LLM) was used exclusively for polishing the writing, It was not employed to generate ideas, methods, contributions.
23

---

Published as a conference paper at ICLR 2026
Ours, aesthetics
Original, FLUX schnell
drinking a pint beer
a blach dog sitting between bush and pair of green pants
standing up with nobody inside them
comic about two cats doing research
long shards of broken mirror or reflecting the eyes of great horned mel
Two cups of coffee, one with latte art of heart. The other hus latte of stars.
triangle with smiling face
Ours, complexity
Original, FLUX schnell
A cartnon tiger face
doorknocker shaped like lion 's head
wearing formal clothes, wearing tophat holding The holding garbage bag. Oil painting in the style abstract cubism.
custle made of tortilla chips, mude sulsa.
There tiny burritos walking around the castle
Figure 17: Visual comparisons for FLUU schnell model
24

---

Published as a conference paper at ICLR 2026
Ours, aesthetics
Original, COSMOS
bowl soup that Looks like monster made of plasticine
plaant growing the side brick wall
A boy holds and girl paints piece of wood.
doorknocker shaped like lion's to''s head
black i-shirr with the peace sign
harp without any strings
Ours, complexity
Original, COSMOS
a horse in ffiled of flowers
child eating birthday cake near some palm trees
A castle made carilborrd
bat landing a baseball bat
Figure 18: Visual comparisons for COSMOS model
25

---

Published as a conference paper at ICLR 2026
Ours, aesthetics
Original, HiDream
small kitchen with white goat it
smiling banana wearing bandana
photo of a four-loof clover made of water.
A smiling sloth wearing leather jacket, cowboy hat, kilt and howtie. The sloth is holding quarterstaff and hig book
ARAAAA
A close-up up two beetles wearing karate uniforms and fighting, jamping waterfall.
A bonet of soup that looks like a monster made out of plasticine
Ours, complexity
Original, HiDream
OOPS
oops!
drawiin of a house on a mountain
A glass of red wine tipped over on a couch, with stain that writes 'OOPS' on couch.
LLAME
black busehall flane docal
doorknocker shaped head
Figure 19: Visual comparisons for HiDream-Fast model
26

---

Published as a conference paper at ICLR 2026
Ours, aesthetics
Original, FLUX
bench without any
green heart with shadow
ceiling fon with five brown blades
grandmother reading book to her grandson and granddaughter
kar
we Baa tastoo
a smiling banana wearing bundana
casile made of cardboard
Ours, complexity
Original, FLUX
close-up of two beetles wearing karate uniforms and fighting, jumping over a waterfall.
a cartoon of a bear birthday party
A hovel of soup that Looks like monster made out of plasticine
draving of
Figure 20: Visual comparisons for FLUX model
27

---

Published as a conference paper at ICLR 2026
Ours, aesthetics
Original, SD3.5 Large
Don't Panic!
Don't Panic!
green heart
book with words 'Don' Panic!
man gives a womaa laprop and boy book.
boy going to school
herd of buffalo stampeding at the Kremlin
baby daikon radish in tuta walking a dog
Ours, complexity
Original, SD3.5 Large
city in 4-dimensianal space-time
A banel soup that lonks like mude of plasticine
BONEZ- BONEZ
Fimm
dog weuring busehull cap buckwards writing BONEZ
A custle made cardbbaadl
chalkboard
Figure 21: Visual comparisons for SD3.5 Large model
28

---

Published as a conference paper at ICLR 2026
Ours, hands
Original, FLUX schmell
photo showing hands in studio
photo of woman showing hands during sunset
photo of child showing hands to the camera under the rain
A photo child showing hands in crowded street
photo raising hands cory
harp without any strings
Ours, object counting
Original, FLUX schnell
five shoep grazed in the pasture
three butterflies danced in the garden
frogs leaped into the water
four apples were picked from the free
Figure 22: Visual comparisons for FLUX schnell model
29

---

Published as a conference paper at ICLR 2026
CausVid Ours
a bear climbing a tree
train speeding down the tracks
a bicycle gliding through a snowy field
bird soaring gracefully in the sky
sheep taking a peaceful walk
Figure 23; Visual comparisons for Caus Vid video model
30

---

Published as a conference paper at ICLR 2026
Prompt:
A medieval Scottish castle with grey stone walls and turrets, positioned mirror-like loch under misty sky, Highland
cows grazing in the foreground, rugged mountains rising in the distance.
Image
Which image better according the instructions?
9
The images are uncomparable
Quality: Brightness and contrast
Quality: Acidic unnatural colors
Quality: Glow
Image 1 is better
Image is better
Image better
Image 2 is better
Image is better
Image 2 is better
The images are equal in this aspect
The images are equal in this aspect
The images are equal this aspect
images were not evaluated this aspect
The images were evaluated for this aspect
The images were not evaluated for this aspect
Quality: Image is better
Quality: Image better
Aesthetics: Visibility of the main objects
Aesthetics: Background and environment
Aesthetics: Image detail
Image better
Image is better
Image better
Image better
Image 2 is better
Image better
The images equal in this aspect
The images are equal in this aspect
The images are equal in this aspect
The images were not evaluated for this aspect
The images were not evaluated for aspect
The images were not evaluated for this aspect
3
Aesthetics: Image is better
Aesthetics: Image 2 is better
Can't decide
Figure 24: Human evaluation interface for aestheties.
31

---

Published as a conference paper at ICLR 2026
cakk door and cowhiide leather belt
Defects in composition and watermarks
Image
Image in better
Images are equal
style
The imaes have the same style
mages images uncomparable
The verdict based staps
of objects
Image is better
Image is better
verdict based the previous
of secondary objects
Image la better
Image to better
based previous
1
Image 1 better
Image 2 better
The uncomparable
Enor loading images
Figure 25: Human evaluation interface for defects.
32

---

Published as a conference paper at ICLR 2026
oak door and cowhide leather belt
Main objects
there are more main objects Image
there are more main objects image
both images have the number of main objects
Final
Image is better
Image better
decide
loading images
Main objects
there are more main objects in Image
there are more main objects in Image
both images have the number of main objects
Secondary objects
there are more secondary objects in Image
there are more secondary objects in Image 2
both images have the same number of secondary objects ?
Image better in
effect objects
Image 2 is better in terms of the effect extra objects
both images have the same effect of extra objects ?
Final
Image
Image better
Can't decide
Error loading images
Figure 26: Human evaluation interface for relevance.
33

---

Published as a conference paper at ICLR 2026
Image1
Image
Which image more complex according to the instructions?
Image
Image 2 better
Can't decide
Figure Human evaluation interface for complexity.
34


Reviews:
[{"review_id": "jfW3PnDEAA", "reviewer": "Reviewer_Jrir", "summary": "This paper revisits the role of global (pooled) text conditioning in diffusion transformers, which has recently been discarded in favor of attention-only conditioning. The authors first demonstrate through empirical analysis that the pooled CLIP embedding contributes little to generation quality in several state-of-the-art models (e.g., FLUX, HiDream-Fast), especially with long prompts. However, they propose a novel perspective: repurposing the pooled embedding for modulation guidance—a training-free, plug-and-play technique that steers the diffusion process toward desirable visual properties (e.g., aesthetics, complexity, hand realism) by extrapolating in the modulation space using positive/negative prompt pairs. The method is simple, incurs negligible overhead, works with or without classifier-free guidance (CFG), and can be retrofitted into models that originally lack pooled embeddings. Extensive experiments across text-to-image, text-to-video, and image editing tasks show consistent improvements in human evaluations and automatic metrics.", "questions": "NA", "limitations": "", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 1, "strengths": "1. The paper provides a clear and convincing empirical investigation into why global text conditioning appears ineffective in current models, filling an important gap in understanding.\n\n2. Modulation guidance is training-free, easy to implement, computationally lightweight, and broadly applicable across architectures and tasks.", "weaknesses": "1. The core idea of using pooled embeddings for guidance resembles prior work on semantic directions in GANs (e.g., StyleGAN) and recent methods like TokenVerse or Concept Sliders, though the application to modulation space in diffusion transformers is new.\n\n2. The method introduces new hyperparameters (guidance scale w, layer indices), requiring tuning for different tasks—though ablations help, this adds complexity compared to plug-and-play baselines.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "0j9v8PmFBs", "reviewer": "Reviewer_Tihg", "summary": "This paper revisits the role of global text conditioning in diffusion transformers. In response to the prevailing trend of abandoning modulation mechanisms in favor of attention-only approaches, the authors demonstrate through analysis that while conventionally used pooled text embeddings contribute limited benefits, repurposing them as a guidance mechanism can effectively adjust the diffusion trajectory toward more desirable attributes. This approach, termed modulation guidance, is training-free, straightforward to implement, and enhances performance across multiple tasks including text-to-image, text-to-video generation, and image editing.", "questions": "Please see my **Weaknesses** part.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The revisiting and discovery that global text conditioning can be leveraged as a powerful control signal—rather than being merely a passive input—is novel. The proposed dynamic modulation guidance demonstrates a clear ability to address classic and stubborn challenges in T2I generation, such as hand synthesis and object counting, which is a significant finding.\n\n2. The paper is impressive in its extensive experimental scope, demonstrating effectiveness across a diverse set of tasks—including text-to-image, text-to-video, and instruction-guided editing—and model architectures, encompassing transformer-based DMs and the CLIP-free COSMOS model.\n\n3. The paper is well-structured and easy to follow.", "weaknesses": "I have the following two major questions:\n\n1. I noticed that different hyperparameters are used for different tasks and generation types/styles In Tab.5. Could the authors provide more detailed guidance on the process of selecting the appropriate strategy and its associated hyperparameters for a **new, unseen task**? Is this process largely heuristic, requiring manual search for each new situation, or are there general principles or a methodology that can be derived from the observations in Figure 3 to make this selection more systematic?\n\n2. **Connection and Distinction to h-space Methods:** The work [1] demonstrates that diffusion models possess a semantic latent space (h-space) and that rescaling the difference in latent features ($\\Delta h$) can control attribute strength. Could the authors discuss the primary distinction between their method and this prior work? Specifically, is it possible to achieve a similar guidance effect by simply rescaling $\\Delta h$, analogous to Equation 3 in this paper, instead of explicitly using the CLIP embedding to compute $y(p^+, t) - y(p^-, t)$?\n\n[1] Mingi Kwon, Jaeseok Jeong, Youngjung Uh. \"Diffusion Models Already Have a Semantic Latent Space\". *ICLR*, 2023.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "DUaPJaHQ6m", "reviewer": "Reviewer_rF5b", "summary": "This paper investigates the role of the CLIP modulation component within Text-to-Image (T2I) diffusion models. The authors begin by questioning its necessity, demonstrating through ablation that its removal has a minimal impact on overall generation performance. Despite this finding, the authors argue that this component enable controllable shifts in the generation process. They propose a novel method that involves altering the modulation guidance at different blocks of the diffusion transofmer. This claim is supported by a thorough quantitative analysis across several T2I models and is further extended to text-to-video models. The authors show that their method improves generation quality in terms of aesthetics and complexity. They also demonstrate that it mitigates common generation failures, such as incorrect object counts and anomalous finger generation.", "questions": "- The analysis suggests that CLIP modulation is more influential for short prompts compared to long prompts. Do the authors have a hypothesis for this observed phenomenon?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "The paper is clearly written and well-structured. A primary strength lies in its comprehensive experimental validation. The experiments are thorough and are conducted on 4 T2I models that are trained with CLIP modulation, and even included additional model that was not using CLIP, training it to incorporate CLIP modulation. Furthermore, they include text-to-video models, thereby broadening the applicability of their findings.", "weaknesses": "The primary weakness is the limited novelty of the method. This method (with the exception of choosing the dynamic modulation strategies) was already presented in [1] as a naive approach (Equation 2). If the authors disagree, I would be happy to discuss and understand the novelty better.\n\nA second, smaller weakness, concerns the justification for the proposed dynamic modulation strategies. These strategies are heuristically derived from observed attention patterns within the model. This reliance is a potential weakness, as attention weights are not always a reliable or faithful indicator [2] of a model's internal semantic processing at different hierarchical levels. While the authors attempt to validate these attention-driven heuristics through an ablation study, the results appear inconclusive and fail to provide a definitive justification for the chosen strategies.\n\n[1] TokenVerse: Versatile Multi-concept Personalization in Token Modulation Space (Garibi and Yadin et al. 2025)\n[2] Attention is not Explanation. (Jain et al. 2019)", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "VNvvap7JTv", "reviewer": "Reviewer_DbAo", "summary": "This paper explores the impact of modulation-based text conditioning on text-to-image diffusion models. The authors demonstrate that this technique is an important factor in increasing the quality of generated images. They introduce a simple, training-free method that improves performance across various diffusion models without imposing any additional computational burden at inference time.", "questions": "- The quality improvement shown in Figure 5 appears to stem largely from guiding the outputs to look more photographic, particularly with the introduction of blurred backgrounds. While this is often desirable, it raises a question about the trade-off between this aesthetic guidance and prompt fidelity. For example, if a user explicitly asks for a plain background, will the modulation override this request to produce a more \"image-like\" result with depth of field? Have the authors evaluated this trade-off?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "- The paper is well-written, presenting its concepts and results with clarity.\n- The proposed approach is easy to apply, computationally inexpensive, and demonstrably improves generation quality.\n- The method is validated through a sound evaluation on state-of-the-art models, confirming its effectiveness.\n- A significant advantage is the method's broad applicability, as it can be used even with models that do not rely on a CLIP text encoder.", "weaknesses": "- Based on the observation presented in Table 1 that adding CLIP embeddings can increase the quality of images generated from short prompts, it would be beneficial if the authors would also separate their evaluation based on the criteria of prompt length, to demonstrate that modulation can increase generation quality even with long prompts. A more detailed analysis would strengthen the method's reliability.\n- The evaluation lacks a comparison to common, practical methods for quality enhancement. For example, many users simply add phrases like \"good quality image\" or \"very detailed image\" to their prompts or use negative guidance, which is a standard feature in many diffusion model interfaces.\n- A potential trade-off between the enforced modulation and prompt fidelity is not explored. If the aesthetic qualities introduced by the modulation contradict the user's explicit request in a prompt, it could negatively impact prompt-following. An exploration of this dynamic would strengthen the submission.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)