Diffusion Adaptive Text Embedding for Text-to-Image Diffusion Models
Authors: Byeonghu Na
Minsang Park
Gyuwon Sim
Donghyeok Shin
HeeSun Bae
Mina Kang
Se Jung Kwon
Wanmo Kang
Il-chul Moon
Venue: NeurIPS 2025 poster ()
Keywords: Diffusion models
text-to-image diffusion models
text-to-image generation
Primary area: deep_learning
TL;DR: We propose Diffusion Adaptive Text Embedding (DATE), which improves text-to-image diffusion models by dynamically refining text embeddings throughout the diffusion sampling process.

Abstract: Text-to-image diffusion models rely on text embeddings from a pre-trained text encoder, but these embeddings remain fixed across all diffusion timesteps, limiting their adaptability to the generative process. We propose Diffusion Adaptive Text Embedding (DATE), which dynamically updates text embeddings at each diffusion timestep based on intermediate perturbed data. We formulate an optimization problem and derive an update rule that refines the text embeddings at each sampling step to improve alignment and preference between the mean predicted image and the text. This allows DATE to dynamically adapts the text conditions to the reverse-diffused images throughout diffusion sampling without requiring additional model training. Through theoretical analysis and empirical results, we show that DATE maintains the generative capability of the model while providing superior text-image alignment over fixed text embeddings across various tasks, including multi-concept generation and text-guided image editing. Our code is available at https://github.com/aailab-kaist/DATE.

Diffusion Adaptive Text Embedding for Text-to-Image Diffusion Models
Byeonghu Na
Minsang Park'
Gyuwon Sim
Donghyeok Shin'
HeeSun Bae'
Mina Kang' Se Jung Kwon Wanmo Kang' I1-Chul Moon
KKISST NAVER Cloud, 'summary.ai
lbyeonghu na, pagemu, , g8wlaks486, tlsehdgurO,catteooo cat2507, kasongl3)@kkaiss (easskkaaatte.. se jenung www.@naverecorppppoooo com {wanmo. kang, (amoon)@kaist .c.kr
Abstract
Text-to-image diffusion models rely on text embeddings from a pre-trained text encoder, but these embeddings remain fixed across all diffusion timesteps, limiting their adaptability to the generative process. We propose Diffusion Adaptive Text Embedding (DATE), which dynamically updates text embeddings at each diffusion timestep based on intermediate perturbed data. We formulate an optimization problem and derive update rule that refines the text embeddings at each sampling step to improve alignment and preference between the mean predicted image and the text. This allows DATE to dynamically adapts the text conditions to the reverse-diffused images throughout diffusion sampling without requiring additional model training. Through theoretical analysis and empirical results, we show that DATE maintains the generative capability of the model while providing superior text-image alignment over fixed text embeddings across various tasks, including multi-concept generation and text text-guided image editing. Our code is available at https: gighub........ com/ com/aaill -al--aaist/DDTT
Introduction
Text-to-image generation has recently received significant attention due its capability generate realistic and semantically accurate images from textual prompts. This progress has been largely driven by diffusion models [18, 53], particularly with large-scale models such DALL-E [45] and Stable Diffusion [47]. These models use pre-trained text encoders like CLIP [42] and T5 [43] to encode prompts into embeddings, providing crucial semantic information to diffusion models. Notably, the quality and semantic aligmment of the generated images heavily depend on these embeddings [48].
Despite their success, pre-trained diffusion models often struggle with semantic alignment and human preferences. Recent studies have addressed this using external reward functions, either through preference fine-tuning [3, 31, 56] through applying guidance directly denoised images during sampling [2]. However, these methods focus on model parameters or intermediate latent variables and overlook text embeddings. Most text-to-image diffusion models fixed text embeddings throughout the sampling process (upper part of Figure la), limiting their adaptability to the evolving generation process. Since different diffusion timesteps influence generation different ways [6. 59], static embeddings can fail to capture evolving semantics, leading to suboptimal text-image alignment.
To address this limitation, propose Diffusion Adaptive Text Embedding (DATE), which dynami- cally updates text embeddings at each diffusion sampling step based on the current denoised image (lower part of Figure la). By continuously tuning the embeddings to maximize alignment between the text prompt and the mean predicted image, DATE captures evolving semantics without extra model training or architectural changes. Notably, DATE operates entirely at time by simply inserting embedding updates into existing sampling procedures. Our theoretical and empirical results demon-
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Fixed text embedding
buseball player with red glove
Fladd
Corz
(910 speaa
uplass
unboikeP7 (1me02242200
1.21
Xi
Xo
0.43
text embedding
Adaptive
baseboll player with red glove
y
ImngsRowant
0.
(ours)
Cora
Opt.
Opt.
Opt.
(X)-a
(xc))
Diflusion timestep
(b) ImageReward for mean pre- dicted images across timesteps
(a) Comparison of fixed and adaptive text embeddings diffusion sampling
Figure 1: (a) Overview of the conventional fixed text embedding and the proposed adaptive text embedding during the text-to-image diffusion sampling process. Green shapes represent the diffusion model network, orange shapes represent the text encoder, and gray boxes labeled Opt. indicate our text embedding optimization, detailed in Figure 3. (b) ImageReward [57], text-to-image generation metric, for mean predicted images. Red triangles mark the timesteps where text embedding is updated.
strate that DATE improves text-image alignment while preserving the model's original generative capabilities. When evaluated across various diffusion models and samplers, DATE outperforms fixed text embeddings consistently, indicating its agnostic characteristics to both models and samplers. Fur- thermore, DATE can be effectively integrated into various downstream tasks, such as multi-concept generation and text-guided image editing, highlighting its broad applicability.
Preliminaries
2.1 Diffusion models
Diffusion models consist of two diffusion processes: forward process and its corresponding reverse process [18]. The forward process is typically defined as fixed Markov noise process, which perturbs the data instance Xg ~ ((xo) by adding Gaussian noise:
( = II g(xx [x.-)), where u(xx|xx-1) = V
B.I).
(1)
Here, X1.7 are latent variables for perturbed data, and Be is the variance schedule parameter. Diffusion models aim to approximate the reverse process via trainable Markov chain with Gaussian transitions:
Po (xoT) = (x(x)) IT= po (xxe,, where px(x 1xx)) = (xx-11; (xe, (), of
(2)
Ho (x,, t) is the parametrized mean, of is the time-dependent variance, and py the prior distribution.
The mean function mo(x;,t) t) is trained by minimizing the upper bound of the negative log- likelihood [18]. Notably, this mean function can be equivalently expressed, up to a constant, as score network So 5x(,, t) that approximates the score function Vx, log qi (xx) [53]:
min E,Ex, [||so (xe, t) Vx, log (x,) [12].
Once the transition kernel po is trained, we sample iteratively from T to near-O using Eq. (2).
2.2 Text-to-image diffusion models
Text-to-image generation aims to produce high-quality images that are semantically aligned with given textual description. Recent advances in diffusion models have greatly improved this task [39, 44, 47]. Text-to-image diffusion models can be formulated as learning a score network with an additional text-conditional input to approximate a conditional score function:
min Ex, ||s) c, t) Vx, log q: (x 13)
0

---

Here, с is the text embedding of the text prompt typically obtained from a pre-trained text encoder [11, 33, 48, 61, 62].
Despite their impressive realism, pre-trained diffusion models often struggle to maintain precise semantic alignment or satisfy human preferences. As the formulation of conditional score network suggests, text-conditional diffusion models can be improved by targeting three components: model parameters 0, perturbed data Xt, and text embedding c. Improvements in each component offer complementary benefits for overall quality and text-image consistency.
Fine-tuning and data-space guidance Most prior works focus optimizing the model parameters through fine-tunig [3, 12, 24, 31, 56]. These approaches adjust diffusion models using additional curated datasets or reward signals to improve alignment or human preference satisfaction. However, they require extensive retraining and substantial computational cost. Another direction modifies the perturbed data X2 via external guidance functions. Classifier Guidance [9] steers samples using gradients of time-dependent classifier, while Universal Guidance [2] approximates this approach with time-independent classifier. While effective, such methods require careful guidance scaling across timesteps, and the guidance component needs to be expressed as a classification probability.
Prompt optimization Another line of work focuses the text-conditioning component of diffusion models. Prompt-level optimization targets the input text to produce better conditioning signals [15, 36]. These methods use reinforcement learning with external reward models train language models that generate refined prompts for diffusion models. However, they are costly to train and lack adaptability when the backbone reward function changes.
Text embedding update Beyond prompt tuning, the text embedding e itself plays central role in aligning images with textual intent. Most diffusion models use frozen text encoders and apply the same fixed embeddings across all timesteps, limiting their ability to capture the evolving semantics [6, 27, 27J 59J. We hypothesize that dynamically adapting text embeddings, by transforming static e into time-dependent C,, can improve semantic alignment.
Recent works have begun to investigate this direction. EBCA [40] updates text embeddings at each cross-attention layer via an energy-based objective but lacks global semantic control. P2L [8] directly optimizes text embeddings for an inverse problem objective. Other works focus on special token tuning, such as Textual Inversion [14] and MinorityPrompt [55], which learn special token embeddings for personalized or minority instance generation, sometimes extended to time-dependent variants [55]. However, these methods are task-specific and limited to special tokens. In contrast, DATE provides a general, training-free framework that dynamically updates the entire text embedding throughout the diffusion sampling process, enabling fine-grained semantic adaptation without retraining.
2.3 Evaluation on text-to-image generation
Given the trained model for text-to-image generation, it is necessary to define an evaluation function that measures how well the generated outputs match the text-conditional inputs. We refer to this as the text-conditionel evaluation function, which can also serve as an objective function to guide generation. Common metrics like CLIP score [16] and ImwgeReward [57] are well-suited for this purpose and are easily applicable in the data space, illustrated Figure Formally, this function can be expressed as (xx: y) E R, and h measures the alignment between the image Xo and the corresponding text input y. For simplicity, assume that higher h values indicate better alignment, implying stronger semantic consistency between images and texts.
Text conditiidio evaluation function
Image Encoder
Cosine Similarity
0.7
Xi
and monkey
Text Encoder
N(xo:y)
Figure 2: Examples of text-conditioned evaluation function.
Figure 1b plots the evaluation score changes over time for the mean predicted image, comparing 100 samples generated with fixed text embeddings and proposed dynamic text embeddings using Stable Diffusion. We observe two limitations of the previous approach: 1) the evaluation function h used only for evaluation without being incorporated into the sampling process, and 2) h is only assessed at the final sample. To address these, we directly leverage h as the learning objective during the intermediate periods of the sampling process. This update to the text embedding improves the evaluation function value, ultimately enhancing the quality of text-to- text-to-image generation.
3

---

3 Methods
3.1 Diffusion Adaptive Text Embedding (DATE)
In this section, we propose an objective function to optimize text embeddings during the sampling process. The diffusion sampling process can be expressed as follows:
XT Pr (xr).x,- ~ pe X2, c₁) for t = T, I......
(5)
ct) is the distribution of the intermediate sample x;-1 at timestep t - 1, generated from the sample x, at timestep t and the text embedding C,. Typically, the text embedding e, is obtained from a pre-trained text encoder Lø. i.e., Corg == 1 (y). and remains fixed across all diffusion timesteps, i.e., C2 Corg for all f (upper part of Figure la). However, we claim that the text embedding that is capable of producing effective generation output varies depending on the diffusion timestep & and the current perturbed data x, (lower part of Figure La). We refer to this dynamic text embedding as Diffusion Adaptive Text Embedding (DATE).
where pe (xx,
Our goal is to find the text embedding cę (x())' that maximizes the evaluation function h for samples generated by the diffusion sampling process pe. The objective of DATE is expressed as follows:
max E Xoor 01:7
1~ 1- Pe(x, xx,,,,) [x(x y)]-
(6)
should be noted that Eq. (6) optimizes the alignment h only at the final reverse diffusion step, so the other latent variables Xi.7 are not directly regularized in any specific direction. However, the below derivation from Eqs. (7) to (10) shows that the optimization direction for each X, can be derived from Eq. (6), which optimizes only Xo. considering that Xo is the sequential sampling result from XI:T.
Since the sampling process of diffusion models proceeds iteratively from timestep T to O, we likewise aim to iteratively update the text embedding O according to this sequential order. Hence, this sequential sampling, which is required for practical implementation. renders Eq. (6) into Eq. (7).
Specifically, the sequential nature of diffusion sampling requires two reformulations of Eq. (6).
The adaptive text embedding C₁ needs to be determined sequentially because the corresponding image is estimated at timestep t. Thus, the sequential decision on C₂ decomposes the joint optimization into the step-wise optimization. Particularly, the diffusion sampling process requires specific order in sequential decisions, i.e., from T to 0. This motivation of decomposed and ordered optimization is reflected in the separated max operator Eq. (7).
Motivation 1:
The evaluation of h is performed at the final sampling step 0, not at intermediate step For simplicity, we need calculate the text-image alignment h on the final generation result while maintaining C₁ This motivates maintaining c, until timestep 0 of the data distribution, which is reflected the equality constraints on the feasible set C2 of each max operator in Eq. (7).
Motivation 2:
max- max max E Exp=pr Xo.T 1 po x, e,) (xx: y)
cj EC c,EC, erECr
Here, C. = (c, ||e, forgll25 S p, C, = c, VT < f}, Cong is the original text embedding, and is scale hyperparameter. The constraint ||c, Corgl 12 S p keeps the optimized text embedding c; does not deviate significantly from the original embedding Corg, preserving its original semantic meaning.
Specifically, the optimization problem for C2 in Eq. (7) can be derived follows: max c,EC,
Exp--1--I:-_ pa(x-- lx,, [M(x;: v)]
<
Coillee 1 x o.) (M(Xo: y)]-
max E
(8)
On the left-hand side of Eq. (8), the terms from timesteps from f 1 to T are eliminated since C++11-7 are set by the inner optimizations. On the right-hand side, the simplification arises from the constraint that all text embeddings from to 1 are identical. Therefore. at timestep f, our objective can be expressed as the expectation of the text-conditioned evaluation function with respect to pe fxxx) xx/ c.).
However, solving Eq. (8) is computationally challenging. Evaluating the objective requires Monte Carlo sampling of xg from x,, and each sample involves iterative sampling with multiple network eval- uations, resulting in high computational cost. To address this, we apply a first-order Taylor approxima- tion of the text-conditioned evaluation function h around Xg & (x,, C; f) Ex (xo (xo). a technique commonly used in previous studies [2. 8]:
Ex, [xxxx; y)] ~ h(xo: v) + Exp (xo x 7 V2h(X00 (xx: b)] = h(xo: v). For simplicity, we omit the dependency x; and write (x:) es whenever ambiguity arises.
(9)

---

Corg
Algorithm Diffusion Sampling with DATE
El
Vehu Vabell
XT pr(-)
h
2: Corg < (b(
X₁
%a Corgi
ih(Kois) s)
for to do
baseballl player with rod glowe
Nu(xs, Curzi y.
4: if E {text update steps} then
Smmi..
t Corg + PIIVa (1
Figure 3: Update step of DATE at timestep t. The symbol with an inverted triangle inside a circle repre- sents the normalized gradient with respect to c, and g denotes summation.
end
X,-1 X, + B so c)
end for
The equality in Eq. (9) holds because Ex, [Xo Xol = 0. Therefore, through the approximation in Eq. (9), we propose the following alternative objective instead of Eq. (8):
h(xo (x+, c;; 0); (x,, Ce
(10)
cc:l|e,- Corg CorgllaSp
This objective Eq. (10) optimizes text-conditioned evaluation function on the mean predicted image Xo given the current perturbed image and the text embedding c, Using the Tweedie's formula [10], the mean predicted image Xy can be computed via single score network evaluation:
Xy (xe, C₁: 0) = (x + (1 - āi (al)ss((x (x1, C₁ t)) /Va.
(11)
where Se (x,,,, C₁2 t) a conditional score network, and ã₁ = II' - Br) is the constant related to the variance schedule of the diffusion process.
3.2 Update process of DATE
We present an update process of text embeddings at each timestep to optimize the objective in Eq. (10) using the current perturbed data. Since evaluating this objective requires score network propagation (see Eq. 11), computational cost increases with each update. Therefore, inspired by sharpness- aware minimization [13], we estimate the updated text embedding in single update.
We decompose the updated embedding C, into its original embedding Corg and an update direction ex:
C₁ Corg + Es-
(12)
By expressing c, in terms of an update direction ét, we reformulate the optimization problem in Eq. (10) into an equivalent problem over et, shown in Eq. (13). Next, approximate h.(:, Corg
:)) using first-order Taylor expansion around 6, = 0 (see the appendix for derivation):
== arg max fig (Xe: Corg + Eri 0) arg max V.eeee Corgi y, 0) Es. |
(13)
||e.|l.5
The solution to this optimization problem can be derived using the Cauchy- Consequently, the estimator for the optimal text embedding é is given by
Cauchy-Schwarz Schwarz inequality.
(x Corg y. 0)
Vehe(Xi, Corgi V. 0)
E = 11 ch Corgi y, 0)]l2
é = Corg + V Vlli((X,, Corgi y, 0)]l2
(14)
This update step refines C, by adjusting it along the normalized gradient direction of h, maximizing semantic alignment at the diffusion timestep under the current perturbed data x;. As a result, the updated text embedding è dynamically adapts to the specific diffusion timestep and the corresponding perturbed data. Figure 3 visualizes this update step, and Algorithm presents the overall algorithm for the diffusion sampling process with DATE. We introduce text embedding update steps in lines 4-6 of Algorithm 1, and the denoising process (line 7) can be performed using various diffusion samplers.
3.3 Theoretical analysis
We provide theoretical analyses of DATE, including performance guarantees and its influence on the data space. We provide proofs and additional approximation error analyses in Appendix A.
First, we show that both unconstrained and constrained optimizations of the text embedding in Eqs. (6) and (7) produce better text embedding than the fixed text embedding Corg-
5

---

Proposition 1. Let h([1. .... cr) = Ex [ v)] where No-T-1 ~ IT Po (x Cr),
Xy ~ pr, and C = {c, : ||c -org-e
Corg l2 S p. + C₁ VT < t}. Then,
max max, CT) max max max ((1 ... ,C) Cir
(15)
e:
e
er
2 max ... max ... max l((1,, .. , 2 la ... Corg) EC c,Ec.
(16)
erECr
The first equation shows that sequential maximization in Eq. (15) (corresponding Motivation 1) attains the same optimum as the joint maximization. Introducing the constraints in Eq. (16) (cor- responding Motivation 2) restricts the feasible set and can lower the optimum. Nonetheless, Proposition 1 guarantees that both the unconstrained optimum (Eq. 6) and the constrained optimum (Eq. 7) yield value at least as high as the fixed embedding. Because DATE derived by approxi- mating the optimization problem in Eq. (7), it is expected to improve the text-image alignment of the generated images compared the fixed embedding.
Next, we present Theorem 2 to illustrate how the DATE update influences the perturbed data. Theorem 2. The score function for the text embedding Ct updated by DATE be expressed as:
Xx po, Xx, log (x( + VVxx [[Vehe(M
Vehe(xi,eux)"
V log po (x (ors)) O(p²). (17)
According to Theorem 2, the updated text embedding C₁ can be interpreted as introducing guidance term the original score function, under a sufficiently small p. This guidance effectively improves the alignment between the evaluation function h2 and the model likelihood from the perspective of the text embedding. Therefore, embedding- -baee guidance balances semantic alignment with the underlying model distribution, enhancing prompt fidelity without reducing generation quality.
3.4 Practical implementation
Reducing computational cost Updating the text embedding via Eq. (14) requires computing the gradient of h, through both diffusion and evaluation networks, which increases computational cost, update embeddings only at subset
discussed in Appendix D.1. To mitigate this overhead, of sampling steps (line 4 in Algorithm 1) and reuse the last update embeddings between updates, balancing performance and efficiency as shown in Figure 9 of Section 4.2. In addition, general computationally efficient strategies such as half-precision inference can be applied during sampling. As shown in Table DATE remains compatible with such efficiency techniques, effectively reducing runtime and GPU memory consumption while incurring only a slight performance degradation.
Selection of original text embedding We explore two strategies for choosing the original text embedding Corg at each update step: (1) always use the pre-trained text encoder output, (y), preserving semantic integrity, and (2) use the embedding from the previous step, C++1, allowing broader exploration of the embedding space. To prevent semantic drift in the second approach, we add an L2 regularizer ||c, - [2)]ll2 to the objective in Eq. (10). This term ensures update embeddings remain close to the original. Ablation results comparing these strategies also appear in Figure 9.
Experiments
We evaluate DATE for text-to-image generation primarily using U-Net-based Stable Diffusion (SD) v1.5 [47] with a pre-trained CLIP ViT-L/14 text encoder [42]. Additionally, to demonstrate broader applicability, we include evaluations the latest transformer-bast based model, PixArt-a [5]. Unless stated otherwise, we set the text-conditioned evaluation function to CLIP score, the scale hyperparameter p 0.5, and the embedding from the previous update the original text embedding Corg for each update step. Additional details are in Appendix C.
4.1 Quantitative results
Following previous evaluation protocols [38, 39, 58), we generate 5,000 images from randomly sampled captions in the COCO [30] validation set. We use DDIM [51] and DDPM [18] for SD, and DPM-Solver [32] for PixArt-a. We evaluate the image quality and semantic alignment using several metrics, including zero-shot FID [17], CLIP score (CS) [16], and ImageReward (IR) [57]. with detailed explanations provided in Appendix C.I.

---

Table Performance on the COCO validation set. Sampling steps are indicated in parentheses, and Time is the average sampling time (min.) for 64 samples. Bold values indicate the best performance.
Backbone
Method
Time
FID
CLIP scoret
ImageReward"1
Fixed text embedding (50 steps)
5.64
18.66
0.3204
0.2132
Fixed text embedding (70 steps)
7.87
18.27
0.3199
0.2137
EBCA [40]
8.10
25.85
0,2877
-0.3128
SD v1.5 [47]
Universal Guidance [2]
8.25
18.56
0.3216
0.2221
w/ DDIM [51]
DATE (50 steps)
10% update with CLIP score
7.82
17.90
0.3237
0,2364
all updates with CLIP score
24.20
17.22
0.3292
0.2277
10% update with lewaeReward
7.82
18.61
0.3224
0,4792
all updates with ImageReward
24.20
18.17
0.3224
1,2972
Fixed text embedding (20 steps)
4.35
31.07
0,3201
0.8140
PixArt-a [5]
Fixed text embedding (45 steps)
9.03
30.62
0.3199
0.8174
w/ DPM-
DATE (20 steps)
Solver [32]
50% update with CLIIP
8.93
30.55
0.3237
0.8287
50% update with ImageReward
8.95
31.07
0.3221
0.9514
Table 2: Results COCO using SD v1.5 with various evaluation functions. Bold values indicate the best performance, while italic values denote cases that underperform the fixed embedding.
C8
IR
0.085
CS
0.085
0.502
0.465
Fidelity FIDI
Semantic CS
Preference IR1
0.5
Method
ASt
PS
IR
0.180
0.502
0.437
Fixed embedding (50 steps)
18.66
5.38
0.3204 0.3199
0.2132
21.51
Fixed embedding (70 steps)
18.27
5.37
0.2137
21.50
0.465
0.437
DATE (50 steps, 10% update)
0.0
with Aesthetic Score (AS)
18.82
5.58
0.3169
0.1910
21.46
with CLJIP Score (CS)
17.90
5.35
0.3237
0.2364
21.53
Figure 4: Pairwise Pearson correlations between evalua-
with ImageReward (IR)
18.61
5,40
0.3224
0.4792
21.53
with PickScore (PS)
18.49
5.42
0.3225
0.2745
21.93
tion functions.
Main results Table compares the performance of different text embeddings across various back- bones and samplers. Tables 8 and 9 in Appendix D. further extend this comparison to different classifier-free guidance scales and sampling methods. Across all settings, DATE consistently outper- forms the fixed text embedding, even with matched sampling time for fair comparison. Notably, DATE improves the metric used for its evaluation function h, as well as other metrics, suggesting that it enhances the overall text-conditional generation quality beyond optimizing a single objective.
Compared to recent methods, EBCA [40] applies the energy-based optimization to text embeddings within cross-attention layers but yields suboptimal performance. Universal Guidance [2], which injects basea guidance directly into the data space, yields performance gains but still falls short of DATE. In contrast, DATE explicitly updates text embeddings optimize the evaluation objective, achieving better semantic alignment (CS, IR) while preserving generative quality (FID), as supported by theoretical analysis.
To better understand the role of evaluation functions, we analyze DATE under different functions. Specifically, examine pairwise correlations among different functions and evaluate DATE when each metric, individually or in combination, serves as the evaluation function. We consider Aesthetic Score (AS) [49] for image fidelity, CS for semantic alignment, and two human- prefenneefereencee-e basee metrics, IR and PickScore (PS) [26]. Each provides per-instance scores for generated images, enabling both correlation analysis and direct integration with DATE.
Evaluation function
Figure presents Pearson correlations computed from 1,000 Stable Diffusion samples, and Figure 5a visualizes instance-level relationships between CS and IR. AS and CS exhibit minimal correlation,
FID is excluded since measures distribution-leve similarity and lacks instance- scores.

---

Table 3: Results COCO across backbones.
Table 4: Results COCO with half- precision.
Backhone
Methods
FID1
CS
IRT
Time
Memory
FIDL
SD3 [1]]
Fixed
26.00
0.3337
1.0018
Fixed embedking (50 steps)
5.64
2400
Fixed embedding (70 steps)
7.87
24.0
18.27
0.3199
0.2137
DATE (ours)
26.00
0.3340
1.0457
DATE (50 steps 10% update)
FLUX [28]
Fixed
29.59
0.3257
0.9634
with CLIP Score (CS)
7.82
61.5
17.90
0.3237
0.2364
DATE (ours)
29.41
0.3283
0.9768
4.40
32.9
with Insaeekeaad (IR)
7.82
61.5
18.61
SDXL [41)
Fixed
18.27
0.3368
0.7284
with ImageReward (IR) (FP16)
4,02
30.6
18.03
0.3222
0.4773
DATE (ours)
18,03
0_3382
0,9096
Fixed
DATE with IR
Revorn difffccon proces
Peursion come cue 0.502
wATE
CS-IR
ImageRewart (IR)
Test corbedding updrt
0.30
0.523 CLI soore (CS)
CLIP score (CS)
(a) Correlation b/w metrics
(b) Results of combined metrics
peorape bearddd Aniding nufloard
Figure 5: (a) Correlation between CLIP score (CS) and Im- ageReward (IR). Blue dots represent individual samples, and the red line indicates their linear regression line. (b) Trade-off weighted
Figure 6: Mean predicted im-
ages No(xt, Ct) for each diffusion timestep and text embedding during the sampling process with DATE.
between CS and IR. For DATE with CS+/R, sum of CS and IR as the evaluation function h, and we plot the performance changes as the weights are varied.
suggesting independence between aesthetic quality and semantic alignment. IR and PS correlate moderately with CS and weakly with AS, indicating that they capture both fidelity and alignment aspects. IR and PS themselves correlate moderately, reflecting their distinct sensitivities.
With these metrics as the evaluation function, DATE generally improves all metrics over the fixed embedding, shown in Table 2. However, when AS is used, other metries often degrade, consistent with its low semantic correlation. Moreover, combining multiple evaluation functions, such as weighted sum of CS and IR, improves both metries simultancously, as shown in Figure 5b and Table 10. DATE thus remains compatible with multi-metric objectives, where adjusting weights balances trade-off. Interestingly, the combined objective can yield even higher CS than for CS alone, demonstrating the synergistic potential among evaluation functions during test-time optimization.
Applicability to other backbones DATE can be applied on top of powerful base models. We evaluate it on recent architectures, including SD3 [11]. FLUX [28], and SDXL [41], following the default configurations described in Appendix C. As shown in Table 3, DATE consistently improves text-image aligmment and generation quality across all these models, demonstrating its robustness and broad applicability to modern diffusion architectures.
Computational efficiency To mitigate the increased computational cost introduced by gradient computations, we explore memory-efficient sampling strategies. In particular, apply saff-precision inference during sampling, which substantially reduces runtime and memory consumption while maintaining competitive performance, shown in Table Notably, casting the CLIP model used in the evaluation function to half-precision led to performance degradation, so we retain it in full precision; however, since the diffusion model dominates computational cost, applying half-precision to remaining components still provides significant savings. These results demonstrate that DATE remains fully compatible with standard memory-efficient strategies.
4.2 Analysis of DATE
Generation process The green boxes in Figure 6 show the generation process of DATE, Fixed text embeddings misinterpret 'a man' as 'two men', but DATE corrects this by dynamic updates.
8

---

Table 5: Ablation studies.
-eeoo--hoi shot FTD
ImageRRward
0.25
CLIP scare
Method
FID
CS*
IRt
Fixed
Fixed embedding
16
0.20
Random update
18.66
0.3204
0.2136
0.32
h(x;,) 1/)
18.80
0.3200
0.2121
0.000.1
0.5075
0.050.1
0.50.75
0.050.1
0.50.75
Unnormalized gradient
18.46
0.3212
0.2225
P
DATE (ours)
17.91
0.3220
0.2229
Figure 7: Sensitivity analysis on the scale p.
Sampling time (min.)
1000
1030
850
630
420
210
20
Zero-shot shot FID
CLIP score
0.33
InageReward
0.26
0.24
10
0.22
17
0.32
0.0
0.5
1.0
0.0
0.5
1.0
0.0
0.5
1.0
0.0
0.5
1.0
Update proportion
Update proportion
Update proportion
Update proportion
Fixed
DATE (org.)
DATE (prev.)
Figure 8: Cosine similar- ity of update directions é, between timesteps.
Figure 9: Comparison of sampling time and performance based on the proportion of update steps total sampling steps.
Time- and instance-adaptive text embedding We analyze the time-dependent text embeddings by measuring the cosine similarity between update directions & at different timesteps, averaged over 100 samples, in Figure 8. Most timestep pairs show near-zero similarity, with about 85% of pairs below 0.1, indicating that optimal embeddings differ across timesteps. In contrast, adjacent timesteps generally show positive similarity, suggesting that reusing the embedding from the previous step can reduce runtime with moderate loss in performance.
We also examine instance-specific adaptation by measuring the cosine similarity of update directions across different samples of the same prompt at each timestep. The similarity remains close to zero (below 0.05) across all timesteps, showing that each instance has distinct text embedding updates, reinforcing the need for adaptive embeddings in text-to-image generation.
Ablation studies Table presents several ablations for the text embedding update. Random update, which replaces the gradient with random Gaussian vector, performs similarly to the fixed embedding. This indicates that our update is not just a perturbation, but plays meaningful role in text-image alignment. Alignment with perturbed data, which aligns the text prompt with the perturbed data X, using (xx:; y). results in degraded performance, likely because the evaluation function h lacks explicit information in the perturbed data space. Unnormalized gradient performs better than fixed embeddings but remains inferior to DATE. This suggests that using the unnormalized gradient still serves gradient ascent method to improve ht, but single-step update is suboptimal.
Sensitivity analysis of p Figure presents sensitivity analysis of the scale hyperparameter p. which controls the magnitude of é. The performance consistently outperforms the fixed embedding, but higher p causes degradation in some regions. This is likely due to errors in the Taylor approximation from an expanded feasible region in Eq. (13). Based on this, we set to 0.5 in our experiments.
Selection of original embedding Figure analyzes the effect of selecting the original embedding at each update step, discussed in Section 3.4. Initializing with the previously updated embedding generally improves the CLIP score, likely due to broader exploration of the embedding space. Based on this, we adopt this strategy in our experiments.
Embedding update steps Figure 9 also shows the sampling time and performance based on the number of update steps. We observe that even a few updates improve performance. Increasing the update proportion tends to further improve performance, but it also increases the sampling time.
Figure compares different update strategies while keeping the total number of updates the same, using Reward as the objective. Here, "r is the fixed embedding; c" refers uniform updates; and ct, c. and cl correspond to updates at early, middle, and late sampling steps, respectively. We find that updating at any sets improves performance over no update, with mid-to-late updates (c and c!) being more effective for text-image alignment. This suggests that adjusting text embeddings during the fine-gra meninned detail refinement phase in the later sampling steps is more effective.

---

peorge
object
coption
Sets of steps
embedding update steps Zero-shot FID
IeageReward
Avengg similariy
9.55
6:35
AAAAA
cm
AAAAA
0.30
0:30
A
4
0.21
0:25
0.74
AAAAA
c0r
0:20
0.75
1000 800
600
400
200
18.0
18.5
03155
0.325
0.0
0.5
Diflusion timestep
SD
CONNPRRM
SD CONFORM DATE
Figure 10: Comparison of update steps. On the left, the red triangles mark update steps, while the bars in the remaining graphs show the performance for each set of update steps.
Figure 11: Average similarity on the AnE dataset [4] for multi- concept generation.
SD
DATE
EDD CONFORM
DATE
Source image
DDPM ins.
DATE
Fined
6.25
1PPPS
scutpeure castle castle
6.20
A lon and monkey
6:35
0.28
0.30
A cartoon of cat An originn of dng
CLPP score
dog and blue balloon
(a) Examples of edited images
(b) LPIPS vs. CS
Figure 12:
Examples of generated images on the AnE dataset for multi-concept generation.
Figure 13: Fixed embedding versus DATE in DDPM inversion [21] for text-guided image editing.
4.3 Applications
Multi-concept generation Multi-concept generation aims to generate multiple concepts (e.g.. objects and attributes) within a single text prompt. We evaluate DATE on the AnE dataset [4] with 1) base SD and 2) SD with CONFORM [35], method for multi-concept generation. Following previous work [4, 35], we generate 64 images per prompt and evaluate similarity between text and images using various metrics. Additional details provided in Appendix C.2.
Figure 11 compares performance with and without DATE and shows that applying DATE consistently improves performance across all metrics. CONFORM provides better performance than DATE alone, but requires explicit annotation of attribute-ohject pairs, and applying DATE to CONFORM can further improve performance. Figure 12 illustrates that DATE improves object representation under the same prompt and random seed. These results highlight the effectiveness of DATE in refining concept representation and improving text-image alignment. Additional results are in Appendix D.3.
Text-guided image editing Text-guided image editing modifies an input image based on text prompts, allowing natural language adjustments [34, 37]. We apply DATE to DDPM inversion [21], diffusion- based image editing model: and evaluate it on 30 source images from ImageNet-R- R-T21 [54],
each modified with five target prompts. Evaluation is based on LPIPS [60] for perceptual similarity with the source image and CLIP score for text-image alignment with the target prompt.
Figure 13b compares LPIPS and CLIP score across different guidance scales. The result shows that DATE achieves better trade-off between fidelity to the source image and alignment with the target text. Figure 13a presents edited images for each method. These results suggest that DATE improves text-guided image editing by better balancing content preservation and textual modifications.
Conclusion
We propose Diffusion Adaptive Text Embedding (DATE), which improves text-to-image diffusion models by dynamically refining text embeddings throughout the diffusion sampling process. Unlike conventional methods with fixed embeddings from frozen text encoder, DATE adaptively updates text representations at intermediate steps, effectively addressing semantic misinterpretations and improving text-image alignment. Experiments show that DATE consistently outperforms fixed embeddings across various tasks and methods involving text-to-image diffusion models, highlighting the potential of time- and instance-dependent text embeddings to improve text-to-iman to-image generation.
10

---

Acknowledgments and Disclosure of Funding
This work was supported by the ITTP (Institute of Information & Communications Technology Planning & Evaluation)-ITRC (Information Technology Research Center) grant funded by the Korea government (Ministry of Science and ICT)
References
[1] Brian DO Anderson. Reverse-time diffusion equation models. Stochastic Processes and their Applications, 12(3):3133326, 1982.
[2] Arpit Bansal, Hong-Min Chu, Avi Schwarzschild, Soumyadip Sengupta, Micah Goldblum, Jonas Geiping, and Tom Goldstein. Universal guidance for diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, pages 843-852, 2023.
[3] Kevin Black, Michael Janner, Yilun Du, Ilya Kostrikov, and Sergey Levine. Training diffusion models with reinforcement learning. In The Twelfth International Conference on Learning Representations, 2024.
[4] Hila Chefer, Yuval Alaluf, Yael Vinker, Lior Wolf, and Daniel Cohen-Or. Attend-and-excite: Attention-based semantic guidance for text-to-image diffusion models. ACM Transactions on Graphics (TOG), 42(4):1-10, 2023.
[5] Junsong Chen, Jincheng YU, Chongjian GE, Lewei Yao, Enze Xie, Zhongdao Wang, James Kwok, Ping Luo, Huchuan Lu, and Zhenguo Li. Pixart-StalphaS: Fast training of diffusion transformer for photorealistic text-to-image synthesis. The Twelfth International Conference on Learning Representations, 2024.
[6] Jooyoung Choi, Jungbeom Lee, Chachun Shin, Sungwon Kim. Hyunwoo Kim, and Sungroh Yoon. Perception prioritized training of diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 11472-1 1481, 2022.
[7] Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay. William Fedus, Yunxuan Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, al. Scaling instruction-finetume language models. Journal of Machine Learning Research, 25(70):1-55 2024.
[8] Hyungjin Chung, Jong Chul Ye, Peyman Milanfar, and Mauricio Delbracio. Prompt-tuning latent diffusion models for inverse problems. In Ruslan Salakhutdinov, Zico Kolter, Katherine Heller, Adrian Weller, Nuria Oliver, Jonathan Scarlett, and Felix Berkenkamp, editors, Proceedings of the st International Conference on Machine Learning, volume 235 of Proceedings of Machine Learning Research, pages 8941-8967, PMLR, 21-27 Jul 2024.
[9] Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans on image synthesis. Advances Neural Information Processing Systems, 34:8780-877 8794, 2021.
[10] Bradley Efron. Tweedie's 's formula and selection bias. Journal of the American Statistical Association, (06(496):1602-1611 2011.
[11] Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas Müller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, et al. Scaling rectified flow transform- ers for high- resh-uesstution image synthesis. In Forty-first International Conference on Machine Learning, 2024.
[12] Ying Fan, Olivia Watkins, Yuqing Du, Hao Liu, Moonkyung Ryu, Craig Boutilier, Pieter Abbeel, Mohammad Ghavamzadeh, Kangwook Lee, and Kimin Lee. Dpok: Reinforcement learning for fine-tuning text-to-image diffusion models. Advances in Neural Information Processing Systems, 36:79858-798868 79885, 2023.
[13] Pierre Foret, Ariel Kleiner, Hossein Mobahi, and Behnam Neyshabur. Sharpness-aware min- imization for efficiently improving generalization. In International Conference on Learning Representations, 2021.

---

[14] Rinon Gal, Yuval Alaluf, Yuval Atzmon, Or Patashnik, Amit Haim Bermano, Gal Chechik, and Daniel Cohen-or. An image worth one word: Personalizing text-to-image generation using textual inversion. In The Eleventh International Conference on Learning Representations, 2023.
[15] Yaru Hao, Zewen Chi, Li Dong, and Furu Wei. Optimizing prompts for text-to-image generation. Advances in Neural Information Processing Systems, 36:66923-66939, 2023.
[16] Jack Hessel, Ari Holtzman, Maxwell Forbes, Ronan Le Bras, and Yejin Choi. Clipscore: A reference-free evaluation metric for image captioning. Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 7514 2021.
[17] Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilibrium. Advances in neural information processing systems, 30, 2017.
[18] Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in Neural Information Processing Systems, 33:6840-6855 2020.
[19] Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance. In NeurlPS 2021 Workshop on Deep Generative Models and Downstream Applications, 2021.
[20] Yushi Hu, Benlin Liu, Jungo Kasai, Yizhong Wang, Mari Ostendorf, Ranjay Krishna, and Noah A Smith. Tifa: Accurate and interpretable text-to-image faithfulness evaluation with question answering. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 20406-20417, 2023.
[21] Inbar Siiterrann-Siegglgghe Vladimir Kulikov, and Tomer Michaeli. An edit friendly ddpm noise space: Inversion and manipulations. In Proceedings of the IEEE/CVE Conference on Computer Vision and Pattern Recognition, pages 12469-12477 2024.
[22] Tero Karras, Mika Aittala, Timo Aila, and Samuli Laine. Elucidating the design space of diffusion-based generative models. In Alice H. Oh, Alekh Agarwal, Danielle Belgrave, and Kyunghyun Cho, editors, Advances in Neural Information Processing Systems, 2022.
[23] Dongjun Kim, Yeongmin Kim, Se Jung Kwon, Wanmo Kang, and Il-Chul Moon. Refining generative process with discriminator guidance in score-based diffusion models. In Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett, editors, Proceedings of the 40th International Conference on Machine Learning, volume 202 of Proceedings of Machine Learning Research, pages 16567-16598, PMLR, 23-29 Jul 2023.
[24] Gwanghyun Kim, Taesung Kwon, and Jong Chul Ye. Diffusionclip: Text-guided diffusion models for robust image manipulation. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 2426-2435, June 2022.
[25] Diederik Kingma, Tim Salimans, Ben Poole, and Jonathan Ho. Variational diffusion models. Advances in neural information processing systems, 14:21696-21700 2021.
[26] Yuval Kirstain, Adam Polyak, Uriel Singer, Shahbuland Matiana, Joe Penna, and Omer Levy. Pick-a-pic: An open dataset of user preferences for text-to-image generation. Advances in neural information processing systems, 36:36652-36663, 2023.
[27] Mingi Kwon, Jaeseok Jeong, and Youngjung Uh. Diffusson models already have a semantic latent space. In The Eleventh International Conference on Learning Representations, 2023.
[28] Black Forest Labs. Flux. https://githhnn com/
woombback--a wwwakllfcrreeeeeeeeeeeeeeeeeeaae forest-----oo labs/flux, 2024.
[29] Junnan Li, Dongxu Li, Caiming Xiong, and Steven Hoi. Blip: Bootstrapping language- image pre-training for unified vision- vision-language understanding and generation. In International conference on machine learning, pages 12888-12900. PMLR, 2022.
[30] Tsung-Yi Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In Computer ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6- 12, 2014, Proceedings, Part V 13, pages 740-755. Springer, 2014.
12

---

[31] Luping Liu, Chao Du, Tinyu Pang. Zehan Wang, Chongxuan Li, and Dong Xu. Improving long- text alignment for eext-to-image diffusion models. In The Thirteenth International Conference on Learning Representations, 2025.
[32] Cheng Lu, Yuhao Zhou, Fan Bao, Jianfei Chen, Chongxuan Li, and Jun Zhu. Dom-solver: A fast ode solver for diffusion probabilistic model sampling in around 10 steps. Advances in Neural Information Processing Systems, 35:5775-5787, 2022.
[33] Bingqi Ma, Zhuofan Zong. Guanglu Song. Hongsheng Li, and Yu Liu. Exploring the role of large language models in prompt encoding for diffusion models. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024.
[34] Chenlin Meng, Yutong He, Yang Song, Jiaming Song, Jiajun Wu, Jun-Yan Zhu, and Stefano Ermon. SDEdit: Guided image synthesis and editing with stochastic differential equations. In International Conference on Learning Representations, 2022.
[35] Tuna Han Salih Meral, Enis Simsar, Federico Tombari, and Pinar Yanardag. Conform: Contrast all you need for high-fidelity text-to-image diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9005-9014, 2024.
[36] Wenyi Mo, Tianyu Zhang, Yalong Bai, Bing Su, Ji-Rong Wen, and Qing Yang. Dynamic prompt optimizing for text-to-image generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 26627-26636, 2024.
[37] Ron Mokady, Amir Hertz, Kfrr Aherman, Yael Pritch, and Daniel Cohen-Or. Null-text inversion for editing real images using guided diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 6038-6047, 2023.
[38] Byeonghu Na, Yeongmin Kim, Minsang Park, Donghyeok Shin, Wanmo Kang, and Il-Chul Moon. Diffusion rejection sampling. In Ruslan Salakhutdinov, Zico Kolter, Katherine Heller, Adrian Weller, Nuria Oliver, Jonathan Scarlett, and Felix Berkenkamp, editors, Proceedings of the s1 International Conference on Machine Learning, volume 235 of Proceedings of Machine Learning Research, pages 37097-37121, PMLR, 21-27 Jul 2024.
[39] Alexander Quinn Nichol, Prafulla Dhariwal, Aditya Ramesh, Pranav Shyam, Pamela Mishkin, Bob Megrew, Ilya Sutskever, and Mark Chen. Glide: Towards photorealistic image genera- tion and editing with text text-guided diffusion models. In International Conference on Machine Learning, pages 16784-16804. PMLR, 2022.
[40] Geon Yeong Park, Jeongsol Kim, Beomsu Kim, Sang Wan Lee, and Jong Chul Ye, Energy- cross attention for bayesian context update in text-to-image diffusion models. In Thirty-seventh Conference on Neural Information Processing Systems, 2023.
[41] Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas Müller, Joe Penna, and Robin Rombach. SDXL: Improving latent diffusion models for high resolution image synthesis. In The Twelfth International Conference on Learning Representations, 2024.
[42] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning. pages 7748-87655 PMLR, 2021.
[43] Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lec, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. Exploring the limits of transfer learning with unified text-to-text transformer. Journal of machine learning research, 21(140):1-67, 2020.
[44] Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text-conditional image generation with clip latents. arXiv preprint arXiv:2204.06125, 2022.
[45] Aditya Ramesh, Mikhail Pavlov, Gabriel Goh, Scott Gray, Chelsea Voss, Alec Radford, Mark Chen, and Ilya Sutskever. Zero-shot text-to-image generation. In International conference on machine learning pages 8821-8831 Pmlr, 2021.
13

---

[46] Royi Rassin, Eran Hirsch, Daniel Glickman, Shauli Ravfogel, Yoav Goldberg, and Gal Chechik. Linguistic binding in diffusion models: Enhancing attribute correspondence through attention map alignment. In Thirty-seventh Conference Neural Information Processing Systems, 2023.
[47] Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. High- resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 10684-10695, 2022.
[48] Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily L Denton, Kamyar Ghasemipour, Raphael Gontijo Lopes, Burcu Karagol Ayan, Tim Salimans, Jonathan Ho, David J Fleet, and Mohammad Norouzi. Photorealistic text-to-image diffusion models with deep language understanding. In S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh, editors, Advances Neural Information Processing Systems, volume 35, pages 36479-36494, Curran Associates, Inc., 2022.
[49] Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Wortsman, et al. Laion- 5b: An open large-scale dataset for training next generation image-text models. Advances in Neural Information Processing Systems, 35:25278- 25294, 2022.
[50] Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. In International Conference on Learning Representations, 2015.
[51] Jiaming Song. Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. In International Conference on Learning Representations, 2021.
[52] Yang Song, Conor Durkan, Iain Murray, and Stefano Ermon. Maximum likelihood training of score-based diffusion models. Advances in Neural Information Processing Systems, 34:1415- 1428, 2021.
[53] Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. In International Conference on Learning Representations, 2021.
[54] Narek Tumanyan, Michal Geyer, Shai Bagon, and Tali Dekel. Plug-and-pla diffusion features for text-driven image-to-image translation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1921-1930, 2023.
[55] Soobin Um and Jong Chul Ye. Minority-focused text-to-image generation via prompt opti- mization. In Proceedings of the Computer Vision and Pattern Recognition Conference, pages 20926-20953 2025.
[56] Xiaoshi Wu, Yiming Hao, Manyuan Zhang, Keqiang Sun, Zhaoyang Huang, Guanglu Song, Yu Liu, and Hongsheng Li. Deep reward supervisions for tuning text-to-image diffusion models. In European Conference Computer Vision, pages 108-124 Springer, 2024.
[57] Jiazheng Xu, Xiao Liu, Yuchen Wu, Yuxuan Tong, Qinkai Li, Ming Ding. Jie Tang, and Yuxiao Dong. Imagereward: Learning and evaluating human preferences for texl-to-image generation. In Thirty-seventh Conference on Neural Information Processing Systems, 2023.
[58] Yilun Xu, Mingyang Deng, Xiang Cheng, Yonglong Tian, Ziming Liu, and Tommi S. Jaakkola. Restart sampling for improving generative processes. In seirth Conference on Neural Information Processing Systems, 2023.
[59] Zhongqi Yue, Jiankun Wang, Qianru Sun, Lei Ji, Eric I-Chao Chang, and Hanwang Zhang Exploring diffusion time-steps for unsupervised representation learning. In The Twelfth Interna- tional Conference on Learning Representations, 2024.
[60] Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang. The unrea- sonable effectiveness of deep features as perceptual metrie. In Proceedings of the IEEE conference computer vision and pattern recognition, pages 586-595, 2018.
14

---

[61] Shihao Zhao, Shaozhe Hao. Bojia Zi, Huaizhe Xu, and Kwan- Yee K. Wong. Bridging different language models and generative vision models for text-to-image generation. Ales Leonardis, Elisa Ricci, Stefan Roth, Olga Russakovsky, Torsten Sattler, and Gül Varol, editors, Computer Vision ECCV 2024, pages 10,86, Cham, 2024. Springer Nature Switzerland.
[62] Le Zhuo, Ruoyi Du, Han Xiao, Yangguang Li, Dongyang Liu, Rongjie Huang. Wenze Liu, Xiangyang Zhu, Fu- Yun Wang, Zhanyu Ma, Xu Luo, Zehan Wang, Kaipeng Zhang, Lirui Zhao, Si Liu, Xiangyu Yue, Wanli Ouyang, Yu Qiao, Hongsheng Li, and Peng Gao. Lumina-next Making lumina-t2x stronger and faster with next-dit. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024.
15

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
The abstract and introduction clearly state the scope of DATE, which aligns with the contributions experiments presented throughout the paper.
Justification:
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
Limitations are discussed in Appendix E.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
16

---

Justification:
The assumptions and proofs are given in Appendix A.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification: Please check details in Section and Appendix C.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make clear how to reproduce that algorithm. (b) the contribution is primarily a new model architecture, the paper should describe the architccture clearly and fully.
(c) If the contribution is new model (e.g., a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reprodueibility. the case of closed-source models, may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
17

---

reproducible, they Answer: [Yes]
Justification: The paper indicates that the implementation is publicly available and provides sufficient experimental details in Appendix C. relies publicly available models such as Stable Diffusion v1.5 and CLIP.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (kttps://nips.co
for more details.
While encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https: //nips. wttpccccpppcccccccccccccccccccct wwwpc/ggusss/.ee/eo SCt////ooesuesseeeeeeeeee ssionPolicy) for more details.
The authors should provide instructions data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (c.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary understand the results?
Answer: [Yes]
Justification: Please check details in Section and Appendix C. Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [Yes]
Justification: Statistical significance of the experimental results for the main claim are provided Appendix D.1.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results accompanied by error bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
18

---

The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative emrr rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
Justification: Please check Appendix C.I for details.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https: / /ttp///eerr//.///ee//eee/ cc wttpblic/Ethisauuideeeooo iines???
Answer: [Yes]
Justification:
The research is consistent with the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. if there is a special consid- eration due to laws or regulations their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: Positive societal impacts and negative impacts are discussed in Appendix
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
19

---

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [Yes]
Justification: We discuss the misuse risk of the generative model and mention the safeguards used by the base model, in Appendix E.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g.. code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: The paper cites the original sources of all datasets and pre- trained models used.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
20

---

If this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: The code and model checkpoints is publicly available at https://githhtn com/ htt/aaallaakaiss/DANA
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/mmodl as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Question:
Answer: [NA]
Justification:
This paper does not include crowdsourcing.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country institution) were obtained?
Answer: [NA]
Justification: This paper does not include crowdsoureing.
Guidelines:
The answer NA means that the paper does involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, not include any information that would break anonymity (if applicable), such the institution conducting the review.
21

---

16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology. scientific rigorousness, originality of the research, declaration is not required.
Answer: [NA]
Justification: The paper does not involve large language models (LLMs) as part of the proposed method, experiments, or core contributions.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy (https: / /ttp///erss////// tteurips.cc/Confeeeeeeee /2te 22LL/LLM) for what should or should be described.
22

---

A Proof and additional theoretical analysis
A.1 Proof of Proposition 1
Proposition 1. Lef h(c] .... CT) Exx r [ (x: y)] where No:T-1 IT==I (x XT Pr, and C. = {cr : C|c - Corgll2 I P. = Ct VT < <t}. Then,
max 1 ... CT) max maxh(ci,.... .r) max max c1:
(15)
es
or
"",,,"",,"""" 2 max ... max... max h(c1, 1. cT) 2 h(Corg !.. Corg). (16)
c c,E0
erECr
Proof. The first equality of Eq. (15) holds because the order of the maximization problems in the LHS of Eq. (15) can be interchanged. The second inequality, from the RHS of Eq. (15) to the LHS of Eq. (16), follows since both problems have the same objective function, but the LHS of Eq. (16) has more restrictive feasible set. Finally, the last inequality from Eq. (16) holds because (Corg."" Corg) belongs to the feasible of the LHS of Eq. (16), which ensures that the optimal value of the L.SS of Eq. (16) is equal to greater than hCCorg:"" Corg))
A.2 Proof of Theorem 2
Theorem 2. The score function for the text embedding c₁ updated by DATE can be expressed as:
Vx, (og = Vx, log Po Corg) + /Vx, [[Vee(M
Ve log (xe + 0(p)). (17)
Proof. By applying first-order Taylor expansion, we obtain the following:
Vehi(xe; Corg)
log po (xx (.) = log po X Corg + P I ch (xe, Corg )
(18)
12
(xe, Corg '
log po (x Corg) + [IVchi Corg) ll2
Po (o(x) + O(p²).
(19)
Taking the gradient with respect to x, on both side then confirms the statement:
Vx, logpo (xe(e(
(20)
Velte(Xes CorR) "| ch (Xx, Corg)
Vx, log po
X Corg.
(21)
Vx, log log + Vx {0 Vehe(xee Corg)
Ve (x + O(p)). (22)
Cork)
A.3 Detailed derivation of Eq. (13)
By applying a first-order Taylor approximation of h1 around e, = Eq. (23), we derive the expansion shown in Eq. (24). Since the first term in Eq. (24) is independent of e,, it can be omitted from the optimization objective, as shown Eq. (25).
E arg max (X Corg Eci y, 0) lee
(23)
~ arg max fy (x,, Corg' v, 0) + e. e.V.Vll Vele Corgi y. 01}
(24)
= arg max ef V Veh(((x (x,, Corgi 0) =: &c.
(25)
lleilaasp
23

---

A.4 Theoretical analysis of approximation errors
When the rext-conditioned evaluation function h is the CLIP score, we analyze the approximation error for Eq. (9) analogously to the theoretical analysis presented in [8].
Proposition 3. Let h(xo; y) = g(f (xn); fr(y)) is the CLIP score, where f and fy are CLIP image and text encoder, respectively, and g is the cosine similarity. Assume that there exists constant K >> 0 such that [ff (xo)]l 2 K for all Xo E No. where Ng is the support of pe pe(xolXx, ct). Then, the approximation error of Eq. (10) upper bounded by:
(x)lx (h(xo: v)] 1(xo: S
max X,EX,
m1,
(26)
K
where my == ||x0 No Xollp(xo|xx X4, dx)dxo is the mean deviation of the conditional distribution of Xp-
Proof. First, prove the following lemma for the property of the cosine similarity.
Lemma 4. Assume that there exists constant K > 0 such that ||x|| 2 K for all Then,
lo(x:y) g(x': = K
| x'11-
(27)
Proof of Lemma. The gradient of g with respect to x is:
vxg(x;y) ||||||>||| (v (x*x)
||x|12 x
(28)
Therefore, the norm of gradient can be derived as follows:
(y"x) x)
1
[[Vx(xx;>)|
|yl|
(29)
|
|
|x|||
K
Therefore,
1 K
19(x: - g(x':y)] (max
(x: ||xxxx11-
1|x
||x
x'1
(30)
Note that the first inequality comes from the mean value inequality.
From Lemma 4, we can derive the approximation error as follows.
Texarpo(xxlxx.co (1(xx: ><xx: - / (1(xx;)) (aniiii)--(R::::0: 1(xo: cx)
(31)
= / (4)
co)
(32)
1
/ /11
< K
cx)xo
(33)
1
|| / ||x0
<
max X,EXi
xollpo Ct )dxx
K
(34)
1
= max
V
(x))l mı
(35)
K
X,E
In the upper bound of the approximation error in Eq. (26), K is the minimum norm of CLIP image encoder outputs, which is about 25 in our experiments. Also, max [ reflects the sharpness of CLIP image encoder, since the encoder is composed of neural networks, this value is finite, and
24

---

smoother image encoders result in a lower approximation error. mi decreases becomes smaller. Therefore, as approaches the upproximation error is reduced.
Additionally, by Taylor's theorem, the approximation error for text embedding updates in Eq. (13) is
1 E. Hn, (c)) R1 =
(36)
2
where HA, (é) is the Hessian of d1 evaluated at some è between Corg and Corg + 6t. Since ||aa|| S this error is O(p²). Therefore, tuning P trades off approximation accuracy against optimization flexibility, We empirically analyze this trade-off in Figure
A.5 Theoretical analysis with convex text-conditioned evaluation function
In Section 3.1 of the main manuscript, we reformulate the optimization problem Eq. (8) into Eq. (10) using the linear approximation in Eq. (9), At this point, when the text-conditioned evaluation function h is convex, the objective functions of the two optimization problems satisfy the following inequality.
Proposition 5. If h is convex with respect Xo, then,
hy (x4. Cti Ct 0) = c(;):)) I Lxx Exo-po(xolM... (hxxxx v)],
(37)
where No (X,, Cy 0) Exxo-po(xo/xxc)))xx))
Proof. This follows directly from Jensen's inequality for the convex function h.
Consequently, optimizing the text embedding c₁ to maximize the proposed objective in Eq. (10) expected to inerease the target value Ex ex)lxx.ee) (xo: y)] as well. However, Proposition relies on the assumption of the convexity of f with respect to Xp. This assumption may not always hold in practice, since often contains pre- ireitrained neural network that introduces non-convexity.
B Related works
B.1
Diffusion models
We additionally provide explanation of the stochastic differential equation (SDE) formulation of diffusion models. In continuous time space, the diffusion process can generalized by formulating the forward and reverse processes as SDEs [53]. The forward process is formulated as:
dx, f(x,, f)dt + g(f)dw,,
(38)
where f and are the drift and volatility functions, respectively. Here, W, denotes the standard Wiener process and & [0, 2]. Based on Eq. (38), a data instance Xo is gradually perturbed towards
Once f and of the forward process specified, the reverse process is uniquely defined as shown in Eq. (39), following the previous work [1]:
dx, [f(x, f) (t) log (x;))dF + g(t)d
(39)
where (x()) is the marginal probability distribution of x, at timestep t, and w, denotes the reverse- time Wiener process.
To generate samples, the reverse process requires an intermediate score function Vx, log qi (xt). which is generally intractable. Instead, neural network s (x,x, t) is used to approximate the score function via score matching loss [53]. Note that the score matching objective serves as an upper bound on the negative log-likelihood under certain temporal weighting functions [52]. This score matching objective equivalent to the noise prediction [18, 9] or the data reconstruction [25, 22] objectives.
25

---

Table 6: Comparison of the diffusion guidance methods.
Method
Guidance target
Guidance module
Classifier free guidance [19]
Perturbed data X; Perturbed data X;
Unconditional score network se (x,, 2,t)
Classifier guidance [9]
Time-dependent classifier hy (x,, c)
Universal guidance [2]
Perturbed data X2
indeieddeenddnt classifier h(Xo (Xx,, c))
DATE (ours)
Text embedding C2
Time- Aime-independent classifier (xXo (xe,, ct))
B.2
Guidance methods for diffusion models
Conditional diffusion models have been developed to generate samples that align with given condition y. These models approximate the conditional score Vx, log qe (x/ly) by incorporating the condition additional input to the score network [9, 22]. In this paper, represent this input as the condition embedding c, which encodes the information
To further improve conditional generation, diffusion models often incorporate guidance term into the unconditional score function, as summarized Table Classifier guidance (CG) [9] introduces the gradient of time-dependent classifier to encode conditional information.
(x c, = so (x,, Ø, () + v log hx(
(40)
In contrast, classifier-fres guidance (CFG) [19] eliminates the need for an explicit classifier by leveraging the difference between conditional and unconditional score estimates.
Scro (x,, C, t) so 2,t) + w (so c) -x,, 2,t)
(41)
Universal guidance (UG) [2] approximate the time-dependent guidance using a time-independent classifier, thereby avoiding time-dependent training.
SUG X,, c, so(x,, 2) Vx, log (x c))-
(42)
Similar UG, our method also uses a -ndepenndeen classifier to derive guidance. However, instead of applying this guidance to the perturbed data, we use it to directly adjust the text embeddings, thus modifying the conditioning information as source. The theoretical implications of this text embedding guidance are further analyzed in Theorem 2 from Section 3.3.
B.3 Improving sampling process in fixed diffusion models
Recent studies have explored refining the sampling process in fixed diffusion models [23, 58, 38]. DG [23] adjusts the score network by incorporating an auxiliary term from a discriminator that differentiates real and generated samples, reducing network estimation errors during sampling process. Restart [58] alternates between reverse and forward steps at fixed time intervals. first denoises samples with a deterministic sampler up to a predefined timestep. then injects noise to introduce stochasticity, and repeats this process to mitigate accumulated errors. DiffRS [38] evaluates sample quality at intermediate sampling steps using discriminator, applies rejection sampling scheme, and refines rejected samples by injecting instance-dependent stochastic noise.
Several studies have explored improving the sampling process in fixed diffusion models, specifically tailored for text-to-image diffusion models [4, 46, 35J. AnE [4] improves subject representation by up- dating the intermediate perturbed latent to maximize attention scores for subject tokens. SynGen [46] adjusts the intermediate perturbed latent to enforce linguistic binding between entities and their visual attributes by aligning the attention maps of paired tokens while differentiating the attention maps of unrelated word tokens. CONFORM [35] similarly updates the intermediate perturbed latent using contrastive loss attention maps.
Unlike these methods, our approach does not require additional training of auxiliary components (e.g., discriminator), updates the text embedding, and does not require additional information about the structure of the text prompt (e.g., binding token pairs). Moreover, our method can be integrated with existing approaches, demonstrated in Section 4.3 with experiments using CONFORM.
26

---

C Additional experimental settings
C.1 Experiments COCO dataset
This subsection provides the experimental settings for Sections 4.1 and 4.2, where DATE is evaluated on the COCO validation set [30]. We use Stable Diffusion v1.5, pre-trained on LAION-5B [49]. with fixed CLIP ViT-L/114 text encoder [42] at a 512 512 resolution. We implement DATE on the Stable Diffusion pipeline with the Restart codebase," 3 built on Diffusers." 4 For EBCA [40], we use the official EBCA codebase," which is also built on Diffusers, and its provided hyperparameters. We conducted most experiments on a single NVIDIA A100 GPU with CUDA 11.4, and some ablation studies were performed on single Intel Gaudi v2 using SynapseAI 1.18.0. Our implementation is publicly available at https:/ / thub. com/ aaiialllll wailaab-ee Sab-kaist/DATE
We use DDIM [51] with 50 sampling steps the default sampler using classifier-free guidance [19], and experiments with the DDPM [18] sampler are provided in Appendix D.1. We set the guidance scale to 8 by default and provide results for various guidance scales in Appendix D.I. For DATE settings, unless otherwise specified, we set the text-conditioned evaluation function to CLIP score, computed using CLIP V from the Hugging Face library. If we set h to ImageReward, compute lewae using the BLIP-basee checkpoint from the official ImageReward codebase." We set the scale hyperparameter P to 0.5 and use the embedding from the previous update as the original text embedding Corg. For the ablation studies in Table 5, the text embedding is updated every 10% of the total sampling steps; and for the sensitivity analysis on p Figure 7, the text embedding is updated at every step.
We perform experiments across multiple backbones and samplers. For each backbone, we adopt the default sampler and configuration provided by the diffusers library. For PixArt-a [7] as the text encoder is used. Sampling follows the default setup, employing DPM-Solver [32] with 20 steps and a classifier-free guidance scale of 4.5. SD3 [11] incorporates CLIP-L/14, CLIP-bigG/14, and 1 utilizing floh-natching Euler sampler (28 steps) with guidance scale of 7.0. FLUX [28] adopts a rectified flow transformer paired with CLIP-L/14 and T5-XXL text encoders, using a flow-matching Euler sampler (28 steps) and a guidance scale of 3.5. SDXL [41] relies on CLIP-L/14 and CLIP-bigG/14 as text encoders with a DDIM sampler, 50 steps, and a guidance scale 5.0.
Implementations for PixArt-a, SD3, FLUX, and SDXL are based respectively on the PixArtAl- phaPipeline," Piibieiifiissoon3Piplleeee FluxPipeline," and MtableeiifuusionXLPPillleee 10 available in the Diffusers library.
Following the evaluation protocol of previous studies [39, 58, 38], we generate 5,000 images from randomly sampled captions in the COCO validation set. We fix the captions and random seed in all experiments. We evaluate text-to-image generation performance using zero-shot FID, CLIP score, and Reward Zero-shot FID (Fréchet Inception Distance) [17, 45] measures the distributional similarity between real and generated images with the same text prompt in feature space. Lower zero-shot FID values indicate that the generated images are more realistic and closer to the real image distribution. CLIP score [16] quantifies semantic alignment between generated image and its text prompt by computing the cosine similarity between their embeddings in CLIP space [42]. A higher CLIP score suggests better text-image alignment. ImageReward [57] is learned reward model trained on human preference data. Using a BLIP-based vision-language model [29], evaluates text-image alignment and fidelity based on human judgment. A higher ImageReward score indicates that the generated image is more likely to be well aligned with human preferences.
3 http:////gghtaacccoe/eee///////s
4 http:///gwhhbbcccccggggggaaaa/a/
stable_diiffusionn_
10 httpss//ggggttococeeccccccllslls /ut. mabbleddiffusion._.l
27

---

Table 7: Prompt categories and examples on AnE dataset [4].
Prompt Category
Template
Example
# of prompts
Animal-Animal
[animalA] and [animalB]
a monkey and frog
66
Animal-Object
a [animal] and [color][object]
a monkey and red car pink crown and purple bow
144
Object-Object
[colorA/[objectA] and [colorB][objecth]
66
C.2 Multi-concept generation
We conduct our experiments on the Attend-and-Excite (AnE) dataset proposed by [4]. There are three types of prompts: 1) Animal-Animal: "a [animalA] and a [animalB]", 2) Animal-Object: "a [animal] and a [color][object]", and 3) Object-Object: "a [oober]]oobjectA and a [color]][ojject3]]" Detailed examples are provided in Table
For baseline comparison, we evaluate our approach against base Stable Diffusion and CON- FORM [35]. We implement the CONFORM-based methods using its official codebase," which is built on Diffusers. We use DDIM with 50 sampling steps using classifier-free guidance scale of 8. We set h as CLIP score, the scale hyperparameter p to 0.5, the embedding from the previous update as the original text embedding Corg, and text embedding is updated at every step.
We follow the quantitative evaluation protocol from previous studies [4, 35J. For each prompt, generate images using 64 random seeds and evaluate performance based on text-image similarity and text-text similarity in CLIP space. Full prompt similarity measures the CLIP-based similarity between the entire prompt and the generated image. This metric measures the overall alignment, but it may not fully capture whether all concepts represented. Minimuam object similarity is computed by splitting the prompt into two sub-prompts and taking the lower CLIP similarity score between them, ensuring that even the least-represented concept is considered. For these similarities, we employ the CLIP ViT-B/16 model. The text embedding for each prompt is obtained by averaging the CLIP embeddings of 80 predefined prompt templates (e.g., "a good photo of (prompt)." ).", photo of clean (prompt] J. "." These similarities are then computed as the average similarity between this text embedding and the CLIP embeddings of the 64 generated images. For lext-caption similarity, generate captions for the 64 generated images using pre-trained BLIP image-captioning model [29]. Then, compute the CLIP similarity between the prompt's 's text embedding (obtained as described above) and the embeddings of the generated captions. The resulting similarity score is averaged across all generated images. To compute these metrics, we use the official implementation of AnE,
C.3 Text-guided image editing
We integrate DATE with DDPM inversion [21] on the ImageNet-R -aieet-RTTI R-TI2I dataset introduced in PnP [54]. DDPM inversion recent method that memorizes all latent vectors while tracing the inverse trajectory of diffusion process. It generalizes DDIM inversion in the perspective of DDPM sampling framework. Our implementation is based on the official DDPM inversion codebase." 13
For evaluation, we follow the parameter setting of DDPM inversion and vary the classifier-free guidance scale. Specifically, we Stable Diffusion v1.4 with 100 sampling steps. DDPM inversion is evaluated with guidance scale of (9, 12, 15, 18, 213. while DATE is tested with a guidance scale of {6,9, 9, 12, 15). We set / as CLIP score, set p to 0.5, initialize each step with the embedding from the previous one, and update the text embedding at every step. We report LPIPS [60] for perceptual similarity with the source image and CLIP score for alignment with the target prompt. LPIPS quantifies perceptual similarity using feature representations from a pre- pre-trained VGG network [50], and CLIP score evaluates the cosine similarity between the target prompt embedding and the modified image embedding from a pre-traeiie tre-trained CLIP model.
28

---

Table 8: Performance on the COCO validation set with Stable Diffusion v1.5 using the DDPM sampler with a classifier-fres guidance scale of 8. Sampling steps are 50 unless otherwise specified. Time the average sampling time (min.) for 64 samples, and NFE is the number of score network evaluations. Bold values indicate the best performance.
Method
Time
NFE
Zero-shot FID
CLIP scoret
ImageReward
Fixed text embedding
5.76
100
21.94
0.3223
0.2567
Fixed text embedding (steps=70)
7.91
140
21.11
0.3212
0.2589
EBCA [40]
8.13
100
30.95
0,2851
-0.2843
DATE (ours)
10% update with CLIP score
7.90
105
20.78
0.3239
0.2630
all updates with CLIP score
24.21
150
20.68
0.3312
0.2712
10% update with ImageReward
7.90
105
21.14
0.3246
0.4913
all updates with ImageReward
24.21
150
21.33
0.3240
1.3262
Table 11: Computation time per sampling step for a batch size of 4.
Table 12: GPU memory
usage for batch size of 4.
Operation
Time (sec.)
Uncond. and cond. score network evaluation (base sampling)
0.33
Text embedding update (including gradient computation) Updated score network evaluation
1.07
Method
Memory (GB)
0.28
Fixed
24.0
DATE
61.5
D Additional experimental results
D.1 Additional experimental results on COCO dataset
Other sampler and guidance scale Table shows the experimental results of the baseline and DATE using the DDPM sampler, and Table 9 and Figure 15 show the results over different classifier- free guidance scales. Note that changing the classifier-free guidance scale does not affect the sampling time and NFE, We observe that DATE achieves performance improvements over the baseline in most metrics. These results demonstrate that our method consistently improves text-image alignment for generated images across various samplers and guidance scales.
Additional experiment results on multi-objective optimization Table 10 reports results when using each metric, or their weighted combinations, is used as DATE's evaluation function. In most cases, DATE improves performance over the fixed embedding. regardless of whether given metric is used as the evaluation function. One exception is when AS is included, where performance on other metrics often decreases, likely because AS, being independent of the text input, offers little synergy with semantic alignment metrics, as shown in Figure 4. Consistent with Figure 5b in the main text, the combined objective can yield higher values than using single metric alone indicated by the bold numbers in Table 10, demonstrating the synergistic potential of combining metrics during text embedding optimization.
Updating embeddings at each timestep increases the computational costs. As shown in the leftmost graph in Figure 9 of the main text, sampling time increases with update propor- tion. The overhead stems from extra score network evaluations for Xo. and gradient computations through h and diffusion model, as mentioned in Section 3.4. A breakdown of the time required for each these operations is provided Table 11, and GPU memory usage is reported Table 12.
Computational costs
Despite the added computational cost, Figure 14 shows that DATE consistently achieves better performance than fixed embeddings comparable sampling times. We generate samples using text prompts of the COCO validation set with Stable Diffusion v1.5 using the DDIM sampler. Fixed embedding adjusts the number of sampling steps, and DATE adjusts both the number of sampling steps and embedding updates. DATE outperforms the fixed embedding on all evaluation metrics and sampling times. Notably, simply increasing the number of sampling steps the fixed embedding setup yields only marginal improvements.
29

---

Table 9: Performance on the COCO validation set with Stable Diffusion v1.3, varying classifier-free guidance (CFG) scales. Sampling steps are 50 unless otherwise specified. For DATE, apply 10% update with CLIP score. Bold values indicate the best performance for each sampler and guidance scale.
Sampler
CFG scale
Method
Zero-shot FID
CLIP scoref
ImageReward
DDIM
2
Fixed text embedding
15.90
0.2915
-0.3616
Hixed text embedding (steps=70)
16.49
0,2905
40,3664
EBCA [40]
28.41
0.2492
-0.9913
DATE (ours)
15.00
0.2959
-0.2838
Fixed text embedding
14.04
0.3065
+0.0947
Fixed text embedding (steps=70)
14.14
0.3056
+0.0981
EBCA [40]
20.67
0,2710
0067988
DATE (ours)
13.70
0.3089
+0.0551
Fixed text embedding
16.14
0.3163
0.1165
Fixed text embedding (steps=70)
15.70
0,3155
0.1072
EBCA [40]
20.98
0,2842
-0.4133
DATE (ours)
15.24
0,3182
0.1296
8 Fixed text embedding Fixed text embedding (steps=70)
18.66
0.3204
0.2132
18.27
0.3199
0.2137
EBCA [40]
25.85
0.2877
+0.3128
DATE (ours)
17.90
0.3237
0.2364
DDPM
2 Fixed text embedding
14.07
0,3008
+0,2173
Hixed text embedding (steps=70)
13.58
0,2999
40,2125
EBCA [40]
22.97
0,2629
40,8076
DATE (ours)
13.77
0,3033
-0.1864
Fixed text embedding
15.17
0.3129
0.0315
Fixed text embedding (steps=70)
14.85
0.3120
0.0482
EHCA [40]
21.51
0.2779
40.5186
DATE (ours)
15.04
0.3158
0.0717
5 Fixed text embedding
18.72
0.3199
0.1941
Fixed text embedding (steps=70)
18.27
0,3190
0.1974
EBCA [40]
25.07
0,2862
-0.3178
DATE (ours)
18.32
0.3225
0.2051
8 Fixed text embedding
21.94
0,3223
0.2567
Fixed text embedding (steps=70)
21.11
0,3212
0,2589
EBCA [40]
30.95
0.2851
-0.2843
DATE (ours)
20.78
0.3239
0.2630
Zero- shot FID
ImageReward 1 0.75
49
0.324
CLIP score
0.50
0.322
48
0.25
0.320
5
10
10
5
10
Sampling time (min.)
Sampling time (min.)
Sampling time (min.)
Fixed
DATE with ImageReward
Figure 14: Performance comparison between fixed text embedding and DATE with ImageReward over different sampling times (minutes per 64 samples). FID values are computed using 1,000 samples, unlike the 5,000 samples used in the main text, which causes scale discrepancy.
30

---

Table 10: Results on COCO with SD v1.5 (extension of Table the main text), including various combined evaluation functions. Columns 2-5 indicate which metrics are used in h. Higher and lower weights columns 2-5 are denoted by @ and o,, respectively; a blank indicates the metric not used. Bold numbers mark cases where combined metrics outperform the target single metric alone.
Evaluation function used in h
Metrics CSt
Method
AS
CS
IR
PS
FID
ASt
IR
PSt
Fixed (50 steps)
18.66
5.38
0.3204
0.2132
21.51
Fixed (70 steps)
18.27
5.37
0.3199
0.2137
21.50
DATE (50 steps, 10% update)
with a single evaluation function
AS
18.82
5.58
0.3169
0.1910
21.46
CS
17.90
5.35
0.3237
0.2364
21.53
IR
18.61
5.40
0.3224
0.4792
21.53
PS
18.49
5.42
0.3225
0.2745
21.93
with two combined evaluation functions
AS+CS
18.77
5.58
0.3171
0.1911
21.46
18.15
5.38
0.3219
0.2179
21.43
AS+IR
18.90
5.57
0.3176
0.2428
21.48
18.15
5.43
0.3216
0.4575
21.53
AS+PS
18.81
5.58
0.3175
0.2091
21.54
18.67
5.44
0.3219
0.2705
21.91
CS+IR
17.94
5.39
0.3241
0.4430
21.52
18.04
5,40
0.3225
0.4756
21.53
CS+PS
18.33
5.40
0.3241
0.2839
21.87
18.61
5.42
0.3224
0.2753
21.93
IR+PS
18.15
5.41
0.3226
0.4774
21.63
18.54
5.42
0.3227
0.3126
21.93
with three combined evaluation functions
AS+CS+IR
18.87
5.57
0.3179
0.2487
21.48
18.50
5.46
0.3208
0.3332
21.50
18.07
5.43
0.3219
0.4557
21.53
AS+CS+PS
18.96
5.57
0.3179
0.2092
21.55
P
18.62
5,47
0.3211
0.2450
21.69
18.69
5.44
0.3221
0.2705
21.91
AS+IR+PS
18.93
5.57
0.3186
0.2602
21.56
18.33
5.44
0.3222
0.4599
21.61
18.72
5.44
0.3224
0.3071
21.91
CS+IR+PS
18.32
5.41
0.3244
0.4192
21.81
18.16
5.41
0.3229
0.4812
21.63
18.68
5.42
0.3228
0.3138
21.92
Statistical significance To assess the statistical significance of the CLIP score improvements introduced by using ImageReward as h, we conduct paired t-test. Comparing samples generated with fixed embeddings (sampling time = 6.34 minutes) and DATE (sampling time 6.03 minutes), we obtain a p-value of 0.00056, indicating a statistically significant improvement at comparable sampling costs.
D.2 Additional analysis of DATE
Multiple text embedding updates We hypothesize that performing multiple text embedding updates per sampling step can expand the search space beyond the initial p-ball, potentially leading to improved performance. Figure 16 indicates that the CLIP score generally increases with more update iterations. However, each additional iteration incurs extra forward and backward passes, resulting in linear increase in sampling time.
31

---

Sampling time
1.8
18
-Zzo-loo FTD
-Zzoo-soll FID
CLIP score
177
12
10
0.322
1.
15
Update iter. per step
Update iter. per step
0.30
0.31
0.32
-1.2
0.0
0.2
CLIP score (CS)
Imag:Reward (IR)
Figure 16: Sensitivity analysis the number of up- date iterations per sampling step with a 50-step DDIM sampler and classifier-free guidance scale of 8. For DATE, we apply 10% update with CLIP score. When the number of iterations is O, identical fixed em- bedding.
Fixed
Fixed (70 stepa)
DATE
Figure 15: Performance between FID and text-image alignment metrics vary- ing the classifier-free guidance scale with the DDIM sampler.
Text prompt: A bearded man in wetsuit holding surfboard.
AIPE
AP
EnPL
[z()) - Cs C1 (ours)
[v)
C5 C4
C3
C2
C₁
Figure 17: Generated images DATE and various fixed text embeddings. The bottom image each column is the generated image, while the images above are zoomed-in views of the boxed regions in the generated image. (g) represents the text embedding from the original text encoder, while Cs, ..., Ci represent the text embeddings updated during the intermediate sampling steps of DATE, with larger indices indicating earlier stages of sampling. The leftmost image is generated using DATE with dynamic updates, while the remaining images are generated with fixed text embeddings.
Time-adaptive text embedding To analyze the updated text embeddings of DATE, we inject the each updated embedding into the entire sampling process. Figure 17 shows the generated images from DATE and several fixed text embeddings. With the embedding obtained after the middle sampling step (c,)) we observe that the information 'two men' is transformed into man' in the text embedding. Furthermore, when using the updated text embeddings at later sampling steps (C2, C1), we observe that the face region of the generated image appears distorted. This suggests that the final updated text embedding is not necessarily globally optimal, and that an appropriate text embedding may exist at each diffusion timestep.
D.3 Additional results for applications
Multi-concept generation In addition to the main evaluation metrics, we also assess model perfor- mance using the TIFA score [20], a recently proposed metric designed to measure the faithfulness of generated images to their textual prompts. TIFA leverages a Visual Question Answering (VQA) model to quantify alignment between image content and prompt, providing an evaluation that is independent of CLIP.
Table 13 presents the full evaluation results across all prompt types in the multi-concept generation setting. We also include results for DATE with ImageReward as h. DATE consistently improves the TIFA score across all tested cases. Furthermore, when using ImageReward outperform the baseline across all evaluation metrics, highlighting its robustness and effectiveness regardless the chosen evaluation function. In addition, Figure 18 shows more generated images for various prompts from the AnE dataset. These results demonstrate that DATE effectively applies to multi-concept generation methods, enabling the generation of images that accurately reflect the given concepts.
h, DATE continues to
32

---

Table 13: Performance comparison for multi-concept generation on the AnE dataset, compared across
Stable Diffusion, DATE with ImageReward, and DATE with CLIP score.
Prompt type
Method
Full prompt
Min. object
Text-caption
TIFA score
Stable Diffusion
0.3123
0.2174
0.7677
0.6847
Animal-Ane Animal
DATE (ImageReward)
0.3219
0.2371
0.7858
0.7948
DATE (CLIP score)
0.3282
0.2398
0.7888
0.8159
Stable Diffusion
0.3443
0.2480
0.7925
0.8223
Animal-Object
DATE (ImageReward)
0.3454
0.2512
0.8009
0.8486
DATE (CLIP score)
0.3530
0.2568
0.8009
0.8420
Stable Diffusion
0.3377
0.2404
0.7684
0.6402
Object-
DATE (ImageReward)
0.3391
0,2454
0.7706
0.6910
DATE (CLIP score)
0.3503
0.2544
0.7728
0.6643
SD
DATE
EDD + CONFORM
DATE
DATE
SD CONFORM
DATE
bear and nuttle
bear and horse
monkey
dog purple car
A frog with a crown
A yellow how and a pink bowl
Figure 18: Additional generated images the AnE dataset for multi-concepl generation.
Source image
DDPM
DATE
Source image
DDPM inv.
DATE
sheich a penguin A video-gume of pengain
tuy of hunky - painting of husky
A toy of a jeepp cartoon of a jeep
sculpture a cat An embroidery a bear
A shatcc of panda sculpture of dog
An embroidery of castle sculpnure of pagoda
Figure 19; Additional examples of edited images the ImageNet-R-T12. dataset for text-guided image editing.
33

---

Text-guided image editing Figure 19 shows several examples of text-guided image editing using DDPM inversion with and without DATE applied the source images. For each method, we present the processed image obtained with the hyperparameter that makes LPIPS less than 0.25. These examples demonstrate that applying DATE to DDPM inversion improves text-guided image editing by better preserving the structure of the source image while improving alignment with the target prompt.
E Limitation and broader impact
Limitation One primary limitation of our approach is the additional computational overhead introduced by updating text embeddings during the sampling process. These updates require gradient computations, resulting in increased sampling time and memory usage. However, experiments demonstrate that DATE outperforms fixed embeddings under equivalent sampling-tine budgets, suggesting favorable trade-off between efficiency and effectiveness. Nevertheless, repeated gradient computations present challenges in terms of memory and computing efficiency, especially in resource- constrained settings. Developing memory-efficien techniques for gradient updates is important direction for future work.
Another potential limitation is the dependence on the evaluation function. While we explore the use of multi-objective evaluation functions and provide empirical evidence that DATE does not overfit the evaluation function itself, the overall generation quality can still be influenced by the design and reliability of the evaluation function. This highlights broader challenge text-to-image generation: the field continues to lack fully reliable, general-purpos evaluation metrics. Continued research on evaluation protocols and their integration with guidance mechanisms crucial for advancing robust and generalizable generation frameworks.
Broader impact Improving conditional embeddings in diffusion-based generative models remains an underexplored area, despite being key component of conditional generation. Our work addresses this gap by proposing a general and effective method to refine text embeddings during sampling, thereby enhancing alignment between the prompt and the generated image. A significant advantage of our approach that it operates without requiring any additional model training and agnostic to the backbone model and sampler. This makes it readily applicable to wide range of text-to-image generation systems.
However, using external modules introduces potential vectors for misuse. For example, adversarial manipulation of these components could compromise model safety and lead unintended harmful outputs. To mitigate these risks, appropriate safeguards could be incorporated into the evaluation functions and sampling process. Responsible deployment of such systems should account for these concerns.
F License information
Our implementation will be publicly released under standard community licenses. In addition, we provide the license information for the datasets and models used in this paper:
SD v1.5:
PixArt-a: http:///ghh.t.oooo.aaaaaa/aaa///
CLIP: http://ghthhaacconeenea////////2
ImageReward: http:///tthh..oorrr///////eee//2
TIFA: COCO: htte:///coodtaaee...g///tttttttt AnE:
http:///ttt.aaaaa/aaaaaaaa//tttt LICENSE
ImageNet-R-T12I:
Restart:
34

---

EBCA:
CONFORM:
DDPM Inversion:
35


Reviews:
[{"review_id": "XlI45jKTCQ", "reviewer": "Reviewer_1BqT", "summary": "This paper introduces Diffusion Adaptive Text Embedding (DATE), a framework for improving text-to-image diffusion models by dynamically updating the text embedding at each diffusion timestep. Unlike traditional models that use a fixed text embedding across all timesteps, DATE adapts the embedding based on the current denoised image, enhancing semantic alignment. The update process is guided by maximizing a text-image alignment metric using a gradient-based optimization. Theoretical analysis supports that DATE improves alignment without degrading generative quality. Experiments show consistent performance gains across tasks such as multi-concept generation and text-guided image editing.", "questions": "Q1. The authors used two metrics (CLIPScore and ImageReward) for optimization. Can another metric can be applied for optimization?\n\nQ2. For further questions and concerns, please refer to the Major and Minor Weaknesses sections above.", "limitations": "yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "**Strengths**\n\nS1. This paper presents a clearly motivated framework for text-to-image generation using diffusion models. Guiding the diffusion model with adaptive text embeddings is both intuitive and effective, since it does not require the training of the diffusion model parameters.\n\nS2. The proposed method is based on strong mathematical formulation (Sec. 3.1. ~ 3.2.), supported by theoretical analysis (Sec. 3.3., Appendix A), which makes the proposed method solid.\n\nS3. The experimental results (including supplementary materials) are impressive. DATE outperforms baselines in FID [1], CLIP Score [2] and ImageReward [3]. Notably, optimizing the method with a specific metric (e.g., CLIP Score [2]) still leads to improvements across other evaluation metrics, including FID [1] and ImageReward [3].\n\nS4. The proposed method demonstrates good applicability and generalizability. DATE can be applied for both text-guided image editing and multi-concept generation, and can be incorporated into orthogonal prior work (e.g. DDPM Inversion [4]), further extending its versatility.\n\n**Major Weakness**\n\nW1. The comparison with prior work is limited. Baselines including prompt optimization-based methods [5, 6, 7], reward-based methods [8], and other guidance strategies [9, 10] should be involved for a more comprehensive and strong evaluation. Is optimizing the text embedding truly the most effective approach for improving text-to-image diffusion models, compared to other techniques?\n\nW2. Along with W1, the paper should better clarify its novelty compared to prior prompt optimization-based works, especially with regard to what DATE offers beyond existing strategies.\n\n\n**Minor Weakness**\n\nW3. The paper would benefit from visualizing diverse qualitative results from both the baseline algorithms and the proposed method for a more fair comparison.\n\nW4. DATE requires x2.5 GPU memory for a batch size of 4 (Table 8) compared to the method using fixed text embedding. Is there a lightweight method (revised version) that can reduce the GPU memory usage?\n\nReferences\n\n[1] Heusel, Martin, et al. \"Gans trained by a two time-scale update rule converge to a local nash equilibrium.\" in NeurIPS (2017).\n\n[2] Hessel, Jack, et al. \"Clipscore: A reference-free evaluation metric for image captioning.” in EMNLP (2021).\n\n[3] Xu, Jiazheng, et al. \"Imagereward: Learning and evaluating human preferences for text-to-image generation.\" in NeurIPS (2023).\n\n[4] Huberman-Spiegelglas, Inbar, Vladimir Kulikov, and Tomer Michaeli. \"An edit friendly ddpm noise space: Inversion and manipulations.\" in CVPR (2024).\n\n[5] Mo, Wenyi, et al. \"Dynamic prompt optimizing for text-to-image generation.\" in CVPR (2024).\n\n[6] Um, Soobin, and Jong Chul Ye. \"Minority-Focused Text-to-Image Generation via Prompt Optimization.\" in CVPR (2025).\n\n[7] Hao, Yaru, et al. \"Optimizing prompts for text-to-image generation.\" in NeurIPS (2023).\n\n[8] Kim, Semin, et al. \"Reward-Agnostic Prompt Optimization for Text-to-Image Diffusion Models.\" arXiv preprint (2025).\n\n[9] Sadat, Seyedmorteza, et al. \"CADS: Unleashing the diversity of diffusion models through condition-annealed sampling.\" in ICLR (2024).\n\n[10] Um, Soobin, and Jong Chul Ye. \"Self-guided generation of minority samples using diffusion models.\" in ECCV (2024).", "was_revised": true, "final_justification": "During the rebuttal, my major concerns regarding (1) comparison with additional baselines, (2) novelty, and (3) computational cost were resolved. Therefore, I have increased my score to 5.", "rebuttal": "We appreciate the thorough reviews and valuable comments. We address the concerns below.\n***\n> ### **W1 & W2: [Previous works]**\n\nThank you for pointing out these relevant lines of work. We agree that a more comprehensive comparison with prompt optimization and guidance strategies would strengthen the paper. In the revised version, we include both a detailed discussion and additional experiments covering the following areas, as below.\n\n**(Prompt optimization method)**\n\nSeveral learning-based approaches [5, 7] generate static prompts via training, often with reward models or reinforcement learning. These methods are effective but costly, and fixed prompts lack adaptability during sampling. In contrast, DATE is training-free and dynamically updates text embeddings at inference time across timesteps. This enables fine-grained control and responsiveness to intermediate generations.\n\n[6] is a concurrent method that also performs inference-time prompt optimization by updating a placeholder token for minority instance generation. While [6] focuses on a specific task and optimizes a single token, DATE is a general framework that updates the full embedding vector and is applicable across tasks.\n[8] proposes a training-free, gradient-free method that leverages two LLMs to iteratively refine prompts. While this method avoids training, it requires multiple LLM inferences and image generations per prompt, making it computationally expensive. In contrast, DATE uses a single image trajectory per instance and updates the text embedding sequentially at each timestep (Motivation 1 from Section 3.1 in the main paper).\n\nWe evaluate DATE individually and in combination with Promptist [7] (trained prompts for aesthetics) using SD v1.4 and ImageReward:\n\n**[Table: Results with the prompt tuning method, Promptist [7]. Note: FID values may differ in scale from those in the main paper due to the smaller sample size (1k).]**\n\n| Method             | FID-1K↓ | CLIP score↑ | ImageReward↑ | Aesthetic score↑ |\n|--------------------|---------|-------------|--------------|------------------|\n| Fixed    | 48.81   | 0.3206      | 0.1801       | 5.40             |\n| **DATE**             | 48.09   | 0.3233      | 0.4680       | 5.43             |\n| Promptist        | 57.75   | 0.2785      | -0.0380      | 6.23             |\n| Promptist + **DATE** | 56.23   | 0.2827      | 0.2471       | 6.20             |\n\nPromptist focuses on improving aesthetic quality (represented by Aesthetic score), often at the cost of other metrics like CLIP score, FID, and ImageReward. In contrast, DATE improves all metrics, including Aesthetic score, which is not used in its objective, indicating good generalization. When combined with Promptist, DATE helps recover image-text alignment and overall quality while maintaining high aesthetic quality. This suggests DATE can complement prompt tuning methods by improving balance and robustness during inference.\n\n**(Guidance strategies)**\n\nAs discussed in the main paper, Universal Guidance (UG) applies external functions to guide sampling in the perturbed data space. [10] targets minority sample generation by proposing a metric that can be internally computed within the diffusion model and applying guidance in the data space.  \n\nIn contrast, DATE applies guidance in the text embedding space, which introduces a fundamental distinction from data-space guidance methods. In Section 4.3, Theorem 2 provides a theoretical foundation showing that DATE updates the text embedding in a direction that aligns with the evaluation function while simultaneously accounting for the model’s original text-conditional likelihood. This enables DATE to strike a better balance between semantic alignment and preservation of the diffusion model’s distribution. Experimentally, we also show in Table 1 that DATE outperforms UG.\n\nMeanwhile, CADS [9] shows that injecting time-dependent Gaussian noise into the conditional embedding can improve generation diversity. We empirically compare DATE with CADS and evaluate their combination. The experiments follow the same setup as Table 1 in the main paper, with DATE using 10% update with ImageReward configuration. Given CADS’s known effectiveness in improving diversity, we include Recall as an additional diversity metric.\n\n**[Table: Performance on the COCO validation set with CADS [9], including Recall metric.]**\n\n| Method      | FID-5K↓   | CLIP score↑ | ImageReward↑ | Recall↑    |\n|-------------|-----------|-------------|--------------|------------|\n| Fixed       | 18.66     | 0.3204      | 0.2132       | 0.8132     |\n| **DATE**        | 18.61     | **0.3224**  | **0.4792**   | 0.8225     |\n| CADS        | 16.62     | 0.3119      | -0.0534      | 0.8328     |\n| CADS + **DATE** | **16.52** | 0.3122      | 0.1973       | **0.8434** |\n\nAs shown in the table, CADS (third row) improves diversity, as reflected by better FID and Recall, but at the cost of reduced text-image alignment, with lower CLIP score and ImageReward. In contrast, DATE (second row) improves both alignment and diversity metrics. When CADS and DATE are combined (last row), the result shows a synergistic effect: FID and Recall improve further, while alignment metrics also increase relative to CADS alone. This suggests that while CADS excels in enhancing diversity, it does not improve text-image alignment, which can be effectively complemented by combining it with DATE.\n\n***\n\n> ### **W3: [Qualitative results]**\n\nThank you for the suggestion. We currently provide qualitative comparisons for our application tasks in the main text (Figures 11 and 12) and in the appendix (Figures 17 and 18). The review system does not allow uploading additional images during the rebuttal period. In our revision, we will add diverse generated samples, e.g., COCO-prompt images corresponding to Table 1, to provide clearer visual comparisons. \n\n***\n\n> ### **W4: [Lightweight method]**\n\nThe increase in GPU memory usage for DATE is primarily due to the gradient computations, as noted in the Limitations section. To address this, we explore memory-efficient techniques. In particular, we apply half-precision (FP16) inference during the sampling process. As shown in the below table, this reduces runtime and memory consumption, albeit at the cost of slightly decreased performance. Nevertheless, our methods still outperform fixed embedding methods.\n\n**[Table: Computational resources and performance on the COCO validation set. This is an extended version of Table 1 in the main text, including results with half-precision.]**\n\n|                             | Time | Memory | FID↓  | CLIP score↑ | ImageReward↑ |\n|-----------------------------|------|--------|-------|-------------|--------------|\n| Fixed (50 steps)            | 5.64 | 24.0   | 18.66 | 0.3204      | 0.2132       |\n| Fixed (70 steps)            | 7.87 | 24.0   | 18.27 | 0.3199      | 0.2137       |\n| **DATE** (50 steps, 10% update) |      |        |       |             |              |\n| &nbsp;with CLIP score         | 7.82 | 61.5   | 17.90 | 0.3237      | 0.2364       |\n| &nbsp;with CLIP score (FP16)  | 4.40 | 32.9   | 17.99 | 0.3229      | 0.2265       |\n| &nbsp;with ImageReward        | 7.82 | 61.5   | 18.61 | 0.3224      | 0.4792       |\n| &nbsp;with ImageReward (FP16) | 4.02 | 30.6   | 18.03 | 0.3222      | 0.4773       |\n\nOne important note is that in our implementation, casting the CLIP model used in the evaluation function to FP16 resulted in degraded performance. Therefore, we kept this component in full precision. Nevertheless, since the diffusion model accounts for most of the computational overhead, applying FP16 to the rest of the pipeline still provides substantial savings.\n\nIt is also worth noting that FP16 is a general memory-efficient strategy that can be applied across different sampling methods. In this context, our results demonstrate that DATE remains compatible with such strategies, despite its additional gradient computations. However, we do not claim that FP16 integration represents a fundamental improvement to DATE itself in terms of memory or speed. Rather, it shows that DATE can effectively incorporate standard efficiency techniques without loss of compatibility.\n\n***\n\n> ### **Q1: [Another metric for optimization]**\n\nOur method is compatible with any evaluation metric that provides access to model weights. We experiment with Pickscore (Kirstain et al., 2023), another popular text-to-image evaluation metric, as the optimization objective. Similar to our results with CLIPScore and ImageReward, using PickScore also led to consistent improvements across all evaluation metrics. Additionally, when evaluating images optimized with other metrics using PickScore, our method still outperformed the fixed embedding baseline. \n\n**[Table: Performance on the COCO validation set using Stable Diffusion v1.5 with DDIM sampler. This is an extended version of Table 1 in the main text, including results with PickScore.]**\n|                             | Time | FID↓      | CLIP score↑ | ImageReward↑ | PickScore↑  |\n|-----------------------------|------|-----------|-------------|--------------|-------------|\n| Fixed (50 steps)            | 5.64 | 18.66     | 0.3204      | 0.2132       | 16.8590     |\n| Fixed (70 steps)            | 7.87 | 18.27     | 0.3199      | 0.2137       | 16.8646     |\n| **DATE** (50 steps, 10% update) |      |           |             |              |             |\n| &nbsp;with CLIP score         | 7.82 | **17.90** | **0.3237**  | 0.2364       | 16.8728     |\n| &nbsp;with ImageReward        | 7.82 | 18.61     | 0.3224      | **0.4792**   | 16.8750     |\n| &nbsp;with PickScore          | 8.23 | 18.49     | 0.3225      | 0.2745       | **16.8761** |\n\n(Kirstain et al., 2023) Pick-a-Pic: An Open Dataset of User Preferences for Text-to-Image Generation, NeurIPS 2023."}, {"review_id": "ijgbVHWhFR", "reviewer": "Reviewer_nC2W", "summary": "This paper proposes Diffusion Adaptive Text Embedding (DATE), a method for dynamically updating text embeddings at each diffusion sampling timestep in text-to-image diffusion models. Existing methods typically use a fixed text embedding throughout the reverse diffusion process, which may lead to suboptimal semantic alignment. DATE refines the text embedding per timestep based on intermediate perturbed images by optimizing a differentiable image-text alignment score (e.g., CLIP score or ImageReward). The paper formulates this as a constrained optimization problem and proposes a lightweight gradient-based update rule that does not require additional model training. DATE is model-agnostic, efficient (can be applied at test time), and improves performance on tasks such as multi-concept generation and text-guided image editing.", "questions": "1. How well does DATE generalize to long and complex prompts? Are there cases where it introduces instability or performance degradation? \n2. Does the stronger text-image alignment induced by DATE reduce image diversity or cause overfitting to the reward function? \n3. Can DATE be effectively combined with prompt tuning or soft prompt methods? Would such combinations be complementary or interfere with each other?", "limitations": "Yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths: \n\n* The idea of introducing time-varying text embedding updates during the diffusion sampling process is both innovative and intuitive Furthermore, optimizing the alignment score at intermediate steps of the diffusion process, rather than only at the final output, is a novel training strategy. The method is accompanied by theoretical derivations. \n* The paper presents well-designed experiments and extensive ablation studies.\n\nWeaknesses:\n\n* Although the authors employ strategies such as sparse updates to mitigate runtime, DATE still incurs a noticeable overhead compared to standard diffusion sampling (over 30% inference time). This may limit its applicability in resource-constrained settings. \n\n* The effectiveness of DATE is dependent on the choice of reward function (CLIP, ImageReward). There exists a risk of overfitting to proxy metrics, which may not fully reflect human judgment or semantic correctness.", "was_revised": true, "final_justification": "I have read the authors' rebuttal. Although some of the explanations and experiments provided are not sufficiently thorough, after considering the rebuttal and the feedback from other reviewers, I believe the work still meets the acceptance criteria. I appreciate the additional experiments and analyses, but they do not fully resolve the main weaknesses. Therefore, I will maintain my original score.", "rebuttal": "We thank the reviewer for the valuable feedback. Below are our responses to the concerns.\n***\n> ### **W1: [Computational overhead]**\n\nThe increased GPU memory usage in DATE mainly stems from gradient computations, as noted in the Limitations section. To address this, we apply half-precision (FP16) inference during sampling. As shown below, this reduces runtime and memory with only a slight performance drop. Still, DATE outperforms fixed embedding.\n\n**[Table: Results on COCO with SD v1.5  of Table 1 in the main text, including FP16.]**\n\n||Time|Memory|FID↓|CLIP score↑|ImageReward↑|\n|---|---|---|---|---|---|\n|Fixed (50 steps)|5.64|24.0|18.66|0.3204|0.2132|\n|Fixed (70 steps)|7.87|24.0|18.27|0.3199|0.2137|\n|**DATE** (50 steps, 10% update)|\n|&nbsp;with CLIP score|7.82|61.5|17.90|0.3237|0.2364|\n|&nbsp;with CLIP score (FP16)|4.40|32.9|17.99|0.3229|0.2265|\n|&nbsp;with ImageReward|7.82|61.5|18.61|0.3224|0.4792|\n|&nbsp;with ImageReward (FP16)|4.02|30.6|18.03|0.3222|0.4773|\n\nOne important note is that in our implementation, casting the CLIP model used in the evaluation function to FP16 resulted in degraded performance. Therefore, we kept this component in full precision. Nevertheless, since the diffusion model accounts for most of the computational overhead, applying FP16 to the rest of the pipeline still provides substantial savings.\n\nIt is also worth noting that FP16 is a general memory-efficient strategy that can be applied across different sampling methods. In this context, our results show that DATE remains compatible with such strategies, despite its gradient computations. However, we do not claim that FP16 integration represents a fundamental improvement to DATE itself in terms of memory or speed. Rather, it shows that DATE can effectively incorporate standard efficiency techniques without loss of compatibility.\n***\n> ### **W2: [Dependency on reward function]**\n\nWe acknowledge that DATE depends on the reward function, as discussed in Appendix E. Here, we offer theoretical, empirical, and practical perspectives.\n\n**(Theoretical perspective)**\n\nTheorem 2 shows DATE aligns text embeddings with the reward function while also regularizing with the model's text-conditional likelihood. This means that DATE does not blindly overfit to the reward function but instead applies regularized guidance informed by the underlying diffusion model. This effect can reduce the risk of overfitting to the proxy metrics.\n\n**(Empirical perspective)**\n\nEmpirically, as shown in Table 1 of the main text, DATE improves performance on both the reward function used for optimization and other metrics not used during optimization (e.g., FID). To further validate this generalization, we conduct additional experiments using PickScore (Kirstain et al., 2023) as both an optimization objective and an external evaluation metric. As shown in the table, DATE consistently improves performance across all metrics. This suggests that DATE’s improvements reflect broader enhancements in generation quality. Ongoing research continues to explore more faithful evaluation metrics for text-to-image generation, and DATE can be readily combined with such advances.\n\n**[Table: Results on COCO with SD v1.5 of Table 1 in the main text, including PickScore.]**\n||Time|FID↓|CLIP score↑|ImageReward↑|PickScore↑|\n|---|---|---|---|---|---|\n|Fixed (50 steps)|5.64|18.66|0.3204|0.2132|16.8590|\n|Fixed (70 steps)|7.87|18.27|0.3199|0.2137|16.8646|\n|**DATE** (50 steps, 10% update)|\n|&nbsp;with CLIP score|7.82|**17.90**|**0.3237**|0.2364|16.8728|\n|&nbsp;with ImageReward|7.82|18.61|0.3224| **0.4792**|16.8750|\n|&nbsp;with PickScore|8.23|18.49|0.3225|0.2745|**16.8761**|\n\n(Kirstain et al., 2023) Pick-a-Pic: An Open Dataset of User Preferences for Text-to-Image Generation, NeurIPS 2023.\n\n**(Practical perspective)**\n\nOur method also supports multi-objective optimization, which can further mitigate overfitting to a single metric. In Figure 4 of the main text, we show results using a combined objective of CLIP score and ImageReward. Interestingly, this combination outperforms the use of each metric individually. This result highlights the potential of our method to leverage synergy between multiple objectives for more robust guidance.\n\nIn summary, while reward function dependency is an important consideration, DATE incorporates both theoretical regularization and practical flexibility to address it effectively.\n***\n> ### **Q1: [Long and complex prompts]**\n\nText encoders in diffusion models are often limited by a fixed token length, which can pose challenges for handling long and complex prompts. For example, many diffusion models using CLIP encoders are restricted to 77 tokens, and the tokens beyond this limit are typically truncated and ignored. Since DATE provides guidance through an external function and does not rely solely on the internal text encoder. This allows DATE to influence the generation process beyond the model’s fixed token-length limitation, making it a promising solution for handling long and complex prompts.\n\nTo investigate this, we perform experiments on the LongAlign (Liu et al., 2025), which contains prompts of substantial length and complexity. We sample 300 prompts and use ImageReward (20% updates) as the evaluation function. For ImageReward, each long prompt is split into individual sentences, and at each update step, one sentence is randomly selected to compute the score and guide the embedding update. We use Denscore (Liu et al., 2025) for the metric.\n\n**[Table: Results for long prompts on LongAlign dataset (SD v1.5)]**\n|Method|Denscore↑|\n|---|---|\n|Fixed|0.2034|\n|**DATE**|**0.2042**|\n\n**[Table: Average Denscore by sentence position]**\n||S1|S2|S3|S4|S5|S6+|\n|---|---|---|---|---|---|---|\n|Fixed|**0.2597**|**0.2478**|**0.2234**|0.1961|0.1678|0.1764|\n|**DATE**|0.2561|0.2465|0.2230|**0.1991**|**0.1729**|**0.1799**|\n\nThese results show that DATE improves Denscore compared to the baseline. Notably, the improvement is more significant for sentences appearing later in the prompt, suggesting that DATE can inject additional semantic guidance in regions where the base model typically underperforms due to token truncation. These results suggest that DATE can serve as an effective mechanism for injecting auxiliary information into diffusion models, especially in scenarios where prompt length exceeds the native capacity of the text encoder.\n\n(Liu et al., 2025) Improving Long-Text Alignment for Text-to-Image Diffusion Models, ICLR 2025.\n***\n> ### **Q2: [Diversity]**\n\nAs discussed in our response to W2, our theoretical analysis based on Theorem 2 suggests that DATE is regularized by the underlying diffusion model, which helps mitigate overfitting to the reward function. Additionally, since DATE does not modify model weights and instead performs instance-specific updates during inference, we believe its impact on sample diversity is limited.\n\nTo empirically validate this, we measure diversity using Recall, the metric for evaluating the diversity of generated images in generative models.\n\n**[Table: Results on the COCO from Table 1 in the main paper with SD v1.5 including Recall.]**\n|Method|FID↓|CLIP score↑|ImageReward↑|Recall↑|\n|---|---|---|---|---|\n|Fixed (50 steps)|18.66|0.3204|0.2132|0.8132|\n|Fixed (70 steps)|18.27|0.3199|0.2137|0.8154|\n|EBCA|25.85|0.2877|-0.3128|0.6960|\n|Universal Guidance|18.56|0.3216|0.2221|0.8048|\n|**DATE** (50 steps)|\n|&nbsp;10% update with CLIP score|17.90|0.3237|0.2364|0.8178|\n|&nbsp;all updates with CLIP score|**17.22**|**0.3292**|0.2277|0.8208|\n|&nbsp;10% update with ImageReward|18.61|0.3224|0.4792|0.8225|\n|&nbsp;all updates with ImageReward|18.17|0.3224|**1.2972**|**0.8558**|\n\nAs shown in the results, DATE improves both text-image alignment and sample diversity. We speculate that giving the model the opportunity to dynamically update the text embedding across timesteps, rather than relying on a fixed embedding, may have encouraged greater exploration during sampling. However, the underlying mechanism behind this effect warrants further investigation.\n***\n> ### **Q3: [Combined with prompt tuning]**\n\nOur method can be combined with prompt tuning and soft prompt methods. Several learning-based approaches (Hao et al., 2023, Mo et al., 2024) have explored prompt optimization using reward models and reinforcement learning. While such approaches focus on optimizing prompt tokens or prompt-generating models, DATE operates at inference time by directly adjusting the text embeddings without any additional training. Therefore, they can be used together: refined prompts from these methods can serve as the initial input, and DATE can further refine the embedding during sampling to better align with the evaluation function.\n\nWe conduct additional experiments by combining DATE with Promptist (Hao et al., 2023), a representative prompt tuning method. We use ImageReward as the evaluation function for DATE.\n\n**[Table: Results with the prompt tuning, Promptist (Hao et al., 2023). FID values may differ in scale due to the smaller sample size (1k).]**\n|Method| FID-1K↓ |CLIP score↑ | ImageReward↑ | Aesthetic score↑ |\n|---|---|---|---|---|\n|Fixed|48.81|0.3206|0.1801|5.40|\n|**DATE**|48.09|0.3233|0.4680|5.43|\n|Promptist|57.75|0.2785|-0.0380|6.23|\n|Promptist+**DATE**|56.23|0.2827|0.2471|6.20|\n\nPromptist focuses on improving aesthetic quality (represented by Aesthetic score), often at the cost of other metrics like CLIP score, FID, and ImageReward. In contrast, DATE improves all metrics, including Aesthetic score, which is not used in its objective, indicating good generalization. When combined with Promptist, DATE helps recover image-text alignment and overall quality while maintaining high aesthetic quality. This suggests DATE can complement prompt tuning methods.\n\n(Hao et al., 2023) Optimizing Prompts for Text-to-Image Generation, NeurIPS 2023.   \n(Mo et al., 2024) Dynamic Prompt Optimizing for Text-to-Image Generation, CVPR 2024."}, {"review_id": "8nBuM0N2LT", "reviewer": "Reviewer_dcFc", "summary": "This paper proposes Diffusion Adaptive Text Embedding (DATE), a novel method for improving text-to-image diffusion models by dynamically updating text embeddings during the sampling process.  Experimental results across several tasks, including t2i generation, multi-concept generation, and text-guided image editing.", "questions": "See the Weakness part.", "limitations": "yes.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Thanks to the authors for their efforts in this work. \n\n**Strengths**    \nThis manuscript is well-presented, with comprehensive experimental results and solid theoretical analysis. The idea that updates the textual embedding with an optimization strategy is novel. The experiments are conducted across different network architectures (e.g., Stable Diffusion v1.5 and PixArt-α), and include detailed ablation studies. In addition, the authors provide a clear theoretical formulation of their method, including optimization objectives and update strategies.\n\n**Weaknesses**    \n1. Despite the effective optimization formulation, the proposed framework relies on the evaluation function $h$, such as CLIP score,  which is known to exhibit biases [1]. Such reliance may limit the robustness or generalizability of the method under varying evaluation metrics. while it also has a trade-off of the Clip score and ImageReward.\n2. Some arguments are overclaims; e.g., the related work section (lines 70–72) lacks discussion of relevant approaches such as SD3 [2], which utilizes multiple text embeddings.\n3. There is also some related work on text embedding tuning [3], text-to-image generation based on the evaluation function [4], which needs to be discussed.\n4. A key concern is that, although the experimental results demonstrate that the proposed embedding tuning improves performance somehow, it alters the embedding space of the original input text. In other words, it may deviate from the user's original intent in generation.\n\n$\\small{\\text{ [1] Ruiz, Nataniel, et al. \"Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation.\" Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2023.}}$\n$\\small{\\text{ [2] Esser, Patrick, et al. \"Scaling rectified flow transformers for high-resolution image synthesis.\" Forty-first international conference on machine learning. 2024.}}$   \n$\\small{\\text{ [3] Gal, Rinon, et al. \"An image is worth one word: Personalizing text-to-image generation using textual inversion.\" arXiv preprint arXiv:2208.01618 (2022).}}$     \n$\\small{\\text{ [4] Fan, Ying, et al. \"Dpok: Reinforcement learning for fine-tuning text-to-image diffusion models.\" Advances in Neural Information Processing Systems 36 (2023): 79858-79885.}}$", "was_revised": true, "final_justification": "The author addressed my concerns on reliance on evaluation function, multiple text embeddings, and \\etc. They also provided a detailed analysis of the reward model, which is very important in fine-tuning Text-to-Image Diffusion Models. Therefore, I increase my rating from borderline reject to borderline accept.", "rebuttal": "We thank the reviewer for thoughtful comments. We address the questions in the following response.\n***\n> ### **W1: [Reliance on evaluation function]**\n\nWe acknowledge that DATE depends on the reward function, as discussed in Appendix E. Here, we offer theoretical, empirical, and practical perspectives.\n\n**(Theoretical perspective)**\n\nTheorem 2 shows DATE aligns text embeddings with the reward function while also regularizing with the model's text-conditional likelihood. This means that DATE does not blindly overfit to the reward function but instead applies regularized guidance informed by the underlying diffusion model. This effect can reduce the risk of overfitting to the proxy metrics.\n\n**(Empirical perspective)**\n\nEmpirically, as shown in Table 1 of the main text, DATE improves performance on both the reward function used for optimization and other metrics not used during optimization (e.g., FID). To further validate this generalization, we conduct additional experiments using PickScore (Kirstain et al., 2023) as both an optimization objective and an external evaluation metric. As shown in the table, DATE consistently improves performance across all metrics. This suggests that DATE’s improvements reflect broader enhancements in generation quality. Ongoing research continues to explore more faithful evaluation metrics for text-to-image generation, and DATE can be readily combined with such advances.\n\n**[Table: Results on COCO with SD v1.5 of Table 1 in the main text, including PickScore.]**\n||Time|FID↓|CLIP score↑|ImageReward↑|PickScore↑|\n|---|---|---|---|---|---|\n|Fixed (50 steps)|5.64|18.66|0.3204|0.2132|16.8590|\n|Fixed (70 steps)|7.87|18.27|0.3199|0.2137|16.8646|\n|**DATE** (50 steps, 10% update)|\n|&nbsp;with CLIP score|7.82|**17.90**|**0.3237**|0.2364|16.8728|\n|&nbsp;with ImageReward|7.82|18.61|0.3224| **0.4792**|16.8750|\n|&nbsp;with PickScore|8.23|18.49|0.3225|0.2745|**16.8761**|\n\n(Kirstain et al., 2023) Pick-a-Pic: An Open Dataset of User Preferences for Text-to-Image Generation, NeurIPS 2023.\n\n**(Practical perspective)**\n\nOur method also supports multi-objective optimization, which can further mitigate overfitting to a single metric. In Figure 4 of the main text, we show results using a combined objective of CLIP score and ImageReward. Interestingly, this combination outperforms the use of each metric individually. This result highlights the potential of our method to leverage synergy between multiple objectives for more robust guidance.\n\nIn summary, while reward function dependency is an important consideration, DATE incorporates both theoretical regularization and practical flexibility to address it effectively.\n***\n> ### **W2: [Multiple text embeddings]**\n\nThank you for the thoughtful feedback. As we present dynamically adapted text embedding to the diffusion process, our method (DATE) is distinct from a static set of multiple text embeddings, such as SD3.\n\nRegarding the related work section, we agree that methods like SD3 are relevant to the goal of improving text embeddings, and we will include a discussion of such methods in the revised version to provide a more comprehensive context.\nThat said, our intention in lines 70-72 was to highlight a specific distinction: while methods like SD3 improve text embeddings (e.g., by using multiple text encoders), they generally do not adapt these embeddings dynamically across diffusion timesteps. In contrast, DATE is motivated by this limitation and proposes an inference-time mechanism for updating the text embedding dynamically over timesteps.\nImportantly, this difference makes DATE complementary to approaches like SD3. We demonstrate that DATE can be applied on top of SD3, SD-XL, and FLUX, all of which use multiple text embedding. For each model, we follow the default configurations provided in the HuggingFace diffusers library and use ImageReward as the evaluation function.\n\n**[Table: Results on COCO with recent stronger base models]**\n|Backbone|Method| FID↓|CLIP score↑|ImageReward↑|\n|---|---|---|---|---|\n|SD3| Fixed|26.00|0.3337|1.0018|\n||**DATE**|26.00|**0.3340**|**1.0457**|\n|FLUX|Fixed|29.59|0.3257|0.9634|\n||**DATE**|**29.41**|**0.3283**|**0.9768**|\n|SDXL|Fixed|18.27|0.3368| 0.7284|\n||**DATE**|**18.03**|**0.3382**|**0.9096**|\n\nThe results above follow the evaluation setup in Table 1 of the main text. Across all modern architectures, DATE consistently improves performance on all metrics compared to the fixed text embedding baseline. These results demonstrates that DATE can be effectively integrated with models that utilize multiple text embeddings, providing a scalable and complementary improvement.\n***\n> ### **W3: [Related works]**\n\nWe agree that a more comprehensive comparison including methods for prompt optimization, text embedding tuning, and reward-based fine-tuning would strengthen the paper. In the revised version, we include a detailed discussion and additional experiments covering the following areas, as below.\n\nAs discussed in Section 2.2 of the main text, methods for refining text-to-image generation can generally target one of the following: model parameters, perturbed data, or text embeddings. The work in [4], as well as (Black et al., 2024) cited in our paper, falls into the first category. These approaches fine-tune the model parameters using a reward function. However, they require significant training time and computational resources.\n\nPrompt-level optimization targets the input text. Several learning-based approaches (Hao et al., 2023, Mo et al., 2024) have explored prompt optimization using reward models and reinforcement learning. These methods train a language model to generate prompts optimized for a diffusion model based on reward signals. However, such methods typically require significant training time and computational resources. Additionally, if the diffusion backbone or reward function changes, the model could be retrained. In contrast, DATE is training-free and operates at inference time. Moreover, these learning-based previous works could be used in conjunction with DATE during test-time for complementary benefits.\n\nThere is also related work focusing on tuning text embeddings directly. For example, [3] learns a special token embedding using a small number of personalized images, which is then appended to prompts for personalized image generation. Similarly, (Um et al., 2025) introduce a method for generating minority instances by learning a special token embedding, further extending it to be time-dependent during sampling. While [3] and (Um et al., 2025) focus on specific tasks, DATE proposes a general-purpose framework for improving text-to-image alignment. Additionally, a key distinction lies in the granularity of updates: [3] and (Um et al., 2025) optimize a special token embedding, whereas DATE updates the entire text embedding vector.\n\nWe conduct additional experiments by combining DATE with Promptist (Hao et al., 2023), a representative prompt tuning method. We use ImageReward as the evaluation function for DATE.\n\n**[Table: Results with the prompt tuning, Promptist (Hao et al., 2023). FID values may differ in scale due to the smaller sample size (1k).]**\n|Method| FID-1K↓ |CLIP score↑ | ImageReward↑ | Aesthetic score↑ |\n|---|---|---|---|---|\n|Fixed|48.81|0.3206|0.1801|5.40|\n|**DATE**|48.09|0.3233|0.4680|5.43|\n|Promptist|57.75|0.2785|-0.0380|6.23|\n|Promptist+**DATE**|56.23|0.2827|0.2471|6.20|\n\nPromptist focuses on improving aesthetic quality (Aesthetic score), often at the cost of other metrics. In contrast, DATE improves all metrics, including Aesthetic score, which is not used in its objective, indicating good generalization. When combined with Promptist, DATE helps recover image-text alignment and overall quality while maintaining high aesthetic quality. This suggests DATE can complement prompt tuning methods.\n\n(Black et al., 2024) Training Diffusion Models with Reinforcement Learning, ICLR 2024.   \n(Hao et al., 2023) Optimizing Prompts for Text-to-Image Generation, NeurIPS 2023.   \n(Mo et al., 2024) Dynamic Prompt Optimizing for Text-to-Image Generation, CVPR 2024.   \n(Um et al., 2025) Minority-Focused Text-to-Image Generation via Prompt Optimization, CVPR 2025.\n***\n> ### **W4: [Deviation from the original intent]**\n\nWe acknowledge the potential risk that adapting the text embedding might deviate from the user’s original intent. However, we believe DATE can actually help better reflect that intent through a feedback-driven refinement process.\nOur method updates the text embedding based on **the expected image output $\\bar{x}_0$** generated from the current embedding, guided by an external evaluation function. This allows the embedding to shift in a direction that better aligns with the user’s intent as captured by the evaluation signal. In contrast to fixed text embeddings, which rely solely on the static encoding of the input text, DATE introduces a feedback loop between the intermediate generated image and the text embedding, making it more adaptive to the desired semantics.\n\nMoreover, Theorem 2 provides a theoretical justification showing that DATE updates the text embedding by aligning with the evaluation function while also accounting for the original diffusion model’s text-conditional likelihood. This regularization effect helps prevent overfitting to the evaluation function or losing alignment with the original prompt semantics.\n\nThat said, we agree that since the base diffusion model was not trained with dynamically updated text embeddings, there may be a training-inference gap. To address this, we explicitly introduce a constraint on the embedding update: $||c-c_{org}||_2 \\leq \\rho$ in Eq. (7). This constraint allows us to balance preserving the original text semantics while adapting to the evaluation feedback. As shown in the sensitivity analysis in Figure 6 of the main text, performance improves when $\\rho$ is set appropriately, supporting the effectiveness of this trade-off."}, {"review_id": "yFIoa2NbT3", "reviewer": "Reviewer_TJGJ", "summary": "The paper introduces Diffusion Adaptive Text Embedding (DATE), a method that improves text-to-image diffusion models by dynamically updating text embeddings at each diffusion timestep. Unlike traditional approaches with fixed embeddings, DATE refines embeddings based on intermediate image data during sampling, enhancing text-image alignment without additional model training. Theoretical and empirical results show that DATE maintains generative quality while improving performance in tasks like multi-concept generation and text-guided editing.", "questions": "Can the method be transferred to other stronger base models like SD-XL, FLUX or other DiT-based image generation models? If so, I would like to see how actually it improves.", "limitations": "Please see the weaknesses.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Pros:\n1. The paper presents an interesting formulation of the problem and offers a unique perspective.\n2. The overall writing is clear and easy to follow.\n3. The results seem promising in some cases.\n\nCons:\n1. The results presented in this paper are limited to some easy cases. As the authors mainly focus on optimizing the text embedding, they should put more attention on complex, long texts to show the improvement they claim to achieve.\n2. Additionally, there are actually many powerful image generation models like GPT-4o, Gemini, FLUX that can achieve good image generation quality while accurately following the text prompts. The authors should compare with them (although some are not open-sourced, they can be inferenced for comparison).\n3. They authors use CLIP Score or ImageReward as the optimization function but also use them as the evalution may lead the optimization into sub-optimal and overfitiing. The authors should explain its validity.", "was_revised": true, "final_justification": "My concerns are addressed, hence I maintain my rating.", "rebuttal": "We are grateful for the reviewers' thoughtful evaluations and detailed comments. Our responses to the raised points are presented below.\n***\n> ### **W1: [Long and complex prompts]**\n\nText encoders in diffusion models are often limited by a fixed token length, which can pose challenges for handling long and complex prompts. For example, many diffusion models using CLIP encoders are restricted to 77 tokens, and the tokens beyond this limit are typically truncated and ignored. Since DATE provides guidance through an external function and does not rely solely on the internal text encoder. This allows DATE to influence the generation process beyond the model’s fixed token-length limitation, making it a promising solution for handling long and complex prompts.\n\nTo investigate this, we perform experiments on the LongAlign (Liu et al., 2025), which contains prompts of substantial length and complexity. We sample 300 prompts and use ImageReward (20% updates) as the evaluation function. For ImageReward, each long prompt is split into individual sentences, and at each update step, one sentence is randomly selected to compute the score and guide the embedding update. We use Denscore (Liu et al., 2025) for the metric.\n\n**[Table: Results for long prompts on LongAlign dataset (SD v1.5)]**\n|Method|Denscore↑|\n|---|---|\n|Fixed|0.2034|\n|**DATE**|**0.2042**|\n\n**[Table: Average Denscore by sentence position]**\n||S1|S2|S3|S4|S5|S6+|\n|---|---|---|---|---|---|---|\n|Fixed|**0.2597**|**0.2478**|**0.2234**|0.1961|0.1678|0.1764|\n|**DATE**|0.2561|0.2465|0.2230|**0.1991**|**0.1729**|**0.1799**|\n\nThese results show that DATE improves Denscore compared to the baseline. Notably, the improvement is more significant for sentences appearing later in the prompt, suggesting that DATE can inject additional semantic guidance in regions where the base model typically underperforms due to token truncation. These results suggest that DATE can serve as an effective mechanism for injecting auxiliary information into diffusion models, especially in scenarios where prompt length exceeds the native capacity of the text encoder.\n\n(Liu et al., 2025) Improving Long-Text Alignment for Text-to-Image Diffusion Models, ICLR 2025.\n***\n> ### **W2 & Q: [More powerful generation models]**\n\nThank you for the suggestion. First, we would like to emphasize that DATE is an inference-time method designed to improve existing image generation models by guiding the sampling process. It can be applied on top of powerful base models to further improve their performance without requiring retraining or access to large-scale paired data. That said, we do not claim that our approach outperforms methods that rely on extensive training with massive datasets and computing resources. Rather, our goal is to provide an adaptive improvement that can complement such powerful image generation models, even when full retraining is impractical or infeasible.\n\nWhile DATE does not modify model weights, it does require access to model weights due to computation of gradients, which makes it challenging to apply to closed-source models such as GPT-4o or Gemini.\nOn the other hand, for open-source models, DATE is readily applicable. As shown in Table 1 of the main text, DATE consistently improves performance across all metrics when applied to PixArt-alpha, which is a DiT-based image generation model.\nTo further evaluate DATE’s applicability to a broader range of strong base models, we additionally apply DATE to FLUX (guided-distilled version), SD-XL and SD3.  For each model, we follow the default configurations provided in the HuggingFace diffusers library and use ImageReward as the evaluation function.\n\n**[Table: Results on COCO with recent stronger base models]**\n|Backbone|Method| FID↓|CLIP score↑|ImageReward↑|\n|---|---|---|---|---|\n|SD3| Fixed|26.00|0.3337|1.0018|\n||**DATE**|26.00|**0.3340**|**1.0457**|\n|FLUX|Fixed|29.59|0.3257|0.9634|\n||**DATE**|**29.41**|**0.3283**|**0.9768**|\n|SDXL|Fixed|18.27|0.3368| 0.7284|\n||**DATE**|**18.03**|**0.3382**|**0.9096**|\n\nThe results above follow the evaluation setup in Table 1 of the main text. Across all modern architectures, DATE consistently improves performance on all metrics compared to the fixed text embedding baseline. These results suggest that DATE can be transferred to a range of stronger base models and consistently enhance their generation quality.\n\n***\n> ### **W3: [Reliance on evaluation function]**\n\nWe acknowledge that DATE depends on the reward function, as discussed in Appendix E. Here, we offer theoretical, empirical, and practical perspectives.\n\n**(Theoretical perspective)**\n\nTheorem 2 shows DATE aligns text embeddings with the reward function while also regularizing with the model's text-conditional likelihood. This means that DATE does not blindly overfit to the reward function but instead applies regularized guidance informed by the underlying diffusion model. This effect can reduce the risk of overfitting to the proxy metrics.\n\n**(Empirical perspective)**\n\nEmpirically, as shown in Table 1 of the main text, DATE improves performance on both the reward function used for optimization and other metrics not used during optimization (e.g., FID). To further validate this generalization, we conduct additional experiments using PickScore (Kirstain et al., 2023) as both an optimization objective and an external evaluation metric. As shown in the table, DATE consistently improves performance across all metrics. This suggests that DATE’s improvements reflect broader enhancements in generation quality. Ongoing research continues to explore more faithful evaluation metrics for text-to-image generation, and DATE can be readily combined with such advances.\n\n**[Table: Results on COCO with SD v1.5 of Table 1 in the main text, including PickScore.]**\n||Time|FID↓|CLIP score↑|ImageReward↑|PickScore↑|\n|---|---|---|---|---|---|\n|Fixed (50 steps)|5.64|18.66|0.3204|0.2132|16.8590|\n|Fixed (70 steps)|7.87|18.27|0.3199|0.2137|16.8646|\n|**DATE** (50 steps, 10% update)|\n|&nbsp;with CLIP score|7.82|**17.90**|**0.3237**|0.2364|16.8728|\n|&nbsp;with ImageReward|7.82|18.61|0.3224| **0.4792**|16.8750|\n|&nbsp;with PickScore|8.23|18.49|0.3225|0.2745|**16.8761**|\n\n(Kirstain et al., 2023) Pick-a-Pic: An Open Dataset of User Preferences for Text-to-Image Generation, NeurIPS 2023.\n\n**(Practical perspective)**\n\nOur method also supports multi-objective optimization, which can further mitigate overfitting to a single metric. In Figure 4 of the main text, we show results using a combined objective of CLIP score and ImageReward. Interestingly, this combination outperforms the use of each metric individually. This result highlights the potential of our method to leverage synergy between multiple objectives for more robust guidance.\n\nIn summary, while reward function dependency is an important consideration, DATE incorporates both theoretical regularization and practical flexibility to address it effectively."}]

Author rebuttal:

Meta-review: The authors present Diffusion Adaptive Text Embedding (DATE), which learns text embeddings in text-to-image diffusion, unlike previous methods that kept embeddings frozen. The experimental results encompass several tasks, including text-to-image generation, multi-concept generation, and text-guided image editing.

**Weaknesses:** Reviewers raised concerns about the ablations and evaluations. For instance, performance on complex prompts showed only marginal improvements in the authors' long-text benchmark. Comparisons to prompt-tuning methods were limited, as the chosen baseline (Promptist) was weak.  Additional concerns included the extra computational cost.

**Recommendation:** Accept (poster). Despite these drawbacks, all reviewers moved their scores to accept, agreeing on the importance of the technique to the community. I concur that studies probing optimization in pre-trained representations are valuable. I see no reason to intervene in the reviewers' decision.

**Discussion summary:** **1BqT**: Asked for more baselines, novelty, visuals, and efficiency. Authors added comparisons with Promptist & CADS, a FP16 lightweight variant, and PickScore generalization. Concerns resolved. **nC2W**: Flagged overhead, reward-function dependence, and diversity risks. Authors demonstrated FP16 gains, performed theoretical/empirical checks, presented long-text & GenAI-Bench results, Explored Recall diversity, and discussed prompt-tuning combinations. Some concerns remain. Score unchanged.   **dcFc**: Concerned about reward bias, missing related work, and intent deviation. The authors analyzed multiple reward models, added correlation studies, discussed SD3/multi-embedding, and showed theoretical safeguards. Rebuttal resolved issues. Score raised.   **TJGJ**: Wanted strong baselines (GPT-4o, FLUX), complex prompts, and metric justification. Authors applied DATE to SDXL/FLUX/SD3, ran long-text + complex benchmarks, and showed metric consistency. Concerns addressed; score maintained.
Decision: Accept (poster)