Aligning Text to Image in Diffusion Models is Easier Than You Think
Authors: Jaa-Yeon Lee
ByungHee Cha
Jeongsol Kim
Jong Chul Ye
Venue: NeurIPS 2025 poster ()
Keywords: diffusion model
flow model
contrastive learning
representation alignment
soft token
Primary area: deep_learning

Abstract: While recent advancements in generative modeling have significantly improved text-image alignment, some residual misalignment between text and image representations still remains. Some approaches address this issue by fine-tuning models in terms of preference optimization,  etc., which require tailored datasets. Orthogonal to these methods, we revisit the challenge from the perspective of representation alignment—an approach that has gained popularity with the success of REPresentation Alignment (REPA). We first argue that conventional text-to-image (T2I) diffusion models, typically trained on paired image and text data (i.e., positive pairs) by minimizing score matching or flow matching losses, is suboptimal from the standpoint of representation alignment. Instead, a better alignment can be achieved through contrastive learning that leverages existing dataset as both positive and negative pairs. To enable efficient alignment with pretrained models, we propose SoftREPA—a lightweight contrastive fine-tuning strategy that leverages soft text tokens for representation alignment. This approach improves alignment with minimal computational overhead by adding fewer than 1M trainable parameters to the pretrained model. Our theoretical analysis demonstrates that our method explicitly increases the mutual information between text and image representations, leading to enhanced semantic consistency. Experimental results across text-to-image generation and text-guided image editing tasks validate the effectiveness of our approach in improving the semantic consistency of T2I generative models.

Aligning Text to Image in Diffusion Models is Easier Than You Think
Yeon Lee"!
Byunghee Cha" Jeongsol Kim² Jong Chul Ye'
Kim Jaechul Graduate School of AI, KAIST
Department of Bio and Brain Engineering, KAIST
{jaayeon, paulcha1025, jeongsol, jong-yes yeeeyyykkkkaa.... "Equal contribution
sign that reads "Othello""
"stone building" - "wooden lighthouse"
Ofthe
O
Othello
C
"A fire hydrant has some writing on its side"
"yellow building" + "red building, open door"
"knolling of a drowing tools"
"car" "motorcycle"
SD3
SD3 (Ours)
Source
Edited
Edited (Ours)
Figure 1: Representative results for image generation and image editing. SoftREPA provides much improved text-to-image alignment by introducing negligible size of learnable soft tokens.
Abstract
While recent advancements in generative modeling have significantly improved text-image alignment, some residual misalignment between text and image rep- resentations still remains. Some approaches address this issue by fine-tuning models in terms of preference optimization, etc., which require tailored datasets. Orthogonal to these methods, we revisit the challenge from the perspective representation alignment-an approach that has gained popularity with the cess of REPresentation Alignment (REPA) [6). We first argue that conventional text-to-image (T2I) diffusion models, typically trained on paired image and text data (i.e., positive pairs) by minimizing score matching or flow matching losses, is suboptimal from the standpoint of representation alignment. Instead, better
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

alignment can be achieved through contrastive learning that leverages existing dataset both positive and negative pairs. To enable efficient alignment with pretrained models, we propose SoftREPA lightweight contrastive fine-tuning strategy that leverages soft text tokens for representation alignment. This approach improves alignment with minimal computational overhead by adding fewer than IM trainable parameters to the pretrained model. Our theoretical analysis demon- strates that our method explicitly increases the mutual information between text and image representations, leading to enhanced semantic consistency. Experimental results across ext-to-image generation and text-guided image editing tasks validate the effectiveness of our approach in improving the semantic consistency of T2I generative models. Project Page:
gi thub.
Introduction
Achieving effective alignment between modalities is essential in multimodal generative modeling. Specifically, latent diffusion models [33 enable various conditioning mechanisms, such as text, during the generation process for the semantic alignment between the two modalities. For UNet- based denoisers 33 29 31, image representations are updated to align with fixed text representations encoded by pre- praited CLIP models through cross-attention layers. In contrast, transformer- based denoisers 30 S jointly update both text and image representations by concatenating them and processing them through self-attention layers. However, there is still room for further improvement, and to mitigate the remaining misalignment between text and image representations, a more effective representation learning approach is necessary.
Recent advancements in representation alignment for Diffusion Transformers (DiT) [0 have notably enhanced their ability to learn semantically meaningful internal representations 46 I.I Notably,
REPA 46 showed that aligning the internal representations of DiT with an external pre-trained visual encoder during training significantly improves both discriminative and generative performance. To further enhance text-image alignment in eet-to--maae (T2I) generative models, we leverage these ideas for aligning DIT's internal representations.
Most vision-language generative foundation models have focused on improving multimodal alignment through architectural modifications. These include MLP-based projection layers (e.g.. LLa VA-stlee visual instruction tuning) 225 26]. cross wwtsss-atentiion-bet based fusion mechanisms (e.g., Flamingo, Stable Diffusion 1.5) 12 ]. and early fusion models that integrate text and vision features within shared representation space (c.g.. Chameleon, Stable Diffusion 3. FLUX) 37 In fact, many of these approaches have been largely adopted in T21 generative models. However, we explore an efficient yet effective way of further enhancing the representation alignment for given pre-trained models.
Specifically, we focus on the contrastive learning framework, a widely adopted strategy for multi- modal alignment in vasion-language representation learning, as seen in models such CLIP (32). ALBEF (21), BLIP (22). and ALIGN [12]. One of the key contributions of this work is the introduc- tion of soft text tokens, which are optimized via contrastive image-text training. These soft tokens allow the model to dynamically adapt its text representations, improving alignment with generated images without requiring full model fine-tuning. This simple yet effective approach significantly enhances text-to-image alignment in both text-to-image generation and text-guided image editing tasks. Notably, the method highly flexible and can be seamlessly integrated with any pretrained text-to-image (T2I) generative model. From theoretical standpoint, we show that this approach explicitly increases the mutual information between text and image representations, resulting in improved semantic consistency across modalities. We refer to our method as SoftREPA, short for Soft REPresentation Alignment. Our contributions can be summarized as follows.
We propose SoftREPA, a novel text-image representation alignment method that leverages a lightweight fine-tuning strategy with soft text tokens. This approach improves text-image alignment while adding fewer than IM additional parameters, ensuring efficiency with minimal computational overhead.
SoftREPA simple yet flexible so that it can be used with any pretrained T21 generative models to improve performance of image generation, editing, etc.

---

We show that our method explicitly increases mutual information between image and text, leading better semantic consistency in representations.
Preliminaries
Suppose that we have access to samples from target distribution Xo q and source distribution X1 ~ p. The goal of flow model is to generate Xo starting from X₁. Specifically, we define velocity field u2 (ax) of flow (x)) [0. 1] R Rd that satisfies ()x) X, and 4/1 (Xx) = X1. Here, the the is uniquely characterized by a flow ODE:
Flow Models
d (a:) = ve (x))
where the flow velocity vs is fitted the parameterized neural network vr,0 via flow matching:
CFM = [0t_
- (x,)(==)))
(2)
However, this is computationally expensive to solve due to the integration with respect to Xo. so the authors in [24] proposed conditional flow matching that has the same gradient with the original objective function:
CCFM (o, 1) (1 ao)) ve,0 (1,0(3)))) (3x:)]1
where ( defines conditional flow (2 satisfying (1) (x1)))) Zt- particular, the linear conditional flow defines the flow as æ, = ii (æ) (x)) = -1)20 + tæ Then, the conditional
velocity field is given by vn (a (zo) vh (2x = 21 244 Thus, the conditional
flow matching loss defined as
[0 ((x1 -oo) U2 0xx)(x)))1
Considering marginal velocity field,
Ve (x(e)) = f (x (20 )dzo = [V (ce, (20)) [xe]
(5)
18[x1 - xo [x]] 15x [xc]] [x1]
the generation process via flow ODE is
dx; velxx)dtt = (E[x21 [xx]
E ( dt)dt
(6)
Accordingly,
Zt
(1
t)z0
22 - 1E(20| [xe] dt
dx:
E
t
when Φ ~p: (O, Ia). In other which is the probability- fllo ODE (PF-ODE) of DDIM B3 15
words, flow models and score-based models can be used interchangeably.
Contrastive Representation Learning Contrastive learning aims to maximize the similarity between semantically related text-image pairs while pushing apart unrelated pairs in shared rep-
resentation space 6 32). Formally, given a batch of N image-text pairs (I; T.) contrastive learning optimizes contrastive loss function, typically a variant of the InfoNCE loss:
1 N
exp(sim L, TD/T)
LCUP
log ビj=1 (I,
where sim(I, I, T,) is a similarity metric such as cosine similarity or inner product between image and text embeddings, and T denotes temperature parameter that controls the distribution sharpness. This contrastive loss encourages image and text representations to form a joint multimodal representation space, where aligned pairs are close together, and unaligned pairs are separated. Several fundamental models have successfully applied contrastive learning for multi-modal representation learning 32
12 21 22 and self-supervised learning [6].
3

---

Po (ol""dog")
Pe(e) dog' .1
photo of dog
"oat")
"oar")
Text Encodar
s(t)
Contrastive T21 Alignment
"cow")
"cow")
Block
00:0
Noise space
Block N
fo ,y)
5a photo
uoee
(a) Conceptual Figure of Soft Tokens
(b) SoftREPA
Figure 2: Network architecture and algorithmic concept of SoftREPA. (a) Learnable soft tokens of each layer are prepended to the text features across the upper layers. (b) The soft tokens are optimized contrastively match the score with positively conditioned predicted noise while repelling the score from negatively conditioned predicted noise. This process implicitly sharpens the joint probability distribution of images and text by reducing the log probability of negatively paired conditions.
Despite the success of contrastive learning in representation learning, its application in generative models remains underexplored due to key challenges. First, there a fundamental mismatch between discriminative and generative features; conventional contrastive learning optimizes representation space for discriminative tasks, whereas generative models focus on realistic sample synthesis. Second, text-image alignment in diffusion- based generative models is often implicit, relying on denoising objectives rather than explicit representation learning.
To bridge this gap, we propose contrastive learning framework that aligns both representation learning and generative objectives, ensuring effective text-image representation alignment while preserving generative quality.
3 SoftREPA
3.1
Contrastive T2I Alignment Loss
Let ((xx(0, ( ( denotes the matched image and text pairs as training data set. Then, contrastive text-to-image alignment T2I model can be generally formulated as follows:
1 -=- 72 Σ log E, exp(1(x(()) y())
(xp(I(x y ())
where I(-5-) represents the similarity measure. It is important to note that unlike the standard T2I model training that only consider positive pairs, i.e. (x((), ())), our SoftREPA employs the
contrastive T2I alignment that additionally considers negative image and text pairs (x((), y())),, i This significantly improves the alignment performance.
Furthermore, define (()())
()) as the logit from the variation of the denoising score matching loss for the case of T2I diffusion models (1I).
1(x: y) = e B. Similarly, in conditional flow matching, the logit ((x y) can be formalized as:
(10)
1(, v) (mm (c (+(()))
(11)

---

Here, T(t) represents time scheduling parameter. One might consider using the denoising score matching loss, -E,,cllle,(21.1 - as a similarity measure instead of relying on its logit value. However, our experiments revealed that this formulation can introduce instability during training due the unbounded nature of the similarity measure. To address this, SoftREPA introduce an exponential function to constrain the logit values, effectively stabilizing the training process.
Intuitively, as illustrated in Fig. b), training a diffusion model with this loss encourage the predicted noise to align with the true noise when conditioned on a matching text description. Conversely, when conditioned on a mismatched text, the predicted noise deviates further from the true noise. This sharpens the conditional probability distribution of the trained model and increases the distinction between different text conditions, which can enhance image-text alignment in generated images.
3.2 The Soft Tokens
To distill the contrastive score matching objective above, we introduce a learnable soft token s, which is the only trainable parameter while pretrained model remains frozen. In particular, as illustrated in fig. 2 we introduce a set of learnable soft tokens that vary between layers and are indexed by time t. The soft token layer l is defined using an embedding function with the number of soft tokens as m:
s (k.t) = Embedding(k: t) IR mxd
(12)
At the layer k; the text representation is updated by concatenating the time-indexed soft token with text hidden representation from the previous layer: ( = [s((:(); (=(0 E (m)++a) The soft token is prepended on text features across the layers, leading to modified denoising function: vg t, y, s). By adopting the soft tokens, we can adjust the image and text representations to get better semantic alignment. Additionally, for computational efficiency. we approximate the expectation within (xx y) using a single Monte Carlo sample, effectively replacing it as
f(, y, 8) = (a.. v.)- (c-))l)) /+(()
(13)
which can stabilize the computation using the same e and f in the same batch. The resulting contrastive loss function with soft tokens can be defined as follows:
exp(i(x, y, 8)
LSAEREPA (s) = -E (E (-1.))
log Σ, exp(1(æ, yb),
(14)
Then, the learnable token is optimized by minimizing CSoRREPA (8). During inference, the trained soft tokens are leveraged with fixed text representation from the text encoder across layers and timesteps. The detailed procedure during the image generation process based on MM-DiT architecture described in algorithm [ firo appendix
3.3 Relationship with Mutual Information
Although the relationship between contrastive loss and mutual information has been previously studied, we revisit it here to provide a more complete theoretical perspective. Additionally, we emphasize the critical role of the logit formulation in the score matching loss, highlighting its importance in ensuring stable and effective optimization.
According to 18), pointwise mutual information (PMI) over image x and text pairs can be defined as follows:
log po( y)) log
po(x yy)
( y)
(15)
po
Ep(c) [Po (x/cc
where the second equality follows from the definition of conditional expectation. Song et al. 36 and Kong et [T8) further show that assuming the diffusion model is an optimal denoiser, the conditional likelihood pe (az y) can be described as (see appendix A):
(a (xxl)) y) = exp(i y)) where Î( v) - l'
X(t) v, C
(16)
By approximating the denominator of PMI via Monte Carlo sampling, we arrive at:
exp(i(x,)) y))
i(x y) R log
(17)
N D. exp(((xx,))
5

---

COCO val5K
Efficiency
Human Preference
Text Alignment
Image Quality
PM
Latency
Model
ImageReward?
PickScoret
CLIP
HPS
FID4
LPIPS)
(GB)
(sec/image)
SD1.5
17.72
21.47
26.4
25.08
24.59
43.80
2.621
1.526
SDL.5 +Ours
32.89
21.50
27.33
25.18
23.43
43.38
2.621
1.547
SDXL
75.06
22.38
26.76
27.35
24.69
42.05
10.45
4.059
SDXL +Ours
85,29
22.62
26.80
28.30
26.04
42.39
10.45
4.060
SD3
94.27
22.54
26.30
28.09
31.59
42.43
18.77
4.637
SD3 +Ours
108.5
22.55
26.91
28,91
36.21
42.88
18.77
4.695
GenEval
Model
Meant
Singlet
Twot
Counting?
Colorst
Position"
Color Attribution!
SD3
0.68
0.99
0.86
0.56
0.85
0.27
0.55
CaPO 20
0.71
0.99
0.87
0.63
0.86
0.31
0.59
RankDPO [66
0.74
1.00
0.90
0.72
0.87
0.31
0.66
SD3 + Ours
0.70
1.00
0.95
0.29
0.92
0.34
0.68
Table 1:
Quantitative evaluation of T2I generation with learnable soft tokens SD1.5, SDXL, and
SD3. Generation quality is evaluated on the COCO-val 5K (23 and GenEval (9 benchmark. Peak
GPU memory(PM) usage and per-image latency are measured to assess computational efficiency. ImageReward, CLIP, HPS, and LPIPS are scaled by x x10?.
This formulation closely resembles cross-entropy loss, where the conditional log-likelihood i(x, y) can be interpreted as logits. Now, from the fact that mutual information is the expectation of the pointwise mutual information, we have
exp([(x yy)) 1 Σ log E, exp(((x((1), y()))) 71
I(X
+ D
(18)
where (x) represents the i-th sample in the data, n denotes the number of samples in the data, and D indicates the constant. Notably, Eq. (18) is very similar in form to the contrastive objective in Eq. (
where the logit is the negative of the diffusion loss. This implies that minimizing our contrastive learning objective is closely related to maximizing the mutual information between image-text pairs under the diffusion model.
4 Experiments
Implementation details. In our experiments, to confirm the flexibility of SoftREPA, we utilize various open-source T2I diffusion models, including Stable Diffusion 1.5, Stable Diffusion XL, and Stable Diffusion 3 to evaluate text and image alignment in both image generation and text-based image editing tasks. For the training of softs token, the batch size iterations using two 100 GPUs. For Stable Diffusion 3, we set the length of the soft token to 4 and used 5 layers to attach the soft tokens for further experiments. In the of Stable Diffusion 1.5 and Stable Diffusion XL, soft tokens are applied only to the Down or Middle block layers of the UNet. Refer further implementation details in appendix B
16 with less than 30,000
Text to Image Generation We conducted text-to-image generation experiments on SD1.5, SDXL, and SD3, training soft tokens using the text-image paired COCO dataset [23] The implementation
details are provided in appendix B To evaluate the generated images, we assessed human preference
scores 45 [7]. text-image aligmment 48 44), and image quality [47 34] on COCO-val 5K dataset.
Furthermore, we conducted evaluation on GenEval [9]. compared with RankDPO [16], which is
fine-tuned via preference optimization. As shown in table in COCO dataset, the proposed method generally outperforms baseline approaches in both diffusion and rectified flow models regardless of the model architecture. Additionally, in GenEval, SD3 with SoftREPA mostly outperforms RankDPO [16]. The drop in the counting metric can be primarily attributed to the tendency of diffusion models to interpret text alignment in a way that encourages the generation of multiple instances of the referenced object. Further analysis and potential mitigation strategies are discussed in appendix [ In fig. 3. detailed textual descriptions are more accurately reflected in the generated images, demonstrated in the qualitative results.

---

Human Preference
Text Alignmnet
Stracture
Background Preservation
Image RRwardd
Pick Score!
CLIP Edited
CLIP
LPIPS/
Inversion
Method
Whole
HPST
Distance
PSNRT
Whole
SSIMT
udim
Pul
32.29
21.53
22.56
25.
26.11
27.32
22.31
14.48
79,58
ddim.
PnP (Ours)
36,78
21.59
22.70
15.78
26.35
27.43
22.27
14.55
79.36
PIEBench
direet
PaP (Ours)
43.32
21.70
22.80
25.83
26.76
23.41
22.40
13.53
79.93
ddim
MasaCarl
-13.44
21.20
24.
23.59
14.59
80.41
ddim
MasaClil (Ours)
-12.76
21.05
21.27
24.44
23,69
27.36
22.27
14.5
80.27
direct
MasaCtrl
4.77
21.39
21:47
24.54
24.92
23.61
22.82
12.21
82.02
MasaCirl
FlowEdit
87,70
22.16
23.19
26.72
28.04
25.48
24.37
12.55
FlowFidir (Ours)
102.24
22.31
23,60
27.19
28.58
24.07
24.99
12,60
88.73
udim
PuP
-9.72
21.10
26.07
24.76
32.39
18.89
ddim
PoP
-7.75
26.18
18.88
dircel
PnP
-6.86
21.17
25.09
29.15
17.96
DIV2K
direct
PnP (Ours)
-5.56
21.22
26.15
25.17
28.82
17.99
ddim
MasaCarl
23.37
17.40
ddim
MasaCurl (Ouns)
-57.41
20.59
23.46
21.64
28.79
17.03
direct
MasaCtri
-42.33
20.80
23.65
22.7
71.16
31.41
MasaCurl
Howfidit
38.08
21.74
26.05
25.53
39.68
15.48
FlowEdit (Ouns)
46.68
21.88
26.39
26.03
35.66
14.98
C2D
FlowEdit
93.70
20.72
22.51
26.68
33.41
19.92
HowlHdit
Table 2: Quantitative evaluation of image editing performance with the use of soft tokens PnP [8]
MasaCtri [4]. Direct inversion [1]. and FlowEdit [19]. FlowEdit is based on SD3, and others are
based on SD1.5 architecture. The editing methods are evaluated on PlEBench [14, DIV2K [1. and
Cat2Dog (C2D). ImageReward, CLIP, HPS, LPIPS, and SSIM are scaled by x x10² and Distance is
scaled by x103.
stor relisonee reflecting mirrar."
"A
motorrycle
"Seffe her Non plains"
person standing deach with skyy
plate
fantasy concept god
eleon bed in hotel more with townk and scop loid out."
'4 pellow and grem object with brown bird on lop of
Lobeeiiie daning oxtumn with pointte of Seey colors trews
Three aebras in feld near bashes."
cosss picture of the Goodraayes Boy School
group older people around lang restouant toble
S03
SD3 (Ours)
SD3
SD3 (Durs)
SD3
SD3 (Durs)
Figure 3: The qualitative results of text-to-image generation comparing SD3 and SD3 with proposed method. The given text is from COCO and Pixart dataset.

---

"...motorcycle.. - > "i.bi.e..."
"..tulips... " "... "..diisies...
"..logs..." ...crates...
"...rocky beach..." "...y beach..."
...coooiies, yellow flowers... > brownies, pink roses...
"..bbicyclists... " skateboorders...
"..bbue white..." " "...red gold..."
"_ ..Barbie.... Ken..."
Source
Edited
Edited (ours)
Source
Edited
Edited (aurs)
Figure 4: The qualitative results of text guided image editing comparing on SD3 with the proposed method. The FlowEdit [19 is used as the editing method for both baseline and SoftREPA
CAT200G
0.20
RowEdit RouEoiiOUus)
0.23
0.19
UPIP5
0.18
0.21
0.5
644
0.0
0.83
0.17
0.20
02
Imagelzand
0.42
0.42
0.16
0.18
TloEdR RowEilROOurs
LMPS
0.15
Number Layers
Number Tokens
0.265
0.270
0.235
CLIP Score
CLIP Score
Figure 6: Ablation study the number of layers and token length for soft tokens. Metrics are normalized to: ImageReward [0.7, 1.1), CLIP score [0.25, 0.28), and LPIPS [0.42, 0.44].
Figure Quantitative comparison of baseline and SoftREA DIV2K and Cat2Dog editing using CLIP Score and LPIPS with various CFG scales.
Memory and Speed Comparison In table we report the peak GPU memory usage (PM) and per-image latency for models with and without soft tokens, including SD1.5, SDXL, and SD3. All models were evaluated in f6oat16 precision. The of soft tokens introduces no change in peak memory and results in only a negligible increase in latency.
Text Guided Image Editing To further demonstrate the improved text-image alignment achieved by our method, we conducted text rext-guided image editing experiments using PnP [38]. MasaCtrl 4
with DDIM (35), Direct Inversion [14), and FlowEdit [19]. FlowEdit employs SD3, while the other
methods use SD1.5. For evaluation, we used PlEBench [14) DIV2K []. and Cat2Dog datasets.
For DIV2K editing dataset, we selected 800 high-quality images and generated detailed source and
target captions using LLaVA (25;; dataset construction details are provided in appendix C In table 2

---

Human Preference
Text Alignment
Image Quality
Model
ImageReward
PickScore
CLIPt
HPST
FIDI
LPIPS]
SD3
90.89
22.50
26.26
27.97
56.87
42.47
SD3 LoRA + Contrastive
97.72
22.57
26.42
28.44
70.33
42.71
SD3 + Soft tokens Contrastive loss (Ours)
106.32
22.49
26.92
28.70
60.76
42.99
Table 3: Quantitative comparison of T2I generation (COCO-val 1K (231)
between soft tokens and
LoRA on SD3. ImageReward, CLIP, HPS, and LPIPS are scaled by x10º
the quantitative results of human preference, text alignment, and structure distance show consistent enhancements with the use of soft tokens without compromising structure and background fidelity.
Considering that FlowEdit performs text-based image editing, where performance heavily depends on text-image alignment, this result indicates improved efficiency in modeling the joint text-image distribution using the proposed method. Furthermore, our method with soft tokens achieves more efficient editing by reducing the number of editing steps to 30, compared to 33 in FlowEdit. As shown in fig. 5. incorporating soft tokens across various CFG scales consistently enhances text-image alignment and image quality. Figure 4 further demonstrates that performance has been improved on both single and multi-concept editing compared to vanilla SD3. Additional results and details are provided in appendix F.
We conducted an ablation study on SD3 using the COCO val dataset 23 to evaluate the effects of the number of layers using soft text tokens and token length on generation quality (fig. (). A value of O corresponds to vanilla SD3, and the number of layers is counted from the first. Comparisons were made using either a fixed token length of 8 or a fixed depth layers. Our findings indicate that incorporating soft tokens beyond layer 7 severely degrades image generation quality. The optimal performance, aligned with human preferences (ImageReward 45)), was observed when soft tokens were applied within layers 2 5. This aligns with the architecture. where early layers handle text-image alignment and later ones enhance fidelity [27]. Notably, CLIP scores continued to improve more layers used soft tokens. indicating stronger adherence to text prompts, while overall image quality remained stable. Supporting qualitative results are provided in appendix G For token length, using more than 8 tokens negatively impacted quality, suggesting overfitting to the fine-tuning data. Shorter lengths (1-4 tokens) preserved perceptual similarity (LPIPS), while moderate lengths (4-8 tokens) significantly improved text- text-image alignment (CLIP and ImageReward). Qualitative examples are shown in appendix
Ablation Study
Comparison with LoRA To further assess the effectiveness of our design, we examined whether contrastive learning alone could achieve comparable improvements. Specifically, we conducted an ablation study replacing the soft tokens with LoRA. As shown in table the combination of soft tokens and contrastive loss consistently outperforms all other variants across evaluation metrics. This result demonstrates the strong synergy between soft tokens and contrastive learning in enhancing text-image alignment. These findings validate the core motivation of SoftREPA explore the text feature space for improved alignment between textual and visual representations. Additional comparisons with different LoRA ranks and layer configurations presented in appendix H
Complementarity with Diffusion RL methods We examined the complementarity between SofRREA and diffusion reinforcement learning (RL) methods. Since these approaches are not mutually exclusive, we tested whether SoftREPA can enhance diffusion RL frameworks. Specifically, we integrated pretrained soft tokens from SoftREPA into two representative
Sithhods--ifffsionn DPO [33, which leverages preference optimization, and DDPO B. which applies policy optimization with an external reward- -usingusnngggg a plug-ant and---lay approach without additional training. As shown in [able incorporating SoftREPA's soft tokens consistently improves performance. This suggests strong complementarity while diffusion RL methods align outputs via preference or reward signals, SoftREPA refines internal text-image representations through contrastive learning. Combining the two effectively unites explicit preference alignment with robust representation learning to enhance generation quality.

---

Human Preference
Text Alignment
Image Quality
Method
ImageRewardf
PickScoref
CLIPT
HPSt
FIDI
LPIPS,
Diffusion-DPO
22.26
21.68
26.51
25.60
54.19
43.96
Diflusion-DPO Ours
33,10
21.64
27.41
25.55
53.72
43.64
DDPO
-8.83
21.20
26.21
23.45
$4.83
43.85
DDPO + Ours
8.07
21.25
27.13
23.75
54.18
43.59
Table 4: Complementarity of SofREEA with diffusion RI. methods (Diffusion-IPO [43], DDPO 3) Soft tokens are integrated in a plug-and-play manner during inference. Generation quality evaluated on the COCO-val IK dataset [23]. ImageReward, CLIP, HPS, and LPIPS are scaled by x x10?.
5 Related Works
Several methods have been proposed to address the text-image misalignment. Training free approach such as CFG++ 77 addresses this problem the off-manifold phenomenon of conventional CFG and reformulates it to a mantfold-conssrained CFG to get better text-to-image alignment. Other training-fres approach, attend-ande -xxeiit [5]. adjust attention activations to enhance text-image consistency. DOODL [22 applies guidance to end-to-end manner. Meanwhile, training-bai based approaches such as Diffusion-DPO [43 introduce alignment losses during training to both enforce stronger text-conditioning and visual realism. Following Diffusion-DPO. denoising diffusion policy optimization(DDPO) [3] enhances the alignment using feedback from the pre-trained vision-language model. RankDPO [16] presents fully synthetic, ranking- based preference datasets generated by reward models instead of humans. However, these methods require either extensive full-finctuning or an additional annotation dataset. In this paper, to get better text and image alignment, proposing a novel SoftREPA on soft tokens.
the prior knowledge of diffusion model itself
To enable efficient prompt-level fine-tuning, VPT [3 introduced learnable tokens for vision trans- formers while keeping the backbone frozen. CoOp 50 and CoCoOp 49) proposed prompt learning methods for CLIP-like vision-languag, models by optimizing prompts to distinguish between predefined set of classes in image classification tasks. In recent works, Minority
(39] and
Optical-Prome (28] learned sample-specific prompts to improve diversity and temporal coherence during sampling steps, respectively. contrast to these approaches, which typically tailor prompts to sample-independer soft tokens. These tokens are optimized over a distribution of text-image pairs, (X, Y). effectively guiding the propagation of text features to better align with the joint text-image distribution.
individual samples specific class sets, we propose learning veneral-purport
Conclusion
In this work, we introduce SoftREPA, novel approach for improving text-image alignment in generative models. Building on the success of prior representation alignment techniques, we extend the idea of internal representation alignment to the text-image alignment without relying on external encoders. Our method effectively integrates contrastive learning and score matching loss to enhance text-image representation learning while preserving the generative capabilities of diffusion models. Also, we introduced soft text tokens, which can effectively adjust the image-text representations through contrastive T21 alignment, making it lightweight and efficient. We demonstrate that our approach enhances text-image alignment both image generation and editing tasks while maintaining minimal parameter overhead. Our experiments span various models, including Stable Diffusion 1.5, XL, and 3.
Limitation and Potential Negative Impacts. While soft tokens effectively enhance text-image aligmment, they may risk overemphasizing textual guidance at the expense of faithfully preserving the user's 's intent in the prompt. Future work could explore regularization techniques to mitigate this issue. Additionally, this work does not address potential risks such as data poisoning or adversarial attacks during soft token training. Although these concerns are beyond the scope of this paper, they merit consideration in future research. Moreover, our method inherits any potential negative impacts of the underlying backbone models.
10

---

Acknowledgments and Disclosure of Funding
This work was supported by the Institute of Information & communications Technology Planning & Evaluation (ITTP) grant funded by the Korean government(MSIT) (No. RS-2024-00457882 Al Research Hub Project). This work was supported by the National Research Foundation of Korea under Grant R5-2024-00336454.
References
[1] Firikur Agustsson and Radu Timofte. Ntie 2017 challenge on single image super-resolution: Dataset and study. In Proceedings of the IEEF conference on computer vision and pattern recognition workshops, pages 126-135, 2017.
[2] Jean-Baptiste Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech, lain Barr, Yana Hasson, Karel Lenc, Arthur Mensch, Katherine Millican, Malcolm Reynolds, et al. Flamingo: a visual language model for few-shot learning. Advances in neural information processing systems, 35:23716-23736, 2022.
[3] Kevin Black, Michael Janner, Yilun Du, Ilya Kostrikov, and Sergey Levine. Training diffusion models with reinforcement learning. In International Conference on Learning Representations. 2024.
[4] Mingdeng Cao, Xintao Wang. Zhongang Qi. Ying Shan, Xiaohu Qie, and Yingiang Zheng. Masactrl: Tuming-free free mutual celf-attention control for consistent image synthesis and editing. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 22560 22570, October 2023.
[5] Hila Chefer. Yuval Alaluf. Yael Vinker, Lior Wolf, and Daniel Cohen-Or. Attend-and -nd-eiitee: Attention- based semantic guidance for text-to-image diffusion models. ACM transactions Graphics (TOG), 42(4): 1-10, 2023.
[6] Ting Chen, Simon Komblith, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning visual representations. arXiv preprint arXiv:2002.05700 2020.
[7] Hyungjin Chung, Jeongsol Kim, Geon Yeong Park, Hyelin Nam, and Jong Chul Ye. Cfg++: Manifold- constrained classifier free guidance for diffusion models. arXiv preprint arXiv:2466. 080700, 2024.
[8] Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas Müller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, et al. Scaling rectified flow transformers for high-resolution image synthesis. In Forty-first international conference on machine learning, 2024.
[9] Dhruba Ghosh, Hannaneh Hajishirzi. and Ludwig Schmidt. Geneval: An objuct-focused framework for evaluating lext-to-image alignment. Advances in Neural Information Processing Systems, 36:52132-52152. 2023.
[10] Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al- Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. The llama 3 herd of models. arXiv preprint SiXi:22007.21883 2024.
[11] Hyconho Jeong, Chun-Hao Paul Huang, Jong Chul Ye, Niloy Mitra, and Duygu Ceylan. Track4gen: Teach- ing video diffusion models to track points improves vidco generation. arXiv preprint arXiv:2412.06016. 2024.
[12] Chao Jia, Yinfei Yang, Xia, Yi-Ting Chen, Zarana Parekh, Hieu Pham, Quoc Le, Yun-Hsuan Sung, Zhen Li, and Tom Duerig. Scaling visual and vision-language representation learning with noisy text supervision. In International conference machine learning, pages 4904-4916. PMI.R, 2021.
[13] Menglin Jia, Luming Tang, Bor-Chun Chen, Claire Cardie, Serge Belongie, Bharath Hariharan, and Ser- Nam Lim. Visual prompt tuning. In European conference computer vision, pages 709-727. Springer, 2022.
[14] Xuan Ju, Ailing Zeng. Yuxuan Bian, Shaoteng Liu, and Qiang Xu. Pup inversion: Boosting diffusion-based editing with 3 lines of code. International Conference on Learning Representations (ICLR), 2024.
[15] Tero Karras, Miika Aittala, Timo Aila, and Samuli Laine. Elucidating the design space of diffusion- generative models. In Proc. NewrIPS, 2022.
[16] Shyamgopal Karthik, Huseyin Coskun, Zeynep Akata, Sergey Tulyakov, Jian Ren, and Anil Kag. Scalable ranked preference optimization for text-to-image generation. arXiv preprint, 2024.
11

---

[17] Yuval Kirstain, Adam Polyak, Uriel Singer. Shahbuland Matiana. Joe Penna, and Omer Levy. Pick-a-pic: An open dataset of user preferences for text-lo-image generation. 2023.
[18] Xianghao Kong, Ollie Liu, Han Li, Dani Yogatama, and Greg Ver Steeg. Interpretable ditfusion via information decomposition. arXiv preprint arXiv:2310.079722 2023.
[19] Vladimir Kulikov, Matan Kleiner, Inbar Huberman--Sieeglllas, and Tomer Michaeli. Flowedit: Inversion- free text-based editing using pre-trained flow models. arXiv preprint urXiv:2412.08629 2024.
[20] Kyungmin Lee, Xiahong Li, Qifei Wang. Junfeng He, Junjie Ke, Ming- Hsuan Yang. Irfan Essa, Jinwoo Shin, Feng Yang, and Yinxiao Li. Calibrated multi-preference optimization for aligning diffusion models. In Proceedings of the Computer Vision and Pattern Recognition Conference, pages 18465-18475, 2025.
[21] Junnan Li, Ramprasaath Selvaraju, Akhilesh Gotmare, Shafiq Joty, Caiming Xiong, and Steven Chu Hong Hoi. Align before fuse: Vision and language representation learning with momentum distillation. Advances in neural information processing systems, 34:9694 1495994-9705, 2021.
[22] Junnan Li, Dongxu Li, Caiming Xiong, and Steven Hoi. Blip: Bootstrapping language-image pre-training for unified dision-language understanding and generation. In International conference on machine learning, pages 12888-12900. PMLR, 2022.
[23] Tsung- Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In Computer Vision-EED ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part V 13, pages 740-755. Springer, 2014.
[24] Yaron Lipman, Ricky T. Q. Chen, Heli Ben-Hamu, Maximilian Nickel, and Malthew Le. Flow matching for generative modeling. In The Eleventh International Conference on Learning Representations, 2023. URL |tttps://openreviee http:/ppprrrrvee..eeeettorrrrr// http:/ooruumndd-PPPPd0333.339
[25] Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jac Lee.. Visual instruction tuning. Advances in neural information procssing systems, 36:34892-34916, 2023.
[26] Haotian Liu, Chunyuan Li, Yuheng Li, Bo Li, Yuanhan Zhang, Sheng Shen, and Yong Jae Lee, Llavanext: Improved reasoning, ocr, and world knowledge, 2024.
[27] Benyuan Meng, Qianqian Xu, Zitai Wang, Xiaochun Cao, and Qingming Huang. Not all diffusion model activations have heen evaluated as discriminative features. Advances in Neural Information Processing Systems, 37:55141 15-5111--5,77, 2025.
[28] Hyelin Nam. Jaemin Kim, Dohun Lee, and Jong Chul Ye. Optical flow guided prompt optimization for coherent video generation. arXiv preprint 1555522111.1540, 2024.
[29] Suraj Patil, Pedro Cuenca, Nathan Lambert, and Patrick von Platen. Stablc diffusion with diffusers. Hugging Face Blog, 2022. https://hugginggfae..ooooooggggu
difffusion.
[30] William Peebles and Saining Xie. Scalable diffusion models with transformers. In Proceedings of the IEEE/CVF international conference on computer vision, pages 4195 4205, 2023.
[31] Dustin Podell, Zion English, Kyle Laccy, Andreas Blattaann, Tim Dockhor, Jonas Müller, Joe Penna, and Robin Rombach. Sdxl: Improving latent diffusion models for high-resolution image synthesis. arXiv preprini arXiv:2307.01952, 2023.
[32] Alec Radford, Jong Wook Kim, Chris Hallacy. Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transfcrable visual models from natural language supervision. In International conference on machine learning, pages 7748-8863. PmL.R,
2021.
[33] Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. High-resolution image synthesis with latent diffusion models. Proceedings the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages (0684-10695, 2022.
[34] Maximilian Seitzer. pytorch-fid: FID Score for PoTor.. http:://ggthubb.ccmmnsttttt pytorch- August 2020. Version 0.3.0.
[35] Jiaming Song. Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. In Interna- tional Conference on Learning Representations. 2021. URL https:///peenrevier wwwp:/wwweeeeoeteieiio. iid- StigiarCHLPP
12

---

[36] Yang Song, Conor Durkan, lain Murray, and Stefano Ermon. Maximum likelihood training of score-based diffusion models. Advances in Neural Information Processing Systems, 34, 2021.
[37] Chameleon Team. Chamelcon: Mixed-modal early-fusion foundation models. arXiv preprint arXiv:2405.0988 099188, 2024.
[38] Narek Tumanyan, Michal Geyer, Shai Bagon, and Tali Dekel. Plug-and-play diffusion features for text- driven image-to-image translation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 1921 1930, June 2023.
[39] Soobin Um and Jong Chul Ye. Minorityprompt: Text minority image generation via prompt optimization. arXiv preprint 2rXiv22410.78838. 2024.
[40] Rejin Varghese and M Sambath. Yolov8: A novel object detection algorithm with enhanced performance and robustness. 2024 International Conference on Advances in Data Engineering and Intelligent Computing Systems (ADICS), pages 1-6, IEEE, 2024.
[41] Pascal Vincent. A connection between score matching and denoising autoencoders. Neural computation. 23()):1611-167, 2011.
[42] Bram Wallace, Akash Gokul, Stefano Ermon, and Nikhil Naik. End-to-end diffusion latent optimization improves classifier guidance. In ICCV, 2023.
[43] Bram Wallace, Meihua Dang, Rafael Rafailov, Linqi Zhou, Aaron Lou, Senthil Purushwalkam, Stefano Ermon, Caiming Xiong, Shafiq Joty, and Nikhil Naik. Diffusion model alignment using direct preference optimization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8228 8238, 2024.
[44] Xiaoshi Wu, Yiming Hao, Keqiang Sun, Yixiong Chen, Feng Zhu, Rui Zhao, and Hongsheng Li. Human preference v2: A solid benchmark for evaluating human preferences of text-to-image synthesis. arXiv preprint arXiv:2306. 09341, 2023.
[45] Jiazheng Xu, Xiao Liu, Yuchen Wu, Yuxuan Tong, Qinkai Li, Ming Ding. Jie Tang, and Yuxiao Dong. Imagereward: learning and evaluating human preferences text-to-image generation. Proceedings of the 37th International Conference on Neural Information Processing Systems, pages 15903-15935, 2023.
[46] Sihyun Yu, Sangkyung Kwak, Huiwon Jang. Jongheon Jeong, Jonathan Huang. Jinwoo Shin, and Saining Xie. Representation alignment for generation: Training diffusion transformers easier than you think. In International Conference on Learning Representations, 2025.
[47] Richard Zhang, Phillip Isola, Alexci A Efros, Eli Shechtman, and Oliver Wang. The unrcasonable effectiveness of deep features as perceptual metrie. In CVPR, 2018.
[48] SUN Zhengwentai. clip-score: CLIP Score for PyTorch. March 2023, Version 0.1.1.
[49] Kaiyang Zhou, Jingkang Yang, Chen Change Loy, and Ziwei Liu. Conditional prompt learning for vision- language models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 16816 16825, 2022.
[50] Kaiyang Zhou, Jingkang Yang, Chen Change Loy, and Ziwei Liu. Learning to prompt for vision-language models. International Journal of Computer Vision, 13009)22337-25 -244,, 2022.
13

---

NeurIPS Paper Checklist
1. Claims
Question:
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:
We clearly claim the SoftREPA's contribution and scope in the abstract and introduction.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification:
We discussed the limitations of SoftREPA in the experiments, and appendix.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g.. independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't 't acknowledged judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
the paper. The authors should use their best
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
14

---

Justification:
We provide clear assumptions and complete proof of concept.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless whether the code and data are provided or not)?
Answer: [Yes]
Justification:
We fully disclose the information about experiments for reproducibility.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make clear how to reproduce that algorithm. (b) the contribution is primarily a new model architecture, the paper should describe the architccture clearly and fully.
(c) If the contribution is new model (e.g., a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reprodueibility. the case of closed-source models, may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
15

---

Answer: [Yes]
Justification:
The code will provided.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips. cc/
for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
/
sttpiiisssooooiocyyy. for more details.
The authors should provide instructions on data access and preparation, including how access the raw data, preprocessed data, intermediate data, and generated data,
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification:
We specified all the training and test details.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core the paper to a level detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined other appropriate information about the statistical significance of the experiments?
Answer: [No]
Justification: reporting results, we have been following existing literature in this domain, evaluating various methods and metrics over 5K images in the generation experiment, and reporting aggregate scores.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g., Normally distributed errors).
16

---

It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification:
We provide information on the computer resources.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't 't make it into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https///waurrr...pppp..cccttttat
Answer: [Yes]
Justification:
We reviewed the NeurIPS Code of Ethics.
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification:
We mentioned the societal impact of our work in the conclusion.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no socictal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
17

---

The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms monitor how system learns from feedback over time, improving the efficiency and accessibility of MI..
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
This paper poses such risks.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Lieenses for existing assets
Are the creators or original owners of assets (e.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Question:
Answer: [Yes]
Justification: This paper cites the original paper that produced dataset and models.
Guidelines:
The answer NA means that the paper does not existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
18

---

If this information is not available online, the authors are encouraged to reach out to the asset's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: This paper does not release new assets.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/mmoel as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent obtained from people whose asset is used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
This paper does not involve crowdsourcing nor research with human subject.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question:
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: This paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
16. Declaration of LLM usage
19

---

Question: Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods this research? Note that if the LLM used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Answer: [NA]
Justification: The core method development in this research does not involve LLMs.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs any important, original, non-standard components.
Please refer to our LLM policy fattps://neurd/: /00p://ccc...cc.eeeee//////////0 for what should or should not be described.
20

---

SoftREPA for better Text-to-Image Alignment
Supplementary Materials
A Information Theoretical Analysis of Diffusion Models
Several studies have explored diffusion models from an information- -ateorotttthhcticcccccoo viewpoint. Accord- ing to [8], the mutual information between random variables X and Y can be expressed as the expectation of pointwise mutual information over their joint distribution:
T(X, Y) = =Bp(2.))) [(a y)].
(19)
where the pointwise mutual information (PMI) is defined the difference between the conditional and marginal log-likelihoods of given sample a::
i( y
log p(z) y) log p(z).
(20)
This quantity measures how much the presence of condition y affects the probability distribution near x.
Song et al. 336 and Kong al. 18 showed that, under the assumption of an optimal diffusion model trained on given data, the log-likelihood of an image a can be formulated as:
log log)(()) = / A(t)Ex|llco(mz - <ll +
(21)
This result also extends to conditional diffusion models, where the likelihood of æ given condition (e.g., text embeddings) is given by:
/ ((l) c|l3]ct C
logp(zly) = -
(22)
B Implementation Details
Architecture Details For Stable Diffusion 3, the main experiments use soft tokens of length 4, applied across 5 layers. Stable Diffusion 1.5, soft tokens of length 4 are applied only to the Down block layers of the UNet's conditional text embeddings. For Stable Diffusion XI., soft tokens of length 8 are used, applied to both the Down and Middle block layers of the UNet's conditional text embeddings, without incorporating timestep dependency.
Training Details The hyperparameters used during training listed in table 5 For Stable Dif- fusion 3, the soft tokens are optimized solely using the contrastive score matching loss (TsSoRePA)) In contrast, for Stable Diffusion 1.5 and Stable Diffusion XL, optimization combines the con- trastive score matching loss (LSaRREPA) with a small weighting of the denoising score matching loss (hoss))table 6 shows that the model consistently achieves improved performance regardless of whether the denoising score matching loss is applied. We observed that initializing the soft tokens with random distribution led to performance degradation Stable Diffusion 1.5. To address this, we initialized the soft tokens using unconditional text embeddings. The code is publicly available at https:/ gi thub. com/ softtepa/ SoftREPA
batch size (positive, negative)
Models
Ir
wd
iterations
loken init
optimizer
Ir scheduler
SD1.5
1c-3
lc-4
32(4, 28)
26,000
Ø
Adam
Cosine Aaaiaaennncaiinggg Restrsst
SDXL
le-3
le-4
16(1, 15)
30,000
N0 0.02)
Adam
Cosine Annealing Warmtestarts
le-3
le-4
16(4, 12)
(.000.02)
Adam
Cosine Annealing WermRestarts
Table 5: The implementation details for training.
21

---

Human Preference
Text Alignment
Image Quality
Model
loss
ImageReward""
PickScoret
CI.IPT
HPS1
FID.
IPIPS
17.72
21.47
26.45
25.08
24.59
43.80
SD1.5
SORRRIPA + LDSM
32.89
21.50
27.33
25.18
23,43
43.38
LSSOREPA
32.63
21.61
27.10
25.94
29.25
44.09
75.06
22.38
26.76
27.35
24.69
42.05
SDXL
SOSREEP + LDSM
85.29
22.62
26.80
28.30
26.04
42.39
IISAREEAA
81.09
22.59
26.87
28.32
26.42
42.69
Table 6: Quantitative comparison of T21 generation between contrastive score matching loss (LsoRrepa) with and without denoising score matching loss (LDSM). Generation quality is eval- uated on the COCO-val IK [23]. ImageReward, CLIP, HPS, and LPIPS are scaled by x10².
Algorithm Image Generation with Soft Tokens in MM-DiT
Require: Gaussian noise (0.
Require: Text Y Piuta
Require: Soft
S Embedding(k; t)
Require: Number of layers N, Threshold layer L
Require: Time steps (1r, br-,,.,,,))
Initialize H (0.T)
1:
ing
2: Initialize H TextFncoder(Y)
3: n= (
for in (Ir, 11. fo
5: for = 1 to N do
6: if k < I, then
7:
8 g +- Embedding((:,
8:
A - (s(htt); ( (1-))
else
10:
Ho (k-111 (8-1.0)
11:
end
12:
H (k.t)
(k.1) <- Layer,(Fi (FI (k00 Fr 11(0-110)
13:
H (K.A)
(A.A)
H (k.4)) [- n :.
Drop soft tokens
14:
end for
15:
end for
16: return X Decoder H ing
(N))
Inference Details detailed deseription the image generation algorithm on MM-Dit is provided in algorithm [I] At each layer, a distinct set of soft tokens is prepended to the text features and used exclusively within that layer. These soft tokens are not carried over to subsequent layers; instead, new soft tokens are introduced or omitted as appropriate. Their primary role is to guide the text tokens toward better text-image alignment, particularly during the early stages of the model.
Memory Efficiency For the memory efficiency metrics reported in table 1 values were measured by averaging results over 50 runs with an A 100 GPU. Image resolution was set to 512 x 512 for SD1.5 and 1024 x 1024 for SDXI. and SD3.
C Prompt for Editing dataset
To generate source/target text prompt from images, we leverage LLaVA 25 and Llama 10 sequentially. Specifically, we extract source text description from 800 training images of DIV2K dataset by giving each image and following text prompt to LLaVA.
"Describe the object and background in the image"
We use checkpoint from 1biv/llaaa-1.5--13bb.
We use Llama-3.1 3.1-BBB.BBBBB..... IS-Irrrruut model.
22

---

Then, we generate the target text prompt that has only single different concept compared with the source text prompt using Llama with the following instruction.
"You are an Al assistant for generating paired text prompts for real image editing tasks. Your goal is to modify a given text description by replacing an object with other while strictly following these rules:
1) Modify only one object (i.e., a single meaningful concept such as an object).
It could be small object.
2) The replacement must be significantly different from the original concept but contextually appropriate. Avoid unrealistic substitutions (e.g., changing "rabbit on grass" to "rocket on grass").
3) Ensure diversity in word choices across different modifications.
4) Preserve all other words exactly as they are. Do not change sentence
structure, introduce new elements, or modify additional details.
5) Do not provide any additional words words-output only the modified text de- scription.
6) Do not change or add colors. Specifically, when modifying a building, change only the appearance, not the type of building (e.g., do not change "building" to "church" or "lighthouse").
7) Modify only one feature at time. If changing an object (e.g.. "starfish" "sea turtle"), do not alter its color, shape, or other attributes.
Example:
Input: The image features a close-up a brown dog with a blue nose. The dog is standing in grassy field, and the background is blurred, creating focus on the dog's face. The dog's 's ears are perked up, and its eyes are open, giving a curious and attentive expression. The dog's 's fur is brown, and the grass in the background is green, creating a natural and vibrant scene.
Output: The image features a close-up up of a brown fox with a blue nose. The fox is standing in a grassy field, and the background is blurred, creating focus the fox's 's face. The fox's 's ears are perked up, and its eyes are open, giving it a curious and attentive expression. The fox's fur is brown, and the grass in the background is green, creating a natural and vibrant scene." "
D Discussion on counting metric
Adopting Soft Tokens Only A Few Layers As shown in section 4 our model demonstrates notable improvements generating single and multiple objects, as well as in color and position-awe aware synthesis. However, it exhibits relative decline in accurately generating the specified number of objects described in the text prompts. We hypothesize that this drop in the counting metric stems from the soft tokens excessively emphasizing textual cues, which can lead to overgeneration of object instances. To address this, we limited the use of soft tokens to the early layers of the model. As presented in table applying soft tokens only to layers 1 ~ 2 preserves strong overall performance while mitigating the degradation in counting accuracy observed when soft tokens are applied across layers 1 ~ 5.
Incorporating Counting Loss during training To further enhance counting fidelity, we trained SoftREPA with a lightweight object-counting loss. Specifically, we employ a mean squared error (MSE) loss between the predicted object count, which is derived from denoised images, and ground- truth labels using the lightweight object detection module, YOLOv8 [40]. This variant, shown in the last row of table 7 shows improved counting accuracy without sacrificing overall generation quality, suggesting that it effectively reduces the tendency to replicate objects due to textual overemphasis.
E
Additional Results on T2I Generation
Additional qualitative results for SD1.5 and SDXL on the COCO-va15K dataset are presented in fig. 7 and fig. respectively. Furthermore, qualitative examples on the GenEval benchmark are provided
in fig.
23

---

GenEval
Model
Layers
Mcant
Singlet
Twot
Countingt
Colorst
Position
Color Attribution
SD3
0.68
0.99
0.86
0,56
0.85
0.27
0.55
SD3 (Ours)
0.70
0.99
0.91
0.51
0.89
0.31
0.58
SD3 (Ours)
1~2
0.69
0.89
0.50
0.88
0.34
0.56
SD3 (Ours)
0.29
0.92
SD3 (Ours) count
0.59
0.86
0.55
Table Additional quantitative comparison on the GenEval [9] benchmark. Bold indicates the best performance, and underline denotes the second best. "Layers" refers to the transformer layers where soft tokens are applied.
ucronn tuiling Mele kitoy plane fut hevul."
Some children arevidyy or mint orange Inser"
"A cat rosing en on spon lptio computer.
bip puppe bur pow parting sear"
bow' with braczal and parale ralin
chickion meal comotr drocoal and nor"
appR
cow gettgggatkeee es milkee!"
"woo peatus jelly hel"
screon TV embesided in reutroom mirerr"
truck athang
ponen Jugling in wave poo!
beor aitiin
prass
dlwhing large
observing enegeti dog
501.5
SD1.5 (Ours)
$01.5
$01.5 (Ours]
$01.5
$01.5 (Ours)
Figure 7: The qualitative results of text-to-image generation comparing SD1.5 and SDL.5 with proposed method. The given text is from COCO dataset.
24

---

"Man preformigg mix wilt disk tey."
24 proup of people string arownd wooden roble
The nit trcct plowing the mow.
red stop sige betsed chase lidk ferse.
"A acuisture of two womenasinng stin...
aApolp with women carpying o place lagaage"
STOP
"A mun and child Acxt to Nurse"
"a red and block traie coming down the cruck"
to dog Jumping to catch fridee o yord"
mon stonslng near a light and sips."
'Ano Viee wift three erango, white and binch futt"
"A brown pless on the noble for disalyarr
LUGU
TTe for hydtont some srtting on
wearing
penio naking street with a inffle light sbow
young bay shaped Uhe sksteebooed
ssucc rable dinenaaee and televiionn
"Semvaner Pring rove anwinp muchine and plants-
SDXL
SOXL (Ours)
SOXL
SDXL (Ours)
SDXL
SOXL (Ours)
Figure 8: The qualitative results of text-to-image generation comparing SDXL and SDXL with proposed method. The given text is from COCO dataset.
25

---

"t of mirowave.
sioto o broccok and vose
"A photo of roler computer mouse
of pluce plus and bunck
5A pioooo of Aanalaas and a rehipertor"
"Aptuto of two swars."
Xatoww dirce oabra..
AAphoo of two Beaks"
Apleea of four beets""
of sebra
"Aphooo of red dog
pAphoto of brown refigerato."
piooto of the right of
54 peoto applr above
Mithto of bosebal piave below an
pnoto of parple sheep and pink benana.
Mateoo ofa blu vare and a
photo of yelow blodle matorryas
S03
SD3 (Ours)
503
SD3 (Ours)
503
SD3 (Ours)
Figure 9: The qualitative results of text-to-image generation comparing SD3 and SD3 with proposed method. The given text from GenEval dataset.
26

---

Additional Results on Image Editing
Qualitative comparisons of SoftREPA across various editing methods, including PnP 38, Mas- aCtrl [4] with DDIM (35, Direct Inversion [14), and FlowEdit [19], on the PIEBench dataset are presented in fig. [1 corresponding to the quantitative results in table 2
Quantitative results on the DIV2K and Cat2Dog datasets under varying target CFG scales are reported in table [ and table 9 respectively. The corresponding qualitative results for different CFG scales also shown table Additionally, qualitative examples across different numbers of editing steps are provided in fig.
and fig. 10
Human Preference
Text Alignment
Image Quality
Model
NMAX
target CFG
ImageReward
PickScoref
CLIPt
HPST
FID
LPIPS.
FlowEdit
33
13.5
0.380
21.749
0.261
0.255
52.038
0.154
FlowEdit
33
16.5
0.466
21.824
0.263
0.259
55.371
0.180
FlowEdit
33
0.510
0.261
58.069
FlowEdit Ours
30
0.466
21.884
0.263
0.260
52.633
0.149
FlowEdit +Ours
30
11
0.564
21.985
0,268
0.265
56.900
0.178
Hoowhdit +Ours
30
13
0.627
22.050
0.272
0.270
59.710
0.201
Table Quantitative results of image editing regarding image quality and target alignment of generated images with various target CFG scales on DIV2K dataset.
Human Preference
Text Alignment
Image Quality
Model
NMAX
target CFG
ImageRewardf
PickScore
CLIP1
HPS
FID
LPIPS
FlowEdit
33
13.5
0.937
20.726
0.225
0.266
197.1
0.199
FlowEdit
33
16.5
0.908
20.641
0.229
0.272
201.05
0,217
FlowEdit
33
19.5
0.882
20.572
0,232
0.276
202.92
0.234
FlowEdit +Ours
30
1.111
21.143
0,226
0.269
198.71
0.173
FlowEdit +Ours
1.144
21.221
0.234
0.283
208.18
0.204
Flow/dit +Ours
30
1.144
21.224
0.239
0.290
214.72
0.230
FlowEdit +Ours
30
13
1.135
21.200
0,241
0.293
219.35
0.252
Table Quantitative results of image editing regarding image quality and target text alignment of generated images with various target CFG scales on Cat2Dog dataset.
close up photo of cat." close up photo of dog."
Source
Edited
Edited (Ours)
Source
Edited
Edited (Ours)
Figure 10: Additional qualitative results of text-guided image editing on Cat2Dog dataset.
27

---

source
ddim-PnP
direct-PnP
ddim-MasaCtri
direct-MasaCtrl
FlowEdit
Baseline
cortoon pointing
Ours
...wetercooor painting..."
Baseline
"...o waterpainting woman...
Baseline
woman...
Ours
"...o
Baseline
young bay-
Ours
"kids trayon drowing young
Figure 11: The qualitative results of various editing methods including PnP, MasaCtrl, Flow Edit with DDIM and Direct inversion on PIEBench.
G Additional Results on Ablation Study
Quantitative results for varying the number of layers and soft tokens are shown in table 10 with corresponding qualitative examples in fig. 13 and fig. 14] As seen in fig. [3 text prompts are increasingly emphasized as more layers adopt soft tokens. The same input image used in fig. to illustrate the effect of different token counts.
28

---

"...truck..." " > sp.rts bike..." "
...egeen and white caterpillar... "...blue and yellow butterfly..."
"...a fox or a wolf..." "...a bear..."
"...a pineapple..'
"...a watermelon..." "
"...a man..." > "...a woman..."
source
edited
edited1 (ours)
edited2 (ours)
edited3 (ours)
Figure 12:
The additional qualitative results on editing using FlowEdit. The edited1-3 using soft tokens are vary the number of editing steps, 27, 29, 30 out of 50 total timesteps, respectively.
H Additional Results on comparing with LoRA
In table II we conducted various conditions on LoRA finetuning, including top-5-layer and full-layer LoRA, well the corresponding number of trainable parameters and the chosen LoRA rank. Specifically, LoRA with top-5 layers uses 0.4M parameters, LoRA applied to all layers uses 2.3M parameters, and our soft token approach uses 0.9M parameters for layers. We adopt LoRA rank of 4, which is commonly used for diffusion model finetuning.
To enable direct comparison with similar number of parameters, additionally report results for LoRA applied to the top-10 layers (0.9M parameters) and top-5 layers with rank 8 (0.9M parameters). Among these settings, LoRA the top-5 layers with rank 4 yields the best results within LoRA baselines. Importantly, the combination of Soft Tokens and contrastive Loss consistently achieves the strongest performance across key metrics, supporting our claim that both techniques independently and meaningfully improve text-image alignment.
29

---

"A person skating on some pieces of wood."
shower curtain open in empty and clean bathroom.
"A collection of apples growing on tree"
is loading luggage suitcases onto the cart the parking lot."
white the top of tiled building,"
"A desk with bunch of electronics on
Text
baseline
Layer #1-2
Layer #1
Layer #1-6
Layer #1-7
Figure 13: Qualitative results of the ablation study on the number of layers adopting soft tokens.
Human Preference
Text Alignment
Image Quality
#l tokens
# of layers
ImageReward?
PickScuret
CLIPt
HPPS
FIDJ
LPIPS
0.993
22.556
0.263
72.253
0.428
1.009
22.464
0.266
0.285
72.308
0.424
8
0.984
22.403
0.267
0,286
73.840
0.427
8
0.944
22.197
0.269
0.280
74,546
0.425
1
1.054
22.492
0.272
0.283
58.430
0.426
4
1,063
22,493
0.269
0.287
60.766
0.429
8
1.056
22.516
0.268
0.288
62.644
0.431
16
0.801
22.095
0.265
0.278
60.743
0.431
32
0.675
21.876
0.257
0.275
61.623
0.426
Table 10: Quantitative results of image generation ablation study on the number of soft tokens and the number of layers. The evaluation conducted on COCO val IK dataset using SD3 backbone.
30

---

person skoting some pieces of wood."
"A shower curtoin sits open in on empty and clean bathroom."
"A collection opples growing tree"
"A mon is laading luggage suitcases onto the parking lot."
"A white clock tower at the top of tiled building."
desk top with bunch electronics
Text
baseline
Takens #2
Tokens #4
Takens #B
Takens #16
Figure 14: Qualitative results of the ablation study on the number of tokens adopting soft tokens.
Human Prefenence
Text Alignment
Image Quality
Layers
ImngeReward
PickScore
LPUSS]
SD3
90.89
22.50
26.26
27.97
56.87
42.47
SD3 LoRA
Conrrastive loss
1-5
0.4M
97.72
22.57
26.42
25.44
70.33
42.71
SD3 LoRA
Contrastive loss
1-5
0.9M
95.92
22.55
26.28
25.28
70.30
42.55
Contrastive loss
SD3 LoRA
Courrssive loss
all layers
2.3M
$0.89
22.20
26.70
25.97
71.57
41.54
SD3 Soft tokens
Contrstivve loss (Ours)
1-5
8.9M
106.32
22.49
26.92
28.70
60.76
42.99
Table 11: Quantitative comparison T2I generation between soft tokens and various configurations
of LoRA SD3. Generation quality is evaluated the COCO-val 1K (23). ImageReward, CL.IP, HPS, and LPIPS are scaled by x102
31


Reviews:
[{"review_id": "zDRJ6vvYnY", "reviewer": "Reviewer_1MZz", "summary": "This paper introduces SoftREPA, a novel and lightweight method to significantly improve text-to-image alignment in diffusion models. The authors argue that conventional training is suboptimal for representation alignment and instead propose a contrastive learning framework that trains a small set of learnable \"soft tokens\" (fewer than 1M parameters) to better distinguish between matching and non-matching text-image pairs, while keeping the base model's weights frozen. This approach, theoretically grounded in maximizing mutual information, is shown to be highly effective and efficient, enhancing prompt fidelity for both image generation and text-guided editing across various models like Stable Diffusion 1.5, XL, and 3, with negligible additional computational overhead.", "questions": "1. To better isolate the contribution of the contrastive loss from the soft token architecture, have you considered applying the same loss function with an alternative parameter-efficient tuning method like LoRA?\n2. How does it perform on prompts requiring complex compositional understanding, such as intricate spatial relationships (e.g., \"a red cube on top of a blue sphere to the left of a green pyramid\") or abstract concepts?\n3. The paper positions SoftREPA as an alternative to preference-tuning methods like DPO [1] and DRTune [2]. Could these two approaches be complementary? \n4. Did the authors explore the impact of different negative sampling strategies, such as using \"hard negatives\" (e.g., pairing an image with a very similar but incorrect text prompt) versus the current batch-based random sampling?\n\n[1] Wallace B, Dang M, Rafailov R, et al. Diffusion model alignment using direct preference optimization[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2024: 8228-8238.\n[2] Wu X, Hao Y, Zhang M, et al. Deep reward supervisions for tuning text-to-image diffusion models[C]//European Conference on Computer Vision. Cham: Springer Nature Switzerland, 2024: 108-124.", "limitations": "Yes.", "rating": 3, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 2, "significance": 1, "originality": 2, "strengths_and_weaknesses": "Strengths:\n1. The approach demonstrates significant and broadly generalizable performance gains across various major diffusion models and tasks.\n2. Its effectiveness is validated through comprehensive experiments using a wide range of metrics, including human preference scores and detailed ablation studies.\n3. The core concepts are presented with exceptional clarity, aided by intuitive figures that simplify complex mechanisms for the reader.\n\nWeaknesses:\n1. The main weakness is that the two components are not strongly related. The author should first test fine-tuning with LORA to better illustrate the usefulness of contrastive loss, as the soft tokens are relatively independent.\n2. This method may slightly reduce perceptual image quality metrics like FID and LPIPS because it prioritizes stricter adherence to text.\n3. It can introduce new failure modes, such as decreased counting accuracy, by over-interpreting alignment cues in text prompts.\n4. The title \"Easier Than You Think\" is somewhat exaggerated, as implementing the method still requires a significant contrastive training setup on large datasets.", "was_revised": true, "final_justification": "According to their rebuttal, the results for both components do not significantly outperform the baselines. Consequently, diffusion-DPO + LoRA yields results similar to those in this paper, which greatly diminishes its value. Additionally, it omits a method I mentioned earlier—DRTune—which represents gradient backpropagation instead of DPO that only utilizes value signals.\n\nMore importantly, the main issue is that the two methods presented in this paper are isolated and lack a clear connection.", "rebuttal": "We sincerely thank the reviewer for their thoughtful and constructive feedback. The comment helped us gain deeper insight and refine the contribution of our work. Below, we address the concern in detail.\n\n---\n### **W1/Q1: Isolating the Contribution of Contrastive Loss**\nWe appreciate the reviewer’s suggestion to examine the effect of contrastive loss independent of the soft token architecture. In response, we conducted an ablation study using **LoRA** as an alternative in place of soft tokens. Specifically, we applied LoRA to the top 5 transformer blocks, which we found to be more effective than applying LoRA across all layers.\n\nAs shown in **Table 1**, the combination of *Soft Tokens + Contrastive Loss* consistently outperforms other variants across metrics such as *ImageReward*, *CLIP*, *HPS*, and *FID*, and performs comparably on *PickScore* and *LPIPS*. This result highlights that both *soft tokens* and *contrastive learning* contribute independently and positively to text-image alignment.\n\nMoreover, these findings support the underlying motivation of SoftREPA: to explore the text feature space to better text-image representation alignment. While LoRA modifies the attention projection layers in both the text and image encoders, this direct parameter modification may lead to suboptimal image generation, as evidenced by a higher *FID* score (70.33) compared to our soft token training approach (60.76).\n\n**Table 1. Quantitative comparison of parameter-efficient tuning methods**  \n| Method                         | ImageReward | PickScore |  CLIP  |  HPS  |  FID  |  LPIPS  |\n|-------------------------------|--------------|-----------|--------|-------|--------|---------|\n| SD3 (base)                    | 90.89        | 22.50     | 26.26  | 27.97 | **56.87**  | **42.47**   |\n| LoRA + Contrastive Loss       | 97.72        | **22.57** | 26.42  | 28.44 | 70.33  | 42.71   |\n| Soft Token + Contrastive Loss | **106.32**| 22.49|**26.92**|**28.70**| 60.76 | 42.99   |\n\n---  \n### **W2: Drop in FID and LPIPS** \nWe acknowledge that our method may affect FID and LPIPS, as it emphasizes stricter text-image alignment. However, these metrics have limitations in evaluating generation quality alone—FID is influenced by both diversity and quality, while LPIPS is more suited to measure similarity between a real image and a generated image on the given text, rather than assessing image quality. \n\nInstead,  other metrics such as ImageReward, PickScore, and HPS  reflect the image quality aligned with human preferences. These metrics consistently demonstrate that our method maintains perceptual quality while improving text-image alignment.\n\n---\n### **W3Q4: Counting Accuracy and Hard Negatives**  \nWe thank the reviewer for suggesting alternative approaches to improve counting accuracy. To address this, we trained SoftREPA with hard negatives added to each sample. Specifically, for each COCO caption, we generated three hard negative examples focusing on the following aspects: object count, color, and position. Using GPT-4.1-mini, we crafted prompts to minimally modify the original caption by altering the number of objects (counting), the color of an object (color), or the position of an object (position). During training, for each training sample, these hard negatives were incorporated alongside other negative samples. \n\nAs shown in ***Table 2***, we did not observe a significant improvement from adding hard negatives to the training data. This may be due to the limited number of hard negatives, as we generated only one altered caption per attribute. We hope to conduct further experiments on the impact of hard negatives in future work.\n\n**Table 2: Effect of Incorporating Hard Negatives into Our Method**\n| Method                        | Mean | Single | Two | Counting | Colors | Position | Color Attr.|\n|-------------------------------|----------|--------|----------|-----------|-----------|------------|---------------|\n| SD3                              | 0.68  |  0.99  | 0.86    | **0.56**     | 0.85     |   0.27     |    0.55      |\n| Ours                            | **0.70**  |  **1.00**   | **0.95**    | 0.29     | **0.92**     |  **0.34**     |   **0.68**      |\n| Ours+Hardneg          | 0.68  |  **1.00**   | 0.86    | **0.56**     | 0.84     |   0.28     |   0.53      |\n\n\n\n---\n### **W4: Title May Be Overstated**  \nWe will tone down and  revise it to better reflect the scope of our method.\n\n---\n### **Q2: Complex Compositional Understanding**  \nWe thank the reviewer for this important question regarding compositional understanding. We address this capability primarily in the Editing tasks. To evaluate this aspect, we used LLaVA [1] to generate long, detailed text prompts for images in the DIV2K dataset. These prompts contain nuanced edits involving spatial relationships, attribute modifications, and object substitutions, requiring complex compositional reasoning. \n\nBelow, we present two representative prompts we used.  In each case, the target prompt modifies multiple attributes and compositional elements from the original source prompt, such as changing object positions, attributes (e.g., material, color), or introducing new objects in specific spatial contexts.\n\nIn the **DIV2K Editing** benchmark, our model with trained soft tokens consistently outperforms the base model on metrics that evaluate background preservation and alignment with the target prompt (refer to Table 2 in the paper). These results suggest that the incorporation of soft tokens enhances the model’s ability to reason over complex, multi-object compositions and produce more faithful edits.\n\n--- \n**Example 1**: The image features a vase filled with a variety of flowers, including pink, white, and red ones. The vase is placed on **{a table, and it is a beautiful blue color $\\rightarrow$  a wooden pedestal, and it is a rich, dark brown color**}. The flowers are arranged in a bouquet, creating an eye-catching display. The combination of the vibrant colors and the elegant *blue vase* makes the scene visually appealing.\n\n**Example 2**: The image features a room with a wooden wall and a wooden table. On the wall, there are several framed pictures, including a picture of **{a man wearing a suit $\\rightarrow$ a woman wearing a dress}**. The table is covered with a variety of items, such as books, a vase, and a bottle. There is also a **{jacket hanging $\\rightarrow$ a guitar hanging}** on the wall, adding to the room's decor. The room appears to be a combination of a living space and a workspace, with the wooden wall and table providing a cozy and functional atmosphere.\n\n---\n**References**  \n[1] Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. Advances in neural information processing systems, 36:34892–34916, 2023.  \n\n---\n### **Q3: Diffusion RL + SoftREPA Complementarity**  \nWe thank the reviewer for this insightful question regarding the potential complementarity between SoftREPA and existing preference-tuning methods. We agree that these approaches are not mutually exclusive, and we explored this direction with additional experiments. To investigate whether SoftREPA can be combined with diffusion-RL methods, we conducted experiments using two representative Diffusion RL frameworks: Diffusion-DPO [1], which performs preference optimization, and DDPO [2], which utilizes policy optimization based on an external reward (e.g., BERTScore [3]).\n\nTo test complementarity, we adopted a **plug-and-play** approach: we integrated pretrained soft tokens from SoftREPA into the inference pipelines of these diffusion-RL models **without further training**.\n\nAs shown in **Table 3**, the addition of SoftREPA's soft tokens leads to notable performance gains in both cases. *Diffusion-DPO + Soft Tokens* shows improvements in *ImageReward*, *CLIP*, *FID*, and *LPIPS*, and remains comparable in *PickScore* and *HPS*. *DDPO + Soft Tokens* yields consistent improvements across all metrics, indicating strong synergy between policy-based optimization and our contrastive alignment objective.\n\nThese results suggest that SoftREPA and Diffusion-RL methods are complementary, while Diffusion-DPO and DDPO aim to align outputs using preference or reward supervision, SoftREPA enhances internal representation alignment through contrastive learning. Their combination allows for leveraging strengths from both strategies—efficient text-image alignment and explicit preference feedback—to further improve generation quality.\n\n**Table 3. Complementarity with Diffusion RL methods**  \n| Method                                   | ImageReward | PickScore |  CLIP  |  HPS  |  FID  |  LPIPS  |\n|----------------------------------------|--------------------|----------------|----------|---------|---------|-----------|\n| Diffusion-DPO[1]                    | 22.26          | **21.68** | 26.51  | **25.60** | 54.19 | 43.96    |\n| Diffusion-DPO[1] + Ours        | **33.10**  | 21.64 | **27.41**  | 25.55 | **53.72** | **43.64** |\n| DDPO[2]                                 | -8.83               | 21.20         | 26.21  | 23.45 | 54.83 | 43.85 |\n| DDPO[2] + Ours                     | **8.07** |**21.25**|**27.13**|**23.75**|**54.18**| **43.59** |\n\n**References**  \n[1] Wallace, Bram, et al. \"Diffusion model alignment using direct preference optimization.\" Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2024.  \n[2] Black, Kevin, et al. \"Training diffusion models with reinforcement learning.\" arXiv preprint arXiv:2305.13301 (2023).  \n[3] Zhang, Tianyi, et al. \"Bertscore: Evaluating text generation with bert.\" arXiv preprint arXiv:1904.09675 (2019)."}, {"review_id": "hHL8oJYzxI", "reviewer": "Reviewer_2H9G", "summary": "This work takes a different approach by focusing on representation alignment, inspired by the success of REPA. The authors propose SoftREPA, a lightweight contrastive fine-tuning method that uses soft text tokens to improve alignment with minimal computational cost (less than 1M additional parameters). Theoretical analysis shows it enhances mutual information between text and image features, leading to better semantic consistency. Experiments confirm its effectiveness in both text-to-image generation and editing tasks.", "questions": "1. The logic chain from SoftREPA to Soft Tokens is not clearly demonstrated in the paper. Why use Soft Tokens instead of other finetuning techniques like LoRA? Furthermore, LoRA does not put more burden upon time cost by merging them into the model.\nPlease also refer to Weaknesses part.", "limitations": "Please refer to Weaknesses part.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths:\n1. The proposed SoftREPA is a lightweight contrastive fine-tuning method that introduces fewer than 1M additional trainable parameters, making it highly efficient and easy to integrate with existing pretrained models.\n2. The method is backed by a solid theoretical analysis showing that it explicitly increases mutual information between text and image representations, which leads to improved semantic consistency.\n3. The approach is evaluated on both text-to-image generation and text-guided image editing tasks, demonstrating consistent improvements in semantic alignment and task performance across multiple benchmarks.\n\nWeaknesses:\n1. The paper lacks sufficient comparison between the proposed Soft Tokens and other fine-tuning techniques like LoRA. Otherwise, the authors should provide more explanations on why Soft Tokens are more suitable for the settings.\n2. The qualitative and quantitative results seem to be contradictory with each other. For example, according to the Table 1, the performance of SD3 in Counting is better than SD3 + Ours. However, the conclusion is not supported by the results of 'Three zebras' in Figure 3.", "was_revised": true, "final_justification": "The author well clarified the relationship between Soft Tokens and SoftREPA and provided results between LoRA and Soft Tokens. After considering the other reviewers’ remarks, I believe that the inclusion of LoRA only partially diminishes the paper’s overall significance. Accordingly, I have raised my rating to 4.", "rebuttal": "### **W1Q1: Comparison with LoRA and Motivation for Soft Tokens**  \nWe appreciate the reviewer giving us the opportunity to clarify the motivation for soft tokens. The motivation of SoftREPA comes from REPA [1], which shows that aligning the internal representations of DiT with an external pre-trained visual encoder during training significantly improves both discriminative and generative performance. In this paper, we aim to enhance text-image alignment in  **pretrained** text-to-image (T2I) generative models, leveraging these ideas to align DiT’s internal representations through contrastive learning, which is a core principle of representation learning.\n\nThe architectural motivation to use soft tokens is primarily inspired by CoOP [2], as stated in our related works. CoOP proposed prompt learning methods for CLIP-like vision-language models that replace the fixed prompt “a photo of a [CLS]” with learned soft prompts that better distinguish between a predefined set of [CLS]. This implies that we can learn an additional feature space that maximizes the discrepancy between general texts, which further enhances alignment with images. While CoOP learns tokens only in the prompt space, we expand this into the feature space within the model and remove the constraint of a predefined class set. This enables more flexible and implicit text-image representation alignment.\n\nTo evaluate the effectiveness of our design, we further investigated whether contrastive learning alone could yield similar benefits. For this, we incorporated *LoRA* as an alternative tuning strategy in place of soft tokens. LoRA was applied to the top 5 transformer layers, which we found to be a more stable and effective configuration than applying it to the full model.\n\nAs presented in **Table 1**, the combination of *Soft Tokens + Contrastive Loss* consistently achieves stronger performance across key metrics such as *ImageReward*, *CLIP*, *HPS*, and *FID*, while remaining competitive on *PickScore* and *LPIPS*. These results affirm that both soft tokens and contrastive learning contribute independently and meaningfully to enhancing text-image alignment.\n\n**Table 1. Quantitative comparison of parameter-efficient tuning methods**  \n| Method                        | ImageReward | PickScore |  CLIP  |  HPS  |  FID  |  LPIPS  |\n|-------------------------------|--------------------|----------------|----------|---------|---------|-----------|\n| SD3                              | 90.89               | 22.50         | 26.26  | 27.97 | **56.87** |**42.47** |\n| LoRA + Contrastive loss | 97.72             | **22.57**    | 26.42  | 28.44 | 70.33 | 42.71 |\n| Soft token + Contrastive loss | **106.32** |22.49 | **26.92**  | **28.70** | 60.76 | 42.99 |\n\n**References**  \n[1] ​​Yu, Sihyun, et al. \"Representation alignment for generation: Training diffusion transformers is easier than you think.\" arXiv preprint arXiv:2410.06940 (2024).  \n[2] Kaiyang Zhou, Jingkang Yang, Chen Change Loy, and Ziwei Liu. Learning to prompt for vision-language models. International Journal of Computer Vision, 130(9):2337–2348, 2022.\n\n---\n### **W2: Qualitative vs Quantitative Results Contradictory**   \nWe thank the reviewer for pointing out this discrepancy. In Figure 3,  the original motivation to include the example of \"Three zebras\" was to qualitatively illustrate improvements in **image quality and fidelity**, rather than to make claims about counting ability. While the SD3 baseline struggles to generate complete and realistic zebras, our method produces more coherent and visually accurate outputs, including improved background details. To avoid confusion, we will replace this with other example in the final paper."}, {"review_id": "j8JpeFJm3x", "reviewer": "Reviewer_hkeq", "summary": "In this paper, the authors approach the task of improving text image alignment in generative models. The paper proposes a method termed SoftREPA that attempts to align the features using a contrastive objective. \nMore concretely, the authors introduce “soft text tokens” that are concatenated to the regular text tokens, and fine tuned using a contrastive objective, where the positive pair corresponds to the aligned text image pair, and the negative samples correspond to unaligned text-image pairs. The soft text tokens vary at different layers and different denoising steps, and are the only learnable parameter, the rest of the model is kept frozen. \nThe authors also show that minimizing this contrastive objective resembles maximizing the mutual information between the text and image pairs.", "questions": "My questions for the authors mostly stem from the weaknesses i listed above. \n- could the authors comment on the need for / importance of the soft text tokens.\n- How is the LPIPS metric implemented for measuring the quality of text-to-image generation?", "limitations": "yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n- The contrastive loss to align the features is well motivated and the idea of using learnable soft text tokens is interesting and unique, that I have not seen in prior papers. \n- The method proposed in this paper is quite general, and the authors demonstrate this by applying it to many different base model architectures (SD1.5, SDXL, SD3), and two tasks (text-to-image and image editing). \n\nWeaknesses:\n- Importance of the soft text tokens. It is unclear to me whether the improvement shown in the paper is primarily due to the contrastive objective or because of the use of soft text tokens. It is unclear if you would get similar, better, or worse results if you simply finetune the generative models with a small LoRA. \n\n- Evaluation of text-to-image generation. I have a couple of doubts based on the quantitative results shown in table 1. First is that the authors only compare to the pre-trained non-finetuned model for evaluating the human preference, text alignment, and image quality. Even in this limited comparison, it seems like the base model has higher image quality. \nSecond, how is LPIPS used to evaluate the image quality? LPIPS is a network that is used to compare two images. It is unclear to me how this can be used to evaluate the quality of a text-to-image model. Perhaps the authors can add some implementation details about this metric. \n\n- The authors discuss several other related works that attempt this task of improving the text-image alignment (cfg++, attend-and-excite, DOODL, DiffusionDPO, …). However the authors do not compare to any of these prior works in Table 1. It would be useful to see how the proposed method compares to other methods in the literature.", "was_revised": true, "final_justification": "My initial concerns about the importance of the soft text tokens was addressed and therefore i keep my original score of borderline accept.", "rebuttal": "### **W1/Q1: Necessity and Role of Soft Text Tokens**  \n\nWe appreciate the reviewer’s question regarding the specific contribution of soft text tokens apart from the contrastive learning objective. To investigate this, we performed an ablation study using **LoRA** as an alternative parameter-efficient tuning method in place of soft tokens. We applied LoRA to the top 5 transformer layers, as this configuration proved more effective than applying LoRA across all layers.\n\nAs shown in **Table 1**, the combination of *Soft Tokens + Contrastive Loss* consistently achieves stronger results across multiple metrics, including *ImageReward*, *CLIP*, *HPS*, and *FID*, and performs comparably on *PickScore* and *LPIPS*. This demonstrates that both the architectural design of **soft tokens and the contrastive loss independently contribute to improved text-image alignment**.\n\nIn addition, we would like to clarify the role and **motivation behind the soft text tokens**. Our architectural design is primarily inspired by CoOP [1], which proposed a prompt learning strategy for CLIP-based vision-language models that replaces the fixed prompt (“a photo of a [CLS]”) with learned soft prompts capable of distinguishing among a set of predefined classes. This suggests that prompt-tuning enables the model to explore richer semantic embeddings. While CoOP focuses on learning prompts in the input space, we generalize this idea by propagating learned soft tokens through the network, allowing the model to align text and image representations in the internal feature space without relying on predefined class labels.\n\nThis architectural choice enables a more flexible and implicit alignment mechanism, encouraging the model to maximize the mutual information between text and image embeddings by refining the internal feature dynamics of the model.\n\n**Table 1. Quantitative comparison of parameter-efficient tuning methods**  \n| Method                        | ImageReward | PickScore |  CLIP  |  HPS  |  FID  |  LPIPS  |\n|-------------------------------|--------------------|----------------|----------|---------|---------|-----------|\n| SD3                              | 90.89               | 22.50         | 26.26  | 27.97 | **56.87** | **42.47** |\n| LoRA + Contrastive loss | 97.72             | **22.57**    | 26.42  | 28.44 | 70.33 | 42.71 |\n| Soft token + Contrastive loss | **106.32** |22.49 | **26.92**  | **28.70** | 60.76 | 42.99 |\n\n**References**  \n[1] Kaiyang Zhou, Jingkang Yang, Chen Change Loy, and Ziwei Liu. Learning to prompt for vision-language models. International Journal of Computer Vision, 130(9):2337–2348, 2022.\n\n---\n### **W2Q2: Clarification on LPIPS and Image Quality**  \nWe appreciate the reviewer’s concern regarding the use of LPIPS and other image quality metrics. In our study, we computed the LPIPS metric to measure the perceptual distance between the generated image and the real image corresponding to the given prompt. While LPIPS can indicate the similarity between the target and source images in editing tasks, we initially believed it could also reflect image quality in generation tasks. However,  other metrics such as ImageReward, PickScore, and HPS, which effectively reflect image quality,  demonstrate that our method preserves image quality while improving text-image alignment. \n\n---\n### **W3: Lack of Comparison with Alignment-Improving Baselines**  \n\nWe would like to note that our approach is grounded in contrastive learning, which shares a deep theoretical connection with recent preference optimization methods in diffusion models. As highlighted in recent work [1], both Reinforcement Learning from Human Feedback (RLHF) and Direct Preference Optimization (DPO) can be reinterpreted as forms of mutual information maximization—a foundational principle also underlying contrastive learning. Motivated by this connection, we compare our method, SoftREPA, with the diffusion-RL-based methods RankDPO [2] and CaPO [3], which are built on SD3 backbone. Notably, CaPO has been reported to outperform Diffusion-DPO [4], which we cited in our related work.\n\nIn addition, we include CFG++ [5], a training-free approach that improves generation quality by refining the classifier-free guidance (CFG) step. Although CFG++ is implemented on SD1.5 and SDXL, we follow the official guidance provided in their public repository to ensure a consistent comparison.\n\nTo evaluate alignment quality, we report results on the GenEval benchmark. As shown in **Table 2**, our method performs competitively overall and demonstrates superior results in several categories, including *Single*, *Two*, *Colors*, *Position*, and *Color Attribute*. These results support that SoftREPA offers a robust and effective framework for improving text-image alignment compared to both preference-optimized and training-free baselines.\n\n\n**Table 2. GenEval benchmark comparing related works**  \n| Method              | Mean | Single | Two  | Counting | Colors | Position | Color Attr. |\n|---------------------|-------|--------|------|----------|--------|----------|--------------|\n| SD3                 | 0.68  | 0.99   | 0.86 | 0.56     | 0.85   | 0.27     | 0.55         |\n| CFG++ [5] (w=0.6)   | 0.64  | 0.98   | 0.79 | 0.51     | 0.83   | 0.21     | 0.52         |\n| CFG++ [5] (w=0.8)   | 0.67  | 0.98   | 0.83 | 0.53     | 0.84   | 0.26     | 0.58         |\n| RankDPO [2]         | 0.74  | **1.00** | 0.90 | **0.72** | 0.87   | 0.31     | 0.66         |\n| CaPO [3]            | 0.71  | 0.99   | 0.87 | 0.63     | 0.86   | 0.31     | 0.59         |\n| **Ours**            | 0.70  | **1.00** | **0.95** | 0.29     | **0.92** | **0.34** | **0.68**     |\n\n---\n\n**References**  \n[1] Lv, Xufei, et al. *The Hidden Link Between RLHF and Contrastive Learning.* arXiv preprint arXiv:2506.22578 (2025).   \n[2] Karthik, Shyamgopal, et al. *Scalable Ranked Preference Optimization for Text-to-Image Generation.* arXiv preprint arXiv:2410.18013 (2024).   \n[3] Lee, Kyungmin, et al. *Calibrated Multi-Preference Optimization for Aligning Diffusion Models.* CVPR 2025.  \n[4] Wallace, Bram, et al. *Diffusion Model Alignment Using Direct Preference Optimization.* CVPR 2024.  \n[5] Chung, Hyungjin, et al. *CFG++: Manifold-Constrained Classifier-Free Guidance for Diffusion Models.* arXiv preprint arXiv:2406.08070 (2024)."}, {"review_id": "Jon82728lI", "reviewer": "Reviewer_v3Q6", "summary": "The paper introduces SoftREPA; a novel and lightweight fine-tuning method for improving t2i alignment in pre-trained t2i diffusion models. The method freezes the pre-trained models and introduces a small number of learnable \"soft tokens\". These learnable tokens are optimized with contrastive loss that leverages the denoising objective to better differentiate between positive and negative text-image pairs. The authors provide theoretical analysis by showing that this approach is related to maximizing the mutual information between the text and image pairs. The authors demonstrated empirically an enhanced semantic consistency for both image generation and editing tasks.", "questions": "As discussed in the weaknesses above.\n\n1. How would you choose a finetuning dataset especially that Table 1. shows significant drop in counting.\n2. How would you better compare this method to other relevant methods? For instance, the author mentioned several training-free techniques (like CFG++) which could be potential candidates to compare to.", "limitations": "Yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths\n\n* Pluggable light-weight module to pretrained t2i models which can used to improve t2i alignment with light-weight finetuning.\n* Repurposing the flow matching network output for contrastive learning.\n* Provide convincing theoretical analysis that SoftREPA is implicitly maximizes the mutual information between text-image pairs.\n\nWeaknesses\n\n* The paper didn't discuss how the choice of positive/negative pairs will impact the learning, not how to select a finetuning dataset.\n* The paper didn't provide convincing comparisons to relavant techniques, instead they compared to RankDPO which is orthogonal method.\n* Table 1. shows significant drop in counting metric.", "was_revised": false, "final_justification": "", "rebuttal": "---\n### **W1W3Q1: Lack of Discussion on Positive/Negative Pair Selection, Counting Drop, and Dataset Choice**  \nPer reviewer's request, we conducted an additional experiment by training SoftREPA with hard negative samples. For each COCO caption, we automatically generated three hard negatives using GPT4.1-mini, by minimally modifying the original caption with respect to: **Counting** (e.g., altering the number of objects), **Color** (e.g., changing the color of one object), **Position** (e.g., modifying spatial arrangement). During training, for each training sample, these hard negatives were incorporated alongside other negative samples used in our original method. \n\nAs shown in ***Table 1***, we did not observe a significant improvement from adding hard negatives to the training data. This may be due to the limited number of hard negatives, as we generated only one altered caption per attribute. We hope to conduct further experiments on the impact of hard negatives in future work.\n\n**Table 1: Effect of Incorporating Hard Negatives into Our Method**\n| Method                        | Mean | Single | Two | Counting | Colors | Position | Color Attr.|\n|-------------------------------|----------|--------|----------|-----------|-----------|------------|---------------|\n| SD3                              | 0.68  |  0.99  | 0.86    | **0.56**     | 0.85     |   0.27     |    0.55      |\n| Ours                            | **0.70**  |  **1.00**   | **0.95**    | 0.29     | **0.92**     |   **0.34**     |   **0.68**      |\n| Ours+Hardneg          | 0.68  |  **1.00**   | 0.86    | **0.56**     | 0.84     |   0.28     |   0.53      |\n\n\n---\n### **W2Q2: Comparisons to Relevant Methods**  \nTo provide a more comprehensive comparison, we conducted additional experiments covering both **(1) parameter-efficient fine-tuning methods** and **(2) approaches aimed at improving text-to-image alignment**.\n\n**(1)** In terms of lightweight tuning strategies, we compare *Soft Tokens* with *LoRA*, applied to the top 5 transformer layers for optimal performance. As shown in **Table 2**, the combination of *Soft Tokens + Contrastive Loss* consistently outperforms LoRA-based tuning across multiple metrics, including *ImageReward*, *CLIP*, *HPS*, and *FID*, while performing competitively on *PickScore* and *LPIPS*. This confirms the effectiveness of our architectural design in conjunction with contrastive learning.\n\n**(2)** Regarding alignment-oriented approaches, we would like to clarify that contrastive learning is strongly relevant to Reinforcement Learning from Human Feedback(RLHF) and Direct Preference Optimization (DPO) frameworks [1]. Recent work [1] shows that both RLHF and DPO can be reframed as contrastive learning objectives via mutual information maximization. Since our approach shares the underlying principle, we compare SoftREPA against RankDPO [2] and additionally CaPO [3] in **Table 3**. CaPO [3] is a recently proposed diffusion-RL method built upon SD3, which is reported to outperform Diffusion-DPO [4], as noted in our related works.\n\nAdditionally, we include CFG++ [5], **a training-free technique** that improves generation quality by refining the classifier-free guidance step. Since CFG++ is originally implemented on SD1.5 and SDXL, we followed the official implementation details provided in their repository.\n\nWe evaluate all models on the GenEval benchmark. As presented in **Table 3**, our method outperforms baselines on several key categories, including *Single*, *Two*, *Colors*, *Position*, and *Color Attribute*, confirming that SoftREPA not only provides an effective tuning method but also achieves strong performance in challenging alignment settings.\n\n---\n**Table 2. Quantitative comparison of parameter-efficient tuning methods**  \n\n| Method                        | ImageReward | PickScore |  CLIP  |  HPS  |  FID  |  LPIPS  |\n|-------------------------------|--------------------|----------------|----------|---------|---------|-----------|\n| SD3                              | 90.89               | 22.50         | 26.26  | 27.97 | **56.87** | **42.47** |\n| LoRA + Contrastive loss | 97.72             | **22.57**    | 26.42  | 28.44 | 70.33 | 42.71 |\n| Soft token + Contrastive loss | **106.32** |22.49 | **26.92**  | **28.70** | 60.76 | 42.99 |\n---\n**Table 3. GenEval benchmark comparing related works**  \n| Method                        | Mean | Single | Two | Counting | Colors | Position | Color Attr.|\n|-------------------------------|----------|--------|----------|-----------|-----------|------------|---------------|\n| SD3                              | 0.68  |  0.99  | 0.86    | 0.56     | 0.85     |   0.27     |    0.55      |\n| CFG++[5]  (w=0.6)      | 0.64  |  0.98   | 0.79    | 0.51     | 0.83     |   0.21     |   0.52      |\n| CFG++[5]  (w=0.8)       | 0.67  |  0.98   | 0.83    | 0.53     | 0.84     |   0.26     |   0.58      |\n| RankDPO[2]                 | 0.74  |**1.00**  | 0.90    | **0.72** | 0.87     |   0.31     |   0.66  |\n| CaPO[3]                      | 0.71  |  0.99  | 0.87    | 0.63     | 0.86     |   0.31     |    0.59      |\n| Ours                            | 0.70  | **1.00** | **0.95** | 0.29  | **0.92**| **0.34** | **0.68** |\n\n---\n**References**  \n[1] Lv, Xufei, et al. \"The Hidden Link Between RLHF and Contrastive Learning.\" arXiv preprint arXiv:2506.22578 (2025).  \n[2] Karthik, Shyamgopal, et al. \"Scalable ranked preference optimization for text-to-image generation.\" arXiv preprint arXiv:2410.18013 (2024).  \n[3] Lee, Kyungmin, et al. \"Calibrated multi-preference optimization for aligning diffusion models.\" Proceedings of the Computer Vision and Pattern Recognition Conference. 2025.  \n[4]Wallace, Bram, et al. \"Diffusion model alignment using direct preference optimization.\" Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2024.  \n[5] Chung, Hyungjin, et al. \"Cfg++: Manifold-constrained classifier free guidance for diffusion models.\" arXiv preprint arXiv:2406.08070 (2024)."}, {"review_id": "SKCRqQlZq0", "reviewer": "Reviewer_F1wd", "summary": "This paper tries to improve the text-image alignment in diffusion-based T2I model. Following REPresentation Alignment(REPA), the authors introduce contrastive representation loss to diffusion model training. The authors define the similarity measure using the the variation of the denoising score matching loss and introduce extra soft tokens at each layers. The experimental results show improvement of the finetuned model over the baseline model.", "questions": "1. According to equation 10 and 11, the similarity measure is defined as the variance of denoising score matching loss. As an expectation calculation is included in the equation, do the authors estimate the expectation with Monte Carlo method? If so, how many samples are needed for this estimation for one text-image pair? And will the number of samples (using in Monte Carlo estimation) influenced the performance? When doing the estimation, what is the distribution for timestep sampling?\n\n2. What is the \"time scheduling parameter\" $\\tau(t)$ used in the paper? Are the experimental results sensitive to the choice of $\\tau(t)$?\n\n3. What data source do the author use to train the model?", "limitations": "Yes, the authors have adequately addressed the limitations and potential negative societal impact of their work.", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "***Strengths***\n\nThe idea of introducing contrastive representation loss makes senses. And the experimental results suggest improvement over the baseline model. \n\n\n***Weaknesses***\n\na. The main idea of introducing contrastive representation directly follow REPresentation Alignment(REPA) [1]. And implementation-wise,  introducing and optimizing some special text token are also not new in diffusion model (e.g. [2][3]) \n\nb. The design of the similarity measure is new for this paper. However, I have some concerns regarding this design choice:\n\n1. The original REPA [1] work introduces the pretrained self-supervised visual representations to calculate the similarity. Yet this work purely relies on the diffusion model itself to define the similarity. Using pretrained self-supervised visual representations make more sense to me as the pretrained model brings extra knowledge during the training. It might be better if the authors can experimentally compare their design with the one using pretrained self-supervised visual representations (i.e. original REPA).\n\n2. Also, the authors mainly compare their finetuned model with the unfinetuned baseline model. I think it is worth adding an ablation which keeps the structure design of the soft-token, but finetune the model with the original diffusion loss and same data. This mainly eliminates the possibility that the improvement comes from continue training on high quality data and can demonstrate that contrastive learning really plays an important role here.\n\n3. This design introducing several new hyper-parameters, including the number of soft tokens, the time scheduling parameter, the layer to include the soft tokens. Also, in appendix B, it seems that for different models, one might need to decide whether to include original denoising loss. These make me concern about the generalizability of the proposed design.\n\n[1] Representation Alignment for Generation: Training Diffusion Transformers Is Easier Than You Think\n[2] Null-text Inversion for Editing Real Images using Guided Diffusion Models\n[3] DreamBooth: Fine Tuning Text-to-Image Diffusion Models for Subject-Driven Generation", "was_revised": true, "final_justification": "The rebuttal has addressed my concerns. I would like to increase my rating to 4.", "rebuttal": "### **Wa: Novelty over REPA and Prompt Optimization**  \nWe thank the reviewer for the opportunity to clarify our contributions. While inspired by REPA [1], our method differs in both objective and application. REPA aims to accelerate training by aligning DiT features with external discriminative representations.  In contrast, our work targets text-image alignment by implicitly aligning internal representations of two different domains through contrastive learning.\n\nThere are several works from prompt optimization. CoOP [2] introduces learnable soft prompts for CLIP models at the input level by replacing fixed prompt templates (e.g., “a photo of a [CLS]”) with learnable embeddings that help differentiate between predefined class labels. However, our approach generalizes this idea by adopting soft tokens within the model's internal layers, independent of predefined classes. This extension enables fine-grained, layer-wise modulation of semantic alignment signals.\n\nTo our knowledge, this is the **first attempt to optimize soft tokens in diffusion models by modulating internal features beyond input embeddings**. Our results show that these tokens significantly improve generative quality conditioning on given text by influencing feature propagation.\n\nTo validate this design, we compare with LoRA applied to the top 5 layers, which outperforms full-layer tuning. As shown in **Table 1**, our method—*Soft Tokens + Contrastive Loss*—achieves the best performance on *ImageReward*, *CLIP*, *HPS*, and *FID*, while remaining competitive on *PickScore* and *LPIPS*, demonstrating the complementary strength of our architectural and training choices.\n\n**Table 1. Quantitative comparison of parameter-efficient tuning methods**  \n| Method                        | ImageReward | PickScore |  CLIP  |  HPS  |  FID  |  LPIPS  |\n|-------------------------------|--------------------|----------------|----------|---------|---------|-----------|\n| SD3                              | 90.89               | 22.50         | 26.26  | 27.97 | **56.87** | **42.47** |\n| LoRA + Contrastive loss | 97.72             | **22.57**    | 26.42  | 28.44 | 70.33 | 42.71 |\n| Soft token + Contrastive loss (Ours) | **106.32** |22.49 | **26.92**  | **28.70** | 60.76 | 42.99 |\n\n**References**  \n[1] Yu, Sihyun, et al. \"Representation alignment for generation: Training diffusion transformers is easier than you think.\" arXiv preprint arXiv:2410.06940 (2024).  \n  [2] Kaiyang Zhou, Jingkang Yang, Chen Change Loy, and Ziwei Liu. Learning to prompt for vision-language models. International Journal of Computer Vision, 130(9):2337–2348, 2022.\n\n---\n### **Wb1: Compare the design with the original REPA**  \nWe sincerely thank the reviewer for the thoughtful suggestion. We agree that pretrained self-supervised visual encoders provide powerful external knowledge, and we appreciate the insight into comparing with such representations as explored in REPA [1]. However, we would like to highlight a key distinction in the **training objectives** and **representation targets** between REPA and our proposed method.\n\nREPA aims to accelerate **pretraining stage** by explicitly aligning the internal features of the diffusion model (DiT) with external discriminative features obtained from a pretrained visual encoder. This is feasible because the internal generative features and the external discriminative features exhibit natural similarity in the image domain.\n\nIn contrast, SoftREPA is designed to enhance text-image alignment of **pretrained models** by introducing learnable soft text tokens that explore a richer textual feature space within the model. As the reviewer pointed out, our method operates entirely within the latent space of the diffusion model itself, relying solely on the model’s internal priors. Since the soft tokens are applied to textual features, there is no inherently sufficient alignment with visual representations from external encoders, making direct supervision from such representations less applicable in our setting.\n\nThat said, we believe the reviewer’s suggestion opens up an interesting future direction. It may be promising to explore soft image token tuning using pretrained visual features, which could potentially enhance the image representation space in a complementary way. While this is beyond the scope of our current work, we are excited by this possibility. Once again, we thank the reviewer for the thoughtful suggestion.\n\n---\n### **Wb2: Finetuning Baseline Without Contrastive Loss**  \nWe thank the reviewer for raising this insightful point, which directly addresses the contribution of soft tokens and the contrastive learning objective. To further clarify their individual roles, we conducted an additional ablation study as suggested.\n\nTo assess whether the observed improvements stem from continued training on high-quality data rather than the contrastive loss itself, we performed a new ablation where the model retains the soft token architecture but is fine-tuned using the original denoising score matching (DSM) loss on the same dataset. This setup isolates the effect of contrastive learning from soft tokens. As shown in **Table 2**, this variant consistently underperforms compared to our full method, confirming that the gains are not merely due to extended training.\n\n**Table 2. Quantitative comparison of extra DSM training**  \n| Method                        | ImageReward | PickScore |  CLIP  |  HPS  |  FID  |  LPIPS  |\n|-------------------------------|--------------------|----------------|----------|---------|---------|-----------|\n| SD3                              | 90.89               | **22.50**         | 26.26  | 27.97 | **56.87** | **42.47** |\n| Soft token + DSM loss  | 86.98               | 22.39        | 26.29  | 27.78 | 70.39 | 42.65 |\n| Ours | **106.32** |22.49 | **26.92**  | **28.70** | 60.76 | 42.99 |\n\n---\n### **Wb3: Concerns About Hyperparameters and Generalization**  \n\nWe would like to clarify that although new parameters are introduced, we observed **consistent trends** across different models (SD3, SD1.5, SDXL). For instance:\n- In terms of layer placement, all models showed best performance when soft tokens were applied only to earlier blocks (SD3) or down blocks of UNet(SD1.5, SDXL).\n- The time scheduling parameter was kept constant across all models, and we did not observe high sensitivity to its value.\n- The optimal number of soft tokens for SD3 lies between 4 and 8, and this range also covers the optimal settings for SD1.5 and SDXL, which show similarly strong performance across a broader range.\n\nThese consistent tendencies suggest that our method behaves robustly across architectures, requiring minimal model-specific tuning—similar to other generation techniques like classifier-free guidance (CFG).\n\nFurthermore, to evaluate the generalizability of SoftREPA, we conducted experiments on SD1.5 and SDXL by training only with the contrastive loss, **omitting the denoising score matching (DSM)** term entirely. As shown in **Table 3**, even without DSM, the model performs competitively, further supporting that our contrastive learning framework is effective and broadly applicable.\n\n**Table 3. Quantitative comparison on the generalizability of SoftREPA**\n| model | loss term  | ImageReward | PickScore |  CLIP  |  HPS  |  FID  |  LPIPS  |\n|----------|---------------|--------------------|----------------|----------|---------|---------|-----------|\n| SD1.5    | baseline                  | 17.72 | 21.47   | 26.45  | 25.08 | 24.59 | 43.80 | \n| SD1.5    | DSM + Contrastive | **32.89** | 21.50  | **27.33**  | 25.18 | **23.43** | **43.38** | \n| SD1.5    | Contrastive             | 32.63 | **21.61**    | 27.10  | **25.94** | 29.25 | 44.09 | \n\n---\n| model | loss term  | ImageReward | PickScore |  CLIP  |  HPS  |  FID  |  LPIPS  |\n|----------|---------------|--------------------|----------------|----------|---------|---------|-----------|\n| SDXL    | baseline                  | 75.06 | 22.38    | 26.76  | 27.35 | **24.69** | **42.05** | \n| SDXL    | DSM + Contrastive | **85.29** | **22.62**    | 26.80  | 28.30 | 26.04 | 42.39 | \n| SDXL    | Contrastive             | 81.09 | 22.59    | **26.87**  | **28.32** | 26.42 | 42.69 | \n\n---\n### **Q1: Expectation Estimation via Monte Carlo**  \nAs described in Section 3.2 of the paper, we approximate the expectation in the logit computation l(x,y) using a single Monte Carlo sample for efficiency. To further stabilize training, we use the same sampled timestep and noise across all pairs within a minibatch when computing the loss.\nDuring training, the timestep ttt is sampled from a uniform discrete distribution over [0,999], which is a standard choice in diffusion-based models. During inference, we use a fixed sequence of timesteps determined by the scheduler: DPM-Solver++ for SD3, and DDIM for SD1.5 and SDXL.\nWe did not observe significant performance degradation with a single sample in practice, though increasing the number of samples could potentially improve stability at the cost of computational overhead.\n\n---\n### **Q2: Time Scheduling Parameter $\\tau(t)$**   \nThe scheduling parameter $\\tau(t)$ used in constructing the SoftREPA logit is set to a constant value of 0.07, which serves as a temperature parameter to scale the logits. This value is commonly used in contrastive learning literature and was found to work well empirically. We did not observe strong sensitivity to this hyperparameter in our experiments.\n\n---\n### **Q3: Data Source for Training**  \n We used the COCO dataset to train the soft tokens, as described in Section 4 (Text-to-Image Generation) of the paper."}]

Author rebuttal:

Meta-review: This paper introduces SoftREPA, a novel method to improve text-to-image alignment in diffusion models via a contrastive learning setup that trains a set of learnable soft tokens. The approach is theoretically well grounded and can work effectively across multiple existing models. Authors have clarified most of the questions raised by the reviewers, in particular comparison of the method against LoRA. I recommend acceptance for this work.
Decision: Accept (poster)