Improving Long-Text Alignment for Text-to-Image Diffusion Models
Authors: Luping Liu
Chao Du
Tianyu Pang
Zehan Wang
Chongxuan Li
Dong Xu
Venue: ICLR 2025 Poster (2025)
Keywords: Long Text Alignment
Diffusion Models
Preference Optimization
Text-to-Image Generation
Primary area: generative models
TL;DR: This paper presents a segment-level encoding method and a decomposed preference optimization method to enhance the alignment of T2I diffusion models with long text inputs, outperforming existing models.

Abstract: The rapid advancement of text-to-image (T2I) diffusion models has enabled them to generate unprecedented results from given texts. However, as text inputs become longer, existing encoding methods like CLIP face limitations, and aligning the generated images with long texts becomes challenging. To tackle these issues, we propose LongAlign, which includes a segment-level encoding method for processing long texts and a decomposed preference optimization method for effective alignment training. For segment-level encoding, long texts are divided into multiple segments and processed separately. This method overcomes the maximum input length limits of pretrained encoding models. For preference optimization, we provide decomposed CLIP-based preference models to fine-tune diffusion models. Specifically, to utilize CLIP-based preference models for T2I alignment, we delve into their scoring mechanisms and find that the preference scores can be decomposed into two components: a text-relevant part that measures T2I alignment and a text-irrelevant part that assesses other visual aspects of human preference. Additionally, we find that the text-irrelevant part contributes to a common overfitting problem during fine-tuning. To address this, we propose a reweighting strategy that assigns different weights to these two components, thereby reducing overfitting and enhancing alignment. After fine-tuning $512 \\times 512$ Stable Diffusion (SD) v1.5 for about 20 hours using our method, the fine-tuned SD outperforms stronger foundation models in T2I alignment, such as PixArt-$\\alpha$ and Kandinsky v2.2. The code is available at https://github.com/luping-liu/LongAlign.

Published as a conference paper at ICLR 2025
IMPROVING LONG-TEXI ALIGNMENT FOR TEXT-TO-IMAGE DIFFUSION MODELS
Luping Liu Chao 2u Tianyu Pang". Zehan Wang Chongxuan Li Dong Xu 'The University of Hong Kong: "Sea Al Lab, Singapore; RRemmin University of China;
*Zhejiang University; BBejjing Key Laboratory Big Data Management and Analysis Methods luping. .orng....ue liu@connect hku.hk; duchaoEsea. com; tianyupangésoa com; wangzehan01 jzju. eew..edu.cco....mm en; chengxuanlif edu. en; dongxuGhku.hi
ABSTRACT
The rapid advancement of text-to-image (T2I) diffusion models has enabled them to generate unprecedented results from given texts. However, as text inputs be- come longer, existing encoding methods like CLIP face limitations, and aligning the generated images with long texts becomes challenging. To tackle these issues, we propose LongAlign, which includes segment- encoding method for pro- cessing long texts and decomposed preference optimization method for effective alignment training. For segment-level encoding, long texts are divided into multi- ple segments and processed separately. This method overcomes the maximum in- put length limits of pretrained encoding models. For preference optimization, we provide decomposed CLIP-based preference models to fine-tune diffusion models. Specifically, to utilize CLIP-based preference models for T2I alignment, we delve into their scoring mechanisms and find that the preference scores can be decom- posed into two components: a text-relevant part that measures T2I alignment and text-irrelevant part that assesses other visual aspects of human preference. Ad- ditionally, we find that the text-irrelevant part contributes to a common overfitting problem during fine-tuning. To address this, propose reweighting strategy that assigns different weights to these two components, thereby reducing overfit- ting and enhancing alignment. After fine-tuning 512 x 512 Stable Diffusion (SD) v1.5 for about 20 hours using our method, the fine-tuned SD outperforms stronger foundation models in T2I alignment, such PixArt-a and Kandinsky v2.2. The code is available at https//gitub..com/luigg www.//pinggiiuu/oogglliiiii liu/LongAlign.
INTRODUCTION
Recent advancements in diffusion models (Sohl-Dickstein et al., 2015; Ho et al., 2020; Song et al., 2021a;b) have significantly enhanced text-to-image (T2I) generation (Schuhmann et al., 2022; Rom- bach et al., 2022; Saharia et al., 2022; Ramesh et al., 2022). While text-based conditioning provides flexibility and eser-friendliness, current models struggle with long and complex text descriptions that often span multiple sentences and hundreds of tokens (Chen al., 2023a; Zheng et al., 2024). Effectively encoding such lengthy text conditions and ensuring precise alignment between text and generated images remains critical challenge for generative models.
To encode text descriptions, contrastive pretraining encoders such as CLIP (Radford et al., 2021) are widely used in T21 diffusion models. However, text length increases, the maximum token limit of CLIP becomes a significant constraint, making it infeasible to encode long descriptions. Recent works have explored large language model (LLM)-based encoders like T5 (Raffel al., 2020), leveraging their ability to handle longer sequences (Saharia et al., 2022: Chen et al., 2023a; Hu et al., 2024). Nevertheless, contrastive pretraining encoders retain a key advantage: their text encoders are specifically trained to align with images, potentially offering superior alignment between text representations and generated images (Saharia et al., 2022: Li et al., 2024b).
"Work done during Luping Liu and Zehan Wang's associate memberships at Sea Al Lab. *Corresponding authors.

---

Published as a conference paper at ICLR 2025
SD-1.3
PixArt-
Kandinsk--222
longSD (Ours)
Prompe: lh image exseease hut of Advocitz and profenienntll phaitgragh of grawsy fult "The dogh gor is direrd hnwarts the camura stitng oine of enggenmne to the image Choriil, the hos goougnedl lo pooride esentaal informatinn about the product a cloi and cincae mamner while slio erprawiinn the ingestaae safey.
beatlwom bostreen degs
fhost the los (2) afforeed black and whils des, niso sppeans (3)
V- V
Syetied Woodpskes, land krown for distective tlick and white
captired
is martee of feading The bid is (1) perched so wire hird feeder, whech a (2) iilled with a mis of beown and white tartheed, The feeder coperded from a the brasch.
prositing a portect egot the wandpeckerm sooes to mool lookgroand
of grece fallage seagosing woodpeciee peoens (raating
Spcies
Brvubaation: So
; ki V
Prompt: The image is digitll artw depisting fastirtical witeer cceay the censer the compositio (1) lange.
elecater andhway made while, cocrocecernd branchon archonar (2) adoemne with red berric and grees stillg arclongy
ground Mannketed in caick lanthoppe trees ore disted with seew, fber branches hanry with the sacw, Th evenal effer: is srers, water worddrladd. Evmibsation: of of PP 5 v
The
the
Prompe:
image prosenee
fae iirts, (2) desicted stades of red, green, and white. includs dremes, pares, and shows, augasing wide rage of cleching and secoummres avaiable se pueduac the bear wr agaisal bockdrap red and white stripe. adling louch abvertiennn. everal looou the arpoedic organinot Evalastion:
heart diled with
The
MENT KAEIIN
chopping mooe.
Figure 1: Generation results of our long Stable Diffusion and baselines. We highlight three key facts for each prompt and provide the evaluation results at the end. In each evaluation line, the four group results are arranged in order of model presentation, with S representing SD-1.5, and so on. Additionally, three / or maintain the order of the key facts corresponding to each prompt.
Beyond encoding, current T21 diffusion models struggle to accurately follow long-text descriptions, often generating images that only partially reflect the intended details, as demonstrated in Figure 1. Inspired by advances in aligning LLMs (Ouyang et al., 2022: Rafailov et al., 2024), one potential solution is preference optimization, which generates and utilizes preference feedback when only parts of a target are satisfied. Recent works have explored collecting human preferences from T2I users and leveraging them train preference models (Kirstain et al., 2023; Wu et al., 2023c;b), enabling preference optimization in T2I diffusion models. However, since these models are typi- cally fine-tuned from CLIIP, they face the same token limit constraints. Moreover, existing human preferences blend text alignment with visual factors such photorealism aesthetics, which only partially support the goal of accurately aligning long and detailed text descriptions.
In this paper, to support long-text inputs for the two scenarios mentioned, we explore segment-level encoding, which involves dividing lengthy texts into shorter segments (e.g., sentences), encoding each one separately, and then merging the results for subsequent tasks. The main challenge is ef- fectively combining these segment outputs to merge their diverse information without causing con- fusion. To address this, for text encoding in diffusion models, we opt for concatenating segment embeddings and explore optimal adjustments to the unintended repetition of special token embed- dings from different segments. For preference models, we implement a segment-level preference training loss alongside segment-level encoding, allowing preference models to handle long inputs while generating both segment-level scores and an overall average score.
To enable preference optimization for long-tex, text alignment, we analyze the scoring mechanisms of preference models and use these models (with segment-level encoding) to fine-tune T2I diffusion models. We find that the desired T21 alignment scores can be separated from general human prefer- ence scores. Specifically, preference scores can be divided into two components: a text-relevant part that assesses T2I alignment and text-inrelevant part that evaluates other factors (c.g.. aesthetics). In addition, we discover that the remaining text- -ext-irrelcvan part leads to a common overfitting issue (Wu et al., 2024) during fine-tuning, To mitigate this, we propose reweighting strategy that assigns different weights to these two components, which reduces overfitting and enhances alignment.
By integrating the methods mentioned above, we propose LongAlign. Our experiments show that p segment-level encoding and training enable preference models to effectively handle long-tex inputs and generate segment-level scores. Additionally, our preference decomposition method allows these
2

---

Published as a conference paper at ICLR 2025
models to produce T2I aligmment scores alongside general preference scores. After fine-tuning the 512 x 512 Stable Diffusion v1.5 (Rombach et al., 2022) using LongAlign for about 20 hours on 6 A100 GPUs, the obtained long Stable Diffusion (longSD) significantly improves alignment (see Figure 1), outperforming stronger foundation models in long-text alignment, such as PixArt-a (Chen al., 2023a) and Kandinsky v2.2 (Razzhigaev al., 2023). Our contributions are follows:
We propose segment-level encoding method that enables encoding models with limited input lengths effectively process long-text inputs.
We propose preference decomposition that enables preference models to produce T21 alignment scores alongside general preference, enhancing text alignment fine-tuning in generative models.
After about 20 hours of fine-tuning, longSD surpasses stronger foundation models in long-text alignment, demonstrating significant improvement potential beyond the model architecture.
2
BACKGROUND
We provide an overview of diffusion models and T2I models. Next, we discuss preference models for fine-tuning T2I diffusion models, followed by an introduction the reward fine-tuning process.
2.1 DIFFUSION MODEL
Diffusion models (Sohl-Dickstein et al., 2015; Ho et al., 2020; Song et al., 2021a;b) construct transformation from a Gaussian distribution to target data distribution through multi-step diffu- sion denoising process. Given a data distribution zo ((2n), the diffusion process satisfies:
Tt. 04 + Sie,
(1)
where e ~ (0, 1), 1 {0, I,... T). T is the maximum timestep, ař + 3 = and 3r controls the speed of adding noise. The loss function is typically defined as follows:
= ||e - ca + t)e.
(2)
where the model εạ aims to predict the noise added to the clean data 2o. Once ey is learned, according to DDIM (Song et al., 2021a), the denoising process from T to satisfies:
2 (xt (ico(st,t))/ar It-1 04-1
BP of (x1, + azc,
(3)
where FT 1), and x; is the predicted clean image timestep l. When o₁ 0, this is (8t-1/8) at / this is the DDPM denoising
the DDIM denoising process. When process. In this paper, we choose aa 0. The corresponding DDIM process be accelerated using numerical solvers such as PNDM (Liu et al., 2022) and DPM-Solver (Lu et al., 2022).
Stable Diffusion. Among different T2I diffusion models, Stable Diffusion (Rombach et al., 2022) plays a crucial role, which integrates a VAE (Kingma & Welling, 2013), a CLIP, and a diffusion model Eg. During training, the pretrained VAE compresses the image x into a latent z, while the pretrained CLIP model encodes the text prompt The diffusion model Eş then learns to fit this new distribution of latent variables z, conditioned on the text p. In the sampling process, the diffusion model co first generates the latent z based on the text prompts and then uses the VAE to decode the latent z to obtain the final image æ. For simplicity, we will omit the VAE in the following.
2.2 PREFERENCE MODEL
Preference optimization (Ouyang et al., 2022; Rafailov et al., 2024) has shown its effectiveness in aligning L.Mss with humans using preference feedback. To facilitate this for T2I diffusion models, prior work (Kirstain et al., 2023: Wu et al., 2023b) fine-tunes these models with T2I preference mod- els that evaluate human preferences for an image given a text prompt p. represented as R(ar, p)- We focus on preference models fine-tuned from pretrained CLIP models. To prepare the preference dataset for fine-tuning, prompts are paired with two generated images, and the preferred image is annotated. For I; preferred over z, (denoted as i > j). the preference training loss is:
exp(R(x:,)))
Lur = cxp(R(x,, p)) (xp(R((E,,))))
(4)
where R(x,1) p) Cx(x) Cp(p) is the dot product of the image embeddings Cx and the text embeddings Cp)). After training, the preference model can be used to evaluate preferences or fine- tune generative models. Similar CLIP, these preference models don't 't support long- text inputs.
3

---

Published as a conference paper at ICLR 2025
2.3 REWARD FINE-TUNING
When fine-tuning T21 diffusion models with the preference models mentioned above, previous works (Black et al., 2023; Fan et al., 2024) typically treat preference models as reward signals defined by L. = Ep
1 p). However, fine-tuning the generator eg of diffusion models us- ing these signals poses two challenges. First, backpropagating gradients through the entire itcration process is problematic. Second, overfitting is concern. Previous works (Prabhudesai et al., 2023; Clark et al., 2023) have employed gradient checkpointing and LoRA (Hu et al., 2021) to facilitate gradient backpropagation, but these methods skip consecutive time steps at the beginning or end of the diffusion iteration to accelerate computation, which inevitably introduces optimization bias.
Recently, DRTune (Wu et al.. 2024) shows a new method that allows for training on a uniform subset of all sampling steps. Specifically, when apply the preference model R( p) to calculate the reward signal L, 1 Eer , (Z p), the corresponding gradient is follows:
T-1 Σ (B.-1//0 -aaii-i---//saaaaa 3//aa) ). (S)
0b(1 Ery R(2g. E p)) (0); p)
DRTune truncates the gradient of all {sg(I:)) ). allowing it to optimize uniform portion of the remaining gradient ((ggc)) }.) In this paper, we also utilize this as our gradient back- propagation method. However, overfitting remains concern that cannot be fully addressed by early stopping alone, necessitating further analysis and additional solutions.
SEGMENT-LEVEL TEXT ENCODING
In this section, we introduce our segment-level encoding method, designed for the long-text encod- ing of both diffusion models and preference models. This approach divides the text into segments, encodes each one individually, and then merges the results. For diffusion models, we explore an em- bedding concatenation strategy during merging. For preference models, we present a segment-level loss alongside the new encoding to handle long inputs and generate detailed preference scores.
3.1 TEXT ENCODING OF DIFFUSION MODEL
For text encoding in diffusion models, contrastive pretraining encoders like CLIP (Rombach et al., 2022) are commonly used. However, the input text length increases, CLIP's maximum token limitation becomes a significant issue. As a result, recent works (Saharia et al., 2022; Chen et al., 2023a; Hu al., 2024) have shifted using LLMs like T5 instead of CLIP, overlooking CLIP's distinct advantage (Radford et al., 2021) in image-text alignment pretraining, To leverage CLIP's capabilities for long text, we introduce segment-level encoding. Our method divides the text into segments (e.g., sentences), encodes each one into embeddings like the original '21 diffusion models, and then merges these embeddings. Figure 9 illustrates this segment-level text encoding.
In the merging process, we initially use direct embedding concatenation, which results in poor gen- erated images (see Figure 10). This occurs because each segment includes special tokens such as <sot>, <eot>, and <pad> during individual encoding, leading the unintended repeated presence of their embeddings in the concatenated embedding. To address this, we conduct ablation experiments on whether keep, remove, replace special token embeddings. The final embedding excludes the <pad> embeddings and introduces a new unique <pad*> embedding to meet the target length. retains all <sot> embeddings while removing all <eot> embeddings, resulting in the format "<sot> Text1. <sot> Text2. <pad*>". More details about this experiment can be found Appendix A.
We can now both CLIP and T5 for long-text encoding in T2I diffusion models. The next chal- lenge is accurately representing all text segments in the generated images. To tackle this issue, we further fine-tune diffusion models with large-scale long texts paired with their corresponding images. We start with supervised training employing & loss. However, we observe a clear optimization limit during training, with even the best version falling short of perfection (see the longSD(S) column in Table 2). This problem prompts us to explore additional preference optimization for long-text align- ment alongside general supervised training, inspired by its success in L.MMs (Ouyang et al., 2022).
3.2 SEGMENT PREFERENCE MODEL
In the context of preference optimization in T2I diffusion models, previous studies (Kirstain et al., 2023; Wu al., 2023b) have employed CL.I--based human preference models to better align T21
4

---

Published as a conference paper at ICLR 2025
Inre:
Trat Enheldigs
Creaso Binoda
Cip Flct
1000
Doase
Frequac
740
909
14
02
250
5. 0.6
Projection
Figure 2: (a) Schematic results for text embeddings. (b) Statistics of the projection scalar v) for three models. (c) The relationship between the original score and the two scores after decomposition using our Denscore. In the three score tables, the diagonal represents the scores for paired data, while the off-diagonal positions indicate the scores for unpaired data.
diffusion models with human preferences. Since our objective- text alignment-iss a key component of human preference, and CLIP-basec structures effective for large-scale training, we aim to adapt such preference models for our long-text alignment task.
The first step towards achieving the above goal is enable CLIP- preference models to accept long-tex text inputs. As shown in Section 2.2, these models introduce a new human preference training objective into the CLIP framework. However, they still have the same limited maximum input length and may struggle to reflect the varying impacts different segments (e.g., sentences) with a single final score. To solve these problems, we split the long-text condition p into K segments, denoted as K {Dx [ūx]<= Then, the new segment-level preference training loss is:
exp(E R(x;,
seg
(6)
exp(E 1= R (x, Pr. K) + exp(E R(x,,.)/K)
The above loss enables weakly supervised learning (Zhou, 2018) during training, eliminating the need for additional segment-level annotations. In comparison its single-value counterpart, the new preference model supports long-text inputs and generates more detailed segment-level scores (R(x,, (&) h: along with an average final score Σ R(x, / = Cx(x) Σ Cr(pk)/K, K. Thus, computing this average score equivalent to first segment-level encoding the text input and using the average embedding. denoted as Op = Σæ= (px)) KV to compute the score. We refer this score as Denscore, which will also function reward signal in the following section. there is no confusion, we may omit the segment label in Cp3 (p).
4 PREFERENCE DECOMPOSITION
In this section, we explore preference optimization using the preference models mentioned ahove. We find that their preference scores comprise a text-relevant component and a text-irrelevant com- ponent, with the latter often causing overfitting in fine-tuning diffusion models. To address this, we propose a reweighting strategy for both components that reduces overfitting and enhances alignment.
4.1 ORTHOGONAL. DECOMPOSITION
As shown in Section 2.2, the CL.P--bsed preference model R(x,,) = Cx(x)) Cp(p) evaluates an image æ against text with respect to human preferences. Some preferences concern whether the text condition p is accurately represented in the image x, while others focus on visual factors such as photorealism and aesthetics, making them irrelevant to the text. We find a direct structural cor- respondence between these two types of preferences within the text embedding Cy(p). Specifically, different Cp(p) display common direction, as illustrated in Figure (a). This common direction corresponds to text-irrelevant preferences, while the remainder reflects text-relevant preferences.
To support this statement, we use text embeddings from large prompt dataset P to compute the common text-irrelevant direction: := PC ppppppppp (pp))//llpppPp(pIlll We then decompose the text embedding (p) into two orthogonal parts: (+()) + V,,, where = Cp(p) V the projection scalar of Cp onto V. For the value of n,, we test CLIP, Pickscore and our Denscore, presenting the results in Figure 2 (b). We observe that Cp exhibits a strong positive scalar projection onto V (n) >> 0.4 for CLIP and 17 >> 0.6 for the others), forming core in representation space. The presence of V is referred to as the cone effect (Gao et al., 2019; Liang et al., 2022), which results from both model initialization and contrastive training.
5

---

Published as a conference paper at ICLR 2025
High
ripott
Pickscore
Dunscore
Figure 3: Retrieval results with low or high text-irrelevant scores, using three CLIP-based models.
To better understand the differing roles of these two components in the preference score R(x, (), we experiment with a 5k image-text dataset. Figure (c) presents a score table illustrating the relationship between Cxx Cp(p). Cx(x)) Cp (p) and Cx) V7 A more detailed version of this figure and real data statistics for the three scores can be found in Appendix B.2. For the second score Cx (x) (p), our experiments in Section 5.2 and Appendix B.3 show that it eliminates the influence of text-irrelevant components and focuses on measuring T2I alignment, which aligns with our objective. We refer to this aspect of Denscore as Denscore-O.
For the third score Cx (7 = nn(xx (e) V), the scalar vj is analyzed in the above two paragraphs. The remaining text-irrelevant term Cx (x)- V still needs further clarification. We provide images with low or high text-irrelevant scores in Figure to visualize the score standard. Notably, the three mod- els assign low scores to images that share similar characteristics, marked by dull visuals and large blank spaces. Conversely, while the high-scoring images selected by CLIP lack distinct features, the two preference models show strong alignment. Their high-scoring images feature rich details, well-organized layouts, and overall higher quality. This highlights the role of training Equation 4, which provides a clear training objective-genett human preference- for text-irrelevant scores. In contrast, CLIP's original contrastive training does not incorporate this aspect.
4.2
REWIINT REWARD FINE-TUNING
To use the preference model (xx, p) as reward signal for fine-tuning T2I diffusion models, there are two main challenges. One challenge is backpropagating the gradient through the entire multi- step iteration process. This can be addressed by the method mentioned in Section 2.3. However, previous works still struggle with overfitting, where fine-tuned images exhibit similar patterns (see Figure 6). While these patterns may enhance reward scores, they often lead to unsatisfactory images. We find that this is primarily because the text-irrelevant parts V constitute a significant portion of the optimization direction. Specifically, we fine-tune the generator Eg using the reward signal L, = 1 Exy.p R.r..pRRa R(z p). with the gradient calculated as follows:
00 - Eer R(26, p)) -0ples x (Cp(p) Cx(x))) = En,
EVV( + B,,
where the item V Cp (p) controls the gradient direction of cg. According to Section 1, the text- irrelevant component V comprises large portion of the entire item, overwhelming the gradient and producing similar output patterns regardless of the text input p. To mitigate this overfitting problem, we fine-tune the generator eg using a reweighted gradient:
(8) where w is the reweighting factor for the common direction V. This addresses the overfitting prob- lem mentioned above. Additionally, this analysis clarifies why the original CLIP is ineffective for reward fine-tuning (see Figure 6). The issue arises because its text-irrelevant component V is well-trained (see Section 4),, resulting in an undefined optimization direction.
0g - Exy R( p)) xs - (V((((VVV) + (+)) Oo,)) (2z))),
By combining the methods from the sections above, arrive our complete LongAlign method. Detailed algorithms are available in Appendix A and C.
5 EXPERIMENT
In this section, we first provide our experimental setup, including models, training strategies, and evaluation metrics. We then present the results of our segment preference models. Next, detail

---

Published as a conference paper at ICLR 2025
Table 1: R@l results for 5k text-to-image retrieval using different CLIP-based models.
CLIP-H
HPSv2
Pickscore Single
Denscore
Single
Average
Single
Average
Average
Single
Average
Cr(p) Cp (p)
86.10 85,80
80.40
42.34
16.72 64.28
54.00
31.84 64.00
83.96 87.24
75.90
85.14
67.94
67.60
91.86
22. 0
25
TS-mlp
21.5
CUP-T5
Denscore-
30
21.0
29
20.5
28
21
27
CLI--cat
19.5-
Ts-mip
CLI--cat
20-
26
CUP+T5
19.0
CUP+T3
20k
12k
20k 24x 28k
Step
Step
Step
Figure 4: FID and Denscore results for diffusion models with different text encodings.
our long-text encoding results for diffusion models using various encoding methods. The advantages of gradient reweighting fine-tuning across different reward signals are then discussed. Finally, we present the generation results using our entire Long Align alongside those of the baselines.
5.1 EXPERIMENTAL SETUP
Model. Our experiments cover three types of models: text encoders, Unets, and preference models. Specifically, we utilize the pretrained CLIP and T5 models as our text encoders. To align the embed- ding dimensions, append two-layer MLP to T5's output, following La Vi-Bridge (Zhao al., 2024a). The Unet derived from the pretrained Stable Diffusion v1.5 Instead of full fine-tuning, fine-tune the Unets using LoRA with rank of 32 and update both the ResNet blocks and atten- tion blocks within the Unets. For the preference model, we select two pretrained models: Pickscore and HPSv2. Additionally, we introduce our new segment-level preference model, Denscore. All three models are fine-tuned from pretrained CLIP models.
Training. (1) For training the Unet, we utilize a dataset of approximately million images, including 500k from SAM (Kirillov et al., 2023), 100k from COC02017 (Lin et al., 2014), 500k from LLaVA (a subset the LAION/CC/SBU dataset), and million from JourneyDB (Sun et al., 2024). We randomly reserve 5k images for evaluation. All images recaptioned using LLaVA-NexI (Liu et al., 2023) or ShareCaptioner (Chen al., 2023b) and resized to 512 x 512 pixels. We optimize the model using the AdamW optimizer with learning rate of x 10 -5, 2k-step warmup, and total batch size of 192. Training is conducted on 6 A A100-40G GPUs for 30k steps over 12 hours. (2) For the reward fine-tuning (RFT) stage of the Unet, size of 96 and 4k total training steps over hours. (3) For training the segment preference model, the same settings for Pickscore (Kirstain et al., 2023), employing CLIP-H Pickscore's training data, along with LLaVA-Next captions and our new segment-level loss function. More details about training the preference model can be found in Appendix B.
the same settings before but with batch
Evaluation. We evaluate our methods using the FID, Denscore, Denscore-O and VQAscore (Lin al., 2024) metrics on the 5k-image evaluation dataset. FID evaluates the distribution distance between the dataset and generated images. Denscore assesses human preference for generated im- ages, while Denscore-O and VQAscore focuses on the text alignment of those images. Additionally, we employ GPT-4o (OpenAI, 2024) to evaluate 1 images against baselines, mitigating the risk of overfitting Denscore. The GPT-4o evaluation template can be found in Appendix All our experiments employ UniPC (Zhao et al., 2024b) with 25-step sampling, maintaining consistent classifier-free guidance factor as per the original papers. In Appendix C.3, we also evaluate our models two additional benchmarks, including DPG-Bench (Hu al., 2024), to assess our T2I generation performance on prompts of varying structures and lengths.
5.2 SEGMENT PREFERENCE MODEL
To demonstrate that our analysis of the preference model generally applicable, we compare four CLIP- models: the pretrained CLIP, the single-value preference models Pickscore and HPSv2,
httpps: / http::///hgggggaeeeee co/ stable-de fuus on-vl hts//ttaalledeeeee ffust on-

---

Published as a conference paper at ICLR 2025
Table 2: FID and Denscore results for 512 x 512 image generation using different models. PlayG
KanD-2.2 and ELLA are from Li et (2024a), Razzhigaev al. (2023) and Hu et al. (2024).
Model
SD-1.5
SD-2.1
PlayG-2
PixArt-a
2.2
ELLA
longSD (S)
longSD (S+R)
FID-5k
24.96
25.80
23.92
22.36
20.04
24.38
20.09
19.63/24.28
Denscore-O
29.20
30.15
28.80
33.48
33.30
32.92
31.29
32.83/35.26
Denscore
20.29
20.91
21.22
22.78
22.70
22.11
21.72
22.74/23.79
VQAscore
84.57
85.61
85.26
86.96
86.31
86.85
86.18
87.11/87.24
29
7.5k
7.5k
30-
7.5k
15k
28-
Denscore-o
26
32
26
22
22
0.0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
Weight
Weight
Wright
36
0.0
0.3
34
33
23
32
30-
31
22
30
1k
3k
7k
9k
11k
13k
15k
1k
36
5k
7k
9k
11k
13k
15k
1k
5k.
7k
9k
11k
13k
15k
Stop
Stop
Stop
Figure 5: FID and Denscore results for diffusion models using different gradient reweighting factors.
as well as our new segment-level preference model, Denscore. According to Section 4.1, the text- irrelevant embedding V reflects the text-irrelevant preference, while the text-relevant embedding Cp(p) reflects the T21 alignment, which is our objective.
To assess the ability of the text-relevant part Cp. (p). we provide the R@ retrieval accuracy for the four models the 5k evaluation dataset in Table Here, either utilize the embedding encoding from the first 77 tokens in a single-pass approach (Single) or take the average of segment-level embeddings (p) (Average). The final score is computed using the image embedding Cx (x) with
either the full text embedding Cp or solely the text-relevant part Cp+ (p). We find that: (1) Across all preference models, the text-relevant embedding outperforms the original embedding, as it climinates the impact of rext-irrelevant factors. (2) Only Denscore with segment-level training yields better segment-level retrieval accuracy, while the other models don't 't benefit from, and are confused by, the extra information provided by averaging multiple embeddings. (3) Although T2I alignment is only partial optimization objective in Denscore, the Denscore model outperforms CLIP. This highlights importance of segment-level encoding strategies and preference decomposition.
In Appendix B.3, to better evaluate Denscore's performance under varying text lengths, we employ different maximum sentence prompts to obtain R retrieval accuracy. Additionally, we conduct an experiment in the same appendix to identify specific misaligned segments in long inputs, showing that segment-level scoring provides more detailed information.
5.3 LONG-TEXT ENCODING
Here, we provide FID and Denscore results for supervised fine-tuning (SFT), comparing various text encoding methods: CLIP with concatenation (CLIP-cat). T5 with an additional two-layer MLP (T5-mlp)?, and combination of CLIP and T5 (CLIP+T5). The results in Figure show that CLIP- cat and T5-mlp perform similarly, while the CLIP+T5 model significantly outperforms them. This suggests that the CLIP+T5 model is preferable, as it leverages the strengths of both CLIP's image- text paired pretraining and T5's pure long-text encoding capabilities. This finding is consistent with
We utilize La Vi-ihiddge'ss (Zhao et al., 2024a) pretrained ML.,, which means only include the short-to- long fine-tuning.
8

---

Published as a conference paper at ICLR 2025
Origin
Reweight (ours)
CLIIP
HP8v2
Pickseore
Demeore
CLIP
HPSv2
Pickseore
Denseore
Figure 6: Generation results using different reward signals, with and without gradient reweighting. The corresponding text conditions can be found in Appendix F.
the results of previous works (Saharia et al., 2022; Li et al., 2024b), while these earlier models still face input length limitations due CLIP's maximum token count.
5.4 REWARD FINE-TUNING
To simplify and accelerate ablation studies of reward fine-tuning (RFT), we leverage both LCM- LoRA (Song et al., 2023; Luo et al., 2023) and DRTune (Wu et al., 2024) to speed up fine-tuning using only CLIP-cat encoding for optimal strategy identification in this subsection.
Weighting Factor. In Figure 5, we provide the FID and Denscore results for different gradient reweight factors and training steps. According to the first three experiments with different reweight factors, the FID results exhibit parabolic shape, with the last three experiments with different training steps, only w = 0.3 maintains relatively stable FID results while simultaneously improving both Denscore and Denscore-O results. In contrast, the other two options improve only one of the Denscore or Denscore-O metrics, accompanied by significant increase in FID, indicating apparent overfitting problem. Additionally, please note that the optimal value of w can vary depending on the model and training strategy used.
0.3 achieving the best FID results. In
provide visual results showing the benefits of gradient reweight- ing on reward signals from various preference models, including CLIP, HPSv2, Pickscore, and our Denscore. Notably, this method also partially addresses the limitation of using reward signals from pretrained CLIP. However, CLIP cannot yet outperform preference models because its text-irrelevant component V not well-trained, which aligns with our analysis Section 4.2.
Reward Signal. In Figure
5.5 GENERATION RESULT
Foundation Model. this subsection, we apply our
entire LongAlign approach to train long Stable Dif- fusion (longSD) and compare against other baselines. Here, present the results of SFT (S) at 28k steps and SFT+RFT (S+R) with w = 0.3, at 1 1.25k and 3.75k steps. As shown in Table 2, SFT+RFT clearly outperforms both the original SD-1.5 and the basic SFT version. Compared other advanced foundation models, our longSD model surpasses them in terms of FID and Denscore metrics. Furthermore, we use GPT-4o (OpenAI, 2024) evalu- ate 1k IongSD(S+RR) results, comparing them to baselines and mitigating the risk of overfitting. The new results (see Figure 7) consistent with previous scores. We also provide visualizations in Figure 8. All these findings highlight the effectiveness of our method for generating high-quality images from long texts, demonstrating significant potential beyond altering the model structure.
708
619
643
Loss
600
500
429
448
.36
408
362
300
208
100
SD-1.5
SD-2.1
PlayG-2
PisArta
Kunb-22
Figure 7: GPT-4o evaluation results of T21 alignment across different models.
Alignment Strategy. addition to varying model structures, there are other methods that improve T2I alignment. Some of these approaches incorporate the assistance of LLMs, such Ranni (Feng et al., 2024) and RPG-Diffusers (Yang et al., 2024), while others employ improved training strate- gies, such Paragraph-to-Image (P2I) diffusion (Wu et al., 2023a). L.LM- methods that

---

Published as a conference paper at ICLR 2025
SD-1.5
longSD (Ours)
Figure 8: The generation results show comparison of performance before and after our fine-tuning on SD-1.5. The corresponding text conditions are provided in Appendix F.
involve additional LLM assistance increase computational requirements and encounter significant out-of-distribution (OOD) issues with long-text inputs. More details on OOD issues can be found in Appendix E. Regarding improved training strategies, our approach is orthogonal to P2I diffusion, so we compare the original P2I diffusion with its fine-tuned version using method. The results in Appendix Table 11 also show clear improvements in terms of FID, Denscore, and GPT-4o.
6 RELATED WORK
Diffusion-based Generation. The popularity of diffusion models has surged, driven by break- throughs in fast sampling methods (Song et al., 2021a; Lu al., 2022: Zhang Chen, 2022) and text-conditioned generation (Saharia et al., 2022; Chen et al., 2023a). Moreover, cascaded and latent space models (Ho et al., 2022; Rombach al., 2022) have enabled the creation of high-resolution images. These advancements have also unlocked various applications beyond T21 generation, in- cluding the ability to create content with a consistent style (Hertz et al., 2023) and image editing tools (Hertz al., 2022; Brooks et al., 2023). Furthermore, advancements in foundation models like U-ViT (Bao et al., 2023) and DiT (Peebles & Xie, 2023) suggest even greater potential.
Text-to-Imase Evaluation. Traditional metrics, such as Inception Score (IS) (Salimans al., 2016) and Fréchet Inception Distance (FID) (Heusel et al., 2017), have limitations in evaluating the quality of T2I generation. To address these limitations, three approaches have emerged. One approach employs Perceptual Similarity Metrics like LPIPS (Zhang et al., 2018), which utilize pretrained models assess image similarity from human perspective. Another approach involves using detection models (Huang et al., 2023) to extract and analyze key object alignments. A third approach fine-tunes human preference models (Xu al., 2024; Kirstain et al., 2023; Wu al., 2023b) on datasets of human preferences for images generated from specific prompts, effectively turning them into proxies for human evaluation.
Reward Fine-funing Training text-to-inn tortttoiiggeeeee models with a reward signal can be effective in target- ing specific outcomes. Two primary approaches have emerged. One leverages reinforcement learn- ing to optimize rewards that are difficult to calculate using traditional methods, such as DPOK (Fan et al., 2024) and DDPO (Black et al., 2023). Another approach, exemplified by DiffusionCLIP (Kim et al., 2022), DRaFT (Clark et al., 2023) and AlignProp (Prabhudesai et al., 2023), involves back- propagation through sampling. This method leverages human preferences and other differentiable reward signals to optimize diffusion models for specific targets. Recently, DRTune (Wu al., 2024) further improves training speed by stopping the gradient of the diffusion model's ' input.
7 DISCUSSION
In this paper, we propose LongAlign to enhance long-text to image generation. We examine segment-level text encoding strategies for processing long-text inputs in both T2I diffusion mod- els and preference models. In addition, we enhance the role of preference models by analyzing their structure and decomposing them into text-relevant and text-irrelevant components. During reward fine-tuning, we propose gradient reweighting strategy to reduce overfitting and enhance alignment. In our experiments, we utilize the classical SD- 1.5 and effectively fine-tune it to outperform stronger foundation models, demonstrating significant potential beyond designing new model structures.
The limitation of this paper is that, while we have made improvements, we still do not fully address the generation of long prompts with complex contextual dependencies those requiring strong mantic understanding, partly due to CLIP's constraints. In the future, we will explore more powerful training strategies beyond using CL.P Cl.P--hased preference models.
10

---

Published as a conference paper at ICLR 2025
ACKNOWLEDGMENTS
This work is partially supported by the Hong Kong Research Grants Council (RGC) General Re- search Fund (17203023). the RGC Collaborative Research Fund (C5052-23G), the National Natural Science Foundation of China/RGC Collaborative Research Scheme (CRS HKU703/24), the Hong Kong Jockey Club Charities Trust under Grant 2022-0174, UBTECH Robotics funding, and The University of Hong Kong's Startup and Seed Funds for Basic Research for New Staff. Additionally, C. Li is supported by the National Natural Science Foundation of China (No. 924701 18), Beijing Natural Science Foundation (No. 1.247030), and Beijing Nova Program (No. 20230484416).
ETHICS STATEMENT
This research acknowledges the ethical implications of advancements in text-to-image (T2I) diffu- sion models. We commit to responsible practices by ensuring transparency in our methodologies and findings. Our segment-level encoding and preference optimization techniques are designed to enhance alignment while minimizing bias and overfitting.
We prioritize inclusivity and societal values by actively engaging with diverse stakeholders to under- stand the cultural impacts of our work. Our ongoing commitment to ethical research practices aims to ensure that our technologies contribute positively to society while mitigating potential harms.
REFERENCES
Fan Bao, Shen Nie, Kaiwen Xue. Yue Cao, Chongxuan Li, Hang Su, and Jun Zhu. All are worth words: A vit backbone for diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Patern Recognition, pp. 22669 22679,, 2023.
Kevin Black, Michael Janner, Yilun Du Ilya Kostrikov, and Sergey Levine. Training diffusion models with reinforcement learning. arXiv preprint arXiv:2305. htt:::3305.330000 13301, 2023.
Tim Brooks, Aleksander Holynski, and Alexei A Efros. Instructpix2pix: Learning to follow image editing instructions. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 18392-18402, 2023.
Junsong Chen, Jincheng Yu, Chongjian Ge. Lewei Yao, Enze Xie, Yue Wu, Zhongdao Wang, James Kwok, Ping Luo, Huchuan Lu, et al. Pixart-alpha: Fast training of diffusion transformer for photorealistic text-to-image synthesis. arXiv preprint arXiv:2310.00422, 2023a.
Lin Chen, Jisong Li, Xiaoyi Dong, Pan Zhang, Conghui He, Jiaqi Wang, Feng Zhao, and Dahua Lin. Sharegpt4v: Improving large multi-modal models with better captions. arXiv preprint arXiv:2311.12793, 2023b.
Kevin Clark, Paul Vicol, Kevin Swersky, and David Fleet. Directly fine-tuning diffusion models on differentiable rewards. arXiv preprint arXiv:2309. 2023.
Ying Fan, Olivia Watkins, Yuqing Du Hao Liu, Moonkyung Ryu, Craig Boutilier, Pieter Abbeel, Mohammad Ghavamzadeh, Kangwook Lee, and Kimin Lee. Reinforcement learning for fine- tuming text-to-image diffusion models. Advances in Neural Information Processing Systems, 36, 2024.
Yutong Feng, Biao Gong, Di Chen, Yujun Shen, Yu Liu, and Jingren Zhou. Ranni: Taming text-to- image diffusion for accurate instruction following. In Proceedings of the IEEE/CVE Conference Computer Vision and Pattern Recognition, pp. 4744-4751 2024.
Jun Gao, Di He, Xu Tan, Tao Qin, Liwei Wang, and Tie-Yan Liu. Representation degeneration problem in training natural language generation models. arXiv preprint
2019.
Amir Hertz, Ron Mokady, Jay Tenenbaum, Kfir Aberman, Yacl Pritch, and Daniel Cohen-Or. Prompt-to-promp image editing with cross attention control. arXiv preprint 22X1 2022.
Amir Hertz, Andrey Voynov, Shlomi Fruchter, and Daniel Cohen-Or. Style aligned image generation via shared attention. arXiv preprint arXiv:23/2.02133, 2023.

---

Published as a conference paper at ICLR 2025
Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to local nash equilibrium. Advances in neural information processing systems, 30, 2017.
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. In Advances in Neural Information Processing Systems, volume 33, pp. 6840-6851, 2020.
Jonathan Ho, Chitwan Saharia, William Chan, David Fleet, Mohammad Norouzi, and Tim Sali- mans. Cascaded diffusion models for high fidelity image generation. J. Mach. Learn. Res., 23 (47):1-33, 2022.
Edward Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang. and Weizhu Chen. Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685, 2021.
Xiwei Hu, Rui Wang, Yixiao Fang, Bin Fu, Pei Cheng, and Gang Yu. Ella: Equip diffusion models with Ilm for enhanced semantic alignment. arXiv preprint arXiv:2403.05/133 2024.
Kaiyi Huang, Kaiyue Sun, Enze Xie, Zhenguo Li, and Xihui Liu. T2i-compbench: A compre- hensive benchmark for open-world compositional text-to-image generation. Advances in Neural Information Processing Systems, 36:78723-78741 2023.
Gwanghyun Kim, Taesung Kwon, and Jong Chul Ye. Diffusionclip: Text-guided diffusion models for robust image manipulation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2426-2435, 2022.
Diederik Kingma and Max Welling. arXiv:1312.6114 14, 2013.
Auto-encoding variational
bayes. arXiv preprint
Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloc Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander Berg, Wan-Yen Lo, et al. Segment anything. In Proceed- ings of the IEEE/CVF International Conference Computer Vision, pp. 4015 2023.
Yuval Kirstain, Adam Polyak, Uriel Singer, Shahbuland Matiana, Joe Penna, and Omer Levy. Pick- a-pic: An open dataset of user preferences for text-to-image generation. Advances in Neural Information Processing Systems, 36, 2023.
Daiqing Li, Aleks Kamko, Ehsan Akhgari, Ali Sabet, Linmiao Xu, and Suhail Doshi. Playground v2. 5: Three insights towards enhancing aesthetic quality in text-to-image generation. arXiv preprint 4rX.v:2402.17245 2024a.
Zhimin Li. Jianwei Zhang, Qin Lin, Jiangfeng Xiong, Yanxin Long. Xinchi Deng, ... and Qinglin Lu. Hunyuan-dit: A powerful multi-resolution diffusion transformer with fine- ine-irai chinese understanding. arXiv preprint S33i::2405.08746 2024b.
Victor Weixin Liang, Yuhui Zhang, Yongchan Kwon, Serena Yeung, and James Y Zou. Mind the gap: Understanding the modality gap in multi-modal contrastive representation learning. Ad- vances Neural Information Processing Systems, 35:17612-17725, 2022.
Tsung-Yl Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In Computer Vision-ECCV 2014: 13th European Conference, Zurich, Switzerland, September 6-12, 2014, Proceedings, Part V 13, pp. 740-755, Springer, 2014.
Zhiqiu Lin, Deepak Pathak, Baiqi Li, Jiayao Li, Xide Xia, Graham Neubig, Pengchuan Zhang, and Deva Ramanan. Evaluating text-to-visual generation with image-to-text generation. In European Conference on Computer Vision, pp. 366-384. Springer, 2024.
Haotian Liu, Chunyuan Li. Yuheng Li, and Yong Jae Lee, Improved baselines with visual instruction tuning. arXiv preprint arXiv:2310.0:77 2023.
Luping Liu, Yi Ren, Zhijic Lin, and Zhou Zhao. Pseudo numerical methods for diffusion models on manifolds. In International Conference on Learning Representations, 2022.
12

---

Published as a conference paper at ICLR 2025
Cheng Lu, Yuhao Zhou, Fan Bao, Jianfei Chen, Chongxuan Li, and Jun Zhu. Dpm-solver: A fast ode solver for diffusion probabilistic model sampling in around 10 steps. In Advances in Neural Information Processing Systems, 2022.
Simian Luo, Yiqin Tan, Suraj Patil, Daniel Gu, Patrick Platen, Apolinário Passos, Longbo Huang, Jian Li, and Hang Zhao. Lem-lora: A universal stable-diffusion acceleration module. arXiv preprini arXiv:231 1.05556, 2023.
OpenAL. Gpt-4o system card, 2024.
URL
gpt-4o-systen-co
Long Ouyang. Jeffrey Wu, Xu Jiang. Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, al. Training language models fol- low instructions with human feedback. Advances in neural information processing systems, 35: 27730-27744, 2022.
William Peebles and Saining Xie. Scalable diffusion models with transformers. In Proceedings of the JEEF/CVF International Conference on Computer Vision, pp. 4195-4205, 2023.
Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhom, Jonas Müller, Joe Penna, and Robin Rombach. Sdxl: Improving latent diffusion models for high-resolution image synthesis. arXiv preprinl arXiv:2307.01953 2023.
Mihir Prabhudesai, Anirudh Goyal, Deepak Pathak, and Katerina Fragkiadaki. Aligning text-to- image diffusion models with reward backpropagation. arXiv preprint arXiv:2310.03739, 2023.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pp. 8748-8763, PMLR, 2021.
Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model is secretly a reward model. Advances in Neural Information Processing Systems, 36, 2024.
Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. Exploring the limits of transfer learning with a unified text-to-text transformer. The Journal of Machine Learning Research, (1 (1):5485-5551) 2020.
Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text- conditional image generation with clip latents. arXiv preprint arXiv:2204.06125, 2022.
Anton Razzhigaev, Arseniy Shakhmatov, Anastasia Maltseva, Vladimir Arkhipkin, Igor Pavlov, Ilya Ryabov, Angelina Kuts, Alexander Panchenko, Andrey Kuznetsov, and Denis Dimitrov. Kandin- sky: an improved text-to-image synthesis with image prior and latent diffusion. arXiv preprint arXiv:2310.03502, 2023.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. High- resolution image synthesis with latent diffusion models. Proceedings of the IEEE/VVF Con- ference Computer Vision and Pattern Recognition, pp. 10684-10695, 2022.
Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily L. Denton, Kamyar Ghasemipour, Raphael Gontijo Lopes, Bureu Karagol Ayan, Tim Salimans, et Photorealistic text-to-image diffusion models with deep language understanding. Advances in Neural Informa- tion Processing Systems, 35:36479-36494, 2022.
Tim Salimans, Ian Goodfellow, Wojciech Zaremba, Vicki Cheung, Alec Radford, and Xi Chen. Improved techniques for training gans. Advances in neural information processing systems, 29, 2016.
Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade W Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Wortsman, et al. Laion- 5b: An open large-scale dataset for training next generation image-text models. In Thirty-sixth Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2022.
13

---

Published as a conference paper at ICLR 2025
Jascha Sohl-Dickstein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsupervised learning using nonequilibrium thermodynamics. In International Conference on Machine Learn- ing. pp. 2256-2265, PMLR, 2015.
Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. In Interna- tional Conference on Learning Representations, 2021a.
Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. In Interna- tional Conference on Learning Represenlations, 2021b.
Yang Song, Prafulla Dhariwal, Mark Chen, and lya Sutskever. Consistency models. arXiv preprint arXiv:2303.01469, 2023.
Keqiang Sun, Junting Pan, Yuying Ge, Hao Li, Haodong Duan, Xiaoshi Wu, Renrui Zhang, Aojun Zhou, Zipeng Qin, Yi Wang, al. Journeydb: A benchmark for generative image understanding. Advances in Neural Information Processing Systems, 36, 2024.
Weijia Wu, Zhuang Li, Yefei He, Mike Zheng Shou, Chunhua Shen, Lele Cheng, Yan Li, Tingting Gao, Di Zhang, and Zhongyuan Wang. Paragraph-to-image generation with information-enrichel diffusion model. arXiv preprint arXiv:2311.1428: 2023a.
Xiaoshi Wu, Yiming Hao, Keqiang Sun, Yixiong Chen, Feng Zhu, Rui Zhao, and Hongsheng Li. Human preference score v2: A solid benchmark for evaluating human preferences of text-to- image synthesis. arXiv preprint (..iv::3006093444 2023b.
Xiaoshi Wu, Keqiang Sun, Feng Zhu, Rui Zhao, and Hongsheng Li. Human preference score: Better aligning ext-to-image models with human preference. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 2096-2105, 2023c.
Xiaoshi Wu, Yiming Hao, Manyuan Zhang, Keqiang Sun, Zhaoyang Huang, Guanglu Song, Yu Liu, and Hongsheng Li. Deep reward supervisions for tuning text-to-image diffusion models. arXiv preprint arXiv:2405.0077 2024.
Jiazheng Xu, Xiao Liu, Yuchen Wu, Yuxuan Tong, Qinkai Li, Ming Ding, Jie Tang, and Yuxiao Dong. Imagereward: Learning and evaluating human preferences for text-to-image generation. Advances in Neural Information Processing Systems, 36, 2024.
Ling Yang, Zhaochen Yu, Chenlin Meng, Minkai Xu, Stefano Ermon, and CUI Bin. Mastering text- to-image diffusion: Recaptioning, planning, and generating with multimodal 1lms. In Forty-first International Conference Machine Learning, 2024.
Qinsheng Zhang and Yongxin Chen. Fast sampling of diffusion models with exponential integrator. arXiv preprint arXiv:2204. /3902, 2022.
Richard Zhang, Phillip Isola, Alexci A Efros, Eli Shechtman, and Oliver Wang. The unreasonable effectiveness of deep features as a perceptual metric. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 586-595, 2018
Shihao Zhao, Shaozhe Hao, Bojia 7i, Huaizhe Xu, and Kwan- Yee K Wong. Bridging differ- ent language models and generative vision models for text-to-image generation. arXiv preprint arXiv:2403.07860, 2024a.
Wenliang Zhao, Lujia Bai, Yongming Rao, Jie Zhou, and Jiwen La. Unipe: unified predictor- corrector framework for fast sampling of diffusion models. Advances in Neural Information Processing Systems, 36, 2024b.
Wendi Zheng, Jiayan Teng. Zhuoyi Yang, Weihan Wang. Jidong Chen, Xiaotao Gu, Yuxiao Dong, Ming Ding, and Jie Tang. Cogview3: Finer and faster text-to-image generation via relay diffusion. arXiv preprint urXiv:2403.05/22 2024.
Zhi-Hua Zhou. brief introduction to weakly supervised learning. National science review, 5(1):
44 53, 2018.
14

---

Published as a conference paper at ICLR 2025
A CONCATENATION STRATEGY
For the CLIP encoder in T2I diffusion models, we first show the pipeline of our new segment- level encoding in Figure In this pipeline, segment-level encoding and concatenation may seem straightforward, but the optimal concatenation strategy remains unclear, as shown in Figure 10. This because each segment contains special tokens, such <sot>, <eot>, and <pad>, leading to their corresponding embeddings" unintended repeated presence in the final concatenated ennaaenndeedebeddinnnn embedding. This raises the question of whether to retain, remove, or replace them in the final embeddings.
For <pad> tokens in each segment, we omit them in the final embeddings since they lack alignment information. However, we sometimes need to introduce a new <pad> embedding to ensure aligned token sequence lengths. To address this, assign unique embedding to each required <pad> token's s position. Specifically, our unique <pad> embedding is the average value of all <pad> tokens of empty sentence, which denote <pad*>. For <sot> and <eot> tokens, we then experiment with pretrained diffusion models to find the optimal strategy. As shown in Figure 10, our results indicate that the optimal approach is to keep all <sot> token embeddings and remove all <eot> token embeddings. Therefore, our final embeddings used in this paper take the form "<sot> Textl. <sot> Text2... ... <pad*>"
Target length
Final Embeddings
texti
<
<>ot>
...
"pad
<pad
Concatemation Taken adjustment
Segment Output Embeddings
text1
conts spars
text2
cents Sparts
<sot> text3
pants
CLIP
CLIP
CLIP
Segment
Input
<cot>
text1
Sent>
spad>
<eat>
text2
Sent> <pad>>
<sot> text3
<ent> spad>
Tokens
CLIP's maximum length
Figure 9: The visualization of our new segment-leve text encoding for diffusion models is presented.
origin
Text
Text
Text
segment
multi segments
Figure 10: Generation results under different embedding concatenation strategies.
The last column of the experiment above shows that removing all <sot> tokens destroys the gen- erated results. In Figure 11, we also compare the effects of keeping only the first <sot> token (1-sot) versus retaining all <sot> tokens (ours). Additionally, we investigate different segmentation strategies by testing the difference between treating each sentence a segment (ours) and grouping several consecutive sentences into a segment (multi), provided their total token count remains under 77. We find that these two ablation studies do not show significant differences in the results.
15

---

Published as a conference paper at ICLR 2025
Ours
1-sot
multi
Figure 11: Generation results of SD-1.5 using various segment encoding strategies.
Furthermore, in Figure 12, we analyze the cross-attention map for both original and segment-level encodings. The results show similar interaction behaviors between them. When the input prompt labels objects (e.g., dog and duck, clock and pen) and references them across different segments, the model accurately aligns these objects across those segments. This demonstrates that T21 diffusion models with segment encoding can handle cross-sentence information.
dog
dog
duck
duck
duck 3
The scene captures joyful moment between spirited dog and its beloved plastic duck The dog lies on the grass, tail wagging furiously, as it playfully nudges the bright yellow duck with nose. duck slightly weathered but still vibrunt, rests nearby, This juxtaposition highlights their playful bond.
endless entertainment.
clock
clock 2
pen
pen 2
SD
Ours
The scene depicts serene moment between vintage alarm clock and sleek fountain pen 1. polished finish softly ticking hands, elegantly wooden desk, casting warm glow in the afternoon light. Beside it, the fountain pen 2, with its deep blue color and intricate design, rests poised as ready to capture thoughts and dreams.
Figure 12: The visualization of the generation results and cross-attention maps for both original and segment-level encodings using SD-1.5 and our fine-tuned version. The 1,2,3 behind the nouns are for readability and do not be included in the input prompt.
16

---

Published as a conference paper at ICLR 2025
B SEGMENT PREFERENCE MODEL
Here, we provide detailed analysis of our segment preference models with respect to training, visualization, and evaluation.
B.1 TRAINING
We train a segment preference model by incorporating long and detailed text conditions generated by LLaVA-Next alongside our segment-level preference loss function. Based on the analysis in Section 1, we choose to combine the segment-level loss for refining text-relevant aspects with the original loss for improving aspects unrelated to text, such as aesthetics. The loss function bxe (where "a" denotes addition) is:
K
Cr(())) - [(Cx(t))CGG(b)/k)) Co(())K)))
(9)
k=1
(Cx Cp(p) - (xz) Cp(p)),
where (a:) = Ite-= is the sigmoid function and e" Cp(ps) with (ps) help the new segment-level loss focus on the T2I alignment part and avoid influencing the text-irrelevant part. The new loss function - (where "o" means orthogonal) is:
tm o(m n). In addition, we substitute
C [E Σ (cx(m)))c(($6//1)))
Σ CCCx()))-C((bb)/
(10)
Expo (2x) Cp(p) - Cx (xz) Cp(p)),
where } represents the segments split from the long text generated by LLava-Next, and p is the original short text.
If prioritize alignment, C is the better choice; if want to balance both alignment and aestheties, - is the preferable option. This because Equation 10 eliminates the influence of the first loss item the text-imelevant part V, allowing it to better focus factors unrelated to the text, such as aesthetics. To support this, we present retrieval results with the highest scores used text-irrelevant components V trained with two different loss functions. The results from Equation 10 align more closely with human preferences. Therefore, in this paper, we choose Lizj train our new segment preference model.
segeo in Equation 10
Eq 10
JohnWetton
FRANK DOONETIONINAL THE BEIERIMS
HHUC
COTTOOII  COIANIAL MARINES
ALIENS
ADO ABOUP NOTHING
Eq
ODKERT
Figure 13: The retrieval results with the highest scores used text-irrelevant components V trained with two different loss functions.
B.2 VISUALIZATION OF SCORE
After training the new segment preference models, want to analyze the behavior of three scores Cx(x) Cp(p). Cx (x) Cp (p) and (x()) Vy We are interested the relationships among them, and additionally, in the differences in these scores between different CLIP-based models. Here, choose CLIP, Pickscore and our Denscore as our experimental targets, as they all use the same model structure, while the last
fine-tuned on Pickscore's human preference dataset. We present the
17

---

Published as a conference paper at ICLR 2025
results in Figures 14 and 15. The first figure uses 20 data pairs to visualize the relationships and differences, while the second figure displays the actual statistics based on 5,000 data pairs.
For the original score Cx)x). Cp(p). only the original CLIP behaves in such way that its score can be close to zero when the image and text imputs are unpaired, while the other two models still assign relatively high scores to such unpaired inputs. This is because the preference models assess the image not only based on text alignment but also on other purely visual factors, such as aesthetics.
For the text-relative score (x) Cp (p). after removing the influence of the text- ext-irrellvant part, all three models provide nearly zero scores for unpaired input data, which supports our analysis that this score focuses on the T2I alignment and explains why this score achieves the best retrieval results for preference models, as shown in Table 1.
For the third score Cx (,)) we have analyzed the scalar 77 in the main paper, which is determined entirely by the text input and is image-inrelevnt. Here, we focus on the text-irrelevant score Cx (z)- V. We find that this score is strongly positive for the two preference models. This corresponds to pure visual factors in preference, as shown in Figure 3. On the other hand, for the original CLIP, this score is nearly zero This indicates that the common direction of text embedding almost orthogonal to the image embeddings, which do not contribute to the final score of CLIP.
Iaage
1eaps C.
I
G
Tal C.
Best
03
03
CLIP
Pkkran
Durront
Figure 14: Logit results for different models, both before and after orthogonal decomposition.
Original Test Embeddings
Original Test Embeddings
Original Tixt Embeddings
Psired
Pained Unpained
Faind
698
Lingurred
600
Uipaind
Frequeocy
Frequrac
480
200
200-
200
Text-irdderat Embreddings
Test-iendovas Emsheddinge
Test-irrehvaol Embeddings
800
600
Unpaiced
Ungaized
Ungaire
Frequeeyy
Frequmay
200
200
200
ding
Embedding
Embrdding
Proquess
Feequarcy
400
400
200
208
200
Seare
Figure 15: The real data statistics for the diagonal paired data and the off-diagonal unpaired data.
18

---

Published as a conference paper at ICLR 2025
B.3 EVALUATION
To better support the above analysis, we design two additional experiments to test the effectiveness of our score alongside the experiment in Section 5.2.
Firstly, to evaluate Denscore's s performance under varying text lengths, we employ different maxi- mum sentence prompts to obtain @@l retrieval accuracy, shown Table 3. We find that our Den- score, using segment-level training, consistently outperforms the others, except in the one-sentence setting. In this one-sentence setting, our segment-level training becomes less meaningful, but our results still outperform other existing preference models.
Additionally, to identify specific misaligned segments in long inputs, we conduct an experiment, illustrated in Figure 16, to show that segment-level scoring provides more detailed information. In some cases, certain segments align better with the first image, while other segments align better with the second image. This makes the overall score relatively meaningless, while segment-level scores continue to perform well.
Table 3: R results for 5k text-to-image retrieval with varying maximum numbers of sentences.
max number
2
3
4
6
8
CLIP
53.06
70.90
76.70
79.62
83.00
84.
HPSv2
41.86
53.66
56.48
59.14
62.58
63.96
Pickscore
42.34
53.86
57.56
60.22
63.60
63.54
Denscore
52.72
72.70
78.78
83.10
88.16
89.94
C DECOMPOSED PREFERENCE OPTIMIZATION
C.I
PSEUDOCODE
Here, we provide the pseudocode in Algorithm 1 for the entire decomposed preference optimization pipeline discussed in this paper.
Algorithm Decomposed Preference Optimization for T2I Diffusion Models
Input: Long-text input T, Initial T2I diffusion model M, Preference model
2: Output: Fine-tuned T2I diffusion model M
3: S Segment(T) (Step 1: Divide T into segments (e.g., sentences) S {81, .2, 8n
4: for each segment 8, do
5: E; + Encode((s:) (Step 2: Encode segment s; (as shown in Section 3.1 and Figure 9)]
6: end for
7: E Concatenate E1, E2,.... F,,) (Step 3: Concatenate segment embeddings (as shown in
Section 3.1 and Figure 9)}
8: I (-M (Step 4: Generate image from embeddings E using the T2I diffusion model]
9: Einage; Exegment & R S) (Stpp 5: Compute the Preference Embeddings of segments S and
image (as shown in Section 3.2)
10: Eoverall f- Ei= Esegment (Compute overall average Embedding)
(Elextreleevatt Erexe-imelevant imelevant 7scaler) Decompose(E Evvenu)) (Step Decompose overall score into relevant and irrelevant components (as shown in Section 4.1))
Taos <- Finage Fiext relevant + Wimelevant Fimuge Thsaaer Faest inrelevant (Step 7: Calculate adjusted loss (as shown in Section 4.2))
13: M +- VineTuner Lines (Step 8: Fine-tune T2I model using computed loss (as shown in Section 2.3))
Return A7
19

---

Published as a conference paper at ICLR 2025
The image depicts large, omate building at night, illuminated by combination natural and artificial light. 296875,,,, (.976666]
The building has classical architectural style, with columns and arches, and is bathed in a warm, orange glow that highlights its facade.
[20.214844, 26.757812]
In front of the building, there well-maintained lawn with a path leading up to the entrance.
(10,498047, 15. 15.820312] The overall atmosphere of the image one
grandeur and mystety, enhanced the dramatic lighting and the fantastical elements in the sky. [22.949219, 17.578125]
The Image depicts dramatic and apocalyptic scene. (26.367188, 20.800781]
Al the cenler of the image, Ihere massive volcanic eruption with a bright orange and yellow lava flow cascading down the sides of a mountain.
[20,507812, 29. 20.16]))]]
The sky filled with dark clouds and smoke, suggesting a catestrophic event.
[22.65625, 20.800 800800781]
In the foreground, there is group of people standing on rocky outcropping. [20.11 88, 8.398438]
The image is digital artwork that festures stylized female figure with fantastical and ethereal appearance.
[28.101562, 562, 26.5625
The figure long, flowing baic that cascades down her back, intertwined with hat appears to be delicale, pink flowers.
[33, 789062, 31 88833388
The figure's face cbaracterized by pale complexion, Jarge, expressive eyes, and subtle, serune expression.
[20,70312) 125, 20.507812]
gaze is directed off to the side, giving the impression of contemplation or daydreaming.
Her lips are partad slightly,
(2) 679688, 1 18.945312]
The image is dominated by television screen mounted on the wall. (25.78125, 23.339844]
The screen is alive with the image of elderly woman, who is elegantly dressed In a white blouse and pearl necklace.
[23.632812, 24.316406]
The itself is dimly lit, creating atmosphere of tranquility.
[20. 800781, 14.257812]
The television, being the central object, drams the viewer's attention,
while the plant in the background adds depth the
[24.21675, 28.125]
The image footuros 3D rendering of robot charactor. (29. 10156,, 28.710938]
The robol has prodominanlly white body with gold accenls, including gold chest plate and gold accents on and legs.
[33.007812, 33. 20320312
The robot's oyes are large and round, with a blue light inside them. (20.800781, 22.558594]
The background of the image is plain, light gray color, providing neutral backdrup that contrasts with the robot's colorful design. (25.78125, 20.21 4844)
The image presents scene dominated by black leather sofa, which the central object in the frame. 127.14843,, 26.953125]
sofa, with its three cushions, is positioned against stark white background, creating striking contrast.
[29. 882812, 28.320312]
The cushions, like sofa, black appear to leather, suggesting uniform color scheme throughout the plece. [26.5625, 27.929688)
wooden frame, adding touch warnth to the otherwise monochrome setting.
The sofa is designed
[24.511719, 20.117188]
Figure 16: Identifying the best-aligned image and segment pairs using segment-level preference scores.
20

---

Published as a conference paper at ICLR 2025
C.2 OPTIMIZATION VISUALIZATION
In Figure 17, we present additional visualizations of the generation results with and without our reweighting strategies various ratios demonstrate the effectiveness of our method. A ratio of indicates the original loss, resulting in significant overfitting, where all images exhibit similar patterns regardless of the imputs, A ratio of O means that the loss only considers the text-relevant part, leading to low image quality that does not align with human preferences. We observe that ratio of 0.3 yields the best image quality. These experiments and the results Section 5.4 demonstrate that our reweighting strategy effectively reduces overfitting and improves alignment.
0.0
0.2
0.3
0.6
Figure 17: The generation results with and without our reweighting strategies at various reweighting ratios. When ratio I, means the original loss.
C.3 MORE EVALUATION
Here, conduct additional evaluations across various structures and prompt lengths.
For structural variation, we use DPG-Bench (Hu et al., 2024), which includes test prompts for cat- egories such as entity, attribute, relation, and count. The results in Table 4 show that our longSD model outperforms others.
For prompt length, evaluate generation results for prompt lengths of about N tokens, where N E [15, 60, 60, 120, 240, 500). Specifically, for shorter prompts (N S 240), we use our test dataset maximum of N tokens. If sentence is truncated, it is discarded entirely. For a prompt is formatted as XXX.XXXXXXXXNXXXXX and the N token appears in the middle of sentence, we only retain "xxX.XXX.". For longer prompts around N 500 tokens, we utilize GPT- 4o for generation. During the T2I generation, the baseline models may truncate the prompt if the input exceeds their maximum limit, whereas the two evaluation methods still assess the generated images based the entire given prompt. The evaluation results in Table demonstrate that our method consistently surpasses current baselines.
and restrict the length
Both additional evaluations highlight the effectiveness of our segment-level encoding and preference optimization strategics.
21

---

Published as a conference paper at ICLR 2025
Table 4: Evaluation results DPG-Bench of different models.
Model
Average
Global
Entity
Attribute
Relation
Other
SD-2
68.09
77.67
78.13
74.91
80.72
80.66
PlayG-2
74.54
83.61
79.91
82.67
80.62
81.22
PixArt-a
71.11
74.97
79.32
78.60
82.57
76.96
KanD-2.2
70.12
77.07
80.01
77.55
80.94
78.64
SD-1.5
63.18
74.63
74.23
75.39
73.49
67.81
ELLAus 74.91
84.03
84.61
83.48
84.03
80.79
longSD 77.58
76.27
83.00
86.40
86.52
86.21
Table 5: Denscore-O and VQAscore results for 512 x 512 image generation using different models and different maximum prompt lengths.
Token
Metric
SD-1.5
SD-2.1
PlayG-2
PixArt-a
KanD-2.2
ELLA
longSD(S)
longSD(S+R)
Denscore-O
25.31
26.51
23,90
27.84
29.04
27.13
26.36
29.07/30.12
15
VQAscore
88.32
90.27
88.32
91.12
91.88
90.28
90.71
92.25/92.52
Denscore-O
30.42
31.93
30,23
34.01
35.13
33.68
33.08
36.48/37.14
60
VQAscore 84.54
87.04
86.13
88.73
87.93
87.98
87.90
89.03/89.29
Denscore-O 29.49
30.59
29.26
33.72
33.75
33.65
31.71
34.52/35.57
120
VQAscore 83.63
85.33
84.78
86.81
86.02
86.93
85.88
87.25/87.49
Denscore-O 29.26
30.10
28.8
33,48
33.18
32.94
31.20
34.29/35.18
VQAscore
84.69
85.81
85.38
87.19
86.58
86.90
86.27
87.26/87.45
Denscore-O 15.11
15.12
13.22
16.27
16.78
16.83
15.90
19.01/19.36
VQAscore 81.14
82.69
80.42
84.79
84.94
85,84
84.34
86.65/87.24
C.4 RESIL-EESOOTION GENERATION
To evaluate our method for high-resolution generation with the latest models, we conducted exper- iments using SDXL (Podell et al., 2023) resolution of 1024 1024. We assess the results with FID, Denscore, VQAscore, and GPT4o, comparing them to those from the previously presented SD1.5 version. The results shown in Table 10.
According to the results, we can see that (1) our methods significantly improve both SD1.5 and SDXL, demonstrating their robustness. (2) Among the two final fine-tuned versions, longSDXL. clearly outperforms longSD1.5 1.5 in terms of long text alignment, indicating that stronger foundation model achieves better performance limits, (3) The improvement is more pronounced in SD1.5, this is because the pretrained version of SDXL is already better than that of SD1.5.
In the following tables, we also provide visual results using SDXL:
22

---

Published as a conference paper at ICLR 2025
Table 6: Comparison of Image Outputs with and without LongAlign (1/6) Image (w/o LongAlign)
Image 2 (w LongAlign)
HSBC
HSBBC
The image captures the exterior an HSBC bank branch. Dominating the scene a gray building. facade punctuated by a large window on the right side. This window, framed in black, is divided into six panes, each reflecting the world outside. Above this window, a red and white sign proudly displays the HSBC logo. The letters "HSBC" are written in black, standing out against the white background of the sign. To the right of the logo, red diamond-shaper symbol adds splash of color the scene. The image a blend of urban architecture and corporate branding, a snapshot a moment the life the city.
The image presents 3D rendering of futuristic car, which the central focus of the composition. The car is predominantly black, with a glossy finish that reflects the surrounding environment. It's equipped with a large, transparent bibe dome on the roof, through which small astronaut can be seen. The astronaut, dressed in black suit with helmet, is floating in space, surrounded by stars and planets. The car is not just vehicle but spacecraft, and celestial backdrop. The car's design is sleek and modern, with curved front and pointed rear. The wheels are large and silver, adding to the futuristic aesthetic. The background of the image a dark blue, providing stark contrast to the black car and the astronaut. This contrast further emphasizes the car and the astronaut, making them the focal points of the image. Overall, the image is blend of science fiction and modern design, creating visually striking and imaginative scene.
indicated by the presence of the astronaut
23

---

Published as a conference paper at ICLR 2025
Table 7: Comparison of Image Outputs with and without LongAlign (2/6) Image (w/o LongAlign)
Image 2 (w LongAlign)
The image is a close-up portrait of a man with a serious expression. His hair is styled a slicked-back manner, and his facial features are highlighted by the lighting, which casts dramatic shadow on his face. The man's eyes directed towards the camera, and his eyebrows to the intensity his expression. The most striking clement the image the fire that appears emanating from the man's neck shoulders. The fire is depicted a realistic style, with orange and yellow hues that suggest a bright, intense flame. The fire is not contained within the image; it seems to be flowing outward, creating sense of movement and energy. The background of the image is dark, which serves highlight the man and the fire. The darkness also helps emphasize the contrast between the man's skin and the fiery elements in the image. Overall, the image dramatic portrait that combines realistic elements with artistic flair. The use of fire as a visual motif adds layer of intrigue and mystery to the image, inviting the viewer to wonder about the story behind the scene.
slightly furrowed, adding
powerful and
ORAGE AKRILISTED EPEZART
26
In the center of the image, a daring motorcyclist is captured in mid-air, performing a thrilling stunt on orange and black dirt bike. The rider, clad in black and white helmet, grips the handlebars tightly, demonstrating control and precision. The bike is tilted slightly the left, adding the sense of motion and excitement. The setting is large indoor stadium, filled with crowd of spectators who are watching the spectacle unfold. Their faces are a blur anticipation and awe. The background is adorned with various advertisements. adding a splash of color and life to the scene. Despite the action action-packed nature of the image, there's 's certain harmony to it. The motorcyclist, the bike, the crowd, and the stadium all come together create snapshot of moment filled with adrenaline and excitement. It's testament to skill and courage of the rider, and thrill sport.
24

---

Published as a conference paper at ICLR 2025
Table 8: Comparison of Image Outputs with and without LongAlign (4/6) Image (w/o LongAlign)
Image 2 (w LongAlign)
The image a digital artwork depicting a nighttime scene. In the foreground, there are three silhouetted trees with curved branches, suggesting a tranquil setting. The trees are set against dark sky, which is filled with numerous stars and faint, milky-way-like nebula. The colors in the sky transition from deep blues at the top lighter purples and pinks near the horizon, indicating either dawn dusk. The overall atmosphere of the image serene and somewhat my mystical, with sense of depth conveyed by the starry sky. There are no visible texts or distinguishing marks that provide additional context information about the image. The style of the artwork is realistic with focus on creating peaceful and somewhat ethereal nighttime landscape.
vastness
In the image, there's a lively scene unfolding in a room with wooden floor and white wall in the background. Two individuals
engaged in discussion, standing in front of bulletin board. The bulletin hoard, made of wood, is adorned with two posters. One poster white with black text, while other is black with white lext. The person on the left, in green shirt, is gesturing with their hands, perhaps emphasizing a point or explaining something. On the right, the other person, wearing black jacket, is attentively listening, their gaze fixed on the person in green. The interaction between the two individuals suggests discussion presentation of some sort. The bulletin board, with its two posters, serves the backdrop for this exchange.
25

---

Published as a conference paper at ICLR 2025
Table 9: Comparison of Image Outputs with and without LongAlign (6/6) Image (w/o LongAlign)
Image 2 (w LongAlign)
The image depicts a fantastical scene featuring large ship with a distinctive design. The ship is predominantly dark in color, with intricate carvings and decorations that suggest historical or mythical inspiration. The most striking feature of the ship is the presence of a large, elephant-lig head the bow, which facing towards viewer. This head is detailed and realistic, with tusks and trunk that are prominently displayed. The ship is sailing on body of water, with clear sky above calm, reflective surface below. In the background, there is a rocky outcrop that adds to the sense of a natural, outdoor setting. On the deck of the ship, there are several people visible, although they are too small to discern any specific details about them. The ship also adorned with multiple flags, which are attached the mast and the bow. These flags or insignias. The overall style of the image is realistic with a touch of fantasy, as evidenced by the elephant head and the elaborate carvings on the ship. The lighting and shadows suggest that the image is set during the daytime, with the sun casting warm glow the scene. There visible texts brands in the image.
not detailed enough to identify any specific symbols
The image presents dramatic and surreal scene set against dark, cloudy sky. Dominating the center of the image large, black ring, which appears to portal vortex. This ring encircled by halo of light, creating stark contrast against the dark sky. The light seems to emanate from the center of the ring, suggesting source power or energy. Beyond the ring, the sky is filled with clouds that are illuminated by the light from the ring. The clouds are dense and appear to be in motion, adding sense of dlnammism the scene. The colors in the image predominantly dark and black, with the light from the ring providing stark contrast. The overall composition of the image is balanced, with ring centrally located and clouds filling rest of frame. The use of light and shadow creates sense of depth and dimension, drawing the viewer's eye towards the center of the image. Despite the fantastical elements, the image is grounded in realistic aesthetic. The clouds and sky are rendered with high level of detail, and the colors naturalistic. This combination of realism and fantasy creates visually striking image that invites the viewer imagine what lies beyond the ring.
26

---

Published as a conference paper at ICLR 2025
Table 10: Evaluation results our methods on two foundation models: SD1.5 and SDXL.
Model
FID
Denscore-O
Denscore
VQAscore
GPT-4o
SD1.5
24.96
29.29
20.29
84.57
195
IongSD1.5
24.28
35.26
23.79
87.24
668
SDXL
21.18
33.52
22.79
86.89
268
longSDXL
23.88
37.33
25.33
87.30
416
D GPT-40 EVALUATION
The input for evaluation consists of a prompt and two images generated for this prompt using differ- ent models. We utilize the multimodal evaluation capabilities of GPT-4o and employ the following Python code for evaluation:
def eval_fn (prompt, imagel, image2) template = g""" (prompt)
Which image is more consistent with the above prompt? Please respond with "first", "second" "tie", no explanation needed. *** completion client. client.chat.cco chat. wwwphttttm.metio......mcmo ierplletoonsccrateee... proee==gpt-4o-2024--0 messages=11
"role": "system", "content": "You image generation
evaluation expert.",
"role": "user", "content";
("type": "text", "text"; templateh,
("type": "image_url", Simage_url") ( ("url": f"data: inage//peg;basee4, {imagel]"]]
("type": "image_ur1", "image_uri" url": ("url": f"data:
image/ jaep: [mmage2)"1]
111)
Table 11: Evaluation for comparison in the P21 diffusion framework.
768
1024
Method
P21
+ours
P21
+ours
FID>55k
20.36
21.60
19.78
20.84
Denscore-O
34.45
38.71
34.78
38.51
Denscore
23.43
25.39
23.47
25.41
GPT-4o
240
583
289
536
E OUT-OF-DISTRIBUTION PROBLEM
Some of these approaches incorporate the assistance of LLMs, such Ranni and RPG-Diffusers. LLM-based methods that involve additional LLM assistance increase computational requirements and encounter significant dittt--distributionn (OOD) issues with long-text inputs.
There are several reasons why existing LLM-based methods struggle with long-text text inputs. Long contain more facts than shorter ones, and LLMs rely on injecting different subprompts into specific subareas. Some facts may be disjoint, while others overlap (e.g., detailed descriptions of single object), complicating the separation of information in pixel space. Additionally, some L.LM- based methods fine-tune the LLM model that may not include datasets with long and detailed inputs. As result, these models often cannot manage such situations for generating effective layout plans to assign different subprompts across different areas.
While current LLM-based methods are not fully effective, we believe this OOD problem can be resolved in the future. Additionally, LLM-based methods complement approach: while concentrate on training more powerful foundation models, these techniques can further enhance results during the sampling stage.
27

---

Published as a conference paper at ICLR 2025
Here some examples of OOD problems:
Ranni. In our experiment with Ranni, approximately half of the prompts resulted in errors before generating the final output. We selected examples from the remaining successful prompts. The example prompt is generated from the first image in Figure 18, Although Ranni generates the layout initially, it creates too many subareas that do not contribute clearly to the final result, as shown in the second image in Figure 18.
Figure 18: The reference image and the corresponding generated image using Ranni.
The complete input prompt "The image a digital artwork featuring a female character standing in a rocky environment. The character is dressed in lantasy-style armor with a predominantly dark color scheme, highlighted by accents of blue and purple. The armor includes a corset-like bodice, a skirt, and arm guards, all adorned with intricate designs and patterns. The character's hair is styled in a high ponytail, and she has serious expression on her face. The armor illuminated by a purple glow, which appears to emanate from the character's body, creating a contrast against the darker elements of the armor and the surrounding environment. The glow also casts a soft light on the character's 's face and the armor, enhancing its details and textures. The background consists of a rocky landscape with a purple huc, suggesting a magical or otherworldly setting. The rocks jagged and uneven, with some areas appearing to be fire, adding to the dramatic and intense atmosphere of the image. There are no visible texts or logos in the image, and the style of the artwork is realistic with a focus on fantasy elements. The image is likely intended for gaming or fantasy-themed context, given the character's 's attire and the overall aesthetic."
RPG-Diffusers. For RPG-Diffusers, the example prompt is generated from the first image in Fig- ure 19. However. they assign two subareas: the left side and the right side of the image. Using these subprompts- black leather sofa is stylishly centered the frame, showcasing and elegant design, the distinct cushions maintain the streamlined look of the piece, offering com- fort against the stark contrast of the white background." and "The three luxurious black cushions are artfully arranged, emphasizing the sofa's inviting nature while seamlessly blending into the co- hesive monochrome theme of the rresuns.""essulss repetition of "soft" the generated image, as shown in the second image in Figure 19.
plush texture
Figure 19: The reference image and the corresponding generated image using RPG-Diffusers.
The complete input prompt is "The image presents a scene dominated by a black leather sofa, which is the central object in the frame. The sofa, with its three cushions, is positioned against stark white background, creating a striking contrast. The cushions, like the sofa, are black and appear to be made of leather, suggesting uniform color scheme throughout the piece. The sofa is designed
28

---

Published as a conference paper at ICLR 2025
with wooden frame, adding touch of warmth to the otherwise monochrome setting. The frame is visible on both the front and back of the sofa, providing stability and support. The arms of the sofa, like the frame,
made of wood, maintaining the overall aesthetic of the piece. The sofa is set against a white background, which accentuates its black color and wooden frame, making it the focal point of the image. There are other objects in the image, and text is present. The relative position of the sofa is central, with ample space around it, further emphasizing its importance in the image. The image does not depict any actions, but the stillness of the sofa suggests a sense of tranquility."
F TEXT CONDITION FOR VISUAL RESULT
In this section, provide the text conditions for the visual results in Figure and Figure 6. The text conditions are as follows:
The text conditions for Figure are as follows:
1. The image presents a 3D rendering of a horse, captured in a profile view. The horse depicted in a state of motion, with its mane and tail flowing behind it. The horse's body is composed of a network of lines and curves, suggesting a complex mechanical structure. This intricate design is further emphasized by the presence of gears and other mechanical components, which are integrated into the horse's body. The background of the image is a dark blue, providing a stark contrast to the horse and its mechanical components. The overall composition of the image suggests a blend of organic and mechanical elements, creating a unique and intriguing visual.
2. The image presents a close-up view of a human eye, which is the central focus. The eye is surrounded by a vibrant array of flowers, predominantly in shades of blue and purple These flowers are arranged in a semi-cirele around the eye, creating a sense of depth and perspective. The background of the image dark blue sky, which contrasts with the bright colors of the flowers and the eye itself. The overall composition of the image suggests a theme of nature and beauty.
3. The image presents a detailed illustration of a submarine, which is the central focus of the art- work. The submarine is depicted in a three-quarter view, with its bow facing towards the right side of the image. The submarine is constructed from wood, giving it a rustic and aged appear- ance. It features dome-shaped conning tower, which is common feature on submarines, and large propeller at the front. The submarine is not alone in the image. It is surrounded by a variety of sea creatures, including fish and sharks, which are swimming around it. These creatures add a sense of life and movement to the otherwise static image of the submarine. The background of the image is light beige color, which provides neutral backdrop that allows the submarine and the sea creatures to stand out. However, the background is not devoid of detail. It is adorned with various lines and text, which appear to be map or a chart of some sort. This adds an element of intrigue to the image, suggesting that the submarine might be on a mission or an expedition. Overall, the image is a detailed and intricate piece of art that captures the essence of a submarine voyage, complete with the submarine, the sea creatures, and the map in the background. It's snapshot of a moment in time, frozen in the image, inviting the viewer to imagine the stories and adventures that might be taking place beneath the surface of the water.
In the image, there's a charming scene featuring green frog figurine. The frog, with its body painted in a vibrant shade of green, is the main subject of the image. It's wearing a straw hat, adding a touch of whimsy its appearance. The frog is positioned in front of white window, which is adorned with a green plant, creating a harmonious color palette with the frog's 's body. The frog appears to be looking directly the camera, giving the impression of friendly encounter. The overall image exudes a sense of tranquility and simplicity.
5. The image portrays female character with fantasy-inspired design. She has long, dark hair that cascades down her shoulders. Her skin is pale, and her eyes are a striking shade of blue. The character's 's face is adorned with intricate gold and pink makeup, which includes elaborate patterns and designs around her eyes and on her cheeks. Atop her head, she wears a crown made of gold and pink roses, with the roses arranged in a circular pattern. The crown is detailed, with each rose appearing to have a glossy finish. The character's 's attire consists of gold and pink dress that is embellished with what appears to be feathers or leaves, adding to the fantasy aesthetic. The
29

---

Published as a conference paper at ICLR 2025
background of the image is dark, which contrasts with the character's pale skin and the bright colors of her makeup and attire. The lighting in the image highlights the character's i features and the details of her makeup and attire, creating a dramatic and captivating effect. There are no visible texts or brands in the image. The style of the image is highly stylized and artistic, with focus the character's beauty and the intricate details of her makeup and attire. The image is likely a digital artwork or concept illustration, given the level of detail and the fantastical elements present.
6.
The image captures scene of a large, modern building perched on cliff. The building, painted in shades of blue and gray, stands out against the backdrop of a cloudy sky. The cliff itself is a mix of dirt and grass, adding a touch of nature to the otherwise man-made structure. In the foreground, a group of people can be seen walking along a path that leads up to the building. Their presence adds sense of scale to the image, highlighting the grandeur of the building. The sky above is filled with clouds, casting soft, diffused light over the scene. This light enhances the colors of the building and the surrounding landscape, creating a visually striking image. Overall, the image presents a harmonious blend of architecture and nature, with the modern building seamlessly integrated into the natural landscape.
The text conditions for Figure are as follows:
1. The image is a digital artwork featuring a female character standing in a rocky environment. The character is dressed in a fantasy-style armor with a predominantly dark color scheme, highlighted by accents of blue and purple. The armor includes a corset-like bodice, a skirt, and arm guards, all adorned with intricate designs and patterns. The character's hair is styled in high ponytail, and she has a serious expression on her face. The armor is illuminated by a purple glow, which appears emanate from the character's body, creating a contrast against the darker elements of the armor and the surrounding environment. The glow also casts a soft light on the character's face and the armor, enhancing its details and textures. The background consists of a rocky land- scape with a purple hue, suggesting a magical or otherworldly setting. The rocks are jagged and uneven, with some areas appearing to be on fire, adding to the dramatic and intense atmosphere of the image. There are no visible texts or logos in the image, and the style of the artwork is realistic with a focus on fantasy elements. The image is likely intended for a gaming or cuntasy-themer context, given the character's 's attire and the overall aesthetic.
2. The image presents a scene of elegance and luxury. Dominating the center of the image is brown Louis Vuitton suitcase, standing upright. The suitcase is adorned with a gold handle and a gold lock, adding touch of opulence to its appearance. Emerging from the top of the suitcase is a bouquet of pink and white roses, interspersed with green leaves. The roses, in full bloom, seem to be spilling out of the suitcase, creating a sense of abundance and luxury. The entire scene is set against a white background, which accentuates the colors of the suitcase and the roses. The image does not contain any text other discernible objects. The relative position of the objects is such that the suitcase is in the center, with the bouquet of roses emerging from its top.
The image captures the grandeur of the Toledo Town Hall, a renowned landmark in Toledo, Spain. The building, constructed from stone, stands tall with two prominent towers either side. Each tower is adorned with a spire, adding to the overall majesty of the structure. The facade of the building punctuated by numerous windows and arches, hinting the intricate architectural details within. In the foreground, a pink fountain adds a splash of color to the scene. A few people can be seen walking around the fountain, their figures small in comparison to the imposing structure of the town hall. The sky above is clear blue, providing a beautiful backdrop to the scene. The image is taken from low angle, which emphasizes the height of the town hall and gives the viewer sense of being in the scene. The perspective also allows for detailed view of the building and its surroundings. The image does not contain any discernible text. The relative positions of the objects confirm that the town hall is the central focus of the image, with the fountain and the people providing context to its location.
30


Reviews:
[{"review_id": "kax49zb3Nf", "reviewer": "Reviewer_GqtG", "summary": "This paper presents a novel approach to enhance the alignment between long text descriptions and generated images in text-to-image diffusion models, introducing segment-level encoding to overcome input length limitations and decomposed preference optimization to mitigate overfitting and improve text-relevant alignment during fine-tuning.", "questions": "see Weaknesses", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The motivation for using preference models is well-founded, and the paper is well-written.\n2. It is interesting to identify two distinct focuses within preference models, and the analysis provided is both reasonable and thorough.", "weaknesses": "weakness\n1. I am unsure why multiple <sot> tokens are retained; regarding the retention or removal of tokens, a more detailed explanation or analysis is needed, as it currently leaves me confused.\n2.After reweighting, whether there will be a noticeable difference in the aesthetic quality of the generated results (due to text-irrelevant components) remains unclear. For Appendix B.1, it would be beneficial to provide some visualizations of the outcomes from the two loss functions.\n3. Segmenting to leverage CLIP's alignment effect is an intuitive innovation, but does this become irrelevant in light of the development of Vision-Language Models (VLMs)? Can the current innovation still contribute to VLMs?\n4. On line 363, it mentions mitigating the risk of overfitting to Denscore. Could you clarify where the potential source of this overfitting lies?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "tClVJWd1jI", "reviewer": "Reviewer_9HBG", "summary": "This paper proposes a novel method to improve text-to-image (T2I) diffusion models in handling long text inputs. Due to the input length limitations of existing encoders like CLIP, it becomes challenging to accurately align generated images with long texts. To address this issue, the authors propose a segment-level encoding strategy, which divides long texts into segments and encodes them separately, combined with a decomposed preference optimization method to reduce overfitting and enhance alignment. Experimental results show that the fine-tuned model surpasses several existing foundation models in long-text alignment, demonstrating significant improvements in handling long text inputs.", "questions": "[1] Does your proposed segment-level encoding strategy demonstrate significant effectiveness for texts of varying lengths? Specifically, how does the model perform with very short texts (fewer than 10 words) or very long texts (over 500 words)? Could you provide additional experiments to show comparative results under different text length conditions to verify the generalizability of the segment-level encoding strategy?\n[2] You mentioned using a reweighting strategy to mitigate the model's overfitting issue, but the description of this process in the paper is rather brief. Could you provide detailed steps or pseudocode to explain the implementation of this strategy? Additionally, does this method have any quantitative results to demonstrate its effectiveness in reducing overfitting in specific scenarios? Could you include comparative data from the experiments to validate the impact of this strategy?\n[3] How were the 5k images in the test set specifically selected from datasets like SAM and COCO2017?\n[4] Could you briefly explain the selection of models like CLIP-H and HPSv2 in the experimental section of Chapter 5, as well as the chosen evaluation metrics?", "limitations": "", "rating": 3, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "[1] It introduces a segment-level encoding strategy that effectively handles long text inputs by dividing and separately encoding segments, overcoming traditional model input limitations and enhancing text-to-image alignment. \n[2] The preference model is innovatively decomposed into text-relevant and text-irrelevant components, with a reweighting strategy to reduce overfitting and improve alignment precision.\n[3] The paper conducts extensive experiments, demonstrating significant improvements in long-text alignment over existing models like PixArt-α and Kandinsky v2.2, proving the method's effectiveness for complex text generation tasks.", "weaknesses": "[1] The paper proposes a segment-level encoding strategy to handle long texts but does not thoroughly validate the performance of this strategy under different text length conditions. For very short or very long texts, can the segment-level encoding still maintain the same alignment effectiveness? The lack of fine-grained comparative experiments makes it difficult to adequately demonstrate the applicability of segment-level encoding across a wide range of text lengths.\n[2] The paper proposes a reweighting strategy to address overfitting, but lacks detailed experimental data to demonstrate its effectiveness, failing to adequately prove its specific impact on reducing overfitting.\n[3] The segment-level encoding and preference optimization strategies proposed in this paper show excellent performance in the experiments, but lack an analysis of the method's limitations. It would be beneficial to discuss whether these segment-level encoding methods might lose part of their alignment effectiveness when dealing with texts that have complex contextual dependencies or require strong semantic understanding.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ZiYOl72O7n", "reviewer": "Reviewer_hkae", "summary": "This paper presents a method for enhancing the prompt following of text-to-image models specifically in the case of long prompts. The key contribution for tackling this problem is twofold: a) using a combination of CLIP and T5 encoders (as is becoming increasingly common these days e.g. SD3, Flux) b) the introduction of a preference model tailored for long prompts (Denscore) and applying reward fine-tuning with this Denscore model to enhance the prompt following of SD1.5 models for long prompts.", "questions": "I apologize in advance if I missed it, but I do not really see clear details about the training of the Denscore model. B.1 has details on the training objectives and the fact that captions are generated by LLaVA-Next, but beyond this I do not see other implementation details (dataset, other choices etc.), so it would be great if the authors could point me to this.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 4, "presentation": 2, "contribution": 3, "strengths": "The paper tackles the crucial challenge of long prompt following in a very effective manner. Using a text encoder that can take the entire long prompt is a sound idea, and the Denscore preference model looks like a useful contribution in general. \nApart from this, the reward fine-tuning with the orthogonal decomposition and the gradient reweighting looks like a good idea to deal with the \"reward-hacking\" problem.\nFinally, the results also appear quite strong from the evaluations presented in the paper.", "weaknesses": "An important paper that is missed here is ELLA[Hu et al. 2024] for a couple of reasons. The first is that they propose replacing the CLIP encoder of SD1.5 with a T5-XL model and get significantly improved results (far superior numbers to those reported by Lavi-Bridge whose MLP adapter is used here). Therefore, this model might be a valid comparison (although the training cost of ELLA is a bit higher: 7 days with 8 A100s for SD1.5). Alternatively, the adapter provided by ELLA would have probably been a better alternative to the one used in the paper (from Lavi-Bridge). \n\nApart from the comparison/use of adapter, there's also DPG-Bench introduced in the paper which is a good benchmark for long prompt following (as compared to existing benchmarks like T2I-Compbench, DSG, TIFA etc.). Evaluating on DPG-Bench would be a useful addition since the 5k evaluation set of this paper is not fully understood and only a few models have been evaluated here. Additionally, from an evaluation standpoint, even on this 5k evaluation set, VQAScore[1] might be a good option to consider, since it uses a T5-XXL model which can take long prompts, and has shown some promising results for text-to-image evaluations. \n\nAnother aspect which is missing here is that all the experiments in this paper are conducted on SD1.5 which is a relatively older model, and there have been newer models in the past 2 years (e.g. SDXL). Therefore, it would have been nicer to also have results with any of the newer, more performant models, but I can understand that this might be a bit more computationally expensive (especially if the training has to be done at 1024 resolution). \n\nOverall, I dolike the paper, but I believe that incorporating these aspects (especially strengthening the paper with additional evaluations) could improve the paper significantly. \n\n[1] Lin et al. \"Evaluating Text-to-Visual Generation with Image-to-Text Generation\", ECCV 2024", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "83lrSI8PMl", "reviewer": "Reviewer_Hfc6", "summary": "This paper presents a new approach for long text inputs on text-to-image (T2I) alignment since the clip text encoding only allows 77 tokens. The authors address limitations of existing encoding methods like CLIP by proposing segment-level encoding, where long texts are divided and processed in parts to bypass input length constraints. They further introduce a decomposed preference optimization that separates alignment-related and non-alignment components of CLIP-based preference scores. By reweighting these components, the method reduces overfitting, achieving superior T2I alignment after fine-tuning Stable Diffusion v1.5, outperforming models such as PixArt-α and Kandinsky v2.2.", "questions": "(1) Please better explain why the proposed split-and-merge approach can address the long-text alignment issue.\n(2) Please provide the ablation study clearly.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "(1) Comprehensive Survey of Related Work.\nThe paper presents a thorough and comprehensive survey of existing work in text-to-image (T2I) diffusion models, demonstrating an impressive grasp of the field. By delving deeply into previous approaches and their limitations, the authors effectively set the stage for their contributions, clarifying the gaps their method aims to fill. This background provides readers with valuable context and insight into the evolution of T2I models, particularly in handling longer, complex textual inputs. The comprehensive nature of this survey also reinforces the authors' understanding of the field's current challenges and strengths, building confidence in the relevance and timeliness of the proposed approach.\n\n(2) Importance of the Problem and a Reasonable, Well-Motivated Solution.\nThe authors tackle a critical issue in T2I diffusion models: the difficulty of aligning generated images with longer text prompts. As the demand for complex, high-fidelity image generation grows, the ability to handle longer text inputs accurately is essential. The segmentation approach, paired with decomposed preference optimization, offers a well-motivated solution to this problem. Segmenting long text into manageable parts allows for better processing within the confines of existing encoding models, while the decomposed preference optimization fine-tunes the alignment, addressing the unique challenges posed by long prompts. The design choices reflect a reasonable and methodical approach to tackling these limitations, and the paper articulates the rationale for each component clearly. This structured approach suggests the authors have carefully considered the problem’s nuances, offering a solution that is not only effective but also grounded in sound methodology.\n\n(3) Demonstrated Superiority over State-of-the-Art Models.\nOne of the paper's significant strengths is the demonstrated performance improvement over state-of-the-art models. Through rigorous experimentation, the authors show that their method surpasses current leading models like PixArt-α and Kandinsky v2.2 in T2I alignment, particularly for long-text prompts. By fine-tuning Stable Diffusion v1.5 with their approach, they achieve superior alignment, reducing overfitting while preserving text-relevant information in the generated images. This achievement underscores the potential of the proposed method to set a new benchmark for handling longer, more detailed textual inputs within T2I models. The improvement over established models validates the effectiveness of the segmentation and preference optimization strategy, indicating that this approach could meaningfully advance the state of the art in T2I diffusion modeling.", "weaknesses": "Although the proposed method solved an important issue, three major issues remain as listed below.\n(1) Limitations and Ambiguities in the Segmentation and Merging Methodology. The segmentation and merging technique proposed in this work introduces a unique approach to handling longer text inputs but raises questions regarding its effectiveness and generalizability. When text inputs exceed 77 tokens, this method still encounters limitations, as it is fundamentally restricted by the underlying model’s capacity to handle “long” sequences since the split-and-merge process does not solve the problem. This constraint is particularly concerning as longer text inputs are common in real-world applications and often essential to producing detailed and contextually accurate image generations. The current approach of segmenting and then merging these sections seems like a workaround rather than a robust solution to handling extended texts, which may inherently limit its scalability and versatility. Furthermore, the mechanics of how segmentation and merging affect the underlying model's cross-attention dynamics remain underexplored. Cross-attention is a critical component in the alignment process between text and image features, and segmenting inputs may disrupt this alignment, especially as certain semantic connections might be lost or diluted across segmented inputs. Investigating the cross-attention differences between the original, unsegmented approach and the segment-and-merge methodology could shed light on any distortions introduced by this technique. A more thorough analysis of cross-attention’s role here could help refine segmentation methods to better retain textual coherence and improve image alignment fidelity, ultimately benefiting downstream performance.\n\n(2) Dependency on an Outdated Baseline Model (Stable Diffusion v1.5):\nThe use of Stable Diffusion v1.5 as the primary evaluation model poses a significant limitation, given that the field has moved toward more advanced versions like SD-3 and SDXL. These newer versions incorporate improved architectures and training techniques, yielding enhanced performance, especially in terms of image quality and alignment with textual inputs. The reliance on an outdated model not only limits the relevance of the study’s results but also restricts the potential impact of the proposed method. Using v1.5 as the baseline reflects well on the approach’s applicability to older architectures, but it leaves unanswered questions about its efficacy on more sophisticated models that incorporate advancements in diffusion techniques, training scale, and multimodal alignment mechanisms. \nMoreover, maintaining SD-1.5 as a standard for comparison could inadvertently hold back progress within the research community. As models continue to evolve, it’s essential to align benchmark tests with the latest technologies to ensure that methods are relevant and that advancements reflect real-world capabilities. Preliminary results from newer models, such as SD-3, have demonstrated considerable improvements in T2I alignment, indicating that the proposed method may benefit even further from these architectural updates. Testing on newer models would better position the approach in the context of current technological standards, ensuring that it remains relevant and applicable as diffusion models evolve. Future work should include evaluations on SD-3 and SDXL to substantiate claims of superiority over other methods in a more current setting. The test of SD-3 with the prompt used in the first example of Fig. 1 is shown below.\nhttps://ibb.co/CWyKQTZ\n\n(3) Over-reliance on Long Prompt Training and Lack of Generalizability Testing.\nThe proposed method seems to rely heavily on training with long prompts, which could limit its flexibility and adaptability. While training on extended text inputs may enhance alignment for similar prompts, it raises concerns about the model's performance on shorter or more varied prompts. In real-world scenarios, prompt lengths and structures vary significantly, and a robust model should perform consistently across this spectrum. By focusing predominantly on long-prompt alignment, the current approach may overfit to specific input lengths, making it less effective for shorter or less detailed prompts where segmentation might not be necessary or where text segments are not sufficiently complex to benefit from this treatment.\nTo address this potential limitation, it would be valuable to conduct experiments that vary prompt lengths and structures systematically, assessing whether the model’s performance holds across different scenarios. Additionally, testing with alternative segmentation designs could reveal whether simpler or more complex methods yield better alignment. These experiments would enhance our understanding of how adaptable the proposed method is, providing insights into its generalizability and robustness. The community would benefit from such insights, as they could guide further development of segmentation-based approaches for T2I tasks.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)