Enhancing Text-to-Image Diffusion Transformer via Split-Text Conditioning
Authors: Yu Zhang
Jialei Zhou
Xinchen Li
Qi Zhang
Zhongwei Wan
Duoqian Miao
Changwei Wang
Longbing Cao
Venue: NeurIPS 2025 poster ()
Keywords: Diffusion Transformer
Text-to-Image Generation
Multimodel Learning
Large Language Model
Primary area: applications

Abstract: Current text-to-image diffusion generation typically employs complete-text conditioning. Due to the intricate syntax, diffusion transformers (DiTs) inherently suffer from a comprehension defect of complete-text captions. One-fly complete-text input either overlooks critical semantic details or causes semantic confusion by simultaneously modeling diverse semantic primitive types. To mitigate this defect of DiTs, we propose a novel split-text conditioning framework named DiT-ST. This framework converts a complete-text caption into a split-text caption, a collection of simplified sentences, to explicitly express various semantic primitives and their interconnections. The split-text caption is then injected into different denoising stages of DiT-ST in a hierarchical and incremental manner. Specifically, DiT-ST leverages Large Language Models to parse captions, extracting diverse primitives and hierarchically sorting out and constructing these primitives into a split-text input. Moreover, we partition the diffusion denoising process according to its differential sensitivities to diverse semantic primitive types and determine the appropriate timesteps to incrementally inject tokens of diverse semantic primitive types into input tokens via cross-attention. In this way, DiT-ST enhances the representation learning of specific semantic primitive types across different stages. Extensive experiments validate the effectiveness of our proposed DiT-ST in mitigating the complete-text comprehension defect. Datasets and models are available.

Enhancing Text-to-lmage Diffusion Transformer via Split-Text Conditioning
Yu Zhang'
Jialei Zhou'
Xinchen Li'
Qi Zhang Zhongwei Wan
Duoqian Miao
Changwei Wang
Longbing Cao
Tongji University
The Ohio State University
Macquarie University
Original Caption
LLh--Ehharced Caption (CompleteTTazt)
Our Caption
(Complete-Texc)
(Split-Text))
Stable Diffusion v3
DIT-ST-M M (Ouns)
On the marble zable, there yellow tablecloth life and fock, and several sples. Among them one of the apples im the bout apears to be unripe and partly color.
Same custamers art sctocting from market salls mied with neelyy amsanged colociul vegelabees and truit, with prices of goods next to them.
rod bockpock adomed with a small pink ribbon how messs on an old wooden bench along a quict city street, shaded by green tres.
racing drive sitingg in the garage with his helmel on green cat, markings sprayed body. placed agalnstt steel plate wall.
(a)
(b)
Figure 1: (a) Images generated by MM-DiT 8B-E using different forms of the same caption. Our split-text caption enables the model to notice details, such unripe and partly green. pink ribbon bow, and display them the generated image. (b) Comparison text-to-image generation results between Stable Diffusion v3 and ours. Obviously, ours has better semantic details and mitigates the attribute misbinding such as green car.
Abstract
Current text-to-image diffusion generation typically employs complete-text condi- tioning. Due to the intricate syntax, diffusion transformers (DiTs) inherently suffer from a comprehension defect of complete-text captions. One-fly complete-text input either overlooks critical semantic details causes semantic confusion by simultaneously modeling diverse semantic primitive types. To mitigate this defect of DiTs, we propose a novel split-text conditioning framework named DTT-ST. This framework converts a complete-tex caption into a split-text caption, a collection of simplified sentences, to explicitly express various semantic primitives and their interconnections. The split-text caption is then injected into different denoising stages of DiT-ST in hierarchical and incremental manner. Specifically, DIT-ST leverages Large Language Models to parse captions, extracting diverse primitives and hierarchically sorting out and constructing these primitives into split-text input. Moreover, we partition the diffusion denoising process according to its differ- ential sensitivities diverse semantic primitive types and determine the appropriate
"Corresponding Author
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

timesteps to incrementally inject tokens of diverse semantic primitive types into imput tokens via cross-attention. In this way, DiT-ST enhances the representation learning of specific semantic primitive types across different stages. Extensive experiments validate the effectiveness of our proposed DiT-ST in mitigating the complete-text comprehension defect. Datasets and models are available.
Introduction
In recent years, diffusion models [ have achieved unprecedented breakthroughs B Leveraging the Transformer 77 backbone, the Diffusion Transformer (DiT) [8] has rapidly become the mainstream paradigm for text-to-image generation and achieves impressive performance.
4 6
Current DiTs for text-to-image generation mostly employ complete-text captions as conditioning, which are sourced from human-curated datasets generated by large language models (LLMs) or multimodal large language models (MLLMs). Such complete-text captions, particularly long ones, typically involve complex syntax with intertwined semantic primitives (object, relation, and attribute) and potential redundancy. When dealing with this form of captions, DiTs exhibit an inherent complete-text comprehension defect, reflecting in issues such attribute misbinding (|] 10 style dominance [2], semantic blending [3] 14], and semantic entanglement [3]]. The comprehension defect stems from two key aspects: insufficient semantic analysis and premature information exposure.
On the one hand, insufficient semantic analywis arises from multiple factors. For instance, (/) text length bottleneck [16] CLIP text encoder [17] accepts at most 77 tokens, yet effective tokens seldom exceed 20, making it difficult to process detailed text and lcading to the truncation or loss of tail information in long captions; (ii) softmax competition [18 [9]: tokens compete against each other in the softmax function, causing the representation capability to decrease as token number increases; (iii) positional bias (20]: CLIP prioritizes the first item, making it difficult to attend to later items and leading to semantic distortion. These factors collectively hinder models from accurately amalyzing semantic primitives and highlighting important semantic information from long complete-text captions. Consequently, models tend to overlook and omit crucial semantic information within complete-text caption. However, split-text caption derived by hierarchically sorting out the semantic primitives within the complete-text caption may reduce the syntactical complexity and comprehension difficulty for models, helping improve semantic analysis.
ribbon
teckly bear
WBaTS
Objectand relation al step 25
Onject.and relation at step 75
(a)
(b)
(a) Attention maps 1b] for various semantic primitives. Caption: A teddy bear wearing a red ribhon around its neck. Attentions exhibit significant overlap between the object primitive 'ribbon' and relation primitive "wears', resulting in semantic entanglement. (b) Superimposed attention maps of object primitive type and relation primitive type denoising timesteps 25 and 75, respectively, Notably, the model focuses more on object primitives during the earlier stage and shifts more attention to relation primitives in the later stage.
Figure 2:
On the other hand, the one-time input of a complete-text caption leads to various semantic primitives being modeled simultaneously. Since these primitives cannot be disentangled effectively within the shared representation space, ultimately competing for representation within the same image regions. This is the main reason for phenomena such semantic entanglement, as illustrated in Figure (a). We summarize this problem as premature information exposure, where excessive fine-grained details are prematurely exposed to the model before the model establishes stable primary semantic concepts. Existing studies [21 22 23 (4 indicate that diffusion models establish primary semantic concepts during the early denoising stage, while improving fine-gre -me-graided details in the later stage. Details mainly originate from attribute primitives, while primary semantic concepts derive from object and relation primitives. In addition, as illustrated Figure ( (b), different denoising stages have diverse sensitivities of semantic primitive types. We can conclude that the prioritization order for semantic primitive types is paapcct-rllaiio-ttttiibie Therefore, we propose incrementally injecting diverse types of semantic primitives by prioritization order to improve the proportion of sensitive

---

primitive types at each stage, enabling the model to enhance the representation learning of a specific sensitive primitive type the corresponding stage. As discussed above, since a split-text caption is derived by hierarchically sorting out the semantic primitives and possesses a distributed structure, it can effectively group the same type of semantic primitives together and be injected incrementally, naturally helping mitigate premature information exposure.
In this paper, we propose novel framework, DiT-ST, to enhance the text-to-image DiT via split- text caption. A split-text caption is essentially a hierarchical collection of simplified sentences that explicitly express various semantic primitives and their interconnections, effectively reducing the syntactical complexity to alleviate insufficient semantic analysis and grouping the same type semantic primitives be injected incrementally to address premature information exposure. Briefly, we first employ LLM [25] for caption parsing to extract various semantic primitives and assemble a hierarchical caption parsing graph. We then construct the hierarchical collection of simplified sentences according to the graph to realize the conversion from complete-text caption to split-text caption. We refine the encoding of the MM-DiT backbone [26], resulting in a DiT-ST with richer semantic level. In addition, determine appropriate injection timesteps for three semantic primitive types by analyzing cross-attention convergence phenomena (22 and identifying the inflection point in the signal-to-noise ratio curve during denoising, After that, incrementally injecting object-relation- attribute tokens to input tokens via cross-attention to increase the proportions of corresponding sensitive semantic primitive types, further enhancing representation learning of specific semantic primitives at different stages.
Extensive experiments confirm the benefits of our split-text captioning. On GenEval, our DiT-ST-M achieves 0.69 overall accuracy and 11.3% gain over SDv3 Medium. On COCO-5K, it records the highest average CLIPScore (34.09) and the lowest FID (22.11) 1)), delivering performance competitive with SDv3.5 Large. All results and visualizations demonstrate that the method is both parameter- efficient and architecture-agnostiic, yieldding finer detail and stronger semantic fidelity.
2 Related Work
2.1 Diffusion Transformers
Diffusion models 2 27 reverse a fixed noising process by imitating the denoising trajectory
through learned optimization. When integrated with multimodal learning [28] 29 30
31 [3] and
other techniques 33
34
35
36
37 38, they offer a powerful generation framework for visual
content 33 40
and other downstream tasks 44
46
48]. Recent advances,
Diffusion Transformer g replaces U-Net [49] with Transformer 7 enabling its strong capability for long-range dependency modeling, scalability, and flexibility. DiT models, exemplified by DeepFloyd- IF 3. Flux [50], PixArt-a [51] and DreamEngine [52). have successfully achieved remarkable generation performance, making DiT the mainstream paradigm for text-to-image generation.
2.2 Complete-Tees Text Comprehension Defect
Complete-tex comprehension defect refers to models' difficulty in effectively analyzing and under- standing complete text, stemming from inherent encoder limitations or structural deficiencies, such as text length bottleneck [6]. softmax competition 18 19, J. positional bias 20), incorrect tokeniza- tion [53] and frequency bias [54). This defect results in suboptimal generation quality and generated content that struggles to faithfully correspond to the prompt. Manifestations of complete-text compre-
hension defect include attribute misbinding (55] object missing [56), style dominance [II, semantic
blending [3], semantic entanglement [15] and etc.
Many existing methods have mitigated specific manifestations. Attend-and-Excict [9] activates domi- nant token attention to guide the generation process in adherence the textual prompt. WiCLP (57) and research [00 address attribute misbinding and object omission by refining the text embedding space and reweighting token-level attention enhance compositional fidelity during generation. DeaDiff [12) mitigates style dominance by employing exclusive subsets of cross cross-attention layers for disentangling style and semantics. LongAlign (58 enhances semantic coherence by introducing
explicit linguistic structures. In contrast, SCoPE (59 operates at the sub-prompt level, beginning
with the simplest sub-puon sub-prompt, then progressively introducing more complex variants and determining injection timesteps using proportional similarity ratios to mitigate alignment degradation.
3

---

While these methods indeed mitigate the complete-text comprehension defect from various perspec- tives, they primarily optimize for specific manifestations. Instead of addressing individual encoder limitations, we target the fundamental cause of the comprehension defect- inherent inability of models to handle complex syntax within complete-text captions. Convert the form of captions to pursue comprehensive mitigation strategy for the complete-text comprehension defect.
3
Methodology
The overall framework of DiT-ST is illustrated in Figure 3 which incorporates three key components: Caption Parsing, Hierarchical Caption Input, and Incremental Primitive Injection. The original caption Cer adopts a complete-text structure. To sort out and refine semantic primitives, DiT-ST employs LLM V8 as for caption parsing, extracting various semantic primitives and assembling them into a caption parsing graph G. According to the relationships among various semantic primitives in graph G. we construct them into a hicrarchical collection of simplified sentences the split-text caption CST-whice is subsequently input to the rext-to-image DiT (MM-DiT). During the DiT denoising process, we calculate and determine appropriate timesteps for injecting each the three semantic primitive types. Following the dejecttrelation-attiibe semantic primitive tokens into input tokens via cross-attention thereby enhancing the representation learning of specific semantic primitives across different stages.
order, incrementally inject encoded
CCT
Larkiing
G
cat. rug. packaging. Nintendo Switch. eyes. cat sitting rug.
CST
Aitting
cute beige. and white cat with large, curious eyes sitting on a rug next to new Nintendo Switch and its original packaging. inquisitively looking
meet
nug
Nintendo Switch
packaging
to cat looking up
Parsing
Construct
Ninmendo Switch cat is cute. cot is beige and white. Nintendo Switch packaging is original eyes are curious.
MM-DIT
eyes
new
original
curious
cute
beige and white
Hierarchica Caption Input
Denoising Process
NM-DIT Encoder
object
relation
attribute
Caption Parsing
Incremental Text Injection
Figure 3: The overall framework of DiT-ST, Three colors represent object, relation and attribute, respectively.
3.1
Caption Parsing
Caption parsing aims to split the complete-text structure of captions, resulting in a caption parsing graph G. The reason why we choose the graph as the target product is that hierarchical tree-like structure [60] can clearly represent various semantic primitives and their intercomnections, naturally resolving existing problems of DiTs in this aspect. Therefore, we leverage LLM (specifically Qwen- Plus & [61] in this work) as a semantic analyzer and graph generator to extract various semantic primitives and assemble these semantic primitives to obtain the caption parsing graph G.
We define a caption parsing graph as G = GV,V where V denotes the set of N nodes and F denotes the set of edges. Specifically, V (1; i- = 1, 2, ..., N). Each node v, is defined v; (o;, A.), with o, denoting the i-th object primitive and A, {a,} representing the set of attributes associated with object primitive og. O is the object set, and A is the attribute set. Relation set R consists of M semantic relationships: R = {rk k = 1, 2, M Each edge is modeled as a labelled triple e (U:, Uy rk) E V V R. the set of edges E C V X R.
During the graph assembly, we use the root node to represent the entire caption, and recursively decompose it into various object primitive nodes. Relations between object primitives are represented by edges between nodes. Atribbuee are represented as child nodes attached to their corresponding object primitive nodes. Therefore, for a given complete-text caption Cor, the process obtain the caption parsing graph G can be formalized as follows:
LLM Parsing (O,R,A) A) Graph Assembly G.
CCT
(1)
In this way, we convert complete-text caption into split and hierarchical caption parsing graph.

---

3.2 Hierarchical Caption Input
The purpose of hierarchical caption input to construct a split-text caption based on its caption parsing graph. In addition, in order to make the model better adapt to the split-text caption and enrich the overall semantic level of the input, we also modify the existing text encoding method in text-to-image DiT to better encode the split-text caption. Thus, the hierarchical caption input process consists of two components: split-text caption construction and DiT text encoding refinement.
3.2.1 Split-Text Caption Construction
The hierarchical structure of the caption parsing graph facilitates us to construct a split-text caption, which is a hierarchical collection of simplified sentences. Considering that inherent positional bias in text encoders (e.g., CLIP) is unavoidable during subsequent split-text encoding, we also take into account the order of semantic primitives, i.e., position the important semantic primitives the beginning achieve better performance. Specifically, set O descending order based on the node degree and frequency of occurrence the caption, obtaining reranked object set O'. We then rerank the primitives in the relation set R. and attribute set A ensure that relations or attributes involving object primitives maintain consistency with the object primitive order in O², resulting in R' and A'.
first rerank object primitives in the object
Subsequently, based on the reranked sets O² R', A' and following the object-relation-atriboe hierar- chical order, we generate corresponding simplified sentences for each primitive within these sets. The forms of simplified sentences corresponding objects, relations, and attributes present respectively [OBJECT] object [RELATION] object relation object j and [ATTRIBUTE] object
attribute_i_ The collection comprising all these simplified sentences constitutes the split-text caption Csr. The process to construct the split-text caption can be formalized as follows:
(O, A,
Acc.to G to Rerank
"O',R',A)
Construct
CST-
3.2.2 DiT Text Encoding Refinement
We employ the current mainstream multimodal diffusion transformer, MM-DiT (6), for text-to- image generation. MM-DiT adopts three text encoders: CLIP-L/14, CLIP-G/14, and T5 XXL. For split-text caption CST. after three encoders' text encoding, three token sequences can be obtained: TST E BLLD1/14 TST E RRL220/14 and T RLDD where L the token sequence length L/14 (I 77), DL/14, Pa/11 and 1) are the dimensions of the three token sequences.
According to the original design, the concatenation of se- quences 1./14 SS and TST yields new token sequence whose still dimension remains smaller than D. Given that
CLIP-L/14
7/11
C/14
MM--DIT Blocks
the new token sequence must be appended with T for input into the MM-DiT blocks, the dimension capacity re- mains underutilized. Therefore, we consider fully utilizing this underutilized dimension capacity by incorporating the complete-texx caption CCT to enrich the overall semantic level of the input. As shown in Figure 4 We supplement it by adding CCT encoded through T5 XXL and followed by linear projection, generating the token We perform dimensional concatenation
CST
CLIP-G/14
Projection
CCT
T5 XXL
Figure 4: DiT text encoding refinement.
sequence T E 4' where D' D D1/14 Da/14, and sequence append as follows:
Teoncate Comcate (Ti,/4: TS TCT E RRD 1/11: G/14
(3)
T Append (Teoncate: IST)) D2LXD
(4)
Through the above operations, we ultimately construct the input token sequence which possesses a hierarchical structure of semantic primitives and rich semantic representation.
3.3 Incremental Primitive Injection
As previously discussed, due to different denoising stages have differential sensitivities of semantic primitive types, prematurely exposing fine-grained detail information before the model has established
5

---

stable primary semantic concepts is detrimental to generation. We propose incrementally injecting diverse semantic primitive types to enhance differential representation learning for diverse semantic primitive types at different stages. Therefore, incremental primitive injection involves both injection timestep selection and semantic primitive injection.
3.3.1 Injection Timestep Selection
Since diffusion models' prioritization order for diverse semantic primitive types during the denoising process generally follows object-relation-attiiivee
we should, ideally, identify accurate timesteps for injecting semantic primitives. However, it is practically impossible for several reasons. First, semantic emergence occurs gradually rather than in discrete stages, lacking clear natural boundaries. Second, there are differences across samples, and the semantic emergence process can shift or stretch on the timeline with changes in prompts. resolution, and random seeds. Furthermore, adaptively selecting specific timestep for each sample will incur substantial computational costs. Therefore, we consider statistical rules across samples to identify approximately appropriate injection timesteps.
Determine the injection timestep for attribute primitives. Research [2] indicates that cross- attention outputs converge to fixed point after several inference steps. This point divides the denois- ing process into two stages: semantic-planning and iidliit-iimprvvin Since fidelity-improvin stage primarily relies on attribute information, we select the timestep corresponding this convergence point as the injection timestep for attribute primitives. Considering statistical rules across a batch of samples, for sample, we first define the attention weights output by the h-th attention head in the m-th cross-attentin layer at inference step as Attn, (m.A)) E RQm. x Km,, where Qm and Km represent the number of query tokens and key tokens, respectively. Subsequently, we caleulate the difference of single-head attention between adjacent inference steps using the Frobenius norm:
||Attm
(m.)) Attn (m.h))
(m.N)
(m,,)
||Attm (mm.h)
||
+ 0
where serves as numerical stabilizer, typically set to 10- Then, we average across all H attention heads and all M cross-attention layers to compute the cross-attention difference for the sample:
H
(sumple)
1 1 M
3ΣΣ (m..))
A(m.))
By calculating the average cross-attention across all samples in the batch, we obtain the average cross- cosss-attntion difference A, at t inference step. We employ the moving average method to calculate the average cross-attention Å across multiple inference steps and set threshold to identify the inference step t" at which cross-attention converges:
A. 0m0
t" -=min t: A < r}.
=
where denotes the size of the moving window. Thus, the timestep corresponding inference step t" is selected as the injection timestep Satty for attribute primitives.
Determine the injection timestep for relation primitives. As previously discussed, given the entire denoising process consists of S timesteps, the first S Sattr timesteps constitute the cemantic-planning stage, during which the samples maintain relatively high signal-to-noise ratio (SNR). Research [2]] indicates that semantic concepts are primarily established at high SNR condition. By analyzing the variation of SNR across timesteps, we observe an initial rapid decline followed by a gradual decrease, as shown F Considering the prioritization order for semantic primitives object-relation- attribute, intuitively speaking, the inflection point of the SNR
Demantic-Planninng Stage
50
infection point
Timesteps (1)
Figure Inflation point of SNR.
within the first S Sattr timesteps can serve appropriate injection timestep for relation primitives.

---

We calculate the average SNR s at each timestep t across all samples in the batch for the first S Sattr timesteps, and construct discrete curve function Y/+ = g(stiry), where 0, 1..... Sattr 1. We employ the discrete maximum-curvature method to identify the inflection point. First, we calculate the first-order and second-order finite differences as follows:
Ay
yi++
Vt-1
A yt
-t-1 2yrc + Vt-1
(8)
STLF 2+1
5707
(3 -38T
Then, we calculate the discrete curvature:
A
(1 + (Ayt )3)8/2
In this way, we obtain the index of inflection point # arg max K2, which also is the injection timestep Srel for relation primitives.
Determine the injection timestep for object primitives. Considering the limited number of timesteps before Sret. further partition of these timesteps lacks theoretical and computational evidence. For simplicity, we set the timestep
the injection timestep Sob; for object primitives.
By identifying the convergence point of cross-attention and the inflection point in SNR curve during denoising, we able to adaptively determine the appropriate injection timesteps according to statistical rules and characteristics of the sample set.
3.3.2 Semantic Primitive Injection
Semantic primitive enhancement aims to increase the proportion of corresponding semantic primitives to align the semantic primitive type sensitivity of each denoising stage. We employ the cross- attention mechanism and incrementally inject each primitive type Cprim of the split-text caption CST_ Cprim is also encoded by three text encoders (CLIP-L/14, CLIP-G/14, and T5 XXL), obtaining Torim RLLxD./M Terim E RLxDaan and Tyr E L where DL/14 + DÆ/11 D' D. We then concate these token sequences along the dimension obtain the primitive injection token sequence Tprim, which is used for rross-attention calculation with T to enhance representation learning of the corresponding semantic primitives the stage.
3.4 Training Strategy
The training objective comprises two components while employing mixing coefficient A:
= LCHM + ALaun-
(10)
The first is conditional flow matching loss CCFM. which supervises denoising prediction, following the setting in MM-DiT. The other is Lamn which constrains the staged cross-attention by aligning injected semantic primitives with split text while regularizing attention behavior for stable semantic injection. It is composed of three components, weighted with empirically determined ratios of 0.6, 0.25, and 0.15:
Caun = CeCinjeca B. + T)
(11)
The first term, Linject, enforces the alignment between the cross-attention outputs and their target semantics at each activated injection stage:
Linject
Ey,
Σ st()) (Cross/Attm: Cnjject
Gaarget
(i)
(12)
The second term, Leons supervises the evolution the attention signal-to-noise ratio (SNR) to ensure proper convergence timing:
Leomy = E,
Σ
(NN ( (1) - (NR(()
(13)
7

---

The third term, Lmutex penalizes spatial overlap between attention maps from different injection stages maintain independence and prevent semantic interference:
Lmnnex = Et
Σ
(ttbb Attnaa,)) 174
(14)
Together, these components ensure stable multi-stage semantic injection by enforcing semantic precision, convergence regularity, and spatial exclusivity, which is crucial for achieving fine-gres ane-grained semantic alignment in Split-Text Conditioning.
Experiment
4.1 Experiment Settings
Impementation Details and Datasets To ensure fair comparison beyond the short-text bias of MM-DiT's original training data (CC12M (62] and ImageNet (63]), we adopt 100K samples from the SAM-LLaVA dataset [5T], which contains semantically rich, long captions suitable for our split-text method. Extended training on this dataset yields MM-DiT 2B-E and MMDiT 8B-E. Following MM-DiT 26 we use 24 transformer layers with CLIP-L, CLIP-G, and T5-XXL as text encoders. Complete captions are parsed into hierarchical inputs via Qwen-Plus [6], with ross-attentiio incrementally injected at selected
denoising steps. Using the same training protocol, we obtain two variants of our models: DiT-ST M (2B) and DiT-ST L. (8B). For evaluation, we follow Cog View3 (64] and A-DiT (65] to construct COCO-SK, assessing performance under varying text lengths and complexities, with results reported from our DiT-ST M and other competing method
Evaluation Metries
We evaluate DiT-ST using both quantitative and qualitative metrics. For quantitative analysis, we adopt GenEval [66] to assess semantic comprehension, along with FID 67 and CLIPScore [68] measure image quality and text-image alignment.
4.2 Main Results
In this section, we compare our method with MMDIT 26 and its variants baselines, given architectural similarity and shared training setup. To further evaluate the generalizability of our split- text strategy in mitigating complete-text comprehension detect, we benchmark against competitive methods, including PixArt-a [51], Flux. Dev [50,, and the state-of-the-an DreamEngine [2].
Effectiveness of Split-text Caption Form Ta- ble compares caption utilization forms terms of CLIPScore and FID, showing that split-text
Table I: Comparison of the caption forms COCO-5K.
Caption Form
CLIPScore1
FIDJ
Original Caption
31.68
24.61
caption leads to consistently better performance.
T.LM-Enhanced Caption 32.13
24.46
By parsing prompts into organized semantic primitives, our method effectively mitigates the complete-text defect, raises CLIPScore by 7.6 % and lowers FID by 10.2 % relative the other two caption forms used on MM-DiT 2B-E, while still providing +6.1 % -9.6 % gains over the L.LM-Enhanced caption, underscoreing that the observed gains primarily attributable to the caption structure, rather than to external LLM intervention.
Split- Caption
34.09
22.11
Multi- Santi-DDimnnional Semantic Comprehension Performance Table reports comparisons on the GenEval benchmark, which evaluates dimensions of semantic understanding: single-object recog- nition, multi-object reasoning, counting, color accuracy, spatial positioning, and attribute binding. Our method achieves competitive overall accuracy of 69%, matching the state-of-the-ard DreamEngine and closely approaching SDv3.5 Large (71%). despite a 4x smaller parameters. Notably, our model outperforms the SDv3 Medium baseline across all subcategories. with marked advantages in multi- object aassoning++0144 and attribute binding(+0.. 10), indicating stronger semantic comprehension. Compared Flux.1 Dev and DALL-E 3, which excel specific areas, our approach offers more balanced and robust performance across all axes due to text-split strategy.

---

Table 2: Performance on GenEvalf benchmark,whil underlined is the second best performance.
Singel object
Two object
Attribute Binding
Method
Parameter
Counting
Colors
Position
Overall
PixArt-a
0.6B
0.98
0,50
0.44
0.80
0.08
0.07
0.48
DALL-E3
0.96
0.87
0.47
0.83
0.43
0.45
0.67
SDv3 Medium
2B
0.98
0.74
0.63
0.67
0.34
0.36
0.62
Flux.1 Dev
12B
0.98
0.81
0.74
0.79
0.22
0,45
0.66
SDv3.5 Large
0,98
0.89
0.83
0.34
0.47
0.71
Dream Engine
1.00
0.94
0.64
0.81
0.27
0.49
0.69
DIT-ST Medium
2B
0.99
0.88
0.70
0.75
0.36
0,46
0.69
Beyond GenEval, we employ CLIPScore and VQAScore 66 on the curated COCO-5K dataset. These two metrics provide comprehensive evaluation: CLIPScore measures global semantic similarity, while VQAScore leverages VQA model to directly probe for complex compositional alignment via probabilistic "Yes/No" query. As results listed in Table 3 DiT-ST Medium attains CLIPScore of 34.09, eclipsing competing methods by average 8.9 %. and even markedly surpassing SDv3.5 Large (32.74) by about 4.1 %. On VQAScore, our DiT-ST Medium achieves 76.12, par with the larger SDv3.5 Large, while surpassing SDv3 Medium (75.37) and other baselines. These results consistently validate that our split-text conditioning strategy enhances both compositional semantic fidelity and cross-modal alignment, enabling the model to generate images that more accurately capture the intended meaning of complex prompts while maintaining high perceptual quality.
Table 3: Performance of CLIPScore and VQAScore on COCO-SK.
Method
PixArt-a
SDv3 Medium
Flux.1 Dev
SDv3.5 Large
DiT-ST Medium
CLIPScoreT
32.58
31.31
31.52
32.74
34.09
VQAScoret
68.86
75.37
75.19
76.49
76.12
Robust Performance Caption Length Table Texposes marked length sensitivity in existing models: PixArt-a benefits from increasing caption length before plateauing, whereas SD-series models peak on mid-length prompts and deteriorate longer ones (e.g.. SD v3 Medium falls to 27.7 in the [45, 55) subband). By contrast, our 2B-parameter model sustains a CLIPScore over 32 across all bins, rises 35.81 for the longest captions, and secures the highest overall score (34.09), outperforming SDv3 Medium and SD v3.5 Large by 8.9% and 4.1%, respectively. These results highlight the robustness and effectiveness of our split-text conditioning strategy in handling auriable-length and semantically complex prompts, enabling more stable text-image alignment across diverse input distributions while remaining parameters effcient.
Table 4: CLIPScore performance comparisons on various caption length in Selected COCO-5K.
Method
Parameter
[10.15]
[15.25)
[25,35)
[35,45)
[45,55]
Average CLIPScore
PixArt-a:
0.6B
30.99
31.99
33.26
33.37
33.29
32.58
SDv3 Medium
2B
31.91
32.69
33.37
30.86
27.73
31.31
Flux.1 Dev
12B
30.59
31.16
32.36
31.08
32.83
31,52
SDv3.5 Large
8B
32.17
32.96
34.10
32.93
31.53
32.74
DIT-ST Medium
2B
32.28
32.47
33.84
34.76
35.81
34.09
4.3 Ablation Analysis
Incremental Text Injection Incremental injection improves the text-split pipeline by scheduling semantic primitives at the appropriate timesteps. As Table this schedule rises CLIPScore by 3.9 % and simultancously reduces FID by 7.3 %. decisively outperforming the single-shot alternative. The
Table Comparison of incremental
injection under CLIPScore and FID.
Method
CLIPScoret
FIDI
w/ injection
34,09
22.11
injection
32.81
23.85
progressive delivery deepens semantic grounding, alleviates attention saturation and inter-tohe token interference, enhancing both text alignment and visual fidelity.

---

Hierarchical Caption Input Hierarchical caption input aligns naturally with the text-split strategy, it organizes diverse semantic prim- itive types into structured format for gener- evaluate ation. To assess its effectiveness, models under three input settings; original cap- tion, LLM-Enhancee caption, and hierarchical input. As Table shows, merely swapping the caption format to the hierarchical variant lifts CLIPScore by 3.6 % and reduces FID by 3 % for each architecture, with additional 2 % gains over the
Table 6: Comparison among input strategies under CLIP- Score and FID Metrics on COCO-SK.
Method
CLIPScore
FID.
Original Caption
31,31
24.66
SDv3 Medium
L.M--hhncced Caption
31.95
24.46
Hicrarchical Input
32.42
24.05
Original Caption
31.68
24.61
DIT-ST
LLM-Enhanced Caption
32.13
24.43
Medium
Hierarchical Input
32.81
23.85
captions. Crucially, SDv3 Medium enjoys the same absolute improvement as our own model, despite zero fine-tuning, demon- strating that the benefit is inherent the representation and broadly transferable across models.
More A blation Analysis
To further validate the effectiveness our model and design choices, conduct additional experiments the orders of semantic primitive injection, timestep selection, hyperparameter sensitivity, and other ablation studies. Please refer to the appendix for more details.
4.4 Visualization
High-Quae Quality Generation Visualization Figure showcases representative image samples gener- ated by our DiT-ST, with rich visual details and strong semantic alignment across diverse prompts.
(d)
(+) rustic breakfast withpoached eggs and avocado on sourdough toast, served on a ceramic platewith coffee, bathed in softmorning sunlight. purple blossoms. (c) At dusk, villagers stroll leisurely along charming cobblestone street adomed decorated with flower in the rain with their dogs.
vibrant butterfly orange
perched blooming
flower,
(d) small robin with a bright orange chest perches on arain drencheed branch, surrounded by wet green leaves.
Figure 6: High-quality generation visualization of DiT-ST Large.
More Visualiztion We provide additional visual content, including results of different caption forms (e.g. Figure (a)), performance against SDv3.5 (e.g. Figure((b)) and other competing models as well as our high-quality images at multiple resolutions, which are available in the Appendix.
5 Conclusion
This paper introduces DiT-ST, a split-text conditioning framework that alleviates the complete-tex comprehension defect in DiTs. It comprises three components. (i) Caption parsing, use LLMs to extract and organize primitives; (ii) Hierarchical input construction, construct split-text inputs and enrich input semanties; (iii) Incremental primitive injection, inject different primitives into appropriate denoising stages to improve stage-specific representation learning. Extensive experiments demonstrate the effectiveness of our split-text caption design and the excellent performance of our proposed DIT-ST. We hope this work will offer some inspiration to the diffusion community.
Acknowledgement
This work supported by the National Natural Science Foundation China (No. 62576251, No. 62376198, and No. 62576247), the National Key Research and Development Program of China (No. D022YFB1104700) and the Fundamental Research Funds for the Central Universities. The authors would like to thank Tianyu Wang and Jiu for generous support.
10

---

References
[1] Jascha Sohl-Dickstein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsuper- vised learning using nonequilibrium thermodynamics. In International conference machine learning, pages 2256-2265. pmlr, 2015.
[2] Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840-6851, 2020.
[3] Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily L Denton, Kamyar Ghasemipour, Raphael Gontijo Lopes, Bureu Karagol Ayan, Tim Salimans, al. Photorealistic text-to-in eet-tooimaae diffusion models with deep language understanding. Advances in neural information processing systems, 35:36479-360 -64944, 2022.
[4] Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High- resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684-10695, 2022.
[5] Ling Yang. Zhilong Zhang, Yang Song. Shenda Hong, Runsheng Xu, Yue Zhao, Wentao Zhang, Bin Cui, and Ming-Hsuan Hsuan Yang. Diffusion models: comprehensive survey of methods and applications. ACM Computing Surveys, 56(4):1-39) 2023.
[6] Hui Shen, Jingxuan Zhang, Boning Xiong, Rui Hu, Shoufa Chen, Zhongwei Wan, Xin Wang, Yu Zhang, Zixuan Gong, Guangyin Bao, et al. Efficient diffusion models: A survey. arXiv preprint arXiv:2502.06805. 2025.
[7] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention all you need. Advances in neural information processing systems, 30, 2017.
[8] William Peebles and Saining Xie. Scalable diffusion models with transformers. In Proceedings of the IEEF/CVF international conference on computer vision, pages 4195-4205, 2023.
[9] Hila Chefer, Yuval Alaluf, Yael Vinker, Lior Wolf, and Daniel Cohen-Or. Attend-and-excite Attention-based semantic guidance for text-to-image diffusion models. ACM transactions on Graphics (TOG), 22(4):1-10, 2023.
[10] Weixi Feng, Xuchai He, Tsu-Jui Fu, Varun Jampani, Arjun Akula, Pradyumna Narayana, Sugato Basu, Xin Eric Wang, and William Yang Wang. Training-free free structured diffusion guidance for compositional text-to-image synthesis. arXiv preprint arXiv:22 25002, 2022.
[11] Wen Li, Muyuan Fang, Cheng Zou, Biao Gong, Ruobing Zheng, Meng Wang, Jingdong Chen, and Ming Yang. Styletokenizer: Defining image style by single instance for controlling diffusion models. In European Conference on Computer Vision, pages 110-126. Springer, 2024.
[12] Tianhao Qi, Shancheng Fang, Yanze Wu, Hongtao Xie, Jiawei Liu, Lang Chen, Qian He, and Yongdong Zhang. Deadiff: An efficient stylization diffusion model with disentangled representations. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pages (693,8702, 2024.
[13] Omri Avrahami, Ohad Fried, and Dani Lischinski. Blended latent diffusion. ACM transactions on graphics (TOG), 42(4):1-11, 2023.
2023.
[14] Lorenzo Olearo, Giorgio Longari, Simone Melzi, Alessandro Raganato, and Rafael Peñaloza. How blend concepts in diffusion models. arXiv preprint 1xXX::2407.14280. 2024.
[15] Raphael Tang, Linging Liu, Akshat Pandey, Zhiying Jiang, Gefei Yang. Karun Kumar, Pontus Stenetorp. Jimmy Lin, and Ferhan Ture. What the daam: Interpreting stable diffusion using cross attention. arXiv preprint arXiv:22 10.04885, 2022.
[16] Beichen Zhang, Pan Zhang, Xiaoyi Dong, Yuhang Zang, and Jiaqi Wang. Long-clip: Unlocking the long-text capability of clip. In European Conference on Computer Vision, pages 310-325. Springer, 2024.
11

---

[17] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference machine learning, PmL.R, 2021.
pages 8748-8766
[18] Zhilin Yang, Zihang Dai, Ruslan Salakhutdinov, and William W Cohen. Breaking the softmax bottleneck: A high-rank mn language model. arXiv preprint arXiv:1711.03955, 2017.
[19] Amita Kamath, Jack Hessel, and Kai-We Wei Chang. Text encoders bottleneck compositionality in contrastive vision-language models. arXiv preprint arXiv:2305 2023.
[20] Reza Abbasi, Ali Nazari, Aminreza Sefid, Mohammadali Banayeeanzade, Mohammad Hossein Rohban, and Mahdieh Soleymani Baghshah. Clip under the microscope: fine-grained analysis of multi-object representation. arXiv preprint wwXiv:2502.198//4 2025.
[21] Jooyoung Choi, Jungbeom Lee, Chaehun Shin, Sungwon Kim, Hyunwoo Kim, and Sungroh Yoon. Perception prioritized training of diffusion models. In Proceedings of the IEEE/CVF Conference Computer Vision and Patlern Recognition, pages
[22] Wentian Zhang, Haozhe Liu, Jinheng Xie, Francesco Faccio, Mike Zheng Shou, and Jürgen Schmidhuber. Cross-attention makes inference cumbersome text-to-image diffusion models. arXiv e-prints, pages arXiv-2404, 2024.
[23] Grace Luo, Lisa Dunlap, Dong Huk Park, Aleksander Holynski, and Trevor Darrell. Diffusion hyperfeatures: Searching through time and space for semantic correspondence. Advances in Neural Information Processing Systems, 36:47500-44511 2023.
[24] Amir Hertz, Ron Mokady, Jay Tenenbaum, Kfir Aberman, Yael Pritch, and Daniel Cohen-Or. Prompt-to-pe tompp--t--pompp image editing with cross attention control. arXiv preprint 0rX1v:2208.01626, 2022.
[25] Baichuan Inc. Qwen: A scalable and multilingual language model family, 2024.
https
7 //huggingface. wtwpaieee..o/eeeeeeeeeeeeeee Qwen- 14B-Plus
[26] Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas Müller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, al. Scaling rectified flow trans- formers for high-resolution image synthesis. In Forty-first international conference on machine learning. 2024.
[27] Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. CoRR, abs/2010.02500 2020.
[28] Yu Zhang, Qi Zhang, Zixuan Gong, Yiwei Shi, Yepeng Liu, Duoqian Miao, Yang Liu, Ke Liu, Kun Yi, Wei Fan, et al. Mlip: Efficient multi-perspective language-image pretraining with exhaustive data utilization. arXiv preprint 240.v:2406.0144 2024.
[29] Bokun Wang, Yang Yang, Xing Xu, Alan Hanjalic, and Heng Tao Shen. Adversarial cross- modal retrieval. In Proceedings of the 25th ACM international conference on Multimedia, pages 164-162, 2017.
[30] Yepeng Liu, Yiren Song, Hai Ci, Yu Zhang, Haofan Wang, Mike Zheng Shou, and Yuheng Bu. Image watermarks are removable using controllable regeneration from clean noise. arXiv preprint arXiv:2410.054700 2024.
[31] Rongtao Xu, Jiguang Zhang, Jiaxi Sun, Changwei Wang, Yifan Wu, Shibiao Xu, Weiliang Meng, and Xiaopeng Zhang. Mrftrans: Multimodal representation fusion transformer for monocular 3d semantic scene completion. Information Fusion, 11 11:102493, 2024.
[32] Zixuan Gong, Qi Zhang, Guangyin Bao, Lei Zhu, Yu Zhang. Ke Liu, Liang Hu, and Duoqian Miao. Lite-mind: Towards efficient and robust brain representation learning. ACM Multimedia 2024, 2024.
[33] Zhongwei Wan, Xinjian Wa, Yu Zhang, Yi Xin, Chaofan Tao, Zhihong Zhu, Xin Wang, Siqi Luo, Jing Xiong, and Mi Zhang. D2o: Dynamic discriminative operations for efficient generative inference of large language models. arXiv preprint 2rXiv:2406.130000 2024.
12

---

[34] Kexue Fu, Mingzhi Yuan, Changwei Wang, Weiguang Pang, Jing Chi, Manning Wang, and Longxiang Gao. Dual focus-attention transformer for robust point cloud registration. In Proceedings of the Computer Vision and Pattern Recognition Conference, pages 11769-11778. 2025.
[35] Zhongwei Wan, Zhihao Dou, Che Liu, Yu Zhang, Dongfei Cui, Qinjian Zhao, Hui Shen, Jing Xiong, Yi Xin, Yifan Jiang, et al. Srpo: Enhancing multimodal IIm reasoning via reflection- aware reinforcement learning, arXiv preprint arXiv:2506. 01713, 2025.
[36] Jinzhou Lin, Han Gao, Xuxiang Feng, Rongtao Xu, Changwei Wang, Man Zhang, Li Guo, and Shibiao Xu. Advances in embodied navigation using large language models: A survey, arXiv preprint arXiv:2311.005.0
2023.
[37] Yu Zhang, Yepeng Liu, Duoqian Miao, Qi Zhang, Yiwei Shi, and Liang Hu. Mg-vit: multi muati-graaullritt method for compact and efficient vision transformers. Advances in Neural Information Processing Systems, 36:69328-69347 2023.
[38] Yepeng Liu and Yuheng Bu. Adaptive text watermark for large language models. arXiv preprint AXKi::441111322 2024.
[39] Prafulla Dhariwal and Alex Nichol. Diffusion models beat gans on image synthesis. CoRR, abs/2105.05233 055203,..053,.,. 2021.
[40] Jonathan Ho, Tim Salimans, Alexey Gritsenko, William Chan, Mohammad Norouzi, and David J. Fleet. Video diffusion models, 2022.
[41] Weimin Qiu, Jicke Wang, and Meng Tang. Self-cross diffusion guidance for text-to-image synthesis similar subjects. In 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 23528 23538, 2025.
[42] Jinjin Zhang, Qiuyu Huang, Junjie Liu, Xiefan Guo, and Di Huang. Diffusion-4k: Ultra-high- resolution image synthesis with latent diffusion models. In 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 23464-23473, 2025.
[43] Junyang Chen, Jinshan Pan, and Jiangxin Dong. Faithdiff: Unleashing diffusion priors for faithful image super-resolution. In 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 28188 281188288197 28197, 2025.
[44] Zixuan Gong, Guangyin Bao. Qi Zhang, Zhongwei Wan, Duoqian Miao, Shoujin Wang, Lei Zhu, Changwei Wang, Rongtao Xu, Liang Hu, Ke Liu, and Yu Zhang. Neuroclips: Towards high-fidelity and smooth fmrri-o-video reconstruction. In A. Globerson, L. Mackey, D. Belgrave, A. Fan, U. Paquet, Tomczak, and C. Zhang, editors, Advances in Neural Information Processing Systems, volume 37, pages $1655-51683. Curran Associates, Inc., 2024.
[45] Bencheng Liao, Shaoyu Chen, Haoran Yin, Bo Jiang, Cheng Wang, Sixu Yan, Xinbang Zhang, Xiangyu Li, Ying Zhang, Qian Zhang, and Xinggang Wang. Diffusiondrive: Truncated diffusion model for end-to-end autonomous driving. arXiv preprint arXiv:2411. 2. 15139, 2024.
[46] Nanxin Chen, Yu Zhang, Heiga Zen, Ron J Weiss, Mohammad Norouzi, and William Chan. Wavegrad: Estimating gradients for waveform generation. arXiv preprint urXiv:2009.00713 2020.
[47] Kunpeng Qiu, Zhiqiang Gao, Zhiying Zhou, Mingjie Sun, and Yongxin Guo. Noise-consistent caaeese-diffsiion for medical image synthesis and segmentation, 2025.
[48] Xiaomin Li, Mykhailo Sakevych, Gentry Atkinson, and Vangelis Metsis. Biodiffusion: A versatile diffusion model for biomedical signal synthesis, 2024.
[49] Olaf Ronneherger, Philipp Fischer, and Thomas Brox. biomedical image segmentation, 2015.
U-net: Convolutional networks for
[50] Black Forest Labs. Flux. https: /atp::://ittbbt http:p///Ahhthhcooo//aaaddddeoes labs/flux 2024.
13

---

[51] Junsong Chen, Jincheng Yu, Chongjian Ge, Lewei Yao, Enze Xie, Yue Wu, Zhongdao Wang, James Kwok, Ping Luo, Huchuan Lu, and Zhenguo Li. Pixart-o: Fast training of diffusion transformer for photorealistic text-to-image synthesis, 2023.
[52] Liang Chen, Shuai Bai, Wenhao Chai, Weichu Xie, Haozhe Zhao, Leon Vinci, Junyang Lin, and Baobao Chang. Multimodal representation alignment for image generation: Text-image interleaved control easier than you think, 2025.
[53] Dixuan Wang, Yanda Li, Junyuan Jiang, Zepeng Ding, Guochao Jiang, Jiaqing Liang, and Deqing Yang. Tokenization matters! degrading large language models through challenging their tokenization. arXiv preprint arXivv2405.17066 2024.
[54] Richard Dichl Martinez, Zébulon Goricly, Andrew Caines, Paula Buttery, and Lisa Beinborn. Mitigating frequency bias and anisotropy in language model pre- pre-training with syntactic smooth- ing. arXiv preprint arXiv:2410. 11462, 2024.
[55] Tsung- Han Wu, Long Lian, Joseph E Gonzalez, Boyi Li, and Trevor Darrell. Self-correcting llm-controlled diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 6327 6336, 2024.
[56] Tuna Han Salih Meral, Enis Simsar, Federico Tombari, and Pinar Yanardag. Conform: Contrast is all you need for high-fidelity text-to-image diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9005-90144 2024.
[57] Arman Zarei, Keivan Rezaei, Samyadeep Basu, Mehrdad Saberi, Mazda Moayeri, Priyatham Kattakinda, and Soheil Feizi. Improving compositional attribute binding in text-to-image generative models via enhanced text embeddings, 2025.
[58] Luping Liu, Chao Du, Tianyu Pang, Zehan Wang, Chongxuan Li, and Dong Xu. Improving long-text alignment for texi-to-image diffusion models, 2025.
[59] Ketan Suhaas Saichandran, Xavier Thomas, Prakhar Kaushik, and Deepti Ghadiyaram. Progres- sive prompt detailing for improved alignment in texl-to-image generative models, 2025.
[60] Ziwei Yao, Ruiping Wang, and Xilin Chen. Hifi-score: Fine-grained image description eval- uation with hierarchical parsing graphs. In European Conference on Computer Vision, pages 441-458. Springer, 2024.
[61] An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, et al. Qwen2. 5 technical report. arXiv preprint 1rXiv:2412.15115. 2024.
[62] Soravit Changpinyo, Piyush Sharma, Nan Ding, and Radu Soricut. Conceptual 12m: Pushing web-scale image-text pre-training to recognize long-tail visual concepts, 2021.
[63] Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large- scale hierarchical image database. In 2009 IEEE Conference on Computer Vision and Pattern Recognition, pages 248-255, 2009.
[64] Wendi Zheng, Jiayan Teng, Zhuoyi Yang, Weihan Wang, Jidong Chen, Xiaotao Gu, Yuxiao Dong, Ming Ding, and Jie Tang. Cogview3: Finer and faster text-to-image generation via relay diffusion, 2024.
[65] Pengtao Chen, Mingzhu Shen, Peng Ye, Jianjian Cao, Chongjun Tu, Christos-Savvas Bouganis, Yiren Zhao, and Tao Chen. 8-dit; A training-free acceleration method tailored for diffusion transformers, 2024.
[66] Dhruba Ghosh, Hanna Hajishirzi, and Ludwig Schmidt. Geneval: An fojused framework for evaluating text-to-image alignment, 2023.
[67] Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by two time-scale update rule converge to a local nash equilibrium, 2018
14

---

[68] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agar- wal. Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural language supervision, 2021.
[69] Zhiqiu Lin, Deepak Pathak, Baiqi Li, Jiayao Li, Xide Xia, Graham Neubig, Pengchuan Zhang, and Deva Ramanan. Evaluating text-to-visual generation with image-to-ter generation. In European Conference on Computer Vision, pages 366-384 Springer. 2024.
15

---

A Dataset Details
We further introduce the datasets used our work from two aspects: training and evaluation.
For training, we observe that existing datasets used by baseline models MM-DiT 26
are suboptimal
for complex caption-conditioned generation. Specifically, datasets like CC12M
62 have relatively
short captions (average length 10.3), and ImageNet [63] provides only class- supervision. Inspired by Pixart-a [51], which employs LLaVA refine raw captions into high- www.iiimmtttondeenaaee text, we adopt the SAM-LLaVA dataset provided by Pixart-a to enhance textual supervision. We select 100K pairs from SAM-ILa LL (available in our released CSV files) to further train MM-DiT 2B and 8B, aiming to improve their capacity on long-text generation and ensure fair comparisons.
For evaluation, we construct COCO-5K, following settings from Cog View3 [64 and A-DiT (65] It comprises 5,000 image-text pairs sampled across multiple caption-length intervals to assess performance under varying textual complexity. Additionally, we report gFID scores based on COCO- 30K to assess distribution-lvel image fidelity. Complete results are provided in Appendix
B Implementation Details
In terms of implementation, we focus on the settings of key thresholds and corresponding code-level realizations. We set the moving average window size to 3, allowing each A. to consider the current and two preceding steps for better capturing the SNR trend. To ensure numerical stability when computing normalized differences, we set 0 = 10 s For identifying the convergence point, we use a convergence threshold т = 10-4, During inference, we select 40 denoising steps from the full diffusion range (0~1000) via uniform random sampling. When determining the injection layers, we allow flexible injection intervals. For example, around timestep 50, we inject attribute primitives within 10 timestep range, while around timestep 400, we permit broader window of +40 timesteps, reflecting the varying stability of attention dynamics at different diffusion stages.
C Further Evaluation and Model Comparison
To reduce metric variance and ensure fair model comparison, we construct a fixed 30K image-text subset from COCO for consistent evaluation across architectures and training scales. We evaluate and compare our method against several representative baselines, including SDv3 Medium, SDv3.5 Large, Flux 1 Dev, Pixart-a. The evaluation results, in terms of FID, summarized in Table 7 On the fixed COCO-30K subset, our DiT-ST Large model obtain the lowest FID scores. At the 2B scale, DiT-ST Medium reaches 18.78, narrowly beating SD v3 Medium (18.82) by 0.2 %. With 8B parameters, DiT-ST Large achieves 17. 16,16, improving on SD v3.5 Large (17.31) by 0.9 %, a modest yet consistent gain that shows split-text conditioning still reduces late-stage artefacts even short COCO captions. Flux 1 Dev (3).10) and PixArt-a (27.35) post FIDs about 40% higher than DiT-ST Large. Flux. tuned for fast continuous-token sampling and lacking COCO fine-tuning, yields texture artefacts; PixArt-a, trained on long, stylised captions, mismatches short, photorealistic domain. Thus, DiT-ST equals or surpasses SD baselines and clearly outperforms models trained on divergent data.
Table 7: Performance of FID on COCO-30K.
Method
PixArt-a
SDv3 Medium
Flux. Dev
DiT-ST Medium
SDv3.5 Large
DiT-ST Large
Param. 0.6B
2B
12B
2B
FID]
27.35
18.82
32.10
18.78
17.31
17.16
D Ablation on Injection Order of Semantic Primitives.
In this experiment, we investigate how varying the injection order of semantic primitives, namely object, relation, and attribute, affects model performance. While our default configuration follows an objecctrelation-attiiiuee order based on semantic granularity and generation stability, we test alternative permutations to examine the sensitivity of DIT-ST Medium to ordering strategies. Table g reveals the cost of premature information exposure. Advancing attribute tokens from their intended attribuee-objectrrlltion cuts CLIPScore to 29, 19.41 and
late-stage slot to
16

---

31.10, drops of 14 % and 9 % from the default order. Early injection forces fine- teee-ggaandd colour and texture cues into an unstable attention landscape: later denoising steps overwrite or mis-bind these details, sharply reducing semantic fidelity.
Table 8: Comparison of different injection order for semantic primitives.
Order
CLIPScoret
attribute-relation obbjct
29.41
cerribut--ojeet-rrlateee
31.10
relation-objer ciailooo-obec--tttiiii
32.52
relation-aiis caation-atribbut-oojjee
30.16
object-attributte-rlaaie
30.79
object-relation -atrrbbbe (our setting)
34.09
E Ablation on step selection strategy for semantic primitive injection.
We further investigate the impact of injection timestep choices for different semantic primitive types in the diffusion denoising process. Given that semantic primitives differ in abstraction level and generation dependency, we design a progressive injection schedule tailored to each type:
Object tokens are injected the early stages of denoising (timesteps O, 10, and 20), where the model primarily focuses on establishing global layout and coarse structural elements. We experiment with these positions to compare their impact and identify the most effective timing for guiding the foundational composition of the generated image.
Relation tokens are injected during mid-stage denoising, Specifically, we begin with timestep 25 the midpoint of [0, 50], and extend timestep 75 with five evenly spaced steps (30, 40. 50, 60, 70). We experiment with these positions to assess their impact and determine the optimal timing for enhancing spatial and compositional coherence.
Attribute tokens are injected during the transition from semantic interpretation visual detail refinement, when the model shifts focus from global structure to fine-grained features. To study their impact on visual modeling, we take the midpoint of the [50, 400] range as reference and select timesteps at 100-step intervals spreciliipee 200, 300, 400, 500, and 600-for evaluation.
This step allocation strategy reflects the assumption that lower-level semanties (e.g., object identity) should be introduced early to guide coarse synthesis, while higher-level or localized attributes benefit from later-stage injection when image fidelity and semantic detail are resolved.
As shown in Table deviating from the default impairs text- eext-imaee alignment. Postponing object injection roughly 5-6 %, underscoring the need for early global cues. Advancing relation tokens to f = 30 reduces the score by about %, as premature relational reasoning diverts attention from still-forming objects. The most severe loss, nearly 8 %, occurs when attribute tokens are introduced at / - 200; exposing fine-grained details before the scene stabilised disrupts subsequent refinement as claimed in our main paper the question of premature information exposure. In aggregate, mis-timed injections can degrade alignment by average 5%, confirming that stage-aware scheduling is critical for maintaining semantic fidelity.
relation-attributtt (0 (O-R-A) schedule markedly 10-20 lowers CLIPScore by
Table Comparison of Different Step Selection Strategy by CLIPScore(CS))
Step
O(10)
O(20)
R(30)
R(40)
R(60)
R(70)
A(200)
A(300)
A(500)
A(600)
Default
CSt
32.71
32.24
32.57
32,73
32.76
32.45
31.26
31.75
32.96
32.83
34.09
F Ablation on Sliding Window Size
Table T0 shows that CLIPScore peaks at 34.09Our when the SNR-smoothing window is set to w/ 3.A narrow window (wr 1) amplifies local noise, shifting the detected inflection point and trimming the score to 33.16 (-2.7 %). Expanding to 2 alleviates some volatility, yet still lags
17

---

the optimum at 33.55 (-1.6 %). Oversmoothing with = 4 and = 5 postpones relation-toke token injection; scores decline 33.72 (-1.1 %) and 33.43 (-1.9 %), respectively. These results confirm that an intermediate window is narrow enough to capture the true SNR consitionnthereil yielding the most consistent relation-level guidance and highest text-image alignment.
wide enough to suppress spurious curvature spikes yet
Table 10: Comparison of different sizes of sliding window.
Window Size
CLIPScoret
33.16
2
33.55
3
(our setting)
34.09
33.72
5
33.43
G
Ablation on Text Encoding Refinement.
As detailed in Section 3.2.2 our design includes refinement for the classical text encoder of MM- DiT. Specifically, we fully utilize the previously wasted dimension capacity by incorporating the complete caption to enrich the overall semantic information of the input. To further validate the effectiveness of this refinement, we conduct an ablation experiment on our DiT-ST Medium model, trained for 5 epochs on 50K SAM-LLaVA dataset. The results are presented in Table 11 Without text encoding refinement, the model performance has declines of 0.62% in CLIPScore and 0.54% in FID metrics. The result indicate that although the improvement from adding complete text captions modest, it indeed contributes positively to model performance without introducing additional computational or architectural overhead.
Table 11: Comparison of w/ and w/o Text Encoding Refinement.
Method
CLIPScore*
FIDI
w/ Text Encoding Refinement
32.33
24.18
w/o Text Encoding Refinement
32.13
24.31
H
Limitation
The injection timestep selection operates the batch level rather than the sample level. This is because sample-level adaptation would significantly increase computational cost and reduce inference efficiency, coupled with the current lack of research on sample-level adaptation in the conditioning of denoising processes. Therefore, this study adopts batch-level adaptation design compromise.
The split-text caption may weaken cross-primitive semantic dependency. Some special long- range dependencies (e.g., irony) could be split, leading to biased semantic understanding in the model. To mitigate this issue, DiT-ST introduces the text encoding refinement design, which incorporates original caption to compensate for the loss. Furthermore, existing works in the same field have not explored evaluating the semantic fidelity of special cases such as irony.
The evaluation metrics lack human subjective assessment. The evaluation DiT-ST primarily relies CLIPScore and FID, aligning with other works in the same filed. These metrics lack human perception, particularly in fine- innineri details (e.g., color tone, detail consistency). To compensate for this limitation, similar other prior works, provide extensive visualizations to facilitate subjective evaluation by readers.
18

---

I
More Visual Comparison Across Caption Forms
Original Caption (Complete-Ted) Textt)
LLM Enhanced Caption (Complete-Teer Text)
Our Caption (Split Text)
Orighna:: quiet beach with soft sond, scattered white seashells, One small bird stonds alons near the water, its feet in the shallow weves, looking down et its reflection. Sentle woves roll behind it. LLM--rbeneet There seashells bird. [OBEECT] beach, its reclection, waves roll behind the bird. (ATTRIBUTTE) The beach is quiet. The sand soft. The seashells are white and scattered. The bird small and alone. The woves are gentle and shallew.
beach with soft sund and scottered white seashells. small bird stonds alone near the water, looking ot reflection. feet woves, bird looks
beach has and seashells, bird
Original quiet forest filled with tall pine trees, young deer with dots on its back stands still the soft moss underfoot.
LLM-Bhhanrd There are quiet forest, tall pine treas and young deer. The deer stonds still on soft moss fleor. Our OERECT) forest. deer. [RELATION] forest has tall trees. deer stands [ATTRIBUTE] The forest is quiet. The
and pine.
young and with moss is soft.
Original: yellow school bus drove through raity I.M.Paaanodd
night. The damp road reflects the yellow shop lights roodside, heavy falling.
yellow school bus driving through a roiny city street at nighT. The wet road reflects the yellow lights of nearby shop
lights, The rain falls heavily and steadlly
Oun (OSIECT] school bus. street. shop. lights. rain, RRE ATION bus drives on damp street. shop has lights. street reflects lights, [ATTRIBUTE] The yellow. The street domp. The lights yallow. The heavy.
Original: At morning oofé, mon flipping through newspaper by the window, with worm sunlight shining ocross the wooden toble and coffee beside
man sitting by the morning café, reading newspoper. Worm sunlight shines across
table,
cup of coffee beside him. Our: (OBJECT) cofé, mon, newspoper. window, sunlight. toble, coffee, [RELATION] man sits by the window: man flips through newspoper. sunlight
shines the table. coffee beside the (ATTRBUTTE The café marning. The table weoden. The sunlight
19

---

Original Caption (Complete-Tee
LLM- Enhanced Caption (Complete- Text)
Our Caption (Split- Text)
Original: furry white cat quietly osleep, nestled a worm, brown blonket adorned with delicate embroider ombroidered floral patterns and roised stitching, LLM--aaaanodd There furry white cat sleeping in worm br brown blonket. The blankest is deconated with delicate embroidered floral potterns and roised stitching.
[DBLECT] blanket. patteres. stitching [RELATION] sleeps in blonket. blonket adorned with floral potterns, blanket has roised stitching. (ATTRIUTT) cat furry and white. The blonkett worm brown, flonel potterns are delicate and embroidered. stitching is raised.
CCCCCr
table beside blonk white spiral-bourd notebook, notebooks lie nearby. LLM--aasnodd There silver pen without cap resting on a wooden table next to blank white spiral-bourd, notebook. Several other notebooks ore scatfered nearby.
Orighnal: pen without cap
Out (OBJECT pen. table, notebook. [RELATION] rests on table, pen beside notebook, other notebooks lie nearby. (ATTRIBUTE) The pen silver uncapped. table wooden. The notebook blank, white, spiral-bound.
hebefom.
2C
Original: A crispy fried chicken burger with lettuce, pickles, and cream souce on top, placed beside container of golden French fries on
weeden table.
crispy
burger topped lettuce, pickles, cream sauce, sauce, sitting next container golden French
fries en wooden table.
Our (CBIECT, burger. lettuce. pickles. cream souce. fries. containen. table [RELATION] burgar has lettuce, pickles, and cream souce. burger is
beside fries. fries cream souce on top. fries are golden, weoden.
container. burger and fries the table. (ATTRIBUTE) The burger crispy and fried. The lettuce, pickles and
Original: healthy meal with two egg yolk facing fried eggs, fresh half cut avocados, and few slices of tocsted whole wheat bread, lightly seasored with herb.
LLM--ahanock There healthy meal consisting
sunny sunny-side-up side-up eggs, fresh evocados, a few slices of toested wheat
bread, lightly seasoned with herb.
Our: (OSEECT) meal, avecodas bread. herb, (RELATION] meal includes avecades, and bread. meal is seasoned with herb, [ATTRIBUTE] The meal healthy. The eggs friid and yalks The avocados fresh and halved, The bread whole wheat teasted.
Figure 7: More comparisons of different caption form and corresponding performances.
20

---

More Visual Comparison Across Different Models
In this section, present more qualitative comparisons across different models. Specifically, we employ our large-scale model DiT-ST Large and compare with the baseline MM-DiT 8B-E under identical prompt conditions. As shown in Figure 8 DiT-ST Large produces more compositionally coherent and semantically aligned results, especially in complex scenes with diverse attributes.
SDv3.5 Large
Our DIT-ST Large
SDv3.5 Large
Our DIT-ST ST-STLan Large
AA KYX
K
cat sits calmly the table in front of keyboard and monitor, scattered papera, and few books. Through the window is green grass touch.
A collection of vintage animal figurines arranged on a wooden floor front dark wooden cabinet filled toy animals
white cat gently stared at a bird perched on clay pot before the
pair of elegant bronze women states sits on bench, each holding handbag green hedge.
black dog sitting driver's of bright red vintage truck, looking out of the window with its resting on the door.
A brass ceiling fan with wooden blades and built-in light fixture, mounted above white bed in minimal, modern bedroom.
little baby standing near to the apple boxes touching apples, the apples mixed of green color. Figure Comparisons between MM-DIT 8B-E (left) and our DIT-ST Large (right)
setting up blue sundy beach dense green foliage with man in yellow T-shirt watching by
We further compare DiT-ST Large with several hate-of-the-art text-to-image generation models, including PixArt-a, Flux, and HunyuanDiT. As illustrated in Figure 10 our model demonstrates competitive or superior visual quality, with stronger object fidelity, spatial consistency, and semantic expressiveness.
21

---

Our DI
Flux Dev
PixArt-a
HunywanDiT
A person is skiing on snow, wearing gloves, helmet, goggles, and a jacket, with a flag and trees in the background.
bread roll with a golden, crispy exterior, coated in pink icing and sprinkled with white sugar pearls
group of toddy bears wrapped in blanket, with pillows underneath.
Under shed with wearing cap working behind table with piles of oranges and juice machine.
The box contains three types of food, with many fried items placed grease-abworbing paper lining.
6 4
A multi-colered wall clock is mounted on the wall, surro unded by few sculptures, lubel above reads "1106." " Figure 9: Comparisons among DiT-ST Large, Flux, PixArt-a and HunyuanDiT.
22

---

K
More High Quality Images Generated by Our DiT-ST Large
Figure 10: High-quality 1024x 1024 images generated by our DiT-ST Large
23

---

Besides the standard 1024x 1024 images shown in Figure 10 we further generate multi-scale samples put Figure confirming that our DiT-ST Large maintains visual fidelity and semantic alignment across different resolutions and formats.
Figure 11: High-quality and multi- scale generation results DiT-ST Large
24

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification:The abstract and introduction clearly outline the core contributions of our work DiT-ST, including the motivation, methodological innovations, and performance gains.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation long is clear that these goals are not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: We acknowledge the limitations of our work and will provide a dedicated discussion in the appendix. Rather than aiming surpass all existing baselines, contribu- tion focuses on developing an effective and generalizable method that performs consistently across tasks and domains. We believe this design choice better serves long-term robustness and transferability.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed not penalize honesty concerning limitations.
3. Theory assumptions and proofs
25

---

Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
Justification:
The paper includes the necessary theoretical results with all assumptions
clearly stated and referenced in the respective sections. All proofs and lemmas are either presented within the main paper
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proof' can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification: To ensure full reproducibility, we release all necessary resources, including model weights, training and evaluation code, and preprocessed datasets. Detailed instructions for environment setup and running experiments are provided in GitHub repository.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often good way to accomplish this, but reproducibility instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of model checkpoint, other means that appropriate to the research performed.
also be provided via detailed
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) the contribution is primarily a new algorithm, the paper should make clear how to reproduce that algorithm.
(b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
26

---

(d)
We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes]
Justification: We release both source code and necessary data to facilitate faithful repro- duction of our results. The code repository includes training and inference scripts and environment configuration files. In addition, we provide access to all datasets used in our experiments on huggingface.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips... cc/
for more details.
While we encourage the release of code and data, we understand that this might be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https http///oooboossiooooooooooo Pololloicci for more details.
/ //nips ptaaps.cc//ublcc/guddde
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing much information possible in supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Question:
Answer: [Yes]
Justification: We describe all experimental settings both the main paper and the appendix. Key implementation details such as data splits, model architecture, optimizer type, learning rate schedules, and hyperparameter configurations are summarized in the main text to support result interpretation.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
27

---

Answer: [Yes]
Justification: We evaluate our method using commonly adopted standard metrics, including [e.g.. FID, CLIPScore, or others], ensure comparability with existing baselines. In addition to quantitative evaluation, we also provide visualizations of generated outputs to support qualitative assessment and highlight consistency across samples. These visual results, alongside statistical metrics, offer comprehensive understanding of model behavior.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by enror bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
OK to report 1-sigma error bars, but one should state The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that out of range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification: We specify in the main paper that all experiments were conducted on NVIDIA A100 Each training run took approximately 12-24 GPU hours.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics //te:///neuupppe htth:///iiiorrrraoo.//oo///a///a
Answer: [Yes]
Justification: We have carefully reviewed the NeurlPS Code of Ethics and confirm that our research fully conforms to principles. Our work does not involve sensitive user data, human subjects, or harmful applications. All datasets used are either publicly available or properly licensed, and we ensure transparency, reproducibility, and responsible reporting throughout the paper.
28

---

Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special cireumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. if there is special consid- eration due to laws or regulations their jurisdiction).
Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: Our work introduces novel and generalizable perspective for improving existing generative models, aiming to address foundational challenges in a model-agnostic manner. Rather than optimizing for narrow task-specific gains, our approach seeks uncover transferable mechanisms that can broadly applied across architectures and domains. This may positively influence future research toward more unified and interpretable generative modeling. While our method is not designed for deployment in sensitive or high-risk applications, acknowledge the broader implications of advancing generation quality, and support responsible usage and release protocols.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., degloyment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For cxample, it is legitimate to point out that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated relcase of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [Yes]
Justification: We acknowledge that high high-quality generative models may pose risks of misuse, such as unauthorized image synthesis or disinformation. To mitigate such risks, we commit to releasing our pretrained models and code under a non-commercial research-only license. We also include usage guidelines outlining responsible applications, and plan to implement content safety filters restrict harmful outputs. Furthermore, we will monitor the downstream use of our models and reserve the right to restrict access if misuse is identified.
Guidelines:
29

---

The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators original owners of assets (e.g.. code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: We make proper attribution to all external assets used in our work. Specifically, we use the COCO 2017 dataset, which publicly available under Creative Commons Attribution 4.0 License (CC-BY 4.0), and we follow its terms of use. Our experiments are based on the Stable Diffusion v3 model provided by Stability AI. which is released under custom non-commereial license; we cite the official source and follow the stated restrictions. We also build on open-source libraries from Hugging Face, such as Transformers and Diffusers, which are released under the Apache 2.0 license.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (c.g.. website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of in the package should be provided. For popular datasets, paperswi tecoiihooodeeooo com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that re-packaged, both the original license and the license of the derived asset (if has changed) should provided.
If this information is not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: We do not introduce any new assets in this work. All experiments are conducted using publicly available datasets (e.g.. COCO 2017) and existing pretrained models (e.g., Stable Diffusion v3). We ensure that these assets are properly cited and used in accordance with their respective licenses.
Guidelines:
The answer NA means that the paper does not release assets.
Researchers should communicate the details the dataset/code/model. as part their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
30

---

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsoureing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: Our work does not involve any form of crowdsourcing or research with human subjects. All experiments are conducted using publicly available datasets and pretrained models without collecting new sumann-rovided annotations or feedback.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: This work does not involve research with human subjects crowdsourcing. All experiments are conducted using publicly available datasets and pretrained models, without collecting new human annotations, feedback, or participation. Therefore, no IRB or equivalent ethical approval is required.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and expect authors adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, originality of the research, declaration is not required.
Answer: [Yes]
Justification: We used the Qwen Plus API, large language model, to assist in processing and structuring textual data used during pre-processing. This usage does not directly affect the core model architecture training process and can be replaced by any other LLM models, but facilitates intermediate data organization and annotation required for our pipeline. We describe this usage and its scope in Appendix G ensure transparency and reproducibility.
31

---

Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to our LLM policy for what should or should not be described.
32


Reviews:
[{"review_id": "yMNbdsViYt", "reviewer": "Reviewer_6Z8X", "summary": "This paper introduces a new framework DiT-ST which aims to handle the inherent limitations of existing text-to-image models. Specifically,  understanding complex long texts through a technique called split-text conditioning is not trivial. \nThis framework leverages a large language model to parse a long textual description into a set of simpler, hierarchical short phrases. \nThese phrases are then incrementally injected into different stages of the denoising process. \nThe experiments show that it significantly improves the generated image's detail accuracy and semantic consistency and the method outperforms or rivals state-of-the-art models with minimal parameter increasement.", "questions": "The authors are encouraged to discuss the differences to the refered paper and complement necessary experiments.", "limitations": "Yes.", "rating": 3, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths:\n- This paper uses a simple appraoch to let the model more accurately understand and generate fine-grained details from the input. The method is splitting complex text into simple subject-relation-attribute meta-phrases. \n- The method is parameter-efficient and architecture-agnostic as the core idea lies in preprocessing the input text and injecting it in stages without requiring major modifications to the model architecture. \n- Experiments show that this method performs well in very long prompts with satisfying CLIPScore performance. Also on the comprehensive benchmark GenEval the DiT-ST model outperforms the SDv3 Medium baseline in many dimensions. \n\nWeaknesses:\n\n- This paper lacks an important reference of \"Progressive Prompt Detailing for Improved Alignment in Text-to-Image\nGenerative Models\" (https://arxiv.org/pdf/2503.17794v1) which has a very similar method on the same issue. This papper should discuss the differences. \n- The evaluation is not enough. The evaluation metrics are not comprehensive. Though FID is adopted to evaluate the image quality while it is limited. Also, this paper only adopts GenEval and CLIPScore to test the text-image alignment without a user study. \n- The experiments are not comprehensive. Only one LLM is adopted for the experiments. Other LLMs should also be tested for the effectiveness and generalization. Also, this paper does not discuss if finetuning LLM works better for this task. \n- The experiment does not include failure cases and analyze them. \n- The captions should be more detailed to better illustrate the content and the motivation of the figures and tables.", "was_revised": true, "final_justification": "Thank the authors for partially addressin the concerns and the reviewer decided to raise the score to 3.", "rebuttal": "Hi, Reviewer 6Z8X. We sincerely appreciate your careful review and so many constructive comments. We value each of them and provide responses below:\n\n>**W1: Difference between our work and SCoPE**\n\nThank you for the vital reference! We were already aware of this work and cited it in Line 104. We fully agree with your suggestion and summarize detailed differences in **method** and **experiment**: \n\n**+Method**\n\n① Method process.\n\nBoth DiT-ST (Ours) and SCoPE aim to mitigate semantic overload through progressive injection, thereby enhancing the model's understanding of captions and improving generation quality, However, the two methods differ fundamentally in design.\n\nSCoPE employs LLM to segment captions into multiple sub-captions progressing from coarse to fine-grained. The timestep for progressive injection is determined by calculating similarity among sub-captions and applying proportional similarity ratios. While SCoPE achieves caption segmentation through this training-free strategy, it cannot distinguish semantic primitives (object, relation, attribute), thereby remaining vulnerable to semantic competition and concept coupling problems outlined in our introduction.\n\nWe employ LLM to split captions into three semantic primitives—objects, relations, and attributes—constructing independent encoding streams. By utilizing SNR curvature and cross-attention convergence, we can easily identify optimal injection timesteps, enabling the model to first establish object concepts before incrementally incorporating relation and attribute. Our method effectively mitigates semantic competition and significantly enhances both generation quality and semantic consistency.\n\nTherefore, SCoPE tends to focus on coarse-grained input control during the inference stage, whereas ours represents a fine-grained structural optimization method targeting internal model architecture and modeling processes. DiT-ST demonstrates superior interpretability and controllability.\n\n② Model structure.\n\nSCoPE, as a method based on the U-Net architecture, has a multi-scale convolutional design that is restricted by a local receptive field, which makes it difficult to adequately capture long-distance dependencies and has limited scalability. SCoPE tends to be a heuristic inference enhancement rather than a scalable training framework.\n\nOurs is based on the Diffusion Transformer, which is naturally adapted to larger scales and more complex scenarios. Global self-attention mechanism has capabilities for long-distance semantic modeling and complex scenario understanding, and makes our method preliminarily follows the scaling law.\n\n**+Experiment**\n\n① Evaluation metrics. \n\nSCoPE uses CLIPScore and VQAScore. \n\nWe use FID, CLIPScore, GenEval and VQAScore (Inspired by the reference you provided). \n\n② Base models. \n\nSCoPE uses U-Net-based diffusions: SD 1–4, SD 2.1, and SDXL. \n\nWe use Transformer-based diffusions: SDv3 Medium and SDv3.5 Large. Thus, our method preliminarily follows the scaling law (Table7, `Reviewer kbep` L1 and `Reviewer kbep` W3 & Q2). \n\n③ Experiment evaluation. \n\nSCoPE only evaluates VQAScore and CLIPScore for proposed methods and base models, without more main results or any ablation studies. \n\nOur main experiments evaluate FID, CLIPScore, GenEval, and VQAScore (added in W2 based on your suggestion), along with model performance under different caption lengths and model sizes. We include not only base models (SD) but also Transformer-based diffusions such as PixArt-α, DALL-E 3, Flux.1 Dev, Dream Engine. In addition, extensive ablation studies further validate our design's effectiveness.\n\n④ Method performance. \n\nSCoPE improves CLIPScore by up to 1.3% over the base model (SDv2.1).\n\nOurs improve CLIPScore by up to 2.78% over the base model (SDv3 Medium).\n\nAs two methods aren’t directly comparable, we use CLIPScore gain as a rough reference. Notably: 1.SCoPE is training-free, which is its advantage. 2.SCoPE adopts prompt enhancement, it’s unclear how much of the gain comes from their design instead of the added information. 3.SCoPE generates eight candidate outputs per prompt and selects the best-aligned one, while we use a single random one.\n\n---\n\nIn addition, our models and datasets are now publicly available. Compared to 4-page reference, our paper offers greater detail, completeness and more visualizations. Generally, SCoPE is still an interesting work. We will incorporate above analysis into the final version to strengthen our study.  Sincerely thank you for your insightful input!\n\n&nbsp;\n\n>**W2: More comprehensive evaluation**\n\nWe fully agree with you. CLIPScore and FID have inherent limitations, as they cannot adequately evaluate image naturalness, fidelity, and rationality. Therefore, we additionally adopted GenEval for more comprehensive evaluation, such as semantic consistency, fidelity, clarity, diversity, and plausibility. While many other metrics exist, these three are the most widely adopted in the field. Unfortunately, when seeking to conduct broad comparative analysis, the number of applicable universal metrics becomes limited. Fortunately, VQAScore from the reference you provided is also a general benchmark similar to GenEval. In order to reserve enough space for responding to your other valuable feedback, we kindly direct you to Reviewer `4VWJ` W4 for our VQAScore evaluation. Thank you for the inspiration!\n\nAs for the user study, we acknowledge this limitation in the appendix (lines 969–973). As it's a common limitation in all similar works [9][10][22][26][39][45] (same reference in our submission) [r1][r2][r3][r4], we regret we couldn't conduct one. As an alternative, we offer extensive visualizations and available models to facilitate your subjective perception and evaluation. We apologize once again for our inability and sincerely appreciate your understanding. \n\n[r1] Reconstruction vs. Generation: Taming Optimization Dilemma in Latent Diffusion Models, CVPR 2025\n\n[r2] Language-Guided Image Tokenization for Generation, CVPR 2025\n\n[r3] SANA: Efficient High-Resolution Text-to-Image Synthesis with Linear Diffusion Transformers, ICLR 2025\n\n[r4] SinSR: Diffusion-Based Image Super-Resolution in a Single Step, CVPR 2024\n\n&nbsp;\n\n> **W3: LLM-related experiments**\n\nThank you for pointing out our overlooked experiments. We add 2 experiments:\n\n① Different LLMs. Besides initial Qwen-plus, we also use GPT-3.5-turbo and Gemini-2.0-flash and evaluate them on COCO-5K. (50K dataset, 5 epochs, base model: SDv3 Medium)\n\n|LLM|CLIPScore↑|FID↓|\n|-|-|-|\n|w/o (Original caption)|31.05|24.70|\n|Qwen|32.33|24.18|\n|GPT|32.28|24.15|\n|Gemini|32.22|24.23|\n\nWe observe that using any LLM improves performance, with Qwen performing best. This demonstrates the effectiveness and generalization of LLMs in our method.\n\n② LLM finetuning: Prompting is viewed as LLM finetuning. Besides our prompt, we also compare a simple prompt and w/o prompt on COCO-5K. The simple prompt is: *Extract all objects, relation between object and attributes from the description.* (50K dataset, 5 epochs, base model: SDv3 Medium)\n\n|Method|CLIPScore↑|FID↓|\n|-|-|-|\n|Our prompt|32.33|24.18|\n|Simple prompt|32.18|24.41|\n|w/o prompt|30.45|24.52|\n\nIt shows that prompt-based LLM finetuning improves performance, with better prompts yielding better results. Without prompting, LLM tends to generate low-quality expansions [r5], hindering semantic understanding of captions.\n\n[r5] Exploring the Role of Large Language Models in Prompt Encoding for Diffusion Models, NeurIPS 2024\n\n&nbsp;\n\n> **W4: Failure case analysis**\n\nFirst, we sincerely apologize that due to NeurIPS 2025 policy restrictions, we are prohibited from using or updating any links in the rebuttal, so we cannot show you visualized images. We appreciate you pointing out this issue, which is very necessary. A comprehensive analysis needs to include failure cases.\n\n+**Prompt of High Complexity**\n\nThe first case uses a prompt with complex syntax and layered semantics, *describing a studio scene with ivy-covered windows, a weathered violin, wine glass, lace-covered table, and and a hidden cat watching dust*. Both our model and SDv3 failed to fully capture some attributes (e.g., the violin’s aged texture), likely due to implicit and long-range phrasing. Still, our model better preserves spatial cues such as the cat’s occlusion and gaze, often missed by baselines. While imperfect, our method generally retains more semantic detail in such complex prompts.\n\n+**Prompt of Irrelevant Information**\n\nThe second case shows failure due to missing visual grounding in the input. The prompt *You may come for your appointment at 9 a.m. tomorrow.* contains no visual object or scene. Although users may expect a hospital-related image, all models—including ours—produce irrelevant outputs. This reflects a limitation of T2I models when prompts lack explicit semantics.\n\nWe hope these cases help clarify the scope and limitations of our method under different prompt conditions.\n\n&nbsp;\n\n> **W5: More detailed captions**\n\nThanks for your suggestion! We promise to detail captions of tables and figures to better illustrate the content and the motivation.\n\n&nbsp;\n\n&nbsp;\n\nOnce again, we sincerely thank you for dedicating your valuable time to review our submission and offering so many insightful and constructive suggestions. We are committed to incorporating the above analyses and experiments you recommended into the final version, as a way to truly honor your time and effort. Honestly, you are the most influential reviewer for the outcome of this submission, and we eagerly look forward to engaging with you further during the discussion period and hope to gain your continued guidance and encouragement. If you have any other questions or feel that any of our explanations are unclear, please kindly let us know—we will be happy to provide further detailed clarifications. Thank you very much once again!\n\n&nbsp;\n\nBest wishes and regards,\n\nAll authors of Submission 510"}, {"review_id": "2BXWUmjUyF", "reviewer": "Reviewer_Dd4v", "summary": "Text-to-image diffusion transformers are typically conditioned on the entire caption during inference. When the caption is long or describes complex relationships between objects, the model struggles to understand the full input. This is due to how tokens compete with each other and how they are positioned in the sentence.\nTo address this, the paper proposes a method that splits the caption into a hierarchical structure based on its semantic components and their relationships. It has been shown that the convergence of cross-attention in diffusion transformers indicates when semantic objects are formed and when details are refined during inference. By analyzing this convergence, the proposed method decides when each part of the hierarchical structure should be given as condition to the model during inference.\nExperiments show that the method performs better on the COCO-5K benchmark and achieves results comparable to state-of-the-art methods on the GenEval benchmark.", "questions": "Given that the caption is divided hierarchically, we would expect each text input to the model to be shorter. Since the method uses a pretrained encoder (three encoders, in fact), part of the encoder input might not be used. The paper mentions that the full caption is still provided as input to fully use the encoder. Does this actually improve model performance?\n\nAre there any changes in memory or computational cost when using the proposed method?\n\nCould the method also be applied during training? I'm not asking the authors to implement this, but I would appreciate a discussion of the potential challenges and benefits.", "limitations": "Limitations are discussed in the appendix.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "The paper is clearly written and easy to follow. The proposed method is simple to implement and can be applied to multiple models at inference time.\n\nThe idea of conditioning the model on different semantic parts of the caption at different inference steps is original, and using cross-attention convergence to guide this process seems sound.\n\nThe experimental results are convincing, and the ablation studies show that the method’s components contribute effectively.\n\nOverall, this is a solid paper. I look forward to seeing the other reviews and the authors’ responses.", "was_revised": true, "final_justification": "The authors addressed my questions. I have read the other reviews and replies and I can confidently recommend this paper to be accepted.", "rebuttal": "Hi, Reviewer Dd4v. We sincerely appreciate you taking time for review and providing   encouraging and constructive feedback! We value each of them and provide responses below:\n\n>**Q1: Ablation study for Text Encoding Refinement**\n\nThank you for pointing out this issue, and we sincerely apologize for the absence of this necessary ablation study. During our investigation, we discovered that when utilizing MM-DiT's encoder, the concatenation of two token sequences results in partial dimension capacity remaining underutilized. Therefore, we incorporated complete-text captions in this component, which is our proposed Text Encoding Refinement. Consequently, we conduct ablation experiment on Text Encoding Refinement with DiT-ST Medium. (50K dataset, 5 epochs, base model: DiT-ST Medium)\n\n|Method|CLIPScore↑|FID↓|\n|-|-|-|\n|w/ Text Encoding Refinement|32.33|24.18|\n|w/o Text Encoding Refinement |32.13|24.31|\n\nAs shown in the table, without Text Encoding Refinement, the model performance has declines of 0.62% in CLIPScore and 0.54% in FID metrics. Although the improvement from adding complete-text captions is modest, this demonstrates that Text Encoding Refinement indeed contributes positively to model performance. We commit to incorporating this ablation study into the final version. Thank you once again for helping make our paper more complete and solid!\n\n&nbsp;\n\n>**Q2: Analysis of memory usage and computational cost**\n\nFirst, we analyze the additional memory overhead introduced by Split-Text Conditioning by listing the formula. The additional memory cost mainly arises from text embedding caching and cross-attention operations. \n\nWe employ text encoders such as CLIP-L $[B,77,768]$, CLIP-G $[B,77,1280]$ and T5-XXL $[B,256,4096]$ with $B$ for batchsize, $[77,256]$ for the number of token respectively and $[768, 1280,4096]$ for embedding  dimension. Given a full-text length $L$, object, relation, and attribute segments $L_O, L_R, L_A$, embedding dimension $d$ , batch size $B$, and fp16 precision, the memory required for a single text embedding is $M_{\\text{text, single}} \\approx B \\cdot L \\cdot d \\cdot \\mathrm{sizeof(fp)}$ .  After O-R-A segmentation, we cache three segmented embeddings and the full-text embedding, yielding $M_{\\text{text, ORA}} \\approx 4 \\cdot B \\cdot L \\cdot d \\cdot \\mathrm{sizeof(fp)}$ . Since $L_O + L_R + L_A \\approx L$ , the main increase comes from storing multiple embeddings rather than longer sequences.\n\nFor cross-attention, which dominates the additional memory overhead, let the number of image tokens be $N = (imgsize // 2)²=512 \\times 512$,  the number of attention heads $H=28$, and the per-head dimension $d_h=64$, giving a total hidden dimension $d=H \\times d_h=1792$ . The key-value (KV) cache memory for a single cross-attention layer is $M_{\\text{KV}} \\approx B \\cdot H \\cdot N \\cdot L_{\\text{avg}} \\cdot d_h \\cdot \\mathrm{sizeof(fp)}$ . Since our method injects three cross-attention stages during denoising, the total KV cache memory is $M_{\\text{KV, total}} \\approx 3 \\cdot B \\cdot H \\cdot N \\cdot L_{\\text{avg}} \\cdot d_h \\cdot \\mathrm{sizeof(fp)}$ , while the attention logits require an additional $M_{\\text{logit}} \\approx 3 \\cdot B \\cdot H \\cdot N \\cdot L_{\\text{avg}} \\cdot \\mathrm{sizeof(fp)}$ . Therefore, the overall memory cost is $M_{\\text{total}} \\approx 4 \\cdot B \\cdot L \\cdot d + 3 \\cdot B \\cdot H \\cdot N \\cdot L_{\\text{avg}} \\cdot d_h + 3 \\cdot B \\cdot H \\cdot N \\cdot L_{\\text{avg}}$ , where the first term corresponds to text embedding caching, the second to KV cache, and the third to attention logits, with cross-attention being the dominant factor while convergence analysis is negligible. \n\nIn terms of computational cost, evaluated on an NVIDIA H200 GPU, the overhead of Split-Text Conditioning is modest and results are as follow:\n\n|Strategy|Throughout(img./s)↑|GFLOPs(G/img.)↓|\n|-|-|-|\n|DiT-ST w/ original caption w/o injection (512×512)|0.2262|605|\n|DiT-ST w/ hierarchical input w/o injection (512×512)|0.2248|710(+17.4%)|\n|DiT-ST w/ hierarchical input w/ injection (512×512)|0.2161|727(+20.2%)|\n|DiT-ST w/ original caption w/o injection (1024×1024)|0.0572|7310|\n|DiT-ST w/ hierarchical input w/o injection (1024×1024)|0.0571|7410(+1.4%)|\n|DiT-ST w/ hierarchical input w/ injection (1024×1024)|0.0558|7530(+3%)|\n\nAt 512×512, replacing the original caption with hierarchical input increases GFLO\nPs by 17.4% but has negligible impact on throughput (0.2262 → 0.2248 img/s), as text processing is a minor component of the overall cost. Adding injection further raises GFLOPs by 20.2% and slightly reduces throughput (0.2248 → 0.2161 img/s) due to the additional cross-attention computations.\n\nAt 1024×1024, where image generation dominates the cost, hierarchical input adds only 1.4% GFLOPs with no throughput change, while injection incurs an extra 3% GFLOPs with a small throughput drop (0.0571 → 0.0558 img/s).\n\nOverall, text-related overhead becomes negligible at high resolution, and injection introduces modest, manageable additional cost.\n\n&nbsp;\n\n>**Q3: Discussion of challenges and benefits during training**\n\nThank you for your question. Our DiT-ST model **already incorporates staged semantic injection during training**, and this design has provided clear empirical benefits. First, training-time injection explicitly guides the model to learn the temporal scheduling of object, relation, and attribute conditioning, leading to more robust semantic disentanglement compared to inference-only strategies. Second, it improves generalization and controllability: by integrating injection into the optimization process, the model not only performs well within the training distribution but also exhibits stronger interpretability and stability for long prompts and complex scenes. Third, training-time injection stabilizes the behavior of cross-attention modules, reducing the need for additional heuristic scheduling at inference time.\n\n However, we also note that this approach introduces additional costs, including higher memory consumption and longer training time due to multi-stage injection and the associated cross-attention parameters and KV caches. Moreover, the scheduling hyperparameters for injection require careful tuning during training.\n\nFor comparison, ControlNet, a well-known approach that **also integrates conditioning during training**, demonstrates similar benefits: it enables the model to directly learn the correspondence between control signals (e.g., edges, depth, pose) and image generation, yielding strong controllability at inference. Yet, ControlNet also faces limitations, such as increased parameter count from additional conditional branches and the need for specialized annotated data, which reduces its flexibility.\n\nIn contrast, SCoPE performs **interpolation only at inference time**, which avoids the training overhead but fails to expose the model to semantic injection during optimization. As a result, SCoPE does not fully resolve semantic entanglement and lacks explicit modeling of different semantic types (objects, relations, and attributes). This inference-only design can lead to less stable performance, especially for long prompts or compositionally complex scenes.\n\nIn summary, incorporating injection during training—as we do in DiT-ST—offers clear advantages for semantic disentanglement, interpretability, and controllability, distinguishing it from inference-only approaches such as SCoPE.\n\n\n&nbsp;\n\n&nbsp;\n\nOnce again, we sincerely thank you for dedicating your valuable time to review our submission and providing such positive feedback. We are committed to incorporating the above analyses and experiments you recommended into the final version, as a way to truly honor your time and effort. Your encouragement is an important reason why we are willing to continue participating in the rebuttal process. We eagerly look forward to your continued support, which will play a crucial role in the success of this submission. Therefore, we very much anticipate further discussion with you during the discussion period. If you have any other questions, please kindly let us know, and we will be happy to response them enthusiastically. Thank you!\n\n&nbsp;\n\nBest wishes and regards,\n\nAll authors of Submission 510"}, {"review_id": "1hCD9pjsZl", "reviewer": "Reviewer_1VDF", "summary": "Summary\nThis paper proposes DiT-ST, a novel split-text conditioning framework for text-to-image diffusion models. Instead of injecting the full caption all at once—which can be semantically dense and challenging for the model to parse—DiT-ST decomposes the caption into multiple semantic primitives (e.g., objects, attributes, relations) using a large language model. These primitives are then hierarchically sorted and injected at different denoising timesteps via cross-attention during the generation process. The motivation is that the progressive nature of diffusion is better suited for staged semantic guidance. Experiments show consistent improvements over strong baselines, demonstrating improved generation quality and semantic alignment", "questions": "Please refer to the weakness section.", "limitations": "yes", "rating": 3, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths\n\n• The paper is clearly written, and the core idea—decomposing long captions and aligning semantic primitives with different timesteps—is intuitive and elegant.\n\n• The proposed semantic scheduling strategy represents a meaningful extension of earlier works that perform conditioning at fixed stages. By tailoring the conditioning granularity and injection timing, DiT-ST introduces a more structured and interpretable control mechanism.\n\n• The method is modular, requires no changes to the underlying diffusion backbone, and can be readily adapted to existing models.\n\n• The experiments are comprehensive, spanning both qualitative visualizations and quantitative results on multiple datasets and text complexity levels. The SNR-based analysis is particularly insightful and well-motivated.\n\n• The paper includes ablations on primitive types, text parsing, and injection schedules, helping the reader understand where the gains come from\n\nWeaknesses\n\n• The formulation in Equation (10) lacks clarity—particularly, the attention loss term L_{\\text{attn}} is not defined in full. It’s unclear what form it takes, what layers it applies to, and how it affects optimization. This should be made explicit for reproducibility.\n\n• The choice of injection timesteps for different semantic primitives (e.g., attributes) is motivated by the convergence of cross-attention outputs, but this assumption is not backed by empirical visualizations. As with Figure 5 (which illustrates the SNR inflation point), the paper would benefit from an analogous figure showing how cross-attention actually converges over time.", "was_revised": true, "final_justification": "After reading the SCoPE paper mentioned by another reviewer, I believe the novelty of this work is indeed weakened, though I still find some merits in the work, so I will change my recommendation to borderline reject.", "rebuttal": "Hi, Reviewer 1VDF. We sincerely appreciate you taking time for review and providing encouraging and constructive feedback! We value each of them and provide responses below:\n\n>**W1: More detailed elaboration on Equation 10**\n\nWe thank the reviewer for the question regarding the design of our attention loss. Our proposed loss term $L_{attn}$  is specifically designed to regulate cross-attention behavior in the multi-stage semantic injection process of diffusion models, ensuring stable training, semantic alignment, and independence between injection stages. The loss is formulated as: \n\n$L_{attn} = \\alpha \\cdot L_{inject} + \\beta \\cdot L_{conv} + \\eta \\cdot L_{mutex}$\n\nwhere $\\alpha = 0.6, \\ \\beta = 0.25, \\ \\eta = 0.15$ , and a global scaling factor $\\lambda = 0.25$ are used in practice.\n\nThe first term, $L_{inject} = E_{t,x_0} \\left[ \\sum_{i=1}^3 \\delta_i(t) \\, ( CrossAttn_i(c_{base}, c_{inject}^{(i)}) - c_{target}^{(i)} )^2 \\right]$ , enforces the alignment of cross-attention outputs with the target semantics at each activated injection stage, thereby improving semantic precision.\n\nThe second term, $L_{conv} = E_t \\Big[ \\sum_{i=1}^3 \\big( SNR_{attn}^{(i)}(t) - SNR_{target}^{(i)}(t) \\big)^2 \\Big]$, supervises the evolution of the signal-to-noise ratio (SNR) of attention across timesteps to ensure timely convergence, avoiding premature or delayed semantic injection that could otherwise lead to generation bias.\n\nThe third term, $L_{mutex} = E_t \\Big[ \\sum_{i \\neq j} \\delta_i(t) \\cdot \\delta_j(t) \\cdot \\text{Overlap}(\\text{AttnMap}_i, \\text{AttnMap}_j) \\Big]$ , prevents spatial overlap between attention maps of different stages, thereby reducing semantic interference and preserving the independence and interpretability of each injection phase. \n\nTogether, these components form a principled objective that not only stabilizes training but also enhances fine-grained semantic alignment, which is critical for the effectiveness of our split-text conditioning strategy in complex prompt scenarios.\n\n&nbsp;\n\n\n>**W2: Visualization of attention at various injection timesteps**\n\nFirst, we sincerely apologize that due to NeurIPS 2025 policy restrictions, we are prohibited from using or updating any links in the rebuttal, so we cannot show you visualized images. We appreciate you pointing out this issue, which is very necessary. A comprehensive analysis needs to include failure cases.\n\nTo determine the attribute injection timestep $T_{\\mathrm{inject}}$, we analyze cross-attention maps during denoising. For each candidate $T_{\\mathrm{inject}}$, we inspect both the map at this timestep and the following 25 noise level $(T_{\\mathrm{inject}}, T_{\\mathrm{inject}}+25)$ to verify convergence. The cross-attention map is defined as:  \n\n$A_t = \\frac{1}{M \\cdot H} \\sum_{m=1}^M \\sum_{h=1}^H \\mathrm{Softmax}\\left(\\frac{Q_t^{(m,h)} K_t^{(m,h)T}}{\\sqrt{d_h}}\\right)$,\n\nwhere we further compute entropy $H(A_t)$ for dispersion and contrast ratio $C(A_t) = \\frac{\\max(A_t)}{\\mathrm{mean}(A_t)}$ for focus.  \n\nAt $T_{\\mathrm{inject}}$, entropy drops and contrast rises sharply, indicating a shift from diffuse global attention to localized attribute focus. Over the next 25 noise levels, these metrics stabilize, confirming effective attribute-level conditioning. Furthermore, we compare attention maps at three candidate ranges: 350–375, 400–425, and 450–475. \n\nIn the first range (350–375), the model remains in a global layout construction phase, with attention maps characterized by **broad, low-intensity coverage** lacking attribute-specific focus. In contrast, in the last range (450–475), the model has already transitioned into an overly localized state, where attention maps exhibit **prematurely concentrated patterns with reduced semantic plasticity**, indicating that attribute injection would come too late. Only the 400–425 range displays a clear transition from global to localized attribute-focused attention, showing **sharp, high-contrast regions** precisely aligned with attribute semantics. Based on this observation, we select 400 as the attribute injection step in the main text.\n\nThese results validate our injection-step selection and provide interpretable evidence of stage-wise semantic alignment. \n\nWe sincerely thanks for your valuable suggestions and agree on the importance of this analysis. We apologize once again for not being able to include direct visualizations in the rebuttal due to NeurIPS 2025 policy restrictions. We commit to providing comprehensive visualizations and detailed analyses in the camera-ready version to address this point thoroughly.\n\n\n\n&nbsp;\n\n&nbsp;\n\nOnce again, we sincerely thank you for dedicating your valuable time to review our submission and for providing such encouraging feedback. We are committed to incorporating the analyses and experiments you recommended into the final version, as a way to truly honor your time and effort. Although the rebuttal policy limits our ability to provide relevant visualizations that could further enhance the completeness and interpretability of our work, we genuinely hope to receive your continued support in the subsequent stages, which will be crucial to the success of this submission. We look forward to further discussions with you during the discussion period. Thank you again and we eagerly await your response.\n\n&nbsp;\n\nBest wishes and regards,\n\nAll authors of Submission 510"}, {"review_id": "9QWp8dxRW1", "reviewer": "Reviewer_kbep", "summary": "The paper approaches to improve T2I diffusion models by splitting text captions into smaller, independent, sentences that contain different (hierarchical) semantic primitives, e.g., general image layout vs vs details for specific objects. These primitives are then injected into the diffusion model at different timesteps, starting with more general information at the beginning of the sampling process and ending with detailed object information at the end of the sampling process. The evaluation shows improved scores on GenEval, CLIP score, and FID.", "questions": "The approach needs additional processing both at training and test time. It would be interesting to do some ablations comparing the effect of different ways of doing the split-text caption construction. It seems relatively involved right now and sounds somewhat brittle.\n\nSimilarly, there is some extra design involved in how and when to inject the different kinds of captions. One interesting baseline would be to just feed the split-text captions into a \"normal\" pretrained models (without any finetuning) at the given timesteps to check the kinds of improvements this would lead to (if any).\n\nIt also seems like right now the conditioning also always includes (a projection of) the full caption. How important is that? Does it still work if it only conditions on the split-text conditions? \n\nHow will this approach affect potential down-stream tasks (such as editing) which would be finetuned from a pretrained T2I model?", "limitations": "The finetuning of the models was done with a dataset that only contains 100K examples. This is relatively small-scale and injecting human biases is likely to help here. It is not clear to me that we would see the same kinds of improvements when a general diffusion model is trained on a large-scale dataset of longer and more complex prompts. Table 2 also shows that models trained on a dataset of more variety also perform well in most settings. It is not clear that using the split-text condition is really helpful compared to training on a larger dataset with diverse, long, and detailed captions.\n\nAdditionally, the approach requires some specific design choices (e.g., how exactly to create the split-text captions, when and how to inject them, etc) and it is not obvious to me that this will easily generalize across different datasets and model architectures.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "The paper addresses an important topic of improving text-image alignment in diffusion models. The breaking down of a complex instruction into various smaller sub-instructions makes intuitive sense. Structuring the creation of the sub-instructions with the well-known behavior of diffusion models to first generate broad layouts and later on focus on semantic details also makes sense.\n\nThe paper finds ways to automatically address these, by automatically creating sub-captions and proposing an automated way to find when to inject them into the model. That being said, the evaluation is only done on one kind of model so it is not clear if this transfers easily to other models and whether the injection approach also works on other models.\n\nHowever, there are many ways to split a caption into sub-sections and it's not clear that the one proposed here is the best one or transfers to different models. It might also depend on the kind of data that a model is trained on. \n\nThis also leads to extra compute requirements, both at train and at test time, since the input captions have to be processed to the split-text format. It also requires additional finetuning of the model, though the approach could also work without finetuning (though this is not tested in the paper I think).", "was_revised": true, "final_justification": "Thank you for the detailed rebuttal. It has addressed my concerns and I have updated my rating to accept.", "rebuttal": "Hi, Reviewer kbep. We sincerely appreciate your careful review and so many constructive comments, especially for pointing out some limitations that we had not previously recognized. We value each of them and provide responses below:\n\n>**W1: Transferability to other models**\n\nWe appreciate you highlighting this issue. We recognize that this experiment is essential for validating the transferability and generality of our method. Therefore, we conduct a comparative study. Our method initially employs SD as the base model, now, we expand our evaluation to include other Diffusion Transformer models. We apply our method to Flux .1 Dev and HunyuanDiT to assess its effectiveness across other models.\n\n|Method|CLIPScore↑|FID↓|\n|-|-|-|\n|SDv3 |30.84|24.77|\n|Ours-SDv3 (i.e. DiT-ST Medium)|32.33|24.18|\n|Flux|31.52|26.69|\n|Ours-Flux|33.25|25.16|\n|HunyuanDiT|32.04|25.24|\n|Ours-HunyuanDiT|32.93|25.07|\n\nExperiment setting: 50K dataset, 5 epoch, base model: SDv3 Medium. The experiment results demonstrate that applying our method to other models, namely Flux and HunyuanDiT, also yields good performance. This demonstrates the effective transferability of our method across same model architectures Given that these models share a common transformer-based architecture, they can similarly leverage cross-attention mechanisms for injection.\n\n&nbsp;\n\n>**W2 & Q1: Different caption splitting strategies**\n\nThank you for raising this question. First, our caption splitting strategy can be transferred to other models and achieves good performance (as jointly demonstrated by Table 5, your W1, your W3 & Q2 experiment results). Second, We conduct additional comparative experiments involving two alternative splitting strategies that were explored but ultimately not adopted during this research. One is simple sentence splitting, and the other is progressive semantic primitive extraction. Specifically, the simple sentence splitting employs LLM to split all complex sentences in the caption into simple sentences. The progressive semantic primitive extraction employs LLM to extract all objects, relations, and attributes, then reconstructs the original caption into a set of three progressive sentence collections following O∪O-R∪O-R-A. The comparative results against our final adopted  strategy are as follows:\n\n|Method|CLIPScore↑|FID↓|\n|-|-|-|\n|Original Caption |31.46|24.65|\n|Ours (i.e. Hierarchical Input)|32.33|24.18|\n|Simple Sentence Splitting|32.18|24.41|\n|Progressive Primitive Extraction |31.74|24.56|\n\nExperiment setting: 50K dataset, 5 epoch, base model: DiT-ST Medium. As demonstrated, our adopted splitting strategy with hierarchical input achieves the best performance. Regarding your concern in the weakness section about whether our proposed strategy is the best one. We firmly believe that more effective splitting strategies will be developed in the future. However, among all the strategies we have explored thus far, our splitting strategy delivers the best results. If you have better ideas, we would be delighted to engage in further discussion with you, and even potentially collaborate on future research in this area.\n\n&nbsp;\n\n>**W3 & Q2: Results without model fine-tuning**\n\nThis is an interesting question. Building upon Table 5, we supplemented our evaluation with experiments using models without any fine-tuning. Specifically, we compare: vanilla SDv3 Medium, SDv3 Medium fine-tuned on our 100K dataset with original captions for 20 epochs (corresponding to SDv3 Medium in Table 5), and DiT-ST Medium trained on our 100K dataset using our method (corresponding to DiT-ST Medium without primitive injection in Table 5). The comparative results for both original caption and split caption are presented below.\n\n|Method|CLIPScore↑|FID↓|\n|-|-|-|\n|Vanilla SDv3+ Original Caption|30.84|24.77|\n|Vanilla SDv3+ Split Caption|31.52|24.52|\n|Finetuning SDv3+ Original Caption|31.31|24.66|\n|Finetuning SDv3+ Split Caption|32.42|24.05|\n|DiT-ST+ Original Caption|31.68|24.61|\n|DiT-ST+ Split Caption|32.81|23.85|\n\nWe found that compared to original captions, split captions can achieve better performance even in vanilla SD without any fine-tuning, but fine-tuned models provide greater performance gains.\n\n&nbsp;\n\n>**Q3: Ablation study for Text Encoding Refinement**\n\nThank you for pointing out this issue, and we sincerely apologize for the absence of this necessary ablation study. During our investigation, we discovered that when utilizing MM-DiT's encoder, the concatenation of two token sequences results in partial dimension capacity remaining underutilized. Therefore, we incorporated complete-text captions in this component, which is our proposed Text Encoding Refinement. Consequently, we conduct ablation experiment on Text Encoding Refinement with DiT-ST Medium. (50K dataset, 5 epochs, base model: DiT-ST Medium)\n\n|Method|CLIPScore↑|FID↓|\n|-|-|-|\n|w/ Text Encoding Refinement|32.33|24.18|\n|w/o Text Encoding Refinement |32.13|24.31|\n\nAs shown in the table, without Text Encoding Refinement, the model performance has declines of 0.62% in CLIPScore and 0.54% in FID metrics. Although the improvement from adding complete-text captions is modest, this demonstrates that Text Encoding Refinement indeed contributes positively to model performance. We commit to incorporating this ablation study into the final version. Thank you once again for helping make our paper more complete and solid!\n\n&nbsp;\n\n>**Q4: Effect on downstream tasks**\n\nThank you for raising this question. We agree that assessing a method’s generality and applicability to downstream tasks is important. Our approach is well-suited for such tasks—including editing—due to its lightweight and modular design. Specifically, split-text conditioning does not alter the pretrained diffusion backbone, but instead redesigns the conditioning mechanism by injecting semantic components (object, attribute, relation) incrementally during denoising, rather than feeding the full prompt at the initial step. This progressive injection aligns better with the evolving nature of generation and ensures compatibility with downstream pipelines.\n\nTaking image editing as an example, which often requires localized modifications (e.g., changing object color or inserting attributes), our method provides two main benefits. First, semantic disentanglement during training allows fine-tuning to target specific elements without affecting unrelated regions, enabling precise compositional edits. Second, staged injection introduces temporal disentanglement, permitting edits at the right step (e.g., modifying attributes without disturbing layout), thus improving controllability. As an architecture-agnostic, plug-and-play module, our approach naturally complements methods like LoRA or T2I-Adapter for editing tasks.\n\n&nbsp;\n\n>**L1: Scaling law for dataset size**\n\nThank you for pointing out this overlooked issue. We design two related experiments to investigate the scaling law regarding data scale. We use DiT-ST Medium as the base model and, using the same method employed to create our 100K dataset, generate additional 50K and 500K datasets. We trained the base model on datasets of 50K, 100K, and 500K scales for 5 epochs each, then conduct comparative analysis.\n\n|Dataset|CLIPScore↑|FID↓|\n|-|-|-|\n|50K|32.33|24.18|\n|100K|32.71|23.92|\n|500K|33.84|23.40|\n\nBoth metrics show clear benefits from data scaling: CLIPScore increases and FID reduces, indicating gains in both semantic alignment and visual quality as training data grows.\n\n|Dataset|[10,15]|[15,25)|[25,35)|[35,45)|[45,55]|Average CLIPScore|\n|-|-|-|-|-|-|-|\n|50K|31.84|32.46|33.40|31.63|32.32|32.33|\n|100K|31.87|32.53|33.60|32.42|33.12|32.71|\n|500K|32.36|33.01|34.04|34.45|35.68|33.84|\n\nBy comparing the CLIPScore performance of models trained on different data scale across various caption lengths, we observe that larger training datasets lead to better overall performance. Moreover, models trained on larger datasets demonstrate particular improvements on longer and more complex captions.\n\nTo be honest, these two experiments are insufficient to fully demonstrate the model's scaling law with respect to data scale. However, due to resource and funding limitations, we are unable to create larger datasets for further exploration. Therefore, we believe our method **preliminarily** follows the scaling law regarding dataset. We will incorporate this limitation into the final version with more detailed analysis. Thank you for the inspiration!\n\n&nbsp;\n\n>**L2: Limitation of universality**\n\nWe appreciate this important reminder. Although our discussion in W1 demonstrates that our method can be transferred to Flux and HunyuanDiT, we honestly acknowledge that our method may not be applicable to all diffusion models—a common limitation shared by similar works. Therefore, we decide to add this limitation discussion to the limitation section in the final version to enhance the depth of our research and analysis. Thank you once again for your constructive and insightful feedback.\n\n&nbsp;\n\n&nbsp;\n\nOnce again, we sincerely thank you for dedicating your valuable time to review our submission and for providing so many constructive comments. We are committed to incorporating the analyses and experiments you recommended into the final version, as a way to truly honor your time and effort. To be honest, you are a pivotal reviewer whose feedback could be decisive for the outcome of this paper. We sincerely hope to earn your encouragement and support. Due to the character limit, we can only provide brief responses at this stage. We very much look forward to engaging in further discussion with you during the discussion period. If you have any additional questions or find any of our explanations unclear, please don't hesitate to let us know—we would be happy to provide further clarifications. Thank you once again!\n\n&nbsp;\n\nBest wishes and regards,\n\nAll authors of Submission 510"}, {"review_id": "yWSutilLG5", "reviewer": "Reviewer_4VWJ", "summary": "This paper proposes a text-to-image generation method using split-text as a condition. Existing text-to-image models struggle to comprehend complex semantic information, so the authors attempt to refine the text using large language models, constructing multiple complex sub-texts based on three dimensions: object, relation, and attribute. The authors also note that these three types of prompts have different impacts on the model during various stages of denoising in the diffusion model, leading to the design of a staged injection mechanism. Experiments show that this design effectively improves the model's ability to follow prompts.", "questions": "Please refer to the weakness.", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "Strengths:\n\n1. The author's use of a graph to model the text is an interesting approach. This method of normalizing the prompt can be helpful for weak text encoders.\n\n2. The author conducted experiments on multiple models to further validate the effectiveness of the approach.\n\n3. The paper is easy to understand.\n\nWeaknesses:\n\n1. Due to the limited text understanding capabilities of models like CLIP and T5, many existing works use powerful LLMs as text encoders for diffusion models[1,2,3]. Can this design further improve performance when used with LLMs with stronger understanding abilities?\n\n2. In the paper, the authors inject object, relation, and attribute into the denoising process in three separate stages. However, since object and relation jointly determine the layout of the generated image, could using only the object prompt in the early denoising stages affect the accuracy of the relation, potentially leading to extra objects or incorrect relationships? The authors should analyze the output results from denoising with different prompts.\n\n3. Regarding the cost analysis of Injection Timestep Selection, for different diffusion models, will these injection timesteps differ significantly, and what kind of time and resource overhead will this bring?\n\n4. The evaluation benchmarks are weak. ClipScore and FID do not reflect the model's understanding of complex prompts. The authors should use additional benchmarks with LLMs as evaluators for ablation studies. They should also provide more showcases to demonstrate that the fine-tuned model has improved prompt comprehension.\n\n5. Using a graph to model prompts lacks completeness. Some prompts don't contain fully defined semantic information, which is common in platforms like MidJourney. For a prompt that doesn't specify an object, what kind of output will the model generate?\n\n[1] https://arxiv.org/abs/2406.11831\n\n[2] https://arxiv.org/abs/2410.10629\n\n[3] https://arxiv.org/abs/2505.10046", "was_revised": true, "final_justification": "After reading the author's rebuttal, my concerns have been addressed. I choose to raise my score.", "rebuttal": "Hi, Reviewer 4VWJ. We sincerely appreciate your careful review and so many professional and constructive comments, especially providing many useful and insightful references. We value each of them and provide responses below:\n\n>**W1: Powerful LLMs as encoders**\n\nThank you for this interesting question. We reference these three works, with particular focus on the Complex Human Instruction from [2] to enhance image-text alignment. We conduct experiments using Qwen2-VL-1.5B and Gemma2 as text encoders. (50K dataset, 5 epochs & 20 epochs, base model: DiT-ST Medium)\n\n|TextEncoder|CLIPScore↑|FID↓|\n|-|-|-|\n|5 epochs|||\n|Ours|32.33|24.18|\n|Replaced by Qwen2-VL-1.5B|33.01|23.78|\n|Replaced by Gemma2|32.93|23.81|\n|20 epochs|||\n|Ours|33.51|23.37|\n|Replaced by Qwen2-VL-1.5B|34.74|21.64|\n|Replaced by Gemma2|34.59|21.75|\n\nAs shown in the table, replacing our original text encoder with LLMs for 5 epochs training yields slight improvements in both CLIPScore and FID. Considering the short training duration, the model performance may be unstable and cannot provide strong evidence. Therefore, we further tested the model performance with 20 epochs of training. We found that as the scale increases, the effects become more obvious, indicating that using LLMs as text encoders can further improve generation quality. This is a very interesting and worthwhile question for deeper investigation. Thank you for providing this valuable insight.\n\n&nbsp;\n\n>**W2: Analysis of different injection processes**\n\nThank you for your inspiration. Although we discussed the impact of different injection orders on performance in Table 8 in the appendix, we acknowledge that as you mentioned, object and relation jointly determine the layout of the generated image. Therefore, we supplement the following experiments based on Table 8: We replace the object injection at t=0 with 0.5object+0.5relation, 0.8object+0.2relation, and 0.2object+0.8relation respectively, to explore the joint effect of object and relation at the beginning of denoising.\n\n|Injection Order|CLIPScore↑|FID↓|\n|-|-|-|\n|O-R-A (ours)|34.09|22.11|\n|0.5O+0.5R|33.37|23.52|\n|0.8O+0.2R|33.85|23.84|\n|0.2O+0.8R|32.23|24.28|\n\nSurprisingly, the experimental results are counterintuitive. The effect of jointly injecting object and relation at the beginning of denoising is not as good as injecting object alone. Due to time constraints, we have not yet found the clear reason for this phenomenon. We speculate that this should be closely related to the semantic entanglement we discussed in the introduction, and ensuring independent modeling of objects at the beginning is crucial. Furthermore, for the 1000-step denoising process of SD, injecting object at t=0 and relation at t=50 can both be viewed as early-stage injection. For our O-R-A injection order, it can both ensure that object and relation jointly determine the layout of the generated image in the early stage, while also taking into account the decoupled modeling of object concepts, thus achieving better results. This is a very interesting topic, and we will continue to explore it in the future to verify the above speculation. Thank you again for your inspiration.\n\n&nbsp;\n\n>**W3: Injection timestep selection and overhead analysis**\n\n+**injection timestep selection**\n\nTo validate the generality of our cross-attention and SNR-guided injection strategy, we conduct experiments on two additional architectures: Flux .1 Dev and HunyuanDiT-v1.2. By analyzing cross-attention convergence and SNR curves, we determine the O-R-A injection steps for each model and compare them with alternative steps. As shown below, injecting at the identified steps consistently yields the best performance, while other steps lead to degradation, confirming the method’s robustness.\n\n|Flux Step|O(10)|O(20)|R(20)|R(30)|R(50)|R(60)|A(100)|A(200)|A(400)|A(500)|default|\n|-|-|-|-|-|-|-|-|-|-|-|-|\n|CLIPScore |32.56|32.38|32.40|32.46|32.49|32.31|31.51|31.89|32.64|32.56|33.25|\n\n|Hunyuan Step|O(10)|O(20)|R(80)|R(90)|R(110)|R(120)|A(300)|A(400)|A(600)|A(700)|default|\n|-|-|-|-|-|-|-|-|-|-|-|-|\n|CLIPScore |32.37|32.22|32.47|32.55|32.50|32.31|31.76|32.08|32.48|32.41|32.93|\n\nAlthough the specific injection positions vary with model architectures—e.g., Flux (0, 40, 300) shifts earlier due to faster convergence, while Hunyuan (0, 100, 500) shifts later—our strategy remains effective across all tested models.\n\n+**overhead analysis**\n\nBefore training, we determine Attribute and Relation injection steps via a one-time offline analysis, enabling phase-wise semantic conditioning without extra supervision or dynamic control.\n\nSpecifically, for the Attribute injection timestep \\$s\\_{attr}\\$, we detect the earliest point where cross-attention stabilizes. For each sample and every layer \\$m=1,\\dots,M\\$ and head \\$h=1,\\dots,H\\$, we compute the normalized difference of the cross-attention matrix \\$Attn\\_t^{(m,h)} \\in R^{Q \\times K}\\$ at step \\$t\\$ (see Eq. 5), average across heads and layers, apply a moving average over the batch, and use thresholding to determine \\$s\\_{attr}\\$. The computational complexity is \\$T\\_{s\\_{\\mathrm{attr}}} = O(N \\times S \\times M \\times H \\times QK)\\$, where \\$N\\$ is the sample size, \\$S\\$ the diffusion steps, \\$M\\$ and \\$H\\$ the numbers of layers and heads, and \\$Q\\$, \\$K\\$ the attention dimensions. As this one-time offline process uses few samples (e.g., \\$N=100\\$), its cost is negligible relative to a training epoch.\n\nFor Relation, we leverage the static SNR (Signal-to-Noise Ratio) curve provided by the diffusion scheduler, defined as  $\\mathrm{SNR}(t) = \\alpha_t^2 / \\sigma_t^2$,  and compute its discrete curvature over the interval $[s_{\\mathrm{attr}}, S]$ to locate the turning point corresponding to the optimal Relation injection timestep $s_{\\mathrm{rel}}$ (see Eqs. 8 and 9). This method requires no model inference or input data, and only involves simple discrete operations over a one-dimensional sequence of length  $S' = S - s_{\\mathrm{attr}}$.  Its computational complexity is: $T_{s_{\\mathrm{rel}}} = O(S - s_{\\mathrm{attr}})$. Since it only performs basic operations over a short vector of scalar values, the runtime is significantly lower than that of the matrix-based analysis used for $s_{\\mathrm{attr}}$, and can be considered negligible.\n\n&nbsp;\n\n>**W4: More comprehensive evaluation**\n\nWe fully agree with you. CLIPScore and FID have inherent limitations, as they cannot adequately evaluate image naturalness, fidelity, and rationality. Therefore, they cannot reflect the model's understanding of complex captions. We additionally adopted GenEval for more comprehensive evaluation, such as semantic consistency, fidelity, clarity, diversity, and plausibility. While many other metrics exist, these three are the most widely adopted in the field. Unfortunately, when seeking to conduct broad comparative analysis, the number of applicable universal metrics becomes limited. Inspired by Reviewer `6Z8X`, VQAScore can serve as a general benchmark similar to GenEval. The following presents the evaluation of our method and other baselines on VQAScore.\n\n|Method|VQAScore(\\*)↑|VQAScore↑|\n|-|-|-|\n|SDv3 Medium|75.37|58.56|\n|SDv3.5 Large|76.49|60.77|\n|PixArt-α|68.86|50.83|\n|Flux.1 Dev|75.19|55.15|\n|DiT-ST Medium|76.12|62.73|\n|DiT-ST Large|76.25|64.30|\n\n\\* denotes the simplified VQAScore used in [r1]. The other VQAScore is standard. We utilize the semantic primitives from the split-text to construct our standard VQA questions: *\"Is [OBJECT] visible in the image?\" \"Does the [OBJECT] have [ATTR]?\" \"Does the [OBJECT] appear [REL] the [OBJECT]?\"*\n\nWe can see that in the simplified VQA, the gap between models is small. However, in the standard VQA, DiT-ST shows a clear lead, demonstrating better capability in handling complex semantics and captions. Additionally, we provide extensive visual showcases in **Figures 7-11 in the appendix** for your evaluation. Once again, thank you for your suggestion and inspiration!\n\n[r1] Progressive Prompt Detailing for Improved Alignment in Text-to-Image Generative Models, arxiv 2025\n\n&nbsp;\n\n>**W5: Output of semantically incomplete captions**\n\nFor prompts that lack explicit objects, we view this as a data-related issue rather than a limitation of our method. Our model is designed to parse complex prompts into structured semantic primitives and generate faithful images when sufficient information is provided. When a prompt omits core content (e.g., just like“dreamlike atmosphere” or “golden hour”), our method naturally falls back to global style or scene-level cues without object-level construction. Importantly, this challenge is not unique to our approach—existing models targetting at solving complex prompts would also behave similarly under such under-specified prompts. Addressing such cases typically requires prompt enrichment or additional context, rather than changes to the underlying generation framework.\n\n&nbsp;\n\n&nbsp;\n\nOnce again, we sincerely thank you for dedicating your valuable time to review our submission and for providing such insightful and professional suggestions. We are fully committed to incorporating the analyses and experiments you recommended into the final version, as a way to respect your time and effort. To be frank, you are a pivotal reviewer whose feedback may play a decisive role in the outcome of this submission, and we genuinely hope to earn your encouragement and support. Due to rebuttal policy restrictions, we regret that we are currently unable to provide the relevant visualizations for your review. However, if you have any further questions or feel that any of our explanations are unclear, please don’t hesitate to let us know—we would be more than happy to provide additional detailed clarifications. Thank you once again!\n\n&nbsp;\n\nBest wishes and regards,\n\nAll authors of Submission 510"}]

Author rebuttal:

Meta-review: ### Summary of Claims and Findings

The paper claims that standard text-to-image diffusion models struggle with complex and long-form text prompts. To address this, the authors propose a "split-text conditioning" framework that uses a LLM to parse the complex text prompt into a collection of simpler sentences. These primitives are then injected incrementally into different stages of the diffusion model's denoising process. The experimental results show that their method significantly outperforms the strong baselines on several metrics (CLIPScore, FID, GenEval) particularly on complex text prompts.

### Strengths of the Paper
All reviewers agree that this paper is written well, the proposed method is simple and effective, and shows strong empirical results with sufficient ablations.

### Weaknesses of the Paper
The primary weaknesses were pointed out by reviewer 6Z8X.

- Limited novelty due to a concurrent work: Reviewer 6Z8X pointed out a concurrent work, SCoPE (arxiv.org/pdf/2503.17794v1), which shares similar ideas. This led to a debate about the paper's novelty.
- Lack of User Study.

### Reasons for Recommendation
I am recommending this paper for acceptance. The authors provided a strong rebuttal that addressed most of the weaknesses raised by reviewers. While the concerns about novelty are valid, they are outweighed by the paper's technical merits, the strength of the rebuttal, and the official NeurIPS policy on contemporaneous work. 

SCoPE should be considered contemporaneous work under the NeurIPS 2025 policy (as it appeared online after the March 1st deadline) and that a submission should not be rejected solely on this basis. More importantly, they provided a detailed, five-point differentiation, highlighting that their method operates at a more granular "semantic primitive" level, while SCoPE operates at a "sub-prompt" level. Their approach for determining injection timesteps is also more deeply tied to the internal dynamics of the diffusion model. AC would strongly suggest the authors to add this discussion to the final paper.

### Summary of Discussion and Rebuttal
The discussion period was productive and crucial for the final decision. The key points were:
- Novelty vs. SCoPE: This was the main topic. Reviewer 6Z8X initially flagged this, and Reviewers 1VDF and 4VWJ later echoed the concern, with Reviewer 1VDF stating it "weakened the novelty." The authors responded by detailing the technical differences and referencing the conference's contemporaneous work policy. I, as the AC, reminded reviewers of this policy to ensure a fair evaluation. While some reviewers remained concerned, Reviewer kbep (who is more senior) agreed it should be treated as concurrent work. My final decision weighs the authors' technical differentiation and the official policy more.
- Evaluation scope: Reviewers 6Z8X and 4VWJ requested more comprehensive evaluation, particularly a user study. The authors acknowledged the lack of a user study as a common limitation in the field and compensated by adding a new VQAScore benchmark analysis to their rebuttal, which was a strong response.
- Generalizability: Reviewer kbep questioned the method's transferability. The authors addressed this directly by running new experiments showing positive results when applying their framework to two other Transformer-based diffusion models.
- Technical Clarity: Reviewer 1VDF requested clarification on a loss function (Equation 10) and visualizations for attention convergence. The authors provided a detailed mathematical breakdown and a clear textual description of the convergence dynamics, which satisfied the reviewer.

Overall, the authors addressed the reviewers' concerns comprehensively with new experiments and detailed explanations. This significantly improved the paper's quality, justifying the decision to accept the paper.
Decision: Accept (poster)