Title: Enhancing Text-to-Image Diffusion Transformer via Split-Text Conditioning
Venue: NeurIPS 2025
Accepted: yes
Keywords: Diffusion Transformer

Text-to-Image Generation

Multimodel Learning

Large Language Model
Abstract: Current text-to-image diffusion generation typically employs complete-text conditioning. Due to the intricate syntax, diffusion transformers (DiTs) inherently suffer from a comprehension defect of complete-text captions. One-fly complete-text input either overlooks critical semantic details or causes semantic confusion by simultaneously modeling diverse semantic primitive types. To mitigate this defect of DiTs, we propose a novel split-text conditioning framework named DiT-ST. This framework converts a complete-text caption into a split-text caption, a collection of simplified sentences, to explicitly express various semantic primitives and their interconnections. The split-text caption is then injected into different denoising stages of DiT-ST in a hierarchical and incremental manner. Specifically, DiT-ST leverages Large Language Models to parse captions, extracting diverse primitives and hierarchically sorting out and constructing these primitives into a split-text input. Moreover, we partition the diffusion denoising process according to its differential sensitivities to diverse semantic primitive types and determine the appropriate timesteps to incrementally inject tokens of diverse semantic primitive types into input tokens via cross-attention. In this way, DiT-ST enhances the representation learning of specific semantic primitive types across different stages. Extensive experiments validate the effectiveness of our proposed DiT-ST in mitigating the complete-text comprehension defect. Datasets and models are available.

Enhancing Text-to-Image Diffusion Transformer via Split-Text Conditioning

Abstract

Current text-to-image diffusion generation typically employs complete-text conditioning. Due to the intricate syntax, diffusion transformers (DiTs) inherently suffer from a comprehension defect of complete-text captions. One-fly complete-text input either overlooks critical semantic details or causes semantic confusion by simultaneously modeling diverse semantic primitive types. To mitigate this defect of DiTs, we propose a novel split-text conditioning framework named DiT-ST. This framework converts a complete-text caption into a split-text caption, a collection of simplified sentences, to explicitly express various semantic primitives and their interconnections. The split-text caption is then injected into different denoising stages of DiT-ST in a hierarchical and incremental manner. Specifically, DiT-ST leverages Large Language Models to parse captions, extracting diverse primitives and hierarchically sorting out and constructing these primitives into a split-text input. Moreover, we partition the diffusion denoising process according to its differential sensitivities to diverse semantic primitive types and determine the appropriate

timesteps to incrementally inject tokens of diverse semantic primitive types into input tokens via cross-attention. In this way, DiT-ST enhances the representation learning of specific semantic primitive types across different stages. Extensive experiments validate the effectiveness of our proposed DiT-ST in mitigating the complete-text comprehension defect. Datasets and models are available.

1 Introduction

In recent years, diffusion models [1, 2] have achieved unprecedented breakthroughs [3, 4, 5, 6]. Leveraging the Transformer [7] backbone, the Diffusion Transformer (DiT) [8] has rapidly become the mainstream paradigm for text-to-image generation and achieves impressive performance.

Current DiTs for text-to-image generation mostly employ complete-text captions as conditioning, which are sourced from human-curated datasets or generated by large language models (LLMs) or multimodal large language models (MLLMs). Such complete-text captions, particularly long ones, typically involve complex syntax with intertwined semantic primitives (object, relation, and attribute) and potential redundancy. When dealing with this form of captions, DiTs exhibit an inherent complete-text comprehension defect , reflecting in issues such as attribute misbinding [9, 10], style dominance [11, 12], semantic blending [13, 14], and semantic entanglement [15]. The comprehension defect stems from two key aspects: insufficient semantic analysis and premature information exposure .

On the one hand, insufficient semantic analysis arises from multiple factors. For instance, ( i ) text length bottleneck [16]: CLIP text encoder [17] accepts at most 77 tokens, yet effective tokens seldom exceed 20, making it difficult to process detailed text and leading to the truncation or loss of tail information in long captions; ( ii ) softmax competition [18, 19]: tokens compete against each other in the softmax function, causing the representation capability to decrease as token number increases; ( iii ) positional bias [20]: CLIP prioritizes the first item, making it difficult to attend to later items and leading to semantic distortion. These factors collectively hinder models from accurately analyzing semantic primitives and highlighting important semantic information from long complete-text captions. Consequently, models tend to overlook and omit crucial semantic information within a complete-text caption. However, a split-text caption derived by hierarchically sorting out the semantic primitives within the complete-text caption may reduce the syntactical complexity and comprehension difficulty for models, helping improve semantic analysis.

(a)

(b)

Object and relation at step 25

Object and relation at step 75

ribbon

wears

On the other hand, the one-time input of a complete-text caption leads to various semantic primitives being modeled simultaneously. Since these primitives cannot be disentangled effectively within the shared representation space, ultimately competing for representation within the same image regions. This is the main reason for phenomena such as semantic entanglement, as illustrated in Figure 2 (a). We summarize this problem as premature information exposure , where excessive fine-grained details are prematurely exposed to the model before the model establishes stable primary semantic concepts. Existing studies [21, 22, 23, 24] indicate that diffusion models establish primary semantic concepts during the early denoising stage, while improving fine-grained details in the later stage. Details mainly originate from attribute primitives, while primary semantic concepts derive from object and relation primitives. In addition, as illustrated in Figure 2 (b), different denoising stages have diverse sensitivities of semantic primitive types. We can conclude that the prioritization order for semantic primitive types is object-relation-attribute. Therefore, we propose incrementally injecting diverse types of semantic primitives by prioritization order to improve the proportion of sensitive

primitive types at each stage, enabling the model to enhance the representation learning of a specific sensitive primitive type at the corresponding stage. As discussed above, since a split-text caption is derived by hierarchically sorting out the semantic primitives and possesses a distributed structure, it can effectively group the same type of semantic primitives together and be injected incrementally, naturally helping mitigate premature information exposure.

In this paper, we propose a novel framework, DiT-ST, to enhance the text-to-image DiT via splittext caption. A split-text caption is essentially a hierarchical collection of simplified sentences that explicitly express various semantic primitives and their interconnections, effectively reducing the syntactical complexity to alleviate insufficient semantic analysis and grouping the same type semantic primitives to be injected incrementally to address premature information exposure. Briefly, we first employ LLM [25] for caption parsing to extract various semantic primitives and assemble a hierarchical caption parsing graph. We then construct the hierarchical collection of simplified sentences according to the graph to realize the conversion from complete-text caption to split-text caption. We refine the encoding of the MM-DiT backbone [26], resulting in a DiT-ST with richer semantic level. In addition, we determine appropriate injection timesteps for three semantic primitive types by analyzing cross-attention convergence phenomena [22] and identifying the inflection point in the signal-to-noise ratio curve during denoising. After that, incrementally injecting object-relationattribute tokens to input tokens via cross-attention to increase the proportions of corresponding sensitive semantic primitive types, further enhancing representation learning of specific semantic primitives at different stages.

Extensive experiments confirm the benefits of our split-text captioning. On GenEval, our DiT-ST-M achieves 0.69 overall accuracy and 11.3% gain over SDv3 Medium. On COCO-5K, it records the highest average CLIPScore (34.09) and the lowest FID (22.11), delivering performance competitive with SDv3.5 Large. All results and visualizations demonstrate that the method is both parameterefficient and architecture-agnostic, yielding finer detail and stronger semantic fidelity.

2 Related Work

2.1 Diffusion Transformers

Diffusion models [2, 4, 27] reverse a fixed noising process by imitating the denoising trajectory through learned optimization. When integrated with multimodal learning [28, 29, 30, 31, 32] and other techniques [33, 34, 35, 36, 37, 38], they offer a powerful generation framework for visual content [39, 40, 41, 42, 43] and other downstream tasks [44, 45, 46, 47, 48]. Recent advances, Diffusion Transformer [8] replaces U-Net [49] with Transformer [7], enabling its strong capability for long-range dependency modeling, scalability, and flexibility. DiT models, exemplified by DeepFloydIF [3], Flux [50], PixArtα [51] and DreamEngine [52], have successfully achieved remarkable generation performance, making DiT the mainstream paradigm for text-to-image generation.

2.2 Complete-Text Comprehension Defect

Complete-text comprehension defect refers to models' difficulty in effectively analyzing and understanding complete text, stemming from inherent encoder limitations or structural deficiencies, such as text length bottleneck [16], softmax competition [18, 19], positional bias [20], incorrect tokenization [53], and frequency bias [54]. This defect results in suboptimal generation quality and generated content that struggles to faithfully correspond to the prompt. Manifestations of complete-text comprehension defect include attribute misbinding [55], object missing [56], style dominance [11], semantic blending [13], semantic entanglement [15], and etc.

Many existing methods have mitigated specific manifestations. Attend-and-Excite [9] activates dominant token attention to guide the generation process in adherence to the textual prompt. WiCLP [57] and research [10] address attribute misbinding and object omission by refining the text embedding space and reweighting token-level attention to enhance compositional fidelity during generation. DeaDiff [12] mitigates style dominance by employing exclusive subsets of cross-attention layers for disentangling style and semantics. LongAlign [58] enhances semantic coherence by introducing explicit linguistic structures. In contrast, SCoPE [59] operates at the sub-prompt level, beginning with the simplest sub-prompt, then progressively introducing more complex variants and determining injection timesteps using proportional similarity ratios to mitigate alignment degradation.

While these methods indeed mitigate the complete-text comprehension defect from various perspectives, they primarily optimize for specific manifestations. Instead of addressing individual encoder limitations, we target the fundamental cause of the comprehension defect-the inherent inability of models to handle complex syntax within complete-text captions. Convert the form of captions to pursue a comprehensive mitigation strategy for the complete-text comprehension defect.

3 Methodology

The overall framework of DiT-ST is illustrated in Figure 3, which incorporates three key components: Caption Parsing, Hierarchical Caption Input, and Incremental Primitive Injection. The original caption C CT adopts a complete-text structure. To sort out and refine semantic primitives, DiT-ST employs LLM as for caption parsing, extracting various semantic primitives and assembling them into a caption parsing graph G . According to the relationships among various semantic primitives in graph G , we construct them into a hierarchical collection of simplified sentences-the split-text caption C ST -which is subsequently input to the text-to-image DiT (MM-DiT). During the DiT denoising process, we calculate and determine appropriate timesteps for injecting each of the three semantic primitive types. Following the object-relation-attribute order, we incrementally inject encoded semantic primitive tokens into input tokens via cross-attention thereby enhancing the representation learning of specific semantic primitives across different stages.

cat. rug. packaging.

cat sitting on rug.

cat looking up.

cat is cute.

cat is beige and white.

packaging is original.

eyes are curious.

A

cute

beige

and

white  cat  with  large,

curious eyes is sitting

on a rug next to a new

its  original  packaging,

inquisitively looking up.

root

new

eyes

curious

cute

beige and white

original

cat

rug

packaging

sitting on

next to

Parsing

Construct

MM-DiT

Caption Parsing

MM-DiT Encoder

Hierarchical

Denoising Process

object

relation

attribute

Incremental Text Injection

3.1 Caption Parsing

Caption parsing aims to split the complete-text structure of captions, resulting in a caption parsing graph G . The reason why we choose the graph as the target product is that a hierarchical tree-like structure [60] can clearly represent various semantic primitives and their interconnections, naturally resolving existing problems of DiTs in this aspect. Therefore, we leverage a LLM (specifically QwenPlus [61] in this work) as a semantic analyzer and graph generator to extract various semantic primitives and assemble these semantic primitives to obtain the caption parsing graph G .

We define a caption parsing graph as G = ( V, E ) , where V denotes the set of N nodes and E denotes the set of edges. Specifically, V = { v i | i = 1 , 2 , . . . , N } , Each node v i is defined as v i = ( o i , A i ) , with o i denoting the i -th object primitive and A i = { a j } representing the set of attributes associated with object primitive o i . O is the object set, and A is the attribute set. Relation set R consists of M semantic relationships: R = { r k | k = 1 , 2 , . . . , M } . Each edge e is modeled as a labelled triple e = ( v i , v j , r k ) ∈ V × V ×R , the set of edges E ⊆ V × V ×R .

During the graph assembly, we use the root node to represent the entire caption, and recursively decompose it into various object primitive nodes. Relations between object primitives are represented by edges between nodes. Attributes are represented as child nodes attached to their corresponding object primitive nodes. Therefore, for a given complete-text caption C CT , the process to obtain the caption parsing graph G can be formalized as follows:

In this way, we convert a complete-text caption into a split and hierarchical caption parsing graph.

3.2 Hierarchical Caption Input

The purpose of hierarchical caption input is to construct a split-text caption based on its caption parsing graph. In addition, in order to make the model better adapt to the split-text caption and enrich the overall semantic level of the input, we also modify the existing text encoding method in text-to-image DiT to better encode the split-text caption. Thus, the hierarchical caption input process consists of two components: split-text caption construction and DiT text encoding refinement.

3.2.1 Split-Text Caption Construction

The hierarchical structure of the caption parsing graph facilitates us to construct a split-text caption, which is a hierarchical collection of simplified sentences. Considering that inherent positional bias in text encoders (e.g., CLIP) is unavoidable during subsequent split-text encoding, we also take into account the order of semantic primitives, i.e., position the important semantic primitives at the beginning to achieve better performance. Specifically, we first rerank object primitives in the object set O in descending order based on the node degree and the frequency of occurrence in the caption, obtaining a reranked object set O ′ . We then rerank the primitives in the relation set R and attribute set A to ensure that relations or attributes involving object primitives maintain consistency with the object primitive order in O ′ , resulting in R ′ and A ′ .

Subsequently, based on the reranked sets O ′ , R ′ , A ′ and following the object-relation-attribute hierarchical order, we generate corresponding simplified sentences for each primitive within these sets. The forms of simplified sentences corresponding to objects, relations, and attributes present respectively [OBJECT] object_ i , [RELATION] object_ i relation_ k object_ j , and [ATTRIBUTE] object_ i is attribute_ i . The collection comprising all these simplified sentences constitutes the split-text caption C ST . The process to construct the split-text caption can be formalized as follows:

3.2.2 DiT Text Encoding Refinement

We employ the current mainstream multimodal diffusion transformer, MM-DiT [26], for text-toimage generation. MM-DiT adopts three text encoders: CLIP-L/14, CLIP-G/14, and T5 XXL. For a split-text caption C ST , after three encoders' text encoding, three token sequences can be obtained: T ST L/ 14 ∈ R L × D L/ 14 , T ST G/ 14 ∈ R L × D G/ 14 and T ST T 5 ∈ R L × D , where L is the token sequence length ( L ≤ 77 ) , D L/ 14 , D G/ 14 and D are the dimensions of the three token sequences.

According to the original design, the concatenation of sequences T ST L/ 14 and T ST G/ 14 yields a new token sequence whose still dimension remains smaller than D . Given that the new token sequence must be appended with T ST T 5 for input into the MM-DiT blocks, the dimension capacity remains underutilized. Therefore, we consider fully utilizing this underutilized dimension capacity by incorporating the complete-text caption C CT to enrich the overall semantic level of the input. As shown in Figure 4, We supplement

it by adding C CT encoded through T5 XXL and followed by a linear projection, generating the token sequence T CT T 5 ∈ R L × D ′ , where D ′ = D -D L/ 14 -D G/ 14 . We perform dimensional concatenation and sequence append as follows:

Through the above operations, we ultimately construct the input token sequence T which possesses a hierarchical structure of semantic primitives and rich semantic representation.

3.3 Incremental Primitive Injection

As previously discussed, due to different denoising stages have differential sensitivities of semantic primitive types, prematurely exposing fine-grained detail information before the model has established

stable primary semantic concepts is detrimental to generation. We propose incrementally injecting diverse semantic primitive types to enhance differential representation learning for diverse semantic primitive types at different stages. Therefore, incremental primitive injection involves both injection timestep selection and semantic primitive injection.

3.3.1 Injection Timestep Selection

Since diffusion models' prioritization order for diverse semantic primitive types during the denoising process generally follows object-relation-attribute, we should, ideally, identify accurate timesteps for injecting semantic primitives. However, it is practically impossible for several reasons. First, semantic emergence occurs gradually rather than in discrete stages, lacking clear natural boundaries. Second, there are differences across samples, and the semantic emergence process can shift or stretch on the timeline with changes in prompts, resolution, and random seeds. Furthermore, adaptively selecting specific timestep for each sample will incur substantial computational costs. Therefore, we consider statistical rules across samples to identify approximately appropriate injection timesteps.

Determine the injection timestep for attribute primitives. Research [22] indicates that crossattention outputs converge to a fixed point after several inference steps. This point divides the denoising process into two stages: semantic-planning and fidelity-improving. Since fidelity-improving stage primarily relies on attribute information, we select the timestep corresponding to this convergence point as the injection timestep for attribute primitives. Considering statistical rules across a batch of samples, for a sample, we first define the attention weights output by the h -th attention head in the m -th cross-attention layer at inference step t as Attn ( m,h ) t ∈ R Q m × K m , where Q m and K m represent the number of query tokens and key tokens, respectively. Subsequently, we calculate the difference of single-head attention between adjacent inference steps using the Frobenius norm:

where θ serves as a numerical stabilizer, typically set to 10 -8 . Then, we average across all H attention heads and all M cross-attention layers to compute the cross-attention difference for the sample:

By calculating the average cross-attention across all samples in the batch, we obtain the average cross-attention difference ¯ ∆ t at t inference step. We employ the moving average method to calculate the average cross-attention ̂ ∆ t across multiple inference steps and set a threshold τ to identify the inference step t ∗ at which cross-attention converges:

where w denotes the size of the moving window. Thus, the timestep corresponding to inference step t ∗ is selected as the injection timestep s attr for attribute primitives.

Determine the injection timestep for relation primitives.

As previously discussed, given the entire denoising process consists of S timesteps, the first S -s attr timesteps constitute the semantic-planning stage, during which the samples maintain a relatively high signal-to-noise ratio (SNR). Research [21] indicates that semantic concepts are primarily established at a high SNR condition. By analyzing the variation of SNR across timesteps, we observe an initial rapid decline followed by a gradual decrease, as shown in Figure 5. Considering the prioritization order for semantic primitives as object-relationattribute, intuitively speaking, the inflection point of the SNR

within the first S -s attr timesteps can serve as an appropriate injection timestep for relation primitives.

We calculate the average SNR snr t at each timestep t across all samples in the batch for the first S -s attr timesteps, and construct a discrete curve function y t = g ( snr t ) , where t = 0 , 1 , . . . , S -s attr -1 . We employ the discrete maximum-curvature method to identify the inflection point. First, we calculate the first-order and second-order finite differences as follows:

Then, we calculate the discrete curvature:

In this way, we can obtain the index of inflection point t # = arg max κ t , which also is the injection timestep s rel for relation primitives.

Determine the injection timestep for object primitives. Considering the limited number of timesteps before s rel , further partition of these timesteps lacks theoretical and computational evidence. For simplicity, we set the timestep at t = 0 as the injection timestep s obj for object primitives.

By identifying the convergence point of cross-attention and the inflection point in SNR curve during denoising, we are able to adaptively determine the appropriate injection timesteps according to statistical rules and characteristics of the sample set.

3.3.2 Semantic Primitive Injection

Semantic primitive enhancement aims to increase the proportion of corresponding semantic primitives to align the semantic primitive type sensitivity of each denoising stage. We employ the crossattention mechanism and incrementally inject each primitive type C prim of the split-text caption C ST . C prim is also encoded by three text encoders (CLIP-L/14, CLIP-G/14, and T5 XXL), obtaining T prim L/ 14 ∈ R L × D L/ 14 , T prim G/ 14 ∈ R L × D G/ 14 , and T prim T 5 ∈ R L × D ′ , where D L/ 14 + D G/ 14 + D ′ = D . We then concate these token sequences along the dimension to obtain the primitive injection token sequence T prim , which is used for cross-attention calculation with T to enhance representation learning of the corresponding semantic primitives at the stage.

3.4 Training Strategy

The training objective comprises two components while employing a mixing coefficient λ :

The first is conditional flow matching loss L CFM, which supervises denoising prediction, following the setting in MM-DiT. The other is L attn, which constrains the staged cross-attention by aligning injected semantic primitives with split text while regularizing attention behavior for stable semantic injection. It is composed of three components, weighted with empirically determined ratios of α = 0 . 6 , β = 0 . 25 , and η = 0 . 15 :

The first term, L inject, enforces the alignment between the cross-attention outputs and their target semantics at each activated injection stage:

The second term, L conv, supervises the evolution of the attention signal-to-noise ratio (SNR) to ensure proper convergence timing:

The third term, L mutex, penalizes spatial overlap between attention maps from different injection stages to maintain independence and prevent semantic interference:

̸

Together, these components ensure stable multi-stage semantic injection by enforcing semantic precision, convergence regularity, and spatial exclusivity, which is crucial for achieving fine-grained semantic alignment in Split-Text Conditioning.

4 Experiment

4.1 Experiment Settings

Impementation Details and Datasets To ensure fair comparison beyond the short-text bias of MM-DiT's original training data (CC12M [62] and ImageNet [63]), we adopt 100K samples from the SAM-LLaVA dataset [51], which contains semantically rich, long captions suitable for our split-text method. Extended training on this dataset yields MM-DiT 2B-E and MMDiT 8B-E. Following MM-DiT [26], we use 24 transformer layers with CLIP-L, CLIP-G, and T5-XXL as text encoders. Complete captions are parsed into hierarchical inputs via Qwen-Plus [61], with cross-attention incrementally injected at selected SNR-based denoising steps. Using the same training protocol, we obtain two variants of our models: DiT-ST M (2B) and DiT-ST L (8B). For evaluation, we follow CogView3 [64] and ∆ -DiT [65] to construct COCO-5K, assessing performance under varying text lengths and complexities, with results reported from our DiT-ST M and other competing method

Evaluation Metrics We evaluate DiT-ST using both quantitative and qualitative metrics. For quantitative analysis, we adopt GenEval [66] to assess semantic comprehension, along with FID [67] and CLIPScore [68] to measure image quality and text-image alignment.

4.2 Main Results

In this section, we compare our method with MMDiT [26] and its variants as baselines, given architectural similarity and shared training setup. To further evaluate the generalizability of our splittext strategy in mitigating complete-text comprehension detect, we benchmark against competitive methods, including PixArtα [51], Flux.1 Dev [50], and the state-of-the-art DreamEngine [52].

Effectiveness of Split-text Caption Form Table 1 compares caption utilization forms in terms of CLIPScore and FID, showing that split-text caption leads to consistently better performance. By parsing prompts into organized semantic primitives, our method effectively mitigates the

complete-text defect, raises CLIPScore by 7.6 % and lowers FID by 10.2 % relative to the other two caption forms used on MM-DiT 2B-E, while still providing +6.1 % / -9.6 % gains over the LLM-Enhanced caption, underscoreing that the observed gains are primarily attributable to the caption structure, rather than to external LLM intervention.

Multi-Dimensional Semantic Comprehension Performance Table 2 reports comparisons on the GenEval benchmark, which evaluates six dimensions of semantic understanding: single-object recognition, multi-object reasoning, counting, color accuracy, spatial positioning, and attribute binding. Our method achieves a competitive overall accuracy of 69%, matching the state-of-the-art DreamEngine and closely approaching SDv3.5 Large (71%), despite a 4 × smaller parameters. Notably, our model outperforms the SDv3 Medium baseline across all subcategories, with marked advantages in multiobject reasoning(+0.14) and attribute binding(+0.10), indicating stronger semantic comprehension. Compared to Flux.1 Dev and DALL·E 3, which excel in specific areas, our approach offers more balanced and robust performance across all axes due to text-split strategy.

Beyond GenEval, we employ CLIPScore and VQAScore [69] on the curated COCO-5K dataset. These two metrics provide a comprehensive evaluation: CLIPScore measures global semantic similarity, while VQAScore leverages a VQA model to directly probe for complex compositional alignment via a probabilistic "Yes/No" query. As results listed in Table 3, DiT-ST Medium attains a CLIPScore of 34.09, eclipsing competing methods by average 8.9 %, and even markedly surpassing SDv3.5 Large (32.74) by about 4.1%. On VQAScore, our DiT-ST Medium achieves 76.12, on par with the larger SDv3.5 Large, while surpassing SDv3 Medium (75.37) and other baselines. These results consistently validate that our split-text conditioning strategy enhances both compositional semantic fidelity and cross-modal alignment, enabling the model to generate images that more accurately capture the intended meaning of complex prompts while maintaining high perceptual quality.

Robust Performance to Caption Length Table 4 exposes marked length sensitivity in existing models: PixArtα benefits from increasing caption length before plateauing, whereas SD-series models peak on mid-length prompts and deteriorate on longer ones (e.g., SD v3 Medium falls to 27.7 in the [45 , 55) subband). By contrast, our 2B-parameter model sustains a CLIPScore over 32 across all bins, rises to 35.81 for the longest captions, and secures the highest overall score (34.09), outperforming SDv3 Medium and SD v3.5 Large by 8.9% and 4.1%, respectively. These results highlight the robustness and effectiveness of our split-text conditioning strategy in handling variable-length and semantically complex prompts, enabling more stable text-image alignment across diverse input distributions while remaining parameters effcient.

4.3 Ablation Analysis

Incremental Text Injection Incremental injection improves the text-split pipeline by scheduling semantic primitives at the appropriate timesteps. As Table 5 reports, this schedule rises CLIPScore by 3.9 % and simultaneously reduces FID by 7.3 %, decisively outperforming the single-shot alternative. The progressive delivery deepens semantic grounding, alleviates

attention saturation and inter-token interference, enhancing both text alignment and visual fidelity.

Hierarchical Caption Input Hierarchical caption input aligns naturally with the text-split strategy, as it organizes diverse semantic primitive types into a structured format for generation. To assess its effectiveness, we evaluate models under three input settings: original caption, LLM-Enhanced caption, and hierarchical input. As Table 6 shows, merely swapping the caption format to the hierarchical variant lifts CLIPScore by 3.6 % and reduces FID by 3 %

for each architecture, with additional 2 % gains over the LLM-Enhanced captions. Crucially, SDv3 Medium enjoys the same absolute improvement as our own model, despite zero fine-tuning, demonstrating that the benefit is inherent to the representation and broadly transferable across models.

More Ablation Analysis To further validate the effectiveness of our model and design choices, we conduct additional experiments on the orders of semantic primitive injection, timestep selection, hyperparameter sensitivity, and other ablation studies. Please refer to the appendix for more details.

4.4 Visualization

High-Quality Generation Visualization Figure 6 showcases representative image samples generated by our DiT-ST, with rich visual details and strong semantic alignment across diverse prompts.

(a)

(@)

(C)

(d)

(a)Arusticbreakfastwithpoachedeggs andavocadoonsourdoughtoast,servedonaceramicplatewithcoffee,bathed insoftmorningsunlight. (b)Avibrantmonarchbutterflywithorangeandblackwingsperchedonabloominglavenderflower,surroundedbyotherpurpleblossoms.

More Visualiztion We provide additional visual content, including results of different caption forms (e.g. Figure 1 (a)), performance against SDv3.5 (e.g. Figure 1 (b)) and other competing models as well as our high-quality images at multiple resolutions, which are available in the Appendix.

5 Conclusion

This paper introduces DiT-ST, a split-text conditioning framework that alleviates the complete-text comprehension defect in DiTs. It comprises three components. ( i ) Caption parsing, use LLMs to extract and organize primitives; ( ii ) Hierarchical input construction, construct split-text inputs and enrich input semantics; ( iii ) Incremental primitive injection, inject different primitives into appropriate denoising stages to improve stage-specific representation learning. Extensive experiments demonstrate the effectiveness of our split-text caption design and the excellent performance of our proposed DiT-ST. We hope this work will offer some inspiration to the diffusion community.

References

Jascha Sohl-Dickstein, Eric Weiss, Niru Maheswaranathan, and Surya Ganguli. Deep unsupervised learning using nonequilibrium thermodynamics. In International conference on machine learning , pages 2256-2265. pmlr, 2015.

Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems , 33:6840-6851, 2020.

Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily L Denton, Kamyar Ghasemipour, Raphael Gontijo Lopes, Burcu Karagol Ayan, Tim Salimans, et al. Photorealistic text-to-image diffusion models with deep language understanding. Advances in neural information processing systems , 35:36479-36494, 2022.

Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. Highresolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 10684-10695, 2022.

Ling Yang, Zhilong Zhang, Yang Song, Shenda Hong, Runsheng Xu, Yue Zhao, Wentao Zhang, Bin Cui, and Ming-Hsuan Yang. Diffusion models: A comprehensive survey of methods and applications. ACM Computing Surveys , 56(4):1-39, 2023.

Hui Shen, Jingxuan Zhang, Boning Xiong, Rui Hu, Shoufa Chen, Zhongwei Wan, Xin Wang, Yu Zhang, Zixuan Gong, Guangyin Bao, et al. Efficient diffusion models: A survey. arXiv preprint arXiv:2502.06805 , 2025.

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems , 30, 2017.

William Peebles and Saining Xie. Scalable diffusion models with transformers. In Proceedings of the IEEE/CVF international conference on computer vision , pages 4195-4205, 2023.

Hila Chefer, Yuval Alaluf, Yael Vinker, Lior Wolf, and Daniel Cohen-Or. Attend-and-excite: Attention-based semantic guidance for text-to-image diffusion models. ACM transactions on Graphics (TOG) , 42(4):1-10, 2023.

Weixi Feng, Xuehai He, Tsu-Jui Fu, Varun Jampani, Arjun Akula, Pradyumna Narayana, Sugato Basu, Xin Eric Wang, and William Yang Wang. Training-free structured diffusion guidance for compositional text-to-image synthesis. arXiv preprint arXiv:2212.05032 , 2022.

Wen Li, Muyuan Fang, Cheng Zou, Biao Gong, Ruobing Zheng, Meng Wang, Jingdong Chen, and Ming Yang. Styletokenizer: Defining image style by a single instance for controlling diffusion models. In European Conference on Computer Vision , pages 110-126. Springer, 2024.

Tianhao Qi, Shancheng Fang, Yanze Wu, Hongtao Xie, Jiawei Liu, Lang Chen, Qian He, and Yongdong Zhang. Deadiff: An efficient stylization diffusion model with disentangled representations. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 8693-8702, 2024.

Omri Avrahami, Ohad Fried, and Dani Lischinski. Blended latent diffusion. ACM transactions on graphics (TOG) , 42(4):1-11, 2023.

Lorenzo Olearo, Giorgio Longari, Simone Melzi, Alessandro Raganato, and Rafael Peñaloza. How to blend concepts in diffusion models. arXiv preprint arXiv:2407.14280 , 2024.

Raphael Tang, Linqing Liu, Akshat Pandey, Zhiying Jiang, Gefei Yang, Karun Kumar, Pontus Stenetorp, Jimmy Lin, and Ferhan Ture. What the daam: Interpreting stable diffusion using cross attention. arXiv preprint arXiv:2210.04885 , 2022.

Beichen Zhang, Pan Zhang, Xiaoyi Dong, Yuhang Zang, and Jiaqi Wang. Long-clip: Unlocking the long-text capability of clip. In European Conference on Computer Vision , pages 310-325. Springer, 2024.

Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning , pages 8748-8763. PmLR, 2021.

Zhilin Yang, Zihang Dai, Ruslan Salakhutdinov, and William W Cohen. Breaking the softmax bottleneck: A high-rank rnn language model. arXiv preprint arXiv:1711.03953 , 2017.

Amita Kamath, Jack Hessel, and Kai-Wei Chang. Text encoders bottleneck compositionality in contrastive vision-language models. arXiv preprint arXiv:2305.14897 , 2023.

Reza Abbasi, Ali Nazari, Aminreza Sefid, Mohammadali Banayeeanzade, Mohammad Hossein Rohban, and Mahdieh Soleymani Baghshah. Clip under the microscope: A fine-grained analysis of multi-object representation. arXiv preprint arXiv:2502.19842 , 2025.

Jooyoung Choi, Jungbeom Lee, Chaehun Shin, Sungwon Kim, Hyunwoo Kim, and Sungroh Yoon. Perception prioritized training of diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 11472-11481, 2022.

Wentian Zhang, Haozhe Liu, Jinheng Xie, Francesco Faccio, Mike Zheng Shou, and Jürgen Schmidhuber. Cross-attention makes inference cumbersome in text-to-image diffusion models. arXiv e-prints , pages arXiv-2404, 2024.

Grace Luo, Lisa Dunlap, Dong Huk Park, Aleksander Holynski, and Trevor Darrell. Diffusion hyperfeatures: Searching through time and space for semantic correspondence. Advances in Neural Information Processing Systems , 36:47500-47510, 2023.

Amir Hertz, Ron Mokady, Jay Tenenbaum, Kfir Aberman, Yael Pritch, and Daniel Cohen-Or. Prompt-to-prompt image editing with cross attention control. arXiv preprint arXiv:2208.01626 , 2022.

Baichuan Inc. Qwen: A scalable and multilingual language model family, 2024. https: //huggingface.co/Qwen/Qwen-14B-Plus .

Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas Müller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, et al. Scaling rectified flow transformers for high-resolution image synthesis. In Forty-first international conference on machine learning , 2024.

Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. CoRR , abs/2010.02502, 2020.

Yu Zhang, Qi Zhang, Zixuan Gong, Yiwei Shi, Yepeng Liu, Duoqian Miao, Yang Liu, Ke Liu, Kun Yi, Wei Fan, et al. Mlip: Efficient multi-perspective language-image pretraining with exhaustive data utilization. arXiv preprint arXiv:2406.01460 , 2024.

Bokun Wang, Yang Yang, Xing Xu, Alan Hanjalic, and Heng Tao Shen. Adversarial crossmodal retrieval. In Proceedings of the 25th ACM international conference on Multimedia , pages 154-162, 2017.

Yepeng Liu, Yiren Song, Hai Ci, Yu Zhang, Haofan Wang, Mike Zheng Shou, and Yuheng Bu. Image watermarks are removable using controllable regeneration from clean noise. arXiv preprint arXiv:2410.05470 , 2024.

Rongtao Xu, Jiguang Zhang, Jiaxi Sun, Changwei Wang, Yifan Wu, Shibiao Xu, Weiliang Meng, and Xiaopeng Zhang. Mrftrans: Multimodal representation fusion transformer for monocular 3d semantic scene completion. Information Fusion , 111:102493, 2024.

Zixuan Gong, Qi Zhang, Guangyin Bao, Lei Zhu, Yu Zhang, Ke Liu, Liang Hu, and Duoqian Miao. Lite-mind: Towards efficient and robust brain representation learning. In ACMMultimedia 2024 , 2024.

Zhongwei Wan, Xinjian Wu, Yu Zhang, Yi Xin, Chaofan Tao, Zhihong Zhu, Xin Wang, Siqi Luo, Jing Xiong, and Mi Zhang. D2o: Dynamic discriminative operations for efficient generative inference of large language models. arXiv preprint arXiv:2406.13035 , 2024.

Kexue Fu, Mingzhi Yuan, Changwei Wang, Weiguang Pang, Jing Chi, Manning Wang, and Longxiang Gao. Dual focus-attention transformer for robust point cloud registration. In Proceedings of the Computer Vision and Pattern Recognition Conference , pages 11769-11778, 2025.

Zhongwei Wan, Zhihao Dou, Che Liu, Yu Zhang, Dongfei Cui, Qinjian Zhao, Hui Shen, Jing Xiong, Yi Xin, Yifan Jiang, et al. Srpo: Enhancing multimodal llm reasoning via reflectionaware reinforcement learning. arXiv preprint arXiv:2506.01713 , 2025.

Jinzhou Lin, Han Gao, Xuxiang Feng, Rongtao Xu, Changwei Wang, Man Zhang, Li Guo, and Shibiao Xu. Advances in embodied navigation using large language models: A survey. arXiv preprint arXiv:2311.00530 , 2023.

Yu Zhang, Yepeng Liu, Duoqian Miao, Qi Zhang, Yiwei Shi, and Liang Hu. Mg-vit: a multi-granularity method for compact and efficient vision transformers. Advances in Neural Information Processing Systems , 36:69328-69347, 2023.

Yepeng Liu and Yuheng Bu. Adaptive text watermark for large language models. arXiv preprint arXiv:2401.13927 , 2024.

Prafulla Dhariwal and Alex Nichol. Diffusion models beat gans on image synthesis. CoRR , abs/2105.05233, 2021.

Jonathan Ho, Tim Salimans, Alexey Gritsenko, William Chan, Mohammad Norouzi, and David J. Fleet. Video diffusion models, 2022.

Weimin Qiu, Jieke Wang, and Meng Tang. Self-cross diffusion guidance for text-to-image synthesis of similar subjects. In 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages 23528-23538, 2025.

Jinjin Zhang, Qiuyu Huang, Junjie Liu, Xiefan Guo, and Di Huang. Diffusion-4k: Ultra-highresolution image synthesis with latent diffusion models. In 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages 23464-23473, 2025.

Junyang Chen, Jinshan Pan, and Jiangxin Dong. Faithdiff: Unleashing diffusion priors for faithful image super-resolution. In 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages 28188-28197, 2025.

Zixuan Gong, Guangyin Bao, Qi Zhang, Zhongwei Wan, Duoqian Miao, Shoujin Wang, Lei Zhu, Changwei Wang, Rongtao Xu, Liang Hu, Ke Liu, and Yu Zhang. Neuroclips: Towards high-fidelity and smooth fmri-to-video reconstruction. In A. Globerson, L. Mackey, D. Belgrave, A. Fan, U. Paquet, J. Tomczak, and C. Zhang, editors, Advances in Neural Information Processing Systems , volume 37, pages 51655-51683. Curran Associates, Inc., 2024.

Bencheng Liao, Shaoyu Chen, Haoran Yin, Bo Jiang, Cheng Wang, Sixu Yan, Xinbang Zhang, Xiangyu Li, Ying Zhang, Qian Zhang, and Xinggang Wang. Diffusiondrive: Truncated diffusion model for end-to-end autonomous driving. arXiv preprint arXiv:2411.15139 , 2024.

Nanxin Chen, Yu Zhang, Heiga Zen, Ron J Weiss, Mohammad Norouzi, and William Chan. Wavegrad: Estimating gradients for waveform generation. arXiv preprint arXiv:2009.00713 , 2020.

Kunpeng Qiu, Zhiqiang Gao, Zhiying Zhou, Mingjie Sun, and Yongxin Guo. Noise-consistent siamese-diffusion for medical image synthesis and segmentation, 2025.

Xiaomin Li, Mykhailo Sakevych, Gentry Atkinson, and Vangelis Metsis. Biodiffusion: A versatile diffusion model for biomedical signal synthesis, 2024.

Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation, 2015.

Black Forest Labs. Flux. https://github.com/black-forest-labs/flux , 2024.

Junsong Chen, Jincheng Yu, Chongjian Ge, Lewei Yao, Enze Xie, Yue Wu, Zhongdao Wang, James Kwok, Ping Luo, Huchuan Lu, and Zhenguo Li. Pixartα : Fast training of diffusion transformer for photorealistic text-to-image synthesis, 2023.

Liang Chen, Shuai Bai, Wenhao Chai, Weichu Xie, Haozhe Zhao, Leon Vinci, Junyang Lin, and Baobao Chang. Multimodal representation alignment for image generation: Text-image interleaved control is easier than you think, 2025.

Dixuan Wang, Yanda Li, Junyuan Jiang, Zepeng Ding, Guochao Jiang, Jiaqing Liang, and Deqing Yang. Tokenization matters! degrading large language models through challenging their tokenization. arXiv preprint arXiv:2405.17067 , 2024.

Richard Diehl Martinez, Zébulon Goriely, Andrew Caines, Paula Buttery, and Lisa Beinborn. Mitigating frequency bias and anisotropy in language model pre-training with syntactic smoothing. arXiv preprint arXiv:2410.11462 , 2024.

Tsung-Han Wu, Long Lian, Joseph E Gonzalez, Boyi Li, and Trevor Darrell. Self-correcting llm-controlled diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 6327-6336, 2024.

Tuna Han Salih Meral, Enis Simsar, Federico Tombari, and Pinar Yanardag. Conform: Contrast is all you need for high-fidelity text-to-image diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 9005-9014, 2024.

Arman Zarei, Keivan Rezaei, Samyadeep Basu, Mehrdad Saberi, Mazda Moayeri, Priyatham Kattakinda, and Soheil Feizi. Improving compositional attribute binding in text-to-image generative models via enhanced text embeddings, 2025.

Luping Liu, Chao Du, Tianyu Pang, Zehan Wang, Chongxuan Li, and Dong Xu. Improving long-text alignment for text-to-image diffusion models, 2025.

Ketan Suhaas Saichandran, Xavier Thomas, Prakhar Kaushik, and Deepti Ghadiyaram. Progressive prompt detailing for improved alignment in text-to-image generative models, 2025.

Ziwei Yao, Ruiping Wang, and Xilin Chen. Hifi-score: Fine-grained image description evaluation with hierarchical parsing graphs. In European Conference on Computer Vision , pages 441-458. Springer, 2024.

An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, et al. Qwen2. 5 technical report. arXiv preprint arXiv:2412.15115 , 2024.

Soravit Changpinyo, Piyush Sharma, Nan Ding, and Radu Soricut. Conceptual 12m: Pushing web-scale image-text pre-training to recognize long-tail visual concepts, 2021.

Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A largescale hierarchical image database. In 2009 IEEE Conference on Computer Vision and Pattern Recognition , pages 248-255, 2009.

Wendi Zheng, Jiayan Teng, Zhuoyi Yang, Weihan Wang, Jidong Chen, Xiaotao Gu, Yuxiao Dong, Ming Ding, and Jie Tang. Cogview3: Finer and faster text-to-image generation via relay diffusion, 2024.

Pengtao Chen, Mingzhu Shen, Peng Ye, Jianjian Cao, Chongjun Tu, Christos-Savvas Bouganis, Yiren Zhao, and Tao Chen. δ -dit: A training-free acceleration method tailored for diffusion transformers, 2024.

Dhruba Ghosh, Hanna Hajishirzi, and Ludwig Schmidt. Geneval: An object-focused framework for evaluating text-to-image alignment, 2023.

Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilibrium, 2018.

Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural language supervision, 2021.

Zhiqiu Lin, Deepak Pathak, Baiqi Li, Jiayao Li, Xide Xia, Graham Neubig, Pengchuan Zhang, and Deva Ramanan. Evaluating text-to-visual generation with image-to-text generation. In European Conference on Computer Vision , pages 366-384. Springer, 2024.

A Dataset Details

We further introduce the datasets used in our work from two aspects: training and evaluation.

For training, we observe that existing datasets used by baseline models MM-DiT [26] are suboptimal for complex caption-conditioned generation. Specifically, datasets like CC12M [62] have relatively short captions (average length 10.3), and ImageNet [63] provides only class-level supervision. Inspired by Pixartα [51], which employs LLaVA to refine raw captions into high-information-density text, we adopt the SAM-LLaVA dataset provided by Pixartα to enhance textual supervision. We select 100K pairs from SAM-LLaVA (available in our released CSV files) to further train MM-DiT 2B and 8B, aiming to improve their capacity on long-text generation and ensure fair comparisons.

For evaluation, we construct COCO-5K, following settings from CogView3 [64] and ∆ -DiT [65]. It comprises 5,000 image-text pairs sampled across multiple caption-length intervals to assess performance under varying textual complexity. Additionally, we report gFID scores based on COCO30K to assess distribution-level image fidelity. Complete results are provided in Appendix D.

B Implementation Details

In terms of implementation, we focus on the settings of key thresholds and corresponding code-level realizations. We set the moving average window size w to 3, allowing each ̂ ∆ t to consider the current and two preceding steps for better capturing the SNR trend. To ensure numerical stability when computing normalized differences, we set θ = 10 -8 . For identifying the convergence point, we use a convergence threshold τ = 10 -4 . During inference, we select 40 denoising steps from the full diffusion range (0-1000) via uniform random sampling. When determining the injection layers, we allow flexible injection intervals. For example, around timestep 50, we inject attribute primitives within a ± 10 timestep range, while around timestep 400, we permit a broader window of ± 40 timesteps, reflecting the varying stability of attention dynamics at different diffusion stages.

C Further Evaluation and Model Comparison

To reduce metric variance and ensure fair model comparison, we construct a fixed 30K image-text subset from COCO for consistent evaluation across architectures and training scales.We evaluate and compare our method against several representative baselines, including SDv3 Medium, SDv3.5 Large, Flux .1 Dev, Pixartα . The evaluation results, in terms of FID, are summarized in Table 7.On the fixed COCO-30K subset, our DiT-ST Large model obtain the lowest FID scores . At the 2B scale, DiT-ST Medium reaches 18.78, narrowly beating SD v3 Medium (18.82) by 0.2 %. With 8B parameters, DiT-ST Large achieves 17.16, improving on SD v3.5 Large (17.31) by 0.9 %, a modest yet consistent gain that shows split-text conditioning still reduces late-stage artefacts even on short COCO captions. Flux .1 Dev (32.10) and PixArtα (27.35) post FIDs about 40% higher than DiT-ST Large. Flux, tuned for fast continuous-token sampling and lacking COCO fine-tuning, yields texture artefacts; PixArtα , trained on long, stylised captions, mismatches short, photorealistic domain. Thus, DiT-ST equals or surpasses SD baselines and clearly outperforms models trained on divergent data.

D Ablation on Injection Order of Semantic Primitives.

In this experiment, we investigate how varying the injection order of semantic primitives, namely object, relation, and attribute, affects model performance. While our default configuration follows an object-relation-attribute order based on semantic granularity and generation stability, we test alternative permutations to examine the sensitivity of DiT-ST Medium to ordering strategies.Table 8 reveals the cost of premature information exposure. Advancing attribute tokens from their intended late-stage slot to attribute-relation-object or attribute-object-relation cuts CLIPScore to 29.41 and

31.10, drops of 14 % and 9 % from the default order. Early injection forces fine-grained colour and texture cues into an unstable attention landscape; later denoising steps overwrite or mis-bind these details, sharply reducing semantic fidelity.

E Ablation on step selection strategy for semantic primitive injection.

We further investigate the impact of injection timestep choices for different semantic primitive types in the diffusion denoising process. Given that semantic primitives differ in abstraction level and generation dependency, we design a progressive injection schedule tailored to each type:

Object tokens are injected at the early stages of denoising (timesteps 0, 10, and 20), where the model primarily focuses on establishing global layout and coarse structural elements. We experiment with these positions to compare their impact and identify the most effective timing for guiding the foundational composition of the generated image.

Relation tokens are injected during mid-stage denoising. Specifically, we begin with timestep 25 as the midpoint of [0, 50], and extend to timestep 75 with five evenly spaced steps (30, 40, 50, 60, 70). We experiment with these positions to assess their impact and determine the optimal timing for enhancing spatial and compositional coherence.

Attribute tokens are injected during the transition from semantic interpretation to visual detail refinement, when the model shifts focus from global structure to fine-grained features. To study their impact on visual modeling, we take the midpoint of the [50, 400] range as reference and select timesteps at 100-step intervals-specifically 200, 300, 400, 500, and 600-for evaluation.

This step allocation strategy reflects the assumption that lower-level semantics (e.g., object identity) should be introduced early to guide coarse synthesis, while higher-level or localized attributes benefit from later-stage injection when image fidelity and semantic detail are resolved.

As shown in Table 9, deviating from the default object-relation-attribute (O-R-A) schedule markedly impairs text-image alignment. Postponing object injection to t = 10 -20 lowers CLIPScore by roughly 5-6 %, underscoring the need for early global cues. Advancing relation tokens to t = 30 reduces the score by about 4 %, as premature relational reasoning diverts attention from still-forming objects. The most severe loss, nearly 8 %, occurs when attribute tokens are introduced at t = 200 ; exposing fine-grained details before the scene is stabilised disrupts subsequent refinement as claimed in our main paper the question of premature information exposure. In aggregate, mis-timed injections can degrade alignment by average 5% , confirming that stage-aware scheduling is critical for maintaining semantic fidelity.

F Ablation on Sliding Window Size

Table 10 shows that CLIPScore peaks at 34.09Our when the SNR-smoothing window is set to w = 3 .A narrow window ( w = 1 ) amplifies local noise, shifting the detected inflection point and trimming the score to 33.16 (-2.7 %). Expanding to w = 2 alleviates some volatility, yet still lags

the optimum at 33.55 (-1.6 %). Oversmoothing with w = 4 and w = 5 postpones relation-token injection; scores decline to 33.72 (-1.1 %) and 33.43 (-1.9 %), respectively. These results confirm that an intermediate window is essential-wide enough to suppress spurious curvature spikes yet narrow enough to capture the true SNR transition-thereby yielding the most consistent relation-level guidance and highest text-image alignment.

G Ablation on Text Encoding Refinement.

As detailed in Section 3.2.2, our design includes a refinement for the classical text encoder of MMDiT. Specifically, we fully utilize the previously wasted dimension capacity by incorporating the complete caption to enrich the overall semantic information of the input. To further validate the effectiveness of this refinement, we conduct an ablation experiment on our DiT-ST Medium model, trained for 5 epochs on 50K SAM-LLaVA dataset. The results are presented in Table 11. Without text encoding refinement, the model performance has declines of 0.62% in CLIPScore and 0.54% in FID metrics. The result indicate that although the improvement from adding complete text captions is modest, it indeed contributes positively to model performance without introducing additional computational or architectural overhead.

H Limitation

The injection timestep selection operates at the batch level rather than the sample level. This is because sample-level adaptation would significantly increase computational cost and reduce inference efficiency, coupled with the current lack of research on sample-level adaptation in the conditioning of denoising processes. Therefore, this study adopts a batch-level adaptation design as a compromise.

The split-text caption may weaken cross-primitive semantic dependency. Some special longrange dependencies (e.g., irony) could be split, leading to biased semantic understanding in the model. To mitigate this issue, DiT-ST introduces the text encoding refinement design, which incorporates original caption to compensate for the loss. Furthermore, existing works in the same field have not explored evaluating the semantic fidelity of special cases such as irony.

The evaluation metrics lack human subjective assessment. The evaluation of DiT-ST primarily relies on CLIPScore and FID, aligning with other works in the same filed. These metrics lack human perception, particularly in fine-grained details (e.g., color tone, detail consistency). To compensate for this limitation, similar to other prior works, we provide extensive visualizations to facilitate subjective evaluation by readers.

I More Visual Comparison Across Caption Forms

Original: A quiet beach with soft sand, scattered white seashells. One small bird stands alone near the water, its feet in the shallow waves, looking down at its reflection. Gentle waves roll in behind it.

LLM-Enhanced: There is a quiet beach with soft sand and scattered white seashells. A small bird stands alone near the water, looking at its reflection. Our: [OBJECT] beach. sand. seashells. bird. waves. [RELATION] beach has sand and seashells. bird stands near water. feet are in waves. bird looks at its reflection. waves roll in behind the bird. [ATTRIBUTE] The beach is quiet. The sand is soft. The seashells are white and scattered. The bird is small and alone. The waves are gentle and shallow.

Our: [OBJECT] forest. trees. deer. moss. [RELATION] forest has tall pine trees. deer stands in moss. [ATTRIBUTE] The forest is quiet. The trees are tall and pine. The deer is young and with dots. The moss is soft.

SCHOOLBUIS

SOHOOLBUS

Original: A yellow school bus drove through a rainy city street at night. The damp road reflects the yellow shop lights on roadside, heavy rain is falling. LLM-Enhanced: There is a yellow school bus driving through a rainy city street at night. The wet road reflects the yellow lights of nearby shop lights. The rain falls heavily and steadily.

Our: [OBJECT] school bus. street. shop. lights. rain. [RELATION] bus drives on damp street. shop has lights. street reflects lights. [ATTRIBUTE] The bus is yellow. The street is damp. The lights are yellow. The rain is heavy.

Original: At a morning café, a man was flipping through newspaper by the window, with warm sunlight shining across the wooden table and a cup of coffee beside him.

LLM-Enhanced: There is a man sitting by the window in a morning café, reading newspaper. Warm sunlight shines across the wooden table, and a cup of coffee beside him.

Our: [OBJECT] café. man. newspaper. window. sunlight. table. coffee. [RELATION] man sits by the window. man flips through newspaper. sunlight shines on the table. coffee is beside the man. [ATTRIBUTE] The café is morning. The table is wooden. The sunlight is warm.

Original: A furry white cat quietly fell asleep, nestled in a warm, brown blanket adorned with delicate embroidered floral patterns and raised stitching. LLM-Enhanced: There is a furry white cat sleeping in a warm brown blanket. The blanket is decorated with delicate embroidered floral patterns and raised stitching.

Our: [OBJECT] cat.  blanket.  patterns.  stitching. [RELATION] cat  sleeps  in  blanket.  blanket  is  adorned  with  floral  patterns.  blanket  has  raised stitching. [ATTRIBUTE] The  cat  is  furry  and  white.  The  blanket  is  warm  and  brown.  The  floral  patterns  are  delicate  and  embroidered.  The stitching is raised.

Original: A silver pen without its cap rests on a wooden table beside a blank white spiral-bound notebook. A few other notebooks also lie nearby. LLM-Enhanced: There is a silver pen without cap resting on a wooden table next to a blank white spiral-bound notebook. Several other notebooks are scattered nearby.

Our: [OBJECT] pen. table. notebook. [RELATION] pen rests on table. pen is beside notebook. other notebooks lie nearby. [ATTRIBUTE] The pen is silver and uncapped. The table is wooden. The notebook is blank, white, and spiral-bound.

Original: A crispy  fried chicken burger with lettuce, pickles, and cream  sauce on top, placed beside  a container of golden  French fries on  a wooden table.

LLM-Enhanced: There is a crispy fried chicken burger topped with lettuce, pickles, and cream sauce, sitting next to a container of golden French fries on a wooden table.

Our: [OBJECT] burger. lettuce. pickles. cream sauce. fries. container. table. [RELATION] burger has lettuce, pickles, and cream sauce. burger is beside fries. fries are in a container. burger and fries are on the table. [ATTRIBUTE] The burger is crispy and fried. The lettuce, pickles and cream sauce is on top. The fries are golden. The table is wooden.

Original: A healthy meal with two egg yolk facing fried eggs, fresh half cut avocados, and a few slices of toasted whole wheat bread, lightly seasoned with herb.

LLM-Enhanced: There is a healthy meal consisting of two sunny-side-up eggs, fresh halved avocados, and a few slices of toasted whole wheat bread, lightly seasoned with herb.

Our: [OBJECT] meal. eggs. avocados. bread. herb. [RELATION] meal includes eggs, avocados, and bread. meal is seasoned with herb. [ATTRIBUTE] The meal is healthy. The eggs are fried and yolks up. The avocados are fresh and halved. The bread is whole wheat and toasted.

rinebcfam.

2C

J More Visual Comparison Across Different Models

In this section, we present more qualitative comparisons across different models. Specifically, we employ our large-scale model DiT-ST Large and compare it with the baseline MM-DiT 8B-E under identical prompt conditions. As shown in Figure 8, DiT-ST Large produces more compositionally coherent and semantically aligned results, especially in complex scenes with diverse attributes.

We further compare DiT-ST Large with several state-of-the-art text-to-image generation models, including PixArtα , Flux, and HunyuanDiT. As illustrated in Figure 10, our model demonstrates competitive or superior visual quality, with stronger object fidelity, spatial consistency, and semantic expressiveness.

Our DiT-ST Large

Flux .1 Dev

PixArt-

α

HunyuanDiT

A multi-colored wall clock is mounted on the wall, surrounded by a few sculptures, a label above reads "1106."

K More High Quality Images Generated by Our DiT-ST Large

BA

Besides the standard 1024 × 1024 images shown in Figure 10, we further generate multi-scale samples put in Figure 11, confirming that our DiT-ST Large maintains visual fidelity and semantic alignment across different resolutions and formats.

NeurIPS Paper Checklist

1. Claims

Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?

Answer: [Yes]

Justification:The abstract and introduction clearly outline the core contributions of our work DiT-ST, including the motivation, methodological innovations, and performance gains.

Guidelines:

The answer NA means that the abstract and introduction do not include the claims made in the paper.

The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.

The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.

It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper.

2. Limitations

Question: Does the paper discuss the limitations of the work performed by the authors?

Answer: [Yes]

Justification: We acknowledge the limitations of our work and will provide a dedicated discussion in the appendix.Rather than aiming to surpass all existing baselines, our contribution focuses on developing an effective and generalizable method that performs consistently across tasks and domains. We believe this design choice better serves long-term robustness and transferability.

Guidelines:

The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.

The authors are encouraged to create a separate "Limitations" section in their paper.

The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.

The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

The authors should reflect on the factors that influence the performance of the approach. For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.

The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.

If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.

While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.

3. Theory assumptions and proofs

Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?

Answer: [Yes]

Justification: The paper includes the necessary theoretical results with all assumptions clearly stated and referenced in the respective sections. All proofs and lemmas are either presented within the main paper

Guidelines:

The answer NA means that the paper does not include theoretical results.

All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced.

All assumptions should be clearly stated or referenced in the statement of any theorems.

The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition.

Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.

Theorems and Lemmas that the proof relies upon should be properly referenced.

4. Experimental result reproducibility

Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?

Answer: [Yes]

Justification: To ensure full reproducibility, we release all necessary resources, including model weights, training and evaluation code, and preprocessed datasets. Detailed instructions for environment setup and running experiments are provided in GitHub repository.

Guidelines:

The answer NA means that the paper does not include experiments.

If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.

If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.

Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.

While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example

(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.

(b) If the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.

(c) If the contribution is a new model (e.g., a large language model), then there should either be a way to access this model for reproducing the results or a way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).

(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.

5. Open access to data and code

Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?

Answer: [Yes]

Justification: We release both source code and necessary data to facilitate faithful reproduction of our results. The code repository includes training and inference scripts and environment configuration files. In addition, we provide access to all datasets used in our experiments on huggingface.

Guidelines:

The answer NA means that paper does not include experiments requiring code.

Please see the NeurIPS code and data submission guidelines ( https://nips.cc/ public/guides/CodeSubmissionPolicy ) for more details.

While we encourage the release of code and data, we understand that this might not be possible, so 'No' is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).

The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines ( https: //nips.cc/public/guides/CodeSubmissionPolicy ) for more details.

The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.

The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only a subset of experiments are reproducible, they should state which ones are omitted from the script and why.

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).

Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted.

6. Experimental setting/details

Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?

Answer: [Yes]

Justification: We describe all experimental settings in both the main paper and the appendix. Key implementation details such as data splits, model architecture, optimizer type, learning rate schedules, and hyperparameter configurations are summarized in the main text to support result interpretation.

Guidelines:

The answer NA means that the paper does not include experiments.

The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.

The full details can be provided either with the code, in appendix, or as supplemental material.

7. Experiment statistical significance

Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?

Answer: [Yes]

Justification: We evaluate our method using commonly adopted standard metrics, including [e.g., FID, CLIPScore, or others], to ensure comparability with existing baselines. In addition to quantitative evaluation, we also provide visualizations of generated outputs to support qualitative assessment and highlight consistency across samples. These visual results, alongside statistical metrics, offer a comprehensive understanding of model behavior.

Guidelines:

The answer NA means that the paper does not include experiments.

The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.

The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).

The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)

The assumptions made should be given (e.g., Normally distributed errors).

It should be clear whether the error bar is the standard deviation or the standard error of the mean.

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified.

For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).

If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.

8. Experiments compute resources

Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?

Answer: [Yes]

Justification: We specify in the main paper that all experiments were conducted on NVIDIA A100 GPUs.Each training run took approximately 12-24 GPU hours.

Guidelines:

The answer NA means that the paper does not include experiments.

The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.

The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.

The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).

9. Code of ethics

Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines ?

Answer: [Yes]

Justification: We have carefully reviewed the NeurIPS Code of Ethics and confirm that our research fully conforms to its principles. Our work does not involve sensitive user data, human subjects, or harmful applications. All datasets used are either publicly available or properly licensed, and we ensure transparency, reproducibility, and responsible reporting throughout the paper.

Guidelines:

The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.

If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.

The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).

10. Broader impacts

Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?

Answer: [Yes]

Justification: Our work introduces a novel and generalizable perspective for improving existing generative models, aiming to address foundational challenges in a model-agnostic manner. Rather than optimizing for narrow task-specific gains, our approach seeks to uncover transferable mechanisms that can be broadly applied across architectures and domains. This may positively influence future research toward more unified and interpretable generative modeling. While our method is not designed for deployment in sensitive or high-risk applications, we acknowledge the broader implications of advancing generation quality, and support responsible usage and release protocols.

Guidelines:

The answer NA means that there is no societal impact of the work performed.

If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.

Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.

The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.

If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).

11. Safeguards

Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?

Answer: [Yes]

Justification: We acknowledge that high-quality generative models may pose risks of misuse, such as unauthorized image synthesis or disinformation. To mitigate such risks, we commit to releasing our pretrained models and code under a non-commercial research-only license. We also include usage guidelines outlining responsible applications, and plan to implement content safety filters to restrict harmful outputs. Furthermore, we will monitor the downstream use of our models and reserve the right to restrict access if misuse is identified.

Guidelines:

The answer NA means that the paper poses no such risks.

Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.

Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.

We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort.

12. Licenses for existing assets

Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?

Answer: [Yes]

Justification: We make proper attribution to all external assets used in our work. Specifically, we use the COCO 2017 dataset, which is publicly available under a Creative Commons Attribution 4.0 License (CC-BY 4.0), and we follow its terms of use. Our experiments are based on the Stable Diffusion v3 model provided by Stability AI, which is released under a custom non-commercial license; we cite the official source and follow the stated restrictions. We also build on open-source libraries from Hugging Face, such as Transformers and Diffusers, which are released under the Apache 2.0 license.

Guidelines:

The answer NA means that the paper does not use existing assets.

The authors should cite the original paper that produced the code package or dataset.

The authors should state which version of the asset is used and, if possible, include a URL.

The name of the license (e.g., CC-BY 4.0) should be included for each asset.

For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.

If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset.

For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.

If this information is not available online, the authors are encouraged to reach out to the asset's creators.

13. New assets

Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?

Answer: [NA]

Justification: We do not introduce any new assets in this work. All experiments are conducted using publicly available datasets (e.g., COCO 2017) and existing pretrained models (e.g., Stable Diffusion v3). We ensure that these assets are properly cited and used in accordance with their respective licenses.

Guidelines:

The answer NA means that the paper does not release new assets.

Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.

The paper should discuss whether and how consent was obtained from people whose asset is used.

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.

14. Crowdsourcing and research with human subjects

Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?

Answer: [NA]

Justification: Our work does not involve any form of crowdsourcing or research with human subjects. All experiments are conducted using publicly available datasets and pretrained models without collecting new human-provided annotations or feedback.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper.

According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.

15. Institutional review board (IRB) approvals or equivalent for research with human subjects

Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?

Answer: [NA]

Justification: This work does not involve research with human subjects or crowdsourcing. All experiments are conducted using publicly available datasets and pretrained models, without collecting new human annotations, feedback, or participation. Therefore, no IRB or equivalent ethical approval is required.

Guidelines:

The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.

Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.

We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution.

For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.

16. Declaration of LLM usage

Question: Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.

Answer: [Yes]

Justification: We used the Qwen Plus API, a large language model, to assist in processing and structuring textual data used during pre-processing. This usage does not directly affect the core model architecture or training process and can be replaced by any other LLM models, but facilitates intermediate data organization and annotation required for our pipeline. We describe this usage and its scope in Appendix G to ensure transparency and reproducibility.

Guidelines:

The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non-standard components.

Please refer to our LLM policy ( https://neurips.cc/Conferences/2025/LLM ) for what should or should not be described.

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a text-to-image generation method using split-text as a condition. Existing text-to-image models struggle to comprehend complex semantic information, so the authors attempt to refine the text using large language models, constructing multiple complex sub-texts based on three dimensions: object, relation, and attribute. The authors also note that these three types of prompts have different impacts on the model during various stages of denoising in the diffusion model, leading to the design of a staged injection mechanism. Experiments show that this design effectively improves the model's ability to follow prompts.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\n1. The author's use of a graph to model the text is an interesting approach. This method of normalizing the prompt can be helpful for weak text encoders.\n\n2. The author conducted experiments on multiple models to further validate the effectiveness of the approach.\n\n3. The paper is easy to understand.\n\nWeaknesses:\n\n1. Due to the limited text understanding capabilities of models like CLIP and T5, many existing works use powerful LLMs as text encoders for diffusion models[1,2,3]. Can this design further improve performance when used with LLMs with stronger understanding abilities?\n\n2. In the paper, the authors inject object, relation, and attribute into the denoising process in three separate stages. However, since object and relation jointly determine the layout of the generated image, could using only the object prompt in the early denoising stages affect the accuracy of the relation, potentially leading to extra objects or incorrect relationships? The authors should analyze the output results from denoising with different prompts.\n\n3. Regarding the cost analysis of Injection Timestep Selection, for different diffusion models, will these injection timesteps differ significantly, and what kind of time and resource overhead will this bring?\n\n4. The evaluation benchmarks are weak. ClipScore and FID do not reflect the model's understanding of complex prompts. The authors should use additional benchmarks with LLMs as evaluators for ablation studies. They should also provide more showcases to demonstrate that the fine-tuned model has improved prompt comprehension.\n\n5. Using a graph to model prompts lacks completeness. Some prompts don't contain fully defined semantic information, which is common in platforms like MidJourney. For a prompt that doesn't specify an object, what kind of output will the model generate?\n\n[1] https://arxiv.org/abs/2406.11831\n\n[2] https://arxiv.org/abs/2410.10629\n\n[3] https://arxiv.org/abs/2505.10046\n\nQUESTIONS:\nPlease refer to the weakness.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-06-30"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper approaches to improve T2I diffusion models by splitting text captions into smaller, independent, sentences that contain different (hierarchical) semantic primitives, e.g., general image layout vs vs details for specific objects. These primitives are then injected into the diffusion model at different timesteps, starting with more general information at the beginning of the sampling process and ending with detailed object information at the end of the sampling process. The evaluation shows improved scores on GenEval, CLIP score, and FID.\n\nSTRENGTHS AND WEAKNESSES:\nThe paper addresses an important topic of improving text-image alignment in diffusion models. The breaking down of a complex instruction into various smaller sub-instructions makes intuitive sense. Structuring the creation of the sub-instructions with the well-known behavior of diffusion models to first generate broad layouts and later on focus on semantic details also makes sense.\n\nThe paper finds ways to automatically address these, by automatically creating sub-captions and proposing an automated way to find when to inject them into the model. That being said, the evaluation is only done on one kind of model so it is not clear if this transfers easily to other models and whether the injection approach also works on other models.\n\nHowever, there are many ways to split a caption into sub-sections and it's not clear that the one proposed here is the best one or transfers to different models. It might also depend on the kind of data that a model is trained on. \n\nThis also leads to extra compute requirements, both at train and at test time, since the input captions have to be processed to the split-text format. It also requires additional finetuning of the model, though the approach could also work without finetuning (though this is not tested in the paper I think).\n\nQUESTIONS:\nThe approach needs additional processing both at training and test time. It would be interesting to do some ablations comparing the effect of different ways of doing the split-text caption construction. It seems relatively involved right now and sounds somewhat brittle.\n\nSimilarly, there is some extra design involved in how and when to inject the different kinds of captions. One interesting baseline would be to just feed the split-text captions into a \"normal\" pretrained models (without any finetuning) at the given timesteps to check the kinds of improvements this would lead to (if any).\n\nIt also seems like right now the conditioning also always includes (a projection of) the full caption. How important is that? Does it still work if it only conditions on the split-text conditions? \n\nHow will this approach affect potential down-stream tasks (such as editing) which would be finetuned from a pretrained T2I model?\n\nLIMITATIONS:\nThe finetuning of the models was done with a dataset that only contains 100K examples. This is relatively small-scale and injecting human biases is likely to help here. It is not clear to me that we would see the same kinds of improvements when a general diffusion model is trained on a large-scale dataset of longer and more complex prompts. Table 2 also shows that models trained on a dataset of more variety also perform well in most settings. It is not clear that using the split-text condition is really helpful compared to training on a larger dataset with diverse, long, and detailed captions.\n\nAdditionally, the approach requires some specific design choices (e.g., how exactly to create the split-text captions, when and how to inject them, etc) and it is not obvious to me that this will easily generalize across different datasets and model architectures.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nSummary\nThis paper proposes DiT-ST, a novel split-text conditioning framework for text-to-image diffusion models. Instead of injecting the full caption all at once—which can be semantically dense and challenging for the model to parse—DiT-ST decomposes the caption into multiple semantic primitives (e.g., objects, attributes, relations) using a large language model. These primitives are then hierarchically sorted and injected at different denoising timesteps via cross-attention during the generation process. The motivation is that the progressive nature of diffusion is better suited for staged semantic guidance. Experiments show consistent improvements over strong baselines, demonstrating improved generation quality and semantic alignment\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n\n• The paper is clearly written, and the core idea—decomposing long captions and aligning semantic primitives with different timesteps—is intuitive and elegant.\n\n• The proposed semantic scheduling strategy represents a meaningful extension of earlier works that perform conditioning at fixed stages. By tailoring the conditioning granularity and injection timing, DiT-ST introduces a more structured and interpretable control mechanism.\n\n• The method is modular, requires no changes to the underlying diffusion backbone, and can be readily adapted to existing models.\n\n• The experiments are comprehensive, spanning both qualitative visualizations and quantitative results on multiple datasets and text complexity levels. The SNR-based analysis is particularly insightful and well-motivated.\n\n• The paper includes ablations on primitive types, text parsing, and injection schedules, helping the reader understand where the gains come from\n\nWeaknesses\n\n• The formulation in Equation (10) lacks clarity—particularly, the attention loss term L_{\\text{attn}} is not defined in full. It’s unclear what form it takes, what layers it applies to, and how it affects optimization. This should be made explicit for reproducibility.\n\n• The choice of injection timesteps for different semantic primitives (e.g., attributes) is motivated by the convergence of cross-attention outputs, but this assumption is not backed by empirical visualizations. As with Figure 5 (which illustrates the SNR inflation point), the paper would benefit from an analogous figure showing how cross-attention actually converges over time.\n\nQUESTIONS:\nPlease refer to the weakness section.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 2, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-04"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nText-to-image diffusion transformers are typically conditioned on the entire caption during inference. When the caption is long or describes complex relationships between objects, the model struggles to understand the full input. This is due to how tokens compete with each other and how they are positioned in the sentence.\nTo address this, the paper proposes a method that splits the caption into a hierarchical structure based on its semantic components and their relationships. It has been shown that the convergence of cross-attention in diffusion transformers indicates when semantic objects are formed and when details are refined during inference. By analyzing this convergence, the proposed method decides when each part of the hierarchical structure should be given as condition to the model during inference.\nExperiments show that the method performs better on the COCO-5K benchmark and achieves results comparable to state-of-the-art methods on the GenEval benchmark.\n\nSTRENGTHS AND WEAKNESSES:\nThe paper is clearly written and easy to follow. The proposed method is simple to implement and can be applied to multiple models at inference time.\n\nThe idea of conditioning the model on different semantic parts of the caption at different inference steps is original, and using cross-attention convergence to guide this process seems sound.\n\nThe experimental results are convincing, and the ablation studies show that the method’s components contribute effectively.\n\nOverall, this is a solid paper. I look forward to seeing the other reviews and the authors’ responses.\n\nQUESTIONS:\nGiven that the caption is divided hierarchically, we would expect each text input to the model to be shorter. Since the method uses a pretrained encoder (three encoders, in fact), part of the encoder input might not be used. The paper mentions that the full caption is still provided as input to fully use the encoder. Does this actually improve model performance?\n\nAre there any changes in memory or computational cost when using the proposed method?\n\nCould the method also be applied during training? I'm not asking the authors to implement this, but I would appreciate a discussion of the potential challenges and benefits.\n\nLIMITATIONS:\nLimitations are discussed in the appendix.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-08"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper introduces a new framework DiT-ST which aims to handle the inherent limitations of existing text-to-image models. Specifically,  understanding complex long texts through a technique called split-text conditioning is not trivial. \nThis framework leverages a large language model to parse a long textual description into a set of simpler, hierarchical short phrases. \nThese phrases are then incrementally injected into different stages of the denoising process. \nThe experiments show that it significantly improves the generated image's detail accuracy and semantic consistency and the method outperforms or rivals state-of-the-art models with minimal parameter increasement.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n- This paper uses a simple appraoch to let the model more accurately understand and generate fine-grained details from the input. The method is splitting complex text into simple subject-relation-attribute meta-phrases. \n- The method is parameter-efficient and architecture-agnostic as the core idea lies in preprocessing the input text and injecting it in stages without requiring major modifications to the model architecture. \n- Experiments show that this method performs well in very long prompts with satisfying CLIPScore performance. Also on the comprehensive benchmark GenEval the DiT-ST model outperforms the SDv3 Medium baseline in many dimensions. \n\nWeaknesses:\n\n- This paper lacks an important reference of \"Progressive Prompt Detailing for Improved Alignment in Text-to-Image\nGenerative Models\" (https://arxiv.org/pdf/2503.17794v1) which has a very similar method on the same issue. This papper should discuss the differences. \n- The evaluation is not enough. The evaluation metrics are not comprehensive. Though FID is adopted to evaluate the image quality while it is limited. Also, this paper only adopts GenEval and CLIPScore to test the text-image alignment without a user study. \n- The experiments are not comprehensive. Only one LLM is adopted for the experiments. Other LLMs should also be tested for the effectiveness and generalization. Also, this paper does not discuss if finetuning LLM works better for this task. \n- The experiment does not include failure cases and analyze them. \n- The captions should be more detailed to better illustrate the content and the motivation of the figures and tables.\n\nQUESTIONS:\nThe authors are encouraged to discuss the differences to the refered paper and complement necessary experiments.\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-08"}