Rare Text Semantics Were Always There in Your Diffusion Transformer
Authors: Seil Kang
Woojung Han
Dayun Ju
Seong Jae Hwang
Venue: NeurIPS 2025 poster ()
Keywords: generative models
diffusion transformer
text-to-vision
Primary area: applications
TL;DR: Variance scale-up of text-token embeddings in MM-DiTs before joint attention lets rare prompts emerge without retraining, extra data, or denoising tweaks, boosting text-to-image, video, and editing

Abstract: Starting from flow- and diffusion-based transformers, Multi-modal Diffusion Transformers (MM-DiTs) have reshaped text-to-vision generation, gaining acclaim for exceptional visual fidelity. As these models advance, users continually push the boundary with imaginative or rare prompts, which advanced models still falter in generating, since their concepts are often too scarce to leave a strong imprint during pre-training. In this paper, we propose a simple yet effective intervention that surfaces rare semantics inside MM-DiTs without additional training steps, data, denoising-time optimization, or reliance on external modules (e.g., large language models). In particular, the joint-attention mechanism intrinsic to MM-DiT sequentially updates text embeddings alongside image embeddings throughout transformer blocks. We find that by mathematically expanding representational basins around text token embeddings via variance scale-up before the joint-attention blocks, rare semantics clearly emerge in MM-DiT’s outputs. Furthermore, our results generalize effectively across text-to-vision tasks, including text-to-image, text-to-video, and text-driven image editing. Our work invites generative models to reveal the semantics that users intend, once hidden yet ready to surface.

Rare Text Semantics Were Always There in Your Diffusion Transformer
Seil Kang"
Woojung Han"
Dayun Ju
Scong Jae Hwang
Yonsei University
{seil, dnwjddl, juda0707, seongjae] @yonsei. ac kr
Text-to-lmage Generation
Text-Driven Image Editing
trophy
Blux
with cuzly haiz
srable Flow
Ours
covezed
Ours
Ours
bioluning Scent nushroors
Eiffel Tower made of water
A dragon made of suternelon slices breathing flakes of snow into the sky
Stable Flow
Ours
Text-to-Video Generation
Ours CogVideoX
rides
drifting
cozy roon,
slaped person in safesuft resting slice triangular piace the with crumbs scattared around
captured photoreal
Figure 1: Our method, ToRA, achieves superior semantic alignment in text-to- rext-to-vision outputs for rare prompts while requiring neither finetuning, optimization, nor additional modules; Misfired phrases in the baseline and existing method outputs are highlighted in red.
Abstract
Starting from flow- and diffusion- based transformers, Multi-modal Diffusion Trans- formers (MM-DiTs) have reshaped text-to-vision generation, gaining acclaim for exceptional visual fidelity. these models advance, users continually push the boundary with imaginative or rare prompts, which advanced models still falter in generating, since their concepts are often too scarce to leave strong imprint during pre-training. In this paper, propose a simple yet effective intervention that surfaces rare semantics inside MM-DiTs without additional training steps, data, denoising -tme optimization, reliance external modules (e.g.. large language models). In particular, the joint-attention mechanism intrinsic to MM-DiT sequen- tially updates text embeddings alongside image embeddings throughout transformer blocks. We find that by mathematically expanding representational basins around text token embeddings via variance scale-up before the joint-attention blocks, rare semantics clearly emerge MM-DiT's outputs. Furthermore, our results general- ize effectively
text-to-vision tasks, including text-to-image, text-to-video. and text-driven image editing. Our work invites generative models to reveal the semanties that intend, once hidden yet ready surface.
"qual contribution
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Endedcrngs
Latert Entooddngg
L.M
child rictng Swing
M
D
Joint
Joint
Block
Block
0
Base
Curs Results contracted by proript and
(4) Plactuning Diffision Transfemete
Optlaiizaiinn besed Diltusson Thansformer
12M grounded
(4) Ours
Transfoemer
Figure 2: Comparison of existing -rrnsfoomtt methods: (a) Finetuning, (b) Optimization- based, (c) LLM- -LM-grounded guidance, and (d) Ours. (e): Contrastive results between rare and common prompts, comparing baseline and ours. The rotating arrow in (b) shows the latent-vector update loop at each timestep.
1 Introduction
Recent progress in text-conditioned generative models has spurred interest in diffusion-based models for generative vision tasks [1-4]. Today, numerous studies have introduced methods based on Multi- modal Diffusion Transformers (MM-DiT), providing sophisticated mechanisms for producing images or videos from textual prompts [3, 4]. As these models advance, users increasingly push the boundary by exploring rare or highly imaginative prompts. However, Park et al. [5] note that such rare concepts (e.g., "an Eiffel tower made of water"' in Fig. 1) appear scarcely in pretraining datasets, often causing generative outputs to misalign with their text semantics. This inherent limitation can typically be overcome by finetuning [6-10], applying optimization techniques during inference [5, 10, 11], or leveraging the capabilities of large language models (LLMs) [5, 8, 12] (refer to Fig. 2(a)-(c)).
However, illustrated in Fig. 2(e), we observe that base MM-DiT accurately captures the semantics of a common prompt such as "A child riding a swing", whereas struggles to represent the meaning of rare prompt like "A carrot riding swing". This observation suggests that the model indeed recognizes specific semantics (e.g.. "riding a swing") but faces challenges when visually reflecting rare textual semantics. Consequently, we suspect that the semantics of rare prompts exist within the text embeddings but remain inaccessible to the model, leading us to pose the following question: Can rare textual semantics naturally emerge within MM-DiT embeddings?
The studies natural language processing provide valuable insights into this analogous problem. Prior studies [13-16] have shown that text embeddings from transformer based models naturally exhibit anisotropy, meaning they are dominantly distributed along specific directions in their semantic space, resulting in high cosine similarities. In other words, anisotropic text embeddings geometrically form a hypercone semantic space. However, Cai al. [15] demonstrated that many models [17-19] naturally exhibit global anisotropy yet display local isotropy (evenly distributed), enabling them to achieve strong contextual understanding by distinguishing between textual embeddings. Inspired by these findings, we examine how anisotropy and isotropy behave across MM-DiT's joint-attention blocks and how they affect rare textual semantic emergence.
We start with the intuition that clarifying text embeddings in the semantic space could preserve distinct semantics and enhance visual generation. Interestingly, our analysis reveals that variance scale-up of text token embeddings amplifies the local isotropy in each joint-attention block, facili- tating the clearer emergence of rare semantics. For example, Fig. 1, when processing the prompt "an Eiffel tower made of water," increasing the representational distance between each text to- ken (e.g., "tower" and "water") helps preserve distinct semantics throughout generation process. Conversely, from global perspective, the variance scale-up increases global anisotropy, yet does not necessarily hinder semantic emergence, in line with prior findings [20-23]. Furthermore, we demonstrate that this intervention benefits contextualization during wist self-attention within joint-attentin Nevertheless, while promising, we clarify that variance scale-up may inadvertently amplify embedding directions unrelated to meaningful semantics. Considering all these aspects, seek a practical approach that capitalizes on the advantages of variance scale-up while accounting for its potential pitfalls, aiming for semantic emergence robustly in text-conditioned generation.
We introduce ToRA, short for Token Spacing and Residual Alignment. Instead of retraining the model or using external modules (e.g., LLMs) as shown in Fig. 2(a)-(c), ToRA has small interventions

---

(Fig 2(d)). We consider the text embedding space into two distinct subspaces: principal space and a residual space can be partitioned via Principal Component Analysis (PCA). Token Spacing scales up variance within the principal space, thereby enhancing semantic distinguishability. Con- currently, Residual Alignment refines the embeddings by rotating their residual space toward the semantic direction [24-26], mitigating unintended side effects from the token spacing. Together, these complementary methods ensure that rare semantic tokens are not overshadowed by other tokens.
To sum up, we bear out that rare textual semantics can naturally emerge within MM-DiT embeddings with our intervention. ToRA unleash dormant semantics seamlessly integrating into standard MM- DiT inference. Its exceptional generalizability is validated across text-to-vision tasks by the impressive outcomes in video generation and image editing shown in Fig. 1. With the help of its low-overhead and compatobp features, ToRA can be easily incorporated with existing methods [5, 27, 28]. Moreover, ToR also consistently improves semantic coherence and generative quality across a broad spectrum of textual prompts, including common prompts. Thus, our work shifts the paradigm from attempting to externally impose semantic alignment onto generative models to naturally revealing semantics that were already there, hidden yet waiting to emerge.
2 Preliminaries
2.1 Joint attention mechanism in MM-DiT.
Multi-modal Diffusion Transformer (MM-DiT) extends Diffusion Transformer (DiT) [29] to multi- modal text-to-vision by jointly processing textual and visual representations, conditioning on CLIP (L/14, G/14) [30] and T5-XXL [31] in SD 3.0 [4] and TS-XXL alone in FLUX.1 [3]. Given text tokens, N latent-noise tokens, and hidden dimensions d, the initial text embedding e!nit E RRVx xd and initial latent-noise embedding init E IR xd are concatenated and passed through B joint-attention blocks, at each of which the joint-attention mechanism updates both the text condition and latent noise. Detailed formal explanations are in the $B.
2.2 Probing semantic geometry in transformer embeddings.
We primarily focus on concepts of anisotropy and isotropy in text semantic spaces [13-16]. Anisotropy describes embeddings dominated by few directions, resulting in high cosine similarity regardless of contextual relations, while isotropy indicates embeddings evenly distributed across various directions. Previous studies quantifying these properties within transformer layers reveal that the semantic space of text embeddings is characterized by high global anisotropy [13, 14]. Conversely, when these embeddings are clustered by some clustering algorithms [32-34], each cluster exhibits low cosine similarity, indicating local isotropy [15]. With these insights, we examine the beneficial effects of variance scaling on the anisotropic and isotropic properties of text semantic spaces within MM-DiT blocks.
3 Analyses
Here, we explore how methodically scaling text embedding variance within joint-attention blocks gives rise to semantic emergence, enabling the model to bring concepts from text prompts with greater visual fidelity, particularly for rare concepts. To confirm the reliability of the observed improvements, we perform an in-depth examination of the text semantic space's s isotropic properties, considering both local and global perspectives [13-16]. Furthermore, discuss and analyze potential limitations, noting that variance scale-up may not always be beneficial. Additional setup details are in the $C. 1.
3.1 Rare Text Semanties Emerges via Variance Scale-Up
We begin with a straightforward intuition [35, 36]: text embeddings should remain distinct in semantic space to effectively represent their underlying concepts, rather than becoming opaque within the model. other words, we propose that specific semantics text tokens inherently exist within a set of prompt embeddings, but generative models fail to visually represent these semantics as they cannot effectively retrieve the intended meanings. To address this, we suggest scaling the variance text embeddings before foint-attention computations, ensuring each embedding has clear representational
3

---

SD original
FLUX origial
Sumool Sunof Elgennoke
b
variznce scale down
var zorianee scale-up
variance scale down
Localtetrous Score G
(d)
tigez striped golden retriever and oval shaped table
candelion zunning with legs
Bioc Index
Figure 3: Effects of variance scaling. blocks, (b) Generated images illustrating visual outcomes, (c) Local isotropy scores across joint- attention blocks, and (d) Visualization of self-attention maps for text embeddings. Results are shown for variance scale-down, original, or scale-up.
(a) Sum of eigenvalues of text embeddings across joint-
margin within the semantic space. Let ex denote the embeddings at block b, with mean eb and variance Var(e"). Then, the scaled embeddings é¹ are given by: éh
e"
Var(e")
To empirically validate the effect of variance scaling, Fig. 3(a) compares the eigenvalue summation of embeddings under variance scale-down (a 1), original, and variance scale-up (a >> 1) conditions. Fig.3(b) illustrates that variance scale-up significantly enhances the visual representation of rare semantics (e.g., tiger striped golden retriever). To further understand how variance scale- up contributes to these visual improvements, we examine its effectiveness in capturing contextual relationships among text tokens by comparing the averaged self-attention maps of original and variance-scalee scaled embeddings across timesteps (Fig. 3(d)). The results indicate that variance scale-up intensifies self-attention activations among text tokens, thereby enhancing inter-token relationships. Subsequently, we conduct detailed analysis of the semantic space's 's isotropic properties, considering both global and local perspectives [13-16], as elaborated in the following sections.
3.2 Variance Scale-Up Makes The Text Semantic Space More Isotropic
Prior research [15] has shown that transformer models inherently exhibit global anisotropy, whereas local isotropy tends to better capture meaningful semantic distinctions. Herein, motivated by this insight, we initially focus on local isotropy. To do so, we first quantify local isotropy at each joint- attention block of MM-DiTs using metrics introduced by Cai et al. [15]:
Stocat (e)
[[BBiy/coss(e ef]],,
where indices i.j represent token embeddings, and indexes local semantic clusters identified using a Gaussian Mixture Model [34]. The centered embedding é; is computed as é; = e; Tel Ejer ej- Higher scores indicate increased isotropy, suggesting evenly distributed semantic directions. Fig. 3(c) compares the local isotropy scores for variance scale-down, original, and variance scale-up embed- dings, empirically demonstrating that variance scale-up improves local isotropy across joint-attention blocks. Moreover, Fig. 3(a) and (c) exhibit closely corresponding trends between the block- bisck-wise local isotropy of text embeddings and their eigenvalue summation. As result, we can confirm that variance scale-up enhances isotropy in the text semantie space, improving the model's text-conditioned visual generation process and resulting in better semantic alignment. Further analysis, other metrics [23], and derivations for local isotropy are in the $C.2.
3.3 High Global Anisotropy Is Harmless to Semantic Emergence
Similar to our previous experiments, we also evaluated the effect of variance scaling on global anisotropy [13]. As illustrated in Fig. 4, variance scale-up increases global anisotropy, yet sub- stantially improves the visual representation of specific text tokens (e.g., horned; annotated by blue). This aligns with the machine learning community's 's finding [20-23] that anisotropy of global perspective allows networks to generalize better to unseen examples, and it can be naturally extended

---

A <
ELIE
Conddrin
Olaa OkbalArtoorppp Sore
0.10
(N son 003 wtw.:/o000.oooo.o Seppoetnin
5.8
* Ar CIS
92-
0.00
Abtoned Mlephort
Block Indix
Vf
Figure 4: Global anisotropy scores across joint-attention blocks, comparing SD 3.0 and FLUX under three text embedding variance settings: scale-down, original, and scale-up.
Figure 5: Comparison of Ar across 500 seeds un- der identical prompt conditions. Each star repre- sents pair of generated images (original vs. scale- up). Red (+) = 0.45, Blue (*) = 0.79 (CLIPRex))
to MM-DiT's [3, 4]. On the other hand, a prior study [14] indicated that high global anisotropy in text semantic spaces could induce semantic opacity in language models. This study proposed reducing global anisotropy by removing principal components identified via PCA [37] to enhance contextual clarity (see $C.3 for more details). However, when we applied this method to MM-DiT's global anisotropy significantly decreased (yellow lines in Fig. 4), but the visual outputs severely deteriorated (yellow-boxee boxed images), resulting in entirely noisy results. Thus, we conclude: (1) variance scale-up effectively enhances rare semantic emergence by amplifying local isotropy in $3.2, and (2) increased
global anisotropy through variance scale-up does not negatively affect rare semantics in MM-DiTs.
3.4 Pitfalls of Variance Scale-Up via Semantic Vector Analysis
While variance scale-up for text token embeddings can sharpen inter-token distinctions ($3.1), it raises concerns about potentially deviating from the core semantic direction of prompt. To explore this possibility, we define the semantic vector a as the difference between a conditional text embedding Gcond. which captures prompt-specific conditions, and its unconditional (null) counterpart eø in classifier free guidance (CFG) [24-26]: - Econd Eg. The vector indicates the key shift from unconditional to conditional meaning in the MM-DiT's s text embedding space. Through semantic vector measure its impact through the cosine-alignmenn alignment change Α cos(s, é) cos s, e). We evaluated Ay and CLIP scores across 500 random seeds using identical prompts, as illustrated in Fig. 5. The results indicate that variance scale-up successfully led By in approximately 75% of random seeds; however, there remain cases where it resulted in Ay < O, often failing to render rare semantic elements (e.g., "hand shaped") in the generated images, resulting in lower CLIP scores compared to instances where Ay > 0. Moreover, we show the mathematical proof for this phenomenon in $C.4. As result, the variance scale-up alone cannot guarantee the preservation of semantic alignment, motivating the strategy of Residual Alignment introduced in $4.2. We also analyze with different examples that are provided in the $C.4 similar to Fig. 5.
4 ToRA: Token Spacing and Residual Alignment
Our previous analyses delivered valuable insights regarding the manipulation of text embeddings within MM-DiT, showing the necessity of scaling up the variance of their token embeddings to support semantic distinguishability. Motivated by these findings, we introduce ToRA, shown in Fig. 6. Specifically, ToRA leverages Principal Component Analysis (PCA) to decompose the text semantic space into two distinct segments: (1) the principal space, which captures the dominant semantic dimensions that characterize text tokens, where we apply Token Spacing to amplify variance and enable a clear distinction between token embeddings; (2) residual space comprising less informative or noisy components, where we apply Residual Alignment to counterbalance undesirable side effects of token spacing as considered in $3.4. Note that, for a single prompt, we perform PCA independently for each block at each timestep, with no sharing across blocks timesteps.
5

---

ToRA
Taxt Embeddings
Latent Embeddings
V IV pe G.V.
8
U-UV
Samantic vectoor
ToRA
8 proj
Principal Space
Residual Space
Joint Allention Block
Σ diag Ai So......
Ru)
U uk+ +1
2 Joh. A.. 12
it ii
(a) Token Spacing
(b) Residual Alignment
X B
Figure 6: Overview of TORA. (Left) ToRA applied text embeddings before joint-attention blocks. (Right) ToRA operates in two complementary steps: (a) Token Spacing, which expands distances among token embeddings by scaling singular values in the principal semantic space; and (b) Residual Alignment, which rotates the residual space to align with target semantic vector.
4.1 Token Spacing
We propose Token Spacing to enhance the distinguishability among token embeddings, illustrated in Fig. 6(a). At each timestep and each joint-attention block b, we perform PCA before applying joint-attention on the e", which is set of text token vectors. This procedure continues sequentially in every block and timestep. This intervention is uniformly applied across all blocks and timesteps, denoting e (for number of blocks 6) simply e for brevity. Initially, we decompose the space of original text embeddings e within each joint attention block using Principal Component Analysis (PCA): UΣV where U and V are orthogonal matrices, and Σ is diagonal matrix comprising singular values arranged in descending order (A1 2 X 2 ... 2 An) Subsequently, we automatically determine the principal subspace by identifying an optimal cutoff point for top-k dimensions, known as the elbow point. This elbow point, denoted by k, is obtained using the Maximum Distance to Chord (MDC) method [38] (see $D.1 for the detailed methodology). Within this identified subspace, Token Spacing is realized by selectively scaling the top & singular values as follows:
2, Soii Ae,
ifisk k ifi
(2)
where > 1 denotes the scaling factor. Finally, the enhanced text embeddings é UUVT where As visually demonstrated by the transformation from e to èt in Fig. 6(a), this scaling explicitly increases the embedding distances (dorgg - dgeate). directly enhancing semantic distinguishability among tokens. We further show that our method, Token Spacing, produces similar effect to amplifying the isotropy characteristics of the text semantic space discussed in $3, as detailed in the $D.2 with detailed explanations and additional analyses thereof.
E = diag(X1, Xx,..., An).
4.2 Residual Alignment
To mitigate potential side effects from variance scale-up ($3.4), such as semantic distortion or unintended amplification of spurious representations the embedding space, we introduce Residual Aligmment targeting the residual space beyond the principal dimensions, as illustrated in Fig. 6(b). Specifically, we rotate the residual subspace spanned by singular vectors V, [Uk+I....., v, using Givens rotation [39]. Given target semantic vector 8, defined the difference between the conditioned text embedding vector Gcund generated from a specific textual context and the unconditioned text embedding vector generated without context, we project it onto the residual subspace by removing its principal space component, as follows: Sprej = 8 Vpr V 8, where VP = [w,...., vy.. As shown in Fig. 6(b), we then perform a Givens rotation Go within the residual subspace to align the first residual singular vector Uk+1, the most significant direction in the residual

---

50 R2F
9030 Ours
FLU R2F
FLUK
Fey
schnel
Ours
FLUX
Property
hozned lenon
wigged pig
haizy Ezog
Shape
butterfly shaped airplane
shaped
Single Object
Texture
zebra striped tree
grape made mazble
A shrimp made of steel
Action
A niczobe walking with legs
smiling shoes
An eel driving car
Complex
11on dressed as eopoleon skiing in the desert
huran face crying with diamond haix
lour ained ninja with white hair in blue gt
Concat
A heart shaped moon and star shapod spaceship
A nustachiond ronkey and zobra striped zock
A bearded frog and two mustachised parzots
Relation
Multi Object
wooly shazk is swirming around banana shaped punpicin
A thorny snake is coiling azound staz shaped dzum
A dancing vase is sharing space with banana shaped
Complex
raboit in nedieval atrOz pajanas milling the
knighted turtle and wizarding dabating philosophy underwater librazy
hozned gizafle reading nesspaper while sitting hot flying over a city rade of giant mushroons
Figure 7: Qualitative comparison between SD3, FLUX-schnell, FLUXX-dvv, R2F [5], and our method on RareBench [5]. Results are organized by category; Red indicates baseline semantic failures.
subspace, with the projected semantic vector Sproj. Formally, the rotation matrix is defined as:
8pmj
Go
|proj
We note that Go is not arbitrary matrix, but rather deterministically constructed orthogonal matrix [39], designed to align the dominant residual component UĄ++1 with the semantic direction by the angle 0 between them. Finally, we rotate the residual subspace vectors accordingly as Vi V.s. The enhanced embedding is then reconstructed as: ē UÊ(V, VinsTT Details on Residual Alignment and Givens rotation, along with results addressing the side effects in Fig. 5, are provided in the $D.3.

---

Table 1: RareBench performance comparison across various models and categories in Image and Video generation. Darker cells indicate the best best-performing model in each column, while lighter cells indicate the second-best scores. GPT-4o was evaluated using the same seed setting as in R2F [5].
Modeb
Ragely
Stape
Coln
Sagg
Textasc
Coaok
Reizion
Complex
GPT4
Hassa
GPT4
Hamas
GPT4c
Hurrn
GPT4o
Bussar
GP74e
Hasras
GPP
Hassas
GPT4: T4o
Hursa
GPT4
Hassas
MAAUNT dicndiiet
FLAIN-der
71.9
63.3
50.7
0
70.3
63.8
12.3
67.5
68.5
08.4
51.9
68.7
60.0
15.3
70.0
67.8
68.
65.5
62.5
53.6
76.1
73.
R7F
89.4
75.6
19.8
75.
81.9
70.8
80.
78.0
12.5
T0.7
7U.0
66.0
58.8
90.1
73.8
65.2
MM- TORA
Tea
89.4
72-2
$1.6
85.2
$1.9
16.
$0.5
72-3
63.8
$4.6
K2.3
83.8
$4,5
85.9
93.2
4233
15.5
15.1
RIL5
77.1
604
75.0
65.5
76.9
08.1
60.5
61.7
00.6
24.4
76:8
7800
23.3
$2.9
63.9
9811
73.8
78.6
70.6
74.1
75.0
75.L
RiFacstatee Ouri
753
91.
714
78.0
125
73.6
67.6
69.0
15.1
74.5
74.3
75.1
$7.8
58.8
70.5
69.3
CigValerX-22
15.8
36.5
12.
46.
542
548
4b.4
44.1
51.9
54.5
N2.7
68.3
28.4
41.2
620
656
CogVabatX-st
4427
16.3
553
52.6
57.6
90.6
5822
49.6
34.8
56.3
66.5
60.9
38.6
39.6
68.9
64.9
05.
57.
10.5
72.
5 Experiments
Benchmarks. We evaluate our method primarily Text-to-Image generation, while demonstrating its versatility in Texeoto-Video generation and Text-driven Image Editing. For image generation, we use RareBench [5] for rare concept evaluation, and 221-ComBeench [40] and GenEval [41] for compositional tasks with common concepts. We further validate generalizability through video generation and image editing evaluations using RareBench.
Implementation details. We apply our method to joint- foint-attention-bal based MM-DiT models, evaluating text-to-image -mage tasks on Stable Diffusion 3.0 [4], FLUX-dev, and FLUX-schnell [3], and extending to CogVideoX (2B/5B) for text-to-video and Stable F2ow[28] for image editing. All hyperpa- rameters maintained at their default settings except for single scaling factor (a = 1.3). Further implementation details are provided in the $D.1.
5.1 Main Results
RareBench in text-to-image generation. Table shows that our method, ToRA, improves the performance baseline models [3-5] across categories in both GPT-4o [42] and human evaluations on RareBench [5]. Additionally, Fig. 7 provides quantitative visual comparisons of generated samples. Our model notably enables the successful emergence and alignment of rare semantics across numerous imaginative prompts. Furthermore, the improved results of R2F + Ours in Table emphasize model's simplicity and compatibility with existing methods. Furthermore, additional qualitative comparisons with GPT-4o [42] are provided in $H, with additional visualizations across various seeds and prompts in $F.
Compositional alignments in text-to-image generation. To assess the model's broader applicability. we extend our evaluation to 121-CohpBench [40] and GenEval [41], which focus on common prompts rather than rare concept cases. As Table 3 shows, the model enhanced with our method outperforms the baseline across categories on these benchmarks, mirroring its gains on RareBench. This confirms that our approach not only improves performance on rare text inputs but also preserves the intrinsic meaning of common prompts. Qualitative results are provided in the $F.
5.2 Generalizability Across Various Text-to- Vision Tasks
RareBench in text-to-video generation. To verify the effectiveness of our method across di- verse tasks, we extended our approach to text-to-video generation using the MM-DiT-based Cog Videix [27] and evaluated rare-prompt performance on RareBench [5] with GPT40-MT scores [43] and a human study (Table. 1). In most evaluation categories, our method improves performance by over 20%p, reflecting its effectiveness in enhancing semantic alignment across diverse visual As illustrated Fig. 8, our method enables the baseline model to effectively capture rare concepts from the prompts (e.g., "...fox playing piano..."), allowing these concepts to emerge clearly across temporal motion and action each frame of the generated videos. Additional results and ablation studies for video generation are provided in the SF.
8

---

Care Cogiobook
50X playing plano in desest at gclden houz, with distant golden zetsiever in Santa butfit descends the saals next to dust STDEn and dremtic, hypezresl Lighting.
house pezched on the clows
CagVabook
50 13061 in Rbadssan shirt walks
thouggh 3 zashy
Eusopean-
Taccoon 1n a trench cont zeading systery ROYEL UnGeZ BLOASRO
style street, holding
ratrbow unbrella
FUSTEDOR
Figure 8: Qualitative comparison of Text-to- Video generation results between the standard CogVideok [27] and Cog VideoX with our method applied, evaluated on custom rare prompts.
fish
Table 2: Performance compari- son between baseline and ours Image Editing.
Reference Image
a can made a cxystal
can weazing xed sweater
Metric
Stable Flow
+ Ours
CLIP img
0.87
0.80
CLIP
0.23
0.28
CLIPair dir
0.08
0.20
Human
50.8
87.7
Reference Image
GPT4o 62.1
82.8
Stable Flow
Ours
Stable Flow
Ours
Figure 9: Qualitative comparison of Text-driven Image Edit- ing results between the Stable Flow [28] and our method, evaluated custom rare prompts.
RareBench in text-driven image editing. We investigated the applicability of our method to text- driven image editing with Stable Flow [28]. Stable Flow edits images using reference images generated from simple prompts like "a photo of [OBJ]," where [OBJ] denotes objects from RareBench, allowing us to edit with rare concept prompts. As shown in Fig. 9, our approach helps the model generate rare semantics. Edited images were evaluated using three following Stable Flow's 's evaluation framework. In Table 2, our method improved upon the baselines across all evaluation metrics. Notably, the CLIP img. which compares reference and generated images, yielded relatively lower scores due to significant visual changes natural to RareBench prompts. See the FF for detailed evaluation metries, extended results, and ablations image editing.
metrics, GPT-4o, and human study,
5.3 Ablation Studies
Effects of a. To assess the impact of the token spacing parameter we varied from 1.0 to 1.5 and measured RareBench scores each setting. As shown Fig. 10(a), a = 1.3 yields the highest performance across all categories, whereas both lower and higher values produce lower scores. To understand this behavior, we visualized the attention maps in Fig. 10(c) and found that, at o = 1.3. the model consistently concentrates the correct regions corresponding attributes like "bearded" and "spotted," demonstrating that this range best facilitates fine- tine-grained semantic alignment. See the $F, for additional results of various values.
Effects of residual alignment. In Fig. 10(b), we evaluated Residual Alignment, showing improved performance across all categories when combined with Token Spacing compared to Token Spacing alone. Additionally, Fig. 10(c) indicates that 'w/o Residual Alignment' results in attention maps with appropriately positioned yet weak attention. insufficient for the effective emergence of desired semantic elements in the images. Therefore, Residual Alignment is identified as a erucial technique for guiding embeddings toward semantic directions discussed in $3.4.

---

Table 3: Comparison results broader applicability in Text-to-Image generation using GenEval [41] and T21- 22--ComBBench [40]. Darker cells indicate best scores; Lighter cells are second-best.
Gostrall
Middl
Stngle
Cuumtigg
Poesion
Biadings
CPTE
Sponial Usitet
Sparal
Coepphen
S03.
0.56
0.34
0.63
0.67
0.34
0.35
10.3
84.5
76.2
63.3
91.3
50.1
12.0
34.0
58.5
31.4
85.2
47.7
FLUX-sclael
0.98
0.77
0.70
0.63
0.18
0.29
88.7
82.5
71.2
59.4
90.0
73.1
73.0
35.0
83.7
31.7
43.4
45.5
PLAIN-dre
0.81
0.76
0.79
0.22
91.7
90.0
16.5
12.1
95.1
R2F
0.76
0.65
06k
0.36
90.5
45.6
891
32.5
25.3
47.9
01.90
0.27
0.59
061
0.20
0.45
67.4
$2.5
70.9
58.5
96.4
18.7
74.7
41.4
90.2
12.0
82.0
45.4
MIR MRC of TORA
Oupaame
0.98
0.27
0.66
0.70
0.50
06
90.5
94.5
72.5
66.1
91.4
82.0
77.3
45.5
68.9
33.8
86.0
48.1
0.99
0384
088
0.7
02.3
84.6
78.9
46.0
35.3
369
Meriin Haliic sencen May
(a)
(b)
(c)
brazdoo
Spotted shinn
MAY
Figure 10: Ablation studies RareBench [5]: (a) Token Spacing with varying a; (b) impact of Residual Alignment; (c) qualitative results and corresponding attention maps.
6 Related Work
Transformer in diffusion models. Recent advancements in diffusion models have brought to the forefront transformer-basee based architectures like Diffusion Transformer (DiT) [29], especially Multi-modal Diffusion Transformers (MM-DiTs). MM-DiTs integrate text and visual information through joint-attention mechanisms, significantly enhancing semantic alignment between prompts and generated visuals. In particular, FL.UX [3] and SD3.0 [4] now represent state-of-the-ar performance, surpassing previous UNet-based approaches [1]. Despite these advances, accurately visualizing complex textual descriptions remains challenging, leading researchers to explore both finetuning methods [6-10] and training-free techniques [5. 10, 11] for improved semantic coherence. Our work shares similarities with these efforts, focusing on pretrained MM-DiT models through simple interventions that require neither structural modifications nor retraining.
Improving text embeddings in diffusion models. A fair amount of practical work has explored various strategies to improve text embeddings in UNet-based diffusion models, underpinning better semantic fidelity, compositional accuracy, and controllability. Ahmed and Mittal [44] and Zarei al. [45] have demonstrated improved prompt fidelity and attribute binding through finetuning embeddings. Other recent studies [5, 46-48] have integrated L.LMs, garnering deeper semantic insights and substantially improving complex prompt understanding. Moreover, other approaches have focused on inference-time interventions, such as regulating cross-attention dynamics [49-51], to better reflect prompt semantics. Beyond common prompts, our work investigates how text embedding variance scale-up up influences semantic alignment for imaginative and rare prompts, focusing on the relatively underexplored MM-DiT architecture.
7 Discussion and Conclusion
In this paper, we have presented ToRA, a simple yet effective intervention for improving MM-DiT models for vision generation tasks of rare text semantics. In particular, we investigated whether text embeddings can naturally emerge their specific meanings in MM-DiT and whether models can effectively apply these semantics to vision generation. We showed ToRA can effectively elicit this semantic emergence, leading to improved generation performance. Through extensive evaluation, we confirmed our method's generalizability across text-to-image, text-to-video, and texi-driven image editing lasks. Lastly, we hope our findings help shift the focus from imposing external semantic constraints to naturally revealing the embedded semantics already there within MM-DiT; We provide more discussion in the SG.
10

---

Acknowledgements
This work was supported in part by the IITP RS-2024-00455882 (AI Research Hub Project), IITP
2020-11201361, NRF RSS2022--00345866, NRF RS-2023-00219019, and NRF RS-2023-002620.

---

References
[1] Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. High- resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684-10695, 2022.
[2] Alex Nichol, Prafulla Dhariwal, Aditya Ramesh, Pranav Shyam, Pamela Mishkin, Bob McGrew, Iya Sutskever, and Mark Chen. Glide: Towards photorealistic image generation and editing with text-guided diffusion models. arXiv preprint arwiv::112..000000 2021.
[3] Black Forest Labs. Flux. https://githie cow/black forest labs/flux, 2024.
[4] Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas Müller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, al. Scaling rectified flow trans- formers for high-resolution image synthesis. In Forty-first international conference machine learning, 2024.
[5] Dongmin Park, Sebin Kim, Tachong Moon, Minkyu Kim, Kangwook Lee, and Jaewoong Cho. Rare-to- foeeuuuett:: Unlocking compositional generation power of diffusion models on rare concepts with Ilm guidance. arXiv preprint arXiv:2410.22376, 2024.
[6] Jingfeng Yao, Cheng Wang, Wenyu Liu, and Xinggang Wang. Fasterdit: Towards faster diffusion transformers training without architecture modification. Advances in Neural Information Processing Systems, 97:56166-5611 2024.
[7] Gongfan Fang, Xinyin Ma, and Xinchao Wang. Remix-dit: Mixing diffusion transformers for multi-expert denoising. Advances in Neural Information Processing Systems, 37:107494- 107512, 2024.
[8] Zhen Han, Zeyinzi Jiang, Yulin Pan, Jingfeng Zhang, Chaojie Mao, Chenwei Xie, Yu Liu, and Jingren Zhou. Ace: All-round creator and editor following instructions via diffusion transformer. arXiv preprint arXiv:2410.00088 2024.
[9] Wangbo Zhao, Yizeng Han, Jiasheng Tang, Kai Wang. Yibing Song, Gao Huang, Fan Wang, and Yang You. Dynamic diffusion transformer. arXiv preprint arXiv:2410.03345,, 2024.
[10] Chang Zou, Xuyang Liu, Ting Liu, Siteng Huang, and Linfeng Zhang. Accelerating diffusion transformers with token-wise feature caching. arXiv preprini 10.0::2410.05317. 2024.
[11] Yuseung Lee, Tachoon Yoon, and Minhyuk Sung. Groundit: Grounding diffusion transformers via noisy patch transplantation. Advances in Neural Information Processing Systems, 37: 36610-58636 2024.
[12] Shoufa Chen, Mengmeng Xu, Jiawei Ren, Yuren Cong, Sen He, Yanping Xie, Animesh Sinha, Ping Luo, Tao Xiang, and Juan-Manuel Perez-Rua. Gentron: Diffusion transformers for image and video generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 6441-6451, 2024.
[13] Kawin Ethayarajh. How contextual are contextualized word representations? comparing the geometry of bert, elmo, and gpt-2 embeddings. Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP,, pages 55 65, 2019.
[14] Jiaqi Mu, Suma Bhat, and Pramod Viswanath. All-but-the-top: Simple and effective postpro- cessing for word representations. arXiv preprint 170.v:/70 2017.
[15] Xingyu Cai, Jiaji Huang, Yuchen Bian, and Kenneth Church. Isotropy in the contextual embed- ding space: Clusters and manifolds. In International conference on learning representations, 2021.
[16] Sara Rajaee and Mohammad Taher Pilehvar. A cluster-based approach for improving isotropy in contextual embedding space. In Proceedings of the 59th Annuul Meeting of the Association for Computational Linguistics and the 1 11th International Joint Conference on Natural Language Processing (Volume 2: Short Papers), pages 575-584, 2021.
12

---

[17] Jacob Devlin, Ming- wei Chang, Kenton Lee. and Kristina Toutanova. Bert: Pre-trainin tre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies, volume (long and short papers), pages 4171-41866 2019.
[18] Justyna Sarzynska- Aleksander Wawer, Aleksandra Pawlak, Julia Szymanowska, Izabela Stefaniak, Michal Jarkiewicz, and Lukasz Okruszek. Detecting formal thought disorder by deep contextualized word representations. Psychiatry Research, 304: 14135, 2021.
[19] Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, al. Language models are unsupervised multitask learners.
[20] Zhanxing Zhu, Jingfeng Wu, Bing Yu, Lei Wu, and Jinwen Ma. The anisotropic noise in stochastic gradient descent: Its behavior of escaping from sharp minima and regularization effects. In International Conference on Machine Learning, pages 7654-7663. PMLR, 2019.
[21] Alessio Ansuini, Alessandro Laio, Jakob H Macke, and Davide Zoccolan. Intrinsic dimension of data representations in deep neural networks. Advances in Neural Information Processing Systems, 32, 2019.
[22] Stefano Recanatesi, Matthew Farrell, Madhu Advani, Timothy Moore, Guillaume Lajoie, and Eric Shea-Brown. Dimensionality compression and expansion in deep neural networks. arXiv preprint 19%6v:1906.00443, 2019.
[23] William Rudman and Carsten Eickhoff. Stable anisotropic regularization. In The Twelfth International Conference on Learning Representations.
[24] Hyungjin Chung, Jeongsol Kim, Geon Yeong Park, Hyelin Nam, and Jong Chul Ye. Cfg++: Manifold-constrained classifier free guidance for diffusion models. arXiv preprint arXiv:2406.08077 080000 2024.
[25] Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance. arXiv preprint arXiv:2207.12598 2022.
[26] Tero Karras, Miia Aittala, Timo Aila, and Samuli Laine. Elucidating the design space of diffusion-based generative models. Advances in neural information processing systems, 35: 26565-26577, 2022.
[27] Zhuoyi Yang. Jiayan Teng, Wendi Zheng, Ming Ding, Shiyu Huang, Jiazheng Xu, Yuanming Yang. Wenyi Hong, Xiaohan Zhang, Guanyu Feng, et al. Cogvideox: Text-to-video diffusion models with expert transformer. arXiv preprint 2xXi:22408.0607. 2024.
[28] Omri Avrahami, Or Patashnik, Ohad Fried, Egor Nemchinov, Kfir Aberman, Dani Lischinski, and Daniel Cohen-Or. Stable flow: Vital layers for training-fiee free image editing. arXiv preprint arXiv:2411.1443 14430, 2024.
[29] William Peebles and Saining Xie. Scalable diffusion models with transformers. In Proceddings of the IEEE/CVF international conference on computer vision, pages 4195-4205, 2023.
[30] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pages 9748-8763. PmLR, 2021.
[31] Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Yunxuan Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, et al. Scaling instruction-finetune language models. Journal of Machine Learning Research, 25(70): 2024.
[32] Xin Jin and Jiawei Han. K-Means Clustering, pages 563-564. Springer US, 2010.
[33] Martin Ester, Hans-Peter Kriegel, Jorg Sander, Xiaowei Xu, et al. A density- based algorithm for discovering clusters in large spatial databases with noise. In kdd, volume 96, pages 226-231, 1996.
13

---

[34] and others. Gaussian mixture models.
[35] Tianyu Gao, Xingcheng Yao, and Danqi Chen. Simese: Simple contrastive learning of sentence embeddings. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 6894-6910, 2021.
[36] Bohan Li, Hao Zhou, Junxian He, Mingxuan Wang, Yiming Yang, and Lei Li. On the sentence embeddings from pre-trained language models. In Proceedings of the 2020 Conference Empirical Methods in Natural Language Processing (EMNLP), pages 9119-9130, 2020.
[37] Andrzej Maćkiewicz and Waldemar Ratajczak. Principal components analysis (pca). Computers Geosciences, 19(3):303-344 342, 1993.
[38] David H. Douglas and Thomas K. Peucker. Algorithms for the reduction of the number of points required to represent a digitized line its caricature. Cartographica: The International Journal for Geographic Information and Geovisualization, 10:1 10:112-122, 1973.
[39] Wallace Givens. Computation of plane unitary rotations transforming general matrix to triangular form. Journal of the Society for Industrial and Applied Mathematics, 6:26-50, 1958. ISSN 03684245. URL http: / /ww. http://ww..srrrrroo/o....o . org/ wwt.aleeee988e666
[40] Kaiyi Huang, Kaiyue Sun, Enze Xie, Zhenguo Li, and Xihui Liu. T2icoompeenh: A compre- hensive benchmark for open-world compositional text-to-image generation. Advances in Neural Information Processing Systems, 36:78723-78747, 2023.
[41] Dhruba Ghosh, Hannanch Hajishirzi, and Ludwig Schmidt. Geneval: An object-focused framework for evaluating text-to-image alignment. Advances in Neural Information Processing 2023.
Systems,
[42] Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, et al. Gpt-4o system card. arXiv preprint arXiv:2410.21276, 2024.
[43] Shenghai Yuan, Jinfa Huang, Yongqi Xu, Yaoyang Liu, Shaofeng Zhang, Yujun Shi, Ruijie Zhu, Xinhua Cheng, Jiebo Luo, and Li Yuan. Chronomagic-bench: A benchmark for metamorphic evaluation of text-to-time-lapse video generation. arXiv preprint arXiv:2406..883 2024.
[44] Yeruru Asrar Ahmed and Anurag Mittal. End-to-end training for text-to-image synthesis using dual-text embeddings. arXiv preprint S50i::2502.010000 2025.
[45] Arman Zarei, Keivan Rezaci, Samyadeep Basu, Mehrdad Saberi, Mazda Moayeri, Priyatham Kattakinda, and Soheil Feizi. Understanding and mitigating compositional issues in text-to- image generative models. arXiv preprint arXiv:2406.00 078444 2024.
[46] Shanshan Zhong, Zhongzhan Huang, Weushao Wen, Jinghui Qin, and Liang Lin. Sur-adapter: Enhancing text-to-image pre-trained diffusion models with large language models. In Proceed- ings of the 31st ACM International Conference on Multimedia, pages 567-578, 2023.
[47] Mushui Liu, Yuhang Ma, Zhen Yang, Jun Dan, Yunlong Yu, Zeng Zhao, Zhipeng Hu, Bai Liu, and Changjie Fan. Llm4gen: Leveraging semantic representation of 1lms for -ext-to-image generation. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pages 5523-5531, 2025.
[48] Xiwei Hu, Rui Wang, Yixiao Fang, Bin Fu, Pei Cheng, and Gang Yu. Ella: Equip diffusion models with I1m for enhanced semantic alignment. arXiv preprint arXiv:2403.05135, 2024.
[49] Hila Chefer, Yuval Alaluf, Yael Vinker, Lior Wolf, and Daniel Cohen-Or. Attend-and-exoii Attention-based semantic guidance for text-to-image diffusion models. ACM transactions on Graphics (TOG), 42(4):1-10, 2023.
[50] Royi Rassin, Eran Hirsch, Daniel Glickman, Shauli Ravfogel, Yoav Goldberg, and Gal Chechik. Linguistic binding in diffusion models: Enhancing attribute correspondence through attention map alignment. Advances in Neural Information Processing Systems, 36:3536-3559, 2023.
14

---

[51] Yang Zhang, Teoh Tze Tzun, Lim Wei Hern, and Kenji Kawaguchi. Enhancing semantic fidelity text-to-image synthesis: Attention regulation in diffusion models. In European Conference on Computer Vision, pages 70-86 Springer, 2024.
[52] Alec Helbling, Tuna Han Salih Meral, Ben Hoover, Pinar Yanardag, and Duen Horng Chau. Conceptattention: Diffusion transformers learn highly interpretable features. arXiv preprint W0X1v:2502.043220 2025.
[53] Sihyun Yu, Sangkyung Kwak, Huiwon Jang, Jongheon Jeong, Jonathan Huang, Jinwoo Shin, and Saining Xie. Representation alignment for generation: Training diffusion transformers is easier than you think. arXiv preprint arXiv:2410006940 2024.
[54] Luping Liu, Chao Du, Tianyu Pang, Zehan Wang, Chongxuan Li, and Dong Xu. Improving long-text alignment for text-to-image diffusion models. arXiv preprint 2rXi:22400.18177, 2024.
15

---

Technical Appendices
Additional Material: Media Gallery
17
The Join-Attention in Multi-Modal Diffusion Transformers
17
Analyses Details
18
C.I Setup Details
18
C.2 Further Discussion for Local Isotropy
18
C.3 Further Discussion for Global Anisotropy
20
C,4 Pitfalls of Variance Scale-Up via Semantic Vector Analysis
................................
21
D Methods Details
22
D.1 Implementation Details and GPU Time/Memory Analysis
22
D.2 Effects of Token Spacing for Local Isotropy
23
D.3 Effects of Residual Alignment for Mitigating the Side Effects from Token Spacing
23
Evaluation Details
24
E.1 Human Study
24
E.2
GPT-4o Evaluation
24
E.3
Evaluation Prompts for Each Benchmarks
26
F
Additional Experiement Results
26
F.1
Additional Ablation Studies
26
F22 More qualitative results
29
G Limitations and Further Discussions
29
G.I Limitations and Future Works
29
G.2 Further Discussions
H Qualitative Comparison with GPT-40
30
16

---

A Additional Material: Media Gallery
As displaying video content frame by frame within the paper offers only limited insight into temporal coherence and visual quality, we provide a Media Gallery Page featuring the full video outputs from both the main and additional experiments. This page allows for a more faithful assessment of motion consistency and prompt alignment. The results can be viewed at: https: hwtp:///ssrsi225-111511100 1573. github. io/
B
The Join-Attention in Multi-Modal Diffusion Transformers
Multi-modal Diffusion Transformer (MM-DiT) extends Diffusion Transformer (DiT) to the multi- modal text-to-vision setting, jointly processing textual and visual representations. Conditioning text embeddings come from CLIP (L/14, G/14) [30] and T5-XXL [31] in Stable Diffusion 3 (SD3) [4], and from XS-XXL alone in FLUX.1 [3]. The initial text embedding inii E RVxd, where denotes the number of text tokens, and the latent-noise embedding xi RNxxd where N represents the number image tokens, are processed through B joint- attention blocks.
For every block b E {1,2,.... .... B}. the text condition e²-1 and latent noise -²-11 are updated. These dual- stralms soint-attention blocks (also referred to as multi-modal attention layers or MMATTNs in some literature [52]) are designed to keep the image and text modalities in separate residual streams while allowing for interaction. The process within each block can be detailed as follows:
1. Adaptive Layer Normalization (AdaLN); The input image patch representations x Nxd and prompt token embeddings e E R RVV are first processed by adaptive layer norm (AdaLN) operations. As described by Peebles and Xie [29] for DiTs, these AdaLN layers are conditioned on the diffusion time-step and global CLIP vector embedding. The AdaLN operation effectively applies LayerNorm and then modulates the normalized em- beddings using learned affine transformation parameters (scales and shifts 3, though some- times only scales are used) derived from t and the CLIP vector. Let h = AdaLN(e"-)) and h 1 Adal.N((*") ((x"_) be the modulated embeddings. These y, B parameters (or just 7) are also used to scale the residual connections later.
2. Query, Key, and Value Projections: Separate learned projection matrices are used for text and image modalities generate queries (Q), keys (K), and values (V). These applied to the modulated embeddings from the AdaLN step:
Q D h W
g.
K b-1
# 1/)
Wh
k
Vb-1 h
h-1 --b
for each modality μ E {txt, img]. Here, W quur specific projection weights at block
whb
Rajo W
E Rddd are the modality-
3. Joint Attention Mechanism: The core of the block the attention operation. The attention outputs for text (oe) and image (0x) are computed as:
Q "ar'(K' (K--1)" img
Qua (Kkk1)
O softmax
softmax
(4)
Vd
Vd
Q Q (I
QQi=1 (KK b-
O2 = softmax
softmax
(5)
Vd
Vd
where the [ operation indicates that the attention scores (maps) from cross-modal attention (e.g.. text queries attending to image keys) and self-modal attention (e.g., text queries attending to text keys) are combined (e.g., concatenated or summed) before being applied to the values of the query's own modality.
4. Output Linear Layer and Residual Connection: The attention outputs Og and are then passed through another linear projection layer. These projected outputs are then added back to the original input embeddings of the block (before AdaLN, i.e., e and x =-11),
17

---

scaled by a factor (e.g., Yew Mm derived from the AdaLN modulation parameters, thus completing the residual path for this attention stage:
e e" h-1 + Tha Linear (oe)
x x h-1 Ving Linear
It's important to note that full joint-attention block, similar to standard Transformer blocks, would typically also include a position-wise Feed-Forward Network (FFN) or MI.P, itself conditioned via AdaLN and with its own residual connection, for both the text and image streams after the attention mechanism described above. The description provided focuses on the multi-modal attention aspects.
This dynamic embedding evolution distinctly differentiates MM-DiT from traditional UNet-based diffusion models. These joint- coint-attention blocks operate sequentially across B layers at each timestep for latent noise prediction.
C Analyses Details
C.1 Setup Details
Rare Prompts (70)
Common Prompts (30)
Property
Single Object
Shape
Categories
Texture
Action
GPT-4o Generated Rare Prompts
GPT-4o Generated Common Prompts
Single Complex
Multi Object
Concat
Relation
Complex
RareBench (40) (5 prompps / 1 category)
Created by GPT-4o (60)
strictly under RareBench's Rules
Figure 11: Distribution diagram of prompt samples used in the analysis.
As stated in the main paper, we retain the default settings provided by the baseline models [3, 4, 27, 28] and only adjust the values derived from our proposed method through ablation studies. In our analysis. we utilize both prompts from the original RareBench [5] and additional prompts generated using GPT-4o [42].
In total, our analysis considers 100 text prompts. 40 prompts come directly from RareBench's 's eight benchmark categories (five per category) [5], and the remaining 60 prompts generated with GPT-4o [42], comprising 30 rare prompts, created strictly under RareBench [5]'s rarity guidelines, and 30 common prompts added concise overview of the entire prompt set appears in Fig. 11. While the main manuscript reports results with random seed 42 for reproducibility, following the evaluation protocol of R2F [5], the present analysis probes robustness by sampling additional seeds other than 42, thereby capturing stochastic variability that could influence generation quality.
bias check to ensure that our analysis is not limited to cases.
Due to the intrinsic characteristics of the MM-DiT architecture, the initial text embeddings are consistently fed into the first block of each timestep. Additionally, we observed that embedding patterns across the 24 blocks within each timestep exhibit strong similaritics from one timestep to the next. Based on this insight, our analyses primarily focus on the behavior observed the block level.
C.2 Further Discussion for Local Isotropy
To rigorously verify the isotropic properties of the text semantic space discussed in Section 2.3, we experimentally analyze how variance scale-up impacts isotropy using the IsoScore metric proposed by Rudman and Fickhoff [23], noting that this metric does not locally measure isotmpy. Additionally, beyond the examples presented in Fig. 3, provide supplementary examples consisting of generated
18

---

icaloup Fo scdwep
icalosa
1.0
0.8
of Eigenvalues
IsoScore
0.6
0.4
Block Index
Block
Figure 12:
Results of measuring isotropy based on the IsoScore [23] metric with variance scaling.
Left: A comparative plot the sum of eigenvalues for the original and variance-scaled data. Right: comparison of IsoScore values before and after variance scaling.
images and their corresponding text self-attention maps to further assess the generality of our approach across diverse prompts.
Another metric measure isotropy: IsoScore [23]. While our analysis Section 2.3 verified the isotropic properties of textual semantics by calculating local isotropy through embedding clustering, Rudman and Eickhoff [23] provides a complementary global perspective by introducing IsoScore, an isotropy metric computed without relying clustering cosine similarity-bascd methods. Specifically, given a text token embeddings represented by the matrix e E VVd with sample mean IsoScore is calculated as follows:
First, embeddings are projected onto their first principal components to obtain PCA E RVk and the covariance diagonal Ep is computed:
Ep = Vk- diag(Cor (EA)))
(6)
||diag Cov( [PCA))]]]
where ||. is the Euclidean norm. We determine the number of principal components & for each block using the Maximum Distance to Chord (MDC) method [38], described in Section 3.1. Then, isotropy defect 5(e) and dimension occupancy ((e) are defined as:
DED - 1|| 22(1: 2(k: vk)
5(e)
where 1 = 1I, E E
(pe) = (k: - 5(e) vR))²
(8)
22
Finally, the metric is rescaled to range between 0 and 1, defining IsoScore" as:
k (p() -1
EssnSScoee (e)
(9)
k
Here, EisoScore E [0, 1]. with EisoScore = -1 indicating perfect isotropy (uniform distribution across all principal directions) and values near 0 indicating high anisotropy.
Fig. 12 shows the block-wise IsoScore results computed from MM-DiTs using the procedure de- scribed above. The results with variance scale-up exhibit higher isotropy across blocks compared to the original and scale-down. This aligns with our findings using other isotropy metries, confirming that variance scale-up improves the isotropic properties of the text semantic space.
More analysis results for local isotropy. In Fig. 3 of Section 2.2, we observed that variance scale-up in MM-DiT's facilitates rare semantic emergence for rare prompts- phenomenon not occurring under the original configuration. Additionally, variance scale-up intensifies activation among text
19

---

tokens, as reflected in the text self-attention maps, thereby enhancing inter-token relationships. Fig. 18 provides supplementary results supporting this observation, presenting additional examples and corresponding text self-attention maps from SD 3.0 [4] and FLUU [3], beyond those included in the main manuscript.
C.3 Further Discussion for Global Anisotropy
In the main manuscript we demonstrated, from a global standpoint, that the text-semantic space of MM-DiT exhibits pronounced block-wise anisotropy. Empirically, this anisotropy does not impair the fidelity of the generated images; rather, an overly aggressive attempt to eliminate it produces almost entirely noisy outputs (see Section 2.4 in the main paper). In this section, we present mathematical derivation of the global anisotropy reeuuttopy-ruutiionn procedure adopted in our study, namely the proseernnssingg technique proposed by Mu et [14]. We further examine related studies[20- 23] to explain, from a global perspective, why text semantic space's anisotropic representations arise naturally in transformer architectures and how moderate regularization, instead of wholesale suppression, preserves the useful structure of the embedding space.
Reducing global anisotropy via principal component removal. Mu et al. [14] identify pronounced global anisotropy in standard word-embedding spaces, showing that a shared mean vector and a few nigh-variance principal directions dominate the geometry. They propose a lightweight processogs step, mean subtraction followed by removal of the top principal components, that markedly restores isotropy and enhances downstream performance. As outlined earlier in our main manuscript, we apply the same strategy to MM-DiT; the detailed mathematical derivation used in our implementation presented below.
Let the original embedding matrix be -ee
TTp BRIixd
M
1
ē
Vi:
é; = ē.
(10)
i=1
1 VI
C
V Σ
c = UAUT,
(11)
where U = jui..... us holds cigenvectors in descending order of eigenvalue. Choosing D = [d/100], we form Up= [u....., up] and project out the dominant subspace:
e é; UD
(12)
Following Mu et al. [14]. we use D-15 when 1-1536 in Stable Diffusion 3.0 [4]. The resulting block- wise anisotropy scores and corresponding generations shown in Fig. 4 of our main manuscript, highlighted in yellow.
Global anisotropy revisited: Extended discussion and related work. In this section, we further explore the nature of anisotropy within the text semantic space of MM-DIT, building upon long- standing discussions in the Natural Language Processing (NLP) community.
Intuition might suggest that anisotropy. where text embeddings are predominantly distributed across a few key dimensions, could negatively impact downstream tasks. However, our experiments, detailed in Section 2.4 of the main manuscript, reveal its harmlessness concerning the semantic emergence effect we proposed. We aim to provide more comprehensive explanation, drawing upon related works [20-23], of why this characteristic, particularly in transformer models, can be a natural global phenomenon that doesn't 't necessarily correlate with downstream model performance.
The Implication of Global Anisotropy Neural Networks; Contrary to certain traditional NLP literature [13, 14], global anisotropy in text semantic spaces is not inherently detrimental to down- stream task performance. Instead, it increasingly recognized as an intrinsic property of transformer architectures and a direct consequence of stochastic gradient descent (SGD) optimization [20]. This perspective posits that anisotropy, arising from SGD, facilitates the model's convergence to flat minima in the loss landscape, which is known to promote superior generalization compared to sharp minima [20, 23]. Furthermore, extensive research indicates that neural networks inherently compress data into swwer-dimensionll manifolds [22, 23]. These compressed representations often lead to
20

---

(b) Distribution of 4
(a) Distribution of A
after Residual Alignment
0.14
1.0
0.14
1.0
Curulaative line (CDF)
Curnulative line (CDF)
0.12
0.8
0.8
0.08
06
0.08
0.5
0.06
04
0.06
0.4
02
0.2
0.02
0.02
0.00
0.00
0.0
Ay
Ar
Figure 13: Pitfalls of variance scale-up as seen through Ay in additional examples (random sampled 50 prompts from Fig. 1I). (a) Analysis of extended samples: It presents results from 50 randomly selected samples used in our analysis. The distribution illustrates the average frequency value for each sample, with error bars indicating the standard deviation. (b) Impact of residual alignment on side effects: We examine the change in Ay values across samples after applying Residual Alignment, demonstrating its effect on mitigating side effects.
enhanced performance downstream tasks. Notably, a lower intrinsic dimensionality (ID) in the final layers has been identified as a robust predictor of classification accuracy on test data [21]. This perspective directly contrasts with earlier claims suggesting that increasing global isotropy (decreasing global anisotropy) improves model representations, potentially at the cost of perfor- mance degradation [23]. Conversely, growing body of evidence suggests that heightened anisotropy, through this very compression into lower-dimensional manifolds, can lead to performance gains [20].
Local Isotropy and Generalization: Several studies [20, 23] corroborate these observations: SGD intrinsically introduces anisotropic noise, which aids in escaping sharp minima and converging to flatter, generalizable minima. Deep neural networks progressively compress data representations into manifolds progressively lower intrinsic dimensionality in later layers. This dimensional compression in later layers strongly correlates with improved generalization performance [20]. The low-dimensional representations learned by neural networks are thus believed to prevent overfitting and facilitate generalization by effectively discarding task-irrelevant dimensions. These discrepancies collectively motivated our investigation into local isotropy to accurately characterize the text semantic space.
Our Conclusion; In conclusion, the evidence strongly suggests two key points. First, global anisotropy naturally emerges from neural network training. Second, the compression of representations into lower intrinsic dimensions intrinsically linked to enhanced generalization performance. These findings hold true across general neural network learning and classification tasks, supported by existing literature [20-23]. Lastly, we find that these insights extend naturally to flow- and diffusion- based generative models for text-to-vision tasks that utilize natural language input.
C.4 Pitfalls of Variance Scale-Up via Semantic Vector Analysis
In Scction 2.5 our main manuscript, we argued that, although variance scale-up demonstrably benefits semantic emergence, it does not invariably producce positive results for every text prompt random seed. Here, we provide a mathematical derivation that clarifies this limitation and present additional experimental examples beyond Fig. in the main paper.
Mathematical derivations. We restate the key definitions from the main paper before delving into new analyses. We first introduce the semantic vector: Econd
the difference between the conditional text embedding econd and its unconditional (null) counterpart eg used in classifier- free guidance (CFG). Let be the original text embedding and é its variance scale-up version. The cosine difference quantifies the effect of variance scale-up on semantic alignment: Ay = cos( e) cos(s, e). A positive value of Ay indicates that variance scale-up enhances alignment
21

---

with the semantic direction 8. whereas negative value signals degradation. Ay can be rewritten as
8.é
é.é. ||e|| e
||ël|
A cos é) cos(s, e)
(13)
||||| |||||
|||
|le|
||s
|e|
|||||
Because the denominator is strictly positive, the sign of Ay is governed solely by the numerator
è ||e|| e e-
Now let the comparison vectors share common component e, the mean of e, and differ in a residual direction a e-é_. writing é = e + ou and e = é + u with scaling factor a > O.
Var(e)
Substituting these forms gives the inner products
é. a(s u) and e Keeping the norms symbolic, ||ē|| and ||e|| depend on é. u, and a but remain positive, we obtain
é ||e|||
ellè|l
ē + as w)lle| ē 8 uu]lē|
(s E)(eeel ||êl]) + (s u) - -(el))
(14)
Hence,
Sgg(A>)) sign((s ( [|ě|) (s u)r -((]))
(15)
In words, whether co,(s))) è) exceeds cos(s, e) is determined by weighted gaps: the shared- component inner product e multiplied by the difference of norms, and the residual inner product multiplied by the scaled norm gap. Plugging explicit formulas for the norms, if desired, turns this boxed condition into concrete inequalities for Ay > 0 or An < O. This sign rule shows that scaling the residual (a) improves alignment (Ay 0) when more aligned with the residual direction u, and worsens it (Ay < O) when is closer to the mean component é. Consequently, as we mentioned in the main manuscript, even though variance scale-up often helps in broad sense, does not automatically improve alignment along the semantic direction mathematically. Ay can still turn negative, so a semantic gain is not guaranteed.
More analysis results. Fig. 13(a) extends the analysis of the Ay distribution for text embeddings, originally presented in Fig. 5 of the main manuscript, to a larger sample set (50 prompts and 100 random seed per prompt). When Ay < 0, the average CLIP score registered at CLIP = 0.16
whereas for Ay 0, it consistently reached CLIP text 0.45 + 0.1.
D Methods Details
In this section, we provide an analysis of our proposed method, Token Spacing and Residual Alignment. Specifically, investigate how isotropy, previously analyzed in the main paper in Section 2, is affected by our proposed approach. We further analyze how Residual Alignment mitigates potential side effects arising from Token Spacing, quantifying its effectiveness in enhancing semantic alignment within the embedding space.
D.1 Implementation Details and GPU Time/Memory Analysis
Our experiments and results derivation were based on Stable Diffusion 3.0 [4] and FLUX. I [3], both publicly available on HuggingFace We reproduced the results of R2F [5] using their publicly accessi- ble code from their official GitHub repository. It is worth noting that R2F exclusively supports SD 3.0 and FLUX-schned within the MM-DiT model family, and does not provide support for FLUX-des Our computational resources included a single NVIDIA 48GB A6000 GPU for general experiments and results, with a single NVIDIA 80GB A 1000 GPU dedicated Text-to- Video generation.
Table 4: Evaluation of GPU performance across diverse methods for text-to-image generation with Stable Diffusion 3.0.
Models
SD 3.0
R2F [5]
Ours
Peak Memory (GB)
22.02
38.49
22.17 44.74 + 1.9
GPU Time (sec)
21.30 1.1
78.13 = 2.36
22

---

(b)
se3 tolan apecing
505 oijinal
1.0
wdil boown speciig
500 gingial
Sum of Eigenvalues
Fux orginal
Local Isotropy Score
Bux ooooriiinlll
trom spectig
Rus toomn spacing
0.8
2-
Block Index
Block Index
Figure 14: We analyzed the geometric properties of the text embedding space after applying Token Spacing. (a) The left plot shows the sum of eigenvalues for the original text embeddings compared to those with Token Spacing applied. (b) The right plot illustrates the local isotropy score.
Beyond this, for practical demonstration of algorithms' performance, analyzed their GPU time and memory usage on an NVIDIA 48GB A6000 GPU, conducting ex- periments ourselves. We measured the resources needed to generate the complex prompt "a horned lion and a wigged e elephant", which features two rare concepts, consistent with the experimental setup in Park et al. [5]. For each measurement, performed 100 trials using different random seeds for the same prompt and report the average values and their standard deviations in Table. 4. These results are limited to the diffusion sampling steps.
D.2 Effects of Token Spacing for Local Isotropy
We analyze our method, Token Spacing, focusing particularly its influence on local isotropy and inter-token relationships. We demonstrate that increasing the distances between embeddings enhances the local isotropy, effectively improving semantic emergence. Here, investigate whether Token Spacing yields comparable benefits. To briefly summarize Token Spacing, we first perform Principal Component Analysis (PCA) the embedding space, decomposing it into principal and residual spaces. Token Spacing then expands the distances between token embeddings within the principal space, effectively amplifying their variance along principal directions. While this approach conceptu- ally parallels a simple variance scaling, our goal is to specifically examine whether Token Spacing also improves local isotropy, an important factor associated with clearer semantic differentiation among tokens.
As shown in Fig. 14, observe results that closely align with our intended outcomes from the variance scale-up analysis in Fig. of the main manuscript. Token Spacing elevates local isotropy to level comparable with direct variance scale-up, validating that Token Spacing effectively boosts the isotropic structure within local cluster neighborhoods.
D.3 Effects of Residual Alignment for Mitigating the Side Effeets from Token Spacing
To mitigate potential side effects (see Section 2.5 and C.4) introduced by variance scaling, we propose an additional technique named Residual Alignment. Residual Alignment involves rotating the residual space, previously obtained through PCA, toward meaningful semantic direction, enhancing semantic coherence while preserving variance-increased embeddings.
Specifically, as depicted in Fig. 13(a) and (b), compare the values of Ay before and after applying Residual Alignment. Here, Ay is defined as cos(s, é) cos(s, e), where é denotes the adjusted embedding. More explicitly, before Residual Alignment, è represents embeddings modified solely by Token Spacing. After applying Residual Alignment, however, ê reflects embeddings adjusted by both Token Spacing and Residual Alignment. This distinction allows us to directly evaluate the incremental impact of Residual Alignment on semantic coherence. The results clearly illustrate that Residual Alignment effectively reduces discrepancies, refining semantic alignment and demonstrating its complementary role alongside Token Spacing.
23

---

E Evaluation Details
We follow the same evaluation protocol as RareBench [5] for both Human Study and GPT-4o evaluation [42]. In the following sections, we describe the scoring criteria for Human Study and GPT-4o, as well as provide brief explanation of the prompts used for each task. Basically, our evaluation methodology followed Park et al. [5]: all evaluations were initially scored on [1, 2, 4, point scale by GPT-4o and Human, which were then normalized [0, 25, 50, 75, 100] for reporting the final benchmark performance.
E.1 Human Study
For the human study, we recruited 23 distinct participants. Participants evaluated the alignment between the given prompt and the generated images using scores ranging from I to 5, where a score of 5 indicates a perfect alignment between the image and text, while score of means that the image fails to capture any aspect of the prompt. We utilized the same prompt categories defined by RareBench, and participants assessed outputs generated by our model and baseline models under identical conditions within each prompt category. To ensure unbiased evaluation, model identities were anonymized, and the presentation order of generated outputs was randomly shuffled within each category and prompt. The detailed scoring guidelines are identical to those used in the GPT-4o evaluation; please refer to the GPT-4o evaluation example provided below.
F.2
GPT-4o Evaluation
We conducted an identical evaluation procedure using GPT-4o, employing the same scoring guide- lines and assessment protocol in the human evaluation described above. To ensure consistency and reproducibility, we set the random seed to 42, following the exact methodology outlined in RareBench [5].
24

---

GPT-4o Instruction: RareBench for Text-to-Image
You are my assistant evaluating the correspondence of an image to a given text prompt. Focus specifically on:
Objects in the image and their attributes (e.g., color, shape, texture)
Spatial layout and positioning
Action relationships among objects
Evaluate how well the provided image aligns with the following prompt:
[PROMPT]
Assign score from based on the criteria below:
5 Image perfectly matches the content of the text prompt with discrepancies.
4 Image portrays most of the content with minor discrepancies.
3 Image depicts some elements, but omits key parts or details.
Image depicts few elements, omitting many key parts or details.
Image fails to convey the main scope of the text prompt.
Provide your evaluation clearly within 20 words using the format below:
### SCORE: [your score]
### EXPLANATION: [brief justification]
GPT-4o Instruction: RareBench for Text-to-Video
You my assistant evaluating the correspondence of time-lapse video to given text prompt.
You will receive eight key frames extracted from the video, each filename indicating its position a sequence. Focus specifically on:
Objects in each frame and their attributes (e.g., color, shape, texture)
Spatial layout and positioning
Action relationships among objects
Consistency and appearanceddiaapperanne elements across frames
Evaluate how well the provided video aligns with the following prompt:
[PROMPT]
Assign score from to 5 based the criteria below:
5 All frames perfectly match the text prompt with no discrepancies.
Most content matches, but minor discrepancies exist in few frames.
Some key elements match, but several important details are missing or incorrect.
Only few prompt elements appear; many key details are absent or wrong.
The video largely fails convey the prompt's 's content.
Provide your evaluation clearly within 20 words using the format below:
### SCORE:
[your score]
###
EXPLANATION: [brief justification]
25

---

GPT-4o Instruction: RareBench for Text-Driven Image Editing
You are my assistant evaluating the effectiveness of text-driven editing from a reference image (first) an edited image (second), guided by the following text prompt:
[PROMPT]
Focus specifically on:
Changes in objects and their attributes (e.g., color, shape, texture)
Adjustments in spatial layout and positioning
Modifications in action relationships among objects
Evaluate how effectively the edited image reflects the intended transformation described by the prompt compared to the reference image. Assign score from based on the criteria below:
5 Edited image perfectly matches the intended transformation described by the prompt.
Edited image effectively couveys the transformation with minor discrepancies.
3 Edited image captures some intended transformations but misses key details.
Edited image reflects few intended changes, omitting many key transformations. Edited image fails to convey the intended transformation from the reference image.
Provide your evaluation clearly within 20 words using the format below:
### SCORE: [your score]
### EXPLANATION: [brief justification]
E.3 Evaluation Prompts for Each Benchmarks
RareBench. We primarily utilize RareBench [5] to evaluate rare prompts. RareBench comprises prompts featuring single objects across five categories: property, shape, texture, action, and complex. Multi-object prompts are categorized into concat, relation, and complex. Each category contains 40 diverse prompts, totaling a comprehensive set for evaluation.
I21-Compbench. To assess performance on more common prompts, we also evaluate using T21- CompBench [40]. T21-CompBench offers holistic evaluation framework, encompassing six cate- gories: color, shape, texture, spatial relationships, non-spatial relationships, and complex compositions. Each category provides tailored prompts, such "a blue bench and green cake" for the color category. For evaluating attributes like color, shape, and texture, we employ BLIP. Spatial relation- ships are assessed using UniDet for object detection, while non-spatial relationships are evaluated with CLIP. Complex compositions are analyzed using the 3-in-1 evaluation method proposed by 72l-CompBench.
GenEval. We also evaluated with GenEval [41], which an object-focused framework designed to evaluate compositional image properties, including object co-occurrence, position, count, and color. leverages object detection models to verify the presence and attributes objects in generated images, facilitating fine-grained, instance-level analysis. GenEval's evaluation pipeline includes tasks such as single object recognition, two-object co-occurrence, counting, color classification, position assessment, and attribute binding. This comprehensive approach allows for detailed evaluation of text-to-image models' capabilities in generating semantically accurate and compositionally coherent images.
F Additional Experiement Results
F.1 Additional Ablation Studies
Experiments for extended range Text-to-Visiee Fig. 15 presents extended ablation study, building upon Fig. 10 of the main manuscript, to thoroughly investigate the impact of the hyper- parameter on both Image and Text-to-Viis Video tasks. In our method, ToRA, o is critically
26

---

(a) Text-to-Image
(b) Text-to-Video
100
Roperty
100
Pizsrty
Tecture
RareBench Porformance
02
Single
Action
RareBennhh Performance
20
Actin
Sing Comple
62
Raistion Nati Densie
60
HMU Comple.
Palition
60
20-
07
27
15 14
Figure 15: Ablation study investigating the impact of varying the hyperparameter o on our method's performance across the RareBench [5] benchmark. (a) Results plot depicting Text-to-Image perfor-
mance for each category RareBench, generated using the SD 3.0 [4]. (b) Corresponding results plot for Text-to- performance across RareBench categories, utilizing the Cog VideoX-5B [27].
In both tasks, o 1.3 was consistently identified as the optimal hyperparameter setting.
(a) Text-to-Image
(b) Text-to- Video
Taltan Spacing
Todure
Tolien Spacing
Residual Alignment Takan Spacing Only Base
Residua Aligmment
A3500
Stape
Action
Shipe
Base
Taken Spazing Only
Kareitench Performance
Rareilench Performance
Singic
rooely TV
Single
roperty
CoRox
Concat
Ratetion
Figure 16: Ablation studies investigating the impact of Residual Alignment within our proposed method, ToRA, evaluated RareBench [5]. (a) For the Text-to-Image task, incorporating Residual Alignment consistently delivered superior performance across all categories, as measured on the SD 3.0 [4]. (b) Similarly, the Text-to- Text-to-Video Video task also demonstrated the best quantitative results when Residual Alignment was utilized, with measurements taken on the Cog VideoX-5B [27].
utilized for Token Spacing. Our experiments consistently demonstrate that optimal performance for both tasks is achieved at 1.3. A notable observation is the significant performance degradation when a < 1.0. Specifically, for a S 0.7 or a 2 1.9, the generated outputs consistently exhibit characteristies close pure noise or bear relevance to the input prompt whatsoever, resulting in the lowest possible score of 0.0. While our method exhibits some sensitivity to the value of a, is crucial to highlight that within the range of 1.0 baseline across all evaluated categories.
1.5, model consistently outperforms the
Effects of Residual Alignment on Text-to- Fig. 16 presents additional quantitative results demonstrating the impact of residual alignment on our method, building upon the insights from Fig. 10. As evidenced in Fig. 16(b), the integration of residual alignment significantly enhances performance in Text-to- Video generation when combined with token spacing within our framework. This outcome demonstrates that employing only token spacing in method, ToRA, does not achieve optimal performance due to the side effects we previously identified in Section 2.5 of the main paper and Appendix C.4. Consequently, this quantitative ablation study further validates that our proposed residual alignment effectively mitigates these undesirable symptoms.
Ablations on -Diven Image Editing. Table 5 presents an extended ablation study, building upon Table 2 from the main manuscript, to comprehensively examine the influence the hyperparameter on text-driven image editing performance. Consistent with the findings reported in the main manuscript, the CLIP ing scores generally show lower values compared the baseline across various settings. However, method achieves the highest directional alignment, measured by CLIP dir. at
27

---

Table CLIP similarity scores image, text, and directional alignment across different a values of Stable Flow [28] + Our approach on Text-Driven Image Editing. The analysis also demonstrates the impact of residual alignment in our methods. The values highlighted in yellow are taken from our main paper, and those marked in blue indicate the highest-performing results per metric.
Experiments
CLIP CLiP
CLIPar1
GPT-4of
Ablation Studies for Various a (WV residual alignment)
0.7
0.77
0.23
0.13
55.7
0.8
0.82
0.25
0.14
68.4
0.9
0.82
0.25
0.13
1.0
11.3
0.25
0.14
73.4
1.1
0.82
0.28
0.17
77.2
1.2
0.81
0,28
0.18
79.3
1.3
0.80
0.28
0.20
82.8
1.4
0.81
0.28
0.20
84.5
1.5
0.82
0,29
0.18
87.2
1.7
0.82
0.29
0.16
81.4
1.8
0.81
0.29
0.17
1.9
0.80
0,29
0.17
68.4
Ablarion Studies for Residual Allgument (0 1.3)
w/o residual alignment
0.83
0.25
0.14
= 1.3 and 1.4. Additionally, we observe clear trend where increasing a consistently improves CLIP text and CLIP mm performance, whereas significantly lower values, such as a 0.7, result in substantial performance degradation. For GPT-4o score, performance rises when the scale factor lies between 1.3 and 1.6; pushing below 1.3 or above 1.6 consistently degrades performance, trend that matches what we observe on other diverse tasks.
Looking at these results, what we find particularly noteworthy is that, given that Stable Flow [28] only controls a subset of layers rather than all layers simultaneously, the variations in performance relative to baseline methods appear limited. Nevertheless, applying either Token Spacing or Residual Alignment methods improves the baseline performance. As previously discussed, choosing an appropriate value remains crucial for optimal model performance, and it's important to note that while 1.3 demonstrates strong results overall, does not universally produce superior performance across all metrics.
Evaluating robustness across random seeds. To check if our method consistently performs well no matter the random seeds, we tested its strength on 20 prompts from RareBench [5]. We used 50 different random seeds for each prompt and then took the average score. As shown in Fig. 17, our approach consistently bolsters the baseline's 's reliability, Moreover, when integrated with other methods [5], we observe general uplift benchmark performance.
100
Single Object (Avg.)
Multi Objec: (Awgi)
RareBench Performance
71.1
#2.8
65.0
14.0
67.7
+3.6
55.8
19.2
20.4
Base
Base +Ours
R2F
R2F +Ours
Figure 17: Quantitative results evaluating robustness across random seeds on RareBench.
28

---

F.2 More qualitative results
In this section, we present further qualitative results. These include insights into compositional aligmment, alongside Text-to-Image Image generation, Text-to-Vide generation, and Text-Driven Image Editing with rare prompts.
Note. Full-page figures are placed at the bottom of the document.
F.2.1 T2ICompbench
Qualitative results in Fig. 19-20 demonstrate how our method, ToRA, influences the model's compositional alignments in Texi-to-Image generation.
F.2.2
Text-to-Image Generation
Supplementary Text-to-Image generation results for rare prompts are shown in Fig 21-29.
F22.3 Text-to- Video Generation
We present further Text-to- Videeo generation outcomes for rare prompts in Fig. 30-34.
F.2.4 Text-driven Image Editing
Fig. 35 and 36 illustrate additional Text-Driven Image Editing results for rare prompts.
G Limitations and Further Discussions
G.1 Limitations and Future Works
Architectural Scope: Our technique is presently tailored to generative models with joint image self-attention. Extending it to U-Net backbones [1, 2] and other emerging diffusion variants is natural next step the architectural landscape evolves. Moreover, as Diffusion Transformers are an actively researched area in generative modeling, similar to REPA [53], it is essential to further explore our method's is applicability to these emerging architectures.
Prompt Length: We have not yet stress-tested extremely long prompts [54]. A systematic evaluation of this regime will clarify the method's s conditioning limits. Investigating such cases would provide further insights into prompt robustness and conditioning boundaries.
Broader Applicability for Training Phase with ToRA: Our method was developed as a training- -free approach. It would be interesting to investigate how its underlying principles might transfer to the training process itself. Extending our approach to diverse learning settings remains an important direction for future work.
Deeper Theoretical Insights for ToRA: Finally, while our empirical findings reveal the role of isotropy and anisotropy in semantic representations, exploring deeper theoretical insights into why variance scaling is effective could more concretely explain these observations. Such insights might also bring clarity the phenomenon of semantic emergence, referring to how meaningful semantic properties arise through the interplay of these representational characteristics, offering an exciting avenue for future exploration.
G.2 Further Discussions
Our investigation centered on the semantic emergence within text embeddings in vision generative models, particularly MM-DiT. We found that this phenomenon, where intrinsic meaning naturally surfaces within the model, can be effectively induced through a relatively simple yet potent technique: variance scale-up. This effect, suggest, be explained by the established properties of isotropy and anisotropy as discussed in natural language processing research.
The significance of this semantic emergence finding lies in its potential to facilitate successful textual semantic and often elusive compositional alignment internally within the model's embedding space, achieved through a simple yet effective intervention. This is accomplished without the need for
29

---

external modules, such as large language models (LLMs), to forcibly inject semantic alignment. This inherent capability appears to offer broad generalizability, allowing for seamless integration with other methods. Moreover, its applicability is not confined to a single output data type or task, extending to wide array of text-to-vision tasks that leverage natural language input. We believe this work thus presents an exciting avenue for elevating the intrinsic capabilities and potentially pushing the upper bounds of performance for MM-DiT architectures, a vibrant area of research in modern generative modeling.
H Qualitative Comparison with GPT--0
As GPT-4o [42] currently represents the state-of-the-arr in generative models, we evaluate our method in direct comparison to it. Although GPT-4o has recently demonstrated remarkable performance across variety of generative tasks, its closed-source nature limits direct reproducibility and in- tegration. Our method is built entirely on open-source components, yet achieves results that qualitatively comparable to GPT-4o across a wide range of prompts, as shown in Fig. 37-40. This highlights the potential open models narrow the performance gap while maintaining accessibility and transparency. We also find that in certain instances where prompts require precise numerical understanding, such "A four armed ninja" in Fig. 40, GPT-4o tends to misrepresent the intended structure. In contrast, our method accurately depicts the number of arms, highlighting improved semantic fidelity in these challenging cases.
Note. Despite being developed entirely with open-source components, our method achieves performance that is comparable to GPT-4o, the kind of state-of-the-art in text-to-image gen- eration. Unlike GPT-4o, which often produces visually similar outputs for given prompt, our method exhibits greater generative diversity across different random seeds, showing its robustness and flexibility. Furthermore, its plug-and-play design allows seamless integration into various generative pipelines, enabling easy experimentation and broader applicability. We believe this accessibility positions our method as practical and versatile contribution toward advancement of future generative modeling research.
30

---

Scale-Down
Original
Scale-Up
Scale-Down
Original
Scale-Up
A mustachioed monkey is hanging from thorny hotdog
A bearded giraffe is admiring a butterfly shaped dish
SD
A wrinkled globe is resting beside an ax shaped chocolate
mustachioed lizard
FLUX
A heart shaped basketball
1mmp running with legs
A pencil made of diamonds
A crying pizza
Figure 18: Further results illustrating the influence of variance scaling (Scale-Down and Scale-Up) on text embeddings within the original configuration, examining its effects on both generated images and text-to-text self-attention maps.
31

---

Left: Original (SD 3.0)
Right: Original + Ours
woman behind a man
computer in front of a cow
A giraffe behind a sofa
A sheep in front of cat
train behind chair
red cylinder and blue cube
big balloon maxble
big bathtub square showerhead
brown and green log
black gold white vase sitting on counter
A dog sitting on couch and watching TV with its owner
Four trucks
Six bowls
Two plates
Eight guitars
The fluffy marshmallow melted over the warm cocoa and the crunchy grahan cracker
The fuzzy sphere nestled between the spiky cube and the smooth cylinder
long rectangular table and small circular vase were placed in the center the
Figure 19: Qualitative comparisons of text-to-image compositional alignments: baseline (SD 3.0) vs.
baseline + our method,
32

---

Left: Original (FLUX-schnell)
Right: Original Ours
A brown dog and yellow horse
brown cow and
pink sheop
A blue boat and red car
Three helicopters
Six apples
Four tomatoes
horse on side of a bicycle
boy on the left of balloon key on the left butterfly
A metallic earring and leather sofa
metallic ring and wooden knife
A cubic block and cylindrical container of hand
circular rug and triangular coffee table
A phone hidden by frog
A fish hidden by a sofa
gold backpack and blue clock
soft white feathers of the owl contrasted with the sharp black talons
glossy apple sat next fuzzy peach and the prickly pear
Figure 20: Qualitative comparisons of text-to-image compositional alignments: baseline (FLUX-
schnell) vs. baseline + our method.
33

---

3.0
3.0 R2F+Ours
SD 3.0
R2F
Ours
hairy clam
A wooly banana
An shaped guitar
An oval shaped soccer ball
Figure 21: Further Text-to-Image generation comparisons for rare prompts.
34

---

3.0
SD 3.0 Ours
SD 3.0 R2F+Ours
SD 3.0
R2F
A leopard made of plastic
A horned pelican and a spotted lion
A flying cheesecake
soccer ball with triangular check pattern
Figure 22: Further Text-to-Image generation comparisons for rare prompts.
35

---

SD 3.0 R2F
SD 3.0 Ours
SD 3.0 R2F+Ours
SD 3.0
A wrinkled globe is resting beside an ax shaped chocolate bar shaped cheesecake
A hairy octopus dancing with a zebra striped duck is sitting on top of a star
NA
red dragon with mane its back and unicorn made of diamond rollerblading through a neon-lit alien cityscape high speed
A ballet-dancinn ing polar bear with red toe shoes and jazz-playing kangaroo with a blue trumpet performing together on a stage made of ice
Figure 23: Further text-to-Image generation comparisons for rare prompts.
36

---

FLUX-schnell
FLUX-schnell
FLUX-schnell R2F+Ours
FLUX-schnell
R2F
Ours
A bearded giraffe
mustachioed clownfish
A donut shaped train
burning ice сream
Figure 24: Further Text-to-Imape generation comparisons for rare prompts.
37

---

FLUX-schnell
FLUX-schneil
FLUX-schnell R2F+Ours
FLUX-schnell
R2F
Ours
A zebra striped palm tree
A sunflower made of diamonds
A black white checkered crocodile
A wooly alligator is guarding a banana shaped bottle Figure 25: Further Text-to-Iman generation comparisons for rare prompts.
38

---

FLUX-schnell
FLUX-schmell
FLUX-schnell R2F+Ours
FLUX-schnell
R2F
Ours
thorny dolphin leaping over a star shaped bed
two decorated indian elephants playing chess floating island surrounded by pink dolphins painting rainbows in sky
blue ant hides in the forest in military uniform aiming gun at red enemy on horseback
pirate sailing on pyramid- -prapid-shaped boat
Figure 26: Further Text-to-Iman Image generation comparisons for rare prompts.
39

---

FeUX-dev Ours
FLUX-dev
horned pelican
horned elephant
butterfly shaped bowl
A banana shaped watermelon
Figure 27: Further Text-to-Image Image generation comparisons for rare prompts.
40

---

FLUX-dev Ours
FLUX-dev
P
A cactus made of steel and a flower patterned mirror
An orange made of marble red bird with blue fish tail
skyblue unicorn doing kung fu
Figure 28: Further Text-to-Image generation comparisons for rare prompts.
41

---

FLUX-dev Ours
FLuX-dev
A shrimp made of steel and a spotted dog
dancing bulldog is staring at black white checkered pineapple
A flower patterned dumpling
A thorny kangaroo riding a unicycle on tightrope stretched across canyon filled with floating jellyfish emitting soft glowing lights
Figure 29: Further text-to-Image generation comparisons for rare prompts
42

---

CogVideoX
Ours OopWobeot
Ared dragon mane its back and unicomn made of diamond rollerblan bladilinng through neon-lit allen cityscape at high speed
CogVideoX
Ours Cepvideaa
snail wearing a racing on candy racetrack
CogVideoX
Ours Coghieok
DSLR photo of a frog wearing sweater
CogVideoX
Ours Cnglideex
A dog piloting sonut-shaped spaceship through galaxy of sprinkles
Figure 30: Further Text-to- Video generation comparisons for rare prompts.
43

---

CogVideoX
Ours DapWobeot
squirrel wizard stirring a magical potion teacup floating leaf
CogVideoX
Ours Cepiideat
human Face crying with diamond
CogVideoX
Ours Ciyheokk
Aaccoon a trench reading a mystery novel under a gloowng mushroom
CogVideox
Ours CngWideek
A foxhith sunglasses driving a bumper car through a glowing tunnel
Figure 31: Further Text-to- Video generation comparisons for rare prompts.
44

---

CogVideoX
Ours CapWobeot
A bicycle and an airplane
CogVideoX
Ours Cepviteat
Gryffindor common room on a snowy evening, warm fireplace, students sipping butterbocr. an owl slceping on window lodge
CogVideoX
Ours Coyheook
Ahairy penguin on candy
CogVideoX
Ours Cnglideek
golden retriever splashes through a shallow river at sunset, chasing after stick with droplets flying every direction. The orange sky reflects off the water's surface, willow branches sway gently the
Figure 32: Further Text-to-Vide generation comparisons for rare prompts.
45

---

CogVideoX
Ours DxpWoboot
miniature traveler climbs full-sized napkin folded like mountain, with spoon resting dinner plate beside them.
CogVideoX
Ours Cepiideat
A horned clephant
CogVideoX
Ours Eigheeok
A DSLR photo of corgi taking a selfie
Cog VogVideoX
Ours Coglideek
Aquurrel pilots a bee eeee-pttteeed airplane through forest made of giant flowers, while a squad of ant soldiers chases after him
Figure 33: Further Text-to- Video generation comparisons for rare prompts.
46

---

CogVideoX
Ours CapWobeat
tennis racket bottte
CogVideoX
Ours Capiiteat
golden retriever with sweater and smiling boy pajamas riding a scooter through a glowing candy forest
CogVideoX
Ours CoyWeesk
A kitten rides car drifting upward in cozy room, captured photoreal detail.
CogVideoX
Ours Coglidee
A cloaked traveler climbs a steep hill toward glowing castle in the distance, as thunderclouds gather and lightning splits the sky
Figure 34: Further Text-to- Video generation comparisons for rare prompts.
47

---

Reference Image
Stable Ours
Stable Flow
A basket ball
A donut
shaped
basket ball
bottle
banana shaped bottle
mirror
flower patterned mirror
building
oval shaped building
koala
A dancing koala
Figure 35: Further Text-Driven Image Editing comparisons for rare prompts.
48

---

Reference Image
Stable Flow Ours
Stable Flow
sunflower
sunflower walking with legs
A watermelon
A banana shaped watermelon
table
A oval shaped table
chocolate
ax shaped chocolate
A dog
dog cheerleading
Figure 36: Further Text-Driven Image Editing comparisons for rare prompts.
49

---

FLUX-des dev Ours
Prompt
GPT-4o
FLUX-dev
thorny shark and a mustachioed dolphin
A thorny snake is coiling around star shaped drum
A hairy octopus dancing with zebra striped duck is sitting on the top of star shaped cheesecake
A flower patterned deer flying with black white checkered pineapple is resting on an oval table
Figure 37: Qualitative comparisons with GPT-44-generated images.
50

---

FeUX-dev
Prompt
GPT-4o
FLUX-dev
Ours
horned whale walking with black white checkered dragonfly is leaning against mustachioed lizard
A bearded giraffe swimming with dancing bulldog is smiling banana shaped bottle
A bearded parrot
A mustachioed clownfish
Figure 38: Qualitative comparisons with GPT-44-generated images.
51

---

FLUX-dev
Prompt
GPT-4o
FLUX-dev
Ours
thorny hotdog
A wrinkled globe
zebra striped rock
stork made of diamonds
Figure 39: Qualitative comparisons with GPT-4o-generated images.
52

---

FLUX-dev
Prompt
GPT-4o
FLUX-dev
Ours
leopard made of plastic
XI XI
clock with tree roots growing out it
VII
A dandelion zunning with legs
foum- ninja with white hair in blue gi
Figure 40: Qualitative comparisons with GPT-4o-gan -ege-ro-geeertted images.
53

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: We have detailed all claims mentioned in the abstract and introduction in the remaining part of the paper and the Appendix.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long as is clear that these goals not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification: Yes, please Appendix Sec. G. for limitations.
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create a separate "Limitations" section their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model sell-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low images taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't It acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play impor- tant role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [Yes]
54

---

We detail the assumption and proof of theoretical result on anisotropy and isotropy Sec. and Appendix Sec. C.
Justification:
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced in the statement of any theorems.
The proof' can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should properly referenced.
4. Experimental Result Reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification:
We will publicly release all code recover instructions and the results.
Guidelines:
The answer NA means that the paper does include experiments. If the paper includes experiments, No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how reproduce that algorithm. (b) If the contribution is primarily
model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome deseribe the particular way they provide for reproducibility, In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
55

---

Answer: [Yes]
Justification: Yes, we will release our codes and instructions once the blind review period is finished.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips... cc/
Bublie/guudee//DdeSSttiii..
iooolloo for more details.
While we encourage the release of code and data, we understand that this might not be possible, "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for a new open-source benchmark).
The instructions should contain the exact command and environment needed to run reproduce the results. See the NeurIPS code and data submission guidelines (https: ggtgg/udes////// htt.a//ddcggdddeeedduuuubaaaaaia "Codeeubmissiee ssionnolicyyy for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information possible in supplemental material (appended to the paper) recommended, but including URLs to data and code is permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: Yes. We include all evaluation recipes in the main paper and Appendix, and will additionally include them in the code release.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment Statistical Significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [NA]
Justification: We only test the models during inference steps. However, as results could vary due to random seed effects, we provide an analysis of their impacts Appendix Sec.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results accompanied by enror bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
56

---

The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
is OK to report I-sigma error bars, one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative eтror rates).
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures tables in the text.
8. Experiments Compute Resources
Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Answer: [Yes]
Justification: Yes, please see Appendix Sec. D.
Guidelines:
The answer NA means that the paper does include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g.. preliminary or failed experiments that didn't make into the paper).
9. Code Of Ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips: httpp////ccppppc.e.ccccoo....ddd
Answer: [Yes]
Justification:
Our research conform NeurlPS Code of Ethics.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [No]
Justification: We currently focus primarily on the positive impacts of this work; however, if reviewers find it necessary, we are open to including discussion of potential harms associated with generative models, particularly regarding the generation of rare semantics.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
57

---

The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, scraped datasets)?
Question:
Answer: [No]
Justification: We used pre-trained Multi- Modill Diffusion Transformers as generative models, which already have safeguards.
Guidelines:
The answer NA means that the paper poses such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification: Yes, credited them in appropriate ways.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (c.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
58

---

If this information is not available online, the authors are encouraged to reach out to the asset's 's creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification: The paper does not release new assets.
Guidelines:
The answer NA means that the paper does release new assets.
Researchers should communicate the details the dataset/code/moodel as part their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: This paper does not involve crowdsourcing nor research with human subjects.
Guidelines:
The answer NA means that the paper does involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Ethies, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification: No, our study did not involve any potential risks to participants, and thus no risk disclosures or Institutional Review Board (IRB) approvals were necessary.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Depending the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, not include any information that would break anonymity (if applicable), such the institution conducting the review.
59

---

16. Declaration of LLM usage
Does the paper describe the usage of LLMs if it is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, or formatting purposes and does not impact the core methodology. scientific rigorousness, originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification: The method development in this study does not make use of LLMs.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non-standard components.
Please refer to L.LM policy (https: http:://neuuuuueeeeee wttpiipp....cooffeeeeee http://ss..e//oofeeeeneeee/////s for what should or should not be described.
60


Reviews:
[{"review_id": "DG9RhYT08k", "reviewer": "Reviewer_KirX", "summary": "This work introduces TORA, a training-free method to enhance rare semantic emergence by expanding text token embeddings’ representational basins via variance scale-up before joint-attention blocks. By amplifying local isotropy and refining residual alignment, TORA improves semantic distinguishability without external modules or retraining. It achieves robust generalization across text-to-image, text-to-video, and image editing tasks, demonstrating superior performance on rare prompts while preserving common concept accuracy. The method’s low computational overhead and compatibility with existing pipelines make it a practical solution for boosting MM-DiT’s semantic fidelity. All code will be publicly released.", "questions": "1. Does ToRA work for other problems like missing objects or wrong numbers of objects?", "limitations": "yes, the limitations have been addressed", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n1. TORA achieves rare semantic alignment without requiring additional training, data, or external modules.\n2. TORA enhances local isotropy in text embeddings by expanding representational basins through variance scale-up and refining residual alignment.\n3. TORA generalizes effectively across diverse text-to-vision tasks (text-to-image, text-to-video, and image editing) and achieves state-of-the-art performance on rare prompts.\n\nWeaknesses:\nHard to find many weaknesses, it's a strong paper that is well written, with a clear argument, clear analysis, clear solution that seems to work after both quantitative and qualitative results.", "was_revised": true, "final_justification": "This work introduces TORA, which is a highly effective, training-free approach to improve semantic alignment. By enhancing the local isotropy of text embeddings through variance scale-up and residual alignment, it successfully addresses a key challenge without needing extra data or modules. Its effectiveness is clearly demonstrated by strong generalization across text-to-image, text-to-video, and editing tasks, alongside state-of-the-art performance on rare prompts. These significant contributions justify my score of 5.", "rebuttal": "We sincerely thank you for your thoughtful and constructive feedback. Below, we address each of your comments in turn. \n\n---\n\n### **[Q1] Does ToRA work for other problems like missing objects or wrong numbers of objects?**\n\nYes . ToRA performs well in mitigating missing objects and incorrect object counts. In `Figure 7` in our main manuscript, for example, for the prompt *\"a thorny snake is coiling around a star‑shaped drum\"*, baseline models fail to produce the *snake*, whereas ToRA successfully generates both the *snake* and the *drum*. Likewise, for *\"a four‑armed ninja with white hair in a blue gi\"*, competing models generate fewer than *four arms*, but our methods generates exactly *four*, faithfully matching the text. `Table 3` quantifies this advantage on the GenEval benchmark [1], where ToRA outperforms baselines on the *Single‑Object*, *Multi‑Object*, and *Counting* categories. \n\nFurthermore, to thoroughly validate the performance gains on RareBench [2], we carefully categorized prompts involving multiple objects' presence and numerical counts, reassessing benchmark performance against the baseline. Specifically, we refined the GPT-4o instructions in `Appendix Section E.2` to explicitly evaluate multiple objects presence and numerical counts accuracy by posing two precise yes/no questions: *“Are all objects mentioned in the text present in the image?”* and *“Is the specified number in the text correctly reflected?”* We conducted experiments with seed=42 and $\\sigma=1.30$, consistent with our main manuscript protocols. The resulting scores, which clearly demonstrate the model’s enhanced capability, are presented in the table below.\n\n| | $\\text{SD3.0}$ (base) | $\\text{R2F}_\\text{SD3.0}$ | $\\text{Ours}_\\text{SD3.0}$ | $\\text{FLUX-schnell}$ (base) | $\\text{R2F}_\\text{FLUX-schnell}$ | $\\text{Ours}_\\text{FLUX-schnell}$ | $\\text{FLUX-dev}$ (base) | $\\text{Ours}_\\text{FLUX-dev}$ |\n| :--- | :---: | :---: | :---: | :---: | :---: | :---: | :---: | :---: |\n| **GenEval (common)** |\n| Counting (max: 1.00) | 0.63 | 0.65 | **0.66** | 0.70 | 0.59 | **0.77** | 0.74 | **0.74** |\n| Two Objects (max: 1.00) | 0.74 | 0.76 | **0.77** | 0.77 | 0.77 | **0.84** | **0.81** | 0.80 |\n| **Rarebench** |\n| Numerical Counts (max: 100.0) | 72.1 | 72.5 | **76.4** | 74.4 | 72.6 | **75.1** | 77.3 | **79.0** |\n| Multi-Objects Presence (max: 100.0) | 81.0 | 80.8 | **82.1** | 83.7 | 83.7 | **84.9** | 83.9 | **85.1** |\n\nTogether, these results confirm that ToRA is effective not only for compositionality but also for missing‑object and miscount problems. These results will be included in our revised version.\n\n---\n\n*[1] Ghosh, Dhruba, Hannaneh Hajishirzi, and Ludwig Schmidt. \"Geneval: An object-focused framework for evaluating text-to-image alignment.\" Advances in Neural Information Processing Systems 36 (2023): 52132-52152.*    \n*[2] Park, Dongmin, et al. \"Rare-to-frequent: Unlocking compositional generation power of diffusion models on rare concepts with LLM guidance.\" arXiv preprint arXiv:2410.22376 (2024).*"}, {"review_id": "xLntVE9HrV", "reviewer": "Reviewer_bpoM", "summary": "This paper proposes a simple method to help Multi-modal Diffusion Transformers better handle rare or unusual prompts in text-to-vision tasks. By adjusting the text embeddings before attention using the proposed principle of variance scale up, the model can bring out rare concepts more clearly, without needing extra data, training, or tools.", "questions": "see Strengths And Weaknesses.", "limitations": "N/A", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "# Strengths\n\n1. The paper is overall well written and clearly structured.\n2. The proposed method operates without requiring external knowledge, enhancing its general applicability across tasks.\n3. The method demonstrates superior performance compared to baseline approaches in the reported experiments.\n\n# Weaknesses\n\nThese are not necessarily weaknesses, but addressing them may help strengthen the paper:\n\n1. It appears that the parameter $k$ in Equation (2) plays a significant role in the proposed method. An ablation study on $k$ would help clarify its impact on performance.\n2. In line 175, the authors mention \"decomposing the space of original text embeddings\" using PCA. PCA typically requires a set of vectors—could the authors clarify what this set consists of in this context?\n3. The exposition in Section 3.2 could benefit from further revision. In particular, the definition of $G_{\\theta}$ in Equation (3) is unclear. Do the authors intend for $G_{\\theta}$ to denote any matrix satisfying Equation (3), or is it defined in a more specific way?\n4. What would be the performance if $\\sigma < 1$, for example $\\sigma = 0.5$ or $\\sigma = -1.3$? Some clarification or empirical discussion would help readers understand the the impact of the variance scaling up under such settings.", "was_revised": true, "final_justification": "The authors’ responses address the technical details. I would like to maintain my initial scores.", "rebuttal": "We sincerely thank you for your thoughtful and constructive feedback. Below, we address each of your comments in turn. If you need to check the technical appendix, it can be found in the submitted supplementary materials.\n\n---\n### **[W1] An ablation study of the parameter $k$.**\n\nAs described in `lines 178–181` of our manuscript, **we automatically determine $k$**, the number of top principal components, via the Maximum Distance to Chord (MDC) method [1], which identifies an optimal cutoff, also referred to as the *elbow point*. We chose this adaptive strategy instead of using a fixed $k$ because the number of meaningful dimensions in the principal and residual subspace can vary significantly across input prompts, especially given their variable user's text lengths. \n\nTo empirically validate our choice of an adaptive strategy, we conducted an ablation comparing our MDC-based automatic selection of $k$ with fixed-$k$ alternatives. To this end, we tested fixed $k$ values ranging from 10 to 60 in increments of 10 (see Tables below).  For additional insight into the typical scale of dimensionality, we computed the average MDC-derived $k$ for each task, resulting in approximately ~42.0 for Text-to-Image (T2I) and ~23.0 for Text-to-Video (T2V).\n\n- *Fixed $k$ ablation studies for text-to-image (Rarebench)*\n\n| $k$ | SD3.0 | FLUX-schnell | FLUX-dev |\n|---|---|---|---|\n| **ours** | **76.7** | **76.0** | **75.3** |\n| 10 | 73.2 | 73.3 | 70.0 |\n| 20 | 74.0 | 73.8 | 71.2 |\n| 30 | 74.8 | 74.6 | 73.0 |\n| 40 | 75.8 | 75.1 | 74.2 |\n| 50 | 74.1 | 72.5 | 70.8 |\n| 60 | 65.2 | 70.3 | 65.2 |\n\n- *Fixed $k$ ablation studies for text-to-video (Rarebench)*\n\n| $k$ | CogVideo-5B |\n|---|---|\n| **ours** | **66.6** |\n| 10 | 63.5 |\n| 20 | 65.3 |\n| 30 | 64.7 |\n| 40 | 60.0 |\n| 50 | 55.7 |\n| 60 | 51.8 | \n\nAs these tables show, we can see that the automatically selected $k$ in ToRA outperforms any fixed $k$. Furthermore, we found that using ToRA yields higher scores than not using it in most cases, regardless of the value of $k$ we chose. (*e.g.,* the original RareBench average for SD 3.0 is 60.9).\n\nInspired by the reviewer’s insightful suggestion, we recognized that additional ablation studies could be conducted, incorporating adaptive shifts of ±15% per sample from the determined $k$ derived by MDC method for single prompt to evaluate performance.  This experiment truly strengthens our paper’s standing. The results of these ablation studies of the adaptive approach are presented below.\n\n- *Adaptive $k$ ablation studies for text-to-image (Rarebench)*\n\n| Adaptive $k$ | SD3.0 | FLUX-schnell | FLUX-dev |\n|---|---|---|---|\n| -15% | 72.9 | 73.8 | 73.1 |\n| -10% | 73.4 | 74.1 | 73.7 |\n| -5% | 75.2 | 75.4 | 74.4 |\n| **ours** | **76.7** | **76.0** | **75.3** |\n| +5% | 72.5 | 73.3 | 69.3 |\n| +10% | 69.8 | 70.9 | 65.0 |\n| +15% | 66.1 | 69.5 | 62.2 |\n\nAdditional implementation details and expanded discussions will be provided in the Appendix of the revised manuscript.\n\n*[1] David H. Douglas and Thomas K. Peucker. Algorithms for the reduction of the number of points required to represent a digitized line or its caricature. Cartographica: The International Journal for Geographic Information and Geovisualization, 10:112–122, 1973.*\n\n---\n### **[W2] Need clarifying the target vector set for PCA decomposition**\n\nYes, PCA requires a set of vectors. In our context, this set corresponds to the **sequence of text‑token embeddings from the user’s single input prompt**.\nTo provide a concrete example using the SD 3.0 architecture, the text encoder outputs a tensor of token embeddings, denoted as $\\mathbf{e} \\in \\mathbb{R}^{L \\times D}$, where $L=333$ is fixed sequence length (composed by the user's prompt tokens and special tokens like [SOS] and [PAD]), and $D=4096$ is the embedding dimension.\n\nThese embeddings are then propagated through a stack of a joint-attention blocks. At each block $b$, we extract the input embedding matrix $\\mathbf{e}^b$, representing the text features before entering the block, which can be viewed as a set of $L=333$ vectors, each with $D=4096$ dimensions.\nWe then apply PCA to this matrix $\\mathbf{e}^b$. Technically, this PCA is implemented via Singular Value Decomposition (SVD), where the matrix $\\mathbf{e}^b$ is decomposed by $U\\Sigma V^T$. In this decomposition, the rows of the matrix $V^T \\in \\mathbb{R}^{D \\times D}$ consist of the eigenvectors that represent the principal components of the embedding space. This meant that the target vectors for PCA is a set of text token embeddings from a single prompt. Following PCA, if we apply Token Spacing method to these vector set, it leads to an increased separation of text tokens along the principal components $V^T[:k, :]$.\n\n---\n\n### **[W3] Unclear interpreting the role and construction of $\\mathbf{G}_\\theta$ in `Equation (3)`**\n\nWe clarify that $G_\\theta$ is not an arbitrary matrix satisfying the `Equation (3)`, but rather a **specifically constructed orthogonal matrix** (a Givens rotation [2]), designed to align the dominant residual component with the semantic direction. Givens rotation is a type of orthogonal transformation represented by orthogonal matrix. We utilize this matrix because the eigenvectors must preserve their orthogonal property.\n\nMore concretely, after decomposing the text embedding space via PCA, we obtain the residual subspace spanned by the singular vectors {$v_{k+1}, \\dots, v_n$}, where $v_{k+1}$ is the most significant among the residual directions. \nWe define the semantic vector $s = e_{\\text{cond}} - e_\\emptyset$, \nrepresenting the difference between conditioned and unconditioned embeddings (as used in classifier-free guidance). We then project $s$ onto the residual subspace (denoted $s_\\text{proj}$) to remove its principal components.\n\nThe Givens rotation matrix $G_\\theta$ is explicitly constructed to rotate the vector $v_{k+1}$ toward the normalized projected semantic vector $\\frac{s_\\text{proj}}{\\Vert s_\\text{proj} \\Vert}$, while leaving all other residual basis vectors orthogonal and unchanged. Note that, $\\theta$ is not an argument but instead refers to the resulting angle derived from the alignment process. This operation, as expressed in `Equation (3)`, ensures semantic alignment within the residual subspace.\nIn short, the matrix $G_\\theta$ is a *rotation operator* constructed to align the dominant residual direction with the projected *semantic vector*, and it can be computed directly without any learning or iterative optimization. We will revise `Section 3.2` to clearly reflect this clarification.\n\n*[2] Wallace Givens. Computation of plane unitary rotations transforming a general matrix to\ntriangular form. Journal of the Society for Industrial and Applied Mathematics, 6:26–50, 1958. ISSN 03684245.*\n\n---\n\n### **[W4] Additional ablation studies on how a more diverse range of $\\sigma$ affects performance.**\n\nThe reviewer raised the question regarding a broader range of $\\sigma$ values, including those with negative signs. \n\n- **Negative $\\sigma$ values (*e.g.,* $\\sigma = -1.3$):** In our method, $\\sigma$ serves as a scaling factor that amplifies or suppresses singular values. Since both variance and singular values are mathematically **non-negative**, multiplying them by a negative $\\sigma$ would reverse the direction of the corresponding semantic axes. This reversal could severely distort semantic relationships between tokens, leading to a disruption of the semantic space and ultimately causing mode collapse. For this reason, we do not consider negative $\\sigma$ values.\n\n- **Performance results of more various $\\sigma$ values in $0< \\sigma < 1$ :** We provide an ablation study on $\\sigma$ in `Appendix Figure 15`. While we did not test values below $0.7$ in the original submission, the results at $\\sigma = 0.7$ already show complete failure, with outputs collapsing to fully noise. In response to the reviewer’s interest, we have conducted additional experiments with $\\sigma = 0.5$, which also resulted in mode collapse of joint-attention in MM-DiT, producing null outputs. Qualitative evidence of this phenomenon can be found in `Fig. 4`. As highlighted in the yellow boxes of `Fig. 4`, setting $\\sigma$ too low reduces variance excessively, leading to failed image and video generation. We attached the quantitative results associated with these ablation studies in the table below.\n\n| $\\sigma$ | $0.1$ | $0.2$ | $0.3$ | $0.4$ | $0.5$ | $0.6$ | $0.7$ |\n|:---|:---:|:---:|:---:|:---:|:---:|:---:|:---:|\n| Rarebench avg. score (SD 3.0) | $0.0$ *(fully noise)* | $0.0$ (fully noise) | $0.0$ *(fully noise)* | $0.0$ *(fully noise)* | $0.0$ *(fully noise)* | $0.0$ *(fully noise)* | $0.0$ *(fully noise)* |"}, {"review_id": "kQYIQc1sx2", "reviewer": "Reviewer_41Hj", "summary": "This paper investigates the generation of rare prompts in multi-modal diffusion transformers. Without requiring additional training or external modules, this paper finds that rare semantics can merge from MM-DiT by proposing a variance scale-up method. Experimental results show that the proposed method achieves strong performance in various tasks, from text-to-image generation to image editing.", "questions": "Please refer to the strength and weaknesses.", "limitations": "This paper discusses the limitations but does not address the broader impact.", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "**Strength:**\n1. The paper presents an insightful analysis on the isotropic properties of rare semantics.\n2. Without requiring additional training, the proposed ToRA method significantly improves text-image alignment for rare prompts.\n3. This work conducts extensive experiments to support its claims and performance improvements. \n3. The paper is clearly written and well presented. \n\n**Weaknesses:**\n1. The proposed ToRA method performs well on rare prompts. How does it perform on common prompts?\n2. Do common and rare prompts require different parameters or settings? If so, in real applications, how does the system determine whether a given prompt is rare or common?\n3. Equation 2 shows that ToRA scales the top-k singular values. Is k determined by the entire Rarebench dataset, or set differently for different prompts? How does k influence the performance using different groups of prompts, similar to Figure 10(a)?", "was_revised": true, "final_justification": "I have carefully reviewed the paper and read the comments from other reviewers.  As also pointed by Reviewer bpoM and Reviewer VqR3,  some important clarifications and implementation details are missing in the original submission. I appreciate the authors’ efforts during the rebuttal to provide additional explanations, which have solved my concerns. I am glad to raise my rating to 5.", "rebuttal": "We sincerely thank you for your thoughtful and constructive feedback. Below, we address each of your comments in turn. If you need to check the technical appendix, it can be found in the submitted supplementary materials.\n\n\n---\n\n### **[W1&W2] Performance of ToRA on common prompts and prompt-type sensitivity**\n\nIn `Table 3`, we report ToRA’s performance on two benchmarks dominated by common prompts; GenEval and T2I‑CompBench and `Fig. 19 & 20` provide the corresponding qualitative examples. Across both datasets, ToRA matches or exceeds every baseline, showing that its advantages are not confined to rare‑prompt scenarios.\n\n$\\sigma$ is the only hyperparameter in our methods, and we keep it fixed at $1.3$ for all experiments, rare and common alike. Since the same setting works well across the board, a deployed system need not decide whether a prompt is *rare* or *common*; it simply applies ToRA with the default $\\sigma$. Although the method is particularly effective on truly rare prompts, it still improves common benchmarks by sharpening token semantics and reducing errors such as missing objects and attribute mismatches, as seen in `Table 3`.\n\nTo further strengthen our evaluation, we have benchmarked our method on an additional benchmark, named HEIM [1], with the results presented in the table below.\n\n| Model | Alignment | Quality | Aesthetics | Reasoning | Knowledge | Art Styles |\n|:---|:---:|:---:|:---:|:---:|:---:|:---:|\n| SD3.0 | 0.81 | 0.65 | 0.63 | 0.53 | 0.72 | 0.48 |\n| SD3.0 + Ours | 0.90 | 0.65 | 0.65 | 0.56 | 0.80 | 0.48 |\n\nThese results corroborate a central claim of our paper: our method demonstrates pronounced strengths in categories such as *alignment*, *reasoning*, and *knowledge*. These are precisely the categories that demand a deep semantic understanding of the input prompt. A more detailed discussion of these findings will be incorporated into the appendix. We are grateful to the reviewer for this insightful suggestion, which we believe has materially improved the paper.\n\n\n*[1] Lee, Tony, et al. \"Holistic evaluation of text-to-image models.\" Advances in Neural Information Processing Systems 36 (2023): 69981-70011.*\n\n---\n### **[W3] Is k determined by the dataset? How does k influence the performance using different groups of prompts?**\nTo begin with, we do not set the $k$ manually, nor do we assign a benchmark‑specific constant. Instead, $k$ is computed automatically for every prompt, at every diffusion timestep, and in every transformer block.\nAs described in `lines 178–181` of our main text, we determine $k$ adaptively for each individual prompt and layer, using the Maximum Distance to Chord (MDC) method [2]. This approach identifies an *elbow point* in the singular value spectrum, which varies significantly across different prompts due to differences in length, composition, and semantic diversity. Thus, $k$ is not fixed globally across the Rarebench dataset, but instead dynamically computed per sample to better capture the effective dimensionality of the text embeddings in that context.\n\nTo address the reviewer’s potential questions, we conducted an extensive fixed‑$k$ ablation.\n\n- *Fixed $k$ ablation studies for text-to-image (Rarebench)*\n\n| $k$ | SD3.0 | FLUX-schnell | FLUX-dev |\n|---|---|---|---|\n| **ours** | **76.7** | **76.0** | **75.3** |\n| 10 | 73.2 | 73.3 | 70.0 |\n| 20 | 74.0 | 73.8 | 71.2 |\n| 30 | 74.8 | 74.6 | 73.0 |\n| 40 | 75.8 | 75.1 | 74.2 |\n| 50 | 74.1 | 72.5 | 70.8 |\n| 60 | 65.2 | 70.3 | 65.2 |\n\n- *Fixed $k$ ablation studies for text-to-video (Rarebench)*\n\n| $k$ | CogVideo-5B |\n|---|---|\n| **ours** | **66.6** |\n| 10 | 63.5 |\n| 20 | 65.3 |\n| 30 | 64.7 |\n| 40 | 60.0 |\n| 50 | 55.7 |\n| 60 | 51.8 |\n\nWe tested fixed $k$ values ranging from 10 to 60 in increments of 10. By checking the values in the tables above, we can see that the automatically selected $k$ in ToRA outperforms any fixed $k$. \nFurthermore, we found that using ToRA yields higher scores than not using it in most cases, regardless of the value of \n we chose. (e.g., the original RareBench average for SD 3.0 is 60.9). For additional insight into the typical scale of dimensionality, we computed the average MDC-derived $k$ for each task, resulting in approximately ~42.0 for Text-to-Image (T2I) and ~23.0 for Text-to-Video (T2V).\n\nTo confirm the robustness of our approach, we varied the automatically selected $k$ by −15 \\% to +15 \\% and assessed performance (see the table below). The results demonstrate the effectiveness of our MDC‑based method.\n\n- *Adaptive $k$ ablation studies for text-to-image (Rarebench)*\n\n| Adaptive $k$ | SD3.0 | FLUX-schnell | FLUX-dev |\n|---|---|---|---|\n| -15% | 72.9 | 73.8 | 73.1 |\n| -10% | 73.4 | 74.1 | 73.7 |\n| -5% | 75.2 | 75.4 | 74.4 |\n| **ours** | **76.7** | **76.0** | **75.3** |\n| +5% | 72.5 | 73.3 | 69.3 |\n| +10% | 69.8 | 70.9 | 65.0 |\n| +15% | 66.1 | 69.5 | 62.2 |\n\nFinally, although we employ exactly the same procedure irrespective of dataset or benchmark, we investigated whether the choice of  $k$ affects performance across benchmarks by testing on T2I‑CompBench, which features a larger share of common‑language prompts. \n\n- *Adaptive $k$ ablation studies for text-to-image (T2ICompbench)*\n\n| $k$ | SD3.0 | FLUX-schnell | FLUX-dev |\n|---|---|---|---|\n| ours | 85.4 | 84.5 | 84.1 |\n| 10 | 84.0 | 83.1 | 82.9 |\n| 20 | 84.5 | 83.8 | 83.5 |\n| 30 | 85.1 | 84.0 | 84.0 |\n| 40 | 84.2 | 83.0 | 83.5 |\n| 50 | 83.9 | 82.8 | 83.2 |\n| 60 | 83.1 | 82.5 | 80.8 |\n\nAs the table above shows, our method again achieves the best results: just like on RareBench, the ToRA variant outperforms nearly every fixed‑$k$ setting, independent of the $k$ values chosen (note, for instance, that SD 3.0’s baseline T2I‑CompBench average is 56.8). For context, the average values of MDC‑derived $k$ on T2I‑CompBench are 37.5 for T2I and 21.0 for T2V.\nWe will add these findings to the Appendix.\n\n*[2] David H. Douglas and Thomas K. Peucker. Algorithms for the reduction of the number of points required to represent a digitized line or its caricature. Cartographica: The International Journal for Geographic Information and Geovisualization, 10:112–122, 1973.*\n\n---\n\n### **[L] This paper discusses the limitations but does not address the broader impact.**\n\nWe appreciate the reviewer's point regarding broader impact. In the revised version, we'll include a discussion on this. We believe our method enhances semantic robustness and mitigates mode collapse in rare-concept generation, thereby improving the inclusivity and reliability of generative models for real-world applications.\n\nSpecifically, by preserving the semantic distinctiveness of less common concepts, even unseen ones, our approach enables the model to generalize more faithfully to novel prompts. Its adaptive nature may also reduce overfitting to dominant concepts, contributing to fairness across diverse prompt types. However, like all generative models, ours carries the potential for misuse (e.g., generating disinformation from rare semantics) depending on user intent."}, {"review_id": "CUwoxbbEdn", "reviewer": "Reviewer_VqR3", "summary": "This paper proposes ToRA (Token Spacing and Residual Alignment), a training-free method for improving the generation of rare semantic concepts in Multi-modal Diffusion Transformers (MM-DiTs). The core insight is that rare semantics already exist within the text embeddings but are not effectively accessible during generation. The method operates by expanding representational basins around text token embeddings via variance scale-up before joint-attention blocks. TORA consists of two components: (1) Token Spacing, which scales up variance in the principal semantic space using PCA and singular value decomposition, and (2) Residual Alignment, which rotates the residual space toward semantic directions to mitigate potential side effects. The method is evaluated on text-to-image generation (using RareBench, T2I-CompBench, GenEval), text-to-video generation (CogVideoX), and text-driven image editing (Stable Flow).", "questions": "**PCA**: When is the PCA computed in practice? Is it pre-computed using a fixed dataset, computed per-batch, or computed dynamically for each input?\n\n**Computational overhead**: What is the actual computational cost of applying TORA during inference? The paper claims \"low-overhead\" but provides no quantitative analysis.\n\n**Failure mode analysis**: Under what conditions does the method fail or produce worse results? The authors do not expand on the failure modes.", "limitations": "The discussion of limitations is deferred to the appendix. Naturally, the clarity of the writing is not discussed as a limitation, although it is the strongest limitation in my opinion.", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 1, "significance": 3, "originality": 3, "strengths_and_weaknesses": "## Strengths\n\n**Strong empirical performance across multiple tasks**: The method shows consistent improvements across text-to-image generation (Table 1), text-to-video generation (Figure 8), and text-driven image editing (Table 2). On RareBench, TORA achieves substantial gains over baselines across all categories when evaluated with both GPT-4o and human evaluation.\n\n**Training-free**: Unlike existing methods that require finetuning (Figure 2a), optimization during inference (Figure 2b), or external LLM modules (Figure 2c), TORA requires no additional training.\n\n\n**Comprehensive evaluation framework**: The evaluation spans multiple benchmarks (RareBench for rare concepts, T2I-CompBench and GenEval for common prompts) and includes both automated metrics and human evaluation, providing robust evidence for the method's effectiveness.\n\n**Good compatibility with existing methods**: The paper demonstrates that TORA can be combined with existing approaches like R2F, showing improved performance in \"R2F + Ours\" configurations (Table 1), indicating practical compatibility.\n\n## Weaknesses\n**Clarity**: the manuscript is sometimes vague and misses details. For example:\n* In Section 3.1, the authors are not specific enough regarding when PCA is computed, whether it is precomputed using many samples and fixed afterwards, or whether it is computed dynamically for each new batch. Additionally, it is not clear whether the PCA decomposition is shared between transformers blocks, although I assume a different PCA is computed for each layer.\n* The writing style is non-standard and difficult to follow on a first read. The paper would benefit from a clearer separation between background, methodology, and experiments. For example, Section 2 mixes background information with experimental results, making it hard to distinguish prior work from novel contributions. This lack of structure is especially challenging for readers unfamiliar with interventions on the neural network activations, such as those proposed in this manuscript.\n* Another area lacking clarity is the discussion of anisotropy and isotropy in the introduction (third paragraph). The explanation of these concepts is vague, particularly regarding why, after clustering, the cosine similarity within clusters is low while the global average similarity is high. This is counterintuitive, as clustering based on cosine similarity should group similar tokens together. The authors should provide a clearer explanation for this phenomenon in the background.\n\n**Baseline comparisons**: The paper primarily compares against base models and R2F. It would be relevant to compare ToRA with methods that fine-tune the diffusion transformer or perform inference optimizations. Even if the method does not beat those baselines, it would be important for practitionners to understand how large the gap would be.\n\n**Figure quality issues**: Figure 2 contains elements that are not clearly explained. Specifically, it is not clear what the \"rotating arrow\" in Figure 2b represents.\n\n\n\nOverall, the empirical results seem strong, however the background section deserves a major rewrite in my opinion.", "was_revised": true, "final_justification": "I am grateful to the authors for taking the time to answer my questions, and for performing additional experiments. Their answers have cleared my concerns, besides the structure of the paper, which cannot be updated during the rebuttal because of the NeurIPS policy.\n\nI will raise my score from 3 to 4, but I believe this submission would strongly benefit from a second round of review for its writing. Two of the reviewers agree that writing should be improved, while the other two appreciate the structure. For this reason, I do not intend to raise my score to 5.", "rebuttal": "We sincerely thank you for your thoughtful and constructive feedback. Below, we address each of your comments in turn. If you need to check the technical appendix, it can be found in the submitted supplementary materials.\n\n---\n### **[W1-1 & Q1] Further detail is required for the implementation details of PCA**\n\nThe reviewer questioned implementation details of PCA. PCA is recomputed at each diffusion timestep and each block, we take the current token‑embedding matrix $e_b$ of joint‑attention block $b$, perform SVD, and apply Token Spacing / Residual Alignment before attention is executed. Because nothing is pre‑computed over an external corpus, the decomposition only reflects the block‑ and timestep‑specific embeddings.  Consequently, each transformer block receives its own PCA decomposition, and no basis is shared between blocks.\n\nWe will revise our paper as *\"We perform PCA **separately** for every block at every timestep; the resulting bases are **not shared** across blocks or timesteps.\"* Together, we will insert the following sentence after `line 174`.\n> *\"At each timestep and each block, we perform PCA on the $e^b$ of each joint‑attention block $b$ before applying Token Spacing; this intervention is uniformly applied across all blocks and timesteps, and we denote $e^b$ simply as e for brevity.\"*\n\n---\n### **[W1-2] Concerns about the paper's readability for the first reader.**\n\nThe reviewer raises a concern about the paper's readability on a first pass. Unfortunately, due to the NeurIPS 2025 rebuttal policy, re-uploading a revised version is not permitted. Therefore, we will illustrate the structure of the updated manuscripts with examples.\n\nIn the current manuscript, `Sec. 2` presents our analysis, while `Sec. 3` introduces the method derived from this analysis. In the revised version, we will adopt the structure you suggested by:\n\n- introducing a Sec. 2: Background,\n- moving our empirical findings to Sec. 3: Analyses, and\n- presenting the technical approach in Sec. 4: Methodology.\n\nWe will also adjust the section titles to the more intuitive *Background*, *Analyses*, and *Method* so that readers can immediately distinguish prior work from our novel contributions, thereby improving readability for those unfamiliar with techniques similar to ours.\n\n---\n###  **[W1-3] Lack of clarified explanation for the concepts of anisotropy and isotropy.**\n\nWe fully understand the source of confusion. As we thought, the reviewer's counter‑intuitive impression arises because two seemingly opposing phenomena coexist in transformer text‑embedding spaces and the peculiar property of contextual embedding spaces [1,2,3,4]:\n\n- **Global Anisotropy.** Contextual embeddings tend to collapse into a narrow hyper‑cone, a property reported by Cai et al. (2021) [1]. This geometry yields high cosine similarity between almost any pair of vectors, even when they are semantically unrelated.\n- **Local Isotropy.** To measure local isotropy, clustering methods are generally based on calculating a specific distance metric (typically Euclidean distance for K-means or Mahalanobis distance for Gaussian Mixture Models), rather than cosine similarity.\nAlso, in each cluster, the cluster‑wise mean is subtracted. This centering step removes the dominant global direction. Within each cluster, the directions diversify, so pair‑wise cosines drop toward a lower value. This procedure is a conceptual overview of `Equation 1` in our main manuscript.\n\nIt is precisely because our clustering relies on distance metric which capture absolute proximity, **not angular alignment**. In short, the embedding space resembles a single hyper‑cone at the global level, yet becomes isotropic once examined from the “center of each cluster”. It is also mentioned in `line: 123` of our manuscript.\n\nFor your better understanding, we use a celestial analogy to explain this concept. Global anisotropy is analogous to the cosmic web, where these galaxies align along the same filaments share a common axis. This alignment causes even semantically unrelated vectors (stars) in the embedding space (galaxies) to have uniformly high cosine similarity.\n\nWithin any single galaxy, however, the situation changes. If we treat the solar system as one cluster and set the Earth at its centroid, the planets around us appear in every direction. This is our local isotropy.\n\nLastly, of course, we will revise the manuscript to make this distinction clear.\n\n*[1] Cai, Xingyu, et al. \"Isotropy in the contextual embedding space: Clusters and manifolds.\" ICLR 2021*\n\n*[2] William Rudman and Carsten Eickhoff. Stable anisotropic regularization. ICLR 2024*\n\n*[3] Kawin Ethayarajh. How contextual are contextualized word representations? comparing the geometry of bert, elmo, and gpt-2 embeddings. EMNLP 2019*\n\n*[4] Sara Rajaee and Mohammad Taher Pilehvar. A cluster-based approach for improving isotropy in contextual embedding space. ACL 2021*\n\n---\n\n### **[W2] Requiring baseline comparison with fine-tuned and optimization methods**\n\nBecause RareBench is designed around prompts the model has barely seen during pre-training in the open-sourced MM-DiT models, each prompt is, by definition, *rare*. Finetuning on these prompts would defeat the justification of the benchmark, and the absence of *ground‑truth images* makes supervised fine‑tuning infeasible. If there is a fair evaluation protocol, we will gladly adopt it.\n\nIn contrast, benchmarks such as GenEval and T2I‑CompBench contain more conventional prompts, allowing direct comparisons with finetuned or inference‑optimized models. The table below reports ToRA’s performance alongside these state‑of‑the‑art baselines.\n\nFinally, we emphasize that **ToRA can be applied to any MM-DiT‑based model, including all finetuned or inference‑optimized variants**, highlighting a key contribution of our work.\n\n- *GenEval*\n\n|Models|Single Obj.|Two Obj.|Count|Colors|Position|Attr. Bindings|\n|--|--|--|--|--|--|--|\n|**MM-DiT w/ Optimization**|\n|$\\text{CaPO}_\\text{SD3.0}$ [1]|0.99|0.87|0.63|0.86|0.31|0.59|\n|$\\text{RankDPO}_\\text{SD3.0}$ [2]|1.00|0.90|0.72|0.87|0.31|0.66|\n|$\\text{GRAT}_\\text{FLUX-dev}$ [3]|0.98|0.84|0.48|0.75|0.26|0.37|\n|**Ours (ToRA)**|\n|$\\text{Ours}_\\text{SD3.0}$|0.98|0.77|0.66|0.70|0.50|0.61|\n|$\\text{Ours}_\\text{FLUX-schnell}$|0.99|0.84|0.77|0.85|0.47|0.55|\n\n- *T2ICompbench*\n\n|Models|Color|Shape|Texture|Spatial|Non-Spaital|Complex|\n|--|--|--|--|--|--|--|\n|**MM-DiT w/ Finetuning**| \n|$\\text{CoT-Diff}_\\text{FLUX-schnell}$ [4]|78.1|61.1|69.1|55.8|31.3|50.0|\n|**MM-DiT w/ Optimization**|\n|$\\text{CaPO}_\\text{SD3.0}$ [1]|78.8|57.2|73.1|23.0|31.3|50.9| \n|$\\text{RankDPO}_\\text{SD3.0}$ [2]|83.3|63.5|78.7|36.5|31.3|48.7| \n|$\\text{RENO}_\\text{FLUX-schnell}$ [5]|80.0|60.0|75.0|-|-|-| \n|**Ours (ToRA)**|\n|$\\text{Ours}_\\text{SD3.0}$|84.5|64.2|82.0|45.5|33.8|48.1| \n|$\\text{Ours}_\\text{FLUX-schnell}$|84.6|65.7|81.2|46.0|35.8|47.7| \n\n*[1] Lee, Kyungmin, et al. \"Calibrated multi-preference optimization for aligning diffusion models.\", CVPR 2025.*\n\n*[2] Karthik, Shyamgopal, et al. \"Scalable ranked preference optimization for text-to-image generation.\" arXiv 2024.*  \n\n*[3] Ren, Sucheng, et al. \"Grouping First, Attending Smartly: Training-Free Acceleration for Diffusion Transformers.\" arXiv 2025.*  \n\n*[4] Liu, Zheyuan, et al. \"CoT-lized Diffusion: Let's Reinforce T2I Generation Step-by-step.\" arXiv:2507.04451 (2025).*  \n\n*[5] Eyring, Luca, et al. \"Reno: Enhancing one-step text-to-image models through reward-based noise optimization.\" NeurIPS 2024*\n\n---\n### **[W3] Unclear figure quality (e.g., `Fig. 2b`)**\n\nIn `Fig. 2b`, the *rotating arrow* was meant only as a simple way to show the latent‑vector update loop at each timestep. As the icon might be unclear to readers, we’ll replace it with the label *''Iterative update over timesteps $t = 1 \\dots T$''* and also add this to the caption for clarity. We are happy to incorporate any further revisions you may suggest.\n\n---\n### **[Q2] Questions on claimed low overhead.**\n\nThe reviewer pointed out the absence of a detailed computational cost analysis. As shown in `Appendix D.1` and `Table 4`, we provide a comprehensive comparison. For convenience, herein, we imported the key results comparing our method against the baseline (SD 3.0) and the R2F method below:\n\n|Models|Peak Memory (GB)|GPU Time (sec)|\n|--|--|--|\n|SD3.0|$22.02$|$21.30 \\pm 1.1$|\n|R2F|$38.49$|$78.13 \\pm 2.36$|\n|Ours|$22.17$|$44.74 \\pm 1.9$|\n\nAs this results illustrates:\n- Memory: TORA's peak vRAM usage is only marginally higher than the baseline, representing an increase of less than 0.1% ($\\sim×1.007$).\n- Inference Time: While our method does increase GPU time relative to the baseline, it is substantially more efficient than R2F ($44.74\\ sec$ vs. $78.13\\ sec$).\n\nTo improve visibility of these results, we will also summarize the relevant analysis in the main text. We believe our validation is thorough, but we welcome any suggestions for further constructive experiments.\n\n---\n### **[Q3] Necessity of failure mode analysis**\n\nWe appreciate the reviewer’s comment for the detailed failure mode analysis. Due to policy restrictions on external links, we offer a descriptive account of observed failure cases instead:\n\n> **Case 1:** As noted in our limitations in `Sec. G`, our method can struggle with long prompts where an entity is mentioned multiple times (e.g., with a pronoun). This issue can be exacerbated as our method's increased variance in the representation space may weaken the semantic link between these words.\n\n> **Case 2:** As discussed in `Sec. 2.5`, scaling up variance to better distinguish between tokens can occasionally misalign embeddings from their intended semantic direction. Our Residual Alignment strategy significantly mitigates this (`Fig. 13`), but cannot entirely eliminate it.\n\nWe will add a visualization of these cases to the appendix in the revised version to further clarify these points."}]

Author rebuttal:

Meta-review: This work introduces TORA, a simple yet effective method to enhance rare semantics inside MM-DiT. Without additional training, denoising-time optimization, or reliance on external modules, TORA expands text token embeddings’ representational basins via variance scale-up before joint-attention blocks. Promising results are demonstrated across text-to-vision tasks, including text-to-image, text-to-video, and text-driven image editing.

Initially, the reviewers raised several concerns, which are briefly outlined below:

* Reviewer KirX: The generalization to other problems.

* Reviewer bpoM: More ablation studies, and PCA details.

* Reviewer 41Hj: Performance on common prompts, and ablation on k.

* Reviewer VqR3: Writing issue, method details, baselines, compute overhead, and failure mode discussion.

The rebuttal and subsequent author-reviewer discussions effectively addressed most of the reviewers' concerns. After carefully considering the reviews, rebuttal, and discussion, the AC concurs with the reviewers’ assessment and thus recommends acceptance of the paper. Finally, the authors are encouraged to incorporate the rebuttal experiments into the manuscript and address the reviewers’ feedback (e.g., the writing refinement) in the final revision.
Decision: Accept (poster)