Representation Entanglement for Generation: Training Diffusion Transformers Is Much Easier Than You Think
Authors: Ge Wu
Shen Zhang
Ruijing Shi
Shanghua Gao
Zhenyuan Chen
Lei Wang
Zhaowei Chen
Hongcheng Gao
Yao Tang
jian Yang
Ming-Ming Cheng
Xiang Li
Venue: NeurIPS 2025 oral ()
Keywords: Diffusion Model Acceleration; Representation Entanglement; Diffusion Transformers
Primary area: deep_learning

Abstract: REPA and its variants effectively mitigate training challenges in diffusion models by incorporating external visual representations from pretrained models, through alignment between the noisy hidden projections of denoising networks and foundational clean image representations. We argue that the external alignment, which is absent during the entire denoising inference process, falls short of fully harnessing the potential of discriminative representations. In this work, we propose a straightforward method called $\textit{$\textbf{R}$epresentation $\textbf{E}$ntanglement for $\textbf{G}$eneration}$ ($\textbf{REG}$), which entangles low-level image latents with a single high-level class token from pretrained foundation models for denoising. 
REG acquires the capability to produce coherent image-class pairs directly from pure noise, substantially improving both generation quality and training efficiency.
This is accomplished with negligible additional inference overhead, requiring only one single additional token for denoising (<0.5\% increase in FLOPs and latency).
The inference process concurrently reconstructs both image latents and their corresponding global semantics, where the acquired semantic knowledge actively guides and enhances the image generation process.
On ImageNet 256$\times$256, SiT-XL/2 + REG demonstrates remarkable convergence acceleration, achieving $\textbf{63}\times$ and $\textbf{23}\times$ faster training than SiT-XL/2 and SiT-XL/2 + REPA, respectively. 
More impressively, SiT-L/2 + REG trained for merely 400K iterations outperforms SiT-XL/2 + REPA trained for 4M iterations ($\textbf{10}\times$ longer). Code is available at: https://github.com/Martinser/REG.

Representation Entanglement for Generation: Training Diffusion Transformers Is Much Easier Than You Think
Ge Wul
Zhenyuan Chen 1.2 Ming-Ming Cheng NKKIAI,, Shenzhen Futian, VCIP, CS, Nankai University, Moo Technology, Harvard University,
Shen Zhang*
Ruijing Shi
Shanghua Gao
Lei Wang 1.2
Zhaowei Chen
Hongcheng Gao
Yao Tang²
Jian Yang²
Xiang Li
University of Chinese Academy of Sciences
Figure 1: Representation Entanglement for Generation demonstrates excellent image quality.
Abstract
REPA and its variants effectively mitigate training challenges in diffusion models by incorporating external visual representations from pretrained models, through alignment between the noisy hidden projections of denoising networks and founda- tional clean image representations. We argue that the external alignment, which is absent during the entire denoising inference process, falls short of fully harnessing the potential of discriminative representations. In this work, we propose straight- forward method called Representation Entanglement for Generation (REG), which entangles low-level image latents with single high-level class token from pre- trained foundation models for denoising. REG acquires the capability to produce coherent image-class pairs directly from pure noise, substantially improving both generation quality and training efficiency. This is accomplished with negligible additional inference overhead, requiring only single additional token for denois- ing (<0.5% increase FLOPs and latency). The inference process concurrently reconstructs both image latents and their corresponding global semantics, where the acquired semantic knowledge actively guides and enhances the image genera- tion process. On ImageNet 256x256 SiT-XL/2 REG demonstrates remarkable convergence acceleration, achieving 63x and 23x faster training than SiT-XL/2 and SiT-XL/2 REPA, respectively. More impressively, SiT-L/2 + REG trained for merely 400K iterations outperforms SiT-XI/2 + REPA trained for 4M iterations (10x longer). Code is available at: https: https:///tt t/gpttbg.ttttttt/// thub. comm/arrtinser/RB.....
"ork is done when Ge Wu is an intern at JIIOV Technology.
1 CCorespondiig author.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

feature
soised class taken
class token
feature
SiT Blocks
C randos noise
Blocks
96
SiT-XL/2
O SET-XL/2
Alignment
Aligmenn
O SiT-XL/2 REG
G00
DMOM
20
Pretrained Visual Encoders
Pretrained Visual Encoders
Blocks
63, x faster
Blocks
Entanglerent
8:3
23 x faster
4.0
Set
110c
170r
2500
4080
(a) REPA Training
(c) REG Training (Ours)
(e) Training Steps FID-50K
estinated class token
SAT
SAT
SiT
tirestep 1
tirestep N
timestep
timestop N
(b) REPA Inference
(d) REG Inference (Ours)
Figure 2: Comparison between REPA and our Representation Entanglement for Generation
(REG). (a) During training, REPA [1] indirectly aligns the intermediate denoising features of SiT [2] with pretrained foundation model representations during training. (b) The external alignment of REPA is absent during actual denoising inference, limiting the effectiveness of discriminative information. (c) REG entangles low-level image latents with pretrained foundation model's 's class token in training, providing discriminative semantic guidance to SiT. (d) REG's inference process jointly reconstructs both image latents and their associated global semantics from random noise initialization. The incorporated semantic knowledge continues to guide generation, actively enhancing image quality during inference. (e) On ImageNet 256x256 SiT-XL/2 + REG achieves substantial acceleration in convergence, training 63x x and 23x faster than SiT-XL/2 and SiT-XL/2 + REPA, respectively.
Introduction
Generative models have undergone significant evolution [3, 4, 5, 7]. demonstrating remarkable success across diverse applications [8, 9. 10, 11, 12, 13, 14]. Recent progress in high-fieelity image
synthesis has been driven by several key innovations: Latent Diffusson Models (LDM) [15] introduced
a stable two-phase training framework, while Diffusion Transformers (DiT) [16] enhanced scalability through transformer-ben based architectures. Building upon these developments, Scalable Interpolant Transformers (SiT) [2] further unified the approach through continuous- stochastic interpolants for diffusion training. Despite these advances, achieving high-fidelity synthesis remains substantial resource for model convergence. While recent techniques such as masked training [17, 18] and multi- scale optimization [19] partially alleviate computational costs and accelerate model convergence. However, fundamental optimization challenges persist when relying solely on architecture changes.
Recent studies demonstrate that enhanced generative models can acquire more discriminative repre- sentations, positioning them as capable representation learners [20, 1, 21J. However, as quantified by CKNNA metries [22], these features still underperform compared those from pretrained vi- sion models [23, 24, 25]. This performance gap has motivated approaches leveraging pretrained visual encoder features to accelerate generative model training convergence. For example, REPA [1] employs implicit feature-space alignment between diffusion models and foundation vision models (see Fig. 2(a)), while REPA-E [21] extends this alignment by enabling end-to-end VAE tuning, and quantitatively demonstrates that enhanced alignment (via increased CKNNA scores directly improves generation fidelity.) However, the external alignment of REPA, which is absent during the entire denoising inference process, falls short of fully harnessing the potential of discriminative information

---

(see Fig. 2(b)). We suggest this structure likely impedes further advancements in discriminative semantic learning and overall generative capability.
To address these limitations, we propose a straightforward method called Representation Entanglement for Generation (REG), efficient framework that unleashes the potential of discrimi- native information through explicitly rellows discriminative information into the generation process (see Fig. (())). REG entangles low-level image latents with single high-level class token from pretrained foundation models during training by applying synchronized noise injection to both of them with spatial concatenation. The denoising inference process concurrently reconstructs both image latents and their corresponding global semantics from random noise initialization, where the acquired semantic knowledge actively guides and enhances the image generation process (see Fig. 2(d)). REG achieves significant improvements in generation quality, training convergence speed, and discriminative semantic learning, all while introducing minimal computational cost through the addition of just one token (less than 0.5% FLOPs and latency in Tab. 4). On class-conditional ImageNet benchmarks at 256 resolution (see Fig. 2())). SiT-XL/2 + REG achieves 63 and 23x faster training convergence compared to SiT-XL/2 and SiT-XL/2 + REPA, respectively. Notably, SiT-L/2 + REPA trained for 400K iterations surpasses the performance of SiT-XL/2 + REPA trained for 4M iterations (see Tab. 1).
In summary, our specific contributions are as follows:
We propose REG. an efficient framework that entangles low-level image latents with single high-level class token from pretrained foundation models for denoising.
REG significantly enhances generation quality, training convergence speed, and discrimina- tive semantic learning while introducing negligible computational overhead.
On ImageNet generation benchmarks, REG achieves 63x and 23x faster training conver- gence than SiT and REPA.
2
Related work
Generative models for image generation. Traditional approaches such as DDPM [5] and DDIM [26] perform iterative noise removal in pixel space, while LDM [15] operates in compressed latent spaces through pretrained autoencoders. Architecturally, early U-Net-based diffusion models [5. 27, 15] rely on iterative denoising, whereas modern transformer-bast based frameworks like DiT [16] and SiT [2] leverage self-attention mechanisms for superior spatial pattern modeling. Despite these advances, existing methods typically require extensive training iterations to achieve convergence. Current acceleration techniques often necessitate significant architectural modifications, such as masked training paradigms [17, 18] or multi-scale optimization strategies [28, 19]. In contrast, we propose REG, which achieves dual improvements in generation quality and training efficiency while introduc- ing minimal inference overhead (requiring just one additional token during denoising). Crucially, REG accomplishes these gains while preserving the original model architecture, demonstrating that superior training dynamics can be achieved without structural compromises.
Generative models as representation learners. Extensive research has established that interme- diate features in diffusion models inherently encode rich semantic representations [1, 21], with demonstrated discriminative capabilities across diverse vision tasks including semantic segmen- tation [29, 30, 31], depth estimation [32], and controllable image editing [33, 34, 35]. Recent advancements have further developed knowledge transfer paradigms from diffusion models to efficient networks through techniques like RepFusion's 's dynamic timestep optimization [36] and DreamTeacher's s cross-men model feature distillation [37]. Notably, DDAE [20] confirms that improved diffusion models yield higher-quality representations, establishing direct correlation between gener- ation capability and representation learning performance. Building upon these insights, we propose to systematically integrate discriminative representations into the generative forward process, enabling persistent discriminative guidance throughout denoising inference.
Generative models with external representations. Prior research [38, 39,40] has explored augment- ing diffusion models through auxiliary components. For example, RCG [41] employs a secondary diffusion model to generate the class token for N-condiicni [42] in unconditional generation. In contrast, our approach eliminates the need for additional models by leveraging single class token as part of the input to provide discriminative guidance, simultancously enhancing both dis-
3

---

criminative semantic learning and conditional generation performance. Recent advancements have incorporated visual representations from foundation models accelerate diffusion training. REPA [1] improves semantic representation quality through feature alignment between early diffusion layers and pretrained vision features, while REPA-E [21] extends this framework by enabling end-to-end VAE tuning. However, these methods rely on external alignment mechanisms that do not take the discriminative representations as the input and denoising, which are unable to produce discriminative representations during inference to guide the generation process. Our proposed REG framework structurally integrates spatial visual representations with semantic class embeddings derived from foundation models. This architectural design enables the denoising phase to concurrently refine local- ized pattern restoration and holistic conceptual representation, thereby establishing context-aware semantic steering that persists throughout the entire generative process.
3 Method
We propose REG, an efficient framework that provides discriminative guidance by entangling image latents with foundation model class token (Fig. 2(c, d)). Section 3.1 covers preliminaries, followed by REG's ' detailed description in Section 3.2.
3.1 Preliminaries
Our work is based Scalable Interpolant Transformers (SiT) [2], which provide unified perspective to understand flow and diffusion models. We first introduce the relevant preliminaries. Flow and diffusion models both leverage stochastic processes gradually transform Gaussian noise e ~ ((, 7) into data samples x., This process can be unified as
X oxX, + O2C,
(1)
where ae is decreasing and 0y increasing function of time t. Flow-based models typically interpolate between noise and data over finite interval, while diffusion models define a forward stochastic differential equation (SDE) that converges to Gaussian distribution as t -
Sampling from these models can be achieved via either reverse-time SDE or a probability flow ordinary differential equation (ODE), both of which yield the same marginal distributions for x,. The probability flow ODE is:
*; v(x,,t) f),
(2)
where the velocity field v(x, t) can be formulated by the conditional expectation:
v(x () E[x; X, = x] = d E(x X, x] G, x; = =x].
To synthesize data, we integrate Eqn. (3) in reverse time, initializing from random Gaussian noise e ~ (0, I). This process yields samples from (x(),, serving as an approximation to the true data distribution p(x). This velocity be estimated by model vo(x,, t), which is trained minimize the following loss function:
Ex. ex..e.:[llvooxx (llvo(xx,)) - áxx 6.
The reverse-time SDE can describe the probability distribution pe (x)) of x, at time f, which can be expressed as:
dx, v(x,, wxs((xe - l)dl windwe
(5)
with s(x, t) denoting the score that can be computed via the conditional expectation:
s(x,, t) -,, Elee| x x].
(6)
The score can be reformulated in terms of the velocity v(x, 1):
s(x,t) o. dev(x, 1) - 0x ay²; âid,
We can learn the velocity field v(x,1) and use to compute the score s(x, 1) when using an SDE for sampling.

---

3.2 Representation entanglement for generation
REG training process. Given the clean input image x., we obtain image latents z, E R Dc via VAE encoder [15] and image feature f, E R xx x from vision foundation encoder (e.g. DI- NOv2 [23]]. where Cg x C₂ denotes the latent spatial resolution. and D₂ is the channel dimension. Besides, N represents the number of visual tokens, and Dos is the embedding dimension of vision foundation encoder. In REPA, the absence of the ability to autonomously generate discriminative representations to guide generation in inference may reduce the leverage of discriminative informa- tion effectively. We introduce the class token cls, E R from the vision foundation model to entangle with image latents for providing the discriminative guidance. Here are the specific details;
We inject noise into both the class token and image latents as a paired input for the SiT forward process [2]. Specifically, given two Gaussian noise samples ~ (0, I) and Eals ~N I) with sizes R. P_ xC, xC, and I. Pes respectively, we perform interpolation operations at continuous time
t E [0, 1] as follows:
24 ===0 022, + 04b2; cls, afcls, + offels:
This defines intermediate states 2; (noised latents) and cls, (noised class token) in the forward diffusion process, where 0g and d, control the generation trajectory. Then, we patchify into zi R NxD' D_ is the embedding dimension. Afterwards, the class token cls, is projected into the same embedding space via linear layer obtain cls; E REID= Finally, we concatenate them to form h, [cls/; z E TR (R(++))x'' which serves as the input to the subsequent SiT blocks. We perform alignment at specific transformer layers n. where n = 4 for SiT-B/2 REG and n = 8 for all other variants, maintaining consistency with REPA. Specifically,
align the projected hidden state feature (H((")) E IR (N) with the reference representation y. R ( which is concatenated by cls, and f.. h!"l R(N+1xxD denotes output of the n-th SiT block, ha is a trainable MLP projection, and sim(..) represents the cosine similarity. The alignment loss is defined as:
CREPA (0, == -x,e,
sim ho (h
(h,")))
In addition to alignment, the training objective includes velocity prediction for both the noised image latents Z₁ and class token cls,. The prediction loss is formulated as:
Ppeed Ex.,c,t [|v (z.. d.z, o,e2 2| + 3 ||v (cls,, t) - álc.s. orEcss1?)
(10)
where v (-, t) is the velocity prediction function, and / >> 0 controls the relative weighting between the image latents and class token denoising objectives. The final training loss integrates both prediction and alignment objectives, where X > 0 governs the relative weight of the alignment loss compared to the denoising loss. Specifically, the total loss Lioul is formulated as:
Listal Spred ACREPA
(11)
REG inference process. The framework requires no auxiliary networks to generate the class token. REG jointly reconstructs both image latents and corresponding global semantics from random noise initialization. It acquired semantic knowledge actively guides and enhances the generation quality.
In general, REG demonstrates three key advantages over existing approaches: (1) Improved uti- lization of discriminative information. REG directly integrates discriminative information as part of the input during training, enabling both autonomous generation and consistent application of semantic guidance during inference. This design aims to address a limitation of REPA, which cannot autonomously generate discriminative representations to guide generation in inference. Because relies on an external alignment mechanism during training to utilize discriminative features, rather than incorporating them as input and applying the corresponding denoising task. (2) Minimal computational overhead. This design introduces only single global class token, providing efficient and effective discriminative guidance while incurring an almost negligible computational overhead of less than 0.5% FLOPs and latency at 256x 256 resolution (see Tab. 4). (3) Enhanced performance across metrics. REG improves superior performance generation fidelity, accelerating training convergence, and discriminative semantic learning. As shown in Fig. 2(e), REG achieves up to 23 and 63x faster FID convergence than REPA and SiT, significantly reducing training time. Fig. further shows consistently higher CKNNA scores across training steps, network layers, and timesteps.
5

---

Table FID comparison across train- ing iterations for accelerated align- ment methods. All experiments are con- ducted on ImageNet x56x256 without classifier-free guidance (CFG).
Table Comparison of the performance different methods on ImageNet 256x256 with CFG. Perfor- mance metrics are annotated with (higher better) and + (lower is better).
Method
Epochs
FIDI
sFIDL
IST
Pre.7
Rec.1
Method
#Params
Iter.
FIDL
Pixel diffusion
SiT-B/2
130M
400K
33.0
ADM-U [43]
400
3.94
6.14
186.7
0.82
0.52
REPA
130M
400K
24.4
VDM+- [44]
560
2.40
225.3
REG (ours)
132M
400K
15.2
Simple diffusion [45]
800
2.77
211.8
CDM [46]
2160
4.88
158.7
SiT-L/2
458M
400K
18.8
Latent diffasion, U-Net
+ REPA
458M
400K
9.7
1.DM-4[1]]
200
3.60
247.7
0.87
0.48
REPA
458M
700K
8.4
Latent diffasion, Transformer U.Ner hybrid
REG (ours)
460M
100K
U-VIT-H/2 [47]
240
2.29
5.68
263.9
0.82
0.57
REG (ours)
460M
400K
4.6
DifET [48]
276.5
0.80
SiT-XL//
675M
400K
17.2
2L
1080
1.58
4.52
314.7
0.79
0.65
SiT-XL/2
675M
7M
8.3
Latent diffassion, Transformer
REPA
675M
200K
11.1
MaskDIT [49]
1600
2.28
5.67
276.6
0.80
0.61
REPA
675M
400K
7.9
SD-DiT [50]
REPA
675M
IM
6.4
DIT-XIJ2 [16]
1400
2.27
4.60
278.2
0.83
0.57
REPA
675M
4M
5.9
SIT-XL/2 [2]
1400
2.06
4.50
270.3
0.82
0.59
REG (ours)
677M
200K
5.0
REPA
4.70
305.7
0.80
REG (ours)
677M
400K
3.4
REG (ours)
1.86
4.49
321.4
0.76
0.63
REG (ours)
677M
1M
2.7
REG (ours)
160
1.59
4.36
304.6
0.77
0.65
REG (ours)
677M
2.4M
2.2
REG (ours)
4.24
296.9
0.77
+ REG (ours)
677M
4M
1.8
REGG
4.25
299.4
0.77
4
Experiments
In this section, investigate three key research questions to evaluate the effectiveness and scalability of REG through comprehensive experimentation:
Model performance. Can REG simultaneously accelerate training convergence and enhance generation quality? (Sec. 4.2)
Ablation analysis. Verify the effectiveness of the REG different designs and hyperparameters. (Sec. 4.3)
Discriminative semanties. Can REG improve the discriminative semantics of generative models? (Sec. 4.4)
4.1 Setup
Implementation details.
We adhere strictly to the standard training protocols of SiT [2] and REPA [1]. Experiments are conducted the ImageNet dataset [51], with all images preprocessed to 256x resolution via center cropping and resizing. following the ADM framework [43]. Each image is encoded into a latent representation using the Stable Diffusion VAE [15]. Model architectures B/2. L/2, and XL/2 (with 2 x 2 patch processing) follow the SiT specifications [2]. For comparability, we fix the training batch size 256 and adopt identical learning rates and Exponential Moving Average (EMA) configurations as REPA [1]. Additional implementation details are provided in the Appendix.
Evaluation protocol.
To comprehensively evaluate image generation quality across multiple dimensions, we employ rigorous set of quantitative metrics including Fréchet Inception Distance (FID) [52] for assessing realism, structural FID (sFID) [53] for evaluating spatial coherence, Inception Score (IS) [54] for measuring class- conditionl diversity, precision (Prec.) for quantifying sample fidelity, and recall (Rec.) [55] for evaluating coverage of the target distribution, computed on standardized set of 50K generated samples to ensure statistical reliability. We further supplement these assessments with CKNNA [22] for analyzing feature-space characteristics. Sampling follows REPA [1], using the SDE Suler-Maruyan solver with 250 steps. Full evaluation protocol details provided in the Appendix.

---

Table Verify the effects of various target representation (Target Repr. [23, 25], the depth of supervision (Depth), and the loss weight (3). Experiments employ SiT-B/2 architectures trained for 400K iterations on ImageNet x56x256, Performance metries (with J/f denoting preferred directions) are computed using an SDE Fuler-Maruyama sampler (NFF=250) without classifier-fre guidance. REPA indicates our local reproduction of the original method's reported results.
Method
Target Repr.
Depth
FDD
sHID
IST
Pre.f
Rec.1
SIT-B/2
33.00
6.46
43.70
0.53
0,63
+ REPA
DINOv2-B
4
24.40
6.40
59.90
0.59
0.65
REPA
DINOv2-B
4
22.38
6.98
66.65
0.59
0.65
CLIP-L
21.30
6.51
70.14
0.61
0.63
DINOv2-B
0.03
15.22
6.89
94.64
0.64
0.63
DINOv2-L
4
0.03
17.36
7.02
89.88
0.63
0.63
DINOv2-B
2
0.03
18.19
6.67
83.96
0.62
0.64
DINOv2-B
4
0.03
15.22
6.89
94.64
0.64
0,63
DINOv2-B
0.03
16.31
7.11
91.72
0.63
0.64
+ REG
DINOv2-B
0.03
17.31
7.23
87.78
0.63
DINOv2-B
0.01
15.76
6.69
93.75
0.66
0.61
DINOv2-B
4
0.02
15.64
6.70
93.86
0.66
0.63
DINOv2-B
4
0.03
15.22
6.69
94.64
0.64
0.63
DINOv2-B
4
0.05
16.28
6.97
92.39
0.64
0.64
DINOv2-B
0.10
18.41
7.44
84.79
0.61
0.64
4.2 Improving the performance of generative models
Accelerating training convergence. Tab. provides detailed comparison between REG, SiT [2], and REPA [1] across multiple model scales on ImageNet 256x 256 without CFG. The proposed REG framework consistently achieves the lowest FID scores while substantially accelerating training across all configurations. For smaller models, SiT-B/2 + REG outperforms B/T-B/2 + REPA by 9.2 FID points and surpasses SiT-L/2 trained for 400K iterations by 3.6 points. In the large-scale models. SiT-L/2 + REG achieves an FID of 4.6 at 400K steps, outperforming both SiT-XL/2 + REPA (4M steps) by 1.3 points and SiT-XL/2 (7M steps) by 3.7 points, while requiring only 10.0% and 5.71% of their respective training costs. Similarly, SiT-XL/2 + REG achieves comparable performance SiT-XL/2 (7M steps) and REPA-XL/2 (4M steps) in just 110K and 170K steps, respectively, demonstrating 63x and 23x faster convergence (see Fig. 2(e)). At 4M steps, REG achieves record-low FID of 1.8, demonstrating superior scalability and efficiency across model sizes.
Comparison with SOTA methods. Tab. 2 presents comprehensive comparison against recent SOTA methods utilizing classifier-free guidance. Our framework achieves competitive performance using the REPA's same guidance interval [56] with significantly reduced training cost. REG matches SiT-XL's quality just 80 epochs (17x faster than SiT-XL's 1400 epochs) and surpasses REPA's 800-epoch performance at 480 epochs, highlighting its superior training efficiency and convergence properties. Additional experiments the Appendix include the results of more training steps, validating the REG's robustness, scalability. and cross-task task generalization.
Computational cost comparison. We compare the computational ef- ficiency of REG and REPA under the same model scale (SiT-XI//2) in Tab. 4. REG introduces only marginal increase in parameter count (+0.30%) and FLOPs (+0.38%) rel- ative to REPA, while maintaining nearly identical latency (6.21s vs. 6818s. +0.49%). Despite the minimal computational overhead, REG yields substantial improve- ments in generation quality, achieving a 56.46% relative reduction FID, alongside $0.50.99 19%
Table 4: Computational cost and performance comparison. This table compares REPA and REG on ImageNet 256x256, detailing model size, FLOPs, sampling steps, latency, and generation quality metrics. REG achieves substantially hetter sample quality with negligible increases in computational cost.
Method
FLOPS]
Latency (s)L
HID
SIT-X1 REPA
675
114.46
6.18
7.90
122,60
677
114.90
6.1
3.44
184.13
SiT-XL/2 REG
(40.30%)
(+0.38%)
(+0.49%)
(456.46%)
(450.19%)
increase in IS. These results demonstrate that REG simultaneously improves generation quality and computational efficiency, highlighting effectiveness as a general-purposs enhancement for generative models.

---

4.3
Ablation Studies
Different discriminative guidance. We systematically investigate the impact of different pretrained vision encoders and their corresponding class tokens target representations in Tab. 3. Among all configurations, DINOv2-B achieves the best performance with the lowest FID (15.22) and highest IS (94.64). Notably, all evaluated target representations consistently surpass the REPA, providing empirical evidence that class tokens derived from self-supervised models enhance generation fidelity,
Alignment depth. As shown in Tab. 3. compare the effects of applying the REPA loss at different network depths. Our analysis reveals that applying the loss in earlier layers yields superior results, which is consistent with REPA's findings. Notably, our method demonstrates consistent improvements over REPA across all configurations, achieving FID reductions ranging from 4. to 7.16 points. We attribute these gains the direct insertion of the class token, which provides discrete global guidance to all layers. This enables adaptive integration of discriminative semantics throughout the network, in contrast to REPA's indirect supervision mechanism, where only selected features are aligned with the target representation. As result, REG allows remaining layers to capture richer high-frequency details than REPA, contributing to the observed improvements.
Effect of B. Tab. 3 systematically evaluates the impact of varying the loss weight 3, which controls the contribution of the class token alignment loss. Among the tested values, B = 0.03 achieves the best overall performance across all evaluation metrics. Consequently, this value was adopted as the default parameter for all subsequent experiments.
Entanglement signal variants. Tab. 5 system- atically evaluates the impact of different entan- glement signals on generation quality through concatenative operation. Concatenating noised latent features with either a learnable token or the average of latent features provides limited improvement, likely due to the lack of rich dis- criminative semantic information In contrast, in- corporating discriminative signals yields sub- stantial gains: averaged DINOv2 features signif- icantly reduce FID to 16.86, while the DINOv2 class token achieves the best performance, low- ering FID by 9.18 and increasing IS to 94.64.
Table 5: Ablation study on different entanglement signals. All experiments are conducted on ImtgeNet 256x256, using SiT-B/2 models trained for 400K iter- ations. This experiment adopts the best configuration from Tab. 3 and focuses solely on the impact of different entanglement signals on generation quality.
Method
FID.
sHID
ISt
SiT-B/2 REPA
24.40
6.40
59.90
one learnable token
23.31
6,48
63.44
avg (latent features)
24.12
6.52
60.78
avg (DINOv2 features)
16.86
6.67
84.91
DDINOv2 class token
15.22
6.69
94.64
These results yield two key insights: (1) high-level discriminative information (class token) sub- stantially enhances generation quality, and (2) the entanglement methodology critically governs performance improvements. The demonstrated efficacy of class token concatenation reveals that global discriminative information effectively regularizes the generative latent space, simultaneously boosting both semantic and output quality while maintaining computational efficiency.
Effectiveness of entanglement alone. Tab. evaluates the impact of incorporating class to- kens from various pretrained self-supervised en- coders into SiT-B/2 without applying represen- tation alignment. The results demonstrate that class token entanglement alone consistently en- hances generation quality, with FID improve- ments ranging from 0.95 to 6.33 points across all variants. Notably, DINOv2-B B delivers optimal performance, achieving 19.18% FID reduction and 35.86% IS improvement compared to the SiT-B/2 baseline. These findings indicate that the model can effectively leverage high-level semantic guidance from the class token, even in the absence of explicit alignment, highlighting the robustness and general utility of class loken-based entanglement for generative modeling.
Table 6: Ablation study on different class token entan- glement [23, 25] without representation alignment. This table investigates the effectiveness of class token entanglement in the absence of explicit representation aligmment. All experiments are conducted on ImageNet 256x 256 at 400K iterations.
Method
Class token
FID
sFID4
IS
SIT-B/2
33.0
6.46
43.70
CLIP-L
32.05
6.76
47.61
+
Entanglement
DINOv2-B BB
26.67
6.88
59.37
DINOv2-L
30.16
6.91
52.86
4.4 Enhancing the discriminative semantic learning of generative models
We systematically measure REG, SiT, and REPA's CKNNA scores across training steps, network layers, and timesteps to assess the discriminative semanties of dense features. For fair comparison, we follow REPA's evaluation protocol: We compute CKNNA scores exclusively between spatially
8

---

8.8
8.6
SIT-XL/2
SIT-XL/2 REPA
50-
511-XL/2
8.7-
0.5
51T-X1/2 RFG
9.6
2,5354
Better Generation
CKMM Score.
CKWNA Score
0..5
6.0728
0.1266
306
9.3
F10
9.3
28
8.2
0.2
18
0.1
4025
0.0
0.40
0.45
0.58
0.55
0.69
9.65
1-0.0
t-0.25
t-9.5
Layen Indox
(a)Correlation: FID & CKNNA Score
(b)Layer-wiss Evolution of CKNNA
(c)Timestep Stability of CKNNA
Figure 3: Analysis of Discriminative Semanties. (a) Correlation between CKNNA and FID across training steps (color-coded by progression). REG demonstrates superior discriminative semantic learning, achieving higher CKNNA scores alongside lower FID values compared to REPA. (b) Layer-wise CKNNA progression at 400K training steps (t=0.5). REG consistently enhances CKNNA across all network layers, indicating robust discriminative semantics learning. (c) Timestep-wise CKNNA variation 400K training. REG improves semantic alignment uniformly throughout the training process, outperforning baselines at all timesteps.
averaged generative model dense features and averaged DINOv2-g dense features, while class token are not involved in calculations. Here are the specific situations:
Training steps analysis. Fig. 3(a) shows the positive correlation between CKNNA and FID scores across training steps at layer (t=0.5). It reveals that both REPA and REG achieve improved semantic alignment (higher CKNNA) with better generation quality (lower FID). Notably, REG consistently outperforms REPA in both metrics throughout training, demonstrating its superior capacity for discriminative semantic learning through discriminative semantics guidance.
Layer-wiss progression. At 400K training steps (I=0.5) in Fig. 3(b), both REG and REPA exhibit similar CKNNA patterns: semantic scores gradually increase until reaching the peak at layer n=8 (where alignment loss is computed), then progressively decrease. Crucially, REG achieves con- sistently higher semantic scores than REPA and SiT across all network layers. This improvement stems from REG's innovation of entangling low-level image latents with high-level class token from pretrained foundation models. Through attention mechanisms, REG effectively propagates these discriminative semantics guide the model in understanding low-level features in early layers, while later layers subsequently focus on predicting high-frequency details.
Timestep robustness, Evaluation of CKNNA layer (400K steps) demonstrates REG's consistent superiority across all timesteps in Fig. 3(c). This robustness confirms its stable, high-level semantic guidance capability throughout the entire noise spectrum, enabling reliable discriminative semantic performance regardless of noise intensity during generation.
Conclusion
This paper presents Representation Entanglement for Generation (REG), a simple and efficient framework that firstly introduces image-class denoising paradigm instead of the current pure image denoising pipeline, which fully unleashes the potential of discriminative gains for generation. REG entangles low-level image latents with single high-level class token from pretrained foundation models, achieved via synchronized noise injection and spatial concatenation. The denoising process simultaneously reconstructs both image latents and corresponding global semantics, enabling active semantic guidance that enhances generation quality while introducing minimal computational cost through the addition of just one token. Extensive experiments demonstrate REG's superior perfor- mance in generation fidelity, accelerating training convergence, and discriminative semantic learning, validating its effectiveness and scalability.

---

Acknowledgements
This research was supported by the Fund of the National Natural Science Foundation of China (Grant No. 62576177), Shenzhen Science and Technology Program (QNXMB2033000100000
JCYJ202506041 184027034, JCYJ202408133 00000811112370444 14237048), the Fundamental Research Funds for the Central Universities 070-63253222, and the Tianjin Key Laboratory of Visual Computing and Intelligent Perception (VCIP). Computation supported by the Supereomputing Center of Nankai University (NKSC). "Science and Technology Yongjiang 2035" key technology breakthrough plan project (20252053), Chinese covernment-guiiee guided local science and technology development fund projects (scientific and technological achievement transfer and transformation projects) (254Z0102G),
National Science Fund of China under Grant No. 066670
References
[1] Sihyun Yu, Sangkyung Kwak, Huiwon Jang, et al. Representation alignment for generation: Training diffusion transformers is easier than you think. In arXiv preprint 100:0:2410.069400, 2024. 2, 3, 4, 6, 7, 14, 15, 16
[2] Nanye Ma, Mark Goldstein, Michael S Albergo, et al. Sit: Exploring flow and diffusion-based generative models with scalable interpolant transformers. ECCV, 2024. 3, 4, 5, 6, 7, 13,
15, 16
[3] Katherine Crowson, Stella Biderman, Daniel Kornis, al. Vggan-clip: Open domain image
generation and editing with natural language guidance. In ECCV, 2022. 2
[4] Keyu Tian, Yi Jiang, Zehuan Yuan, et al. Visual autoregressive modeling: Scalable image generation via sext-scale prediction. In NeurlPS, 2024.
[5] Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. In NeurIPS, 2020. 2.3
[6] Alexander Tong, Kilian Fatras, Nikolay Malkin, et al. Improving and generalizing flow-based generative models with minibatch optimal transport. In arXiv preprint arXiv:2302.00482, 2023.
[7] Aditya Ramesh, Mikhail Pavlov, Gabriel Goh, et al. Zero-shot text-to-image generation. In ICML, 2021. 2
[8] Taihang Hu, Linxuan Li, Joost van de Weijer, et al. Token merging for training-fres semantic binding in text-to-image synthesis. In NeurIPS, 2024.
[9] Mariam Hassan, Sebastian Stapf, Ahmad Rahimi, et al. Gem: generalizable ego-vision multimodal world model. In arXiv preprint arXiv:2412.11198 2024. 2
[10] Hongru Liang, Haozheng Wang, Jun Wang, et al. Jtav: Jointly learning social media content rep-
resentation by fusing textual, acoustic, and visual features. In arXiv preprint arXiv:1806.01488 2018.
[11] Erik Nijkamp, Bo Pang, Hiroaki Hayashi, et al. Codegen: An open large language model for code with multi-turn program synthesis. In arXiv preprint 2203 2022. 2
[12] Shen Zhang, Yaning Tan, Siyuan Liang, Zhaowei Chen, Linze Li, Ge Wu, Yuhao Chen, Shuheng Li, Zhenyu Zhao, Caihua Chen, et al. Ledit: Your length-extrapolataae diffusion transformer without positional encoding. arXiv preprint arXiv:2503.044 00:.:0000000444,, 2025.
[13] Taihang Hu, Linxuan Li, Kai Wang, Yaxing Wang, Jian Yang, and Ming-Min ding-Ming Cheng. Anchor token matching: Implicit structure locking for training-free image editing. arXiv preprint 2504. 10434, 2025.
[14] Tao Liu, Dafeng Zhang, Gengchen Li, Shizhuo Liu, Yongqi Song, Senmao Li, Shiqi Yang, Boqian Li, Kai Wang, and Yaxing Wang. From cradle to cane: two-pass framework for high-fidelity lifespan face aging. arXiv preprint arXiv:2506.20977, 2025. 2
10

---

[15] Robin Rombach, Andreas Blattmann, Dominik Lorenz, et al. High-resolution image synthesis with latent diffusion models. In CVPR, 2022. 2, 6, 13, 15
[16] William Peebles and Saining Xie. Scalable diffusion models with transformers. In ICCV, 2023. 2, 3, 6, 15, 16
[17] Shanghua Gao, Pan Zhou, Ming-Ming Cheng, al. Masked diffusion transformer is strong image synthesizer. In ICCV, 2023. 2,3
[18] Shanghua Gao, Pan Zhou, Ming-Ming Cheng, et al. Mdtv2: Masked diffusion transformer is a strong image synthesizer. arXiv preprint arXiv:2303. 144399993..4,9,, 2023. 2, 3, 6, 15
[19] Qihao Liu, Zhanpeng Zeng, Ju He, et al. Alleviating distortion in image generation via multi- resolution diffusion models. In arXiv preprint arXiv:2406. 16, 2024. 2,3
[20] Weilai Xiang, Hongyu Yang, Di Huang, and Yunhong Wang. Denoising diffusion autoencoders are unified self-supervised learners. In ICCV. 2,
[21] Xingjian Leng, Jaskirat Singh, Yunzhong Hou, et al. Repa-e: Unlocking vae for end-to-end tuning with latent diffusion transformers. In arXiv preprint drXiv:2504.10483, 2025, 2, 3,
[22] Minyoung Huh, Brian Cheung, Tongzhou Wang, et The platonic representation hypothesis. In arXiv preprint arXiv:2405.07988 2024. 2, 6
[23] Maxime Oquab, Timothée Darcet, Théo Moutakanni, et al. Dinov2: Learning robust visual features without supervision. In arXiv preprint arXiv:2304.07193, 2023. 2, 5, 7, 8, 13, 15, 16
[24] Xinlei Chen, Saining Xie, and Kaiming He. An empirical study of training self-supervised vision transformers. In ICCV, 2021. 2
[25] Alec Radford, Jong Wook Kim, Chris Hallacy, et Learning transferable visual models from natural language supervision. In ICML, 2021. 2, 7,
[26] Jiaming Song. Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. In arXiv preprint arXiv:2010.02500, 2020. 3
[27] Alexander Quinn Nichol and Prafulla Dhariwal. Improved denoising diffusion probabilistic models. In ICMI., 2021.
[28] Xizhou Zhu, Xue Yang, Zhaokai Wang, al. Parameter-inverted image pyramid networks. In arXiv preprint 14XX:::2406.4433300 2024. 3
[29] Dmitry Baranchuk, Ivan Rubachev, Andrey Voynov, et al. Label-efficient semantic segmentation with diffusion models. In arXiv preprini 12.01221 2021.
[30] Wenliang Zhao, Yongming Rao, Zuyan Liu, et al. Unleashing text-to-image diffusion models for visual perception. In /CCV, 2023.
[31] Changyao Tian, Chenxin Tao, Jifeng Dai, al. Addp: Learning general representations for image recognition and generation with alternating denoising diffusion process. In arXiv preprint arXiv:2306.05423 2023.
[32] Ming Gui, Johannes Schusterbauer, Ulrich Prestel, et Depthfm: Fast monocular depth estimation with flow matching. In arXiv preprint arXiv:2403. 13788, 2024. 3
[33] Amir Hertz, Ron Mokady, Jay Tenenbaum, et al. crompt-to-prompe image editing with cross attention control. In arXiv preprint arXiv:2208.01622 2022.
[34] Konpat Preechakul, Nattanat Chatthee, Suttisak Wizadwongsa, et al. Diffusion autoencoders:
Toward meaningful and decodable representation. In CVPR, 2022. 3
[35] Zijian Zhang, Zhou Zhao, and Zhijie Lin. Unsupervised representation learning from pre-trained diffusion probabilistic models. In arXiv preprint arXiv:22/1./1036 2022, 3
[36] Xingyi Yang and Xinchao Wang. Diffusion model as representation leamnr. In ICCV, 2023.

---

[37] Daiqing Li, Huan Ling, Amlan Kar, et al. Dreamteacher: Pretraining image backbones with deep generative models. In ICCV. 2023.
[38] Yongxin Zhu, Bocheng Li, Hang Zhang, et al. Stabilize the latent space for image autoregressive modeling: A unified perspective. In arXiv preprint 1xXiv:2410.124900 2024. 3
[39] Lijun Yu, Yong Cheng. Zhiruo Wang, et al. Spae: Semantic pyramid autoencoder for multimodal generation with frozen 1ms.. In NeurlPS, 2023. 3
[40] Jingfeng Yao, Bin Yang. and Xinggang Wang. Reconstruction vs. generation: Taming op- timization dilemma in latent diffusion models, In arXiv preprint arXiv:2501.01423, 2025.
3
[41] Tianhong Li, Dina Katabi, and Kaiming He. Return of unconditional generation: A self- supervised representation generation method. NeurIPS, 2024.
[42] Ethan Perez, Florian Strub, Harm De Vries, et al. Film: Visual reasoning with general conditioning layer. In AAAI, 2018. 3
[43] Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans image synthesis. In NeurlPS, 2021. 6, 15, 16
[44] Diederik Kingma and Ruiqi Gao. Understanding diffusion objectives as the elbo with simple data augmentation. In NeurlPS. 6, 15, 16
[45] Emiel Hoogeboom, Jonathan Heek, and Tim Salimans. Simple diffusion: End-to-end diffusion for high resolution images. In ICML. 6, 15, 16
[46] Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance. In arXiv preprint arXiv:2207.12541 12598. 6, 15
[47] Fan Bao, Shen Nie, Kaiwen Xue, Yue Cao, Chongxuan Li, Hang Su, and Jun Zhu. All are worth words: A vit backbone for diffusion models. In CVPR. 6, 15
[48] Ali Hatamizadeh, Jiaming Song, Guilin Liu, Jan Kautz, and Arash Vahdat. Diffit: Diffusion vision transformers for image generation. In ECCV. 6, 15
[49] Hongkai Zheng, Weili Nie, Arash Vahdat, and Anima Anandkumar. Fast training of diffusion models with masked transformers. In arXiv preprint arXiv:2306.09305. 6, 15, 16
[50] Rui Zhu, Yingwei Pan, Ychao Li, Ting Yao, Zhenglong Sun. Tao Mei, and Chang Wen Chen, Sd-dit: Unleashing the power of self-supervised discrimination in diffusion transformer. In CVPR. 15
[51] Jia Deng, Wei Dong, Richard Socher, et al. Imagenet: A large-scale hierarchical image database.
In CVPR, 2009. 6
[52] Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, et al. Gans trained by two time-scale update rule converge to a local nash equilibrium. In NeurIPS, 2017.
[53] Charlie Nash, Jacob Menick, Sander Dieleman, al. Generating images with sparse representa- tions. In arXiv preprini 10XKv::213.0332: 2021. 6
[54] Tim Salimans, Ian Goodfellow, Wojeiech Zaremba, et Improved techniques for training gans. In NeurlPS, 2016. 6
[55] Tuomas Kynkäänniemi, Tero Karras, Samuli Laine, et al. Improved precision and recall metric for assessing generative models. In NewrIPS, 2019. 6
[56] Tuomas Kynkäänniemi, Miika Aittala, Tero Karras, Samuli Laine, Timo Aila, and Jaakko Lehtinen. Applying guidance in limited interval improves sample and distribution quality in diffusion models. In arXiv preprint arXiv:2404.077224 2024. 7, 14, 16
[57] Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint
11X1991412.6980, 2014. 16
[58] Ulya Loshchilov and Frank Hutter. Decoupled weight decay regularization, arXiv preprint
axXiv:171..55101, 2017. 16
12

---

Representation Entanglement for Generation: Training Diffusion Transformers Is Much Easier Than You Think Supplementary Materials
A Discriminative semantics in inference
We posit that REG's semantic reconstruction capability during inference stems from key design elements: (1) the architectural entanglement of class token with image latents during training, and (2) the consistent application of SiT's [2] velocity prediction loss to both them. Comparative analysis reveals: (I) The One learnable Token (OLT) method (see Fig. 4(a)) concatenates noised latents with a learnable token and only calculates velocity prediction loss Ly on dense features. In contrast, REG (see Fig. 4(c)) entangles one high-level noised class token with low-level noised latent features while computing velocity prediction losses Epred for both components.
b
pred
feature
token
token
M randon noise
learnable token
SiT Blocks
SiT Blocks
Alignnent
Alignnent
SiT Blocks
Pretrained Visual Encoders
Pretrained Visual Encoders
SiT Blocks
Entanglement
(a) OLT Training
(c) REG Training (Ours)
Feu discrininative senantics
Rich discrininative semantics
SiT
SiT
tinostep 1-N
timastep 1-N
(b) OLT Inference
(d) REG Inference (Ours)
Figure 4: Comparison between the One Learnable Token (OLT) method and REG for generation. (a) During training. OLT simply concatenates noised latents with a learnable token while computing velocity prediction loss L, on dense features. (b) OLT's output learnable token demonstrates minimal discriminative semanties after multi-step denoising. (c) REG entangles one high-level noised class token with low-level noised latent features while computing velocity prediction losses Cped for both components. (d) REG can reconstruct the corresponding global semantics of image latents with rich discriminative semantics.
To quantitatively validate the two methods' discriminative semantics in inference, we use 10,000 ImageNet validation images as input processed through identical noise injection via the VAE en- coder [15]. REG's inference integrates noised latents with the noise-initialized class token through concatenation before multi-step denoising (see Fig. 4(d)), while OLT similarly processes noised latents with its learnable token (see Fig. 4(b)). Then, we process these ImageNet validation images through DINOv2 [23] to obtain the reference class token. Fig. 5 computes both CKNNA and cosine
13

---

S1T-B/2 OLT
S1T-B/2 REG
0.514
8.458
0.3
+0.50
0.2
0.1
8.814
Cosine Sinilarity
CKNA Score
Figure 5: Quantitative evaluation of cosine similarity and CKNNA score between OLT's learnable token, REG's class token, and DINOv2-g's reference token. Results demonstrate REG's superior semantic retention during inference compared to OLT.
similarity between the reference class token and the output from REG's class token/OLT's learnable token using SiT-B/2 as backbone. The results demonstrate a significant disparity: OLT's learnable token achieves only 0.007 cosine similarity and 0.014 CKNNA scores, while REG's class token reaches 0.458 and 14514, respectively. This empirical evidence confirms REG's superior capacity to preserve discriminative semanties compared OLT's limited representation capability in inference.
B Analysis of training overhead in REG
We summarize the total training overhead in Tab. 7, reporting the costs required to reach the same performance upper bounds claimed in the original SiT and REPA papers. All experiments are conducted on 8 NVIDIA A40 GPUs. Our results show that REG requires only 110K training steps to reach the performance level of SiT trained for 7M steps, reducing GPU hours by 98.36%. Moreover, compared with REPA, REG achieves the performance of 4M counterpart with only 170K iterations, reducing GPU hours by 95.72%. These results highlight the training efficiency of REG, demonstrating faster convergence and significantly lower training overhead compared to prior methods.
Table 7:
Training overhead comparison. REG achieves comparable performance to other models while significantly reducing training time.
Model
FID|
Training stepl.
All GPU hours]
SIT-XLAX
8.3
7M
2380
+ REG (ours)
8.2
110K
39 (-98.36 %
+ REPA
5.9
4M
1800 (-95.72 %)
+ REG (ours)
5.6
170K
C 256x256 ImageNet
Tab. presents extended training results with CFG using the REPA's same guidance interval [56]. demonstrating REG's excellent performance with a 1.40 FID at 480 epochs; it achieves better performance comparable to REPA [1] at epochs while requiring fewer than 40% the training iterations. In addition, Tab. 9 presents more specific performance details of SiT + REG, further highlighting its superior robustness and accelerated convergence. Tab. presents quantitative performance metrics of SiT-XL REG under varying classifier-free guidance scale w.
14

---

Table 8: Extended REG training on ImageNet 256x256 with CFG demonstrates progressive perfor-
mance gains.
Method
Epochs
FID
sFIDI
IS1
Pre.1
Rec.1
Pixel diffiusion
ADM-U [43]
400
3,94
6.14
186.7
0.82
0.52
VDM++ [44]
560
2.40
225.3
Simple diffusion [45]
800
2.77
211.8
CDM [46]
2160
4.88
158.7
Latent diffusion,, U-Net
LDM4 [15]
200
3.60
247.7
0.87
0.48
Latent diffusion, Transformer U-Net hybrid
U-VIT-H/2 [47]
240
2.29
5.68
263.9
0.82
0.57
D [48]
1.73
276.5
0.80
0.62
MDTv2-XX XL/2 [18]
1080
1.58
314.7
0.79
0.65
Latent diffasion, Transformer
MaskDiT [49]
1600
2.28
5.67
276.6
0.80
0.61
SD-DIT [50]
3.23
DIT-XI/2
1400
2.27
4.60
278.2
0.83
0.57
SiT-XL/2 [2]
1400
2.06
4.50
270.3
0.82
0.59
REPA
800
1.42
4.70
305.7
0.80
0.65
REG (ours)
80
1.86
4.49
321.4
0.76
0.63
REG (ours)
160
1.59
4.36
304.6
0.77
0.65
REG (ours)
300
1.48
4.31
305.8
0.77
0.66
REG (ours)
480
1.40
4.24
296.9
0.77
0.66
REG (ours)
800
1.36
4.25
299.4
0.77
0.66
Table 9: More performance analysis of SiT
REG across model scales without CFG.
Model
#Params
Iter.
FID
sFID.,
ISt
Prec.
Rec.
SiT-B/2 [2]
130M
400K
33.0
6.46
43.7
0.53
0.63
REPA
130M
400K
24.4
6.40
0.59
0.65
REG (ours)
132M
50K
64.7
9.47
23.2
0.40
0.51
REG (ours)
132M
100K
36.1
7.74
45.5
0.53
0.61
REG (ours)
132M
200K
22.1
7.19
72.2
0.60
0.63
REG (ours)
132M
400K
15.2
6.69
94.6
0.64
0.63
SiT-L/2 [2]
458M
400K
18.8
5,29
72.0
0.64
0.64
REPA
458M
400K
10.0
5.20
109.2
0.69
0.65
REG (ours)
460M
50K
30.1
8.92
52.6
0.58
0.57
REG (ours)
460M
100K
11.4
5.36
108.8
0.70
0.60
REG (ours)
460M
200K
6.6
5.16
145.4
0.73
0.63
REG (ours)
460M
400K
4.6
5.21
167.6
0.75
0.63
SiT-XL/2 [2]
675M
7M
8.3
6.32
131.7
0.68
0.67
REPA
675M
4M
5.9
5.73
157.8
0.70
0.69
REG (ours)
677M
50K
26.7
16.49
59.2
0.60
0.54
REG (ours)
677M
100K
8.9
5.50
125.3
0.72
0.59
REG (ours)
677M
200K
5.0
4.88
161.2
0.75
0.62
REG (ours)
677M
400K
3.4
4.87
184.1
0.76
0.64
REG (ours)
677M
IM
4.93
201.8
REG (ours)
677M
2.4M
2.2
4.79
219.1
0.76
0.66
REG (ours)
677M
4M
1.8
4.59
230.8
0.77
0.66
D 512x512 ImageNet
To further validate REG's effectiveness, we conduct experiments at 512x512 resolution following REPA's protocol [1]. The RGB images are processed through the VAE [15] to yield 64x64x3 latents, with DINOv2 [23] providing both dense features and class token from 448x448 inputs. As demonstrated in Tab. 11, REG surpasses the performance of REPA trained for 200 epochs and
SiT-XI-/2 trained for 600 epochs terms of FID at only 80 epochs, demonstrating its superior effectiveness.
15

---

Table 10:
The results of SiT-XL REG 2.4M training iterations under varying classifier-fres
guidance scale w, employing the guidance interval method [56].
Model
#Paarms
Iter.
Interval
w
FID.
sFID
1st
Prec.t
Rec.1
SiT-XL/2 [2]
675M
7M
[0. 1]
1.50
2.06
4.50
270.3
0.82
0,59
REG (ours)
675M
2.4M
[0, 0.8]
2.4
1.45
4.32
280.44
0.77
0.67
REG (ours)
675M
2.4M
[0, 0.85]
2.4
1.41
4.24
299.65
0.77
0.67
REG (ours)
675M
2.4M
[0. 0.9]
2.4
1.61
4.21
334.50
0.79
0.64
REG (ours)
675M
2.4M
[0. 0.85]
2.5
1.43
4.25
303.11
0.77
0.67
REG (ours)
675M
2.4M
[0, 0.85]
2.4
1.41
4.24
299.65
0.77
0.67
REG (ours)
675M
2.4M
[0, 0.85]
2.3
1.40
4.24
296.93
0.77
0.66
REG (ours)
675M
2.4M
[0. 0.85]
2.2
1.40
4.25
293.57
0.77
0.67
Table 11: Performance comparison on ImageNet 512x512 with CFG.
Model
Epochs
FID,
sFID4
ISt
Pre.f
Rec.1
Pixel diffusioo
VDM++ [44]
2.65
278.1
ADM-G, ADM-U [43]
400
2.85
5.86
221.7
0.84
0.53
Simple diffusion (U-Net) [45]
800
4.28
171.0
Simple diffusion (U- VIT, L) [45]
800
4.53
205.3
Latent diffusion, Transformer
MaskDiT [49]
800
2.50
5.10
256.3
0.83
0.56
DIT-XL/2 [16]
600
3.04
5.02
240.8
0.84
0.54
SiT-XL/2 [2]
600
2.62
4.18
252.2
0.84
0.57
REPA
80
2.44
4.21
247.3
0.84
0.56
REPA
100
2.32
4.16
255.7
0.84
0.56
REPA
200
2.08
4.19
274.6
0.83
0.58
REG (ours)
80
1.68
3.87
306.9
0.80
0.63
E
Experimental setup
Hyperparameter setup. Tab. 12 presents the hyperparameter configurations of SiT REG across different model scales. Following REPA'S experimental protocol [1], we employ AdamW [57, 58] optimization with batch size of 10-1 and adopt DINOv2-B [23] as the optimal alignment model, maintaining 250 denoising steps for all inference processes.
CKNNA score. We adopt REPA's CKNNA computation [1]. caleulating scores exclusively between spatially averaged dense features from both the generative model and DINOv2-g representations [23]. To ensure fair comparison, the class token is explicitly excluded from all CKNNA calculations.
Limitations
Due to limitations in computing resources, we plan conduct extended training of REG with additional iterations at higher resolutions and under varied experimental configurations.
G
Broader Impacts
REG provides a principled framework for rethinking discrininative-geeraaiie model integration, demonstrating how strategic utilization of pretrained vision representations systematically enhance diffusion model performance while maintaining computational efficiency.
H More visualization results
We present more visualization results of REG in Fig. 6 25 with CFG (w 4.0).
16

---

Table 12: Hyperparameter settings across different model scales.
Backbone
SiT-B
SiT-L
SiT-XL
Architecture
#Params
132M
460M
677M
Input
32x32xx
32x32x4
32x32x4
Layers
12
24
28
Hidden dim.
768
1,024
1,152
Num. heads
12
16
16
REG settings
0.03
0.03
0.03
0.5
0.5
0.5
Alignment depth simm..-)
8
cOs. sim. DINOv2-B
cOs. sim. DINOv2-B
cos. sim. DINOv2-B
Encoder E
Optimization
Batch size Optimizer
256 AdamW W 0.0001 (0.9, 0.999)
256 Adam W 0.0001 (0.9, 0.999)
256 Adam W 0.0001 (0.9, 0.999)
(3),
Interpolants
1 -
1 -
1 - t dt V-prediction Buler-Maruyama 250
dz Wy Training objective Sampler
V-prediction Euler-Maruyama 250
V-prediction Euler-Maruyama 250
Sampling steps
Figure 6: The visualization results of SiT-XL/2 REG use CFG with w 4.0, and the class label is "Great white shark" (2).
17

---

Figure 7: The visualization results of SiT-XL/2 + REG use CFG with w 4.0, and the class label is
"Bald eagle" (22).
Figure 8: The visualization results SiT-XL/2 + REG use CFG with w 4.0, and the class label is "Great grey owl" (24).
Figure 9: The visualization results of SiT-XL/2 REG use CFG with w 4.0, and the class label is "Macaw" (88).
18

---

Figure 10: The visualization results of SiT-XL/2 + REG use CFG with w 4.0, and the class label is "Suuphur-crested cockatoo" (89).
Figure 11: The visualization results SiT-XL/2 + REG use CFG with w = 4.0, and the class label is "Koala" (105).
Figure 12: The visualization results of SiT-XL/2 + REG use CFG with 4.0, and the class label is "American coot" (137).
19

---

Figure 13: The visualization results of SiT-XL/2 + REG use CFG with w 4.0, and the class label is
"Lesser panda" (156).
Figure 14: The visualization results of SiT-X1-/2 + REG use CFG with wo = 4.0, and the class label is "Border collie" (232).
Figure 15: The visualization results of SIT-XL/2 + REG use CFG with w 4.0, and the class label is "Timber wolf" (269).
20

---

Figure 16: The visualization results of SiT-XL/2 + REG use CFG with w 4.0, and the class label is
"Polecat" (358).
Figure 17: The visualization results SiT-XL/2 + REG use CFG with w = 4.0, and the class label is "Lesser panda" (387).
Figure 18: The visualization results of SIT-XL/2 + REG use CFG with 4.0, and the class label is "Giant panda" (388).
21

---

Figure 19: The visualization results of SiT-XL/2 + REG use CFG with w = 4.0, and the class label is
"Castle" (483).
Figure 20: The visualization results of SiT-X1//2 + REG use CFG with w = 4.0, and the class label is "China cabinet" (495).
Figure 21: The visualization results of SiT-XL/2 + REG use CFG with w 4.0, and the class label is "Convertible" (511).
22

---

Figure 22: The visualization results SiT-XL/2 + REG use CFG with w = 4.0, and the class label
"Bubble" (971).
Figure The visualization results of SiT-XL/2 + REG use CFG with w 4.0, and the class label is "Geyser" (974).
Figure 24: The visualization results of SIT-XL/2 + REG use CFG with 4.0, and the class label is "Lakeside" (975).
23

---

Figure 25: The visualization results of SiT-XL/2 + REG use CFG with w 4.0, and the class label is "Volcano" (980).
NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
Justification: The abstract and introduction clearly outline the paper's scope and contribu- tions, with a strong focus on our key findings and experimental results.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize other settings.
It is fine to include aspirational goals as motivation as long as is clear that these goals attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations of the work performed by the authors?
Answer: [Yes]
Justification:
See the supplementary material.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g. if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.

---

The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't It acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proof's
Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?
Answer: [NA]
Justification:
This paper does not include theoretical proofs.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification:
See the supplemental material and code.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
25

---

While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make clear how to reproduce that algorithm. (b) the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.
(c)
If the contribution is new model (e.g., , a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g., registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions faithfully reproduce the main experimental results, described in supplemental material?
Answer: [Yes]
Justification:
See the supplemental material and code.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips.c cc/
Muuides/CodeSubbrn
wwwooooliioe PoPollcyyy icy for more details.
While encourage the release of code and data, we understand that this might not be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https: / //nips. cccccccppooo http//ubliccceeeee weciiii/guieseseee wtteeeeeeeoooeeeeeiieeeieieeeeee
for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification:
See the supplemental material and code.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented the core the paper to level of detail that is necessary to appreciate the results and make sense of them.
26

---

The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Question:
Answer: [NA]
Justification: This line of experimentation represents distinct research direction from ours.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the enror bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall
with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma eгror bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification:
See the experiment part and supplemental material.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of individual experimental runs as well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't 't make it into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics https://neurri: http:////c.ppppcceiicteettteeee.
Answer: [Yes]
Justification: Yes, our research methodology has been conducted in full accordance with the ethical guidelines established by NeurlPS.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
27

---

If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g., there a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive socictal impacts and negative societal impacts of the work performed?
Answer: [Yes]
Justification: See the supplemental material.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g.. deploymentt of technologics that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied partieular applications, let alone deployments. However, if there is direct path any negative applications, the authors should point it out. For example, it is legitimate to point that improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Question:
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Answer: [NA]
Justification: Null.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
28

---

Answer: [Yes]
Justification: The construction of derivative datasets and subsequent research applications were predicated on obtaining express written consent from all originating data stewards. Each contributing entity formally validated their approval via email correspondence, unequivocally supporting our academic pursuits. These verified permission records, with all personally identifiable information redacted, will be disclosed alongside the published work.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package or dataset.
The authors should state which version of the asset used and, possible, include a URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswi thcode. com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
this information not available online, the authors are encouraged to reach out to the asset's s creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: We have meticulously recorded the complete experimental methodology and manuscript development procedures.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
At submission time, remember to anonymize your assets (if applicable). You can either create anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification: Null.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
29

---

15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country institution) were obtained?
Answer: [NA]
Justification: Null.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with
human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
Declaration of LLM usage
Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology, scientific rigorousness, or originality of the research, declaration is not required.
Question:
Answer: [NA]
Justification: Null.
Guidelines:
The answer NA means that the method development in this research does not involve LLMs as any important, original, or non-standard components.
Please refer to our LLM policy for what should or should be described.
30


Reviews:
[{"review_id": "yVRnkeTQZK", "reviewer": "Reviewer_nn5b", "summary": "The paper proposes a new method for training diffusion models for image synthesis. \n\nA one-dimensional vector, typically obtained from self-supervised vision foundation models, that represents the \"semantic contents\" of the image is chosen and is noised similarly to the latents (from a VAE) corresponding to the image. \n\nThe DiT is then task to denoise (predict the flow field of) both the noisy latents and the noisy representational feature. \n\nThrough this auxiliary token injection into the image sequence and modeling of the task, the authors show that the representation quality and generation quality of the DiT thus obtained is greatly improved. Furthermore, they also show that training convergence is also dramatically accelerated.", "questions": "How is it different than https://arxiv.org/abs/2504.16064v1?\n\nWhat happens to the representation quality in the standalone case (when you don't have the REPA loss)?\n\nI feel like a concrete explanation/investigation into how exactly the extra token and task helps image quality is lacking. It is mentioned that the key limitation of REPA is that it doesn't have some alignment option during inference, but it's not obvious as to why that is nedded or is a limitation in general?\n\nHow do you think the Vision foundational model chosen for the representation affects learning in the REG case compared ot REPA?", "limitations": "Yes", "rating": 3, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "Strengths:\nThorough experimental study.\nGood ablations and analysis.\nWell-written and easy to understand. \n\nWeakness:\nEffectiveness of entanglement alone: While there are results showcasing the effectiveness of entanglement in terms of the generation quality, it's not clear how the representation quality benefits due to the addition of extra token and denoising task. \n\nREPA shows that the representation quality in DIT and performance improves with better vision encoder. This study seems to contradict that with the performance decreasing with bigger encoder (DINOv2-L). This might because of the hyperparam chosen / having the same timestep sampling schedule for the representation part as well as the latents. But there's some missing stuff here. \n\nWhile comparison with respect training iterations is helpful, it's also important to compare against SiT with respect to training FLOPs. REG, like REPA, has an alignment loss which requires additional flops compared to the standard SiT formulation. \n\nEvaluation of the generated representation: Experiments indicating the ability of the DiT to generate an accurate representation is missing.", "was_revised": false, "final_justification": "", "rebuttal": "Thank you very much for your valuable comments and the time you devoted to reviewing our manuscript. \nWe address each of your comments in detail below.\n\n---\n\n### **[W1, Q2]: The effectiveness of the an extra noised class token and the denoising task for it.**\n\nAblation studies (Table 6) evaluate entanglement alone by removing alignment loss (REPA loss), assessing the extra noised class token and its denoising task (velocity prediction loss).\nResults show introducing a noised class token without a matching denoising objective causes a large performance drop (FID 98.67). \nThis is because, without class token denoising, the model cannot autonomously generate a discriminative class token to guide inference.\nMoreover, **there is an inconsistency between training and inference**. \nThis discrepancy disrupts the generation process. In inference, no explicit class token is provided; the diffusion model cannot rely on it as in training. \n\nIn contrast, introducing the noised class token with a dedicated training denoising task yields optimal performance (FID 26.67). \nThis is because class token denoising loss enables the **diffusion model to autonomously generate a semantically discriminative class token during inference.**\nThe generated class token can effectively guide the generation of other tokens. Moreover, this setup ensures consistency between training and inference. \n\n\n\n| Method  | Class token | Velocity prediction loss for class token | FID↓  | sFID↓ | IS↑   |\n|:-|:---|:-|:-|:------|:------|\n| SiT-B/2 | ×           | ×                                        | 33.0  | 6.46  | 43.70 |\n| _+ Entanglement_ |\n| DINOv2-B | ✓           | ×                                        | 98.67 | 9.80  | 12.36 |\n| DINOv2-B | ✓           | ✓                                        | 26.67 | 6.88  | 59.37 |\n\n**Therefore, using the class token alone without corresponding denoising task (velocity prediction loss) yields no substantial benefit, while employing both jointly leads to clear improvements.**\n\n\n\n---\n\n### **[W2]: The explanation and analysis about REG use DINOv2-B which perform better than DINOv2-L.**\n\nThank you for your question. We provide the explanation and analysis for this issue below:\n\n**REG performs better with DINOv2-B, which is consistent with REPA.**\nThe observation that REG performs better with DINOv2-B than with DINOv2-L is consistent with findings reported in the REPA study.\nREPA does not exhibit a performance improvement when using larger encoders. \nPlease refer to the content reproduced below from Table 2 of the original REPA paper, along with our experimental results.\n\n\n\n\n\n| Mehtod  | Target Repr. | FID↓ | \n|:-------:|:------------:|:----:|\n| SiT-L/2 |      -       | 18.8 |\n| + REPA  |   DINOv2-B   | **9.7**  | \n| + REPA  |   DINOv2-L   | 10.0 | \n\n\n**Alignment difficulty and hyperparameter selection affect performance.** \nSpecifically, aligning **shallow-layer representations of SiT (e.g., 4 layers) with** **deep-layer representations of DINOv2 (the final layer)** \nmay result in representational mismatches and increased training instability, thereby hindering effective alignment.\nFor REG, final converged loss values reflect alignment and generation quality. Ablations on REG + SiT-B/2 (targeting DINOv2-B/L) use REPA’s loss formulation, \nreporting denoising (velocity prediction), alignment (negative cosine similarity), and total losses.\n\n\nSpecifically, Lower velocity prediction/total losses mean better generation; alignment loss ([−1,1]) closer to −1 indicates stronger alignment between SiT and DINOv2, reflecting more effective semantic understanding. \nResults show all three losses are lower when aligning with DINOv2-B than DINOv2-L.\n**This highlights that SiT-B/2 aligns more easily and effectively with DINOv2-B, resulting in superior generative performance relative to DINOv2-L.**\n\n| Method          | Target Repr.| FID↓    | Velocity prediction loss↓ | Alignment loss↓ | Total loss↓ |\n|:----------------|:------------|:--------|:--------------------------|:----------------|:------------|\n| REG + SiT-B/2   | DINOv2-B    | 15.22   | **0.77**                  | **-0.31**       | **0.46**    |\n| REG + SiT-B/2   | DINOv2-L    | 17.36   | 0.78                      | -0.29           | 0.49        |\n\nFurthermore, our hyperparameter ablation experiments for DINOv2-L show that, with appropriate tuning of β (0.03->0.05), which is the weight of the class token velocity prediction loss, \nthe performance gap between DINOv2-B and DINOv2-L can be significantly reduced.\n\n| Mehtod  | Target Repr. |  β   | FID↓  | \n|:-------:|:------------:|:----:|:-----:|\n| SiT-B/2 |      -       |  -   | 33.00 | \n|  + REG  |   DINOv2-B   | 0.03 | 15.22 | \n|  + REG  |   DINOv2-L   | 0.01 | 22.23 | \n|  + REG  |   DINOv2-L   | 0.03 | 17.36 | \n|  + REG  |   DINOv2-L   | 0.05 | 15.53 | \n|  + REG  |   DINOv2-L   | 0.07 | 16.76 | \n\nOverall, alignment difficulty and hyperparameter selection affect performance. \nTo ensure a fair comparison, **we follow the same experimental setup as REPA and adopt DINOv2-B** as the alignment target across all models in our experiments.\n\n---\n\n### **[W3]: Analysis of Training Overhead in REG.**\n\n\nDuring training, REG requires alignment with DINOv2-B representations. **We pre-process the DINOv2-B features offline and store them locally**. \nAs a result, during the training of the diffusion model, the DINOv2-B features can be directly loaded through the dataloader **without almost no additional FLOPs**.\n\nWe summarize the total training overhead in a below table, \nreporting the costs required to **reach the same performance upper bounds** claimed in the original SiT paper.\nAll experiments are conducted on 8 NVIDIA A40 GPUs.\nOur results show that REG requires only 110K training steps to reach the performance level of SiT trained for 7M steps, \n**reducing GPU hours by 98.36%**. In addition, the single-step training speed of the two models shows no significant difference.\nThese results highlight the training efficiency of REG, \ndemonstrating faster convergence and significantly lower training overhead compared to prior methods.\n\n\n|      Model       | FID↓ | Training Step |  All GPU hours↓  | Single step training speed↓ | \n|:----------------:|:----:|:-------------:|:-:|:---------------------------:|\n|     SiT-XL/2     | 8.3  |    7M         |       2380       |      1.22 s/step            |\n| **+ REG (ours)** | **8.2**  |   **110K**    | **39 (-98.36%)** |         1.28 s/step         |\n\n\n---\n\n### **[W4]: The linear probing accuracy of the generated representation.**\nFollowing REPA’s setup, we compare REG and REPA’s linear probing accuracy on ImageNet. \nResults show REG’s superior performance, reflecting stronger representational understanding.\n\n|           Model           |   Acc↑   |\n|:-------------------------:|:--------:|\n|   REPA + SiT-XL/2         |   74.6   |\n| **REG + SiT-XL/2 (ours)** | **76.3** |\n\n---\n\n### **[Q1]: What the differences between REG and ReDi?**\nREG and ReDi [1] are proposed during the same period. REG is simpler, faster in both training and inference, and achieves superior performance.\nWe provide a detailed comparison below:\n\n1. **Motivation**：\n   - REG: Overcomes REPA’s unaddressed issue: external alignment makes REPA **unable to produce discriminative representations during inference to guide generation**.\n   - ReDi: Investigates integrating representation learning with generative modeling, enforcing the diffusion model to learn data-latent joint distributions.\n \n2. **Representation selection and usage methods**:\n   - **_REG (Representation Entanglement with Class Token)_**: This method employs **a single global class token** which **is concatenated** with low-level latent tokens as the input.\n   - **_ReDi (Representation Fusion with Dense Tokens)_**: In contrast, this approach utilizes **multiple semantic dense tokens**, which **are fused** with low-level latent tokens via Principal Component Analysis (PCA) to construct the input.\n\n3. **Computational overhead of REG is lower**:\n   - Extra cost of REG is for handling one class token.\n   - Extra cost of ReDi is PCA on all dense tokens.\n \n4. **REG performs significantly better than ReDi:**:\n    \n    | Model | Iter | FID↓ |\n    |:-----:|:----:|:---:|\n    | ReDi  |  1M  | 5.1 |\n    | ReDi  |  3M  | 3.7 |\n    |  REG  | 400K | 3.4 |\n    |  **REG**  |  1M  | **2.7** |\n\n\n[1] ReDi: https://arxiv.org/abs/2504.16064v1\n\n---\n\n\n\n\n\n### **[Q3]: Mechanism Difference between REPA and REG.**\n\nREPA aligns with discriminative representations during training, \nwithout incorporating them as input and applying the corresponding denoising loss. \nThis design causes REPA to be **unable to produce discriminative representations during inference** to guide the generation process.\nIn contrast, REG treats the discriminative class token as part of the input and applies a corresponding denoising loss, allowing it to guide training.\nDuring inference, REG can **explicitly generate the discriminative class token and leverage it** to guide the generation process.\n\n---\n\n### **[Q4]: The vision foundational model chosen compared with REPA.**\n\nThe vision foundational model chosen strategy of REG is consistent with REPA, based on response to W2. \nSpecifically, DINOv2-B performs better than DINOv2-L in both cases of REG and REPA, according to Table 2 of REPA and the ablation experiments of REG.\nFurthermore, to ensure a fair comparison, **we adopt the same vision foundational model as used in REPA.**"}, {"review_id": "Yc9dcJMV2B", "reviewer": "Reviewer_1Q9L", "summary": "REPA allows training diffusion models faster through representation alignment with a pre-trained vision foundation models such as DINO. However, during sampling, models trained with REPA are not conditioned on any information from the vision backbone used for alignment. REG concatenates a noised class token from foundation models (e.g., DINOv2) with noised image latents as input to the diffusion transformer, enabling joint reconstruction of both image and semantic content. Experiments on ImageNet 256×256 show significant training acceleration and improved generation quality.", "questions": "**Noise schedule**: have the authors considered using a different noise scheduler for the diffusion process on the class embedding? Intuitively, I would expect that using a noise scheduler that is quicker to denoise the class token could be beneficial. Indeed, it means that the other tokens (that estimate the score for image tokens) could rely on the class embedding more, potentially leading to increased performance. I would appreciate if you could include such an experiment in your submission.\n\n**Additional parameters**: Why does REG requires 2M mode parameters than REPA (Table 1)? I did not see an explanation for this fact.\n\n**Failure Cases**: Under what conditions does REG perform poorly? Are there specific classes or image types where the method struggles? Including a more fine-grained evaluation (eg performance per class) would be interesting, for example compared to REPA of SiT.", "limitations": "The only limitation discussed by the authors is that models might be under-trained. No other limitations are discussed. One could argue that while their approach is effective, just as REPA, it requires using an external model, hence the approach is not end-to-end.", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "## Strengths\n**Simplicity**: REG is simple, general and easy to implement. It does not seem to require intensive hyperparameter tuning\n\n**Clarity**: The background section and explanation of the method are easy to follow, and can be understood on the first read. Figure 2 is easy to understand, and allows the reader to quickly understand their approach.\n\n## Weaknesses\n**Training overhead**: while the author discuss the sampling overhead, starting at line 219, they do not discuss the *training* overhead compared to training a regular SiT. In particular, they do not report the total cost of training with REG compared to a vanilla SiT without alignment.\n\n**Unfair claims**: occasionally, the authors make claims that I would judge to be unfair and that could mislead the reader. I would like the authors to revise their manuscript to clarify the following parts. Namely:\n- Lines 99-100\n> However, these methods rely on external alignment mechanisms that remain inactive during denoising inference, fundamentally limiting their capacity to fully exploit discriminative representations.\n\nREG does not use the external alignment mechanisms either during sampling. The above suggests that it does, hence it is misleading to the reader.\n- Lines 136-138\n>  To address the problems of REPA, the external alignment is absent during the entire denoising inference process, falling short of fully harnessing the potential of discriminative information. \n\nSimilarly, this is misleading, since REG does not rely on the external alignment module either.\n- Lines 165-166:\n> Full utilization of discriminative information.\n\n\"Full\" suggests that REG cannot be improved further. This cannot be proved empirically, hence it is not a fair claim.\n- Lines 169-189\n> This addresses a critical limitation of REPA, which relies on the external alignment mechanism that is absent during denoising inference.\n\nThis is not a fair claim: REG does not rely on the external alignment mechanism either during inference, only on the class token.", "was_revised": true, "final_justification": "I believe the paper should be accepted: it's simple, well-written, well-tested, and works well. I do not think it is ground-breaking though, which is why I did not give it a higher score.", "rebuttal": "Thank you very much for your valuable comments and the time you devoted to reviewing our manuscript. \nWe address each of your comments in detail below.\n\n---\n\n### **[W1]: Analysis of training overhead in REG.**\nDuring training, REG requires alignment with DINOv2-B representations. **We pre-process the DINOv2-B features offline and store them locally**. \nAs a result, during the training of the diffusion model, the DINOv2-B features can be directly loaded through the dataloader **without almost no additional FLOPs**.\n\nWe summarize the total training overhead in a below table, \nreporting the costs required to **reach the same performance upper bounds** claimed in the original SiT paper.\nAll experiments are conducted on 8 NVIDIA A40 GPUs.\nOur results show that REG requires only 110K training steps to reach the performance level of SiT trained for 7M steps, \n**reducing GPU hours by 98.36%**. In addition, the single-step training speed of the two models shows no significant difference.\nThese results highlight the training efficiency of REG, \ndemonstrating faster convergence and significantly lower training overhead compared to prior methods.\n\n\n|      Model       | FID↓ | Training Step |  All GPU hours↓  | Single step training speed↓ | \n|:----------------:|:----:|:-------------:|:-:|:---------------------------:|\n|     SiT-XL/2     | 8.3  |    7M         |       2380       |      1.22 s/step            |\n| **+ REG (ours)** | **8.2**  |   **110K**    | **39 (-98.36%)** |         1.28 s/step         |\n\n---\n\n### **[W2]: Modify some claims.**\n\nThank you very much for pointing out the inappropriate phrasing in our manuscript. \nWe will revise our manuscript accordingly in the final version to ensure clarity and accuracy.\n\n- _Lines 99-100_ _revision_: However, these methods rely on external alignment mechanisms \n  that do not take the discriminative representations as the input and denoising, \n  which are unable to produce discriminative representations during inference to guide the generation process.\n \n- _Lines 136-138_ _revision_: In REPA, the absence of the ability to autonomously generate discriminative representations \n  to guide generation in inference may reduce the leverage of discriminative information effectively.\n  \n- _Lines 165-166_ _revision_: Improved utilization of discriminative information.\n\n- _Lines 169-189_ _revision_: This design aims to address a limitation of REPA, which cannot autonomously generate discriminative representations to guide generation in inference. \n  Because it relies on an external alignment mechanism during training to utilize discriminative features, rather than incorporating them as input and applying the corresponding denoising task.\n\n---\n\n### **[Q1]: Faster denoising noise scheduler for class token.**\n\nThank you for your suggestion. Following your recommendation, we explore using different types of noise schedulers for different tokens in transforming velocity predictions to estimate the score. \nSpecifically, we apply a cosine path type to the class token, aiming to achieve faster denoising. For the other tokens, we retain the linear path type, assuming they could benefit from stronger reliance on the class token during the generation process.\nWe offer the detailed ablation results below. \n\n|      Model      | FID↓ | class token | the other tokens | \n|:---------------:|:----:|:-----------:|:----------------:|\n| REG + SiT-XL/2  | **1.8**  |   linear    |      linear      | \n| REG + SiT-XL/2  | 2.2  |   cosine    |      linear      | \n\n\nHowever, this modification does not lead to performance improvements. \nWe infer that the lack of gain stems from **the training–inference inconsistency**. During training, all tokens, including the class token, use the linear interpolant path for noise scheduling. Therefore, employing a different noise scheduler for the class token during inference creates a mismatch that negatively affects the model's performance.\n\nDue to time constraints, we have not yet retrained the model under consistent training and inference configurations. In future work, we plan to retrain a model where the class token is trained and inferred under a faster denoising scheduler to further evaluate the effectiveness of this strategy.\n\n\n\n---\n\n### **[Q2]: Explanation of the additional parameters in REG.**\n\nThank you for your comment. We explain the additional parameters (2M) introduced by REG below. \nSpecifically, REG includes the following extra components: a Linear layer is used to map the class token from the DINOv2-B representation to the SiT space, \na LayerNorm layer is applied to the mapped class token, and an additional Linear layer at the end of the network to decode the class token into the final output prediction. \nThese components collectively enable the transformation and utilization of the class token within the REG framework. \nThe detailed implementation and parameter calculations are provided below:\n```\n#DINOv2-B hidden_size\ncls_token_dim = 768 \n\n#SiT-XL hidden_size\nhidden_size = 1152\n\n# project the class token into the same embedding space via a linear layer to obtain \nself.cls_projectors2 = nn.Linear(in_features=cls_token_dim, out_features=hidden_size, bias=True)\n\n# apply normalization to the mapped class token\nself.norm = nn.LayerNorm(hidden_size, elementwise_affine=True, eps=1e-6)\n\n# decode class token into the final output prediction\nself.linear_cls = nn.Linear(hidden_size, cls_token_dim, bias=True)\n\nParam = self.cls_projectors2 + self.norm + self.linear_cls\n      = (768 × 1152 + 1152) + (1152 + 1152) + (1152 × 768 + 768) = 1.77M\n      ≈ 2M\n```\n\n---\n\n### **[Q3]: Discussion on failure cases.**\n\nThank you for your suggestion. We conducted a large-scale comparison of the visualization results between REPA and REG.\nOverall, REG demonstrates a substantial improvement in image quality compared to REPA, \nHowever, for a small subset of challenging samples, such as tench (a kind of fish), both REG and REPA perform poorly.\nWe attribute this to the extra noise in the ImageNet training data. For instance, these images depict a person holding a tench, but the provided label corresponds only to the tench. \nThis discrepancy can introduce ambiguity during training and hinder the model’s ability to learn accurate class-conditional alignment.\nDue to time constraints, we will make a more fine-grained evaluation in the future. \n\n---\n\n### **[Q4]: Discussion on limitations.**\nThank you for your comment. We agree that the reliance on an external model, such as DINOv2, limits the end-to-end nature of our approach. In addition, REG has not yet been extensively explored in terms of architecture and task diversity. There are several promising directions for future work:\n- Extending to broader tasks and models, such as video generation and integration with autoregressive generative models, which may benefit from more complex discriminative guidance.\n- Adopting stronger variants of VAE to enhance the model's ability to generate more accurate and diverse representations, potentially improving the overall performance in generative tasks.\n\nWe plan to investigate these directions, along with improved integration schemes and generalization to more diverse domains in future work."}, {"review_id": "MDyOjQ9e2h", "reviewer": "Reviewer_vBTu", "summary": "The authors propose REG (Representation Entanglement for Generation) which improves/accelerates diffusion model training by integrating a high-level class token from pretrained vision models (e.g., DINO v2) into low-level image latents. Unlike REPA, which implicitly aligns the discriminative clean features with the image latents only during training, REG extends the alignments to be more explicit, by entangling the low-level image latents with the high-level class token from the pretrained vision models. This enables generation of coherent image-class pairs from noise, enhancing quality and training speed with minimal computational cost.", "questions": "- Can this method be still effective in T2I task? I guess the text embedding includes rich information as much as dino cls embeddings, and therefore the proposed method might not be as effective as in the simple class conditional setting.  \n- Where do you get the parameters for the linear layer mapping $cls$ to $cls’$? Is it trained together?\n- $F_0 = [f_0; cls_{0}]$ in $(N+1)\\times D_z$? How? $f_0 \\in \\mathbb{R}^{N \\times D_{vf}}$ and $cls_t \\in \\mathbb{R}^{1 \\times D_{vf}}$.\n- Dimension of $F_0$ and the output of $h_\\theta$. Are they in $\\mathbb{R}^{(N+1) \\times D_{vf}}$? I guess it should be, as $F_0$ is kinda proxy ground truth. Now, how do you maximize cos similarity? Inner product might yield $(N+1) \\times (N+1)$. Do you only maximize the diagonal terms (perhaps 1 if normalized)? \n- Related works involving the conditioning signal to the diffusion process, as this paper proposes, i.e., the condition is noised and denoised together. Can this be only used for training from scratch? Can it be used for finetuning pretrained large models?\n- How is FID measured?\n    - Are the same models used for Tab 1 and Tab 2?\n    - For Tab 1, I guess it is unconditional generation? 50k for synthetic and 50k for real?\n    - For Tab 2, it is confusing. CFG requires the model to take a conditional signal. Is it given through cross attention layer? Is one-hot class conditioned? How many samples are used to measure?\n- “Applying the loss in earlier layers yields superior results.” Why is this?", "limitations": "- Performance in T2I setting is not reported.\n- Applicability to pretrained T2I large models, e.g., SD1, SD3, FLUX.\n- Comparison with REPA-E", "rating": 6, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 4, "significance": 3, "originality": 4, "strengths_and_weaknesses": "- Strengths:\n    - The paper is written clearly.\n    - The proposed method is simple but effective.\n    - Good visualizations of the proposed method.\n    - Experiment results are impressive.\n- Weaknesses\n    - Performance in T2I setting is questionable.\n    - Applicability of the proposed method to the widely-used pretrained diffusion models in T2I setup.\n    - Some details are not clear.\n    - Missing baseline; REPA-E.\n\nOverall, this paper lacks a baseline comparison with REPA-E, and performance report in T2I setting, which is more widely used than class-conditional setting. It would be much more impactful if the proposed method can be applicable in T2I setting, and even to the pretrained large models. However, the proposed method has a reasonable motivation with a suitable method, and shows good performance in class-conditional setting compared to the recent baseline. \n\nI think the proposed method is good enough to give accept. \nHowever, I am not sure if it could be impactful as the performance in T2I setting (which is more widely used in practice) is questionable.\nIf the authors can clarify some of my concerns, especially performance in T2I setting, I would raise my score to strong accept.", "was_revised": true, "final_justification": "The author response during the rebuttal has resolved my concerns.\n\nTo my understanding the strengths of this paper are:\nOn top of REPA (, which proposed a soft regularization that can boost diffusion training speed by leveraging the pretrained knowledge of DINO v2) this paper further suggests a method to incorporate the class token of pretrained large scaled discriminative models into diffusion models. The experiment results show that their idea yields non-trivial benefits (faster training and better performance). They also show that their methods can be generalized beyond the classifier-guidance setting.\n\nThus, I recommend to accept.", "rebuttal": "Thank you very much for your valuable comments and the time you devoted to reviewing our manuscript. \nWe address each of your comments in detail below.\n\n---\n\n### **[W1, Q1]: Performance in T2I setting.**\nThank you for your question. We add text-to-image (T2I) results in the Appendix by adopting the same experimental configurations as REPA and UViT [1], utilizing MMDiT [2] as the backbone network. \nThe results are presented below. REG demonstrates superior generation quality compared to alternative approaches on MS-COCO [3].\n\n| Method                      | Type               | FID↓  |\n|:----------------------------| :----------------- | :---- |\n| AttnGAN                     | GAN                | 35.49 |\n| DM-GAN                      | GAN                | 32.64 |\n| VQ-Diffusion                | Discrete Diffusion | 19.75 |\n| DF-GAN                      | GAN                | 19.32 |\n| XMC-GAN                     | GAN                | 9.33  |\n| Frido                       | Diffusion          | 8.97  |\n| LAFITE                      | GAN                | 8.12  |\n| U-Net                       | Diffusion          | 7.32  |\n| U-ViT-S/2                   | Diffusion          | 5.95  |\n| U-ViT-S/2 (Deep)            | Diffusion          | 5.48  |\n| MMDiT (SDE; NFE=250)        | Diffusion          | 5.30  |\n| MMDiT + REPA (SDE; NFE=250) | Diffusion          | 4.14  |\n| MMDiT + REG (SDE; NFE=250)  | Diffusion          | **3.07**  |\n\n[1] All are Worth Words: A ViT Backbone for Diffusion Models, CVPR 2023  \n[2] Scaling Rectified Flow Transformers for High-Resolution Image Synthesis, ICML 2024   \n[3] Microsoft COCO: Common Objects in Context, ECCV 2014\n\n---\n\n### **[W2, Q5]: Applicability to pretrained T2I large models.**\n\nREG is not limited to training from scratch; it can also be applied to fine-tuning pretrained large-scale models. \nHowever, due to time constraints and limited computational resources, \nwe currently lack sufficient resources to fine-tune REG on pretrained large-scale text-to-image (T2I) models (e.g., SD1, SD3, FLUX). \nIn future work, we plan to progressively extend REG to these models in order to evaluate its generalization capabilities and scalability in more complex generative settings.\n\n---\n\n\n###  **[W3, Q2]: Explanation about the linear layer.**\nThe parameter of the linear layer is randomly initialized and is trained with the diffusion model. \nThe specific process is: The class token $c l s_t \\in \\mathbb{R}^{1 \\times D_{v f}}$ of DINOv2-B is projected by a learnable linear layer (Linear: $ \\mathbb{R}^{1 \\times D_{v f}}$ → $ \\mathbb{R}^{1 \\times D_{z}}$) to get $c l s_t^{\\prime} \\in \\mathbb{R}^{1 \\times D_z}$. \nThis linear layer is jointly trained with the entire diffusion model (SiT), to minimize the total loss function $\\mathcal{L}_{\\text {total }}$ (Equation 11).\n\n---\n\n### **[W3, Q3, Q4]: Details of calculating negative cost similarity.**\nThank you for your question. In the final version of the manuscript, we will incorporate further revisions to improve clarity and facilitate better understanding. The following provides a detailed description:\n* $F_0 \\in \\mathbb{R}^{(N+1) \\times D_{vf}}$ is the reference representation of DINOv2 (pseudo ground truth). It is obtained by concatenating $f_0 \\in \\mathbb{R}^{N \\times D_{v f}}$ and $cls_0 \\in \\mathbb{R}^{1 \\times D_{v f}}$.\n* $H_t^{[n]} \\in \\mathbb{R}^{(N+1) \\times D_{z}}$ represents the output of the n-th SiT block, which is mapped by a trainable MLP projection $h_\\phi$ to get $h_\\phi\\left(H_t^{[n]}\\right)  \\in \\mathbb{R}^{(N+1) \\times D_{vf}}$ \n* $\\mathcal{L}_ {REPA}$ (Equation 9) computes negative cosine similarity between $F_0$ and ${h_\\phi\\left(H_t^{[n]}\\right)}$. It is minimized to encourage alignment.\n* We compute negative cosine similarity only between corresponding token positions (i.e., the diagonal elements) to get the $(N+1)$ matrix rather than $(N+1)×(N+1)$.\n* The detailed pseudocode is present below. `F_0` denotes $F_0$, `W` denotes $h_\\phi\\left(H_t^{[n]}\\right)$, `D_vf` represents the dimension $D_{vf}$.\n    \n```\n# B: Batchsize   N+1: Length   D_vf: Dimension\n# F_0: [B, N+1, D_vf]  - reference features (e.g., from SiT)\n# W:   [B, N+1, D_vf]  - predicted features (e.g., from DINOv2)\n\n# Step 1: Normalize both feature sets along the feature dimension\nF_0 = F.normalize(F_0, dim=-1) # [B, N+1, D_vf]\nW = F.normalize(W, dim=-1) # [B, N+1, D_vf]\n\n# Step 2: Compute element-wise dot product between corresponding vectors\ntemp = F_0 * W  # [B, N+1, D_vf]\n\n# Step 3: Sum across feature dimension to compute negative cosine similarity for each position\nalign_loss = -torch.sum(temp, dim=-1) # [B, N+1]\n\n# Step 4: Average across all tokens and batches to get scalar alignment loss\nalign_loss = align_loss.mean() # Final scalar loss (e.g., -0.26)\n```\n\n---\n\n### **[Q6]: How is FID measured?**\n    \nWe follow the evaluation protocol of REPA for computing the Fréchet Inception Distance (FID) [1], using the Inception-V3 model to extract 2048-dimensional features from both generated and real images. \nThe FID is then calculated based on the mean and covariance of these feature distributions over 50,000 samples.\n\n[1] Gans trained by a two time-scale update rule converge to a local nash equilibrium, NeurIPS 2017\n\n---\n\n### **[Q6.1]: Are the same models used for Table 1 and Table 2?**\nYes, the same models are used for both Table 1 and Table 2. Table 1 reports results without using Classifier-Free Guidance (CFG), whereas Table 2 reports results with CFG.\n\n---\n\n### **[Q6.2]: Explanation of Table 1 and Table 2.**\nTable 1 and Table 2 all are trained for **conditional generation using label inputs**.\nIn both tables, 50,000 images are generated and evaluated against real images using standard quantitative metrics.\n\n---\n\n### **[Q6.3]: Details of Classifier-Free Guidance (CFG) in REG.**\n\nClassifier-Free Guidance enhances the model's responsiveness to conditioning signals during inference by combining conditional and unconditional predictions. \nThis improves the quality and diversity of generated images while avoiding the complexity of external classifier guidance.\nWe sample 50K images to measure which follows SiT and REPA.\n1. **Form of conditioning input**\n   - In REG + SiT, the conditioning input is formed by the **sum of the timestep embedding and the label embedding** following DiT [1]. This combined vector serves as the conditioning signal.\n2. **Condition injection mechanism (AdaLN)**  \n   - The conditioning vector is passed through an MLP and fed into the **Adaptive Layer Normalization (AdaLN) [1]** modules of each SiT block.  \n   - AdaLN maps the conditioning vector to layer-wise `scale` $\\gamma$ and `shift` $\\beta$ parameters to modulate the feature maps:  \n     $\\text{AdaLN}(x) = \\gamma(\\text{cond}) \\cdot \\text{LN}(x) + \\beta(\\text{cond})$\n\n[1] DiT: Scalable Diffusion Models with Transformers, ICCV 2023\n\n\n---     \n\n### **[Q7]: Further explanation about applying the alignment loss in earlier layers yields superior results.**\nREG is built upon REPA and adopts the same configuration. We provide detailed ablation studies on alignment depth below.\nFirst, REG consistently outperforms REPA across different alignment depths, \nand applying the alignment loss in earlier layers yields superior results, which is consistent with REPA’s findings.\n\n| Mehtod  | Target Repr. | Depth | FID↓  | sFID↓ |  IS↑  | Pre.↑ | Rec.↑ |\n|:-------:|:-----------:|:-----:|:-----:|:-----:|:-----:|:-----:|:-----:|\n| SiT-B/2 |      -      |   -   | 33.00 | 6.46  | 43.70 | 0.53  | 0.63  |\n| + REPA  |   DINOv2-B  |   4   | 24.40 | **6.40**  | 59.90 | 0.59  | **0.65**  |\n|  + REG  |   DINOv2-B  |   2   | 18.19 | 6.67  | 83.96 | 0.62  | 0.64  |\n|  + REG  |   DINOv2-B  |   4   | **15.22** | 6.89  | **94.64** | **0.64**  | 0.63  |\n|  + REG  |   DINOv2-B  |   6   | 16.31 | 7.11  | 91.72 | 0.63  | 0.64  |\n|  + REG  |   DINOv2-B  |   8   | 17.31 | 7.23  | 87.78 | 0.63  | 0.63  |\n\n\nWe believe this reflects an inherent trade-off: **the alignment depth determines the diffusion model’s ability to understand and generate.** \nAligning at earlier layers allows the diffusion model to acquire DINOv2 semantics in its early stages, \nenabling subsequent layers to build on a strong representation and focus on capturing more high-frequency details for generation.\nHowever, aligning too early is not always better. For example, \"depth = 2\" performs worse than \"depth = 4\" indicating a trade-off between representation understanding and generation. \nOverall, earlier layers tend to yield superior results.\n\n---\n\n### **[Q8]: Comparison with REPA-E.**\nThank you for your suggestion. Following the REPA-E paper, we compare the results of REG and REPA-E, and find that REG achieves better performance on ImageNet $256{\\times}256$.\n\n| Model      | Tokenizer | Epochs | FID↓  | sFID↓    | IS↑   |\n|:-----------|:----------| :----- |:------|:---------|:------|\n| REPA-E     | SD-VAE*   | 20     | 12.83 | 5.04     | 88.8  |\n| REPA-E     | SD-VAE*   | 40     | 7.17  | **4.39** | 123.7 |\n| REPA-E     | SD-VAE*   | 80     | 4.07  | 4.60     | 161.8 |\n| **REG (ours)** | SD-VAE    | 20     | 8.90  | 5.50     | 125.3 |\n| **REG (ours)** | SD-VAE    | 40     | 5.02  | 4.88     | 161.2 |\n| **REG (ours)** | SD-VAE    | 80     | **3.44**  | 4.87     | **184.1** |\n\n'*' indicates that VAE is updated during end-to-end training by REPA-E"}, {"review_id": "kFRZdnHtNv", "reviewer": "Reviewer_xw43", "summary": "This paper proposes an improvement to REPA, a regularization designed to accelerate the training of diffusion transformers. While the original REPA aligns the spatial features of vision encoders like DINO with the features of the diffusion transformer, the proposed method, REG, further aligns the class token from DINO as well. In addition, the diffusion transformer is trained to predict not only the latent tokens but also an additional class token. This approach is simple, introduces minimal computational overhead, and significantly improves training speed, as demonstrated on ImageNet generation experiments.", "questions": "- In the appendix, ImageNet 512×512 results are presented, but there is no EDM2 [B] baseline, and REG underperforms compared to EDM2. While outperforming EDM2 is not necessarily the goal, it would be valuable to include a discussion on REG’s applicability to EDM2 or on why EDM2 performs better.\n\n- In Table 3, why does DINO-B outperform DINO-L? An explanation or analysis would help clarify this counterintuitive result.\n\n[B] Karras et al., Analyzing and Improving the Training Dynamics of Diffusion Models, CVPR 2024.", "limitations": "Yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "[Strengths]\n\n- The proposed method is simple and effective.\n- The paper is clearly written and easy to understand.\n- Experimental validation is comprehensive, including both unguided and guided generation comparisons, ablation studies, and CKNNA analysis.\n\n[Weaknesses]\n\n- The paper only presents results for shorter training iterations compared to SiT and REPA. Although REG demonstrates higher training efficiency, it is unclear whether the performance remains strong or even improves with extended training. For instance, would training SiT-L/2 to 700K iterations or SiT-XL/2 to 4M iterations yield better results (Table 1)? Similarly, would REG benefit from training to 800 epochs (Table 2)?\n\n- REG incorporates four loss terms: velocity prediction and representation alignment for both latent and class tokens. However, ablation studies only remove the class token’s representation alignment loss (Table 6). There are no experiments that remove the class token’s velocity prediction loss or employ alignment using only class tokens. More detailed ablation is necessary to identify which loss components contribute most to REG’s performance.\n\n- Both REPA and this paper on REG omit a simple but strong baseline: allocating more training computation to timesteps that contribute more to performance. For example, the EDM [A, B] or Stable Diffusion 3 [C] paper samples timesteps from a logit-normal distribution instead of uniform, and emphasizes higher noise levels through timestep shifting for high-resolution images. Given that the core motivation of REG is to improve training speed, excluding this baseline weakens the claim.\n\n- Table 2 lacks baseline CFG performance without guidance interval, making comparisons incomplete.\n\n[A] Karras et al., Elucidating the Design Space of Diffusion-Based Generative Models, NeurIPS 2022.\n\n[B] Karras et al., Analyzing and Improving the Training Dynamics of Diffusion Models, CVPR 2024.\n\n[C] Esser et al., Scaling Rectified Flow Transformers for High-Resolution Image Synthesis, ICML 2024.", "was_revised": true, "final_justification": "The extended training results and the 512×512 results are impressive, and the ablation studies are thorough.", "rebuttal": "Thank you very much for your valuable comments and the time you devoted to reviewing our manuscript. \nWe address each of your comments in detail below.\n\n---\n\n### **[W1]: REG improves the performance with extended training.**\nWe further train REG, and as the training progressed, it consistently shows performance improvements across different task settings.\nFor example, the two tables below present the ImageNet $256{\\times}256$ results without and with CFG, respectively.\n\nImageNet $256{\\times}256$ without CFG. \nCompared to other approaches, REG achieves the best performance, reaching an FID of 1.8 at 4M training steps, which is 4.1 lower than REPA.\n\n| Model       | Iter | FID↓ |\n| :---------- | :--- |:-----|\n| SiT-XL/2    | 400K | 17.2 |\n| SiT-XL/2    | 7M   | 8.3  |\n| +REPA       | 200K | 11.1 |\n| +REPA       | 400K | 7.9  |\n| +REPA       | 1M   | 6.4  |\n| +REPA       | 4M   | 5.9  |\n| **+ REG (ours)** | 200K | 5.0  |\n| **+ REG (ours)** | 400K | 3.4  |\n| **+ REG (ours)** | 1M   | 2.7  |\n| **+ REG (ours)** | 2.4M | 2.2  |\n| **+ REG (ours)** | 4M   | **1.8**  |\n\nImageNet $256{\\times}256$ with CFG. REG exhibits consistently improving performance with extended training time, \nachieving its best FID score of 1.36 at 800 epochs.\n\n|      Model       | Epochs |  FID↓  |  sFID↓  |   IS↑  | Pre.↑ | Rec.↑ |\n|:----------------:|:----:|:-----:|:----:|:------:|:-----:|:----:|\n|     SiT-XL/2     | 1400 | 2.06  | 4.50 | 270.3  | **0.82**  | 0.59 |\n|      + REPA      | 800  | 1.42  | 4.70 | 305.7  | 0.80  | 0.65 |\n| **+ REG (ours)** |  80  | 1.86  | 4.49 | **321.4** | 0.76  | 0.63 |\n| **+ REG (ours)** | 160  | 1.59  | 4.36 | 304.6  | 0.77  | 0.65 |\n| **+ REG (ours)** | 480  | 1.40  | **4.24** | 296.9  | 0.77  | **0.66** |\n| **+ REG (ours)** | 800  | **1.36** | 4.25 | 299.4 | 0.77  | **0.66** |\n\n---\n\n### **[W2]: The effectiveness of the an extra discriminative class token and the denoising task for it.**\n\nAblation studies (Table 6) evaluate entanglement alone by removing alignment loss (REPA loss), assessing the extra noised class token and its denoising task (velocity prediction loss).\nResults show introducing a noised class token without a matching denoising objective causes a large performance drop (FID 98.67). \nThis is because, without class token denoising, the model cannot autonomously generate a discriminative class token to guide inference.\nMoreover, **there is an inconsistency between training and inference**. \nThis discrepancy disrupts the generation process. In inference, no explicit class token is provided; the diffusion model cannot rely on it as in training. \n\nIn contrast, introducing the noised class token with a dedicated training denoising task yields optimal performance (FID 26.67). \nThis is because class token denoising loss enables the **diffusion model to autonomously generate a semantically discriminative class token during inference.**\nThe generated class token can effectively guide the generation of other tokens. Moreover, this setup ensures consistency between training and inference. \n\n\n\n| Method  | Class token | Velocity prediction loss for class token | FID↓  | sFID↓ | IS↑   |\n|:-|:---|:-|:-|:------|:------|\n| SiT-B/2 | ×           | ×                                        | 33.0  | 6.46  | 43.70 |\n| _+ Entanglement_ |\n| DINOv2-B | ✓           | ×                                        | 98.67 | 9.80  | 12.36 |\n| DINOv2-B | ✓           | ✓                                        | 26.67 | 6.88  | 59.37 |\n\n**Therefore, using the class token alone without corresponding denoising task (velocity prediction loss) yields no substantial benefit, while employing both jointly leads to clear improvements.**\n\n---\n\n### **[W3, Q1]: REG achieve better performance in higher-resolution experiments on ImageNet $512{\\times}512$.**\nFollowing your suggestion, we further train REG on ImageNet $512{\\times}512$ and compare it with EDM2 [1]. \nAmong all methods in the table below, REG achieves the best performance.\n\n| Model                         | Epochs | FID↓ | sFID↓  | IS↑   | Pre.↑ | Rec.↑ |\n|:------------------------------|:-------|:-----|:-------|:------| :---- | :---- |\n| _Pixel diffusion_                |\n| VDM++                         | —      | 2.65 | —      | 278.1 | —     | —     |\n| ADM-G, ADM-U                  | 400    | 2.85 | 5.86   | 221.7 | 0.84  | 0.53  |\n| Simple diffusion (U-Net)      | 800    | 4.28 | —      | 171.0 | —     | —     |\n| Simple diffusion (U-ViT, L)   | 800    | 4.53 | —      | 205.3 | —     | —     |\n| EDM2                          | —    | 1.81 | —      | —     | —  | —  |\n| _Latent diffusion, Transformer_  |\n| MaskDiT                       | 800    | 2.50 | 5.10   | 256.3 | 0.83  | 0.56  |\n| DiT-XL/2                      | 600    | 3.04 | 5.02   | 240.8 | 0.84  | 0.54  |\n| SiT-XL/2                      | 600    | 2.62 | 4.18   | 252.2 | 0.84  | 0.57  |\n| + REPA                        | 80     | 2.44 | 4.21   | 247.3 | 0.84  | 0.56  |\n| + REPA                        | 100    | 2.32 | 4.16   | 255.7 | 0.84  | 0.56  |\n| + REPA                        | 200    | 2.08 | 4.19   | 274.6 | 0.83  | 0.58  |\n| **+ REG (ours)**              | 80     | **1.68** | **3.87**   | **306.9** | 0.80  | **0.63**  |\n\n\n[1] Karras et al., Analyzing and Improving the Training Dynamics of Diffusion Models, CVPR 2024.\n\n---\n\n### **[W4]: REG perform better in CFG performance without the guidance interval.**\nFollowing your suggestion, we compare the missing CFG performance without guidance interval results in Table 2, and find that REG performs better than REPA.\n\n|             Model             | Epochs |   FID↓   | sFID↓ |    IS↑    |    Pre.↑    | Rec.↑ |\n|:-----------------------------:|:------:|:--------:|:-----:|:---------:|:-----------:|:-----:|\n|           SiT-XL/2            |  1400  |   2.06   | 4.50  |   270.3   |  **0.82**   | 0.59  |\n|   + REPA w/o interval         |  200   |  1.96    | 4.49  |   264.0   |  **0.82**   | 0.60  |\n| **+ REG w/o interval (ours)** |  200   | **1.68** | **4.36**  | **312.3** |    0.80     | **0.62**  |\n\n---\n\n### **[Q2]: The explanation and analysis about REG use DINOv2-B which perform better than DINOv2-L.**\n\nThank you for your question. We provide the explanation and analysis for this issue below:\n\n**REG performs better with DINOv2-B, which is consistent with REPA.**\nThe observation that REG performs better with DINOv2-B than with DINOv2-L is consistent with findings reported in the REPA study.\nREPA does not exhibit a performance improvement when using larger encoders. \nPlease refer to the content reproduced below from Table 2 of the original REPA paper, along with our experimental results.\n\n\n| Mehtod  | Target Repr. | FID↓ | \n|:-------:|:------------:|:----:|\n| SiT-L/2 |      -       | 18.8 |\n| + REPA  |   DINOv2-B   | **9.7**  | \n| + REPA  |   DINOv2-L   | 10.0 | \n\n\n**Alignment difficulty and hyperparameter selection affect performance.** \nSpecifically, aligning **shallow-layer representations of SiT (e.g., 4 layers) with** **deep-layer representations of DINOv2 (the final layer)** \nmay result in representational mismatches and increased training instability, thereby hindering effective alignment.\nFor REG, final converged loss values reflect alignment and generation quality. Ablations on REG + SiT-B/2 (targeting DINOv2-B/L) use REPA’s loss formulation, \nreporting denoising (velocity prediction), alignment (negative cosine similarity), and total losses.\n\n\nSpecifically, Lower velocity prediction/total losses mean better generation; alignment loss ([−1,1]) closer to −1 indicates stronger alignment between SiT and DINOv2, reflecting more effective semantic understanding. \nResults show all three losses are lower when aligning with DINOv2-B than DINOv2-L.\n**This highlights that SiT-B/2 aligns more easily and effectively with DINOv2-B, resulting in superior generative performance relative to DINOv2-L.**\n\n| Method          | Target Repr.| FID↓    | Velocity prediction loss↓ | Alignment loss↓ | Total loss↓ |\n|:----------------|:------------|:--------|:--------------------------|:----------------|:------------|\n| REG + SiT-B/2   | DINOv2-B    | 15.22   | **0.77**                  | **-0.31**       | **0.46**    |\n| REG + SiT-B/2   | DINOv2-L    | 17.36   | 0.78                      | -0.29           | 0.49        |\n\nFurthermore, our hyperparameter ablation experiments for DINOv2-L show that, with appropriate tuning of β (0.03->0.05), which is the weight of the class token velocity prediction loss, \nthe performance gap between DINOv2-B and DINOv2-L can be significantly reduced.\n\n| Mehtod  | Target Repr. |  β   | FID↓  | \n|:-------:|:------------:|:----:|:-----:|\n| SiT-B/2 |      -       |  -   | 33.00 | \n|  + REG  |   DINOv2-B   | 0.03 | 15.22 | \n|  + REG  |   DINOv2-L   | 0.01 | 22.23 | \n|  + REG  |   DINOv2-L   | 0.03 | 17.36 | \n|  + REG  |   DINOv2-L   | 0.05 | 15.53 | \n|  + REG  |   DINOv2-L   | 0.07 | 16.76 | \n\nOverall, alignment difficulty and hyperparameter selection affect performance. \nTo ensure a fair comparison, **we follow the same experimental setup as REPA and adopt DINOv2-B** as the alignment target across all models in our experiments."}]

Author rebuttal:

Meta-review: The paper introduces REG, a method for accelerating the training of diffusion models. It is a simple extension to REPA that aligns spatial information like in REPA, but additionally denoises the class tokens from the pretrained vision models. This approach significantly accelerates training and improves generation quality with minimal overhead. The method is simple and effective, and the proposed empirical evaluation is convincing. Because the approach is very simple, and the topic very timely, I think this paper makes a very interesting contribution. All reviewers have all acknowledged the quality of this submission. A shared concern about ViT-{L/B} results was properly addressed by the authors. Because of all the above, I recommend accepting this paper as an oral at NeurIPS 2025.
Decision: Accept (oral)