What matters for Representation Alignment: Global Information or Spatial Structure?
Authors: Jaskirat Singh
Xingjian Leng
Zongze Wu
Liang Zheng
Richard Zhang
Eli Shechtman
Saining Xie
Venue: ICLR 2026 Poster (2026)
Keywords: repa
representation learning
repa-e
Primary area: generative models

Abstract: Representation alignment helps generation by distilling representations from a pretrained vision encoder to intermediate diffusion features. We investigate a fundamental question - `what aspect of the target representation matters for generation, its global information (measured by Imagenet1K accuracy) or its spatial structure (pairwise cosine similarity between patch tokens)''? Prevalent wisdom holds that stronger global performance leads to better generation as a target representation.  To study this, we first perform a large-scale empirical analysis across 27 different vision encoders and different model scales. The results are surprising - spatial structure, rather than  global performance drives the generation performance of a target representation. To further study this, we introduce two straightforward modifications, which specifically accentuate the transfer of spatial information. We replace the standard MLP projection layer in REPA with a simple convolution layer and introduce a spatial normalization layer for the external representation. Surprisingly, our simple method (implemented in <4 lines of code), termed iREPA, consistently improves convergence speed of REPA, across a diverse set of vision encoders, model sizes, and training variants (such as REPA, REPA-E, meanflow, JiT etc). Our work motivates revisiting the fundamental working mechanism of representational alignment and how it can be leveraged for improved training of generative models.

Published as a conference paper at ICLR 2026
WHAT MATTERS FOR REPRESENTATION ALIGNMENT: GLOBAL INFORMATION OR SPATIAL STRUCTURE?
Jaskirat Singh" 12
Xingjian Leng²
Zongze Wu'
Liang Zheng²
Richard Zhang'
Eli Shechtman New York University
Saining Xie
Adobe Research
ANU
ABSTRACT
Representation alignment (REPA) guides generative training by distilling rep- resentations from strong, pretrained vision encoder to intermediate diffusion features. We investigate a fundamental question: what aspect of the target repre- sentation matters for generation, its global semantic information (e.g., measured by ImageNet-IK accuracy) or its spatial structure (i.e. pairwise cosine similarity between patch tokens)? Prevalent wisdom holds that stronger global semantic performance leads to better generation as a target representation. To study this, we first perform a large-scale empirical analysis across 27 different vision encoders and different model scales. The results are surprising spatial structure, rather than global performance, drives the generation performance of target representa- tion. To further study this, introduce two straightforward modifications, which specifically accentuate the transfer of spatial information. We replace the standard MLP projection layer in REPA with simple convolution layer and introduce spatial normalization layer for the external representation. Surprisingly, our simple method (implemented in <4 lines of code), termed iREPA, consistently improves convergence speed of REPA, across a diverse set of vision encoders, model sizes, and training variants (such as REPA, REPA-E, Meanflow, JiT etc). Our work moti- vates revisiting the fundamental working mechanism of representational alignment and how it can be leveraged for improved training of generative models.
gPID (SIT-xL//u)) Bok steges
BEED 1ST7 2 100K stepa)
Pearson cor 0,260
Pnarson corT. -0,822
STEXU + REMA
S/EXL/2 + IREA
BFID (400K stapp)
Low correlation
High cocrelation
10.4
10.0
0.5
0.1
26-
7.7
7.4
10
DINOwO-S16 DINGYS-MIG+ HIGe
PB.C
WHIAH MaCCor LangPS-G
18
Linear Probing ACc.
Spetial Structure (LDS)
Vision Encoder
Figure 1: What matters for representation alignment? Left: Correlation analysis across 27 diverse vision encoders. Surprisingly, contrary to the prevailing wisdom, we find that spatial structure, rather than global performance (measured by linear probing accuracy), drives the generation performance of a target representation. Right: We further study this by introducing two simple modifications to accentuate the transfer of spatial features from target representation to diffusion model. Our simple approach consistently improves the convergence speed of REPA across diverse settings.
INTRODUCTION
Representation alignment has emerged as a powerful technique for accelerating the training of diffusion transformers (Ma et al., 2024; Peebles & Xie, 2023). By aligning internal diffusion
'oon during internship Adohe Research
Project: hetps: /ttpa:::///dddeddeee/ treattn waw.nn...tttttt 10/1 repa

---

Published as a conference paper at ICLR 2026
Retto Dlond
Detar Gibbal
We Gereration
You on
Liby
Livi
SeatoPt:
RG.%%
9:88
30.3
0.33
26.1
0.1
23.0
Gaayyingge
Beter Sautel
Gusiall
Detar Genannan
Ap9D SoB cvaiiiy Vousborio
Sprsasse contons, Sealirtton
Figure 2: Motivating examples spatial structure matters. Metries comparison showing inverse relationship between ImageNet accuracy and generation quality. Left: PE-G, despite having signifi- cantly higher validation accuracy (82.8% vs. 53.1%). shows worse generation quality compared to SpatialPE-B (Bolya et al., 2025). Right: Similarly, WebSSL-IB (Fan et al., 2025) also shows much better global performance (76.0% 53.1 1%), but worse generation. Spatial Self-Similarity: We find that spatial structure instead provides a better predictor of generation quality than global performance. See $3 for spatial structure metric. All results reported 100K using SiT-XL/2 and REPA.
representations with pretrained self-supervised visual encoders, recent methods have demonstrated significant improvements in the convergence speed and final performance (Yu et al., 2024; Leng al., 2025a). However, despite these empirical successes, there remains limited understanding of the precise mechanisms through which self-supervised features enhance diffusion model training. A fundamental question persists: is the improvement primarily driven by incorporating better global semantic information, as commonly measured through linear probing performance, or does it stem from better capturing spatial structure, characterized by the relationships between patch token representations?
Understanding these mechanisms is erucial for advancing generative model training, it directly impacts one's ability to select the optimal target representation and maximize its benefits. Currently, a prevalent assumption is that encoder performance for representation alignment correlates strongly with ImageNet-11 K validation accuracy, a proxy measure of global semantic understanding (Oquab et al., 2024; Chen et al., 2021). That is, target representations with better ImageNet performance hypothesized to lead to better generation (Yu et al., 2024). As such, increases in linear probing accuracy of diffusion features are frequently cited as evidence for the effectiveness of representa- tion alignment, reinforcing the emphasis global semantic information the primary driver of improvement. The following quote from REPA (Yu et al., 2024) captures the current understanding:
"When a diffusion transformer is aligned with a pretrained encoder that offers more seman- tically meaningful representations (i.e., better linear probing results), the model not only captures better semanties but also exhibits enhanced generation performance, as reflected by improved validation accuracy with linear probing and lower FID scores."
In this paper, we challenge this conventional wisdom by systematically investigating what truly drives representation alignment: global semantic information or spatial structure? Through large-scale empirical analysis across diverse vision encoders, including recent large vision foundation models such as WebSSL (Fan et al., 2025), DINOv3 (Siméoni et al., 2025), perceptual encoders (Bolya et al., 2025), C-RADIO (Heinrich et al., 2024), we uncover surprising findings.
Higher validation accuracy does not imply better representation for generation. Contrary to prevailing assumptions, vision encoders with higher global semantic performance, measured by ImageNet-1K K linear probing accuracy, can often underperform for generation. For instance, consider PE-Spatial-BB a spatially-tuned model derived from PF-Core-G (Bolya et al., 2025). We find that while PE-Spatial-B shows much worse validation accuracy' on patch tokens than PE-Core-G (53. % vs. 82.8%), it leads to better generation with REPA (Figure 2). Similarly, WebSSL-IB (Fan et al., 2025) shows much better global performance (76.0% vs. 53.1%), but worse generation. In Section $2. we find that this pattern holds across various target representations, suggesting fundamental principle in how representation alignment benefits diffusion training.
Spatial structure not global information determines generation performance. To quantify this insight, we consider several straightforward metries to measure the spatial self-similarity structure
Similar REPA (Yu et al., 2024), we use linear probing accuracy patch tokens to measure global semantic performance of external representation only patch tokens are used for representation alignment.
2

---

Published as a conference paper at ICLR 2026
(Shechtman & Irani, 2007) between patch tokens ($3). We then perform large-scale analysis comput- ing the correlation between generation FID with REPA, linear probing accuracy, and spatial structure across 27 vision encoders and model sizes (SiT-B, SiT-L, SiT-XL). Surprisingly, all spatial structure metrics exhibit remarkably better correlation (Pearson |r| > 0.852) with generation FII scores, far exceeding the predictive power of ImageNet-16 K validation accuracy (Ir) = 0.26). These findings are also supported by additional experiments showing that external representations with very limited global information can be used to get significant gains with REPA. For instance, SAM2 leads better generation with REPA than other encoders with ~ 60% higher ImageNet accuracy (Fig. 3). Similarly, while not the best, we find that classical spatial features such as SIFT (Lowe, 1999) and HOG (Dalal & Triggs, 2005) can also be used to achieve decent gains with representation alignment ($3).
Accentuating spatial features improves convergence performance. To further study this, we next introduce two straightforward modifications (<4 lines of code), which specifically accentuate the transfer of "spatial" information from target representation to diffusion model: (1) We first introduce spatial regularization layer which boosts the spatial contrast of the target representations. (2) Next, we show that the standard MLP-based projection layer (used to map diffusion features to target representation dimensions) causes loss of spatial information (Figure 6a). To avoid this, replace with a simple convolution layer. The results are surprising: our simple method, termed iREPA, consistently improves convergence speed of REPA across diverse variation in encoders, model sizes, and training recipes e.g., REPA-E (Leng et al., 2025a) and Meanflow (Geng et al., 2025) w/ REPA.
We highlight the main contributions of this paper below:
We perform large-scale empirical analysis showing that spatial structure and not global semantic information drives the effectiveness of representation alignment.
We introduce a straightforward and fast-to-compute Spatial Structure Metric (SSM), which shows significantly higher correlation with downstream FID performance than linear probing scores.
We propose simple modifications to better accentuate the transfer of spatial information from the target representation to diffusion features. Our simple method shows consistent improvements in the convergence speed over REPA across variations target representation, model architectures as well as training recipe variants (REPA, REPA-E, Mcanflow w/ REPA, JiT w/ REPA etc).
2 MOTIVATION: GLOBAL INFORMATION MATTERS LESS
We first provide several motivating examples showing that target representation with higher global performance (ImageNet-Il IK accuracy) does not imply better generation performance with REPA. We later show in $3, that these rrviously-uncxplained observations can instcad be better explained by measuring the spatial structure of the target representation.
Recent vision encoders. We examine different recent vision encoders, including Perceptual Encoders (Bolya al., 2025), WebSSL (Fan et al., 2025), and DINOv3 (Siméoni et al., 2025). Consider PE-Spatial-B (80M), a small spatially tuned model derived from PE-Core-G (1.88B) (Bolya et al., 2025). As seen in Figure 2, we find that while PE-Core-G achieves much higher ImageNet- K accuracy (82.8% vs 53.1 %), it performs worse when used as the target representation for REPA (FID 32.3 vs 21.0). Similarly, WebSSL-IB IB (1.2B) achieves much higher ImageNet- IK accuracy (76.0% vs 53.1%) but performs worse when used as target representation for REPA (FID 26.1 vs 21.0).
SAM2 outperforms vision encoders with much higher ImageNet- accuracy. To further understand how little global information impacts generation performance, we amalyze SAM2-S vision encoder (46M) (Ravi et al., 2024) small model with very little global information and validation accuracy of only 24.1 (Fig. 3a). Surprisingly, when used for REPA, SAM2-S achieves better FIDD than other vision encoders with significantly higher ImageNet- K accuracy egg., PF--Core-G (82.8%).
Larger models within same encoder family can have similar or worse generation performance. common perception is that larger models within the same encoder family have better representations (measured by ImageNet- IK accuracy). However, for representation alignment, larger model variants often lead to similar (DINOv2) or even worse (PE, Cradio) generation performance (Fig. 3b). Notably (Yu et al., 2024) also make similar observation for DINOv2 and explain it as "we hypothesize is due to all DINOv2 models being distilled from the DINOv2-g model and thus sharing similar representations" We later show that this trend can be better explained using spatial structure ($3).
3

---

Published as a conference paper at ICLR 2026
GFID (S:T 1OOK steps
PE-G
aRID XLL-0L stops)
a=0.5
32-
Worse geseration FID
Higber Val. Accuracy
P1-11
LanghB-G
Highor Accuracy Worse generation
22
23-
8
Higher Val. Aucuracy Worse geseration FDD
SAME
24
PF-B
00=0.0
Validation Accuracy
Validation Accuracy (%)
Validation Accuracy
(ST 100K stepa)
RPID (SST-XL (XXXLL1000 oOK stepa)
GRLD 1000 steps)
a 0.0.5
12.
Higher Spatial Structure
Hetter generation FID
Spatial Structure Better generatioa FID
22
23
28
22
WebSEL
SAM2
24
Higher Spetial Structure
Better generaation FTD
PF-B
0,0
0.10 Spatial Structure (LDS)
0.30
0:30
Spatial Structure
Special Siructure
(a) SAM2 "better" encoders.
(b) Larger models but worse gFID.
Adding global info. hurts FID.
Figure 3: Higher global information does not imply better REPA performance. Top row: Several trends showing global performance does not correlate well with generation FID when using REPA. (a) SAM2-S with only validation accuracy of 24.7% results in better generation performance with REPA compared with other models with 60% higher validation accuracy. (b) Larger encoders within same family can have better validation but worse generation performance. (c) Adding global information to patch tokens via CLS token improves global performance but hurts generation. Bottom row: We show that spatial structure rather then global performance provides a better indicator for generation. Please see $3 for large-scale detailed analysis across different spatial structure metrics. All results reported w/o classifier free guidance, S/TXL/2 w/ REPA and 250 NEE (Yu et al., 2024) for inference.
Adding more global information can hurt generation. To test whether additional global information benefits representation aligmment, we conduct controlled experiments that inject global semantics, using the CLS token, into local patch tokens (with DINOv2 as encoder). The CLS token mixing operation is p.t. P: + a where p; denotes the nth patch token, e the CLS token, and a E [0, 0.5) controls the mixing strength. As shown in Figure 3c, as a increases from 0 to 0.5, linear probing accuracy improves monotonically from 70.7% to 78.5%. However, generation quality deteriorates significantly, with FID scores worsening from 19.2 at a = 0 to 25.4 at a = 0.5.
The above observations highlight that global performance of representation is not good indicator of its REPA performance. We next show ($3) spatial structure instead provides a better signal.
Finding 1. Better global semantic information (e.g., ImageNet-IK accuracy) does not imply a better representation for generation.
3 SPATIAL STRUCTURE MATTERS MORE
We hypothesize that spatial structure, rather than global information, drives the generation perfor- mance with a target representation. To quantify this, we first consider several straightforward metrics to measure the spatial self-similarit structure (Shechtman & Irani, 2007) of target representations. We then show that all spatial structure metrics not only correlate much higher with gFID than ImageNet- 1K accuracy, but can be also used to explain previously unexplained observations in $2.
Measuring spatial self-similarity structure. Given an image I and vision encoder E, we define:
D be the extract patch representations, with T = H x W patches. Let kernel Kx(;,) :) E R measure appearance self-similarrty (Shechtman & Irani, 2007) (x,)x.)
Self-similarity Let X' = EII E RTx
between patch tokens. Here, we use the cosine kernel Kx (t, t') = |
Spatial distance. Let E NTx2 be the spatial location of each of the T tokens in coordinate space and d(-s-) :) E N be the Manhattan distance between pairs of tokens.
We then measure spatial self-similarity metric as how self-similarity Kxx varies with lattice distance d between patch tokens. Intuitively,, larger values indicate stronger spatial organization (closer patches
4

---

Published as a conference paper at ICLR 2026
MPEED croOk stape)
DETD spo0k 20090)
0.260
0.052
Y- 0.005
r- 0,847
0.580
OPIO croOk
Linoar Probing Acturary
Spatial Structurn (LBS)
Spotial Stroctume (5R158)
Spenial Stractume (CDSS)
Spatial Structure (RMIS)
Figure 4: Spatial structure shows higher correlation with generation quality than linear probing.
Correlation analysis across 27 diverse vision encoders, SYT-XIJ2 and REPA. Linear probing shows weak correlation with FID (Pearson |rl 0.260), while all spatial structure metries: LDS (Ir) = 0.852), SRSS (Irl 0.885), CDS (rr 0.847), and RMSC (rrl 0.888), demonstrate much stronger correlation with generation performance. See Fig. 10 for detailed plots with encoder labels.
more similar to each other than patches further away). By default, we use a simple correlogram contrast (local vs. distant) metric (Huang et al., 1997):
LDS(X, P) = (d(t,)))(0, (x t') Ed(t, tt).
The final spatial self-similarity score (LDS) is computed as the expectation over patch representation X. We use Uneur = flar H/2 here, though we found correlation to be robust to their exact choices. Please see Appendix B for exact details and alternative spatial metrics (CDS, SRSS, RMSC) which also perform effectively (see Fig. 4).
Spatial structure correlates much higher with generation performance than linear probing. We next perform large-scale correlation analysis across diverse vision encoders. As shown Figure 4, we find that while typically used linear probing shows very weak correlation (Pearson |r| = 0.26), all SSM metrics show much higher correlation with generation performance (Pearson |r] > 0.85).
Generalization across model scales. We verify the correlation across different model scales (SiT-B, SiT-L, SiT-XL) in Figure 5. Linear probing shows weak correlation across model scales ([rl < 0.306), while spatial structure shows much higher correlation with generation performance (Ir) 0.826).
Finding 2. Spatial structure correlates much higher with generation performance than linear probing. We next use spatial metrics to explain previously unexplained trends.
Spatial structure explain previously unexplained trends. As discussed in $2, global per- formance (ImageNet-1k IK accuracy) does not serve as a predictive measure of effectiveness for representation alignment. We find that instead the above spatial metrics serve as better predictors.
(1) PE-Spatial-B vs PE-Core-G: Figure 2 shows that PF-Core-G achieves much higher ImageNet-1K accuracy (82.8% 53.1 1,), but performs worse when used target representation for REPA (FII) 32.3 vs 22.0). Looking at spatial structure metric makes things clearer. As seen in Figure 2, despite lower global performance, PE-Spatial-B shows much better spatial structure than PE-Core-G; leading to better generation performance as observed.
(2) SAM2 outperforms "better" vision encoders: $2 shows that while SAM2 achieves much lower ImageNet- KK accuracy (24.1%), it leads to better generation than encoders with 60% higher accuracy. As in Fig. 3a; these gains can be directly explained through SAM2's better spatial structure.
(3) Larger models in same encoder family underperform: As shown in Fig. 3b, while larger models in same encoder family show better ImageNet- 1 K accuracy, they can have worse spatial structure, leading to worse generation performance with REPA. This also aligns with observations from Yu et al. (2024), where they find that larger models can have similar or worse generation performance.
4) Adding global information to patch tokens via CLS token hurts generation: In Figure 3c, we observe that increasing global information by mixing the CLS token with patch tokens improves global performance. However, mixing CLS token reduces spatial contrast among patch tokens. This leads to patch tokens having high similarity with otherwise unrelated tokens (e.g., from foreground object to background). The reduced spatial structure thus leads to worse generation performance.
If spatial structure matters more, can we use SIFT or HOG features for REPA? Surprisingly, yes. We find that while certainly not the best, classical spatial features like SIFT (Lowe, 1999), HOG (Dalal & Triggs, 2005) and intermediate VGG features (Simonyan & Zisserman, 2014)
12:92
3.84
5

---

Published as a conference paper at ICLR 2026
PPernie Fore
Peemon Elers
Teamon CAPT
Peenzo Ciey
Pearrzo Caer
Peepun CArr
SEEL/2
Proting fecurccy
Triling
Squati
Figure 5: Correlation analysis across model scales. Across different model scales, we find that spatial structure (right) consistently shows higher correlation with gFID than linear probing (left).
all lead to performance gains with REPA. This provides further evidence that representation alignment can benefit from spatial features alone without need for additional global information.
Can we use spatial metrics to explain gains with REPA? Yes. As shown, the introduced spatial metries be used to explain both gains with REPA as well as our improved training recipe (iREPA) which we introduce next in $4.
4 iREPA: IMPROVING REPRESENTATION ALIGNMENT BY ACCENTUATING WHAT MATTERS
To further investigate the role of spatial structure in representation alignment, we introduce two straightforward modifications to the original REPA training recipe, which enhance the transfer of spatial features from the teacher (vision encoder) to the student (diffusion transformer) model.
Convolutional projection layer instead of MLP. The standard REPA approach uses a 3-layer MLP projection to map diffusion feature dimensions to that of the external representation. However, as shown in Figure 6a, we observe that this projection is lossy and diminishes the spatial contrast between patch tokens. We therefore replace the MLP with a lightweight convolutional layer (kernel size 3, padding 1), that operates directly on the spatial grid. The convolutional structure naturally preserves local spatial relationships through its inductive bias.
Spatial normalization layer. Similar to Siméoni et al. (2025), we find that patch tokens of pretrained vision encoders consist a significant global component. This is evidenced by the high linear probing scores for the mean of patch tokens (Figure 14). Also, we see in Figure 6b that while mixing of this global information (mean of patch tokens) with the patch tokens helps improve global performance, it reduces the spatial contrast between individual patch tokens. This leads tokens (e.g., foreground object) showing high similarity with otherwise unrelated tokens (e.g., background).
Given results from $3, we hypothesize that we can sacrifice this global information (mean of patch tokens) to improve the spatial contrast between the patch tokens. The improved contrast should provide better spatial signal (pairwise similarity between patch tokens) - leading to better REPA performance. To this end, we add a simple spatial normalization layer (Ulyanov et al., 2016) to the patch tokens of the target representation:
X
[x
Var[x] +
where x E BBTxD represents the patch tokens, the expectation and variance are computed across the spatial dimension, and 10-6 for numerical stability.
Algorithm 1 Summary of key iREPA modifications to standard REPA training
1. Conv projection instead of MLP proj_ nn. Con2dddddD__n,, D_out, kernel_size-3 padding-1)
2. Spatial normalization on enceder features [B, T, D]
gamma x.mean (dim=1, keepdim-True) x sw.stdddierd keepdin-True) + 1e- 6)
4.1 EXPERIMENTS
In this section, we validate the performance of the improved training recipe through extensive experiments Imagenet 256 256. In particular, investigate the following research questions:

---

Published as a conference paper at ICLR 2026
Target Representation
Didfusion Peatures (MLP Projection Layer)
Deftusion Features (Cony Projection
(a) Simpler prejection layer for REPA. Standard MLP projection layer in REPA (middle) loses spatial information while transferring features from target representation (left) to diffusion features. Instead using simpler convolution layer leads to better spatial information transfer (right),
wo Spatial Normalisation Layer
w/ Spatial Normalization Layer
McCoV3-1 DINOV3-8
1.20
4075
0.25
0.20
(b) Spatial normalization layer. Patch tokens of pretrained vision encoders have global component which limits spatial contrast. This causes the tokens in one semantic region (e.g.. tomato) to show quite decent cosine similarity with unrelated tokens (e.g., background cucumber), We hypothesize that information to improve the spatial contrast between patch tokens Icading to better generation performance.
sacrifice this global
Standand REMA
After Spertal Modiftcattons JREPA
Eituao Pustroe
(c) Overall impact of improved training recipe (iREPA) on spatial structure of diffusion features.
Figure 6: Motivating two straightforward modifications (iREPA) to enhance spatial feature transfer from target representation to diffusion features. All results reported with SiTL/2 with REPA 100K.
Can iREPA consistently improve the convergence speed of diffusion transformers over REPA
across diverse external representations? (Figure 8, 12, 13, 15)
Is iREPA scalable in terms of model size and generalize across variations in training settings?
(Table 1 [a, 4, 5, 6, and, Figure 12, 16)
Does iREPA generalize across more recent representation alignment methods such as REPA-E (Leng et al., 2025a), McanFlow w/ REPA (Geng al., 2025)? and pixel-space diffusion models such as JiT w/ REPA (Li & He, 2025by? (Table 3 [a.b])
Convergence Speed. We evaluate the convergence behavior of iREPA across diverse vision encoders (DINOv3-B, WebSSL-1BB PE-Core-G, G, CLIP-L, MoCov3, PE-Lang-G), and model sizes (SiT-XL/2 and SiT-B/2). Results are shown in Fig. 7. We find that iREPA consistently helps achieve faster convergence over baseline REPA across variations in both target representation and model sizes.
Target representation. We analyze the generalization of iREPA across different vision encoders in Fig. and Table 4. We observe that iREPA consistently improves the generation quality across all vision encoders. Additional comparisons across all 27 encoders are provided in Appendix C, E.
Enc. Size
1st
FID sFID) Pr: Re.t
Model Size 181 FID sFID Pr. Re.t
Aln. Depth 1ST FIDJ sFID) Pr. Re.7
PE-B (90M)
59.6 22.5
6.23
0.63 0.60
SiT-B HIREPA
27.5 49.50 50
7.00
0.46 0.59
Layer HREPA
28.1 49.0
6.83
0.46 0.60
HREPA
17.5
6.04
43.37
7.87
0.50
0.60
41.7
36.0
0.52 0.62
(320M)
6:25
Layer HREPA
50.6
0.46 0.59
HREPA
76.3
17.6 6,36
0.65 0.61
HIREPA
66.9 20.28
6.14
0.63 0.62
42.4 35.3
6.90
0.53 0.62
(1.88)
42.7 32.3
6:70
0.57 0.59
SiT-XI.
70.3 19.06
5.83
0.65 0.61
Layer
24.7 54.8
7.35
0.44 0.58
HRREAA
70.8
19.5 6.35
0.64 0.61
HIREPA
77.9 16.96
6.26
0.66 0.61
+IREPA
38.3 38.9
7.19
0.51 0.62
(a) Vision Encoder size
(b) Model size
(c) Encoder depth (SiT-B/2)
Table Variation in training settings. We find that iREPA leads to consistent gains over baseline REPA across diverse training settings. Unless otherwise specified all results are reported using Dinov2 as encoder, SiTXL/2, 100K steps and vanilla-REPA as baseline.
Encoder size. We analyze the generalization of iREPA across different encoder sizes. Table la shows results analyzing generalization iREPA across different encoder sizes; PE-B (90M), PE-L (320M), PE-G (1.88B). We see that of iREPA consistently helps improve the performance across
7

---

Published as a conference paper at ICLR 2026
DEA
JO IEP
GOERO COIM
GITAL Eaegtt
OTTAR PECANA
GEAD PAOOLL11
FON
For
48
X0E 606
200 CBB
JCC
Figure 7: Accentuating spatial features helps consistently improve convergence speed.
SIFXL-2 RERA SITXL-2 IREPA (Duns)
REPA Augi 24.2
IREPA Avg: 17.8
RPID Score (100K Steps)
DENINZ
DENON3
WABSSE
Langre
SpatielPE
CHADIO
DINO CLIPP
MUCCOS)
JEPA
MANE
Lis HEreE
Figure 8: Variation in target representation. Across all 27 vision encoders, we find that accentuating transfer of spatial features from target representation to diffusion features (iREPA) leads to consistent improvements in convergence speed. See Appendix C for more results across diverse settings.
all encoder sizes. Interestingly, the percentage improvement also increases with increasing encoder size (22.2% for PE-B, 38.8% for PE-L, 39.6% for PE-G).
Scalability, We analyze the scalability of iREPA across different model scales in Table lb. We observe that the spatial improvements not only consistently improve performance, but larger percentage gains are seen with larger models; showing that spatial improvements are scalable with model size.
Encoder depth. Table Ic analyzes generalization of iREPA across different alignment depths. All results are with S/T-B/2 at 100K iterations using DINOv3-B B as target representation. We observe consistent improvements over baseline REPA across different alignment depths.
Abalation on different components. We also study the role of different components in iREPA in Table We observe that both spatial normalization and convolution projection layer significantly improve the generation quality over baseline REPA; with the best results achieved by using both.
Training recipe. Lastly, we analyze the generalization of iREPA across different training recipes such as REPA-E (Leng et al., 2025a) and MeanFllw w/ REPA (Geng et al., 2025). Table 3 shows that spatial improvements with iREPA lead to convergence gains across different training recipes.
frrs guidance. We evaluate the generation quality of iREPA with CFG in Table 4. presents results across different vision encoders at 400K training iterations. We find that across different encoders, iREPA leads to faster convergence both with and without classifier-fres guidance.
Pixel-space diffusion (JiT). We also evaluate iREPA on pixel-space diffusion models such as JiT- B (Li & He, 2025b). Figure 9 shows results across with both REPA and iREPA using JiT (Li & He, 2025b). We observe that accentuating transfer of spatial information consistently achieves faster convergence with REPA across various vision encoders (e.g., DiNOy2, DINOv3, PE etc.).
8

---

Published as a conference paper at ICLR 2026
BETA IREM
RIN AEP
00.4
01.3
61.5
BBTS
IRERA
- REM
-O- IRERA
sPI (ity (1101100)
DINOv2-B
DINOv3-8
DINOvS-It+
LangPE-G
PE-Care-G
42.7
41.3
44.3
400
47.2
42.9
20.
40.4
43.3
40.3
43.0
100
2018
200%
500%
400
30CK
S00K
200%
500C
600
Training Iterations
Training Berations
Training Iterations
Training Iteraionn
Training Iteratioon
Figure 9: Convergence comparison with pixel-space diffusion (JIT). FID convergence curves comparing REPA vs iREPA with pixel space JiT (Li & He, 2025b) across different vision encoders. Accentuating transfer of spatial structure helps consistently improve the convergence speed of across different vision encoders for pixel-space diffusion models. All results are reported with JiTB/16 (Li & He, 2025b), 256 batch size and without dassifier-free guidance. Refer Table 10 for further results.
DINOv2-B
DINOr3-B
WebSSL-1BB
PE-Core-G
Method
FID1
ISt
sFID]
FIDI
181
sFID
FID4
181
sFID.
FID4
181
sPID
Baseline REPA
19.06
70.3
5.83
21.47
63.4
6.19
26. 10
53.0
6,90
32.35
42.7
6.70
iREPA spotial norm)
18.52
17.76
6.21
iREPA (w/ conv proj)
17.66
72.8
6.03
18.28
70.8
6.18
18.44
71.0
6.22
21.72
61.5
6.26
IREPA (full)
16.96
77.9
6.26
16.26
78.8
6.14
16.66
77.5
6.18
18.19
75.0
6.03
Table 2: Ablation on different components. We that across different encoders both spatial
normalization layer and convolution projection layer ($4) lead to significant gains in convergence speed; with best results obtained using both. All results reported 100K steps and SiT-XL/2.
Finding 3. Accentuating transfer of spatial structure from target representation diffusion features helps boost convergence speed with representation alignment (REPA).
5 RELATED WORK
We discuss the most relevant related work here and provide detailed discussion in Appendix H.
Representation alignment for generation. Many recent works explore use of external represen- tations for improving diffusion model training (Pernias et al., 2023; Fuest et al., 2024). Notably, recent works (Yu al., 2024; Yao Wang, 2025; Leng et al., 2025a;b; Kouzelis et al., 2025) show that significant performance gains can be achieved by aligning internal diffusion features with clean image features from pretrained vision encoder. (Zhang et al., 2025; Wu et al., 2025) extend this idea to video generation and 3D generation respectively. (Ma et al., 2025a) shows that representation alignment can be used to improve training of unified models. Despite these emperical successes, there remains limited understanding of the precise mechanisms through which self-supervised features enhance diffusion model training. In this paper, we try to understand what aspect of the target representation matters for generation, and use it to propose improved training recipe.
Spatial vs global information tradeoff in pretrained vision encoders. Recent works explore the tradeoff between global and spatial information in pretrained vision encoders (Bolya al., 2025; Siméoni et al., 2025). (Siméoni et al., 2025) show that continued training of self-supervised vision representations can lead increased similarity between global CLS token and patch tokens leading to worse performance on dense spatial tasks. (Bolya et al., 2025; Heinrich et al., 2024) specifically train spatial-tune -uneed models for dense spatial tasks. In this paper, we show that for generation, spatial structure of vision encoder matters more then its global information. We hope this motivates future research on better selecting and training external representations for generation.
6 CONCLUSION
In this paper, we study what truly drives the effectiveness of representation alignment, global information the spatial structure of the target representation? Through large-scale empirical analysis

---

Published as a conference paper at ICLR 2026
Encoder
18t
FIDJ
sFIDL
Pres:f
Rec:f
w/o CFG
w/ CFG
We
5288
26.5
5.20
0.620
0.585
4 NNF
INFE
ANFR
I
NFF
Encoder
HREPA-E
87.0
13.2
5.28
0.699
0,598
181
FIDL
1ST
FID
1St
FIDJ
18
FID
PE-G
50.9
25.9
5.68
0.612
0.576
WebSSI.11B
27.22
51.40
24.14
58.67
87.85
16.59
69.06
23.74
+RREPAEE
80.0
16.4
5.40
0.667
0.616
+IREPA
31.48
45.67
27.33
55.70
100.74
13.89
78.67
20.69
DINOv3-B
82.2
14.4
4.68
0.694
0,596
DINOv3-B
28.36
49.58
25.47
57.01
93.30
15.56
72.41
22.55
+RREPA-E
93.6
11.7
0.613
+REPA
44.52
124.54
98.93
17.32
DINOv2-B
87.5
12.9
5.40
0.708
0.586
DINOv2-B
28.77
48.87
25.43
56.63
94.40
15.28
71.95
22.17
+REEP-E
91.3
12.1
4.86
0.712
0.602
+REPA
33.63
44.52
29.67
53.75
111.44
12.59
90.25
18.62
(a) REPA-E (SIT-XL/2)
(b) MeanFlow w/ REPA (SIT-B/2)
Table 3: Variation in training recipes. We apply our spatial improvements on top of REPA-E (a)
and Mcanflow with REPA (b); achieving consistent gains. For Mcanflow, we report results at I and NFEs. CFG value of 2.0 is used for Meanflow. All results are reported at 100K training iterations.
CFG
CFG
Vision Encuder
Steps
1St
FID4
xFII..
Prec.
Rec.
1S1
FIDJ
SFID
Prec.
Rec.
DINOv2-B
100K
69.20
19.3
5.89
0.64
0.61
157.2
6.35
5.91
0.769
0.536
+RRPP
100K
77.92
16.9
0.66
179.3
5.15
6.23
0.783
DINOv2-B
400K
127.4
7.76
5.06
0.70
0.66
263.0
1.98
4.60
0.799
0.610
+IREPA
400K
128.6
7.52
4.89
0.71
0.65
268.8
1.93
4.59
0.799
0.600
DINOv3-B
100K
63.64
21.4
6.14
0.63
0.60
144.0
7.57
6.09
0.762
0.526
+IREPA
100K
78.79
16.2
6.14
0.66
0.61
181.9
4.87
6.10
0.780
0.547
DINOv3-B
400K
126.7
8.10
5.06
0.70
0.66
261.2
1.99
4.58
0.799
0.609
+IREPA
400K
132.9
7.13
4.93
0.71
0.66
272.4
1.89
4.58
0.799
0.600
WeNSSL-
100K
53.87
25.5
6.57
0.61
0.59
124.0
9.59
6.37
0.756
0.515
+RREPA
100K
77.47
16.6
6.18
0.66
0.61
177.6
5.09
6.11
0.787
0.538
WebSSL-1B
400K
116.9
9.39
5.14
0.70
0.64
250.5
2.24
4.61
0.809
0.580
+RREPA
400K
130.8
7.48
4.91
0.70
271.6
4.58
0.798
PE-Coro-G
100K
42.74
32.3
6.70
0.57
0.59
97.2
14.1
6.56
0.714
0.525
+RRPA
100K
75.01
18.1
6.03
0.64
0.61
176.8
5.66
6,08
0.771
0.544
PE-Core-G
400K
109.4
10.4
5.00
0.69
0.64
238.4
2.44
4.57
0.805
0.585
+REPA
400K
132.0
5.02
0.70
275.4
4.59
0.796
Table 4: Results across different encoders with and w/o classifier-free guidance. SiT-XL/2 is used base model. See Appendix E for detailed results on SiT-B/2 & SiT-L/2 across all encoders.
we uncover a surprising finding: spatial structure and not global information, drives the effectiveness of representation alignment. We further study this by introducing two simple modifications which accentuate the transfer of spatial information from target representation to diffusion features. Our simple method, termed iREPA, consistently improves convergence speed with REPA across diverse variations in vision encoders and training recipes. We hope our work will motivate future research to revisit the fundamental working mechanism of representational alignment and how we can better leverage it for improved training of generative models.
ACKNOWLEDGEMEENT
We thank You Jiacheng (@ YouJiacheng), Shuming Hu (@ (ShumingHuu) @gallabytes whose com- ments on X motivated the exploration this direction (Jiacheng, 2025a;b; Hu, 2025). The authors were glad to find out their original predictions were wrong, which opened the door to new insights.
We also thank Zhengyang Geng for providing the meanflow with REPA implementation and for useful discussion on hyperparameter configuration. We also thank Boyang Zheng, Fred Lu, Nanye (Willis) Ma and Sihyun Yu for insightful discussions and guidance on RAE experiments.
10

---

Published as a conference paper at ICLR 2026
REPRODUCIBILITY STATEMENT
We provide all implementation details and hyperparameters in Appendix G. We also open-source our code, model checkpoints and analysis results.
REFERENCES
Kristian Abstreiter, Sarthak Mittal, Stefan Bauer, Bernhard Schölkopf, and Arash Mehrjou. Diffusion- based representation learning. In ICML 2021 Workshop on Unsupervised Reinforcement Learning, 2021.
Stability AL Improved autoencoders ... od-vae-Bf ft-mse, n.d. Accessed: April 11, 2025.
Anthropic. Claude code. /ttps://elant
ai/ code, 2025.
Mahmoud Assran, Quentin Duval, Ishan Misra, Piotr Bojanowski, Pascal Vincent, Michael Rabbat, Yann LeCun, and Nicolas Ballas. Sulf--upe learning from images with joint-embedding predictive architecture. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 15619-15629, 2023.
Fan Bao, Shen Nie, Kaiwen Xue, Chongxuan Li, Shi Pu, Yaole Wang, Gang Yue, Yue Cao, Hang Su, and Jun Zhu. All are worth words: vit backbone for diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 22669-22679, 2023.
Herbert Bay, Tinne Tuytelaars, and Lue Van Gool. Surf: Speeded up robust features. In Computer Vision-ECCV 2006: 9th European Conference on Computer Vision, Graz, Austria, May 7-13, 2006. Proceedings, Part I 9. pp. 404-417. Springer, 2006.
Daniel Bolya, Yao Huang, Peize Sun, Jang Hyun Cho, Andrea Madotto, Chen Wei, Tengyu Ma, Jiale Zhi, Jathushan Rajasegaran, Hanoona Rasheed, Junke Wang, Marco Monteiro, Hu Xu, Shiyu Dong, Nikhila Ravi, Daniel Li, Piotr Dollár, and Christoph Feichtenhofer. Perception encoder: The best visual embeddings are not at the output of the network. arXiv:2504.13181 2025.
Shoufa Chen, Mengmeng Xu, Jiawei Ren, Yuren Cong, Sen He, Yanping Xie, Nicu Sebe, and Wai Lam. Gentron: Diffusion transformers for image and video generation. arXiv preprint arXiv:2312.04557 2023.
Shoufa Chen, Chongjian Ge, Shilong Zhang, Peize Sun, and Ping Luo. Pixelflow: Pixel-space generative models with flow. arXiv preprint vrXiv:254..07963 2025.
Xinlei Chen, Saining Xie, and Kaiming He. An empirical study of training self-supervised vision transformers. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 0640-9649, 2021.
Xinlei Chen, Zhuang Liu, Saining Xie, and Kaiming He. Deconstructing denoising diffusion models for self-supervised learning. arXiv preprint arXiv:2401. 2024.
Jang Hyun Cho, Andrea Madotto, Effrosyni Mavroudi, Triantafyllos Afouras, Tushar Nagarajan, Muhammad Maaz, Yale Song, Tengyu Ma, Shuming Hu, Hanoona Rasheed, Peize Sun, Po- Yao Huang, Daniel Bolya, Suyog Jain, Miguel Martin, Huiyu Wang, Nikhila Ravi, Shashank Jain, Temmy Stark, Shane Moon, Bahak Damavandi, Vivian Lee, Andrew Westbury, Salman Khan, Philipp Krähenbühl, Piotr Dollár, Lorenzo Torresani, Kristen Grauman, and Christoph Feichtenhofer. Perceptionlm: Open-access data and models for detailed visual understanding. arXiv:2504.1 13180, 2025.
N. Dalal and B. Triggs. Histograms of oriented gradients for human detection. In 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR 05), volume 1, pp. 886-893 vol. 2005. doi: 2t09///2..200.....
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large- acage hierarchical image database. In 2009 JEEE conference on computer vision and pattern recognition, pp. 248-255. leee, 2009.
11

---

Published as a conference paper at ICLR 2026
Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans image synthesis. Advances in neural information processing systems, 34:8780-8799 2021.
Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas Müller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, et al. Scaling rectified flow transformers for high-resolution image synthesis. In Forty-first International Conference on Machine Learning, 2024.
David Fan, Shengbang Tong, Jiachen Zhu, Koustuv Sinha, Zhuang Liu, Xinlei Chen, Michael Rabbat, Nicolas Ballas, Yann LeCun, Amir Bar, et al. Scaling language- visual representation learning. arXiv preprint 2501:2250..00117 2025.
Michael Fuest, Pingchuan Ma, Ming Gui, Johannes Schusterbauer, Vincent Tao Hu, and Bjom Ommer. Diffusion models and representation learning: A survey. arXiv preprint arXiv:2407.00783, 2024.
Zhengyang Geng, Mingyang Deng, Xingjian Bai, J Zico Kolter, and Kaiming He. Mean flows for one-step generative modeling. arXiv preprint arXiv:2505. 2tt:::555..34444 /3447, 2025.
Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial networks. Communications of the ACM, 63(11))139-14) 2020.
Greg Heinrich, Mike Ranzinger, Hongxu, Yin, Yao Lu, Jan Kautz, Andrew Tao, Bryan Catanzaro, and Pavlo Molchanov. Radiov2.5: Improved baselines for agglomerative vision foundation models, 2024. URL https:://aaaaa
org/ abs 22412. 07679.
Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilibrium. Advances in neural information processing systems, 30, 2017.
Shuming Hu. X post on representation alignment discussion. https: //xp::/xx... /xx com/ Stnn//hhuingg/// 2025. Accessed: 2025.
status /
Jing Huang, S.R. Kumar, M. Mitra, Wei-Jing Zhu, and R. Zabih. Image indexing using color correlograms. In Proceedings of IEEE Computer Society Conference Computer Vision and Patter Recognition, pp. 762-768, 1997. doi: 10.1 109/CVPR.19977.69911
Drew A Hudson, Daniel Zoran, Mateusz Malinowski, Andrew K Lampinen, Andrew Jaegle, James L McClelland, Loic Matthey, Felix Hill, and Alexander Lerchner. Soda: Bottleneck diffusion models for representation learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 19461-19471, 2024.
Sadeep Jayasumana, Srikumar Ramalingam, Andreas Veit, Daniel Glasner, Ayan Chakrabarti, and Sanjiv Kumar. Rethinking fid: Towards a better evaluation metric for image generation. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition (CVPR), pp. 9307-9315, June 2024.
You Jiacheng. X post representation alignment. https:
1957073 0257073253772 66388225, 2025a. Accessed: 2025.
You Jiacheng. X on www.srreeeeeenetnaaiiipooaoooooa apppstatttn worernnmmenn-ffooooouop.-
sfatuss/1951112244670100608 2025b. Accessed: 2025.
Dengyang Jiang, Mengmeng Wang, Liuzhuozheng Li, Lei Zhang, Haoyu Wang, Wei Wei, Guang Dai, Yanning Zhang, and Jingdong Wang. No other representation component is needed: Diffusion transformers provide representation guidance by themselves. arXiv preprint 02280 2025.
Jiatao Jing, Gu Wu, Yuanpeng Zhou, Yuxin Xu. Zixum Sun, Xinhao Ni, Yazhou Wang, Chunmeng Li, Hao Tang, Ruihua Chen, et al. Dart: Denoising autoregressive transformers for scalable text-to-image generation. arXiv preprint arXiv:2410.08/55 2024.
12

---

Published as a conference paper at ICLR 2026
Minguk Kang, Richard Zhang, Connelly Baress Nga Tran, Saurabh Kar, and Jun- Yan Zhu. Distilling diffusion models into conditional gans. arXiv preprint S.xiv:2405.05967,, 2024.
Diederik P Kingma and Jimmy Ba. Adam: method for stochastic optimization. arXiv preprint 111.6944, 2014.
Theodoros Kouzelis, Efstathios Karypidis, Ioannis Kakogeorgiou, Spyros Gidaris, and Nikos Ko- modakis. Boosting generative image modeling via joint image-featur synthesis. arXiv preprint arXiv:2504. /6064, 2025.
Tuomas Kynkäänniemi, Tero Karras, Samuli Laine, Jaakko Lehtinen, and Timo Aila. Improved precision and recall metric for assessing generative models. Advances in neural information processing systems, 32, 2019.
Tuomas Kynkäänniemi, Tero Karras, Miika Aittala, Timo Aila, and Jaakko Lehtinen. The role of imagenet classes fréchet inception distance. In International Conference on Learning Reprexentations, 2023.
Xingjian Leng, Jaskirat Singh, Yunzhong Hou, Zhenchang Xing, Saining Xie, and Liang Zheng. Repa-e: Unlocking vae for end-to-end tuning with latent diffusion transformers. arXiv preprint arXiv:2504. 10483, 2025a.
Xingjian Leng, Jaskirat Singh, Ryan Murdock, Ethan Smith, Rebecca Li, Yunzhong Hou, Zhenchang Xing, Saining Xie, and Liang Zheng. Family of End-to-End Tuned VAEs for Supercharging T21 Diffusion Transformers. https: https://endZend /end2endddddddd di diffusion...iiii github io/ lerappa eeeaa-e-ae e-t2i/, 2025b.
Tianhong Li and Kaiming He. Back to basics: Let denoising generative models denoise. arXiv preprint arXiv:2511.13720, 2025a.
Tianhong Li and Kaiming He. Back basics: Let denoising generative models denoise. arXiv preprint arXiv:2511.1372 137220, 2025b.
Loshchilov. Decoupled weight decay regularization, arXiv preprint arXiv: 2017.
David G Lowe. Object recognition from local scale-invariant features. In Proceedings of the seventh IEEE international conference computer vision, volume 2, pp. 1150-1157. leee, 1999.
Nanye Ma, Mark Goldstein, Michael S Albergo, Nicholas M Boffi, Eric Vanden-Eijnden and Saining Xie. Sit: Exploring flow and diffusion- based generative models with scalable interpolant transformers. In European Conference on Computer Vision, pp. 23 40. Springer, 2024.
Yiyang Ma, Xingchao Liu, Xiaokang Chen, Wen Liu, Chengyue Wu, Zhiyu Wu, Zizheng Pan, Zhenda Xie, Haowei Zhang, Xingkai Yu, et al. Janusflow: Harmonizing autoregression and rectified flow for unified multimodal understanding and generation. In Proceedings of the Computer Vision and Puttern Recognition Conference, pp. 7739-7751, 2025a.
Zehong Ma, Longhui Wei, Shuai Wang, Shiliang Zhang, and Qi Tian. Deco: Frequency-decoupled pixel diffusion for end-to-end image generation. arXiv preprint a1X1v:2511.19665, 2025b.
Charlie Nash, Jacob Menick, Sander Dieleman, and Peter Battaglia. Generating images with sparse representations. In International Conference on Machine Learning, pp. 7958-7968. PMLR, 2021.
OpenAl. Gpt-5, https::////// //pp:////p//.a/.//////e// com, 2025.
Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy Vo, Marc Szafraniec, Vasil Khalidov, Pierre Fermandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et al. Dinov2: Learning robust visual features without supervision. Transactions on Machine Learning Research Journal, pp. 1-31, 2024.
Sayak Paul. Cmmd-pytorch: Pytorch implementation of cmmd metric. https://github.. sayakpaul cttl://mmmppppp coommnd-pytorhhh 2024. Accessed: 2025.
William Peebles and Saining Xie. Scalable diffusion models with transformers. In Proceedings of the IEFF/CVF International Conference on Computer Vision, pp. 4195-4205, 2023.
13

---

Published as a conference paper at ICLR 2026
Pablo Pernias, Dominic Rampas, Mats Leon Richter, Christopher Pal, and Mare Aubreville. Würstchen: An efficient architecture for large-scar scale text-to-image diffusion models. In The Twelfth International Conference Learning Representations, 2023.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pp. PMLR, 2021.
Nikhila Ravi, Valentin Gabeur, Yuan-Ting Hu, Ronghang Hu, Chaitanya Ryali, Tengyu Ma, Haitham Khedr, Roman Rädle, Chloe Rolland, Laura Gustafson, Eric Mintun, Junting Pan, Kalyan Vasudev Alwala, Nicolas Carion, Chao- Yuan Wu, Ross Girshick, Piotr Dollár, and Christoph Feichtenhofer. Sam Segment anything images and videos. arXiv preprint 408002248..0074, 2024. URL https: / aaxiiiiii.. org/ abs/ / 2408 00714.
Ethan Rublee, Vincent Rabaud, Kurt Konolige, and Gary Bradski. Orb: An efficient alternative sift or surf. In 2011 International conference on computer vision, pp. 2564-2571. Ieee, 2011.
Tim Salimans, Ian Goodfellow, Wojeiech Zaremba, Vicki Cheung, Alec Radford, and Xi Chen. Improved techniques for training gans. Advances in neural information processing systems, 29, 2016.
Axel Sauer, Katja Schwarz, and Andreas Geiger. Projected gans converge faster. In Advances in Neural Information Processing Systems, volume 34, pp. 17480-17492, 2021.
Axel Sauer, Katja Schwarz, and Andreas Geiger. Stylegan-xl: Scaling stylegan to large diverse datasets. In ACM SIGGRAPH 2022 conference proceedings, pp. 1-10, 2022.
Axel Sauer, Tero Karras, Samuli Laine, Andreas Geiger, and Timo Aila. Stylegan-t: Unlocking the power of gans for fast lcage text-to-image synthesis. arXiv preprint arXiv:2301.095/5 2023.
Axel Sauer, Dominik Lorenz, Andreas Blattmann, and Robin Rombach. Adversarial diffusion distillation. In European Conference Computer Vision, pp. 218-234. Springer, 2024.
Eli Shechtman and Michal Irani. Matching local self-similarities across images and videos. In 2007 IEEE Conference Computer Vision and Pattern Recognition, pp. 1-8, 2007. doi: 10.1 109/ CVPR.2007.383. 188..8398...
Oriane Siméoni, Huy V. Vo, Maximilian Seitzer, Federico Baldassarre, Maxime Oquab, Cijo Jose, Vasil Khalidov, Mare Szafraniec, Scungeun Yi, Michaël Ramamonjisoa, Francisco Massa, Daniel Haziza, Luca Wehrstedt, Jianyuan Wang, Timothée Darcet, Théo Moutakanni, Leonel Sentana, Claire Roberts, Andrea Vedaldi, Jamie Tolan, John Brandt, Camille Couprie, Julien Mairal, Hervé Jégou, Patrick Labatut, and Piotr Bojanowski. DINOv3, 2025. URL https:////// / /ttp/ixxxvoorg.//// abs/2508.. 1004.
Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv: 1409. 1556, 2014.
George Stein, Jesse C. Cresswell, Rasa Hosseinzadeh, Yi Sui, Brendan Leigh Ross, Valentin Ville- croze, Zhaoyan Liu, Anthony L. Caterini, J. Eric T. Taylor, and Gabriel Loaiza-Ganem. Exposing flaws of generative model evaluation metrics and their unfair treatment of diffusion models. In Advances in Neural Information Processing Systems, 2023.
Shitao Tang, Fuyang Chen, Aviral Joshi Bose, Kun Zhang, Shengjie Bi, Zhenyu Wang. Jiacheng Feng, Haozhe Cai, Alexander Liu, Antonio Torralba, et al. Ardit: Autoregressive diffusion transformers. arXiv preprint arXiv:2406.008.0 2024.
Dmitry Ulyanov, Andrea Vedaldi, and Victor Lempitsky. Instance normalization: The missing ingredient for fast stylization. arXiv preprint arXiv: 160.00027 2016.
Runqian Wang and Kaiming He. Diffuse and disperse: Image generation with representation regularization. arXiv preprint arXin:2506.09007, 2025.
14

---

Published as a conference paper at ICLR 2026
Shuai Wang, Ziteng Gao, Chenhui Zhu, Weilin Huang, and Limin Wang. Pixnerd: Pixel neural field diffusion. arXiv preprint arXiv:2507.23268, 2025a.
Shuai Wang, Zhi Tian, Weilin Huang, and Limin Wang. Ddt: Decoupled diffusion transformer. arXiv preprint ar arXiv:2504.057. 2025b.
Ziqiao Wang, Wangho Zhao, Yuhao Zhou, Zekai Li, Zhiyuan Liang, Mingjia Shi, Xuanlei Zhao, Pengfei Zhou, Kaipeng Zhang, Zhangyang Wang, Kai Wang. and Yang You. Repa works until it doesn't. t: Etrly-stopped, holistic alignment supercharges diffusion training. arXiv preprint 1177722505.6679, 2025c.
Haoyu Wu, Diankun Wu, Tianyu He, Junliang Guo, Yang Ye, Yueqi Duan, and Jiang Bian. Geometry forcing: Marrying video diffusion and 3d representation for consistent world modeling. arXiv preprint NxXi::2077.09982. 2025.
Mengping Yang, Ceyuan Yang. Yichi Zhang, Qingyan Bai, Yujun Shen, and Bo Dai. Revisiting the evaluation of image synthesis with gans. In Advances in Neural Information Processing Systems, 2023.
Jingfeng Yao and Xinggang Wang. Reconstruction vs. generation: Taming optimization dilemma in latent diffusion models. arXiv preprint arXiv:2501.014223 2025.
Sihyun Yu, Sangkyung Kwak, Huiwon Jang, Jongheon Jeong, Jonathan Huang, Jinwoo Shin, and Saining Xie. Representation alignment for generation: Training diffusion transformers is easier than you think. arXiv preprint arXiv:2410.069400, 2024.
Xiangdong Zhang, Jiaqi Liao, Shaofeng Zhang, Fanqing Meng, Xiangpeng Wan, Junchi Yan, and Yu Cheng. Videorepa: Learning physics for video generation through relational alignment with foundation models, arXiv preprint arXiv:2505. 23656, 2025.
15

---

Published as a conference paper at ICLR 2026
A
DETAILED CORRELATION ANALYSIS
Tecadke Happlng
8
88
61
90
3
(a) Linear Probing vs FID (SiT-XL-2)
(b) Linear Probing vs FID (SiT-B-2)
Happling
0.852
0.852 = 24
27
106 Oipatel Spramm
125 Sharral Mredtnd
(c) Spatial Structure FID (SIT-XL-2)
(d) Spatial Structure vs FID (SiT-B-2)
Figure 10: Detailed correlation analysis with encoder labels. Top row shows linear probing accuracy vs FID correlation for SiT-XL-2 and SiT-B-2 models. Bottom shows spatial structure (LDS) vs FID correlation. Each point is labeled with its corresponding encoder number (see legend). The spatial structure metric demonstrates significantly stronger correlation with generation quality across both model scales.
16

---

Published as a conference paper at ICLR 2026
B SPATIAL SELF-SIMILARITY METRICS
Figure 11: Explaining semantic region self-similarity (SRSS) metric. Visual explanation of SRSS metric: For each image, we compute mask using SAM2 and select anchor cechor-positive-neaaiv triplets. The SSM measures the difference in cosine similarity between anchor- and unchor-negatioe pairs (outside mask). Intuitively, larger values indicate that patches the same semantic region are more similar than patches on unrelated regions - indicating better spatial structure preservation.
pairs (within mask)
Setup. Given an image I and vision encoder E, we extract patch tokens X' = E(() E RTxD with T H W and indices E [T] placed H W lattice. Let d [T] x [T] N be the Manhattan distance, and let the (cosine) self-similarity kernel be
(x,, Xe
Kx t) = X la |
[-1,1],
a standard local self-similarity measure (Shechtman & Irani, 2007).
Patch token representation: X EI) Tx where T H x W is the spatial grid of patches
(patch index set [T]). Also let d [T] x [T] + N be Manhattan distance on the H x W grid.
Self-similarin kernel: K [T] x [T] + R measuring self-similarity (Shechtman & Irani, 2007)
between patch tokens. We use the cosine kernel (x (') = (x,, xx: ((]x/[[[xlla||xx ||2).
Spatial self-similarity metric: functional m x D + IR measures how self-similarity K between patch tokens varies with lattice distance d. Intuitively, larger values indicate stronger
spatial organization (near patches more similar to each other than far away patches).
We next discuss several straightforward metrics for measuring the spatial self celf-similarity structure of the target representations.
Local Distant Similarity (LDS).. We first consider simple correlogram contrast (local vs. distant) metric (Huang et al., 1997):
LDS(X) = (t, t') t(t,t') < Tnearl E[Kx(t, (t, d(t, 2 Ttarl-
where F Tnear and Ffar are the hyperparameters. By default we use Tnear Tfar = H/2. We found correlation to be robust to their exact choices. Intuitively, larger values indicate that average, patches that are closer to each other are more similar than patches that are further away.
Correlation Decay Slope (CDS). Given the patch tokens X' E(I) RTXD compute the spatial correlogram gæ (8) = Et,x(t,, l') (')ld(t, (') = 6) for distances E (1,...,A). We then fit
least-squares line 0x (6)
,38 and define:
CDS(X')= = -
where 3 is the fitted slope. Larger CDS values indicate faster similarity decay with distance, suggesting stronger spatial organization.
Semantic-Region Self-Similarity (SRSS). For each image, we first sample a binary mask M # containing random object using SAM2 (Ravi et al., 2024) and downsampled to H x W. We then select three types of points: anchor, positive, and negative. The anchor and positive points are sampled from within the mask (should have similar semantics), while the negative point is sampled from outside the mask (less related to anchor). Conceptually, if the encoder feature representation preserves the spatial structure, the anchor and positive points should have higher cosine similarity,
17

---

Published as a conference paper at ICLR 2026
while the anchor and negative points should have lower cosine similarity. Thus, define the Spatial Structure Metric (SSM) as:
SSM(P) M [cos| Punchorr Ppos) cos Panchors Pneg)
where positive points are within Manhattan distance d S Tnear from the anchor within the mask, and negative points are at distance d 2 ffar outside the mask.
RMS Spatial Contrast (RMSC). Finally, consider simple contrast metric capture the spatial contrast between patch token representations. Given normalized features &; x 2l2 for each patch E, we compute:
RMSC(2)
12
T
where x x= E X is the mean of normalized features across all spatial locations. Higher RMSC values indicate greater spatial diversity in the feature representations, suggesting preserved spatial structure, whereas lower values indicate more uniform feature distributions indicating a loss of spatial structure (typically happens global information dimnishes spatial structure (Siméoni et al., 2025)).
18

---

Published as a conference paper at ICLR 2026
C
VISION ENCODER VARIATION
S108-2 (REM
KERA
0.
40.8
01.1
56.6 001
46.5
$3.5
5006
50.5
500
66A
ML1
BHNON2
DONOV
WHANKL
LangiS
SpataiPe
CHAADOO)
DEND
CSEPP
HAAR
(a) SiT-B-2 2 (130M parameters)
51060 JREIN
REDA
IREPA 21.1
45.0
Sirpo
32.1
37.6
213
221
412 213
223
2OLD
Lif
2916
(b) SiT-L-2 (458M parameters)
SIT-XL-2 REPA
Avg:
Aug:
SIT-L-22 IREPA (Ours)
gFID Scare (100K Steps)
25.2
25L1
134 713
29.4
20.0 22.0
21.2
K1
15 T74
DINOv3
Rito
2016
(c) ST--XL-2 (675M parameters)
Figure 12: Variation in vision encoders. Ablation studies showing iREPA improvements across different vision encoders for SiT-B-2, SiT-L-2, and SiT-XL-2 models. iREPA consistently improves generation quality across all encoders and model sizes.
19

---

Published as a conference paper at ICLR 2026
D ENCODER DEPTH VARIATION
S128-2 (REM
KERA
0.
40.8
01.1
56.6 00.
46.5
$3.5
50.5
500
6AA
ML1
BHNON2
DONOV
WHANKL
Langgi
SpataiPe
CHAADOO)
DENO
HAAR
(a) S-T-B-2 with encoder layers
S17822
Auge
Auge
515 a0a
JOLS
53.0
25-7
10.01
TLB
OOLS
SEA.
513
6.6
136
(b) SiT-B-2 with 8 encoder layers
Figure 13: Effect of encoder depth on iREPA performance. Comparison of SiT-B-2 performance with different encoder depths (6 8 layers) across various vision encoders. The results show consistent improvements with iREPA regardless of encoder depth, with slightly better performance observed with 8 encoder layers. Both configurations demonstrate significant FID improvements when spatial regularization is applied.
20

---

Published as a conference paper at ICLR 2026
E COMPREHENSIVE RESULTS ACROSS VISION ENCODERS
E SIT-B/2 RESULTS
Vision Encoder
ISt
FID4
sFID
Prec.t
Rec.
VIT-L
59.44
24.84
6.46
0.584
0.645
+IREPA
69.70
21.26
6.64
0.601
0.644
DINOv2-B
64.92
22.75
6.54
0.593
0.653
+IREPA
70.88
21.40
6.77
0.597
0.653
DINOv2-L
66.65
22.46
6.53
0.594
0.650
+IREPA
72.41
20.99
6.84
0.600
0.650
DINOv3-B16
62.57
23.91
6.77
0.584
0.647
+IREPA
75.48
19.65
6.68
0.606
0.649
DINOv3-H16
61.91
24.82
6.57
0.579
0.645
+RREPA
71.76
21.80
6.84
0,592
0.655
DINO-B
58.37
24.71
6.32
0.586
0.635
+iREPA
57.55
24.41
6.43
0,587
0.638
LangPE-G
58.42
25.37
6.32
0.583
0.641
+iREPA
66.88
21.62
6.65
0.604
0.649
MoCov3-B
57.61
25.60
6.25
0.579
0.640
+iREPA
61.41
22.80
6.46
0.596
0.639
PE-B
58.86
25.78
6.44
0.58
+iREPA
69.50
21.15
6.66
0.601
0.659
PE-G
56.09
26.84
6.45
0.568
0.641
+iREPA
70.43
21.52
6.58
0.598
0.651
SpatialPE-B
59.19
24.28
6.37
0.586
0.646
+iREPA
61.50
22.71
6.41
0.604
0.644
SpatialPE-L
62.13
23.87
6.51
0.59
0.65
+iREPA
68.83
21.07
6.59
0.607
0.651
WebSSL-1B
63.20
23.48
6.33
0.593
0.646
+iREPA
73.54
19.74
6.63
0.612
0.646
WebSSL-2B
66.56
22.27
6.69
0.596
0.654
+iREPA
72.13
20.47
6.89
0.609
0.648
WebSSL-300M
62.06
23.64
6.48
0.590
0.646
+iREPA
66.83
21.46
6.71
0.606
0.653
WebSSL-3B
65.91
22.55
6.53
0.598
0.654
+iREPA
71.98
20.64
6.71
0.607
0.650
Table 5: SiT-B/2 performance across vision encoders 400K iterations. iREPA consistently improves generation quality. All baselines are reported using vanilla-REPA (Yu et al., 2024) for training.
21

---

Published as a conference paper at ICLR 2026
E.2 SIT-L/2 RESULTS
Vision Encoder
1st
FIDL
sFID
Prec.1
Rec.
CLIP-ViT-L
107.7
10.6
5.20
0.682
0.643
+iREPA
115.8
9.41
5.15
0.689
0.653
DINOv2-B
113.3
9.61
5.11
0.684
0.653
+REPA
113,9
9.36
5.11
0.690
0.655
DINOv3-B16
115.3
9.41
5.24
0.687
0.651
+iREPA
118.1
9.06
5.25
0.685
0.650
DINOv3-HI6+
110.7
10.6
5.32
0.677
0.652
+iREPA
120.5
9.64
5.29
0.681
0.663
LangPE-G
102.8
11.2
5.13
0.684
0.637
+iREPA
112.7
9.37
5.02
0.690
0.653
MoCov3-B
102.1
11.0
5.14
0.685
0.635
+iREPA
104.7
10.1
5.00
0.690
0.641
PE-G
98.38
12.6
5.31
0.670
0.646
+iREPA
117.1
9.65
5.33
0.683
0.651
WebSSL-IB
107.9
10.5
5.13
0.686
0.644
+iREPA
121.5
8.62
5.01
0.696
0.653
Table 6: SiT-L/2 performance across vision encoders 400K iterations. iREPA consistently improves generation quality across all encoders, with particularly strong gains in FID and metrics. All baselines reported using vanilla-REPA (Yu et al., 2024) for training.
22

---

Published as a conference paper at ICLR 2026
F ADDITIONAL EXPERIMENTAL RESULTS
Do
Linear Prolonn Valehater
PP
Langre
CRAIRO
JPP4
14A0
Figure 14: Global information in mean patch tokens. We find that in addition to [CLS] token, mean of patch tokens also contains substantial global semantic information. While this helps improve global performance, it reduces the contrast between individual patch tokens, potentially hindering spatial structure transfer. We find that we can remove some of this global information (through mean of patch tokens) to improve spatial structure transfer during representation alignment ($44).
Vision Encoder
1St
FID
sFID
Prec.t
Rec.t
DINOv2-B
262.97
1.98
4.60
0.799
0.610
+iREPA (Ours)
268.79
1.93
4.59
0.799
0.600
DINOv3-B
261.18
1.99
4.58
0.799
0.609
+iREPA (Ours)
272.41
1.89
4.58
0.799
0.600
WebSSL-1BB
250.53
2.24
4.61
0.809
0.580
+iREPA (Ours)
271.59
1.90
4.58
0.798
0.609
PE-G
238.37
2.44
4.57
0.805
0.585
+iREPA (Ours)
275.36
1.93
4.59
0.796
0.606
Table 7: Generation quality with classifier-free guidance. Comparison of REPA vs iREPA with CFG (scale 2.0) across different vision encoders. All experiments use SiT-XL/2 trained for 400K iterations with 250 sampling steps. iREPA consistently improves both IS and FID metrics across all encoders.
SCTELL 190M
SERLLS IROM
SURELG + 19TM
30
SERLSS IROM
DINOv2-B
DINOC3-B
20-
Wel68L-1R
PE-Core-O
F10 Score
811
35
7.7
106K
2006
3005 411K
100K
2016
2015 455k
106K
2016
3005 421k
300K
20M5 455k
Traiting Stege
Traintng Steps
Training Stegs
Traintng Steps
SESLL REN
SETALO R21%
SERLL REN
-On RERUL RIT -O- Nor Si1 Hoom
-O+ SUT Join
-O+ NITSLO J0mm
CLIPA
20
DINOv3--E+
LangPE-
MoCord-B
F1O Score
3.0
10
10.0
150
/8
74
3.0
100K
455K
2018
400K
100K
423K
2001 Training
Transing Steps
Traising
Traiting Steps
Figure 15: Accentuating spatial features helps consistently improve convergence speed. Results for SiT-XL/2 with REPA and iREPA.
23

---

Published as a conference paper at ICLR 2026
w/ Standard Projection
Lightsssight Conv Projection
who Spenial Beig.
wel Spotial Keg.
Figure 16: Visualizing the impact of two straightforward improvements to enhance spatial fea- ture transfer. First, we find that standard MLP projection layer (top-left) losses spatial information while transferring features from the pretrained encoder (after projection) to diffusion model (before projection). Instead using a simpler convolution layer better preserves the spatial information transfer (top-right). Second, we observe that vision encoder features often have limited spatial contrast ($2. This causes the tokens in one semantic region (e.g., dog) to show quite decent cosine similarity with unrelated tokens (e.g., background). We address this with simple spatial regularization layer, leading to better generation
which accentuates the spatial contrast in the learned representation performance, Best results are obtained while using both (refer Table 2).
24

---

Published as a conference paper at ICLR 2026
G IMPLEMENTATION DETAILS
Training setup. We follow the same setup as in REPA (Yu et al., 2024) unless otherwise specified. All training is conducted on the ImageNet (Deng et al., 2009) training split. For preprocessing, we adopt the protocol from ADM (Dhariwal & Nichol, 2021), where each image is center-cropped and resized to 256 x 256 resolution. We use stabi haabbiittai/ssie wettttaiidddaaeeeeeeee /ss-/a-aaf--ttteee our diffusion training and inference. For spatial normalization layer we use E [0.6, 0.8) and for projection layer we use a convolutional layer with kernel size and padding For optimization, use AdamW (Kingma & Ba, 2014; Loshchilov, 2017) with constant learning rate of 1 x 10-4. and a global batch size of 256. During training, we use bfloat16 mixed precision and torch. - comp e to accelerate training, and gradient clipping and exponential moving average (EMA) to the generative model for stable optimization.
VAE (AI, n.d.) throughout
For REPA-E (Leng et al., 2025a) and JiT (Li & He, 2025a) experiments, source implementation. For MeanFlow (Geng et al., 2025) experiments, we received the implementa- tion (including REPA) from original authors and adapt it introduce the two straightforward changes for iREPA (54).
the official open-
Evaluation. For image generation evaluation, we strictly follow the ADM setup (Dhariwal & Nichol, 2021). We report generation quality using Fréchet inception distance (gFID) (Heusel et al., 2017), structural FID (sFID) (Nash al., 2021), inception score (IS) (Salimans al., 2016), precision (Prec.) and recall (Rec.) (Kynkäänniemi et al., 2019), measured on 50K generated images. For sampling, we follow the approach in REPA (Yu al., 2024), using the SDE Euler-Maruyama sampler with 250 steps. For JiT (Li & He, 2025a), we use 50 inference steps following official implementation.
H MORE DISCUSSION ON RELATED WORK
Classical spatial features in computer vision. Spatial feature extraction has long been fundamental to computer vision. Classical methods like SIFT (Lowe, 1999), HOG (Dalal & Triggs, 2005), SURF (Bay al., 2006), and ORB (Rublee et al., 2011) providing robust local descriptors for tasks ranging from object detection to image matching. While these handcrafted features excel at capturing local spatial patterns and geometric invariances, recent work in generative modeling has primarily focused on representations from modern self supervised methods that demonstrate strong global classification performance, such as DINOv2 (Oquab et al., 2024) and CLIP (Radford et al., 2021). Our findings suggest a different perspective: since spatial structure preservation is critical for generation quality, even classical spatial features could potentially improve diffusion training when properly aligned. This highlights the potential of leveraging the full spectrum of spatial feature extractors, from traditional handcrafted descriptors to modern learned representations, provided that they maintain strong spatial coherence.
Pretrained visual encoders for generative models. Pretrained visual encoders have supported generative models in several capacities: as discriminators to accelerate GAN convergence (Goodfellow et al., 2020; Sauer et al., 2021; 1; 2022; 2023; Radford et al., 2021), as teachers in adversarial distillation for diffusion models (Sauer et al., 2024; Kang et al., 2024), and more recently alignment targets. In GANs (Goodfellow et al., 2020), pretrained features have not only improved convergence speed but also enabled scaling large datasets, as demonstrated by StyleGAN-XL (Sauer et al., 2022) and StyleGAN-T (Sauer et al., 2023) with CLIP (Radford et al., 2021) features. For diffusion models, adversarial distillation leverages pretrained encoders guide student networks toward higher-fideltty samples, showing clear improvements in perceptual quality. In particular, REPA (Yu al., 2024) aligns diffusion features with external encoders, demonstrating that representation alignment can improve both generation convergence and quality. Building this direction, we focus not only on alignment but specifically on spatial structure perseverance rather than their discriminative capabilities.
Denoising transformers. Transformer architectures have become the dominant backbone for scalable generative modeling, with various formulations including diffusion transformers (Peebles & Xie, 2023) and flow matching variants (Ma et al., 2024). Recent architectures like GenTron (Chen et al., 2023) scale transformers to over 3B parameters for text-to-image synthesis. U-ViT (Bao et al., 2023) demonstrates that pure transformer backbones without U-Net inductive biases can achieve competitive performance. ARDIT (Tang et al., 2024) and DART' (Jing al., 2024) explore
25

---

Published as a conference paper at ICLR 2026
autoregressive formulations that combine denoising with sequential generation, enabling flexible trade-offs between quality and speed. Despite these architectural advances, training these models from scratch remains computationally expensive, often requiring millions of iterations to achieve good generation quality. While representation alignment methods like REPA (Yu et al., 2024) have shown that pretrained features can dramatically accelerate convergence, the mechanisms behind this improvement remained unclear. Our analysis reveals that the key benefit comes from preserving spatial structure rather than semantic alignment, explaining why certain encoders provide stronger acceleration than others and guiding the design of more effective alignment strategies. While (Wang et al., 2025c) propose improvements to REPA training through early stopping. they hypothesize that "REPA predominantly distills global semantic information while leaving structural information untapped." In contrast, our analysis reveals that spatial structure (not global semantic information) already plays very significant role in the effectiveness of REPA ($2, $3). Thus, while we indeed find that spatial structure remains underexploited ($44) in REPA, we surprisingly find that majority of improvements in REPA are already coming from introducing an inductive bias for spatial structure (not global information).
Denoising as self-supervised learning task. The connection between denoising and representation learning has been explored from multiple perspectives. Early work by Abstreiter et al. (2021) extended diffusion objectives for representation learning, demonstrating that denoising naturally learns meaningful features. SODA (Hudson et al., 2024) introduces a diffusion model with an information bottleneck to learn compact representations. Chen et al. (2024) deconstruct diffusion models and find that a simple denoising autoencoder suffices for strong self-supervised performance. Wang & He (2025) introduce a dispersive loss encourage internal representation diversity in diffusion models, improving generative performance without external encoders. Similarly, Jiang et al. (2025) propose Self- Reff-Representttinn Alignment (SRA) to align diffusion transformer's latent features across noise levels, providing self-guidance without an auxiliary model. These works establish denoising as fundamental self-supervised task that naturally encourages learning robust features. Our findings complement this view by showing that when diffusion models are aligned with strong spatial representations from self-supe encoders, both the generative and discriminative capabilities improve, suggesting that spatial structure preservation is a key factor in this synergy.
Scaling self-supervised vision encoders. Recent years have seen remarkable progress in scaling self-supervised vision models to unprecedented sizes and datasets. DINOv3 (Siméoni et al., 2025) trains 7B parameter VIT on 1.7 billion images without labels by aligning representations from different augmentations. WebSSL (Fan et al., 2025) demonstrates that visual models trained more than 2 billion images can match language-supervised models like CLIP (Radford et al., 2021) 1) on vision-language tasks without language supervision. C-RADIO (Heinrich et al., 2024) combines multiple teacher models through distillation to create versatile encoders that excel across diverse visual domains. I-JEPA (Assran et al., 2023) explores predictive architectures that learn by predicting masked regions in representation space, instead of reconstructing pixels directly. SAM (Ravi et al., 2024) specializes in promptable segmentation through large-scale supervised training. The Perception Encoder family (Bolya et al., 2025; Cho et al., 2025) shows that intermediate features often outperform final representations for dense prediction tasks. While these models are typically evaluated on global tasks like image classification, discriminative benchmarks does not necessarily translate to better generation quality. Instead, we find that encoders preserving spatial structure, regardless of their classification securacy-proviie the most benefit for diffusion training. This suggests that the evaluation metrics for self-supervised encoders should be reconsidered when targeting generative applications, with spatial coherence being as important as semantic understanding.
work reveals crucial insight: strong performance
I NOTE ON LLM USAGE
We use GPT-5 (OpenAI, 2025) for considering different variations of the spatial structure metrics discussed in the paper $3. Furthermore, all figures in the paper are directly generated from our experiment logs and checkpoints using Claude-Code (Anthropic, 2025). Additionally we use LLM help for searching and formulating relevant work in Appendix H. We use cursor in some parts to help with paper writing.
26

---

Published as a conference paper at ICLR 2026
J ADDITIONAL DISCUSSION ON SPATIAL STRUCTURE METRICS
Spatial structure metrics can be used to measure the representation gap. Yu et al. (2024) use linear probing measure the representation gap, using the increase in validation accuracy to explain the effectiveness of REPA. We find that spatial structure metrics (SSM) can also be used to measure the representatton gap and explain the improvements across models. As shown in Figure 17, we see that representation alignment helps close the gap between SSM performance of a pretrained encoder like DINOv2 and the diffusion features.
DO0O Ebooker ABN
6,42
1,35 Sogre
LD% Siore
Scase
0.2
6.1
0.5
12
D0402-8 PB- BINb
SIA22
SI7-2
ST3L2
Lager Indoxx
Layper 20:0e
VIsaE Encaders
Model SI0
100
DINDIG Spuiii 2ag
Figure 17: Representation alignment as bridging the spatial feature gap. a) We find that representation alignment can also be seen as bridging the spatial feature gap between diffusion and vision encoder patch features. b) iREPA ($4) accentuates the spatial features of the vision encoder (at cost of some global information) - helping achieve better generation performance. e) iREPA helps consistently improve performance across different vision encoders. d) Spatial structure improvements scale with model size. All results are reported with SiT-XL/2 at 400k iterations.
0,260
4.118
0,852
8,802 21
V 27
Figure 18: Spatial structure better correlates with generation quality than linear probing. Correlation analysis across 27 vision encoders. Left two: Linear probing accuracy vs FID for SiT-XL-2 (Pearson = -0.26) and SiT-B-2 (r -0.12) shows weak correlation. Right two:
Spatial structure (LDS) FID for SiT-XL-2 (r = -0.55) and SiT-B-2 (r = 0.899 shows strong correlation. See Figure in Appendix for detailed plots with encoder labels.
27

---

Published as a conference paper at ICLR 2026
K
ADDITIONAL ANECDOTAL COMPARISONS
SHAC--MG 20000 S0000
Uiced Prllng VM Aoo (%0
2611
10 16 S
BID Sove
wwbos--ttoob
02
Figure 19: Motivating anecdotes from recent SSL encoders spatial structure matters. Top row: Metrics comparison showing inverse relationship between ImageNet accuracy and generation quality. Left: PE-g achieves higher accuracy (82.8% vs 72.4%) but worse FID (32.3 vs 22.0) than Spatial-PE-g, with much lower spatial structure (LDS: 0.1 0.4). Right: WebSSL-dinov higher accuracy (76.0% vs 71.7%) but worse FID (26.1 vs 19.3) than DINOv2-b, with weaker spatial structure (LDS: 0.2 vs 0.4). Bottom row: Spatial cosine similarity visualizations confirm that encoders with better generation (Spatial-PE-g. DINOv2-b) maintain clear spatial coherence, while those optimized for classification (PE-g, WebSSL) lose spatial structure.
shows
0831
SUTXL2 + PG-G
SIPX2 SpatslFB-D
Prolinn 0001
SIFXL2 WebsssL11
STFXL/2 + SpaiolFE-D
Spattel Structune 11
82.6
33.3
5-3
35.0
26.1
63
583
21,1
20-
02
12
8.1
20-
FS-0
SpatsiPE
Figure 20: Motivating examples spatial structure matters. Top: Metrics comparison showing inverse relationship between ImageNet accuracy and generation quality. Left: PEcore-Gi -G, despite having significantly higher validation accuracy (82.8% vs. 53.1 1)), shows worse generation quality compared to -Espatial- (Bolya al., 2025). Right: Similarly, WebSSL- (Fan et al., 2025) also shows much better global performance (76.0% vs. 53. %), but worse generation. Bottom:
find that spatial structure instead provides better predictor of generation guality than global performance. See $3 for spatial structure metric. All results reported at 100K using SiT-XL/2 and REPA.
28

---

Published as a conference paper at ICLR 2026
L FURTHER DISCUSSION
Role of Spatial Normalization We visualize the impact of spatial normalization on vision encoder representations. Vision encoders can have significant global components overlays that limit the contrast between spatial tokens. For example, tokens in one semantic region can be highly correlated with tokens in unrelated regions (e.g.. background), reducing the spatial distinctiveness of features. Spatial normalization removes this global overlay to enhance contrast between different spatial tokens, allowing the model to better preserve local spatial structure while reducing the dominance of global information that can interfere with generation quality.
The following figures demonstrate this effect across different examples, showing how spatial normal- ization transforms the feature representations to emphasize spatial structure:
w/o Spatial Normalization Layer
w/ Spatial Normulizaaion Layer
PE-B McCoVa-B MAE-L DINOV3-B
0.4
0.2
Figure 21: Visualizing impact of spatial normalization (Example 1). The heatmaps show token similarity patterns before and after spatial normalization. Without normalization (left), global components create high correlations across unrelated regions. With spatial normalization (right), local spatial structure is enhanced while reducing global interference, resulting in more distinct semantic boundaries.
Spatial Normalization Layer
Spatial
Layer
PE-B McCoV3-B MAB-L DINOV3-B
1.0
8.4
-02
Figure 22: Visualizing impact of spatial normalization (Example 3). The heatmaps show token similarity patterns before and after spatial normalization. Without normalization (left), global components create high correlations across unrelated regions. With spatial normalization (right), local spatial structure is enhanced while reducing global interference, resulting in more distinct semantic boundaries.
29

---

Published as a conference paper at ICLR 2026
w/o Spatial Normalization Layer
w/ Spatial Normalization Layer
PE-8 McCoV3-8 MAE-1 DINOV3-B
9.6
Figure 23: Visualizing impact of spatial normalization (Example 2). Feature similarity maps demon- strate how spatial normalization improves spatial contrast. The original features (left) exhibit a global overlay that reduces distinction between foreground and background regions. After normalization (right), spatial tokens become more locally coherent, with clearer separation between different se- mantic regions.
w/o Spatial Normalization Layer
w/ Spatial Normalization Layer
PE-B MoCoV3-8 MAB-L DINOCS-B
0.
0.2
-22
Figure 24: Visualizing impact of spatial normalization (Example 4). The heatmaps show token
similarity patterns before and after spatial normalization. Without normalization (left), global components create high correlations across unrelated regions. With spatial normalization (right), local spatial structure is enhanced while reducing global interference, resulting in more distinct semantic boundaries.
30

---

Published as a conference paper at ICLR 2026
M ADDITIONAL RESULTS
M.1 ADDITIONAL. RESULTS AT HIGHER RESOLUTIONS
Setup. We conduct further experiments on ImageNet-512 (Deng et al., 2009) to evaluate the generalization of thr proposed spatial improvements at higher resolutions. We follow the same setup as REPA (Yu et al., 2024), and report results across different choice of pretrained encoders (DINOv2, DINOv3, WebSSL, PE, etc.). All results are reported using SiT-XL and SiT-B using 50 NFE at inference w/o classifier free guidance. Fig. 25 shows the results with REPA before and after application of spatial improvements (iREPA). We observe that the spatial improvements also generalize to higher resolutions. Furthermore, consistent gains are observed across different choice of pretrained encoders (DINOv2, DINOv3, WebSSL, PE, etc.).
SIGD DINDVOU
SIE DIMONSU816
SIEB RECOre
KTT WAASSL 15
43.1
37.0
91.
32
DEE
1927%
STAUU MSCom
SprtattE-
11.3
123
B1
12.5
33
10.4
161
11.
Zaga
Treniig
Figure 25: Convergence results at 512x512 resolution. Accentuating spatial features helps consis- tently improve convergence speed across different resolutions for both Imagenet 256 (Figure 7) and Imagenet 512 (above).
31

---

Published as a conference paper at ICLR 2026
M.2 ADDITIONAL RESULTS ON MULTIMODAL T21 TASKS
Setup. To further study the generalizability of the proposed spatial improvements beyond ImageNet, also perform extensive experiments multimodal T21 tasks. Following REPA (Yu al., 2024), we adopt MMDiT (Esser et al., 2024) as the diffusion backbone and apply REPA with various pretrained vision encoders (DINOv2, CLIP, WebSSL, PE. etc.). Same as REPA (Yu al.. 2024), all models are trained for 150K steps with a batch size of 256, and evaluated using an ODE sampler with 50 NFE. Fig. 26 shows the results with REPA before and after application of spatial improvements (iREPA). We observe that the spatial improvements also generalize to multimodal T2I tasks. Furthermore, consistent gains are observed across different choice of pretrained encoders (DINOv2, CLIP, WebSSL, PE, etc.).
REPA Avg: 5.00
IREPA Avg: 4.72
T21
IREPA (Ours)
gFID Score (150K Stepp)
6.26
3.00
0.30
3.25
375
BIG
HIOPLIS
M00M
18
Figure 26: Text-to-image generation across encoder variants. Accentuating transfer of spatial features from the target representation to the diffusion features consistently improves convergence speed for both imagenet (refer $4) and multimodal T2I tasks (above). Furthermore, consistent gains are observed across different choice of pretrained encoders (DINOv2, CLIP, WebSSL, PE, etc.).
32

---

Published as a conference paper at ICLR 2026
M.3 CORRELATION ANALYSIS WITHOUT OUTLIERS (MoCOV3-L LL AND MAE-L)
We repeat the correlation analysis from Section after removing MoCOv3-L and MAE-L, which identified outliers. Figure 28 shows the updated correlations across different model sizes. We observe that spatial structure still shows much higher correlation with generation performance over linear probing accuracy. Interestingly, after removing the outliers linear probing actually shows small positive correlation with gFID (i.c., as linear probing performance increases, the generation becomes worse). This trend is consistent with the observations discussed in Sec. 2, wherein often target representations with higher global semantic performance (linear probing accuracy) show similar or worse generation performance with representation alignment (REPA).
Poarsce Corr Smraa
Poursen Coey
Rearsen Cary
Rearsco Cory
Poarson Cory
Pourson Cory
singl
SHTXL
Lsear Accurnzy
Lieer Frohiny Accurecy
Frehing
Spetial acturo ILDS)
Spettal auture (LBS))
Spettal Stradtaee (LBS)
Correlation analysis without outliers. Across all model sizes (B, L, XL) spatial structure still shows much higher correlation (Pearson Ir| 0.85) with generation performance over linear probing accuracy (Pearson |r| < 0.38) after removing the outliers (MoCOv3-L and MAE-L). Interestingly, after removing the outliers linear probing actually shows a small positive correlation with gFID (i.e., as linear probing performance increases, the generation becomes worse). This trend consistent with the observations discussed in Sec. 2, wherein often target representations with higher global semantic performance (linear probing accuracy) show similar worse generation performance with representation alignment (REPA).
Figure 27:
33

---

Published as a conference paper at ICLR 2026
M.4 ADDITIONAL QUALITATIVE RESULTS
IREPA
REDA
IRETA
IREPA
Figure 28: Additional Qualitative Results comparing generation outputs before and after application of spatial improvements with REPA. All results are reported using PE-G (Bolya et al., 2025) as the pretrained vision encoder, 400K steps (80 epochs) and with classifier-free guidance scale of 4.0. Similar to quantitative improvements ($4), we also observe that spatial improvements (iREPA) also help improve the visual quality and coherence of the generated outputs.
34

---

Published as a conference paper at ICLR 2026
M.5 ADDITIONAL RESULTS WITH ALTERNATIVE EVALUATION METRICS
In addition to traditional evaluation metrics (Inception Score, FID, sFID, Precision, Recall), we also verify the robustness of the proposed findings with alternative evaluation metries for generation quality (Stein et al., 2023; Yang et al., 2023; Kynkäänniemi et al., 2023; Jayasumana et al., 2024). In particular, we use the CMMD metric (Jayasumana et al., 2024) with the PyTorch implementation from (Paul, 2024). Standard reference set from OpenAI ADM evaluation suite (Dhariwal & Nichol, 2021) is used as reference images. We next verify the robustness of the key findings from both $3 and $4 with the CMMD metric.
Spatial structure correlates much higher with generation performance than linear probing. To study robustness of analysis from $3, we repeat the large-scale correlation analysis across different vision encoders with CMMD metric (Jayasumana al., 2024). Results are shown in Figure 29, All results are reported using SiT-B/2 (100K steps) and REPA. All spatial metrics show much higher correlation (Pearson |r| > 0.88) with generation performance (CMMD) than linear probing (Pearson 0.074) demonstrating that key empirical findings from $3 are robust to the choice of evaluation metric (FID or CMMD).
|r|
turpe
Peecton Lorn
CHIDD CLO0K Stago
Premson Cart
CMMO D100K stppol
Pearson Carr
Pearmon
CMOD 0100K stog0)
Phwmson Corr
= 0.074
=- 0.887
- 0.805
E- 00,503
L 0,901
CMMD CTOOK
21
X2 O
Probing haturayy
Figure 29: Correlation analysis with CMMD metric. We repeat the correlation analysis from $3 with CMMD metric (Jayasumana et al., 2024) instead of gFID. All results are reported using SiT-B/2 (100K steps) and REPA. All spatial metries show much higher correlation (Pearson |r] > 0.88) with generation performance (CMMD) than linear probing (Pearson |r| 0.074). This demonstrates that key empirical findings from $3 are robust to the choice of evaluation metric.
Accentuating transfer of spatial features helps improve generation quality. We next study the generalizability of analysis from $4 with the CMMD (Jayasumana et al., 2024) metric addition to traditional evaluation metrics (gFID, sFID, IS, etc.). Results are shown in Table 8. Across various vision encoders, accentuating transfer of spatial features (iREPA) helps improve convergence speed with both CMMD and traditional evaluation metrics (IS, FID, sFID, Prec., Recall). Furthermore, consistent improvements are observed both with and without classifier-fre guidance (CFG).
35

---

Published as a conference paper at ICLR 2026
w/o CFG
w/ CFG
Vision Encoder
CMMDJ
ISt
FID
sFID
Prec.f
Rec.f
CMMD
ISt
FID
sFID
Prec.t
Rec.t
DINOv2.B
0.702
5.89
0.61
0.529
157.2
6.35
5.91
0.54
+REEPA
100K
0.652
77.92
16.9
6.26
0.66
0.61
0.484
179.3
5,15
6.23
0.78
0.54
DINOv2.B B
400K
0.455
127.4
7.76
5.06
0.70
0.66
0.320
263.0
1.98
4.60
0.80
0.61
#REPA
400K
0.438
128.6
7.52
4.89
0.71
0.65
0.310
268.8
1.93
4.59
0.80
0.60
63.64
21.4
0.60
#iREPA
100K
0
78.79
16.2
6.14
0.66
0.61
0.481
181.9
4,87
6.10
0.78
0.55
DINOv3-B
400K
0.474
126.7
8.10
5.06
0.70
0.66
0.336
261.2
1.99
4.58
0.61
#REPA
400K
0,441
132.9
7.13
4.93
0.71
0.66
0.314
272.4
1.89
4.58
0.80
0.60
WebSSL-LB
100K
0.825
53.87
25.5
6.57
0.61
0.59
0.622
124.0
9.59
6.37
0.76
0.52
#REPA
100K
0.683
77.47
16.6
6.18
0.66
0.61
0.484
177.6
5.09
6.11
0.79
0.54
WeASSL-IB
0.512
116.9
9.39
5.14
0.64
0.354
250.5
2.24
4.61
0.58
+REPA
400K
0.445
130.8
7.48
4.91
0.70
0.65
0.311
271.6
1.90
4.58
0.80
0.61
PE-Core-G
100K
0.922
42.74
32.3
6.70
0.57
0.59
0.714
97.2
14.1
6.56
0.71
0.53
+REPA
100K
0.697
75.01
18.1
6.03
0.64
0.61
0.525
176.8
5.66
6.08
0.77
0.54
Core-G
#REPA
400K
0.458
132.00
7.78
5.02
0.70
0.65
0.322
275.4
1.93
4.59
0.80
0.61
CLIP-
100K
0.790
54.07
25.2
6.65
0.61
0.60
0.605
124.5
9.77
6.59
0.75
0.52
+REPA
100K
0.637
74.46
17.8
6.33
0.65
0.61
0.489
172.0
5.67
6.33
0.78
0.54
CLIP-L
+REPA
400K
0.442
130.1
7.42
5.03
0.71
0.65
0.307
271.8
1.96
4.64
0.80
0.61
Table 8: Additional results with alternative evaluation metrics. We provide additional results with CMMD metric (Jayasumana et al., 2024). All results are reported using SiT-XL/2, 256 batch size and traditional REPA as baseline. We adopt the pytorch implementation from (Paul, 2024) for computing the CMMD metric. Standard reference set from OpenAl ADM evaluation suite (Dhariwal & Nichol, 2021) is used reference images. Across various vision encoders, accentuating transfer of spatial features (iREPA) helps improve convergence speed with both CMMD and traditional evaluation metries (IS, FID etc.). This demonstrates that key empirical findings from $4 robust to the choice of evaluation metric.
36

---

Published as a conference paper at ICLR 2026
M.6 SPATIAL IMPROVEMENTS WITH SAM2
Table 9 and Figure 30 study the impact of spatial normalization ($4) on SAM2-S (46M) encoder features*. As shown in Table 9, observe that for SAM2, while spatial normalization layer helps improve performance, the improvements can be marginal. This is because spatial normalization relies on removing the global component (mean of patch tokens) enhance spatial contrast. Since SAM2 already has little to no global information (validation accuracy < 24%), spatial normalization only slightly improves the spatial contrast (see Figure 30).
Spatial Normalization Layer
Spatial Normalization Layer
SAM2-S SAM2-B
Costoe Samilarity
Figure 30: Impact of spatial normalization on SAM2. We observe that for SAM2, while use of spatial normalization layer does help enhance the spatial contrast, improvements can marginal. This is because spatial normalization ($4) relics removing the global component (mean of patch tokens) to enhance spatial contrast. Since SAM2 already has little to no global information (validation accuracy < 24%), spatial normalization only slightly improves the spatial contrast.
Vision Encoder
Steps
1S1
FID
sFID.
Prec."
Rec.
SAM2-S
100K
50.69
25.32
6.52
0.631
0.588
+IREPA
100K
53.14
24.52
6.28
0.629
0.591
SAM2-S
400K
110.77
9.54
4.93
0.699
0.638
+IREPA
400K
114.62
9.10
4,89
0.704
0.640
Table 9: Impact of spatial improvements on SAM2. All results are reported with SiT-X1/2, without
classifier-free guidance, SAM2-S (46M) vision encoder and with traditional REPA as the baseline.
NNote that similar to (Bolya et al., 2025). we use the intermediate output of vision encoder for SAM2 features and not the mask logits. As shown in (Bolya et al., 2025), while mask logits lead to sharper spatial maps, the mask logits itself not suitable target representation.
37

---

Published as a conference paper at ICLR 2026
M.7 ADDITIONAL RESULTS WITH FULL-FINETUNING ACCURACY INSTEAD OF LINEAR PROBING ACCURACY
Correlation analysis with full-finetuning accuracy instead of linear probing accuracy. We repeat the correlation analysis from $3 with the validation accuracy after full-finetuning instead of linear probing. Results are shown in Fig. 31. All results are reported using SiT-XL/2 (100K steps) and REPA. For full-fine finetuning validation accuracy, instead of linear probing, we perform full-finctuning of vision encoder with learning rate 5e 5, warmup ratio of 0.1, and total of 30 epochs. All spatial metrics show much higher correlation with generation performance than full-finetuning accuracy. Interestingly, gFID actually shows a weak positive correlation with the validation accuracy after full-fnettnning (Pearson
0.317), i.e., as validation accuracy increases, the gfid increases, and generation performance becomes worse. This is consistent with the observations discussed $2, wherein often target representations with higher global semantic performance show similar worse generation performance with representation alignment (REPA).
cepal
Pearson P- 4.811
GETO F100X stogst
- 0..64
- 4,850
-0.04
p- 4..00
AG
Spottal
Spotaa
Sparial
Spottal
Figure 31: Correlation analysis with full-finetuning accuracy instead of linear probing accuracy. We repeat the correlation analysis from $3 with the validation accuracy after full- full-finttnnnng instead of linear probing. All results are reported using SiT-XL/2 (100K steps) and REPA. All spatial metrics show much higher correlation with generation performance than full-finetuning accuracy. Interestingly, gFID actually shows a weak positive correlation with the validation accuracy after full-fnnetuning (Pearson generation performance becomes worse. This is consistent with the observations discussed $2, wherein often target representations with higher global semantic performance show similar worse generation performance with representation alignment (REPA).
0.317), i.c., validation accuracy increases, the gfid increases, and
'Please note that while the final findings remain similar, in context REPA linear probing might be more accurate for estimating global information in "pretrained" vision encoders. This is because REPA uses the "pretrained" encoder features themselves for regularization. Finetuning the encoder itself can impact the amount of global information. Same as REPA (Yu et al., 2024), we therefore use linear probing accuracy default for mcasuring global information.
38

---

Published as a conference paper at ICLR 2026
M.8 ADDITIONAL RESULTS ON PIXEL-SPACE DIFFUSION MODELS
Method
#Params
ISt
FIDL
SFIDL
Prec.
Rec.t
PixelFlow (Chen et al., 2025)
4S9M
24.67
54.33
0.58
PixDDT (Wang et al., 2025b)
434M
36.24
46.37
17.14
0.63
PixNerd (Wang et al., 2025a)
458M
43.01
37.49
10.65
0,62
DeCo w/o LyregiM
426M
46.44
34.12
10.41
0.64
DeCo REPA (Ma al., 2025b)
426M
48.35
31.35
9.34
0.65
JiT (Li & He, 2025a)
459M
29.37
49.06
11.21
0.40
0.62
JiT+ (Ours)
459M
50.72
29.19
9.42
0.51
0.65
Table 10: Generation performance of pixel-space diffusion models. Despite its simplicity, when combined with JiT (Li & He, 2025a) iREPA outperforms recently proposed state-of-art pixel-space diffusion methods e.g., DeCo (Ma et al., 2025b). All results are reported with 200K training iterations with batch size of 256 and evaluated using 50-step Euler sampling without classifier- guidance. Results for DeCo are obtained directly from (Ma et al., 2025b).
39


Reviews:
[{"review_id": "Dyo09rtLeH", "reviewer": "Reviewer_6Vka", "summary": "This paper investigates REPA and systematically analyses how to best guide diffusion model training. With external representations. Through a large-scale empirical study, the authors show that spatial structure correlates more strongly with generation quality than global semantic information and based on these findings, the paper introduces iREPA. The method is a minimal but effective change to the existing pipeline that shows consistent improvements to REPA by replacing the MLP with a convolutional layer and adding spatiall normalization layer to enhance spatial contrast.", "questions": "1. **Generalization to multimodal or semantically guided setups**: Given that iREPA explicitly downweights global semantic signals, how might the approach generalize to multimodal or text-conditioned diffusion models where semantic correspondence is critical? Would some hybrid form of spatial and global alignment be beneficial in that setting?\n2. While the results clearly show that preserving spatial correlations improves sample realism, the mechanism behind this effect is not fully explained. Do the authors believe that maintaining local spatial dependencies helps the diffusion model reconstruct structure more coherently during denoising, or that it serves as a regularizing inductive bias for patch-level consistency?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "- **Fundamental insight that provides stable gains**: the work provides large-scale evidence that spatial structure rather than semantic quality determines usefulness of pertained visual features for generative alignment and uses this insight to implement a simple and minimal intervention, that improves convergence and generative quality consistently. \n- **Generalization and robustness:** the proposed method is able to show improvements across multiple architectures / encoders (DINOV2, SAM2, CLIP, etc.), model scales (-B to -XL) and training variants \n- **Extensive experimentation and validation:** provides extensive correlation studies, ablations, and analyses to support the central claim", "weaknesses": "**Effect of removing global semantics:** iREPA intentionally suppresses the global semantic component of pretrained representations to enhance spatial contrast. While this clearly benefits diffusion-based generation, it remains uncertain how much this trade-off might affect tasks that depend on higher-level semantic coherence or multimodal conditioning.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "VPtOFWSBTH", "reviewer": "Reviewer_S2CT", "summary": "First, this paper investigates the underlying factors that make REPA effective. Through a large-scale empirical study involving 27 different vision encoders, the paper demonstrates that spatial structure is more important than global information. Second, the paper introduces a simple and efficient metric, termed Spatial Structure Metric (SSM). Third, the paper proposes an improved iREPA method which is designed to better preserve and transfer spatial information. Extensive experiments show that iREPA consistently accelerates convergence and improves final generation quality over the baseline REPA across a wide variety of encoders, diffusion model sizes, and training variants.", "questions": "Q1. Following on from W1, could the authors show how iREPA's convergence acceleration compares to other popular methods for improving generative model training? Could it work together with other training methods?\nQ2. If the outliers are removed, will the conclusion corresponding to Figure 1 still hold?\nQ3. The iREPA modifications are praised for their simplicity. However, given the strong conclusion that spatial structure is key, did the authors experiment with more sophisticated mechanisms or network structure to maximize spatial information transfer?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 4, "strengths": "S1. The paper's primary contribution—that spatial structure, not global semantic accuracy, is the key driver for REPA's success—is a significant and non-obvious finding. The authors have conducted a comprehensive set of experiments to validate their claims.\nS2. The proposed iREPA method is elegantly simple (noted as <4 lines of code) yet highly effective. The two modifications (convolutional projection and spatial normalization) are well-motivated by the paper's core finding and are easy to implement, which significantly increases the practical value and potential for adoption.\nS3. The experimental results clearly demonstrate the effectiveness of iREPA across multiple evaluation aspects. Specifically, iREPA exhibits consistent performance gains in scalability (handling larger model sizes), robustness across different encoder depths, and improvements in generation quality.", "weaknesses": "W1. The paper primarily focuses on improving REPA and its variants. While this is a valid contribution, the proposed method is not benchmarked against other, orthogonal techniques for improving generative model training. \nW2. In the correlation plot between linear probing accuracy and gFID (Fig 1 left), there are two noticeable outliers (points 25: Mocov3-l, 27: MAE-l) that have both low accuracy and poor (high) gFID. These high-leverage points significantly influence the linear regression.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "GUMCfxlzyp", "reviewer": "Reviewer_fK5t", "summary": "This work presents an empirical study and enhancement of representation alignment (REPA), which refers to a constraint that aligns a generative model’s representations with those of a pre-trained model during learning. Specifically, the authors extensively experiment with various pre-trained encoders and conclude that representations with “spatially structured” properties contribute more to generation quality than those excelling in classification tasks. The spatial structure is measured using the pairwise cosine similarity between patch tokens. Inspired by this finding, the authors propose a strategy to replace the MLP in REPA with convolutional layers and introduce spatial normalization layers, thereby encouraging the generative model to better learn from the spatial structure of the encoder’s representations.", "questions": "1. My primary concern revolves around the uncertainty introduced by the evaluation metrics, which affects the reliability of the findings and conclusions. The authors may consider addressing this by incorporating the suggestions outlined in the Weaknesses section, such as providing quantitative results with alternative evaluation metrics and on other datasets, as well as including qualitative results to better substantiate the claims.\n\n2. Would the use of encoders trained on segmentation or depth prediction tasks lead to better performance? Intuitively, such representations are likely to exhibit stronger spatial structure, and it would be valuable to examine whether they bring further improvements in the proposed framework.\n\n3. This is an open discussion, not a must: From the perspective of learning spatial structure, would it be beneficial to not apply the REPA constraint at all denoising steps?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The paper systematically examines the relationship between the spatial structure of representations and metrics such as FID by testing an extensive set of encoders, and further strengthens this conclusion using Pearson correlation coefficients.\n\n2. Building on the above findings, the authors propose two strategies to encourage more effective learning of the spatial structure of representations. Both approaches are visually supported through attention map visualizations and are substantiated by ablation studies.", "weaknesses": "1. The conclusions of this work rely on the premise that metrics such as FID and IS accurately reflect model generation capability. However, several prior studies[1,2,3, 4] have argued and demonstrated that FID/IS may not fully capture model performance. This introduces some uncertainty to the findings of this work. Thus, it is recommended that the authors provide additional evaluation results based on other metrics - for instance, by replacing the feature extractor used for FID computation.\n\n2. Regarding the experimental setup, this work conducts training and evaluation exclusively on ImageNet at 256×256 resolution. The absence of results on other datasets or at different resolutions (e.g., 512×512) limits the generalizability of the claims. Moreover, since the Inception and VGG models used for FID/sFID/IS/Precision/Recall are also pre-trained on ImageNet, this further compounds the uncertainty in the evaluation.\n\n3. Given the potential limitations of the quantitative evidence raised in the previous points, the paper is notably lacking in qualitative results. Neither generated results of the proposed iREPA method nor qualitative comparisons with other approaches are provided, which would help better assess the actual improvements in generation quality.\n\n4. In the ablation study table (Table 2), under the DINOv2/DINOv3 settings, the sFID results for iREPA (full) are not the best. The authors are expected to explain this observation. Additionally, it is recommended not to format the specific numbers in bold to avoid misunderstanding.\n\n[1] Stein, George, et al. \"Exposing flaws of generative model evaluation metrics and their unfair treatment of diffusion models.\" Neurips 2023.\n\n[2] Yang, Ceyuan, et al. \"Revisiting the evaluation of image synthesis with GANs.\" Neurips 2023.\n\n[3] Kynkäänniemi, Tuomas, et al. \"The role of imagenet classes in Fr\\'echet Inception Distance.\" ICLR 2023.\n\n[4] Jayasumana, Sadeep, et al. \"Rethinking FID: Towards a better evaluation metric for image generation.\" CVPR 2024.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "j2YBPjf3Ha", "reviewer": "Reviewer_HkpS", "summary": "The paper investigates whether global semantic information or spatial/local information of the external encoder contributes more to the generation performance of REPA (Representation Alignment for diffusion models).\nFirst, the authors show that the amount of global semantic information (quantified by linear probing accuracy) of the external encoder may negatively correlates with REPA generation quality (measured by FID).\nSecond, they introduce a metric for spatial information of the external representations, defined by how much closer patches exhibit higher feature similarity than distant ones, and show that this spatial inductive bias positively correlates with REPA generation performance.\nFinally, the authors propose two simple modifications to improve REPA: (1) replacing the MLP projection layer with a convolutional projection layer to preserve spatial structure, and (2) introducing a spatial normalization to encourage feature consistency among nearby patches. These enhancements consistently improve generation performance across multiple external encoders.", "questions": "- Have you tested whether your proposed spatial normalization or convolutional projection still helps when using already spatially rich encoders like SAM2?\n- Can you discsuss how doe the previous work's insight situates with your findings? [1] \n- Could you clarify the experimental setup for Figure 3: specifically, how many diffusion steps were used and which diffusion backbone the experiments were based on?\n\n[1]Wang, Ziqiao, et al. \"REPA Works Until It Doesn't: Early-Stopped, Holistic Alignment Supercharges Diffusion Training.\" arXiv preprint arXiv:2505.16792 (2025).", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The studied problem is well-motivated:analyzing what are the important factor in REPA of diffusion models.\n- The analysis is comprehensive. span across a lot of diverse external recent advanced external encoders and various sizes of diffusion models.\n- The empirical findings are clear and well-supported: global semantic information(as measured by linear-probe accuracy) weakly correlates with REPA generation performance(measured by FID), while spatia structure(measured by the spatial metrics) strongly correlates.\n-  The simple adjustments proposed in Section 3 are well-motivated, lightweight, and lead to consistent FID improvements across all encoders.\n- The qualitative examples in figure 2 gives a nice intuition on the spatial information of generated image.\n- The writing is clear and well-structured, making the analysis easy to follow despite the large experimental scope.\n\nOverall, the work provides practical and actionable insights for improving REPA performance in diffusion-based generative models.", "weaknesses": "- Figures 2, 3a, and 3c all show a negative trend between global information and generation quality, and Sections 1–2 carry that narrative. But Figure 4 actually shows the correlation is weakly positive among the diverse encoders, as opposed to completely negative. The paper’s wording early on makes it sound stronger than what the data supports.\n - The paper should mention prior findings that REPA tends to improve generation quality at first and then degrade after more training epochs[1] That paper already pointed out that \"REPA injects semantic anchors but underuses structural information, which overlaps a lot with this paper’s main claim.\n- Defining the linear probing accuracy as \"global information\" is not entirely intuitive as global information often refer to high-frequency features in respresentation learning domain. \"Semantic information\" is a better way to describe it. \n- Missing details in experimental setup: Figure 3 is lacking some experimental setup details:for example, how many diffusion steps were used, or what diffusion backbone used. These details are important for interpreting the results and comparing across models.\n\n[1]Wang, Ziqiao, et al. \"REPA Works Until It Doesn't: Early-Stopped, Holistic Alignment Supercharges Diffusion Training.\" arXiv preprint arXiv:2505.16792 (2025).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)