What Secrets Do Your Manifolds Hold? Understanding the Local Geometry of Generative Models
Authors: Ahmed Imtiaz Humayun
Ibtihel Amara
Cristina Nader Vasconcelos
Deepak Ramachandran
Candice Schumann
Junfeng He
Katherine A Heller
Golnoosh Farnadi
Negar Rostamzadeh
Mohammad Havaei
Venue: ICLR 2025 Poster (2025)
Keywords: Geometry
Diffusion models
VAE
Generative Models
Guidance
Memorization
Out-of-Distribution Detection
Primary area: generative models
TL;DR: We show that the local geometry of generative models is indicative of generation aesthetics, artifacts, diversity, and memorization.

Abstract: Deep Generative Models are frequently used to learn continuous representations of complex data distributions by training on a finite number of samples. For any generative model, including pre-trained foundation models with Diffusion or Transformer architectures, generation performance can significantly vary across the learned data manifold. In this paper, we study the local geometry of the learned manifold and its relationship to generation outcomes for a wide range of generative models, including DDPM, Diffusion Transformer (DiT), and Stable Diffusion 1.4. Building on the theory of continuous piecewise-linear (CPWL) generators, we characterize the local geometry in terms of three geometric descriptors - scaling ($\psi$), rank ($\nu$), and complexity/un-smoothness ($\delta$). We provide quantitative and qualitative evidence showing that for a given latent vector, the local descriptors are indicative of post-generation aesthetics, generation diversity, and memorization by the generative model. Finally, we demonstrate that by training a reward model on the 'local scaling' for Stable Diffusion, we can self-improve both generation aesthetics and diversity using geometry sensitive guidance during denoising. Website: https://imtiazhumayun.github.io/generative_geometry.

Published as a conference paper at ICLR 2025
WHAT SECRETS Do YOUR MANIFOLDS HOLD? UNDERSTANDING THE LOCAL GEOMETRY OF GENERATIVE MODELS
Ahmed Imtiaz Humayun 1.2, Ibtihel Amara 2.3, Cristina Vasconcelos', Deepak Ramachandran', Candice Schumann', Junfeng He*, Junffnn Golnoosh Farnadi', Negar Rostamzadeh' Mohammad Havaei 'Google Research, "Rice University, "MeGill University, *Google Deepmind imt carce..r..... edu, mhavaei@googgee.o com
Katherine Heller',
ABSTRACT
Deep Generative Models are frequently used learn continuous representations of complex data distributions by training on a finite number of samples. For any generative model, including pre-trained foundation models with Diffusion or Trans- former architectures, generation performance can significantly vary across the learned data manifold. In this paper, we study the local geometry of the learned manifold and its relationship to generation outcomes for wide range of generative models, including DDPM, Diffusion Transformer (DiT), and Stable Diffusion 1.4. Building the theory of continuous piecewise-linear (CPWL.) generators, we characterize the local geometry in terms of three geometric deseriptors - scaling (ő). We provide quantitative and qualitative evidence showing that for a given latent vector, the local descriptors are indicative of post-generation aesthetics, generation diversity, and memorization by the generative model. Finally, we demonstrate that by training reward model on the local scaling for Stable Diffusion, we can self self-improve both generation aes- thetics and diversity using geometry sensitive guidance during denoising. Website: www.hmmyunngithhub....... github. 10/ gene gateratieeeaaeeeeemme. leree_eeoeerrrrrr
(v)) rank (v), and complexityyun-smootnnee
imtiaz www.aaaammmuunccmmmmm
a Australian terrier on rhodesian ridgeback with a cobblestone street
p
p
p 1.5
a city in the background
P 0
-=-1 P=-3
Figure 1: Geometry sensitive diffusion guidance. During the reverse diffusion process, increasing (top-row) or decreasing (bottom-row) the local scaling, i.e., volume dilation, denoising trajectories using geometry reward based guidance with strength results in increased (top-row) or decreased (bottom-row) visual complexity. Increasing local scaling results in more background elements coming into view and a decrease in the focus the subject, vice-versa when local scaling is decreased. Here we use a reward model trained on the local scaling computed for 800K Stable Diffusion (Rombach et al. 2021) latents. More examples in appendix fig. 22 fig.
fig. 24

---

Published as a conference paper at ICLR 2025
INTRODUCTION
In recent years, deep generative models have emerged a powerful tool in machine learning, capable of synthesizing realistic data across diverse domains (Karras et al. 2019; 2020; Rombach et al. 2021). However, the performance of such generative models may vary based on the latent vector or senditiooni used for generation. Recent studies have demonstrated that models like Stable Diffusion can exhibit generation biases in terms of reduced generation fidelity diversity, when certain demographic groups are mentioned in the prompts /Zhao et al. (2018); Luccioni et al. (2023). Especially for /) models trained with large heterogeneous data distributions, and for 2) generative models pre-trained on unknown training distributions, the aforementioned observations, i.e., latent/prompt specific behaviors can become hard interpret reason. Since generative models are essentially approximating generative function mapping a latent space to the data manifold, one way to explain downstream performance could be in terms of the characteristics of the approximated function that can be measured via the model internals, i.e., activations and weights. In this regard, we pose the following research question:
Research Question. For any deep generative model pre-trained on an arbitrary dataset, how is the local geometry of the generator function related to downstream generation?
The theory of continuous piecewise-linear generators Balestriero et al. (2020) suggests that a large class of generative models can be considered continuous piecewise linear (CPWL) operators, implying that such generative models can be fully characterized in terms of their weights and architecture. We consider it the framework of choice to find answers to the aforementioned question and propose using three local geometric descriptors to quantify local characteristics of any pre-trained generative model:
Local rank (v), that characterizes the local dimensionality of the learned manifold.
Local scaling (10), that characterizes the local change of volume by the input output mapping of the generative model.
Local complexity (6), that approximates the un-smoothness of the generative model in terms of second order changes in the input-output mapping.
Geometric descriptors such as local scaling, complexity or rank, have previously been used to measure function complexity of Deep Neural Networks (DNN) (Hanin & Rolnick 2019)) and DNN expressivity (Poole et al., 2016 Raghu et al.] (017), to evaluate the quality of representations learned with supervisdd objective (Garrido al. 2023), ) for interpretability and visualization of DNNs (Humayun et al. 2023), to understand the learning dynamics in reinforcement learning (Cohan et al. 2022), to explain grokking, i.e., delayed generalization and robustness in classifiers (Humayun et al. 2024), sampling of GAN based generative models (Humayun et al.) 2021 20226), and maximum likelihood inference in the latent space (Kuhnel et al. 2018). We provide a more extensive discussion on related works in appendix B
Our contributions. In this paper. through rigorous experiments on large image generative models, we establish correlations between the local geometry of the generative function and the aesthetics of the generation, the diversity of the generation, and the memorization in the generated samples. We demonstrate how these qualities manifest differently for different sub-populations of the learned distribution. We also show that the geometry of the data manifold is heavily influenced by the training data, which enables applications in oft-o--distribution detection and reward modeling to control the output distribution. Our empirical results lead to the following conclusions.
C1. We present the first large-scale analysis of the local geometry of foundational text-to- image latent diffusion models and establish correlations between local geometric descriptors and downstream aesthetic quality, diversity, and memorization (Sec 4.).
C2. For small diffusion models and foundational image generative models, we show that the local geometry the generative model manifold is distinct from the off-manifold geometry and can help distinguish the domain of generative model (Sec. 3).
C3. By training an auxiliary model on precomputed local geometric descriptors of Stable Diffusion, we present a novel framework for reward guidance on diffusion model to increase/decrease sampling diversity or control aesthetic qualities (Sec 5).
2

---

Published as a conference paper at ICLR 2025
2 LOCAL DESCRIPTORS OF GENERATIVE MODEL MANIFOLDS
We start by introducing the geometric descriptors we will use in our study and provide insight into what aspect of the generative model manifold geometry each of the descriptors quantity.
2.1 CONTINUOUS PIECEWISE-LINEAR GENERATIVE MODELS
Consider generative network 9, which can be the decoder of a Variational Autoencoder (VAE) (Kingma & Welling, 2013). the generator of Generative Adversarial Network (GAN) (Good
fellow et | 2014). or an unrolled denoising diffusion implicit model (DDIM) (Song et al. 2020). Suppose, g R + RDD is deep neural network with T. layers, input space dimensionality E and output space dimensionality D. For any such generator, the layers comprise affine operations such convolutions, skip-connections, or max//vv-ppooling and the non-linearities are continuous piecewise-linear (CPWL) such as leaky-ReLU Xu (2013). ReLU, or periodic triangle, then the generator is a continuous piecewise- linear operator (Balestriero & Baraniuk] 2018a Humayun et al. 2023). This implies that the g : R + R ? the input space into linear regions 52 with each region from the input domain being mapped to the output via an affine operation. The continuous data manifold or image of the generator lm(G) can be written the union of sets:
mapping can be expressed in terms of a subdivision of
lm(g)
U (A V2 w)) YWEE
(I)
where, Q is the partition of the latent space E into continuous piecewise-linear regions, A_ and b. are the slope and offset parameters of the affine mapping from latent space vectors E w to the data manifold. For the class of continuous piecewise-linear (CPWL) neural network based generative models, Q2 Aw. and b., are functions of the neurons/parametent of the network. For generator with L layers, A_ and bw can be expressed in closed-form in terms of the weights and the region-wise activation pattern of neurons for each layer. We refer the readers to Lemma of (Humayun et al. 2023) for details.
To help build intuition, without loss of generality lets consider a CPWI. toy generator that is trained on a handcrafted task where the target function f: TR2 -> TR33 a mapping between TR22 and a mixture of five gaussian functions. Since the learned function is continuous piecewise-affin spline operator, we use SplineCAM (Humayun et al.) 2023) to analytically compute the function learned by the generator and visualize the learned manifold, as well as the input space piecewise-linear partition learned by the generator in Fig. 2 middle-left and left. Each convex region w bounded by the black lines, is mapped to Im via per region parameters as described in Equation [ The input-output mapping operation by the generator is affine region-wise, therefore any given input space region can be scaled, rotated or translated with continuity constraint between regions, while going from the input to the output. For CPWL generators there are three characteristics of the learned manifold that can be studied: i) the affine scaling induced per region, ii) the number of dimensions that are retained after scaling, i.e,, local dimensionality of the learned manifold, and iii) the local smoothness of the CPWL partition. We now introduce local descriptors that can be used characterize these quantities.
2.1.1 LOCAL SCALING, vb
We first introduce local scaling as a target descriptor to be used in our study that measures the local scaling performed on region w by CPWL generator.
For a CPWL manifold produced by generator G, the local scaling thw is constant within each region w, and measures the log-scaling of the volume induced by the affine slope A., for all latents E w. Local scaling for w is expressed as
Definition 1.
th. = loggly det (let(AAA))) Σ
where, (o are the non-zero singular values of A
Refering back to the example in Fig. 2 each region on the CPWL manifold (middle-left) and in the input space (left) is colored by the, with darker shades indicating higher ei_. Suppose & has a uniform latent distribution, meaning every region has uniform probability density in the latent space.
3

---

Published as a conference paper at ICLR 2025
Data Manifold. lm(g) E R
Latent Domain, Z E R²
2 ~ U(Z)
I - g(=)
Figure 2: The geometry of a continuous piecewise- -inear toy generator. For a CPWL generator : + R. provide analytically computed visualization of the input space partition, ie.. arrangement of linear regions (left) and learned CPWL manifold (middle-left). Each piece for this example, is colored by the piecewise-constane scaling induced by G. Uniform samples from the latent domain (middle-right) and generated samples (right) are also presented, colored by the density of the output distribution estimated using gaussian kernel density estimator in R We that for any sample E the estimated density (f green) is inversely proportional to the scaling (4 green) for region w.
Under an injectivity assumption for any input space region w and S = (A, E w). according to Theorem in Humayun et (2022a), the output density on S, ps())) x vWw Therefore, local scaling diw is proportional to the negative log- -igliikeihoodd of the generative model for any E w. We can validate this by using a kernel density estimator (KDE) to estimate the density of generated samples the data manifold from a uniform latent distribution Fig. 2 In fig. 3 for toy dinosaur manifold and fig. 32 a toy mixture of gaussians, we demonstrate how the local scaling is lower around the modes of the learned distribution and higher for anti-modes
2.1.2 LOCAL RANK, V.
The second descriptor we study is the rank of the region-wise slope matrix A which represents the dimensionality of the manifold learned by a CPWL generator.
Definition 2. For CPWL manifold produced by generator G, local rank vw is the exponent of the Shannon entropy the spectral distribution of the per-region affine slope A and can expressed
Σ ai log(a:)
Vw exp
Oi
where ag
Σ, 0s
Here, kosk are non-zero singular values of A.,, and 10 10-30 is constant. The local rank V can be shown to be equivalent to the dimensionality of the tangent space on the data manifold at z.
2.1.3 LOCAL COMPLEXITY, 5
An important geometric notion to characterize any manifold locally is the local smoothness of the manifold. However, smoothness requires computing the hessian of the input-output mapping making computationally intractable for large generative models. We therefore consider local complexity as a proxy for sharpness of the manifold locally for our study, Based on the notion of complexity for CPWL neural networks (Hanin & Rolnick 2019), we define local complexity of CPWL generator as the following.
Definition 3. For a CPWL generator with input partition Q, the local complexity 8, for P. dimensional neighborhood of radius around latent vector is
82
Σ
(5)
VunV 70
where V- fxr R
B(x 2)]l1 <r).
(6)
4

---

Published as a conference paper at ICLR 2025
Here, B is an orthonormal matrix of size P E with P P E, |1-111 11 is the & norm operator and is radius parameter denoting the size of the locality to compute ó for. Here we consider a P dimensional neighborhood instead of the full dimensionality of the latent space to reduce computational complexity. The sum over regions w E V= requires computing f2 n V= which can be computationally intractable for high dimensions. A proxy for computing the partition for V2 with small is counting the number of non-lincarities within V., since for small the one can assume that the non non-linearities do not fold inside v. therefore providing an upper bound on the number of regions according to Zaslavsky's 's Theorem (Zaslavsky, 1975). To compute local complexity, we use the method introduced by Huumayun al., (2024) for general neural networks. In appendix/C]we discuss the relationships between the geometric descriptors and present examples for a VAE trained on MNIST.
2.2 EXTENDING BEYOND CONTINUOUS PIECEWISE-LINEAS GENERATORS
Computing jacobians for large networks. For any latent vector E A can be obtained by computing the input-output jacobian of the network. Computing the singular values of the full input-output jacobian is significantly expensive for large networks. Therefore, when computing local scaling and rank we obtain singular values via randomized SVD Halko et al. (201I). First we obtain a random projection matrix with orthonormal rows W with shape k such that WW 4Ą. Here is the dimensionality of the outputs generated by the network. We therefore approximate local scaling as:
(trunc)
(trunc)) where o,
(trunc) are the non-zero singular values of WA
For any if W forms basis for the range of A. then o; o (trunc) 1,2... k [4]. Therefore WA would provide us a low-rank approximation of Aw..
In our experiments we have tried two methods to obtain the projection matrix W 1) by obtaining the eigenvectors for the covariance matrix for set of 50K randomly generated samples. This was suggested in Halko et al.] (2011). 2) by performing QR decomposition of a randomly initialized matrix. We see that the performance difference between methods 1) and 2) are negligible therefore consider the cheaper alternative 2) and consider fixed pre-computed W with k=120 for all A. in our Stable Diffusion experiments.
Networks with smooth activations. While the descriptors are defined for CPWL mappings, modern generative models employ a mixture CPWL. and non-CPWL operations. For networks with smooth activation functions or non-piecewise-linen non-linearities, our descriptors are first order Taylor approximations. For example, Stable Diffusion employs the GeLU activation funetion for which we perform the bulk of our experiments in following sections. Smooth activation functions induce a soft VQ partitioning of the latent space compared to the hard VQ partitioning induced by CPWL map Balestriero & Baraniuk (2018b), retaining much of the local linear structure expect in CPWL maps. Recent work has also empirically verified the local linearity for a large class of image based diffusion models [Chnn al. (2024) suggesting the reliability of first-order approximations.
CHARACTERIZING THE LOCAL GEOMETRY OF PRE-TRAING MODELS VIA DESCRIPTORS
In this section, we explore the geometry of pre-trained generative models by characterizing the latent space to output manifold mapping in terms of the local geometric descriptors mentioned in the previous section. We are interested the following questions: How does the on manifold local geometry vary from the off manifold local geometry? ii) How does the local geometry vary across the input domain?
3.1 ON AND OFF MANIFOLD GEOMETRY FOR DENOISING DIFFUSION PROBABILISTIC MODELS
Setup. To study the on and off manifold geometry of diffusion models, we train denoising diffusion probabilistic model (DDPM) (Ho et al. 2020) on a toy datase to visualize how the local geometry varies for 1) different noise levels f, and 2) different training iterations.
hntg:///ummiigrrveessstttaaaaaau
5

---

Published as a conference paper at ICLR 2025
22202 8t=0,220 Pc=0.225
EM[Wh] [xle
BMrl6dl
Exd(un]
E Pr
1000
1390
124.0
Figure 3: Local geometric descriptors computed over the input domain of DDPM trained on samples from a toy dinosaur manifold M IR? conditioned on 0.22T (left three columns). Denoising dynamics of local descriptors (right three columns) for different number of training optimization steps. For a well trained diffusion model, local descriptors can distinguish between on and off manifold vectors in the input space. See fig. for more.
10"2
In Fig. 3 B-eeammaps we present the local complexity 8 local scaling vi and local rank vs computed for differ- ent input space vectors æ using the DDPM conditioned on noise levels t. Here T is the highest noise level in the forward diffusion process. We also present the dif- ference between the expected descriptor values on and off the manifold, [BM[$ E, [ VΦ E {8" 5 w'} at different training iterations (right). We consider the set of input vectors within 0.05 units of the training data as manifold M and rest as off the manifold M.
10"3
10-1-
10"3
riflctr colebc)
celebal
10.3
Atoowers celeba)
riffe celebu)
10-31
t/T E [0.5.
Figure 4: Statistical significance the difference between local scaling distribu- tions for in-domain (blue) vs out-of oo-mnn (red) datasets when conditioned noise levels for SD Unet trained the CelebAHQ dataset. Here ((,, denotes t-test between local scaling distributions for dataset a and dataset b.
Observations. The first observation is that with longer training, the maximum absolute difference between on and off manifold local geometry max, [| [$] - EM [$]] in- creases. Since with more training we see higher distinction between the on and off manifold geometry, this difference can be an indicator of learning in diffusion models. We see that for well trained models, apart from > 0.17T, v/ and V decreases and 8% increases with decreasing t, Vr E M.
different
This means, the likelihood on the manifold increases as noise levels are reduced, the smoothness decreases and the dimensionality of the manifold decreases as well. The quantity E. (b) E,y [$] also minimized at 0.177. This indicates that there can exist a noise level conditioned on which diffusion model local scaling, rank and complexity have the highest distinction geometrically between on and off manifold vectors from the input space. In fig. 4 we see statistically significant off manifold geometry at smaller t for Stable Diffusion (SD). These result indicate that the local geometry can allow directly probing which parts of the input space are on the learned manifold to possibly perform one step denoising or propose novel guidance schedules.
difference between the
6
1/
Figure 5: Geometry of the Stable Diffusson latent space. Geometric descriptors (left, middle-left, middle-right) visualized on 2D latent space subspace, that passes through the latent representations of "a fox", "a cat" and "a dog" (right), denoted via markers on the 2D subspace descriptor. In Appendix, we provide denoised images for different high/low descriptor regions from the subspace. We that in the convex hull of the three anchor latent vectors that in the convex hull, the local rank v undergoes sharp changes which are not visible towards the edges of the domain.
and å 1. Moreover

---

Published as a conference paper at ICLR 2025
3.2 THE LOCAL GEOMETRY OF LATENT DIFFUSION MODELS
In Sec. 3.1 we see that the local geometry in the input domain of a ddpm can be distinctive of its learned manifold. In this section we study the local geometry of the Stable Diffusion (SD) latent space, to explore whether there exists relationship between the local geometry and the domain of the SD decoder.
we could visualize the whole input domain of the diffusion model, for SD we can only visualize a subspace of the SD latent space. We use three prompts "a cat", "a dog" and "a fox" to generate three latent vectors using the SD diffusion model and consider 2D slice in the latent space, going through the three denoised latents as our domain to visualize. Note that since this is 2D subspace of the latent space, can expect part of be in-domain for the SD decoder, whereas part of it would be out-of-domain.
Setup. While in Sec. 3.1
Observations. We observe that 1) In the convex hull of the three denoised latents used as anchors for the 2D subspace being visualized, we have higher complexity, lower rank and higher local scaling. The decoded images from the convex hull may contain artifacts but are legible generations. 2) Local rank does not smoothly vary across the latent space, especially with sharp changes in the local rank within the convex hull of the anchor latent vectors. For the lowest rank regions in the convex hull, decoded images have good fidelity compared to latents with high uncertainty or complexity. 3) If we move away from the convex hull, we see that generated images become more broken and contain heavy artifacts, indicating that such regions are out-of-domain for the SD decoder. However, we see that the local scaling is lower in these regions compared to the convex hull.
Vendi Score
Hum. Pref. Score
0.0
0.85
0.80
0.75
0.65
020
1880
-002
-175
-130
-125
200
-75
-25
Increasing level sets
Figure 6: Local Geometry level sets Ima- genet prompts. Vendi diversity scores and RAHI Liang et al. (2024) aesthetic scores computed for images with classifier free guid- ance (CFG) 7, and 3. Diversity per level set increases and then decrease with increased lo- cal scaling. Aesthetic score slightly increases and then decreases well with increased local scaling.
SECRETS THAT YOUR MANIFOLDS HOLD
Visually complex images have higher local scaling
We selected 20K samples from Imagenet with resolution higher or equal to 512 x 512, encode the samples using the SD encoder, and compute the local descriptors for the SD decoder. In Appendix Fig. 17 each column represents local scaling level set, with the & for columns increasing from left to right. Recall that local scaling is proportional to the negative log-likelihood. In Appendix Fig. 17 can see that for lower local scaling images we have more modal features in the images, i.c. the samples have less background elements and are focused on the subject corresponding to the Imagenet class.
For images with higher local complexity, we see more qualitatively outlier characteristics and higher visual complexity. For images with higher local rank in Appendix Fig. 19] we see that the backgrounds have higher frequency elements compared to lower rank images. For higher rank images,
1g
kar
Figure 7: Local scaling is sensitive to image corruptions. We use 16 corruptions from (Hendrycks & Dietterich) (2019) to corrupt 10K imagenet images and compute the local geometry of SD decoder. see that SD local geometry is sensitive to cor- ruptions, i.e., aesthetic changes to in-domain images (here Ima- genet).
JEPPR
Geo reke
2ewrrty2
3
aevertyy
severit

---

Published as a conference paper at ICLR 2025
the dimensionality of the manifold is higher locally, therefore allowing more noise dimensions the manifold. These results indicate that the local geometry is indeed sensitive to qualitative variations such visual complexity of Imagenet images.
Noise corruptions increase and blurring corruptions decrease local scaling
Fig. 7 illustrates the effect of applying 16 different image distortions (originally proposed in (Hendrycks & Dietterich] 2019)) to 10k ImageNet images. We consider Imagenet as in-domain for Stable Diffusion and encode them to the SD latent space to compute the geometric descriptors for the SD decoder. Samples are uniformly distributed over its classes. The plot shows the local scaling distribution 6 increasing levels of severity E {0. 1, 2, 3, 4, 5). with zero corresponding to no corruptions applied. We observe that corruptions that are associated with reduction of spectral band, and/or reduction to the color range result log-liked rogllikeihhood. We conjecture that this is due to the averaging effect of such distortions which move the corrupted images close to the mean of all images. Conversely, distortions known to be associated with the introduction of freehfreueeccy artifacts therefore uncertainty moving the images away from the mean. The results clearly indicate that the local geometry is sensitive aesthetic changes to images introduced via most of the 16 corruptions.
reduction the local scaling therefore the negative
observed to produce increase in local scaling
Local Scaling, rb
Local Rank, v
Local Complexity,
115.9
Noavar Promuts coco tomm
0.6775
110
Marr alred
Aemorrted Premant
COCO Frompts
COCo pranpos
18
20
30
50
16
23
20
30
315.5
115-1
335.2
6.515
115.2
30 30
43
118.1
30
15
46 45
50 30
0.018
40
Denoising steps
Figure 8: Local geometry of denoising trajectories. Geometric descriptors computed for the SD decoder unconditionally, during 50 stable diffusion denoising steps, for (top) 100 COCO and 100 memorized prompts (Wen et al. (024) with guidance scale 7.5 and (bottom) 100 COCO prompts with varying guidance scales. For each prompt or guidance scale, we start from the same seeds. Shaded region represents 95% confidence interval. We see that the local geometry trajectories are discriminative of memorization, well increased alignment when stronger classifier free guidance is used.
Memorized and aligned denoising trajectories are locally contracted and smooth.
cussifer--rre-guiddan is a method for increasing the alignment between the conditioning text- prompt and generated image Rombach et al. (202I). In Fig. bottomm we see that for higher classifier free guidance scales during denoising, the avg. local scaling is lower, avg. local rank is lower and the avg. local complexity is higher. This indicates that for more conditionally aligned images obtained via -reeguddaace of the SD diffusion Unet, the decoder uncertainty is also lower especially during the final denoising steps. We also compute the local geometric descriptors for denoising trajectories conditioned memorized prompts Wen al. (2024) vs coco captions (Figg&ttoo). We see that the mean local geometry is significantly different for denoising trajectories of 100 memorized prompts vs 100 random coco prompts. We also see that for higher guidance scales local rank // is and local complexity 8 is J as well.
Connections with generation diversity and human preference scores.
In fig. 0 we present Vendi score Friedman & Dieng (2023) and human preference score Liang et al. (2024) (higher is better) aggregates for 50K real and generated Imagenet images with classifier free guidance scales of 7, 5 and 3. The images are sorted in increasing local scaling 1é bins from left to right. We see that for local scaling level sets from the lowest to the middle bins, we have an increase in the diversity of images per bin. For the highest local scaling bins, get images from the highest uncertainty modes, i.c., the anti-modes, which result in drop in the diversity of images (see Appendix fig. 32 for toy illustration). We observe similar trends in diversity for an Imagenet
8

---

Published as a conference paper at ICLR 2025
trained Diffusion Transformer (DiT) Peebles & Xie 2023
discussed in Appendix appendix D.1 For generated images we see human preference score marginally increasing from left to right. For the highest local scaling bins, especially for lower CFG, we see drop in the human preference scores. For real images, human preference scores have a significantly different trend compared to generated images. Note that the RAHF human preference model Liang al. (2024) is trained on synthetic images and therefore might be less reliable for real images.
5 GUIDING GENERATION USING GEOMETRY AS A REWARD
In the previous sections, we have presented qualitative and quantitative evidence, establishing the connection between geometric descriptors and downstream generation. Among the three descriptors we find that local complexity has the highest sensitivity to aesthetic changes in images due to corruptions (fig. 7 and correlates with visual complexity (fig. [7). We also observe in fig. 6 higher local scaling level sets for samples generated using a classifier free guidance scale of 7.5, have higher diversity while maintaining higher predicted human preference scores. Based on these results, we wish to explore whether local scaling can be used to guide generation.
Recently proposed instance-level uni- versal guidance method (Bansal al.) (023), effectively influence the latents in the reverse process of a latent diffusion mod- els produce desired changes. Directly using local scaling to guide generation using such methods, require calculating the input-output Hessian since local scaling is a first-order measure that requires comput- ing the onput-output jacobian. To avoid computing the Hessian we train a reward model as a proxy and use the reward model gradients directly. Instead of training continuous local scaling values in a regression task, we transform it into a local scaling level set classification task. We discretize the range of local scaling values into bins and use the bin indices training labels.
339
Local Scaling
-128-
Vendi Score
Pref:
Hum:
Jen. 482c
483c
50c
50c
Pre-guidance local scaling level set
Figure 9: Guidance via reward models trained with increasing number of training samples. We observe that even with when only 50K samples are used in training, reward models can increase generation diversity, local scaling and human preference scores. Here we present the change of vendi score, average local scaling and average aesthetic score for pre-guidance local scaling level sets increasing from left to right on the x-axis.
Data preparation. We obtain training data for the reward model by i) sampling N images from Imagenet and encoding them to the Stable Diffusion latent space ii) adding noise using the forward diffusion process up to randomly chosen noise levels iii) for each latent computing the local scaling descriptor.
To evaluate the performance of the reward model and dependency the reward model on the number of training samples, we train multiple models for N = 50K, 200K, 400K, 800K For evaluation, we generate 2560 samples using the dreambooth live subject prompt templates Ruiz al. (2023), with Howard (2019) dogs subjects. While Imagewoof dog classes are present Imagenet therefore possibly in the training data, the dreambooth prompt templates contain a variety of settings that are not generally present in Imagenet, e.g., 'a <subject> top of pink fabric'.
Hoowarl
Evaluation Setup. We first sample Stable diffusion without any reward guidance and with classifier- free guidance of 7.5 to obtain baseline samples. We partition the range of local scaling values obtained for the baseline samples into n = 10 bins fig. where each bin contains images from a local scaling level set. Following that we use the same seed and prompts as the baseline samples to generate images using reward guidance to increase local scaling. For each bin or pre- pre-guidance local scaling level set, compare between the baseline samples and corresponding reward guided generations in the following three axes: i) change of local scaling ii) change of vendi (diversity) score iii) the change of human preference score (RAHF Liang al. (2024) aesthetic score).

---

Published as a conference paper at ICLR 2025
p= 1=1
p= 2
p= p p= p -3
Ausatrall tenier with city in background
mouut. in the lkkg.
Samoyyed
p=0 0 1=1 P 2
P=0
P 3
a.Nutrallan tomee on asstellsiion street
Besgle Pasileeweet with city in haskgronne
Figure 10: Reward guidance on stable diffusion. Local scaling reward guidance increases from left right in each picture (left-panel) and decreases left to right in each picture (right-panel), with the first image showing no reward guidance. We observe maximizing the reward leads to sharper details, improved sharpness and contrast, and higher diversity in the images. Decreasing the local scaling leads to minimized uncertainty, resulting in noticeable blurring effect and loss of details especially the background of the image.
Results. In fig. we present the mean local scaling per bin with 95% confidence interval. Here the blue line represents the mean pre-guidance local scaling values, increasing from left to right. In fig. 9 present vendi scores and average predicted human preference scores. For any bin, we present results for the reward guidance scale that maximizes the local scaling. We see that even for a model trained with 50K samples, aesthetic score for most of the bins, Changes in local scaling, vendi and aesthetic scores are higher for the lower pre-guidance local scaling level set bins compared the higher pre-guidance local scaling level set bins.
have considerable increase in the local scaling, diversity and
Our experiments reveal that maximizing local scaling in the manifold of stable diffusion model directly correlates with adding texture to the generated images. Moreover, this approach reduces the likelihood on the manifold for single images. By optimizing the local scaling descriptor, the generative model is guided towards producing more varied and textured outputs.
This approach is notable because traditional methods for diversity guidance generally function the distribution level. Our method, however, focuses on maximizing the inherent diversity as preserved by the model within its learned manifold, effectively steering the generated images towards the extremities of the distribution. This instance-level intervention allows for more detailed and precise enhancement of diversity, presenting novel approach guiding generative models.
As seen from Fig. 10 (left-aanel) maximizing the reward results in added details form sharpening the image, adding texture and contrast. We also observe that if we move towards minimizing the reward, the images tend to loose fine-graine ame-grained details as seen in Fig. I0 (right-panel). Please refer to the supplementary material for more visual results.
CONCLUSION & FUTURE DIRECTIONS
In this paper, we present empirical evidence that the local geometric descriptors - local scaling (v), local rank (1)) and local complexity (8) - can effectively characterize the local geometry and distinguish between downstream qualitative aspects of generated samples such as generation quality, aesthetics, diversity, and memorization. Such descriptors only utilize the model's architecture and weights to characterize the behavior of generative models. We acknowledge two main limitations that warrant further investigation. First, the geometry of the learned manifold inherently influenced by the training dynamics of the model. A deeper understanding of this relationship is needed to fully leverage geometric analysis for models. Second, the computational complexity of our method, particularly the calculation of the Jacobian matrix, may pose practical challenge, especially for large-scale models. Future work should explore more efficient algorithms or approximations to address this limitation.
10

---

Published as a conference paper at ICLR 2025
REFERENCES
Georgios Arvanitidis, Lars Kai Hansen, and Søren Hauberg. Latent space oddity: on the curvature of deep generative models. arXiv preprint 1rXiv: 113,,,1911,, 2017.
Randall Balestriero and Richard Baraniuk. spline theory of deep learning. In International Conference on Machine Learning. pp. 374-383. PMLR, 2018a.
Randall Balestriero and Richard G Baraniuk. From hard to soft: Understanding deep network nonlinearities via vector quantization and statistical inference. arXiv preprint arXiv:1810.09277 2018b.
Randall Balestriero, Sehastien Paris, and Richard Baraniuk. Max-affine spline insights into deep generative networks. arXiv preprint arXiv:2002.11912 2020.
Arpit Bansal, Hong-Min Chu, Avi Schwarzschild, Soumyadip Sengupta, Micah Goldblum, Jonas Geiping, and Tom Goldstein. Universal guidance for diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 843-852, 2023.
Siyi Chen, Huijie Zhang, Minzhe Guo, Yifu Lu, Peng Wang, and Qing Qu. Exploring low-dimensional subspaces in diffusion models for controllable image editing. arXiv preprint arXiv:2409.022 02374, 2024.
Setareh Cohan, Nam Hee Kim, David Rolnick. and Michiel van de Panne. Understanding the evolution of linear regions in deep reinforcement learning. Advances in Neural Information Processing Systems, 35: 10891-10903, 2022.
Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans image synthesis. Advances in neural information processing systems, 34:8780 8794, 2021.
Dan Friedman and Adji Bousso Dieng. The vendi score: diversity evaluation metric for machine learning. Transactions on Machine Learning Research, 2023.
Quentin Garrido, Randall Balestriero, Laurent Najman, and Yann Lecun. Rankme: Assessing the downstream performance of pretrained self-supervised representations by their rank. International Conference on Machine Learning. pp. 10929 10974. PMLR, 2023.
IJ Goodfellow, J. Pouget-Abadie,
Mirza, B. Xu, Warde-Farley, S. Ozair, Courville, and Y. Bengio. Generative adversarial nets. In Proceedingy of the 27th International Conference on Neural Information Processing Systems, pp. 2672-2680. MIT Press, 2014.
Nathan Halko, Per-Gunnar Martinsson, and Joel A Tropp. Finding structure with randomness: Probabilistic algorithms for constructing approximate matrix decompositions. SIAM review, 53(2):217-288, 2011.
Boris Hanin and David Rolnick. Complexity of linear regions in deep networks. arXiv preprint arXiv: 2019.
Dan Hendrycks and Thomas Dietterich. Benchmarking neural network robustness to common corruptions and perturbations. In International Conference Learning Representations, 2019. URL https://
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840 6851, 2020.
Jeremy Howard. Imagewoof: subset of 10 classes from imagenet that aren't easy to classify. Gitllub, March 2019. http::/t: httpt/tttttaat..///e wtw.:ttab.ccmccocceem fastai iwwgenetteeeeeeemmmmmm.emeee itt.teeemmmmooooo
Ahmed Imtiaz Humayun, Randall Balestriero, and Richard Baraniuk. Magnet: Uniform sampling from deep generative network manifolds without retraining. International Conference on Learning Representations. 2021.
Ahmed Imtiaz Humayun, Randall Balestriero, and Richard Baraniuk. MaGNET: Uniform sampling from deep generative network manifolds without retraining. In ICLR, 2022a. URL httphtttps.//opeeeeeeew n [forum?
d=r ddmmm?ddrrrggne Lilliiiiwi Ywf9
Ahmed Imtiaz Humayun, Randall Balestricro, and Richard Baraniuk. Polarity sampling: Quality and diversity control of pre-trained generative networks via singular values. In CVPR, pp. 10641-10650, 2022b.
Ahmed Imtiaz Humayun, Randall Balestriero, Guha Balakrishnan, and Richard G Baraniuk. Splinecam: Exact visualization and characterization of deep network geometry and decision boundaries. In Proceedings of the IFFE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3789-3798, 2023.
11

---

Published as a conference paper at ICLR 2025
Ahmed Imtiaz Humayun. Randall Balestriero, and Richard Baraniuk. Deep networks always grok and here is why. arXiv preprint arXiv:2402.15555, 2024.
Zahra Kadkhodaie, Florentin Guth, Eero P Simoncelli, and Stéphane Mallat. Generalization in diffusion models arises from ceometry-adaptice harmonic representation, arXiv preprint arXiv:2310.02557, 2023.
Hamidreza Kamkari, Brendan Leigh Ross, Jesse Cresswell, Anthony I. Caterini, Rahul G Krishnan, and Gabriel Saaiza-Ganem. A geometric explanation of the likelihood ood detection paradox. arXiv preprint arXiv: 20::::203./9000 08900, 2024.
Tero Karras, Samuli Laine, and Timo Aila. A style-based generator architecture for generative adversarial networks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4410- 2019.
Tero Karras, Samuli Laine. Miika Aittala, Janne Hellsten, Jaakko Lehtinen, and Timo Aila. Analyzing and improving the image quality of stylegan. In Proc. CVPR, pp. 8110-8119, 2020.
Diederik P Kingma and Max Welling. Auto-encoding variational bayes. arXiv preprint arXiv: 2013.
Line Kuhnel, Tom Fletcher, Sarang Joshi, and Stefan Sommer. Latent space non- non-linear statistics. arXiv preprint arXiv:1805.0777 2018.
Youwei Liang, Junfeng He, Gang Li, Peizhao Li, Arseniy Klimovskiy, Nicholas Carolan, Jiao Sun, Jordi Pont- Tuset, Sarah Young, Feng Yang, et al. Rich human feedback for text-to-image generation. In Proceedings of the IEEE/CVF Conference Computer Vision and Puttern Recognittin,, pp. 19401-19411, 2024.
Alexandra Sasha Luccioni, Christopher Akiki, Margaret Mitchell, and Yacine Jernite. Stable bias: Analyzing
societal representations diffusion models. arXiv preprint arXiv:2303. J /408, 2023.
Hila Manor and Tomer Michaeli. On the posterior distribution in denoising: Application to uncertainty quantification. arXiv preprint arXiv:2309. 2023.
William Peebles and Saining Xie. Scalable diffusion models with transformers. Proceedings of the IEEE/CVE International Conference on Computer Vision, pp. 4195-4200 2023.
Ben Poole, Subhaneil Lahiri, Maithreyi Raghu, Jascha Sohl-Dickstein, and Surya Ganguli. Exponential expressivity in deep neural networks through transient chaos. In Advances In Neural Information Processing Systems, pp- 3360-3368, 2016.
Maithra Raghu, Ben Poole. Jon Kleinberg, Surya Ganguli. and Jascha Sohl Dickstein. On the expressive power of deep neural networks. In ICML, pp. 2847-2854, 2017.
Salah Rifai, Pascal Vincent, Xavier Muller, Xavier Glorot, and Yoshua Bengio. Contractive auto-encoders: Explicit invariance during feature extraction. In Proceedings of the 28th international conference on machine learning (ICML-II), pp. 833-840 840, 2011.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High-resolution image synthesis with latent diffusion models. 2022 ieee. In CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 10674-10685, 2021.
Nataniel Ruiz, Yuanzhen Li. Varun Jampani, Yacl Pritch, Michael Rubinstein, and Kfir Aberman. Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pottern Recognition, pp. 22500-22510, 2023.
Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. arXiv preprint 2rX10:2010.02502, 2020.
Jan Stanezuk, Georgios Batzolis, Teo Deveney, and Carola- -iibiane Schönlieb. Your diffusion model secretly knows the dimension of the data manifold. arXiv:2212.1266 2022.
Yuxin Wen, Yuchen Liu, Chen Chen, and Lingjuan Lyu. Detecting, explaining, and mitigating memorization in diffusion models. In The Twelftt International Conference on Learning Representations, 2024. URL
httts://cpenreviiet net/ for 2d= F 4n33wwkTb
Bing Xu. Empirical evaluation of rectified activations in convolutional network. arXiv preprint arXiv: 1505.0083. 2015.
Thomas Zaslavsky. Facing up to arrangements: Face-count formulas for partitions space by hyperplanes: Face-counl ficce-cuulformmllas for partitions of space by hyperplanes, volume 154, American Mathematical Soc., 1975.
Shengjia 7hao, Hongyu Ren, Arianna Yuan, Jiaming Song, Noah Goodman, and Stefano Ermon. Bias and generalization in deep generative models: An empirical study, arXiv preprint 1rXiv:1811.03255 2018.
12

---

Published as a conference paper at ICLR 2025
A COMPUTATION TIMES FOR LOCAL DESCRIPTORS
The computation times for the local scaling and local rank computation (since both require one randomized SVD computation for one latent vector) ends up being 3929s for 1000 samples. For local complexity we require 113s for 1000 samples. All the estimates are for a JAX implementation of Stable Diffusion on TPUv3.
Note that to train a reward model, we require the descriptors be computed only once for each pre-trained model. If we compute the local scaling for 100k samples we require 173.1 TPU v3 hours which is equivalent to 54.58 V100 hours (according (2021)). Compared to 79,000 A 100 hours required for Stable Diffusion training 24000 hours with enterprise level optimization f the computation required for the descriptors and reward model training is significantly small. The computation time for the local descriptors can be further reduced by using smaller & for our projection matrix W, or by using non-jacobian based methods, c.g., estimating the local scaling by measuring the change of volume for a unit norm e f1-ball in the input space. We leave exploration of these directions for future work.
Appendix A.3 Dhariwal & Nichol
B RELATED WORKS
Local geometry pre-diffusion. Early applications of the local geometry of generative models involved improving the generation performance and/or utility of generative models via geometry inspired methods. For example, iniRiai et al. (2011) the authors proposed regularizing the contraction of the local geometry to learn better representations in autoencoders trained on MNIST and CIFAR I0. The regularization penalty is employed via the norm of the input-output jacobian in Rifai et al. (2011) is an upper bound for local scaling presented our paper. In Arvanitidis et al. (2017) the authors provided visualizations on the curvature of pre-trained VAE latent spaces and proposed using an auxiliary variance estimator neural network to regularize the latent space geometry during generation. In Kuhnel et al. (2018) the authors perform latent space statistical inference problems, e.g., maximum likelihood inference, by training a separate neural network to approximate the Riemannian metric. In Humayun al. (2022a) the authors proposed a novel latent space sampling distribution based on the latent space geometry that allows uniformly sampling the learned data manifold of continuous- piecewise affine generators. The authors showed downstream benefits with fairness and diversity for such latent space samplers. While most of these methods discuss pre-diffusion architectures, their results are early demonstrations of how the local geometry can affect downstream generation. also employ auxiliary Neural Networks to model an intrinsic property of pre-trained generator, similar to how we propose using a reward model for Stable Diffusion.
Local intrinsic dimensionality of diffusion models. The local geometry of diffusion models and possible applications have garnered significant interest in recent years. In Stanczuk al. (2022) the authors propose a method to compute the intrinsic dimensionality of diffusion models using the assumption that the score field is perpendicular to the data manifold. For any vector x on the data manifold, the method requires computing the dimensionality of the score field around x and subtracting it from the ambient dimension. To do that, the authors perform one step of the forward diffusion process k times for x, denoise the & noisy samples using the diffusion model and compute the rank of the data matrix containing denoised samples to obtain the intrinsic dimensionality. Compared to this method, we compute the dimensionality directly via a random estimation of the input-output jacobian SVD. We do not require any assumption on the score function vector field being perpendicular to the data manifold, which may not hold for diffusion model that is not optimally trained or highly complex training datasets like LAION.
In Kamkari et al. 2024) the authors compute rank using the method proposed in Stanczuk et al. (2022) and show that local intrinsic dimensionality can be used for outoo--distribution (OOD) detection. This is analogous to our analysis in Sec 3 on the local geometry or off the manifold. We can see that the intuition authors provided in Kamkari et al. (2024) for diffusion models trained on smaller models and datasets e.g.. FMNIST, MNIST, translate larger scale models like Stable Diffusson trained on LAION as we have presented fig. 5 fig. [7 and See 4. Especially in fig. 7 we show that creating OOD samples with corruptions on Imagenet data (in-distribution), we can have increase or decrease
13

---

Published as a conference paper at ICLR 2025
in negative-log likelihood (estimated via local scaling), with decrease for blurring corruptions and increase in noising corruptions.
Concurrent work Kamkari et al. (2024) has also shown the relationship between the intrinsic dimen- sionality (local rank) of Stable Diffusion scale models and the texture/visual complexity of generated images. We believe our analysis is much more holistic with three different geometric properties being measured compared to only local dimensinality. We i) show quantitatively how diversity measured via vendi score is higher for higher local scaling and rank values (fig. (6). We have explored how rank and scaling evolves continuously across the latent space in fig. 5 We have presented how the geometry distribution varies as we continually perturb images via noise or blurring operations fig. [ig-[7 And finally in Sec 5 we have presented a method to guide generation using the local geometry to obtain downstream generation benefits.
Misc. Apart from the aforementioned works, Kadkhodaie et al. (023) show that the emergence of generalization in diffusion models when two networks separately trained on the same data learn the same mapping- can be attributed to the cigenspectrum and eigenvectors of the input-output jacobian. While we do not study the training dynamics of the local geometric descriptors in our paper, Kadkhodaie et al. (203) suggests that the local geometry can be an important indicator of memorization and generalization emergence in diffusion models. In Manor Michaeli (2023) the authors use the posterior principal components of a denoiser for uncertainty quantification. This work suggests that components with larger eigenvalues result in larger uncertainty which is directly related to the local scaling descriptors as it measures the product of non-zero singular values. While in Manor& & Michaeli (2023) the authors propose using it for only a single image denoiser, we show that it generalizes for any diffusion model including Stable Diffusion scale text-to-image diffusion models.
C CORRELATIONS BETWEEN THE THREE DESCRIPTORS
Local scaling characterizes the change of volume by the affine slope A going from the latent space to the data manifold. Local rank characterizes the number of dimensions retained on the manifold after the network locally scales the latent space. Both local rank and scaling quantify first order properties of the CPWL operator. Local complexity approximates the 'number of unique affine maps' within a given neighborhood Humayun et al.] (022) by computing the number of CPWL knots intersecting an & ball in the input/latent space. Therefore local complexity is measure of 'un-smoothness' and quantifies local second-order properties of a CPWL. operaooe
Correlations between local scaling to and local rank v. By definition, local scaling and local rank correlated, since both characterize the change of volume by the network input- map at any input space linear region also evident in eq. (2) and eq. (3). Local scaling is also upper bounded by
local rank, the < og- where on is the largest singular value of A The correlation is evident for our low dimensional DDPM setting presented in fig. 3 local rank and local scaling are highly correlated in their spatial distribution. There are indications suggesting that the correlations persists throughout training as can be seen in fig. 3 righttost column top and bottom. However in fig, 5 we can see that in the high-dimensional Stable Diffusion latent space, local scaling and rank are correlated but local rank has sharper changes spatially compared to local scaling.
Correlations between local complexity 8 and rank v. There also exist correlations between local complexity and local rank due to the continuity of CPWL maps between two neighboring linear regions W1 and W2, the corresponding slope matrices Aw, and A differ by at most one row. Therefore between two neighboring regions w1 and W2, [ww, //wyll <= 1. Informally, the local rank in neighborhood V is lower bounded by the number of non-linearities in neighborhood V. This is evident in the empirical results presented in fig. and fig. [ In both figures, for input space neighborhoods with higher local complexity, we see decrease in local rank. However, observe sharp changes in local complexity as we observe in local rank in fig. [ In fig. 33] we see that local rank is more discriminative of the data manifold compared local complexity. Their training and denoising dynamics differ significantly as seen in fig. rightmost column.
do not
Qualitative and quantitative results on correlations. We train a beta- VAE unconditionally on MNIST and present in Fig. IT samples from increasing local descriptor level sets from left to right along the columns. In Fig. [2] we present joint distributions of local scaling, complexity, rank and mean squared reconstruction error for training and test samples. We see that while local scaling,
14

---

Published as a conference paper at ICLR 2025
complexity and rank have some linear correlation, the classwise distribution in fig. 12 very different between the three. We also present in fig. [3 the vendi score for increasing local scaling level sets and evidence that the population means for the descriptors don't follow the same pattern between sub-populations.
9.44
0114
012
0.12
0.12
04
8.9
0.10
6:00
0/96
0.06
36
2.
-00
-70
-53
-26
-10
3
23 68 83 300 -43
-78
-20
-50
-40 -30
-73
-50
-50
Figure 11: Joint distributions for local scaling and MSE, local rank and MSE, local complexity and MSE, local scaling and local complexity, and local scaling and local rank. We observe that local complexity is linearly correlated wth MSE, with higher complexity images incurring higher error. Local scaling, rank and complexity have correlations between them as well.
Increasing vi
Increasing 5
Increasing v +
0 697
///
54 547077
Train
6
2 11
1/770
20200
2 21:211
2. 3
7 5
4 /
2 1
/ 1/1776
4644
12
2
1
1
20
11228
466404000
3
/2111
1104 6 i
22 2
2
14053
11457
7
22022
33075 97559472220
16004040 1 &
Gen.
60342 ( 11
31245111 1 / / (7543+8033 1111101000
/ 07711300002100 1240022
# 9 140 2 1 P1
f 1 4 24 /
6333
121110611
0//111020221 234422221
111/111 10041100100000
1111119000000000
21ra5/ wnd.nngnnn
Figu 3
From
40. (5 Rengl procent teaiig Sanples (10 1 gonertied -wwelteee-iiip ( RoVa
for linearly increasing level sets of local scaling (t)) from 1-80--22 local complexity (8) from
[0, 120] and local rank (v) from [1.5, Not all level sets had an equal number of samples from training/generater distributions. We see that for higher v. we have more outlier samples whereas for lower v we have modal samples. For increasing â we see that the quality of generated samples decreases and the diversity of samples is reduced as well. For higher digits become more regularly shaped.
Tain
10
Gen.
5
80
60
50
100
4
80
60
50
100
4
Figure 13: (Left panel) Vendi score (Friedman Dieng 2023) calculated for samples from different local descriptor level sets of Beta- Beta-VAE. We take upto 150 samples from each level set and compute vendi score seperately for the MNIST train dataset, test dataset and generated samples. (Right panel) Sub-population differences of local descriptors in training data. We see that the order of sub- tobp-opulation means for the three classes, are not the same for all three descriptors.
D ADDITIONAL EXPERIMENTS
D.1 LOCAL SCALING FOR TRANSFORMER BASED DIFFUSION MODEL
Since we are based on the CPWL formulation NNs, our framework would generalize to models of any scale and any architecture with CPWL non-linean linelerriss.s. Empirically we have shown it to generalize for non-CP CPn- architectures like Stable Diffusion v1.4 and DDPM that employs non CPWL. non-linearities such attention, GeLUU and much more. We have performed additional
15

---

Published as a conference paper at ICLR 2025
experiments with DiT-XL Peebles & Xie (2023) trained Imagenet-256. For the DiT compute the descriptors for the transformer network, conditioned on noise level = 0, i... zero noise level. We generate 5120 images conditioned on (irngeeoof/Howar (2019) classes and present in fig. [4] increasing local scaling level sets from left to right. We see that similar to fig. 17 from the, DiT exhibits a qualitative correlation between visual complexity and local scaling. For additional analysis we repeat the Stable Diffusion experiments on the relation between diversity and local scaling for DiT. We see that similar to Stable Diffusion, for increasing local scaling level sets, the diversity of images increase and then drop for the highest local scaling level sets.
12
Figure 14: Left: Vendi score and membership counts for increasing local scaling level sets, computed for DiT transformer. We see that similar to Stable Diffusion, local scaling increases from lower to higher local scaling level sets, then drops for very high local scaling level sets. Right: Generated samples from each level the left panel. Sample sets from higher local scaling level sets, tend be more diverse.
0.667
t 0.447
t = 0.221
t= 0
EM (
400
6.3
66
E4
1134
160
+4.
+16
1055 12:500
Figure 15: Local geometric descriptors computed the input domain of pre-trained toy diffusion model trained to produce samples from a dinosaur manifold M E R22 Descriptors are computed by conditioning the diffusion model noise level We consider the of input vectors within 0.05 units of the training data as on manifold M and rest as off the manifold M. We present the difference between the expected descriptor values on and off the manifold, E.M [b] (165, 8t. v²} at different training iterations (right). We also present the descriptor computed over [-6, 6]2 for different noise levels t after 125000 training iterations (rest). We observe that vf is lower, 8¹ is higher and w' is lower on the manifold than off the target manifold for lower noise levels, especially after the model is considerably trained. This indicates that for well trained diffusion model, ie., learned manifold M x M, local descriptors can distinguish between on and off manifold vectors in the input space.
[9), vΦ
D.2 VAE TRAINING DYNAMICS FOR MNIST
Setup. We train Variational Auto Encoder (VAE) the MNIST dataset with width 128 and depth 5 for both encoder and decoder. We add Gaussian noise with standard deviation (0, 0.0001, 0.001, 0.01, 0.1) to the training data. Initialization not kept fixed. In Fig. 16
16

---

Published as a conference paper at ICLR 2025
present plots showing the training dynamics of local complexity and scaling, averaged over all test dataset points from MNIST.
Figure 16: Training dynamics of geometric descriptors for a VAE trained on MNIST with additive noise. As training progresses local complexity 5 increases and local scaling v decreases suggesting an increase in expres- sivity and decrease in uncertainty on the data manifold. At latter time-steps, 1/ + and ó if noise std. is increased.
203
-50
-70
-30
2048 5006 7909 1000 1259
1966 3
2530
5800
1200 a000 12500 Lodeo
optimization steps
Observations. By increasing the noise control the puffiness of the target manifold. We observe that as the noise standard deviation increased there 1) increase indicating the manifold becomes less smooth 2) decrease in local scaling indicating that the uncertainty decreases. We can also observe an initial dip in both local complexity and local scaling. This similar to what was observed for discriminative models in (Humayun al., 2024) where double descent behavior was reported the local complexity training dynamics of classification models. Based on these results, contrary the observation in (Humayun al. 2024), generative models do have double descent in local complexity however we do observe double ascent in local scaling, Our observations suggest that the training dynamics need be taken into account, when comparing the local manifold geometry between two separately trained models.
E ENTROPY DIFFERENCE BETWEEN TWO REGIONS
Suppose have injective Z X mapping learned by CPWL generator G. Any linear region in the latent space CPWL partition Q is mapped to unique region on the output manifold. We define S as:
S = G(z) A.Z +bw Vz E
The change of volume from w + is den T A.)) Therefore for any latent = and output (zz):
Pgl (x) Evwen / ddet(A. A...
(x)
H =EW
For any 21 E Wi the sum from the above equation can be ignored, since for all other regions the value would be zero.
Taking negative log and expectation on both sides the conditional entropy becomes
(pg(x)); E w1)= (Pz(21)) + log( / det((.., T TAw,))
For a uniform latent distribution and two regions w1 and W2, substituting the second term above with dhan
H (pg(xx)):z E w1) H (pg(x2); E w2) = thes Shee
F BROADER IMPACT STATEMENT
Our proposed framework for assessing and guiding generative models through manifold geometry offers several potential benefits to society. By providing more objective and automated approach, we can significantly reduce the cost and time associated with human evaluation, making the auditing and mitigation of biases in large-scale models more accessible and efficient. This has implications for promoting fairness and equity Al systems, particularly in domains where biases can have significant societal consequences.
Furthermore, our approach can empower researchers and practitioners to better understand the relationship between the geometry of learned representations and various aspects of model behavior,
17

---

Published as a conference paper at ICLR 2025
such generation quality, diversity, and bias. This deeper understanding inform the development of more robust and reliable generative models, leading to advancements in various fields, including art, design, healthcare, and education.
However, we recognize that our approach not without limitations and potential risks. While it can be valuable tool for identifying and mitigating biases, should not and cannot fully replace human annotators, especially in high-risk domains where human judgment and contextual understanding are crucial. Our method focuses on reducing costs and improving the auditing process, but it should be used as a standalone approach.
Moreover, the increased automation enabled by approach raises concerns about the potential displacement of human annotators, leading to job losses and economic disruptions, While our method addresses some aspects of model evaluation, model behavior. Therefore, it should be used with caution and in conjunction with other evaluation methods, including human expertise.
comprchensive and cannot assess all facets of
G EXTRA FIGURES
Increasing Ab
Figure 17: Local Scaling is sensitive to natural image variations. ImageNet images ordered along the columns (from left to right), with increasing local scaling of the Stable Diffusion decoder learned manifold. We observe that ImageNet samples with lower values of s contain simpler backgrounds with modal representation of the object category. Conversely for higher have increasing diversity both in background and foreground features.
18

---

Published as a conference paper at ICLR 2025
Figure 18: Images generated during 50 diffusion denoising steps for top bottom, COCO prompts generated with guidance scale 1,5,9 and memorized prompts generated with guidance scale 7.5. Higher guidance scale images, as well as memorized images, tend to resolve faster during the denoising process.
19

---

Published as a conference paper at ICLR 2025
Low local rank, +
High local rank, v
Figure 19: Influence of the local rank descriptor value on overall image perception. Images with the lowest (left) and highest (right) local rank from of 20000 randomly sampled ImageNet dataset samples. Low rank images contain simpler textures for every class compared to the high rank samples. This is because for images with higher local rank, the learned manifold is higher dimensional therefore allowing higher independent degrees of variations locally for the generated images.
Low local scaling, &b 1
High local scaling, s/
Figure 20: Influence of the local scaling descriptor.mmagnntt images with high and low local scaling for the stable diffusion decoder. Each coordinate in both left and right image grids, correspond the same imagenet class.
20

---

Published as a conference paper at ICLR 2025
1=
t= 17
t= 28
L= 39
50
Figure 21: 2D Data Manifold Geometry, A toy Example. After 11395 optimization steps. Geometry of diffusion model onput-output mapping, trained to on toy 2D distribution. Local scaling lower around data manifold, local complexity higher around manifold, rank is lower around manifold as well. t=50 has considerably low variance in local scaling showing that final timestep has a diminishing change of density.
21

---

Published as a conference paper at ICLR 2025
Figure 22: Reward guidance on stable diffusion (maximizing the reward). We observe significant increase in both background detail and artifact diversity within the generated images.
0
p=1
p = 2
p= 3
Australian terrier with a city in the background
a Australian terrier with city the background
P=-1
p=-3
p p=-5
Figure 23: Controlling mage diversity with local scaling. Using Reward guidance to increase (top row) and decrease diversity (bottom row) using same initial seed.
22

---

Published as a conference paper at ICLR 2025
2
Australian terrier the
0 -=
Australian terrier the snow
Figure 24: Controlling mage diversity with local Sealing.Using Reward guidance to increase (top row) and decrease diversity (bottom row) using same initial seed.
Figure 25: Decoded images (right) using 20 latents (left) from the 2D subspace, with highest v. Each image bounding box (right) is color coded according to the corresponding latent vector (left).
23

---

Published as a conference paper at ICLR 2025
Figure 26: Decoded images (right) using 20 latents (left) from the 2D subspace, with lowest vh. Each image bounding box (right) is color coded according to the corresponding latent vector (left). Selected latents lie outside the domain of the VQGAN latent space.
Decoded images (right) using 20 latents (left) from the 2D subspace, with highest v. Each image bounding box (right) is color coded according to the corresponding latent vector (left). Selected latents lie outside the domain of the VQGAN latent space.
Figure 27:
24

---

Published as a conference paper at ICLR 2025
Figure 28: Decoded images (right) using 20 latents (left) from the 2D subspace, with lowest v. Each
image bounding box (right) is color coded according the corresponding latent vector (left).
Figure 29: Decoded images (right) using 20 latents (left) from the 2D subspace, with highest 5. Each image bounding box (right) is color coded according to the corresponding latent vector (left).
Figure 30: Decoded images (right) using 20 latents (left) from the 2D subspace, with lowest 8, Each image bounding box (right) is color coded according the corresponding latent vector (left).
25

---

Published as a conference paper at ICLR 2025
Chest Xray
magenet
10
-10
Figure 31: UMAP visualization of the aggregated local geometry descriptors (local smoothness, local rank, and local complexity). This reveals distinct, non-overlapping clusters, clearly separating the Imagenet, Imagenet Corrupted with Gaussian Noise, and Chest X-ray datasets. This visual evidence underscores the effectiveness of aggregating the descriptors to capture unique patterns within each dataset, demonstrating its ability provide meaningful and interpretable representation of the underlying data
Training Samples
Generated Samples
Local scaling, M
Figure 32: Highest local scaling level sets are anti-modes. In all of our experiments, we have observed that samples from the highest local scaling level sets have lower vendi score. We repeat the experiments from Fig. 15 for mixture of nine gaussians. We see that the regions between the gaussian modes inside the domain of the data, i.e., the anti-modes, have higher local scaling, with the highest local scaling values the center of the anti-modes.
26


Reviews:
[{"review_id": "2eyKAWmcjo", "reviewer": "Reviewer_WPMu", "summary": "This paper, \"What Secrets Do Your Manifolds Hold? Understanding the Local Geometry of Generative Models\", explores the role of local geometry in deep generative models and its effect on generation quality, diversity, and memorization. The authors propose three geometric descriptors—local scaling ($\\psi$), rank ($v$), and complexity ($\\delta$)—to characterize the latent space of generative models, focusing on models with Continuous Piecewise-Linear (CPWL) mappings. They empirically demonstrate that these descriptors are correlated with various aspects of generative performance and propose a reward model trained on these descriptors to guide sample generation in diffusion models, particularly Stable Diffusion.", "questions": "**1.** I am curious about the computational costs such as GPU hours per 1k samples to calculate each descriptor value, as well as the data preparation of reward models. I think this would give readers a more comprehensive understanding of the empirical impact of the proposed approach.\n\n**2.** I think it would be good if authors could provide more studies on DiT or MM-DiT-based text-to-image diffusion models. The UNet-based Stable Diffusion 1.4 model is kind of weak when compared with current strong DiT-based models such as Stable Diffusion 3 or Flux. This study will give readers an in-depth understanding of how compatible the approach is across different neural network architectures. \n\n**3.** It would be good if authors could provide more intuitive discussions on each of the three descriptors. This will help readers understand the intuitions behind them. \n\n**4.** The author should provide quantitative metrics such as FID or human preference scores when using reward guidance sampling.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "**1. Interesting Indicators of Geometry**: This paper introduces an interesting use of geometric descriptors to understand and control generative models, providing new insights into latent space structure.\n\n**2. Comprehensive Experiments**: The paper includes extensive experiments across various models, from toy models to Stable Diffusion.\n\n**3. Practical Framework for Control**: The reward model offers a practical way to influence generation characteristics, such as diversity and aesthetic quality, by guiding sampling in the latent space based on geometry based on the trained scalar model of the geometry descriptor.", "weaknesses": "**1. Computational Demands**: The method relies on calculating Jacobians on each linear piece of model manifold. This may be computationally intensive for large models, limiting its practical application. Such a limitation potentially indicates the approach is less scalable. Besides, since the computation of each geometric descriptor is computationally demanding, generating data for training latent space reward models is also computationally demanding and therefore possibly not scalable. \n\n**2. Descriptor Interpretation**: Some descriptors, particularly local complexity ($\\delta$), lack intuitive interpretation when applied to high-dimensional latent spaces, which could be further clarified. This makes the paper more like a pure empirical computation of each descriptor instead of an in-depth study of the latent geometry of generative models.  \n\n**3. Weak Evaluations**: A major drawback of this paper is the weak lack of quantitative evaluations. For instance, the paper should provide quantitative metrics such as FIDs or Human preference scores to evaluate the performances of the proposed geometry descriptor guided sampling instead of merely qualitative plots.\n\n**4. Poor Compatibility for Networks with Smooth Activation Functions**: If I do not misunderstand, I think the approach is only properly defined for models with piece-wise linear neural networks instead of those networks using smooth activation functions such as SiLU, GELU or SwiGLU. This might limit the broader usage of the proposed approach and the impacts of the study.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Cx4FVPYO4v", "reviewer": "Reviewer_fW1y", "summary": "This paper aims to study the local geometry of pretrained diffusion models and explore how different local geometry indicates different properties. It proposes three descriptors scaling (ψ), rank (ν), and smoothness (δ) to characterize the local geometry in diffusion models based on the continuous piecewise-linear (CPWL) assumption. The paper uses these descriptors to study the local geometry in classical DDPM and stable diffusion models. The paper conducts various experiments to show the following statements: 1. Local geometry of off manifold is different from on manifold. 2. Increasing local scaling leads to the generation of more complex image content. 3. Local geometry of Stable Diffusion is sensitive to image corruptions. 4. Local geometry of Stable Diffusion is sensitive to memorization of text prompts. Lastly, the paper proposes to guide generation with gradients of the descriptors (specifically local scaling). To avoid heavy computation of Hessian, they actually train a reward model to approximate local scaling. With this geometric guidance, the paper can control content complexity in the generated images.", "questions": "(a)  Where does equation (3) come from?\n\n(b) In equation (7), how is B defined? (i. How to choose number of dimensions P, is it the same aound any latent $z$? ii. How to choose columns in B? iii. Why a constant projection B can be used for all neighbor around the specific $z$?)\n\n(c) What explicitly are the input space vectors for DDPM and Stable Diffusions? Are they the inputs to the denoising UNet, or could they be features in the middle layers of the UNet?\n\n(d) In section 3.1 L250, \"input vectors within 0.05 units of the training data\", how is 0.05 chosen and why it is a reasonable value?\n\n(e) In Figure 4, how are the three anchors (representing latent from 3 text prompts) labeled in the latent space and what are other points?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 3, "strengths": "1. The paper is studying the local geometry from a comprehensive perspective, modeling scaling (ψ), rank (ν), and smoothness (δ).\n\n2. The paper shows experiment results from several perspectives including off and on manifold, content complexity, image corruption, and text prompt memorization.\n\n3. The paper utilizes local scaling to control the generated image complexity.", "weaknesses": "1. Lack of literature review. There exists prior work [1] related to out-of-domain detection with local geometry, and [2] related to uncertainty quantification with local geometry. Even though the metrics are not exactly the same, it is worth discussing these previous works, as well as other ones related to the manifold of diffusion models.\n\n2. Lack of clarity and technical details. See the questions part.\n\n3. Lack of Justification of the continuous piecewise-linear (CPWL) assumption: The paper does not provide justification that CPWL can be a good approximation for DDPM and stable diffusion. In contrast, recent work [3] has experimentally verified such local linearity.\n\n4. Redundancy of metrics: Though all three metrics are interesting, it seems rank (ν), and smoothness (δ) are redundant since most observations and applications can be correlated only with scaling.\n\n[1] A Geometric Explanation of the Likelihood OOD Detection Paradox. Hamidreza Kamkari et al.\n\n[2] On the Posterior Distribution in Denoising: Application to Uncertainty Quantification. Hila Manor et al.\n\n[3] Exploring Low-Dimensional Subspaces in Diffusion Models for Controllable Image Editing. Siyi Chen et al.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "kLIA2UPqMm", "reviewer": "Reviewer_KSLy", "summary": "This paper studies the local geometry of the generative model and its relationship to the quality of the generation. Built upon the theory of continuous piecewise-linear generators, the authors propose to use three metrics: (1) local rank; (2) local scaling; and (3) local complexity to quantify the geometry of the generative model. Leveraging these descriptors, they try to understand how the local geometry relates to the generation process. Specifically, they show that local scaling is related to the complexity of the generated scenes, and can be used as a tool to detect whether the data is on the input (training) manifold. In addition, they propose to train a reward model to approximate the local scaling descriptor and present results showing more objects and complex backgrounds while increasing the value of the local scaling.", "questions": "1. Eqn.2: Should the singular values here be normalized?\n\n2. Line 164: What's the definition of $S$, why the local scaling is proportional to the NLL of the generative model when the relationship between $\\psi$ and $S$ is injective?\n\n3. Lines 262-269: Where are the corresponding results and visualizations? Why do results at 0.17T show the minimized discrepancy between the on-manifold and off-manifold input space? And why does the minimized discrepancy indicate ``local geometry indicators have the highest distinction geometrically between on and off manifold vectors from the input space'' (should it be the opposite)?\n\n4. Figure 3: What does $[-6, 6]^2$ mean? What's the x-axis of the plot on the right, is it the noise scale?\n\n5. Figure 3 and Figure 4: What are the axes of the local generator? Does each pixel correspond to one input image? How is this being calculated?\n\n6. Figure 4: What images correspond to the pixels in the center of the geometric desciptors?\n\n7. Section 5: How is the reward function defined? Why the calculation of the Hessian matrix is required in the first place? The paper describes three local geometric descriptors, but why does the reward model only approximate the local scaling value?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "The idea is novel and the authors conduct extensive experiments in multiple scenarios. It is very interesting to understand the geometry of the model and its role in the generation process.", "weaknesses": "1. My biggest concern regarding this paper is the approachability to the audience. The three local geometric metrics discussed in the paper are very interesting. However, in terms of the method presentation, particularly the reward model built upon the local scaling descriptor, the current presentation lacks detailed descriptions, which makes me concerned about the reproducibility of the results.\n\n2. The claim that \"local geometry trajectories are discriminative of memorization\" is relatively weak for the local complexity metric when the trend of the metrics is almost un-distinguishable when increasing the guidance scale.\n\n\nMinors:\nLine 323: I think it should be capitalized DDPM instead of \"ddpm\".", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "9Vz5NWD7iG", "reviewer": "Reviewer_dk5T", "summary": "Building on the continuous piecewise-linear hypothesis, this paper introduces three local geometric descriptors for generative models at specific stages of generation: scaling, rank, and smoothness. Using these descriptors, the paper empirically examines the local geometry of both a toy dataset and Stable Diffusion, drawing several conclusions about the learned image manifold. Finally, leveraging these descriptors, the paper proposes a novel reward model that guides the generation in Stable Diffusion, allowing control over specific local geometric properties.", "questions": "1. In line 115, the paper discusses unrolling the sampling process of the diffusion model. Does this mean that when calculating the descriptor at a large timestep $t$, you need to compute the Jacobian iteratively over the neural network multiple times because the sampling steps are large? From my experiments, computing the Jacobian over the UNet in Stable Diffusion takes hours, even without unrolling. What is the computational cost of your implementation?\n\n2. Are these three proposed descriptors redundant? From Figures 5, 12, and 13, it appears that as local scaling, local rank, and local complexity increase, the visual complexity of the images all increases. Seems we only need one descriptor for the image visual complexity. What specific, distinct properties do each of these three descriptors capture?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 4, "strengths": "This paper represents a valuable empirical contribution to understanding the local geometric properties of the image manifold learned by diffusion models. Based on the hypothesis that the output space of neural networks exhibits a continuous piecewise-linear structure, the learned data manifold can be approximated as a union of affine transformations across local regions. This foundation makes the proposed local geometric descriptors both intuitive and well-motivated. In empirical experiments, these descriptors align closely with human visual evaluation, underscoring their effectiveness. I believe this paper will significantly impact the study of image manifold geometry.", "weaknesses": "1. This paper lacks discussion with related works [1, 2, 3, 4]. The local dimensionality of the diffusion model-based image manifold has also been studied in those papers. However, as concurrent work, this won't significantly weaken the contribution of this paper. Better discussion about the relation and differences with those works could highlight the contribution of this paper.\n2. The proposed method is not very strong. The authors provide only qualitative results without any quantitative evaluation or comparison to related works. \n3. Some parts of the writing are unclear. I suggest including the numerical method used to approximate the calculation of local complexity (Definition 3) in this paper. Although this method was introduced in previous work, including it here would make the paper more complete and easier to understand.\n\n\n\n\n\n\n[1] Stanczuk, Jan Pawel, Georgios Batzolis, Teo Deveney, and Carola-Bibiane Schönlieb. \"Diffusion Models Encode the Intrinsic Dimension of Data Manifolds.\" In Forty-first International Conference on Machine Learning.\n\n[2] Kamkari, Hamidreza, Brendan Leigh Ross, Rasa Hosseinzadeh, Jesse C. Cresswell, and Gabriel Loaiza-Ganem. \"A Geometric View of Data Complexity: Efficient Local Intrinsic Dimension Estimation with Diffusion Models.\" arXiv preprint arXiv:2406.03537 (2024).\n\n[3] Wang, Peng, Huijie Zhang, Zekai Zhang, Siyi Chen, Yi Ma, and Qing Qu. \"Diffusion models learn low-dimensional distributions via subspace clustering.\" arXiv preprint arXiv:2409.02426 (2024).\n\n[4] Chen, Siyi, Huijie Zhang, Minzhe Guo, Yifu Lu, Peng Wang, and Qing Qu. \"Exploring low-dimensional subspaces in diffusion models for controllable image editing.\" arXiv preprint arXiv:2409.02374 (2024).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)