What Secrets Do Your Manifolds Hold? Understanding the Local Geometry of Generative Models

## Abstract

Abstract Deep generative models learn continuous representations of complex data manifolds using a finite number of samples during training. For a pre-trained generative model, the common way to evaluate the quality of the manifold representation learned, is by computing global metrics like Fréchet Inception Distance using a large number of generated and real samples. However, generative model performance is not uniform across the learned manifold, e.g., for foundation models like Stable Diffusion generation performance can vary significantly based on the conditioning or initial noise vector being denoised. In this paper we study the relationship between the local geometry of the learned manifold and downstream generation.
Based on the theory of continuous piecewise-linear (CPWL) generators, we use three geometric descriptors
– scaling ( ψ 𝜓 \psi italic_ψ ), rank ( ν 𝜈 \nu italic_ν ), and complexity ( δ 𝛿 \delta italic_δ ) – to characterize a pre-trained generative model manifold locally.
We provide quantitative and qualitative evidence showing that for a given latent, the local descriptors are correlated with generation aesthetics, artifacts, uncertainty, and even memorization. Finally we demonstrate that training a reward model on the local geometry can allow controlling the likelihood of a generated sample under the learned distribution.

## 1 Introduction

In recent years, deep generative models have emerged as a powerful tool in machine learning, capable of synthesizing realistic data across diverse domains [, , ]. With the rapid increase in generation performance of such generative models, we have seen a demand for more fine-grained methods for evaluation. For example, global metrics of generation such as Fréchet Inception Distance (FID) are sensitive to both the fidelity and diversity of generated samples – precision and recall [] were proposed to disentangle these two factors and quantify them separately. With the advent of foundational generative models like Stable Diffusion [], human evaluation of images in a fine-grained manner has become the modus operandi for evaluation. In such cases, evaluating generative models extend beyond assessing mere sample fidelity and conditional alignment, to broader concerns about bias and memorization/duplication. The trend therefore shows a strong need for fine-grained understanding of the generative model behavior apart from what we can derive from global generation metrics. In this paper we look for answers to the following question:

##### Question.

For any sample generated using a pre-trained generative model, how is the local geometry of the learned manifold related to qualitative aspects of the sample?

We believe this question is especially significant 1) for models trained with large heterogeneous data distributions. Due to the heterogeneity, generation performance can vary significantly based on the sampling rate of the data manifold when training data is prepared. 2) For any pre-trained generative model without access to the training distribution, since computing global metrics like FID, precision or recall require the training data as ground truth.

##### How do we characterize the learned manifold?

To answer the question above, we require methods to derive local characteristics of a pre-trained generative model manifold for any given latent-sample pair $\{\bm{z},\bm{x}\}$.
A large class of generative models – comprised of convolutions, skip-connections, pooling, or any continuous piece-wise linear (CPWL) operation – fall under the umbrella of CPWL generative models. Any CPWL operator, can be characterized exactly in-terms of the 1) knots/regions in the input space partition of the CPWL operator, where every knot denotes the location of a second-order change in the function and 2) the region-wise linear/affine operation, which scales, rotates, and translates any input region while mapping it to the output []. We therefore propose using the three geometric descriptors to characterize the learned manifold of a CPWL generator:

- •
Local rank ($\nu$), that characterizes the local dimensionality of the learned manifold around $x$.
- •
Local scaling ($\psi$), that characterizes the local change of volume by the generative model for an infinitesimal volume around $z$.
- •
Local complexity ($\delta$), that approximates the smoothness of the generative model manifold in terms of second order changes in the input-output mapping.

Geometric descriptors such as local scaling, complexity or rank of Deep Neural Networks, have previously been used to measure function complexity [] and expressivity [, ], to evaluate the quality of representations learned with a self-supervised objective [], for interpretability and visualization of DNNs, [], to understand the learning dynamics in reinforcement learning [], to explain grokking, i.e., delayed generalization and robustness in classifiers [], increase fairness in generative models [], control sampling in generative models [], and maximum likelihood inference in the latent space [].

##### Our contributions.

In this paper, through
rigorous experiments on large text-to-image latent diffusion models and smaller generative models, we demonstrate the correlation between the local geometric descriptors with generation aesthetics, diversity, and memorization of examples. We provide insights into how these manifest for different sub-populations under the generated distribution. We also show that the geometry of the data manifold is heavily influenced by the training data which enables applications in out-of-distribution detection and reward modeling to control the output distribution. Our empirical results present the following major observations, which can also be considered novel contributions of this paper:

- •
C1. The local geometry on the generative model manifold is distinct from the off manifold geometry.
Local scaling and local rank are indicative whether a $\{z,x\}$ pair is out-of-distribution.
- •
C2. A strong relationship exists between the geometric descriptors and generation fidelity, aesthetics, diversity amd memorization in large text-to-image latent diffusion models.
- •
C3. By training a surrogate model on the local geometry of Stable Diffusion, we can perform reward guidance to increase/decrease sampling uncertainty.

The paper is organized as follows: In Sec. 2 we define the local descriptors from first principles of CPWL generators. In Sec.3 we first show that for a toy DDPM [] C1. holds. Following that we present visual and quantitative results showing C1. for Stable Diffusion. We also provide qualitative evidence for C2. for samples from the real Imagenet manifold. In Sec. 3.3.4 we draw connections between the local descriptors and 1) generation quality 2) uncertainty, and 3) memorization for Stable Diffusion. Finally in Sec. 4 we demonstrate how local geometry can be used as a reward signal to guide generation, providing evidence for C3.

## 2 Geometric Descriptors of the Learned Data Manifold

### 2.1 Continuous Piecewise-Linear Generative Models

Consider a generative network $\mathcal{G}$, which can be the decoder of a Variational Autoencoder (VAE) [], the generator of a Generative Adversarial Network (GAN) [] or an unrolled denoising diffusion implicit model (DDIM) []. Suppose, $\mathcal{G}:\mathbb{R}^{E}\rightarrow\mathbb{R}^{D}$ is a deep neural network with $L$ layers, input space dimensionality $E$ and output space dimensionality $D$. For any such generator, if the layers comprise affine operations such as convolutions, skip-connections, or max/avg-pooling, and the non-linearities are continuous piecewise-linear such as leaky-ReLU, ReLU, or periodic triangle, then the generator is a continuous piecewise-linear or piecewise-affine operator [, ]. This implies that the $\mathcal{G}:\mathbb{R}^{E}\rightarrow\mathbb{R}^{D}$ mapping can be expressed in terms of a subdivision of the input space into linear regions $\Omega$ and each region $\omega$ being mapped to the output data manifold via an affine opearation.
The continuous data manifold or image of the generator $\text{Im}(\mathcal{G})$ can be written as the union of sets:

$$ $\text{Im}(\mathcal{G})=\bigcup_{\forall\omega\in\Omega}\{\bm{A}_{\omega}z+\bm{ b}_{\omega}\forall z\in\omega\},$ (1) $$

where, $\Omega$ is the partition of the latent space ${\mathbb{R}}^{E}$ into continuous piecewise-linear regions, $\bm{A}_{\omega}$ and $\bm{b}_{\omega}$ are the slope and offset parameters of the affine mapping from latent space vectors $z\in\omega$ to the data manifold. For the class of continuous piecewise-linear (CPWL) neural network based generative models, $\Omega$, $\bm{A}_{\omega}$, and $\bm{b}_{\omega}$ are functions of the neurons/parameters of the network. For a generator with $L$ layers, $\bm{A}_{\omega}$ and $\bm{b}_{\omega}$ can be expressed in closed-form in terms of the weights and the region-wise activation pattern of neurons for each layer. We refer the readers to Lemma 1 of [] for details.

### 2.2 Local Geometry of Continuous Piecewise-Linear Generative Models

For any CPWL function, the input-output mapping can be expressed in terms of the input space subdivision $\Omega$ and the set of piecewise-affine parameters $\{\bm{A}_{\omega},\bm{b}_{\omega}\forall\omega\in\Omega\}$. In this section, we discuss how the local properties of the manifold can also be defined in terms of the regions $\omega$ or the region-wise affine parameters.

#### 2.2.1 Local complexity, δ 𝛿 \delta italic_δ

For CPWL functions, a general notion of complexity is based on the number of intervals or pieces of the function. For CPWL neural networks, a similar notion of complexity based on number of linear regions was proposed by []. We define local complexity of a CPWL generator as the following.

Definition 1. For a CPWL generator with input partition $\Omega$, the local complexity $\delta_{z}$ for a $P$-dimensional neighborhood of radius $r$ around latent vector $\bm{z}$ is

$$ $\displaystyle\delta_{\bm{z}}$ $\displaystyle=\sum_{\forall\omega\cap V_{\bm{z}}\neq\emptyset}\mathds{1}_{\omega}$ (2) $\displaystyle\text{where }V_{\bm{z}}$ $\displaystyle=\{\bm{x}\in{\mathbb{R}}^{E}:||\mathbf{B}(\bm{x}-\bm{z})||_{1}<r\}.$ (3) $$

Here, $\mathbf{B}$ is an orthonormal matrix of size $P\times E$ with $P\leq E$, $||.||_{1}$ is the $\ell_{1}$ norm operator and $r$ is a radius parameter denoting the size of the locality to compute $\delta$ for. The sum over regions $\omega\in V_{\bm{z}}$ requires computing $\Omega\cap V_{\bm{z}}$ which can be computationally intractable for high dimensions. A proxy for computing the partition for $V_{\bm{z}}$ with small $r$ is counting the number of non-linearities within $V_{\bm{z}}$, since for small $r$, the one can assume that the non-linearities do not fold inside $V_{\bm{z}}$, therefore providing an upper bound on the number of regions according to Zaslavsky’s Theorem [].
To compute local complexity, we use the method described in [] to estimate the number of knots intersecting $V_{\bm{z}}$.

Local complexity of CPWL neural networks have previously been connected to the expressivity of Deep Neural Networks [, ]. From a geometric standpoint local complexity tells us the smoothness of the learned function for a given locality. While local complexity is defined in terms of the neighborhood of a latent vector $\bm{z}$, in this paper we use it interchangeably with the local complexity of the data manifold around point $\bm{x}=\mathcal{G}(\bm{z})$.

#### 2.2.2 Local scaling, ψ 𝜓 \psi italic_ψ

##### Definition 2.

For a CPWL manifold produced by generator $\mathcal{G}$, the local scaling $\psi_{\omega}$ is constant for every region $\omega$ denoting logarithm of the scaling induced by the affine slope $\bm{A}_{\omega}$ for all vectors $\bm{z}\in\omega$. Local scaling for $\omega$ can therefore be expressed as

$$ $\displaystyle\psi_{\omega}$ $\displaystyle=\log(\sqrt{\det(\bm{A}_{\omega}^{T}\bm{A}_{\omega})})=\sum_{i}^{ k}\text{log}(\sigma_{i})\mathds{1}_{\{\sigma_{i}\neq 0\}},$ (4) $$

where, $\{\sigma_{i}\}_{i=0}^{i=k}$, are the non-zero singular values of $\bm{A}_{\omega}$.

Suppose $\mathcal{G}$ has a uniform latent distribution, meaning every region $\omega$ has a uniform probability density function at the input. Under an injectivity assumption for an input space region $\omega$ and $S=\{\bm{A}_{\omega}\bm{z}+\bm{b}_{\omega}\forall\bm{z}\in\omega\}$ the output density on $S$, $p_{S}(\bm{x})\propto\frac{1}{e^{\psi_{\omega}}}$. Therefore for an injective mapping, local scaling can be considered an un-normalized likelihood measure, where higher local scaling for any given $z\in\omega$ signals lower posterior density. For two regions $\omega\in\Omega$ and $\omega^{\prime}\in\Omega$, the difference in uncertainty can therefore be written as:

$$ $H_{\omega}-H_{\omega^{\prime}}=\psi_{\omega}-\psi_{\omega^{\prime}},$ (5) $$

where, $H_{\omega},H_{\omega^{\prime}}$ are the conditional entropy on the manifold for input space regions $\omega$ and $\omega^{\prime}$.

#### 2.2.3 Local rank, ν 𝜈 \nu italic_ν .

Definition 3. For a CPWL manifold produced by generator $\mathcal{G}$, local rank $\nu_{\omega}$ is the smooth rank of the the per-region affine slope $\bm{A}_{\omega}$ and can be expressed as:

$$ $\displaystyle\nu_{\omega}$ $\displaystyle=\exp\left(-\sum_{i}^{k}\alpha_{i}\log(\alpha_{i})\right)$ $\displaystyle\text{where }\alpha_{i}$ $\displaystyle=\frac{\sigma_{i}}{\sum_{i}^{k}\sigma_{i}}+\epsilon.$ $$

Here, $\{\sigma_{i}\}_{i=0}^{i=k}$ are non-zero singular values of $\bm{A}_{\omega}$ and $\epsilon=10^{-30}$ is a constant. The local rank $\nu_{\omega}$ therefore denotes the dimensionality of the tangent space on the data manifold.

Figure: Figure 3: Local geometric descriptors $\{\psi^{t},\delta^{t},\nu^{t}\}$ computed over the input domain of a 2D toy diffusion model producing distribution $z^{t}$ via reverse diffusion at noise level $t$, for an initial $z^{T}=\mathcal{N}(\mathbf{0},\mathbf{I})$. For any input vector, descriptors are computed by conditioning the diffusion model on $t$. We see that with decreasing $t$, local scaling $\psi$ and local rank $\nu^{t}$ decreases and local complexity $\delta^{t}$ increases in the proximity of the support of $z^{0}$, i.e., the learned manifold by the diffusion model. Therefore, at any $t$ during the reverse diffusion process, we can use the local descriptors computed for lower noise scales, for example $t=0.22T$, to guide generation towards or away from $z^{0}$.
Refer to caption: extracted/5787659/figures/tinydiffusion/exps_dino_save_itr2_model_s_124999_it_5_samples.jpg

### 2.3 Example: Toy generator trained on a f : ℝ 2 → ℝ 3 : 𝑓 → superscript ℝ 2 superscript ℝ 3 f:\mathbb{R}^{2}\rightarrow\mathbb{R}^{3} italic_f : blackboard_R start_POSTSUPERSCRIPT 2 end_POSTSUPERSCRIPT → blackboard_R start_POSTSUPERSCRIPT 3 end_POSTSUPERSCRIPT task

Setup.
We train a toy generator $\mathcal{G}:{\mathbb{R}}^{2}\rightarrow{\mathbb{R}}^{3}$ with depth 3 and width 20, to map the 2-dimensional latent domain $[-10,10]^{2}$ onto a toy 2-manifold in a 3-dimensional output space. We train the generator with a regression loss on a ground truth manifold defined as a mixture of five gaussian functions. In Fig. 2-left, we present analytically computed visualization [] of the piece-wise linear manifold learned by the generator as well as the latent space partition $\Omega$ represented by dark lines. Every black line represents a non-linearity of the function that folds/bends the latent space while going from ${\mathbb{R}}^{2}$ to ${\mathbb{R}}^{3}$. Therefore, the black lines are knots of the continuous piecewise affine spline generator. Each convex region $\omega$ formed by the intersection of the black lines, is mapped to $\text{Im}(\mathcal{G})$ via per region parameters as described in Equation 1. Each region in the input and output partition is colored by $\psi_{\omega}$. We also use a kernel density estimator (KDE) to estimate the density of generated samples (right) on the data manifold for a uniform latent distribution (middle-right), and color samples with the estimated density.

Observations. After training the generator, the pre-activation zero-level sets (Fig. 2) of neurons are positioned where changes to the slope of the manifold is required. The density of linear regions, i.e., local complexity is higher in the center and lower towards the edges. $\bm{A}_{\omega}$ is full-rank $\forall\omega\in\Omega$. Contrasting the KDE estimate of density and local scaling, we can see that for higher $\psi$ (blue hue) we have lower estimated density (blue hue) and vice-versa.

## 3 Exploring Generative Model Manifolds using Descriptors

In this Section we will be exploring the geometry of the data manifolds learned by various generative models, e.g., denoising diffusion probabilistic models, latent diffusion models like Stable Diffusion [].

### 3.1 DDPM trained on toy f : ℝ 2 → ℝ 2 : 𝑓 → superscript ℝ 2 superscript ℝ 2 f:\mathbb{R}^{2}\rightarrow\mathbb{R}^{2} italic_f : blackboard_R start_POSTSUPERSCRIPT 2 end_POSTSUPERSCRIPT → blackboard_R start_POSTSUPERSCRIPT 2 end_POSTSUPERSCRIPT generation task.

Setup. We train a denoising diffusion probabilistic model [] on a toy dataset(^1^11https://jumpingrivers.github.io/datasauRus) to visualize how the local geometry of the learned manifold varies with 1) de-noising steps/noise levels 2) gradient descent steps.
In Fig. 3 we present the local complexity $\delta^{t}$, local scaling $\psi^{t}$ and local rank $\nu^{t}$ at timesteps $t\in\{6,17,28,39,50\}$. We consider a grid of points in the 2D data space and compute the local descriptors at each point. Note that we drop the $\omega$ notation as we don’t compute $\Omega$ and no longer compute the scaling and rank in a region-wise fashion. To obtain $\bm{A}_{\omega}$ at any vector $z\in\omega$ where $\omega$ is unknown, we can take the input-output jacobian at $z$ (see Eq. 1). We add a superscript $t$ to denote the descriptors being computed for the learned output manifold at denoising step $t$. $\psi^{t}$ and $\nu^{t}$ therefore denotes the local scaling and rank by the mapping from $t$ to $t+1$.

Observations. We see that the local complexity is higher around the support of $z^{t}$ for all noise levels $t$ and both training steps shown in Fig. 3 and Fig. 19. Both local scaling and rank are lower around the data manifold until the last few diffusion timesteps, when the variance of $\psi$ and $\nu$ diminishes.
The qualitative results suggest that given two diffusion models (here one is an early checkpoint and the other is a late checkpoint), local geometry around the training data for the better trained model would have higher $\delta^{t}$, and lower $\psi^{t},\mu^{t}$.

### 3.2 VAE training dynamics for MNIST

Setup. We train a Variational Auto Encoder (VAE) on the MNIST dataset with width 128 and depth 5 for both encoder and decoder. We add Gaussian noise
with standard deviation $\{0,0.0001,0.001,0.01,0.1\}$ to the training data. Initialization was not kept fixed.
In Fig. 4, we present plots showing the training dynamics of local complexity and scaling, averaged over all test dataset points from MNIST.

Figure: Figure 4: Training dynamics of geometric descriptors for a VAE trained on MNIST with additive noise. As training progresses local complexity $\delta$ increases and local scaling $\psi$ decreases suggesting an increase in expressivity and decrease in uncertainty on the data manifold. At latter time-steps, $\psi\downarrow\text{ and }\delta\uparrow\text{if noise std. is increased}$.
Refer to caption: x1.png

Observations. By increasing the noise we control the puffiness of the target manifold.
We observe that as the noise standard deviation is increased there is 1) increase in $\delta$ indicating the manifold becomes less smooth 2) decrease in local scaling indicating that the uncertainty decreases. We can also observe an initial dip in both local complexity and local scaling. This is similar to what was observed for discriminative models in [] where a double descent behavior was reported in the local complexity training dynamics of classification models. Based on these results, contrary to the observation in [], generative models do not have a double descent in local complexity however we do observe a double ascent in local scaling. Our observations suggest that the training dynamics need to be taken into account, when comparing the local manifold geometry between two separately trained models.

### 3.3 Pre-trained Latent Diffusion Foundation Model

In this section we explore the local geometry of a pre-trained latent diffusion model namely Stable Diffusion v1.4 []. Stable Diffusion (SD) comprises of an unconditional vector quantized VAE (VQVAE) and a conditional diffusion model in the latent space of the VQVAE. The diffusion model is trained to map a gaussian distribution to the latent domain of the decoder, therefore we focus our experiments on studying the geometry of the unconditional decoder.

Figure: Figure 5: Geometry of the Stable Diffusion latent space. Geometric descriptors (left, middle-left, middle-right) visualized on a 2D latent space subspace, that passes through the latent representations of "a fox", "a cat" and "a dog" (right), denoted via markers on the 2D subspace descriptor. In Appendix, we provide denoised images for different high/low descriptor regions from the subspace. We see that in the convex hull of the three anchor latent vectors $\psi\uparrow$, $\nu\downarrow$ and $\delta\uparrow$. Moreover we see that in the convex hull, the local rank $\nu$ undergoes sharp changes which are not visible towards the edges of the domain.
Refer to caption: extracted/5787659/figures/dog_cat_fox_slice_clx.png

Figure: Figure 6: Local Scaling for encoded ImageNet images. ImageNet images ordered along the columns (from left to right), with increasing local scaling $\psi$ of the Stable Diffusion decoder learned manifold. We observe that ImageNet samples with lower values of $\psi$ contain simpler backgrounds with modal representation of the object category. Conversely for higher $\psi$ we have increasing diversity both in background and foreground features.
Refer to caption: extracted/5787659/figures/imagenet_sorted_scaling.png

#### 3.3.1 Visualizing the local geometry

##### Setup.

We use three prompts "a cat", "a dog" and "a fox" to generate three latent vectors using the SD diffusion model. We consider a 2D slice in the latent space, going through the three denoised latents, and compute the local descriptors on a $512\times 512$ uniformly spaced grid. We compute $\delta$ for a $4$-dimensional neighborhood of radius $r=0.00001$. To make $A_{\omega}\mathds{1}_{z\in\omega}$ computation tractable, for any latent vector $z$ we take a $120$-dimensional random orthonormal projection of the SD decoder manifold [] and compute the input-output jacobian with the projection as the output. We use a JAX implementation of Stable Diffusion on TPU resulting in $3.6s$ required for computing jointly $\delta$, $\psi$, and $\nu$ for one latent vector $z$.
In Fig. 5 we present the $\delta,\psi,\nu$, on the 2D slice, along with decoded versions of the three latent vectors, used to compute the 2D slice. In Appendix Figs. 25, 26, 21, 22, 23, and  24 we present generated images from the high or low local descriptor regions from the 2D slice.

##### Observations.

Similar to what was observed for a DDPM trained on toy data (see Sec. 3.1), we observe that 1) in the convex hull of the three denoised latents, we have higher complexity indicating its proximity to the SD decoder data manifold 2) lower rank in the convex hull affirms that the convex hull has proximity to the data manifold as well. However we see a sparse collection of latents for which the rank drops further 3) as we move inside the convex hull, local scaling increases. This indicates that a convex combination of "a fox", "a cat" and "a dog" is higher uncertainty w.r.t the SD decoder. However if we move away from the convex hull we see that both $\delta$ and $\psi$ decreases while rank increases. This indicates that as we move away from the data manifold, the local scaling descriptor collapses as such regions are no longer within the support/latent domain of the SD decoder. This is further illustrated through qualitative examples in Appendix Fig. 21 and Fig. 22.

#### 3.3.2 Local geometry of ImageNet

##### Setup.

Given the observations from the previous sections, we ask the question "What can local geometry of a foundation model say about a dataset?". To find an answer, we consider $20K$ samples from Imagenet with resolution higher or equal to $512\times 512$, encode the samples using the SD encoder, and study the local geometry of the SD decoder manifold for the encoded Imagenet latents.
In Fig. 6 we present samples from Imagenet ordered with increasing $\psi$. In Appendix Fig. 14 we present images where the manifold dimensionality is the lowest and highest for the Imagenet samples considered.

##### Observations.

In Fig. 6 each column represents a local scaling level set, with increasing values from left to right. Recall that in Eq. 5 we show that increase in local scaling is equivalent to increase in uncertainty. In this figure, we can see that for lower uncertainty images we have more modal features in the images, i.e., the samples have less background elements and are focused on the subject corresponding to the Imagenet class. For higher uncertainty images, we see that images have more outlier characteristics. For images with higher local in Fig. 14, we see that the backgrounds have higher frequency elements compared to lower rank images. For higher rank images, the dimensionality of the manifold is higher locally, therefore allowing more noise dimensions on the manifold. In Appendix. Fig. 15 we present class-wise examples for high and low local scaling images.

Figure: Figure 7: Geometry distinguishes out-of-distribution regions. We encode chest X-ray vs Imagenet images and present here local scaling (left) and local rank (right) distributions computed for the Stable Diffusion decoder. Here chest X-ray images are representative of out-of-distribution samples with respect to the Stable Diffusion learned distribution, and Imagenet images are considered in-distribution. We see that for OOD samples, the expected $\psi$ is higher, showing higher uncertainty under the learned distribution. The local rank is also higher for OOD samples, which correlates with our observations from Fig. 5. These results indicate that the local geometry can be used to characterize samples that are out-of-distribution with respect to a pre-trained generative model.
Refer to caption: x3.png

#### 3.3.3 Detecting Out-of-Distribution Samples

##### Setup.

To study if the geometric descriptors are discriminative of out-of-distribution samples, we consider evaluating the metrics for $20K$ randomly selected Imagenet samples representing in-domain images for Stable Diffusion and $20K$ X-Ray images from the CheXpert [] dataset representing out-of-domain images for Stable Diffusion. We present the distributions in Fig. 7.

##### Observations.

We see that X-ray images have higher uncertainty and higher local rank in expectation compared to Imagenet images. Especially for local rank $\nu$, Chest X-ray images from Chexpert and imagenet image distributions are almost separable. This corroborates with our visualizations from Fig. 5, where away from the convex hull of the denoised latents, we see an increase in $\nu$.

#### 3.3.4 Relationship with Quality, Diversity & Memorization

Quality and diversity of generation for a pre-trained model, is directly connected to the estimation of the target manifold. In this section, we study the relationship between the geometric descriptors and generation quality (good estimation of the manifold), generation diversity (large support of the learned distribution) and memorization (interpolation of the target manifold at a data point).

##### Setup.

The experiments are organized as such:
$\bullet$ We denoise a fixed set of latent vectors using a) varying guidance scale $\{0,1,3,5,7.5,9\}$ to control quality-diversity trade-off or, b) with memorized/non-memorized prompts by Stable Diffusion. We obtain memorized prompts for SD from [] and non-memorized prompts from the COCO dataset validation set.
$\bullet$ For a) $50K$ real ImageNet images b) $50K$ synthetic images generated by SD from increasing descriptor level sets we compute the Vendi Score [] of clip embeddings, the aesthetic score predicted by an aesthetic reward model and a artifact reward model. For the synthetic Imagenet images, we use a guidance scale of $7.5$ and the prompt template "an image of $p$" where $p$ is a randomly class from Imagenet.

##### Observations.

We see in Fig. 8-bottom that for higher classifier free guidance scales during denoising, uncertainity is lower especially during the final denoising steps. This shows a clear correlation with quality since higher guidance scales result in higher quality images []. We also see that for higher guidance scales local rank $\nu$ is $\downarrow$ and local complexity $\delta$ is $\downarrow$ as well. For memorized prompts, in Fig. 8-top, we see that the $\psi$ is significantly lower than non-memorized prompts, indicating very low uncertainity in memorized samples compared to non-memorized samples. We also observe the manifold to be higher dimensional for memorized prompts and smoother.

Figure: Figure 8: Local geometry of denoising trajectories. Geometric descriptors computed for the VQGAN decoder, during $50$ stable diffusion denoising steps, for (top) 100 COCO and 100 memorized prompts [] with guidance scale $7.5$ and (bottom) 100 COCO prompts with varying guidance scales. For each prompt or guidance scale, we start from the same seeds. Shaded region represents 95% confidence interval. We see that the local geometry trajectories are discriminative of memorization, as well as increased quality when stronger classifier free guidance is used.
Refer to caption: x5.png

Figure: Figure 9: Local Geometry level sets for Imagenet images. Vendi scores, Aesthetic scores and Artifact scores computed for 50k generated and real samples of ImageNet from local scaling (top) and local rank (bottom) descriptor level sets of Stable Diffusion. Scores are computed for each level set separately. Synthetic samples are generated using a guidance scale of $7.5$. We see that diversity increases for higher scaling and rank level sets for both real and synthetic data. But the aesthetic/artifact score remains more or less constant for synthetic data, since a high guidance scale is used. This shows that by using local geometry to guide generation, we can increase the diversity of generation while maintaining generation aesthetics.
Refer to caption: extracted/5787659/figures/imagenet_real_generated_score.png

In Fig.9, we see that with increasing $\psi$ and $\nu$ diversity increases, however for very high $\psi$, i.e., very uncertain images, the diversity collapses. This is due to the higher $\psi$ level sets converging towards the anti-modes of the learned distribution. We see that for real images, aesthetic and artifact scores (higher if without artifacts) get reduced for higher local scaling and rank level sets. However for generated images for higher uncertainty the rewards saturate. This is an indication that even with less perceptual/aesthetic changes, we can generate higher uncertainty images.

## 4 Guiding Generation With Geometry

Reward Modeling.
In the previous chapters, we established that geometric descriptors are correlated with out-of-distribution detection, memorization, and aesthetics. Our primary focus has been on analyzing the geometry of the learned data manifold. In this section, we demonstrate that these geometric descriptors can be utilized to intervene in the latent representations of generative models, resulting in meaningful effects on the generated examples. Leveraging recent instance-level guidance methods, such as those described in the Universal Guidance paper [], we can effectively influence the latents to produce significant alterations in the generated outputs. To guide the generation process, we employ local scaling as our geometric descriptor. We use classifier guidance to perform gradient ascent with step size $\rho$ on gradients derived from local scaling. This method allows us to steer the generative process by maximizing the local scaling descriptor, thus altering the generated examples in a controlled manner.

Figure: Figure 10: Reward guidance on stable diffusion (maximizing the reward). Local scale reward guidance increases from left to right in each picture, with the first image showing no reward guidance. We observe maximizing the reward leads to sharper details, improved sharpness and contrast, and higher diversity in the images. Prompts used for generation from top to bottom: “a Rhodesian ridgeback with a city in the background”, “a Beagle in the snow”, “a Australian terrier on a cobblestone street”
Refer to caption: extracted/5787659/figures/reward_guidance/positive_guidance/image_2_19.png

Figure: Figure 11: Reward guidance on stable diffusion (minimizing the reward). Local scale reward guidance decreases from left to right in each picture, with the first image showing no reward guidance. We observe that decreasing the local scale leads to minimized uncertainty, resulting in a noticeable blurring effect and loss of details in the image. Prompts used for generation from top to bottom:“a Beagle with a city in the background”, “a Samoyed with a mountain in the background”, “a Australian terrier on a cobblestone street”.
Refer to caption: extracted/5787659/figures/reward_guidance/negative_guidance/image_2_29.png

To enable universal guidance, we need to compute the gradient of the reward function. In our context, this entails calculating the Hessian matrix, which is computationally expensive. Instead, we train a reward model to approximate local scaling, given noisy latents at time step $t$ of the diffusion chain. We transform the regression task of estimating a continuous local scaling value into a classification task by first determining the maximum and minimum range of local scaling values for the ImageNet dataset. We then discretize this range into five uniform bins.
Our training data collection process is as follows: For each image in the ImageNet dataset, we sample 10 time steps, apply noise to the encoded latents according to the sampled time step, and compute the corresponding local scaling. Subsequently, we train a classification model on pairs of latents and their associated local scaling bins.

Our experiments reveal that maximizing local scaling in the manifold of a stable diffusion model directly correlates with adding texture to the generated images. Moreover, this approach enhances diversity based on single images. By optimizing the local scaling descriptor, the generative model is guided towards producing more varied and textured outputs.
This approach is notable because traditional methods for diversity guidance generally function at the distribution level. Our method, however, focuses on maximizing the inherent diversity as preserved by the model within its learned manifold, effectively steering the generated images towards the extremities of the distribution. This instance-level intervention allows for a more detailed and precise enhancement of diversity, presenting a novel approach to guiding generative models.

As seen from Fig. 10 maximizing the reward results in added details in form of sharpening the image, adding texture and contrast. We also observe that if we move towards minimizing the reward, the images tend to loose fine-grained details as seen in Fig. 20.

## 5 Conclusion & Future Directions

In this paper, we proposed a novel self-assessment approach to evaluate generative models using geometry-based descriptors – local scaling ($\psi$), local rank ($\nu$) and local complexity ($\delta$) - effectively while utilizing only the model’s architecture and weights. Our approach characterizes uncertainty, dimensionality, and smoothness of the learned manifold without requiring original training data or human evaluators.
Our experiments demonstrated how these descriptors relate to generation quality, aesthetics, diversity, and biases. We showed that the geometry of the data manifold impacts out-of-distribution detection, model comparison, and reward modeling, enabling better control of output distribution.
While using the geometry of manifolds offers a novel approach to self-assess generative models, we acknowledge two main limitations that warrant further investigation. First, the geometry of the learned manifold is inherently influenced by the training dynamics of the model. A deeper understanding of this relationship is needed to fully leverage geometric analysis for model assessment and improvement. Second, the computational complexity of our method, particularly the calculation of the Jacobian matrix, poses a practical challenge, especially for large-scale models. Future work should explore more efficient algorithms or approximations to address this limitation.

## Appendix A Appendix / Supplemental material

## Appendix B Broader Impact Statement

Our proposed framework for assessing and guiding generative models through manifold geometry offers several potential benefits to society. By providing a more objective and automated approach, we can significantly reduce the cost and time associated with human evaluation, making the auditing and mitigation of biases in large-scale models more accessible and efficient. This has implications for promoting fairness and equity in AI systems, particularly in domains where biases can have significant societal consequences.

Furthermore, our approach can empower researchers and practitioners to better understand the relationship between the geometry of learned representations and various aspects of model behavior, such as generation quality, diversity, and bias. This deeper understanding can inform the development of more robust and reliable generative models, leading to advancements in various fields, including art, design, healthcare, and education.

However, we recognize that our approach is not without limitations and potential risks. While it can be a valuable tool for identifying and mitigating biases, it should not and cannot fully replace human annotators, especially in high-risk domains where human judgment and contextual understanding are crucial. Our method focuses on reducing costs and improving the auditing process, but it should not be used as a standalone approach.

Moreover, the increased automation enabled by our approach raises concerns about the potential displacement of human annotators, leading to job losses and economic disruptions. While our method addresses some aspects of model evaluation, it is not comprehensive and cannot assess all facets of model behavior. Therefore, it should be used with caution and in conjunction with other evaluation methods, including human expertise.

## Appendix C Extra Figures

Figure: Figure 12: Images generated during 50 diffusion denoising steps for top to bottom, COCO prompts generated with guidance scale 1,5,9 and memorized prompts generated with guidance scale 7.5. Higher guidance scale images, as well as memorized images, tend to resolve faster during the denoising process.
Refer to caption: extracted/5787659/figures/trajectories/coco_guidance_scale_1.jpg

Figure: Figure 13: Sub-population differences of local descriptors in generated data. Separation between the $\psi$ distributions for $1000$ images generated for each prompt in a photo of a {rich, poor} swiss person (left) and a photo of a {bartender, crane operator} (right). We see that the uncertainty as expressed by $\psi$ can vary between sub-populations indicating an underlying bias in generation diversity.
Refer to caption: x11.png

Figure: Figure 14: Images with the lowest (left) and highest (right) local rank $\nu$ from a set of $20000$ randomly sampled ImageNet dataset samples. Low rank images contain simpler textures for every class compared to the high rank samples. This is because for images with higher local rank, the learned manifold is higher dimensional therefore allowing higher independent degrees of variations locally for the generated images.
Refer to caption: extracted/5787659/figures/low_rank_imagenet-min.png

Figure: Figure 15: Imagenet images with high and low local scaling for the stable diffusion decoder. Each coordinate in both left and right image grids, correspond to the same imagenet class.
Refer to caption: extracted/5787659/figures/classwise_imagenet_lowscaling-min.png

Figure: Figure 16: Bivariate $(\epsilon,\psi)$ and $(\epsilon,\nu)$ distributions for training samples in green (top), and $(\delta,\psi)$, $(\nu,\psi)$ distributions for training in green and generated samples in red (bottom). Local complexity $\delta$ exhibits linear correlation with $\epsilon$ and $\psi$. Lowest $\epsilon$ samples also have lowest $\delta$, indicating that good reconstruction requires linear interpolation of the data manifold. Local rank $\nu$ uncorrelated with $\epsilon$. Both $\delta$ and $\nu$ exhibit some linear relationship with $\psi$. Note that the modes of real and generated samples for the $(\psi,\delta)$ and $(\psi,\nu)$ distributions are not completely aligned.
Refer to caption: x13.png

Figure: Figure 17: Local scaling dynamics while fine-tuning a latent diffusion model on Chest-Xray images. The local scaling is computed for generated chest-xray images, where we can see a reduction of local scaling, i.e., uncertainty.
Refer to caption: extracted/5787659/figures/dreambooth_demo.png

Figure: Figure 18: Geometric descriptors of the data manifold. Level sets of data manifold descriptors for a Beta-VAE trained unconditionally on MNIST. From left to right, we present training samples (top row) and generated samples (bottom row) for linearly increasing level sets of local scaling ($\psi$) from $[-80,-42]$, local complexity ($\delta$) from $[0,120]$ and local rank ($\nu$) from $[1.5,5.5]$. Not all level sets had an equal number of samples from training/generated distributions. We see that for higher $\psi$, we have more outlier samples whereas for lower $\psi$ we have modal samples. For increasing $\delta$ we see that the quality of generated samples decreases and the diversity of samples is reduced as well. For higher $\nu$ digits become more regularly shaped.

Figure: Figure 19: After 11395 optimization steps. Geometry of a diffusion model input-output mapping, trained to on a toy 2D distribution. Local scaling lower around data manifold, local complexity higher around manifold, rank is lower around manifold as well. t=50 has considerably low variance in local scaling showing that final timestep has a diminishing change of density.
Refer to caption: extracted/5787659/figures/tinydiffusion/exps_dino_save_itr2_model_s_11395_it_5_samples.jpg

Figure: Figure 20: Reward guidance on stable diffusion (maximizing the reward).
Refer to caption: extracted/5787659/figures/reward_guidance/positive_guidance/image_1_0.png

Figure: Figure 21: Decoded images (right) using $20$ latents (left) from the 2D subspace, with highest $\psi$. Each image bounding box (right) is color coded according to the corresponding latent vector (left).
Refer to caption: extracted/5787659/figures/2d_slice/highscaling_slice.png

Figure: Figure 22: Decoded images (right) using $20$ latents (left) from the 2D subspace, with lowest $\psi$. Each image bounding box (right) is color coded according to the corresponding latent vector (left). Selected latents lie outside the domain of the VQGAN latent space.
Refer to caption: extracted/5787659/figures/2d_slice/lowscaling_slice.png

Figure: Figure 23: Decoded images (right) using $20$ latents (left) from the 2D subspace, with highest $\nu$. Each image bounding box (right) is color coded according to the corresponding latent vector (left). Selected latents lie outside the domain of the VQGAN latent space.
Refer to caption: extracted/5787659/figures/2d_slice/highrank_slice.png

Figure: Figure 24: Decoded images (right) using $20$ latents (left) from the 2D subspace, with lowest $\nu$. Each image bounding box (right) is color coded according to the corresponding latent vector (left).
Refer to caption: extracted/5787659/figures/2d_slice/lowrank_slice.png

Figure: Figure 25: Decoded images (right) using $20$ latents (left) from the 2D subspace, with highest $\delta$. Each image bounding box (right) is color coded according to the corresponding latent vector (left).
Refer to caption: extracted/5787659/figures/2d_slice/highclx_slice.png

Figure: Figure 26: Decoded images (right) using $20$ latents (left) from the 2D subspace, with lowest $\delta$. Each image bounding box (right) is color coded according to the corresponding latent vector (left).
Refer to caption: extracted/5787659/figures/2d_slice/lowclx_slice.png

Figure: Figure 27: Vendi score [] calculated for samples from different local descriptor level sets of a Beta-VAE. We take upto $150$ samples from each level set and compute vendi score seperately for the MNIST train dataset, test dataset and generated samples.
Refer to caption: x20.png

Figure: Figure 28: Sub-population differences of local descriptors in training data. Class-wise local descriptor distributions for a Beta-VAE trained unconditionally on MNIST. We use the VAE encoder to obtain latent vectors for the training dataset and compute descriptors using the VAE decoder. We can observe class specific separation in the distributions indicating that geometric properties may be different for different sub-populations within the training dataset.
Refer to caption: x23.png

## Peer Reviews

confidence: 3
contribution: 3
initial_rating: 6
presentation: 2
questions: **1.** I am curious about the computational costs such as GPU hours per 1k samples to calculate each descriptor value, as well as the data preparation of reward models. I think this would give readers a more comprehensive understanding of the empirical impact of the proposed approach.

**2.** I think it would be good if authors could provide more studies on DiT or MM-DiT-based text-to-image diffusion models. The UNet-based Stable Diffusion 1.4 model is kind of weak when compared with current strong DiT-based models such as Stable Diffusion 3 or Flux. This study will give readers an in-depth understanding of how compatible the approach is across different neural network architectures. 

**3.** It would be good if authors could provide more intuitive discussions on each of the three descriptors. This will help readers understand the intuitions behind them. 

**4.** The author should provide quantitative metrics such as FID or human preference scores when using reward guidance sampling.
soundness: 3
strengths: **1. Interesting Indicators of Geometry**: This paper introduces an interesting use of geometric descriptors to understand and control generative models, providing new insights into latent space structure.

**2. Comprehensive Experiments**: The paper includes extensive experiments across various models, from toy models to Stable Diffusion.

**3. Practical Framework for Control**: The reward model offers a practical way to influence generation characteristics, such as diversity and aesthetic quality, by guiding sampling in the latent space based on geometry based on the trained scalar model of the geometry descriptor.
summary: This paper, "What Secrets Do Your Manifolds Hold? Understanding the Local Geometry of Generative Models", explores the role of local geometry in deep generative models and its effect on generation quality, diversity, and memorization. The authors propose three geometric descriptors—local scaling ($\psi$), rank ($v$), and complexity ($\delta$)—to characterize the latent space of generative models, focusing on models with Continuous Piecewise-Linear (CPWL) mappings. They empirically demonstrate that these descriptors are correlated with various aspects of generative performance and propose a reward model trained on these descriptors to guide sample generation in diffusion models, particularly Stable Diffusion.
weaknesses: **1. Computational Demands**: The method relies on calculating Jacobians on each linear piece of model manifold. This may be computationally intensive for large models, limiting its practical application. Such a limitation potentially indicates the approach is less scalable. Besides, since the computation of each geometric descriptor is computationally demanding, generating data for training latent space reward models is also computationally demanding and therefore possibly not scalable. 

**2. Descriptor Interpretation**: Some descriptors, particularly local complexity ($\delta$), lack intuitive interpretation when applied to high-dimensional latent spaces, which could be further clarified. This makes the paper more like a pure empirical computation of each descriptor instead of an in-depth study of the latent geometry of generative models.  

**3. Weak Evaluations**: A major drawback of this paper is the weak lack of quantitative evaluations. For instance, the paper should provide quantitative metrics such as FIDs or Human preference scores to evaluate the performances of the proposed geometry descriptor guided sampling instead of merely qualitative plots.

**4. Poor Compatibility for Networks with Smooth Activation Functions**: If I do not misunderstand, I think the approach is only properly defined for models with piece-wise linear neural networks instead of those networks using smooth activation functions such as SiLU, GELU or SwiGLU. This might limit the broader usage of the proposed approach and the impacts of the study.

---

confidence: 4
contribution: 3
initial_rating: 5
presentation: 2
questions: (a)  Where does equation (3) come from?

(b) In equation (7), how is B defined? (i. How to choose number of dimensions P, is it the same aound any latent $z$? ii. How to choose columns in B? iii. Why a constant projection B can be used for all neighbor around the specific $z$?)

(c) What explicitly are the input space vectors for DDPM and Stable Diffusions? Are they the inputs to the denoising UNet, or could they be features in the middle layers of the UNet?

(d) In section 3.1 L250, "input vectors within 0.05 units of the training data", how is 0.05 chosen and why it is a reasonable value?

(e) In Figure 4, how are the three anchors (representing latent from 3 text prompts) labeled in the latent space and what are other points?
soundness: 2
strengths: 1. The paper is studying the local geometry from a comprehensive perspective, modeling scaling (ψ), rank (ν), and smoothness (δ).

2. The paper shows experiment results from several perspectives including off and on manifold, content complexity, image corruption, and text prompt memorization.

3. The paper utilizes local scaling to control the generated image complexity.
summary: This paper aims to study the local geometry of pretrained diffusion models and explore how different local geometry indicates different properties. It proposes three descriptors scaling (ψ), rank (ν), and smoothness (δ) to characterize the local geometry in diffusion models based on the continuous piecewise-linear (CPWL) assumption. The paper uses these descriptors to study the local geometry in classical DDPM and stable diffusion models. The paper conducts various experiments to show the following statements: 1. Local geometry of off manifold is different from on manifold. 2. Increasing local scaling leads to the generation of more complex image content. 3. Local geometry of Stable Diffusion is sensitive to image corruptions. 4. Local geometry of Stable Diffusion is sensitive to memorization of text prompts. Lastly, the paper proposes to guide generation with gradients of the descriptors (specifically local scaling). To avoid heavy computation of Hessian, they actually train a reward model to approximate local scaling. With this geometric guidance, the paper can control content complexity in the generated images.
weaknesses: 1. Lack of literature review. There exists prior work [1] related to out-of-domain detection with local geometry, and [2] related to uncertainty quantification with local geometry. Even though the metrics are not exactly the same, it is worth discussing these previous works, as well as other ones related to the manifold of diffusion models.

2. Lack of clarity and technical details. See the questions part.

3. Lack of Justification of the continuous piecewise-linear (CPWL) assumption: The paper does not provide justification that CPWL can be a good approximation for DDPM and stable diffusion. In contrast, recent work [3] has experimentally verified such local linearity.

4. Redundancy of metrics: Though all three metrics are interesting, it seems rank (ν), and smoothness (δ) are redundant since most observations and applications can be correlated only with scaling.

[1] A Geometric Explanation of the Likelihood OOD Detection Paradox. Hamidreza Kamkari et al.

[2] On the Posterior Distribution in Denoising: Application to Uncertainty Quantification. Hila Manor et al.

[3] Exploring Low-Dimensional Subspaces in Diffusion Models for Controllable Image Editing. Siyi Chen et al.

---

confidence: 3
contribution: 3
initial_rating: 6
presentation: 2
questions: 1. Eqn.2: Should the singular values here be normalized?

2. Line 164: What's the definition of $S$, why the local scaling is proportional to the NLL of the generative model when the relationship between $\psi$ and $S$ is injective?

3. Lines 262-269: Where are the corresponding results and visualizations? Why do results at 0.17T show the minimized discrepancy between the on-manifold and off-manifold input space? And why does the minimized discrepancy indicate ``local geometry indicators have the highest distinction geometrically between on and off manifold vectors from the input space'' (should it be the opposite)?

4. Figure 3: What does $[-6, 6]^2$ mean? What's the x-axis of the plot on the right, is it the noise scale?

5. Figure 3 and Figure 4: What are the axes of the local generator? Does each pixel correspond to one input image? How is this being calculated?

6. Figure 4: What images correspond to the pixels in the center of the geometric desciptors?

7. Section 5: How is the reward function defined? Why the calculation of the Hessian matrix is required in the first place? The paper describes three local geometric descriptors, but why does the reward model only approximate the local scaling value?
soundness: 3
strengths: The idea is novel and the authors conduct extensive experiments in multiple scenarios. It is very interesting to understand the geometry of the model and its role in the generation process.
summary: This paper studies the local geometry of the generative model and its relationship to the quality of the generation. Built upon the theory of continuous piecewise-linear generators, the authors propose to use three metrics: (1) local rank; (2) local scaling; and (3) local complexity to quantify the geometry of the generative model. Leveraging these descriptors, they try to understand how the local geometry relates to the generation process. Specifically, they show that local scaling is related to the complexity of the generated scenes, and can be used as a tool to detect whether the data is on the input (training) manifold. In addition, they propose to train a reward model to approximate the local scaling descriptor and present results showing more objects and complex backgrounds while increasing the value of the local scaling.
weaknesses: 1. My biggest concern regarding this paper is the approachability to the audience. The three local geometric metrics discussed in the paper are very interesting. However, in terms of the method presentation, particularly the reward model built upon the local scaling descriptor, the current presentation lacks detailed descriptions, which makes me concerned about the reproducibility of the results.

2. The claim that "local geometry trajectories are discriminative of memorization" is relatively weak for the local complexity metric when the trend of the metrics is almost un-distinguishable when increasing the guidance scale.


Minors:
Line 323: I think it should be capitalized DDPM instead of "ddpm".

---

confidence: 3
contribution: 4
initial_rating: 6
presentation: 3
questions: 1. In line 115, the paper discusses unrolling the sampling process of the diffusion model. Does this mean that when calculating the descriptor at a large timestep $t$, you need to compute the Jacobian iteratively over the neural network multiple times because the sampling steps are large? From my experiments, computing the Jacobian over the UNet in Stable Diffusion takes hours, even without unrolling. What is the computational cost of your implementation?

2. Are these three proposed descriptors redundant? From Figures 5, 12, and 13, it appears that as local scaling, local rank, and local complexity increase, the visual complexity of the images all increases. Seems we only need one descriptor for the image visual complexity. What specific, distinct properties do each of these three descriptors capture?
soundness: 3
strengths: This paper represents a valuable empirical contribution to understanding the local geometric properties of the image manifold learned by diffusion models. Based on the hypothesis that the output space of neural networks exhibits a continuous piecewise-linear structure, the learned data manifold can be approximated as a union of affine transformations across local regions. This foundation makes the proposed local geometric descriptors both intuitive and well-motivated. In empirical experiments, these descriptors align closely with human visual evaluation, underscoring their effectiveness. I believe this paper will significantly impact the study of image manifold geometry.
summary: Building on the continuous piecewise-linear hypothesis, this paper introduces three local geometric descriptors for generative models at specific stages of generation: scaling, rank, and smoothness. Using these descriptors, the paper empirically examines the local geometry of both a toy dataset and Stable Diffusion, drawing several conclusions about the learned image manifold. Finally, leveraging these descriptors, the paper proposes a novel reward model that guides the generation in Stable Diffusion, allowing control over specific local geometric properties.
weaknesses: 1. This paper lacks discussion with related works [1, 2, 3, 4]. The local dimensionality of the diffusion model-based image manifold has also been studied in those papers. However, as concurrent work, this won't significantly weaken the contribution of this paper. Better discussion about the relation and differences with those works could highlight the contribution of this paper.
2. The proposed method is not very strong. The authors provide only qualitative results without any quantitative evaluation or comparison to related works. 
3. Some parts of the writing are unclear. I suggest including the numerical method used to approximate the calculation of local complexity (Definition 3) in this paper. Although this method was introduced in previous work, including it here would make the paper more complete and easier to understand.






[1] Stanczuk, Jan Pawel, Georgios Batzolis, Teo Deveney, and Carola-Bibiane Schönlieb. "Diffusion Models Encode the Intrinsic Dimension of Data Manifolds." In Forty-first International Conference on Machine Learning.

[2] Kamkari, Hamidreza, Brendan Leigh Ross, Rasa Hosseinzadeh, Jesse C. Cresswell, and Gabriel Loaiza-Ganem. "A Geometric View of Data Complexity: Efficient Local Intrinsic Dimension Estimation with Diffusion Models." arXiv preprint arXiv:2406.03537 (2024).

[3] Wang, Peng, Huijie Zhang, Zekai Zhang, Siyi Chen, Yi Ma, and Qing Qu. "Diffusion models learn low-dimensional distributions via subspace clustering." arXiv preprint arXiv:2409.02426 (2024).

[4] Chen, Siyi, Huijie Zhang, Minzhe Guo, Yifu Lu, Peng Wang, and Qing Qu. "Exploring low-dimensional subspaces in diffusion models for controllable image editing." arXiv preprint arXiv:2409.02374 (2024).

Meta-review: This paper explores the local geometry of generative models and its impact on generation quality. Using the theory of continuous piecewise-linear generators, the authors introduce three metrics: (1) local rank, (2) local scaling, and (3) local complexity. They demonstrate that local scaling correlates with scene complexity and can detect whether data lies on the input (training) manifold. Additionally, they train a reward model to approximate local scaling, showing that increasing its value leads to more objects and complex backgrounds in generated outputs.

All reviewers agree that this work is interesting and above the acceptance bar.

Decision: Accept (Poster)