An analytic theory of creativity in convolutional diffusion models
Authors: Mason Kamb
Surya Ganguli
Venue: ICML 2025 oral (2025)
Keywords: Diffusion models
Creativity
Inductive Biases
Theory
Interpretability
Primary area: deep_learning->theory
TL;DR: We obtain an end-to-end analytic, interpretable and predictive theory of creativity in convolutional diffusion model by solving the optimal score-matching problem under the conditions of locality and equivariance.

Abstract: We obtain an analytic, interpretable and predictive theory of creativity in convolutional diffusion models. Indeed, score-matching diffusion models can generate highly original images that lie far from their training data.  However, optimal score-matching theory suggests that these models should only be able to produce memorized training examples. To reconcile this theory-experiment gap, we identify two simple inductive biases, locality and equivariance, that: (1) induce a form of combinatorial creativity by preventing optimal score-matching; (2) result in fully analytic, completely mechanistically interpretable, local score (LS) and equivariant local score (ELS) machines that, (3) after calibrating a single time-dependent hyperparameter can quantitatively predict the outputs of trained convolution only diffusion models (like ResNets and UNets) with high accuracy (median $r^2$ of $0.95, 0.94, 0.94, 0.96$ for our top model on CIFAR10, FashionMNIST, MNIST, and CelebA). Our model reveals a {\it locally consistent patch mosaic} mechanism of creativity, in which diffusion models create exponentially many novel images by mixing and matching different local training set patches at different scales and image locations. Our theory also partially predicts the outputs of pre-trained self-attention enabled UNets (median $r^2 \sim 0.77$ on CIFAR10), revealing an intriguing role for attention in carving out semantic coherence from local patch mosaics.

An analytic theory of creativity in convolutional diffusion models
Mason Kamb Surya Ganguli
Abstract
Theory
CNN
Theory
CNN
Theory
CNN
Theory
CNN
G
D
We obtain an analytic, interpretable and predic- tive theory of creativity in convolutional diffusion models. Indeed, score-matching diffusion mod- els can generate highly original images that lie far from their training data. However, optimal score-matching theory suggests that these models should only able to produce memorized train- ing examples. To reconeile this gap, we identify two simple inductive biases, lo- cality and equivariance, that: (1) induce form of combinatorial creativity by preventing optimal score- soore-matching:: (2) result in fully analytic, com- pletely mechanistically interpretable, local score (LS) and equivariant local score (ELS) machines that, (3) after calibrating a single time-dependent hyperparameter can quantitatively predict the out- puts of trained convolution only diffusion mod- els (like ResNets and UNets) with high accuracy (median r2 of 0.95, 0.94, 0.94, 0.96 for our top model on I0,AR10, FashionMNIST. MNIST, and CelebA), Our model reveals a locally consistent patch mosaic mechanism of creativity, which diffusion models create exponentially many novel images by mixing and matching different local training set patches at different scales and image locations. Our theory also partially predicts the outputs of pre-trained self-attention enabled UN- ets (median 2 ~ 0.77 on CIFARIO) 10), revealing an intriguing role for attention carving out se- mantic coherence from local patch mosaics.
2
Figure 1. Our analytic theory (left columns) can accurately predict case by case basis the outputs of convolutional diffusion models (right columns), with UNet or ResNet architectures trained MNIST, CIFARIO, FashionMNIST. and CelebA (left to right), even w when these outputs are highly original and far from the train- ing data. See Fig. App. Fig. 10 and Table 2, and App. D, Fig. 13 to Fig. 24 for many more successful eheory-experiment comparisons.
nonetheless clearly original, exhibiting novel combinations of attributes that are represented across disparate training examples. What is the nature and origin of this creativity, and how precisely is it generated from a finite training set? We answer these questions for small convolutional diffusion models of images by deriving an analytic and interpretable theory of their behavior that can accurately predict their outputs case-by-case basis (Fig. 1), and explain how they are created out of locally consistent patch mosaics of the training data.
1. Introduction and related work
A deep puzzle of generative AI lies in understanding how it produces apparently creative output: outputs that pos- sess a meaningful relationship to their training data, but are
Denoising probabilistic diffusion models (DDPMs) were established in Sohl-Dicke Dickstein al. (2015) and Ho al. (2020), and then unified with score-matching (Song & Er- mon, 2019; Song et al., 2020b). Denoising diffusion implicit models (DDIMs), alternative deterministic parameteriza- tion which we primarily use in this paper, were established in Song et al. (2020a). Diffusion models now play an impor- tant role not only in image generation (Dhariwal & Nichol, 2021; Rombach et al., 2022; Ramesh et al., 2022), but also video generation (Ho et al., 2022a;b; Blattmann al., 2023).
'Department of Applied Physies. Stanford University, California, United States. Correspondence Mason Kamb <kambm stanfordd.edu>. Surya Ganguli <sgan- guli@ siii-stanfrd.cdd....
Proceedings of the 42nd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

An analytic theory of creativity in convolutional diffusion models
drug design (Alakhdar et al., 2024), protein folding (Watson et al., 2023), and text generation (Li et al., 2023; 2022).
focus on diffusion models with oolyycoonvlltional back- bone, without the self-attention layers introduced in Ho et al. (2020). We identify two fundamental inductive biases that prevent such models from learning the ideal score-function translational equivariance, due to parameter sharing in con- volutional layers, and locality, due to the model's finite receptive field size. Remarkably, we show these two simple biases are sufficient to quantitatively explain the creative outputs of convolutional diffusion models, after calibrating single time-dependent hyperparameter (the locality scale),
These models are trained to reverse a forward diffusion pro- cess that turns the finite training set distribution (a sum of 5-functions over the training points) into an isotropic Gaus- sian noise distribution, through a time-dependent family of mixtures of Gaussians centered shrinking data points. Dif- fusion models are trained reverse this process by learning and following score function that points in gradient direc- tions of increasing probability. But therein lies the puzzle of creativity in diffusion models: if the network can learn this ideal score function exactly, then they will implement a perfect reversal of the forward process. This, in turn, will only be able to turn Gaussian noise into memorized training examples. Thus, any originality in the outputs of diffusion models must lie in their failure to achieve the very objec- tive they are trained on: learning the ideal score function. But how can they fail in intelligent ways that lead to many sensible new examples far from the training set?
Relatedly, Kadkhodaic ct al. (2023a) also identified local- ity as a limiting constraint in CNN-based diffusion models, but did not attempt to predict their specific outputs. Con- currently with our work, Niedoba et al. (2024) developed (non-equivariant) patch-based local score approximation model of diffusion models similar to ours, although their quantitative success in predicting the outputs of the neural networks they studied was more limited since they did not study CNNs, which have the strongest locality biases. An- other concurrent work (Wang& & Vastola, 2024) also studied Gaussian mixture-based approximation to the reverse pro- cess to predict samples on case-by-case basis. Finally, the results of our analysis exhibit some similarity very early batch-based texture synthesis methods, c.g. Efros & Leung (1999). Our contributions and outline are as follows:
Several theoretical and empirical works study the properties of diffusion models. Some works study the sampling prop- erties of these models under the assumption that they learn the ideal score function exactly for solvable toy class of distributions (Biroli et al., 2024; De Bortoli, 2022; Wang & Vastola, 2023) or up to some small bounded error (Benton et al., 2024). Others establish accuracy guarantees on learn- ing the ideal score function under various assumptions on the data distribution, and the hypothesis class of functions (Lee et al., 2022; Chen et al.. 2023; Oko et al., 2023; Ventura et al., 2024; Cui & Zdeborová, 2023; Cui al., 2023).
1. We review why diffusion models that learn the ideal score function can only memorize (Sec. 2).
2. We derive minimum mean squared error (MMSE) ap- proximations the ideal score function subject to lo- cality, equivariance, and/or partially broken equivari- ance due to image boundaries. Remarkably, we find simple analytic solutions in all cases (Sec. 3.)
As noted above, key limitation of studying diffusion mod- els under the assumption that they (almost) learn the ideal score function is that such models can only generate mem- orized training examples; while memorizing behavior has been observed in trained diffusion models (Gu et al., 2023: Somepalli et al., 2023), the ideal score function predicts the model will always produce memorized examples, at odds with the creativity of diffusion models in practice. For example, they can compose aspects of their training data in combinatorially many novel ways (Sclocchi et al., 2024; Okawa et al., 2024). This observation has motivated studies of mechanisms behind generalization in diffusion models that underfit the score-matching objective (Kadkho- daie et al., 2023b; Zhang et al., 2023; Wang et al., 2024; Scarvelis et al., 2023). Other works connect creativity in diffusion models to the breakdown of memorization in mod- ern Hopfield networks (Ambrogioni, 2023; Hoover et al., 2023; Pham et al., 2024). However, the extent to which these works can quantitatively predict individual samples from trained diffusion model on a base-by-case basis is more limited.
3.
These solutions lead to local score (LS) machine and boundary-broken equivariant local score (ELS) machine, which constitute fully analytic. mechanisti- cally interpretable theories that can transform noise into creative, structured images without the need for any explicit training process. (Sec. 3).
4. We theoretically characterize samples generated by the ELS machine and show how achieves exponen- tial creativity through locally consistent patch mosaics composed of different local training set image patches different locations in each novel sample (Sec. 4).
5. We show our brondn ELS machine is not only analytic and interpretable but also predictive: it can prediet, on case-by-case basis, the outputs of trained UNets and ResNets, achieving median theory- experiment agreements of 1² ~ 0.94, 0.95, 0.94, 0.96 on MNIST, FashionMNIST, CIFAR1O, and CelebA for the best architecture on each dataset (Sec. 5). We show that on 322x3, ResNets are best predicted by
To develop theory beyond the memorization regime, we
2

---

An analytic theory of creativity in convolutional diffusion models
the ELS machine (median r² ~ 0.96), but UNet behav- ior is better predicted by the fully-local LS machine (median p² ~ 0.90).
Tf - (e) for all E [o, Intuitively, this reversal
occurs because, for any finite dataset D, (6) is a mixture of Gaussians centered at shrunken data points,
6. Our comparison between theory and experiment re- veals that trained diffusion models exhibit a coarse-to- fine generation of spatial structure over time and use image boundaries to anchor image generation (Sec. 5).
Te (+)
(1- (1-aa))) Σ PED
(2)
DD
and the score st()) points uphill on this mixture. Thus the second term in (1) flows Φi, as t decreases, towards shrunken data points, and the first term undoes the shrinking.
7. Our theory reproduces the notorious behavior of diffu- sion models to generate spatially inconsistent images at fine spatial scales (e.g. incorrect numbers of limbs) and explains its origin in terms of excessive locality at late times in the reverse generative process, (Sec. 5).
Motivated by this theory, score-based diffusion models at- tempt to sample the data distribution (o( by forming an estimate (Ø) of the score function sy()), and then plug- ging this estimate and initial noise Φr ~ NNO, 7) into the reverse flow in (I) to obtain sample Φg- We consider what happens when the estimate matches the ideal score function so ()()) = S (o) on any finite dataset D. Then the score of the Gaussian mixture x (Ø) in (2), is (App. A):
8. We compare our purely local ELS machine theory to more powerful trained UNet architectures with non- local self celf-attention (SA) layers. Our local theory can still partially predict their non-local outputs (median y² of 0.77 on CIFARI0), but reveal an interesting role for attention in carving out semantically coherent objects from the ELS machine's local patch mosaics (Sec. 6).
sc(Ø) I 1- ār FED
1
Σ (Vaig-o)Wall -
(3)
Overall our work illuminates the mechanism of creativity in convolutional diffusion models and forms a foundation for studying more powerful attention-enabled counterparts.
Vool (1- (1-)))) Wa(pp)) EVEED Voug', ; az)
(4)
2. The ideal score machine only memorizes
When s in (3) is inserted into (1), each term in (3) acts as a force that pulls the sample towards a shrunken data point / Vāgi as t decreases, weighted by the posterior probability (y(o)) that & at time t would have originated from the datapoint yp at time 0 under the forward diffusion.
We first discuss why any diffusion model that learns the ideal score function on finite dataset can only memorize.
The key idea behind diffusion models is to reverse a stochas- tic forward diffusion process that iteratively converts the data distribution (Ø)), where Φ E R.N is any data point, into a sequence of distributions Te (((() (Ø) over time t, such that the final distribution TT(②) at time T is an isotropic Gaus- sian (0,, I). The forward diffusion process usually shrinks the data points toward the origin while adding Gaussian noise, so that when conditioning on any individual data point y ~ Tp. the conditional probability Te (p)pp) becomes the Gaussian V(ol (1 - ()) The noise schedule āe decreases from 1 at t = 0 to 0 mean /arpp of n, (()ss) shrinks over time, and its variance increases, until Te (p)()) ~ (N(0, I) for all initial points yp (see figure 8).
The combined reverse dynamics in (1), (3) and (4), which we call the ideal score machine, has an appealing Bayesian guessing game interpretation: the current sample & at time optimally guesses which data point 4p it originated from in the forward process, thereby forming the posterior belief distribution W)(p))), and then flows to each (shrunken version) of the data points, weighted by this belief.
Importantly, since the reverse flow provably reverses the forward diffusion, To equals the empirical data distribution To. which is sum of delta functions the training set. Thus, the ideal score machine memorizes. The mechanism behind memorization can be explained by positive feedback instabilities in the reverse flow. In particular, the closer the sample & is to a shrunken version of a data point sp, the higher the belief Wi that & originated from y,, and the stronger the force term ( (Vārs- +) (yll)) in (3) pulling & even closer to the shrunken yp. which in turn raises the belief Wt (y]lo) at earlier t. This positive feedback between belief and force causes the posterior belief distribution (v) to rapidly concentrate onto a single data point yo, and so dr. flows to this same point şp under the reverse flow (Fig.2 a.).
T so that the
A simple time reversal of this forward process can be ob- tained by sampling or ~ (0, I) and then flowing it back- wards in time from T to 0 under the deterministic flow
- = 7cbh se(())),
where s0) = V Volog ne ((b) is the score function of the distribution Te (Ø) under the forward process and Y depends on the entire noise schedule ai (see App. A for details). The flow in (1) induces a sequence of reverse distributions 7 (Ø) that exactly reverse the forward process in the sense that
Thus, any diffusion model that learns the true score 8 on a finite dataset D must memorize the training data and cannot
3

---

we extend our theory to An analytic theory of creativity in convolutional diffusion models
In our case of images, G' is the spatial translation group in two dimensions, U is translated image, and U MM is the translated score function. In other words, translating the input translates the outputs of an equivariant model in the same way. CNNs are translation equivariant if we impose periodic boundary conditions on the pixels, so that, for example, left translation of the leftmost pixels move them to the rightmost pixels (i.e. circular padding). However, the common practice of zero- rero-padding images at their boundary breaks translation-equivariann this case in Sec. 3.4.
(a) IS Machine
(b) I.S Machine
(c) ELS Machine
Figure 2. Ideal score-matching under various constraints. (a) In the IS machine, the entire image (hottom) reverse flows to a single training set image from the training set (top stack). (b,c) In both the LS and ELS machines, different local patches of the image flow to different local patches in the training set. In the LS machine this final training patch must be drawn from the same location (b), while the ELS machine, it can drawn from any location (c).
We next turn to locality. For image data, let x be a pixel location, 6(x) E RC be the pixel value of image & at loca- tion x (where C the number of color channels) and let Me [z] E RC denote the model score function evaluated at pixel location z, which informs how the pixel value d(x) should move under the reverse flow. Also, at each pixel lo- cation let S2, denote a local neighborhood of z consisting of a subset of pixels near x, and let øn. E IR [R xC' be the restriction of pixel values of the entire image & to the |12g] pixels in the neighborhood 22g. We define locality as:
creatively generate new samples far from the training data. While we have explained this memorization phenomenon intuitively using the ideal score machine, it has been well established in prior work (e.g. Biroli et al. (2024)).
Definition 3.2. M, [o] is defined to be Q-local if, for all images & and all pixel locations x, Mr [x)) depends on & only through do., i.e. Mx[ Molonn,1 (xr)
3. Equivariant and local score machines
The failure of creativity in the ideal score machine means that it cannot be good model of what realistic diffusion models do beyond the memorization regime. We therefore seek simple inductive biases that prevent learning the ideal score function 8t in (3) on a finite dataset D. By identifying these inductive biases, we hope to obtain a new theory of what diffusion models do when they creatively generate new samples far from the training data.
Thus if Q-local model M, [(b] is used in place of s()) in (I), the instantaneous reverse flow of any pixel value o(x) at location and time I will not depend on pixel values at any locations outside the local neighborhood Ω2; it depends only on the image in neighborhood Ω2 In particular, two pixels at distant locations x and y with non-overlapping neighborhoods 22 and S2y will make completely indepen- dent decisions as to which directions to reverse flow; the portion of the image on,, in the neighborhood 22 of y, can- not instanteously affect the flow direction of the pixel value ob (z),, and vice versa.
The key observation is that many diffusion models use con- volutional neural networks (CNNs) to form an estimate (()) of the score function. Such CNNS have two prominent in- ductive biases. The first is translational equivariance due to weight sharing: translating the input image will correspond- ingly translate the CNN outputs. More generally, networks can be equivariant to arbitrary symmetry groups (e.g. (Co- hen& & Welling, 2016), (Hoogeboom et al., 2022)). The second is locality: since convolutional filters have narrow support, typical outputs of a CNN depend on their inputs only through a small receptive field of neighboring input pix- els. We therefore seek an optimal estimate (()) of the ideal score in (3) subject to locality and equivariance constraints.
Next, we consider the optimal minimum mean squared error (MMSE) approximation to the ideal score function 8 (e) in (3) under locality and/or equivariance constraints. We provide full derivations in App. B, but the final answers, which we state below, are simple and intuitive.
3.1. The equivariant score (ES) machine
We start with formal definitions of equivariance and locality. Let Ma (o) denote a model score function that takes an input image e and outputs an estimated score á (Ø) = (o).
We first impose equivariance without locality. The MMSE equivariant approximation to s(t) in (3)-(4) is identical in form to the ideal score, except the dataset D is augmented to the orbit of D under the equivariance group G, which we denote by GDD). For example, in our case of images, G(D) corresponds to all possible spatial translations of all images in D. Explicitly, the MMSE equivariant score is given by
Definition 3.1. A model M, is defined to be Geequivariant with respect to the action of a group G on data if for any U E G. Mt satisfies [U = (M).
4

---

An analytic theory of creativity in convolutional diffusion models
(see App. B.3 for a proof)
However, an important limitation remains in the LS machine: local image patch do, pixel location x must reverse flow close to some local training image patch yo, drawn from the same location 2; it cannot flow to training image patch ,o.. drawn from a different location x'. We next see that adding equivariance removes this limitation.
(x(d[()) = T- āt (EG(D)
1
Σ
(Varp(s)- ((1))
(5)
N(olvāng, (1)
Wi(+l0)= = Ege Mooly aip aa)))"
3.3. The equivariant local score (ELS) machine
Replacing the ideal score s(t) in (1) with (5) yields the equivariant score (ES) machine. While the ideal score ma- chine memorizes the training data (see Sec. 2), the ES ma- chine on images achieves only limited creativity: it can only generate any translate of any training image.
Further constraining the LS machine with equivariance leads to the ELS machine in which any local image patch at any pixel location x can now flow towards any local training set image patch drawn from any location x' not necessarily equal to x, in the LS machine. This is the local analog of how the IS machine can only generate training set images, but the equivariance constrained ES machine can generate training set images globally translated to any other location.
3.2. The local score (LS) machine
We next impose locality without equivariance. The MMSE Q-local approximation to s(t) in (3)-(4) is given by
To formally express this result, assume all local neighbor- hoods S2g for different x have the same shape f. For con- creteness, one can think of 9 as a P x P square patch of pixels for P odd, with 2g centered at location x. Then let Po(D) denote the set of all possible Ω shaped local training image patches drawn from any training image cen- tered at any location. An element y E (2((D) now lives in I PxCX and denotes the pixel values of some local Ω2 shaped training image patch centered at some location. Now the optimal MMSE approximation to the ideal score in (3), under both equivariance and locality constraints is (App. B):
M1(o(()) = Σ (Vārg((z) - )(x)) W: Wi Ida.).
- āz
FED
N(aa, Varpp, (1 aa)) ExpE (oo.loalaaaaaaa IVa (1 a. [)
Wa(Pa, 100a)=
(8)
Each term in the local M2 [o] (I) in (7) is identical to each term in s(t) in (3). yielding a force pulling the pixel value ((x) towards a shrunken training set pixel value (xx) as before, except for the important change that the global posterior belief (yll)) in (3)(4), that is the same for all pixels x. is now replaced with a local z-dependent be- lief W. (1(, ) in (7)- (Po, l00,)) is the posterior probability that a sample image Φi under the forward process at time t originated from a training image yp at time 0, con- ditioned on the only information the model M, (x) can depend on, namely the restriction on, of the image á to the local neighborhood 22 at location x. The closer the local image patch Φn, is to the co-located training image patch PP.., the larger the posterior WPP,, (@n.) in (8).
(Vārp((0) - +(x)) W((Flo,
(x)
Σ
1 1- az
DEPa(D)
(9)
N(øa, IVonp (1 - 1))
EVERA (oon, Vaai' (1 ā. a
(10)
We note that (9)-(10) for the ELS machine is identical to (7)-(8) for the LS machine except that: (1) the sum over local training set patches in (9)-(10) in determining the flow M, (x) for pixel o(x)) is no longer restricted to training patches centered at the same location as x; and (2) each pixel x must now track a larger posterior belief state W x)) in (10) about which local training set patch at any location x' was the origin of Φn.,. as opposed to the smaller belief state W₂ (yp,, |002,) ) in (8) about which local training set patch at the same location x was the origin of Φ2.- In essence, in the Bayesian guessing game interpretation, equivariance removes each pixel's knowledge of its location æ. so guess the origin of its local image patch Φo,, it must guess both the training image and the location in the training image that it came from under the forward process. This guess then informs the reverse flow. Taken together, the ELS machine can creatively generate exponentially many novel images by
Replacing the ideal score s(t) in (1) with (7) yields the local score (LS) machine. The LS machine can achieve significant combinatorial creativity by allowing local image neighborhoods on, and o of different pixels x and x' to reverse flow close to training image patches ya, and Slay from different training images o and p' (Fig.2b). Indeed the same positive feedback between belief and force that holds for the IS machine at global level (Sec. 2), also holds for the LS machine at a local level, causing the posterior belief' Wa((plén,) ) of all pixels z to concentrate on a unique training image, but this training image could be different for different far away pixels. This flow decoupling of local image patches in dt empowers exponential creativity.
5

---

An analytic theory of creativity in convolutional diffusson models
Training Data
Valid Samples (Locally Consistent Points)
Invalid Samples
(b)
(c)
Figure 3. Exponential creativity through locally consistent patch mosaics. (a) A training set of two images (all black or all white). (b) Original samples from any local score machine (LS or ELS) with a 3 locality window and periodic boundary conditions. Local consistency in this special case means every generated pixel is either black or white, and the majority color of every generated 3 x 3 patch equals the color of its central pixel. (c) We note that samples are generated by numerically integrating the reverse flow (1). If the step size in this integration is too large, one can generate invalid samples with a few cases of broken local consistency (highlighted red patches). In practice in trained diffusion models, this local consistency would only hold approximately.
mixing and matching local training set patches and placing them at any location in the generated image. We call this a patch mosaic model of creativity.
(D( for which Φu is closer in Lg distance (in IR (0, IxC) than other local training set patch 4e E (D).
Intuitively, samples generated from these machines are lo- cally consistent in the sense that they obeying local condi- tions: (1) every pixel x can be uniquely assigned to a local
3.4. Breaking equivariance through boundaries
Due to the common practice of zero padding images at boundaries, CNNs actually break exact translational equiv- ariance. We can modify our ELS machine to handle this broken equivariance (see App. B.2 for details). The key idea is that breaking translation equivariance restores to each pixel some knowledge of its location within the image. For example, if the local image patch Φn around pixel location a contains many 0 values, then the pixel can use these to infer its location with respect to the boundary, and use this knowledge in the Bayesian guessing game that determines the reverse flow. In essence, with additional conditioning about its relation to the boundary, Φn, should only flow to training image patches that are consistent with the observed amount and location of zero-padding. For example, interior, edge, and corner image patches only flow to interior, edge and corner training image patches with the same boundary overlap (Fig. 9). This is partial case of complete equivari- ance breaking in the LS machine, in which pixels know their exact location and the local image patch dn, only flows to training image patches at the same location x (Fig.2b).
training set patch yp; (2) the pixel value (x(x) is exactly equal to the central pixel sp(0) of yp: (3) the rest of the local
generated patch on resembles the local training patch ip more than any other possible training patch. This result characterizes the creative outcome of locally constrained machines as creating locally consistent patch mosaics where every pixel of every local patch in the sample matches the central pixel of the L2 closest local patch in the training set.
4.1. The simplest example of patch mosaic creativity
As the simplest possible example illustrating the locally consistent patch mosaic model of creativity for the LS and ELS machines, consider a training set of only two images: all black and an all white image (Fig.3a). A highly expressive diffusion model trained only on these two images would only generate these two images. However, an LS or ELS machine with local 3 x 3 neighborhoods generates exponentially many new samples that are locally consistent patch mosaies (Fig.3b): every pixel is either black or white, indicating it is assigned to either an all black or all white 3 x 3 local training set patch. And any 3 x 3 local patch of generated sample with a central black (white) pixel is closer to the all black (white) training set patch than the other training set patch. Thus local consistency in this special case reduces to the simple condition that the majority color of any 3 x locally generated patch must equal the color of its central pixel. The reader can check that this local consistency holds (with appropriate circular wraparound) at every pixel in Fig.3b.
4. A theory of creativity after convergence
It is clear that the reverse flow from Gaussian noise Ør to final sample Φn in the ideal score machine converges to single training set image. But what do the LS, ELS or boundary broken ELS machines converge to at the end of the reverse process if they creatively generate novel samples far from the training data? We answer this question by proving a theorem that characterizes the converged samples Φu at the end of the reverse process (App. B.4).
5. Tests of the theory on trained models
Theorem 4.1. For the LS, ELS, and boundary broken ELS machines. assuming lim &e and lim dior exist, then for every pixel x, Φo(x)= = (0)) for the unique patch yp e
We next test our theory on two CNN-based architectures, a standard UNet (Ronneherger et al., 2015) and a ResNet
6

---

An analytic theory of creativity in convolutional diffusson models
T-0,0, UNer
T- 0.50, UNet
1.000, UNet
UNet
MIENT CIARCO
23-
Crira
Time (Forward Process) ResiNet
ResNer
ResNet
1000 ResNet
25-
GPARSO
Receptive
Time (Forward Process)
(b)
Figure 4. Coarse to fine progression of spatial locality in the reverse flow. (a) A heatmap of the average absolute value of the Jacobian from the output score MM Mr[0e](= 0) at the center pixel 0 back to all input pixels o(x') function of r'. This receptive field shrinks from large to small as time progresses from early (large f) to late (small f) in the reverse flow. (b) Optimally calibrated values of the spatial locality scale P of the (F)L.S machine function of time t (sec App. C.2 for details of calibration). (c) A schematic view of the time-dependen LS and ELS machines in which the locality neighborhood shrinks as the reverse time flows from top to bottom.
(He et al., 2016) trained on 4 datasets, MNIST, Fashion- MNIST, CIFARIO, and CelebA (see App. C.I for details of architectures and training). We restrict our attention to these simple datasets because our theory for CNN-based diffusion models only, and more complex diffusion models with attention and latent spaces are required to model more complex datasets.
behavior is more accurately described by the local score machine rather than the ELS machine on CelebA32x32, the former achieving median y² ~ 0.90; we describe this observation in more detail in section 5.4. To our knowl- edge, this is the first time an analytic theory has explained the creative outputs of trained deep neural network- based generative model to this level of accuracy. Importantly, the (E)LS machine explains all trained outputs far better than the IS machine (Fig. 10 and Table 2). See App. D, Fig. 13 to Fig. 22 for many more successful case-by-case theory- experiment comparisons for the nets and 3 datasets.
5.1. Coarse-to-fine time dependent spatial locality scales
To compare our theory of ELS and LS machines with exper- iments, we must first choose a locality scale for the size of the P x P local patch. We measure it in the trained UNet and ResNet and find, importantly, that it changes from large to small scales as time passes from early (large t) to late (small t) in the reverse flow (Fig. 4a). We therefore promote the spatial size of the P x P locality window in our ELS dynamic variable which we calibrate
We also trained circularly padded ResNets MNIST and IIFARIO, and found a good match between the non- boundary broken ELS machine and experiment (Figs. 11, 21 and 22). Interestingly, in both theory and experiment for MNIST, circular padding yields more texture- outputs and less localized digit-like outputs, indicating the funda- mental importance of boundaries in anchoring diffusion models, for MNIST least (compare Fig. 21 and Fig. 13).
and LS machines to the UNet and ResNet (Fig. 4bc). See App. C.2.
5.2. Theory predicts trained outputs case-by-case
5.3. Spatial inconsistencies from excess late-time locality
We first compare the outputs of the scale-calibrated bound- ary broken-ELS machine the outputs of the ResNet and the UNet on a case-by case-by-case basis for the same initial noise samples Ør to both the theory and the ResNet or UNet, and we find an excellent match 5ab). Indeed we find a re- markable and uniform quantitative agreement between the CNN outputs and ELS machine outputs. For ResNets, find median 2 values between theory and experiment of 0.94 on MNIST, 0.90 on FashionMNISTT, 0.90 on CIFAR 10, and 0.96 on CelebA32x32.. For UNets, we find median 2² values of 0.89 on MNIST, 0.93 on FashionMNIST, and
Diffusion models notoriously generate spatially inconsis- tent images at fine spatial scales, e.g. incorrect numbers of fingers and limbs. Indeed, these inconsistencies are consid- ered a tell-tale sign of Al-generated images (Bird & Lotfi, 2024; Shen et al., 2024; Lin et al., 2024). Our trained mod- els on FaShionMNIST also generate such inconsistencies, e.g. pants with too many or too few legs, shoes with more than one toe region, and shirts with incorrect numbers of arms. Remarkably, our theory, since it matches trained model outputs on case by case basis, also reproduces these inconsistencies (Fig. 5c). Since our theory is completely mechanistically interpretable, it provides clear explanation
0.90 on CIFAR1O (see Fig. 10 for the full distribution of 2 values). We find, unlike on other datasets, that the UNet

---

An analytic theory of creativity in convolutional diffusson models
4
1
+
4
7
MNIST
9
1/1
S/
2
0
0
3
9
9
5
CIFARL
FashisoMNNNST
CelebA
(a) Theory (left) vs. ResNet (right)
(b) Theory (left) vs. UNet (right)
(c) Inconsistencies in FashionMNIST
Figure 5. Match between theory and experiment. (a.b) Each pair of images shows a striking match between the output of the boundary broken ELS machine (left image in each pair) and the output of trained CNN diffusion model (right image in each pair) when both models are given the same initial noise input. We compare theory with 2 architectures (ResNet in (a), and UNet in (b)) on 3 datasets (MNIST, CIFARIO and FashionMNIST from top to bottom). See App. D, Fig. 13 Fig. 22 for many comparisons and Fig. 10 and Table 2 for quantitative x² values indicating high match between theory and experiment. (c) Trained CNN diffusion models (right) produce well-known spatial inconsistencies (e.g. legged pants (row 1,4), 3 armed tops (row 3,6), bifurcated shoes (row 2,5)). Remarkably, the ELS theory (left) predicts this behavior and mechanistically explains it through excessive spatial locality at late times in the reverse flow.
for the origin of these inconsistencies in terms of excessive locality at late stages of the reverse flow. The late-time (t < 0.3) locality for all models is less than about 5 pixels (Fig. 4b). With such small locality scale, different parts of the image more than few pixels away must decide whether to develop into e.g. total number of limbs in the image; this process frequently results in incorrect numbers of total limbs.
However, interestingly, for CelebA, an LS machine that fully breaks equivariance better explains the outputs of the UNet, but not the ResNet, compared to the boundary broken ELS (Table 2). Indeed, the UNet creates more structured faces than the ResNet (compare rows and 4 in figure Fig. 7). The less structured faces of the ResNet are better explained by the boundary- broken ELS machine (compare rows and 2 in Fig. 7). while the more structured faces of the UNet are better explained by the LS machine with fully broken equivariance (compare rows 3 and in Fig. 7).
pant leg without knowing the
5.4. UNets can fully break equivariance
We note that for three datasets, MNIST, FashionMNIST and CIFAR10, the best matching theory that explains the outputs of zero-padded CNNs (for both ResNets and UNets) is the boundary- broken ELS machine (see Table and Fig. 5).
An explanation for why the UNet can in principle fully break equivariance, while the ResNet cannot, is that the maximal possible receptive field (RF) size of the ResNet is 17x17 while the image is 32x32. Thus, any instant of time t, the
UNeESA
Theory
Figure 6. Comparison between UNet+SA outputs (top row) and ELS machine outputs (bottom row) for the same noise inputs. For this class of inputs, the UNct+SA appears to carve out more semantically coherent objects out of the closely related ELS patch mosaic.
8

---

An analytic theory of creativity in convolutional diffusson models
entire distribution of xa values).
Theur (EL
Qualitatively, the outputs of the UNet+SA model fall into three rough classes in which the UNet+SA produces: (1) semantically incoherent image which nevertheless strongly resembles the prediction of the ELS machine (Fig. 23a); (2) a semantically coherent image which has some quantitative correlation with. but little qualitative resemblance to, the ELS machine prediction (Fig. 23b); and (3) a semantically coherent image that also has strong resemblance to the less semantically coherent ELS machine outputs (Fig. 6).
ResNa
Theary (LS)
UNet
This third class is the largest and most interesting of the three. Qualitatively, the UNet+SA appears to carve a seman- tically coherent object out of the patch mosaic of the ELS machine (compare top and bottom rows of Fig. 6). For exam- ple, the UNet+SA often cuts out foreground object from the ELS patch mosaic, while smoothing the background and accentuating it from the foreground object.
Figure 7. Comparison between LS, ELS, ResNet, and UNet out- puts on CelebA. Each column represents identical initial noise inputs. Note the Unet produces more spatially structured faces than the ResNet, and the LS machine better explains this higher UNet performance than the ELS machine.
Fig. 24 shows large set of comparisons between the ELS machine and UNet+SA outputs. While these results show that the ELS theory bears in many cases both quantitative and qualitative resemblance the UNet+SA outputs, full quantitative theory of the role of attention in the creativity of diffusion models remains for future investigation. However, the correspondences in Fig. 6, Fig. 23a, and Fig. 24 and the ELS correlations (y-axis) in Fig. 12, suggest the ELS theory provides an important foundation for this endeavor.
ResNet score computation at pixels near the image center cannot depend image data outside this RF. However, the maximal possible RF size of the UNet covers the entire image. Thus, the UNet can in principle use information over the entire image, including the boundary, to infer the absolute location of each pixel when computing the score at that pixel. Indeed, it does this for CelebA, possibly because for CelebA there strong correlations between image neighborhoods and pixel locations (e.g. eyes, ears, mouths and noses all appear in similar locations across the dataset). However, for the other datasets, the UNet does not seem to infer absolute pixel location far from the boundary when computing the score at each instant of time, and so is better described by boundary-broken ELS machine rather than an LS machine with fully broken equivariance.
7. Discussion
Developing mechanistic understanding of how generative models convert their training data into novel outputs far from their training data is an important goal in the field of neural network interpretability. We have developed such an understanding for convolutional diffusion models of im- ages that accurately predicts individual outputs on fixed random inputs in terms of the training data, for standard ar- chitectures (ResNets and UNets), standard datasets (MNIST, FashionMNIST, CIFAR 10, and CelebA), and standard loss functions msthre-ma Moreover, our mechanistically interpretable theory of diffusion models is derived not from intensive and highly detailed analysis of the inner workings of trained networks (modulo matching spatial scales), as in most mechanistic interpretability works, but rather from a first principles approach stemming from analytic solutions for the optimal score subject to only 2 posited inductive biases: locality and equivariance. The strong quantitative agreement between theory and experiment on a case-by-case basis suggests that these two inductive biases are sufficient to explain the creativity of convolution-only diffuusonn models. We hope this work provides a foundation for understanding the creativity of more powerful attention-enabled diffusion models trained on more complex datasets.
6. The relation between theory and attention
While the local theory explains the outputs of CNN-based diffusion models on a case by case basis with high accuracy, many diffusion models also include highly nocal self- attention (SA) layers. For example (Ho al., 2020)) added SA layers UNet (which we call a UNet-SA architecture). The non-locality of SA strongly violates the assumptions of our local theory. This violation raises an important question: do the predictions of our local theory bear any resemblance at all to the non-local outputs of trained UNet+SA models?
To address this question, we compare our existing ELS machine theory with the outputs of publicly available UNet+SA model pretrained on CIFAR10 (Sehwag, 2024). Strikingly, our ELS model, with no modification whatsoever, predicts the UNct+SA outputs on base-by-case basis with a median of , ~ 0.77 on 100 sample images. This is substantially higher than the median , ~ 0.48 of an IS machine baseline on the same images (see Fig. 12 for the
9

---

An analytic theory of creativity in convolutional diffusion models
Acknowledgements
models on low-dimensional data. In International Con- ference Machine Learning, pp. 4672-4712. PMLR, 2023.
M.K. would like to acknowledge the support of the NSF Graduate Research Fellowship. M.K. would like to acknowl- edge the helpful conversations, comments, and feedback from Daniel Kunin, Atsushi Yamamura, and Feng Chen. S.G. thanks the Simons Foundation, Schmidt Sciences Polymath Award, and an NSF CAREER award for funding.
Cohen, T. and Welling, M. Group equivariant convolutional networks. In International conference on machine learn- ing. pp. 2990-2999. PMLR, 2016.
Cui, H. and Zdeborová, L. High-dimensional asymptotics of denoising autoencoders. arXiv [cs.LG], 18 May 2023.
Impact Statement
This paper presents work whose goal advance our un- derstanding of Machine Learning systems. As the scope of the capabilities of these systems increase, and as these systems become more deeply integrated into socially impor- lant applications, it is imperative to develop fundamental understanding of how these capabilities emerge. Unfortu- nately, the development of such fundamental understanding has lagged significantly behind advances in capabilities. Our work helps address this gap by developing a better under- standing of simple but still highly nontrivial deep networks, hopefully paving the way for future studies that move our fundamental understanding of these systems closer to the state-of-the-art of capabilities.
Cui, H., Krzakala, F., Vanden-Eijnden, E., and Zdeborová, L. Analysis of learning flow-based generative model from limited sample complexity. arXiv [stat.ML], 5 October 2023.
De Bortoli, V. Convergence of denoising diffusion mod- els under the manifold hypothesis. arXiv preprint NDIi:2208.0033444 2022.
Dhariwal, P. and Nichol, A. Diffusion models beat gans image synthesis. Advances in neural information processing systems, -18988-8794, 2021.
Efros, A. A. and Leung, T. K. Texture synthesis by non- parametric sampling. In Proceedings of the seventh IEEE international conference on computer vision, volume 2, pp. 1033-1038. IEEE, 1999.
References
Alakhdar, A., Poczos, B., and Washburn, N. Diffusion models de novo drug design. Journal of Chemical Information and Modeling, 2024.
Gu, X., Du, C., Pang, T., Li, C.,, Lin, M., and Wang, On memorization in diffusion models. arXiv preprint arXiv:2310002664 2023.
Ambrogioni, L. In search of dispersed memories: Genera- tive diffusion models are associative memory networks. arXiv preprint arXiv:2309.17299 2023.
He, K., Zhang, X., Ren, S., and Sun, J. Deep residual learn- ing for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770-778, 2016.
Benton, J., Bortoli, V., Doucet, A., and Deligiannidis, G. Nearly d-linear convergence bounds for diffusion models via stochastic localization. 2024.
Ho, J., Jain, A., and Abbeel, Denoising diffusion proba- bilistic models. Advances in neural information process- ing systems, 33:6840-6855 2020.
Bird, J. and Lotfi, A. Cifake: Image classification and ex- plainable identification ai- ti-generated synthetic images. IEEE Access, 2024.
Ho, J., Chan, W., Saharia, C., Whang, J., Gao, R., Gritsenko, A., Kingma, D. P., Poole, B., Norouzi, M., Fleet, D. J., et al. Imagen video: High definition video generation with diffusion models. arXiv preprint arXiv:2210.02300, 2022a.
Biroli, G., Bonnaire, T., De Bortoli, V., and Mézard, M. Dynamical regimes of diffusion models. arXiv preprint mrXiv 2024.
Ho, J.. Salimans, T., Gritsenko, A., Chan, W., Norouzi, M.. and Fleet, D. J. Video diffusion models. Advances in Neural Information Processing Systems, 35:8633-8644 8646, 2022b.
Blattannn A., Dockhorn, T., Kulal, S., Mendelevitch, D., Kilian, M., Lorenz, D., Levi, Y., English, Z., Voleti, V., Letts, A., al. Stable video diffusion: Scaling latent video diffusion models to large datasets. arXiv preprint N..i::2311.15127., 2023.
Hoogeboom, E., Satorras, V. G., Vignac, C., and Welling, M. Equivariant diffusion for molecule generation in 3d. In International conference on machine learning, pp. 8867- 8887. PMLR, 2022.
Chen, M., Huang, K., Zhao, T., and Wang, M. Score approx- imation, estimation and distribution recovery of diffusion
10

---

An analytic theory of creativity in convolutional diffusion models
Hoover, B., Strobelt, H., Krotov, D., Hoffman, J., Kira, Z., and Chau, D. H. Memory in plain sight: A survey of the uncanny resemblances between diffusion models and associative memories. arXiv preprint arXiv:2309.16770, 2023.
Pham, B., Raya, G., Negri, M., Zaki, M. Ambrogioni,
L., and Krotov, D. Memorization to generalization: The emergence of diffusion models from associative memory. 2024.
Ramesh, A., Dhariwal, P., Nichol, A., Chu, C., and Chen, M. Hierarchical text-conditional image generation with clip latents. arXiv preprint arXiv:2204.005 06/25, 1(2):3 2022.
Kadkhodaie, Z., Guth, F., Mallat, S., and Simoncelli, E. P.
Learning multi-scale local conditional probability models of images. arXiv preprint arXiv:2303. 02994, 2023a.
Rombach, R., Blattmann, A., Lorenz, D., Esser, P., and Ommer, B. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF con- ference on computer vision and pattern recognition, pp. 10684-10695, 2022.
Kadkhodaie, Z., Guth, F., Simoncelli, E. P., and Mal- lat, S. Generalization in diffusion models arises from geometry-adaptive harmonic representation. arXiv preprint arXiv:2310.02557. 2023b.
Lee, H., Lu, J., and Tan, Y. Convergence for score-based generative modeling with polynomial complexity. Ad- vances in Neural Information Processing Systems, 35: 22870-22882, 2022.
Ronneberger, O., Fischer, P., and Brox, T. U-net: Con- volutional networks for biomedical image segmenta- tion. In Medical image computing and computer-assisted intervention-MICC. 2015: 18th international confer- ence, Munich, Germany, October 5-9, 2015, proceedings, part III 18, pp. 234-241. Springer, 2015.
Li, X., Thickstun, J., Gulrajani, I., Liang, P. S., and Hashimoto, T. B. Diffusion-Im improves controllable text generation. Advances in Neural Information Process- ing Systems, 35:4328 4343, 2022.
Scarvelis, C., Borde, H. S. O., and Solomon, Closed- form diffusion models. arXiv preprint arXiv:2310.12330 2023.
Li, Y., Zhou, K., Zhao, W. X., and Wen, J.-R. Diffusion models for non con-autoregressivv text generation: survey. arXiv preprint MXX.::333300577. 2023.
Sclocchi. A., Favero, A., and Wyart, M. A phase transition in diffusion models reveals the hierarchical nature of data. arXiv preprint 24X:::2402.1691,, 2024.
Lin, L.,, Gupta, N., Zhang, Y., Ren, H., Liu, C.-H., Ding, F. Wang, X., Li, X., Verdoliva, L., and Hu, S. Detecting multimedia generated by large models: A survey. arXiv preprint arXiv:2402.00045 2024.
Sehwag. V. Minimal implementation of diffusion models. https:/ htop::///ittiee //g/tttbb.... wtt.cco////eeeee VSewwag// minimal-diien diffusiff. 2024. Accessed: 2024-11-01.
Lipman, Y., Chen, R. T., Hamu, H., Nickel, M., and Lc, M. Flow matching for generative modeling. arXiv preprint ar 2rii:22100007777, 2022.
Shen, D., Song, G., Xue, Z., Wang, F.-Y,, and Liu, Y. Re- thinking the spatial inconsistency in classifier-free diffu- sion guidance. In Proceedings of the IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition, pp. 9379, 2024.
Nichol, A. Q. and Dhariwal, P. Improved denoising diffu- sion probabilistic models. In International conference on machine learning, pp. 8162-8171. PMLR, 2021.
Niedoba, M., Zwartsenberg. B., Murphy, K., and Wood, F. Towards mechanistic explanation of diffusion model generalization. arXiv preprint arXiv:2411.19332 2024.
Sohl-Dickstein, J., Weiss, E., Maheswaranathan, N., and Ganguli, S. Deep unsupervised learning using nonequi- librium thermodynamics. In International conference on machine learning, pp. 2256-2266 PMLR, 2015.
Okawa, M., Lubana, E. S., Dick, R., and Tanaka, H. Com- positional abilities emerge multiplicatively: Exploring diffusion models on a synthetic task. Advances in Neural Information Processing Systems, 36, 2024.
Somepalli, G., Singla, V., Goldblum, M., Geiping, J., and Goldstein, T. Diffusion art or digital forgery? investigat- ing data replication in diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pat- 2023.
Oko, K., Akiyama, S., and Suzuki, T. Diffusion models minimax optimal distribution estimators. In Inter- national Conference on Machine Learning, pp. 26517- 26582. PMLR, 2023.
recognition, pp.
Song. J., Meng, C., and Ermon, S. Denoising diffusion im- plicit models. arXiv preprint arXiv:2010.02550 2020a.
Peebles. W. and Xie, S. Scalable diffusion models with transformers. In Proceedings of the IEEE/CVF Interna- tional Conference on Computer Vision, pp. 4195 2023.
Song, Y. and Ermon, S. Generative modeling by estimating gradients of the data distribution. Advances in neural information processing systems, 32, 2019.

---

An analytic theory of creativity in convolutional diffusion models
Song, Y., Sohl-Dickstein, J., Kingma, D. P., Kumar, A., Er- mon, S., and Poole, B. Score-hased generative modeling through stochastic differential equations. arXiv preprint arXiv:2011.13456 2020b.
Ventura, E., Achilli, B., Silvestri, G., Lucibello, C., and Ambrogioni, I.. Manifolds, random matrices and spectral gaps: The geometric phases of generative diffusion. arXiv preprint arXiv:2410.05880 2024.
Wang, B. and Vastola, J. The unreasonable effectiveness of gaussian score approximation for diffusion models and its applications. Transactions Machine Learning Research, 2024.
Wang, B. and Vastola, Diffusion models generate images like painters: an analytical theory of outline first, details later. arXiv preprint PrXfi:2303.02490 2023.
Wang, P., Zhang, H., Zhang, Z., Chen, S., Ma, Y., and Qu, Q- Diffusion models learn low-dimensional distributions via subspace clustering. arXiv preprint arXiv:2409.02226, 2024.
Watson, J. L., Juergens, D., Bennett, N. R., Trippe, B. L., Yim. J., Eisenach, E., Ahern, W., Borst, A. J., Ragotte, R. J., Milles, L. F., al. De novo design protein struc- ture and function with rfdiffusion. Nature, 620(7976): 1089-1100, 2023.
Zhang, H., Zhou, Lu, Y., Guo, M., Wang, P., Shen, L., and Qu, Q. The emergence reproducibility and con- sistency in diffusion models. In forty-first International Conference Machine Learning, 2023.
12

---

An analytic theory of creativity in convolutional diffusion models
Score function St (Φ)
To
TAN
White 7)
Training images SP
Noised Samples dt
Figure 8, A schematic illustration of score score-matehee matching diffusion.
A. Mathematical Preliminaries
A.1. Notation conventions
In what follows, we use the following notation:
D will represent the training set.
yp E N will represent an example from the training set. For images of size L pixels by L pixels by C' channels, we have N = L x L x C.
Φ will represent any arbitrary image (or other data) that we are plugging into the score function/diiftssion model.
x represents a pixel location in an image.
For image data, (zx) and (x()) will represent the pixel values of the images & and at pixel location æ; both elements of gC
MM] R + RM represents a model that takes in an image ó and produces a new image (e.g. an estimate of the score function). We will denote by (z) E ICC the value of the outputs of this model, given an input Ø at the pixel location æ.
Φn. and Pn_ will represent the restriction of images o and yp to neighborhood Qg around a pixel æ. We usually take Qg to be square patch of size P x P, with P odd, containing pixel at the center. In this case, dn, and yo, vectors in R Px
PxPC PxC However, the theoretical framework supports arbitrary assignments from - Ωz
For square image patch with an odd-dimension side length, the value (00) E IRC indicates the pixel at the center of the patch.
(D( will denote the set all Q2shaped patches drawn from elements of D.
(Nlu,, Σ) represents the PDF of the normal distribution with mean μ and covariance Σ. We also use the short-hand
N(,, Σ) when do need refer the name of specific random variable.
A.2. Stochastic differential equations (SDEs) and Probability Flow
In probabilistic modeling, we are often confronted with the problem of sampling from a data distribution whose exact form we do not have access to, or whose form makes direct sampling difficult. Diffusion models are an approach to sampling from such distributions by learning time-ine ineomogeoouos differential equation that transports samples from simple Gaussian distribution the more complex distribution of interest.
13

---

An analytic theory of creativity in convolutional diffusion models
More formally, consider a time-dependent (Itô) stochastic differential equation, given follows:
dø. fe((t) dl + gr. dW.-
(1I)
Here We is a standard Wiener process and dW; is its differential. We call this stochastic process the 'forward' process. starts from the data distribution (() and induces a flow on probability distributions Te for 0 described by associated Fokker-Planck equation:
TT.(0) = - (fe(o) (
(12)
0L
We will imagine that our forward process is constructed so that (or - T for some finite time T), He converges to some tractable o. typically Gaussian with finite variance.
The idea underpinning diffusion models (or, more technically, DDIMs, the deterministic variant of diffusion models considered for the most part in this paper) is to look for deterministic, deeedeenndent vector field ve vo()) that induces the same flow on distributions as (12). Then one can simply reverse this flow to sample from (t(() by first sampling from the simple distribution ΦT ~ ny, then evolving the sample deterministically backwards in time from = T = 0 under the ODE
dør = vr())).
(13)
dt
This ODE induces a flow on probability distributions (Ø) described by the advection equation
θT. Øt
= [ve())
(14)
We want this advection process above to induce the same flow on distributions as the original flow (12), when run in reverse starting, from the simple final distribution тт. (This setup is closely related 'flow matching" models: see (Lipman et al., 2022) for a review). Interestingly, (0)) can be easily identified by rewriting the flow in (12)
@m(()) 0
1 V (lfi()) - 76 V log logTe(o))=.(O))
(15)
By matching (14) and (15), we find
= f(()) 2 log (o).
(16)
This vector field is sometimes known as the 'probability flow.' The function
st (Φ) = log (Φ)
(17)
is known the score function, and contains all of the complicated dependency the initial distribution (e) that would like to capture in our model.
A.3. Diffusion models
The most common choice of forward process (11) an inhomogenous Ornstein-Uhlenneec (OU) process process of the following form:
dø, ~v00: + 22ndW,
(18)
for which the probability flow is given by
Up (o) ( + V log (d)).
(19)
The reason for this choice is that the finite-time marginals मe for this distribution can be sampled from tractably. We can generate samples & ~ Te by computing the following linear linear combination:
Φt Vardo + v1 G,7/
(20)
14

---

An analytic theory of creativity in convolutional diffusion models
with do To sample from the target distribution and v (O0 7) vector of isotropic Gaussian noise. The values of di depend on the choice of 7/2 via the following formula:
a exp -2 f dt
(21)
In practice, the values are typically chosen first and "/r is then specified implicitly by this choice. The choice of â is known as the 'noise schedule' for diffusion model; typically, we choose ãे = 1 (so that = 0 corresponds to uncorrupted sample) and oT = O for some large but finite value of T' (so that the entire reverse process can take place in finite time). At a distributional level, the solution of (12) for this process given by
TØ( - moloo) / (1-a1) (I) don.
(22)
The score function for ne can then be obtained analytically in terms mn:
Toloo) (1 T.)T) (Ør vorso) déo
(+) 1 04.
(+))
(23)
Cr. P(oooo+ = () - v vartho) don-
There is an extremely convenient fact about this particular score function that we can take advantage of order to learn it from data. Given a particular sample Φ; generated by the forward noising process, the score function proportional the conditional expectation of the added noise 77 from (20), given dr:
sc(
(Th pz: = Φ).
(24)
VI ay
This result is known as Tweedie's theorem. A standard result in statistics is that the conditional expectation (m (d)) the functional optimum of the following loss function:
(f( No. (on. m)) my [12]
(25)
for de defined (20); the following slightly rescaled loss can be used if mcore-matching is preferred:
(f( = E. -ooo- f m)) + (1 - az)) 19.
(26)
In practice, model the score using single neural network fo(x, t) for all times t [0, using the following objective:
I(0)= = B..000077.66000o000000l0ll1666 m),t) n. 3).
(27)
A.4. The empirical score function
In practice, we never have direct access to the data distribution To that we are attempting to sample from; we only have access the discrete empirical prior defined by particular training set D:
1 D
To
8(Φ s).
(28)
PED
At finite time I, the empirical distribution of noised training examples is simply mixture of Gaussians centered at the (rescaled) training data points:
(Φ) = Σ
(l (1 a.).).
(29)
D
PED
15

---

An analytic theory of creativity in convolutional difflusion models
The score function (23) for this distribution is then simply given by
(i(0) - āi
Σ (Ø ((ll))
(30)
RED
(1- Wx(Fl1)- MOPED-NoVVarr (1)
(31)
Intuitively, this corresponds to computing the conditional average over the added noise, by averaging the proposed noise vectors 7/ X (ø - vāri) between our observed example & and each training example ip, weighted by the probability ((p () of ip being the training example that & originated from. This probability is in turn computed essentially by Bayes theorem: the probability of starting from a training example ye, given the observed Φ, is given by the likelihood of generating the noise needed to go from yp to o, divided by the likelihood of going from y' to ø for all possible training examples pp.. Appealingly, the weights W (vll)) are given by computing simple soft-max over a simple quadratic loss function 2(1
0 Varep! for every point in the training set.
It should be emphasized at this point that the ideal score function is not representative of real diffusion models. Primarily: always memorizes the training data. More importantly in practice, this memorization property becomes manifest very early in the reverse process for high dimensional data, due to the typically large separation between training points in Fuclidean space. This is a manifestation of the curse of dimensionality- it would require an amount of data exponential in the dimension to provide sufficiently good coverage of the underlying space for the ideal empirical score function to well-approximate the true ideal score function over all inputs over all times.
The failure of the ideal score function as a model for realistic diffusion models suggests that we should try to understand the particular manner in which they fail to optimally solve the task that they are trained on, In particular, we are motivated to look for the implicit and explicit biases and constraints that prevent these models from learning the ideal score function, and then understand what they do instead under these limitations.
B. Formalism
B.1. Optimal local translationally equivariant score matching
Fully translationally equivariant local models M, can be written in the following way:
(x) = f(0a))
(32)
where Φn, is the restriction of & to the neighborhood Q2 around pixel In this section, we will use circular boundary conditions, so that if & is near an image border, the neighhorhood on, includes the pixels on the opposite side of the image near the corresponding border (we will revisit this in the next section). This functional form reflects the locality constraint by making manifest that the output at a pixel location æ depends only on the patch Φo, around it. Equivariance is reflected in the fact that the output of the model at every point æ is determined by the same function of the input patch. f should be of as function mapping CC - C where C' is the number of channels in the image and [22] is the number of pixels in the local patch Ω. The problem of identifying the optimal local/equivariant model can thus be framed as finding the that minimizes the score matching objective:
thought
C
Σ Es [ll(oo..) st [o]
(33)
Writing this out concretely gives
L c= Te( Σ - - s=(e(())|1 do.
(34)
To find the functional optimum, we vary the objective with respect to f(Ø), with & any arbitrary patch, and set this variation to zero This yields the condition
Σ / Te ((()((((ba))
0 0=
)c))(((6 - b) do.
(35)
16

---

An analytic theory of creativity in convolutional diffusson models
We rearrange this into the following form:
Σ / ö(0n. 7))((())))0(()) de
f()) Σ Telon. = =
Σ / blón. -VV d6
Σ V
$)
Here (0) E R is the pixel value in the center of the patch T. Tee (0n, b) indicates the marginal probability under the distribution Te that the patch on, equals the target patch &. The distribution 2n TT Tr(ón, D) is then proportional to the marginal distribution that a mandollyseelcttd f2-shaped asshped-patch in the image & equals Q. Dividing through by this marginal, we obtain
((0) V Vo log Σ Ténn = =$)
(36)
Lee we find that f() is simply the score function of the modified marginal density E. (e =). Since Ti (Ø) is mixture of Gaussians, the marginal Te(0n_ = . can be obtained simply and is given by
Tt (dn = = = Σ
N( Va,pa, (1-))
(37)
MED
Summing over x gives
Σ Telón. = +b
Σ (E)
N(olVarp, (1 a.)
(38)
where (D) is the set of all S patches in the training set D. Finally, taking the derivative with respect to ((0) and substituting on. for & gives us the final answer for I I. which, when inserted into (32), yields the final answer for Mp:
(x(o(()) = 1 - ār Σ (o(x) - (a) (EPA(D)
(39)
N(on, Iooe (1 - āz)
W(olon = E EptEPa(D)) (oo, Vaue' Tz)
(40)
We term the reverse diffusion model parameterized by M, (39) the Equivariant Local Score (ELS) Machine.
This result has a simple intuitive interpretation. Firstly, it should be noted that the form of the resulting approximation to the score function strongly resembles the form of the true score function (30). In that case, the score function computation could be framed as guessing the added noise by finding the necessary added noise for each possible training set clement, computing the likelihood of generating that noise under a Gaussian noise model, and then averaging the possible noises over the entire training set weighted by the Bayesian posterior over each possible noised example.
The ELS machine (39) can be interpreted similarly. However, very important distinction is that the Bayes weights are pixelwise-decoupled. Under the exact computation of the score function, the Bayes weights are computed based all available information in the image, and shared across every pixel; under the independently computes a separate set of Bayes weights for each training set element, based on its local receptive field. This decoupling of the belief states of different pixels means that under the reverse denoising process parameterized by (39), different pixels will be drawn towards different elements of the training set. At scales below the locality scale the final denoised images should (roughly) resemble part of a training set image: however, larger scales, the resulting images will not resemble any particular training set image, but rather a kind of patchwork quilt/mosaic of randomly combined training set images. We make this result more precise in (B.4).
approximation, each pixel
The role played by equivariance can likewise be interpreted very simply as removing each pixel's ability to locate itself within the image. Position is therefore promoted to a latent variable that must be integrated over, in addition to the training set element itself. This results in needing to compute a Bayes weight not only for each gorespondinnlly-lcaat patch in the training set, but every possible patch in the training set.
17

---

An analytic theory of creativity in convolutional diffusion models
Center patch
Edge patch
Comer patch
Figure 9. In the presenee of zero-padded borders, different dictionaries of training set patches are used for the ELS machine computation depending on the contextual information provided by the visible border within the patch. For a central patch (left. red patch) without border information, training set patches (green patches) are sourced from the entire image interior. For edge patches (middle, red patch), training set patches (green patches) are sourced from everywhere along the edge at the same distance from the border. For corner patches (right, red patch), only patches from that exact location used in the computation.
B.2. Adding borders
There is ambiguity about the behavior of a convolutional neural network for pixels enough to the boundary of an image such that the network's receptive field extends past that boundary. One option in that situation is to enforce circular boundary conditions, so that the convolution operation 'wraps around' to the other side upon encountering the boundary. This approach is not typically used in practice; more commonly, 'zero padding' is introduced, wherein pixels outside the image are treated as zeros for the purposes of the convolution operation.
In the presence of zero-padding, the results given above concerning the optimal local equivariant approximation to the score are nearly identical; in fact, the fundamental identity (32) still holds. However, must modify the interpretation of the visible patch on, for pixel near the border. Instead of considering the patch include 'wrapped around' portions of the image, we instead simply extend it with zeros in all locations where it extends past the border.
When the ELS machine takes as input the patch Φn_, it computes the conditional probability that it corresponds to a noising of each particular patch in the training set. Formally, getting an exactly zero value at any pixel location occurs with probability zero. Thus, observing a patch Φn, with zero-padding indicates with probability that the patch is corruption of training set patch that came from location inside the image consistent with the observed border information. We are thus able to write the ELS machine in the presence of rero-padded boundary
(x)
1
(ő(x) - vā(((0)) 0i (D))(D)
Σ
N(nn, (1 EpeEpH N( Vaie' (1 ū,,)))
(41)
The only modification to the ELS machine (39) is that we have replaced the set of all patches (D) in the sum with the r-deependent patch dictionary Ps (D), corresponding to the collection of patches consistent with the border data at location These collections illustrated in figure
B.3. Optimal equivariant score matching for general symmetry group
In many diffusion model applications outside of computer vision, equivariance under more general symmetry groups is built in to the architecture of the backbone model. For instance, molecular diffusion models are sometimes made equivariant under E3)), the group of isometries on Euclidean space (Hoogeboom al., 2022). Diffusion transformers (Peebles & Xie, 2023) are also naturally equivariant under the group of sequence permutations, although this equivariance is broken in controlled way by the inclusion of positional embeddings. We are thus motivated to study the question of optimality under the constraint of equivariance under a general group of symmetries G, which we define as follows:
Definition B.1. Let G be particular group of transformations acting data ø. We say that model M, is G-equivariant if, for any U E G, our model satisfies
MU U.M.@o).
(42)
The result is given here:
18

---

An analytic theory of creativity in convolutional diffusion models
Theorem B.2. The optimal G-equivariant approximation to the empirical score function (3) under the score matching objective (26) is given by the empirical score function for the dataset G(D) consisting of the orbit of the dataset 'D under the group G.
Proof. Let Mj be G-equivariant model. For simplicity, we will assume that M2 is being optimized with the following loss:
L = Bg M| [6] si (
(43)
where s; V log Te (6) is the ideal score function. First consider the orbit of single point do under the group G, given by G[ {o= EU E G U'n Φ). For any @ E G[őo]. there is an element U E G such that U-L = Po. and thus the output of equivariant model .M is simply U M, [oo]. The problem of picking optimal Mc[o] for any ø G[oo] can thus be reduced to standard linear regression for Mr [do]. under the loss
(0EG( L (0
M|[]]
log (
lo Te mrlU-Lpp 60]
[M[0] UV log TI T, (U- 0b) du
(G(ool)
where in the second line we have used the property of unitaries that [ ||=|| Here Ty (G[øol) indicates the probability density assigned to the entire orbit Gløo] by me. We have used the orbit-stabilizer property to write the integral over the orbit as an integral over the entire group. Despite its complexity this formula represents a standard least-squares objective for M, [o]. the minimizer of which is simply the weighted average of the target function UV log Te. TU))o))) weighted by
m. Ahol. In other words, optimal G-equivariant model is
M [
Te (U( Φ)
U log Te d) Jvea (V-))) dV dU.
(44)
Juea
We can do some simple algebra to write this experssion in a more interpretable form:
Sued U Vog n. - ( dU
Svea UV 6) dU Svec T. (U dU
V log /
T-(U-10) dU
Svec oe(U-16)dl dU
JuEG
where in the last step we have used the fact that U = U and that
Φ).
We now note that
J.sc TulU- 5) du
Σ
N(U 6:
(1 N(U d))
(45)
D
UEG
RED
Since U is unitary, it follows that
U0-1 1b vo.+pl12)
(IU-@lpVan, (1-a))) x exp
2(1 ā.)
| x11
exp
2(1 at))
and thus our optimal model is the score function for the empirical noise distribution of the G-augmented dataset, i.c.
a {
1
Σ
N(ù; Vaup' (1 ox)
(46)
D
G(e)
RED
EpED Sa (6
T) di'
M1 = v log _
(0)
(47)
1 - āe
ExeE Saio /olloai (1- d
19

---

An analytic theory of creativity in convolutional diffusion models
B.4. The sample distribution at under local score approximation
When the score is learned optimally, the reverse process concentrates the sample distribution on the training dataset - 0. It is instructive for to ask what the analogous constraint the generated samples is for the localit--constramned models that consider in this paper. The answer is that the flow will concentrate the probability on certain 'locally consistent points' ē, defined as follows. Suppose we are employing an f2-occa approximation Mr to the score function, with each individual pixel a using a (possibly identical) dictionary of patches &p E P2- A 'locally consistent point' & is point such that for every pixel location æ, the value ((x) is equal to the center pixel (o(0) of the 12-closest patch Pa to the patch on., iee the patch that minimizes | do_ ||
over all patches in Po.
The reverse flow approximation parameterized by M, will concentrate on locally consistent points. We can formalize this effect in the following theorem:
Theorem B.3. Suppose sample an initial point Φr from the Gaussian wT, and we evolve this density under the standard reverse process
0.0: = --60e + M.(())
(48)
where
0,ā0
(49)
2ā:
Suppose also that the limits lim oe and lim;-so deor exist for an initial point Ør- Then the limit must be locally consistent point.
Proof. The assumption that lim;-vo or exists entails that for any point o, on a particular trajectory, the values of Φe and (or(o, + M2(o.)) must stay bounded as t + O, which in turn entails that M,,,( must likewise stay bounded as t + 0. This latter quantity is given at pixel location x by
0,a,
lim Mc)o)() = lim (+0
Σ (dr(x) - Varp()))W(((
(50)
20, (1 as) PEPR
Nlon.laare (1 -1-1000 az)
(( =
(51)
E'errg Noon,
/aiip' I
T)
The prefactor goes to oc t- as t -> O, so it follows that for the derivative to have finite limit, the hinh factor must go zero. As de - O, the weights take the limiting values
{
so = arg min (llon - =3
lim W (z)))) r
(52)
else
and thus the limiting value of the sum is simply
(δ(z) - ((())
(53)
where و arg min, (lln, )) This value is zero only when d(x) = -(()). The condition that this holds for all is the definition of a locally consistent point.
C. Empiries
C.1. Experimental details
To test our ELS machine model of CNN-based diffusion, we examine two different architectures:
1. UNet: we use a standard UNet (Ronneberger et al., 2015) with three scales with channel dimensions of 64, 128, 256 respectively. We use residual connections in each UNet block. This model is formally local, but has a maximum receptive field size larger than the 32 32 images we consider.
20

---

An analytic theory of creativity in convolutional difflusion models
2. ResNet: we use a minimal 8-layer convolutional neural network, with an upscaling and downscaling layer and 6 intermediate convolutional layers at channel dimension of 256. Each layer is single convolutional layer with kernel size of x 3 and with residual connections (He et al., 2016) between layers. This model has a formal maximum receptive field size of 17 x 17.
For all experiments, we train each model for 300 epochs with Adam, using an initial learning rate of le-4, a batchsize of 128, and an exponential learning rate schedule that applies a multiplicative factor of 0.999965 to the learning rate with each step (this approximately halves the learning rate over the course of 50 epochs with our batch size of 128).' We do not employ normalization layers in any of our models in order to avoid the possibility of information being exchanged nonlocally throughout the image. We do not use weight decay. We sample images from the theoretical processes (IS/L.S/ELS) using 20-step discretization of the reverse process, using the analytic form prescribed for DDIM-style models in (Song et al., 2020a). We sample from the CNN models using 20-step and 150-step discretizations, and report the respective results in tables 2 and ??. As evidenced, we find that the 150-step discretization outputs better match the theoretical outputs than the 20-step discretization; no theoretical explanation for this has been suggested. We use cosine noise schedule (Nichol & Dhariwal, 2021) for each experiment.
We evaluate each architecture using sero-padded convolutions on the following datasets: MNIST, FashionMNIST, CIFARIO, and CelebA. For each of the latter datasets, we use class-conditioning. We only train uncondiacnii models on MNIST and CelebA. In addition, we evaluate our ResNet architecture using circularly-padded convolutions on CIFARIO (class-conditional) and MNIST (class-nnconditiona)).
For each neural network each dataset, we calibrate associated multiscale LS model and an associated multiscale ELS model of the network using the procedure described in C.2. The (E)LS model inherits the conditioali of the neural network it is modeling. We then compute the outputs of each neural network on 100 distinct random noise inputs for each dataset, drawn iid from an isotropic normalized Gaussian distribution. For class-conditional models, we additionally sample a label for each seed. We compute the outputs of the corresponding ELS machine on the same seeds/labels. For each example, we compute the pixelwise 7² between the (E)LS machine outputs and the network output. We also compute the outputs of the IS machine across all of the same inputs/labels, and compute the pixelwise y² between this baseline and the neural network outputs. For all except the CelebA UNet we find that the ELS machine is the best predictor; for this case, we find that the LS machine is significantly more predictive, indicating that the model is able to use positional information in the interior of the image. We report the median 22 value across the 100 samples for all configurations in table 2, and plot the distribution of ELS correlations/IS correlations in figures 10 and 11 (LS/IS correlations for the CelebA UNet).
We also repeat this analysis procedure for a pretrained 2024), with the exception that we re-use the scales calibrated for the CIFARIO ResNet model rather than re-estimating them for the Salf-AAttetioo-enablle model in order to minimize bias.
-nableed UNet trained on CIFARIO from (Sehwag,
C.2. Identifying multiscale behavior
In order correctly recapitulate the behavior of the models we study, we need account for a crucial empirical observation: convolutional diffusion models exhibit time-dependent effective receptive field sizes. This behavior is illustrated in figure 4. In the left panel, we display an average absolute value of the gradient of the center pixel of the model outputs from two of our CIFARIO 10-trained diffusion models, with respect to the input image. We plot this at various time steps in the reverse process (with the center pixel omitted for visual clarity). This visualization highlights which areas of the image the center pixel's outputs are sensitive to, an indicator of the degree of locality in the model's output. At T = 1.0 (corresponding to an input of initial white noise), the average gradient spans a large range (for the ResNet, a range clearly constrained by its maximal receptive field size). As the noise level reduces throughout the reverse process, the width of the heatmap decreases, until at the last time step the heatmap is almost entirely concentrated in single (3 x 3) square.
To calibrate the time-dependent locality scale of our theoretical model, we compute the reverse trajectories under the CNN-parameterizee neural networks for random validation set. At each time step, we compare the predictions of the model for the added noise and the outputs of ELS machines with a range of scales via cosine similarity. We then pick the representative scale for each time step by picking the median optimal scale across the range of samples. The resulting calibrated scales are shown in the middle panel of figure 4. We see that the UNet better described by larger-scale ELS machine early in the reverse process than its ResNet counterparts, phenomenon that can probably be linked the
Code for the following experiments hosted at
diffusion
21

---

An analytic theory of creativity in convolutional diffusion models
more stringent locality constraints in the latter model. However, as the reverse process continues, both models prefer monotonically smaller scales, until converging the smallest scale (3 for the final few denoising steps. These results are in accordance with the visual evidence from the gradient heatmaps in figure 4.
At this stage we have no a-priori method for predicting the scales that the models choose to use at each time step. However, the general phenomenon where the model initially starts with a large field of view and decreases over the course of the reverse process could be anticipated on general grounds. As the noise variance decreases, the noised training distribution separates into a multimodal distribution with larger and larger number of modes; as + the number of modes converges the (very large!) number of training examples. Since the models we consider are equivariant, an optimal model must also in principle represent not just the modes corresponding to the training set, but also every translated augmentation, which for a 32 32 image results in 1024-fold increase in effective dataset size! However, the emergence of multimodality is delayed when considering only the marginal distributions with respect to a smaller scale, as there are fewer dimensions via which two distinct data points could be distinguished from each other. This suggests that the model may somehow be picking the largest scale that it can a) represent within its receptive field (a constraint more pertinent to the ResNet, which has smaller maximum receptive field size) and b) for which can represent the local/equívariant approximation the score function reasonably parameter efficient way, i.c. for which it need model too many independent modes of the data distribution. However, more work needs to be done in order understand this phenomenon.
D. Results
22

---

An analytic theory of creativity in convolutional diffusion models
Model/Theary Correlation for MNIST ResNett
Model/Theery Correlation for MNIST UNet
P.ELS Rarmin
0.6
0.0
0.0
0.0
0.2
0.4
0.6
0.8
1.0
0.0
0.2
0.6
L0
Machine
Machhnne
Model/Theory Comelation for FashionMNIST ReaNat
Modol//heor Corrolation FashienMNIST UNot
0.8
Mochine
06
ELS Moceun
04
0.0
0.0
0.6
1.0
0.6
P. Machine
P Mochire
Model/Theory Correlation for CIFAR10 ResNet
Model/Theory Correlation for CIFAR10 UNet.
Rockine
ittecine
0.0
0.2
0.4
0.6
0.8
100
0.0
0.2
0.4
0.6
0.8
A.
Mode//hhory ationlation
Madel/Theary Correlation ColobA
1.0
0.8
Rachina
H F.ELS Martine
0.6
0.5
0.4
0.4
0.2
0.2
0.0
0.0
A. Mochine
A. Mochine
Figure 10. Correlations between model outputs and (E)LS machine/IS baseline on cach datasct for zero- models. LS machine is used for the CelebA UNet, ELS machine for other models. axis (E)LS machine 2 axis baseline , for each data point the sample. The (E)LS machine uniformly outperforms the memorizing baseline.
23

---

An analytic theory of creativity in convolutional diffusion models
Modei/hheory Correlation for MNIST ResNet
Model/Theary Correlation for CIFAR10 ResNet
A.ELSaariie
0.6
0.0
0.2
0.4
0.6
0.8
0.0
Machine
e7, Machine
Figure 11. Correlations between model outputs and ELS machine/IS baseline each dataset for circularly-padded models. Y axis is ELS machine r², X axis is IS baseline y² for each data point the sample. The ELS machine uniformly outperforms the baseline. The performance of the ELS machine circular MNIST anomalously lower than other configurations, but the degree of outperformance of the ideal score baseline is higher.
Model/Theary Corrolation for
0.8
RLS
0.4
0.0
0.2
0.4
3.6
0.81
1.0
Figure Correlations between model outputs and ELS machine/IS baseline CIFARI0 for pretrained Attention-enabled UNet. axis is ELS machine r. X axis is IS baseline 2 for each data point in the sample.
24

---

An analytic theory of creativity in convolutional diffusion models
Table A summary of the experimental results the paper for different datasets and model configurations for each architecture across each datasct, with 20 steps in the thcoretical reverse process and 150 steps in the neural network reverse process. Pixelwise rd between theory and neural network images are computed using 100 image samples per configuration; the median across the sample is reported. We compare these results baseline consisting of the correlations of the model with the outputs of ideal score (IS) model, which always outputs memorized training examples. We also report the percentage of samples on which the EL. machine outperforms the output from the ideal score-matching diffusion model.
Dataset
Arch.
Padding
Conditional
ELS Corr.
LS Corr.
IS Corr.
(E)LS > IS %
MNIST
UNet
Zeros
X
0.89
0.88
0.70
0.93
CIFAR1O
UNet
Zeros
0.90
0.87
0.41
0.92
FNSHionMNIST
UNet
Zeros
0.93
0.93
0.80
1.00
CelebA
UNet
Zeros
0.85
0.90
0.55
1.00
MNIST
ResNet
Zeros
X
0.94
0.82
0.61
1.00
MNIST
ResNet
Circular
X
0.77
0.36
0.15
0.92
CIFAR10
ResNet
Zeros
0.95
0.90
0.42
1.00
CIFARIO
ResNet
Circular
0.94
0.83
0.35
1.00
FaShionMNNIST
ResNet
Zeros
0.94
0.88
0.68
1.00
CelebA
ResNet
Zeros
X
0.96
0.90
0.47
1.00
CIFAR1O
UNet SA
Zeros
0.77
0.77
0.48
0.95
Table summary of the experimental results of the paper for different datasets and model configurations for each architecture across each dataset, with 20 steps in the theoretical reverse process and 20 steps in the neural network reverse process.
Dataset MNIST
Arch.
Padding
Conditional
ELS Corr.
LS Corr.
IS Corr.
(E)LS IS
UNet
Zeros
0.84
0.83
0.69
0.92
CIFARIO
UNet
Zeros
0.82
0.80
0.39
0.99
FashionMNISST
UNet
Zeros
0.91
0.91
0.80
0.90
CelebA
UNet
Zeros
X
0.75
0.81
0.51
1.00
MNIST
ResNet
Zeros
0.94
0.84
0.62
0.97
MNIST
ResNet
Circular
0.73
0.33
0.14
0.99
CIFAR10
ResNet
Zeros
V
0.90
0.86
0.43
1.00
CIFARIO
ResNet
Circular
0.90
0.80
0.36
1.00
FaShionMNIIST
ResNet
Zeros
0.90
0.88
0.74
0.97
CelebA
ResNet
Zeros
X
0.92
0.88
0.48
1.00
CIFAR10
UNet SA
Zeros
0.75
0.74
0.47
0.92
25

---

An analytic theory of creativity in convolutional diffusion models
9
7
5
77
22
3
f<
f
Y
7
2
B
2
}
L
2
L
8
5
2
>
4
>
4
3/
NF
1
X
7
7
Z
7
4
f
R
S5
7
7
R
e
J
7
7
2
]
3
3
X
7
J<
1
7
?.1
R
7
B
+
25
3
/
1
1
A
E
3
4
G
4
4
7
7
P
7
L
H
+
LS
55
0
0
4
1r
2
2
4
4
/
A
25
7
>/
7
f
Figure 13.. Further comparison between ResNet (right columns) and ELS Machine (left columns) samples for MNIST. Model is uncondi- tional and has zero padding.
26

---

An analytic theory of creativity in convolutional diffusion models
9
9
6
+
/
4
2
6
2
6
Q
L1
2
7
6
6
7
7
0
U
B
2
2
7
0
7
8
5
4
9
3
5
Q
Q
2
6
9
3
E
6
}
7
7
R
2
7
7
4
P
R
2
Q
4
6
+
2
2
2
3
<
7
2
7
7
3
2
7
771
4
4
/
//
2
7
S
3
9
3
K
9
9
4
4
2
7
5
5
4
2
G
4
6
7
7
S
9
3
7
7
5
7
7
9
9
45
4
0
4
3
24
4
4
4
4
6
3
R
7
5
S
6
6
P
D
7
7
3
1
9
9
E1
E
1
4
Figure 14. Further comparison between UNet (right columns) and ELS machine (left columns) samples for MNIST. Model is unconditional and has zero padding.
27

---

An analytic theory of creativity in convolutional diffusion models
Figure 15. Further comparison between ResNet (right columns) and ELS Machine (left columns) samples for FashionMNIST. Model class conditional and has zero padding.
28

---

An analytic theory of creativity in convolutional diffusion models
Figure 16. Further comparison between UNet (right columns) and ELS machine (left columns) samples for FashionMNISTT. Model class conditional and has zero padding.
29

---

An analytic theory of creativity in convolutional diffusion models
Figure 17. Further comparison between ResNet (right columns) and EL.S machine (left columns) samples for CIFAR1O. Model is class conditional and uses zero padding.
30

---

An analytic theory of creativity in convolutional diffusion models
Figure 18. Further comparison between UNet (right columns) and ELS machine (left columns). Model is class conditional and has padding.
31

---

An analytic theory of creativity in convolutional diffusion models
Figure 19. Further comparison between UNet (right columns) and ELS machine (left columns) CelebA32x32. Model is class conditional and has zero padding. Model samples use 150 timesteps, ELS samples use 20.
32

---

An analytic theory of creativity in convolutional diffusion models
Figure 20. Further comparison between UNet (right columns) and ELS machine (left columns) CelebA32x32. Model is class conditional and has zero padding. Model samples use 150 timesteps, ELS samples use 20.
33

---

An analytic theory of creativity in convolutional diffusion models
S
1
2
7
5
7
7
2
2
Figure 21. Further comparison between ResNet (right columns) and ELS machine (left columns) MNIST. Model is unconditional and has circular padding.
34

---

An analytic theory of creativity in convolutional diffusion models
Figure 22. Further comparison between ResNet (right columns) and ELS machine (left columns) samples CIFAR1O. Model is class conditional and has circular padding.
35

---

An analytic theory of creativity in convolutional diffusion models
(a) Incoherent output
(b) Non-matching output
Figure 23. ELS Machine (left) dnention-enables UNet (right) pairs. Panel (a) shows outputs where the Attentive UNet produces semantically incoherent, uninterpretable outputs, which tend to match strongly with the ELS Machine outputs. Panel (b) shows examples where the Attentive UNet produces samples not obviously matched with the ELS machine.
36

---

An analytic theory of creativity in convolutional diffusion models
Figure 24, Further comparison between attention-enabled UNet (right columns) and ELS machine (left columns) samples CIFARIO.
Model is class conditional and has zero padding.
37


Reviews:
[{"review_id": "H3Hf0FxRNB", "reviewer": "Reviewer_svst", "summary": "This paper presents an analytic theory of generalization in convolutional diffusion models. It identifies that, given a finite empirical dataset, the optimal score function produces a perfect reverse diffusion process, leading to replicas of training samples. The paper then hypothesizes that the creativity of real trained diffusion models arises from the inductive biases of parametric models (e.g., convolutional neural networks). Assuming equivalence and locality properties of the score function in image generation, the authors derive an analytic score model. This enables a case-by-case study of parametric score models (e.g., CNNs, ResNets), showing impressive correlation with actual trained models. This work provides a transparent interpretation of how diffusion models generate novel images.", "questions": "", "limitations": "", "rating": 5, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We appreciate the reviewer’s feedback and the thoughtful suggestions that they made for our paper. Below we address some of these comments and suggestions.\n\n>Perhaps one additional aspect could be explored: in real scenarios, two factors influence the final generation of diffusion models—the inductive biases inherent in the model family (e.g., U-Net) and those introduced by training dynamics (e.g., initialization, optimizers). I wonder whether there is a non-monotonic trend in r2 during the training process, which could provide insight into how optimization choices introduce implicit inductive biases.\n\nIn response to the reviewer’s questions about the importance of training dynamics and initialization in determining model outputs, we retrained our models several times with identical data but different initial seeds, and evaluated the dynamics of the r2 metric with the ELS outputs.  \n\nWe found that the resulting generated outputs did not significantly vary between initialization seeds, with the median CNN-CNN pixelwise r2 between different post-training models achieving 0.9-0.97 on different datasets and architectures. We found that we could improve this somewhat by continuing training further and continuing to anneal the learning rate; e.g., training 600 epochs instead of 300 while continuing to decay the learning rate produced correlations typically > 0.95.\n\nWe found that the ELS/CNN r2 was somewhat oscillatory over short time ranges (if learning rates remained too high near the end of training), but the overall trend was monotonically upwards across the training process. The oscillations we attributed to an instability in the overall image intensities of the model-generated output during the training process, which eventually disappeared as we annealed the learning rate appropriately. This dynamical effect did not seem to affect the output of the optimization process provided the learning rate was decayed sufficiently over the course of the training process. We intend to defer a more detailed study to further work.  But overall there is no strong indication that early stopping consistently helps with achieving a higher r^2 for these datasets.\n\n>The method for determining patch sizes (e.g., Figure 4) appears somewhat heuristic. Did you attempt to derive it theoretically or formulate it as an optimization problem (e.g., minimizing the score-matching loss through cross-validation)? This could eliminate the need to rely on studying the saliency map of a pretrained model.\n\nWhile we considered alternative options, we observed in our work (as shown in figure 4b) that the optimal patch size differed between the UNet model and the ResNet models, which showed that the optimal patch size for each model could not be an intrinsic statistical characteristic of the dataset and therefore could not be completely deduced through a process that did not use additional characteristics of the post-training models.\n\nWe would like to emphasize that the approach that we took to calibrate our theory to the model did not actually use the saliency maps of the post-training models, which were shown in figure (4a) primarily as ancillary evidence for the observed multi-scale behavior. Rather, we performed a direct fit for each time by selecting the maximally performant ELS scale at each time step on a separate validation set.  Performance was defined as the correlation between the ELS machine’s predicted score function at each time, and the UNet or ResNet’s score function.  We chose the patch-size scale of the ELS machine to maximize this correlation at each separate time. Details can be found in our Appendix. We hope this constitutes a principled way to calibrate the time-dependent scale of the ELS machine.", "claims_and_evidence": "The claims in this paper are clear and supported by theoretical derivations as well as convincing empirical evidence.", "methods_and_evaluation_criteria": "The evaluation primarily rely on $r\\^2$, as the main goal of this paper is an analytic interpretation of generation in diffusion models. The $r\\^2$ metric along with visual comparisons, show strong correlation between the theory-predicted results and actual generations from trained parametric diffusion models.", "theoretical_claims": "Yes, I checked most of the proofs (e.g., the derivation of the ELS machine), which appear correct.", "experimental_designs_or_analyses": "The experimental design and analyses are sound. Perhaps one additional aspect could be explored: in real scenarios, two factors influence the final generation of diffusion models—the inductive biases inherent in the model family (e.g., U-Net) and those introduced by training dynamics (e.g., initialization, optimizers). I wonder whether there is a non-monotonic trend in $r\\^2$ during the training process, which could provide insight into how optimization choices introduce implicit inductive biases.", "relation_to_broader_scientific_literature": "This work focuses on the generalization of generative models, which is a core topic in generative AI.", "essential_references_not_discussed": "Several existing works share the intuition of memorization in ideal score functions:\n\n- Gu, Xiangming, et al. \"On memorization in diffusion models.\" arXiv preprint arXiv:2310.02664 (2023).\n- Somepalli, Gowthami, et al. \"Diffusion art or digital forgery? investigating data replication in diffusion models.\" Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2023.", "other_strengths_and_weaknesses": "This work is particularly interesting as it systematically investigates the failure of the ideal score function and studies the problem from a principled perspective. It also suggests the possibility of nonparametric generative models for high-dimensional data—provided we identify the correct inductive biases. The paper is well-written and convincing, with well-designed notations (although not consistent with common conventions).\n\nOne weakness is that while the theory predicts generation to some extent, there remains a clear quality gap. It is unclear whether an (approximate) theoretical solution is still feasible when considering more complex inductive biases, such as attention mechanisms.", "other_comments_or_suggestions": "Typos:\n- In Eq. (5): $\\\\sqrt{\\\\bar{\\\\alpha}\\_t}\\\\varphi(x)-\\phi$ should be $\\\\sqrt{\\\\bar{\\\\alpha}\\_t}\\\\varphi(x)-\\phi(x)$.", "questions_for_authors": "The method for determining patch sizes (e.g., Figure 4) appears somewhat heuristic. Did you attempt to derive it theoretically or formulate it as an optimization problem (e.g., minimizing the score-matching loss through cross-validation)? This could eliminate the need to rely on studying the saliency map of a pretrained model.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ybMj8xSYOl", "reviewer": "Reviewer_TYVX", "summary": "This paper develops a theory for why convolutional diffusion models fail to learn the ideal score function. It is theorized that this is due to locality from small receptive fields and translational equivariance. Under these assumptions, an optimal minimum MSE approximation to the ideal score function is derived subject to locality and broken translational equivariance constraints. The so-called equivariant local score machine shows that convolutional diffusion models compose different patches from training examples together, which the papers calls a locally consistent patch mosaic. There is very high quantitative (r^2 values) agreement between outputs from convolutional diffusion models and the ELS machine, suggesting that the ELS provides a plausible explanation for the mechanisms of convolutional diffusion models. Some of the theory also holds up when self-attention is introduced.\n\n## Update after rebuttal\nI maintain my score of strong accept. I thank the authors for their effort in addressing my comments, especially the Celeb-A experiments.", "questions": "", "limitations": "", "rating": 5, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their detailed feedback and suggestions. We address some of these below:\n\n>How important is each constraint in the analytic solution? […] Is it possible to ablate each constraint and observe the correlation with the diffusion samples?\n\nAn equivariant score machine on its own will only generate memorized training examples, similar to the ideal score machine, but translated a random amount. This is highly discrepant from the observed data, especially in the presence of boundary conditions that fix the spatial position of at least the edges of the image. Due to time constraints we have not yet performed the ablation of locality; however, we expect a priori for the reason outlined before that it will perform uniformly lower across all categories than even the ideal-score baseline.\n\nTo address the part of the reviewer’s question about the performance of the locality constraint without equivariance, we discuss several aspects. Firstly, fully-equivariant circularly-padded models display features clearly inconsistent with patches knowing their position. This is shown clearly in the samples from the circularly-padded ResNet on MNIST, which include line segments that intersect the boundary, a feature that is not present in any training example in MNIST and cannot be elicited from the LS Machine.\n\nSecondly, we evaluated the correlation between the outputs of each model we studied in the paper and the outputs of a corresponding LS Machine. The quantitative results of these results are summarized in the table below:\n\n|Model|LS/CNN r^2|ELS/CNN r^2|\n|---|---|---|\n|MNIST/UNet/Zeros|0.83|0.84|\n|CIFAR10/UNet/Zeros|0.80|0.82|\n|FashionMNIST/UNet/Zeros|0.91|0.91|\n|MNIST/ResNet/Zeros|0.84|0.94|\n|MNIST/ResNet/Circular|0.33|0.77|\n|CIFAR10/ResNet/Zeros|0.86|0.90|\n|CIFAR10/ResNet/Circular|0.80|0.90|\n|FashionMNIST/ResNet/Zeros|0.88|0.90|\n|CIFAR10/UNet+SA|0.74|0.75|\n\nWe find that the ELS machine uniformly outperforms the LS machine for ResNets, and performs above or at par with the LS machine for UNets, but the discrepancy is small for zero-padded models, indicating that locality is the main factor. Qualitatively, however, the LS machine samples are \"grainy\" and the ELS samples are visually much better, which is not reflected in this metric.\n\nWe found one outcome, in response to the suggestion that we study CelebA, which did not follow the trend described above. We describe the results below. \n\n>If possible, I think applying the ELS machine on CelebA-64x64 (or even 32x32 and grayscale) can produce very convincing results.\n\nWe strongly thank the reviewer for this suggestion. To address it, we trained a ResNet and a UNet model on CelebA-32x32 grayscale. Our analysis is not yet complete due to the large computational expense required for the calibration and generation process, but we have been able to perform a preliminary analysis of the LS Machine and ELS Machine, using a) a reduced dataset and b) manually calibrated scales. In these experiments, we have so far found the following.\n\n1) The ELS machine matches the ResNet model, with median r^2 ~ 0.96. However, qualitatively, the samples produced by the ResNet are of poor quality, similarly to CIFAR10.\n2) The UNet model trained on this dataset is able to produce recognizable faces. We found, to our surprise, that it appears to be better fit by the *LS Machine* (r^2 ~ 0.92) as opposed to the ELS Machine (r^2 ~ 0.90). The LS Machine captures the placement of noise, eyes, and mouth in the image; these details are not captured by the ELS machine, which generates similar images overall but which lacks these key human-interpretable features.\n\nWe believe that the explanation for this behavior is as follows. The ResNet is shallow and has a *hard* locality constraint. The UNet receptive field size is formally very large and includes the image border everywhere; the fact that it exhibits local and equivariant behavior is an *emergent phenomenon* that requires pixels to use less information than maximally possible. Restoring positionality while preserving locality is akin to reintroducing some but not all of the information that is discarded.\nMore experiments will be needed to study this capability, but we believe that this will not be fully achievable within the timeframe of the revisions and defer this to future work. However, we intend to incorporate the additional findings on CelebA into our paper in the final revision.\n \n>I think Fig. 1 can be further improved by including a column of the input noise. \n\nWe could in principle pick images for the figure such that the same initial noises could be used for black and white images; however, CIFAR10 images have 3 channels, while FMNIST and MNIST have only one, so we would always need at least two additional columns of noise. We feel that the addition of these multiple columns would dilute the visual impact of the initial figure; thus, if the reviewer agrees, we would like to retain the figure design for figure 1.", "claims_and_evidence": "The claims are strongly supported by theory and empirical results, showing strong agreement between outputs from the ELS machine and from convolutional diffusion models.", "methods_and_evaluation_criteria": "The proposed methods and evaluation criteria make sense. r^2 is measured between the outputs from the ELS machine and convolutional diffusion models to measure agreement.", "theoretical_claims": "The theoretical claims seem to be correct. Ultimately, the empirical results show that the derived analytic solution closely matches with the true outputs from convolutional diffusion models.", "experimental_designs_or_analyses": "The experimental designs and analysis are sound. The analysis of spatial inconsistencies from excess late-time locality is especially insightful, providing an explanation for the well-known phenomenon that diffusion models struggle in generating limbs.", "relation_to_broader_scientific_literature": "The paper gives care to discussing previous work that shows that the ideal diffusion model should memorize its data. This provides a nice setup for the contributions of this paper, which explain why convolutional diffusion models don't in fact memorize their data, and essentially compose patches from different training examples.", "essential_references_not_discussed": "The paper has done due diligence to cite related works such as Kadkhodaie et al., 2023a and concurrent work (Niedoba et al., 2024). So, I do not see any issues in missing essential references.", "other_strengths_and_weaknesses": "**Strengths**\n* I think this is a very strong paper that can have big impact since it provides a theory for why convolutional diffusion models can generalize. This has implications for data attribution, fixing artifacts from diffusion model samples, improving adherence to conditioning, etc. The paper is also very well-written. As a non-theorist, the development of the ELS machine was very intuitive, and I appreciated the illustrations such as Figure 3 to build and intuition. I provide some small suggestions below. \n\n**Suggestions**\n* How important is each constraint in the analytic solution? It is mentioned in the paper that the equivariance constrained machine can only \"generate training set images globally translated to any other location.\" Is it possible to ablate each constraint and observe the correlation with the diffusion samples?\n* I think Fig. 1 can be further improved by including a column of the input noise. This will emphasize the message that the analytic theory predicts almost the same output as a convolutional diffusion model given the same input noise. \n* I understand that the theory can only mainly explain toy, small-scale settings such as MNIST, CIFAR, Fashion-MNIST. But there is not much compositionality in these datasets. CIFAR is a difficult dataset to generate cohesive images, so it is hard to see any \"creativity\" since it is difficult to see any semantics in the images. If possible, I think applying the ELS machine on CelebA-64x64 (or even 32x32 and grayscale) can produce very convincing results. It is a fairly homogeneous dataset, so fitting a convolutional diffusion model should not be difficult. Also, there is a lot of room for compositionality since facial expressions (e.g., smile) or accessories (e.g., glasses) can be patches that are combined with other patches to demonstrate \"creativity.\" I think this would make it more convincing beyond the theory community.", "other_comments_or_suggestions": "In the Supplementary Section D. Samples, the page is blank followed by samples in the next page.", "questions_for_authors": "Please see **Suggestions** above.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "3lpcZ313bX", "reviewer": "Reviewer_CGco", "summary": "This work proposes that biases of translation-equivariance and locality are sufficient to explain novel image generation in fully convolutional diffusion generative models. It does so by showing that a closed-form score model subject to those constraints qualitatively recapitulates the images generated by trained CNNs.\n\n## Update after rebuttal\n\nThe authors' rebuttal answers my questions and concerns, and I remain strongly in favor of acceptance.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their highly detailed feedback and insightful suggestions. Below, we carefully address several of the points that they raised.\n\n>I would encourage the authors to reconsider their use of the term \"creative\", as I think \"novel\" or \"original\" would be somewhat more precise, and would be less burdened with humanistic baggage […] However, this is at least partially a matter of taste, and I leave the decision to the authors' discretion.\n\nWe appreciate the reviewer’s concern about this choice of nomenclature. While a matter of taste, we did arrive at the term by trying to encapsulate a particular technical phenomenon; roughly, “originality subject to constraints embedded in the data.” We felt that it was important to distinguish this behavior from the trivial originality type of e.g. a white noise sampler. In order to make this case clearer, we will add language in our revisions that makes more explicit the particular characteristics that we are attempting to capture with this word choice.\n\nThe term that we felt was otherwise nearest to the concept we were trying to describe is “generalization.” While a less humanistic word choice, we felt that this term too had baggage that we were not totally comfortable with. Generalization is often used to mean something along the lines of “correct generalization” or the gap between training and test set performance. It was not clear to us that this was the right notion for the phenomena in our paper (e.g. the outputs of Figure 5 are \"incorrect\"); as such, we elected to sidestep this by instead using a nontechnical term.\n\n>One concern is that all quantitative comparisons use pixelwise r2, and no clear justification is provided for this choice.\n\nThe reasoning behind the choice of the r2 metric is as follows. The task we are trying to benchmark is a paired image comparison task, and thus taking the median of some image distance metric across the sample set seemed appropriate. Many metrics commonly used in the generative image model literature, such as FID, are *distributional* distance metrics and so we did not consider them as they were too weak for our purpose. Distances between pairs of images in the underlying pixel space seemed both most natural to us and also the most stringent possible test for our model. We considered using the L2 distance directly, however, this metric is sensitive to the overall scale of the image distribution; since we wanted a metric that would allow us to compare the performance across different datasets that have different image statistics, the metric should be invariant to overall scale. The r2 metric is a standard metric which satisfies these criteria. We also felt that the 0-1 scores that the r2 metric assigns were more intuitively understandable than the raw l2 scores.\n\nAnother alternative metric choice that satisfied the criteria outlined above was cosine similarity. This metric is very similar to the r2 metric, and the results are given below:\n\n|Model|ELS/CNN Cos|\n|---|---|\n|MNIST/UNet|0.93|\n|CIFAR10/UNet|0.82|\n|FMNIST/UNet|0.88|\n|MNIST/ResNet/Zeros|0.97|\n|MNIST/ResNet/Circular|0.82|\n|CIFAR10/ResNet/Zeros|0.89|\n|CIFAR10/ResNet/Circular|0.90|\n|FMNIST/ResNet/Zeros|0.92|\n|CIFAR10/UNet+SA|0.84|\n\nUltimately, rather than presenting two separate but highly related metrics, we made the choice to use the r2 metric alone.\n\n>The finding that imposing circular boundary conditions leads to more texture-like generated images is interesting. Can you provide any intuition for why that is? Does it arise purely through interactions between the artificially-joined boundary patches?\n\nThe intuition for the texture-like behavior is that, without any global positional information, local models are not able to robustly coordinate their generation in order to produce coherent images. Roughly, each independent part of the denoised image spontaneously decides to resemble a totally random location in a training image; this naturally produces a bit of a jumble. Adding borders helps coordinate this generation process by ‘pinning down’ the boundary. This helps especially for datasets such as MNIST and FashionMNIST, where the boundary is very regular (a stereotyped black background).\n\n>What happened in the ELS-generated image in the eigth row of the second column of Figure 17? \n\nThis was a plotting bounds issue; the image should appear visually monochromatically black and has been fixed. \n\n>This is something that could be left to future work, but I am curious if the ELS model could recapitulate the results of the forward-backward experiments performed by Sclocchi, Favero, & Wyart.\n\nWe thank the reviewer for this interesting suggestion. We suspect that there may be a connection, and in particular we are curious in particular about whether the ELS machine framework might shed light on what the “natural” analogue of the hierarchy of variables that they study might be in real datasets. However, we will leave detailed investigation up to future work.", "claims_and_evidence": "The claims are largely supported by convincing evidence. Given the broad interest in diffusion generative models, it goes without saying that this paper should be of substantial interest to the ICML audience, and may well be influential. \n\nI would encourage the authors to reconsider their use of the term \"creative\", as I think \"novel\" or \"original\" would be somewhat more precise, and would be less burdened with humanistic baggage. For instance, the paper could alternatively be titled \"An analytic theory of novel image generation in convolutional diffusion models\". However, this is at least partially a matter of taste, and I leave the decision to the authors' discretion.", "methods_and_evaluation_criteria": "The methodology seems sound.", "theoretical_claims": "Most of the derivations in this paper are quite straightforward. The main theoretical result is Theorem 4.1; its proof appears correct.", "experimental_designs_or_analyses": "The experiments are largely well-designed. One concern is that all quantitative comparisons use pixelwise $r^2$, and no clear justification is provided for this choice.", "relation_to_broader_scientific_literature": "This paper is related to the broader literature through the position of the literature on diffusion models within the broader machine learning field.", "essential_references_not_discussed": "I think the authors do a good job addressing relevant literature on theories for how diffusion models generate novel images; no omissions stood out.", "other_strengths_and_weaknesses": "I think the authors should state more clearly in the Introduction that the excellent predictions resulting from their closed-form model require calibration of the time-dependent patch scale based on the quality of the predictions. I don't think this is a substantial limitation - after all this is only a few parameters - but I do think it's important to mention clearly, if only to highlight that the factors determining the schedule of scale decreases is an interesting topic for future work.", "other_comments_or_suggestions": "- There are a number of in-text citations for which \\citet should be used in place of \\citep, e.g. when citing Kadkhodaie et al. in Line 98. \n\n- Please use a 1:1 aspect ratio in Figures 8-10. \n\n- It would be helpful if the authors swapped the column ordering of Figure 19 so that the ELS machine is on the right like in other figures.", "questions_for_authors": "- The finding that imposing circular boundary conditions leads to more texture-like generated images is interesting. Can you provide any intuition for why that is? Does it arise purely through interactions between the artificially-joined boundary patches? \n\n- What happened in the ELS-generated image in the eigth row of the second column of Figure 17? Here the model has produced an image very different from the convolutional model, and indeed one that lacks the expected localized features. \n\n- This is something that could be left to future work, but I am curious if the ELS model could recapitulate the results of the forward-backward experiments performed by Sclocchi, Favero, & Wyart. In particular, could some of the transition points they detect be linked to changes in the patch scale?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "qzFIfjUfbL", "reviewer": "Reviewer_qd8x", "summary": "The paper proposes a formula to predict images generated by convolutional diffusion models. The analysis suggests that biases in convolutional neural networks—such as locality and translational equivariance—prevent diffusion models from learning a perfect score function, encouraging them to generate samples that were not present in the training data.", "questions": "", "limitations": "", "rating": 5, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for their comments and suggestions. Below, we hope to carefully address a few of their concerns:\n\n>The evidence is promising in the cases of MNIST and FashionMNIST. However, the evaluation based on CIFAR-10 is less conclusive. This is still acceptable because the ResNet-based diffusion model itself still struggles to generate high-quality images.\n\nWe would like to emphasize that the primary objective of our paper is to find a predictive theory for small convolutional neural networks, rather than a predictive theory of high performance networks which require more elements like transformers. While it is true that these networks are limited in their performance, particularly on CIFAR-10 (as the reviewer notes), this is a feature of the models we are attempting to study, and a theoretical model that obtained higher performance would not be an accurate description of the model class. Predicting the defects of models is in fact part of the objective of our paper-- for instance, we are interested in understanding the origin of common spatial consistency problems in diffusion models, as shown e.g. in figure 4.\n\n>r^2 measures how well the predicted outputs (ELS machine) correlate with the actual outputs (diffusion model). This is pixel-level measurement. In the image generation, there are multiple choices of metrics that can be used along with pixel-level metrics, such as FID and Inception Score. The only concern is this work does not include these metrics.\n\nWhile FID and Inception Score are commonly used metrics, they are measures of distributional similarity between large sets of unpaired images. However, the primary task in our paper is to measure the similarity between individual pairs of images, and thus FID and Inception Score are not appropriate for the task.  Indeed any *distributional* distance metrics is too weak for our purpose: we don’t just want to show that the ELS machine and the trained UNet/Resnet sample from similar distributions; we want to show a much stronger result that for every single noise realization, the particular image generated by the ELS machine, and the corresponding paired image generated by the UNet/ResNet is very similar. \n\nFeature-wise distances might be considered, but there is less of a standard consensus on the features to use for small black and white datasets such as MNIST and FashionMNIST (InceptionV3, from which FID is derived, was trained on ImageNet, whose statistics do not resemble the statistics of MNIST and thus whose features are unlikely to be useful for capturing the features of that dataset). Rather than look for a nonstandard feature space, we felt it was better to stick with a metric in the underlying pixel space, where the resulting numbers would be clearly interpretable.\n\nThere are other pixelwise distance metrics that are standard in the literature that could be considered, such as L2 distance and cosine similarity. The former is not invariant to rescaling, and since we wanted to have a comparison between the performance of the ELS machine on models trained on image datasets with different statistics, we felt it was better to choose a scale-invariant metric such as r^2 or cosine similarity. These two metrics are very similar quantitatively, and we felt it would be redundant to include both. The choice of r^2 does not privilege our model; the values of the median cosine similarities for various model configurations are presented below for comparison:\n\n| Model                       | ELS/CNN Cosine Similarity |\n|-----------------------------|---------------------------|\n| MNIST/UNet/Zeros            | 0.93                      |\n| CIFAR10/UNet/Zeros          | 0.82                      |\n| FashionMNIST/UNet/Zeros     | 0.88                      |\n| MNIST/ResNet/Zeros          | 0.97                      |\n| MNIST/ResNet/Circular       | 0.82                      |\n| CIFAR10/ResNet/Zeros        | 0.89                      |\n| CIFAR10/ResNet/Circular     | 0.90                      |\n| FashionMNIST/ResNet/Zeros   | 0.92                      |\n| CIFAR10/UNet+SA             | 0.84                      |", "claims_and_evidence": "The evidence is promising in the cases of MNIST and FashionMNIST. However, the evaluation based on CIFAR-10 is less conclusive. This is still acceptable because the ResNet-based diffusion model itself still struggles to generate high-quality images.", "methods_and_evaluation_criteria": "r^2 measures how well the predicted outputs (ELS machine) correlate with the actual outputs (diffusion model).\nThis is pixel-level measurement. In the image generation, there are multiple choices of metrics that can be used along with pixel-level metrics, such as FID and Inception Score. The only concern is this work does not include these metrics.", "theoretical_claims": "Although I did not go into the details of proofs, the high-level idea and motivation sounds promising.", "experimental_designs_or_analyses": "Experimental design is straight-forward so there is no issue with it.", "relation_to_broader_scientific_literature": "There are public debates about whether diffusion models copy art from humans.\nThis work theoretically demonstrates that diffusion models can be creative and generate something no human has seen before. This work is important in relation to the broader scientific literature, as it has the potential to extend the framework to other generative AI models, such as large language models (LLMs).", "essential_references_not_discussed": "-", "other_strengths_and_weaknesses": "I cannot think of the weakness.", "other_comments_or_suggestions": "-", "questions_for_authors": "No question", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: The paper received enthusiastic evaluations from all reviewers. Reviewers appreciate the design of experiments, theoretical derivations, and clear presentation. One of the reviewers suggested replacing the term creativity with another (more technical) term, e.g., generalization. After reading the paper, I share the reviewer’s perspective on this and encourage authors to consider this suggestion. Relevant references are appropriately included. This is a high quality work.
Decision: Accept (oral)