How Diffusion Models Learn to Factorize and Compose
Authors: Qiyao Liang
Ziming Liu
Mitchell Ostrow
Ila R Fiete
Venue: NeurIPS 2024 poster (2024)
Keywords: Representation Learning
Compositional Generalization
Diffusion Models
Generative Models
Primary area: diffusion_based_models
TL;DR: We carry out experiments on conditional DDPMs using synthetic datasets that demonstrate their ability to learn factorized representations and compositionally generalize.

Abstract: Diffusion models are capable of generating photo-realistic images that combine elements which do not appear together in natural images, demonstrating their ability to compositionally generalize. Nonetheless, the precise mechanism of compositionality and how it is acquired through training remains elusive. Here, we consider a highly reduced setting to examine whether diffusion models learn semantically meaningful and fully factorized representations of composable features. We performed extensive controlled experiments on conditional DDPMs trained to generate various forms of 2D Gaussian data. We demonstrate that the models learn factorized, semi-continuous manifold representations that are orthogonal in underlying continuous latent features of independent variations but are not aligned for different values of the same feature. With such representations, models demonstrate superior compositionality but have limited ability to interpolate over unseen values of a given feature. Our experimental results further demonstrate that diffusion models can attain compositionality with a small amount of compositional examples, suggesting a novel way to train DDPMs. Finally, we connect manifold formation in diffusion models to percolation theory in physics, thereby offering insights into the sudden onset of factorized representation learning. Our thorough toy experiments thus contribute a deeper understanding of how diffusion models capture compositional structure in data, paving the way for future research aimed at enhancing factorization and compositional generalization in generative models for real-world applications.

How Diffusion Models Learn to Factorize and Compose
Qiyao Liang Ziming Liu Mitchell Ostrow Ila Fiete
Massachusetts Institute of Technology {qiyao, zmliu, ostrow, fiete) Cmmo..mmiiee.......m.eee edu
Abstract
Diffusion models are capable of generating photo-realistic images that combine elements which likely do not appear together in the training set, demonstrating the ability to compositionally generalize. Nonetheless, the precise mechanism compositionality and how it is acquired through training remains clusive. Inspired by cognitive neuroscientific approaches, we consider a highly reduced setting to examine whether and when diffuusion models learn semantically meaningful and factorized representations of composable features. We performed extensive controlled experiments conditional Denoising Diffusion Probabilistic Models (DDPMs) trained to generate various forms of 2D Gaussian bump images. We found that the models learn factorized but not fully continuous manifold representa- tions for encoding continuous features of variation underlying the data. With such representations, models demonstrate superior feature compositionality but limited ability to interpolate over unseen values of given feature. Our experimental results further demonstrate that by training with independent factors of variation, diffusion models can attain compositionality with few compositional examples, suggesting a more efficient way train DDPMs. Finally, we connect manifold formation diffusion models to percolation theory in physics, offering insight into the sudden onset of factorized representation learning. Our thorough toy experiments thus contribute deeper understanding of how diffusion models capture compositional structure data.
1 Introduction
Large-scale text-to-image generative models can produce photo-realistic synthetic images that com- bine elements in novel fashion (compositional generalization). Nonetheless, the ability of models to do so, as well as their failure modes, are not well-studied systematically in large diffusion models due to the size of the model and the complex and high-dimensional nature of their training datasets. Factorization and compositional generalization have been theoretically and empirically investigated in 7 g 9]. However, these studies have not reached
many deep generative models before [1 2 3 a unanimous conclusion on whether factorized representations learned the intermediate layers of the model promote compositional generalization in the model performance. Specifically, several studies S have found little correlation between factorization and compositionality, contrary to others that suggest that factorization promotes compositionality [TO] 2 8 As result, there no consensus on the exact mechanism of compositionality and how models gain the ability to compositionally generalize. However, these previous studies involve data with complex mix of discrete and continuously varying features, making it difficult to explicitly analyze the model's learned representations beyond the disentanglement score, and thereby hindering deeper understanding of factorization and compositionality.
38th Conference on Neural Information Processing Systems (NcurIPS 2024).

---

Despite the remarkable compositional abilities demonstrated by diffusion based text-to-image gen- erative models, explicit exploration of factorization and compositional generalization of diffusion models has been relatively limited. In recent empirical study of toy diffusion models, Okawa et. al. [6] shows that diffusion models learn to compositionally generalize multiplicatively. They too, however, did not focus on the mechanistic aspect of compositional generalization within the model or analyze the representations learned by the models. To investigate factorization and compositionality in diffussin models, we conduct extensive experiments
conditional denoising diffusion probab- listic models (DDPMs) [12] trained with various 2D Gaussian datasets to characterize their learned representations and generalization performance | Drawing inspiration from traditional cognitive psychology and neuroscience studies on animals and humans, we design straightforward cognitive tasks for models to perform while analyzing both their generation output ("behavior") and their intermediate layer representations ("neural activities"). Specifically, we evaluate the diffusion model's ability to generalize by generating 2D Gaussian bumps centered specified x; y coordinates finite canvas. We deliberately chose a low-dimensional dataset and a simplified experimental setup to enable clear analysis of how the model's representation factorization relates to its generalization capabilities.
Conceptually, the models can adopt different parameterizations of the z, y coordinates for the Gaus- sian bumps, each with varying implications for data efficiency. Consider a K K-dimensional dataset created from the composition of multiple independently varying 1-dimensional (ID) latent features (e.g. a factorizable data distribution P(11, Ø2..... xK) = P(æ1) ... P(xx)). Hypotheti- cally, model may learn either joint representation of all K dimensions P( 22, TK) * P(x1) ((zr) P(x2) P(*x) or learn independent 1D representations former case, if there were N states per dimension, the model would need see approximately N samples during training. If the model could somehow recognize the independence of the different factors, as few as KN training samples might suffice. We term the former coupled representation, while the latter is a factorized one. Factorized representations could be far more sample efficient to learn, potentially permitting better generalization, and are more robust and adaptable for downstream applications. In biological neural networks, features of independent variation are typically represented in factorized manner. For instance, primates encode allocentric pose information in a factorized fashion, leveraging grid cells for two-dimensional location and head direction cells for orientation
P(Ik). In the
18 20, 21 22. Further, spatial information about objects in the environment is factorized into place cells and other spatial coding cells 23 15
[4 [1]] 16
17
Across three related studies, we investigate how diffusion models learn encode independent features of the dataset, how that informs the model's ability to generalize out-of-rraining-distibbbut in terms of composition and interpolation, and how variations in the training data could impact such generalization. To be precise, here we define compositionality as the ability to generate novel combinations of feature values that are not observed during training (e.g.. given {(a, b), (c. d)}- output {(a, d), (c, b))). We define interpolation as the ability to linearly combine different feature values (e.g.. output = 16 given x = 15 and x 17). The failure of the model perform the designed tasks could be due to its inability to handle composition, interpolation, or combination of both. We train the model on 2D Gaussian data and, some cases, explicitly include 1D Gaussian stripes with specified positions that can be combined to form the desired 2D Gaussian. We then pose the following questions:
1. Do diffusion models learn factorized representations? If so, when do these emerge over training?
2. Do the trained models learn to generalize beyond the training distribution, and what kind of training data is sufficient for this generalization?
3. Does the inclusion of few explicitly factorized examples in training (e.g. the ID Gaussian data) improve sample efficiency and generalization?
Contributions. In three experiments, we find that:
1. Given factorized, continuous conditional inputs, diffusion models learn "hyper-factorized" representations that are orthogonal not only for independent features but also different values of the same feature (Section 3.1
A preliminary version of this work was presented at the ME-FoMo and BGPT workshops at ICLR 2024
13

---

2. Models compose well but interpolate poorly. Models can compositionally generalize when they observe the full extent of each independent latent feature along with a few compositional examples (Section 3.2).
3. Models trained on datasets containing isolated factors of variation require an exceptionally small number of compositional examples compositionally generalize, showing remarkable data efficiency (Section 3.2).
4. Formation of a factorized representation for each continuous dimension of variation requires a threshold amount of correlated data, related to percolation theory in physics (Section 3.3)
Our results suggest that diffusion models can learn factorized. though not fully continuous, represen- tations of continuous features with independent variations. These models demonstrate exceptional compositionality but have limited interpolation ability, even with sufficient training and data. Our analysis offers deeper diagnostic insights into the mechanism of factorization and compositionality of diffusion models from a microscopic representation perspective. Moreover, our insights on the data efficiency of training with isolated factors of variation with few compositional examples suggest the possibility of far more data efficient methods of training diffusion models.
2 Methods
Dataset. We generate N x N pixel grayscale images. By de- fault, we set N 32 unless otherwise specified. Each image contains one 2D Gaussian bump ("blob"), the multiplication of
Qaussian Bump
20 Gausskan 908
a vertical and a horizontal ID Gaussian stripe, or one 2D Gaus- sian addition, the sum of vertical and horizontal ID Gaussian stripe ("sum of stripes (SOS's)") at various x, locations. The brightness U(2,z) of the pixel at position (z. y) for 2D Gaus- sian blob centered at (Ma, Mby) with standard deviation (az, σy) 255x x (1 e (e (w- /40 - (00-x))//40
H
is given as v. bump
(m.yy
and as ((z.)) 255x x [1 - 2 (e--(a-na) 1/40 -e- (v- (o_m,)///000 /
Figure 1: Example 32 x 32 image data of 2D Gaussian bump (left) and a 2D Gaussian SOS (right).
for 2D Gaussian SOS with the normalized range of ((z,w) to
be j0, 255). Each image is generated with ground truth label of (pz: Myy). which continuously vary within [0, N unless oth- erwise specified. In our convention of notation, we label the top left corner of the image as (1, 1) while the bottom right corner of the image as (N,N), Sample 32 x 32 images of 2D Gaussian bump and SOS centered at μa = My = 16 with 0r σy 1 are shown in Fig.
A single dataset of these images consist of the enumeration of all possible Gaussians tiling the whole N N canvas increment d in the z-direction and dy in the y-direction. larger de or d, means a sparser tiling of the image space and fewer data while a smaller d₂ or dy result in more data with denser tiling of the total image space. Moreover, each 2D Gaussian have independent spread in the x and y direction given by dr and Tg. with a larger spread leading to more spatial overlap of neighboring Gaussians and a smaller spread less overlap. By parameterically tuning the increments d_ and dy and the spread a₂ and oy. we can generate datasets of various sparsities and overlaps. We provide a more detailed analysis of the various attributes of the data based on these parameters in Appendix A.5) For the majority of the experimental results we show in Sec. [ we have chosen to fix
o == 02 ogy = 1.0 and d := dg = dy = 0.1 unless otherwise specified. A default 32 x 32 dataset of = 0.1 contains 102400 images.
with standard UNet architecture as shown in Appendix Fig. [ For each image in the training dataset, we provide an explicit ground truth label (Mz: My) as an imput to the network. For reference, we investigate the internal representation learned by the model using the output of layer 4 as labeled in Fig. 7 Since each dataset has inherently two latent dimensions, x and y,, we use dimension reduction tools such as PCA or UMAP 26) to reduce the internal representation of the model to a 2D/3D embedding for the ease of visualization and analysis. We defer the details of model architecture, dimension reduction, and experimentation to Appendix A.I and A.2 Briefly, we assess the performance of the model based on the accuracy of
Models & Evaluations. We train a conditional DDPM 12
3

---

its generated images at correctly displaying the center location of the 2D Gaussian. Further details on the precise definition of the accuracy metric can be found in Appendix A.3
3
Results
3.1 Models learn factorized but not fully continuous representations
In this section, we aim to understand the factorization of the model's learned representation via explicit inspection of its topology and geometry. Given a conditional DDPM trained on the 2D Gaussian bump dataset described above, we first investigate whether the model learns coupled or factorized representation. Naively, a 2D Gaussian dataset with two independent features of variation, and y. has 2D plane-like latent representation. Unfortunately, simply inspecting the learned representation would not allow us to easily differentiate between a coupled versus a factorized representation since the Cartesian product of two lines I.- x I R x R RR² is topologically and geometrically equivalent to a plane P2 C TR2. To overcome this issue, we perform a simple modification the Gaussian dataset. We impose periodic boundary conditions in the image space to connect the left-right and top-bottom boundaries of the image such that the latent representation of the dataset forms a torus. Mathematically, a torus is defined by the Cartesian product of two circles S x St. A Clifford Torus is an example of a factorized representation of the torus, with each circle independently embedded in R²2, resulting in a 4-dimensional object. However, the most efficient representation in terms of extrinsic dimensionality is the regular torus T which is embedded in RR. albeit unfactorized. Various 2D projections of the 3D torus and the Clifford (4D) torus are shown in Fig. aa). Due to geometric differences between the Clifford and the 3D torus, we can now distinguish whether the model learns a factorized representation. If the model were to represent r and y independently with two ring manifolds, we expect to see Clifford torus in the neural activations, rather than a different geometry such as the 3D torus.
To apply the geometry tests for differentiating between the 3D and the Clifford torus, we first need to confirm that the model indeed learns a torus representation of the dataset by computing the topological features of the Iearned representation via persistent homology. In Fig. [b), we compare the persistence diagrams of a standard torus (left) with the final learned representation of the model (right). Both diagrams exhibit the similar topological features, rank-1 Ho group, rank-2 Hi group (with overlapping orange points on top in both diagrams), and rank-1 1 H2 group, signaling that the model indeed learns a torus representation.
We then investigate when this torus representation emerges during training. In Fig. 2(e), show the model's performance as measured by the accuracy of its generated images (top) and the corresponding effective dimension (bottom) of the learned representations across training, as measured by the participation ratio ( given by (E, /; X², where X; is the eigenvalue of the i-th principal component (27). Intuitively, the participation ratio counts the dominant aegen-components providing a measure of the effective dimension of the geometric object of interest. A detailed inspection of the effective dimension of the learned representations over the training duration informs us of whether and how the model arrives at factorized, 4D representation. We see in Fig. 2 c) that as training progresses, the model's internal representation first undergoes dimension increase then decrease, eventually converging to around 7 dimensions after 200 epochs. While the dimensionality converged to higher dimension than 4, the top 4 eigenvectors became notably more prominent as the training converges, as indicated in the eigenspectra in Fig. dd) This signals that a 4D, rather than 3D, torus representation is eventually learned. Various PCA projections of the learned representations along the training process are shown in Fig. gi)) where the projections in the last epoch resembles those of the Clifford torus shown in (a).
To confirm whether the model learns Clifford torus, we employ the orthogonality and parallelism tests of tori 28 (details found in Appendix Bp. In an ideal Clifford torus, the rings along the poloidal direction should be parallel with each other, and similarly for those along the toroidal direction. Moreover, the rings along the poloidal direction should be orthogonal with respect to the rings in the toroidal direction. The orthogonality and parallelism tests measure the orthogonality and parallelism of subspaces spanned by various rings on the torus. If the model learns a Clifford torus representation, the subspaces that code for x and should be orthogonal, and the subspaces that code for x (ạ) for any given y (xr) should be equivalent (parallel). Given a pair of a and y, we construct 2D subspaces of rings S(fix z) and S(fix y) spanned by fixing while varying y and fixing y while varying z,

---

(n)
(6)
(d)
3D Torus
Clifford Torus
(b)
Taxs Portsstece Dligian
Treedea Pauch Perstterre Skggran
7pans
Copll Cutt
350
280
250
(e)
(9)
Dediredd kile 10,00
Figure 2: Metries of a model trained using 2D Gaussian bump datasets with periodic boundaries. (a) 2D projections
standard 3D torus (left) and a 4D Clifford torus (right). The 3D torus an example of a coupled representation that can be learned by the model while the 4D torus is factorized one. (b) Persistence diagrams of standard torus (left, the diagram looks the same for Clifford tori) and the learned representation of the model at the terminal epoch (right). There are two overlapping orange points for H1 in both diagrams. (c) Model accuracy (top) and effective dimension (bottom) of representation learned by the model first 15 dimensions) of the model's 's learned representations and their corresponding sample accuracy percentage and explained variance ratio of the top 4 PCs (labeled top right of each panel) at various checkpoints during training. (e)-(g) PCA visualizations of the learned representations at epoch 0, 150, and terminal epoch, respectively.
function of training epochs. (d) PCA eigenspectrum (the
respectively. We perform 3 sets tests: 1) z-on-y tests test orthogonality and parallclism between surfaces S(fix I) and S(fix y) for all combinations of and y; 2) r-on-z tests test orthogonality and parallelism between S(fix x1) and 5(fix x22) for all combinations
different x's; 3) y-on-y tests test orthogonality and parallelism between S(fix y1) and S(ix y2) for all combinations of different y's.
In Fig. 3 we compare the orthogonality and parallelism test statisties between an ideal 3D torus (left), the model's learned representation at the final epoch (center), and an ideal Clifford torus (right). Comparing test statistics (top row) between the model representation and the standard 3D and Clifford tori, we see that the model encodes x and independently in mostly orthogonal subspaces, similar to a Clifford torus. The test 2 and statistics (middle and bottom row) of the model representation, however, does not match those of either the 3D the Clifford torus. In fact, the statistics from tests and of the model representation closely resemble those from test of the 3D torus. This means that the model is encoding different values of in an almost orthogonal, opposed to parallel, fashion, similarly for different values of y. These observations signify that the models "hyper-factone"" factoriiact each individual factor of variation by treating them more similarly to categorical variables with nonzero overlaps between neighboring categories rather than dally-continuous variables, which explains why the effective dimension of the model's learned representation is higher than after convergence in Fig, Cc.. Although these observations are made from a model trained on torus dataset, we expect the model to have similar encoding schemes across all variants of our Gaussian bump datasets.
3.2 Models can compose but not interpolate
In this section, we examine the model's ability to compositionally generalize. Specifically, we train the model on incomplete datasets of 2D Gaussian SOSs, in which we leave out all Gaussian SOSs
5

---

(2)
3D Torus
(b)
Learned Representation at Terminal Epoch
(6)
Ciifford Torus
Deiver Dareert Dewerartt
APch
Aeythe
Figure 3: Comparison of orthogonality and parallelism test statistics between 3D torus, model's learned representation, and Clifford torus. z-on-y (top row), z-on-x (middle row), y-on-y (hottom row) orthogonality (left column) and parallelism (right column) test statistics are compared between (a) an ideal 3D torus (blue), (b) the learned representation by the model (green), and (c) an ideal Clifford torus (orange).
centered in the red-shaded test regions (Fig. 4 D), sampling distributions shown in Fig. Fa), (b)). We then assess the performance of the models generating 2D Gaussian SOSs centered within and outside of the test regions. Here we choose the width of the cuts to be around 6 pixels wide, which roughly corresponds the width of the Gaussian stripes of 1.0. We design the lesions such that we can probe the model's ability at compositionally generalizing out of the training distribution as well as its ability to spatially interpolate in a single variable alone. There are possible outcomes based on the model's ability to compose and interpolate, and we give predictions for the model's s generalization performance in each case: 1) the model cannot compose or interpolate: here, we would see low performance across the test regions; 2) the model interpolates but cannot compose: here, we would see high performance across the test regions; 3) the model composes but cannot interpolate: here, we would see high performance in one dimension but not the other in the non-intere -on-intersecting part of the test regions, low performance in the intersection; 4) the model can compose and interpolate: here, we would see higher performance across the test regions. We note that case (2) and (4) are indistinguishable via the behavior of the model.
In our first experiment, we simply train our model on a 2D Gaussian SOS dataset where all data centered in the test regions are left out, i.e. excluding all Pzz, By E [13, 19 as shown in Fig. 4a). We call this model the 2D model since the model is trained on only 2D Gaussian SOS data. We then examine the terminal accuracy of the 2D model in generating Gaussian SOSs at the correct μa and My in various parts of the test regions, which we section into a horizontal part, a vertical part, both excluding the area of their intersection, and the intersection itself (shown schematically in Fig. 4t)). We note that the 2D model achieves high accuracy in generating My while suffers low accuracy in generating f4 in the vertical section. Similarly, the 2D model achieves high accuracy in generating Pzz while suffers low accuracy in generating By in the horizontal section. The model suffers low accuracy in generating both μz and My the intersection region. These observations have two implications: i) the model is factorized and compositional since it is able to generate the correct ar My irrespective of the other; ii) the model has limited ability to spatially interpolate, which suggests that it does not learn fully continuous manifold in its activation space (see Fig. To(e).. These observations meet our expectation for outcome case (3), when the model can compose but not interpolate and resonates our conclusion from Sec. 3.1 that the model has learned to factorize x and y, but has not learned consistent representation across all x's (and likewise y y's). In Appendix C.3 Fig. IT we quantitatively assess model's ability to interpolate as a function of the held-out range width in the data manifold and found that model's ability to interpolate gradually decrease as a function of the held-out region width.
To further verify our hypothesis that the model is capable of composition but not interpolation, we design an alternative dataset that includes mixture of 2D Gaussian SOS data and 1D Gaussian stripe data (single stripes that are either horizontal or vertical). Specifically, we generate set of 1D Gaussian stripe data across the entire range of a and y without any held-out regions, as shown in Fig. 4b.. Note that the ID Gaussian stripe data respects the 2D structure of the conditional

---

(c)
Vertical
2D Gaussian SOS Dataset
1.00
20
0.50
20+10
Vertical
10
2.25
Horizontal
saction Horizontal
Horizonta
23-
Vertical
0.75
(b)
1D Gaussian Stripe Dataset
0.50
12
0:25
0.00
(
20 Cauzsttnn Bump Seunsin Stripe
(6)
Intersection
0.75
deaaryy
Accurecy
02
0.00
3J. 32 24 20 Br be B7
Sulcompling
Figure 4: Models trained on Gaussian SOS datasets to generalize the test regions. We train three models on various Gaussian SOS datasets to test their ability to compositionally generalize in the red-shaded, held-out test regions shown in the sample image (f). (a) The 2D Gaussian SOS dataset contains all combination of 2D Gaussian SOSs for all x and y between O and 32 except for the held-out range between 13 and 19. (b) The ID Gaussian stripe dataset contains horizontal and vertical ID Gaussian stripes of full range of x and values between 0 and 32. The accuracy of the three models in generating the correct and y location of the Gaussian SOS shown for different sections of the test regions: (c) The vertical section excluding the intersection, (d) the horizontal section excluding the intersection, and (e) the intersection. (f) Sample image of 2D Gaussian SOS with the different test regions labeled. (g) shows the accuracy of models run with various subsampling rate of the 2D Gaussian bump + ID Gaussian stripe dataset.
input embedding. The exact generation procedure for the ID Gaussian stripe dataset is described in Appendix C.I This ID dataset is then combined with the previous 2D Gaussian SOS dataset with the same held-out test regions. The resulting 2D+1D dataset contains ID data all range of æ and y but has not seen 2D data (composition of 1D) in the test regions. The model trained under this dataset, deemed the 2D+ID model, has high performance in generating both Pa and by across all of the test regions. This shows that given the necessary "ingredients" (1D Gaussian stripes of all range of x and y), the model able to compose them a proper manner (2D Gaussian SOSs). These observations again confirm that the learned representation of the model is factorized but not consistent across single feature, meaning that the model able to compose independent features not interpolate within single feature dimension.
Finally, we test whether compositionality can be learned given no compositional examples in the training dataset. To test this, we train ID model on the ID Gaussian stripe dataset as described above. Our results show that while model achieves respectable accuracy in generating the correct My across the test regions, it does not simultaneoush generate the correct Ma, indicating that the model did not learn to properly compose the 1D stripes. This is also evident from the generated image samples from the ID model shown in Fig. c). Without any compositional examples, the model fails to generate the intersection of two ID Gaussian stripes and defaults back to generating one of the stripes at the correct location. While the 1D model has also learned to factorize and as independent concepts, it is not able to generalize here due to the lack of compositional examples. Hence, based the performance of the ID. 2D. and 2D+1D models that we have seen above, conclude that complete compositional generalization requires data of all range of x's and y's and some compositional examples of the two to be present within the dataset.
Building on our previous results, we investigate whether combining the 2D Gaussian bump dataset with ID stripe data can impart different form of compositionality to the model multiplicative rather this, we create a dataset that mirrors the 2D+1D Gaussian SOS set, substituting
than additive. the 2D Gaussian SOS data with 2D Gaussian bump data while maintaining the same test regions. The latent representation of this dataset can be found Fig. 9h) in the Appendix. The resulting model trained under the 2D Gaussian bump + ID Gaussian stripe dataset is able to generate 2D Gaussian
7

---

Bumps
Bumps w/ Stripes
ACCEPNN
Quadert hatbeacc
Redd Geceniy
Taathconery 000 Acopere
000 Aecericy
S27 val di sax aay are
bay Cie ata 120
Sateurri
Parceriage Sutoumm
Image Stee
Figure 5: Exponential sample efficiency gains from training the model on independent factors of variation. (a-b): Results on N = 32 Gaussian 2D bump generation. (a) Model accuracy in generating 2D Gaussian bumps from training on 2D Gaussian bumps, shown as function of the subsampling percentage. (b) Model accuracy in generating 2D Gaussian bumps from training on mixed 2D Gaussian bumps ID Gaussian stripes. Red dashed lines in (a). mark threshold accuracy of 0, 60, and 100%. (c) Log-log plot of dataset size needed to reach 60% threshold accuracy as function of image size N with 2D Gaussian bumps training data (blue) versus mixed 2D Gaussian bumps + ID Gaussian stripe training data (orange): distinct scalings of data efficiency visualized by dashed gray and black lines, which provide linear and quadratic reference, respectively.
bumps within the test regions up to 70% accuracy, suggesting that the model is adept at handling an alternative form of compositionality. We further investigated whether the number of 2D Gaussian bump examples in the dataset influences compositionality by subsampling the 2D Gaussian bumps in the in- tn-distribution regions (Fig. (g)). The results show that models trained on datasets with varying percentages of subsampled 2D Gaussian bumps achieve similar performance across the test regions, even with as little as 20% of the original data. These findings suggest that training for compositional generalization can be efficiently achieved by using full range of data for each individual factor of variation, along with few compositional examples.
To validate the benefit of training with 2D ID datasets, we analyzed the data efficiency scaling of models trained on 2D Gaussian bumps + ID Gaussian stripes, comparing them to models trained solely on 2D Gaussian bumps datasets, we excluded 20% of the 2D Gaussian bumps from the center square region of the N x N canvas for out-of-distributte the percentage of 2D Gaussian bumps subsampled in Fig. 5 a) for the bascline dataset and (b) for the augmented dataset. The results clearly demonstrate that the model trained on the augmented dataset performs better with significantly lower percentage of 2D Gaussian bumps in the training dataset. We then plotted the sizes of the baseline and augmented datasets required to reach a 60% accuracy threshold as a function of N on log-log scale in Fig. c).. We found that models trained on the augmented datasets only require linear number of data in while those trained on the baseline datasets require quadratic number of data in N. This result suggests that augmented datasets with samples of all independent factors plus a few compositional examples provide an efficient training method that has sample complexity linear in the number of factors. In Appendix C.2 and C.4 we further discuss the sample efficiency of learning compositionality from dataset of isolated factors of variation plus few compositional examples.
function of image size N. For both the baseline and augmented
(OOD) evaluations. The accuracy of the models are plotted against
In summary, our findings indicate that the model excels at compositionality but struggles with interpolation. Consequently, effective generalization necessitates that the training set encompasses the full range of each individual factor of variation, alongside compositional examples. Furthermore, we discovered that models trained on datasets featuring explicitly factorized examples exhibit remarkable data efficiency and adaptability to various forms of compositionaalty. These results align with the observed factorization patterns in the model representation discussed in Sec. [3.]] and suggest promising strategics for developing more data-efficient training schemes that utilize tailored datasets.
3.3 Connection between manifold learning and percolation theory
We have established that the model learns factorized (but not fully continuous) and compositional representations. Next, we identify potential connection between manifold learning in diffusion models and percolation theory in physics, which provides a normative explanation as to how factor-
ization (or compositional generalization) emerges. To start, we note that our Gaussian bump datasets can be approximated as a simple Poisson Boolean (Gilbert's disk) model [2] 3D), well-studied system in continuum percolation theory. Fig. 6(a) shows schematic illustration of our dataset of
8

---

Threshost Overape 6 005
Mayo 4110
(c)
(6)
Ground Truth Image
Generated Imace
C-011
0-63.000
PM05.C0
e=161D
Figure 6: Percolation theory of manifold formation and interpolation. (a) Schematic drawing of the Gaussian bumps of various width on a lattice of grid size d, and dy. (b) Overlaps of neighboring Gaussian bumps as a function of the grid size d := d₂ dy for various Gaussian widths. (c) Theoretical simulation of largest connected mass ratio as function of percentage of data in training set À with threshold overlap of 0.005 for various Gaussian widths. (d) Ground truth (left) and
generated image (right) of a sample Gaussian bump data a = 0.1 centered at μtx 16 and My = 16. (e) Terminal accuracy of models as function of À trained with datasets that only differ in the Gaussian bump widths for various Gaussian bump widths.
Gaussian bumps approximated as disks of various widths on a 2D 32 32 lattice of grid spacings da and dy. In percolation theory, the quantity of interest is the critical fraction of nodes that need to have non-zero overlaps in order for the entire system to be interconnected with high probability. For most systems (finite- and infinite-sized), there exist phase transition that occurs at the critical fraction that can be either analytically derived numerically estimated, with the transition becoming sharper as the system size scales. In our context, since the Gaussian bump images are pixelated, overlaps between neighboring Gaussian bumps are smooth. In Fig. b(b), we plot the normalized 12-norm of the neighboring that since the L.2norm measure of overlap normalized to between given Gaussian bump and itself, the overlap between a given Gaussian bump and slightly offset Gaussian bump temporarily goes beyond due to the discrete nature the data.
sesaanee-d-sppratee Gaussian bumps with various spread o ss. We note
We hypothesize that below a threshold amount of training data, the diffusion model cannot construct faithful representation of the training dataset. From the percolation perspective, if there does not exist a large enough interconnected component within the dataset, the model will fail to learn the relative spatial location of the data points, making it hard learn a faithful 2D representation. To test this hypothesis, we first simulated the mass ratio of largest interconnected components as a function of the unit area data density with our 2D Gaussian bump datasets. For simulation, we use a dataset of 1024 data points, corresponding to d == dN dg = 1.0 on a 32 x 32 lattice. We then randomly sample À 1024 points on the lattice to compute the size of the largest interconnected cluster of Gaussian bumps. Here, we define a hyperparameter of threshold overlap beyond which we consider two Gaussian bumps as overlapping.
Fig. ((c). we show the simulation results. averaged over runs, with the chosen threshold overlap of 0.005 for datasets with various o p's. Additional simulation results with various chosen overlap thresholds can be found in Appendix D
Next, we quantify the connection between percolation theory and manifold formation in diffusion models. To account for the stochasticity sampling the datasets and avoid significant overhead in model training, we choose fix set of grid points on the 32 x 32 lattice and generate datasets of varying a's from this fixed set of grid points. Moreover, we choose each sampled fraction of the dataset to be a strict superset of the smaller sampled fractions of the dataset. For example, dataset at 0.2 contains all of dataset at À = 0.1, and dataset at = 0.3 contains all of dataset at À 0.2 and hence all of X 0.1, so on and so forth. For datasets of different a a's, we chose the same grid points such that the only difference between datasets of different a's is just the a itself. This procedure climinates all the stochasticity due to the probabilistic nature of percolation, and ensures

---

that differences in training are only due to the differences in o and X. To ensure fair comparison, we proportionally add training time each model trained with fewer data such that the total amount of training batches are constant across all models.
In Fig. (e), we display the accuracy of models trained with different fractions of a training dataset of a total size of 1024 for various a's. We note that there are high variances in the model performance, 0.3, 0.5. This is due stochasticity in model training itself as well as the gradual as opposed to a sharp phase transition, as shown in simulation Fig. c). In addition, we notice that based on the simulation, o 0.1 will not forge any connected cluster due to minuscule spread. In Fig. dj).. an sample image of Gaussian bumps of a = 0.1 is shown on the left, where it is mere pixel centered at (Keg,/y) Nonetheless, in experiment Fig. [(e. we see that the model is able to learn the o = 0.1 dataset at À = 0.8, 0.9. This is partially due to the fact that the noising process in diffusion training smears out the Gaussian bump of a 0.1 to give it a larger effective width, as shown in the generated bump image in Fig. dj.. In general, our experimental results agree with the percolation theory prediction that smaller similarities between the samples makes it harder for the models to learn meaningful representations, despite the same quanitity of data. The portraits of the learned representations by the models generating Fig. ((()) is shown Fig. 15 of Appendix D.2
especially for
4 Discussion
In the previous section, we have shown that diffusion models are capable of learning factorized and semi-continuous representations. This allows for compositional generalization across factors that have appeared in the training dataset, even given only small number of compositional examples. While we studied a toy system, our results imply that the diffusion model architecture has some inductive biases favoring factorization and compositionality, as seen in astonishing compositional text-to-image generation examples such as an "astronaut riding a horse on the moon". Our results demonstrate further that if the training data include isolated factors of independent variation and some compositional examples, diffusion models are capable of attaining high performance in OOD compositional generalization. While it is true that natural images much more complex, and there can be numerous forms of compositionality within a single image, the datasets and modes of composition we studied were not trivial. Nonetheless, further investigation is necessary understand the compositionality and factorization of models when multiple forms of compositionality are at play,
Furthermore, we showed connection between the models' performance, as related to its ability to learn a faithful representation of the dataset, and percolation theory in physics. Percolation theory provides plausible mechanistic interpretation of the observed sudden emergence in the model's s capability beyond a threshold number of data points. Further work is needed to characterize the precise connection between percolation theory and manifold formation, in both toy settings and realistic settings. In realistic datasets, mutual information or cosine similarity between data points can serve as abstract forms of overlap. Moreover, the idea of percolation can be further extended to study alternative observations of phase transitions in deep learning, such as percolation of the chain-of-knowledd in large language models.
Conclusion
We have shown that diffusion models are capable of learning factorized representations that can compositionally generalize OOD, given data containing the full range of each independent factor of variation and a small amount of compositional examples. Our study suggests that diffusion models have the inductive bias for factorization and compositionality. which are believed to be key ingredients for scalability. We identified that the diffusion models fail to generalize out-of-distrion -istrri-ittiittinn when 1) there are unseen values of given factor of variation for the composition, 2) there are no compositional examples in the training dataset, 3) there is an insufficient quantity of data, and 4) there is an insufficient amount of overlaps between data. Together, our results imply that a more efficient training dataset can be constructed by incorporating samples that feature explicit factorized examples along with few compositional examples that have substantial overlap. By optimizing the training data in this manner, we can enhance the data efficiency scaling of diffusion-basee models. Future investigations should consider controlled experiments with data optimization for improving data efficiency of diffusion models using more realistic data.
10

---

References
[1] Shengjia Zhao, Hongyu Ren, Arianna Yuan, Jiaming Song, Noah D. Goodman, and Stefano Ermon. Bias and generalization in deep generative models: An empirical study. In Samy Bengio, Hanna M. Wallach, Hugo Larochelle, Kristen Grauman, Nicolò Cesa- Desa-Bianchi,, and Roman Gareett, editors, Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information P Processing Systems 2018, NeurlPS 2018, December 3-8, 2018, Montréal, Canada, pages 10815 10824, 2018.
[2] Irina Higgins, Loic Matthey, Arka Pal, Christopher P. Burgess, Xavier Glorot, Matthew M. Botvinick, Shakir Mohamed, and Alexander Lerchner. beta-vae: Learning basic visual concepts with constrained variational framework. In 5th International Conference on Learning Rep- resentations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings. OpenReview.net, 2017.
[3] Christopher Burgess, Irina Higgins, Arka Pal, Loic Matthey, Nick Watters, Guillaume Des- jardins, and Alexander Lerchner. Understanding disentangling in f-vae. CoRR, Abs//180403359, 2018.
[4] Milton Llera Montero, Casimir H. Ludwig, Rui Ponte Costa, Gaurav Malhotra, and Jeffrey Bowers. The role of disentanglement in generalisation. In 9th International Conference Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021. OpenReview.net, 2021.
[5] Zhenlin Xu, Mare Niethammer, and Colin Raffel. Compositional generalization in unsupervised compositional representation learning: A study on disentanglement and emergent language. In Sanmi Koyejo, S. Mohamed, A. Agarwal, Danielle Belgrave, K. Cho, and A. Oh, editors, Advances in Neural Information Processing Systems 35: Anmual Conference on Neural Informa- tion Processing Systems 2022, NeurlPS 2022, New Orleans, LA, USA, November 28- December 9, 2022, 2022.
[6] Maya Okawa, Ekdeep Singh Lubana, Robert P. Dick, and Hidenori Tanaka. Compositional abilities emerge multiplicatively: Exploring diffusion models on synthetic task. CoRR, abs/2310.09336, 2023.
[7] Thaddaus Wiedemer, Prasanna Mayilvahanan, Matthias Bethge, and Wieland Brendel. Composi- tional generalization from first principles. In Alice Oh, Tristan Naumann, Amir Globerson, Kate Saenko, Moritz Hardt, and Sergey Levine, editors, Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10 16, 2023, 2023.
[8] Jeffrey S. Bowers, Ivan I. Vankov, Markus F. Damian, and Colin J. Davis. Why do some neurons in cortex respond to information in a selective manner? insights from artificial neural networks. Cognition, 148:47-63, 2016.
[9] Rahma Chaabouni, Eugene Kharitonov, Diane Bouchacourt, Emmanuel Dupoux, and Marco Baroni. Compositionalliy and generalization in emergent languages. In Dan Jurafsky, Joyce Chai, Natalie Schluter, and Joel R. Tetreault, editors, Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, ACL 2020, Online, July 5-10, 2020, pages 4427 4442. Association for Computational Linguistics, 2020.
[10] Yoshua Bengio, Aaron C. Courville, and Pascal Vincent. Representation learning: A review and new perspectives. IEEE Trans. Pattern Anal. Mach. Intell., 35(8):1798-1888, 2013.
[11] Sunny Duan, Loic Matthey, Andre Saraiva, Nick Watters, Chris Burgess, Alexander Lerchner, and Irina Higgins. Unsupervised model selection for variational disentangled representation learning. In 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020. OpenReview.net, 2020.
[12] Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. In Hugo Larochelle, Mare Aurelio Ranzato, Raia Hadsell, Maria-Florina Balcan, and Hsuan-Tien Lin, editors, Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual, 2020.
11

---

[13] Qiyao Liang, Ziming Liu, and Ila Ficte. Do diffusion models learn semantically meaningful and efficient representations? CoRR, abs/2402.03305, 2024.
[14] Jack W. Lindsey and Elias Issa. Factorized visual representations the primate visual system and deep neural networks. April 2023.
[15] Moser MB. Moser EI, Kropff E. Place cells, grid cells, and the brain's spatial representation system. 2008.
[16] Torkel Hafting, Marianne Fyhn, Sturla Molden, May-Britt Moser, and Edvard I. Moser. Mi- crostructure of a spatial map in the entorhinal cortex. Nature, 436:801-8066 -806, 2005.
[17] Rishidev Chaudhuri, Berk Gerçek, Biraj Pandey, Adrien Peyrache, and Ila Fiete. The intrinsic attractor manifold and population dynamics of canonical cognitive eircuit across waking and sleep. Nat. Neurosci. 22(9)) September 2019.
[18] J Taube, R U Muller, and J B Ranck, Jr. Head-direction cells recorded from the postsubiculum in freely moving rats. i. description and quantitative analysis. J. Neurosci. 10(2):420-435, February 1990.
[19] Jeffrey S Taube. The head direction signal: origins and sensory motor integration. Annu. Rev. Neurosci,, 30: 181-207, 2007.
[20] Torkel Hafting, Marianne Fyhn, Sturla Molden, May-Britt Moser, and Edvard I Moser. Mi- crostructure of spatial map in the entorhinal cortex. Nature, 436(7052):001-806, August 2005.
[21] Hanne Stensola, Tor Stensola, Trygve Solstad, Kristian Frøland, May-Britt Moser, and Edvard I Moser. The entorhinal grid map discretized. Nature, 492(7427):72-77, December 2012.
[22] Yoram Burak and lla R Fiete. Accurate path integration in continuous attractor network models of grid cells. PLoS Comput. Biol., (())ee00002))) February 2009.
[23] May-Britt Moser. David Rowland, and Edvard I. Moser. Place cells, grid cells, and memory. Cold Spring Harbor Perspectives in Biology, 7(2), Feb 2015.
[24] Nanxin Chen, Yu Zhang, Heiga Zen, Ron J. Weiss, Mohammad Norouzi, and William Chan. Wavegrad: Estimating gradients for waveform generation. In 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021. OpenReview.net, 2021.
[25] Chitwan Saharia, Jonathan Ho, William Chan, Tim Salimans, David J. Fleet, and Mohammad Norouzi. Image super-resolution via iterative refinement. IEEE Trans. Pattern Anal. Mach. Intell., 45(4):4713-422 2023.
[26] Leland Mclnnes and John Healy. UMAP: uniform manifold approximation and projection for dimension reduction. CoRR, abs/1802..3346, 2018
[27] Peiran Gao, Eric Trautmann, Byron Yu, Gopal Santhanam, Stephen Ryu, Krishna Shenoy, and Surya Ganguli. A theory of multineuronal dimensionality, dynamics and measurement. bioRxiv, 2017.
[28] Christopher J. Cueva, Adel Ardalan, Misha Tsodyks, and Ning Qian. Recurrent neural network models for working memory of continuous variables: activity manifolds, connectivity patterns, and dynamic codes. CoRR, abs/211 1111111100011 1.01275. 2021.
[29] E. N. Gilbert. Random plane networks. Journal of the Society for Industrial and Applied Mathematics, 94):5333-545 1961.
[30] Karl-Heinz Hanisch. On classes of random sets and point process models. J. Inf. Process. Cybern., 498-002 1980.
[31] Leland Mclnnes, John Healy, Nathaniel Saul, and Lukas Grossherger. Umap: Uniform manifold approximation and projection. The Journal of Open Source Software, 3(29):861, 2018.
12

---

[32] Christopher J Cueva, Adel Ardalan, Misha Tsodyks, and Ning Qian. Recurrent neural network models for working memory of continuous variables: activity manifolds, connectivity patterns. and dynamic codes. arXiv preprint arXiv:2111.012777 2021.
A Experimental Details
A.1 Architecture
We train a conditional denoising diffusion probabilistic model (DDPM) [2] with a standard UNet architecture of downsampling and upsampling blocks, interlaced self-attention layers, and skip con- nections as shown in Fig- 7 Each down/up-sampling blocks consist of max pooling/upsampling layers followed by two double convolutional layers made up by convolutional layers, group normalization, and GELU activation functions.
Network Architecture:
Embedding Scheme:
Layer 0
Down(64,122 128)
Postional encoding for
Possional encocing for
Embedding
Layer
Down(1 28,256)
Positional Encoding:
Layer
Down(256,256)
PF (/11000022)
Self- Self-attention layer of corresponding sizes
1F (1110000*+1))
we 1112-1
Bottleneck
Loyer
Up(51 22,128
Positional Encoding Concatenation:
Layer
Up(256.64)
Time
PE N
PE
Layer 5
Upx128.64)
Figure 7: The UNet architecture of the conditional diffusion model. The schematic diagram of the standard UNet architecture consisting of three downsanpligguupammpiii self-attention layers and skip connections shown on the left. The conditional information consisting of concatenation of positional encodings of timestep and w/y-posiiions is passed in at each block as shown on the right.
blocks with interlaced
The conditional information is passed in at each down/up-samplinn block shown in the schematic drawing. In our experiments, we choose to preserve the continuity the Gaussian position labels passed into the model via explicit positional encoding rather than using separate trainable embedding MLP at each block. Each embedding vector is made by concatenating equal-length vectors of the positional encodings of the timestep, the x-position, and the y-position.
In our experiments, we visualize the outputs of layer as the internal representation of the diffusion model. We have chosen not use the output of the bottleneck layer for our study of the learned latent manifold, as we have observed that the bottleneck layers have diminishing signals most of experiments. This is likely due to the presence of the skip connections. This choice does not affect the validity of our main results, as we are focused on the factorization of the model's 's learned representation.
A.2 Dimension Reduction
We primarily use the dimension reduction technique Uniform Manifold Approximation and Projection for Dimension Reduction (UMAP) (26) to study and visualize the learned representation of the model. Specifically, we collect image samples and their corresponding internal representations (outputs of layer 4 from the architecture described in Sec. A. We then transform the high-dimensional internal representations into 3D embedding as sample of the learned representation, which we visualize and analyze. For an implementation of UMAP, used the Python package [31].
13

---

A.3 Evaluation
We assess the performance of the model using two primary criteria: 1) the quality of the denoised images and 2) the quality of the learned representation.
At given time during after training, we generate 1024 denoised images and their corresponding internal representations of sampled labels based on 32 x 32 grid points. We predict the lahel corresponding each generated image based the x- and y-positions of the generated Gaussian bump/SOS in the image. We then compute the accuracy of predicted labels from the ground-truth labels averaged over 1024 samples as
1024
Accuracy 1024
Σ 1
P1l < 1) 1(1,) pby < 1),
(1)
where 1()) is an indicator function that returns 1 if the expression within holds true, otherwise. Similarly, we can modify this expression to only assess the accuracy of generated z-positions or u-positions separately. Here estimate the center of the Gaussian bump/SOS fúž and jt, by finding the location of the darkest pixel in the image. In the cases where there are no Gaussian bumps/SOS's or more than one bump/SOS, the algorithm defaults back finding the centroid of the image. We then construct the learned representation of the model based on the neural activations of layer corresponding the 1024 sampled images collected the terminal diffusion generation timestep. We note that we have investigated neural activations collected at alternative diffusion generation timesteps and found that they do not noticeably differ from timestep to timestep.
A.4 Training Loss
Diffusion models iteratively denoise a Gaussian noisy image Xy into a noisefree image Xo over diffusion timesteps 10, ,,,,,,..777 T) given the forward distribution q(x:|x--1) by learning the reverse distribution pa xx-1 (x)) Given a conditional cue c, a conditional diffusion model [24 25 reconstructs image from source distribution q(xo c). Specifically,
train our neural network (UNet) to predict the denoising direction eg (x,, 4, c) at a given timestep with conditional cue c with the goal of minimizing the mean squared loss (MSE) between the predicted and the ground truth noise as follows
= (eeegt.....).xx00gtttl........
[||e eo (xo, L, c)]l3],
(2)
where we assume each noise vector to be sampled from a Gaussian distribution (o L.I.D. at each timestep t.
A.5 Datasets
The datasets we used for training the models generating the results Sec. Sec.Bhavee have various increments d and a. Here we briefly comment on the interplay between increments and sigmas, and how they affect dataset densities and overlaps. The ultimate goal of our task of interest learn a continuous 2D manifold of all possible locations of the Gaussian bumps/SOS's. Intuitively, the spatial information necessary for an organized, continuous, and semantically meaningful representation to emerge is encoded in the overlap of the neighboring Gaussian bumps/SOS's s, which is tuned via the parameters and a. As we increase d, the size of the dataset gets scaled quadratically, resulting in denser tilings of the Gaussian bumps/SOS's.
As we scale up a,, the dataset size remains fixed while the overlaps with neighbors are significantly increased. In Fig. b(b), we plot the normalized L2-norm of the product image of neighboring Gaussian bumps as function of increments for various spreads. Specifically, given two inverted grayscale Gaussian bump images, and 6, the normalized L2-norm of their product is given by the formula I|va * / ||al|2, where is element-win wise multiplication and ||-|/2 is the L.2norm. This quantity should give rough measure of the image overlap with the exception increment around 0.5 due to the discrete nature of our data. Moreover, we note that the cusps in the curves occur for the same reason. As we can see, the number of neighbors that a given Gaussian bump has non-trivial overlaps with grows roughly linearly to sub-linearly with the spread. Since the model constructs non-pane non-parallel representation encodings of different values of x's and y more neighbor information results in more overlaps between different values of z's and y's and hence better representation Icarned, as
14

---

shown in the percolation results in Fig. in Sec. 3.3 Overlaps in Gaussian SOS data can be similarly analyzed. albeit the differences in overlaps of neighboring Gaussian SOS's of various spread o's is not as pronounced due to the large coverage area of each Gaussian stripe.
A.6 Training Details
We train the models quad-core Nvidia A 100 GPU, and an average training session lasts around hours (including intermediate sampling time). Each model given an inversely proportional amount of training time as a function of the size of the dataset that it is trained on such that there is sufficient amount of training for the models to converge. For each model, we sample 1024 samples corresponding to the 32 grid points of Ma and My pairs tiling the entire image space. The model trained with the Adam W optimizer with built-in weight decay, and we employ a learning rate scheduler from Pytorch during training. We did not perform any hyperparameter tuning, although would expect our observations to hold regardless.
B Orthogonality and Parallel Test
(a)
Between surfaces S(fix xx)
S(fix y)
S(fix x,) S(fix x²)
Orthogonality
Orthogonality
ffrd aoras 3D Torus
Wesserseen Detance
30
Warserstein Blstance
20
25
20
10
10
Cifford lorus
Tonus
100
150 Fpochs
200
250
terminal
100
150
200
250
terminal
Parallelism
Paral elism
0.8
Wasvestenn Desance
0.5
llfford
Westerster Distance
0.4
0.4-
02-
0.1
0.-
Cliford
Epachs
Epachs
Figure 8: Wasserstein distanees of orthogonality and parallelism tests statistics between learned representations and ideal Clifford/3D tori.
B.1 Motivation for the Analysis
To better identify whether the diffusion models learn to factorize its representations, we choose to study periodic representations of Gaussian bumps, which expect to form toroidal manifolds in the latent space as opposed to planar manifolds. This is because toroidal manifolds have multiple standard realizations which vary based on their degree of entanglement in the two rings. In particular, the standard torus embeds the rings in three dimensions that couple the two periodic variables, whereas the Clifford Torus embeds each ring in its own orthogonal subspace. This difference in geometry motivates the two analyses that we implemented to identify which geometry the diffusion model learned.
B.2 Computation of the Tests
implement the orthogonality and parallelism tests in 32. Briefly, orthogonality test asks whether the rings that code for each variable lie in orthogonal subspaces-if so, this indicates that the
15

---

torus has a Clifford-likk like geometry. The parallelism test asks whether rings in one variable are aligned in the other variable. If so, this also indicates that the torus has -lifford-like geometry.
The analysis first requires identifying each sing-surprisingly, we found that when the rings formed, the top four Principal Components comprised them. In particular, the first ring were consistently spanned by PCs I and 3, and the second was spanned by PCs 2 and 4 (Fig. f and (g)). Thus, we confirmed that we could use PCA for our analysis.
For each analysis, we identify the ring subspaces by fixing one variable of the input, and sweeping the other variable. We do so for each index, arriving at 32 different subsets of the data for variable and 32 different points for variable y. Next, we compute the top two Principal Components for each subset of the data, thereby identifying the ring.
For the orthogonality test, we randomly draw pairs of rings that are fixed in x and y and compute the pairwise angles between each rings' Principal Components via cosine similarity. For the parallelism test, we compute the randomly drawn pairs of rings that are fixed in the same variable. Next, we create projection matrix P' from the subspaces defined by one of the rings, and calculate the reconstruction error of the second ring's 's projection onto that subspace:
hh pp'
In each test, we aggregate histograms of the respective metric. To identify the similarity between our data and either toroidal embedding, we compute the same histograms for both the Clifford and standard torus (based on simulation with similar spacing in x and samples), and compute the Wasserstein distance between the data and these simulations. The Wasserstein distances between the orthogonality and parallelism test statistics of the model's learned representation and those of an ideal Clifford torus and 3D torus are shown in Fig. [ as a function of the training epochs. We note that the model's 's learned representations at smaller training epochs are not necessarily torus, hence the metric comparison at those points has less significance.
Composition and Interpolation
C.1 Gaussian Stripe/SOS Dataset Generation
To generate ID Gaussian stripe dataset shown in Fig. bb) while maintaining the structure of the 2D conditional embeddings, we embed the 32 x 32 data latent space into an extended 44 x 44 latent space while maintaining the image pixel size be 32 32, Functionally, this allows us to mix into the 2D Gaussian SOS datasets ID (and OD) Gaussian data, as defined by the portion of the Gaussian that is actually visible to the model in the 32 x 32 pixels. Examples of OD, ID, and 2D Gaussian SOS data are shown in Fig. 91) (c)-(c), respectively. Fig. (g) then show the actual data latent representations of the 2D. ID. and 2D+1D datasets that we use to train 2D, ID. and 2D+ ID models described in Sec. 3.2 Here, the green points represent each individual 2D Gaussian SOS data, the blue points represent the 1D Gaussian SOS data, and the yellow points the OD Gaussian SOS data, In all three datasets, we leave out the all Gaussian SOS data centered in the red shaded regions. We note that due to the nature of our data generation scheme, the 2D data only exist within the image space (and some spillover) and the ID (and OD) data only exist in the extended data latent space. Because of the nonzero width of the Gaussian stripes, portion of the data along the rim of the 32 x 32 image space is actually 2D with a portion of the overlaps between the horizontal and the vertical stripes visible within the image frame. In training the 1D model shown in Fig. 4 we have removed the rim where any overlap between the horizontal and vertical Gaussian stripes is partially visible within the image, as shown in Fig. [.). The model trained on such a ID dataset without any compositional example is, as discussed in Sec. 3.2 not able to handle the conjunction of the 2 Gaussian stripes, as shown in Fig. 10 a)).
C.2 Generalization with Few Compositional Examples
In Sec. 3.2 we have seen that the models are sufficiently data efficient in learning the compositionality given a few 2D examples. Here we compare the performance of models trained on a pure ID Gaussian stripe dataset versus a 1D Gaussian stripe dataset + a small amount 2D compositional examples along the image rim, where the conjunction of the stripes are only partially visible within the data images. Here we name the pure ID Gaussian stripe dataset as ID stripes without 2D rim and the alternative
16

---

(a)
(b)
1 Gauasian 908 Data
1D Gauesian 806 Data
20 Gauesian S08 Data
20 Gauesian Bump Data
10
20-
20-
20-
20-
40
Eolanded Stape
Ccaen Sed Space
Estended Spaces
Ddlanded Scace
30
20
30
20
(e)
20 Lakeeit
(T)
Latent
(g)
70-10 aleet
(h)
70 Bumpx 10 Siripen Lawn?
Gesoalan tump daussanaas
10 Gessuan a0e
COO
32 32
Figure 9: Sample Gaussian stripe/SOS/bump data images and dataset latent representations used for generating models in Fig. 4 We generate the 1D Gaussian stripes via embedding the 32 x 32 image frame into an extended latent space of 44 x 44, where the extended portion is not visible to the model. (a)-(c) show example images of OD, 1D, and 2D Gaussian SOS data images, where the dimensionality of the image is defined by the dimension of the portion of data visible within the 32 32 image frame. (d) shows the equivalent augmentation of 2D Gaussian bump data in the embedded space. For each data image, we assign its position labels from range 0 to 44 while keeping the image size to be 32 32 by cropping out the "extended space" in the image. Based on these image data, we construct the 2D. ID, and 2D+1D ID datasets of Gaussian SOS's that we use to train the models in Fig. 4 (e)-(g) show the latent representation of the 2D, ID, and 2D+1D datasets respectively, where the green dots represent 2D Gaussian SOS data, the blue dots represent the ID Gaussian SOS data, and the yellow points the OD Gaussian SOS data. We leave all 2D Gaussian SOS centered in the red-shaded region out of the training dataset for our compositional generalization evaluation. (h) shows the equivalent dataset to the 2D+1D dataset with the 2D Gaussian SOS's replace with 2D Gaussian bumps, where the magenta points represent the 2D Gaussian bump data.
as ID stripes with 2D rim. The data latent representations of both datasets are shown in Fig. 10 a). (d) and the overall, in-distribution, and out-of-distributee sampled image accuracy models trained these datasets are shown in (b) and (e). We note from the generation behavior that the model trained without the 2D rim has a hard time generating the portion of the images where the vertical and horizontal stripes overlap. However, given small set of examples of partial overlaps, the model has completely gained the ability to properly compose the stripes. Specifically, the model trained with the 2D rim achieves the same level of accuracy, if not better accuracy, than the 2D 1D model shown in Fig. 4 with ample of 2D compositional examples. This suggests that models can learn compositional structure efficiently from merely few compositional examples. Moreover, we have shown in Sec. 3.2 (g) that this type of sample efficiency is transferable to an alternative type of compositionality.
C.3 Model's Ability to Interpolate
As discussed in Sec. 3.2 the models have poor ability to interpolate evident from the fact that models cannot generalize to unseen values of & and y in the OOD experiments where an entire range of æ and y are removed from the training dataset (Fig. 4 This is because the model Iearned non- non-parallel representations of different values of z's and y s, hence treating æ and similar to categorical values where there are non-zero overlaps between neighboring categories. We have established in Sec. 3.2 Fig. 4 that the 2D model trained on the 2D Gaussian SOS datasets with the red held-out test regions can indeed compose but not interpolate. Here we quantitatively investigate the model's 's ability to interpolate when trained on 2D Gaussian SOS datasets of varying widths in the held-out x range, as illustrated Fig. a Fig. bb), we show the trained models' overall, in-distribution, and out-of-distribution image-generation accuracy as a function of the held-out range widths. We notice that the accuracy in generating images in the OOD region degrades gradually
as function the held-out range width. This is likely due the fact that the models learn parallel
17

---

(a)
Sitoo
(b)
[d)
Strgoo
(e)
Drta
000
(c)
Figure 10: Comparison between 1D Gaussian stripe dataset with and without 2D Gaussian SOS examples. (a) shows the latent manifold pure 1D Gaussian stripe dataset with rim of Gaussian SOS examples removed along the 32 x 32 image space. The overall, ID, and OOD accuracy of the generate images of the model trained under this dataset shown (b) along with 25 randomly selected generated images in (c). (d) shows the latent manifold of the ID Gaussian stripe dataset with 2D Gaussian SOS data included along the image rim. The overall, ID, and OOD accuracy of the generate images of the model trained under this dataset is shown in (e) along with 25 randomly selected generated images in (f).
encodings of different values of x's and y's that have nontrivial overlaps with each other based on the spatial overlap of the Gaussian SOS data. Similarly the learned representations by the model become increasingly disconnected macroscopic generation behavior of the model.
function of the width sizes, which is in agreement with the
C.4 Model's Ability to Compose
In this section, we investigate model's ability to compositionally generalize when we cut out an dittrr-distribution square region of size W x W in the image data manifold. Specifically, interested two scenario: 1) interior compositional generalization where the OOD cut is in the center of the data manifold, and exterior compositional generalization where the OOD cut the corner of the data manifold. In either case, we want to test out diffusion model's ability to generalize function of the OOD region width W. We test models under both settings of
the OOD regions interior and exterior compositional generalization when trained using 2D Gaussian bump dataset or 2D Gaussian SOS dataset. The results show that for the Gaussian bump dataset, the model's 's ability to compositionally generalize in interior regions slowly decay while in exterior regions quickly decay as function the OOD region width. On the other hand, the model's 's performance remains relatively high for all tested OOD region widths in both interior and exterior compositional generalization when trained the Gaussian SOS dataset. These results show that the Gaussian SOS dataset indeed allows the model to acquire compositionality in more robust well as sample efficient manner.
18

---

(a)
(b)
Data Latent Manirold
1.0
0.8
ODO
Accuracy
0.6
0.4
0OD
0.0
Width
(c)
Width=1.0
Width=3.5
Width=8..
22-
30
20
19
15
16
-32
-30
-20
-2 -24
22
24
26
28
10
Figure 11: Investigation of model's ability to interpolate. (a) Data manifold with a vertical lesion in the center the manifold of various width. (b) Model OOD accuracy decays as function of the width of the lesion. (c) The learned representation (UMAP reduced) of models trained with datasets of lesion widths 1.0, 3.5, and 8.0.
D Manifold formation and Percolation Theory
D.1 Percolation Theoretical Simulation
In Sec. 3.3 we have shown simulation results on percolation Fig. ((c) to match with our experimental data in (e). Specifically, we perform the simulation actual 2D Gaussian bump datasets generated in the same way as those used to train the models generating the experimental results. Here we employ seighborhood-findin
algorithm that recursively iterates through and find all island of connected clusters within given sample of a dataset of and size À x 1024. In the neighborhood- -indonnnnd--n algorithm, we define a given set of 2D Gaussian bumps be connected if the normalized L2 metric (introduced in Appendix A.5 and plotted in Fig. o(b)) is beyond given threshold, which becomes hyper-paramette in our simulation. higher threshold overlap means that the neighborhood hetihbbrrood-fidding algorithm has a more stringent standards for judging which set of bumps are connected, and that harder to percolate given the same amount of data. The simulation results averaged over 5 sample datasets per point are shown in Fig. 13 for various e's and threshold overlaps. As we increase the threshold overlap in the simulation, the percolation thresholds of datasets of various o' become delayed. Given large enough threshold overlap, some of the datasets with small a's never percolates. Moreover, we notice that simulation o 0.1 never percolates due to its width of pixel wide. Nonetheless, the model has learned to generated the 0.1 Gaussian bumps with a larger effective width such that the model able to construct a faithful manifold of the 0.1 dataset at large X. In Fig. ofc), we have chosen to display the simulation results with threshold overlap of 0.005 to best match with the observed experimental data. Such a low threshold overlap signifies that the model is sensitive to relative spatial overlaps among data images.
D.2 Percolation Experimental Details
In testing out percolation theory in experiment, we relied on specific data generation procedure to climinate much of the stochasticity in sampling datasets avoid having train a large number of
19

---

(s)
Interior Comp Gen
(6)
2D Gaussian Bump Interior 000
(d)
2D Goussian Addition Interior OOD
32
Accuracy
Tota Accuracy
Accuracy
Total Accuracy
Accuracy
Accuracy
WXW
Accuracy
Accuraty
5.2
ID
5.0
Region Width
Regian Width
(b)
Exterior Comp Gen
(0)
2D Gaussian Bump Exterior 00D
2D Saussian Addition DOD
32 x
32
0.4
Accuracy
Apcuracy
Total Accuracy
ID
0OD Accurasy
Accuracy
lotel Accuracy Arturayy
OOD
0.-
Wxw
300 Accuracy
0.0
8
ODD Regian Width
70
14
16
20
COD Region Width
10
10
Figure 12: Interior and exterior compositional generalization of 2D Gaussian hump and 2D Gaussian SOS datasets. (a) shows schematic drawing of interior compositional generalization where we leave out data points within an interior red-shaded OOD region of the data manifold. (b) shows the a schematic drawing of exterior compositional generalization where we leave out data points within an exterior red-shaded OOD region of the data manifold. The size of the OOD regions in both scenario W x W. (c).(d) shows the accuracy of models trained on 2D Gaussian bump and 2D Gaussian SOS datasets for interior compositional generalization as a function of various OOD region width W. (f),() shows the accuracy of models trained on 2D Gaussian bump and 2D Gaussian SOS datasets for exterior compositional generalization as function of various OOD region width W.
models. In Fig. 6 d), the datasets used to train model different for different X's are the same across all a's except for the a's. The shared underlying latent representations of the datasets are shown in Fig. 14 for different A's. Moreover, we show an example simulation of the largest connected mass ratio of these specific datasets with threshold overlap of 0.05 in Fig. 14.).
Note that here, described in Sec. 3.3 each subsequent dataset of larger À is superset of all preceding datasets of smaller A's. As a result, we know that if the dataset with a percolates at a given Acs then it automatically percolates any À 2 A.. This is important establishment since the models trained with datasets of small a's seem to have high variance in the outcome training accuracy for different X's, shown in Fig. d). Given the fact that all models trained datasets with larger À have superset of information than those trained on datasets with smaller X's, we can attribute the variance in the experimental outcome partially to the instability in the training itself rather than percolation theory.
One hypothesis for the observed instability in models trained datasets with smaller values is that the small interconnected components within the dataset lead to an unstable learned representation, making it more sensitive fluctuations during optimization. The learned representation of the model corresponding to each data point shown in Fig. ((d) are plotted in Fig. 15] As we can see, with fewer data points (smaller A) and smaller a, the learned representation has less rigid structure as compared to those learned with more data and larger a's. Furthermore, we notice that the training and validation loss of the model grows linearly in Fig. c). This suggest that with a larger set of data, the model has less confidence in its learned representation due to the vast number of data it has to accommodate despite learning a higher-quality representation. Given a smaller dataset, the model can easily accommodate all data with relatively small training/validation losses despite learning low-quality representation.
function of the number of training data, as shown
20

---

Threshald Overtop- 0.001
Threshold Over 0.01
Ma5s Ratio
0-
Mass Rabio
0-04
0.05
00
0.4
0.-
Threshold Overlap- 0.05
Threshole Overlap- 0.1
Mass Ratio
Mass Ratio
0.6
Figure 13: Percolation simulation of various o's with different overlap thresholds.
Thrishu d Doer apm 0.25
1.0
10)
Waddeoon
6- Towwin
Figure 14: Supporting figures for percolation simulation and experiments from Fig. 6 (a) Latent representation of datasets used to generate results in Fig. 6 (b) Simulation of largest connecting cluster mass ratio with threshold overlap 0.05 for a's corresponding to those shown in Fig ((c).
(c) The final training and validation losses function of X of models in Fig. k(e)
NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Answer: [Yes]
We have summarized in the abstract the three points of contributions in terms of studying the factorization and composition diffusion models using 2D Gaussian data. The contributions include observing that 1) model learns factorized, semi-continuous manifold representations, 2) models can compose but not interpolate, and 3) manifold formation in diffusion models follows the prediction of percolation theory.
Justification:
toy experiment settings
21

---

A=
A-04
Aw05
A-0.6
Aw07
Amo.0
A
0-01
0-03
Figure 15: Final learned representations corresponding models that generated results in Fig. 6
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
is fine to include aspirational goals as motivation as long is clear that these goals are not attained by the paper.
2. Limitations
Question:
Does the paper discuss the limitations the work performed by the authors?
Answer: [Yes]
Justification: We have briefly commented on the limitation of our studies in the conclusion section. The primary limitation of our study is the restricted scope to synthetic toy datasets, and mentioned that future investigations should consider more complex synthetic datasets that better capture natural image datasets.
Guidelines:
The answer NA means that the paper has limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specificatiin, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g. if the approach was only tested on few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
22

---

The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't It acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory Assumptions and Proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof?
Answer: [NA]
Justification: Our study is purely empirical and contains theoretical results.
Guidelines:
The answer NA means that the paper does not include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated or referenced the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental Result Reproducibility
Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Question:
Answer: [Yes]
Justification:
All experimental details are spelled out in Sec.
Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is specific model and empirical evaluation, it may be necessary to either make possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
23

---

reproducible, they While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm.
(b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) the contribution is a new model (c.g., a large language model), then there should either be way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome describe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (c.g., to registered users), but it should be possible for other researchers to have some path to reproducing verifying the results.
5. Open access to data and code
Question: Does the paper provide open access the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [NA]
Justification: We have performed the standard experiments on standard UNet architectures with all necessary details of data generation and model training provided in Appendix A
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips cc/
public/ public/guides htta://guiides/ceeuumm.ssss
for more details.
While we encourage the release of code and data, we understand that this might be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurlPS code and data submission guidelines (https
7/nips. cc/
httpccccppppcccccecccceeeel.cllc http:ppllllcceeee muubiccguuidesee twtp//gwieeeeeee/eee Swtp://osesoSSeiiiiioiiia.cammee ssioinoliccyy for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. If only subset of experiments should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing much information possible in supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental Setting/Details
Question: Does the paper specify all the training and test details (c.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: We have enumerated all the details above for a standard training session in Appendix A.6 otherwise specified for specific experiments.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented the core the paper to level of detail that is necessary to appreciate the results and make sense of them.
24

---

The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment Statistical Significance
Question:
Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: Most of our quantitative results are shown for qualitative behavior observations of the model and is not for multiple trials with hyperparameter tuning, with the exception of the percolation data. We specifically discussed the variance regarding the percolation results in Sec. 3.3 and Appendix D
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call a library function, bootstrap, etc.)
The assumptions made should given (e.g., Normally distributed errors).
It should be clear whether the error bar is the standard deviation or the standard error of the mean.
OK to report I-sigma error bars, but one should state The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that out of range (e.g. negative error rates).
If error bars reported in tables plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text.
8. Experiments Compute Resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Question:
Answer: [Yes]
Justification:
The experimental compute resources are specified in Appendix A.6
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper).
9. Code Of Ethics
Question: Does the research conducted the paper conform, in every respect, with the NeurlPS Code of Ethics http:////..aaarssccccppp////aaas
Answer: [Yes]
Justification: We have abid by the NeurIPS Code of Ethics.
25

---

Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. there special consid- eration due to laws or regulations in their jurisdiction).
10. Broader Impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification: Our work has no foreseeable negative societal impact to the best of our knowledge.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no societal impact why the paper does not address societal impact.
Examples of negative socictal impacts include potential malicious unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied particular applications, let alone deployments. However, if there is direct path to any negative applications, the authors should point it out. For example, it is legitimate point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification: Our study analyzes toy models that do not require safeguards in general.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
26

---

Question: Are the creators or original owners of assets (e.g.. code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [NA]
Justification: All datasets are created synthetically from scratch and all models are trained from scratch and originally.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g.. website). the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithheeooo thcode. com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license dataset.
For existing datasets that are re-packaged, both the original license and the license of the derived asset (if has changed) should be provided.
this information is not available online, the authors are encouraged to reach out to the asset's ss creators.
13. New Assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
We provide
assets in this paper.
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/model part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
At submission time, remember anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
Crowdsourcing and Research with Human Subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification: Our paper does not contain experiments on human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included in the main paper.
According to the NeurlPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
27

---

15. Institutional Review Board (IRB) Approvals or Equivalent for Research with Human Subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based the requirements of your country institution) were obtained?
Answer: [NA]
Justification: Our paper does not contain experiments on human subjects.
Guidelines:
The answer NA means that the paper does not involve crowdsourcing nor research with
human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere the NeurIPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
28


Reviews:
[{"review_id": "5zwFgTZ9Ra", "reviewer": "Reviewer_u7D1", "summary": "In this work, the authors investigate how diffusion models achieve compositional generalization. Through controlled experiments on conditional DDPMs with 2D Gaussian data, the authors find that these models learn semantically meaningful, factorized manifold representations of composable features. These representations are orthogonal for independent feature variations but not aligned for different values of the same feature, resulting in superior compositionality but limited interpolation over unseen feature values. The study reveals that a small amount of compositional examples can enhance this capability and links the formation of these representations to percolation theory in physics. This work provides insights into the mechanism of compositionality in diffusion models, guiding future research to improve their factorization and generalization for real-world applications.", "questions": "1. Given the dataset contains only ~100K synthetic images, the model can simply memorize the data. This makes it hard to argue the generalization behavior of the model can be extended to real image diffusion models. \n2. Why the output of layer 4 is used to investigate the internal representations? Do the authors try representations from other layers?", "limitations": "The work sufficiently addressed the limitations.", "rating": 5, "confidence": 3, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "1. The work investigate the factorization and compositionality of diffusion model, which is an interesting and valuable problem.", "weaknesses": "1. The major concern is that the work only considers a highly reduced setting, which makes it hard to validate wether the conclusions generalize to real world applications. Since the setting is oversimplified, it's also unclear how the conclusions can be extended to popular applications like conditional text2image generation. \n2. The performance of diffusion is mostly evaluated with customized metrics without comparison to other models. It's hard to get a sense of how well/bad the model performs in term of the metrics.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for taking the time to provide us with constructive feedback. Please find our responses to the specific concerns and questions below.\n\nWeaknesses\n\n1. We thank the reviewer for the feedback, we kindly refer the reviewer to the global response Section “Regarding the toy setting” addressed to all reviewers.  \n\n2. Due to the simplicity of our toy setting, we design the custom metrics such that it is tailored to our objective of investigating the factorization and compositionality in Diffusion models. Towards this goal, we have specifically designed the task and the metrics accordingly such that they target the model’s ability in recovering the spatial location of the Gaussian bumps. \n\nQuestions\n\n1. Naively, when a model resorts to memorization, it typically suffers from poor performance in the out-of-distribution task sets. We have carefully selected our task and model such that the task is not too trivial for the model and that the model can just memorize all the training data. This is indeed shown through the out-of-distribution model performance evaluation in, for example, Fig. 5. We note that even given a subset of the original dataset, the model has similar in-distribution and out-of-distribution performance, which means that the model is not simply memorizing all in-distribution data but rather trying to learn the correct representation. \n\n2. The reason why we have chosen layer 4 of the UNet as our internal representation is because it better reflects the latent structure of the dataset than other layers. Specifically, we have consistently noticed that the bottleneck layer of the UNet gives diminishing signal due to the utilization of the skip connection. As a result, we chose the layer immediately following the bottleneck layer as our internal representation. Moreover, we have investigated all the layers of the UNet and found that layer 4 gives the strongest semantic signal.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "7q0M2d4wqs", "reviewer": "Reviewer_un3W", "summary": "This paper investigates, on a very simple toy dataset, how conditional diffusion models learn factorized representations of the data, and the extent to which they can compositionally generalize out of distribution. Additionally, the authors make a connection to percolation theory in physics.", "questions": "See weaknesses.", "limitations": "The limitations are mentioned in the discussion section. Unfortunately, I believe some of them may be too large to ignore.", "rating": 5, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "The motivation and research questions are very interesting and relevant for the community. They are outlined in a compelling way in the introduction. The experiments are carefully designed and rather interesting.", "weaknesses": "Main high-level weaknesses:\n- This paper takes a promising approach to a crucial research question, but to me it does not deliver. Although using toy data allows for broader experimentation, the evaluation is overall quite limited. For this reason, though the authors make an effort to clarify the relevance of their results for realistic settings (Sec. 4.1), this still sounds unconvincing. \n- Clarity, especially in the presentation and discussion of the results. I found the results and conclusions significantly more difficult to parse than would be expected from toy experiments.\n\n---\n\nExpanding on the above:\n- In general, the design and motivation for the experimental study is a bit lacking. I appreciate toy experiments, but they should reflect more realistic cases as much as possible, and in this case we are assuming that the model observes basically all information necessary to reproduce the data (as opposed to typical cases where the conditioning signal has significantly less information than the data itself). The authors should provide a solid justification for this choice, and more in general argue how such a toy scenario may be informative for more complex settings. The most natural next step would be to include a dataset that comes significantly closer to realistic settings, although of course some investigations will probably be impossible in that case. The trade-off between relevance and controllability is a hard one, and the current paper seems to be heavily on the latter side.\n- Even sticking to the current toy data, a broader evaluation would be possible and useful. There are several degrees of freedom that can be explored further, e.g., the UNet input noise level at evaluation time, the mutual information between condition and data (which could range from perfect - the current unrealistic case - to zero - the unconditional case), different compositionality patterns (as done in some references in the paper), the layer of the UNet at which representations are extracted.\n- The abstract states \"paving the way for future research aimed at enhancing factorization and compositional generalization\". This may be an overstatement, given my point above. What are concretely actionable insights from these experiments?\n- The experiments here investigate the representations in the last layer of a UNet. Why this choice? Representations in diffusion models could also be considered to be the activations at different layers, especially the bottleneck, which has been investigated in the literature. Another representation can be the latent variable deterministically corresponding to the data using the probability flow ODE.\n    - The bottleneck idea is mentioned only in Appendix A.1, and in a negative way. This reinforces my belief that this toy scenario is too far from realistic settings, where the bottleneck is widely used as representation in diffusion models. However, let me still point out that I find the toy scenario a very interesting and promising direction.\n- As far as I can tell, there is no mention of the noise schedule for training DDPM.\n- When evaluating the representations in the UNet, what is the noise level in the input? I would expect this to significantly affect the representation (especially since you're using the last UNet layer and the UNet is trained to predict noise -- but this is just a hunch).\n- The models here are basically trained to convert the $(\\mu_x,\\mu_y)$ conditioning pair to an image that is deterministically determined from such a pair. I would be a lot more interested in investigating the representations of an *unsupervised* model trained on such data, where the labels are inaccessible but can be used for evaluation -- similarly to how e.g. disentanglement is evaluated.\n    - Alternatively, to mimic real-world conditional (e.g. text-to-image) generation, there could be some stochasticity involved, such that the observed data is not trivially obtained from the conditioning signal.\n- To make the experiments and results fully understable and accessible to a wider range of researchers, I would strongly recommend including a quick introduction to the relevant concepts from geometry/topology.\n- In addition, the individual results subsections seem to lack a clear structure, which makes them not particularly easy to follow. Some more intuitive explanation, as well as highlighting the main takeaways from each experiment, might help.\n\nA few minor or more detailed comments:\n- In the contributions: \"differing values of the same feature are also treated similarly\" - what does this mean?\n- In Section 3.1 there's mention of $x$ and $y$: are these actually $\\mu_x$ and $\\mu_y$, since these are the ground-truth generative factors of the data?\n- At the beginning of the results section, the dataset is modified to have a torus topology. Why not define the dataset like this in the first place?\n- Line 124. \"If the model were to parameterize x and y independently with two ring manifolds, we expect to see a Clifford torus in the neural activations, rather than a different geometry such as the 3D torus.\" What does it mean exactly that the model *parameterizes* x and y? My interpretation is that, when generating data conditional on $x$ and $y$ (which I take to be the means), we can observe how the activations in the pre-determined UNet layer change, as we change $x$ and $y$.\n- Line 127 and following: \"we first confirm that the model indeed learns a torus representation of the dataset\". What would the model alternatively learn? Since we're considering such simple toy datasets, I think it's expected that we can get a full intuition of what is going on. In my opinion, this is not the case. The following lines also involve technical terms from topology that are not properly introduced.\n- It's unclear what exactly effective dimensionality is, why it is important here, and what we can learn from it.\n- The conclusions drawn in the last paragraph of page 4 are not very clear. For example: \"These results suggest that x and y are independently encoded in pairwise orthogonal subspaces, but different values of x’s and values of y’s respectively are not encoded in the same way, i.e. in parallel subspaces\".\n- line 170: \"compositionally generalizing out of the training distribution as well as its ability to spatially interpolate in a single variable alone\". What is the precise difference between these two scenarios? In general, interpolation and composition don't seem to be accurately defined (although they can be implicitly inferred e.g. by lines 172-179).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for taking the time to provide us with constructive feedback. Please find our responses to the specific concerns and questions below.\n\nWeaknesses\n\nExpanding on high-level\n\n1. We kindly refer the reviewer to the first two sections of the global response.\n2. We appreciate the reviewer's suggestions. To summarize your suggestions: a) Input noise level for UNet, b) Imperfect conditioning label, c) Different compositionality patterns, d) Different layers of UNet. We have explored a), c), and d). For a), we have studied the bottleneck and layer 4 at various noise levels (diffusion timesteps) and found minimal differences between representations at different timesteps, with the bottleneck having diminishing signals. Thus, we used the final timestep output from layer 4 for our analysis. For c), as shown in Fig. 4, the model learned both multiplicative and additive compositionality with the full range of 1D Gaussian Strips and a few compositional examples. For d), we investigated all UNet layers and found layer 4 better reflects the latent structure of the dataset than other layers, so we used it for our analysis. We have not considered b). While interesting, this doesn't allow precise prompting of the model to output a specific Gaussian at a desired location, complicating evaluation. As a reminder, our focus is on whether the model can even, given perfect label information, organize learned data into an efficient and meaningful representation for generalization. The pursuit of b) falls outside of our research scope. \n3. We thank the reviewer for this feedback. We will modify this sentence in the abstract. \n4. Although Diffusion model bottlenecks have been shown to encode semantic information, in our case, layer 4 acts as the bottleneck due to the skip connection. This does not affect our main point, as we are focused on the factorization of the model's learned representation.\n5. We kindly refer the reviewer to the global response section on “Notes on additional figures - Additional Figure 1: Impact of Annealed Noise Schedule on Learning Rates”. \n6. See response to “Expanding on high-level” 2.\n7. Investigating representations learned by unsupervised models is intriguing. Previous work has focused on disentanglement in unconditional Diffusion models, but evaluating compositional generalization without explicit conditional input is challenging. In terms of stochasticity, our conditional training uses classifier-free guidance, dropping labels 10% of the time to co-train conditional and unconditional models. Moreover, we know that the model isn't solely relying on conditional input, as it sometimes fails to learn even with perfect conditions (e.g., Fig. 4(c)-(e)). Future work should nevertheless evaluate unconditional models or conditional models with imperfect input for their factorization and compositionality.\n8-9. We will refine our manuscript to reflect the suggestions.\n\nMinor\n\n1. By the original sentence, we meant that different values of the same feature (e.g., $x = 13, 14, 15$) are encoded similarly to how different features ($x = 13$, $y = 13$) are encoded, treating $x$ and $y$ more like categorical than continuous variables. We will clarify this in the original sentence.\n2. We refer to $x$ and $y$ as features and $\\mu_x$ and $\\mu_y$ as the center locations of the Gaussian bumps in a data image. The latter is intrinsic to the dataset, while the prior is learned by the model. \n3. Although periodic boundary conditions could be enforced in all datasets, the resulting torus manifold is nonlinear and less straightforward to analyze using linear methods. Otherwise, there's no fundamental difference between datasets with and without these conditions. \n4. By “parameterize $x$ and $y$,” we mean how the model represents the two features. Jointly, the model may learn a 2D look-up table (3D torus), while independently, it may learn $x$ and $y$ as 1D rings (4D Clifford torus). Our geometry/topology test distinguishes between these scenarios.\n5. Our tests detect whether the learned representation is a 3D or Clifford torus. This requires the object to be a torus, which isn't always obvious during training. The persistent homology results in Fig. 2(b) help clarify this. We will refine the manuscript to be more precise.\n6. Effective dimension, computed by the participation ratio (lines 134-135), measures the intrinsic dimensionality of learned representations. Plotting it over training epochs helps us understand if the model learns a Clifford torus (dimensionality of 4) and whether it does so from first learning a 3D torus (dimensionality of 3). Fig. 2(c) shows higher than 4-dimensional effective dimensionality, indicating independent learning of $x$ and $y$. Fig. 2(d) eigenspectra and Fig. 2(g) PCA projections suggest the learned representation isn't a perfect Clifford torus but higher-dimensional and cone-shaped. We will add more details on this in the manuscript. \n7. The model treats $x$ and $y$ as categorical variables with non-zero overlaps. This means $x=16$ is a separate category from $x=17$, not neighboring values of a continuous variable. Technically, if $x$ and $y$ were continuous, different values should be encoded in parallel subspaces, like a Clifford torus. The model learns a hyper-factorized version, representing $x$ and $y$ as categorical. We will clarify this in the manuscript.\n8. \"Compositional generalization\" means combining components in ways not seen in the training set (e.g., given ${(a, b), (c, d)}$, output ${(a, d), (c, b)}$). Interpolation refers to combining values (e.g., output $x=16$ given $x=15$ and $x=17$). These are distinct forms of generalization, which we probe separately in Sec. 3.2. We will define these clearly at the beginning of Sec. 3.2.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "h04PL0ynBc", "reviewer": "Reviewer_RMNd", "summary": "This paper investigates how and when diffusion models learn factorized representations of composable features. To this end, the authors construct controllable synthetic datasets by compositionally combining 1D and 2D Gaussian data and examine the factorized representation and the compositional generalization capability of the diffusion model. Systematic analysis on the controllable dataset indicates that the diffusion model learns orthogonal but not necessarily parallel representations and is capable of compositionally generalization to OOD samples if a few compositional examples are provided. Additionally, the authors draw a connection to percolation theory, suggesting that a certain amount of correlated data is required to learn factorized representations.", "questions": "- Are the conclusions and implications specific to the diffusion model? Although the paper claims to investigate the compositionality of diffusion models, the experiments and analysis do not seem diffusion-specific. Other general generative models might exhibit similar behavior.\n- In the Gaussian Bump + 1D Gaussian Stripes experiment (Figure 4(g)), what is the accuracy when only 2D Gaussian Bump data is used? Comparing this value would help identify the performance boost from compositional generalization.\n- How did you format the inputs when training the diffusion model on 1D Gaussian stripe data? Did you input a null value (e.g., $(\\mu_x, \\phi)$)?\n- How does the model generate OOD samples in 2D Gaussian Bump Data? From the experiments on the 2D Gaussian Addition dataset, it is concluded that the model cannot interpolate well on unseen examples, as implied by the low accuracy in the intersection areas. By adding 1D Gaussian Addition data, the model learns from single 1D stripe patterns and composes that information to generate OOD samples in the test regions. However, in 2D Gaussian Bump Data + 1D Gaussian Addition data, the model never observed the 2D Gaussian Bump samples in the test regions. How could adding 1D Gaussian Addition data lead to improvement?\n- In Figures 4(c), 4(d), and 4(e), interestingly, the model trained only on 1D Gaussian data already predicts reasonably on $y$ values (but very low for $x$ values). Is there any particular reason for this result?", "limitations": "The conclusion section describes the limitation regarding the restricted scope and simplicity of the toy dataset. While the study provides valuable insights, the findings are based on synthetic datasets with simple structures, which may not fully capture the complexities of real-world data.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "- The paper is well-written and easy to follow.\n- The simple yet well-controlled experiments and comprehensive analysis provide a clear understanding of factorization and compositionality in diffusion models.\n- Connecting the empirical findings to percolation theory improves the understanding on the emergence of factorization.", "weaknesses": "- The experiments are conducted solely on a simple dataset. The 2D Gaussian Addition dataset features only basic additive compositions of 1D Gaussian sprites. It remains unclear whether the paper's claims extend to more complex compositions, such as multiplicative compositions involving scale, color, etc.\n- Some of the conclusions, such as those regarding the properties and requirements for compositional generalization, have already been addressed in previous work [1]. [1] provides theoretical analyses and conditions for sufficient compositional support. Therefore, the experiments and conclusions from section 3.2 provide limited additional insights.\n\n[1] Wiedemer et al., “Compositonal Generalization from First Principles”, in NeurIPS 23.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for taking the time to provide us with constructive feedback. Please find our responses to the specific concerns and questions below.\n\nWeaknesses\n\n1. We appreciate the reviewer’s feedback. In our paper, we explored both additive and multiplicative composition with the 2D Gaussian Addition (addition of two 1D Gaussian Stripes) and the 2D Gaussian Bump (multiplication of two 1D Gaussian Stripes) datasets. Fig. 4(g) shows that the model can generalize to new 2D Gaussian Bumps given all 1D Stripes and a few 2D Bumps. This multiplicative composition is similar to transformations like scale, color, and style, as it involves \"masking\" one 1D Stripe with another.\n2. We thank the reviewer for bringing Ref. [1] to our attention. While it similarly concludes that gaps in the support of the training dataset hinder learning, it lacks insights from a manifold formation perspective. Their experiments use sprites with a mix of (semi-)continuous and categorical latent variables, showing general results across all types. Missing support in categorical variables makes sense, as a model cannot generate images of monkeys after only seeing cats and dogs. However, with continuous latent features like $x$- and $y$-positions, one would expect interpolation to be possible (e.g., generating a sprite at $x=16$ after seeing $x=15$ and $x=17$). Our experiments focus on the model’s ability to represent continuous latent features. In Sec. 3.1, we found that the model represents continuous features similarly to categorical variables, with some overlap. Consequently, as shown in Sec. 3.2, the model struggles with interpolation. The main takeaway is that the model excels at composition but fails at interpolation. Contrary to popular belief, Diffusion models can perform well at compositional generalization but struggle with continuous latent features, hindering interpolation and robust generalization. \n\nQuestions\n\n1. While we believe some observations may apply to other generative models, we refrain from general claims, focusing our study on Diffusion models and their factorization and compositionality. Similar studies on vision- or language-based generative models are beyond our current paper's scope.\n2. We thank the reviewer for the constructive suggestion. We refer the reviewer to the global response section on “Notes on additional figures - Additional Figure 2: Data Efficiency Scaling”. \n3. When generating 1D Gaussian Stripes, we embed the data image (32 x 32) into a larger canvas (44 x 44), centering it and creating a 6-pixel border. We generate 2D Gaussian Additions centered in this extended border region, most of which (except the corner ones) will have one Gaussian Stripe partially visible in the 32 x 32 space. We then crop the central 32 x 32 pixels, keeping the label as the center of the 2D Gaussian Addition in the extended space. Thus, 1D Gaussian Stripes have one coordinate outside the 32 x 32 image space, preserving the 2D structure for both 1D and 2D data points. Details of the data generation process are in Sec. C.1 of the Appendix.\n4. The experimental setup is detailed in Sec. C.1 Fig. 8 of the Appendix. The model effectively learned spatial information from the 1D Gaussian Stripes. Given a few examples of 2D Gaussian Bumps, it learned to multiplicatively compose the 1D Gaussian Stripes into Bumps. Combined with the results in response to Question 2, this demonstrates the model's ability to transfer knowledge across different forms of compositionality. \n5. We appreciate the reviewer's detailed observation. The 1D models in Fig. 4(c)-(e) are trained on equal numbers of vertical and horizontal 1D Gaussian Stripes, providing the same training examples for learning $x$ and $y$. While the exact reason for the higher accuracy in generating $\\mu_y$ is unclear, we observed that the model often defaults to generating 1D Stripes when it fails to generate the intersection of two Stripes, possibly skewing the accuracy distribution between $\\mu_x$ and $\\mu_y$.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "zKgEBAIqY1", "reviewer": "Reviewer_VwNJ", "summary": "This paper investigates the capabilities of diffusion models, particularly Denoising Diffusion Probabilistic Models (DDPMs), in learning factorized representations and achieving compositional generalization. The authors aim to quantify this by analyzing mechanisms to train the model, supporting the hypothesis that the architecture of diffusion models has an inductive bias towards such factorized representations. Their results on a toy dataset suggest that, to achieve out-of-distribution compositional generalization, the training set must: i) contain at least a few compositional examples of the factors, and ii) present the factors independently of each others across the full range of their variability. If either of these conditions i) or ii) is not met, the model will fail to generalize out of distribution.", "questions": "N/A", "limitations": "See “Weaknesses”", "rating": 8, "confidence": 3, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "- The manuscript is particularly well-written and presented, with clear and thorough explanations. The methodology and experiments are relevant and consistent, and they are both well-explained and well-illustrated.\n- The originality of this work lies in explaining the factorization capability emerging in the manifold from the perspective of percolation theory. This supports the hypothesis that the training set needs a certain level of correlation among the components of independent features to achieve a faithful representation, which in turn leads to compositional generalize.", "weaknesses": "The only weakness I can think of in this work is that it is conducted in simplistic toy settings. However, I don't believe this discredits the work and interesting experiments presented in the paper. Additionally, the authors acknowledge this issue and suggest exploring more naturalistic and relevant experiments in future studies.\n\nSome typos: \n- Line 7: first time using the abbreviation ‘DDPMs’ in the abstract, maybe precise what it is (Denoising Diffusion Probabilistic Models)\n- Line 198: at the end of the sentence, there is an extra ‘not’", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for taking the time to provide us with constructive feedback. Please find our responses to the specific concerns and questions below.\n\nWeaknesses\n\n1. We thank the reviewer for the very positive feedback. As noted in the global response section “Regarding the toy setting,” using a simple toy setting allows us to carefully examine various effects. Future investigations on more realistic datasets are necessary but could introduce many competing effects. Despite the simplicity, our study provides valuable insights into compositionality and generalization in Diffusion models. Future research should explore why Diffusion models cannot encode continuous latent features continuously and how percolation theory of manifold formation can be applied to natural image data.\n2. We thank the reviewer for pointing out our typos. We will fix them in the manuscript.", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:
We thank all the reviewers for the constructive feedback and suggestions. Below are some of the recurring concerns/questions that we would like to address to all reviewers. 
 
Regarding the toy setting

It is a tradeoff between a simple setting, which allows for controlled studies to isolate all possible effects, and a more complex setting that mimics popular application regimes. We agree with the reviewers that in larger, more realistic settings, richer dynamics absent from the toy setting would emerge. However, this does not undermine the relevance of our observations or the importance of starting with simple, controlled scenarios, especially given the complexity of neural networks.

In fact, We expect many insights from our toy experiments to hold for larger models. For example, our experiments showed that the model's inability to interpolate well is due to the categorical-like encoding of continuous latent features, highlighting the limited ability of Diffusion models to encode continuous quantities key to image generation according to real-world physics. Our observation of the connection between percolation theory and manifold formation should also be relevant to more realistic datasets, where the measure of "overlap" is more abstract (e.g., cosine similarity between images). Using Gaussian bumps, a well-studied system in percolation theory with computable thresholds, validated our observations. Extending this overlap measure to natural image data and studying manifold formation in more realistic settings is a promising future direction.

Finally, interpretability studies of Diffusion models on natural/synthetic datasets show varying degrees of success, especially in compositionality and factorization. Even slightly more realistic synthetic datasets than Gaussian datasets (e.g., dSprites, CLEVR) have led to conflicting conclusions, as noted in the Related Work section. This underscores the need for careful and controlled experiments, motivating our study of the Gaussian dataset, which has a low-dimensional, intuitive latent representation. This allowed us to analyze its geometry and topology to study its factorization explicitly. More importantly, our case study of the Gaussian dataset revealed phenomena of greater scientific interest than we initially expected. These observations, impossible without the toy setting, provide a foundation for understanding larger, real-world models. Future work should aim to validate and extend these observations to larger models with more realistic datasets.

Regarding the specific experimental setup 

The task of image reconstruction of Gaussians given perfect conditional information (center coordinates) is inspired by traditional cognitive psychological studies. In these studies, subjects perform simple tasks while their behavior and brain activities are recorded. For tasks involving continuous latent factors (e.g., angles, color hues), humans and animals can represent these data using continuous attractors (e.g., rings or lines), which are efficient and robust. Our study aims to determine if Diffusion models can similarly learn these continuous attractors in a factorized and generalizable manner, akin to biological brains. Thus, we designed our task to mimic cognitive experiments, analyzing the "brain activity" (Sec. 3.1) and "behavior" (Sec. 3.2) of the Diffusion model during the task. Our findings reveal that, unlike biological brains, Diffusion models do not learn continuous manifolds representing continuous data variation, making them less efficient and robust.


Notes on additional figures

We have included two additional figures in the new supplementary material attached below. 

Additional Figure 1: Impact of Annealed Noise Schedule on Learning Rates

We included a figure showing how an annealed noise schedule affects the learning rates of different concepts. Using a noise schedule similar to the original DDPM paper, we analyzed the relationship between learning rates and signal-to-noise ratios. We hypothesize that high-frequency details (e.g., moles) are drowned out more quickly with an aggressive noise schedule, while low-frequency features (e.g., hair color) persist longer and are easier to learn. This aligns with existing literature observations that low-frequency features are learned before high-frequency ones. To verify, we trained 1D conditional Diffusion models on sinusoidal data with high- and low-frequency components. Figure 1(b) shows the model learns the low-frequency component faster and more accurately due to the annealed noise schedule. Figure 1(c) illustrates that more noising steps result in diminishing signal-to-noise ratios in most noised image samples, making high-frequency details harder to learn. We have previously omitted this from the main paper as it’s not central to our main message.

Additional Figure 2: Data Efficiency Scaling

We included a figure on the data efficiency scaling of Diffusion models across various datasets. Our original paper (Sec. 3.2) shows that combining 2D Gaussian data with 1D Gaussian data reduces the number of compositional examples needed for compositional generalization. Figure 2 in the supplementary material compares data efficiency between models trained on i) 2D Gaussian Bumps, ii) 2D Gaussian Bumps + 1D Gaussian Stripes, and iii) 2D Gaussian Bumps + 2D Gaussian Additions + 1D Gaussian Stripes. Figure 2(a)-(c) show that models trained on ii and iii achieve higher accuracy with fewer compositional examples. We also show in Figure 2(d) that for dataset size $N$, models trained with 1D Gaussian Stripes are more data-efficient, with data needed to reach an accuracy threshold growing linearly rather than quadratically. This signifies that not only can the model learn multiple forms of compositionality, but do so in a more data efficient manner when the dataset is mixed in with the set of 1D components, which potentially suggests a more data-efficient training approach for Diffusion models.

Meta-review: This paper investigates the internal mechanisms of diffusion models, focusing on their ability to learn factorized representations that support compositional generalization. The work provides valuable insights into the inductive biases of diffusion models through controlled experiments on synthetic datasets and offers a normative explanation inspired by percolation theory.

The reviewers had mixed opinions about the reliance on toy datasets, with some concerns about the generalizability of the findings to more realistic settings. However, Reviewer VwNJ strongly supported the paper, noting its straightforward characterization of conditions under which factorization and composition are achieved, and believes it is worth sharing with the community. Reviewer un3W, while acknowledging some limitations, also agrees that the paper has value and supports its acceptance, with the suggestion that the authors explicitly acknowledge the scope of their experiments and improve the clarity of their presentation, particularly in sections involving technical concepts from topology and geometry.


Based on the overall positive feedback and the constructive discussions among the reviewers, I recommend accepting this paper. I encourage the authors to consider Reviewer un3W's suggestions for improving the clarity and accessibility of their work, especially for readers less familiar with the technical terminology used.
Decision: Accept (poster)