Compositional Abilities Emerge Multiplicatively: Exploring Diffusion Models on a Synthetic Task
Authors: Maya Okawa
Ekdeep Singh Lubana
Robert P. Dick
Hidenori Tanaka
Venue: NeurIPS 2023 poster (2023)
Keywords: Diffusion model; Emergence; Emergent capabilities; Science of deep learning; Mechanistic interpretability

Abstract: Modern generative models exhibit unprecedented capabilities to generate extremely realistic data. However, given the inherent compositionality of the real world, reliable use of these models in practical applications requires that they exhibit the capability to compose a novel set of concepts to generate outputs not seen in the training data set. Prior work demonstrates that recent diffusion models do exhibit intriguing compositional generalization abilities, but also fail unpredictably. Motivated by this, we perform a controlled study for understanding compositional generalization in conditional diffusion models in a synthetic setting, varying different attributes of the training data and measuring the model's ability to generate samples out-of-distribution. Our results show: (i) the order in which the ability to generate samples from a concept and compose them emerges is governed by the structure of the underlying data-generating process; (ii) performance on compositional tasks exhibits a sudden "emergence" due to multiplicative reliance on the performance of constituent tasks, partially explaining emergent phenomena seen in generative models; and (iii) composing concepts with lower frequency in the training data to generate out-of-distribution samples requires considerably more optimization steps compared to generating in-distribution samples. Overall, our study lays a foundation for understanding emergent capabilities and compositionality in generative models from a data-centric perspective.

Compositional Abilities Emerge Multiplicatively: Exploring Diffusion Models on a Synthetic Task
Maya Okawa" 1.2, Ekdeep Singh Lubana*1.3 Robert P. Dick Hidenori Tanaka +1.2
CCetter for Brain Science, Harvard University, Cambridge, MA, USA Phhysccs & Informatics Laboratories, NTT Research, Inc., Sunnyvale, CA, USA SEECS Department, University of Michigan, Ann Arbor, MI, USA
Abstract
Modern generative models exhibit unprecedented capabilities to generate extremely realistic data. However, given the inherent compositionality of the real world, re- liable use of these models in practical applications requires that they exhibit the capability to compose novel set of concepts to generate outputs not seen in the training data set. Prior work demonstrates that recent diffusion models do exhibit intriguing compositional generalization abilities, but also fail unpredictably. Mo- tivated by this, we perform a controlled study for understanding compositional generalization in conditional diffusion models in synthetic setting, varying dif- ferent attributes of the training data and measuring the model's ability to generate samples -istiiibiitiii Our results show: (i) the order in which the ability generate samples from a concept and compose them emerges is governed by the structure of the underlying lata-generating process; (ii) performance composi- tional tasks exhibits a sudden "emergence" due to multiplicative reliance on the performance of constituent tasks, partially explaining emergent phenomena seen in generative models; and (iii) composing concepts with lower frequency in the training data generate out-of-distribution samples requires considerably more op- timization steps compared to generating in-distribution samples. Overall, our study lays foundation for understanding emergent capabilities and compositionality in generative models from a data-centric perspective.
Introduction
The scaling of data, models, and computation has unleashed powerful capabilities in generative models, enabling controllable synthesis of realistic images [1-11], 3D scenes [12-16], videos [17- 25], accurate image-editing [26-31], and semantically coherent text generation [32-46]. With increased interest in incorporating these models in day-to-day applications [47-50], e.g., to improve robotic systems via better planning and grounding [51 [51-59], analyzing and improving their reliability has become crucial.
With the motivation above, we study compositional generalization abilities of conditional diffusion models, i.e., diffusion models that are conditioned on auxiliary inputs to control their generated images (e.g., text-conditioned diffusion models [6. 26]). Given the inherent compositionality of the real world, it is difficult to create training dataset that exposes model to all combinations of different concepts underlying the tata-ggnneraiig process. We therefore argue that compositional generalization is central to model reliability in out-of distrritiitiiitionn scenarios, i.e., when the model has to compose novel set of concepts generate outputs not seen in the training data set [60, 61]. As shown by several prior works investigating the compositional generalization capabilities of off-the- shelf text-conditioned diffusion models [62-72], modern diffusion models often compose complicated
*Equal contribution. Email: (mayaokawa, ekdeeplubana, hidenori_tanakaaJe hawvard. edu.
37th Conference on Neural Information Processing Systems (NcurIPS 2023).

---

WWite color- Izard"
"Green-coloree Ieard"
'Whte-coloed goldish
"Green-colo goldfish
"Whitecolored panda"
Green colored panda
Blue-colored "Magenta-coless
"Blae-colored goldish" "Megenta-colordd goldiish "Bue-colored panda" "Magenta-cobere pandia'
Figure I: (Lack of) Compositionality in text-conditioned diffusion models. Images generated using Stable Diffusion v2.1 [73]. We see diffusion models conditioned on text descriptions of the concepts in image often allow generation of novel concepts that absent from the training data, indicating an ability to compose learned concepts and generalize out-of-distributiin (lizard and goldfish panels). However, the model sometimes unpredictably fails to compose concepts has learned, i.e., compositional capabilities depend on the specific concepts being composed. For ex., generating panda in the above figure is difficult for the model, likely because it has not been exposed to different colors of pandas. The model instead alters the background or lighting to change the color.
concepts, producing entirely non-existent objects, but also unpredictably fail compose apparently similarly complicated concepts (see Fig. 1). It remains unclear why models produce novel samples by composing some concepts but not others. Indeed, what concepts does a model learn compose? What differentiates these concepts from the ones the model is unable to compose?
"Model Experimental Systems" Approach with Interpretable Synthetic Data. To address the above questions, we seek take systematic approach to understanding complex systems, such as gener- ative models, by dividing the process into hypoth- esis generation and testing. Specifically, drawing inspiration from the natural sciences, we adopt a nodel-expprimental systems approach, which in- volves studying simplified. controllable, and steer- able experimental systems to develop mechanistic hypotheses applicable more complex systems. For instance, in neuroscience research, model organisms like mice or fruit flies are used study neural mech- anisms and develop hypotheses that can be tested on the larger-scale human brain [74-76]. Similarly, in our work, we design synthetic experimental setup that pursues simplicity and controllability while pre- serving the essence of the phenomenon of interest, ie., compositional generalization. Specifically, our data-generating process tries to mimic the structure of the training data used in text-conditioned diffusion models by developing pairs of images representing geometric objects and tuples that denote which con- cepts are involved in the formation of given image (see Fig. 2). We train diffusion models on synthetic datasets sampled from this data-generating process, conditioning the model on tuples denoting which concepts an object in the image should possess, while systematically controlling the frequencies of concepts the dataset. Thereafter, we study the model's 's ability to generate samples corresponding to novel combinations of concepts by condition- ing the denoising process on a correspondingly novel tuple, thus assessing the model's ability to compositionally generalize. This approach allows to systematically investigate the key properties of a dataset enabling compositional generalization in an interpretable and controlled manner in conditiones conditioned diffusion models. Our main contributions follow.
(a)
Train Set
(c) Test
Tuple
(1.a)
(2,a) (1.b)
(2.b)
?
Image
(b)
(2.a)
(L.a)
(1,b)
Shape
Color
circle, triangle a: red, blue
Figure 2: Compositionality in minimal- istic conditional generation task. (a) We train diffusion models on pairs of images and tuples, where the tuples denote the values of the concepts represented the image (e.g., values of color and shape in the figure). (b) Samples between which only a single tuple el- ement differs simplify the learning of capa- bility to recognize and alter the concept distin- guishing the images. (c) To test the existence of such capabilities of a trained model, we ask the model generate images correspond- ing to novel tuples that are out-of-distribution, hence requiring compositional generalization.
Concept graphs: A simple, synthetic framework for studying conditional diffusion models. We develop a minimalistic abstraction for training conditional diffusion models by introducing a framework called "concept graphs". The framework forms the foundation of our systematic

---

study of how diffusion models learn to generate samples from set of concepts and compose these concepts to generate -itto---iitribution samples. To our knowledge, our results are the first demonstration of perfect compositional generalization on (synthetic) vision problem.
Multiplicatiive emergence of compositional abilities. We use our model system's 's interpretability monitor learning dynamics of diffusion model's capabilities and the ability to compose them data. As we show, a diffusion model first memorizes the training
to produce dataset and then sequentially generalizes to concepts that are greater "distance" from the training distribution. Since progress in learning each capability multiplicatively affects the model's performance in compositional generalization, we find a sudden ability to compose and produce samples dittof--ditributionn "emerges". We thus hypothesize compositionality may partially explain emergent phenomena seen in modern generative models [77-81].
Investigating challenges for compositional generalization. We systematically examine the challenges arising from decreased frequency of specific concepts necessary for learning a cor- responding capability. We further evaluate the effectiveness of using fine-tuning to address misgeneralizzaions in adversarial settings, and find that it is generally insufficient to enable the learning of new capabilities.
Before proceeding, we emphasize the trade-off's and limitations of our model-expenie experimeeli systems approach. Just as neural mechanisms identified in model animals cannot be directly applied to human medical applications, our observations should not be considered definitive conclusions that can be directly transferred to modern large generative models. Instead, our study aims to establish conceptual frameworks, identify data-centric control variables, and formulate mechanistic hypotheses, paving the way for further theoretical and empirical explorations of larger models.
2 Related Work
Diffusion models. Diffusion models are the state-of-the-art method for generating realistic visual data [1-11]. They are often trained using image-text pairs, where the text is processed using large language model to produce semantically rich embeddings that allow controlled generation of novel images and editing of existing images [26-31]. Such conditional diffusion models are easy to test for compositional generalization, as one can directly give a text description that requires composition of concepts the model is likely to have learned (c.g., avocado and chair) to produce images that were not encountered in the model's training data (c.g., avocado chair; see [1, 6)). Such results demonstrate the model's ability compose and generalize emphasize that text-conditioned models may fail to generalize compositionally if the text model is unable to sufficiently disentangle relevant concepts in the text-embedding space. To avoid this pitfall, we use ordered tuples that denote precisely which concepts are involved in an image's 's composition.
However, we
Compositional generalization. Compositionalityy is an inherent property of the real world [82], wherein some primitive such as color can be composed with another primitive such as shape to develop or reason about entirely novel concepts that were not previously witnessed [60]. Notably, compositionality is hypothesized to play an integral role in human cognition, enabling humans to operate in novel scenarios [83-87]. Inspired by this argument, several works have focused on developing [67, 68, 88-93] and benchmarking [64, 66, 90, 94-102] machine learning models to improve and analyze compositional generalization abilities. We especially highlight the works of Lewis et al. [96], who use a similar model experimental system approach as ours to evaluate factors influencing compositionality in contrastive language-image pretraining (CLIP) [34]. Our focus on generative models, instead of representation learning, distinguishes our results however. Relatedly, note the notion of compositionality has been discussed in several recent works in the fields of disentangled and causal representation learning [101, 103-107J. Our framework for systematic understanding compositionality in generative models was heavily influenced by the synthetic dataset design considerations promoted in these papers. We also emphasize that thorough formalization of compositionality in modern machine learning is generally lacking, but highlight the notable exceptions by Hupkes et al. [108] and Wiedemer et al. [106]. To avoid ambiguity, we provide a precise formalization that instantiates our intended meaning of compositionality.
3 Concept Graph: A Minimalistic Framework for Compositionality
In this section, we present the concept graph framework, as illustrated in Fig. 3, which enables us to visually depict compositional structure of our synthetic data and forms the basis for generating
3

---

hypotheses and designing experiments in our work. We begin by defining the essential building blocks of our framework: concept variables and concept values. In the following, we call a specific output of our data-generating process an "object" (see Fig. 3).
Definition 1. (Concept Variables.) A concept variable v; uniquely characterizes specifie property of an object. = {U1, 02...., Un denotes a set of concept variables, such that, given an object vz(x)) returns the value of the ith concept variable in that object.
For instance, for geometric objects shown in Fig. 3, concept variables may include shape color size, location, and angle. These variables take on values from a pre-specified range, called concept values, described below.
Definition 2. (Concept Values.) For each concept variable ot V, C. {Ca,, C22, Cik. denotes the set of Kú possible values that vi can take. Each element of C; is called a concept value.
For example, in Fig. 3, the concept values for the shape variable are in the set {circle, triangle, square). while for the color values are in the set (red, blue, green). Given concept variables, with each variable v; having k concept values, there can be II 1 k distinct combinations that yield a valid object. We variables, Ul,...,Up (p < n), to define the notion of a "concept class".
unique combination of subset of these concept
Definition 3. (Concept Class.) concept class C' is an ordered tuple (v1 = c1, = C2...., Cp). where each C₁ is concept value corresponding the concept variable Uú. If an object belongs concept class C, then (x(x) = c; Vi ,,...,
concept variables are free in the above definition. When these free variables are specified, we get specific object from our data- lata-generating process. That is, a concept class represents a family of objects by specifying the values of pre-defined subset of concept variables. For example, species (concept class) of lizards; here, color of the lizard serves free variable. Similarly, in the geometric objects scenario of Fig. "small red circle" would be concept class for which the shape. color, and size variables have been assigned specific values. Objects are images designated by associating precise values with the remaining concept variables of locati on and angle. Next, introduce the notion of concept distance, which serves proxy to succinetly describe the dissimilarity between two concept classes.
Note that the remaining
Fig. 1, different colored lizards instantiate images (objects) from the
Definition 4. (Concept Distance.) Given two concept classes C(J) (cf" . D...., 2) and = -(2)= -c(2) (2) ..... "....Ca (2), the concept distance d(C(1), C(2)) is defined as the number of elements that differ between the two concept classes: d(((1), ((2)) = Σm I(c,)) cq (2)), where I(ci, 62.) 1 if en # ezi and [cis, 021) 0 otherwise.
...,.Ch)
Concept Variables
Concept Class
Objects
Concept Graph
shape=0tcircle
011
111
size=O:small color-1:blue
shape color size location angle
001
101
010
110
#hape=1:triangl
size-O:large
000
100
color=0:red
Figure 3: Concept graphs. We organize our study using simple but expressive framework called concept graphs. The basis of concept graph is set of primitives called concept variables (e.g.. shape, color. etc.). subset of these variables is instantiated with specific values to yield concept class, c.g., (shape = O, size = O, color = 1} implics a small, blue circle. This is akin to defining broad set objects that share some common properties, e.g., all lizards in Fig. 1 belong to the same species, despite their diverse colors. A specific object in this class is instantiated by fixing the remaining variables, e.g., small, blue circles at different locations. Each concept class corresponds to graph node, where nodes are connected if their concept classes differ by a concept distance of 1.

---

The concept distance quantifies the dissimilarity between two concept classes by counting the number of differing concept values. It is important note that this distance serves only as null model because each axis represents distinct concept variables and each of the variables can assume various possible concept values. We are now ready to define the notion of a concept graph (see Fig. 3), which provides a visual representation of the relationships among different concept classes.
Definition 5. (Concept Graph.) A concept graph G == (N, E) consists of nodes and edges, where each node E N corresponds to concept class, and an edge E. connects two nodes nn and n2 representing concept classes C() and (122, respectively, if the concept distance between the two concept classes is I, i.e., d(C()), C(2))=
A concept graph organizes different concept classes as nodes in the graph, while edges denote pairs of concept classes that differ by a single concept value. An ideal conditional diffusion model, when trained on a subset of the nodes from this graph, should learn capabilities that allow it to produce objects from other concept classes. We formalize this as follows.
Definition 6. (Capability and Compositionality) Consider diffusion model trained to generate samples from concept classes Č {C1; C2. ..... Cr}- We define a capability the ability to alter the value of a concept variable U; to a desired value ci. We say the model compositionally generalizes if it can generate samples from a class Č such that d(O, C.) 2 IAC, E C.
The ideas above are best explained via Fig. 4. Specifi- cally, assume we train diffusion model on data from a subset of concept classes, i.e., a subset of nodes in the concept graph. To fit the training data, the model might just memorize the training data or it may learn the relevant capabilities to alter specific concept variables; e-g., given samples from classes 0000, 0010, and 0001 in Fig. 4 (a), the model may just memorize the data or learn how to alter the third and fourth concept variables. Models that just mem- orize the training data lack the capability to generate samples from out-of disiiitbibttiinn classes (e.g.. 1100). In contrast, the ability to compose concepts would en- able the model to produce nutoo--distriuution samples starting from classes seen in the training data.
Concept Distance
Compositional Goneralization
3
|III
2
1101
1
11000
1010
0110
1001
0101
0011
0000
Training Set
Figure 4: Capabilities and compositional- ity in a concept graph. Consider lattice representation of a concept graph correspond- ing to four concept variables. Blue nodes de- note classes represented in the training data; model can either memorize data from these classes or learn capabilities to transform sam- ples from one class another. If it learns capabilities, it can compose them with data- generating process of samples from a concept class in the training data and produce samples that are entirely outtof-distribution,, denoted as pink and green nodes.
In summary, our concept graph framework provides a systematic approach to representing and understand- ing a minimalistic compositional structure, allowing for an analysis and comparison of different learn- ing algorithms' abilities to generalize across various concept classes.
3.1 Experimental and Evaluation Setup
With the concept graph framework in place, we now have the tools to systematically investigate the impact of different data-centric properties on learning and composing capabilities in conditional diffusion models. Before proceeding further, we briefly define our experimental setup and evaluation protocol. This allows us to interleave our results with our hypotheses, explaining them in context.
Experimental Setup. The detailed setup is presented in Appendix A. In brief, we train con- ditional diffuuson models that follow the U-Net pipeline proposed by Dhariwal and Nichol [1]. Our dataset involves concept classes defined using three concept variables, each with two values; specifically, shape (circle, triangle), color = (red, blue), and size (large, small). Tuples with binary elements are used to condition the diffusion model's training: they serve as an ab- straction of text conditioning. For example, the tuple 000 implies large, red circle is present in the image. To sample images from this process, we simply map the size and color axes to the range [0, 1] and uniformly sample to develop a training dataset of 5000 samples; the precise samples depend on which concept classes are allowed in the data-generating process. In this setup, the minimal required set for learning capabilities alter concepts is four pairs of tuples
5

---

and images, each drawn from one of the following four concept classes: {000, (circle, large) }. {100, (triangle, red, large)), {010, (circle, blue, large)]. {001, (circle, blue, small)). Consider the following two tuples: {000, (circle, red, large)) and {100, (triangle, red, large) J. In this case, we see that the shape concept is encoded in the first elements. Here, O represents a circle and represents a triangle. The remaining concepts are similarly encoded in later elements.
Evaluation Metric. Evaluating whether a generated image corresponds to the desired concept class might require a human in the loop. To eliminate this requirement, we follow methods from literature on disentanglement and train classifier probes to predict whether a generated image possesses some property of interest [103, 109-114]. Specifically, we use the diffusion model training data to train three linear probes, which respectively infer the following three concept variables: shape, , color and size. We define model's 's accuracy for generating images of a given concept class as the product of the probabilities outputted by the three probes that each concept variable matches the value for the concept class. Note that a random classifier for a concept variable will have an accuracy of 0.5. We indicate this random baseline with dotted, gray lines in our plots when useful.
4 Multiplicative Emergence of Compositional Abilities
We first investigate the order in which a model learn capabilities to produce sample from a concept class and how the learning of such capabilities affects the ability to compositionally generalize.
Learning dynamics respect the structure of the concept graph (Fig. 5). We find the ability to compositionally generalize, i.e., produce samples from out-of-distributiin concept classes, emerges at a rate which inversely related to a class's s concept distance with respect to classes seen in training. Specifically, in Fig. 5 (a) we show the learning dynamics of the model, where lightblue nodes denote concept classes within the training dataset, while pink and darkpink nodes respectively denote classes at concept distances of and 2 from classes in the training dataset. As the model learns to fit its training data (lightblue nodes), it learns capabilities that can be composed to produce samples from out-of-diste -f--ssiiirbtttinn concept classes (pink/ / darkpink nodes). Fig. 5 (b) further shows that the learning dynamics of compositional generalization are influenced by the concept distance from the training set: the model first learns the concept classes in the training dataset (lightblue lines) and then generalizes to concept classes with a concept distance of from the training dataset (pink lines). Thereafter, the model suddenly acquires the capability to compositionally generalize to concept class with concept distance of from the training dataset (darkpink line). Fig. 5 (c) shows the images generated by the model over time. We observe that rough shapes and sizes are learned relatively early in training, by the 4th epoch, while the color is highly biased to be red, the majority color in the training dataset, up the 10th epoch. Then, around the 20th epoch, the model learns generate the underrepresented
(a)
(LLLI)
(LLL))
(b)
(c)
(0,0.1)
(1,0.))
Accurscy
Color
10,1.03
(1.1.0)
TEC
Size
(0,0.0) Shape
(1.0.))
Optimizadion Stops
0
Number opocns
6
10
20
40
60
Figure 5: Concept distance from the training set governs the order in which compositional capabilities emerge. (a) Concept graph (cube) depicting training data points (blue nodes) and concept distances for test data points, where pink nodes represent distance = 1, and darkpink nodes represent distance = 2. Each trajectory represents the learning dynamics of generated images given a tuple prompt. Each trajectory represents the learning dynamics of generated images based on each tuple prompt. During every training epoch, 50 images are generated and binary classification is performed to predict each concept, including color, shape, and size. (b) Compositional generalization happens in sequence, starting with concept distance = and progressing to concept distance = 2. The x-axis represents the number of epochs, and the y-axis represents the progress of compositional generalization. (c) Images generated as a function of time clearly show a sudden emergence of the capability to change the color of small, blue, triangles.

---

(a)
100
color (blue) for concept classes with concept dis- tance of I. Finally, around the 40th epoch, the model learns to generate the underrepresented color (blue) for the class at distance 2, showing a sudden emer- gence of capability to generate samples from that class. The observations above generalize to higher- dimensional concept graphs well (see Fig. 16). Specifically, we now introduce a fourth concept vari- able backgroundcolor ={ =(white, black) and again find that compositional generalization occurs in the order govemndd by the distance from the training set: the accuracy hegins to rise for concept distance of 1, continues to increase for distance 2, and peaks for 3.
Mutiplicatve
Accuracy
Acoitive
50
111
10'
10a
10
(b)
100
Accuracy
Size
Shape
Color
50
10'
13
Optimization Steps
Figure 6: Multiplicative influence of indi- vidual capabilities elicits an "emergence" of compositionality. (a) Accuracy of pro- ducing samples from the concept class {111, (triangle, blue, small) 1, which has concept distance of from the training data. A multi- plicative measure (solid line) has score of when all concept variables of shape, color, and size are correctly predicted. Conversely, an additive measure (dashed line) indepen- dently relates each concept variable predic- tion accuracy to the compositional accuracy, deceptively suggesting smooth progress. (b) Learning dynamics of predicting each of the three concept variables: shape (blue), color (orange), and size (green).
Multiplicative influence of capabilities drives the sudden emergence of compositional generaliza- tion (Fig. 6). The interpretability of our experimental setup illustrates that the multiplicative reliance on underlying capabilities is the critical structure behind the sudden emergence of compositional generaliza- tion in our setup. For example, in Fig. 6 (a) we show the dynamics of an additive vs. multiplicative accuracy measure for generating concept class ( (111, (triangle, small, blue)), which has concept distance of 2 from the training data. We observe a sudden improvement of the multiplicative measure. To bet- ter understand the above, in Fig, 6 (b), we plot the accuracy of probes used for predicting each concept variable (shape, size, color). From the plot, we notice that the model fails to acquire strong color transformation capabilities until the final stage of training. effectively bottlenecking compositional generalization of 111 class. This empirical observation leads us the following hypothesis on emergence of compositional abilities in generative models.
Hypothesis. (Compositional Abilities Emerge Multiplicatively.) We hypothesize the nonlinear increase in capability observed in large neural networks as size and computational power scale up is partially driven by the task's 's compesitionality. Models must learn all required concepts, but compositional generalization is hindered by the multiplicative, not additive, impact of learning progress on each concept. This results in rather sudden emergence of capabilities to produce or reason about out-of- offdisstibbution data.
(a)
The core aspect worth noting in our hypothesis is that compositional tasks require the concurrent acquisition of all involved "atomic" capabilities, akin an AND logi- cal condition. Correspondingly, in generative models, the learning of individual atomic abilities leads to a manifesta- tion of emergent capabilities!. We argue this requirement leads to a non-linear increase in a model's capabilities (see also concurrent work making similar claims [78, 117]). We analyze this further via a simple toy model below.
750.05
gm
20
10
(b)
Optiiiztioon Steps
Toy analysis. We consider toy compositional problem which demonstrates how complex capabilities can rapidly develop (" ("emerge") from a set of atomic abilities. Assume there are n atomic abilities, each with a probability p of being learned in a given time step, i.e., the dynamics of learning an ability can be modeled as Bernoulli coin flip: once the coin turns from 0 to 1, the model learns the said ability. The probability that the ability will be learned in steps is thus given by 1 1-(1-p) An implicit assumption this model is that the learning dynamics of
Concept Distance in)
Figure 7: Toy Model of Compositional Emergence. (a) Probability of learning a compositional capability at a concept dis- func-
tance w.n.t. atomic abilities tion of time f. (b) Critical optimization time steps t" required for the above prob- ability to reach a threshold p as a func- tion of concept distance n.
This is related to the notion of "weak emergence" studied in psychology [115] and complex systems [116].
7

---

atomic abilities are independent (hence the name atomic) and that once an atomic ability is learned will not be forgotten. Now, our goal is to characterize the dynamics of learning a capability that involves composing these atomic abilities. The multiplicative emergence hypothesis argues that the learning such a capability by time requires that the model have learned the n atomic abilities by that time. We then get the probability that the compositional capability has been learned by time t is P(n) = (1 - (1 - p)") These dynamics are plotted in Fig. 7 (a) and look strikingly similar to the rapid learning (emergence) we found in our diffusion model setup (see Fig. 5 b)); for increasing degree of compositionality, we see much sharper learning dynamics (i.e., emergence) that is controlled by the concept distance w.r.t. atomic abilities. Further, assuming given threshold probability P at which one claims a capability has been learned, for a degree of composition n, we can solve for
the critical time at which a compositional capability is learned as follows: t" = log(1 -p)
Plotting this in Fig. 7 (b), we see the progress on increasingly more compositional tasks is logarithmic w.r.t. the number of atomic concepts being composed (aka concept distance). Importantly, this implies if we allow the model to train infinitely and it learns several atomic abilities, it will have an explosion of capabilities due to the inherent compositionality of the data generating process-we we further investigate this in a follow up work [118].
4.1 Challenges for Compositional Generalization
We have shown that, given a well-structured synthetic training dataset, conditional diffusion model can learn to compose its capabilities to generate novel inputs not encountered in the training set. Next, we analyze an adversarial setups under which the model fails to learn relevant capabilities to compose concepts and if simple interventions can help mitigate these failures.
Critical frequency for learning capabilities (Fig. 8). We first probe the effect of changing the frequencies of samples from different concept classes in the training data and examine how this affects the model's 's ability to learn and compose capabilities involving that concept class. Results are shown in Fig. 8 and demonstrate how the frequency of color and size concepts in the training data impacts the generalization capabilities of the diffusion model. Specifically, we change the number of samples in the training data from 0 to 300 for concept class 001 (Fig. (a)) and class 100 (Fig. (b)). As can be seen, low frequencies of certain concept values degrade the accuracy of the model in both settings. Notably, training for cuttof-distributionn concept classes (pink lines) requires more samples than that for in-distribution ones (lightblue lines). This suggests that as the sample size grows, memorization occurs first, and generalization achieved superlinearly as a function of data frequency. More importantly, we observe a critical number of samples are required before we can see the onset of capabilities to alter a concept. Specifically, in Fig. 8 (a), we can see that the model rapidly learns the color concept after being provided with 10 samples for a concept of large blue circle, 001. In contrast, in Fig. 8 (b), the model Iearns the shape concept only after reaching a certain threshold in the number of samples with concept of large red triangle, 100.
We believe the results above are especially interesting because an often used strategy to prevent a generative model from learning harmful capabilities, such as the ability to generate images involving sensitive concepts, involves cleaning the dataset to remove images corresponding to such concepts, hopefully hindering the model's ability to generate samples containing the concept [33, 119, 1 120].
(a)
100
(b)
100
80
011
Accuracy
010
60
010
001
103
3
103
150
011
011
8
20
115
110
010
110
100
200
300
100 200
300
Number samples
001
Number samples 100
Figure 8: How does the frequency of data samples impact the learning of capabilities? We
systematically control the frequency of a specific concept class in the training dataset and observe how it affects the model's learning of capabilities. (a) Capability to alter colors is quickly learned after introducing approx. 10 samples with concept class of large blue circle, 001. (b) In contrast, a critical threshold (marked with dotted vertical lines) exists for learning a capability to alter the shape: as we gradually introduce samples, with concept class of large red triangle, 100.
8

---

(a)
(b)
011
111
001
101
010
110
000
100
Adversarial Task Fine-tuning + node "101" + node "010" node "011" Figure 9: Fine-tuning is not remedy for misgeneralization in an adversarial task. (a) The model struggles to generalize when the training setup is configured ad adversarially. As shown in the plot, the model incorrectly produces triangles (instead of circles) for both 011 and 010. (b) The model faces difficulty learning new concepts through fine-tuning. We add node 101 (triangle, large, blue) to the dataset and attempt to fine-tune the model. However, even with large learning rate equal to the one used for training, the model fails to learn the capability to alter object size.
Such filtering is generally expensive and, arguably, statistically impossible to achieve perfectly in a large dataset, i.c., a few samples corresponding the sensitive concept are likely to remain in the data. Our findings suggest that one might better consider the relationship between the probability of learning concept and its frequency of occurence in the data determine the degree of filtering required: if their presence in the training data below a critical threshold, that can be sufficient to deter the model from learning capability generate samples related to that concept, obviating the need for a perfect filtering of the dataset.
Fine-tuning is not remedy for misgeneralization in an adversarial task. (Fig. 9). We next evaluate a setting where concept classes present the training data are not neighbors, i.e., their concept distances are greater than 1. but nonetheless represent all concept variables to allow model to learn relevant capabilities. Specifically, as shown in Fig. 9 (a). we use only the following four pairs of tuples and images for training: {000, (circle, red, large)). {100, (triangle, red, large)), }. (001, (circle, blue, small)]. {111, (triangle, blue, small)}. Arguably, capabilities corresponding to shape change (000 to 100) and color change (000 001) can still be learned as the setup for these is similar to our prior experiments. However, our results show this yields an interesting failure mode; specifically, in Fig. 9 (a) we find the model fails to dissociate second element value of 1 with the shape of a small triangle, and this bias causes the model to generate small triangles for both 011 and 010, when it should have produced small circles.
We further analyze if the failure above can be fixed via mere downstream fine-tuning (see Fig. 9 (b)). Specifically, we fine-tune the trained model dataset that includes the concept class of 101 (left), 010 (middle), and 011 (right). We find the model still generates small triangles for 011 and 010 even after fine-tuning. When the concept classes 010 (middle) and 011 (right) are added, the newly introduced concepts "overwrite" all existing concepts, causing the previously learned concepts (e.g.. the color and shape concept for 101) to be forgotten. These results present further challenge in addressing learned bias.
4.2 Additional Experimental Results with Real Data
To validate our hypotheses in more realistic settings, we conducted experiments using the CelebA dataset. We selected three attributes as the concepts: Gender (categorized as Female and "Male"), Smiling ("Smiling" and Not Smiling), and Hair Color ("Black Hair" and "Blonde Hair"). The concept graph corresponding these attributes in the CelebA dataset is illustrated in Fig. 10 (a). For each individual concept, we trained a 3-layer CNN and adopted the product of accuracies from each concept as the evaluation metric. Although not all nodes achieved an accuracy of 100% due to limited training time, we find that our observations (partially) extend: (i) learning dynamics following the structure of the concept and (ii) delayed generalization of underrepresented attribute (gender), hold true, even at larger, more realistic scales. Fig. 10 (b) illustrates the learning dynamics using the real CelebA dataset. We again observe the sequential pattern in generalization. The accuracy begins to rise at concept distance 1 (denoted by pink lines), followed by concept distance 2 (red line). Notably, the node labeled 011 ovn though is at concept distance 1 1-precedes the memorization stage (represented by blue lines) This exception can be attributed to the gender bias present in the training data. In Fig. 10 (c), we plotted the accuracy for the individual concept of gender (i.e., Female and Male). The Female concept class's 's training curve (red lines) reaches convergence faster

---

(a)
(b)
(c)
Accuracy
Accuracy
630
100
803
280
Training
107
Optimization
Optimization Steps
Figure 10: Concept distance from the training set governs the order in which compositional capabilities emerge. (a) Concept graph (cube) depicting training data points (blue nodes) and concept distances for test data points for CelebA dataset. (b) Compositional generalization happens in sequence, starting with concept distance 1 and progressing to concept distance 2. (c) Delayed emergence of abilities to generate underrepresented attribute (gender) for distant classes. Gender prediction accuracy based on generated samples at each step during training. We observe that the ability to generate underrepresented attributes occurs much later as concept distance increases.
compared the Male concept class (blue lines). This disparity stems from the Female class being more predominant in the dataset, thereby making the Male class underrepresented. This observation offers useful insights for practitioners: mitigate biased outcomes against underrepresented groups, we need to further train the diffusion model beyond its initial convergence on the training dataset.
5 Conclusion
In this work, we introduced a concept graph framework and used controlled interventions with synthetic dataset to formulate and characterize four novel failure modes in the compositional gener- alization of diffusion models. We first hypothesized and verified that compositional generalization to concept class far from the training data, based on concept distance, can fail, as it requires more optimization steps due to the sequential nature of generalization (Fig. 5). We show compositionality emerges in sequence that respects the geometric structure of the concept graph, eliciting multi- plicative emergence effect that manifests as sudden increases in the model's performance to produce outoo-ddistributin data well after it has learned to produce samples from its training distribution. This behavior is reminiscent of the recently observed phenomenon of grokking in language modeling- like objectives [80, 81, 121]. Furthermore, when particular concept value is underrepresented in the training dataset (e.g., an underrepresented color or gender). avoiding misgeneralization of the concept value, and thus achieving compositional generalization, requires far steps, even after achieving perfect performance on the training set. We also discovered a critical threshold for the inclusion of certain concepts in the training data, which is useful for deliberately making the model fail at learning harmful concepts (Fig. 8). Finally, we first analyze misgeneraliza- tions an adversarial task and find that fine-tuning does not provide solution (Fig. 9). Overall, our synthetic data approach allowed us to apply controlled interventions that shed light on four different mechanisms behind the failure of diffusion models in compositional generalization.
additional optimization
Authors' Contributions
ESL initiated the research by defining the problem. ESL and HT formulated the theoretical framework, concept graph, and collaborated with MO on the experimental design. MO conducted most experiments and, along with ESL and HT, produced the visualizations and figures. ESL and HT authored the introduction. ESL was responsible for the literature review and related work. MO, ESL, and HT jointly worked on the results and discussion sections. Throughout the study, MO, ESL, and HT collaborated on analyzing experimental results and refining the manuscript. HT and RPD provided supervisory guidance and feedback.
Acknowledgements
We thank Naomi Saphra, David Krueger and his group, Nikhil Vyas, Mohamed El Banani, and Anna Golubeva for fruitful discussions during the course this project. ESL's time University of Michigan was partially supported via NSF under award CNS-2008
10

---

References
[1] Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans on image synthesis. Advances in Neural Information Processing Systems, 34:8780-877 2021.
[2] Midjourney AL Midjourney, 2023, https://doca www.ooceeedjjorere...oo
[3] Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in Neural Information Processing Systems, 33:6840-6851, 2020.
[4] Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance. arXiv:2207. 12598, 2022.
arXiv preprint
[5] Alexander Quinn Nichol and Prafulla Dhariwal. Improved denoising diffusion probabilistic models. International Conference on Machine Learning, pages 8162-8171. PMLR, 2021.
[6] Alex Nichol, Prafulla Dhariwal, Aditya Ramesh, Pranav Shyam, Pamela Mishkin, Bob Mc- Grew, lya Sutskever, and Mark Chen. Glide: Towards photorealistic image generation and editing with text-guided diffusion models. arXiv preprint arXiv:2112. 2021.
Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text-conditional image generation with clip latents. arXiv preprint arXiv:2204.061 2022.
[8] Aditya Ramesh, Mikhail Pavlov, Gabriel Goh, Scott Gray, Chelsea Voss, Alec Radford, Mark Chen, and Ilya Sutskever. Zero-shot rext-to-image generation. In International Conference on Machine Learning, pages 8821-8831, PMLR, 2021.
[9] Zhihong Pan, Xin Zhou, and Hao Tian. Arbitrary style guidance for enhanced diffusion-basee text-to-image generation. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 4461-4471, 2023.
[10] Chitwan Saharia. Jonathan Ho, William Chan, Tim Salimans, David J Fleet, and Mohammad Norouzi. Image super-resolution via iterative refinement. IEEE Transactions on Puttern Analysis and Machine Intelligence, 2022.
[11] Jiahui Yu, Yuanzhong Xu, Jing Yu Koh, Thang Luong, Gunjan Baid, Zirui Wang, Vijay Vasudevan, Alexander Ku, Yinfei Yang, Burcu Karagol Ayan, et al. Scaling autoregressive models for content-rich text-to-ime -moge generation. arXiv preprint arXiv:2206. 107.9999.....9, 2022.
[12] Ben Poole, Ajay Jain, Jonathan T Barron, and Ben Mildenhall. Dreamfusion: Text-to-3d using 2d diffusion. arXiv preprint arXiv:2209. /4988, 2022.
[13] Chen-Hsuan Lin, Jun Gao, Luming Tang, Towaki Takikawa, Xiaohui Zeng, Xun Huang, Karsten Kreis, Sanja Fidler, Ming-Yi Yu Liu, and Tsung-Ye Yi Lin. Magic3d: High-resolution text-to-3d content creation. arXiv preprint mxXiv2221..10411 2022.
[14] Elad Richardson, Gal Metzer, Yuval Alaluf, Raja Giryes, and Daniel Cohen-Or. Texture: Text-guided texturing of 3d shapes. arXiv preprint arXiv:2302.0172/, 2023.
[15] Jae Hyun Lim, Nikola B Kovachki, Ricardo Baptista, Christopher Beckham, Kamyar Az- izzadenesheli, Jean Kossaifi, Vikram Voleti, Jiaming Song, Karsten Kreis, Jan Kautz, et al. Score-hased diffusion models in function space. arXiv preprint arXiv:2302.07400 2023.
[16] Ian Huang. Panos Achlioptas, Tianyi Zhang, Sergey Tulyakov. Minhyuk Sung, and Leonidas Guibas. Ladis: Language disentanglement for 3d shape editing. arXiv preprint arXiv:22/2.05011, 2022.
[17] Jonathan Ho, William Chan, Chitwan Saharia, Jay Whang, Ruiqi Gao, Alexey Gritsenko, Diederik Kingma, Ben Poole, Mohammad Norouzi, David Fleet, et al. Imagen video: High definition video generation with diffusion models. arXiv preprint arXiv:2210.02303, 2022.
[18] Jonathan Ho, Tim Salimans, Alexey Gritsenko, William Chan, Mohammad Norouzi, and David J Fleet. Video diffusion models. arXiv preprint arXiv:2204.03.. 2022.
11

---

[19] Andreas Blattmann, Robin Rombach, Huan Ling, Tim Dockhorn, Seung Wook Kim, Sanja Fidler, and Karsten Kreis. Align your latents: High-resolution video synthesis with latent diffusion models. arXiv preprint arXiv:2304.088// 2023.
[20] Tsai-Shien Chen, Chieh Hubert Lin, Hung- Yu Tseng, Tsung-Yi Yi Lin, and Ming-Hsuan Yang. Motion-conditioneed diffusion model for controllable video synthesis. arXiv preprini arXiv:2304. 14404, 2023.
[21] Yuming Jiang, Shuai Yang. Tong Liang Koh, Wayne Wu, Chen Change Loy, and Ziwei Liu. Text2pefformer: Text-driven human video generation. arXiv preprint 1rXiv:2304008453 2023,
[22] Sihyun Yu, Kihyuk Sohn, Subin Kim, and Jinwoo Shin. Video probabilistic diffusion models in projected latent space. arXiv preprint arXiv:2302.07665,, 2023.
[23] Kangfu Mei and Vishal M Patel. Vidm: Video implicit diffusion models. arXiv preprint 1xXiv:2212.00235. 2022.
[24] Duygu Ceylan, Chun-Hao Paul Huang, and Niloy J Mitra. Pix2video: Video editing using image diffusion. arXiv preprint arXiv:2303. 12688, 2023.
[25] Chachun Shin, Heeseung Kim, Che Hyun Lee, Sang-gil Lee, and Sungroh Yoon. Edit-a-video: Single video editing with object-aware consistency, arXiv preprint arXiv:2303007945, 2023.
[26] Bahjat Kawar, Shiran Zada, Oran Lang, Omer Tov, Huiwen Chang, Tali Dekel, Inbar Mosseri, and Michal Irani Imagie: Text-based real image editing with diffusion models. arXiv preprint arXiv:2210.09276, 2022.
[27] Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily L Denton, Kamyar Ghasemipour, Raphael Gontijo Lopes, Burcu Karagol Ayan, Tim Salimans, et al. Photorealistic text-to-image diffusionn models with deep language understanding. Advances in Neural Information Processing Systems, 35:36479-36494, 2022.
[28] Vidit Goel, Elia Peruzzo, Yifan Jiang. Dejia Xu, Nicu Sebe, Trevor Darrell, Zhangyang Wang, and Humphrey Shi. Pair-diffusion: Object-level image editing with muucture-and-appearae paired diffusion models. arXiv preprint arXiv:2303. 175.6 2023.
[29] Hareesh Ravi, Sachin Kelkar, Midhun Harikumar, and Ajinkya Kale. Preditor: Text guided image editing with diffusion prior. arXiv preprint arXiv:2302.07979, 2023
[30] Guillaume Couairon, Jakob Verbeek, Holger Schwenk, and Matthieu Cord. Diffedit: Diffusion- based semantic image editing with mask guidance. arXiv preprint arXiv:2210.11422 2022.
[31] Tim Brooks, Aleksander Holynski, and Alexei A Efros. Instructpix2pix: Learning to follow image editing instructions. arXiv preprint 20.i::22/10000000 2022.
[32] Alec Radford, Karthik Narasimhan, Tim Salimans, Ilya Sutskever, et al. Improving language understanding by generative pre-training, 2018
[33] Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. Language models are unsupervised multitask learners. OpenAl blog, 1(8):9, 2019.
[34] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pages 8748-8763. PMLR, 2021.
[35] Jason Wei, Maarten Bosma, Vincent Y Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M Dai, and Quoc V Le. Finetuned language models are zero-shot learners. arXiv preprint arXiv:2/09.01663 2021.
[36] Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, et Palm: Scaling language modeling with pathways. arXiv preprint a:X1v:2204.02311, 2022.
12

---

[37] Hugo Touvron, Thibaut Lavril, Gautier Izacard. Xavier Martinet, Marie-Ami Lachaux, Timo- thée Lacroix. Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, al. Llama: Open and efficient foundation language models. arXiv preprint arXiv: 13971, 2023.
[38] Jordan Hoffmann, Sebastian Borgeaud, Arthur Mensch, Elena Buchatskaya, Trevor Cai, Eliza Rutherford, Diego de Las Casas, Lisa Anne Hendricks, Johannes Welbl, Aidan Clark, et al. Training compute-optimal large language models. arXiv preprint arXiv:2203. /5556, 2022.
[39] Amanda Askell, Yuntao Bai, Anna Chen, Dawn Drain, Deep Ganguli, Tom Henighan, Andy Jones, Nicholas Joseph, Ben Mann, Nova DasSarma, et al. A general language assistant as laboratory for alignment. arXiv preprint arXiv:2 2021.
[40] Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback. Advances in Neural Information Processing Systems, 35:27730-27744: 2022.
[41] Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. Exploring the limits of transfer learning with unified text-to-text transformer. The Journal of Machine Learning Research, 21(1):5485-5551, 2020.
[42] Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova DasSarma, Dawn Drain, Stanislav Fort, Deep Ganguli, Tom Henighan, al. Training helpful and harmless assistant with reinforcement learning from human feedback. arXiv preprint 2r0.v22204.0586., 2022.
[43] Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, al. Evaluating large language models trained on code. arXiv preprint arXiv:2107.03374, 2021.
[44] Erik Nijkamp, Bo Pang, Hiroaki Hayashi, Lifu Tu, Huan Wang, Yingbo Zhou, Silvio Savarese, and Caiming Xiong. Codegen: An open large language model for code with multi-turn program synthesis. arXiv preprint arXhv:2203.13477 2022.
[45] Qinkai Zheng, Xiao Xia, Xu Zou, Yuxiao Dong, Shan Wang, Yufei Xue, Zihan Wang, Lei Shen, Andi Wang, Yang Li, et al. Codegeex: A pre-trained model for code generation with multilingual evaluations humaneval-x. arXiv preprint arXiv:2303. 17568, 2023.
[46] Federico Cassano, John Gouwar, Daniel Nguyen, Sydney Nguyen, Luna Phipps-Coss Costin, Donald Pinckney, Ming-Ho Ho Yee, Yangtian Zi, Carolyn Jane Anderson, Molly Q Feldman, et al. Multipl-e: A scalable and polyglot approach benchmarking neural code generation. IEEE Transactions on Software Engineering, 2023.
[47] Sai Vemprala, Rogerio Bonatti, Arthur Bucker, and Ashish Kapoor. Chatgpt for robotics: Design principles and model abilities. 2023, 2023.
[48] Boston Globe. A Boston Dynamics robot can now be run using ChatGPT. What www.-errrrnnnnnnntatg-t-------e
could go wrong?, 2023.
com/2003/05/033 /5/////bsinees/////////
[49] Kevin Riera, Anne-Laure Rousseau, and Clement Baudelaire. Doctor GPT:3: hype or reality?, 2020. https://ww/ http:///wwww/aaaaaa/aa com/blog/gptt..
[50] Kevin Roose. A Conversation With Bing's 's Chatbot Left Me Deeply Unsettled, 2023. https:
//www. www.:.yttmms.co cttp:/s/ccmmmm////////// http:////ccccooooooooooncccacaaa http--aaaaabb--iiitoooo---eeeeee html.
[51] Michael Janner, Yilun Du, Joshua B Tenenbaum, and Sergey Levine. Planning with diffusion for flexible behavior synthesis. arXiv preprint arXiv:2205. (9990. 2022.
[52] Ishika Singh, Valts Blukis, Arsalan Mousavian, Ankit Goyal, Danfei Xu, Jonathan Tremblay, Dieter Fox, Jesse Thomason, and Animesh Garg. Progprompt: Generating situated robot task 2022.
plans using large language models. arXiv preprint
13

---

[53] Cheng Chi, Siyuan Feng, Yilun Du, Zhenjia Xu, Eric Cousineau, Benjamin Burchfiel, and Shuran Song. Diffusion policy: Visuomotor policy learning via action diffusion. arXiv preprint 2.Xi:22303.04177,, 2023.
[54] Johann Brehmer, Joey Bose, Pim De Haan, and Taco Cohen. Edgi: Equivariant diffusion for planning with embodied agents. arXiv preprint 2rXiv:2303.22410, 2023.
[55] Michael Ahn, Anthony Brohan, Noah Brown, Yevgen Chebotar, Omar Cortes, Byron David, Chelsea Finn, Keerthana Gopalakrishnan, Karol Hausman, Alex Herzog, al. Do can, as i say: Grounding language in robotic affordances. arXiv preprint arXiv:2204.0169), 2022.
[56] Wenlong Huang, Fei Xia, Ted Xiao, Harris Chan, Jacky Liang, Pete Florence, Andy Zeng, Jonathan Tompson, Igor Mordatch, Yevgen Chebotar, al. Inner monologue: Embodied reasoning through planning with language models. arXiv preprint arXiv:2207.05600, 2022.
[57] Wenlong Huang, Pieter Abbeel, Deepak Pathak, and Igor Mordatch. Language models as zero-shot planners: Extracting actionable knowledge for embodied agents. In International Conference on Machine Learning, pages 9118-9147. PMLR, 2022.
[58) Anthony Z. Liu, Lajanugen Logeswaran, Sungryull Sohn, and Honglak Lee. A picture is worth a thousand words: Language models plan from pixels. arXiv preprint arXiv:2303.0093: 2023.
[59] Pratyusha Sharma, Antonio Torralba, and Jacob Andreas. Skill induction and planning with latent language. arXiv preprint arXiv:2110.01517. 2021.
[60] Dinghuai Zhang, Kartik Ahuja, Yilun Xu, Yisen Wang. and Aaron Courville. Can subnetwork structure be the key to duittf-distribution generalization? In International Conference Muchine Learning, pages 12356-12367. PMLR, 2021.
[61] Jivat Neet Kaur, Emre Kiciman, and Amit Sharma. Modeling the data-generating process is necessary for out-of-distribution generalization. arXiv preprint. arXiv:2206.078377 2022.
[62] Gary Marcus, Ernest Davis, and Scott Aaronson. A very preliminary analysis of dall-e 2. arXiv preprint arXiv:2204.13800 2022.
[63] Evelina Leivada, Elliot Murphy, and Gary Marcus. Dall-e 2 fails to reliably capture common syntactic processes. arXiv preprint arXiv:22 10.2200022220 /2889, 2022.
[64] Colin Conwell and Tomer Ullman. Testing relational understanding in text-guided image generation. arXiv preprini arXiv:2208.00000 00005, 2022.
[65] Colin Conwell and Tomer Ullman. A comprehensive benchmark of human-likk relational reasoning for text-to-image foundation models. In ICLR 2023 Workshop on Mathematical and Empirical Understanding of Foundation Models, 2023.
[66] Tejas Gokhale, Hamid Palangi, Besmira Nushi, Vibhav Vineet, Eric Horvitz, Ece Kamar, Chitta Baral, and Yezhou Yang. Benchmarking spatial relationships in text-to-image generation. arXiv preprint 1001v:2212.10015, 2022.
[67] Yilun Du, Conor Durkan, Robin Strudel, Joshua B Tenenbaum, Sander Dieleman, Rob Fergus, Jascha Sohl-Dickstein, Arnaud Doucet, and Will Grathwohl. Reduce, reuse, recycle: Compositional generation with energy-based diffusion models and meme. arXiv preprint urXiv:2302.11552. 2023.
[68] Nan Liu, Shuang Li, Yilun Du, Antonio Torralba, and Joshua B Tenenbaum. Compositional visual generation with composable diffusion models. In Computer Vision-ECCV 2022: 17th European Conference, Tel Aviy, Israel, October 23-27, 2022, Proceedings, Part XVII, pages 423-439. Springer, 2022.
[69] Gautam Singh, Fei Deng, and Sungjin Ahn. Illiterate dall-e learns to compose. arXiv preprint arXiv:2110.J /400/ 2021.
[70] Royi Rassin, Shauli Ravfogel, and Yoav Goldberg. Dalle-2 is seeing double: flaws in word-to- concept mapping in text2image models. arXiv preprint arXiv:2210.10610 2022.
14

---

[71] Weixi Feng, Xuehai He. Tsu-Jui Fu, Varun Jampani, Arjun Akula, Pradyumna Narayana, Sugato Basu, Xin Eric Wang, and William Yang Wang. Training-free -reee structured diffusion guidance for compositional text-to-image synthesis. arXiv preprint urXiv:2212.05032, 2022.
[72] Ben Hutchinson, Jason Baldridge, and Vinodkumar Prabhakaran. Underspecification in scene description-to-depittii tasks. arXiv preprini urXi::2210.008815, 2022.
[73] Stability AI. Stable Diffusion v2.1, 2023. https: /tttts://bet..oe htt.//ttt...rammtuuiio... ai ai/generate.
[74] NYTimes. Emily Anthes. Creature's Brain, 2021.
Why Scientists Have Spent Years Mapping This
wtw.:///..ia..ccc/////////////// Prosophila-fly-braae beabbriicconnetomeee html.
[75] IEEE Spectrum. David Adler. Reverse Engineering the Brain, 2008. https://spectrun.
leee. wwwproeeeree-eniinnneeeeeee engineering- teannng--h-bbbrin
[76] Hidenori Tanaka, Aran Nayebi, Niru Maheswaranathan, Lane McIntosh, Stephen Baccus, and Surya Ganguli. From deep learning to mechanistic understanding in neuroscience: the structure of retinal prediction. Ady. in Neural Information Processing Systems (NeurIPS), 2019.
[77] Jason Wei, Tay, Rishi Bommasani, Colin Raffel, Barret Zoph, Sebastian Borgeaud, Dani Yogatama, Maarten Bosma, Denny Zhou, Donald Metzler, et al. Emergent abilities of large language models. arXiv preprint ar arXivv2206.07682 2022.
[78] Sanjeev Arora and Anirudh Goyal. A theory for emergence of complex skills in language models. arXiv preprint arXiv:2307. 15936, 2023.
[79] Rishi Bommasani, Drew A Hudson, Ehsan Adeli, Russ Altman, Simran Arora, Sydney von Arx, Michael S Bernstein, Jeannette Bohg, Antoine Bosselut, Emma Brunskill, al. On the opportunities and risks of foundation models. arXiv preprint. 1rXiv:2108.07258, 2021.
[80] Alethea Power, Yuri Burda, Harri Edwards, Igor Babuschkin, and Vedant Misra. Grokking: Generalization beyond overfitting on small algorithmic datasets. arXiv preprins arXiv:2201.02177, 2022.
[81] Neel Nanda, Lawrence Chan, Tom Liberum, Jess Smith, and Jacob Steinhardt. Progress measures for grokking via mechanistic interpretability. arXiv preprint arX1v:2301.05217, 2023.
[82] Jonas Peters, Dominik Janzing, and Bernhard Schölkopf. Elements of causal inference: foundations and learning algorithms. The MIT Press, 2017.
[83] Noah D Goodman, Joshua B Tenenbaum, Thomas I. Griffiths, and Jacob Feldman. Composi- tionality in rational analysis: Grammar-based induction for concept learning. The probabilistic mind: Prospects for Bayesian cognitive science, 2008.
[84] Steven Phillips and William H Wilson. Categorial compositionality: A category theory explanation for the systematicity of human cognition. PLoS computational biology, 6(7):
e1000858, 2010.
[85] Steven M Frankland and Joshua Greene. Concepts and compositionality: search of the brain's 's language of thought. Annual review of psychology, 71:273-303. -3733-33,333 2020.
[86] Carlo Reverberi, Kai Görgen, and John-Dylan Haynes. Compositionaality rule representations in human prefrontal cortex. Cerebral cortex, 2266):133711266, 2012.
[87] Nicholas T Franklin and Michael J Frank. Compositional clustering in task structure learning. PLoS computational biology, 14(4)ee 16, 2018.
[88] Yilun Du, Shuang Li, Yash Sharma, Josh Tenenbaum, and Igor Mordatch. Unsupervised learning of compositional energy concepts. Advances Neural Information Processing Systems, 34:15608-1500 2021.
15

---

[89] Guangyue Xu, Parisa Kordjamshidi, and Joyce Chai. Prompting large pre-trained vision- language models for compositional concept learning. arXiv preprint arXiv:2211.05077) 2022.
[90] Mert Yuksekgonul, Federico Bianchi, Pratyusha Kalluri, Dan Jurafsky, and James Zou. When and why vision-language models behave like bag-of-words models, and what to do about it? arXiv preprint arXiv:2210.01936. 2022.
[91] Emanuele Bugliarello and Desmond Elliott. The role of syntactic planning in compositional image captioning. arXiv preprint arXiv:2101.1991, 2021.
[92] Sam Spilsbury and Alexander Ilin. Compositional generalization in grounded language learning via induced model sparsity. arXiv preprini arXiv:2207.025118 2022.
[93] Nupur Kumari, Bingliang Zhang, Sheng- Yu Wang, Eli Shechtman, Richard Zhang, and Jun- Yan Zhu. Ablating concepts in text-to-image diffusion models. arXiv preprint arXiv:2303.13331 2023.
[94] Tristan Thrush, Ryan Jiang, Max Bartolo, Amanpreet Singh, Adina Williams, Douwe Kiela, and Candace Ross. Winoground: Probing vision and language models for visio-linguistic compositionality. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 5238-5248, 2022.
[95] Jacob Andreas. Measuring compositionality in representation learning. arXiv preprint arXiv:1902.07881, 2019.
[96] Martha Lewis. Qinan Yu, Jack Merullo, and Ellie Pavlick. Does clip bind concepts? probing compositionality in large image models. arXiv preprint (xXiv:2212.10537 2022.
[97] Brenden Lake and Marco Baroni. Generalization without systematicity: On the compositional skills of sequence-to-ser soquenennce recurrent networks. In International conference on machine learming, pages
PMLR, 2018
[98] Tian Yun, Usha Bhalla, Ellie Pavlick, and Chen Sun. Do vision-language pretrained models learn primitive concepts? arXiv preprint 22Xix:2003.17777 2022.
[99] Michael A Lepori, Thomas Serre, and Ellie Pavlick. Break it down: Evidence for structural compositionality in neural networks. arXiv preprint arXiv: 10884 2023.
[100] Justin Johnson, Bharath Hariharan, Laurens Van Der Maaten, Fei-Fei, Lawrence Zitnick, and Ross Girshick. Clevr: A diagnostic dataset for compositional language and elementary visual reasoning. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 2901-2910, 2017.
[101] Lukas Schott, Julius Von Kügelgen, Frederik Träuble, Peter Gehler, Chris Russell, Matthias Bethge, Bernaard Schölkopf, Francesco Locatello, and Wieland Brendel. Visual repre- sentation learning does not generalize strongly within the same domain. arXiv preprint arXiv:2107.08221, 2021.
[102] Josef Valvoda, Naomi Saphra, Jonathan Rawski, Adina Williams, and Ryan Cotterell. Bench- marking compositionality with formal languages. In Proceedings of the 29th International Conference on Computational Linguistics, pages 6007-601 6018, 2022.
[103) Irina Higgins, Loic Matthey, Arka Pal, Christopher Burgess, Xavier Glorot, Matthew Botvinick, Shakir Mohamed, and Alexander Lerchner. beta-vae: Learning basic visual concepts with a constrained variational framework. In Proc. Int. Conf. on Learning Representations (ICLR), 2017.
[104] Julius Von Kügelgen, Yash Sharma, Luigi Gresele, Wieland Brendel, Bernhard Schölkopf, Michel Besserve, and Francesco Locatello. Self-supervised learning with data augmentations provably isolates content from style. Adv. in Neural Information Processing Systems (NeurIPS), 2021.
[105] Bernhard Schölkopf, Francesco Locatello, Stefan Bauer, Nan Rosemary Ke, Nal Kalchbrenner, Anirudh Goyal, and Yoshua Bengio. Towards causal representation learning. arXiv preprint. arXiv:2102.111077 2021.
16

---

[106] Thaddäus Wiedemer, Prasanna Mayilvahanan, Matthias Bethge, and Wieland Brendel. Com- positional generalization from first principles. arXiv preprint arXiv:2307.05596 2023.
[107] Shengjia Zhao, Hongyu Ren, Arianna Yuan, Jiaming Song, Noah Goodman, and Stefano Ermon. Bias and generalization in deep generative models: An empirical study, Advances in Neural Information Processing Systems, 31, 2018.
[108] Dieuwke Hupkes, Verna Dankers, Mathijs Mul, and Elia Bruni. Compositionality decomposed: How do neural networks generalise? Journal of Artificial Intelligence Research, 67:757-795, 2020.
[109] Hyunjik Kim and Andriy Mnih. Disentangling by factorising, In International Conference on Machine Learning, pages 2649-2658. PMLR, 2018.
[110] Cian Eastwood and Christopher Williams. A framework for the quantitative evaluation of disentangled representations. In International Conference on Learning Representations, 2018.
[111] Ricky TQ Chen, Xuechen Li, Roger B Grosse, and David K. Duvenaud. Isolating sources disentanglement in variational autoencoders. Advances in neural information processing systems, 31, 2018.
[112] Abhishek Kumar, prasanna Sattigeri, and Avinash Balakrishnan. Variational inference of disentangled latent concepts from unlabeled observations. arXiv preprint arXiv: 2017.
[113] Sjoerd Van Steenkiste, Francesco Locatello, Jürgen Schmidhuber, and Olivier Bachem. Are disentangled representations helpful for abstract visual reasoning? Adv. in Neural Informatio Processing Systems (NeurIPS), 2019.
[114] Francesco Locatello, Stefan Bauer, Mario Lucic, Gunnar Raetsch, Sylvain Gelly, Bernhard Schölkopf, and Olivier Bachem. Challenging common assumptions in the unsupervised learning disentangled representations. Proc. int, conf. on machine learning (ICML), 2019,
[115] Mark A Bedau. Is weak emergence just in the mind? Minds and Machines, 18:443-455 2008.
[116] David G Green. Emergence in complex networks of simple agents. Journal of Economic Interaction and Coordination, pages 1-44, 2023.
[117] Rylan Schaeffer, Brando Miranda, and Sanmi Koyejo. Are emergent abilities of large language models a mirage? arXiv preprint arXiv:2304.15000 2023.
[118] Rahul Ramesh, Mikail Khona, Robert P Dick, Hidenori Tanaka, and Ekdeep Singh Lubana. How capable can a transformer become? a study on synthetic, interpretable tasks. arXiv preprint Stti::2311.1999: 2023.
[119] Karan Desai, Gaurav Kaul, Zubin Aysola, and Justin Johnson. Redcaps: Web-curated image- text data created by the people, for the people. arXiv preprint arXiv:2111.11431, 2021.
[120] Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhari- wal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. Advances in neural information processing systems, 33:1877- 1901, 2020.
[121] Boaz Barak, Benjamin Edelman, Surbhi Goel, Sham Kakade, Eran Malach, and Cyril Zhang. Hidden progress in deep learning: Sgd learns parities near the computational limit, Advances Neural Information Processing Systems, 35:21750-21764, 2022.
[122] Nanxin Chen, Yu Zhang, Heiga Zen, Ron Weiss, Mohammad Norouzi, and William Chan. Wavegrad: Estimating gradients for waveform generation. ICML, 2021.
[123] Diederik P Kingma and Jimmy Ba. Adam: method for stochastic optimization. In ICLR, 2015.
17

---

Supplementary Information
A Experimental details
A.1 Synthetic dataset
The dataset consists of 5,000 rendered images of 2D geometric shapes, along with corresponding concept classes. These synthetic images are generated using Blender' by creating a scene graph and rendering it. Each scene contains a single object placed on a blank background of size 28 x 28. The objects have three types of attributes: size, color, and shape. There two shapes (circle and triangle), three colors (red, blue), and two sizes (large, small), resulting in up to eight different combinations of attributes. Each image is annotated with the corresponding object attributes, which can be utilized as conditional features for image generation. This enables us to directly evaluate the text-to-image generation capability of the diffusion model against ground truth images. Fig. depicts example images with the corresponding concept classes.
000
001
100
010
110
101
111
Color: (0.81,0.111.. (0..020))0224
1..00.1...0.1 1.46
0..19
jo.8t. 25.0.19)
(022,0 20.0.80)
10.16,0.24 24.00.2...85.
10:25,0-16,0/
16.16600.79 1o. 78,0.23.0116) 21.0.19.077) 2.55
Size: 2.55
2.40
2.57
1.28
1.26
1.62
Color:
10.76,0
10.84.0.21.00
21.00.2)
20,0.20.0.23) (0.15.0.16.0.80)
Side: 3.09
0.93
2.92
2.49
0.92
2.50
1.07
1.67
(0.85.0.13.0.1))
10.83.0:25.0235)
20.00.20.0220
(0.18,0 22.0.80
(0.18.0.21.0 2..00.1....21
(0.17.0.18.079)
(0.35..25.0.22) (0.24.0.22.0.63)
Color: (0.83.023.0224) Size: 3.07
22.106)22.016))
10.30,0.2 25,0.25)
0..076
(0.19.0.21.0.84)
(025,0115,0776) (0. 00.799..10,.166 (0.18.0.25.088))
0.84
3.09
2.72
0.87
2.33
1:29
1.07
Figure 11: Examples of data samples with pairs of images and corresponding concept classes.
A.2 Loss function
Diffusion models convert Gaussian noise into samples from data distribution through iterative denoising process. The sampling process starts with a noisy input Xy, and denoised samples are generated through gradual iteration, Xy- Xy-2, until the original input X, is obtained. Conditional diffusion models [10, 122] allow for a denoising process conditioned on texts or class labels. In all the experiments, we used the conditional diffusion model with the form (V)(V)) where x denotes an image and V (U1, 12, ..., Un denotes set of n concept variables. To predict the noise at each timestep E [0, T]. we follow the approach proposed [3] by training a neural network. Specifically, we construct neural network eg (Xt: f, a) and minimize the mean squared error (MSE) between the predicted Gaussian noise and the true noise:
= xxx
[|]e (x &, V)I
where g(xo) denotes the distribution of input image Xo. and (N I) denotes the standard Gaussian distribution.
A.3 Architecture
We the conditional U-Net architecture [1], in [3], for our neural network c. Our architecture comprises two down-sampling and up-sampling blocks, with each block consisting of 3 xx3 convolu- tional layers, GELU activation, the global attention, and pooling layers. The conditional information V are fed through an embedding layer and concatenated with the image feature maps at each stage of the up-sampling blocks. We illustrate our network architecture in Fig. 12.
Patp://www.blender.org
18

---

Conditional U-Net
...
Skip connection
x (T-
Text encoding
XT
XT-1
X
[2622533
[8028
Shape
Color
Element-wise addition
Size
Element-wise multiplication
Timestep
The architecture of the conditional diffusion model. The architecture of the conditional diffusion model involves iterative process comprising noise addition and denoising steps. The model leverages conditioning information, specifically concept classes, to guide the transformation of the input image towards desired state. our implementation, we utilize U-Net to parameterizethe denoising process. The U-Net architecture consists of three upsampling convolutional layers and three downsampling convolutional layers, which are connected through skip connections. Each layer within the U-Net includes pooling layer, a global attention mechanism, and GELU activation function.
Figure 12:
A.4 Optimizer
We implemented the diffusion model using PyTorch and trained it on four Nvidia A100 GPUs. We performed a hyperparameter search based on validation set. We tested batch sizes ranging from 32 to 256, the number of channels in each layer from 64 to 512, leaning rate from 10 and 10-3, the number steps in the diffusion process from 100 to 400. We employed the Adam optimizer [123] with 31 0.9, $2 0.99, and weight decay of 10-5
A.5
Evaluation metric
For the evaluation. follow probing protocol popularly used in several prior works on learning disentangled representations [103, 109-114]. To evaluate the attributes of the images generated by our conditional diffusion model, trained linear classifiers these images for three specific attributes: shape, color, and size. For each attribute, we developed a dedicated classifier: fu (oo) for shape, (fo) for color, and f2 for size. Here in denotes the image generated by the conditional diffusion model. We utilized a cross-entropy loss function to train these classifiers. The output of each classifier fell into of categories: for shape, the categories were circle triangle; for color, blue or red; and for size, large or small. We then calculated the accuracy for each attribute using the corresponding classifier outputs. To quantitatively assess the accuracy of predicted attributes aligning with their corresponding ground- concept classes, we utilize a multiplicative measure. This measure gauses the accuracy of all attributes, and defined by the product of individual accuracies for each attribute as follows:
N
Σ 1 (Jo(a)")) (())) (m)) (())) 1 (J2(1))) ()) m())))
Accuracy
N
where 1(-) is the indicator function, n is the index of the test samples, and N, is the total number of samples used for evaluation. For experiments, we generated N, 50 images for each input of concept classes. 10 (n) (n) and v² denote the actual (ground truth) concepts classes for shape, color, and size, respectively. We trained them over 50 epochs using the training dataset comprising 5,000 pairs of concept classes and images. The trained linear classifiers achieved an accuracy rate of 100% on the test set drawn from the original synthetic dataset.
19

---

Fig. 13, 14, and 15 display the average pixel values of the generated images with the prediction outcomes from the linear probing classifiers for the shape, color, and size attributes these images. respectively. The figures in each subplot represent the ratio of accurately classified images out of total of 50 samples. The predictions made by the linear classifiers align with our intuitive expectations, thereby confirming their suitability for evaluating the generated images.
Ground truth
0.98
1.0
1.0
1.0
1.0
1.0
1.0
1.0
1.0
Ground
1.0
Ground truth
0.22
0.5
0.6
0.32
0.78
0.62
0.86
1.0
0.94
Ground
0.72
Figure 13: Average pixel values of the generated images and prediction outcomes of the linear probing elacsifier for chane
0.2
Ground truth
0.08
0.06
0.0
0.7
1.0
1.0
1.0
1.0
1.0
Ground
Ground truth 0.06
0.0
0.0
0.0
0.26
0.68
0.84
0.0
0.2
Figure 14: Average pixel values of the generated images and prediction outcomes of the linear probing elaccifior for color
Ground truth
1.0
1.0
1.0
1.0
1.0
1.0
Ground truth
1.0
1.0
1.0
1.0
1.0
1.0
Ground truth
0.98
1.0
1.0
1.0
1.0
1.0
1.0
1.0
1.0
Ground truth
1.0
1.0
1.0
1.0
1.0
1.0
Figure 15: Average pixel values of the generated images and prediction outcomes of the linear probing classifier for size.
20

---

A.6 Experiment with real data
We sourced our data from the CelebA dataset". Our training dataset consists of 13,000 facial images, each labeled with its respective concept class. We delineated the concept classes based on these attributes: gender (classified as "Male" or otherwise), facial expression ("Smilingg or not), and hair color (either "Black Hair" or "Brown Hair"). We adjusted the images to resolution of 48 x 48 pixels.
For our evaluation, we designed three classifiers for each concept using three-layer CNN with 64 units. The learning rate was established at 10-3, trained the model for 10 epochs. We employed the ReLU activation function for the hidden layers and the sigmoid function for the output layer. We divided the image dataset into training and test sets at ratio of 80% to 20%. The trained classifier exhibited a 96% accuracy for gender, 86% for facial expression, and 95% for hair color. All additional experimental configurations conformed to synthetic data experiment procedures.
B Additional experimental results
In Fig. 16, we generalize the find- ings in Fig. 5 a larger con-
(6) Conoupt Daturce
Componiiond Gornreloziin
(b)
cept graph, i.e., one with four con- cept variables. Specifically, we now introduce a fourth concept variable backgroundcolor w/hite, black) and again find that composi- tional generalization occurs in the or- der governed by the distance from the training set: the accuracy begins to rise at concept distance of I, continues to increase at a distance of 2, and peaks at 3.
Accurstyy
1100
1001
[100
0000
301]]
(101)
Oysimization Steps
Figure 16: Compositional generalization. (a) Consider a lattice representation of a concept graph corresponding to four concept variables. Blue nodes denote classes rep- resented in the training data; a model can either memorize data from these classes leam capabilities transform samples from one class to another. If it learns capabilities, it can compose them with data-generating process of samples from a concept class in the training data and produce sam- ples that are entirely -itoo--distribution, denoted as pink and green nodes. (b) We find compositional generalization happens in sequence, starting with concept distance of and then progressing to concept distance of 2 and 3.
In Fig. 17, we investigate the bottle- necks that affect compositional gener- alization by examining the prediction accuracy of concept variables such as shape, color, and size, as well as the overall classification accuracy. To eval- uate this, we generated 50 samples in each epoch of training and classified them using a linear probe. Our findings reveal that various concept bottlenecks impede the learning process for different compositional objects. For instance, the generalization of blue-colored objects (011, 101, 111) is hindered by the color concept, indicating that the model struggles correctly identify objects based on their color when it comes to these particular compositions. Similarly, the generalization to small red triangles (111) is bottlenecked by the shape concept, implying that the model faces difficulties in accurately recognizing the shape of these specific compositions. These observations highlight the challenges associated with compositional generalization and shed light on the specific concept bottlenecks that hinder the learning process. Understanding and addressing these bottlenecks can contribute to the development of more robust models capable of achieving better compositional generalization in various contexts.
In Fig. 18. 19, and 20 we present the experimental results for the diffusion model without global attention. Notably, we observe that these results exhibit a similar pattern to the ones obtained from the diffusion model with global attention, which is shown in Fig. 5 and 6. The experimental results consistently support our findings. Firstly. the order of emergence of compositional capabilities aligns with the compositional structure of the data-generating process, as evident in the results shown in Fig. 18. Secondly, we observe a notable delay in the emergence of the ability to generate underrepresented colors as the concept distance within concept class increases. This emphasizes the importance of continued training beyond the point of achieving in-distribution generalization, as demonstrated in Fig. 19. Specifically, observe that the model is capable of generating the majority color (red) much earlier than the underrepresented color (blue). Importantly, the training
https://mmlabii ie www..ccw..k.c.ceeecccoo hww.hp/oojecs/CCetbbbbmmm
21

---

Shape
Color
Size
1.0
Accurcy
0.8
0.6
001
1.0
Accuracy
0.5
011
101
110
111
0.0
10'
10a 10
10
10'
10
10
10'
Optimization
Optimization Steps
Optimization Steps
Optimization
Figure 17: Analyzing the bottlenecks of compositional generalization. We examine the prediction accuracy of concept variables (shape, color, and size) and the overall classification accuracy. In each epoch of training, 50 samples were generated and classified using linear probe. We observe that various concept bottlenecks hinder the learning of different compositional objects. For example, compositional generalization to blue-colored objects (011, 101, 111) is bottlenecked by the color concept, while generalization to small red triangles (111) is bottlenecked by the shape concept.
(a)
(0I.U)
(LLDD)
(b)
(c)
(0.0.1)
(1:0,13
100
Armonayy
Color
010
(1.1,0)
Size
(0,0.0) Shape
(1.0.0)
UP
Number apacha
Figure 18: Concept distance from the training set govern the order in which compositional capabilities emerge. (a) Concept graph (cube) depicting training data points (blue nodes) and concept distances for test data points. (b) Compositional generalization happens in sequence, starting with concept distance = and progressing to concept distance = 2. The x-axis represents the number of epochs, and the y-axis represents the progress of compositional generalization. (c) Images generated as function of time clearly show a sudden emergence of capability to change color for small, red triangles.
time to achieve generalization based concept increases with concept distance. This observation provides important insights for training models with fairness in mind: even once generalization for in- in-distribution concept classes is achieved, stopping the training of a model will likely lead to a failure in generating underrepresented concepts, particularly for compositional generalization. Thirdly, findings reveal multiplicative impact of learning individual concepts performance in compositional tasks, offering an explanation for the sudden emergence of capabilities, as depicted in Fig. 20.
In Fig. 21, we conducted experiments with different configurations by using varied sets nodes in the training data. In Fig. (1( and Fig. (b). inverted the axes of the concept graph from Fig. 9a): (a) from bottom to top and (b) from left to right. In Fig. 21(c) and Fig. 21(d), we inverted the axes of the concept graph from Fig. 5(a): (c) from bottom to top and (d) from left to right. In Fig. 2b(b) and Fig. 21(c), we observe that the model incorrectly generates triangles for both 011 and 010, though circles are expected. In the other cases, the model outputs correct images for all the nodes. Our observations indicate that the model struggles to to dissociate a second element value of 1 with the shape of small triangle, exhibiting bias generate triangles rather than eircles for small objects.
22

---

CO0
60
C10
100
101
20
110
103
Optimization Steps
Delayed emergence of abilities to generate underrepresented colors for distant classes. Prediction accuracy of color based on generated samples at each epoch during training. We observe that the ability to generate underrepresented colors emerges significantly later as the concept distance of concept class increases, highlighting the need for extended training beyond the point of achieving in-distribution generalization. This prolonged training enables effective composition the underrepresented concept and leads improved generalization.
Figure 19:
(a)
100
Accurazy
10a
10'
10
(b)
Optimization Steps
100
Accuracy
Color
Shape
50
Size
10'
10*
10*
Optimization Steps
Figure 20: Multiplicity underlies the sudden emergence of compositional capabilities. (a) Accuracy of producing samples from the concept class (111, (triangle, blue, small)), which has concept distance of from the training data. (b) Learning dynamics of accuracies for predicting each of the three concept variables: shape (blue), color (orange), and size (green).
(a)
(b)
(c)
(d)
011
111
010
110
000
100
Figure 21: The additional experimental results with different configurations by using varied sets of nodes in the training data.
23


Reviews:
[{"review_id": "b7xe6rdn0V", "reviewer": "Reviewer_kruN", "summary": "This work proposes a framework for studying the compositional generalisation abilities of diffusion models (or generative models more broadly). To that end it introduces the notion of a concept graph which the authors use to manipulate simple synthetic datasets. These concept graphs arrange different combinations of concept values and variables in a way that perseveres a notion of distance between different combinations (based on bit flips). By studying how performance changes as models are required to reconstruct concept value combinations that are further apart from the training data, they can draw inferences regarding the emergence of generalisation capabilities in diffusion models.", "questions": "1. Is Multiplicative Emergence unsurprising when the authors are forcing accuracy to be 0 if at least one concept value is incorrect during out-of-distribution generation?\n2. I find it weird that colour takes longer to learn than the other properties. Since the reconstruction error must be very high even if the shape and scale are correct, I would expect cooler to be learned quicker. Do the authors have some insight into why this happens?\n3. Is the relation to grokking really that strong? In grokking, models still exhibit high validation error even if the test error is high. But here ", "limitations": "1. It is incorrect to characterise some of the work discussed in lines 120 and 121 as merely performing benchmarking. Indeed at least [94], [95] and [99] perform manipulations of the training and dataset in much the same way as this present work. Thus, while I definitely like that the authors took a more hypothesis driven approach to their work, it is not by any means the case that this is has not been done before. Other works that do this include [1] and [2]. While these may not be based on generative models, they are still relevant examples of the kind of study that the authors are advocating for.\n\n2. Relatedly, the authors should discuss the relation of their work with [3] and [4]. Unlike the previous references, these do involve compositional generalisation capabilities of generative models. The former also provides one characterisation of compositional generalisation in generative models that is similar to the one used in this work. Thus it is indeed not the case that there have been no previous proposals and that [101] is the only noteworthy attempt (which by its own admission, just summarises previous definitions anyway).\n\n3. The latter uses an approach based on transforming input images which allows it to identify the concepts that vary between images in a way that resembles the example presented in Figure 2, panel b. Their results show a similar pattern as the one in this work where combinations that are close to the training data are easier for the models. I believe that both this work and the previously discussed ones deserve a slightly deeper discussion.\n\n4. Lines 291 and 292 are very close to each other which makes the caption of Figure 7 and the main text hard to distinguish.\n\n\n\n[1] Hermann, K., Chen, T., & Kornblith, S. (2020). The origins and prevalence of texture bias in convolutional neural networks. Advances in Neural Information Processing Systems, 33, 19000-19015. \n\n[2] Geirhos, R., Jacobsen, J. H., Michaelis, C., Zemel, R., Brendel, W., Bethge, M., & Wichmann, F. A. (2020). Shortcut learning in deep neural networks. Nature Machine Intelligence, 2(11), 665-673.\n\n[3] Montero, M. L., Ludwig, C. J., Costa, R. P., Malhotra, G., & Bowers, J. (2020, October). The role of disentanglement in generalisation. In International Conference on Learning Representations.\n\n[4] Montero, M., Bowers, J., Ponte Costa, R., Ludwig, C., & Malhotra, G. (2022). Lost in Latent Space: Examining failures of disentangled models at combinatorial generalisation. Advances in Neural Information Processing Systems, 35, 10136-10149.\n\n** note that I used the same citation numbers the authors used for the other references i.e. [95] above corresponds to reference [95] in the main text.", "rating": 7, "confidence": 5, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "1. The article is very well written and easy to follow. The idea of the concept graph and the manipulation of the dataset is clearly explained and well motivated.\n2. The use of a hypothesis driven research as opposed to the standard benchmarking approach is welcomed change compared to most of the research in AI. That the way the authors present it makes it seem as particularly novel is more a reflection on the field than this particular work (but see later on some suggestions regarding this point).\n3. The point correlations between concept value make learning difficult for the models is important since it highlights how much model still rely on correlations in spite of the fact of their superior generative capabilities.", "weaknesses": "There are no substantial weaknesses, but there are some errors and omissions when discussing the literature which I will discuss below.\n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear Reviewer kruN:\n\nThank you for your positive response! We are delighted that you find our paper \"well written and easy to follow\", the introduced ideas of the concept graph \"clearly explained and well motivated\", and the use of a hypothesis-driven approach \"a welcome change to AI research.\" Below, we address your raised questions and comments.\n\n**Question on the role of the evaluation measure:** Great question! As we note in the appendix, defining a discrete measure that rewards correct concept values is intuitively more appropriate because, e.g., claiming a model can produce \"avocado chairs\" only makes sense if the generated image both looks like an avocado *and* a chair. However, to show our claims also hold for a more continuous measure, we have added a plot showing the progress of cross-entropy between ground-truth concept values and the predicted values via our probes (please see Figure R3). Even with this continuous measure, we observe the sudden emergence, as shown in Figure R3(a). This result is attributable to the multiplicative nature of our task setting, as highlighted in Figure R3(b).\n\n**Question on delayed learning of color:** Our hypothesis is that delayed learning of the capability to manipulate color is due to, in our setup, majority samples in a randomly sampled batch of data possess the same color, i.e., red. Consequently, the model quickly learns to produce the majority color, and then first learns to perform well on producing other concepts, before learning how to produce blue-colored samples, keeping the performance low on generating OOD concepts that require an ability to alter the color values.\n\n**Clarification on relation to grokking:** This is a fair point. The term \"grokking\" is generally supposed to refer to an inability to perform well on test/validation data well after the training error has reduced to zero. In our scenario, since the model is able to generate samples from within the train distribution, grokking is more \"distributional,\" i.e., the model suddenly learns how to generate samples from unseen distributions. We will ensure this point is further clarified in the paper.\n\n---\n\n**Limitations:**\n\n**Response to Comment 1:** We agree with the reviewer that papers like [94], [95], and [99] perform systematic manipulations of training data for evaluating the compositionality of neural networks. Due to space constraints, we were forced to use a reductive assessment of \"benchmarking\", but will add a much more detailed discussion to clarify this. For now, we note that our novelty lies in the use of a model-experimental systems approach for evaluating the compositionality of diffusion models, a thorough formalization of the notion of compositionality in this scenario, relating compositional behavior with emergent phenomena seen in modern generative models, and the consequences of these results on learning biases due to correlations in training data. We will also add discussions on the reviewer's cited works on disentanglement–we were well aware of these papers, but again space constraints disallowed a fruitful discussion of them.\n\n**Response to Comments 2 and 3:** Thank you for pointing out these references on disentanglement! While the goals of these papers are sufficiently different from ours (see the answer to L1 above), we will ensure a thorough discussion is included in the final version of the paper. We especially agree that the dataset structure in [3] is similar to ours, though our provided formalization and the precise results are very different.\n\n**Response to Comment 4:** Thank you for this feedback; we will ensure this is addressed in the final version of the paper.\n"}, {"review_id": "nYJR8lw6hd", "reviewer": "Reviewer_L63E", "summary": "The authors try to understand the compositionality aspects of generative models by training a conditional diffusion model in a toy setting on synthetic data. They show that the models indeed learn to be compositional if we train longer. They also hypothesize that the sudden emergence of compositionality in the later parts of training is due to multiplicity of how the model learns the underlying factors. They also show that it is hard to disentangle and generalize to new concepts if the underlying factors are highly correlated. ", "questions": "1.\tDid you try or have results with synthetic data with > 3 attributes?\n2.\tWhy did you not use synthetic data with real objects instead of colors and shapes? Verifying your claims on even a small real-object dataset would make the paper much more robust and useful. \n3.\tI might have missed this detail, what is the size of the test set, did you look at only the “pink” examples in the figures? \n", "limitations": "N/A", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "The paper is easy to follow. The study of compositionality in diffusion models is a crucial problem to understand. The authors studied it in a controlled setting which is novel. ", "weaknesses": "The authors chose a very simple synthetic dataset to study this problem. The dataset has only 3 attributes. Even though some of the observations make sense, we have no guarantees that any of the results will extend to the real world. \n\nThe figures are quite confusing. Especially the colors. Blue (albeit a different shade), was used as an attribute and to denote the training data. \n\nIn Section 4, the authors used a bit of sensationalist language, such as “sudden emergence” etc, while such terms are not defined nor appropriately cited. They can improve the writing in this section.\n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear Reviewer L63E,\n\nThank you for the insightful review of our paper. We are pleased that you found our research question crucial, our approach novel, and our writing easy to follow. Your specific and constructive recommendations to conduct additional experiments with more than three attributes and more realistic images were greatly appreciated. In the past week, we've committed to further experimentation and created two additional plots (Figs. R1, R2) in response to your feedback.\n\nWe believe these improvements adequately address your concerns. We hope that you will find the revisions satisfactory and that this will solidify your support for the acceptance of our submission.\n\n**Question on Synthetic Dataset with >3 Attributes:** We emphasize that to test our hypotheses in a precise setup, we deliberately chose a simplified data-generating process. However, we agree that extending the results to more complex scenarios is valuable. We have accordingly conducted additional experiments with synthetic data with four attributes, adding “background color” as a new attribute, and have found our claims successfully generalize in this setup as well. Please refer to Figure R1 and the general response for the new experimental results and discussions. We will include these new results in the final version of the paper.\n\n**Question on Using Real Data:** We agree that examining generalization to more complex and realistic images would be a great way to demonstrate the robustness of our findings. Accordingly, we have now run new experiments with more realistic data from CelebA to further verify our claims. Please refer to Figure R2 and the above general response for new experimental results and discussions. We confirmed that our findings on (i) learning dynamics following the structure of the concept and (ii) delayed generalization of minority attribute (gender), hold true, even at larger, more realistic scales. We will include these results in the final version of our paper.\n\n**Question on the size of the test set and whether only “pink” nodes from concept graphs are evaluated:** As noted in Appendix A.5, we generate 50 novel samples via the diffusion model per target concept class, i.e., the class defined by combinations of different concept values, and report the accuracy of probes trained to predict whether a sample contains all underlying concept values that define it. Given 8 nodes, this yields an evaluation of 400 samples per concept graph. That is, the ability to generate samples from both in-distribution (shown as blue nodes in figures) and out-of-distribution (shown as pink nodes in figures) concept classes is evaluated.\n\nWe also note that the “pink” nodes in the figures denote out-of-distribution concept classes, i.e., the model has never seen a sample that comes from that distribution during training. Evaluating the model’s ability to generate samples from such out-of-distribution classes allows us to show that a model can learn to produce samples from classes entirely unseen during training–a primary goal of our work. However, the evaluation does encompass the blue nodes as well (shown as blue lines in learning dynamics plots).\n\n**Clarification on Figures and Colors:**\nThank you for this feedback! We will increase the diversity of colors used to represent different entities to reduce any possible confusion.\n\n**Concerning the use of the term “sudden emergence” in Section 4:**\nWhile we understand the criticism, we stress that the form of sudden generalization wherein a model does exhibit a sharp improvement in performance on a task (herein, generating samples from an out-of-distribution class) is unlike any standard notion of generalization we are taught in classical machine learning. Accordingly, we do firmly believe that a new technical term needs to be devised. To remain consistent with contemporary work [1], we currently use the term “emergence,” but are happy to use a more appropriate term if the reviewer has suggestions. We also emphasize that the use of “emergence” has a long history in related fields: e.g., in cognitive science, where emergence is often used to refer to sharp improvements in a child’s capabilities [2].\n\n[1] Wei et al., 2022. Emergent Abilities of Large Language Models.\n\n[2] Spelke, E.S. and Kinzler, K.D., 2007. Core knowledge. Developmental science.\n"}, {"review_id": "uOipksT6es", "reviewer": "Reviewer_wac1", "summary": "This paper proposes to empirically studied how compositional structure emerges in diffusion model. The paper proposes the abstraction of concept graphs, and illustrates how diffusion models first learn to fit the training data before compositionally generalizing. The paper illustrates how diffusion models have difficulty disentangling data and modeling low data concepts", "questions": "See above", "limitations": "Yes", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The paper is clearly written and the color coding through the text and figures greatly helped comprehension of the paper\n- I enjoyed the introduction of concept graphs and the illustration of compositional distance with respect to graph\n- The analysis in the paper is quite enlightening, illustrating how diffusion models first fit in distribution before generalizing out of distribution, as well as its struggles on finding confounding factors between variable and low data variables.", "weaknesses": "- The evaluation setting is rather simple and focuses on a single synthetic dataset with 3 classes of attributes\n- It would be good to study generalization both to more complex images (for instance photorealistic synthetic scenes rendered using something like Kubric) as well as to more factors larger than 3\n- In practice, a lot of concepts in the world follow the Zipf distribution -- it would be interesting to analyze that\n- It would also be interesting to see how syntatic form would effect compositional generalization\n- Some theoretical analysis on compositional generalization would also be interesting", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear Reviewer wac1,\n\nWe sincerely appreciate the time and effort you invested in your thorough and insightful review of our submission. Your recognition of the paper's well-written aspects, alongside your positive remarks on the analysis and our approach to understanding diffusion models, has greatly encouraged us.\n\nWe are equally grateful for your constructive suggestions to perform new experiments with (i) more than 3 attributes and (ii) more realistic images. Over the past week, we have dedicated ourselves to conducting those experiments and prepared two new plots (Figs. R1, R2) to directly address your feedback.\n\nWith these improvements, we believe we have successfully addressed your concerns. We hope that you are satisfied with the revisions, and that this will strengthen your support for the acceptance of this submission.\n\nBelow, we address your remaining questions and concerns in an effort to provide clarity on any remaining issues.\n\n**Simplicity of Evaluation Setting:**  \nWe emphasize that to test our hypotheses in a precise setup, we deliberately chose a simplified data-generating process. However, we agree that extending the results to more complex scenarios is valuable. In response, we have conducted additional experiments on a more complicated synthetic dataset with four attributes. Please refer to Figure R1 and the general response for the experimental results and observations; we found successful generalization of our claims in this setting as well. We will include these results in the revised manuscript to reflect your feedback.\n\n**Generalization to More Complex Images:**  \nWe agree that examining generalization to more complex and realistic images would be a great way to demonstrate the robustness of our findings. Accordingly, we have now run new experiments with more realistic data from CelebA to further verify our claims. Please refer to the above general response for the experimental results and observations. We confirmed that our findings on (i) learning dynamics following the structure of the concept and (ii) delayed generalization of minority color, hold true, even at larger, more realistic scales.\n\n**Analysis of Zipf Distribution and Role of Syntactic Form:**  \nWe find both these ideas to be very valuable and are keen to explore them further. Currently, we focus only on the goals of demonstrating that compositionality is indeed captured by modern generative models, that it likely underlies some emergent phenomena seen in their training, and the consequences of such sudden learning behaviors when biased datasets (i.e., correlated concepts) are used.\n\n**Theoretical Analysis on Compositional Generalization:**  \nWe emphasize that theoretical characterization of compositional generalization is extremely lacking in contemporary literature; in fact, there is little consensus on basic definitions of what compositionality means, as we note in the paper. By providing a formal framework and defining what we mean by compositionality, we do believe we have taken a step towards making a formal analysis amenable and are keen to follow up on this direction. Currently, we believe that pursuing this would divert attention from the primary motivation of the paper, and we therefore leave this for future work."}, {"review_id": "ZZiZRZk4va", "reviewer": "Reviewer_bYvo", "summary": "The paper studies compositional generation abilities of conditional diffusion models. The main contributions of the paper are the concept graph framework, which is used to examine compositional abilities in a simplified setting, and insights on the learning dynamics of diffusion models.", "questions": "1. When does the sudden emergence of compositionality occur during training, can this be somehow predicted? Does it happen at the same time for different architectures and training configurations (noise schedules, augmentations etc.)? Providing answers to these questions would be valuable to researchers designing inductive biases to models.\n\n2. How is the critical threshold for learning new capabilities in Fig. 8 determined? Is there a practical way of estimating critical thresholds for learning harmful concepts based only on the given training dataset?\n\n3. Why Fig. 8 results could not be interpreted as if you have even a slight amount of harmful concepts in the dataset, then the model can eventually learn these given sufficient amount of training/capacity? From this point of view even a small amount of harmful samples leads to unwanted behavior and filtering training data is very important.\n\nMinor points on improving the clarity:\n\n4. Potential typo at line 188, “0010” corresponds to a sample in the training set.\n\n5. Fig. 5 duplicate sentence. For clarity Fig. 5 b) and c) would benefit having the same units in the x-axis (either optimization steps or epochs).\n\n6. I would remove the bullet point: “if so, under what circumstances does it fail?” from the beginning of Sec. 4 to Sec. 5 because that is where this question is assessed.\n\n7.  Figure 8 lattices are potentially flipped. The lattice of (b) should be the lattice of (a) because in (a) the number of samples from “001” is modified, and vice versa. The lattice (b) is also redundant, I suggest refactoring the figure to contain three subfigures (a) showing the lattice and (b), (c) showing the interventions. Additionally, the x-axis shows the number of samples and the text discusses frequencies. I would switch the x-axis to show frequencies instead of the number of images to improve interpretability in a more general case.\n\n8. Fig. 9 says “In this example, large objects are red, and small objects are blue.”, however in (a) there are large objects in both blue and red.\nIn general, figures would benefit if they were converted to vector graphics format (pdf, svg etc.) because they need to be zoomed in order to view.\n\n", "limitations": "The authors adequately addressed the limitations of their work.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The paper is very well written, motivated, and easy to follow. The problem the authors are investigating is relevant to know the limitations of compositional generation of the current diffusion models.", "weaknesses": "Even though the paper provides a very pleasing framework to investigate compositional capabilities of diffusion models through simple and highly controllable interventions, I think the submission is incomplete. This is mainly because the paper considers only a single architecture and does not provide guidelines how insights from concept graph framework could be further studied or provide validation that they hold in a large scale setting. Additionally, I would like to see a summary answering the questions from the abstract: “What are the reasons underlying this behavior? Which concepts does the model generally find difficult to compose to form novel data?”", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear Reviewer bYvo,\n\nThank you so much for carefully recognizing the strengths of our work while providing us with concrete action items to make our submission more impactful and complete. In response, we have intensively experimented over the last week to generate three new plots for you (Figs. R2, R4, R5) and also drafted a new paragraph to summarize the results of our study. With these additions, we have fully addressed your concerns, and hope that you could now recommend this submission for acceptance.\n\n---\n\n**Weaknesses:**\n- **Exploration of Architectures [Figure R4]:** Thank you for your great suggestion. In response, we conducted experiments with more diverse architectures and model configurations. Please refer to Figures R4 and R5, and the general response. We will include these new results in the final version of the paper.\n- **Lack of Guidelines on Concept Graph Framework [Figure R2]:** We agree that examining generalization to a large-scale setting would be a great way to demonstrate the robustness of our findings. In response, we have now run new experiments with more realistic data from CelebA to further verify our claims. Please refer to Figure R2 and the above general response for new experimental results and discussions. We confirmed that our findings on (i) learning dynamics following the structure of the concept graph and (ii) delayed generalization of minority attribute (gender), hold true, even at larger, more realistic scales. We will include these results in the final version of our paper.\n\n- **Summary Answering the Questions from the Abstract\n\"What are the reasons underlying this behavior (unpredictable failure in compositional generalization)? Which concepts does the model generally find difficult to compose to form novel data?\":**\nThis is a great suggestion. In response, we have drafted a concluding paragraph to be included in the final draft:\n\"*In this work, we introduced a concept graph framework and used controlled interventions with a synthetic dataset to formulate and characterize four novel failure modes in the compositional generalization of diffusion models. We first hypothesized and verified that compositional generalization to a concept class far from the training data, based on concept distance, can fail, as it requires more optimization steps due to the sequential nature of generalization (Figure 5). Furthermore, when a particular concept value is underrepresented in the training dataset (e.g., a minority color or gender), avoiding misgeneralization of the concept value, and thus achieving compositional generalization, requires far more additional optimization steps, even after achieving perfect performance on the training set (Figure 6). We also discovered a critical threshold for the inclusion of certain concepts in the training data, which is useful for deliberately making the model fail at learning harmful concepts (Figure 8). Finally, we found that correlations and biases in concept variables can make compositional generalization difficult, and fine-tuning does not provide a solution (Figure 9). Overall, our synthetic data approach allowed us to apply controlled interventions that shed light on four different mechanisms behind the failure of diffusion models in compositional generalization.*\"\n\n---\n\n\n**Questions:**\n- **Q1A Can we predict the sudden emergence of compositional abilities?** \nGreat question! The key challenge in predicting the timing of the sudden emergence of compositional ability lies in tracking the progress during the plateau phase of learning dynamics, where the compositional loss remains flat. Our insight into multiplicative emergence suggests that we can break down the compositional task into individual tasks to make progress on each sub-task visible.\n- **Q1B Does the emergence of compositional ability happen at the same time for different architectures and training configurations (noise schedules, augmentations, etc.)?** \nAs mentioned in the previous paragraph and in our general response, we conducted experiments with more diverse architectures and configurations. As you can see in Figures R4 and R5, the result shows rich phenomena, including how attention mechanisms accelerate compositional generalization.\n- **Q2 Response to how critical thresholds in Figure 8 are assigned and can they be practically estimated:**\nWe compute the thresholds, represented by a dotted line, as the minimum quantity of samples required to achieve non-zero accuracy in distance 2 compositional generalization for “111.” However, making quantitative estimations of critical thresholds for learning harmful concepts solely based on the available training data may present challenges.\n- **Q3 Response to re-interpretation of results in Figure 8:** \nWe share your perspective. To clarify, in lines 328–331, we emphasize that when the quantity of data falls below a certain critical threshold, additional data filtering becomes unnecessary. Another way to perceive this observation is that when the critical threshold is exceptionally low, a more rigorous data filtering approach becomes imperative. This alternative interpretation of our finding holds equal significance and we will expand the discussion in the paper to highlight the same.\n\n---\n\n**Minor points:**\nThank you so much for making all the suggestions for better presentation in minor points. We will reflect all of your feedback in the final draft."}]

Author rebuttal:
Dear Reviewers,

We thank all reviewers for their diligent efforts in evaluating our submission. We are pleased by the unanimous recognition and support for our scientific approach, aimed at enhancing the understanding of diffusion models using minimal synthetic data. We would also like to thank the reviewers for the constructive and actionable feedback to further strengthen our claims in more diverse and realistic setups. In the attached one-page PDF, we have incorporated **five** new experimental figures that cover all the requested additions. With these enhancements, we are now confident that our paper will constitute a unique and impactful contribution to the NeurIPS conference.

---

**Summary of new experimental figures in response to reviewers' feedback:**

*   **[Figure R1] Sequential Generalization on Concept Graph Holds with 4 (>3) Attributes (L63E, wac1):** We have performed new experiments with 4 attributes. Here, we introduced a new concept of "background color" (ranging from 0 for black to 1 for white) in addition to the three existing concepts of "shape," "object color," and "size." Through this experiment, we confirmed that our hypothesis of “Sequential compositional generalization on concept graphs” generalizes to the case with more than three attributes. Memorization occurs first, as represented by the blue lines. Subsequently, generalization happens in sequence based on the concept distance: Concept distance 1 (depicted by pink lines) comes first, followed by concept distance 2 (red lines). Lastly, concept distance 3, represented by the green line, arises. Overall, our claim holds more generally beyond the concept cube!

*   **[Figure R2] Our Observations Hold in More Realistic Dataset (bYvo, L63E, wac1):** To generalize our hypotheses in more realistic settings, we have performed experiments with the CelebA dataset. We have chosen three attributes for our concepts: Gender (categorized as Female and "Male"), Smiling ("Smiling" and Not Smiling), and Hair Color ("Black Hair" and "Blonde Hair"). The corresponding concept graph for these attributes in the CelebA dataset can be found in Figure R2(a). We trained a 3-layer CNN for each individual concept and used the product of accuracies from each concept as the evaluation measure. While not all nodes reached an accuracy of 100% due to insufficient training time, our results consistently support our primary observations:

      *   **Learning dynamics respect the structure of the concept:** Figure R2(b) illustrates the learning dynamics using the real CelebA dataset. We again observe the sequential pattern in generalization. The accuracy begins to rise at concept distance 1 (denoted by pink lines), followed by concept distance 2 (red line). Notably, the node labeled '011' – even though it's at concept distance 1 – precedes the memorization stage (represented by blue lines). This exception can be attributed to the gender bias present in the training data, which we'll explain below.
      *   **Delayed Generalization of Minority Attribute:** In Figure R2(c), we plotted the accuracy for the individual concept of gender (i.e., Female and Male). The Female concept class's training curve (represented by red lines) reaches convergence faster compared to the Male concept class (depicted by blue lines). This disparity stems from the Female class being more predominant in the dataset, thereby making the Male class a minority. This observation offers useful insights for practitioners: To mitigate biased outcomes against minority groups, we need to further train the diffusion model beyond its initial convergence on the training dataset.

*   **[Figure R3] Emergence Persists with Probabilistic Accuracy Measure (kruN):** In Figure R3, we plot the learning dynamics for the probabilistic accuracy. Here, the probabilistic accuracy is calculated as the product of probabilistic accuracies $p(f(x^{(n)}), v^{(n)})$ from each individual concept, where $n$ is the index of the test sample, $f(\cdot)$ represents the classifier, $x^{(n)}$ is the generated image, and $v^{(n)}$ denotes the actual (ground truth) concept class (i.e., attribute). Using this probabilistic metric, we once again observe the sudden emergence of capability in the test nodes (represented by pink and red lines), especially for node "111" (highlighted by the red line). This confirms that our observation of the emergent capability isn't solely due to the choice of evaluation metric.

*   **[Figure R4] Attention Mechanisms Accelerate Compositional Generalization (bYvo):** In Figures R4(a)-(c), we explored three distinct architectures as the backbone of the diffusion model: U-Net without global attention (Figure R4(a)), U-Net with global attention (Figure R4(b)), and the Transformer model (Figure R4(c)). Our observations indicate that the models with attention mechanisms initially exhibit a slower learning curve. However, once they commence learning, there is a notable surge in their accuracy. Specifically, in Figure R4(c), the Transformer's accuracy initially increased slowly but quickly peaked at 100% shortly after.
*   **[Figure R5] Our Observations Hold Robustly across Diverse Configurations (bYvo):** Figures R5(a)-(c) show the learning dynamics across different hyperparameter settings for the diffusion model. Specifically, we varied the number of units in U-Net, labeled as "units" in the figures, and the number of time steps in the diffusion model, denoted as "time steps" in the figures. We verified that our findings regarding (i) sequential generalization of capability based on concept distance and (ii) sudden emergence of capability remain consistent across all hyperparameter settings. Based on your insightful recommendations, we are carrying out extensive experiments, e.g., using various noise schedules. However, due to page limitations, we have omitted some of these. Importantly, all the results we have now support our primary observations!

Meta-review: This work studies compositional generalization of conditional diffusion models. All four reviewers are positive about this work. The compositionality of diffusion models is crucial and the controlled study is novel, the idea of concept graphs is interesting, and the finding on learning dynamics is intriguing that diffusion models first fit in distribution before generalizing out of distribution. Main concerns are lack of theoretical justification and whether the results and findings extend to large scale realistic world. AC agrees this work sheds new insights into the understanding of diffusion models, and thus recommends acceptance.
Decision: Accept (poster)