Title: Concept Algebra for (Score-Based) Text-Controlled Generative Models
Venue: NeurIPS 2023
Accepted: yes
Keywords: disentanglement; representation learning; text-controlled generative models; diffusion models
Abstract: This paper concerns the structure of learned representations in text-guided generative models, focusing on score-based models. A key property of such models is that they can compose disparate concepts in a 'disentangled' manner.This suggests these models have internal representations that encode concepts in a 'disentangled' manner. Here, we focus on the idea that concepts are encoded as subspaces of some representation space.  We formalize what this means, show there's a natural choice for the representation, and develop a simple method for identifying the part of the representation corresponding to a given concept. In particular, this allows us to manipulate the concepts expressed by the model through algebraic manipulation of the representation. We demonstrate the idea with examples using Stable Diffusion.

Concept Algebra for (Score-Based) Text-Controlled Generative Models

Abstract

This paper concerns the structure of learned representations in text-guided generative models, focusing on score-based models. A key property of such models is that they can compose disparate concepts in a 'disentangled' manner. This suggests these models have internal representations that encode concepts in a 'disentangled' manner. Here, we focus on the idea that concepts are encoded as subspaces of some representation space. We formalize what this means, show there's a natural choice for the representation, and develop a simple method for identifying the part of the representation corresponding to a given concept. In particular, this allows us to manipulate the concepts expressed by the model through algebraic manipulation of the representation. We demonstrate the idea with examples using Stable Diffusion.

1 Introduction

Large-scale text-controlled generative models are now dominant in many parts of modern machine learning and artificial intelligence [ e.g., Bro + 20; Rad + 21; Bom + 21; Koj + 22 ] . In these models, the user provides a prompt in natural language and the model generates samples based on this prompt-e.g., in large language models the sample is a natural language response, and in text-to-image models the sample is an image. These models have a remarkable ability to compose disparate concepts to generate coherent samples that were not seen during training.This suggests that these models have some internal representation of high-level concepts that can be manipulated in a 'disentangled' manner. Broadly, the goal of this paper is to shed light on how this concept representation works, and how it can be manipulated. We focus on text-to-image diffusion models, though many of the ideas are generally applicable.

Our starting point is the following commonly observed structure of representations:

Each data point x is mapped to some representation vector Rep ( x ) ∈ /Rbbb p .

High-level concepts correspond to subspaces (directions) of the representation space.

Perhaps the best known example of this structure is in word embeddings, where semantic relationships such as Rep ( 'king' ) -Rep ( 'queen' ) ≈ Rep ( 'man' ) -Rep ( 'woman' ) suggest that high-level concepts (here, sex) are encoded as directions in the representation space [ Mik + 13 ] . This kind of encoding of concepts has been argued to occur in many contexts, including in the latent space of variational autoencoders [ ZW20; Khe + 20; MFM21 ] and in the latent space of language models [ Bol + 16; GG19; Rad + 21; Elh + 22 ] . We'll call representations of this kind arithmetically composable , because composition corresponds to arithmetic operations on the representation vectors. The goal of this paper is to develop arithmetically composable representations of text for score-based text to image models.

There are two main motivations. First, understanding the structure of the representation space is important for foundational progress on understanding the emergent behavior of text-controlled generative models. It is particularly interesting to study this question in the text-to-image setting 37th Conference on Neural Information Processing Systems (NeurIPS 2023).

(a)

Rep

[

'a portrait of mathematician'

]

]

because the multi-modality of the data makes it straightforward to distinguish concepts from inputs, and because it is not clear a priori that the models themselves build in any inductive bias towards arithmetic structure. The secondary motivation is that having such a representation would allow us to manipulate the concepts expressed by the model through linear-algebraic manipulations of representation of the input text; fig. 1 illustrates this idea.

The development of the paper is as follows:

We develop a mathematical formalism for describing the connection between representation structures and concepts for text-controlled generative models.

Using this formalism, we show that the Stein score of the text-conditional distribution is an arithmetically composable representation of the input text.

Then, we develop concept algebra as a method for manipulating the concepts expressed by the model through algebraic manipulation of this representation. We illustrate the approach with examples manipulating a variety of concepts.

2 A Mathematical Framework for Concepts as Subspaces

The first task is to develop a precise mathematical formalism for connecting the structure of representations and high-level concepts. This is necessary for understanding when such representations are possible, how to construct them, and when they may fail. We must make precise what is a concept, how concepts relate to inputs x , and what it means to represent concepts.

Concepts The real-world process that generated the training data has the following structure. First, images Y are generated according to some real-world, physical process. Then, some human looks at each image and writes a caption describing it. Inverting this process, each text x induces some probability density p ( y | x ) over images Y based on how compatible they are with x as a caption. The (implicit) goal of the generative model is to learn this distribution.

To write the caption, the human first maps the image to a set of high-level variables summarizing the image's content, then uses these latent variables to generate the text X . Let C be the latent variable that captures all the information about the image that is relevant for a human writing a caption. So,

The random variable C captures the information that is jointly relevant for both the image and caption. Variables in C include attributes such as has _ mathematician or is _ man , but not pixel _ 14 _ is _ red . We define concepts in terms of the latent C .

Definition 2.1. A concept variable Z is a C -measurable random variable. The concept /Zcal associated to Z is the sample space of Z .

Now, the full set of all possible concepts is unwieldy. Generally, we are concerned only with the concepts elicited by a particular prompt x .

Definition 2.2. A set of concepts /Zcal 1 , . . . , /Zcal k is sufficient for x if p ( c | X = x , Z 1: k = z 1: k ) = p ( c | Z 1: k = z 1: k ) for all z 1 , . . . , z k ∈ /Zcal 1 × · · · ×/Zcal k .

For example, the concept profession would be sufficient for the prompt 'A nurse'. This prompt induces a distribution on many concepts (e.g., background is likely to be a hospital) but these other concepts are independent of the caption given profession = nurse . Then,

Concept Distributions Following eq. (2), we can view each text x as specifying a distribution p ( z 1: k | x ) over latent concepts /Zcal 1 , . . . , /Zcal k . This observation lets us make the relationship between text and concepts precise.

Definition 2.3. A concept distribution Q is a distribution over concepts. Each text x specifies a concept distribution as Qx = p ( z 1: k | x ) .

That is, we move from viewing text as expressing specific concept values ( is_mathematician = 1) to expressing probability distributions over concepts ( Qx ( is_mathematician = 1 ) = 0.99). The probabilistic view is more general-deterministically expressed concepts can be represented as degenerate distributions. This extra generality is necessary: for example, the prompt 'a person' induces a non-degenerate distribution over the sex concept.

Concept Representations A text-controlled generative model takes in prompt text x and produces a random output Y . Implicitly, such models are maps from text strings x to the space of probability densities over Y . We'll define a representation Rep ( x ) ∈ /Rcal of x as any function of x that suffices to specify the output distribution. We define f r ( · ) as the density defined by r ∈ /Rcal , and assume that the model learn's the true data distribution of Y | X = x :

The key idea for connecting representations and concepts is to move from considering representations of prompts to representations of concept distributions.

Definition 2.4. A concept representation Rep is a function that maps a concept distribution Q to a representation Rep ( Q ) ∈ /Rcal , where /Rcal is a vector space. The representation of a prompt x is the representation of the associated concept distribution, Rep ( x ) : = Rep ( Qx ) .

There are two reasons why this view is desirable. First, defining the representation in terms of the concept distribution makes the role of concepts explicit-this will allow us to explain how representation structure relates to concept structure.

The second reason is that it allows us to reason about representations that do not correspond to any prompt. Every prompt defines a concept distribution, but not every concept distribution can be defined by a prompt. This matters because we ultimately want to reason about the conceptual meaning of representation vectors created by algebraic operations on representations of prompts. Such vectors need not correspond to any prompt.

Arithmetic Compositionality We now have the tools to define what it means for a representation to be arithmetically composable. We define composability for a pair of concepts /Zcal and /Wcal . In the subsequent development, our aim will be to manipulate /Zcal while leaving /Wcal fixed.

Definition 2.5. A representation Rep is arithmetically composable with respect to concepts /Zcal , /Wcal if there are vector spaces /Rcal Z and /Rcal W such that for all concept distributions of the form Q ( z , w ) = QZ ( z ) QW ( w ) ,

where Rep Z ( QZ ) ∈ /Rcal Z and Rep W ( QW ) ∈ /Rcal W .

In words: we restrict to product distributions to capture the requirement that the concepts /Zcal and /Wcal can be manipulated freely of each other (the typical case is that one or both of QZ and QW are degenerate, putting all their mass on a single point). Then, the definition requires that there are fixed subspaces corresponding to each concept in the sense that, e.g., changing only QZ induces a change only in /Rcal Z .

3 The Score Representation

We now have an abstract definition of arithmetically composable representation. The next step is to find a specific representation function that satisfies the definition.

We will study the following choice.

Definition 3.1. The score representation s [ Q ] of a concept distribution Q is defined by:

The centered score representation ¯ s [ Q ] is defined by ¯ s [ Q ] : = s [ Q ] -s [ Q 0 ] .

Here, s [ Q ] is itself a function of y and the representation space /Rcal is a vector space of functions. This is a departure from the typical view of representations as elements of /Rbbb p . The score representation can be thought of as a kind of non-parametric representation vector. The centered score representation just subtracts off the representation of some baseline distribution Q 0 . 1

The main motivation for studying the score representation is that

The importance of this observation is that ∇ y log p ( y | x ) is learnable from data. In fact, this score function is ultimately the basis of many controlled generation models [ e.g., HJA20; Ram + 22; Sah + 22 ] , because it characterizes the conditional while avoiding the need to compute the normalizing constant [ HD05; SE19 ] . Accordingly, we can readily compute the score representation of prompts in many generative models, without any extra model training.

Causal Separability The score representation does not have arithmetically composable structure with respect to every pair of concepts. The crux of the issue is that concepts are reflected in the representation based on their effect on Y . If the way they affect Y depends fundamentally on some interaction between two concepts, the representation cannot hope to disentangle them. Thus, we must rule out this case.

Definition 3.2. We say that Y is causally separable with respect to /Zcal , /Wcal if there exist unique Y -measurable variables Y /Zcal , Y /Wcal , and ξ such that

Y g Y , Y , ξ for some invertible and differentiable function g , and

Informally, the requirement is that we can separately generate Y /Zcal and Y /Wcal as the part of the output affected by /Zcal and /Wcal (and ξ as the part of the image unrelated to Z and W ), then combine these parts to form the final image. That is, generating the visual features associated to a concept /Wcal can't require us to know the value of another concept /Zcal . As an example where causal separability fails, consider the concepts of species /Wcal = { deer , human } and sex /Zcal = { male , female } . It seems reasonable that there is a Y -measurable Y /Wcal that is the species part of the image-e.g., the presence of fur vs skin, snouts vs noses, and so forth. However, there is no part of Y that corresponds to a sex concept in a manner that's free of species. The reason is that the visual characteristics of sex are fundamentally different across species-e.g., male deer have antlers, but humans usually do not. In fig. 8 we test this example, finding that concept algebra fails in the absence of causal separability.

It turns out it suffices to rule out this case (all proofs in appendix):

Proposition 3.1. If Y is causally separable with respect to /Wcal and /Zcal , then the centered score representation is arithmetically composable with respect to /Wcal and /Zcal .

That is: the (centered) score representation is structured such that concepts correspond to subspaces of the representation space.

4 Concept Algebra

We have established that concepts correspond to subspaces of the representation space. We now consider how to manipulate concepts through algebraic operations on representations.

To modify a particular concept /Zcal we want to modify the representation only on the subspace /Rcal Z corresponding to /Zcal . For example, consider changing the style concept to Fauvism . Intuitively, we want an operation of the form:

where proj style is the projection onto the subspace corresponding to the style concept. The idea is that the representation of the original prompt x orig is unchanged except on the style subspace. On the style subspace, the representation takes on the value elicited by the new prompt x new = 'Fauvism style'.

There are two main challenges for putting this intuition into practice. First, because we are working with an infinite dimensional representation, it is unclear how to do the projection. Second, although we know that some /Rcal Z exists, we still need a way to determine it explicitly.

4.1 Concept Manipulation through Projection

Following proposition 3.1, we have that

for some representation functions ¯ s Z and ¯ sW with range in /Rcal Z and /Rcal W respectively. We have that the Z -representation space is

The first obstacle is that /Rcal Z is a function space, making algebraic operations difficult to define. The resolution is straightforward. In practice, score-based models generate samples by running a discretized (stochastic) differential equation forward in time. These algorithms only require the score function evaluated at the finite set of points. At each y , we have that ¯ s ( y ) ∈ /Rbbb m . Accordingly, by restricting attention to a single value of y at a time, we can use ordinary linear algebra to define the manipulations:

Our goal is to find a projection onto /Rcal Z .

Definition 4.1. The Z subspace at y is

and the Z-projection at y , denoted proj Z ( y ) is the projection onto this subspace.

If we can compute proj Z ( y ) then we can just edit the representation at each point y . That is, we transform the score function at each point:

We then draw samples from the stochastic differential equation defined by ¯ s edit .

4.2 Identifying the Concept Subspace

The remaining obstacle is that we need to explicitly identify /Rcal Z ( y ) , so that we can compute proj Z ( y ) . The problem is that the function ¯ s Z in eq. (6) is unknown, so we cannot compute /Rcal Z ( y ) directly. Our strategy for estimating the space is based on the following proposition.

Proposition 4.1. Let Q W be any fixed distribution over the W concept and Q 0 Z be any reference distribution over Z. Then, assuming causal separability for /Zcal , /Wcal ,

The importance of this expression is that it does not require the unknown s Z .

We'll use eq. (10) to identify /Rcal Z ( y ) . The idea is to find a basis for the subspace using prompts x 0 , . . . x k that elicit distributions of the form Qx = QZ Q W . For example, to identify the sex concept we use the prompts x 0 = 'a man' and x 1 = 'a woman', with the idea that

with the same marginal distribution QW . We then use the prompts to define the estimated subspace as

That is, we write k + 1 prompts x 0 , . . . x k designed so that each elicits a different distribution over Z , but the same distribution on W . Then, the estimated subspace is given by eq. (12).

4.3 Concept Algebra

Summarizing, our approach to algebraically manipulating concepts is:

Find prompts x 0 , . . . , x k such that each elicits a different distribution on Z , but the same distribution on W . That is, Qxj = Q j Z QW for each j .

Construct the estimated representation space ˆ /Rcal Z ( y ) following eq. (12), and define proj Z ( y ) as the projection onto this space.

Sample from the discretized SDE defined by the manipulated score representation 2

Implementation of eq. (13) with the diffusion model is described in appendix A.

5 Validity of Concept Subspace Identification

The procedure described in the previous section relies on finding spanning prompts x 0 , . . . , x k for the target concept subspace. These prompts must satisfy Qxj = Q j Z QW for some common QW , and we must have sufficient prompts to span the subspace. The first condition is a question of prompt design, and is often not too hard in practice. However, it is natural to wonder when it's possible to actually recover /Rcal Z using only a practical number of prompts. We give some results showing that the dimension of /Rcal Z ( y ) is often small, and thus can be spanned with a small number of prompts. Note that these results rely on the special structure of the score representation, and may not hold for other representations.

First, the case where /Zcal is categorical with few categories:

This result covers concepts such as sex , which can be spanned with only two prompts.

Proposition 5.1. Assuming causal separability holds for /Zcal , /Wcal . If /Zcal is categorical with L possible values (L ≥ 2 ), then dim ( /Rcal Z ( y )) ≤ L -1 .

The next result extends this to certain categorical concepts with large cardinality, such as style . The idea is that if a concept is composed of finer grained categorical concepts, each with small cardinality, then the representation space of the concept is also low-dimensional. For example, style may be composed of lower-level concepts such as color , stroke , textures , etc.

Proposition 5.2. Suppose /Zcal is composed of categorical concepts { /Zcal k } K k = 1 each with the number of categories L k , in the sense that /Zcal = /Zcal 1 × . . . /Zcal k . Assume Y satisfies causal separability with respect to /Zcal , /Wcal , with Y /Zcal the corresponding Y-measurable variable for /Zcal . Further assume that there exists Y /Zcal -measurable variables Y /Zcal k such that p ( y /Zcal | z ) = Π K k = 1 p ( y /Zcal k | z k ) . Then

Following this result, we might take the spanning prompts for style to be x 0 = 'a mathematician in Art Deco style', x 1 = 'a mathematician in Impressionist style', etc. Each of these prompts elicit a fixed distribution QW over the content, but varies the distribution QZ over style. If style is composed of finer-grained attributes, a relatively small set of such of prompts will suffice.

6 Experiments

We have formalized what it means for concepts to correspond to subspaces of the representation space, and derived a procedure for identifying and editing the subspaces corresponding to particular concepts in the score representation. We now work through some examples testing if this subspace structure does indeed exist, and whether it enables manipulation of concepts.

Many concepts are represented as subspaces First, we check whether the subspace structure does indeed exist. To this end, we generate randomly selected prompts-e.g., 'a black dog sitting on the beach'-and attempt to change binary concepts expressed in the prompt. For example, we change the subject to be a cat by manipulating the concept dog -vs -cat with concept algebra. This, and other examples, are shown in fig. 2. It is clear that we are able to manipulate the target concept-providing evidence that these concepts are represented as subspaces.

Concept Algebra can disentangle hard-to-separate concepts We stress-test concept algebra by using it to sample images expressing combinations of concepts that occur rarely in the training data. Specifically, we look at unlikely subject / style combinations-e.g., 'A nuclear power plant in Baroque painting'. We accomplish this by taking a base prompt that generates a high-quality image, but with the wrong style (e.g., 'A nuclear power plant'). Then we use concept algebra to edit the style. We compare this with directly prompting the model, and with concept composition [ e.g., Du + 21; Liu + 21 ] . The later is a method that adds on a score representation of the style to the base prompt (without projecting on to a target subspace).

Weused each of the three methods to sample images expressing to 49 anti-correlated content / style pairs. Human raters were then presented with the outcomes alongside reference images, and asked to rank them based on adherence to the desired style and content. This evaluation was replicated across 10 different raters. Refer to fig. 3 for illustrative examples. Raters consistently favored images produced by concept algebra, as highlighted in table 1. This aligns with our theory, suggesting concept algebra's adeptness in retaining content while altering style. See appendix C for further details.

Anuclearpower plant

concept-proj

Pre-Raphaelite painting style

Amodern minima

concept-proj

concept

direct-pro

traditional Chinese painting

A modern minimalist house

cone

ept-pro

concept-compose

direct-prompt

An artificial Iintelligencelab

Mixture concept distributions Next, our theory predicts that we can use concept algebra to sample from mixture (non-degenerate) distributions over concept values. Consider sex . Figure 1a shows that x = 'a portrait of a mathematician' almost always generates pictures of men. In fig. 1b we sample from the distribution induced by

and observe that we indeed get a mixture distribution (induced by 'person') over sex.

Non-prompted edits to the subspace edit the concept Concept algebra uses reference prompts (e.g., 'woman' or 'person') to set the target concept distribution. It's natural to ask what happens if we make an edit to a concept subspace that does not correspond to a reference prompt. In fig. 4, we sample from

The representation vector 1 2 s [ 'a male nurse' ]+ 1 2 s [ 'a female nurse' ] need not correspond to any English prompt. We observe that modifications on the subspace still affect just the sex concept though-the samples are androgynous figures!

Mask as a Concept Finally, we consider a more abstract kind of concept motivated by the following problem. Suppose we have several photographs of a particular toy, and we want to generate an image of this toy in front of the Eiffel Tower. In principle, we can do this by fine-tuning the model (e.g., with dreambooth) to associate a new token (e.g., 'sks toy') with the toy. Then, we can generate the image by conditioning on the prompt 'a sks toy in front of the Eiffel Tower'. In practice, however, this can be difficult because the fine-tuning ends up conflating the toy with the background in the demonstration images. E.g., the prompt 'a sks toy in front of the Eiffel Tower' tends to generate images featuring carpet; see fig. 5b.

Intuitively, we might hope to fix this problem by finding a concept subspace that excludes background information. Given such a 'subject subspace', we could mask the subject out of the image, generate the background, and then edit the subject back in. In appendix C.1 we explain how to construct such a subspace using the prompts x 0 = 'a toy' and x 1 = 'a soccer ball'. Figure 5 shows the sampled output.

(a)

s

[

'a portrait of a nurse'

]

(b) ( /Ibbb -proj sex ) s [ 'a portrait of a nurse' ] + proj sex ( 1 2 s [ 'a female nurse' ]+ 1 2 s [ 'a male nurse' ])

7 Discussion and Related Work

We introduced a framework illustrating that concepts align with subspaces of a representation space. Through this, we validated the structure of the score representation and derived a method to identify the subspace for a concept. We then demonstrated concept manipulation in a diffusion model's score representation.

Concepts as Subspaces There has been significant interest in whether and how neural representations encode high-level concepts. A substantial body of work suggests that concepts correspond to subspaces of a representation space [ e.g., Mik + 13; MYZ13; PSM14; GL14; Aro + 15; GAM17; AH19 ] . Often, this work focuses on a specific representation learning approach and is either empirical or offers domain-tied theoretical analysis. For instance, in word embeddings, theories explaining observed structures depend on the unique nature of language [ e.g., Aro + 15; AH19 ] . In contrast, our paper's mathematical development is broad-we merely stipulate that the data have two views separated by a semantically meaningful space. We argue that the concepts-assubspaces structure stems from the structure of probability theory, independent of any specific architecture or algorithm.

Our work also relates to studies that assume training data arises from a specific latent variable model and demonstrate that learned representations (partially) uncover these latent variables [ e.g., HM16; HM17; HST19; Khe + 20; VK + 21; Eas + 22; Hig + 18; Zim + 21 ] . This literature often aims for "disentangled" representations where each latent space dimension matches a single latent factor. Unlike these, we don't presuppose a finite set of latent factors driving the data.

Instead, our representations define probability distributions over latent concepts, not merely recovering them. This non-determinism, as observed, is generally necessary.

Controlling Diffusion Models To demonstrate the concept-as-subspace structure, we developed a method for identifying the subspace corresponding to a given concept and showed how to manipulate concepts in the score representation of a diffusion model. We emphasize that our contribution here is not the manipulation procedure itself, but rather the mathematical framework that makes this procedure possible. In particular, the requirement to manipulate entire score functions is somewhat burdensome computationally. However, the ability to precisely manipulate individual concepts is clearly a useful tool, and it is an intriguing direction for future work to develop more efficient procedures for doing so. We conclude by surveying connections to existing work on controlling diffusion models.

One idea has been to take the bottleneck layer of UNet as a representation space and control the model by manipulating this space [ KJU22; Haa + 23; Par + 23 ] . This work does not consider text controlled models. It would be intriguing to understand the connection to the score-representation view, as moving from manipulation of the score to manipulation of the bottleneck layer would be a large computational saving.

(a)

s

'a toy in front of the Eiffel Tower'

[

]

(c) ( /Ibbb -proj subject ) s [ 'a toy in front of the Eiffel Tower' ] proj s 'a sks toy in front of the Eiffel Tower'

+ subject dreambooth [ ]

Concept algebra can be seen as providing a unifying mathematical view on several methods that manipulate the score function [ e.g., Du + 21; Liu + 21; NBP22; Ano23 ] . Du et al. [ DLM20 ] and Liu et al. [ Liu + 22 ] manipulate concepts via adding and subtracting scores. Negative prompting is a widely-used engineering trick that 'subtracts off' a prompt expressing unwanted concepts. In section 6 and appendix D we compared against these heuristics and show that concept algebra is more effective at manipulating concepts in isolation. Couairon et al. [ Cou + 22 ] use score differences to identify objects' locations in images; this inspired our approach in section 6 and appendix C.1. In each case, we have seen that this kind of manipulation may be viewed as editing the subspace corresponding to some concept.

References

[ AH19 ] C. Allen and T. Hospedales. 'Analogies explained: towards understanding word embeddings'. In: International Conference on Machine Learning . PMLR. 2019 (cit. on p. 9).

[ Ano23 ] Anonymous. 'Reduce, reuse, recycle: compositional generation with energy-based diffusion models and MCMC'. In: Submitted to The Eleventh International Conference on Learning Representations . under review. 2023 (cit. on p. 10).

[ Aro + 15 ] S. Arora, Y. Li, Y . Liang, T . Ma, and A. Risteski. A latent variable model approach to pmi-based word embeddings . 2015. arXiv: 1502.03520 (cit. on p. 9).

[ Bol + 16 ] T. Bolukbasi, K.-W . Chang, J. Y. Zou, V . Saligrama, and A. T . Kalai. 'Man is to computer programmer as woman is to homemaker? debiasing word embeddings'. Advances in neural information processing systems (2016) (cit. on p. 1).

[ Bom + 21 ] R. Bommasani, D. A. Hudson, E. Adeli, R. Altman, S. Arora, S. von Arx, M. S. Bernstein, J. Bohg, A. Bosselut, E. Brunskill, et al. On the opportunities and risks of foundation models . 2021. arXiv: 2108.07258 (cit. on p. 1).

[ Bro + 20 ] T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P . Dhariwal, A. Neelakantan, P . Shyam, G. Sastry, A. Askell, et al. 'Language models are few-shot learners'. Advances in neural information processing systems (2020) (cit. on p. 1).

[ Cou + 22 ] G. Couairon, J. Verbeek, H. Schwenk, and M. Cord. 'Diffedit: diffusion-based semantic image editing with mask guidance'. arXiv preprint arXiv:2210.11427 (2022) (cit. on p. 10).

[ DLM20 ] Y. Du, S. Li, and I. Mordatch. 'Compositional visual generation with energy based models'. Advances in Neural Information Processing Systems (2020) (cit. on p. 10).

[ Du + 21 ] Y. Du, S. Li, Y . Sharma, J. Tenenbaum, and I. Mordatch. 'Unsupervised learning of compositional energy concepts'. Advances in Neural Information Processing Systems (2021) (cit. on pp. 7, 10).

[ Eas + 22 ] C. Eastwood, A. L. Nicolicioiu, J. von Kügelgen, A. Keki´ c, F . Träuble, A. Dittadi, and B. Schölkopf. 'Dci-es: an extended disentanglement framework with connections to identifiability'. arXiv preprint arXiv:2210.00364 (2022) (cit. on p. 9).

[ Elh + 22 ] N. Elhage, T . Hume, C. Olsson, N. Schiefer, T . Henighan, S. Kravec, Z. Hatfield-Dodds, R. Lasenby, D. Drain, C. Chen, et al. 'Toy models of superposition'. arXiv preprint arXiv:2209.10652 (2022) (cit. on p. 1).

[ GAM17 ] A. Gittens, D. Achlioptas, and M. W . Mahoney. 'Skip-gram- zipf + uniform = vector additivity'. In: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) . 2017 (cit. on p. 9).

[ GL14 ] Y. Goldberg and O. Levy. Word2vec explained: deriving mikolov et al.'s negativesampling word-embedding method . 2014. arXiv: 1402.3722 (cit. on p. 9).

[ GG19 ] H. Gonen and Y. Goldberg. Lipstick on a pig: debiasing methods cover up systematic gender biases in word embeddings but do not remove them . 2019. arXiv: 1903.03862 (cit. on p. 1).

[ Haa + 23 ] R. Haas, I. Huberman-Spiegelglas, R. Mulayoff, and T . Michaeli. 'Discovering interpretable directions in the semantic latent space of diffusion models'. arXiv preprint arXiv:2303.11073 (2023) (cit. on p. 10).

[ Hig + 18 ] I. Higgins, D. Amos, D. Pfau, S. Racaniere, L. Matthey, D. Rezende, and A. Lerchner. 'Towards a definition of disentangled representations'. arXiv preprint arXiv:1812.02230 (2018) (cit. on p. 9).

[ HJA20 ] J. Ho, A. Jain, and P . Abbeel. 'Denoising diffusion probabilistic models'. Advances in Neural Information Processing Systems (2020) (cit. on pp. 4, 13).

[ HD05 ] A. Hyvärinen and P . Dayan. 'Estimation of non-normalized statistical models by score matching.' Journal of Machine Learning Research 4 (2005) (cit. on p. 4).

[ HM16 ] A. Hyvarinen and H. Morioka. 'Unsupervised feature extraction by time-contrastive learning and nonlinear ica'. Advances in neural information processing systems (2016) (cit. on p. 9).

[ HM17 ] A. Hyvarinen and H. Morioka. 'Nonlinear ica of temporally dependent stationary sources'. In: Artificial Intelligence and Statistics . PMLR. 2017 (cit. on p. 9).

[ HST19 ] A. Hyvarinen, H. Sasaki, and R. Turner. 'Nonlinear ica using auxiliary variables and generalized contrastive learning'. In: The 22nd International Conference on Artificial Intelligence and Statistics . PMLR. 2019 (cit. on p. 9).

[ Khe + 20 ] I. Khemakhem, D. Kingma, R. Monti, and A. Hyvarinen. 'Variational autoencoders and nonlinear ica: a unifying framework'. In: International Conference on Artificial Intelligence and Statistics . PMLR. 2020 (cit. on pp. 1, 9).

[ Koj + 22 ] T. Kojima, S. S. Gu, M. Reid, Y. Matsuo, and Y. Iwasawa. Large language models are zero-shot reasoners . 2022. arXiv: 2205.11916 (cit. on p. 1).

[ KJU22 ] M. Kwon, J. Jeong, and Y. Uh. 'Diffusion models already have a semantic latent space'. arXiv preprint arXiv:2210.10960 (2022) (cit. on p. 10).

[ Liu + 21 ] N. Liu, S. Li, Y. Du, J. Tenenbaum, and A. Torralba. 'Learning to compose visual relations'. Advances in Neural Information Processing Systems (2021) (cit. on pp. 7, 10).

[ Liu + 22 ] N. Liu, S. Li, Y . Du, A. Torralba, and J. B. Tenenbaum. Compositional visual generation with composable diffusion models . 2022. arXiv: 2206.01714 (cit. on pp. 10, 16).

[ Luo22 ] C. Luo. Understanding diffusion models: a unified perspective . 2022. arXiv: 2208.11970 (cit. on p. 13).

[ Mik + 13 ] T. Mikolov, I. Sutskever, K. Chen, G. S. Corrado, and J. Dean. 'Distributed representations of words and phrases and their compositionality'. Advances in neural information processing systems (2013) (cit. on pp. 1, 9).

[ MYZ13 ] T. Mikolov, W .-t. Yih, and G. Zweig. 'Linguistic regularities in continuous space word representations'. In: Proceedings of the 2013 conference of the north american chapter of the association for computational linguistics: Human language technologies . 2013 (cit. on p. 9).

[ MFM21 ] G. Mita, M. Filippone, and P . Michiardi. 'An identifiable double vae for disentangled representations'. In: International Conference on Machine Learning . PMLR. 2021 (cit. on p. 1).

[ NBP22 ] N. G. Nair, W . G. C. Bandara, and V . M. Patel. Unite and conquer: cross dataset multimodal synthesis using diffusion models . 2022. arXiv: 2212.00793 (cit. on p. 10).

[ Par + 23 ] Y.-H. Park, M. Kwon, J. Jo, and Y. Uh. 'Unsupervised discovery of semantic latent directions in diffusion models'. arXiv preprint arXiv:2302.12469 (2023) (cit. on p. 10).

[ PSM14 ] J. Pennington, R. Socher, and C. D. Manning. 'Glove: global vectors for word representation'. In: Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP) . 2014 (cit. on p. 9).

[ Rad + 21 ] A. Radford, J. W . Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P . Mishkin, J. Clark, et al. 'Learning transferable visual models from natural language supervision'. In: International Conference on Machine Learning . PMLR. 2021 (cit. on p. 1).

[ Ram + 22 ] A. Ramesh, P . Dhariwal, A. Nichol, C. Chu, and M. Chen. Hierarchical text-conditional image generation with clip latents . 2022. arXiv: 2204.06125 (cit. on p. 4).

[ Sah + 22 ] C. Saharia, W . Chan, S. Saxena, L. Li, J. Whang, E. Denton, S. K. S. Ghase mipour, B. K. Ayan, S. S. Mahdavi, R. G. Lopes, et al. 'Photorealistic text-to-image diffusion models with deep language understanding'. Advances in neural information processing systems (2022) (cit. on p. 4).

[ SE19 ] Y. Song and S. Ermon. 'Generative modeling by estimating gradients of the data distribution'. Advances in Neural Information Processing Systems (2019) (cit. on p. 4).

[ VK + 21 ] J. Von Kügelgen, Y. Sharma, L. Gresele, W . Brendel, B. Schölkopf, M. Besserve, and F . Locatello. 'Self-supervised learning with data augmentations provably isolates content from style'. Advances in neural information processing systems (2021) (cit. on p. 9).

[ ZW20 ] D. Zhou and X.-X. Wei. 'Learning identifiable and interpretable latent models of highdimensional neural activity using pi-vae'. Advances in Neural Information Processing Systems (2020) (cit. on p. 1).

[ Zim + 21 ] R. S. Zimmermann, Y. Sharma, S. Schneider, M. Bethge, and W . Brendel. 'Contrastive learning inverts the data generating process'. In: International Conference on Machine Learning . PMLR. 2021 (cit. on p. 9).

A Concept Algebra Algorithms in Diffusion Model

Text-to-image Diffusion Models use score representations in their generation. More specifically, suppose the target is to sample Y = Y 0 ∼ P ∗ , with the corresponding score function denoted as s 0 . The key ingredients for generation are the score function for Yt (denoted as s t ), which is Y noised at different levels, (e.g. Yt = ( 1 -α t ) Y + α t ε t for standard independent Gaussian noise ε ), for t = 0, ..., T . See [ Luo22 ] for more details. To apply our results, we require causal separability with respect to /Zcal , /Wcal holds for all Yt , t = 0, ..., T . Then our theoretical results follow through.

Algorithm 1 is an implementation of Concept Manipulation through Projection based on DDPM [ HJA20 ] (we can also implement different variants). 3 It requires FindSubspaceMethod, for which we can use FindSubspaceBasis(algorithm 2) and FindSubspaceMask(algorithm 3) based on the properties of /Zcal as discussed in the main text. More specifically,

FindSubspaceBasis We calculate the projection matrix (denoted as Π Z ) for the Z -subspace, from a span of K prompts (after subtracting off the baseline) (algorithm 2). In practical computations, we evaluate the m × K matrix /triangle E :

Then, the top K thres left singular vectors are selected as Q . Here, K thres denotes the least number of factors required to surpass a certain proportion of variance explained, denoted as thres. Consequently, we have Π z ← QQ T .

FindSubspaceMask In this context, Π Z signifies a mask. This mask can be calculated from the score difference /triangle ε (refer to algorithm 3). As a practical measure, we may implement noise reduction techniques to fine-tune /triangle ε . One approach is the application of a Gaussian blur to smooth out neighboring pixels.

Algorithm 1 Concept Manipulation through Projection

10:

Algorithm 2 FindSubspaceBasis

Require: y t ∈ /Rbbb m , prompts { x k } K k = 0 1: ˆ /Rcal Z ( y ) ← span ( { εθ ( y t , t | x k ) -εθ ( y t , t | x 0 ) } K k = 1 ) 2: Determine Π Z as the projection matrix onto ˆ /Rcal Z ( y ) 3: return Π Z

,

Algorithm 3 FindSubspaceMask

/negationslash

B Proofs

Proposition 3.1. If Y is causally separable with respect to /Wcal and /Zcal , then the centered score representation is arithmetically composable with respect to /Wcal and /Zcal .

Proof. By assumption in Theorem 3.2, we have

Therefore,

where p Z [ QZ ]( y ) = ∫ p ( y /Zcal | z ) QZ ( z ) d z and p W [ QW ]( y ) = ∫ p ( y /Wcal | z ) QW ( w ) d w . Then, taking the log-derivative with respect to y , we get its score function as follows:

where s Z ( y ) and sW ( y ) are p Z [ QZ ]( y ) 's and p W [ QW ]( y ) 's score functions, and s 0 ( y ) : = ∇ y log /parenleftbig3 p ( ξ ( y )) /barex /barex /barex det ( ∂ g ∂ y ) /barex /barex /barex /parenrightbig3 . So the centered-score is

where Q 0 Z and Q 0 W are the marginal distributions of Z and W of the baseline Q 0 . Then, we can use the fact that

Consequently, the claim follows.

Proposition 4.1. Let Q W be any fixed distribution over the W concept and Q 0 Z be any reference distribution over Z. Then, assuming causal separability for /Zcal , /Wcal ,

Proof. By causal separability we can easily get the /Rcal Z ( y ) in proposition 4.1 is the same as:

The only thing left to show is that /Rcal Z ( y ) remains the same for whatever choice of baseline Q 0 Z . But this is immediate: span ( { s Z [ QZ ]( y ) -s Z [ Q 0 Z ]( y ) : QZ a concept distribution } ) = span ( { s Z [ QZ ]( y ) -s Z [ Q 1 Z ]( y ) : QZ a concept distribution } ) for any two baselines Q 0 Z and Q 1 Z .

Proposition 5.1. Assuming causal separability holds for /Zcal , /Wcal . If /Zcal is categorical with L possible values (L ≥ 2 ), then dim ( /Rcal Z ( y )) ≤ L -1 .

Proof. We denote the possible values that /Zcal can take as { z 0 , z 1 , . . . , z L -1 } . Let δ z i : = δ z i ( z ) represent the delta function in the /Zcal -subspace, which is infinite at z i and zero at all other points. For any distribution QZ over Z and any y ∈ /Rbbb m , we can express s Z [ QZ ]( y ) as a linear combination of s z [ δ z i ] in the following form:

Here, ∑ L -1 l = 0 π l ( y ) = 1. Consider a baseline concept distribution Q 0 Z and its corresponding Z -related score s Z [ Q 0 Z ]( y ) = ∑ L -1 l = 0 c l ( y ) s Z [ δ z l ]( y ) . We can then express the difference s Z [ QZ ]( y ) -s Z [ Q 0 Z ]( y ) as:

where ω l ( y ) = π l ( y ) -c l ( y ) for l = 1,. . . , L -1. Consequently, we can observe that /Rcal Z ( y ) ⊂ span ( /braceleftbig1 s z [ δ z l ]( y ) -s z [ δ z 0 ]( y ) /bracerightbig1 L -1 l = 1 ) , which implies that dim ( /Rcal Z ( y )) ≤ L -1.

Proposition 5.2. Suppose /Zcal is composed of categorical concepts { /Zcal k } K k = 1 each with the number of categories L k , in the sense that /Zcal = /Zcal 1 × . . . /Zcal k . Assume Y satisfies causal separability with respect to /Zcal , /Wcal , with Y /Zcal the corresponding Y-measurable variable for /Zcal . Further assume that there exists Y /Zcal -measurable variables Y /Zcal k such that p ( y /Zcal | z ) = Π K k = 1 p ( y /Zcal k | z k ) . Then

Proof. By assuming that p ( y /Zcal | z ) = ∏ K k = 1 p ( y /Zcal k | z k ) , we can easily derive the following result for any concept distribution QZ over Z :

where QZk represents the concept distribution of /Zcal k for each k , and s Zk [ QZk ]( y ) = ∇ y log /parenleftbig1∫ p ( y /Zcal k | z k ) QZk ( z k ) d z k /parenrightbig1 . Recall that

where Q 0 Z is a baseline. Importantly, it should be noted that /Rcal Z ( y ) is unique regardless of the choice of Q 0 Z as per proposition 4.1.

Let Q 0 Zk denote the /Zcal k -related part of Q 0 Z for k = 1, . . . , K . We define /Rcal Zk ( y ) : = span ( { s Zk [ QZk ]( y ) -s Zk [ Q 0 Zk ]( y ) } ) . Then, we can state that:

Based on proposition 5.1, it follows that dim ( /Rcal Zk ( y )) ≤ L k -1 for each k . Hence, we can conclude that:

C Experiment Details and More Figures

C.1 Concept projection for Dreambooth (Figure 5)

First, we fine-tune the diffusion model using Dreambooth, applying a learning rate of 5 e -6 and setting the number of steps to 800. While there are configurations that could yield a less overfitted model, we intentionally opt for these parameters to generate an overfitted model. Our aim is to verify if it's possible to disentangle the overfitted model by using concept manipulation via projection.

To generate images depicting a sks toy in front of the Ei ff el Tower , we utilize our Dreambooth fine-tuned diffusion model together with the original pretrained Stable Diffusion model. Only for the new prompt, x new = a sks toy', we use the score function from the Dreambooth fine-tuned model. All other prompts are plugged into the score functions from the original pretrained Stable Diffusion model. To create the desired images, we construct a projector using a pair of prompts: ( x 1 , x 2 ) = ( 'a toy', 'a soccer ball' ) . The mask, computed using algorithm 3 with the threshold = 0.1, helps identify specific areas corresponding to the location of the subject. Then, we use the Dreambooth score function s dreambooth ( a sks toy' ) to guide the generation process within the masked region (areas with value 1), while using s ( a toy in front of the Eiffel Tower' ) to guide the generation outside the mask (areas with value 0).

To ensure image fidelity, we exclusively employ the score function s dreambooth ( 'a sks toy' ) for guiding the denoising process for the last 6% of the denoising steps.

It is important to note that due to severe overfitting issues with the fine-tuned model, there is no significant difference between using either the prompt 'a sks toy' or 'a sks toy in front of the Eiffel Tower' for the fine-tuned model. Also, due to the same reason, we apply the original pretrained diffusion model for all score functions except for the sks toy related one.

C.2 The mathematician example (Figure 1)

Our starting point is an original prompt x orig = 'a portrait of a mathematician'. Our objective is to modify the sex and style using concept projection:

To adjust sex , we formulate a corresponding direction using a pair of prompts ( x 1 , x 2 ) = ( 'a man','a woman' ) . Subsequently, we set x new = 'a person'.

To alter the style , we set x new = 'a portrait of a mathematician, in Fauvism style'. We define the concept subspace using prompts of the form 'a portrait of a mathematician in [ x style ] style', where x style takes value from a list of styles. During sampling, the original prompt is utilized in the first 20% of timesteps to better retain the content.

The list of styles is generated by ChatGPT . They are: Art Deco, Minimalist, Baroque, Abstract Expressionist, Cubist, Fauvism, Impressionist, Steampunk, Neoclassical, Japanese Ukiyo-e, Surrealism, Memphis Design, Scandinavian, Bauhaus, Pop Art, Art Nouveau, Street Art, American West, Victorian Gothic, Futurism, Photorealistic, Mannerist, Flemish, Byzantine, Medieval, Romanesque, Trompe-l'œil, and Dutch Golden Age.

C.3 Stress-test experiments (Figure 3)

In this experiemnt, we deliberately chose a diverse array of artistic styles and contrasting content to challenge our model. The styles used are Baroque painting, traditional Chinese painting, Pop art, Gothic stained glass, the Pre-Raphaelite painting style, Victorian botanical illustration, and Japanese Edo period art. In terms of content, we use: a bustling train station, a nuclear power plant, an artificial intelligence lab, a jazz music concert, a modern dance festival, a modern minimalist house, and a contemporary office setting. The rationale behind this selection was to pair styles and content that would rarely co-occur in training data. This rarity poses a significant challenge for the model in generating realistic outputs, testing its capabilities and adaptability to unconventional combinations.

We implement concept algebra the same way as in Figure 1 example. For concept composition, we use the software in [ Liu + 22 ] .

In fig. 6 we show some examples where concept algebra is not preferred.

Pop art

sicconcert

ept

10

Amodern dancefestival

C.4 The nurse example (Figure 4)

We initiate the process with an original prompt, x orig = 'a portrait of a nurse'. Our goal is to perform concept projection to manipulate the sex attribute. Similar to the mathematician example, we define the sex direction using a pair of prompts: ( x 1 , x 2 ) = ( 'a man','a woman' ) . However, instead of setting the distribution of sex as one of the delta functions or a fair one corresponding to the neutral prompt 'a person', we wish to see what the concept's arithmetic average will define. Specifically, we take the sex direction of the average of a female nurse and a male nurse , calculated as 1 2 s ( a female nurse' )+ 1 2 s ( a male nurse' ) . It turns out the arithmetic mean realize the interpolation between two extremal sex points in the sex subspace, and the score function after concept projection returns images of androgynous nurses.

D Additional experiments

Concept Algebra beats negative prompting in simple tasks Negative prompting, aims to eliminate target concept expressions by subtracting relevant scores. Unlike concept algebra and similar to concept composition, this method does not confine manipulations to specific subspaces. As predicted, we see negative prompting inadvertently modify off-target concepts tied to the primary concept, whereas concept algebra succeeds, as in .

Concept algebra fails when causal separability (Theorem 3.2) is violated We show one concrete example of failures.

Figure 8 shows that we are unable to transfer the gender of the nurse when we calculate the score function of a male nurse by ( /Ibbb -proj Z ) s [ 'a portrait of a nurse' ]+ proj Z s [ 'a man' ] where proj Z is computed by s [ 'a buck on the grass' ] -s [ 'a doe on the grass' ] . The target concept Z and W are sex ∈ { male, female } and species ∈ { human, deer } . It's obvious that the sex and species have an interaction effect on the image Y -different species induce different sexual characteristics.

中

(a)

s

[

'a portrait of a nurse'

]

Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper hypothesizes that the latent representations learned by text-guided diffusion models contain structured subspaces corresponding to semantic concepts. Concepts are formalized as latent variables, and the concepts associated with a (text,image) pair are formalized as a bag of these latent variables, such that the distribution over image outputs is conditionally independent of text inputs given these concepts.\n\nNecessary and sufficient conditions (causal separability; Proposition 3.3) are given for concepts to be arithmetically composable; arithmetic composability in turn admits algebraic manipulation of concepts. A proof-of-concept algorithm is given for identifying subspaces of latent representations corresponding to concepts (based on \"spanning prompts\") and examples are provided of identifying and manipulating these subspaces using the Stable Diffusion model.\n\nSTRENGTHS:\nThe conceptual framework constructed in this paper is though-provoking. The exposition is quite clear, striking a good balance between helpful exposition and mathematical precision. Definition 2.5 (arithmetic composability) is interesting, as is Proposition 3.3 (characterizing arithmetic composability in terms of \"causal separability).\n\nThe algorithms proposed in Section 4 for identifying and manipulating concepts are well-motivated. I also appreciate the discussion in Section 5 of the structure/character of concept subspaces. These analyses could be a good starting point for guiding empirical study of the score representation.\n\nDepending on how the effective the proposed algebraic interventions prove to be in practice (see Weaknesses) I think that the abstractions and definitions introduced in this paper have the potential to stimulate an entirely new direction of work on control and interpretability of generative models. Control and interpretability are broadly relevant to the NeurIPS community.\n\nWEAKNESSES:\nThe experiments in this paper are very rudimentary. There are no quantitative results, and the qualitative results consist of a small number of examples provided in Figures 1-5. It is unclear to me how broadly effective, reliable, or robust this method is. There is just not enough information to evaluate whether these ideas work (or could be made to work) in practice; see the Limitations section of my review for additional thoughts about practicality.\n\nMinor: \"Causal Separability\" is strong language, and I'm not convinced that the word \"causality\" accurately describes the relationship defined by this term. At the very least, some argument is needed to justify calling this a causal relationship.\n\nQUESTIONS:\nManipulating the non-parametric representation space via evaluated values along the diffusion of a score-based model is clever, and my understanding is that this is the key to making concept algebra tractable. Could these methods be applied to, e.g., the Parti model? If not, it might be helpful to point this out and be more explicit about how and where these method depend upon the structure of a score-based model.\n\nThis work seems closely related to interpretability research. The concept editing algorithm could be seen as a sort of causal intervention (in the sense of \"do calculus\") on a hypothetical interpretation of a latent representation. I'm curious how this work connects or complements recent interpretability research, e.g., ongoing work on mechanistic interpretability.\n\nIs the definition of \"causal separability\" given here related/consistent with established definitions of \"causally separable processes\"?\n\nLIMITATIONS:\nIt is unclear to me how restrictive the causal separability condition might be (Definition 3.2). Especially for more abstract, complicated concepts, I could imagine that causal separability might never be satisfied (but I am open to being convinced otherwise). Even for simple concepts, it seems like causal separability can be violated in ways that might not be obvious: for example, the inseparability of (male,female) and (deer,human) surprised me (although it became very clear once the reason for inseparability was explained to me). Understanding the how broadly causal separability holds seems essential to the ultimate practicality of this framework.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-07"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper aims to propose a formalization of concept-based algebra for text-to-image models. \nIt presents equations for how prompts are composed of concepts, which can interact additively in order to generate the corresponding images, similar to word embedding analogies by Mikolov.\nSeveral examples are presented to show this framework in action.\n\nSTRENGTHS:\nThe direction of work representing images as latent concepts and then learning representations for these concepts is interesting and useful.\n\nWEAKNESSES:\nThe main part of this paper is the mathematical equations for conditioning image generation on concepts. \nIt starts by claiming to provide a cognitive framework, such as mathematical equations for how humans map images to high-level concepts, which seems to be complete conjecture. There are no references provided to show any evidence that this is actually how humans analyse images. I very much doubt that a human looking at an image first starts by making a list of all the attributes in that image.\n\nThe equations then gradually morph into how concept representations can be learned from examples. However, it is unclear how this is an improvement or a contribution over the previous work. The last paragraph of the paper correctly points out several other works that also do concept-based representations for image generation. I would expect there to be a comparison and evaluation. \n\nThere is currently very little evaluation of the method, each claim seems to be backed up by only a single example.\n\nAlmost all of the examples use image style as the concept that is being modified. In one case this is referred to as \"medium\", but even that is ultimately just image style. This moves the work to the style transfer area, which isn't really addressed in the paper. It also doesn't give any evidence that this method can be used for anything involving actual concepts related to the content of the image.\n\nPages 7-8 claim a strength of this method is to handle images for which no prompt exists. However, it seems the first example could be prompted with \"a portrait of a male or female mathematician\" and the second one with \"an androgynous nurse\". Neither of these seem examples for which no textual prompt exist; or at least it hasn't been shown in the paper that prompting like this wouldn't work.\nAlso, this section makes a rather strong claim that the model is being debiased in terms of gender by adding the \"person\" vector, which is a claim that would need a lot more proof than examples from a single prompt.\n\nThere do not seem to be any details on how the experiments were conducted - what pre-trained models or training data was used, or how the modifications were performed in the context of that particular model.\n\nThe appendix is repeatedly referenced for important details, proofs and examples, but does not seem to be submitted.\n\nLine 88: A text-controlled generative model should not be producing a random output.\n\nQUESTIONS:\nPlease define the novel contribution of this work in the context of the previous work in the area of concept-based representations for image generation.\n\nLIMITATIONS:\nThere does not seem to be any discussion of limitations or possible societal impact.", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 2, "DATE": "2023-07-09"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe paper focuses on encoding abstract concepts (as prompt) and systematically composing them to generate images. They proposed a mathematical framework to generate images based on specific combinations of prompts. Unlike approaches that change the prompt with different text descriptions, this framework encodes each feature independently. It then performs mathematical operations on different features of the prompt and the original content prompt to generate the desired image. They show several successful examples of the proposed framework and compare its performance with direct prompting approaches. Additionally, they emphasize the necessity of the assumption of causal separability.\n\nSTRENGTHS:\n1. The paper introduces an interpretable approach to encoding high-level concepts with LLM representations. By incorporating specific prompts and applying metathetical operations on these prompts and the original content, the framework enables the generation of images that capture desired attributes or concepts. This interpretability allows for more fine-grained control over the generated content. I personally really like this aspect.\n\n2. The paper showed an interesting analysis of the causal relationship keywords used in the framework. By examining the feature differences between terms such as \"buck\" and \"doe,\" the authors demonstrate the limitations of direct application between terms like \"man\" and \"woman\" due to the inherent species difference.\n\nWEAKNESSES:\n1. It's necessary to have a larger-scale evaluation of the generated images. While the qualitative results presented in the paper showcase specific examples, a quantitative evaluation on a larger dataset or with a larger sample size would enhance the reliability and generalizability of the method.\n2. I suspect the impact of prompt length is a key factor in the performance of the proposed methods. If shorter direct prompts yield comparable results to the proposed method, it may reduce the appeal of the proposed approach. Once again, a larger evaluation scale on different prompt lengths would shed light on this aspect. \n3. In line 286, you mention that \"1/2 male nurse and 1/2 female nurse\" does not correspond to any English prompt. I wonder if paraphrasing it into a more neutral term like \"androgynous person\" and re-prompting the model could potentially yield different direct prompting results.\n\nQUESTIONS:\n1. How does \"sex\" encoded in Figure 1 (b)? what's the result of sex \"man\" or \"women\" in this example?\n\nLIMITATIONS:\nNo, the paper doesn't discuss limitations.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 4, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-13"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis research presents a novel mathematical framework that brings clarity to the abstract notion of concepts, enabling their connection to specific subspaces in a representation space. The significance lies in demonstrating the existence of structured concepts in score representations, specifically emphasizing the compositional nature of these concepts within the representation space, which is useful for further analysis regarding generation.\nOne of the key contributions of this work is the introduction of an effective method to identify the subspace associated with a given concept. This approach allows for the manipulation of concepts expressed by the model through algebraic operations on the representation, thereby providing a powerful tool to work with and understand complex concepts in a more tangible and interpretable manner.\nMoreover, the implications of this research extend beyond its immediate domain, as the proposed framework can be extended to other areas, such as natural language processing (NLP), analysing concepts such as topics and semantic meanings.\n\nSTRENGTHS:\n1. Previously, the notion of concepts is often discussed in an abstract level. This work demonstrated the existence of structured concepts in score representations, specifically emphasizing the compositional nature of these concepts within the representation space, which can be helpful in further analysis on abstract notions.\n2. The work proposed a useful method to identify the subspace associated with a given concept, allowing us to manipulate concepts expressed by a model through algebraic operations on the representation. This method can be a powerful tool in terms of understanding complex concepts.\n3. The experimental results verified the efficacy of this method in case studies. This method can be extended to analysis on other areas like NLP.\n\nWEAKNESSES:\n1. The major concern regarding this work lies in the absence of a precise quantitative evaluation to assess the impact of concept extraction and manipulation, particularly in the experiments. Relying largely on case-level analysis may not be sufficient to draw robust conclusions. Such measurement based on the precise quantitative evaluation can be used to make comparisons between various methods relevant to this topic and for further analysis on subsequent improvements.\nIt will be more convincing if the concept manipulation is evaluated on more cases (or on more datasets) and comparisons are made between the proposed method and relevant methods (e.g., methods to extract and manipulate latent features) used in previous research.  \n2. A minor issue that needs attention is the writing style, particularly the accuracy of internal references. Some of the internal references in the work are not precise and should be revised for clarity and consistency.\n\nQUESTIONS:\nPlease pay attention to the clarity of the descriptions.\n1. Some notations are a little bit confusing. E.g., in Definition 2.2,  does “z_{1:k}” refer to “z_1, z_2, …, z_k” regarding Definition 2.2? What is δmale in Equation 11?\n\n2. “Following theorem 3.3” (Line 172).  “theorem 3.3” is not found. Similar cases can also be found in Line 246 and Line 252.\n\n3. It can be worth discussing the selection of the discretized concepts, e.g., how many ks are sufficient for a latent variable C?\n\nLIMITATIONS:\nThe limiatations have not been well discussed in the paper.\nAs the performance of the proposed method is mainly evaluated on case studies, it is suggested that a more quantitive measurement be introduced to judge the robustness of the method.", "TITLE": "Official Review", "RECOMMENDATION": 6, "REVIEWER_CONFIDENCE": 3, "SOUNDNESS_CORRECTNESS": 3, "DATE": "2023-07-22"}

comments: SUMMARY:
This paper suggests concepts are represented in text-guided generative models as an encoded subspaces of the representation space. It also formalizes the description of the representation of concepts in text-controlled generative model in a mathematical way. The paper also shows that the Stein score of the text-conditional distribution is an arithmetic composable representation of the input text, and develops concept algebra, a method to manipulate the concepts via arithmetic manipulation of the representation.

STRENGTHS:
Very well-written paper! All the math looks great and I am excited about the potential alternative to prompt engineering! Hope there is a demo that I could try.

- Section 2 and 3 rigorously defines the mathematical background used in the concept algebra method, and section 4 and 5 is also very well-written.
- I enjoyed reading the figures mentioned in the paper! All of them are informative, and demonstrate the point of each figure clearly.
- As demonstated in section 6, the proposed method could be a strong alternative to prompt engineering! Being much more stable, less random, and with no need to manipulate models trying to act as a painter or other roles, concept algebra seems very promising!

WEAKNESSES:
Nothing much, but I would love to see some limitations of the method discussed in section 7.

QUESTIONS:
* See above

LIMITATIONS:
* If I understand everything correctly, concept algebra can only work on models have access to their representation, therefore, concept algebra can't really work on closed models, right?
* How will it negatively impact the society if everyone can edit the model output using concept algebra?
TITLE: Official Review
SOUNDNESS_CORRECTNESS: 4
DATE: 2023-07-23