Concept Algebra for (Score-Based) Text-Controlled Generative Models
Authors: Zihao Wang
Lin Gui
Jeffrey Negrea
Victor Veitch
Venue: NeurIPS 2023 poster (2023)
Keywords: disentanglement; representation learning; text-controlled generative models; diffusion models

Abstract: This paper concerns the structure of learned representations in text-guided generative models, focusing on score-based models. A key property of such models is that they can compose disparate concepts in a 'disentangled' manner.This suggests these models have internal representations that encode concepts in a 'disentangled' manner. Here, we focus on the idea that concepts are encoded as subspaces of some representation space.  We formalize what this means, show there's a natural choice for the representation, and develop a simple method for identifying the part of the representation corresponding to a given concept. In particular, this allows us to manipulate the concepts expressed by the model through algebraic manipulation of the representation. We demonstrate the idea with examples using Stable Diffusion.

Concept Algebra for (Score-Based) Text-Controlled Generative Models
Zihao Wang', Lin Gui', Jeffrey Negrea", and Victor Veitch 1,2,3
1 Department of Statistics, University of Chicago 2 Data Science Institute, University of Chicago Google Research
Abstract
This paper concerns the structure of learned representations in text-guided generative models, focusing score-based models. A key property of such models that they can compose disparate concepts in a 'disentangled" manner. This suggests these models have internal representations that encode concepts in a 'disentangled' manner. Here, we focus on the idea that concepts are encoded subspaces of some representation space. We formalize what this means, show there's a natural choice for the representation, and develop a simple method given concept.
for identifying the part of the representation corresponding In particular, this allows us to manipulate the concepts expressed by the model through algebraic manipulation of the representation. We demonstrate the idea with cxamples using Stable Diffusion.
Introduction
Large-scale controled generative models are now dominant in many parts modern machine learning and artificial intelligence [e.g., Bro+20; Rad+21; Bom+21; Koj+22]. In these models, the user provides a prompt in natural language and the model generates samples based on this prompt-e.g- in large language models the sample is natural language response, and in text-to-image models the sample is an image. These models have remarkable ability to compose disparate concepts to generate coherent samples that were seen during training. This suggests that these models have some internal representation of high-level concepts that can be manipulated in 'disentangled' manner. Broadly, the goal of this paper is to shed light on how this concept representation works, and how it can be manipulated. We focus on text-to-image diffusion models, though many of the ideas are generally applicable.
Our starting point is the following commonly observed structure of representations:
1. Each data point x is mapped to some representation vector Rep(x) IR..
2. High-level concepts correspond to subspaces (directions) of the representation space.
Perhaps the best known example of this structure is in word embeddings, where semantic relation- ships such as concepts (here, sex) are encoded as directions in the representation space [Mik+133 13]. This kind of encoding of concepts has been argued to occur in many contexts, including in the latent space of variational autoencoders [ZW20; Khe+20;; MFM21] and in the latent space of language models [Bol+16; GG19; Rad+21; Elh+22]. We'll call representations of this kind arithmetically composable, because composition corresponds to arithmetic operations on the representation vectors. The goal of this paper is to develop arithmetically composable representations of text for score-based text to image models.
ss Rep("man")) Reem""")ReRaaara ("woman") suggest that high-level
There are two main motivations. First, understanding the structure of the representation space is important for foundational progress on understanding the emergent behavior text-controlled generative models. It is particularly interesting study this question in the text-to-image setting
37th Conference on Neural Information Processing Systems (NeurIPS 2023).

---

Rep( portrait mathemariclan")
(b) (2- proj..n Repl" la portrait of mathematician +prry Repl "a person"]
(c) (I-projayu) Rep("a portrrit of mathematician" J-pojeyyi Rep(" "in Fauvism style")
Figure 1: We show that high-level concepts such as and artistic style encoded subspaces of suitably chosen representation space. This allows us manipulate the concepts expressed by prompt through algebraic operations on the representation of that prompt. Namely, we edit representation projected on to the subspace corresponding to concept. Note images are paired by random seed.
because the multitmmdallity of the data makes it straightforward to distinguish concepts from inputs, and because it is not clear priori that the models themselves build in any inductive bias towards arithmetic structure. The secondary motivation is that having such a representation would allow us to manipulate the concepts expressed by the model through linear-algebrate manipulations of representation of the input text; fig. 1 illustrates this idea.
The development of the paper is follows:
1. We develop a mathematical formalism for describing the connection between represen- tation structures and concepts for text-controlled generative models.
2. Using this formalism, we show that the Stein score of the text-conditional distribution arithmetically composable representation of the input text.
3. Then, we develop concept algebra as a method for manipulating the concepts expressed by the model through algebraic manipulation of this representation. We illustrate the approach with examples manipulating a variety of concepts.
A Mathematical Framework for Concepts Subspaces
The first task to develop a precise mathematical formalism for connecting the structure of representations and high-level concepts. This is necessary for understanding when such represen- tations are possible, how construct them, and when they may fail. We must make precise what is a concept, how concepts relate to inputs x, and what it means to represent concepts.
The real-world process that generated the training data has the following structure. First, images Y are generated according to some real-world, physical process. Then, some human looks at each image and writes caption describing it. Inverting this process, each text induces some probability density p(y x) over images Y based on how compatible they are with x caption. The (implicit) goal of the generative model is to learn this distribution.
Concepts
To write the caption, the human first maps the image the image's content, then uses these latent variables to generate the text X. Let C be the latent variable that captures all the information about the image that relevant for human writing caption. So,
set of high-level variables summarizing
p(y 1x= xx C x)dc.
(1)

---

The random variable C captures the information that is jointly relevant for both the image and caption. Variables in C include attributes such has mathematician or is man, but not pixel_ 14 is red. We define concepts in terms of the latent C.
Definition 2.1. A concept variable Z is C-measurable random variable. The concept " associated to Z is the sample space of Z.
Now, the full set of all possible concepts is unwicldy. Generally, we are concerned only with the concepts elicited by particular prompt x.
Definition 2.2. A set of concepts 2,,..., 2x is sufficient for x if p(c |X = xx = =z1,) = p(c 1i: 11:x for all 21...., 2 E 2 xxxx
For example, the concept profession would sufficient for the prompt "A nurse". This prompt induces distribution many concepts (e-g., background is likely to be hospital) but these other concepts are independent of the caption given profession = nurse. Then,
ply xx)== Σ ply #1=)p(211 x).
(2)
Fick
Concept Distributions Following eq. (2), we can view each text as specifying a distribution
p(Z x) over latent concepts 2,, This observation lets us make the relationship between text and concepts precise.
Definition 2.3. A concept distribution Q is a distribution over concepts. Each text x specifies a concept distribution Q2 p x).
That is, we move from viewing text as expressing specific concept values (is m___tthhhmattiiiannn 1) to expressing probability distributions over concepts (Q, mathematician 1) 0.99). The probabilistic view is more determal-ddermmiiitticaalll expressed concepts can be represented as degenerate distributions. This extra generality is necessary: for example, the prompt "a person" induces a non-degenerate distribution over the sex concept.
Concept Representations A text-controlled generative model takes in prompt text x and produces a random output Y. Implicitly, such models are maps from text strings to the space of probability densities over Y. We'll define a representation Rep(x) E A of as any function of x that suffices to specify the output clistribution. We define f.(-) as the density defined by E R, and assume that the model learn's the true data distribution of Y y|X- = x:
Treptx = ppy xx)
(3)
The key idea for connecting representations and concepts is to move from considering represen- tations of prompts to representations of concept distributions.
Definition 2.4. concept representation Rep is a function that maps a concept distribution Q to representation Rep(Q) M where A is vector space. The representation of prompt is the representation of the associated concept distribution, Rep(x) == Rep(Ox).
There are two reasons why this view is desirable. First, defining the representation in terms of the concept distribution makes the role of concepts explicit-this will allow us to explain how representation structure relates to concept structure.
The second reason is that it allows us reason about representations that do not correspond to any prompt. Every prompt defines a concept distribution, but not every concept distribution can be defined by a prompt. This matters because we ultimately want to reason about the conceptual meaning of representation vectors created by algebraic operations on representations of prompts. Such vectors need not correspond to any prompt.
Arithmetic Compositionality We now have the tools to define what it means for representa- tion to be arithmetically composable. We define composability for a pair of concepts and w. In the subsequent development, aim will be to manipulate 2 while leaving w fixed.
Definition 2.5. representation Rep arithmetically composable with respect to concepts 2, W if there are vector spaces R, and Ry such that for all concept distributions of the form Q w) = Q=(2
Rep(QzQw) = Repz
(4)
3

---

where Repz(Qz) 2g and (Qw) Rw
In words: we restrict to product distributions to capture the requirement that the concepts 2 and W can be manipulated freely of each other (the typical case is that one or both of Q2 and Qw are degenerate, putting all their mass on single point). Then, the definition requires that there are fixed subspaces corresponding to each concept in the sense that, e.g., changing only Qz induces change only 2.
3 The Score Representation
We now have abstract definition of arithmetically composable representation. The next step is to find a specific representation function that satisfies the definition.
We will study the following choice.
Definition 3.1. The score representation s{Q] of concept distribution Q is defined by:
s[Q]Gr) = = v, log p(y w)Q(z, w)dzdw.
The centered score representation s[Q] is defined by s[Q] = ==[[]]<[[0]
Here, s[Q] is itself a function of and the representation space A is a vector space of functions.
This departure from the typical view of representations as elements of Rº, The score representation can be thought of as kind of non non-paramttric representation vector. The centered score representation just subtracts off the representation of some baseline distribution Qo
The main motivation for studying the score representation is that
V, logp(y [x).
The importance of this observation is that Vy log p(y x) is Icarnable from data. In fact, this score function is ultimately the basis of many controlled generation models [e.g.. HJA20; Ram+22; Sah+22], because it characterizes the conditional while avoiding the need to compute the nor- malizing constant [HD05; SE19]. Accordingly, of prompts in many generative models, without any extra model training.
readily compute the score representation
Causal Separability The score representation does not have arithmetically composable structure with respect to every pair of concepts. The crux of the issue is that concepts are reflected in the representation based on their effect on Y. If the way they affect Y depends fundamentally on some interaction between Thus, we must rule out this case.
concepts, the representation cannot hope to disentangle them.
Definition 3.2. We say that Y is causally separable with respect to H, measuuabbl variables Yz, Yw, and E such that
W if there exist unique
1. Y &Ye, Yu E) for some invertible and diffcrentiable function g, and 2. p(yz. z, w) p =>)(y, w)
Informally, the requirement is that we can separately generate Y& and Y as the part of the output affected by 2 and / (and? as the part of the image unrelated Z and W), then combine these parts to form the final image. That is, generating the visual features associated to a concept w can't require us know the value of another concept Y. As an example where causal separability fails, consider the concepts of species W = (deer, human) and sex 2 = (male, female). It seems reasonable that there is Y-measurable Yw that is the species part of the image-c-g., the presence of fur vs skin, snouts vs noses, and so forth. However, there is no part of Y that manner that's free of species. The reason that the visual characteristics of sex are fundamentally different across species-e.g, e.g., male deer have antlers, but humans usually do not. fig. we test this example, finding that concept algebra fails in the absence of causal separability.
corresponds sex concept
turns out suffices to rule out this case (all proofs appendix):
'The representation space & is the same for all Qo: the choice is arbitrary. We define ă to ensure 0 is an element of 인. This is for theoretical convenience; will see that only required in practice.
4

---

Proposition 3.1. If Y is causally separable with respect to W and 2, then the centered score representation is arithmetically composable with respect w and 2.
That is: the (centered) score representation structured such that concepts correspond to subspaces of the representation space.
Concept Algebra
We have established that concepts correspond subspaces of the representation space. We now consider how to manipulate concepts through algebraic operations on representations.
To modify a particular concept 2 we want modify the representation only on the subspace R corresponding to 2. For example, consider changing the style concept to Fauvism. Intuitively, we want an operation of the form:
(5) where projayte is the projection onto the subspace corresponding to the style concept. The idea is that the representation of the original prompt Xorig is unchanged except on the style subspace. On the style subspace, the representation takes on the value elicited by the new prompt Xnew "Fauvism style".
Sedit - (T-pproayyass(aa portrait of mathematician") "
"Fauvism style"],
There are two main challenges for putting this intuition into practice. First, because we are working with an infinite dimensional representation, it unclear how to do the projection. Second, although we know that some 22 exists, we still need way to determine explicitly.
4.1 Concept Manipulation through Projection Following proposition 3.1, we have that
=
(6)
for some representation functions 8y and 5w with range in Rg and Bw respectively. We have that the Z-representation space is
L span( (3 [Q2]: Qz concept distribution)).
Our goal is to find projection onto Ry-
The first obstacle is that 2 is function space, making algebraic operations difficult to define. The resolution is straightforward. In practice, score-based models generate samples by running discretized (stochastic) differential equation forward in time. These algorithms only require the score function evaluated at the finite set of points. At each y, we have that 5(y) R Accordingly, by restricting attention to single value of y at time, can use ordinary lincar algebra to define the manipulations:
Definition 4.1. The Z subspace at y is
Ry(y) = span((sx(Q211r) Qz a concept distribution))
(8)
and the Z-projection y, denoted proiz (y) is the projection onto this subspace.
we can compute projz(y) then we can just edit the representation at each point y. That is, we transform the score function at each point:
(y) < (I- (
(9)
We then draw samples from the stochastic differential equation defined by S
4.2
Identifying the Concept Subspace
The remaining obstacle is that we need to explicitly identify Rz(y), so that we can compute (y)) The problem is that the function 5z in eq. (6) is unknown, so we cannot compute Ry(y) directly. Our strategy for estimating the space is based on the following proposition.
Proposition 4.1. Let Qw be any fixed distribution over the W concept and Q² be any reference distribution over Z. Then, assuming causal separability for 2, W,
g Zy(y) (pon((ss002Q0 1(y) Qz concept distribution)).
(10)

---

The importance of this expression is that it does not require the unknown sz.
We'll use eq. (10) to identify 8 ( The idea to find a basis for the subspace using prompts ......Xx Xx that elicit distributions of the form Q2 = Qw. For example, to identify the sex concept we use the prompts Xo "a man" and X1 " woman", with the idea that
Qx. 6 male XQw
Qx
6 female xQw>
(11)
with the same marginal distribution Qw- We then the prompts define the estimated subspace as
, = (ponppp((stx1)))ss>>00))))) 1I..,)))
(12)
That is, we write k+1 prompts Xa.... X2 designed so that each elicits a different distribution over Z, but the same distribution W. Then, the estimated subspace is given by eq. (12).
4.3 Concept Algebra
Summarizing, our approach to algebraically manipulating concepts is:
1. Find prompts Xo such that each elicits a different distribution on Z, but the same distribution on W. That is, Qx = 2 for each j.
2. Construct the estimated representation space Ly(y) following eq. (12), and define projz(y) as the projection onto this space.
3. Sample from the discrctized SDE defined by the manipulated score representation"
Sedirly) ())s (y))
(13)
Implementation of eq. (13) with the diffusion model is described in appendix A.
Validity of Concept Subspace Identification
The procedure described in the previous section relies on finding spanning prompts x,, for the target concept subspace. These prompts must satisfy Qx, = Q Qw for some common Qw and we must have sufficient prompts to span the subspace. The first condition is question of prompt design, and is often not too hard in practice. However, it is natural to wonder when it's possible to actually recover Rg using only a practical number of prompts. We give some results showing that the dimension of Z (y) is often small, and thus can be spanned with a small number of prompts. Note that these results rely on the special structure of the score representation, and may not hold for other representations.
First, the case where 2f is categorical with few categories:
Proposition 5.1. Assuming causal separability holds for 2,W. W. If 2 is categorical with possible values (I. 2 2), then Zim(R,z)) < 1-11.
This result covers concepts such as sex, which can be spanned with only two prompts.
The next result extends this to certain categorical concepts with large cardinality, such as style. The idea is that if concept is composed of finer grained categorical concepts, cach with small cardinality, then the representation space of the concept is also low-dimensional. For example, style may be composed of lower-level concepts such as color, stroke, textures, etc.
Proposition 5.2. Suppose 2 is composed of categorical concepts ( ( each with the number of categories Lt in the sense that 2 = x......y Assume Y satisfies causal separability with respect to 2, W, with Yg the corresponding Y-measurable variable for 2. Further assume that there exists Y Yy-measurable variables Ys, such that p(yx x) nK_ P(Y,,, 1xx). Then
1K
dim timl@ R2(()) < (Lx-1)
(14)
We can view this as first editing the centered representation 3: (yd - (x) (( Then add the same baseline on both sides.

---

(a) dog cat
(b) beach forest
formal clothes casual clothes
old
(e) smiling vs. gloomy
n sunny vs. rainy
Figure 2: Binary concepts (e.g. dog-vs-cat, soaling-vs-gloonae correspond to subspaces, and can be easily manipulated with concept algebra.
Following this result, we might take the spanning prompts for style to be Xg "a mathematician in Art Deco style", X1 "a mathematician in Impressionist style", etc. Each of these prompts elicit fixed distribution Qw over the content, but varies the distribution Q2 over style. If style composed of mnainer attributes, relatively small set of such of prompts will suffice.
6 Experiments
We have formalized what it means for concepts to correspond to subspaces of the representation space, and derived a procedure for identifying and editing the subspaces corresponding to particular concepts in the score representation. We now work through some examples testing this subspace structure does indeed exist, and whether it enables manipulation of concepts.
Many concepts are represented as subspaces First, we check whether the subspace structure does indeed exist. To this end, we generate randomly selected prompts-e-g. "a black dog sitting on the beach"-anr attempt to change binary concepts expressed in the prompt. For example, we change the subject to be a cat by manipulating the concept dog-vs-ca VB with concept algebra. This, and other examples, are shown in fig. 2. It is clear that we are able to manipulate the target provept-providing evidence that these concepts are represented subspaces.
Concept Algebra can disentangle hard-ao--eearrte concepts We stress-test concept algebra by using it to sample images expressing combinations of concepts that occur rarely in the training data. Specifically, we look at unlikely subject/ styyle combinations-e.. Baroque painting". We accomplish this by taking a base prompt that generates high-quality image, but with the wrong style (e.g., "A nuclear power plant"). Then we use concept algebra to edit the style. We compare this with directly prompting the model, and with concept composition [e.g., Du+21; Liu+211]. The later is method that adds on score representation of the style to the base prompt (without projecting on to a target subspace).
"A nuclear power plant
We used each of the three methods to sample images expressing to 49 pairs. Human raters were then presented with the outcomes alongside reference images, and asked to rank them based on adherence to the desired style and content. This evaluation was replicated across different raters. Refer to fig. for illustrative examples. Raters consistently favored images produced by concept algebra, as highlighted in table 1. This aligns with our theory, suggesting concept algebra's adeptness in retaining content while altering style. See appendix C for further details.
content sstyle
7

---

partiry
Fidem mocielant Toate
Figure 3: Concept algebra considered more faithful than other methods. Raters are shown rows of images and rank method outputs by how well they produce target style while preserving content. Each row has style-reference (leftmost), content-reference (2nd from left), and left-to-right (here, but randomly permuted in the survey) images generated from concept-algebra, composition and direct prompting. Quantative results table
Direct Prompting
Concept Composition
Concept Algebra
All Bad
Average Proportion 0.162
0.164
0.476
0.198
Standard Error
0.017
0.017
0.023
0.018
Table 1: Raters find concept algebra more faithful to content and style than direct prompting or concept composition
Mixture concept distributions Next, our theory predicts that we can use concept algebra to sample from mixture (non-dcgenerate) distributions over concept values. Consider sex. Figure la shows that x = "a portrait of a mathematician" almost always generates pictures of men. In fig. 1b sample from the distribution induced by
Sedit ( S
(15)
and observe that indeed get mixture distribution (induced by "person") sex.
Non-prompted edits to the subspace edit the concept Concept algebra uses reference prompts (e.g., "woman" or "person") to set the target concept distribution. It's natural to ask what happens if we make edit concept subspace that does not correspond to reference prompt. In fig. 4, we sample from
1 Sedit (I-projaez )s[x]>]rxxxxox)o s( "a male nurse"]+ ss"s female nurse"]).
1
(16)

---

The representation vector =s["a male nurse" 3s female nurse"] need not correspond to any English prompt. We observe that modifications on the subspace still affect just the sex concept though-tte samples are androgynous figures!
Mask as Concept Finally, we consider more abstract kind of concept motivated by the following problem. Suppose we have several photographs of a particular toy, and we want to generate image of this toy in front of the Eiffel Tower. In principle, fine-tuning the model (e.g., with dreambooth) to associate a new token (e.g., "sks toy") with the toy. Then, the Eiffel Tower". In practice, however, this can be difficult because the fine-tuning ends up conflating the toy with the background in the demonstration images. E.g., the prompt "a sks toy in front of the Eiffel Tower" tends to generate images featuring carpet; see fig. 5b.
do this by
generate the image by conditioning on the prompt "a sks toy in front of
Intuitively, we might hope fix this problem by finding concept subspace that excludes background information. Given such a "subject subspace", we could mask the subject out of the image, generate the background, and then edit the subject back in. appendix C. we explain how to construct such a subspace using the prompts Xo "a toy" and X = "a soccer ball". Figure 5 shows the sampled output.
(a) a portralt nurse")
(I-proj portrait of nurse"] +pro ( female nurse']+ js["a male nurse"])
Figure 4: Elements of the X may not correspond any prompt.
7 Discussion and Related Work
We introduced framework illustrating that concepts align with subspaces of a representation space. Through this, we validated the structure of the score representation and derived a method to identify the subspace for concept. We then demonstrated concept manipulation in diffusion model's score representation.
Concepts as Subspaces There has been significant interest in whether and how neural represen- tations encode high-level concepts. substantial body of work suggests that concepts correspond to subspaces representation space [e.g., Mik+13; MYZ13; PSM14; GL14; Aro+15; GAM17;
AH19]. Often, this work focuses on specific representation learning approach and is either empirical or offers domain-tied theoretical analysis. For instance, in word embeddings, theories explaining observed structures depend the unique nature of language [e.g., Aro+ AH19]. In contrast, our paper's mathematical development is broad-we merely stipulate that the data have two views separated by semantically meaningful space. We argue that the concepts-ar subspaces structure stems from the structure of probability theory, independent of any specific architecture or algorithm.
Our work also relates to studies that assume training data arises from specific latent variable model and demonstrate that learned representations (partially) uncover these latent variables [e.g., HM16; HM17; HST19; Khe+200; VK+21; Eas+22; Hig+18; Zim+21J. This literature often aims for "disentangled" representations where each latent space dimension matches single latent factor. Unlike these, we don't presuppose finite set of latent factors driving the data.

---

Instead, our representations define probability distributions over latent concepts, not merely recovering them. This non-determinism, as observed, is generally necessary.
Controlling Diffusion Models To demonstrate the concept-as-subbpaet structure, developed a method for identifying the subspace corresponding to a given concept and showed how to manipulate concepts in the score representation of diffusion model. We emphasize that our contribution here is not the manipulation procedure itself, but rather the mathematical framework that makes this procedure possible. In particular, the requirement manipulate entire score functions somewhat burdensome computationally. However, the ability to precisely manipulate individual concepts is clearly useful tool, and it is an intriguing direction for future work to develop more efficient procedures for doing so, We conclude by surveying connections to existing work on controlling diffusion models.
One idea has been to take the bottleneck layer of UNet as a representation space and control the model by manipulating this space [KJU22; Haa+23; Par+23]. This work does not consider text controlled models. It would be intriguing to understand the connection to the score-representation view, moving from manipulation of the score manipulation of the bottleneck layer would be a large computational saving.
(n) a toy front of the Eiffel Tower")
(b) Fdreanbooll la sks toy in front of the Eiffel Tower")
(c) (1- Pi--prooessse 1s("a toy in front of the Eiffel Tower") Tower"]
+proj adeeantool ("a toy in front
Figure 5: We can manipulate abstract concepts such as 'subject' of the image
Concept algebra can be seen providing unifying mathematical view several methods that manipulate the score function [e.g., Du+21; Liu+21; NBP22; Ano23]. Du et al. [DLM20] and Liu et al. [Liu+22] manipulate concepts via adding and subtracting scores. Negative prompting a uidely-used engineering trick that 'subtracts off' a prompt expressing unwanted concepts. In section 6 and appendix D compared against these heuristics and show that concept algebra more effective at manipulating concepts in isolation. Couairon et al. [Cou+22] use score differences to identify objects' locations in images; this inspired approach in section and appendix C.1. In each case, we have seen that this kind of manipulation may be viewed as editing the subspace corresponding some concept.
10

---

Acknowledgements
This work is supported by ONR grant 100014-23-1-259 and Open Philanthropy.
References
[AH19]
C. Allen and T. Hospedales. "Analogies explained: towards understanding word embeddings". In: International Conference on Machine Learning. PMLR. 2019 (cit. on p. 9).
[Ano23]
Anonymous. "Reduce, reuse, recycle: compositional generation with energy- based diffusion models and MCMC". In: Submitted to The Eleventh International Conference Learning Representations. under review. 2023 (cit. on p. 10).
[Aro+15]
S. Arora, Y. Li, Y. Liang, Ma, and A. Risteski. A latent variable model approach to pmi-based word embeddings. 2015. arXiv: 1502.03520 03520 (cit. on p. 9).
[Bol+16] T. Bolukbasi, K.-W. Chang, J. Y. Zou, V. Saligrama, and A. T. Kalai. "Man is to computer programmer woman to homemaker? debiasing word embeddings". Advances in neural information processing systems (2016) (cit. on p. 1).
[Bom+21] R. Bommasani, D. A. Hudson, E. Adeli, R. Altman, S. Arora, S. von Arx, M. S. Bernstein, J. Bohg, A. Bosselut, E. Brunskill, et al. On the opportumities and risks of foundation models. 2021. arXiv: 2108.0722
(cit. p. 1).
[Bro++00]
T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neclakantan, P. Shyam, G. Sastry, A. Askell, et al. "Language models are few-shot learners". Advances in neural information processing systems (2020) (cit. on p. 1).
[Cou+22]
Couairon, Verbeek, H. Schwenk, and M. Cord. "Diffedit: diffusion based se- mantic image editing with mask guidance". arXiv preprint arXiv:2210. 11122111.11217 (2022) (cit. on p. 10).
[DLM20]
Y. Du, S. Li, and I. Mordatch. "Compositional visual generation with energy based models". Advances in Neural Information Processing Systems (2020) (cit. on p. 10).
[Du+21]
Y. Du, S. Li, Y. Sharma, J. 'eeeebbuu, and I. Mordatch. "Unsupervised learning of compositional energy concepts". Advances in Neural Information Processing Systems
(2021) (cit. on pp. 7, 10).
[Eas+22]
C. Eastwood, A. L. Nicolicioiu, J. von Kügelgen, A. Kekić, E Träublc, A. Dittadi, and B. Schölkopf. "Dci-es: an extended disentanglement framework with connections to identifiability". arXiv preprint arXiv:2210. 00364 (2022) (cit. on p. 9).
[Elh+22]
N. Elhage, T. Hume, C. Olsson, N. Schiefer, T. Henighan, S. Kravec, Z. Hatfield- -oddd, R. Lasenby, D. Drain, C. Chen, et al. "Toy models of superposition". arXiv preprint
arXiv:2209. 10652 (2022) (cit. p. 1).
[GAM17] A. Gittens, D. Achlioptas, and M. W. Mahoney. "Skip-gram- zipf+ uniform= vec- tor additivity". In: Proceedings of the 55th Annual Meeting of the Association for Computationnl Linguistics (Volume 1: Long Papers). 2017 (cit. on p. 9).
[GL14]
Y. Goldberg and O. Levy. Word2vec explained: deriving mikolov et al.'s negative- sampling word-embedding method. 2014. arXiv: 1422.3722 (cit. on p. 9).
[GG19]
H. Gonen and Y. Goldberg. Lipstick a pig: debiasing methods cover up systematic gender biases in word embeddings but do not remove them. 2019. arXiv: 1903.03862
(cit. on P. 1).
[Haa+23]]
R. Haas, I. Huberman-SSieee Spiegelglas, R. Mulayoff, and T. Michaeli. "Discovering inter- pretable directions in the semantic latent space of diffusion models". arXiv preprint arXiv:2303. 11073 (2023) (cit. on p. 10).
[Hig+18] I. Higgins D. Amos, D. Pfau, S. Racaniere, L. Matthey, D. Rezende, and A. Lerchner. "Towards definition of disentangled representations", arXiv preprint arXiv:1812.00 021.02230 (2018) (cit. on p. 9).
[HJA20]
J. Ho, A. Jain, and P. Abbeel. "Denoising diffusion probabilistic models". Advances in Neural Information Processing Systems (2020) (cit. on pp. 4, 13).
[HD05]
A. Hyvärinen and P. Dayan. "Estimation of non-normalized statistical models by score matching." " Journal of Machine Learning Research 4 (2005) (cit. on p. 4).
[HM16]
A. Hyvarinen and H. Morioka. "Unsupervised feature extraction by time-contrastive learning and nonlinear ica". Advances in neural information processing systems (2016)
(cit. on p. 9).
[HM17]
A. Hyvarinen and H. Morioka. "Nonlinear ica of temporally dependent stationary sources". In: Artificial Intelligence and Statistics. PMLR. 2017 (cit. on p. 9).
11

---

[HST19]
A. Hyvarinen, H. Sasaki, and R. Turner. "Nonlinear ica using auxiliary variables and generalized contrastive learning". In: The 22nd International Conference on Artificial Intelligence and Statistics. PMLR. 2019 (cit. on p. 9).
[Khe+20] I. Khemakhem, D. Kingma, R. Monti, and A. Hyvarinen. "Variational autoencoders and nonlinear ica: a unifying framework". In: International Conference on Artificial Intelligence and Statistics. PMLR. 2020 (cit. on pp. 1, 9).
[Koj+22] T. Kojima, S. S. Gu, M. Reid, Y. Matsuo, and Y. Iwasawa. Large language models are sero-shot reasoners. 2022. arXiv: 2205. 11916 (cit. on p. 1).
[KJU22]
M. Kwon, J. Jeong, and Y. Uh. "Diffusion models already have semantic latent space". arXiv preprint arXiv:2210.1090 N. Liu, S. Li, Y. Du, J. Tenenbaum, and A. Torralba. "Learning to compose visual relations". Advances in Neural Information Processing Systems (2021) (cit. on pp- 7,
(2022) (cit. on p. 10).
[Liu+21]
10).
[Liu+22]
N. Liu, S. LI, Y. Du, A. Torralba, and J. B. Tenenbaum. Compositional visual generation with composable diffusion models. 2022. arXív: 01714 (cit. on pp. 10, 16).
[Luo22]
C. Luo. Understanding diffusion models: a unified perspective. 2022. arXiv: 220.
(cit. on p. 13).
[Mik+13]
T. Mikolov, I. Sutskever, K. Chen, G. S. Corrado, and J. Dean. "Distributed repre- sentations of words and phrases and their compositionality" Advances in neural information processing systems (2013) (cit. on pp. 1, 9).
[MYZ13] T. Mikolov, W.-t. Yih, and G. Zweig. "Linguistic regularities in continuous space word representations". In: Proceedings of the 2013 conference of the north american chapter of the association for computational linguistics: Human language technologies. 2013 (cit. on 9).
[MFM21] G. Mita, M. Filippone, and Michiardi. "An identifiable double vae for disentangled representations" In: International Conference on Machine Learning. PMLR. 2021
(cit. on p. 1). [NBP22] N. G. Nair, W. G. C. Bandara, and V. M. Patel. Unite and conquer: cross dataset multimodal synthesis using diffusion models. 2022. arXiv: 2212.00793 (cit. on p. 10). Y.HH. Park, M. Kwon, J. Jo, and Y. Uh. "Unsupervised discovery of semantic latent directions in diffusion models". arXiv preprint arXiv:2302. 12469 (2023) (cit. on p. 10).
[Par+23]
[PSM14]
J. Pennington, R. Socher, and C. D. Manning. "Glove: global vectors for word representation". In: Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP). 2014 (cit. on p. 9).
[Rad+21] A. Radford, J. W. Kim, C. Hallacy, Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark, et al. "Learning transferable visual models from natural language supervision". In: International Conference on Machine Learning. PMLR.
2021 (cit. on 1).
[Ram+22] A. Ramesh, Dhariwal, A. Nichol, C. Chu, and M. Chen. Hierarchical text-conditional image generation with clip latents. 2022. arXiv: 2204.02022222 (cit. on p. 4).
[Sah+22] C. Saharia, W. Chan, S. Saxena, L. Li, J. Whang, E. Denton, S. K. S. Ghase mipour, B. Ayan, S. S. Mahdavi, R. G. Lopes, et al. "Photorealistic text-to-image diffusion models with deep language understanding". Advances in neural information processing systems (2022) (cit. on p. 4).
[SE19]
Y. Song and S. Ermon. "Generative modeling by estimating gradients of the data distribution". Advances in Neural Information Processing Systems (2019) (cit. on
p. 4).
[VK+21]
J. Von Kügelgen, Y. Sharma, L. Gresele, W. Brendel, B. Schölkopf, M. Besserve, and
F. Locatello. -Sperviuper learning with data augmentations provably isolates content from style". Advances in newral information processing systems (2021) (cit. on
p. 9).
[zW20]
D. Zhou and X.-X. Wei. "Learning identifiable and interpretable latent models of high- dimensional neural activity using pi-vae". Advances in Neural Information Processing Systems (2020) (cit. on p. 1).
[Zim+21]
R. S. Zimmermann, Y. Sharma, S. Schneider, M. Bethge, and W. Brendel. "Contrastive learning inverts the data generating process". In: International Conference on Machine Learning. PMLR. 2021 (cit. on p. 9).
12

---

A Concept Algebra Algorithms in Diffusion Model
Text-to-image Diffusion Models use score representations in their generation. More specifically, suppose the target is sample y Y,- Pr with the corresponding score function denoted as 5o- The key ingredients for generation are the score function for Y, (denoted as s2), which noised different levels, (c.g. Y, (1-a, + for standard independent Gaussian noise E), for t = O, T. See [Luo22] for more details. To apply our results, we require causal separability with respect to 2, w holds for all Y, [ O,... T. Then theoretical results follow through.
Algorithm is an implementation of Concept Manipulation through Projection based on DDPM [HJA20] (we can also implement different variants). It requires FundSeuspaceeethhod, for which use (aaauussppccBBssssasgiriiii on the properties of Ø as discussed in the main text. More specifically,
2) and FindSubapaceMMss((gggrnnn 3) based
FindSubspaccBBssis We calculate the projection matrix (denoted as I1z) for the Z- Z-subspace, from span of K prompts (after subtracting off the baseline) (algorithm 2). In practical compu- tations, we evaluate the mxK matrix AE:
AE (Y,, x1) ea (r,,, t .o),
xx) E xo)]
Then, the top Khrre left singular vectors are selected as Q. Here, Kehres denotes the least number of factors required to surpass a certain proportion of variance explained, denoted as thres. Consequently, we have II, + QQ".
FindSubspaceMaskk In this context, IT signifies a mask. This mask can be calculated from the score difference Ae (refer to algorithm 3). As a practical measure, we may implement noise reduction techniques to fine-tune Ae. One approach is the application of a Gaussian blur to smooth out neighboring pixels.
Algorithm Concept Manipulation through Projection
1: Require Diffusion model e,(),, t[x), guidance scale w, covariance matrix I²I,
empty prompt 4" prompts: Xorig xnew>
prompts to build the Z subspace: (xi),
the function for finding the Z subspace: FindSubspaceMethoo(::)
2: Initialize sample Yr 10),
3: for = T,...,1 do
4: Cempty - E """ # unconditional
5: Earig Eee + E E x # conditional scores
6: I2 -FnndSubppceMettood (xx3) )))))))))))))))))))))))))))))) # find the projection matrix
7: Ecand t (I-IIt) ong IIy concept projection
8: e et --Eot(CCondee
apply classifier-free guidance
9: "MI~" MM (y.
10: end for
Algorithm FindSubspaceBasis
Require y, E R' prompts {xx)
1: L2 - span(1e | xx,t xo)]
2: Determine II as the projection matrix onto 8x(y)
3: return I1z
"Note there here we use residual eg t x) instead of the score sa xx) for generation, they are equivalent up to time-varying constant.
13

---

Algorithm FindSubspaceMask
Require: E Rm pair of prompts (x)
1: Ae - Eg (Y,, [ x,)
2: for = 1 to do
3: m + e # 0
4: end for
5: I1 + d my...., mm)
6: return Il
B Proofs
Proposition 3.1. Y is causally separable with respect to W and 2, then the centered score representation is arithmetically composable with respect w and 2.
Proof. By assumption Theorem 3.2, we have
p x, 12,w)= p(v,, E(r) w) det ( 2g yy
2g
=p(x 3z) Iw)p(((r) det
Therefore,
plQlky)=ple=xxww0
=p:l@=l0v/pQewwwcvvpp det( ag yy
where Pz(O2](r)= p(yz Then, taking the -eeivvvvvee with respect y, get its score function follows:
p(y= (=)d and Pw [ww I(y) p x3) (w)dw.
s[QzxQw ](y) (0y))=s([011110> +s I() I(y)+s
(17)
where sy(y) and sw(y) (y) are Pz (Qyr('ss and Pw [Qw J'y)'s score functions, and sn(y)
:=
V, log p((())) det( (g) )). So the centered-score is
xQQ
(()) I(y))
(18)
where Q2 and o are the marginal distributions of Z and W of the baseline Qo- Then, we can use the fact that
R z span((5_(02]-5:(0@}} Qz concept distribution))
span((ss[Q:]]>>((0]] Q₂ concept distribution))
Rw 1pmn Qw concept distribution))
span(ss
[Q [Qw] Qw a concept distribution))
Consequently, the claim follows.
Proposition 4.1. Let Qw be any fixed distribution over the W concept and Q₂ be any reference distribution over Z. Then, assuming causal separability for 2, W
2 (y) = spcmm((tQ0.Qn111>>)ccccc....
Q2 a concept distribution)..
(10)
Proof. By causal separability we can casily get the Ry in proposition 4.1 is the same as: Qz a concept distribution)
Ry(y) = span(ss((Q21l))>=s(001111
The only thing left to show is that Ry(y) remains the same for whatever choice of base- line Q9. But this is immediate:
I(y) Qz concept distribution)) =
(pan((ssQQ2l1)>>{{10000 Q2 concept distribution)) for any two baselines Q2 and Q E
14

---

Proposition 5.1. Assuming causal separability holds for 2, W. If 2' is categorical with possible values (L 2), then 2 (y)) 2 L-1.
Proof. We denote the possible values that can take as (2g, 2,,...,,L-1k Let 8 := (z) represent the delta function in the 2 susbspace, which is infinite at 2, and zero at all other points. For any distribution Q2 over Z and any y IR"", we can express sy[Oz](y) as a linear combination of s,[6,] in the following form:
sx(O2](y) m((7)521,,,1()))
Here, Σl=o ml(y) 1. Consider baseline concept distribution Q and corresponding Z- related score Ixr Ei (01)),[8,,11 Ic)). We can then express the difference sy[0;]0r>~
&x(2210)) as:
<210216))-<<QQ010>>> oiyy (,
where =;(y)=
= m.)) for 10,,= Consequently, can observe that M2( (pan((s_16,,10)))3 )()) which implies that dim (z)()) 1.11
1K each with the number of categories Ly, in the sense that 24- = 2, x,. Assume Y satisfies causal separability with respect to 2, W with Yg the corresponding Y-measurable variable for 2. Further assume that there exists Yy measurabll variables Yz, such that p(yx II 122). Then
Proposition 5.2. Suppose 2 is composed of categorical concepts (2))
K
dim ( S V (1x
(14)
k-1
Proof. By assuming that p(yz 2) = x for any concept distribution Q2 over Z:
k=1 p(y,, 1 zx), we can easily derive the following result
K
sy sx(21(y)=
Σ
$z [2z, I(y),
where Qz, represents the concept distribution of 2g for cach k, and s,,(2x,1r) V, log (ʃ p(yz, z 12 (zx)dz;). Recall that
2 x =
Oz is a concept distribution),
where Q2 a baseline. Importantly, it should be noted that Rz(y) is unique regardless of the choice of Qt per proposition 4.1.
Let 2% denote the re-rele part of O₂ for k 1,...,K. We define 3z (y) span(1s,, 1Q2, 16y)),, 102 I(y))). Then, can state that:
* xy(y)
Rz (y).
1
Based proposition 5.1, it follows that dim (Z L--1 for each k. Hence, can conclude that:
dim( <
(Lx 11).
k-1
15

---

Experiment Details and More Figures
C.1 Concept projection for Dreambooth (Figure
First, we fine-tune the diffusion model using Dreambooth, applying a learning rate of 5e and setting the number of steps to 800. While there are configurations that could yield a less overfitted model, we intentionally opt for these parameters to generate overfitted model. Our aim is to verify if it's possible to disentangle the overfitted model by using concept manipulation via projection.
To generate images depicting sks toy in front of the Eiffel Tower, we utilize our Dreambooth fine-tuned diffusion model together with the original pretrained Stable Diffusion model. Only for the new prompt, new sks toy", model. All other prompts are plugged into the score functions from the original pretrained Stable Diffusion model. To create the desired images, construct projector using pair of prompts: (x,, x2) = ("a toy", "a soccer ball"). The mask, computed using algorithm 3 with the threshold= 0.1, helps identify specific areas corresponding to the location of the subject. Then, we use the Dreambooth score function Sdreambooth (a sks toy") to guide the generation process within the masked region (areas with value 1), while using s(a toy in front of the Eiffel Tower") to guide the generation outside the mask (areas with value 0).
the score function from the Dreambooth fine-tuned
To ensure image fidelity, we exclusively employ the score function Sdreambooth ("a sks toy") for guiding the denoising process for the last 6% of the denoising steps.
It important note that due to severe overfitting issues with the fine-tuned model, there is no significant difference between using either the prompt "a sks toy" or "a sks toy in front of the Eiffel Tower" for the fine-tuned model. Also, due to the same reason, we apply the original pretrained diffusion model for all score functions except for the sks toy related one.
C.2 The mathematician example (Figure 1)
Our starting point is an original prompt Xorig =a portrait of a mathematician". Our objective is to modify the sex and style using concept projection:
To adjust sex, we formulate corresponding direction using a pair of prompts (x1, ("a man", "a woman"), Subsequently, we set Xnew = "a person".
To alter the style, we set Xnew = "a portrait of a mathematician, in Fauvism style". We define the concept subspace using prompts of the form "a portrait of a mathematician in [x style", where Xsylle takes value from a list of styles. During sampling, the original prompt is utilized in the first 20% of timesteps to better retain the content.
The list of styles is generated by ChatGPT. They are: Art Deco, Minimalist, Baroque, Abstract Expressionist, Cubist, Fauvism, Impressionist, Steampunk, Neoclassical, Japanese Ukiyo-e, Surre- alism, Memphis Design, Scandinavian, Bauhaus, Pop Art, Art Nouveau, Street Art, American West, Victorian Gothic, Futurism, Photorealistic, Mannerist, Flemish, Byzantine, Medieval, Romanesque, Trompe-I'cei, and Dutch Golden Age.
C.3 Stress-test experiments (Figure 3)
In this experiemnt, we deliberately chose a diverse array of artistic styles and contrasting content to challenge our model. The styles used are Baroque painting, traditional Chinese painting, Pop art, Gothic stained glass, the Pre-Raphaelite painting style, Victorian botanical illustration, and Japanese Edo period art. In terms of content, we use: a bustling train station, nuclear power plant, an artificial intelligence lab, a jazz music concert, modern dance festival, a modern minimalist house, and a contemporary office setting. The rationale behind this selection was to pair styles and content that would rarely co-occur training data. This rarity poses a significant challenge for the model in generating realistic outputs, testing its capabilities and adaptability to unconventional combinations.
We implement concept algebra the same way as in Figure 1 example. For concept composition, we use the software in [Liu+22].
In fig. 6 we show some examples where concept algebra is not preferred.
16

---

Figure 6: Some examples where concept algebra not preferred. Each row has style-refrrence (leftmost), a content-reference (2nd from left), and left-to-right (here, but randomly permuted the survey) images generated from muccept-algebra composition and direct prompting.
C.4 The nurse example (Figure 4)
We initiate the process with original prompt, X "a portrait of nurse". Our goal is to perform concept projection to manipulate the sex attribute. Similar to the mathematician example, we define the sex direction using a pair of prompts: (x1, ("a man", "a woman") However, instead of setting the distribution of sex as one of the delta functions or a fair one corresponding to the neutral prompt "a person", we wish see what the concept's arithmetic average will define. Specifically, we take the sex direction of the average of female nurse and male nurse, calculated as {s female nurse")+ sss male nurse"), It turns out the arithmetic mean realize the interpolation between two extremal sex points in the sex subspace, and the score function after concept projection returns images of androgynous nurses.
D Additional experiments
Concept Algebra beats negative prompting in simple tasks Negative prompting, aims to eliminate target concept expressions by subtracting relevant scores. Unlike concept algebra and similar concept composition, this method does not confine manipulations specific subspaces. As predicted, we see negative prompting inadvertently modify off-target concepts tied to the primary concept, whereas concept algebra succeeds, as in
Concept algebra fails when causal separability (Theorem 3.2) is violated We show one concrete example of failures.
Figure 8 shows that we are unable to transfer the gender of the nurse when we calculate the score function of male nurse by (I-projz p- portrait of prrji="+ man"] where proj, is computed by s["a buck on the grass"]-s["" doe on the grass"]. The target concept Z and W are sex (male, female) and speciese (human, deer). It's obvious that the sex and species have an interaction effect on the image - different species induce different sexual characteristics.
17

---

Figure 7: Concept-algebe can succeed when negative prompting fails. Images (matched on random seed) from top to bottom: direct prompting "a portrait of a king", negative prompting w/ "male", negative prompting w/ "male" but using concept projection, concept algebra with prompt "female". Note negative prompting does not remove the maleness of the output.
18

---

s("a portrait nurse")
(b) (L- Propooj.h la portrait nurse ++ = where projz computed by buck grass" --s doe the grass"
(c) (L proj, pr "s("a portrait nurse" 1+ p man') where proj,, computed by 3["a ] s-s woman'] ]
Figure 8: Necessity of Assumptions: the validity of concept algebra depends causal separability
19


Reviews:
[{"review_id": "aIozGrLw30", "reviewer": "Reviewer_1z2h", "summary": "This paper suggests concepts are represented in text-guided generative models as an encoded subspaces of the representation space. It also formalizes the description of the representation of concepts in text-controlled generative model in a mathematical way. The paper also shows that the Stein score of the text-conditional distribution is an arithmetic composable representation of the input text, and develops concept algebra, a method to manipulate the concepts via arithmetic manipulation of the representation. ", "questions": "* See above", "limitations": "* If I understand everything correctly, concept algebra can only work on models have access to their representation, therefore, concept algebra can't really work on closed models, right?\n* How will it negatively impact the society if everyone can edit the model output using concept algebra?", "rating": 9, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 4, "strengths": "Very well-written paper! All the math looks great and I am excited about the potential alternative to prompt engineering! Hope there is a demo that I could try.\n\n- Section 2 and 3 rigorously defines the mathematical background used in the concept algebra method, and section 4 and 5 is also very well-written.\n- I enjoyed reading the figures mentioned in the paper! All of them are informative, and demonstrate the point of each figure clearly.\n- As demonstated in section 6, the proposed method could be a strong alternative to prompt engineering! Being much more stable, less random, and with no need to manipulate models trying to act as a painter or other roles, concept algebra seems very promising!", "weaknesses": "Nothing much, but I would love to see some limitations of the method discussed in section 7.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your in-depth engagement with our work on concept algebra.\n\n**Limitations & Future Improvements**:\n- The primary limitations of our method lie in the concept-subspace estimation step, but there's potential for enhancement:\n  - The current approach depends on basis prompts that vary in $Z$ and are invariant in $W$. A more systematic method for selecting these basis prompts, alongside a quantitative evaluation of their disentangling effectiveness, is required.\n  - Estimating the subspace from $K$ score embeddings presents a challenge in high-dimensional estimation. Despite our use of truncated SVD and variance thresholding, this step could benefit from advanced statistical techniques.\n  - The computation-heavy nature of the estimation step at each sampling iteration is a concern. An immediate workaround could involve performing concept-algebra selectively during the sampling process, as high-level concepts often get determined in initial steps. Alternatively, a model approximating the $\\text{Proj}_z(.)$ function could be trained for direct use during sampling, presenting an interesting avenue for future research.\n\n**Demo Details**:\n- In the supplementary materials, we've included a `concept_algebra.ipynb` demo focused on binary concept alterations. This code can be run with Google Colab with GPU (pro is not necessary). For more complex concepts, we've implemented solutions in `code/concept_pj_basis/concept_pj.py`. \n\n**Applicability to Closed Models**:\n- Our approach isn't directly applicable to closed models. However, if one has access to the underlying language model of a closed system, concept-algebra could be potentially be extended to that language model to tweak embeddings (this will be interesting future research). Following this, nearest corresponding prompts to the altered embeddings could be identified, enabling concept manipulation in closed text-to-image models.\n\n**Potential Negative Impact**:\n- While concept algebra can be misused, its capabilities can also serve as a countermeasure. For instance, if one creates NSFW images using a specific subspace, the same subspace could aid in developing a more robust NSFW detection classifier, reducing susceptibility to spurious correlations.\n"}, {"review_id": "VSt9YyvsFJ", "reviewer": "Reviewer_pK6W", "summary": "This research presents a novel mathematical framework that brings clarity to the abstract notion of concepts, enabling their connection to specific subspaces in a representation space. The significance lies in demonstrating the existence of structured concepts in score representations, specifically emphasizing the compositional nature of these concepts within the representation space, which is useful for further analysis regarding generation.\nOne of the key contributions of this work is the introduction of an effective method to identify the subspace associated with a given concept. This approach allows for the manipulation of concepts expressed by the model through algebraic operations on the representation, thereby providing a powerful tool to work with and understand complex concepts in a more tangible and interpretable manner.\nMoreover, the implications of this research extend beyond its immediate domain, as the proposed framework can be extended to other areas, such as natural language processing (NLP), analysing concepts such as topics and semantic meanings.\n", "questions": "Please pay attention to the clarity of the descriptions.\n1. Some notations are a little bit confusing. E.g., in Definition 2.2,  does “z_{1:k}” refer to “z_1, z_2, …, z_k” regarding Definition 2.2? What is δmale in Equation 11?\n\n2. “Following theorem 3.3” (Line 172).  “theorem 3.3” is not found. Similar cases can also be found in Line 246 and Line 252.\n\n3. It can be worth discussing the selection of the discretized concepts, e.g., how many ks are sufficient for a latent variable C?\n", "limitations": "The limiatations have not been well discussed in the paper.\nAs the performance of the proposed method is mainly evaluated on case studies, it is suggested that a more quantitive measurement be introduced to judge the robustness of the method.", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "1. Previously, the notion of concepts is often discussed in an abstract level. This work demonstrated the existence of structured concepts in score representations, specifically emphasizing the compositional nature of these concepts within the representation space, which can be helpful in further analysis on abstract notions.\n2. The work proposed a useful method to identify the subspace associated with a given concept, allowing us to manipulate concepts expressed by a model through algebraic operations on the representation. This method can be a powerful tool in terms of understanding complex concepts.\n3. The experimental results verified the efficacy of this method in case studies. This method can be extended to analysis on other areas like NLP.\n", "weaknesses": "1. The major concern regarding this work lies in the absence of a precise quantitative evaluation to assess the impact of concept extraction and manipulation, particularly in the experiments. Relying largely on case-level analysis may not be sufficient to draw robust conclusions. Such measurement based on the precise quantitative evaluation can be used to make comparisons between various methods relevant to this topic and for further analysis on subsequent improvements.\nIt will be more convincing if the concept manipulation is evaluated on more cases (or on more datasets) and comparisons are made between the proposed method and relevant methods (e.g., methods to extract and manipulate latent features) used in previous research.  \n2. A minor issue that needs attention is the writing style, particularly the accuracy of internal references. Some of the internal references in the work are not precise and should be revised for clarity and consistency.\n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your detailed feedback and appreciation of our work's contributions. We're glad you recognized the novelty and utility of our approach in providing tangible interpretations of abstract concepts within score representations. Our method, as you rightly pointed out, offers a powerful tool for understanding and manipulating these complex concepts, with potential extensions to domains like NLP.\n\n**Experiments & Limitations**:\nPlease refer to the global response for a comprehensive discussion on the experiments. In brief, we believe our experiments substantiate the paper's primary assertions, and we've integrated a broader scale evaluation in the revised manuscript.\n\nRegarding the noted limitations, they have been elaborated in our global response and will be included in the camera-ready version. Primarily, these limitations pertain to the automation, efficiency, and accuracy of estimating the concept-subspace. As highlighted in Section 5, given the low-dimensionality of the concept-subspace (for many practical problems), we're optimistic about overcoming these challenges in future works.\n\n**Selection of Discretized Concepts**:\nFor a specified prompt $x$, we solely require the adequate sets of concepts $Z_1, …, Z_K$. If our objective revolves around modifying $Z_1$ while maintaining the stability of other concepts, we can designate $Z$ as $Z_1$, with $W$ representing the remaining concepts. Thus, even though our method predominantly uses two concepts, $Z$ and $W$, its applicability remains general.\n\n**Clarifications**:\nWe appreciate your attention to detail. Indeed, Thm 3.3 should be referenced as Prop 3.3. In the notation, $z_{1:k}$ indeed represents the sequence $z_1, z_2, …, z_k$. Regarding Eq. 11, the intended interpretation is $Q_{x_1}(z, w) = \\delta_{\\text{male}}(z) Q_w$,  with $\\delta_{\\text{male}}(z)$  implying $P(Z = \\text{male}) = 1$.\n\n"}, {"review_id": "ludGtvrIsd", "reviewer": "Reviewer_RGiq", "summary": "The paper focuses on encoding abstract concepts (as prompt) and systematically composing them to generate images. They proposed a mathematical framework to generate images based on specific combinations of prompts. Unlike approaches that change the prompt with different text descriptions, this framework encodes each feature independently. It then performs mathematical operations on different features of the prompt and the original content prompt to generate the desired image. They show several successful examples of the proposed framework and compare its performance with direct prompting approaches. Additionally, they emphasize the necessity of the assumption of causal separability.\n", "questions": "1. How does \"sex\" encoded in Figure 1 (b)? what's the result of sex \"man\" or \"women\" in this example?\n", "limitations": "No, the paper doesn't discuss limitations.\n", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The paper introduces an interpretable approach to encoding high-level concepts with LLM representations. By incorporating specific prompts and applying metathetical operations on these prompts and the original content, the framework enables the generation of images that capture desired attributes or concepts. This interpretability allows for more fine-grained control over the generated content. I personally really like this aspect.\n\n2. The paper showed an interesting analysis of the causal relationship keywords used in the framework. By examining the feature differences between terms such as \"buck\" and \"doe,\" the authors demonstrate the limitations of direct application between terms like \"man\" and \"woman\" due to the inherent species difference.\n", "weaknesses": "1. It's necessary to have a larger-scale evaluation of the generated images. While the qualitative results presented in the paper showcase specific examples, a quantitative evaluation on a larger dataset or with a larger sample size would enhance the reliability and generalizability of the method.\n2. I suspect the impact of prompt length is a key factor in the performance of the proposed methods. If shorter direct prompts yield comparable results to the proposed method, it may reduce the appeal of the proposed approach. Once again, a larger evaluation scale on different prompt lengths would shed light on this aspect. \n3. In line 286, you mention that \"1/2 male nurse and 1/2 female nurse\" does not correspond to any English prompt. I wonder if paraphrasing it into a more neutral term like \"androgynous person\" and re-prompting the model could potentially yield different direct prompting results.\n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your thoughtful insights and constructive feedback on our approach and its implications.\n\n**Regarding larger-scale experiments**\nAs discussed in the global response: Our main contribution is the mathematical framework. The demonstrations test predictions of the framework---i.e., that the score representation is arithmetically composable, and the subspaces corresponding to concepts can be identified.  (If these results were not true, we would expect to find no examples where linear representation manipulations change concepts in isolation!)  \nWe have also added some additional experiments, including a quantitative comparison with direct prompting and an algebraic composition method that doesn’t account for subspace structure. We find that concept algebra is significantly preferred by human evaluators.\n\n**Regarding Limitations**\nWe added limitations and discussion subsection in the camera ready version. Please see our global response. \n\n**Regarding the effect of prompt length**\nThanks for the interesting suggestion! Indeed, Fig 3(d) example in the main text uses a very long prompt (to describe a detailed scene). However, it does not seem that prompt length is a key factor. In the added experiments (see global response), we use succinct prompts for describing both content and target style; e.g., “A nuclear power plant” or “Baroque painting”. In these short prompt examples, we observe that direct prompting still frequently fails and concept algebra often succeeds (and is clearly preferred by human evaluators). \n\n**Regarding promptless embeddings**\nThe point about figure 2 is to support the claim that the estimated subspace genuinely corresponds to the concept of sex. The vector $\\frac{1}{2}(s[\\text{\\`\\`male nurse''}]+ s[\\text{\\`\\`female nurse''}])$ doesn’t correspond to any English prompt. In the absence of concept subspace structure, we would expect adding such a vector to either result in nonsense—e.g., a white noise image—or to have no effect at all. Instead, we observe that the outputs are semantically sensible, supporting the idea that we’ve found a direction corresponding to the sex concept. \n\nNote: even if the weighted sum of scores very luckily does correspond to the word ''androgynous'', our claim still holds --- because then $\\frac{2}{3}s[\\text{\\`\\`male nurse''}]+ \\frac{1}{3}s[\\text{\\`\\`female nurse''}]$ will definitely not correspond to the same word. However, we can also generate sensible images with this embedding (we didn't show the images due to space limits).\n\nFinally, regarding your last question  “How is \"sex\" encoded in Figure 1 (b)? what's the result of sex \"man\" or \"women\" in this example?”. Could you clarify what you mean by this question?\n"}, {"review_id": "gpGGDY8wbI", "reviewer": "Reviewer_WFH5", "summary": "The paper aims to propose a formalization of concept-based algebra for text-to-image models. \nIt presents equations for how prompts are composed of concepts, which can interact additively in order to generate the corresponding images, similar to word embedding analogies by Mikolov.\nSeveral examples are presented to show this framework in action.", "questions": "Please define the novel contribution of this work in the context of the previous work in the area of concept-based representations for image generation.", "limitations": "There does not seem to be any discussion of limitations or possible societal impact.", "rating": 3, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 1, "strengths": "The direction of work representing images as latent concepts and then learning representations for these concepts is interesting and useful.", "weaknesses": "The main part of this paper is the mathematical equations for conditioning image generation on concepts. \nIt starts by claiming to provide a cognitive framework, such as mathematical equations for how humans map images to high-level concepts, which seems to be complete conjecture. There are no references provided to show any evidence that this is actually how humans analyse images. I very much doubt that a human looking at an image first starts by making a list of all the attributes in that image.\n\nThe equations then gradually morph into how concept representations can be learned from examples. However, it is unclear how this is an improvement or a contribution over the previous work. The last paragraph of the paper correctly points out several other works that also do concept-based representations for image generation. I would expect there to be a comparison and evaluation. \n\nThere is currently very little evaluation of the method, each claim seems to be backed up by only a single example.\n\nAlmost all of the examples use image style as the concept that is being modified. In one case this is referred to as \"medium\", but even that is ultimately just image style. This moves the work to the style transfer area, which isn't really addressed in the paper. It also doesn't give any evidence that this method can be used for anything involving actual concepts related to the content of the image.\n\nPages 7-8 claim a strength of this method is to handle images for which no prompt exists. However, it seems the first example could be prompted with \"a portrait of a male or female mathematician\" and the second one with \"an androgynous nurse\". Neither of these seem examples for which no textual prompt exist; or at least it hasn't been shown in the paper that prompting like this wouldn't work.\nAlso, this section makes a rather strong claim that the model is being debiased in terms of gender by adding the \"person\" vector, which is a claim that would need a lot more proof than examples from a single prompt.\n\nThere do not seem to be any details on how the experiments were conducted - what pre-trained models or training data was used, or how the modifications were performed in the context of that particular model.\n\nThe appendix is repeatedly referenced for important details, proofs and examples, but does not seem to be submitted.\n\nLine 88: A text-controlled generative model should not be producing a random output.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for taking the time to read the paper.\n\nWe’d like to clarify a few misunderstandings:\n\n1. The paper does not provide a cognitive framework. We do posit a latent variable model in the data generating process, which we think is reasonable. Some evidence that this is reasonable: we are able to use it to derive the (surprising!) conclusion that high-level concepts are encoded as subspaces of the Stein-score representation space, and then find empirical examples showing this is true.\n2. In addition to style transfer examples, we test the method on the concept of sex, and changing a generic toy to a specific one (the Dreambooth example)—these are both content variables!\n3. With respect to the literature on style transfer: the point of the experiments is to show that the Stein score representation encodes these elements already, and in an arithmetically composable fashion. In particular, there is no style-transfer specific heuristics or finetuning. The takeaway of those experiments is that we can find style subspaces. Our claim is not that this is the most aesthetically pleasing way to affect style transfer.\n4. The point of the experiments with unpromptable vectors is to show that the subspace structure does indeed correspond to the concept. So, the point of the androgynous figure example is not that this is the best way to produce an androgynous figure. It's that the output samples are semantically sensible! (As opposed to, e.g., having no effect, or producing nonsense images, as we might expect if there was no semantic subspace structure)\n5. The supplementary material included with the submission has both experimental details and demonstration code. Additionally, it's mentioned twice (including in the abstract) that the experiments are based on Stable Diffusion.\n6. The appendix was submitted in the supplementary material.\n7. ''A text-controlled generative model should not be producing a random output.'' These models produce random samples drawn from a distribution defined by the prompt.\n"}, {"review_id": "EZZUA3DVAk", "reviewer": "Reviewer_mXTP", "summary": "This paper hypothesizes that the latent representations learned by text-guided diffusion models contain structured subspaces corresponding to semantic concepts. Concepts are formalized as latent variables, and the concepts associated with a (text,image) pair are formalized as a bag of these latent variables, such that the distribution over image outputs is conditionally independent of text inputs given these concepts.\n\nNecessary and sufficient conditions (causal separability; Proposition 3.3) are given for concepts to be arithmetically composable; arithmetic composability in turn admits algebraic manipulation of concepts. A proof-of-concept algorithm is given for identifying subspaces of latent representations corresponding to concepts (based on \"spanning prompts\") and examples are provided of identifying and manipulating these subspaces using the Stable Diffusion model.", "questions": "Manipulating the non-parametric representation space via evaluated values along the diffusion of a score-based model is clever, and my understanding is that this is the key to making concept algebra tractable. Could these methods be applied to, e.g., the Parti model? If not, it might be helpful to point this out and be more explicit about how and where these method depend upon the structure of a score-based model.\n\nThis work seems closely related to interpretability research. The concept editing algorithm could be seen as a sort of causal intervention (in the sense of \"do calculus\") on a hypothetical interpretation of a latent representation. I'm curious how this work connects or complements recent interpretability research, e.g., ongoing work on mechanistic interpretability.\n\nIs the definition of \"causal separability\" given here related/consistent with established definitions of \"causally separable processes\"?", "limitations": "It is unclear to me how restrictive the causal separability condition might be (Definition 3.2). Especially for more abstract, complicated concepts, I could imagine that causal separability might never be satisfied (but I am open to being convinced otherwise). Even for simple concepts, it seems like causal separability can be violated in ways that might not be obvious: for example, the inseparability of (male,female) and (deer,human) surprised me (although it became very clear once the reason for inseparability was explained to me). Understanding the how broadly causal separability holds seems essential to the ultimate practicality of this framework.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 4, "strengths": "The conceptual framework constructed in this paper is though-provoking. The exposition is quite clear, striking a good balance between helpful exposition and mathematical precision. Definition 2.5 (arithmetic composability) is interesting, as is Proposition 3.3 (characterizing arithmetic composability in terms of \"causal separability).\n\nThe algorithms proposed in Section 4 for identifying and manipulating concepts are well-motivated. I also appreciate the discussion in Section 5 of the structure/character of concept subspaces. These analyses could be a good starting point for guiding empirical study of the score representation.\n\nDepending on how the effective the proposed algebraic interventions prove to be in practice (see Weaknesses) I think that the abstractions and definitions introduced in this paper have the potential to stimulate an entirely new direction of work on control and interpretability of generative models. Control and interpretability are broadly relevant to the NeurIPS community.", "weaknesses": "The experiments in this paper are very rudimentary. There are no quantitative results, and the qualitative results consist of a small number of examples provided in Figures 1-5. It is unclear to me how broadly effective, reliable, or robust this method is. There is just not enough information to evaluate whether these ideas work (or could be made to work) in practice; see the Limitations section of my review for additional thoughts about practicality.\n\nMinor: \"Causal Separability\" is strong language, and I'm not convinced that the word \"causality\" accurately describes the relationship defined by this term. At the very least, some argument is needed to justify calling this a causal relationship.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your thoughtful review! We’re glad that you found the conceptual framework thought provoking, and you think it may have the potential to stimulate new lines of work in control and interpretability.\n\n**Experiments**\nWith respect to the experiments, please see our global-level response. In short: the purpose of the experiments is to test the mathematical framework by testing the prediction that (1) the Stein score is an arithmetically composable representation, and (2) it's possible to find finite dimensional subspaces corresponding to concepts. In our view, the examples in the paper do this—if the subspace structure didn't exist, then we couldn't find examples that demonstrate it! \n\nAdditionally, we have added a new experiment testing whether the concept subspace structure is actually useful or necessary for model control. To that end, we compare with an existing method for style transfer that just adds on score vectors generated by style prompt (not taking subspace structure into account). We also compare with direct English prompting. We find that human raters significantly prefer samples using concept algebra—see the global response for details. \n\n**Causal Separability**\nThe intuitive idea here is that the separability of factors of variation boils down to whether there are “non-ignorable” interactions in the structural equation model that generates the output from the latent factors of variation—hence the name. The formal definition 3.2 relaxes this causal requirement to distributional assumptions. We have added its causal interpretation in the camera ready version. \n\n**Application to Other Generative Models**\nUltimately, the results in the paper are about non-parametric representations (indeed, the results are about the structure of probability distributions directly!) The importance of diffusion models is that they non-parametrically model the conditional distribution, so that the score representation directly inherits the properties of the distribution. \n\nTo apply the results to other generative models, we must articulate the connection between the natural representations of these models (e.g., the residual stream in transformers) and the (estimated) conditional distributions. For autoregressive models like Parti, it’s not immediately clear how to do this. This is an exciting and important direction for future work!\n\n(Very speculatively: models with finite dimensional representations are often trained with objective functions corresponding to log likelihoods of exponential family probability models, such that the natural finite dimensional representation corresponds to the natural parameter of the exponential family model. In exponential family models, the Stein score is exactly the inner product of the natural parameter with $y$. This weakly suggests that additive subspace structure may originate in these models following the same Stein score representation arguments!)\n\n**Connection to Interpretability**\nThis is a great question! Indeed, a major motivation for starting this line of work is to try to understand if the ''linear subspace hypothesis'' in mechanistic interpretability of transformers is true, and why it arises if so. As just discussed, the missing step for precisely connecting our results to this line of work is articulating how the finite dimensional transformer representation (the residual stream) relates to the log probability of the conditional distributions. Solving this missing step would presumably allow the tool set developed here to be brought to bear on the interpretation of transformers.\n\nOne exciting observation here is that linear subspace structure appears to be a generic feature of probability distributions! Much mechanistic interpretability work motivates the linear subspace hypothesis by appealing to special structure of the transformer architecture (e.g., this is Anthropic's usual explanation). In contrast, our results suggest that linear encoding may fundamentally be about the structure of the data generating process.\n\n**Limitations**\nOne important thing to note: the causal separability assumption is required for the concepts to be separable in the conditional distribution itself. This is a fundamental restriction on what concepts can be learned by any method that (approximately) learns a conditional distribution. I.e., it’s a limitation of the data generating process, not special to concept algebra or even diffusion models.   \n\nNow, it is true that to find the concept subspace using prompts we have to be able to find prompts that elicit causally separable concepts. However, this is not so onerous—because sex and species are not separable, we can't elicit the sex concept with ''buck'' and ''doe''. But the prompts ''a woman'' and ''a man'' work well.\n"}]

Author rebuttal:
We appreciate the reviewers' insightful comments.

The reviewers broadly agree that understanding how high-level concepts are encoded in the internal representations of generative models is a timely and important topic, and that the mathematical framework developed here is a significant step in this direction. Several reviewers note that the development in this paper opens up important new directions in the control and interpretation of language guided generative broadly. Further, the reviewers generally found the exposition clear and thought provoking.

## Existing experiments
Reviewers expressed concerns about our experimental evaluation. We assert that our experiments are scientifically robust and align with standard publication norms. Additionally, we've introduced further experiments highlighting concept algebra's advantages over traditional heuristics.

Regarding our scientific contributions, let's distinguish the three claims one might make:
1. Our mathematical framework connects high-level concepts with internal representations, making significant predictions.
2. Concept algebra holds promise for image manipulation in text-guided diffusion models.
3. Our method excels in image manipulation compared to some alternatives.

Our paper mainly champions claim (1), with preliminary evidence supporting claim (2) (which may have created the confusion). As we note explicitly in the paper, our claim is not (3).

Our empirical evaluation aims to ascertain the utility of the mathematical framework. We specifically address the following non-trivial implications of our theory:
1. The Stein score represents concepts in an arithmetically decomposable manner.
2. Subspaces can be discerned using the methodologies from sections 4 and 5, given certain conditions.
3. These conditions aren't merely theoretical; concrete examples of concepts exist where corresponding subspaces can be identified.

To validate these claims, we presented examples within the paper that pinpoint concepts, identify their corresponding subspace, and exhibit their arithmetically composable nature.

We furthered our analysis, which could have been misunderstood. Through two key stress tests:
- First, our experiments revealed that manipulations within the sex subspace are coherent, even when such manipulations aren't directly prompted by English. Importantly, our emphasis is **not** about images cannot be generated via English prompts but that the sex subspace genuinely encapsulates the concept. The example \( $\frac{1}{2} (s[\text{\`\`man''}]+ s[\text{\`\`woman''}])$ \) illustrates this, producing semantically sound images even without a direct English phrase representation.
- Our work emphasizes concept manipulation's effectiveness, even when direct prompts fail. The point here is that the subspace structure is free of the prompt, and so works even for "hard" prompts.

In the camera-ready version, we will expand on the discussion and address several limitations:
- Our primary emphasis is on the mathematical framework. We anticipate its applicability beyond just the text-to-image setting.
- Concept algebra is useful for communicating user intention to the model. It does not generically change the quality of generated images. Accordingly, it is complementary to generative model improvements such as architecture changes or increased scale
- We acknowledge the computational demands of concept algebra and the necessity for handcrafting prompts to pinpoint concept subspaces. These present significant practical challenges. These are both significant practical issues, and an important direction for future work would be overcome this.

## Further Experiments
Beyond this, we've incorporated new experiments (figures and results in the attached pdf) to highlight our theory's advantages over existing heuristic methods. Detailed codes and results have been shared anonymously with the AC.

Concept algebra modifies representation vectors in subspaces aligned with target concepts. To gauge its utility, we contrasted it against methods that don't utilize such structured subspaces. Methods like [Du+21; Liu+21; NBP22; Ano23] (references in main text) employ algebraic manipulations without pinpointing specific subspaces. Another common approach, negative prompting, aims to eliminate target concept expressions by subtracting relevant scores. Unlike concept algebra, these methods don't confine manipulations to specific subspaces. Consequently, our theory posits that these heuristics might inadvertently modify off-target concepts tied to the primary concept, like inducing a medieval theme while aiming for a renaissance style.

We assessed concept algebra's efficacy against composition and direct prompting in style transfer tasks. Using 49 challenging content/style combinations, like "A nuclear power plant in Baroque painting", we employed the three methods on each pair to generate samples. Human raters were then presented with the outcomes alongside reference images, ranking them based on adherence to the desired style and content. This evaluation was replicated across 10 different raters. Refer to Fig 1 in the attached PDF for illustrative examples. **Raters consistently favored images produced by concept algebra**, as highlighted in table 1(c) of the attached PDF. This aligns with our theory, suggesting concept algebra's adeptness in retaining content while altering style.

Furthermore, we demonstrated a comparison between concept algebra and negative prompting. Using the prompt "a portrait of a king", our aim was to transition to "a portrait of a queen". Negative prompting using $x_{-} = \text{\`\`male"}$ was ineffective, while concept algebra employing  $x_{\text{new}} = \text{\`\`female"}$  (refer to equation 13) achieved the desired result. For a fairer comparison, we applied the same negative prompt in conjunction with concept projection, which was also successful. Details can be found in the anonymous link provided to AC.


Meta-review: This paper presents a study that argues that concepts within text-guided generative models are manifested as encoded subspaces within the representation space. It provides a formal mathematical formalization of the representation of these concepts within text-controlled generative models. The work is highly thought-provoking and is commended for its clarity, striking a balance between providing clear explanations and maintaining mathematical precision. Several definitions and propositions introduced in the paper are noted as interesting contributions. The algorithms proposed for identifying and manipulating concepts are well-motivated, offering potential avenues for practical application. Overall, the paper's abstractions and definitions have the potential to inspire new research directions in the control and interpretability of generative models, which is relevant to the NeurIPS community.
Decision: Accept (poster)