Concept Reachability in Diffusion Models: Beyond Dataset Constraints
Authors: Marta Aparicio Rodriguez
Xenia Miscouridou
Anastasia Borovykh
Venue: ICML 2025 poster (2025)
Keywords: Diffusion models
concept learning
steering vectors
Primary area: deep_learning->generative_models_and_autoencoders

Abstract: Despite significant advances in quality and complexity of the generations in text-to-image models, *prompting* does not always lead to the desired outputs. Controlling model behaviour by directly *steering* intermediate model activations has emerged as a viable alternative allowing to *reach* concepts in latent space that may otherwise remain inaccessible by prompt. In this work, we introduce a set of experiments to deepen our understanding of concept reachability. We design a training data setup with three key obstacles: scarcity of concepts, underspecification of concepts in the captions, and data biases with tied concepts. Our results show: (i) concept reachability in latent space exhibits a distinct phase transition, with only a small number of samples being sufficient to enable reachability, (ii) *where* in the latent space the intervention is performed critically impacts reachability, showing that certain concepts are reachable only at certain stages of transformation, and (iii) while prompting ability rapidly diminishes with a decrease in quality of the dataset, concepts often remain reliably reachable through steering. Model providers can leverage this to bypass costly retraining and dataset curation and instead innovate with user-facing control mechanisms.

Concept Reachability in Diffusion Models: Beyond Dataset Constraints
Marta Aparicio Rodriguez'
Xenia Miscouridou'
12 Anastasia Borovykh"
Abstract
(see Figure 1). In addition learning the visual and spatial components of concepts in images, text-to-image models must correctly associate the concept in an image with its corresponding semantic description in the caption (Huang et al., 2023; Ghosh et al., 2023; Chang et al., 2025). When this alignment fails, even overspecifying and re- re-prompting may fail generate the target image.
Despite significant advances in quality and com- plexity of the generations in text-to-image mod- els, prompting does not always lead to the de- sired outputs. Controlling model behaviour by directly steering intermediate model activations has emerged as a viable alternative allowing to reach concepts latent space that may other- wise remain inaccessible by prompt. In this work, we introduce set of experiments to deepen our understanding of concept reachability. We de- sign training data setup with three key obstacles: scarcity of concepts, underspecification of con- cepts in the captions, and data biases with tied concepts. Our results show: (i) concept reach- ability in latent space exhibits distinct phase transition, with only a small number of samples being sufficient to enable reachability, (ii) where in the latent space the intervention is performed critically impacts reachability, showing that cer- tain concepts are reachable only at certain stages of transformation, and (iii) while prompting abil- ity rapidly diminishes with a decrease in quality of the dataset, concepts often remain reliably reach- able through steering. Model providers can lever- age this to bypass costly retraining and dataset curation and instead innovate with user-facing control mechanisms.
Prior work has shown that, an alternative prompt-based sampling, one operate directly model's represen- tation level (Kwon et al., 2023; Ilharco et al., 2023; Wang et al., 2024; Wu et al., 2024). In particular, by editing spe- cific activations, the sampling trajectory of diffusion models can be adjusted towards a particular target (Epstein et al., 2023; Samuel al., 2024; Li et al., 2024a). While these works show that it possible steer towards certain output concepts, we lack concrete framework understand the complexity of guidance, in other words the reachabil- ity of concepts. When can concept be reliably accessed through prompting? prompting fails, under what con- ditions can steering reliably reach the concept? And ulti- mately, what factors render a concept entirely unreachable despite it being some way present in the training data?
imago beaker
an image person walking left
image rainbow in black and white
1. Introduction
The scaling of diffusion models (Sohl-Dickstein et al., 2015; Song & Ermon, 2019; Ho al., 2020; Song al., 2021) has significantly expanded their capacity store and generate vast amounts of complex concepts. While prompts have be- come the de- facfacto manner to control the model output, there are numerous examples when simply prompting falls short
Figure Images generated by Stable Diffusion (Rombach et al., 2022) that fail produce the desired outcome due to hypothesised dataset limits: (L) a scarce concept, (C) underspecification in a caption, (R) biases. See Appendix F22 for details.
'Department of Mathematics, Imperial College London, UK 'Department of Mathematics and Statistics, University of Cyprus, Cyprus. Correspondence Marta Aparicio Rodriguez <marta.aparicio www.apaiciiiooriiueeee22222 innrriguz222@mmeriilll..... icperial.c.ukk>.
Existing work has shown that the dataset structure plays key role in reachability. For GANs and VAEs, Zhao et al. (2018) conclude that biases in the train set will influence the generation ability of models, regardless of the architecture or training algorithm implemented. In diffusion models, Chang et al. (2025) show that certain characteristics such as balanced datasets help mitigate the failure of generated
Proceedings of the ndnd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

Concept Reachability in Diffusion Models: Beyond Dataset Constraints
image match the target prompt. However, the complexity of the real world complicates the construction of balanced training dataset that fully captures the true data-generating process. Figure/ 1 highlights scenarios in which prompt- ing fails to produce the correct output. Underlying these scenarios are three core dataset limitations:
models are unable disentangle, and hence reach, the concepts effectively.
Our work demonstrates both the limits of prompting and the resilience of latent space interventions under three com- monly present dataset limitations. Our work suggests that instead of curating new datasets or retraining models from scratch, model providers can enable users to reach concepts through novel control mechanisms. By shifting focus from data curation to user-driven model steering, providers can enhance model usability, robustness, and accessibility in ways that go beyond the limitations of prompting.'
1. Scarcity of concepts: real-world datasets often exhibit an uneven distribution of concepts, with some being un- derrepresented or appearing infrequently. Such scarcity can hinder model's ability effectively learn these concepts.
2. Underspecification of captions: real-world datasets often include captions that do not describe all the con- cepts present in image. This lack of detailed an- notation can constrain model's ss ability to accurately associate unmentioned concepts with their correspond- ing semantic representations.
2. Prior Work
Control in LLMs Large language models (LLMs), hav- ing been trained on vast amounts of data, can be concep- tualised as powerful information compressors, raising the challenge of how to effectively extract task-specifi infor- mation. Various methods have been proposed to address this challenge without requiring fine-tuning of all model pa- rameters. Some approaches focus on fine-tumie tne-ttning smaller subset of weights (Hu al., 2021; Zaken al., 2022; Il- harco al., 2023; Wu et al., 2024), while others enhance performance by optimising the input for specific tasks (Liu et al., 2022). Existing work additionally explores modify- ing a model's activations by introducing steering vectors at specific layers (Meng et al., 2022; Todd et al., 2024; Panickssery et al., 2024; Marks & Tegmark, 2024; Turner et al., 2024; Li et al., 2024b). In particular, recent advance- ments in Sparse Autoencoders demonstrate their potential in capturing interpretable latent representations, enabling fine- grained control and enhanced interpretability in language models (Cunningham et al., 2023; Templeton et al., 2024).
3. Biases: real-world datasets frequently exhibit biases, where certain concepts consistently co-occur. This inherent correlation can make it challenging for models to disentangle these concepts, limiting their ability to independently generate or represent them.
Real-world images are often highly complex, with many in- tertwined factors that can make it challenging to isolate the exact conditions under which model successfully reaches concept. To address this, we work within synthetic frame- work that allows us to systematically vary the structure of the dataset (for details, see Figure 2 and Section 3.3). This enables us to understand in detail how the mechanics of concept reachability are influenced by the above-mentioned three scenarios. To verify the generality of our main con- clusions, we analyse the impact of the same scenarios real-world data, including Stable Diffusion (Rombach et al., 2022) and CelebA (Liu et al., 2015). We list our contribu- tions as:
Steering text-to-image models Similar LLMs, previ- ous works have successfully steered diffusion models to generate rare concepts, compose concepts and manipulate specific attributes in an image. Samuel et al. (2024) optimise the initial random seed to address generation failures and produce desired outputs. Other methods focus on modifying the U-net output (Wang et al., 2024; Gandikota et al., 2025) or adjusting activations of particular layers. Research has shown that editing the generation process can be achieved by adding vectors or fine-tuning the cross-attention layers that inject semantic information into the model (Hertz et al., 2022; Epstein et al., 2023; Kumari et al., 2023; Gandikota et al., 2024). Furthermore, semantically meaningful direc- tions in the bottleneck layer of the U-net have been identified as a means to steer the generation effectively (Kwon et al., 2023; Li et al., 2024a; Haas al., 2024).
We show that even minor dataset limitations severely decrease the effectiveness of prompts, highlighting an inherent weakness in relying solely on this method of model control.
We demonstrate that concepts remain accessible within the latent space, even under highly corrupted dataset conditions.
We demonstrate a phase shift in reachability: a rapid increase in reachability can be observed as the number of images containing a concept is increased beyond a concept-agnosttc, low threshold.
We identify when concepts cannot be reached: when a concept is not specified in the captions of the train set,
Code is available https: https://github.ee httpb/..hhhhbo/////o ma httppparrrd/ccccp_reeeeeee
2

---

Concept Reachability in Diffusion Models: Beyond Dataset Constraints
Data attribution methods The study of how training data influences model's output (Koh & Liang, 2017; Pruthi et al., 2020; Park et al., 2023; Hammoudeh & Lowd, 2024), particularly in real data, is closely intertwined with the broader question of how dataset constraints govern reach- ability. Data attribution techniques have been extended to image generation in diffusion models by analysing the sampling dynamics (Georgiev et al., 2023: Zheng et al., 2024), or by examining intermediate checkpoints (Xie et al., 2024) or hidden representations obtained during the training process (Brokman et al., 2025). While these approaches provide valuable insights, significant practical challenge persists: rigorously evaluating the accuracy of influence estimation methods. To address this, Wang et al. (2023) introduces a methodology for constructing datasets explic- itly influenced by known datapoints, thereby establishing framework for empirical evaluation. In our work, we adopt synthetic dataset, where the influence relationships are known by design.
noise by minimising the loss function
C = [1. - co (x,;
We implement text-to-image diffusion models EØ(X,, i, y). that additionally condition generation on text prompt y E V, passed through a text encoder & and inputted into the U-net through cross-attention layers (Vaswani et al., 2017; Rombach et al., 2022; Ramesh et al., 2022; Nichol et al., 2022: Epstein et al., 2023). The train set is comprised of image-caption pairs (x. y) E X x y. The sampling process involves the denoising of a latent X ~ NN I) conditioned on an input prompt y'. See Appendix A for details on our choice of architecture and hyperparameters.
3.2. Concepts
We introduce assumptions on the underlying structure of our data, following similar approach to Zhao al. (2018); Llera Montero al. (2021); Okawa et al. (2023); Park et al. (2024); Wang et al. (2024). We assume that the images in the dataset are generated by a set of factors, such as object iden- tity, colour, position or texture. Each factor can take certain values, which we denote as concepts. Formally, we say there exists a set of n concept variables = { (Fi, , F.), that define the image x ~ Pdata (x). Each of the variables Pi are sampled from their respective distributions p(F.). We denote the set of possible values the variables F; can take as Fi, which can be discrete or continuous. The space of concepts is then defined as the set F = Fi x F2 x ... x Fn. containing all possible combinations the factors can take.
Analysis of reachability in generative models Underly- ing the question of whether we can reach a certain concept lies the ability of the model to properly compose concepts seen during training. Research assessing this compositional- ity through prompting concludes that models can compose latent factors in novel ways if trained on sufficiently di- verse data or for extended periods (Deschenaux et al., 2024; Okawa et al., 2023). Studies have also explored the learn- ing dynamics that shape a model's generalisation ability. Mészáros et al. (2024) observe simplicity bias in LLMs, where the learning process prioritises simpler tasks earlier in training, Additionally, Park et al. (2024) identify sudden transitions where diffusion models rapidly acquire the abil- ity to generate specific concepts. In practice, failure cases do remain, with the work of Chang et al. (2025) analysing the impact of the underlying data distribution and lack of cov- erage of unique phenomena. Furthermore, steering vectors in LLMs have been found to sometimes produce unreli- able or even counterproductive results (Tan et al., 2024). Building on these insights, our work investigates the factors contributing to the unreliability of reachability methods.
Each combination of values (f1, f2..... fn) E F uniquely determines an image x, where this relation is defined by injective function g : F -> WxxCCC that transforms the tuple f2,.,.,.... fn) into image x with the target concepts f1, f fn. This function determines the distri- bution of x when conditioned on a combination of factors.
Throughout our work, we focus on a subset of factors assumed to be identifiable for all images sampled from Pdata (x). Without loss of generality, we assume these the first n factors F2. Fm- We refer to the corresponding values of these factors, (fi, , fm). as concepts of interest. We use the notation [fr, fia..... fx where ig E {1,2, m) for k = 1, 1,2,.... 1, to refer to sub- sets of the dataset that share the concepts fax fia.... für for the factors Figh F,,. and have no restrictions along other factors.
3. Background
3.1. Diffusion Models
Denoising diffusion probabilistic models (Ho et al., 2020) approximate the distribution (xata(x) that gives rise to collection of data points X. During training, noise is added to images Xo from a train set X' to give latents X; = Vā,2a + Vā, for E [) and appropri- ate constants ā, dependent on a noise schedule. A U-net (Ronneberger et al., 2015) eg is trained to match the added
We further assume that captions capture the expressiveness of the concepts of interest in an image. For an image x containing concepts of interest (f1, f2, f2...., fm), we assume there is an injective function h : F, x x ....X Fm + R' such that h(f1, fe...., is a string containing the se- mantic information relevant to the concepts f1, f2....,frr fm-
3

---

Concept Reachability in Diffusion Models: Beyond Dataset Constraints
Therefore, image-caption pairs (x, y) X x y/ satisfy that the concepts of interest of x and the tuple h (y) are equal.
3.4. Reachability
In this section, we introduce the notion of reachability and key definitions used in subsequent sections.
3.3. Problem Setting
Definition 3.1 (Concept Reachability). Given a target com- bination of concepts access these, we define the reachability of f. as the accuracy or proportion of images produced by M that contain the concept combination (f1, f2...., :
Our study examines how the presence and relationships within training data influence the ability of diffusion models to reach concepts (Figure 2). To achieve this, we systemati- cally vary the dataset used to train model, and observe the evolution of reachability for specific concept combinations. We vary the following:
fm) and method M to
We also identify cases during generation in which outputs are out-of-distributie
1. Scarcity of concepts: starting with a balanced dataset, the presence of individual concepts is progressively reduced. We evaluate how the reachability of com- binations containing the underrepresented concept is affected.
Definition 3.2 -ft-oo-diisributtion). Given model trained a dataset X' y with concept function g as defined in Section 3.2, generated image x" with concepts of interest (fT, .mm) is out-of-distribution (OOD) if [2, S = 0.
We additionally distinguish between two mechanisms that result in unseen combinations. This distinction provides a valuable framework for understanding differences in reacha- bility levels OOD generalisation scenarios, as in Sections 5.2 and 5.4. In particular, we consider a model's ability to combine known factors into new configuration and model's 's ability to transfer knowledge of attributes from one positional context to another.
2. Underspecification of captions: beginning with fully annotated dataset, semantic information relevant to certain factors removed. The impact of this specifica- tion reduction on the model's ability to reach complete concept combinations is assessed.
3. Biases: from a dataset where two concepts are con- sistently paired, we incrementally introduce data con- taining only one of the concepts. We analyse how the model's ability to independently reach each concept evolves.
Definition 3.3 (Compositionnaly out-of-distribution)) Given an out-of-distribution combination of concepts F, = (fi, f2....... fm) E Fi x F2 X X Fi. we say it com- positionally out-
out-of-aaeteo of distribbution for every concept fy in the combination, there exists a concept combination in the train set whose jth component is fj-
Reduce number of images contaning concept
Definition 3.4 (Positionally nitoo--ditrribution). Given an cuutoof-distrbbutin combination of concepts F, (f1.J2.-..; E F1 x F2 x ... x Fm. we say it is positionally out-of- distiriiinitio if it is not composition- ally out-of-distribution, and there exists a permutation P. : {1,2, ... .m} - {1.2...., m] such that F% (foci): fo()) -................... fp(m)) ) is seen during training.
F2
Reduce label
[ 189)
specifleation
(f8-189)
18
(fP, f (89)
(PP-12")
(129)
Fi
Roduce concepts datn
4. Methodology
4.1. Dataset
10
72
We design a synthetic experimental setup that pursues con- trollability over the tasks outlined in Section 3.3. Our setup is similar to the work of Zhao al. (2018); Llera Montero et al. (2021); Scimeca al. (2022); Wiedemer al. (2023); Okawa et al. (2023); Deschenaux et al. (2024); Park et al. (2024); Chang et al. (2025). We use dataset that contains images of coloured shapes on black background, where one shape is partially covered by the other (Figure 10). See Appendix B for further details.
Figure 2: Visualisation of the structure of the dataset ac-
cording to concepts of interest (in the diagram, three). Each block in the cube represents the collections of images in [f(0) : -/ (k) [x. A darker shade represents higher num- ber of images in the block. Data modifications are as de- scribed in Section 3.3.
Let (circle, triangle, square) and C = {red, green

---

Concept Reachability in Diffusion Models: Beyond Dataset Constraints
blue). The images in our dataset are labelled using the captions, "a {cr} {s1) behind a {cz) {$2)", where $1; 82 E S, and (2) E (C x C) \ {(c,c) : c E CJ. That is, all combinations of any two shapes and any two colours in S and C are admitted, excluding images containing two shapes of the same colour. We refer to the concepts of interest images as (c1, 81; C22 82) (Figure 3). We create the dataset such that each of the combinations of concepts (c1, 81, C22 82) is originally seen equal number of times.
containing the concept combination of interest. Using the starting prompt Vs- we create the image-label pairs (Xo, Us) for every Xo Z.
Prompt space Let Eye) be the text embedding outputted by the text encoder for the prompt ys- We modify the sam- pling trajectory of ys by replacing E(y.) by E(y,) + Vp. where vp is vector of the same dimensionality E(y.). We refer to the output of the U-net after this modification as co (x,, f, y, vp) (Figure 9A). To obtain the vector Vp. the following loss is minimised:
Colour
Front Back
Shape x-position j-position
Front Back
Colourt Cy Shape:
(c,,
Lp = (x.. (x t Vo, vp)ll?
Colour
Colour 02 Shape:
blue circle behind a not trinngl
Shape
X-position
Note that the weights of Eg are frozen. The vector Vp there- fore accounts for the mismatch between the semantic in- formation contained in y, and the visual information in the noisy latents obtained from images in Z.
yegosition
Figure 3: Concepts of interest in the dataset. The concepts of an image are summarised as the tuple (c1, C2 82), where each of these positions refers to back colour, back shape, front colour and front shape respectively. In the diagram, this is (blue, circle, red, triangle).
h-space We also consider steering of the starting prompt Ys on the h-space, following the implementation of Li et al. (2024a). Let en Exe, 4,; Vn.) denote the output of the U-net when inputted the noisy image Xt, the prompt y, and the vector Va. which is added to the bottleneck layer output (Figure 9B). The vector is optimised to minimise the loss:
Our original dataset is comprised of 54 combinations of shapes and colours (C1, 81, C22 #2), each containing 1000 images. During experiments where the number of images containing certain concept combinations is varied, we pre- serve the total size of the dataset by adjusting the size of the remaining combinations.
LA 1... EB (x Uay V2)
4.2. Steering
We employ implementations that add constant vector specific layer of the U-net during sampling. These methods consist of two stages: the optimisation of concept vector using images that contain the target concept combinations, and the addition of the optimised concept vector during sampling. Motivating our choices from existing work on steering (Kwon et al., 2023; Kumari et al., 2023; Li al., 2024a), we consider two spaces in which implement steering: the text encoding of the prompt and the bottleneck layer of the U-net.
Similarly to steering on the prompt space, the vector vn is optimised to reconcile the mismatch between the semantic information in y, and the visual concepts in the images.
4.3. Evaluation Method
Reachability, as defined in Section 5, necessitates the identi- fication of the concepts of interest within sampled images, Identifying concepts in images on large scale can prove challenging task, with most real-data methods relying vision-text models such as CLIP (Radford et al., 2021; Huang et al., 2023; Ghosh et al., 2023). To evaluate the con- cepts within the generated images, we train three classifiers for identifying the back shape, front shape and back-front colour pairs of the two shapes in each image. The result- ing labels are then compared against the target concepts. Images producing incomplete outcomes (such black background with no shapes or an image only showing one shape) are accounted for as incorrect images (see Appendix C). Throughout our experiments, we train four models ini- tialised with different random seeds and report the mean results obtained.
Let Ea denote a conditional U-net trained on the dataset X' x V. For a given combination of concepts (f1, ......... fmn). the model can generate an image containing this combina- tion by using the prompt ye h(ff, f2. ... fm). Alter- natively, we sample from starting prompt y, and steer the generation process towards the desired combination of Note that ys may not satisfy y. yee or that the image produced by the model may not accurately represent the target concepts of interest. Thus, we use steering to enable the model to reach the desired output more accurately. Let 2 be collection of images
concepts (J1 f,...,,,,..
5

---

Concept Reachability in Diffusion Models: Beyond Dataset Constraints
5. Empirical Results
the steerability bias reported by Tan et al. (2024). This sup- ports the role of dataset properties in shaping reachability behaviour. We provide further analysis of reachability on the h-space in Appendix E.I.
5.1. Baseline: Balanced Dataset
Before introducing additional complexities to the data, we establish a clear understanding of the different reachability methods under balanced conditions. Specifically, we train diffusion models and use steering to generate various tar- get combinations, varying the level of complexity between these and the starting prompt. Reachability when directly prompting the target combination is also accounted for, as illustrated in Figure 4.
Overall, we observe that prompting and steering perform effectively in a perfectly balanced dataset. In the following sections, we present results regarding their performance un- der adverse data conditions. Based on the results observed under balanced conditions, we choose the starting prompt that maximises performance when steering on the h-space. Unless stated otherwise, throughout the remaining experi- ments we choose to implement steering from the starting prompts ys which describe the target concepts (0 concepts changed).
ys = a green triangle behind a red triangle
1.0
5.2. Scarcity of Concepts
0.8
We fix (red, triangle, green, square) as a target concept combination and gradually reduce the number of data points in the subset [ci red,x. This process is repeated for the subsets [$1 triangle] x [c2 green] x and [s2 square)x. We present two of these tests in this section and relay the remaining cases to Appendix E.3.
Accuracy
0.6
0.2
Preenpt Prempt space
space
Reachability drops sharply past a critical threshold We implement steering to our chosen target concept combina- tion and observe that, although reachability decreases as the presence of concepts is reduced, this decline remains grad- ual until reaching a threshold at px (f) z 0.01, = 61, 82- Below this, reachability decreases rapidly (Figure 5). This decline is less pronounced for steering, most noticeably for $2 square, highlighting that steering can be more effec- rive at maintaining concept reachability under conditions of scarcity. This behaviour suggests a phase transition-like effect: when concept presence falls below 1%, the system shifts abruptly from a state of high reachability to one of significantly diminished reachability. This phase transition implies that the model requires few data points, relative to the train set size, in order to learn a concept effectively. Increasing the number of data points containing a concept beyond this critical threshold has limited impact on improv- ing reachability.
0.0
0
1
2
3
Number of concepts changed
Figure 4: Reachability to different concept combinations when prompting, as well as steering from the starting prompt y, -'a green triangle behind a red triangle". Target colour- shape combinations (note that the relative position is repre- sentative) are shown at the top of each bar, and are organised according to the number of concepts that differ from the concepts of ya-
Prompt-space steering matches prompting reachability Prompting and steering on the prompt space yield an accu- racy above 0.9, demonstrating their effectiveness in reaching concepts under balanced conditions. This highlights steer- ing as reliable method for accessing desired concepts. In contrast, steering within the h-space exhibits pronounced dependency on the targeted concepts (for fixed y.). No- tably, accuracy on the h-space diminishes as the number of modified concepts increases, suggesting a limitation in its capacity to handle heavily complex modifications.
Positionally OOD combinations can be reachable through steering When the presence of a concept f is such that px (f) = O, generating the combination (red, triangle, green, square) results in a positionally OOD target. For instance, eliminating the data points in (c red]xx. limits the model's exposure to red shapes in the back position, even if red shapes remain present in the front position C2. Our findings reveal that models are unable to reach the target concept combination through prompt- ing (Figure 5), aligning with observations by Chang et al. (2025). In contrast, steering methods can achieve moder-
Reachability bias when steering on the h-space We ob- serve a tendency for reachability to be higher, relative to other concept combinations with the same number of modi- fied concepts, when the target front shape and back shape are identical (second column from the right of Figure 4). We identify this as a property of the dataset, as is consistently observed across different models, and find it aligns with
6

---

Concept Reachability in Diffusion Models: Beyond Dataset Constraints
G1 =
red
for accessing target concepts under reduced specification conditions.
1.00
Accuracy
0.75
0.50
A decrease in specification hinders reachability Despite the improvement in reachability achieved through steering, reducing the specification of captions during training re- sults in a rapid decrease in reachability. Figure 6a shows average accuracy falls below 0.30 for models trained with fewer than two concepts specified. Classifying the outputs after prompting and steering reveals that models tend to generate concepts accurately for the factors specified during training, but fail to correctly generate the missing concepts. An example for one random seed is presented in Figure 6b, When prompting is used, the model's 's accuracy on specified labels is high and on unspecified labels is nearly equivalent to chance-level performance. In contrast, steering yields higher accuracy on the full target concept combination, al- though remaining close to the prompting baseline. These findings show captions hold a pivotal role in organising and structuring a model's latent space, providing a semantic framework that enhances its ability to access concepts. Our results demonstrate that more detailed labeling of training images can improve the success rate of generating target con- cept combinations. On the contrary, given a trained model, overly detailed prompts may not lead to an improvement of reachability.
0.25
0.00
0.00
0.02
0.04
0.06
0.08
Px(C2)
S2 square
1.00
Accurcyy
0.75
0.50
Pouap: spoc
0.25
0.00
0.04
0.06
0.08
px(52)
Figure 5: Accuracy of prompting and steering on the prompt space and h-space to (red, triangle, green, square) for starting prompt ys = "a red triangle behind green square" and varying proportion px of images in the dataset contain- ing the concepts c1 = red (top) and $2 square (bottom) across the dataset. The dashed red line marks the approxi- mate threshold 0.01 of the shift in reachability.
ate reachability, Notably, when reducing [s2 - square|x steering on the prompt space achieves more than 50% ac- curacy, even when no training images contain the reduced concept. This indicates that when prompting fails, alterna- tive mechanisms may enable models to generalise concepts to unseen positions.
We further analyse reachability when using ye containing the full semantic information of the target combination (ci, 81, C2, 82) in Appendix E.4.
5.4. Biases
5.3. Underspecification of Captions
We measure reachability to individual concepts that are con- sistently paired during training. Specifically, we construct a dataset where an image in the training set satisfies c = blue if and only if s1 = circle. To evaluate the model's ability to disentangle these concepts, we measure reachability to concept combinations of the form (blue, 82,02, (2,22, 82). where s * circle, and (c1, circle, 52) where of * blue. Prompt space steering exhibited minimal variation across different starting prompts Ye. whereas on the h-space, start- ing with a prompt describing the target concepts of interest yielded the highest accuracy. We present this in Figure 7.
We reduce the factors of interest of the dataset, initially (ci, $1, C22 s2). For instance, removing the specification of Ci results in captions y containing information only with respect to ($1, C22 $2). See Appendix D for details. We then evaluate the reachability of (cI, 81, 02, 82) via prompting and steering.
We present the results of prompting when describing the full set of target concepts. For example, "a red triangle behind a green square". Steering is performed on top of starting prompts y, that reflect only the concepts seen dur- ing training. For example, when removing back colour (c1), steering is implemented from y, = "a triangle behind green square" to images with the concept combination (red, triangle, green,, square).
Biases in compositional OOD generalisation can be by- passed through steering When no train images contain only blue in the back, attempting to reach combinations where either concept appears separately places the model in a compositionally OOD setting. Figure 7 shows steering, particularly on the prompt space, achieves a moderately high reachability to both concepts (accuracy of black X' s)s), suggesting that models are capable, to some extent, of disen- tangling heavily biased concepts. Prompting fails to achieve this level of disentanglement, revealing its inability to accu-
Steering outperforms prompting under low specifica- tion In all cases of specification reduction, we observe an improvement in reachability with respect to prompting when using either steering method (Figure 6a). Among these, stecring on the prompt space achieves the highest reachability, demonstrating it a more effective approach
7

---

Concept Reachability in Diffusion Models: Beyond Dataset Constraints
Freeret Proonge spoce Pripace
0.4
Red
Square
Pronge Pronpe spece
Accucay
04
100.
100.
Poppece
0.50
a22
Cirele
(6, 05.40.5)
15.00
Iec. (0
B.ue
Label specification
TTiingle
(a)
(b)
S2, c22 92) for different levels of label specification. b) Outputs produced by a model when steering towards (red, circle, green, square) when the factors c1 (left) and $2 (right) are removed from the captions of the train set. Each diagram shows the target concept in the top axis, and the alternative concepts the removed factor can take in the remaining directions. The label X represents any other output, including those that do not produce the target concept for the seen factors. Each diagram displays the proportion of images of the output belonging to each label. A reachability method that correctly generates the target concept combination will produce a high proportion of images on the top axis. If a model only fails to generate the removed concept value, the proportion of images X will be low.
Figure 6: a) Average reachability across 10 randomly chosen concept combinations (c1, $1,
rately reflect model's full capabilities.
Furthermore, we conduct similar analysis to that in Sec- tion 5 using subset of the CelebA dataset. Results presented in Appendix G. Specifically, we use the labelled attributes for Gender (male/female) and Hat (wearing/not wearing a hat) as the concepts to be analysed. Overall, we again find that the structure of the training data strongly influences concept reachability. In particular, reachability declines sharply when the number of training examples con- taining given concept falls below certain threshold, and underspecified labels substantially hinder access to those concepts. Additionally, we observe that reducing biases in the co-occurrence of concepts improves a model's ability to represent and generate individual concepts in isolation. These findings are consistent with those observed in the syn- thetic setting, supporting the generality of these conclusions and highlighting their relevance in real settings.
Increasing the presence of an individual concept im- proves the separate reachability of both tied concepts As the number of images containing non-circular blue shapes increases, the reachability combinations only con- taining G1 blue rises rapidly, evidenced by the large increase in accuracy along the horizontal axis in Figure 7. This shows a similar threshold pattern to that observed in Section 5.2. Interestingly, the lighter X's on the vertical axis tend to achieve the highest accuracy, suggesting that the reachability of blu- circles in the back also increases, albeit with minor variability. This pattern implies that the model learns identify the contrasting concept, likely by inferring its definition through what it is not.
See Appendix E.5 for additional results.
6. Discussion and Conclusion
5.5. Reachability in Real Settings
This paper explores the limits of reachability in diffusion models, examining the challenges accessing target con- cepts. While prompting the default approach, our findings reveal that it often fails fully capture model's poten- tial for generating specific concepts. We demonstrate that steering can enhance the reachability of scarce concepts and disentangle biases inherent in datasets. We also identify key patterns in reachability such as shifts in a model's ability to access concepts and the critical role of proper concept specification in captions. This work provides a valuable framework for evaluating when concepts reachable, and offers insights into how can design train sets to improve
We additionally implement steering on the prompt space of Stable Diffusion in order to explore how reachability is impacted in a real-world setting. Figure 8 presents samples from the same initial random seed X7 as Figure 1, with additional steering implemented. The resulting images show clear improvement, more accurately representing the tar- get concepts. Further examples and analysis of reachability through steering are provided in Appendix F.3. Overall, we observe that steering can extend beyond the reachability of prompting. However, dataset limitations such as underspec- ification continue to hinder reachability, exhibiting patterns consistent with those observed in our synthetic analysis.
8

---

Concept Reachability in Diffusion Models: Beyond Dataset Constraints
Tied concepts: (c1 blue, S1 circle)
it. The two spaces also differ in dimensionality, and the degree of disentanglement of either space may also influence steering effectiveness. We note that, while steering on the h-space does not perform optimally for every choice of Vs. our results show an effective method when ys aligos with the target concept combination.
(-,cirele)
Increase proportion of images dataset contsining only blue in the back
fespain
An intriguing aspect of our experiments concerns a model's ability employ different mechanisms (compositional or positional) to achieve OOD generalisation. As presented in Section 5.2, models face significant challenges in OOD positional generalisation, with instances where reachability across all methods approaches zero. While biased composi- tional OOD generalisation remains a demanding task, we observe moderate levels of reachability using at least one of the methods studied (Section 5.4). Generalising shape posi- tionally from back to front is particularly interesting, it requires the model to synthesise information from multiple images to reconstruct complete, unobstructed shape. We detect no substantial variance in reachability when position- ally generalising in either direction, suggesting that models find both tasks equally demanding.
Pronitt space
10006
60022
(Alue,
Accurcy
Figure 7: Reachability to concept combinations containing non-blue circles in the back (vertical axis) and blue non- cireular shapes in the back (horizontal axis). Results eraged over 6 randomly chosen target concept combinations, with starting prompt Ys describing those concepts. The X' mark the accuracy of reaching either type of concept when prompting, steering on the prompt space and steering on the h-space. lighter colour represents stronger presence of images containing only blue in the back in the dataset.
Despite improvements in reachability, the steering meth- ods explored in this study rely on auxiliary collection of images 2 containing the target concept combinations, which in real data may be hard to acquire. Investigating alternative steering methods that mitigate this dependency and enhance reachability remains an important direction for future research. Additionally, there may exist simple or more efficient alternative methods to steering that outper- form existing approaches and warrant further exploration. Moreover, while our work isolates the impact of each ob- stacle studied, in real data it is likely that problems occur simultaneously and with varying degrees of complexity. Ex- panding our dataset to incorporate additional factors and more intricate relationships can provide a more comprehen- sive analysis of model behaviour.
the generative performance of diffusion models.
The improvement observed in reachability through steering suggests misalignment between the model's 's semantic and visual understanding of concepts. That is, a model may be able to reach a concept, but the prompt fails to access the correct spatial information required to generate it. Moreover, we observe differences in the accuracy of steering on the prompt space and steering on the h-space, which highlight the importance of the stage of the transformation which steering is performed. In particular, the nature of the space in which steering is implemented will impact reachability. For example, while the text encoding is independent of the timestep f, the bottleneck output (h-space) depends on
Acknowledgements
Additional steering on the prompt space
MAR is supported by the Department of Mathematics, Im- perial College London, through the Roth Scholarship.
Impact Statement
This paper focuses studying the robustness methods for model control. It holds direct value for model providers it highlights the delicacy of prompting and instead motivates the usage of alternative latent control methods.
image of beaker
of
minbow in black and while
walking left
Figure 8: Images sampled from the same random seeds used in Figure 1 with steering vectors added to the prompt space during sampling. Using steering, we observe: (L) a more accurate representation of a beaker, (C) a successful change in orientation of the person towards the left, (R) the rainbow generated in greyscale tones.
9

---

Concept Reachability in Diffusion Models: Beyond Dataset Constraints
References
Ghosh, D., Hajishirzi, H., and Schmidt, L. Geneval: An object-focused framework for evaluating text-to-image alignment, 2023. URL https: wttp/aawwi.......aa..aoo/.. 2310.11513.
Brokman, J.. Hofman, O., Vainshtein, R., Giloni, A., Shimizu, T., Singh, I.. Rachmil, O., Zolfi, A., Shabtai, A., Unno, Y., and Kojima, H. Montrage: Monitoring training for attribution of generative diffusion models. In Computer Vision ECCV 2024, pp. 1-17, Cham, 2025. Springer Nature Switzerland. ISBN 978-3-031-73226-3.
Haas, R., Huberman- Seaberman-Spiegggllas I., Mulayoff, R., Graßhoff S., Brandt, S. S., and Michaeli, T. Discovering inter- pretable directions in the semantic latent space of dif- fusson models, 2024. URL https::/arxiv..... abs abss2303..11 11073.
Chang, Y., Zhang. Y., Fang. Z.,, WU, Y. N., Bisk, Y., and Gao, F. Skews the phenomenon space hinder general- ization in text-to-image generation. In Computer Vision ECCV 2024, pp. 422-439, Cham, 2025. Springer Nature Switzerland.
Hammoudeh, Z. and Lowd, D. Training data influence analysis and estimation: survey. Machine Learning, 113(5)23311 March 2024. ISSN 1573-0565. doi: 10. 10.10077/1100944023-006655333559 URL /tsttp::// doi.
Clark, A. Pillow (pil fork) documentation, 2015. URL https: /bui dmeedia.... readthedocs org/ aeetia/pat/pilll//aatteeee
Hertz, A., Mokady, R., Tenenbaum, J., Aberman, K., compt-to-prompp im-
Pritch, Y., and Cohen-Or, control, 2022. URL age editing with cross attention
Cunningham, H., Ewart, A., Riggs, L., Huben, R., and Sharkey, L. Sparse autoencoders find highly interpretable features in language models, 2023. URL https:// arxiv. org/ abs 20o9/2309.00 08600.
https:/ attaaaaivvvv...........a .wwp/orgg/oooo.........oo.. abs 221./222..........
Ho, J., Jain, A., and Abbeel, P. Denoising diffu- sion probabilistic models. In Advances in Neu- ral Information Processing Systems, volume 33, pp. 6840-6851. Curran Associates, Inc., 2020. URL
Deng, J., Dong, W. Socher, R., Li, L.-J,, Li, K., and Fei-Fei, L. Imagenet: A large-scale hierarchical image database. In 2009 IEEE Conference on Computer Vi- sion and Pattern Recognition, pp. 248-255, 2009. doi: 10.1109/CVR.22009. 5206648..
https://proceedings neurips. cc cc/ppaper_fillesppperr///// (ccoffe63344r0006111111100033930 Pape.. pdf.
Deschenaux, J., Krawczuk, I., Chrysos, G. G., and Cevher, Going beyond compositions, ddpms can produce zero-shot interpolations.
Hu, E. J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y.. Wang, S., Wang, L.., and Chen, W. Lora: Low-rank adaptation of large language models, 2021. URL https: /tatpp:///rxivv.. //axxiv. org/ abs 21 2106.0..00 09685.
International Conference on Machine Learning, 2024. URL https: //api. semanticscholaan wtt..accccc....../rorp........ worg/CorrpusDD 270095412.
Huang, K., Sun, K., Xie, E., Li, Z. and Liu, X. T2i- comphench: comprehensive benchmark for open-world compositional text-to-image -mogmaae generation. arXiv preprint arXiv: 2307.06350 2023.
Epstein, D., Jabri, A., Poole, B., Efros, A., and Holynski, A. Diffusion self-guidance for controllable image generation. In Advances in Neural Information Processing Systems, volume 36, pp. 16222-16239, Curran Associates, Inc., 2023.
Ilharco, G., Ribeiro, M. T., Wortsman, M., Gururangan,
S., Schmidt, L., Hajishirzi, H., and Farhadi, A. Editing models with task arithmetic, 2023. URL https:// a arxiv org/ abs 2.12.00000 04089.
Gandikota, R., Orgad, H., Belinkov, Y., Materzyńska, J., and Bau, D. Unified concept editing in diffusion models, 2024.
Koh, P. W. and Liang, Understanding black-box predic- tions via influence functions. In Proceedings of the 34th International Conference Machine Learning Volume 70, ICML'17, pp. 1885-1894, JMLR.org. 2017.
URL https:tt/////m http:///xxxvv.vr... abs//308..... 14766.11466. 14761.
Gandikota, R., Materzyáska, J., Zhou, T., Torralba, A., and Bau, D. Concept sliders: Lora adaptors for precise control in diffusion models. In Computer Vision - ECCV 2024, pp. 172-188, Cham, 2025. Springer Nature Switzerland.
Kumari, N., Zhang, B., Wang. S.-Y., Shechtman, F.,, Zhang, R., and Zhu, J.-Y. Ablating concepts in text-to-image dif- fusion models. In International Conference Computer Vision (ICCV), 2023.
Georgiev, K., Vendrow, J., Salman, H., Park, S. M., and Madry, A. The journey, not the destination: How data guides diffusion models, 2023. URL https:/ /atpos///rxivvv org/ org/abs///e
Kwon, M., Jeong, J., and Uh, Y. Diffusion models already have a semantic latent space, 2023. URL https:/ a arxiv. www..orgaabb.. 22000.221011 10960.
abs 221.. 62005.
10

---

Concept Reachability in Diffusion Models: Beyond Dataset Constraints
Li, H., Shen, C., Tom, P., Tresp, V., and Gu, J. Self- discovering interpretable diffusion latent directions for re- sponsible text-to-image generation, 2024a. URL https: //arxivv org/ 2tab///311 17216.
Park, C. F., Okawa, M., Lee, A., Tanaka, H., and Lubana, E. S. Emergence of hidden capabilities: Exploring learn- ing dynamics in concept space. In Advances in Neural Information Processing Systems, volume 37, pp. 84698- 84729. Curran Associates, Inc., 2024.
Li, K., Patel, O., Viégas, F., Pfister, H., and Wattenberg, M. Inference-time time intervention: Eliciting truthfuul answers from a language model, 2024b. http::/:: org/ hts//bb//a0 2306. 6.00666600 03341.
Park, S. M., Georgiev, K., Ilyas, A., Leclerc, G., and Madry, A. Trak: Atuributing model behavior at scale, 2023, URL https: / www.vi.orrr/aaeeee 2303. 1 1411 4186.
Liu, X., Ji, K., Fu, Y.. Tam, W. L., Du, Z., Yang, Z., and Tang, J. P-tuning v2: Prompt tuning can be comparable to fine-tuning universally across scales and tasks, 2022. URL https://///... /aaariivvvorrro.//a.a orgg abs 2tt//211..00666 07602.
Pruthi, G., Liu, F., Sundararajan, M., and Kale, Estimat- ing training data influence by tracing gradient descent. In Proceedings of the 34th International Conference on Neural Information Processing Systems, NIPS '20, Red Hook, NY, USA, 2020. Curran Associates Inc. ISBN 9781713829546.
Liu, Z., Luo, P., Wang, X., and Tang, X. Deep learning face attributes in the wild. In Proceedings of International Conference on Computer Vision (ICCV), December 2015.
Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., Krueger, G., and Sutskever, Learning trans- ferable visual models from natural language supervi- sion. In Proceedings of the 38th International Con- ference Machine Learning, ICML 2021, 18-24 July 2021, Virtual Event, volume 139 of Proceedings of Ma- chine Learning Research, pp. 3768. PMIR, 2021. URL http: htttt::ppproeeee /ptpp/eeoceeenseeeeeeeee mlr paala.press/v1131 radford21a. html.
Llera Montero, M., Ludwig, C., Ponte Costa, R., Malho- tra, G., and Bowers, J. The role of disentanglement in generalisation. In International Conference on Learning Representations, 2021.
Marks, S. and Tegmark, M. The geometry of truth: Emer- gent linear structure in large language model represen- tations of true/false datasets, 2024. URL https:// arxiv. org/ / abs /abc. 06824.
Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., and Liu, P. J. Exploring the limits of transfer learning with unified text-to-text transformer. J. Mach. Learn. Res., 21(1), January 2020. ISSN 1532-4435.
Meng, K., Bau, D., Andonian, A., and Belinkov, Y. Locating and editing factual associations in gpt. In Proceedings of the 36th International Conference on Neural Information Processing Systems, NIPS '22 Red Hook, NY, USA, 2022. Curran Associates Inc. ISBN 9781713871088.
Ramesh, A., Dhariwal, P., Nichol, A., Chu, C., and Chen, M. Hierarchical text-conditional image generation with clip latents, 2022. URL htips: /awpxiixvvv.//////// org /bs/ 2204. 06125.
Mészáros, A., Ujváry, S., Brendel, W., Reizinger, P., and Huszár, F. Rule extrapolation in language modeling: A study of compositional generalization on ood prompts. In Advances in Neural Information Processing Systems, volume 37, pp. 34870-3498 -4899. Curran Associates, Inc., 2024.
Rombach, R., Blattmann, A., Lorenz, D.,, Esser, P., and Ommer, B. High-Resolution Image Synthesis with Latent Diffusion Models. In 2022 IEEE/CVF Conference Computer Vision and Pattern Recognition (CVPR), Los Alamitos, CA, USA, June 2022. IEEE Computer Society. doi: 10. 11019///VPR222682222222020111100 h 10. N0.1109//CPP522688 2022.01042.
Nichol, A., Dhariwal, P., Ramesh, A., Shyam, P., Mishkin, P., McGrew, B., Sutskever, I., and Chen, M. Glide: Towards photorealistic image generation and editing with text-guided diffusion models, 2022. URL https: //arxivv.rrg/ abs/ 11741.
URL org/
Okawa, M., Lubana, E. S., Dick, R., and Tanaka, H. Com- positional abilities emerge multiplicatively: Exploring diffusion models on synthetic task. In Advances in Neural Information Processing Systems, volume 36, pp- 50173-50195, Curran Associates, Inc., 2023.
Ronneberger, O., Fischer, P., and Brox, T. U-net: Con- volutional networks for biomedical image segmentation. In Medical Image Computing and Cosputer-Assisted In- tervention MICCAI 2015, pp. 234-241, Cham, 2015. Springer International Publishing. ISBN 978-3-319- 24574-4.
Panickssery, N., Gabrieli, N., Schulz, J., Tong, M., Hub- inger, E., and Turner, A. M. Steering llama 2 via contrastive activation addition, 2024. URL https: //arxiv. //aaxiivwrg//aeeeeeee.//..ooooet www.orggbbbb.. httg/abs//20 22122 06681.
Samuel, D., Ben-Ari, R., Raviv, S., Darshan, N., and Chechik, G. Generating images of rare concepts us- ing pre-trained diffusion models. In Proceedings of

---

Concept Reachability in Diffusion Models: Beyond Dataset Constraints
the Thirty-Eighth AAAI Conference on Artificial Intel- ligence and Thirty-Sixth Conference on Innovative Appli- cations of Artificial Intelligence and Fourteenth Sympo- sium on Educational Advances in Artificial Intelligence.
Turner, A. M., Thiergart, L.., Leech, G., Udell, D., Vazquez, J. J., Mini, U., and MacDiarmid, M. Steering language models with activation engineering, 2024, URL https: / //arxiv..../////////////e//a org/ abs / 2200. 10248.
AAAI Press, 2024. ISBN 978-1-57735-887-9 doi:
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, N., Kaiser, u., and Polosukhin, I. Atten- tion is all you need. In Advances in Neural Information Processing Systems, volume 30. Curran Associates, Inc., 2017. URL https::/prrcceeddngs neurips. cc oc//appe_fiiee/pppee//////// 3 1ffa22455776e11100053cc414444484 pdf.
10. ww..9/aaii..3...222222222 URL https::/tpo.otm //dddoiorrg) 10. 1 (0166009/aaa.... v18.5...85..... 28270.
Schuhmann, C., Beaumont, R., Vencu, R., Gordon, C., Wightman, R., Cherti, M., Coombes, T., Katta, A., Mullis, C., Wortsman, M., Schramowski, P., Kundurthy, S., Crow- son, K., Schmidt, L., Kaczmarczyk, R., and Jitsev, J. Laion-5b: an open large-scale dataset for training next generation image-text models. In Proceedings of the 36th International Conference on Neural Information Process- ing Systems, NIPS '22, Red Hook, NY, USA, 2022. Cur- ran Associates Inc. ISBN 9781713871088
von Platen, P., Patil, S., Lozhkov, A., Cuenca, P., Lam- bert, N., Rasul, K., Davaadorj, M., Nair, D., Paul, S., Berman, W., Xu, Y., Liu, S., and Wolf, T. Diftusers: /github. State-of-the-art diffusion models. 2022.
Scimeca, L., Oh, S. J.. Chun, S., Poli, M., and Yun, S. Which shorteut cues will dnns choose? study from the parameter-space perspective, 2022. URL https: //arxiv. org/ abs/ / 2210..1111... 03095.
Wang, S.-Y., Efros, A. A., Zhu, J.-Y,, and Zhang, R. Eval- uating data attribution for text-to-image models, 2023, URL https:://////... //apiii:///xx....//////a//.mm
httpv.xiivvv..o//..//aaa wwwpsiiv.oorrooooooooooooo abs 2200. 09345.
Sohl-Dickstein, J., Weiss, E. A., Maheswaranathan, N., and Ganguli, S. Deep unsupervised learning using nonequilibrium thermodynamics, 2015. URL https: //arxiv..o //aaxxivorrg/aaaaaeeeeeaaaa hbs///ab//// 111./ 03585.
Wang, Z., Gui, L., Negrea, J., and Veitch, V. Concept algebra for (score-based) text-controlled generative mod- els, 2024. URL https::: /atpivv./iivvvv/aa org/ abs/ 03693.
Song. Y. and Ermon, S. Generative modeling by es- timating gradients of the data distribution. CoRR, abs/1907.05600, 2019. URL Abbs/190..0560
Wiedemer, T., Mayilvahanan, P., Bethge, M., and Bren- del, W. Compositional generalization from first princi- ples, 2023. URL httpt:ss: /ttps:///axiiii /awxiv.ivivv/////////a/////aaava org/ abs/2307. 05596.
Song, Y., Sohl-Dickstein, J., Kingma, D. P., Kumar, A., Ermon, S., and Poole, B. Score-based generative model- ing through stochastic differential equations, 2021. URL
Wu, Z., Arora, A., Wang, Z., Geiger, A., Jurafsky, D., Manning, C. D., and Potts, C. Reft: Representation finetuning for language models, 2024. URL https: farxivvorrge abs////////// /2404 03592.
org/ abs 2011 13456.
Tan, D., Chanin, D., Lynch, A., Kanoulas, D., Paige, B., Garriga-Alonse A., and Kirk, R. Analyzing the gener- alization and reliability of steering vectors, 2024. URL https: / arxiv. org/ abs/ / 2407. 12404.
Xie, T., Li, H., Bai, A., and Hsieh, C.-J. Data attribution for diffusion models: Timestep-inducet bias in influence estimation, 2024. URL https: /atp:://axxvvvoi org. abs/ 2401. 09031.
Templeton, A., Conerly, T., Marcus, J., Lindsey, J., Bricken, T., Chen, B., Pearce, A., Citro, C., Ameisen, E., Jones, A., Cunningham, H., Turner, N. L., McDougall, C., MacDiarmid, M., Freeman, C. D., Sumers, T. R., Rees, E., Batson, J., Jermyn, A., Carter, S., Olah, C., and Henighan, T. Scaling monosemanticity: Ex- tracting interpretable features from claude 3 sonnet. Transformer Circuits Thread, 2024. URL https:
Zaken, F. B., Ravfogel, S., and Goldberg, Y. Bitfit: Simple parameter-effficiet fine-tuning for transformer- based masked language lunguage-models, models, 2022. URL https: //arxiv. org abs /aabbs///22.... 10199.099......
Zhao, S., Ren, H., Yuan, A., Song. J., Goodman, N., and Ermon, S. Bias and generalization in deep generative models: An empirical study. In Bengio, S., Wallach, H., Larochelle, H., Grauman, K., Cesa-Bianchi, N., and Garnett, R. (eds.), Advances in Neural Information Processing Systems, volume 31. Curran Associates, Inc., 2018. URL httpps:/ nww.ips...pp. cc/paper_files/// /cc/ypper_.ffleeppppr////////111
pub/2024/ sealiinggmmoosmmmnttiiiiyyyyy// Lnw.:///awannntiiiiiiennee...... html.
Todd, E., Li, M. L., Sharma, A. S., Mueller, A., Wallace, B. C., and Bau, D. Function vectors in large language models, 2024. URL https:/ // /arxiv...v./ org/ abs/ 2310.15213. 15213.
12

---

Concept Reachability in Diffusion Models: Beyond Dataset Constraints
55776679988811186008444488888813 Paper.
pdf.
Zheng, X., Pang, T., Du, C., Jiang, J., and Lin, M. Intriguing properties of data attribution on diffusion models, 2024. URL https: htthtt:///axiie wrw. http:/bb//11...//111 00500.
13

---

Concept Reachability in Diffusion Models: Beyond Dataset Constraints
A. Hyperparameters
A.I. Architecture
For our experiments, we used the Diffusers package (von Platen et al., 2022) to implement the U-net architecture. Our model comprises 3.7 million parameters and features symmetric architecture with four downsampling and upsampling blocks. The channels in each block set to (16, 32, 64, 128), respectively. Each block contains two ResNet blocks.
Additionally, incorporate cross-attention layers in the midblock, enabling the model to integrate information from text-encoding input. For processing the text prompt, we use pre-trained T5Small text encoder (Raffel et al., 2020). The encoder's 's weights are frozen throughout training of the U-net and during the optimisation of concepts vectors.
A.2. Training
Training of the U-net is performed for 70 epochs using Adam with learning rate 0.001 and default parameter values. Additionally, we use an exponential learning rate scheduler with parameter gamma = 0.98. All models are trained using = 1000 and sampled with DDPMScheduler at inference time.
A.3. Concept Vector Optimisation
Concept vectors are initialised at the vector,ctor, and optimised for 5000 steps using Adam, with learning rate 0.02 and default parameter values. This choice observed to be sufficient for the optimised losses Lp and La to converge. A diagram of the different spaces in which the optimisation is implemented is shown in Figure
For steering on the prompt-space, Vp of the same dimensionality as the input prompt ys- In the original dataset with captions "a {ci (s1) behind {02} {s2), this accounts for vp IR1x10x512 and is adjusted for the experiments in Section 5.3.
The dimension of the vectors optimised in the bottleneck of the U-net is Vn E R 1288888 for all experiments.
red circle behind
ys
green square
X =
X
A
Vp
Cross-attention layers in U-net
B
Vh
U-net
Figure 9: Diagram of spaces in the architecture where steering is implemented. A: prompt space, the concept vector is added to the encoding for the text prompt before passing through the cross-attention layers of the U-net. B: ft-space, the concept vector is added to the bottleneck layer the U-net, after the mid-block.
14

---

Concept Reachability in Diffusion Models: Beyond Dataset Constraints
B. Dataset Creation
Images are of dimension 64 x 64 pixels, comprised of two coloured shapes. To create the complete dataset, we generate 1000 images with each of the combinations of the concepts of interest (ci $1, C22 82). Each image created using the Pillow package in Python (Clark, 2015). First, the location of the centre of the back shape determined, and then the position of the front shape is sampled uniformly from neighbourhood of this centre. This neighbourhood ensured reasonable portion of the back shape was always visible for every pair of shapes (thus, facilitating the classification task). The minimum percentage of each back shape visible in the balanced train set for the diffusion models is the following: circle 52.46%. triangle 48.97%, square 59.38%. Examples of train images are presented in Figure 10.
a blue circle behind a rod square
blue square behind green square
green circle behind rod triangle
green square behind red square
blue square behind green cirele
Figure 10: Examples image-caption pairs from our synthetic dataset. Each image shows two shapes of different colours, one partially covered by the other, on a black background.
When reducing the size of some of the combinations of the dataset, the size of the remaining combinations is proportionally increased to ensure that the total dataset size is as close as possible to 54,000 while maintaining uniformity (equal number of images) among the unaffected concept combinations.
C. Classifier and Evaluation Details
We train three different classifiers for identifying the back shape, front shape and back-front colour pairs. The architecture consists of two convolutional layers that increase channel size to 16 and 32 respectively. Both consist of 3 3 kernels with ReLU activation and 2 x max-poolia max-pooling. The flattened output is passed to fully connected layer of 128 units and ReLU, The output vector is three-dimensional in the case of both shape classifiers, and consists of six dimensions for the classifier of back-front colour pairs.
Each classifier trained using Adam with learning rate 0.001 on a dataset comprised of the original 54,000 balanced dataset and equal number of sampled images from trained diffusion models. Training implemented for epochs. When evaluating on 5400 human-labelled images that were generated by prompting from two additional diffusion models trained on balanced data, the classifier obtains an accuracy of 96.63%.
During evaluation of reachability, 100 images are sampled using a reachability method. Each of these labelled according to the outputs of the trained classifiers. Images are accounted for as correct the labels match the target concept combination. In cases where classifier results were uncertain or ambiguous, additional human evaluations were conducted, Furthermore, identify the number of (non-black) colours in an image, and if this number if distinct from 2, we account for the images incorrect. After this, the proportion of correct images is used the reachability value.
D. Reduction of Label Specification on Captions
Our dataset is originally comprised of captions of the form "a {cr {si} behind {c2} {s2)". order to reduce the semantic information of the prompt, we implement the following:
Remove ei: replace the caption with "a {a1] behind (62) {82]"
2. Remove #1: replace the caption with "a {ci shape behind {c2) (82)"
3. Remove Oz: replace the caption with "a {cr} {82) behind {82]"
15

---

Concept Reachability in Diffusion Models: Beyond Dataset Constraints
4. Remove 82 replace the caption with "a { {*2) behind {c2} shape"
5. Remove e and s1; replace the caption with "a {c2] {52]"
6. Remove C1 $1 and O2: replace the caption with "a {s2}"
7. Remove C1, $1, Og and S2: replace the caption with the empty string,
E. Additional Experiments
E.1. Additional Analysis on Balanced Data
Final norm of optimised concept vector is indicative of reachability when steering on the h-space The differences in the accuracies achieved through steering on the h-space suggest that, from given starting prompt ys. certain concept combinations are inherently more reachable than others. While the final loss the end of the optimisation of the steering vector might be considered natural indicator of reachability via steering (lower loss associated with higher reachability), we observe cases where high final loss values correspond to high steering accuracies (Figure 11, teal x x's). Our findings indicate that the final vector norm provides a more reliable indicator, with a larger norm being associated with lower reachability (Figure 11, bottom right). Interestingly, when sampling, adding a larger concept vector norm indicates a greater deviation from the latents that would be obtained from solely prompting ys- Hence, we conclude that reachable concepts remain close in the f--pacc. Steering on the prompt space does not exhibit such patterns (Figure 11, left).
Prompt space
h space
1.00
1.0
Seed 0
Seed 1
Accuracy
0.95
Accuracy
Seed 2
0.5
Seed 3
0.90
0.85
0.0
10
15
20
10
15
20
Loss
Loss
1.00
1.0
Accuracy
0.95
Accuracy
0.90
0.85
0.0
100
1000
1250
Norm
Norm
Figure 11: Relation between accuracy of steering on the prompt space and h-space and final loss (top row) and final vector norm (bottom row) after the optimisation of the concept vector. The different colours show the results for four different models trained on different random seeds.
E.2. Reachability When Removing One Concept Combination from the Dataset
We test the ability of a model to generalise compositionally OOD in the simplest scenario, where only the target concept combination is removed from the dataset. Three examples of this are presented in Figure 12. Our results highlight the variability of accuracy of steering the h-space, also observed in Section 5.1. Moreover, we observe that reachability is generally high and unaffected from the removal of the target combination. Prompting achieves the highest reachability, with steering on the prompt space also providing similar accuracy values. Thus, models are capable of generalising OOD for simple compositional concept combinations based on their ability to recombine learned individual concepts. We note that this is not necessarily extensible to any compositional OOD generalisation, as our experiments in Section 5.4 provide a
16

---

Concept Reachability in Diffusion Models: Beyond Dataset Constraints
complex compositional OOD generalisation task, where models are affected by the biases of combinations seen during
training.
(green, triangle, blue, square)
(red, circle, blue, triangle)
(red, square, green, circle)
1.0
1.0
10
0.8
0.8
0.8
Accuracy
0.6
Accuracy
0.6
Accuracy
0.6
Prompl
0.2
0.2
0.2
Prompt
h-space
0.0
0.0
0.0
a groen trinngle beland blue aquare
rod triangle belandd blue squarre
a groen triangle bahind a cirole
red circle behind bloe triongle
axed cirele belind a green trinugle
sed circlo behind blue trinngle
green cine behind blue triangle
asd cascle behand groes aquare
green triangle behind hine circle
a green square behind blue citele
aed aquare bebiidd a green circle
axdd squaro behind blue trinngle
a blue circle behind green squaree
green tilaglle beluid blas aquare
bine square behind green
Figure 12: Steering accuracies for different starting prompts Ys to (green, triangle, blue, square),
(red, cirele, blue, triangle) and (real square, green, circle) respectively. For each graph, the target concept combination the only concept combination not present the training dataset.
F.3. Additional Results for Scarcity of Concepts
In this section we present additional results for Section 5.2. Figure 13 shows reachability results for different starting prompts y, when reducing [c, red]x and [s₂ square) As observed the baseline analysis, steering on the prompt space presents a more stable outcome, while steering on the h-space shows greater variability. The highest reachability values on the h-space achieved on the starting prompt y. "a red triangle behind green square", that mentions the same concepts those in the target combination. Note that despite the prompt "a red triangle behind a green circle" achieving comparable reachability in the case of reduction of C1, this is not consistent in the reduction of the presence of other concepts.
We also present the effect decreasing the size of the subsets of the dataset [81 triangle)x. (c2 = green) x Figure 14 shows similar sudden decrease in reachability, observed Figure Moreover, we highlight the improvement of reachability by steering on the prompt space over prompting below the threshold, most noticeably when reducing the concept $1 triangle.
E.4. Additional Results for Underspecification
In this section we present additional results for analysing the effect of varying the number of label specification on reachability (Section 5.3). We implement steering from the same 10 randomly chosen target concept combinations and trained models as in Figure 6a, but with starting prompt containing the full semantic information "a {cr ) (81) behind
{oo} { This approach ensures that the size of the vector Vp used for steering on the prompt space remains constant for all the models, regardless the specification level used during training. Figure 15 demonstrates that the reachability results show almost no difference to those in Figure 6a, indicating that models are robust to variations of the size of vp when steering on the prompt space.
17

---

Concept Reachability in Diffusion Models: Beyond Dataset Constraints
Prompt space
h h-space
1.0
1.0
Accuracy
Accuracy
0.2
0.2
0.0
0.0
0.075
0.125
Px(C1)
Px(c1)
1.0
1.0
0.8
0.8
.....
Prompe
Accuracy
red triangle behand
Accuracy
y, blae triangle belind green square
y; a triangle behand groen circle
y/ a blae circle bebiod green square sed triangle behand bluc tinngle
0.2
preen tringle belund square
0.2
y, a square behind green circle
0.0
0.0
0.000
0.025
0.050
0.075
0.100
0.125
0.150
0.000
0.025
0.050
0.075
0.100
0.125
0.150
0.175
Px(52)
Px(S2)
Figure 13: Accuracy of prompting and steering on the prompt space and h-space for different starting prompts and varying level of the proportion px of images in the train set containing the concepts red (top) and square (bottom) across the dataset. The target concept combination for all ys is (red, triangle,green, green, square).
51 triangle
C2 green
Accuracy
Accuracy
0.50
... Protupt
space hapoce
0.00
0.00
0.02
0.00
0.02
0.04
0.06
0.08
px(ss)
Px(C2)
Figure 14: Accuracy of prompting and steering the prompt space and h-space to (red, triangle, green, square) for starting prompt y, "a red triangle behind a green square" and varying the proportion px of images in the training set containing the concepts $1 = triangle (left) and C2 = green (right) respectively. The vertical red line marks the approximate threshold 0.01 that determines the approximate shift reachability.
18

---

Concept Reachability in Diffusion Models: Beyond Dataset Constraints
1.0
Prompt
Prompt space
h-space
Accuracy
0.6
0.4
0.2
(G.$1,
(51,C,.82
(C1, (C1. C2,52,
(C1.$1.52
(11.$1
(C2,52)
(53
Label specification
Figure 15: Average reachability across 10 randomly chosen concept combinations (c1, $1, C2, 82) and varying levels of specification in the captions. Starting prompts ys used to steer to each of the 10 concept combinations are chosen to fully deseribe the target concept combinations.
E.5. Additional Results for Biases
Figure 16 provides an additional example of the scenario studied in Section 5.4. particular, we tie the concepts c2 red and $2 = triangle, and gradually increase the images in the train containing non-triangular red shapes. Similar to Figure 7, we observe sharp increase in reachability for red shapes as their representation in the dataset grows.
Additionally, although with more variability than Figure 7, we note a general improvement across all reachability methods on the remaining tied concept (non-red triangles). This suggests that the model learns to disentangle the concept 82 = triangle as the presence of C2 = red increases. Moreover, steering consistently outperforms prompting in disentangling the concept 82 triangle, demonstrating it is more effective reachability method in such biased scenarios.
F. Stable Diffusion Implementations
F.1 Hyperparameters
All experiments real data are conducted on Stable Diffusion v1.5. We use DDPM scheduler with T 1000 inference steps to generate images.
Steering is implemented on the prompt space, with ys describing the target (e.g., "an image of a beaker", "an image of a person walking left"), and a collection 2 of 200 images containing the correct target concepts. The concept vector initialised at the zero vector and is optimised using Adam for steps. The images required for steering are obtained from openly available datasets such ImageNet (Deng al., 2009) and images sampled from Stable Diffusion and DALLE (Ramesh et al., 2022).
E.2. Train Dataset
Stable Diffusion is primarily trained subsets of the LAION5B and LAION2B-en datasets (Schuhmann et al., 2022). To estimate the presence of concepts within the dataset, we use unigram and bigram frequencies of images labeled with English captions (LAION2B-en), as provided by Samuel et al. (2024).
The approximate number of captions containing the word beaker is 52,000. Despite beakers being easily recognisable and composed simple visual features, the model struggles to generate them, as shown in Figure We hypothesise that this due to the relatively low number images containing beakers compared to the overall dataset size.
19

---

Concept Reachability in Diffusion Models: Beyond Dataset Constraints
Tied concepts: (c2 = red, $2 = triangle)
(- triangle)
0.2222
XO
Increase proportion of images dataset containing only only the front
88
Accuracy
66
-0.1111
047
Prompt space
14644
0.0556
0.0222
Mete
(red,
0000
0,0000
0.2
Ascurey
66
Figure 16: Reachability to concept combinations containing only non-red triangles in the front (vertical axis) and only red non-triangular shapes in the front (horizontal axis), by prompting, steering the prompt space and steering on the h-space. Results are averaged over multiple 6 randomly chosen target concept combinations. When steering, the starting prompt y, describing those same concepts is used.
The bigrams walking left and walking right appear only 431 and 585 captions, respectively, whereas the unigram walking appears in over 3,450,000 captions-a significantly higher occurrence. Despite this imbalance, the model occasionally generate images of people facing either left or right, suggesting inherent understanding of orientation. However, we hypothesise that the lack of explicit directional specification in most captions limits the model's I's ability to reliably generate images of people walking in specific direction.
Finally, the model struggles to generate black-and- images of rainbows, despite successfully handling black-and- white colour representation in other contexts. We hypothesise that this stems from the scarcity of black-and-white rainbow images in the dataset, making it difficult for the model disentangle colour information from the broader concept of a rainbow, ultimately revealing bias in the model's latent space.
F.3. Steering on the Prompt Space
We compare the images generated through sampling using prompting and steering on the prompt space, ensuring both methods the same initial random seed. This evaluation conducted 50 images per studied concept. Below, present seven examples illustrating the impact of steering the final output.
1. SCARCITY OF CONCEPTS
Figure 17 illustrates the effect of steering on image generation towards the target concept: a beaker. We observe that some random seeds initially generate images unrelated to the beaker concept but are successfully guided towards it. contrast, cases where the model already produces a beaker show minimal modification. Overall, the steered images exhibit attributes commonly associated with beakers, such transparency, cylindrical shape, measurement markings, and the presence of liquid.
F.3.2. UNDERSPECIFICATION OF CAPTIONS
Figure 18 illustrates the effect of steering towards generating images of a person walking leftward. Our results indicate that steering in this scenario is limited. In many cases, applying steering on the prompt space does not significantly alter the individual's position compared to images generated without steering. While we occasionally observe improvements in no such the walking direction towards the left, we also find instances where the position shifts towards the
20

---

Concept Reachability in Diffusion Models: Beyond Dataset Constraints
No steering
Steering
Figure 17: Comparison between sampling on Stable Diffusion with no steering (top row) and with additional steering on the
prompt space (bottom row) by using the same random seed. Images are sampled from the prompt "an image of beaker". The steering vector optimised between the same starting prompt ys "an image of beaker" and 200 images of beakers.
occurrences the images used to optimise the steering vector. These findings suggest that reachability is highly constrained when attempting steer towards concepts that not explicitly mentioned in the captions. This observation aligns with discussion in Section 5.3.
No steering
Steering
Figure 18: Comparison between sampling on Stable Diffusion with no stecring (top row) and with additional steering on the prompt space (bottom row) using the same random seed. Images are sampled from the prompt "an image of person walking left". The steering vector is optimised between the starting prompt y/ "an image of a person walking left" and 200 images containing individual walking towards the left side of the image frame.
F.3.3. BIASES
Figure 19 illustrates the behaviour of images when steered towards disentangling rainbows from colour. We observe cases where the model reduces the presence of the rainbow, fading the colours from the image, and at times completely removing it. Other images result in an arc-like black-and-whiee pattern. Interestingly, some steered images resemble black-and-wiln woodgrain patterns, suggesting potential bias in the latent space. This may arise because both rainbows and woodgrain share structure of contrasting light and dark lines, which the model's latent representations might conflate. Finally, in other instances steering proves ineffective, and the model continues to generate images featuring coloured arc.
21

---

Concept Reachability in Diffusion Models: Beyond Dataset Constraints
No steering
Steering
Figure 19: Comparison between sampling on Stable Diffusion with no steering (top row) and with additional steering on the prompt space (bottom row) using the same random seed. Images are sampled from the prompt "an image of a rainbow in black and white". The steering vector is optimised between the same starting prompt y. "an image of rainbow black and white" and 200 greyscale images containing scenes of rainbows.
G. CelebA
We conduct a similar analysis to that presented in Section 5 using synthetic data, this time using a subset of 16,000 images of faces from the CelebA dataset (Liu et al., 2015). Leveraging the dataset's 's existing attribute labels, we assume that the images are generated by underlying factors that take on specific values (concepts), following the structure outlined in Section 3.2. We note that certain factors and biases present in the dataset may remain unaccounted for, as attributes such as posture or lighting are labelled in the dataset. Furthermore, the positional relations investigated in our earlier experiments that support positional generalisation observed in synthetic data. Further challenges may also arise from the use of more entangled and context-dependent words in comparison to the simpler captions in the controlled synthetic setup. Nevertheless, this analysis remains insightful for understanding the model's behaviour in more realistic settings.
not applicable in this context, which may cause deviations from the behaviours
G.1. Hyperparameters
We study the attributes gender (g E (man, woman)) and hat (h E (wearing a hat, without hat))) Captions are composed in the form "a {g} {h}". For example, "a woman wearing a hat", "a without hat". The balanced dataset is comprised of 4,000 images of each of the four possible concept combinations. The images are transformed using random horizontal flip (p = 0.5) and colour jitter with brightness 0.1, contrast 0.1 and saturation 0.1. As in our previous experiments, when varying the concepts in the train set we approximately preserve the total dataset size. The model architecture the same as described in Appendix A. We train our models for 400 epochs using the same optimiser and learning rate as before.
The concept vector for steering on the prompt space and h-space is obtained using Adam with learning rate 0.02 for 11 steps, and 100 images containing the target concept combinations. To evaluate reachability, train 4 diffusion models with different random seeds, use them generate 100 images of the target concept combinations and report the mean results. To help in the evaluations of the generated images, we train two CNNs consisting of three convolutional layers and two linear layers to classify (i) the gender of the person in the image and (ii) if they are wearing a hat or not. Our classifiers obtain an accuracy of 94.6% (gender) and 97.22 97.2% (hat) on a held-out validation set consisting of 2,400 images.
G.2. Baseline
Fixing starting prompt we apply steering towards the different target concept combinations. For comparison, we also evaluate the reachability achieved through direct prompting of the target combinations. The results are presented in Figure 20a. Additionally. Figure 20b illustrates a comparison images generated by the model under no steering (prompting Vs ='a woman wearing hat"") and with additional steering towards the concept combination (man, without a hat).
22

---

Concept Reachability in Diffusion Models: Beyond Dataset Constraints
woman wearing a hat
No steering
Steering
10
Wi
M.H
N
W.
M.H
Accireyy
0.6
0.2
Prempt space
h-spacs
0.0
0
Number concepts changed
(a)
(b)
Figure 20: a) Reachability different concept combinations when prompting and when steering from the starting prompt y/ ='a woman wearing a hat". Target combinations are shown at the top of each bar, and are organised according to the number of concepts that differ from the concepts of The notation M/W refers to the gender (man/woman) and H/H' to wearing hat/without a hat. b) Example of images generated from the prompt "a woman wearing a hat" (no steering) and the images obtained by additionally steering on the prompt space to the concept combination (man, without hat).
Concepts remain reachable when steering from diverse starting prompts Figure 20a demonstrates that the behaviour observed under balanced conditions aligns with the patterns identified in the synthetic setting. Specifically, while concepts generally remain reachable through prompt-space steering, reachability in the h-space declines as the number of altered concepts increases. Overall, steering enables effective access target concepts, achieving level of reachability comparable to that of directly prompting the desired concepts. Notably, we observe systematic decrease in reachability for the concept combination (woman, wearing hat) relative to the other combinations, which may reflect latent biases in the training data distribution.
G.3. Scarcity of Concepts
We decrease the presence of the concept wearing hat, and consider reachability to the concept combination (man, wearing a hat). Results are presented in Figure 21.
h wearing a hat
1.00
Accuracy
0.75
0.50
0.25
Pronpe
Steesing
0.00
0.00
0.02
0.04
0.06
0.08
0.10
0.12
Px(h)
Figure 21: Accuracy of prompting and steering from the starting prompt y. = "a man wearing hat" for a varying proportion Px of images across the dataset containing the concept h wearing a hat. The steering curve shows the results for max (prompt space, h-space). The dotted red line marks the approximate threshold 0.01 of the shift in reachability.
Reachability drops sharply past a critical threshold Figure 21 reveals threshold pattern consistent with that observed in Figure Specifically, we observe a sharp decline in reachability once the proportion of training images containing a given concept falls below a certain threshold. As in the synthetic setting, the model appears able to learn and generalize a concept even from small number of examples, provided this minimal threshold is met. This highlights the importance of
23

---

Concept Reachability in Diffusion Models: Beyond Dataset Constraints
ensuring minimal representation of key concepts in training data ensure reachability.
G.4. Underspecification of Concepts
We vary the level specification of captions the training dataset, and in particular reduce the captions of the form {g}
{h}" to the following:
1. Remove gr replace the caption with "a person {h)"
2. Remove h: replace the caption with "a {g}
3. Remove g and h: replace the caption with the empty string,
We steer from the starting prompt describing only concepts. For example, when removing à from the captions, to steer to (woman, wearing hat) we use 1/s "a woman", Additionally, compare the accuracy of prompting using the complete description of the target concepts. Results are presented in Figure 22a
1.0
Prompt
0.8
Prompt space
0.8
(man, without hat)
h h-space
Prompt
Accuracy
0.6
Accuracy
0.6
Steering
0.4
0.4
(woman, wearing hat)
0.2
0.2
Prompt Steering
0.0
0.0
(g, h)
(h) Label specification
(g)
0.0
0.1
0.2
0.3
Px(man, without a hat)
a
(b)
Figure 22: a) Average accuracy of prompting and steering to the different concept combinations for different levels of concept specification. b) Accuracy of prompting and steering to the concept combinations (man, without a hat) and (woman, wearing hat) under biased conditions. Starting prompts ys describe the target concept combinations, while varying the presence of the concept combination (man, without hat). The steering curve shows the results for max (prompt space, h-space).
Underspecification hinders reachability Similar to Section 5.3, we observe rapid decrease in the reachability of concepts. The reachability is observed to be close to the value of sampling the unspecified concepts randomly: when one factor is unspecified, reachability approximately 50%, and when both are unspecified, reachability approximately 25%.
G.5. Biases
We tie the concepts "man" and "wearing a hat" (which also causes the concepts "woman" and "without a hat" to be tied), and gradually increase the presence images containing the concept combination (man, without a hat), thus reducing the bias. We evaluate reachability to (man, without hat) and (woman, wearing hat). Results are presented in Figure 22b.
Increasing the presence of individual concept increases separate reachability to both concepts Figure 22b shows clear increase in reachability to either concept combination as the bias in the dataset is decreased. Most noticeably, we observe a sudden increase in reachability to (man, without a hat), similar to the threshold patterns previously observed. Moreover, the reachability to (woman, wearing hat) also gradually increases, suggesting that the models become more capable of disentangling concepts. We further note that this concept combination is consistently more reachable through steering.
24


Reviews:
[{"review_id": "7hz3tXebGZ", "reviewer": "Reviewer_7wCq", "summary": "This paper focuses on the limitations of prompting for model control. It shows how steering is a more robust mechanism to enhance concept reachability. The authors study three common dataset issues: concept scarcity, underspecification of captions, and biased co-occurrence of concepts. Experiments are evaluated mostly on a controlled synthetic dataset (colored shapes with specific positional relationships). Their findings demonstrate that steering vectors significantly improve concept reachability, particularly in out-of-distribution or underspecified conditions, and they support this with empirical studies including extensions to real data using Stable Diffusion.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for reading through our work and providing your feedback! We have read through your comments and address your questions below:\n___\n- **Notation of steering vectors**\n\nWe noticed that the notation $\\mathbf{s}$ may also lead to confusion due to the labels $s_1$ and $s_2$ for the factors in our dataset. However, we can change the notation for the steering vectors to $\\mathbf{v}$.\n- **Notation for $[f_{i_1}, f_{i_2}, \\dotsc, f_{i_k}]_X $ and $[f_1^{(i)}, f_2^{(j)}, f_3^{(k)}]_X$**\n\nThe notation **$[f_1^{(i)}, f_2^{(j)}, f_3^{(k)}]_X$** refers specifically to the diagram. This diagram contains only three factors, so the notation refers to the combination of images that contain one specific value or concept for each factor in the dataset. The notation $[f_{i_1}, f_{i_2}, \\dotsc, f_{i_j}]_X$ refers to the $n$-factor case, and only fixes the concept value for a subset of the factors, indexed as $i_1, i_2, \\dotsc i_j$. We will modify the indexing in the diagram to avoid confusion.\n- **Difference between $y_e$ and $y_s$**\n\n$y_e$ describes the properties of the images we steer to (the end target). $y_s$ is the prompt from which we start, that may (if $y_s = y_e$) or may not describe the desired properties. $\\mathbf{x}_0$ is the collection of images containing the target concepts that are used to steer the generation process (they may have been generated by sampling from a model using the prompt $y_e$ or obtained from a test set).\n- **Figure 4: why steer to such extreme combinations?**\n\nWe wanted to explore the behaviour in extreme modifications under balanced conditions in order to understand the different reachability methods. Perhaps it would be expected that as the number of factors changed (with respect to $y_s$) increases, reachability via steering decreased. However, this is not the case in steering on the prompt space, which highlights structural differences in the spaces where steering is implemented. Throughout the remaining experiments presented in the main body, the steering is implemented in the case $y_s = y_e$, which we found to produce the most stable results for steering on the $h$-space. We will clarify this in the revised version of the paper.\n- **Examples on real datasets**\n\nThis is addressed in the response to Reviewer qawW (CelebA).", "claims_and_evidence": "The main claims are supported by well-designed experiments.", "methods_and_evaluation_criteria": "The methodology is appropriate for analyzing robustness of model control. The synthetic dataset allows for precise control over concepts, facilitating clear analysis of reachability mechanics.", "theoretical_claims": "The paper is primarily empirical. There are no formal theorems or proofs.", "experimental_designs_or_analyses": "The experimental design is clean and systematic.", "relation_to_broader_scientific_literature": "The connection to steering literature (including LLMs and diffusion models) is well established.", "essential_references_not_discussed": "Essential references are discussed.", "other_strengths_and_weaknesses": "### Strengths\n\n- The empirical setup is clean and contributes to the steering literature.\n\n- Insightful findings for alternative to prompting for model control.\n\n- Relevance to practitioners seeking more controllable diffusion model behavior.\n\n- The papers is really well-written and organized.\n\n### Weaknesses\n\n- While results are robust within their synthetic setup, the paper would benefit from further discussion on the generalization of these findings to real-world, more complex datasets.\n\n- See questions below.", "other_comments_or_suggestions": "- If possible, figure 9 should be moved to the main paper.\n- The notation for the steering vectors $\\mathbf{c}_p$ and $\\mathbf{c}_h$ can be confused with the concept notation $c$. The authors should use for example $\\mathbf{s}_p$ and $\\mathbf{s}_h$.", "questions_for_authors": "- Are the notations $[f_{i_1}, f_{i_2}, \\ldots, f_{i_j}]_X$ and $[f_1^{(i)}, f_2^{(j)}, f_3^{(k)}]_X$ the same?\n\n- Can you better explain the difference between $y_e$ and $y_s$? Which one is the target combination? And Why do you need $\\mathbf{x}_0$?\n\n- Figure 4, why steering from the initial prompt \"a green triangle behind a red triangle\" to a completely different combination such as \"a red circle behind a blue square\"? Isn't it a too \"extreme\" steering?\n\n- Can you provide more examples on real datasets?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "0sA8ZvOgeN", "reviewer": "Reviewer_qawW", "summary": "This paper explores the influence of three core dataset issues on concept reachability in text-to-image diffusion models. Through a synthetic setup, the paper constructs dataset variations corresponding to the three dataset issues and test concept reachability by evaluating the concepts in generated images after training on the dataset variation. The paper introduces novel perspectives into how data issues impact diffusion model generation and offers insights into the benefit of steering in image generation controls.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your thorough review and thoughtful comments! We address your questions below:\n___\n- **Evidence behind the claim that overly detailed prompts may not help with reachability**\n\nThe starting prompt y_s used to implement the steering is as described in Section 3.5, however the accuracy displayed in Figure 6(a) for prompting is that of the prompt containing the full description of the target concepts. Hence, the graph shows that as the specification of concepts is decreased, the reachability of prompting the full description (which is overspecified with respect to the seen specification level) is severely affected due to the specification of the train set being limited. This will be clarified in the paper.\n- **Rationale behind the selected y_s for each experiment (e.g., how much does it deviate from y_e and why)?**\n\nIn Experiment 1 we vary the deviation (number of concepts changed) between y_s and y_e to understand the behaviour between lower to higher changes in number of concepts. In the remaining sections we choose the case y_s = y_e, as it overall showed the most stable performance with respect to steering on the h-space (prompt space is mostly unaffected), and would be the natural choice when trying to reach a concept in a real data model. That is, if generating something on a model via prompting fails, in most cases one would to try to additionally steer on the prompt that describes the desired outcome.\n- **Quality of the classifier**\n\nThis is addressed in the response to Reviewer saQX.\n- **Reference suggestion**\n\nThank you for the reference! Data attribution methods are certainly related to our setup. We will discuss data attribution methods in diffusion models and the reference provided in the updated version of the paper.\n- **Figure 11**\n\nWe will add a legend labelling the random seed to clarify the figure.\n- **Portion of visible back shape**\n\nThis is addressed in the response to Reviewer saQX (Degree of overlap). Note that the illustrations in Figure 3 and Figure 4 are diagrams - actual samples of the train set are provided in Figure 10 in the Appendix.\n- **Difference between h-space and prompt space in Figure 6(b)**\n\nFigure 6(b) shows an example for one model of the behaviour of prompting and steering when trying to reach red in the back when the label c_1 is not specified in training, or reaching a square in the front when the label s_2 is not specified. We wanted to note that, particularly in the case of prompting and steering on the h-space, the generated output is close to randomly sampling the value of the unspecified label (in the case of c_1, the colour of the front shape in the target combination is green and so there are only two choices for the back colour). Steering on the prompt space, instead, produces higher accuracy on the target combination, leading to a higher value achieved on the top axis (red and square, respectively) than the one achieved by prompting, although remaining close to this value. We will update the explanation of this in the paper. The dimensionality, level of disentanglement, and the dependency of the h-space on the timestep $t$ could potentially impact the observed differences.\n- **CelebA**\n\nWe agree that extensions to real-world data will be valuable. We are currently implementing our framework on the CelebA dataset to assess whether its structure aligns with the properties required for our analysis. As part of this, we are constructing approximately 15-20 dataset variations and training multiple models per dataset. Given the compute we have available we hope to be able to present the full results before the end of the author-reviewer discussion period and include them in the paper.\n\nWe currently have observed similar trends in a balanced dataset as in Experiment 1: reachability on the prompt space is more consistent than reachability on the h-space, which is affected by number of factors changed. Moreover, we have also observed a decrease in reachability as the level of specification of the train set is decreased. We plan to include this in the paper, as well as an analysis of scarcity and biases, with detailed examples of steered images.\n\nWe remark that while CelebA does provide attribute labels for selected factors, other latent factors such as background, lighting and pose are uncontrolled, and it is only the use of synthetic data that guarantees a fine-grained cause effect study of the impact of dataset modifications.", "claims_and_evidence": "The claims made are clear and supported with one small issue: maybe I missed this in the paper, but the evidence behind the claim that overly detailed prompts may not help with reachability on line 373 is not available.", "methods_and_evaluation_criteria": "The method works effectively for the proposed synthetic setup. Although Sec 5.5 and Appx F.3 show the success of steering on Stable Diffusion, these are essentially verifying concept customization but not how dataset issues impact the reachability in general in real world distributions. However, the effectiveness in real-world dataset can be readily tested, e.g., CelebA offers labels of attributes for subsampling to construct the dataset issues.", "theoretical_claims": "No proof or theoretical claims", "experimental_designs_or_analyses": "1. The choice of starting prompt y_s seems rather arbitrary. What is the rationale behind the selected y_s for each experiment (e.g., how much does it deviate from y_e and why)?\n2. The quality of concept classifiers is not shown in the paper. How well does it serve as the evaluator for reachability accuracy?", "relation_to_broader_scientific_literature": "This paper is a novel addition to the data-centric reachability study in diffusion models and offers insights and recommendations for future improvement of text-to-image generative models and generation controls.", "essential_references_not_discussed": "It can be interesting and relevant to discuss the connection between (1) the diminishing reachability as the dataset issues worsen and (2) data attribution for the diffusion models trained on the problematic datasets, such as [1].\n\n[1] Wang, Sheng-Yu, et al. \"Evaluating data attribution for text-to-image models.\" Proceedings of the IEEE/CVF International Conference on Computer Vision. 2023.", "other_strengths_and_weaknesses": "The paper is clearly presented with substantial visualizations and graphs. The proposal enables novel perspectives to root cause reachability in the dataset.", "other_comments_or_suggestions": "Figure 11 in Appx E is missing a legend for dots in the plot, which makes it a bit confusing to read the figure.", "questions_for_authors": "1. Despite that Sec 5.5 and Appx F.3 show the success of steering on Stable Diffusion, these are essentially verifying concept customization but not how dataset issues impact the reachability in general in real world distributions. However, the effectiveness in real-world dataset can be readily tested, e.g., CelebA offers labels of attributes for subsampling to construct the dataset issues. The utility of the discovery in this paper can be more impactful if shown on realistic settings.\n2. The choice of starting prompt y_s seems rather arbitrary. What is the rationale behind the selected y_s for each experiment (e.g., how much does it deviate from y_e and why)?\n3. The quality of concept classifiers is not shown in the paper. How well does it serve as the evaluator for reachability accuracy in the synthetic setting?\n4. The image illustrations in Figure 3 and Figure 4 present different positions of shapes and portions of overlapping. Does the IoU, especially the visible portion of the back shape, have any impact on the result?\n5. It is interesting to see in Figure 6(b) that prompt space and h-space optimization lead to different performance on red and blue colors. Is there any hypothesis why this happens?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "bKgSUlEb26", "reviewer": "Reviewer_fxhd", "summary": "This paper studies the concept reachability in diffusion models and focuses on the effects of three common constraints in datasets: scarcity of concepts, underspecification of captions, and biases. The work shows that although some concepts are reachable for the model, prompting fails to provide sufficient information to reach them. In addition, this paper proposes steering as a novel controlling mechanism for better concept reachability in the generation of diffusion models.\n\n## update after rebuttal\nI would like to keep my original rating. This paper presents insightful findings, but my main concerns regarding evaluation on more complex datasets and clarification on general image generation remain insufficiently addressed during the rebuttal. The authors acknowledged the usage of CelebA in the response to Reviewer qawW but failed to provide results during the rebuttal.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for reading through our paper! We really appreciate your positive feedback, and have addressed the questions/comments you made below:\n___\n- **How is the efficiency of steering in terms of application?**\n\nOur current work did not focus on analysing the computational efficiency of steering precisely, but overall we found that the optimisation of the steering vectors is dependent on the dataset and learning rate used for optimising, with the number of steps required to implement the optimisation varying. Analysing the efficiency of steering under different conditions in large-scale models would be an interesting extension. In Stable Diffusion, the optimisation of the steering vector required less than 20 steps.\n- **CLEVR and synthetic data**\n\nAlthough the Clevr dataset contains more complex images with greater detail and 3D structure, we believe that repeating the experiments with Clevr would yield similar results to those already observed within our synthetic framework. The underlying factor structure of both datasets is essentially the same, and as such, we anticipate that the observed outcomes would not vary significantly. In our work, we opted for a simpler dataset to ensure controllability, robustness, and realisable methods for evaluating accuracy, as well as more scalable model training at a lower computational cost. We further address experiments with more complex data in the response to Reviewer qawW (CelebA).\n- **Figure 7**\n\nFigure 7 shows the reachability to images containing either only circles or only blue shapes in the back, under conditions where these two concepts are tied. As the presence of images containing only blue in the back is increased (thus, reducing the bias), the reachability of images containing only blue in the back increases - this is shown in how the light-coloured X's produce a higher accuracy on the horizontal plane. However, this also leads to a general trend (with some variability) of increase in the accuracy of images containing only circles in the back - note that generally across all methods, on the vertical axis, the lightest coloured X's achieve the highest accuracy. We will clarify the description of this Figure in the paper.", "claims_and_evidence": "The claims are successfully supported by experiments on controlled synthetic data.", "methods_and_evaluation_criteria": "The proposed methods and evaluation criteria make sense under the problem setting.", "theoretical_claims": "There is no major theoretical claim in this paper.", "experimental_designs_or_analyses": "The experiment design is convincing overall, except for several concerns: (1) The experiments focus on the analysis of synthetic data and only show three examples for three challenges for the application in image generation with diffusion models. Is it possible to evaluate the performance on a larger scale for diffusion model generation? (2) How is the efficiency of steering in terms of application?", "relation_to_broader_scientific_literature": "Previous work has proposed to optimize the input of specific task, or introduce steering vectors at specific layers to improve the fine-grained control in LLM or text-to-image model generation. This work studies the concept reachability of steering the textual prompt or in the U-Net bottleneck layer under three scenarios of dataset constrains.", "essential_references_not_discussed": "I didn't identify any such references.", "other_strengths_and_weaknesses": "Strengths:\n\nThis paper is well-written and the contribution is valid by categorizing and analyzing the common constraints in concept reachability. The steering approach is effective in these settings and image generation of diffusion models. The experiments on the synthetic data are comprehensive, with settings that are well-controlled for each constraint.\n\nWeaknesses:\n\nThis work mainly focuses on the analysis of synthetic data, lacking broader evaluation for more general image generation with diffusion models.", "other_comments_or_suggestions": "The notion of Figure 7 is unclear to me, and I didn't fully understand how Figure 7 supports the claims regarding biases.", "questions_for_authors": "Please see the questions in Experimental Designs Or Analyses. Additional question: Will the synthetic dataset be too simplified? CLEVR [1] is a dataset with multiple combinations of attributes with more realistic appearances, and the code to render new images is provided as well. Will CLEVR be a better choice for analysis?\n\n[1] Johnson, Justin, et al. \"Clevr: A diagnostic dataset for compositional language and elementary visual reasoning.\" Proceedings of the IEEE conference on computer vision and pattern recognition. 2017.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "YRmb5mVz0C", "reviewer": "Reviewer_saQX", "summary": "This paper compares the effects of prompting vs. steering on diffusion models, and in particular how well these approaches allow for the control of specific concepts in the generated output.  To provide control in the experiments, a synthetic dataset of overlapping shapes is created, and the concepts of interest are the types of shape in the image and the colors of the shapes.  The paper then explores the effects of bias (frequent co-occurance of a shape/color pair), concept scarcity (the degree to which the concept has been observed), and caption specification (how well the caption associated with an images fully captures the concepts in the image).  The main finding is that commonly used prompting is not sufficient to realize concepts in the presence of badly formed data, whereas steering allows better generation of scarce concepts and allows the disentanglement where spurious correlation/bias exists.  Furthermore, prompt steering rather than steering in the latent space of the model is shown to be more robust and reliable.\n\n### Update\nFollowing the rebuttal I have maintained my score and recommend accepting the paper.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for taking the time to review our work. We are very happy to read your positive review! We have read through the comments and suggestions you made and addressed them below:\n___\n- **Does the underlying accuracy of the classifier not need to serve as the baseline for the reachability measure?**\n\nWhen evaluating on human-labelled images (5400) that were generated by prompting from two diffusion models trained on balanced data, the classifier obtains an accuracy of 96.63%, hence the results would have to consider the possible error of the classifier. We consider this accuracy to be sufficient to reflect the impact of our experiments on reachability. We will add a clarification about this to the paper.\n- **Paper focuses on steering, but there are other methods for reaching the concept than steering**\n\nThis is fully correct: steering falls under the broader category of modifying the latent space and many variations exist on how this latent space modification is performed (which model features are being modified, how this modification is performed). Our focus here was on steering as it has achieved successful results in both diffusion models and autoregressive modelling, while still being relatively simple and efficient to implement (and hence promising also for practical use-cases).\n- **More complex data**\n\nThis is addressed in the response to qawW (CelebA).\n- **Typos in Section 3.4**\n\nThank you for pointing these out! We will correct this in the updated version of the paper.\n- **Does the degree to which the shapes overlap affect the reachability measure?**\n\nOriginally, we encountered challenges when classifying the back shape in case these were heavily occluded. We modified the neighbourhood within which the front shape was sampled in order to ensure a reasonable portion was visible. The minimum percentage of each back shape visible in the balanced train set for the diffusion models is the following: circle 52.46%, triangle 48.97%, square 59.38% - this will be added to the Appendix in the dataset details. Overall, approximately half or more, of each shape is visible. We did not explore this further, however after this modification we didn’t notice any significant difference in reachability with the different shapes in the back.\n- **In Figure 4 — are the shape/color combinations at the top of figure only representative examples, or does the figure relate to these exact samples?**\n\nThese are the colour-shape combinations for the accuracies displayed in the graph. For example, the first column shows a green triangle behind a red triangle, which changes no concepts with respect to the starting prompt y_s. Note that it does not refer to the exact relative position between the two shapes, only the combination (we will clarify this).", "claims_and_evidence": "Yes.  The use of the synthetic data allows specific control over the presence of concepts, and degree to which the prompt specifies the concepts.  The are examples of showing the equivalent effects on real-world samples, albeit limited in number.", "methods_and_evaluation_criteria": "Yes.  The use of the synthetic data removes ambiguity and noise present in real-world samples (such as the degree to which a concept is present in an image, or how other the presence of concepts affect interpretation of a concept).", "theoretical_claims": "N/A.  The paper presents an empirical study.", "experimental_designs_or_analyses": "I have no concerns about the experiments, but had a minor question regarding using the accuracy of a classifier to measure concept reachability.  Does the underlying accuracy of the classifier not need to serve as the baseline for the reachability measure?  For example, if the classifier reports 90% accuracy on the generated data, reachability is reported at this level — however if the classifier itself is only 90% accurate, then might the concept not be considered full reachable?", "relation_to_broader_scientific_literature": "The paper references related work that these experiments build on, and relevant citations are included to support statements made.", "essential_references_not_discussed": "I have no concerns here.  The assume the authors make is that if a concept cannot be reached by steering vectors, then it is not reachable.  However, there are other methods for steering the outputs generated by models, and these might perhaps be able to tease out the concept from the model.  I do think it is fine to focus only on steering vectors, but this limitation/assumption might be worth spelling out.", "other_strengths_and_weaknesses": "Strengths\n+ The authors will release the code upon accept to aid reproducibility.\n+ The experiments are well put together and results support the claims made.\n\nWeaknesses:\n- Perhaps more analysis on trickier real-world cases would make the paper stronger.", "other_comments_or_suggestions": "In Section 4.3:  replace “with not shapes” with “with no shapes”\n\nIn Section 4.3:  replace “report the average results” with “report the mean results”", "questions_for_authors": "Q1:  Does the degree to which the shapes overlap affect the reachability measure?  For example, a heavily occluded background shape might be harder for the classifier to identify, even if the shape itself is correct?  This could mean that reachability is under-estimated in more difficult cases, as opposed to the concept being less reachable.\n\nQ2:  In Figure 4 — are the shape/color combinations at the top of figure only representative examples, or does the figure relate to these exact samples?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {"ethical_review_concerns": "N/A"}}]

Author rebuttal:

Meta-review: This work proposes a thorough analysis on the topic of concept reachability, focusing on text-to-image diffusion models. For that, they design a synthetic (fully controllable) dataset that allows training under the following situations: scarcity of concepts, underspecification of captions, and biases. 
The results show that steering is more effective at reaching concepts under such underrepresented situations, allowing better disentanglement of concepts when bias is present (suprious correlations).


The rebuttal has been engaging and I thank both authors and reviewers for that. Many questions have been solved. The reviewers agree in that this work is very clearly presented, with a thorough evaluation.  However, reviewers fxhd, saQX, qawW, 7wCq  found that more evaluation on realistic data (eg. CelebA) would be beneficial. I also agree with this observation. The authors mention they are running experiments on CelebA, and provide initial results. They commit to add CelebA in the final paper. 


Given the actual scores and commitment of the authors to include CelebA, I believe this paper would reach the quality required for ICML. I think that the scope and findings of this paper will be valuable for the community, given the strong use of T2I Diffusion and the need to satisfy user needs beyoned the nominal behavior (reach any concept).
Decision: Accept (poster)