Uncovering Conceptual Blindspots in Generative Image Models Using Sparse Autoencoders
Authors: Maty Bohacek
Thomas Fel
Maneesh Agrawala
Ekdeep Singh Lubana
Venue: ICLR 2026 Poster (2026)
Keywords: Generative Image Models
Failure Modes
Interpretability
Sparse Autoencoders
Primary area: interpretability and explainable AI

Abstract: Despite their impressive performance, generative image models trained on large-scale datasets frequently fail to produce images with seemingly simple concepts -- e.g., human hands or objects appearing in groups of four -- that are reasonably expected to appear in the training data. These failure modes have largely been documented anecdotally, leaving open the question of whether they reflect idiosyncratic anomalies or more structural limitations of these models. To address this, we introduce a systematic approach for identifying and characterizing "conceptual blindspots" -- concepts present in the training data but absent or misrepresented in a model's generations. Our method leverages sparse autoencoders (SAEs) to extract interpretable concept embeddings, enabling a quantitative comparison of concept prevalence between real and generated images. We train an archetypal SAE (RA-SAE) on DINOv2 features with 32,000 concepts -- the largest such SAE to date -- enabling fine-grained analysis of conceptual disparities. Applied to four popular generative models (Stable Diffusion 1.5/2.1, PixArt, and Kandinsky), our approach reveals specific suppressed blindspots (e.g., bird feeders, DVD discs, and whitespaces on documents) and exaggerated blindspots (e.g., wood background texture and palm trees). At the individual datapoint level, we further isolate memorization artifacts -- instances where models reproduce highly specific visual templates seen during training. Overall, we propose a theoretically grounded framework for systematically identifying conceptual blindspots in generative models by assessing their conceptual fidelity with respect to the underlying data-generating process.

Published as a conference paper at ICLR 2026
UNCOVERING CONCEPTUAL BLINDSPOTS IN GENERA- TIVE IMAGE MODELS USING SPARSE AUTOENCODERS
Matyas Bohacek! Thomas Fe
Maneesh Agrawala'
Ekdeep Singh Lubana
1 Deparrment of Computer Science, Stanford University
"Kempner Institute, Harvard University
*CBS-NTT Program in Physics of Intelligence, Harvard University
ABSTRACT
Despite their impressive performance, generative image models trained large- scale datasets frequently fail to produce images with seemingly simple con- cepts-e.g. human hands or objects appearing in groups of four-that are reasonably expected to appear in the training data. These failure modes have largely been documented anecdotally, leaving open the question of whether they reflect idiosyneratic anomalies more structural limitations of these models. To address this, we introduce a systematic approach for identifying and characterizing "concep- tual bbindspots"-coneee in a model's generations. Our method leverages sparse autoencoders (SAEs) to extract interpretable concept embeddings, enabling quantitative comparison of concept prevalence between real and generated images. We train an archetypal SAE (RA-SAF) DINOv2 features with 32, 000 concepts-the the largest such SAE to date enablin fine-grained analysis of conceptual disparities. Applied to four popular generative models (Stable Diffusion 1.5/2.1, PixArt, and Kandinsky), our approach reveals specific suppressed blindspots (e.g., bird feeders, DVD discs, and whitespaces on documents) and exaggerated blindspots (e-g.. wood background texture and palm trees). At the individual datapoint level, we further isolate memorization artifacts - instances where models reproduce highly specific visual templates seen during training. Overall, we propose theoretically grounded framework for systematically identifying conceptual blindspots in gener- ative models by assessing their conceptual fidelity with respect to the underlying generatnn process. Code: httpps://concppt http:/oocpppuulllllinndtsttttt blindspots.sss..eee ..giisgiii thub. i0
present in the training data but absent or misrepresented
INTRODUCTION
Generative image models trained on large scale datasets have achieved unprecedented capabilities.
allowing their use in applications both within the vision domain OpenAT (2024); Peebles and Xie
(2023); Ramesh al. (2021): Saharia al. (2022) Nichol et al. (2021): : Wang al. (2024) Poole
ct al. )022); Richardson et al. ; Rombach et al. (2022) and well beyond that Ahn et al. (0022):
Huang et al. 2022a b); Rombach al. (002)); Chen al. (2024); Zhong al. (202)) Siddiqui et al.
(2024). Despite this success, several qualitative (Marcus et al.,
[022][Cabree
el al. 2021 Heigl
2025) and quantitative studies (Liu et al., 2023a Conwell et al.
(2024) have shown that, at times.
models can struggle to generate images with relatively simple concepts, e.g., human hands (Lu et al.
2024 Narasimhaswamy al. [2024] /Zhangi al. 2024 Fallah et al.] 2025), objects appearing in groups of four (Cao et al. 2025), and negations or object relations (Conwell and Ullman 2022: Conwell et al.) 2024).. In fact, when prompted to generate images containing such concepts, models tend to produce outputs with related structures, but not precisely the ground truth concept-ed e.g., producing hands with six fingers. These failure modes, which we call "conceptual blindspots can be unintuitive, since may reasonably expect models have had enough exposure
'Equal contribution. Email: maty@stanford.cmm edu. tfelêg. harvard. edu. saneesh@cs.stanfordd..k edu. ekdeeplubana@fass harvard. edu.
WWe borrow the term "blindspots" from psychology literature (Banjji and Greenwald 2016), wherein it is used to refer to scenarios where an agent makes biased decisions despite exposure observations that contradicts the rationale behind those decisions.

---

Published as a conference paper at ICLR 2026
Suppressed
0.25
0.5
0.75
Exaggersted 1
and feeder
luree-off screen
Draganily
Color paint
Badkgro,nd wood
Orange backgro.nd
Taficc sign
Repeosed gride
boos
Figure 8(k) quantifies model's tendency to under-generate concept os compared to its natural-data frequency. We deem concepts with 8(k) < 0.1 as suppressed conceptual blindspots and concepts with o(k) > 0.9 as exaggerated conceptual blindspots. The depicted images, generated by four popular generative image models, show examples of conceptual blindspots as well as aligned concepts. The models are completely unable to generate suppressed blindspots (e.g.. bird feeder), despite diverse prompting and steering strategies. For concepts with 5(k) z 0.25 (e.g.. traffic sign), the models exhibit substantial deficiencies. In contrast, exaggerated blindspots emerge unprompted, at rates far exceeding their distribution in natural images.
to demonstrations accurately detailing such concepts. This raises the question whether such failures reflect intriguing quirks of certain specific concepts, or whether they are demonstration of a more systematic phenomenon under which, for broad spectrum of concepts, models fail to overly likely to produce images containing them.
Answering this question remains infeasible with existing approaches for evaluating generative image models (Stein et al.] ) Wang et al.) (2023a). Specifically, existing approaches generally rely on coarse-grained measures that meant assess image realism, e-g.. FID (Heusel et al. 2017). and
hence do not capture distributional failures. Methods like CLIPScore evaluate generation diversity or distributionncoverage statistics (Hessel et al. 2021 Dombrowski et al. 2024 Hwang al. 2024).
hence offering partial insights to our question, but not at the granularity of fine eeeeeeradndd features or concepts Theis et al. (2015); Naeem et al. 2020), making it difficult to identify conceptual blindspots. Finally, qualitative analy analyses for evaluating generative models, such participant surveys (Cabrera iet al., 2021 Nichol et al. 2021 Petsiuk et al. 2022 Xu et al. 2023 Wu et al. (023a) or open-ended exploration (Bau et a 2019), can identily faalures in models ability to capture certain concepts, but do not offer a scalable methodology that would allow for comparability.
This work. Motivated by the above, argue identifying and analyzing conceptual blindspots in generative image model requires designing a methodology that, in an automated and unsupervised manner, elicit concepts in the data distribution that have mismatch between their odds of generation by the true lata-generating process versus the trained model. Our contributions include:
Formalizing Conceptual Blindspots Generative Image Models. We introduce systematic framework for identifying and quantifying conceptual blindspots in generative image models compared to natural images (Sec. 2}. This formalization moves beyond anecdotal or human- defined concept evaluations, offering principled approach to understand the models' limitations.
A Scalable, Unsupervised Approach for Identifying Conceptual Blindspots using Sparse Autoencoders. We develop a structured methodology using sparse autoencoders (SAEs) to extract and compare concept distributions between natural and synthesized images (Sec. 3). To do so, we employ SAFs, which decompose the high-dimensional activation space models into sparse, human-interpretee concepts; namely, they trained to reconstruct model activations using sparse combination of learned feature directions (concepts). Each concept can then be assigned a human-interpretable label through autointerpretabillty examining high-activating exemplars and prompting an LLM to describe them. To this end, we train and oper-e an archetypal SAE (RA-SAB) on DINOv2 features with 32, 000 concepts, the largest such RA-SAE date.
Exploratory Tool and Analysis. Our exploratory web tool enables both distribution- and datapoint- analysis blindspots across models (Sections 4. 4.44.. We apply our method to four popular models (Section 4 and identify specific instances of both suppressed conceptual blindspots (e.g., bird feeders, DVD discs, and whitespaces on documents) and exaggerated conceptual blindspots (e.g., wood background texture and palm trees), shown in Fig.
2

---

Published as a conference paper at ICLR 2026
2
FORMALIZING CONCEPTUAL BLINDSPOTS IN GENERATIVE MODELS
We begin by formalizing the notion of conceptual blindspots: systematic discrepancies between the conceptual content of natural images and that of model-generated outputs. This formulation enables to derive principled, quantitative measures that characterize which concepts are under or over represented by generative model relative to its data distribution. The process is illustrated in
Fig. 2 While rely standard assumptions in this pursuit (Von Kugelgen al.) [021] Locatello
et al. 2019; Zimmermann et al. 2021] Gresele et al., 2020 2021), empirically we find meaningful phenomenology is elicited even when these assumptions are violated.
Definition Pata-Generating Process). Let C C K denote latent space with a Boltzmann prior p(c) exp( E(c))Z-1, where E(-) denotes energy function that linearly decomposes individual latents and Z is the corresponding partition portition.fucctio function, i.e., E(c) = Dx E(cx) and hence p(c) 1lx p& (ca). A data-generating process (DGP) is invertible function G C X that maps the latents E C to observations x X. i.e. = G(c).
For notational simplicity, we p(-) denote both the latent density p(c) and its push- forward to image space p(x), where x G(c). This is justified by the invertibil- ity of G, which induces a valid distribution over X via the change of variables formula. In essence, the individual dimensions of the latent space reflect the Concepts underlying the data data-distribution Px, defined over some ob- servation space of images X. For example, different latents may correspond to concepts like color, shape, size, location, and so on (Okawa et al. 2023 Park et al., (024). We also let the data- sitiaaagnerattngg process associate text-description t E T with any image sam- pled from the data distribution, but do not ex- plicitly model it. These text descriptions can then be used to train a text-conditioned Gener- ative image model ge, with parameters a set of image-text pairs to map a noise signal n N(,, I) and a text-description of the scene t to produce images æ illustrating the latter.
Observation Space X'
Latent
G
C Space
l
Bo
Ck
G-1
x
(x(x))
Energy Model
Synthesized Images
Figure 2: Intuitive mapping of our framework.
Latent concepts c₁ E C are mapped to observations (x, t) through the (invertible) DGP. ge generates images x' based on t. The energy model & (x) extracts concept representations from both æ and æ', enabling comparison of concept distributions to identify blindspots.
To define conceptual blindspots in the model ge, we must assess the probability mass assigned to a concept by the data- generating process, com- paring it to the mass assigned by the model. To this end, we take an evaluation set natural image-text pairs (Dx, DT) and define a set of synthet- ically generated images D'x using the text descriptions. To estimate the probability of occurrence of a concept, we consider Energy model & X' Rd that maps images to a d-dimensional vector, where each dimension associates a scalar representing the energy in the 14th concept, i.c., (x(x) E(cx). These estimates are feasible because
assume the data eetta-enerating process is invertible. Correspondingly, the probability the data-generating process associates with the image can then be defined as (xx exp( -x &x (a)) (where Zę is the partition function), hence yielding a population-level estimate p& (Dx) = IILe (x)) Using this and the sigmoid map a-)), define below the average energy difference in the 1tth concept between the datasets Dx. D'x- Definition (Energy Difference). Let x E Dx denote real image sampled from the data- generating process G. and lei x' E D'x be synthetic image generated by the model go. Let Eu X + R denote the energy assigned to the 1hth concept by the energy model &. We define the energy difference for concept as:
8 ( (Ear (EL(2') -m (Ex(x)))
Pk (Dxx)
(Dx + P& (D'x'
where the expectations are taken over D'x and Dx. respectively, and Pt: (D) & exp (- EEED Ee(x)) denotes the unnormalized conceptual probability mass of dataset 'D under concept k.
3

---

Published as a conference paper at ICLR 2026
Thus, the energy difference in the 1m concept describes the ratio of the probability concept occurs in a set of observations (here, D'x) compared to a baseline dataset (here, xx). Based on this measure, we can now define conceptual blindspots follows.
Definition 3 (Suppressed Exaggerated Conceptual Blindspots). Given generative image model go. we say, compared to the data gentrainngg process G, Ck is suppressed conceptual blindspot in the model if Gg (k) < Amin and exaggerated 6go++ Amax-
Overall, define conceptual blindspot as a concept whose likelihood of occurrence in generated images deviates markedly, either through suppression or exaggeration, from its prevalence under the data-generating process. Suppressed concepts exhibit disproportionately low activation (e.g.. 5(k) < Amin). whereas exaggerated concepts are overrepresented (e.g., 5(k) > Amax). Throughout our analysis, we adopt threshold values of Amin 0.1 and Amax 0.9 isolate these regimes.
We also note this definition is related to the idea of "mode collapse" studied in past work (e.g., see Bau al. (2019)): the difference is in the granularity which the analysis performed. Specifically, mode collapse focuses on exaggerated / suppressed odds of generating entire images, while we focus on changed odds of specific concepts. For example, if a model fails to produce images of an object with white background, we say this concept is a suppressed conceptual blindspot.
3 METHOD: OPERATIONALIZING THE DEFINITION OF CONCEPTUAL BLINDSPOTS
We next discuss our pipeline for identifying conceptual blindspots a generative model ge. As per Sec. 2 the salient objects we need for this are (i) a set of images sampled from Jo that allow comparison with the ground-truth generative process, and (ii) an energy model which enables said comparison. Below, of non-zero entries (the to pseudo-norm). For a vector or matrix X, X 2 O implies element-wise non-negativity. For n. 0, we let [n] (1, ..,.... n), , and denote the i-th row of matrix A by A..
use ||r to denote the Frobenius norm and |lo to denote the number
From Prompts to Latent Representations. To identify conceptual blindspots in a model go. we compare a dataset Dxx of image-caption pairs (a. t) sampled from the data-generating process G and their synthetic counterparts sampled from the generative model go using the text descriptions.
Specifically, given t. we synthesize counterpart image x' using a pretrained toxt-to-image generator go : T + X, implemented as a denoising diffusion probabilistic model (DDPM) (Razzhigaev et al.. 2023 Stability Al 2022 (m)==
Chen et al. 2023a). Sampling occurs in latent space via a reverse trajectory
YT NO,,)),
h- Va, (7/- vi-a. Eo(T),t,c) o. and n: NO,,
where ag E (0, 1) and 0r IIe ax follow the standard cosine noise schedule. The final latent Yo decoded via a pretrained VAE to yield the synthetic image x' VAB(70). For the remainder of the paper, treat go black box that maps prompts to images: t - x'.
Defining the Energy Model. Building on prior work that shows the ability of self-supervised learning methods to invert the data-generating process and identify the energy function underlying it up linear transformations (Zimmermann et al. /20TT Von Kugelgen al. 2021 Khemakhem et al. 2020; Hyvarinen et al. 2019). we use DINOv2 (Oquab et al. (023) for our analysi Under the expectation that the number of concepts underlying the DGP is larger than the dimensionality of the model's feature space (Elhage et al. 2022 Bricken et al. 2023) we train sparse autoencoders (SAFs)
on its features to identify subspaces corresponding to these concepts (Fel et al. 2025 |Cnniingaam
et al.) 2023) Gao et al. 2025 Templeton et al.) 2024 Rajamanoharan et al. 2024). The intuition here
is that if the concepts underlying the generative process are modeled via approximately orthogonal
We use DINOv2 our energy model because its self-supervised training on large-scale unlabeled data yields emergent, highly structured visual representations that capture broad semantic and geometric regularities without text supervision. These embeddings have proven robust across tasks (classification, segmentation. depth estimation, tracking) and domains (natural, medical, satellite imagery) Oquab et al. (2023): Liu et al. (203b)) Ayzenberg al. 2004).
4

---

Published as a conference paper at ICLR 2026
STAGE 1: Concept Extraction
STAGE 2: Analysi
Distribution Level
Eircsppo lead supprivedd medggggeee
Neteed
Syst-rx
biiqus
Excoder
concept
46
Convetent conoprto hare enegy
Energy Diff ... elA)
Corcaple
Datapoint Level (DP)
Menoriation
Lapactdd
datapoint (x.4,"):
122
DP Distance (85+ (12)
DINOv2 forard ferts.
Co-occurence
Z'z "00o .0.0
Sangpeuted
batch top-t
SAE Concepts Energy
Decomposing 2' 2'2&2'2 22 Symmetry cos (V"U)
(C'c)
compastion
3(x')
ZZ '000
000
sultion
Figure 3: Concept Extraction Pipeline. For triplet (æ, t, x'), the concepts in æ and are extracted by obtaining each image's ' DINOv2 features, which are further processed by RA-SAE into sparse concept embeddings, yielding energy models E(x)) and g(x'), respectively. In particular, {x(x)) E(ck) holds the energy in the km concept.
directions by DINOv2 (as assumed in our independence constraint in Def. T. then an SAE should be able to isolate these concepts along individual dimensions in its latent space (Elhage et al. 2002).. The activation associated by the SAE a dimension will serve as our approximation the ground-truth energy function assigned to the concept modeled by that dimension.
Formally, using f X + Idd to denote our feature extraction module (i.e., the DINOv2 model), we extract features = f(x)= E F from both natural and synthetic images from datasets Dx. D'x- Assuming the count of image-text pairs n, we stack the real and generated features into matrices A, Raad We then decompose each feature vector into sparse combination of learned concept
atoms using an SAE. Specifically, let D E RRdx denote a dictionary of K" concept vectors, and let L R R be the SAE encoder that maps input features to sparse codes. Applying row-wise yields the matrix of activations Z (A) E R=*XK', where each row 2, = (a,) represents the concept decomposition of an image. The decoder reconstructs features via 2D T and the SAE is
trained to minimize the reconstruction error subject to sparsity and non-negativity:
min A (AA)D * D
s.t. (A) 2 0, || (A))lo K'
(2)
Vanilla SAEs often drift toward arbitrarily oriented dictionaries, making downstream analyses highly sensitive to the random seed. To mitigate this instability and make our study reproducible and independent of the seed, we employ the Archetypal SAE (RA-SAE) Fel et al. (2025) on T sparsity Constraint/Gao et al. al. (2025). RA-SAE constrains the dictionary D to be convex combination of training data. Specifically, we write D W A with W E aK the set of row-stochastic
matrices in Ig k'n.
OKK : < W 2 O, W1
Thus every atom D; lies in the convex hull of the data conv(A), (A), and any reconstruction T resides inside the conic hull of the data cone(A) A). This ensures learned concepts remain faithful to the support of the data distribution (Fel et al. 2025). Once trained, the SAE provides a consistent set of sparse codes: Z for real images and Z' for their generated counterparts. These codes capture the sume promgi-conditioned visual semantics in terms of shured, interpretable concepts, with the activation value of the concept serving as energy values for our analysis of conceptual blindspots. In summary then, method defines a structured pipeline that, given prompt and its associated real image (t, x). produces two sparse concept vectors (z, z'), enabling direct comparison the real and generated visual content in common conceptual basis.
This summarizes our full pipeline: starting from a captioned image (t, x). we synthesize a counterpart x' and map both images into shared, sparse concept space via vision encoder and trained SAE, yielding interpretable representations (z, x') that will serve as the foundation for evaluating conceptual shifts induced by the generative process.
5

---

Published as a conference paper at ICLR 2026
4 RESULTS
We analyze four generative image models trained LAION-SB-SII 1.5, SD 2.1, PixArt, and Kandinsky- X 10,000 image-text pairs and their corresponding generations (Appendix[N) N). Our analysis spans three levels (Fig. 3): distribution-level evaluation reveals suppressed and exaggerated concepts; a datapoint-leve analysis surfaces failures tied to ambiguity, omission, and memorization; and a compositional analysis uncovers subtle distortions concept co-occurrence geometry. Our core contribution is an interactive exploratory tool, shown in Appendix C from which all subsequent analyses emerge. Rather than exhaustively studying one phenomenon, we present high-level findings that highlight the tool's versatility and enable broader, customizable exploration.
4.1
THE HEAVY TAIL OF SUPPRESSED CONCEPTS
SD 1.5
SD 2.1
PixArt
Kandinsky
1,0
0.8
Densty
0.6
0.4
10"1
0.2
0.00
0.25
0.50
0.75
1.00
0.00
0,25
0,50
0.75
1.00
0.00
0.25
0.50
0.75
0.00
0.25
0.50
0.75
1.00
Energy Differance
Energy Differance
Energy Lifference
Energy Lifference
Concept Energy Distribution. Log-scale histograms of energy differences 5(k) across 32, 000 concepts, comparing the natural and synthesized distributions for each evaluated model. Values left of 0.5 represent suppressed concepts (under-represennted): values right of 0.5 represent exaggerated concepts (over-represented))
Figure 4:
To assess disparities between the generative models and the natural image distribution, we begin by analyzing the marginal energy difference 8(k) across 32,000 concepts learned using RA-SAF. As defined in Sec. 3 this quantity reflects the relative prevalence of each concept in the synthesized versus natural image sets. A value of 5(k) < 0.1 indicates that concept under-represented (suppressed) in the generated images, while 5(k) 0.9 indicates vverreepresnntttion (exaggerated), Fig. presents the distribution of o(k) for each the four evaluated models. Across all models, observe heavy-tailed histograms with substantial mass both extremes, suggesting systematic discrepancies in concept coverage. Notably, the left mhil-corresponding to suppressed concepts- is denser and longer than the right, indicating consistent tendency of concept suppression. This asymmetry reflected in the negative skewness of the distributions: Skewness -0.54 for SD 2.1, -0.40 for both SD 1.5 and PixArt, and -0.23 for Kandinsky.
We also note that while all models exhibit both suppressed and exaggerated concepts, their specific profiles differ. For instance, PixArt shows wider spread, suggesting more suppressed concept distribution. Nevertheless, the consistent left-skew in all distributions underscores a common tendency toward concept omission, though the specific characteristies of this behavior require further analysis, which we explore in the next Sections.
SD 1.5
SD 2.1
PixArt
Kandinsky
1.0
0.8
0.6
0.4
0.2
0.0
Structure of Concept Energy Differences. UMAP visualizations of 32, 000 concepts. colored according to their energy difference 5(k) between the natural and synthesized distributions. Clusters reveal patterns of conceptual blindspots, with suppressed concepts on the blue end and exaggerated ones on the yellow end.
Figure 5:

---

Published as a conference paper at ICLR 2026
(a)
(b)
1.0
D-0.2
1.0
[-0,1
1.0
r-0.46
5013
C:41
0.5
PoAArt
Kardiask
3031
0.0
0.0
0.0
Kandiaay Pakks
2.44
-1.0
-1.0
-1.0
SD 1.5
2012 2011
Rolt far
Figure 6: Cross-Model Concept Energy Correlation. Pairwise scatter plots of 8(k) across all four evaluated models, with Pearson correlation coefficients reported top left. Strong alignment between SD 1.5 and SD 2.1 contrasts with weaker correlations among other architectures, indicating model-specific blindspots. Correlation Matrix of Conceptual Blindspots. Heatmap of pairwise Pearson correlation coefficients for ó between all models, quantifying the degree of shared conceptual blindspots across these models.
4.2 STRUCTURE AND SPECIFICITY OF CONCEPTUAL BLINDSPOTS
While the previous section quantified marginal discrepancies in concept frequency, here investigate their global structure by embedding the full set of 32,000 concepts into two dimensions using UMAP on the sparse codes, coloring the concepts by their 8(-) values. As shown in Fig. [5] distinct clusters of concepts emerge across all models. These clusters often correspond to contiguous regions of conceptual blindspots, especially for suppressed (blue) concepts, suggesting that blindspots are quite structured -rftturtdrrflletiing shared biases in either training distributions or architectural priors. To assess the consistency of these blindspot patterns across models, we further analyze both the magnitude and structure of concept-level 5(-) values. Fig. presents scatter plots and pairwise Pearson correlation coefficients between the (k) vectors of SD 1.5 and all other models. As expected, SD 1.5 and 2.1 exhibit strong correlation (r 0.82), reflecting their shared architectural and training pipelines. In contrast, their correlations with PixArt and Kandinsky are substantially lower-r 0.41 and 0.46, respectively indicating that these models emphasize different regions of the conceptual space.
Overall, the analysis above reveals that while some blindspots are universally shared likely due to properties of the mataset-other are highly model-specific, emerging from idiosyncrasies training dynamics model capacity. This motivates the need identify and study both blindspots that shared across models and ones that are unique to specific models in subsequent sections.
4.3 QUALITATIVE BLINDSPPT EXAMPLES
SD 1.5
SD 2.1
PixArt
Kandinsky
We next visualize specific examples of both sup- pressed and exaggerated blindspots to gauge what concepts fall under these regimes. Specif- ically, Fig. Xa we show an example ceptual blindspot suppressed by all models find all evaluated models fail to reproduce the concept solid white documents. As can be seen in the figures, despite the caption explic- itly referencing this concept, none of the gener- ated images reflect the intended visual seman- tics, suggesting that this region of the concept across
VEIC
con-
HIS WAY OUT
Cup coffee on momnng paper Eusinees news.
Hon To Doankood And Instal A Forn.
Dust Cup Fibwr for Des Davil F25 Vocumm Part
This Why Out Skan
space is systematically under-sample models. Meanwhile, Fig. [p highlights a model- specific blindspot: the concept pan is accurately captured by three models, yet conspicuously missing from generations produced by Kandin- sky. This reinforces the findings from Sec. [4.1] where cross- moos- agreement was found to be high in some cases but limited others.
Figure 7: Example of an Exaggerated Concep- tual Blindspot. Four synthesized images x' with the wooden surfaces and everyday objects concept are shown alongside the corresponding nat- ural image æ and text prompt t. As expected with exaggerated blindspots, the concept is prominent in x' but absent æ and t.
Conversely, in Fig. we present case of exaggeration, where the concept shadow under animal overly emphasized in generated images. While shadows are mildly plausible, their consistent and pronounced rendering across models, relative to the more nuanced and variable occurrences in
7

---

Published as a conference paper at ICLR 2026
(a) Universal Blindspot
(b) Unique Blindspot
SD 1.5
SD 2.1
PixArt Kandinsky
SD 1.5
SD 2.1
PixArt
Kandinsky
colfuue Eogla
Whitespaces
9
documents
pan
codlrg leaes
deudins
Examples of Suppressed Conceptual Blindspots. The natural images representative O and I, shown alongside four synthesized images æ', generated using Sg. The universal blindspot is present in all evaluated models; the unique blindspot is only present in Kandinsky.
Figure 8:
natural images, suggests overactive prior. Interestingly, despite attempts finding concepts that are uniquely exaggerated by specific model, we did not find any clear examples-thi suggests exaggerations are approximately universal.
Overall, the examples above concretely demonstrate how conceptual blindspots manifest in generated outputs, illustrating that our energy-based diagnostic can surface both shared and model-specific failure modes. Notably, it enables the identification of surprising model limitations as the consistent failure to reproduce clear or solid background elements, like whitespace on documents, across all models. This raises the possibility that certain failure patterns may stem from architectural constraints or training data biases that transcend individual model idiosyncrasies.
While these aggregate-level analyses are informative, they invite deeper question: do these blindspots emerge only in the aggregate across many samples, or they manifest themselves even at the level of individual datapoints? This finer- ener-grained perspective allows us to probe the mechanisms behind blindspots more directly-uncovering cases prompt misinterpretation, latent memorization, or both.
(a) Memorized
(b) Not Memorized
4.4 DATAPOINT-LEVEL ENERGY DIFFERENCE FROM INCONGRUENT TO MEMORIZED IMAGES
To move beyond population-level statistics, examine individual natural vs. generated image pairs for which the 8(.) values averaged across all concepts exhibit the largest and smallest dif- ferences. This analysis aids easy understanding of model success and failures, latter of which we find often arises from prompt ambiguity or mem- orization artifacts. For example, Fig. L shows instances with near-zero difference in average 6(.) values. In these cases, the generated im- ages are conceptually indistinguishable from the original. However, qualitative inspection clearly shows this happen not because the model faith- fully captures the prompt semantics, but from pure replication of memorized templates: we see repetitive visual structures (c.g., outlines of clothing or object arrangements), indicating that the model may be copying from overly frequent patterns in the training data. By contrast, Fig. [9p illustrates samples that are among the largest 5(.) values. These indicate significant conceptual divergence between the synthesized and natural image. While some of these discrepancies can be attributed to underspecified or noisy captions, others reveal genuine blindspots: the prompt describes clear concept faithfully present in Dx. yet the model fails to realize it in D'x- This failure suggests that even when language grounding is adequate, certain concepts fall outside the model's generative abilities. To confirm that these distributional discrepancies reflect genuine failure cases rather than artifacts of poor data quality, we conduct a systematic VI.M-based inspection of high-divergence samples, finding that the majority (56.3% of
Zt Eiittema digectiv patts Sto/ocen...
27475408144
Imege Uhasa
Color Dalle...
perforance
Figure 9: Datapoint-level Conceptual Align- ment. (a) Examples with minimal energy differ- ences where models appear to memorize train- ing patterns. (b) Examples with large differ- ences where significant concept divergences due to prompt ambiguity or model limitations occur.
8

---

Published as a conference paper at ICLR 2026
the 200 most diverging datapoints) constitute genuine blindspots where the caption is sufficient but the model fails to generate the concept (see Appendix for details).
4.5 ANALYZING POST-TRAINING EFFECTS
Post-training protocols, e.g., safety fine-tuning, have been argued to reduce the diversity of model generations (Kirk et al. 2023). Given our pipeline's ability to isolate interesting dif- ferences in a model's generations and the ground-truth DGP, we next use to understand
the effects of DPO-a popular safety fine-tuning protocol (Rafailov et al. 2023). Specifi-
cally, we compare two variants of the SD 1.5 model: one trained with DPO, and one without.
For each image pair (Dx: D'x) the Eg norm of the difference between their inter- nal concept energy vectors, (lE(D')) {(Dx)ll2 Fig. presents histogram of these datapoint- wise energy differences. The DPO-enhanced model exhibits both a lower median and a nar- rower spread, indicating more consistent distri- bution of generated concepts with the ground- truth DGP. This suggests that DPO may serve to regularize the model's concept distribution, encouraging outputs that more closely reflect the semantic content of the seen inputs. While our analysis does not disentangle the specific in- ductive biases introduced by DPO, these results provide empirical evidence that its optimization objective, which favors human-preferred generations, indirectly promotes better match with the training distribution. In particular, it reduces both over and under activation of individual concepts relative to the natural baseline. These findings highlight the utility of our pipeline in characterizing the downstream effects of post-training interventions: not merely in terms of output quality, but in how they reshape the conceptual geometry of the model's output space.
compute
SD 1.5
SD 1.5 DPO
C2
DARSC 0.4755 55205 15005
0,500
CAR55 0,975 DSEO 05501
63500
60
Exppoke
Ditpoke Swegy
Figure 10: Effect of DPO on Concept Fidelity. Histogram of datapoint-wing energy differences hetween the synthesized and natural distribution of SD 1.5 models with and without DPO.
4.6
CONCEPTUAL MISALIGNMENT AS FUNCTION OF EMPIRICAL. FREQUENCY
SD 1.5
SD 2.1
PixArt
Kandinsky
1.0
1.0
2.8
088
2.6
Relibo
02
0.2
0.25
0.50
1,00
0.25
0.50
0.75
3.00
0.25
0.90
0.75
0.00
0.25
0.25
1.00
Ffe
[|Z]]|
of
[2/h
of
N/
Concept Fidelity Across Frequency Spectrum. Scatter plots showing the relationship between concept frequency (X-axis) and the energy difference (y-axis) across four evaluated models. Each point represents concept with size is proportional its activation frequency.
Figure 11:
We previously hypothesized that conceptual blindspots are not merely be architectural artifacts, but may also emerge as a direct consequence of distributional peculiarities of certain features. In this section, we empirically test this hypothesis by examining whether concepts that are rarely activated in natural images, i.e., those in the long tail of the data distribution, lead blindspots in generative models. Specifically, we process the natural dataset Dx through the trained SAE and compute, for each concept kt empirical frequency ||Z.illo. where Z., is the activations of concept i across all our images. We then correlate this with the absolute energy difference observed across generated outputs. Fig. II visualizes this relationship for evaluated models. We find that concepts with higher frequency in natural data tend to show lower energy discrepancies, while rare concepts- concepts-esspccillly suppressed ones (8(k) < 0.5) exhibit sigmificant alignment errors. This suggests that many blindspots stem not from randomness model quirks, but from systematic effects tied to long-tail concept distributions. Addressing these issues may require not just architectural changes but also strategies like data reweighting augmentation.

---

Published as a conference paper at ICLR 2026
5 DISCUSSION
Our analysis reveals multiple conceptual blindspots in four popular generative image models. The results presented here, however, only scratch the surface: each individual finding could warrant its own dedicated investigation. Rather than delving deeply into any of these questions, instead showcase the versatility of our method and exploratory tool. Out of the box, they allow for systematic identification of concepts that models struggle generate, detection of memorization artifacts, discovery of datapoints with insufficient captions, quantification of post-training effects, and characterization of conceptual shifts across model architectures. We thus open space for follow-up work to extend the depth of analysis, scope of evaluated architectures, and inquiry into root causes of conceptual blindspots. Future work could also explore hierarchical representations of concepts in RA-SAE to allow for a more nuanced analysis. The core methodology presented in this paper is modular and agnostic to the specific concept extraction model, and such an analysis would hence require minimal adjustment to the overall process.
Beyond mere exploration and mapping of the conceptual space existing models, our method could also serve grounds for targeted intervention strategies employed during training of new models. Specifically, the energy profiles could inform prioritized sampling or reweighting, increasing the prevalence of suppressed concepts in the training distribution, and more. The energy profiles could also be employed into the training objective as a regularization term, explicitly penalizing deviations from the natural concept distribution.
Limitations. We wish to highlight several limitations of our work. By relying DINOv2 and RA-SAE for concept extraction and representation, our approach inherently constrained to the kinds of concepts these models capture; concepts poorly represented by them will escape analysis. Additionally, while our sample size of 10,000 images is substantial, it may not fully capture the long tail of rare concepts, concept co-occurrence, other compositional statistics (see Appendix KQ).
USE OF LLMS
Large language models (LLMs) were used in parts of the implementation and during the writing of the paper (e.g.. paragraph shortening, transition refinement, etc.). Al-powered search engines were also used help identify some references.
REPRODUCIBILITY STATEMENT
To maximize reproducibility of our work, our code is fully open-sourced at httpp:// conceptual-blindsposs. wir.tt...tttu...... io and the web exploratory web app available at hhtpss www.diifffgiuhiii 1o This repository will also include the Conceptual Blindspots data extracted for the models/datasets used this paper. Furthermore, our experimental setup clearly outlined in Appendix D
10

---

Published as a conference paper at ICLR 2026
REFERENCES
OpenAl. Sora:
Creating video from text, 2024. httppst//ppeaaa..cmm
William Peebles and Saining Xie. Scalable diffusion models with transformers. In Proceedings of the IEEE/CVF international conference on computer vision, pages 4195-4205, 2023.
Aditya Ramesh, Mikhail Pavlov, Gabriel Goh, Scott Gray, Chelsea Voss, Alec Radford, Mark Chen, and Ilya Sutskever. Zero-shot text-to-image generation. In International conference on machine learning, pages 5821-8831. PMLR, 2021.
Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily Denton, Kamyar Ghasemipour, Raphael Gontijo Lopes, Burcu Karagol Ayan, Tim Salimans, et al. Photorealistic text-to-image diffusion models with deep language understanding. Advances neural information processing systems, 15:36479-36494, 2022.
Alex Nichol, Prafulla Dhariwal, Aditya Ramesh, Pranav Shyam, Pamela Mishkin, Bob McGrew, Ilya Sutskever, and Mark Chen. GLIDE: Towards photorealistic image generation and editing with text-guided diffusion models. 1.ii:2112.1104::: 2021.
Chen Wang, Jiatao Gu, Xiaoxiao Long, Yuan Liu, and Lingjie Liu. Geco: Generative image-to-3d within second. arXiv preprint arXiv:2405. 203227,, 2024.
Ben Poole, Ajay Jain, Jonathan T Barron, and Ben Mildenhall. Dreamfusion: Text-to-3d using 2d diffusion. arXiv preprint arXiv:2209. 14988, 2022.
Elad Richardson, Gal Metzer, Yuval Alaluf, Raja Giryes, and Daniel Cohen-Or. Texture: Text-guided texturing 3d shapes. ACM SIGGRAPH 2023 conference proceedings, pages 1-11, 2023.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High- resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF confer- ence on computer vision and pattern recognition, pages 10684-10695, 2022.
Michael Ahn, Anthony Brohan, Noah Brown, Yevgen Chebotar, Omar Cortes, Byron David, Chelsea Finn, Keerthana Gopalakrishnan, Karol Hausman, Alex Herzog, et al. Do as can, not as say: Grounding language in robotic affordances. arXiv preprint arXiv:2204.0169)) 2022.
Wenlong Huang, Fei Xia, Ted Xiao, Harris Chan, Jacky Liang, Pete Florence, Andy Zeng, Jonathan Tompson, Igor Mordatch, Yevgen Chebotar, et al. Inner monologue: Embodied reasoning through planning with language models. arXiv preprint arXiv:2207.0556
2022a
Wenlong Huang, Pieter Abbeel, Deepak Pathak, and Igor Mordatch. Language models as zero-shot planners: Extracting actionable knowledge for embodied agents. International Conference on Machine Learning, pages 91 18-9147. PMLR, 2022b.
Yun-Chun Chen, Selena Ling, Zhiqin Chen, Vladimir G Kim, Matheus Gadelha, and Alec Jacobson. Text-guided controllable mesh refinement for interactive 3d modeling. In SIGGRAPH Asia 2024 Conference Papers, pages 1-11, 2024.
Lei Zhong, Yiming Xie, Varun Jampani, Deqing Sun, and Huaizu Jiang. Smoodi: Stylized motion diffusion model. In European Conference on Computer Vision pages 405-421. Springer, 2024.
Sprinner
Yawar Siddiqui, Antonio Alliegro, Alexey Artemov, Tatiana Tommasi, Daniele Sirigatti, Vladislav Rosov, Angela Dai, and Matthias Nießner. Meshgpt: Generating triangle meshes with decoder- only transformers. In Proceedings of the IEEE/CVF conference on computer vision and pullern recognition, pages 19615-19625, 2024.
Gary Marcus, Ernest Davis, and Scott Aaronson. A very preliminary analysis of dall-e 2. arXiv preprint 12KV::2204.13807, 2022.
Ángel Alexander Cabrera, Abraham Druck, Jason Hong, and Adam Perer. Discovering and validating ai errors with crowdsourced failure reports. Proceedings of the ACM on Human- Computer Interaction, 5(CSCW2)1122, 2021.
11

---

Published as a conference paper at ICLR 2026
Rebecca Heigl. Generative artificial intelligence in creative contexts: systematic review and future research agenda. Management Review Quarterly, pages 1-38, 2025.
Qihao Liu, Adam Kortylewski, Yutong Bai, Song Bai, and Alan Yuille. Discovering failure modes of text-guided diffusion models via adversarial search. arXiv preprint arXiv:2306.0m 30..00000944.. 2023a.
Colin Conwell, Rupert Hawiahh-uashie and Tomer Ullman. Relations, negations, and numbers: Looking for logic in generative text-to-image models. arXiv preprint arXiv:2411.17066 2024.
Wenquan Lu, Yufei Xu, Jing Zhang. Chaoyue Wang. and Dacheng Tao. HandRefiner: Refining malformed hands in generated images by diffusion-based conditional inpainting. In Proceedings of the 32nd ACM International Conference on Multimedia, pages 7085-7093, 2024.
Supreeth Narasimhaswamy, Uttaran Bhattacharya, Xiang Chen, Ishita Dasgupta, Saayan Mitra, and Minh Hoai. HanDiffuser: Texl-to-image generation with realistic hand appearances. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2468 2479, 2024.
Qilong Zhangli, Jindong Jiang, Di Liu, Licheng Yu, Xiaoliang Dai, Ankit Ramchandani, Guan Pang, Dimitris N Metaxas, and Praveen Krishnan. Layout-agnostic scene text image synthesis with diffusion models. In 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 7496-7506. IEEE Computer Society, 2024.
Forouzan Fallah, Maitreya Patel, Agneet Chatterjee, Vlad I Morariu, Chitta Baral, and Yezhou Yang. Viston: Text and prompt complexity driven visual text generation benchmark. arXiv:2503.13730 2025.
Yuefan Cao, Xuyang Guo, Jiayan Huo, Yingyu Liang, Zhenmei Shi, Zhao Song, Jiahao Zhang, and Zhen Zhuang. Text-to-image diffusion models cannot count, and prompt refinement cannot help. arXiv preprint arXiv:2503.0683 2025.
Colin Conwell and Tomer Ullman. Testing relational understanding in text-guided image generation. arXiv preprint
2022.
Mahzarin R Banaji and Anthony G Greenwald. Blindspot: Hidden biases of good people. Bantam, 2016.
George Stein, Jesse Cresswell, Rasa Hosseinzadeh, Yi Sui, Brendan Ross, Valentin Villecroze, Zhaoyan Liu, Anthony I. Caterini, Eric Taylor, and Gabriel Loaiza-Ganem. Exposing flaws of generative model evaluation metrics and their unfair treatment of diffusion models. Advances in Neural Information Processing Systems, 36:3732-3784 2023.
Boheng Wang. Yunhuai Zhu, Liuqing Chen, Jingeheng Liu, Lingyun Sun, and Peter Childs, A study of the evaluation metrics for generative images containing combinational creativity. Al EDAM, 37: el 2023a.
Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilibrium. Advances in neural information processing systems, 30, 2017.
Jack Hessel, Ari Holtzman, Maxwell Forbes, Ronan Le Bras, and Yejin Choi. Clipscore: A reference- free evaluation metric for image captioning. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 7514-7528, 2021.
Mischa Dombrowski, Weitong Zhang, Sarah Cechnicka, Hadrien Reynaud, and Bernhard Kainz. Image generation diversity issues and how to tame them. 1.XX::2411.16117 2024.
Jachui Hwang, Junghyuk Lee, and Jong Seokk Lee. Anomaly score: Evaluating generative models and individual generated images based on complexity and vulnerability. In Proceedings of the IEEE/CVE Conference on Computer Vision and Pattern Recognition, pages 8754-8763, 2024.
Lucas Theis, Aäron van den Oord, and Matthias Bethge. A note on the evaluation of generative models. 111v:1511..1144 2015.
12

---

Published as a conference paper at ICLR 2026
Muhammad Ferjad Naeem, Scong Joon Oh, Youngjung Uh. Yunjey Choi, and Jaejun Yoo. Reliable fidelity and diversity metrics for generative models. In International conference on machine learning, pages 71 7176-7185. PMLR, 2020.
Vitali Petsiuk, Alexander E Siemenn, Saisamrit Surbehera, Zad Chin, Keith Tyser, Gregory Hunter, Arvind Raghavan, Yann Hicke, Bryan A Plummer, Ori Kerret, et al. Human evaluation of text-to- image models on multi-task benchmark. arXiv:2211.22112 2022.
Jiazheng Xu, Xiao Liu, Yuchen Wu, Yuxuan Tong, Qinkai Li, Ming Ding, Jie Tang, and Yuxiao Dong. Imagereward: Learning and evaluating human preferences for text-to-image generation. Advances in Neural Information Processing Systems, 36:15903-15955, 2023.
Xiaoshi Wu, Yiming Hao, Keqiang Sun, Yixiong Chen, Feng Zhu, Rui Zhao, and Hongsheng Li. Human preference score v2: A solid benchmark for evaluating human preferences of text-to-image synthesis. 2023a.
David Bau, Jun- Yan Zhu, Jonas Wulff, William Peebles, Hendrik Strobelt, Bolei Zhou, and Antonio Torralba. Seeing what a GAN cannot generate. In Proceedings of the IEEE/CVF international conference on computer vision, pages 4502-4511, 2019.
Julius Von Kugelgen, Yash Sharma, Luigi Gresele, Wieland Brendel, Bernhard Scholkopf, Michel Besserve, and Francesco Locatello. Self-supervised learning with data augmentations provably isolates content from style. Advances Neural Information Processing Systems (NeurIPS), 2021.
Francesco Locatello, Stefan Bauer, Mario Lucie, Gunnar Raetsch, Sylvain Gelly, Bernhard Scholkopf, and Olivier Bachem. Challenging common assumptions in the unsupervised learning of disentan- gled representations. Proceedings of the International Conference on Machine Learning (ICML), 2019.
Roland S Zimmermann, Yash Sharma, Steffen Schneider, Matthias Bethge, and Wieland Bren- del. Contrastive learning inverts the data generating process. Proceedings of the International Conference on Machine Learning (ICML), 2021.
Luigi Gresele, Paul K Rubenstein, Arash Mehrjou, Francesco Locatello, and Bernhard Scholkopf. The incomplete rosetta stone problem: Identifiability results for multi-view nonlinear ica. Uncertainty in Artificial Intelligence, 2020.
Luigi Gresele, Julius Von Kugelgen, Vincent Stimper, Bernhard Scholkopf, and Michel Besserve. Independent mechanism analysis, new concept? Advances Neural Information Processing Systems (NeurIPS), 2021.
Maya Okawa, Ekdeep S Lubana, Robert Dick, and Hidenori Tanaka. Compositional abilities emerge multiplicatively: Exploring diffusion models on a synthetic task. Advances Neural Information Processing Systems, 36:50173-50195 2023.
Core Francisco Park, Maya Okawa, Andrew Lee, Ekdeep S Lubana, and Hidenori Tanaka. Emergence of hidden capabilities: Exploring learning dynamics in concept space. Advances in Neural Information Processing Systems, 57:84698-84722 2024.
Anton Razzhigaev, Arseniy Shakhmatov, Anastasia Maltseva, Vladimir Arkhipkin, Igor Pavlov, Ilya Ryabov, Angelina Kuts, Alexander Panchenko, Andrey Kuznetsov, and Denis Dimitrov, Kandinsky: An improved text-to-image synthesis with image prior and latent diffusion. arXiv preprint arxiv:2310.03502, 2023.
Stability AI. Stable Diffusion 2.0 Release, 2022.
URL https:/ssabillty.aa/eeee
stable-diifusionn-2-rell.e.
Junsong Chen, Jincheng Yu, Chongjian Ge, Lewei Yao, Enze Xie, Yue Wu, Zhongdao Wang, James Kwok, Ping Luo, Huchuan Lu, et al. Pixart-a: Fast training of diffusion transformer for photorealistic text-to-image synthesis. arXiv preprint arxiv:2310.00420 2023a.
Ilyes Khemakhem, Diederik Kingma, Ricardo Monti, and Aapo Hyvarinen. Variational autoencoders and nonlinear ica: A unifying framework. Proceedings of the International Conference on Machine Learning (ICMI), 2020.
13

---

Published as a conference paper at ICLR 2026
Aapo Hyvarinen, Hiroaki Sasaki, and Richard Tumer. Nonlinear ICA using auxiliary variables and generalized contrastive learning. In The 22nd International Conference on Artificial Intelligence and Statistics, pages 859-866 PMLR, 2019.
Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy Vo, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et al. Dinov2: Learning robust visual features without supervision. ArXiv e-print, 2023.
Jason Xinyu Liu, Ziyi Yang. Ifrah Idrees, Sam Liang, Benjamin Schornstein, Stefanie Tellex, and Ankit Shah. Grounding complex natural language commands for temporal tasks in unseen environments. In Conference on Robot Learning, pages 1084-1110. PMLR, 2023b.
Lev Ayzenberg, Raja Giryes, and Hayit Greenspan. Dinov2 based self supervised learning for few shot medical image segmentation. In 2024 IEFE International Symposium on Biomedical Imaging (ISBI), pages 1-5. IEEE, 2024.
Nelson Elhage, Tristan Hume, Catherine Olsson, Nicholas Schiefer, Tom Henighan, Shauna Kravec, Zac Hatfield-Dodds, Robert Lasenby, Dawn Drain, Carol Chen, Roger Grosse, Sam McCan- dlish, Jared Kaplan, Dario Amodei, Martin Wattenberg, and Christopher Olah. Toy models of superposition. Transformer Circuits Thread, 2022.
Trenton Bricken, Adly Templeton, Joshua Batson, Brian Chen, Adam Jermyn, Tom Conerly, Nick Turner, Cem Anil, Carson Denison, Amanda Askell, Robert Lasenby, Yifan Wu, Shauna Kravec, Nicholas Schiefer, Tim Maxwell, Nicholas Joseph, Zac Hatfield-Dodds, Alex Tamkin, Karina Nguyen, Brayden McLean, Josiah E Burke, Tristan Hume, Shan Carter, Tom Henighan, and Christopher Olah. Towards monosemanticity; Decomposing language models with dictionary learning. Transformer Circuits Thread, 2023.
Thomas Fel, Ekdeep Singh Lubana. Jacob S Prince, Matthew Kowal, Victor Boutin, Isabel Papadim- itriou, Binxu Wang, Martin Wattenberg, Demba Ba, and Talia Konkle. Archetypal SAE: Adaptive and stable dictionary learning for concept extraction in large vision models. 2025.
Hoagy Cunningham, Aidan Ewart, Logan Riggs, Robert Huben, and Lee Sharkey. Sparse autoen- coders find highly interpretable features in language models. ArXiv e-print, 2023.
Leo Gao, Tom Dupre Tour, Henk Tillman, Gabriel Goh, Rajan Troll, Alec Radford, Ilya Sutskever, Jan Leike, and Jeffrey Wu. Scaling and evaluating sparse autoencoders. Proceedings of the International Conference on Learning Representations (ICLR), 2025.
Adly Templeton, Tom Conerly, Jonathan Marcus, Jack Lindsey, Trenton Bricken, Brian Chen, Adam Pearce, Craig Citro, Emmanuel Ameisen, Andy Jones, Hoagy Cunningham, Nicholas L Turner, Callum McDougall, Monte MacDiarmid, C. Daniel Freeman, Theodore Sumers, Edward Rees, Joshua Batson, Adam Jermyn, Shan Carter, Chris Olah, and Tom Henighan. Scaling monosemanticity: Extracting interpretable features from claude sonnet. Transformer Circuits Thread, 2024.
Senthooran Rajamanobaran, Tom Lieberum, Nicolas Sonnerat, Arthur Conmy, Vikrant Varma, Janos Kramar, and Neel Nanda. Jumping ahead: Improving reconstruction fidelity with jumprelu sparse autoencoders. ArXiv e-print, 2024.
Robert Kirk, Ishita Mediratta, Christoforos Nalmpantis, Jelena Luketina, Eric Hambro, Edward Grefenstette, and Roberta Raileanu. Understanding the effects of rlhf on Ilm generalisation and diversity. arXiv preprint 2rXiv:2310.06452, 2023.
Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model is secretly a reward model. Advances in Neural Information Processing Systems, 36:53728- 53741, 2023.
Karen Simonyan, Andrea Vedaldi, and Andrew Zisserman. Deep inside convolutional networks: Visualising image classification models and saliency maps. Proceedings of the International Conference on Learning Representations (ICLR), 2013.
Mukund Sundararajan, Ankur Taly, and Qiqi Yan. Axiomatic attribution for deep networks. Proceed- ings of the International Conference on Machine Learning (ICML), 2017.
14

---

Published as a conference paper at ICLR 2026
Ramprasaath R. Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. Grad-cam: Visual explanations from deep networks via gradient-basee localiza- tion. Proceedings of the IEEE International Conference on Computer Vision (ICCV), 2017.
Julius Adebayo, Justin Gilmer, Michael Muelly, Ian Goodfellow, Moritz Hardt, and Been Kim. Sanity checks for saliency maps. Advances in Neural Information Processing Systems (NIPS), 2018.
Amirata Ghorbani, Abubakar Abid, and James Zou. Interpretation of neural networks is fragile. Proceedings of the AAAI Conference Artificial Intelligence (AAAI), 2017.
Peter Hase and Mohit Bansal. Evaluating explainable ai: Which algorithmic explanations help users predict model behavior? Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL., 2020.
Been Kim, Martin Wattenberg, Justin Gilmer, Carrie Cai, James Wexler, Fernanda Viegas, al. Interpretability beyond feature attribution: Quantitative testing with concept activation vectors (tcav). Proceedings of the International Conference on Machine Learning (ICML), 2018.
David Bau, Bolei Zhou, Aditya Khosla, Aude Oliva, and Antonio Torralba. Network dissection: Quantifying interpretability of deep visual representations. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017.
Thomas Fel, Agustin Picard, Louis Bethune, Thibaut Boissin, David Vigouroux, Julien Colin, Rémi Cadène, and Thomas Serre. Craft: Concept recursive activation factorization for explainability. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2023a.
Matthew Kowal, Achal Dave, Rares Ambrus, Adrien Gaidon, Konstantinos G Derpanis, and Pavel Tokmakov. Understanding video transformers via universal concept discovery. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2024.
Thomas Fel, Victor Boutin, Mazda Moayeri, Remi Cadene, Louis Bethune, Mathieu Chalvidal, and Thomas Serre. A holistic approach to unifying automatic concept extraction and concept importance estimation. Advances in Neural Information Processing Systems (NeurIPS), 2023b.
David Chanin, James Wilken-Smith, Tomas Dulka, Hardik Bhatnagar, and Joseph Bloom. A is for absorption: Studying feature splitting and absorption in sparse autoencoders. ArXiv e-print, 2024.
Martin Wattenberg and Fernanda B Viegas. Relational composition neural networks: A survey and call action. ArXiv e-print, 2024.
Usha Bhalla, Suraj Srinivas, Asma Ghandeharioun, and Himabindu Lakkaraju. Towards unifying interpretability and control: Evaluation via intervention. ArXiv e-print, 2024.
Bart Bussmann, Noa Nabeshima, Adam Karvonen, and Neel Nanda. Learning multi-level features with matryoshka sparse autoencoders. arXiv preprini urXiv:2503.17447 2025.
Vladimir Zaigrajew, Hubert Baniecki, and Przemyslaw Biecek. Interpreting CLIP with hierarchical sparse autoencoders. arXiv:2502.20575 2025.
Bart Bussmann, Patrick Leask, and Neel Nanda. Batchtopk sparse autoencoders. ArXiv e-print, 2024.
Alireza Makhzani and Brendan Frey. K-sparse autoencoders. Proceedings of the International Conference Learning Representations (ICLR), 2014.
Arpita Chowdhury, Dipanjyoti Paul, Zheda Mai, Jianyang Gu, Ziheng Zhang, Kazi Sajeed Mehrab, Elizabeth G Campolongo, Daniel Rubenstein, Charles V Stewart, Anuj Karpatne, et al. Prompt- CAM: A simpler interpretable transformer for fine- ane-grained analysis. arXiv:2501.099: 00333399333 2025.
David Bau, Jun- Yan Zhu, Hendrik Strobelt, Bolei Zhou, Joshua B Tenenbaum, William T Freeman, and Antonio Torralba. GAN dissection: Visualizing and understanding generative adversarial networks. 2018.
Raphael Tang, Linging Liu, Akshat Pandey, Zhiying Jiang, Gefei Yang, Karun Kumar, Pontus Stenetorp. Jimmy Lin, and Ferhan Ture. What the DAAM: Interpreting stable diffusion using cross attention. 2022.
15

---

Published as a conference paper at ICLR 2026
Ali Borji. Qualitative failures of image generation models and their application in detecting deepfakes. Image and Vision Computing, 137:104771, 2023.
Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text- conditional image generation with CLIP latents. arXiv preprint arXiv:2204.06/25, 1(2):3, 2022.
Yang Song. Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations. arXiv preprint arXiv:2011. 13456, 2020a.
Yang Song, Prafulla Dhariwal, Mark Chen, and Ilya Sutskever. Consistency models. 2023.
Alexander Quinn Nichol and Prafulla Dhariwal. Improved denoising diffusion probabilistic models. In International conference on machine learning, pages 8162-8171. PMLR, 2021.
Jonathan Ho, Tim Salimans, Alexey Gritsenko, William Chan, Mohammad Norouzi, and David Fleet. Video diffusion models. Advances in Neural Information Processing Systems, 35:8633-8646 2022.
Haoyu Lu, Guoxing Yang, Nanyi Fei, Yuqi Huo, Zhiwu Lu, Ping Luo, and Mingyu Ding. VDT: General-purpose video diffusion transformers via mask modeling. arXiv preprint arXiv:2305.13311, 2023.
Jiuniu Wang, Hangjie Yuan, Dayou Chen, Yingya Zhang, Xiang Wang, and Shiwei Zhang. Mod- elScope text-to-video technical report. arXiv preprint arXiv:2308.0633 2023b.
Yuwei Guo. Ceyuan Yang, Anyi Rao, Zhengyang Liang, Yaohui Wang, Yu Qiao, Maneesh Agrawala, Dahua Lin, and Bo Dai. AnimateDiff: Animate your personalized text-to-image diffusion models without specific tuning. arXiv preprint arXXiv23007.0472: 2023.
Shanchuan Lin and Xiao Yang. AnimateDiff-Lighninng Cross-model diffusion distillation. arXiv preprint arXiv:2403. 12706, 2024.
Wenyi Hong, Ming Ding, Wendi Zheng, Xinghan Liu, and Jie Tang. Cog Large-scale pretraining for text-to-video generation via transformers. arXiv preprint 22X.v:2205.15866 2022.
Haoxin Chen, Menghan Xia, Yingqing He, Yong Zhang, Xiaodong Cun, Shaoshu Yang, Jinbo Xing, Yaofang Liu, Qifeng Chen, Xintao Wang, et al. VideoCrafterl: Open diffusion models for high-quality video generation. arXiv preprint wxXiv2330..951/1 2023b.
Jay Zhangjie Wu, Yixiao Ge, Xintao Wang. Stan Weixian Lei, Yuchao Gu, Yufei Shi, Wynne Hsu, Ying Shan, Xiaohu Qie, and Mike Zheng Shou. Tune-a-video: One-shot tuning of image diffussion models for text-oo-video generation. In Proceedings the IEEE/CVF International Conference Computer Vision, pages 7623-7633, 2023b.
Chen-Hsuan Lin, Jun Gao, Luming Tang, Towaki Takikawa, Xiaohui Zeng, Xun Huang, Karsten Kreis, Sanja Fidler, Ming-Yu Yu Liu, and Tsung-Yi Yi Lin. Magic3D: High-resolution text-to-3d content creation. In Proceedings of the IEEF/CVF conference on computer vision and pattern recognition, pages 300-309, 2023.
Heewoo Jun and Alex Nichol. Shap-E: Generating conditional 3d implicit functions. arXiv preprint arXiv:2305.02463, 2023.
Haochen Wang, Xiaodan Du, Jiahao Li, Raymond A Yeh, and Greg Shakhnarovich. Score jacobian chaining: Lifting pretrained 2d diffusion models for 3d generation. In Proceedings the IEEF/CVF conference on computer vision and pattern recognition, pages 12619-12629, 2023c.
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in
neural information processing systems, 33:6840-6851, 2020.
Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. arXiv preprint arXiv:2010.02502, 2020b.
Stability AI. Introducing Stable Diffusson 3, 2024.
URL
stable-diffusion-3
16

---

Published as a conference paper at ICLR 2026
Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas Müller, Joe Penna, and Robin Rombach. SDXL: Improving latent diffusion models for high-resolution image synthesis. arXiv preprint arXir:2307.01195, 2023.
Kihyuk Sohn, Nataniel Ruiz, Kimin Lee, Daniel Castro Chin, Irina Blok, Huiwen Chang, Jarred Barber, Lu Jiang, Glenn Entis, Yuanzhen Li, et al. StyleDrop: Text-to-image generation in any style. arXiv preprini arXiv:2306.0092 2023.
Zhihong Pan, Xin Zhou, and Hao Tian. Arbitrary style guidance for enhanced diffusion-based text-to-image generation. In Proceedings of the IEEE/CVF winter conference on applications of computer vision, pages 4461-4477 2023.
Lymin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models. In Proceedings of the IEEE/CVF international conference on computer vision, pages 2836-3847, 2023.
Nataniel Ruiz, Yuanzhen Li, Varun Jampani, Yael Pritch, Michael Rubinstein, and Kfir Aberman. DreamBooth: Fine tuning text-to-image diffusion models for subject-driven generation. In Proceed- ings of the IEEE/CVF conference on computer vision and pattern recognition, pages 22500-22510, 2023.
Darian Tomavsevic, Fadi Boutros, Chenhao Lin, Naser Damer, Vitomir Struc, and Peter Peer. Id- booth: identity-consistent face generation with diffusion models. arXiv preprint 034v22250..07392, 2025.
Stability AI. DeepFloyd IF: A Powerful Open-Source Texte Model, 2023. URL https:
Black Forest Labs. Announcing Black Forest Labs, 2024. URL https://blacfforetllbbb ai/
brrouuniing-blaa--foorss
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image worth 16x16 words: Transformers for image recognition at scale. Proceedings of the International Conference Learning Representations (ICLR), 2020.
Jiahui Yu, Yuanzhong Xu, Jing Yu Koh, Thang Luong, Gunjan Baid, Zirui Wang, Vijay Vasudevan, Alexander Ku, Yinfei Yang, Burcu Karagol Ayan, et al. Scaling autoregressive models for content- rich text-to-image generation. arXiv preprint arXiv:2206. 10789, 2(3):5, 2022.
Joel Hestness, Sharan Narang, Newsha Ardalani, Gregory Diamos, Heewoo Jun, Hassan Kianinejad, Md Mostofa Ali Patwary, Yang Yang, and Yanqi Zhou. Deep learning scaling is predictable, empirically. arXiv preprint arXiv: 1712.00499, 2017.
Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Wortsman, et al. LAION-5B: An open large-scale dataset for training next generation image-text models, Advances in neural information processing systems, 35:25278-25294 2022.
Common Crawl. Common crawl corpus.
2017.
Minwoo Byeon, Beomhee Park, Haecheon Kim, Sungjun Lee, Woonhyuk Baek, and Saehoon Kim. COYO-700M: Image-text pair dataset, 2022.
Piyush Sharma, Nan Ding, Sebastian Goodman, and Radu Sorieut. Conceptual Captions: cleaned, hypemymed, image alt-text dataset for automatic image captioning. In Proceedings of ACL, 2018.
Abeba Birhane, Sanghyun Han, Vishnu Boddeti, Sasha Luccioni, et al. Into the LAION's den: Investigating hate in multimodal datasets. Advances in neural information processing systems, 36: 2023.
Abeba Birhane, Sepehr Dehdashtian, Vinay Prabhu, and Vishnu Boddeti. The dark side of dataset scaling: Evaluating racial classification in multimodal models. In Proceedings of the 2024 ACM Conference Fairness, Accountability and Transparency, pages 1 1224, 2024.
17

---

Published as a conference paper at ICLR 2026
Preethi Seshadri, Sameer Singh, and Yanai Elazar. The bias amplification paradox in text-to-image generation. arXiv preprint arXiv:2308.. 00755, 2023.
Abeba Birhane, Vinay Uday Prabhu, and Emmanuel Kahembwe. Multimodal datasets: Misogyny, pornography, and malignant stereotypes. arXiv preprint arXiv:2110.01963, 2021.
David Thiel. Identifying and eliminating CSAM in generative ml training data and models. Stanford Internet Observatory, Cyber Policy Center, December, 23:3, 2023.
Ali Shirali and Moritz Hardt. 157,69, 2023.
What makes ImageNet look unlike LAION. arXiv preprint
Thao Nguyen, Samir Yitzhak Gadre, Gabriel Ilharco. Sewoong Oh, and Ludwig Schmidt. Improving multimodal datasets with image captioning. Advances in Neural Information Processing Systems, 16:22047-22066 2023.
Bruno A Olshausen and David J Field. Emergence of simple-cell receptive field properties by learning sparse code for natural images. Nature, 1996.
Michael Flad. Sparse and redundant representations: from theory applications in signal and image processing. 2010.
Julien Mairal, Francis Bach, and Jean Ponce. Sparse modeling for image and vision processing. Foundations and Trends in Computer Graphics and Vision, 2014.
David L Donoho. Compressed sensing. IEEE Transactions Information Theory, 2006.
Emmanuel J Candès, Justin Romberg, and Terence Tao. Robust uncertainty principles: Exact signal reconstruction from highly incomplete frequency information. IEEE Transactions on Information Theory, 2006.
Stuart Lloyd. Least squares quantization in pcm. IEEE Transactions on Information Theory, 1982.
Daniel D Lee and H Sebastian Seung. Learning the parts of objects by non-negative matrix factoriza- tion. Nature, 1999.
Nicolas Gillis. Nonnegative matrix factorization. 2020.
Hui Zou, Trevor Hastie, and Robert Tibshirani. Sparse principal component analysis. Journal of Computational and Graphical Statistics, 2006.
Michal Aharon, Michael Elad, and Alfred Bruckstein. K-svd: An algorithm for designing overcom- plete dictionaries for sparse representation. IEEE Transactions on Signal Processing, 2006.
Julien Mairal, Francis Bach, Jean Ponce, and Guillermo Sapiro. Online dictionary learning for sparse coding. Proceedings of the International Conference on Machine Learning (ICML), 2009.
Rodolphe Jenatton, Guillaume Obozinski, and Francis Bach. Structured sparse principal component analysis. International Conference on Artificial Intelligence and Statisticc,, 2010.
Daniel A Spielman, Huan Wang. and John Wright. Exact recovery of sparsely dictionaries. The Journal of Machine Learning Research (JMLR), 2012.
Jean Barbier and Nicolas Macris. Statistical limits of dictionary learning: random matrix theory and the spectral replica method. Physical Review E, 2022.
Vardan Papyan. Yaniv Romano, and Michael Elad. Convolutional dictionary learning via local processing. Proceedings of the IEEE International Conference on Computer Vision (ICCV), 2017.
Alex Tamkin, Mohammad Taufeeque, and Noah D Goodman. Codebook features: Sparse and discrete interpretability for neural networks. ArXiv e-print, 2023.
Berk Tinaz, Zalan Fabian, and Mahdi Soltanolkotabi. Emergence and evolution of interpretable concepts in diffusion models. arXiv:2504, 2025.
18

---

Published as a conference paper at ICLR 2026
Viacheslav Surkov, Chris Wendler, Mikhail Terekhov, Justin Deschenaux, Robert West, and Caglar Gulcehre. Unpacking SDXL turbo: Interpreting -ext-to-image models with sparse autoencoders. ArXiv e-print, 2024.
Akshay Kulkarni, Ge Yan, Chung-En Sun, Tuomas Oikarinen, and Tsui- -sui-Wei Weng. Interpretable generative models through post-hoc concept bottlenecks. arXiv preprint arXiv:2503. 19377, 2025.
Mert Yuksekgonul, Maggie Wang, and James Zou. Post-hoc concept bottleneck models. arXiv preprint arXiv:2205. 15480, 2022.
Can Demircan, Tankred Saanum, Akshay Jagadish, Marcel Binz, and Eric Schulz. Sparse autoencoders reveal temporal difference learning in large language models. arXiv preprint arXiv:2410.01280, 2024.
Matthew Lyle Olson, Musashi Hinck, Neale Ratzlaff, Changbai Li, Phillip Howard, Vasudev Lal, and Shao- Yen Tseng. Analyzing hierarchical structure vision models with sparse autoencoders. In Proceedings of the Computer Vision and Pattern Recognition Conference, pages 4835-4839, 2025.
Karol Gregor and Yann LeCun. Learning fast approximations of sparse coding. In Proceedings of the 27th international conference on international conference on machine learning, pages 399-406, 2010.
Xiaohan Chen, Jialin Liu, Zhangyang Wang, and Wotao Yin. Theoretical linear convergence of unfolded ista and its practical weights and thresholds. Advances in Neural Information Processing Systems, 31, 2018.
Pierre Ablin, Thomas Moreau, Mathurin Massias, and Alexandre Gramfort. Learning step sizes for unfolded sparse coding. Advances in Neural Information Processing Systems, 32, 2019.
Bahareh Tolooshams and Demba Ba. Stable and interpretable unrolled dictionary learning. arXiv preprint arXiv:2106.00 00058, 2021.
Benoît Malézieux, Thomas Moreau, and Matthieu Kowalski. Understanding approximate and unrolled dictionary learning for pattern recovery. arXiv preprint arXiv:2/06.06338, 2021.
Sanjeev Arora, Rong Ge, Tengyu Ma, and Ankur Moitra. Simple, efficient, and neural algorithms for sparse coding. In Conference on learning theory, pages 1 113-149, PMLR, 2015.
Sai Sumedh R Hindupur, Ekdeep Singh Lubana, Thomas Fel, and Demba Ba. Projecting assumptions: The duality between sparse autoencoders and concept geometry. arXiv preprint 21X.02250. 2025.
Tuomas Kynkäänniemi, Tero Karras, Samuli Laine, Jaakko Lehtinen, and Timo Aila. Improved precision and recall metric for assessing generative models. Advances in neural information processing systems, 32, 2019.
Jingyi Xu, Hieu Le, and Dimitris Samaras. Assessing sample quality via the latent space of generative models. In European Conference on Computer Vision, pages 449-464. Springer, 2024.
Iro Laina, Yuki M Asano, and Andrea Vedaldi. Measuring the interpretability of unsupervised representations via quantized reverse probing. MXXiv:220.03326. 2022.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pages 8748-8763. PmLR, 2021.
Pang Wei Koh, Thao Nguyen, Yew Siang Tang, Stephen Mussmann, Emma Pierson, Been Kim, and Percy Liang. Concept bottleneck models. Proceedings of the International Conference on Machine Learning (ICML), 2020.
Chris Hamblin, Thomas Fel, Srijani Saha, Talia Konkle, and George Alvarez. Feature accentuation: Revealing' what' features respond to natural images. ArXiv e-print, 2024.
Colin McDiarmid et al. On the method of bounded differences. Surveys in combinatorics, 141(1): 148-188, 1989.
19

---

Published as a conference paper at ICLR 2026
APPENDIX CONTENTS
Related Work
22
A.I
Explainability in Vision
22
A.2 Generative Image Models
22
A.3 Datasets for Generative Image Models
22
A.4
Concept Discovery and Sparse Coding in Generative Image Models
23
Comparison with Existing Approaches
24
Exploratory Tool
25
Experimental Setup
26
D.I
Observation Space
26
D.2
Synthesized Images
26
D.3
Distribution Level Analysis
26
D.4
Datapoint Level Analysis
27
D.5
Co-occurrence Analysis
27
Computational Resources
28
Custom RA-SAE
29
F1
Training Details
29
Autointerpeetability
30
F3
Examples of Learned Concepts
32
Additional Results: Qualitative Examples of Blindspots
36
Model-Specific Blindspots
................................
37
G.2
Stress Testing
................................
39
G.2.1
Bird Feeder Blindspot in Kandinsky
39
G.2.2
Glossy DVD Disc Blindspot in SD 1.5
40
H
Additional Results:
Higher-order Blindspots with Compositional Discrepancy
41
Additional Results: Caption Noise in High Divergence
43
Methodology
43
Results
43
1.3
Analysis
43
J
Additional Results: SAE Error Cases
44
J.1 Methodology
44
2
Results
44
20

---

Published as a conference paper at ICLR 2026
J.3
Analysis
45
J.3.1
Error Cases: Natural Images
47
J.3.2
Error Cases: Al-Generated Images
47
Concentration bounds for
48
Monotonicity and Calibration-Fres Interpretation of 8
49
M Stability of FID Under SAE Embeddings
50
Additional Examples of Synthesized Images
52
21

---

Published as a conference paper at ICLR 2026
A RELATED WORK
A.1 EXPLAINABILITY IN VISION
Early work in explainable AI, including computer vision, focused on methods for attribution of influential input regions Simonyan et al. (2013); Sundararajan et al.] y0 /Selvarauu et al. (2017). However, these methods offered limited semantic information about learned representations and often produced incorrect explanations Adebayo et al. (2018); Ghorbani et al.
(2017): Hase and Bansal (2020). To address these issues, concept-based interpretability Kim et al. (2018) emerged identify
semantically meaningful directions in neural networks, revealing not just where they look but what concepts and structures they employ Bau et al. (2017): Fel al. (2023a): Kowal al. (2024).
Recent work demonstrates that popular concept-based interpretability methods Ghorbani et al. (017 CRAFT al. (2023a), and Sparse Autoencoders (SAEs [Cuniingham et al. (2023) [Brikken et al. 2023) essentially address the same dictionary learning task under different constraints Fel et al. (2023b). Out of these approaches, sparse autoencoders (SAEs) have emerged as particularly scalable for pancept-bbsedd interpretability. While recent studies reveal some limitations of the original SAEs including overly specific features [Chanin et al. (2024), compositionality challenges Wattenberg and Viegas (2024), and limited intervention effects Bhalla et al. (1024)/improved SAE versions have emerged, including archetypal SAB (RA-SAE) Fel et al. (2025), hierarchical approaches Bussmann
сt al. (2025) : Zaigrajew et al. (2025), and variants addressing specific architectural choices Bussmann et al.
2024 Makhzani and (2014).
Beyond SAEs, other interpretability methods include prompt-based probing Chowdhury et al. (2025). attention map or activation visualizations Bau et al. (2018); Tang et al. (2022), and dataset-level statisties Dombrowski et al. (2024); (wwang et al. (2024) (e.g., diversity distribution coverage metrics) offer only partial insights ghts to answer these questions. Crucially, they lack granularity, focusing on full images or prompts instead of fine-graines features and concepts Theis et al. (2015); Naeem iet al. (2020). Furthermore, they depend on subjective interpretation and do not distinguish between various failure models [Borji (2023). The existing methods and metrics are hence inadequate in systematically identifying feature- and concept-level weaknesses of generative image models Stein
al. (2023).
A.2 GENERATIVE IMAGE MODELS
Diffusion-based methods have become dominant across various modalities in generative vision modeling, including image Saharia et al. (2022) Ramesh et al. (2022) Song et al. (2020a) 2023):
Nichol and Dhariwal (2021), ), video Ho al. (2022) Lu al. (2023) Wang et al.
(2023b) g Guo
et al. 2 Lin and Yang (024); Hong et al. (2022) ; Chen et al. (2023b): Wu et al. (2023b), and Poole al. (2022); Lin et al. (2023); Jun and Nichol (2023); Wang et al. (2023c). In the domain of image generation, this can be traced back to denoising diffusion probabilistic models (DDPMs) Ho
al. 2220)) which were later extended to non-Markov diffusion processes with denoising diffusion implicit models (DDIMs) (Song et al. (2020b).
The Stable Diffusion (SD) Rombach et al. (2022) model family made DDMPs highly accessible both in the research and oppen-source communities. The original SD was followed by several subsequent versions, including SD 2 (Stability Al (2022), SD 3 Stabiiityy Al (022)). SD XL ct al. (2023). Many modifications and extensions of the SD architecture have emerged, enabling additional constraints for the diffusion process (e.g., style [Sohn et al. (2023): Pan ct al. (023).. pose Zhang et al. (2023), , and identity Ruiz et al.) (2023); Tomavsevic et al. 2025 as well different input modalities, such as image-to-in toee-ttt-mmgeee generation. Different Latent Diffusion Models built on top of SD including Kandinsky Razzhigaev al. (2023), PixArt al.
| Podell
2023a DeepFloyd Saharia have also emerged.
et al. 2022); Stability A] (2023), and FLUX Black Forest Labs (2024
A.3 DATASETS FOR GENERATIVE IMAGE MODELS
The recent success of generative vision models is largely attributed to the abundance of computational resources and large-scale internet datasets Dosovitskiy all (2020); Yu et al. (2022); Hesstness et al.) (2017). Specifically, LAION-5B Schuhmann et al. (2022) has played a key role in the training of open-source text-to-image models, including SD and its derivatives. This dataset, scraped from
22

---

Published as a conference paper at ICLR 2026
Common Crawl Common Crawl (2017), contains over 5B image-caption pairs, 2.3B of which are in English. Other prominent datasets include COYO-700M Byeon et al. (2022) and Conceptual
Captions Sharma al. (0018), with 700M and 3M image-caption pairs, respectively.
As LAION-5B gained popularity, concerns grew over its biases Birhane et al. (2023
2024); Seshadri
et al.
(202)): Birhane et al. (2021); Thiel (2023) Despite filtering attempts, harmful content per-
sisted Birhane et al. (2023 2024 Seshadri et al. (2023), including NSFW material Birhane et al.
(2021) and hundreds of CSAM instances Thiel (202)), prompting temporary removal from official
channels. The dataset also suffers from low-quality images Shirali and Hardi (2023) and internet-style
captions (e.g., product descriptions) that misalign with how users prompt trained models Nguyen
itt al. (2023).
A.4 CONCEPT DISCOVERY AND SPARSE CODING IN GENERATIVE IMAGE MODELS
Dictionary learning seeks to find sparse representations of input data, where each sample can be recon- structed using linear combination of few dictionary atoms Olshausen and Field (1996); Elad (2100)
Mairal et al. (0014). Built upon compressed sensing theory Donoho (2006); Candès et al. 2006), the field evol ved from early Vector quantization methods Lloyd (1982) to sophisticated approaches including Non-nen Von-tvegtive Matrix Factorization Lee and Seung 1999), Gillis (2020). Sparse PCA Zou
al. (2006), and K-SVD/Aharon et al./ (2006). Recent advances include online methods MMarall et al.
(2009), structured sparsity Jenatton et al. (2010), and theoretical guarantees SSpielman et al.
2012);
Barbier and Macris (2022). alongside growing connections to deep learning Papyyn et al.
201
Tamkin et al. (2023).
Advances in sparse coding have also been leveraged to study the emergence of high-level concepts inside diffusion models Tinaz al. (2025); Surkov et al. (0024). Prior to diffusion models, concept- grounded interpretability has been deployed to earlier generative architectures through concept- bottleneck models, which require human intervention at training time Kulkarni et al. (202)), and post-hoc detectors that retrofit concept supervision Yuksekgonul et al. (2022). However, both of these approaches require human-defined concepts and hence inherently miss broader trends that the user does not explicitly register. Recent work has also explored SAF applications to understanding temporal dynamics in language models Demircan et al. (2024). hierarchical structure in vision models Olson et al. (2025). and theoretical connections between autoencoders and sparse coding through unrolled optimization approaches Gregor and LeCun 2010); Chen al. (2018); Abbin et al. (2019) Tolooshams and Ba (2021); Malézieux et al. [2021] Arora et al. (2015): Hindupur et al. (0025)
Aggregate metrics (e.g., precision, recall, density, and coverage Kynkäänniemi et al. (011 Naeem ct al. (2020)) and latent density scores, which predict sample quality based on the model's latent space Xu et al. (2024), have emerged to evaluate generative image model capabilities. While these effectively uncover distributional gaps, they offer little insight into specific concepts that are under- or over-represented.
Together, these limitations motivate a scalable, unsupervised framework that can systematically identify and quantify concept-level failure modes in generative image models Laina et al. (2022).
23

---

Published as a conference paper at ICLR 2026
B COMPARISON WITH EXISTING APPROACHES
Table summarizes the existing approaches for evaluating consistency and semantic coverage of generative image models, and compares them to our method.
Fréchet Inception Distance (FID), Heusel al. (2017). This metric embeds images using Inception-v3 and calculates the Wasserstein-2 distance between natural and generated distribu- tions. While it regularly used to encode the overall quality of a model, aggregates many potential failure modes into single scalar. It is hence incapable of surfacing specific conceptual blindspots.
CLIPScore, Radford et al. (021). CLIPScore computes the cosine similarity between the em- beddings text prompt and a generated image to assess their consistency. This method bound by the prompt itself; it cannot detect blindspots for concepts that are not explicitly included in the evaluation prompt set.
Improved Precision and Recall, Kynkäänniemi et al. (2019). This evaluation framework esti- mates the manifold of real and generated data using k-Nearest Neighbor (k-NN) radii to separately quantify precision (fidelity) and recall (coverage). While a drop in recall implies the existence of distributional blindspots (mode collapse), the metric cannot identify which concepts are missing.
GAN Dissection, Bau et al. (2018). This method correlates the activation maps with semantic segmentation masks to identify units responsible for specific concepts. Although offers high granu- larity, it is computationally intensive and limited to the fixed vocabulary of the external segmentation network used for supervision.
Adversarial Search (SAGE), Liu et al. (2023a). SAGE treats the generative image model as an adversary and optimizes text tokens to discover prompts that maximize divergence from surro- gate classifier. While effective at identifying specific error cases, it lacks a structured representation of the full conceptual space, and the iterative optimization makes it prohibitively expensive.
Concept Bottleneck Models (CBMs), Koh al. (2020). These architectures explicitly force the neural network to compress information into layer where neurons correspond to pre-defined human concepts. This requires training models from scratch with concept-labeled data, making it unsuitable for the post-hoc evaluation of pre-trained foundation models.
Table 1: Comparison of Approaches to Identifying Conceptual Blindspots in Image Models.
Method
Specificity
Unsupervised
Scalability
Exaggeration
FID
Precision & Recall
CLIPScore
X
X
GAN Dissection
X
SAGE
X
X
X
CBMs
X
Human Evaluation
X
Our Method
V
Evaluation Criteria
1. Specificity. The method supports notion of discrete concepts (these may be defined in text, via examples, a dictionary or otherwise; there may also exist a taxonomy/hierarcyy).
2. Unsupervised. For specific concept be deemed as a blindspot, the method does not require for the user to explicitly define or describe it.
3. Scalability, The complete conceptual space of the evaluated model, as conceptualized by the framework, can be feasibly searched (given a conceptual space of 2 1,000 concepts).
4. Exaggeration. The method can detect both suppression and exaggeration.
24

---

Published as a conference paper at ICLR 2026
C EXPLORATORY TOOL
Shown in Figure 12 is an overview of the exploratory tool developed alongside this project to facilitate inspection and comparison of concept-level energy differences. The tool is web-based interface built around UMAP projection of concept representations, enabling visualization and comparison of concept-level energy differences. It publicly available at phttpss//saee diff. githubt........ 10/ along with pre-computed energy difference data for the four models evaluated in this work (SD 1.5, SD 2.1, PixArt, and Kandinsky). All subsequent analyses in this paper are derived from insights enabled by this tool. Its primary functionalities, which support these analyses, include:
Contrast different models and architectures. For each evaluated model, the tool provides UMAP visualization spanning all 32,000 concepts from the RA-SAE. Each scatter point represents an individual concept, color-coded by its energy difference.
Inspect concepts. Each concept has card with key statistics, representative real and generated images (æz, æ²), and visualized co-occurrence patterns. Explore blindspots. Beyond the UMAP and per-concept views, the tool features global rankings of suppressed and exaggerated blindspots, helping to highlight the most notable conceptual blindspots.
Contrast various models & architectures
Explore blindspots
Inspect individual concepts
Overview of the Exploratory Tool. The web interface displays a UMAP projection for each evaluated model, where each dot represents concept, color-coded by its energy difference. When a concept selected, a detail panel presents illustrative images, statistics, and the most representative natural and generated images and x'. An ordered list of the concept's 's co-ocurrrences shown alongside global rankings of blindspots.
Figure 12:
25

---

Published as a conference paper at ICLR 2026
D EXPERIMENTAL SETUP
This section details the experimental setup for analysis of four popular generative image models: SD 1.5/2.1, Kandinsky, and Aix,rt, all trained on LAION-5B or its subsets/derivatives. The code available at httpp:://itthbb.. http:////...oeeeefo/eeeeeerreiie
D.1 OBSERVATION SPACE
The observation space is constructed by sampling 10, 000 image-text pairs from the LAION-5B dataset (Schuhmann et al. (2022). which serves as our domain natural images. Due concerns with CSAM and other unsafe content in the dataset, the original data release no longer available. A substitute release of a subset this dataset with additional filtering the unsafe content, available at https://huggingt ace.. htt//daaatss///aaeeeeae paataatts//aaion/eelaaiontt
en- research- safe is used.
The sampling procedure consists of: (1) loading the full LAION dataset using the Hugging Face datasets library, (2) performing validation to ensure proper URL structure and resource availability via HTTP HEAD requests, and (3) employing random sampling with replacement until reaching the target count of 10, 000 valid samples. This approach, yielding Da with (æc tuples, ensures observation space contains accessible image-text pairs for comparative analysis of a dataset of image URLs whose large portion has been made unavailable since original release. Additional examples of synthesized images are shown in Appendix N
D.2 SYNTHESIZED IMAGES
For each of the four evaluated models, we generate a synthetic dataset Dgo to have one-to-one correspondence with De. yielding triplets (æ, x', t). Specifically, given the 10,000 image-text pairs (x. from DG, we use to synthesize counterpart images x' using each generative model go-
The synthesis process follows the standard text-to-image generation pipeline for each model archi- tecture, implemented using the Hugging Face dif diffusers library. where the models are loaded at mixed precision (fp16) 6). All synthetic images are generated at 512 512 pixel resolution with default parameters.
Stable Diffusion 1.5. The checkpoint from https::/huggiingface http:///ninnaaa.cce///naaaaaiiaa stable-diffusion-viie (which is a mirror of the deprecated co/baenammin-painn/ wounayml/stable-one www.sblleedfffiiooo-oooo fusioonv11) used. Inference performed using 50 inference steps. with the guidance scale fixed at 7.5.
co/
Stable Diffusion 1.5 + DPO. The DPO variant of SD 1.5 (used in the analysis in Sec. 4.5) follows the baseline SD 1.5 implementation, but replaces the UNet component with a DPO-trained version from httt/ttt::/huggingeeeee http://wwugung.......commaaaaaa/ co/nho/ng//ppp/////ooooo sd1 5-texx2image--
Stable Diffusion 2.1. The checkpoint from https://hugginggaet http:///waa..c.cttaiiiititttcttt co.stabiliyyai stable-diffusion22 is used. Inference is performed using 50 inference steps, with the guidance scale fixed 7.5.
Kandinsky. The checkpoint from https::/hugginngaac. wwwee/eciccannccccooooooaa ty/ kandinsky-2-11 is used. Inference is performed using 100 inference steps, with the guid- ance scale fixed at 4.0.
PixArt. The checkpoint PixArt-XL -L-2-1024-M6 is used. Inference is performed using 50 inference steps, with the guidance scale fixed 7.5.
from https://huggingfface http://iwinnnaaa...o//aatttattaa co.PixArt-allppall
D.3
DISTRIBUTION LEVEL ANALYSIS
Section 4.1
We compute energy differences 5(-) across all 32,000 concepts for each evaluated model. The sigmoid transformation with temperature T = 0.8 is applied during normalization. The resulting values visualized log-scale density histograms with 100 bins spanning [0, 1).
26

---

Published as a conference paper at ICLR 2026
We embed the complete set of 32,000 concepts into two-dimensional space using UMAP applied to the sparse concept codes. Each point in this UMAP represents an individual con- cept, colored according to its energy difference 8(.), emphasizing both suppressed and exaggerated blindspots. To quantify cross-model consistency, we compute pairwise Pearson correlation coeffi- cients between o()) vectors of all model pairs, producing both scatter plots and correlation matrices. This analysis reveals whether blindspots cluster in conceptual space and identifies model-specific versus universal patterns of conceptual blindspots.
Section 4.2
We rank all 32,000 concepts by their energy difference 8(),, and manually examine the extrema (both suppressed and exaggerated blindspots). For suppressed blindspots, we select concepts with 5(.) < 0.1; for exaggerated blindspots, we choose those with 8(.) 0.9. Presented examples are manually annotated with textual descriptions of the respective concepts through inspection of their most activating images and spatial attention patterns. We outline ongoing efforts automate this concept interpretation in Appendix F.2
Section 4.3
Section 4.5 We compare 1.5 with and without DPO in the following fashion: for each image
pair (z. x') we compute the L.2 norm of the difference between their concept energy vectors [l5(x') - {(x) 12. We apply sigmoid transformation with temperature T = 0.8 to the dement-wise differences before taking their mean. This yields datapoint-wise energy differences that quantify how much each generated image deviates from its natural counterpart in concept space. Finally, these differences are visualized as overlapping histograms, contrasting both model variants.
For each concept Ck. its empirical frequency Zz,aloo (the count of non-zero activations across the natural dataset) is counted. A sigmoid normalization with temperature T 0.4 is then applied to the energy differences 8(.) The analysis is visualized using scatter plots where the x-axis is the empirical concept frequency and the y-axis is the cemooddtransformmen energy difference. The point sizes are proportional to activation frequency and point colors are proportional to the magnitude of energy differences.
Section 4.6
D.4
DATAPOINT LEVEL ANALYSIS
Section 4.4
For each image pair (ær, x²), we compute the L2 norm of the difference between their concept energy vectors [|E(x') {(x)]|2. This yields a scalar measure of conceptual divergence for each image pair. The samples are ranked by their energy differences. Minimal divergence indicate potential memorization artifacts and maximal divergence point to significant conceptual failures. This analysis enables qualitative inspection of specific failure modes.
D.5
CO-OCCURRENCE ANALYSIS
For both the natural and synthesized data DG and Dgo. concept co-occurrence patterns are analyzed through the co-activation matrix 7f 7, which holds pair-wise correlations in concept usage. Spectral analysis performed examine the dominant conceptual directions using eigendecomposition. The alignment between natural and synthetic co-occurrence structures is assessed using cosine similarity heatmaps between the top-100 cigenvectors of each co-occurrence matrix. These 100 x 100 similarity matrices visualized as square heatmaps where perfect diagonal alignment would indicate identical principal concept axes, while off-diagonal patterns reveal would revolve rotations and mismatches in compositional geometry.
Appendix H
27

---

Published as a conference paper at ICLR 2026
E COMPUTATIONAL RESOURCES
This section summarizes the GPU resources used for training and experiments in support of this paper. In total, we used upproximately 202 GPU-hours on NVIDIA H100s and H200s.
RA-SAE. Trained for approximately 24 GPU-hours on three NVIDIA H100s.
Synthesized Images. Generating the full Dg, (see Appendix D took roughly hours per generator when distributed across four NVIDIA H200 GPUs. With five generators, this totaled approximately 100 GPU-hours on a NVIDIA H200.
Distribution-Level Analysis. Extracting energy differences at the distribution level took about hours per generator on a single NVIDIA H200 GPU (total ~ 15 GPU-hours).
Datapoint-Level Analysis. Computing datapoint-level energy differences, ranking concepts per datapoint, and ranking datapoints per concept also took approximately hours per generator on one H200 GPU (total 15 GPU-hours).
28

---

Published as a conference paper at ICLR 2026
F CUSTOM RA-SAE
This section introduces our custom relaxed archetypal sparse autoencoder (RA-SAE), its training configuration, autointerpretabiiiyy pipeline, and examples of learned concepts. The model open- sourced at anonymized
Sparse Autoencoders. SAEs decompose the high-dimensional activation space of models into sparse, human-interpretable concepts. Specifically, SAEs enforce a sparsity constraint so that each activation vector is reconstructed using only small subset of learned feature directions (i.e. concepts), which helps disentangle overlapping information (superposition) into more separable parts. Once trained, each concept is assigned human- human-interpres interpretaable label, either by human annotators or via autointerpretability pipelines (for example using a vision-language model). To assist with this labeling, high-activating exemplars (inputs that yield strong activation for that concept) identified, and recurring visual or semantic patterns across those exemplars are described.
Archetypal SAFs. Regular SAEs suffer from instability: small changes initialization, data, training lead to different learned dictionaries. Archetypal SAEs (A-SAEs) mitigate this by constraining dictionary atoms (feature directions) to lie within the convex hull of the data; that is, each concept vector must be expressible convex combination of actual activation vectors from the data. This geometric anchoring forces the learned features (atoms) to be more directly tied to the underlying data, improving stability. Relaxed Archetypal SAEs (RA-SAEs) loosen this constraint somewhat to allow more flexibility (better reconstruction ability) while retaining much of the stability benefits. Empirically, RA-SAFs have been found to match or outperform regular SAES in benchmarks of plausibility (how well learned directions recover known classification or semantic directions) and identifiability (how well they disentangle synthetic mixtures of concepts), producing more stable and semantically meaningful concepts.
Our Configuration. Our custom RA-SAE was trained top of DINOv2 representations using the Top-K sparsity constraint Gao et al. (2025). has 32, 000 concepts, making it largest RA-SAE to date. Training details are reported App. We describe the autointerpretabiliit pipeline to assign Finally, examples of concepts learned by the RA-SAE are given in
labels to concepts in App. F2
App.
F.1 TRAINING DETAILS
The auto-encoder is trained on the complete ImageNet-Il 1k training split, (< 1.28 M) RGB images. Each image is converted to 261 visual tokens using DINOv2 Oquab et al. (2023): tokens are fed to the SAE without class position embeddings. The total number of training tokens is therefore 50 X 1.28M x 261 x= 1.67 x 10
Dataset.
Dictionary. The dictionary has 32,000 concept dimensions. For the sparse activation rule, top-k masking with & 5, is used; activations outside the largest five per input set to 0. The weights are initialized using Xavier/Glorot. The training is conducted at mixed precision (fp16), with the last ten epochs performed at full precision.
Optimizer and Schedule. The model is trained for 50 epochs using base AdamW (8, 0.9, 32 = 0.999) optimizer is employed with weight decay set to 10 Linear warm-up is applied on the first 5% of steps, followed by cosine decay from 7mmax = 5 10-4 to M/mal = 10-6 MSE loss is used alongside an auxiliary term penalizing activations that never enter the top-k set, where À = 10-5
29

---

Published as a conference paper at ICLR 2026
F.2 AUTOINTERPRETABILIT
In effort to automate the interpretation of concepts identified by the RA-SAE, qualitatively evaluated the ability of "ssion-Language Models (VLMs) to describe them. In particular, we queried ChatGPT 4o via the OpenAl API using various prompts and formats of image x" in which the given concept was salient, recording the description generated by the VLM. In this section, we highlight the findings from exploration. For evaluation purposes, allowed the VLM to provide its full reasoning; to make this a scalable solution, simple modification of the prompt-asking the
VLM provide only the description- filter out the reasoning.
First, we provided the VLM with two images: raw version of x" and x" with a heatmap superim- posed to localize the intensity of the concept:
Input
Output
The concept identifled by the DMNOV based SAE these images sppears to correspond "ring" or "metal ring" as the salient object.
Reasoning:
In the original image, sleek, bruehwdemetal ring centered on worden surfice. In the heatmap overlay, regions that activated (least with bright blocks) righly reflections.
partcularly
You are given two images: (11 raw image and (2) the same image with heattmup seliant
minor scattaned activations, The wooden background mostly ignoved background confirinn that
present the image, as idertffied by DINOv2-based SAE Describe what this
consistent and strong focus metallic, cimular object suggests model mecogniees this the defining element-likely trained to detect the shape, material, and reflectivity typical
As seen in the example, the VLM did not interpret the heatmap visualization correctly. Instead of describing the wood texture in the background the actual salient region it described the ring in the foreground, which received no attention from the RA-SAE.
To determine whether this prompting issue, tested the scenario again-this this time using an expanded prompt that explained the heatmap in greater detail:
Input
Output
The concept idertified by the DINOV2 SAE these images appears be "ring" or specificall "metal band/jawelly ring."
Here's is the reseoning: the heatmap
the red and yellow (most salient) migions wign very doseey with image.
ring the
You are given two images (1) raw image and (2) the same with experinpseed heatmep contain cancept idensfied by DINOv2- besed SAE, Red areas are the most salient the concept).
ring the only distinet ob object the photo and clearly segmented from the
backg round (wooden surface), which mostly marked in blue and green Boast salient.
atanntion doas focus
sexture of background lighting but highly to curved, metalic object.
lcontain the Desoribe what concept.
Thus, the concept is almoet certwily the meta lie ring-its s'ape, reflectviity and material trigger spame SAE dicrionary.
making learned concept
This prompt modification did not resolve the issue, the VLM still identified the ring the prominent concept in the image. Several other prompting techniques, including those describing the heatmap in greater depth (e.g., by including the code used for its generation), were explored with similar results.
30

---

of Published as a conference paper at ICLR 2026
We next tested providing the VLM only with the beatmap-superimposed x" and similar prompt explaining how to interpret the heatmap:
Input
Output
You are given image superimpoted heatmsp of areas that concept icentifen by most sallent Icontain the concept) blue areas are the least saliant (do not contain conceppd) Describa this concapt.
concapt idantified by DINCvZ-baee more broadly, "cicular metallic object' The most salient (rea]) aress are focused fightly around iself tself-orpeecall regions strongy
image uppears to "ring"
ediges and reflective inniciinnn that the
concept.
The surouncling wooden surface mostly blke and green, which shows it not relevant concapt, supporting idea madel isalating ring
Once again, the result failed identify the wooden background as the prominent concept in the image and instead focused on the ring. As the previous case, we experimented with various prompting strategies using this image format, but the results remained largely unchanged.
From these observations, we concluded that the heatmap alone is insufficient to guide the VLM's attention to the salient areas. The presence of less relevant regions appears to distract the model. To address this, we replaced the heaamap--uupernppssed x" with an alpha-masked version of x":
Input
Output
are given an image where only areas that contain concapt idantified by DINOx2-based SAE preserved: concept Describe what this corcept.
preserved image appoars "wood grain." The retained aross show cansistent pattarns typical of wooden surfcces linwer Rirous streaks warthy tones like beige, gray. and brown. The selective masking highlights image SAE learned to iollate the visual festures wood flkklly flooring or table).
concept
texture"
prominent, suggesiing
As shown in the example, the VLM was now able to correctly identify wood texture as the dominant concept. We observed consistent success across a range of concepts and images.
The formatting of x" and the prompt shown above yielded the most reliable results in qualitative evaluation. However, we note that this evaluation is limited by its qualitative nature (due to the absence of ground truth annotations) and its focus single VLM. We hope future work the autointerpretability of SAE concepts can build on and expand this analysis.
31

---

Published as a conference paper at ICLR 2026
F.3 EXAMPLES OF LEARNED CONCEPTS
Shown below is representative sample of concepts learned by our RA-SAE. For each concept, present 12 images for which the concept had the highest activation in the ImageNet dataset (on the left) and localization of the respective concept within those images (on the right). Additionally, epitome constructed using the Feature Accentuation method from Hamblin et al. (2024) is shown bottom left.
As can be seen from this sample, the granularity of concepts varies. We see concepts for objects (e.g., colorful underwater fish), textures and pattemss (e.g., colorful polka dots pattern), composition (e.g., person on the right edge at social gatherings and bright colorful backgrounds), actions (e.g., skiing action and objects), types of images (e.g., comic book illustrations and characters and male portraits in various attire), and more.
snowy slopes and gripping various tools
Colorful underwater fish
6
Epitome
Most-Activating ImageNet Examples
MAD
MAP
4685
Comic book
illustrations and characters
CHAPTEO
TALES
TALES
WHEN WAKES THE
BEAST
Epitome
Most-Activating ImageNet Examples
32

---

Published as a conference paper at ICLR 2026
6577
B
Bright colorful
Ck
backgrounds
Epitome
Most-Activating ImageNet Examples
9606
Colorful polka
C
dots pattern
Epitome
Most-Activating ImageNet Examples
20808
Gripping various tools and objects
shuterstr
shufeesst
Epitome
Most-Activating ImageNet Examples
33

---

Published as a conference paper at ICLR 2026
Symbolic cross
Ck
motifs
FINE & or
FINE MY OHIT
Wirlbor
Mariboro
Epitome
Most-Activating ImageNet Examples
21053
Male portraits
C
various attire
Epitome
Most-Activating ImageNet Examples
9620
Skiing action
snowy slopes
Epitome
Most-Activating ImageNet Examples
34

---

Published as a conference paper at ICLR 2026
14944
Person the right edge at social gatherings
Ck
Epitome
Most-Activating ImageNet Examples
10
Architectural ceilings and arches
C
Epitome
Most-Activating ImageNet Examples
30404
Webpage layouts and design elements
Epitome
Most-Activating ImageNet Examples
35

---

Published as a conference paper at ICLR 2026
G ADDITIONAL RESULTS: QUALITATIVE EXAMPLES OF BLINDSPOTS
Shown below are qualitative examples of suppressed conceptual blindspots in SD 1.5. For each concept, we include a representative image from the natural distribution to illustrate the desired visual depiction. To the right, show four images generated by SD 1.5 using various prompts designed to clicit the concept. Despite using simple, clearly worded prompts, the model consistently struggles to generate these concepts, supporting their identification suppressed conceptual blindspots.
25651
X
Bright orange
C2
backgrounds
apple againman onange backgroound
orange background
onange background
arange background
718
II
x
Directional signs
C:
and symbols
trficc croswerk
ain pointing right
sign pointing to directon exit
anow
25996
Dark screens of various devices
C
dark
Slnck computer
Pawered laptoo screen
laptoo
laptop
turned
Lock h
10430
x
Locking mechanisms and packaging
C
paclock packaging
lock packaging
lack packag ng
In the remainder of this section, we enumerate additional suppressed and exaggerated blindspots for each evaluated model (see App. G. We also deseribe stress-testing procedure, in which attempt to elicit the concepts identified as suppressed blindspots using many prompt variations, in order to validate that these are indeed true blindspots (see App. G.2
36

---

Published as a conference paper at ICLR 2026
G.1 MODEL-SPECIHC BLINDSPOTS
(a) Suppressed Blindspots
(b) Exaggerated Blindspots
ponding
kudlgra
vilde
kewe
Canhed
Sando
Figure 13: Examples of conceptual blindspots in Stable Diffusion 1.5. For each concept, the prototypical natural (for suppressed blindspots) or synthesized (for exaggerated blindspots), based on the highest absolute activation, is shown. The spatial heatmap for the concept is superimposed atop the image.
(a) Suppressed Blindspots
(b) Exaggerated Blindspots
Whlee priddng
Repestad
Soldd
Weul botter
Ridd gootall Desubcerr
Bidacdi R11100 sament
Fredew Tu4iih commrnatinnl Level Masons
Abrasca promeric
'Siing
Uife frumiig
Burge fardh diek
Tanirs mata carter
employtddry
badgpourd
San Bauttn
Figure 14: Examples of conceptual blindspots Stable Diffusion 2.1. For each concept, the prototypical natural (for suppressed blindspots) synthesized (for exaggerated blindspots), based on the highest absolute activation, is shown. The spatial heatmap for the concept is superimposed atop the image.
37

---

Published as a conference paper at ICLR 2026
(a) Suppressed Blindspots
(b) Exaggerated Blindspots
Brd fooker
Pomes Hoaa
Watamak
Oul Facil Lese
Tar'e call
Oue
bese Wheie poi pacogg thet.
Mud carnas nol Fiss SAP and
Egan cocaal thery car. cedge
Nomusddre grea color Instras...
bedi
Ancicus soul Toeeoon
cone vibi by 5933311
cae peogaie imsgle
wode up gandd Jamalcs
bacar
Bunseing PM
Figure 15: Examples of conceptual blindspots in Kandinsky. For each concept, the prototypical natural (for suppressed blindspots) or synthesized (for exaggerated blindspots), based on the highest absolute activation, is shown. The spatial heatmap for the concept is superimposed atop the image.
(a) Suppressed Blindspots
(b) Exaggerated Blindspots
Q
Teee grans
Wood rack
Stake kodyy
Muit white
Sky clecs
Calorhi
Tonil ATTIUE
progrrsh
pery
NOW 71
Has sprikes
Figure 16: Examples of conceptual blindspots in PixArt. For each concept, the prototypical natural (for suppressed blindspots) or synthesized (for exaggerated blindspots), based on the highest absolute activation, is shown. The spatial heatmap for the concept is superimposed atop the image.
38

---

Published as a conference paper at ICLR 2026
G.2 STRESS TESTING
To stress-test the blindspots identified by our method, we gathered range of prompts describing these blindspots and used them to generate many images. We then contrasted the outputs from models in which the concept identified as a blindspot with those in which it was not.
Specifically, ChatGPT-40 was prompted as follows: want to generate an image of the following concept: "<blindspot>". Suggest 50 prompts highlighting this concept to be used input for text-to-image model. Return these as a list of strings in Python. Five images were generated per prompt and analyzed using custom RA-SAE model (see Appendix [F.Ip. which ranked them by the intensity with which the desired concept appeared. All images were then manually reviewed to determine whether the blindspot
successfully depicted.
As seen the following examples of suppressed concepts, while some aspects the target concept occasionally appeared (e.g.. a holder string for the bird feeder blindspot and round hole for the glossy DVD disc blindspot), the models generally failed to generate the full concept. This aligns with our method's 's assessment and supports the validity of the stress test.
G.2.1 Bird Feeder BLINDSPOT KANDINSKY
Suppressed Blindspot
Regular Concapt
Text
Kandinsky
SD 1.5
bird feeder hanging from branch
colorful
feeder attracting many small birds
feeder sunoundad by blooming Howerss
transparent feeder filled with saods
Figure 17: Examples of images generated with various prompts involving the bird feeder concept a part of the stress testing. In Kandinsky, our method identified this concept as suppressed conceptual blindspot, which matches the observed behavior: the model is unable to generate corresponding image. By contrast, SD 1.5, in which this concept was identified blindspot, is able to generate this concept.
39

---

Published as a conference paper at ICLR 2026
G.2.2 Glossy DVD Disc BLINDSPOT IN SD 1.5
Regular Concept Kandinsky
Suppressed elindspot
Text
SD 1.5
gleesy disc on a white background
glossy disc with visible data layer
minimalist glosey DVD disc
glossy DVC disc rambbw spectrum
Figure 18: Examples of images generated with various prompts involving the glossy DVD disc concept part of the stress testing. In SD 1.5, method identified this concept suppressed conceptual blindspot, which matches the observed behavior: the model is unable to generate corresponding image. By contrast, Kandinsky, in which this concept was not identified as blindspot, able to generate this concept.
40

---

Published as a conference paper at ICLR 2026
H ADDITIONAL RESULTS: HIGHER-ORDER BLINDSPOTS WITH COMPOSITIONAL DISCREPANCY
Thus far, our analysis has centered on individual concept activations. Yet visual scenes are rarely composed of isolated concepts; instead, they are structured through rich and structured co-occurrence patterns that encode compositional semantics. We now examine whether generative models capture this higher-order structure by analyzing the co-activation matrix Z Z, which reflects pairwise correlations in concept usage.
LO norm
Unique elements
10
107
10
10
10
Kandinscy
Kandinsty
BwAtt
Uniquu
10?
PwArt
St2
5D 2.1
2.1
10'
Original
10P
Original
10
10
10
0"
10t1
10
10
Epailon
Epoiion
Sparsity and Structural Divergence. On the left: LO norm of the co-occurrence matrix ZZ" as a function e (threshold), indicating how many entries remain active each model. On the right: Number of unique entries in the synthesized distribution relative to the natural distribution. All evaluated models preserve global sparsity structure, but diverge in activation content.
Figure 19:
Surprisingly, when assessed at the level of binary structure, diffusion models approximate the global sparsity of the natural co-occurrence matrix with high fidelity. As shown in Fig. 19 (left), the to norm of Z Z thresholded varying values tracks closely between the natural and synthesized distributions across all models. This indicates that the gross connectivity of the conceptual graph, i... which concepts tend to co-activate all, is well preserved. Formally, one can deem Z Z as the adjacency matrix of weighted, undirected graph over concepts, where edge weights reflect co-activation strength across the dataset.
Eigenvalue Spectrum
10
107
107
Kandinsky
PaAn
10'
SO 1.5
S.2.1
Original
10
10
10
Figure 20: Spectral Structure of Co-occurrence. Log-log plot of the eigenvalue spectra from co-occurrence matrices ~ across models and the natural distribution. All evaluated models match the heavy-tailed decay of the natural distribution.
However, as illustrated in Fig. 19 (right), the specific content of these co-activations diverges: substantial portion of entries in the model-generated Z Z are not shared with the natural baseline. This suggests that while the capacity for compositionality is retained, the identity of active pairings may shift, potentially reflecting model specific inductive biases or training artifacts. To probe the internal structure of these co-occurrence patterns, we turn to spectral analysis. Fig. 20 shows the eigenvalue spectra of the co-occurrence matrices for each model and the natural distribution. All spectra exhibit a heavy-tailed decay, consistent with power-law behavior, indicating that generative models preserve the overall rank structure and variance allocation across conceptual dimensions.
Further, we examine the alignment of dominant conceptual directions via cosine similarity heatmaps between the top 100 eigenvectors of the synthesized and natural co-occurrence matrices (Fig. 21)
41

---

Published as a conference paper at ICLR 2026
SD 1.5
SD 2.1
PixArt
Kandinsky
1.0
Natural Eigvec
20
20
20
0.8
40
40
40
40
0.6
60
90 60
80 60
80 60
0.4
80
0.2
0.0
20 40
60
80
20
40
60
60
20
40
60
80
90
Synthesized Figven
Synthasized Figvec
Synthesized Figuec
Synthesizos izd Figvac
Concept Basis Similarity. Cosine similarity heatmaps between the top 100 eigenvectors of the natural and synthesized co-occurrence matrices Zz". Diagonal structure shows alignment of dominant conceptual directors, with varying degrees of alignment across the four models.
Figure 21:
While all models exhibit partial diagonal aligneet--implying overlap in principal concept axes-the off-diagonal entries reveal rotations and mismatches in higher modes, reflecting evident deviations in compositional geometry.
Together, these findings reveal that diffusion models upproximate the global shape concept co- activation surprisingly well. yet deviate in subtle and structured ways when examined through the spectral lens. Such higher-order discrepancies may underpin failures generating coherent, multi-object scenes relational concepts.
42

---

Published as a conference paper at ICLR 2026
I ADDITIONAL RESULTS: CAPTION NOISE IN HIGH DIVERGENCE
We inspected datapoints with the highest datapoint-level energy differences to determine whether the divergence stems from genuine blindspots or low-quality input data (specifically, noisy captions).
1.1 METHODOLOGG
We isolated the top datapoints with the highest datapoint-level energy difference ||{(x') - <(x)) II2 (see Section 4.4). We analyzes these using Vision-Language Model acting an AutoRater (also referred to -LLMMaa-a-judge"" The VLM was presented with the natural image æ: and the text prompt L, and tasked with classifying the caption quality into three categories:
Noisy. The caption is irrelevant, factual nonsense, or consists purely of meta-data (e.g., filenames, URLss..
2. Underspecified. The caption is technically correct but too vague to identify the specific concepts visible in the image.
3. Sufficient. The caption provides enough semantic detail that a generative model should reasonably be expected to reproduce the main concepts visible in the image.
1.2 RESULTS
We used ChatGPT-4o the VLM and analyzed k-199 images. Out of these, 76 (38.2%) were labeled as noisy, 1 (5.5%) as underspecified, and 112 (56.3%) as sufficient.
1.3 ANALYSIS
While a portion of the highest energy differences are indeed attributable to poor instruction quality, inherent in web-scraped datasets like LAION-5B, the majority of cases have sufficient caption. In these instances, the prompt adequately describes the concept present in the natural image, yet the generative model produces a high-energy difference output. This confirms that while dataset noise a contributing factor, the primary driver of high conceptual divergence remains structural model limitations, supporting the findings in Section 4.4
43

---

Published as a conference paper at ICLR 2026
J ADDITIONAL RESULTS: SAE ERROR CASES
To better understand the specificity and reliability of the SAE used in our experiments to give {(x) and {(x'), we sought to quantify its misfires (false positives, FP) and missing concepts (false negatives, FN).
J.1 METHODOLOGY
We randomly sampled Al-generated and n natural images. We constructed the energy profile (E(x) or {(=')) of each image using the evaluated SAE, and filtered for the top-k concepts with the highest activation values. Each concept was mapped to its autointerpretability label.
We then employed Vision-Language Model (VLM) as AutoRater (also referred to "LLM-as- a-judge"). The VLM was presented with the image (æ or æ') and the list of top-k detected concept descriptions (including their activation strengths), and tasked with two classification objectives:
Misfire (FP) Detection. Identify concepts the detected list that are not visually present in
the image.
2. Missing Concept (FN) Detection. Identify critical visual concepts clearly present in the image but absent from the detected concept list.
The VLM was instructed to weigh activation strength when evaluating misfires, as concepts with very low activations more likely to be spurious detections.
J.2
RESULTS
We conducted the analysis on the top-k=20 concepts from n=1000 natural images and 100 Al- generated images (namely generated using SD 1.5), using ChatGPT-4o as the VLM. The top misfires for natural and Al-generated images reported in Tables 2 and respectively. Structural concepts with incorrect autointerpretability labels (see Section [.3] are shown in italics.
Concept
Count
Concept
Count
1
(Striped marine creatures)
98
(Striped marine creatures)
100
2
(Human interaction with surroundings)
20
(Human interaction with surroundings)
3
Natural textures organic forms
17
Natural textures and organic forms
4
Green apples and playful animals
9
Elegant fashion models in glamorous...
5
Musical instruments and accessories
7
Musical instruments and everyday...
4
6
Objects and symbols representing time
6
Airplanes and clothing textures
7
Airships and vintage photography
5
Bathroom fixtures and sinks
8 Red curtains and theatrical elements Airplanes and clothing textures
4
Leather couch and animals
Smoke and vapor emissions
Elegant fashion models glamorous..
Bookshelves and seating arrangements
Table 2:
Top Misfires for Natural Images. Con- cepts from the top-k==20 concepts of natural im- ages, labeled as misfires in VLM AutoRater analysis (conducted using ChatGPT-4o). Con- cepts in cursive structural concepts with in- accurate autointerpretability labels.
Table 3: Top Misfires for AI Images. Concepts from the top-k==0 concepts of Al-generated im- ages (SD 1.5), labeled misfires in VI.M Au- toRater analysis (conducted using ChatGPT-4o). Concepts in cursive are structural concepts with inaccurate autointerpretability labels.
44

---

Published as a conference paper at ICLR 2026
Figures 22 and 23 show the cumulative misfire rate function of k. Figure 23 excludes structural concepts with incorrect autointerpretability labels, since these are not visibly present in the images and thus cannot be correctly annotated by the VLM; Figure 22 includes all top-k concepts.
Only 8.0% (Al-generated) and 11.1% (natural) of images had critical visual concepts missing from the top-20. The share of concepts that were misfires among the top-20 was 24.04% for natural images and 26.10% for Al-generated images.
100
Target
Target
Images with 21 Misfree (%)
Images Misfire
80
60
40
20
Al-Generated Images
20
Al-Generated Images Images
Images
Top-K Concepts (by activation strength)
Top-K Concepts (by activation strength)
Figure 22: Cumulative Concept Misfire (All). Share of the n=100 images with at least one cept misfire, labeled in a VLM AutoRater analy- sis (conducted using ChatGPT-40), of k. All concepts are included, including struc- tural ones which tointerpretability description is inaccurate. The region highlighted "Target k" corresponds the k hyperparameter of the SAE.
Figure 23: Cumulative Concept Misfire (Vis- ible Concepts Only). Share of the n=100 im- ages with at least one concept misfire, labeled in VLM AutoRater analysis (conducted using ChatGPT-4o), as a function of k. Structural con- visible and where the autoin-
function
visible and where the
cepts that terpretability description is inaccurate are not included. The region highlighted as "Target k" corresponds to the k hyperparameter of the SAE.
J.3 ANALYSIS
Tables andd 3 listing the top misfires in natural and Al-generated images, point to a phenomenon where abstract concepts fire frequently without being tied clear semantic feature visible in the image. As shown Figures [24] 25) these concepts attend to abstract content near the left or right edges of the image, without any particular semantic attachment. Consequently, the autointerpretability descriptions assigned to these concepts by VLM (e.g.. Striped marine creatures and Human interaction wi th surroundings) are not truly descriptive. The VLM AutoRater then marks these concepts as misfires for almost every image because it relies purely on those textual descriptions (Figure 22).
After filtering out such concepts, the misfire profile in Figure 23 shows strong performance under the SAF configuration with k=5. Here, the SAE activates only five sparse concept codes per image, and these are the positions that should be primarily scrutinized. At k= for both natural and Al-generated images, the majority of datapoints exhibit no misfires. This rate increases approximately linearly until it plateaus around k=20. Furthermore, only 8.0% of Al-generated images and 11.1% of natural images had critical visual concepts missing from the top-20 concepts.
Specific examples of datapoints with misfires missing concepts for both natural and Al- Al-generated images are given in Sections J.3. I and J.3.2] respectively.
45

---

Published as a conference paper at ICLR 2026
k
31585
Striped marine
creatures
Epitome
Most-Activating ImageNet Examples
Figure 24: Concept 31585 Detail. Shown above are the autointerpretability description, exemplars, and epitome for the concept.
13101
Human interaction with
C
surroundings
Epitome
Most-Activating ImageNet Examples
Figure 25: Concept 13101 Detail. Shown above are the autointerpretabiliiy description, exceripionnexemmlarrs. and epitome for the concept.
46

---

Published as a conference paper at ICLR 2026
J.3.1
ERROR CASES: NATURAL IMAGES
Top Concepts (Misfires)
Missing Concepts
Bowls of various foods
N/A
pespers, ors, cros-sscctins
Striped marine creatures
Gourmat food presentation
Search result baked chicken lajtos heart healthy Easy and delictous homemaae recipe...
Fresh vegerables dips
Top Extracted Concepts (Misfires)
Missing Concepts
Vibrant market scenes
Children studyng
Seating arrangements crowds
Books and educational materials
Facial expressions cultural diversity
Human interaction with suroundiins
The Economics Of Developing
Countries
Back views of clothing and swimwear
J.3.2
ERROR CASES: AI-GENERATED IMAGES
Extracted Concepts (Misfires)
Missing Concepts
Beds and mattreeses
N/A
Industrial tools and equipment
Bediding pillows and blankets
organic
Memory Mattress Siee Cool Memory Foom Mattress
Bedding and textures
Top Extracted Concepts (Misfires)
Missing Concepts
Covieetion
Website layouts and designs
Underwater cleaning equipment
Tools equipment
Company branding logos
Smoke and emissions
Bright orange backgrounds
2019 MSS CANTATIONCEEANER
Textual branding
47

---

Published as a conference paper at ICLR 2026
K CONCENTRATION BOUNDS FOR
In our experiments, we estimate ((k) using n 10,000 000 paired samples for each concept. While this budget is modest, it raises the natural question of whether it suffices to obtain reliable estimates. To address this, derive concentration bound on the empirical estimator on (k) using McDiarmid's inequality McDiarmid et al. (199).. The resulting bound is tight and demonstrates that even with relatively few samples, we can obtain fast and accurate estimates of concept bias.
Theorem 4 (Concentration of 8n (k)). We assume that the concept score Ex(xt takes values in [a, 6] almost surely for all images drawn from either Dx or D'x. Let paired samples (x,, x' i)i be drawn independently with 2; ~ Dx and x ~ D'x, and define the empirical estimator
(ma) 1 - (u(()))
(k)
Let M := bb- and L. = M/4. Then for every the deviation satisfies
1a ((ān(()) - 8(k)) > 26) exxp
2nE²
1
Proof. The function x (x) is 1 1/4-Lipschitz, since lo'(x)) S 1/4 for all x.
Viewing 5n (k) as function of the 2n independent variables (æ11.... Fy ....... a%y) changing single argument alters the inner difference of means by at most M M/n, /m, and the outer sigmoid scales this by at most 1/4. Hence, the bounded difference constant for each coordinate (M (M/n)(1/4) = L/n.
By McDiarmid's inequality (McDiarmid et al. 1989).
P (().(()) 5(k) >) 2exp
2s
2ne
exp
Σ j=m (L/n)²
T,2
which proves the claim.
Practically, most concept scores 8(k) are sparse, with the majority concentrated near zero and only few reaching values up to 10. The concentration bound shows that even for the largest observed biases, a sample size of 10,000 yields estimates of o, (k) that deviate from the true value by no more than a small ε with high probability. This justifies our sampling strategy and confirms that accurate bias measurements are attainable with limited data.
48

---

Published as a conference paper at ICLR 2026
L MONOTONICITY AND CALIBRATION-FREE INTERPRETATION OF ò
Our goal when analyzing blind spots is rank concepts by the severity of their generative bias. In practice, we use the score 8(k) for this purpose. However, one may wonder whether such a score introduces distortions relative to more direct quantities such as the energy gap the odds ratio. The following result establishes that ((k) a strictly increasing reparameterization both, and therefore inherits their ordering. This guarantees that calibration is needed when using d(k) to rank concepts.
Theorem (Monotonicity and Calibration of &g For every concept index k define the energy gap
A = Dy Ex
the associated odds ratio Pv: = exp(An) and the energy nnergy-difference score
8(k) = exe(((AAs) 1 + Pk
Ph
Then 8(k) a strictly increasing bijection of both DR and pk, so ranking concepts by any one of 5(k), A Pk produces exactly the same ordering.
Proof. The logistic sigmoid satisfies d' d1xx) (zx)) (()) > Var TR; hence and therefore 8(k) a A+) grow strictly with Ag. Because the exponential map is also strictly increasing and bijective + (0, oo), setting Pt = exp(Ax) preserves order and gives AS log Ps. Substituting this identity into yields 6(k)
(log Pk: pt/(1 pk), which is the composition of two strictly increasing bijections and is therefore itself strictly increasing and bijective in Pt. Since strict monotonic functions never reverse inequalities, the three quantities share the same total order concepts.
Thus, ranking concepts by 8(.) is strictly equivalent to ranking them by energy gap by conceptual generation odds Pk- No calibration necessary, and three quantities preserve the same total ordering over concepts.
49

---

Published as a conference paper at ICLR 2026
M STABILITY OF FID UNDER SAE EMBEDDINGS
In this section establish quantitative relationship between the Fréchet Inception Distance (FID) computed in the original activation space (of dimension d) and the FID after applying a (potentially overcomplete) SAE dictionary D Rxx with k > d. Throughout assume that D has orthonormal columns but is not necessarily square, i.e.
DD la.
while
DD DD +
We start by recalling simple fact: if is not overcomplete, orthogonal and k: = d, the we have an isometry between A and Z. implying that the FID perfectly preserved. However, this case is realistic, we will then turn the the overcomplete case, and show we can bound FID by the extreme singular value of D. We will work with the Wasserstein-2 metric W2, noting that FID is just W22 specialised to Gaussians.
For probability measure μ on Hd we write D## for its push-forward under D, i... D= iuDD1l 2). Denote by mii and max the minimal and maximal singular values of D, equivalently the square-roots of the extremal eigenvalues of DD
6 Liin Is DD
3
0 0 Is-
Empirically one usually finds mins T max & 1, but the proof does not rely on that. We will start by a overcomplete.
simple lemma in the case where D
Lemma 1 (Isometry under exact orthogonality), Suppose k = d and DJ D DDT Ia- Then D is an isometry: Dvll2 2 = ||la| for all Rd. Consequently, for any probability measures μs, on R2d with finite second moment,
W #H. DΦv) W)(
Proof. Orthogonality of D implies preservation of the Euclidean norm, and push-forward commutes with the map inside the W2 infimum; the integrand is unchanged, so the infimum value is identical.
This case, however, is quite unrealistic as SAE usually rely on the overcompletness to extract meaningful and interpretable concepts. the overeomplete case, D is no longer orthonormal, but we can still have column column-orthonon orthonormal dictionary. We will use that to show that we can bound using the extremal singular value of DTD.
Theorem 6 (FID under column orthogonal embeddings). Let D E lxd satisfy D D Ia and denote by 0 < O S max the extreme singular values of DD Given two data matrices A, A' Ruxd (rows are sample vectors), set = AD E B and = A'D E BRaxk Then
2 min FID(A,A) < FID(Z. Z') < F FID(A,A))
Proof. Write μe for the empirical measure of A and v for that of A', i.e.
1
8 A
S
For any coupling म E [p, v) (i.e. probability measure on Rd x Rdd with marginals fa, 2) we have, by the extremal singular value bound,
2
| v <
D
E)ll
muxx x y1l..
V(x, y) E Rd Rdd
Integrating with respect to arbitrary coupling π II(H,,)) v) yields
||
y u
dn
/1ID(=-))1B
/ID(( - v)112 dv - o o
/l1
/  y 22122 dr.
min
The middle integral is exactly the transport cost of the pushed-forward forward coupling (D x D) between uD D# l and vD Dav. Because the inequalities hold for every π,, they hold in particular
50

---

Published as a conference paper at ICLR 2026
for the optimal couplings attaining (2 2) and W2(MD, DD), though these two optima need not coincide. Taking the infimum over π term-wise wise makes this explicit:
/ /1DD m
/ /lte 212 dr.
inf
y112 dm I
inf
y)l
o TETLLN)))
inf
mEll(u,
Hence
min W (t, v) & Wg vD) E
v). W3 (.,)) .) for the Gaussian surrogate and plugging in (A, A') (resp. (Z, 2
m W (M,
Recognising (ID(
finishes the proof.
L
Essentially, theorem 6 tells us that applying a column-orthogonal overcomplete SAE dictionary cannot distort Fréchet inception Distance by more than the square of its extremal singular values.
When D nearly orthogonal- - eempirically we usually found that min max R the result implies that FID measured in the SAE feature space is essentially close to the canonical FID. 0
51

---

Published as a conference paper at ICLR 2026
N
ADDITIONAL EXAMPLES OF SYNTHESIZED IMAGES
Text
Natural
SD 1.5
SD 2.1
PixArt
Kandinsky
New Modemm
Fibergibes e
Swimm Pool
Prtrance Step
Cieen
Refreehing tlla Water
Kylie Yorkde
Rolly Teacup Puppies
2F lance Vicfanl
given sterani wind mp3:
GLA
Office Bundle Package
BULE2S BACEFER
package
Rs 2500
Stagocaach Springs Road, Custer image
Christ On The Acrylic Print Delscroix
Outdoor Swing
Chair/ Bed
Gamapy Patterned Arch- Coffee
DOGS S
PEOPLE E
Dogs People
LOYE
carbonara spaghetti on
background
Figure 26: Additional captenn pair examples from LAION-5B with matching images generated with the same prompt by SD 1.5, SD 2.1, PixArt, and Kandinsky.
52


Reviews:
[{"review_id": "L8R9DfAfse", "reviewer": "Reviewer_3MU4", "summary": "The paper introduces a novel framework to systematically discover suppressed or exaggerated conceptual blindspots. In particular, the authors first trained a large sparse autoencoder (RA-SAE) on DINOv2 features to extract an interpretable concept basis. They then define energy differences between the prevalence of each concept in real vs. generated images to pinpoint these blindspots.", "questions": "- I don’t think the title of Section 4.1 captures its contents? Moreover, some works have shown that T2I models are able to generate unseen concepts [1]\n\nReferences:\n[1] Haviv, A., Sarfaty, S., Hacohen, U., Elkin-Koren, N., Livni, R., & Bermano, A. H. (2024). Not Every Image is Worth a Thousand Words: Quantifying Originality in Stable Diffusion. arXiv preprint arXiv:2408.08184.", "limitations": "", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The problem is novel and well-motivated, and the paper is well-written.\n- While I feel like some of the observations are not new (for instance Sec 4.6), I think it is nice that the framework can be used to quantify them.\n- The open-source framework is highly appreciated.", "weaknesses": "- Section 4.4 would benefit more from a more systematic study and quantitative results. For instance, when the authors claim that “While some of these discrepancies can be attributed to underspecified or noisy captions, others reveal genuine blindspots”, I think it would be better to quantify what portion of the images with large energy difference has noisy captions.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "FWepJZXJAP", "reviewer": "Reviewer_NSuB", "summary": "The paper proposes a framework to identify \"conceptual blindspot\" in generative models by using sparse autoencoders to extract interpretable concept and quantitatively comparing the energy between real and generated images. The framework distinguishes between two types of blindspots: suppresed blindspots and exaggerated blindspots, and applies to a range of models.", "questions": "- Dependence of DINOv2 features, have you tried any other feature extractor, how is the method when work with not as good extractor?\n- What about the hierarchical concepts (vs compositional concepts)?\n- Can you elaborate more on the concept dictionary, how it is trained/intialized? How to map those to human understandable concepts for further analysis?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "- The paper offers a new perspective on understanding the failure modes of generative models and provide mathematical definition of conceptual blindspot.  \n- The method can scale easily in an unsupervised manner, free from human inputs or manual checks.\n- The paper is very well written", "weaknesses": "- Including error analysis and failure modes of the approach and discussing those will strengthen the paper, where the SAE produces spurious concepts or misses obvious ones. \n- Baseline comparison is missing. How is the proposed method compared to other framework proposed in the literature. Do the observations agree?\n- What about mitigation? No proposed mitigation strategy on how to reduce the blindspots? How to combine this with classifier-guided generation, for example, use the proposed energy-based model and langevin dynamics to sample better images.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "twiC1mCWku", "reviewer": "Reviewer_dqnN", "summary": "In this work, authors introduce an SAE based approach to systematically identify and characterize \"conceptual blindspots\" in generative image models. In particular, these blindspots are defined to be concepts that are reasonably expected and present in the training data but misrepresented in generations. Authors train RA-SAE's on DINOv2 features to perform fine-graine analysis of conceptual disparities between real and generated images. Authors apply their technique on SD1.5/2.1, PixArt and Kandinsky models and reveal suppressed or exaggerated blindspots.", "questions": "***Questions and Suggestions:***\n* Did the authors observe any redudancies among the concepts learned by the SAEs (e.g., different indices of the concept embedding corresponding to the same concept)? \n\t* If not, is it evident that the adopted SAE training approach inherently ensures the discovery of distinct concepts?\n\t* If such redundancies do exist, consider the following example: suppose the first and second entries of the concept embeddings both correspond to \"concept A\". Is it possible for real images to tend to activate the first entry but not the second, and the other way around for a synthetic images? In that case, the proposed method might incorrectly identify \"concept A\" as a blindspot (in fact as both suppressed and exaggerated).\n* I would recommend discussing Surkov et. al. [1] for completeness in Appendix A.4.\n* In Section 4.2, the authors mention that some blindspots likely stem from dataset characteristics, while others appear to be model-specific. Based on your results and intuition, do you have any promising hypotheses on how these blindspots could be mitigated or eliminated?\n* Figure 11 suggests that blindspots tend to correspond to rare concepts in the natural dataset; however, not all rare concepts are blindspots. Do you have any intuition about why this might be the case?\n* How were the $\\delta$ thresholds $0.1$ and $0.9$ chosen?\n* What motivated the decision to train the SAE on ImageNet rather than LAION?\n* Have you encountered any \"counting\" related blindspots (e.g. number of objects) in any of the models that you've tested ? If so, were there any interesting observations?\n\n***\n***References:***\n\n[1] Surkov, Viacheslav, et al. \"One-Step is Enough: Sparse Autoencoders for Text-to-Image Diffusion Models.\" arXiv preprint arXiv:2410.22366 (2024).", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "* The paper is very well written, and together with the well-designed figures, I truly enjoyed reading the manuscript.\n* The work is contextualized well within the existing literature.\n* The proposed technique for automatically mining the blindspot failure modes of generative image models, as well as the formalism of blindspots itself, is both sound and timely. I believe the ICLR community would find these results, and the interactive tool, highly interesting. \n* In terms of reproducibility, authors did an excellent job in disclosing dataset, implementation and compute resource details.", "weaknesses": "* While not a major concern, there appears to be an implicit conceptual assumption that the discovered concepts in the SAE are non-redundant for energy discrepancy to be meaningful. It is unclear whether this assumption holds in practice. If redundancy exists empirically, it should be discussed in terms of its impact on the interpratation of results. Otherwise, it would be helpful to demonstrate that the discovered concepts are indeed unique/distinct. More on this is discussed in the questions below.\n* Please see the other questions below as well.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "x4i0ww7eRW", "reviewer": "Reviewer_uZCM", "summary": "In this paper, the authors analyzed how text-to-image generative models fail to represent certain visual concepts that are present in real-world data. They define these systematic failures as cases of “conceptual blindness”. The paper provides a formal definition of this phenomenon by modeling the true data-generating process as an invertible function mapping latent concepts to observed images.\n\nTo explore this idea, the authors train a Sparse Autoencoder (SAE) on DINOv2-extracted features to identify latent “concepts” in both real and synthetic images. By comparing the frequency of concept activations across the two distributions, they detect under-represented concepts, which is interpreted as blindspots of the generative model.\n\nThe authors conduct analysis on four text-to-image models (Stable Diffusion 1.5, Stable Diffusion 2.1, PixArt, and Kandinsky), identifying both universal and model-specific blindspots. They further explore correlations across models and connect the discovered blindspots to broader phenomena such as memorization and mode collapse, as well as how post-training techniques influence the distribution of conceptual coverage.", "questions": "- What would be the impact on the thoeretical framework if the assumotions in section 2 are violated?\n- Would the findings generalize to other architecture of text to image models?\n- How are the threshold of 0.1 and 0.9 determined for $\\delta(k)$?\n- Have the authors tested other feature encoders (e.g., MAE, CLIP, or perceptual networks)?\n- Did the authors perform any human evaluation or concept-level annotation to validate that the detected “blindspots” align with human judgment?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "- The paper is well motivated, aiming to better understand the behavior of text-to-image generative models, particularly their tendency to violate some realworld concepts.\n- The authors propose a formal definition of conceptual blindspots that goes beyond qualitative analyses. The experimental setup of using the same textual prompts for real and synthetic images makes the two distributions highly comparable.\n- The paper is well written and easy to follow, with clear structure and visual illustrations that make the methodology and findings intuitive.\n- The use of RA-SAE trained on DINOv2 features provides a scalable and unsupervised approach to discover a large number(32000) of latent concepts without human labeling. The appendix provides detailed procedures for automatic concept labeling with LLMs, adding clarity and reproducibility.\n- The empirical analysis covers 4 recent diffusion models and revealing both universal and model-specific blindspots. The additional analyses such as studying post-training effects are informative and provide practical insights on how representation coverage might be improved.\n- The paper draws meaningful connections between conceptual blindness, memorization, and mode collapse, positioning the work within broader discussions on generalization and coverage in generative modeling.", "weaknesses": "- The analysis focuses exclusively on text-to-image diffusion models. While this is a reasonable starting point, there are other types of text-to-image generators such as autoregressive and GAN-based models. Including at least one non-diffusion baseline would strengthen the generality of the paper’s claims and help assess whether conceptual blindspots are architecture-specific or model-agnostic.\n- The paper’s title refers broadly to “generative image models”, but the experiments and discussion focus only on text-to-image models. Adjusting the title or framing to more accurately reflect this scope would improve clarity and alignment between the claim and the content.\n- In Section 2, the authors assume that the data-generating process (DGP) is an invertible mapping between latent concepts and images, with factorized latent priors. In practice, visual concepts are often correlated and non-invertible (e.g., “wooden texture” co-occurs with “brown color”). The authors briefly mention (L110–113) that empirical findings remain meaningful even when these assumptions are violated, but it is a bit unclear how relaxing these assumptions would affect the validity of their theoretical framework or the interpretation of results.\n- The qualitative examples in Figures 7 and 8 are not fully convincing. For example, in Figure 7, none of the four generated images clearly display the feature “shadow under animal.” In Figure 8a, all models appear to generate reasonable outputs for Following directions on worksheet, though the amount of whitespace varies (and SD 1.5 still shows some). It is unclear whether these examples truly demonstrate conceptual blindspots or instead reflect weak coupling between the text prompt $t$ and the concept $c_k$\n- The analysis relies solely on DINOv2 as the encoder for feature extraction. DINOv2 is trained with a contrastive objective emphasizing invariance to color, rotation, and background attributes that are crucial for generation. As a result, only using DinoV2 may underrepresent certain visual factors. It would be interesting to explore whether using a different encoder, such as MAE or a perceptual encoder, changes the resulting concept decomposition or blindspot patterns.\n\nMinor typos: \n- Figure 2 caption:$S_\\{theta}$ should be $g_\\{theta}$ \n- Figure 4 caption: Values left to 0.5 (L282), not left to zero.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)