The Deleuzian Representation Hypothesis
Authors: Clément Cornet
Romaric Besançon
Hervé Le Borgne
Venue: ICLR 2026 Poster (2026)
Keywords: Mechanistic Interpretability
Concept Extraction
Explainability
Primary area: interpretability and explainable AI
TL;DR: We propose a novel method for extracting interpretable concepts from a model's activations, by representing the differences between samples.

Abstract: We propose an alternative to sparse autoencoders (SAEs) as a simple and effective unsupervised method for extracting interpretable concepts from neural networks. The core idea is to cluster differences in activations, which we formally justify within a discriminant analysis framework. To enhance the diversity of extracted concepts, we refine the approach by weighting the clustering using the skewness of activations. The method aligns with Deleuze's modern view of concepts as differences. We evaluate the approach across five models and three modalities (vision, language, and audio), measuring concept quality, diversity, and consistency. Our results show that the proposed method achieves concept quality surpassing prior unsupervised SAE variants while approaching supervised baselines, and that the extracted concepts enable steering of a model’s inner representations, demonstrating their causal influence on downstream behavior.

fr Published as a conference paper at ICLR 2026
THE DELEUZIAN REPRESENTATION HYPOTHESIS
Clément Cornet, Romaric Besançon & Hervé Le Borgne Université Paris-Saclay, CEA, List, F-91120, Palaiseau, France {clement. cornet, romaric, peneerrcc..ssancooooeeeeeeeeeeae
ABSTRACT
We propose alternative to sparse autoencoders (SAEs) as simple and effective unsupervised method for extracting interpretable concepts from neural networks. The core idea is to cluster differences in activations, which formally justify within a discriminant analysis framework. To enhance the diversity of extracted concepts, we refine the approach by weighting the clustering using the skewness of activations. The method aligns with Deleuze's modern view of concepts as differences. We evaluate the approach across five models and three modalities (vi- sion, language, and audio), measuring concept quality, diversity, and consistency. Our results show that the proposed method achieves concept quality surpassing prior unsupervised SAE variants while approaching supervised baselines, and that the extracted concepts enable steering of a model's s inner representations, demon- strating their causal influence downstream behavior.
INTRODUCTION
Interpretabilitty of neural network representations essential for building trustworthy models, en- abling a deeper understanding of the mechanisms underlying a model's predictions, and promoting fairness and accountability. However, interpreting the internal representations learned by neural net- works remains central challenge in deep learning. Sparse autoencoders (SAEs) (Bricken al., 2023; Cunningham et al., 2023) have emerged as a powerful tool for extracting sparse and seman- tically meaningful features from model activations. Nevertheless, they face challenges that limit their applicability. Notably, they suffer from difficulties in training, and may still yield polyseman- tic features, not corresponding to single interpretable concept. Moreover, sparse autoencoders (and similar methods) rely feature sparsity proxy for interpretability, choice that has been criticized as potentially inadequate (Sharkey et al., 2025).
We introduce alternative to sparse autoencoders (SAEs) for extracting features that correspond to interpretable concepts from neural networks. Drawing inspiration from Deleuze's 's philosophical view of concepts differences, model concepts directions that capture distinctions between representations of individual samples. Specifically, our approach can be seen as an unsupervised
"Winning the prize"
"the Gold Medal"
"the World Record"
(a) Image: Van Gogh's Paintings
(b) Text: Sports Achievements
(c) Audio: Brass Instruments
Figure 1: Our method extracts diverse concepts from image, text and audio models.

---

Published as a conference paper at ICLR 2026
discriminant analysis: it identifies directions in the internal representation that best separate data samples. We estimate those directions by sampling activation differences between pairs of data points, then use K.Mans clustering uncover recurring patterns. Our analysis is further refined using distributional skewness to promote diversity.
Evaluating interpretability methods remains major challenge. SAEs often assessed by their eeconstruction-spartii trade-off, which does not necessarily reflect interpretability. Hence, most recent studies in this field are also evaluated qualitatively, showing their relevance through selected examples. While insightful, such evaluations provide limited support. In contrast, we adopt a quanti- tative evaluation based on probe loss (Gao al.. 2025), which measures the extent to which extracted concepts capture the attributes expected to be present in dataset. To ensure robust evaluation, we apply this metric to a broad set of 874 attributes spanning different tasks, five datasets and five mod- els across three modalities (image, text and audio). Our method captures the desired attributes more effectively than recent SAE-based approaches. In several settings, is competitive with supervised linear discriminant analysis. Beyond the presence of expected attributes, also evaluate cross-run consistency with the Maximum Pairwise Pearson Correlation (MPPC) (Wang et al., 2025), establish- ing a comprehensive evaluation framework for concept evaluation methods. Finally, we demonstrate concept steering on text and image models, showing that manipulating extracted concepts causally influence downstream behavior, without incurring information loss.
Hence, the main contribution of this paper is a novel type of approach of mechanistic interpretabil- ity of neural networks. We investigate the fundamental principle underlying our approach and demonstrate that it achieves globally more compelling results than state-of-the-art sparse autoen- coder (Sse) techniques. Our method is advantageous in its simplicity: it is governed by single, interpretable hyperparameter. The proposed principle is theoretically grounded in discrim- inant analysis and clustering, and further relates to Deleuze's philosophical notion of "concepts." Similar to SAE-based approaches, our method is fully unsupervised and therefore does not require manual specification or annotation of the identified concepts.
2 METHODS
2.1
CRITERIA AND CONCEPTUAL GROUNDING
Our aim is to extract an ontology of "concepts" from a neural network, by analyzing its activations. Before proposing our approach, we first discuss the criteria such concepts should satisfy.
Interpretability; this work aims to extract human- intrrrrattteeee features, that are then re- ferred to as "concepts".
Transparency: in order to gain interpretable insights into the model, the approach itself should be as simple and transparent as possible, not relying on non-interpretable hyperpa- rameters.
Diversity: the extracted concepts should be semantically diverse, in order to represent wide variety of data samples, ideas, and semantic levels.
Consistency: the approach should consistently yield similar concepts when run multiple times with different random seeds.
Existing methods in mechanistic interpretability typically extract unsupervised concepts by recon- structing model activations (Bricken et al., 2023; Cunningham et al., 2023). Because they are trained to minimize reconstruction error, such approaches are driven to capture as much variance in the acti- vation space as possible, subject to sparsity constraints. This framing implicitly presents concepts as universal structural components of the model activations, echoing the classical philosophical view of concepts "the universal essence of fact" (Plato, c. 375 BCE; Hegel, 1816). However, such a representation has been criticized as overly restrictive (Nietzsche, 1889; Sartre & Elkaim--Srrte, 1946). More recent perspectives instead emphasize concepts arising from Difference and Repeti- tion (Deleuze, 1968), rather than universals. Following this idea, our approach does not attempt to model the full variance of activations. Instead, it identifies recurring differences between activations.
2

---

Published as a conference paper at ICLR 2026
Activations
Compute skewness Differences
5. Interpretable projections
corresponding Concepts
A
(X
N
PDF(d.)
Image: Van Gogh's Paintings
2. Sampling Differences
4. Inverse Skewness Weighted KMeans
World Record
winning the prize
the Gold Medal
a(d, C)- 110 d:ll2
Text: Sports Achivements
d2
(s(d)
dj
Audio:
Instruments
Figure 2: Overview of our concept extraction approach. We sample pairwise differences in activa- tion between samples. Then, we use the mnverse-skennees of those differences selected the final concepts, corresponding to vectors in the activation space.
2.2 EXTRACTING REPEATED DIFFERENCES IN ACTIVATION SPACE
Our objective extract concepts from model activations, at given layer with dimensions, over dataset of N samples. To represent repeated differences in activations between data samples, we define D {dj, _,.,. dN} } as set of D-dimensional pairwise differences in activation between samples. Since our approach is fully unsupervised, we cannot restrain D to contrastive pairs between two classes. However, computing all pairwise differences is quadratic in N. To approximate the distribution of differences, we instead randomly sample N pairs, ensuring that each data point is used once each side of the subtraction.
To constrain our concept dictionary to a fixed number of concepts k: we cluster activation differences using KMeans (Lloyd, 1982; Zeng& & Zheng, 2019). However, highly skewed distributions: they remain near-zero for most samples, but occasionally spike to large values. Those differences tend dominate the Euclidean distance used by standard KMeans, and produce redundant clusters (Milligan, 1980). The skewness of distribution X is its normalized third central moment. For concept direction d, consider skewness that of the projection (d: 11
activation differences exhibit
(a(())) Σ/ - --(((-z)))) 2,))
(1)
No -3,33
where uu). and (.) are the mean and standard deviation over the {x, /=1 Since highly skewed co- ordinates tend to produce redundant clusters, we penalize them by assigning weights inversely pro- portional to skewness. In order to avoid ill-defined clustering with negative weights, and to consider opposite directions d, as similar (as are seeking directions, regardless of their orientation), consider -d, for differences with negative skewness. This results variant of Feature- Feature-Weighted KMeans (Huang et al., 2005), in which concept directions are weighted during centroids compu- tation, in order to promote concept diversity. More precisely, this clustering defines the weighted distance between de and its corresponding centroid C' as
d(d, 1 jrs(d.)
|10 d,
The obtained centroids are then used as concept vectors.
Both pair sampling and KMeans clustering run in linear time and memory with respect to dataset size N and activation dimension D, demonstrating scalability of our approach towards large datasets, or large models.
3

---

Published as a conference paper at ICLR 2026
Finally, this procedure retains simple and transparent formulation (Figure 2), that are key proper- ties for interpretability research. Notably, the number of extracted concepts is the only hyperpa- rameter required for our approach, and is itself interpretable.
2.3 CONNECTION DISCRIMINANT ANALYSIS
We aim to extract "concepts" from model activations, defining concept as a difference between ideas. supervised setting, this objective relates closely to discriminant analysis (Fisher, 1936), which identifies direction č orthogonal to the optimal separating hyperplane between two classes. Let ΣΛ and ΣB the class covariances, and μa and μB their means. The separation between classes is maximized by:
Ex (E En) m7n)
Consider two samples and j with activations こ; and E.J, and suppose we seek the optimal separation between clusters with means H; and Zy. distinguished by concept č. In high-dimensional spaces (typically 2 512 dimensions for transformers), we approximate Σ; and Σj as diagonal, containing each dimension's 's variance (Ahdesmäki & Strimmer, 2010).
From equation 2, 7 x T4 - I achieves optimal separation when Σ, x In xx J, i.e. under isotropic cluster distributions. Thus, treating activation differences as the optimal separation between ideas is equivalent to assuming isotropic distributions of concepts in activation space.
Unlike standard LDA, equation 2 does not require homoscedasticity or Gaussianity (MeLachlan, 2005), and naturally extends to multiclass discrimination (Rao, 1948).
In Appendix G. we derive a quadratic extension to our approach. that accounts for anisotropic dis- tribution of concepts. While theoretically interesting, it does not lead to better experimental results. For this reason, we focus on the isotropic approach (i.e Σ, x Σ, x T) the following.
2.4 LOSSLESS STEERING
Sparse autoencoders and related methods allow steering of extracted concepts (Zhou et al., 2025). To do so, they project sample activations in their concept space, apply a steering vector, and projects back into the activation space. The two projections required introduce reconstruction error and information loss. In contrast, our extracted concepts are vectors in the activation space. Therefore, can perform steering directly in the activations space. To steer the embedding of a sample x, with a magnitude a and a concept G, consider its steered representation ă = x + ac, If one steers concept by +a, then by -a, we retrieve exactly the base activation. By avoiding projections into and out of the concept space, our approach enables lossless steering: the modifications affect only the targeted direction and can be exactly reversed.
EXPERIMENTS
Datasets and Models To evaluate our concept extraction methods, we conduct large-scale study spanning five models and five datasets across three modalities (vision, language, and audio), cover- ing a wide variety of semantic attributes.
For text, we use two datasets: IMDB (Maas et al., 2011) and CoNLL-2003 (Tjong Kim Sang & De Meulder, 2003). IMDB provides sentence-level binary sentiment classification labels, while CONLL-20003 provides token-level labels for named entity recognition (NER), part-of-speech (POS) tagging, and syntactic chunking. For vision, we use subset of ImageNet (Russakovsky et al., 2015) with 100 classes and the WikiArt dataset (Baylies, 2020) which contains paintings labeled by artist (129 classes), style (27 classes), and genre (11 classes). Concerning text datasets, IMDB has binary classification labels, while CoNLL-2003 has token-wise labels for NER (9 classes), POS-tagging (47 classes) and chunk tags (23 classes). For audio, we use AudioSet (Gemmeke et al., 2017), with multi- multi-classification labels (527 audio classes).
Our text experiments are conducted on DeBERTa (He et al., 2021) and the encoder of BART (Lewis al., 2020), well as Pythia-70M (Biderman et al., 2023). For vision, we evaluate Dino (Oquab
4

---

Published as a conference paper at ICLR 2026
al., 2023) and CLIP (Radford et al., 2021). For audio, Transformer (AST) (Gong et al., 2021). We only consider encoder models, (including the encoder of BART). This choice allows us evaluate the quality of extracted concepts with respect to supervised labels that are likely represented at the analyzed layer of each model, since our objective to com- pare concept extraction methods. It also enables comparable analyses across multiple modalities. More details on datasets and models are provided in Appendix B.
pretrained Audio Spectrogram
Baselines Sparse autoencoders (SAE) predominant among concept extraction methods. We compare our method to five different types of SAEs:
VanillaSAE (Van-SAE) (Bricken et al., 2023): standard SAE, trained with an L2 recon- struction loss, and enforcing sparsity via an L1 penalty which requires a coefficient A; GatedSAE (Gat-SAE) (Rajamanoharan et al., 2024a): SAE learning activations gates, hence separating feature selection and magnitude estimation;
JumpReLUSAE (JR-SAE) (Rajamanoharan et al., 2024b): SAE with learnable threshold
0: for each concept, designed to minimize the reconstruction error;
MatryoshkaSAF (Mat-SAE) (Bussmann et al., 2025): SAE learning nested dictionaries of concepts, focusing hierarchies of concepts, belonging to multiple semantic levels;
TopKSAE (Tk-SAE) (Gao et al., 2025): SAE enforcing sparsity via a TopK activation function, that sets every activation to zero, except the k highest.
ArchetypalSAE (A-SAE) (Fel et al., 2025): SAE constraining decoder atoms to be combi- nations of activations, gain stability.
Pretrained Sparse Autoencoders (Pretrained): we compare our method with publicly avail- able, pretrained sparse autoencoders on two models. For DinoV2 experiments. we use ViT-Prisma (Joseph et al., 2025), and for Pythia we use sparse autoencoder trained by FleutherAl!.
We also compare our approach to Independant Component Analysis (ICA) (Comon, 1994). that is linear decomposition method maximizing statistical independence between latent dimensions. In addition, as our approach is closely related to discriminant analysis, we also compare it to super- vised Linear Diseriminant Analysis (LDA) (Fisher, 1936) which serves as an upper bound under assumptions of homoscedasticity and normal distribution of concepts.
Evaluation Our primary quantitative evaluation relies on the probe loss metric (Gao al., 2025), which measures the degree to which extracted concepts align with ground-truth annotated attributes. Beyond the quality on individual concepts, we also aim at uncovering a broad set of concepts from model activations. To this end, assess probe loss across tasks characterized by diverse attribute sets, thereby quantifying the capacity of our approach to capture multiple, semantically meaningful concepts. In addition, Maximum Pairwise Pearson Correlation (MPPC) (Wang et al., 2025) is used to measure the consistency of the different methods. Finally, to highlight causal influence of concepts on model predictions, we perform concept steering, and provide qualitative examples. Note that, while prior work on sparse autoencoders has emphasized reconstruction-ssariie trade-offs, these objectives not applicable our framework; therefore exclude them from evaluation. All the reported results are computed using activations from the last transformer block of each encoder, using a concept space with 6144 dimensions, corresponding to 8 times the size of the activations (except for ICA, that is limited to 768).
3.1 EVALUATION OF CONCEPT QUALITY
We evaluate concepts extracted in an unsupervised manner by assessing whether they correspond to interpretable attributes known to exist in the dataset. This correspondence is quantified using Probe Loss (Gao al., 2025). For each attribute, Probe Loss measures how well oineeiimesllnaa logistic probe can recover the ground-truth attribute from the extracted concepts. Specifically, we train separate ID logistic probe for every concept and record the lowest cross-entropy loss achieved. For multi-class attributes, we report the median Probe Loss across all attributes. The results of this evaluation are presented in Table 1.
"https://hugginge:"
/taeeerrrr/aaeeteaeeaeee ssssssaeeppptteeee 70m- 32k
5

---

Published as a conference paper at ICLR 2026
Table 1: Probe Loss evaluation (lower is better) of unsupervised approaches on CLIP and DinoV2 image encoders, DeBERTa and BART text encoders and Audio Spectrogram Transformer on audio. Supervised baseline (LDA) is reported for reference (gray row). Best results are in bold, second in italicy. Bottom right table indicates the average rank of all methods over all datasets (lower is better). "Pretrained" are models independently trained by other teams (see text for details)
CLIP
Dino V2
labels
Method
ImNet
WikiArt
ImNet
WikiArt
Artist
Style
Genre
Artist
Style
Genre
LDA
0.0083
0.0084
0.0465
0.0976
0.0044
0.0101
0.0545
0. 1084
X
ICA
0.0154
0.0141
0.0816
0.2104
0.0161
0,0155
0.0839
0.2035
Van-SAE
0.0264
0.0137
0.0558
0.1531
0.0220
0.0147
0.0722
0.1706
X
Gat-SAE
0.0384
0.0142
0.0747
0.1647
0.0345
0.0151
0.0789
0.1752
JR-SAE
0.0355
0.0138
0.0667
0.1490
0.0327
0.0148
0.0741
0.1723
X
Mat-SAE
0.0216
0.0141
0.0686
0.1588
0.0127
0.0154
0,0767
0.1613
Tk-SAE
0,0154
0.0125
0.0558
0.1360
0.0096
0.0144
0.0718
0.1577
A-SAE
0.0172
0.0130
0.0567
0.1370
0.0143
0.0145
0,0713
0.1429
Pretrained
0.0333
0.0149
0.0787
0.1796
Deleuzian (Ours)
0.0128
0.0119
0.0560
0.1230
0.0055
0.0137
0.0680
0.1538
DeBERTa
BART
CoNLL-20003
CoNLL-2003
Ibbeds
Method
IMDB
IMDB
NER
POS
Chunk
NER
POS
Chunk
LDA
0.6394
0.0429
0.0044
0.0062
0.3473
0.6326
0.3875
0.0870
ICA
0.6936
0.1251
0.0195
0.0126
0.6931
1.4578
0.7143
6.1319
Van-SAE
0.6893
0.0869
0.0252
0.0173
0.5983
0.2719
0.1647
0.0447
Gat-SAE
0.6883
0.1223
0.0251
0.3982
0.6391
0.3982
0.4054
0.3208
JR-SAE
0.6908
0.1150
0.0248
0.0170
0.6931
0.4416
0.2111
0.0883
Mat-SAE
0.6836
0.0868
0.0189
0.0164
0.6931
1.120
0.4954
0.2143
Tk-SAE
0.6858
0.0839
0.0166
0.0167
0.5980
0.3478
0.2045
0.0399
A-SAE
0,6859
0.0775
0.0141
0.0058
0.5547
0.3754
0.1959
0.0415
Deleuzian (Ours)
0.6849
0.0665
0.0161
0.0143
0.5974
0.2148
0.0639
0.0419
AST
Pythia
Avg. Rank 4
labels
Method
AudioSet
CoNLL-2003
NER
POS
Chunk
LDA
0.0164
0.0742
0.0072
0.0089
X
ICA
0.0234
0.1378
0.0331
0.0088
6.85++2.29
X
Van-SAE
0.0177
0.1498
0.0272
0.0083
4.65+1.55
X
Gat-SAE
0.0186
0.1480
0.0231
0.0086
6.65+1.42
X
JR-SAE
0.0181
0.1507
0.0277
0.0085
5.75+0.94
X
Mat-SAE
0,0186
0.1754
0.0320
0.0088
5.70
X
Tk-SAE
0.0169
0.1321
0.0203
0.0082
2.65+1.07
X
A-SAE
0.0169
0.1378
0.0331
0.0088
3.20+1.72
Pretrained
0.1717
0.0344
0.0087
X Deleuzian (Ours)
0.0164
0.1121
0.0133
0.0080
1.65
From Table I, our method globally outperforms all variations of SAE, with the lowest probe loss on 13 of the 20 tested tasks. This indicates high ability to recover attributes expected to be found in datasets, on a wide variety of tasks, models and modalities. On several cases, probe loss is midway between supervised LDA and the second most effective unsupervised method (typically TopKSAE). Note that LDA obtains poor results on BART over CoNLL-2003, which indicates that the additional hypothesis made by LDA compared
method (normal distribution of concepts

---

Published as a conference paper at ICLR 2026
Table 2: Evaluating the consistency of extracted concepts with MPPC on several tasks/datasets including WikiArt (WA), AudioSet (AS).
CLIP
D2
DeBERTa
BART
AST
ImNet
WA
ImNet
WA
IMDB
CoNLL
IMDB
CoNLL
AS
ICA
0,449
0.388
0.264
0.406
0.122
0,440
0.999
0.420
0.296
Van-SAE
0.840
0.918
0.603
0.903
0.986
0.437
0.996
0.439
0.837
Gat-SAE
0.346
0.415
0.264
0.401
0.836
0.453
0.996
0.357
0.399
JR-SAE
0.341
0,440
0.272
0.424
0.894
0.536
0.996
0.439
0.449
Mat-SAE
0.225
0.247
0.201
0.219
0.707
0.339
0.506
0.216
0.274
Tk-SAE
0.757
0.861
0.588
0.824
0.866
0.594
0.996
0.761
0.601
Deleuzian (Ours)
0.821
0.856
0.789
0.843
0.980
0.588
1.0
0.768
0.830
and homoscedasticity) are not satisfied in this particular case. On average over all datasets, our approach significantly the best classified among unsupervised approaches. Significance of the results is detailed in Appendix C.
To complement the quantitative evaluation, we further analyze representative examples, which pro- vide evidence for the relevance and interpretability of the extracted concepts: in addition to the examples provided in Figure 1, we present qualitative results in Appendix
3.2 CONSISTENCY ACROSS RUNS
order to measure consistency of a concept extraction method, we measure Maximum Pairwise Pearson Correlation (MPPC) (Wang al., 2025) 10 times between sets of concepts extracted with different random seeds, and report the average. Therefore, a MPPC closer to indicates higher consistency. We present MPPC in details and discuss its statistical significance in Appendix D.
Results from Table show that our approach generally extracts more consistent concepts than other models, except for VanillaSAE, but this method reaches much lower concept quality and diversity according to Table 1.
3.3 CONCEPT STEERING: QUALITATIVE EVIDENCE OF CAUSAL INFLUENCE
possible use extracted concepts to explicitly modify the behavior of a model, by steering its internal concepts. We provide qualitative examples of steering, using the method described in 2.4, highlighting the causal influence of concepts on the output of model.
Discriminative Steering on CLIP used to perform style transfer, in similar fashion previous works (Wynen et al., 2018). From WikiArt, we consider two concepts corresponding to artistic styles (identified empirically from im- ages), namely Romanticism and Abstract paintings. Starting from romantic painting of sailing ship, we inhibit the Romanticism concept, and boost the Abstract paintings one. The resulting steered embedding shifts the painting's representation such that its nearest neighbors in the WikiArt dataset are abstract sailing ships (Figure 3).
Steering the inner representation of an image encoder may be
4.9 Romanticism
+4.9 Abstract
Figure 3: Stecring a painting style in CLIP activations: target represented by its nearest images. Romanticism is set to zero, while Abstract is steered positively by the same magnitude.

---

Published as a conference paper at ICLR 2026
Steering BART BART (Lewis al., 2020) is text incoder-decodee model which, without fine- tuning, typically reproduces its input sequence. Here, we steer the final transformer layer of its encoder before passing the modified representation into the decoder. We analyze the steering ef- fects of a concept with highest activations corresponding to country names (Figure 4). Inhibiting this concept (a 0) causes BART replace "Rio de Janeiro" with "Fehruary", forming coher- ent sentence with no geographical indication. In the same fashion, its leads to replacing the word "country" by the word "city". Positive values of a encourage the model evoke country names, even in sentences without geographic context. In particular, this leads to frequent mentions of the United States, highlighting potential bias BART.
BART has potertial bias toword the United has rolationship
country inside country in the United States. The United States mamber the United Kingstom. The USA country
Ententemerr
contere takess place Janeira, in the Linited States hasied USA and Canada
United States, USA and Canada (buth countries, respectivaly USA and Canada ialatiorshpp USA Canads
+5
+0-
lakes place Janairo
The country inside city
has potantial
Inbiiio
The conference takes place February
The Vaticen city inside city
BART has potental bls
Figure 4: Steering the concept of countries BART model for three sentences (in gray), using
+5 and
-5 in each case
3.4 ABLATION STUDIES
Table 3: Ablation study in terms of performance (Probe Loss) and diversity (effective rank). Our
approach is the last line.
skewness weighting
probe loss J
effective rank 1
max. pairwise cos.
input space
concept
CLIP
DeBERTa CoNLL
CLIP WikiArt
DeBERTa CoNLL
CLIP
DeBERTa
identif.
WikiArt
WikiArt
CoNLL
acts.
Tk-SAE
X
0.0125
0.0839
96.1
183.9
0.2900
0.3716
acts.
KMeans Tk-SAE
0.0133
0.1184
24.3
14.6
0,8685
0.9195
diff
0.0134
0.1093
340.5
109.2
0.3407
0.1737
diff
KMeans
X
0.0128
0.0841
17.9
5.65
0.6504
0.8357
diff
KMeans
0.0119
0.0665
124.4
182.0
0.5677
0.3908
We conduct ablation study of our method, to assess the impact of three aspects its perfor- mance. First, we evaluate the interest of learning from differences between samples, rather than directly from the samples themselves (i.e. changing the input space). Second, pact of using a clustering to identify the concepts, by replacing the the KMeans clustering of our approach with an SAE, trained the activations the differences. Finally, we evaluate the impact of weighting the KMeans clustering by the inverse skewness. Since the objective of this weight- ing to increase diversity, also report evaluation of the diversity of the extracted concepts, measured by the effective rank (Roy & Vetterli, 2007: Skean et al., 2025), well as the maximum pairwise cosine among concept directions that quantifies redundancy. Results, computed on CLIP activations on WikiArt, and DeBerta on CoNLI NER attributes, are reported in Table 3. These results, most notably those for KMeaas on activations and TopKSAE on differences highlight the impact of representing differences in activations. Moreover, these results highlight the importance of using the inverse skewness of pairwise differences as KMeans weights, allowing the extraction of much larger, and less redundant sets of concepts, according to both effective rank and maximum pairwise cosine metrics.
evaluate the im-
Figure 5 evaluates the performance of method while extracting number of concepts smaller than 6 614.. Only 2000 concepts are needed to outperform every concurrent method on CLIP, WikiArt artist task. This highlights the ability of our method to efficiently recover concepts.
8

---

Published as a conference paper at ICLR 2026
Probe Loss vs. Number of Concepts
0.0135
Deleuzian
0.0134
Archetypal-SAE (0.0130)
Topk-SAE (0.0125)
0.0132
0.0131
0.0130
Probe Loss
0.0128
0.0127
0.0126
0.0124
0.0122
0.0121
0.0120
0.0120
100
250
500
1,000
2,000
4,000
Number of Concepts
Figure 5: Performance of our Deleuzian approach using less than 6144 concepts, on CLIP, WikiArt
artist task.
RELATED WORKS
Concept-Based Interpretability Identifying the internal mechanism of neural network corre- sponding precise concept provides valuable insights into the network's behavior. Arik & Liu (2020) perform clustering on multi-layer activations, in order to determine similar images, not to extract interpretable concepts. Prior studies have investigated the extent which classification probe can be learned directly on model hidden representations (Köhn, 2015). Probe-based concept extraction has been used extensively NLP (Gupta al., 2015). These studies suggest that LLMs linearly represent the truth or falsehood of factual statements (Marks & Tegmark, 2024). Simi- lar analyses have also been applied to computer vision (Alain Bengio, 2017) or reinforcement learning (Lovering et al., 2022). However, probe- based concept extraction only captures correlation (not causation) and heavily relies on curated data extract concepts (Belinkov, 2022). To address this problem, Concept Bottleneck Models (CBM) (Koh et al., 2020) structure the network to make comann-defined concepts, enabling intervention but requiring labeled predictions through a layer Contssst-onnsstten Search probes for an axis in the activation space, corre- concept supervision.
sponding to the presence or absence of concept (Burns et al., 2023), however uses predefined concepts. Similarly, TCAV (Kim et al., 2018)
contrastive groupings, and thus cannot and ACE (Ghorbani et al., 2019) perform concept extraction upon predefined list.
Sparse Autoencoders Sparse autoencoders (SAEs) (Lee et al., 2007) are sparse dictionary learn- ing technique that aims to find sparse decomposition of data into an overcomplete set of features. They typically enforce sparsity via an L1 penalty. In recent years, SAEs have been applied neu- ral networks to learn unsupervised dictionary of interpretable features tied to concepts from hidden representation (Bricken al., 2023; Cunningham al., 2023). Various extensions of sparse autoencoders have been proposed with modified activation functions, such as JumpRel.U (Raja- manoharan et al., 2024b), TopK (Gao et al., 2025), and BatchTopK (Bussmann et al., 2024) sparse autoencoders. Other works seek hierarchies of features by extracting nested dictionaries (Bussmann et al., 2025; Zaigrajew et al., 2025). Analogous methods have been developed order to find rela- tions between different layers of same network, including transcoders (Dunefsky et al., 2024) and crosscoders (Lindsey et al., Archetyrcct (Fel et al., 2025) constrains the decoder training in order gain stability, while Spade (Hindupur et al., 2025) is distance based SAE.
Further use of extracted concepts Identifying the mechanism corresponding to a semantic concept of the analyzed model. For example, studies extracted
within neural network enables

---

Published as a conference paper at ICLR 2026
concepts to analyze the circuits related to specific task (Conmy al., 2023; Dunefsky al., 2024), or to measure the importance of concepts in model inner representations (Fel et al., 2023). Concept extraction techniques can also be used to perform steering, ie. controlling the behavior of model by explicitly modifying its internal concepts (Zhou et al., 2025). When applied to multiple models in parallel, concept extraction methods allow construction of shared concept spaces (Thasarathan al., 2025), automating naming of CLIP concepts (Rao et al., 2024) and quantification of similarities between models (Wang al., 2025).
CONCLUSION
Discussion We present a novel approach for extracting human- cumanninterpreetbble "concepts" from neu- ral network activations and evaluate it across five models and three modalities. Our method is sim- ple and can be interpreted as an unsupervised form of discriminant analysis. Probe loss evaluation shows that the extracted concept space captures attributes expected from labeled datasets, and our approach outperforms existing methods on this metric. Moreover, the concepts are stable across mul- tiple runs, enabling consistent analyses, and the method supports lossless interventions on internal representations. These results suggest that explicitly representing inter-sample differences, in line with Deleuze's ' notion of concepts, can improve both the quality and utility of extracted concepts.
Limitations Although our method is fully unsupervised. its evaluation depends on labeled datasets. Consequently, interpretable concepts that do not align with the available labels may in- cur high probe losses, even they are highly meaningful but subtle or specific. Evaluating without labels would require a theoretically justified proxy for interpretability. which remains lacking; spar- sity alone does not satisfy this criterion (Sharkey et al., 2025).
All evaluations are performed in concept spaces of 6, 6,144 dimensions (8x the activation dimension), except for an ablation. While some studies use even higher-dimensional projections, further creasing dimensionality could bias our evaluation, given the limited number of attributes and data samples relative to the potential size of the concept space. Exploring higher-dimensional spaces could nonetheless reveal additional characteristics of concept extraction methods.
Our approach assumes that concepts can be represented as linear projections. This assumption is empirically validated across five models spanning different categories and modalities. However, model with inner representations that violate this assumption could exist and would require adapting the method.
Perspectives Our method is fully unsupervised and extracts concepts that represent repeated di- rections in a model. Consequently, a method that can automatically name or interpret these concepts would greatly enhance the scope and applicability of the findings, enabling more comprehensive analyses across datasets, modalities, and models. Such generalization could facilitate understanding of model behavior, provide interpretable axes for interventions, and support downstream tasks that leverage concept-level information. We provide qualitative examples of concept steering. As our method allows lossless steering, such intervention on model inner representations could be used larger scale, for example to adapt to specific domain.
REPRODUCIBILITY STATEMENT
Our results can be reproduced, following the method described in section 2 and Appendix A. Cor- responding code provided at
ACKNOWLEDGEMEEN
This work was partially funded by the Agence Nationale de la Recherche (ANR) for the STUDIES project 1AN-233CE33--0014-02 and SHARP (ANR-23-PENL -0088 in the context of the France 2030 program). We would like to thank the reviewers and chairs for their constructive feedback and careful consideration of our rebuttal. We appreciate the significant effort put into the review process, especially given the challenging circumstances.
10

---

Published as a conference paper at ICLR 2026
REFERENCES
Miika Ahdesmäki and Korbinian Strimmer. Feature selection in omics prediction problems using cat scores and false nondiscovery rate control. Annals of Applied Statistics, 4(1):503 519, 2010.
Guillaume Alain and Yoshua Bengio. Understanding intermediate layers using linear classifier probes, 2017. URL https: http///pppeeeieeeeeeeeeeeeeeeeee net wttp://eee/ooommmdddddl...ll..
Sercan Arik and Yu-Han Liu. Explaining deep neural networks using unsupervised clustering. In Proc. Workshop Hum. Interpretabilityy Mach. Learn, pp. 377-389, 2020.
Peter Baylies. Wikiart dataset, 2020.
URL https:/ wwww www.wwwww........... com/ /atasets/
steubk/ steubk/wikiarz wikiart.
Yonatan Belinkov. Probing classifiers: Promises, shortcomings, and advances. Computational Linguistics, 48(1):207-219, 2022.
Stella Biderman, Hailey Schoelkopf, Quentin Gregory Anthony, Herbie Bradley, Kyle O'Brien, Eric Hallahan, Mohammad Aflah Khan, Shivanshu Purohit, USVSN Sai Prashanth, Edward Raff, et al. Pythia: A suite for analyzing large language models across training and scaling. In International Conference on Machine Learning, pp. 2397-2430. PMLR, 2023.
Trenton Bricken, Adly Templeton, Joshua Batson, Brian Chen, Adam Jermyn, Tom Conerly, Nick Turner, Cem Anil, Carson Demison, Amanda Askell, et al. Towards monosemanticity: Decom- posing language models with dictionary learning. Transformer Circuits Thread, 2023.
Collin Burns, Haotian Ye, Dan Klein, and Jacob Steinhardt. Discovering latent knowledge in lan- guage models without supervision. In The Eleventh International Conference on Learning Rep- resentations, 2023.
Bart Bussmann, Patrick Leask, and Neel Nanda. Batchtopk sparse autoencoders. In NeurIPS 2024 Workshop on Scientific Methods for Understanding Deep Learning, 2024.
Bart Bussmann, Noa Nabeshima, Adam Karvonen, and Neel Nanda. Learning multi-level features with matryoshka sparse autoencoders. In Forty-second International Conference Machine Learning. 2025.
Pierre Comon. Independent component analysis, new concept? Signal processing, 36(3):287-314, 1994.
Arthur Conmy, Augustine Mavor-Parke Aengus Lynch, Stefan Heimersheim, and Adrià Garrga-- Alonso. Towards automated circuit discovery for mechanistic interpretability. Advances in Neural Information Processing Systems, 36: 16318-166355 2023.
Hoagy Cunningham, Aidan Ewart, Logan Riggs, Robert Huben, and Lee Sharkey. Sparse autoen- coders find highly interpretable features in language models. arXiv preprint (00::230000000000 2023.
G. Deleuze. Différence et répétition. Bibliothèque de philosophie contemporaine Histoire de la philosophic et philosophie générale. Presses Universitaires de France, 1968. ISBN 9782130585299, URL htlLht:ps://bbe
google.
Jacob Devlin, Ming-We Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: Pre-training of deep bidirectional transformers for language understanding. In Jill Burstein, Christy Doran, and Thamar Solorio (eds.), Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pp. 4171-418 Minneapolis, Minnesota, June 2019. Association for Com- putational Linguistics. doi: 10. (0.18653/v1/N19- 1423. URL https: /ttlls://aaaaattaa hww.aaaaanooooyyy..oooooooooooa. .org/ N1 1423/.
Jacob Dunefsky, Philippe Chlenski, and Neel Nanda. Transcoders find interpretable Ilm feature circuits. Advances Neural Information Processing Systems, 37:24375-24410 2024.

---

Published as a conference paper at ICLR 2026
Thomas Fel, Victor Boutin, Louis Béthune, Rémi Cadène, Mazda Moayeri, Léo Andéol, Mathieu Chalvidal, and Thomas Serre. A holistic approach to unifying automatic concept extraction and concept importance estimation. Advances in Neural Information Processing Systems, 36:54805- 54818, 2023.
Thomas Fel, Ekdeep Singh Lubana, Jacob Prince, Matthew Kowal, Victor Boutin, Isabel Papadim- itriou, Binxu Wang. Martin Wattenberg, Demba Ba, and Talia Konkle. Archetypal sae: Adap- tive and stable dictionary learning for concept extraction in large vision models. In Forty-second International Conference Machine Learning (ICLR), 2025.
R. A. Fisher. The use of multiple measurements in taxonomic problems. Annals of Eugenics, 7(7): 179-188, 1936.
Ronald A Fisher. Frequency distribution of the values of the correlation coefficient in samples from an indefinitely large population. Biometrika, 10(4):507-521, 1915.
Leo Gao, Tom Dupré la Tour, Henk Tillman, Gabriel Goh, Rajan Troll, Alec Radford, Ilya Sutskever, Jan Leike, and Jeffrey Wu. Scaling and evaluating sparse autoencoders. International Confer- ence on Representation Learning (ICLR), 2025.
Jort F. Gemmeke, Daniel P. W. Ellis, Dylan Freedman, Aren Jansen, Wade Lawrence, Channing Moore, Manoj Plakal, and Marvin Ritter. Audio set: An ontology and human-labeled dataset for audio events. In Proc. IEEE ICASSP 2017, New Orleans, LA, 2017.
Amirata Ghorbani, James Wexler, James Y Zou, and Been Kim. Towards automatic concept-based explanations. Advances in neural information processing systems, 32, 2019.
Yuan Gong, Yu-An Chung, and James Glass. Ast: Audio spectrogram transformer. Interspeech 2021, 2021.
Abhijeet Gupta, Gemma Boleda, Marco Baroni, and Sebastian Padó. Distributional vectors encode referential attributes. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pp. 12-21, 2015.
Pengcheng He, Xiaodong Liu, Jianfeng Gao, and Weizhu Chen. DeBERTa: Decoding-enhanced BERT with disentangled attention. In International Conference on Learning Representations, 2021. URL https: / owwnreview..o.. net/
Georg Wilhelm Friedrich Hegel. Wissenschaft der Logik: Die objective Logik, volume 2. Johann Leonhard Schrag, 1816.
Sai Sumedh R Hindupur, Ekdeep Singh Lubana, Thomas Fel, and Demba Ba. Projecting assump- tions: The duality between sparse autoencoders and concept geometry. In ICML 2025 Workshop on Methods and Opportunities at Small Scale, 2025. URL CGrum?1d=xKa32hhIF
n
Joshua Zhexue Huang, Michael K Ng. Hongqiang Rong, and Zichen Li. Automated variable weight- ing in m-manns type clustering. IEEE transactions on pattern analysis and machine intelligence, 27(5):657-66, 2005.
A. Hyvärinen and E. Oja. Independent component analysis: algorithms and applica- tions. Neural Networks, 13(4):411-430, 2000. ISSN 0893-6080. doi: https://doi.on org/ 10. 1016650893-608000000002665 URL https: //tp://wwww. sci wew.eiieeeiiccc rect. wwwtteeeetteeeeeeeeeeeeeeeeeeeee ence/ arricle//p1//0883300000000
Gabriel Ilharco, Mitchell Wortsman, Ross Wightman, Cade Gordon, Nicholas Carlini, Rohan Taori, Achal Dave, Vaishaal Shankar, Hongseok Namkoong, John Miller, Hannanch Hajishirzi, Ali Farhadi, and Ludwig Schmidt. Openclip. July 2021. URL sattps://deee org 5281/ zenodo. 5143773.
Sonia Joseph, Praneet Suresh, Lorenz Hufe, Edward Stevinson, Robert Graham, Yash Vadi, Danilo Bzdok, Sebastian Lapuschkin, Lee Sharkey, and Blake Aaron Richards. Prisma: An open source toolkit for mechanistic interpretability vision and video. arXiv preprint arXiv:2504. 19475, 2025.
12

---

Published as a conference paper at ICLR 2026
Been Kim, Martin Wattenberg, Justin Gilmer, Carrie Cai, James Wexler, Fernanda Viegas, et al. Interpretability beyond feature attribution: Quantitative testing with concept activation vectors PMLR, 2018.
(tcav). In International conference machine learning, pp-
Pang Wei Koh, Thao Nguyen, Yew Siang Tang, Stephen Mussmann, Emma Pierson, Been Kim, and Percy Liang. Concept bottleneck models. In International conference on machine learning, pp- 1338-5345 PMLR, 2020.
Arne Köhn. What's in an embedding? analyzing word embeddings through multilingual evaluation. In Proceedings of the 2015 Conference on Empirical Methods Natural Language Processing, pp. 2067-2007, 2015.
Honglak Lee, Chaitanya Ekanadham, and Andrew Ng. Sparse deep belief net model for visual area v2. Advances in neural information processing systems, 20, 2007.
Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy. Veselin Stoyanov, and Luke 7ettlemoyer. Bart: Denoising sequence-to-sequence pre- training for natural language generation, translation, and comprehension. In Proceedings of the 58th Anmual Meeting of the Association for Computational Linguistics, pp. 7871-7888 2020.
Jack Lindsey, Adly Templeton, Jonathan Marcus, Thomas Conerly, Jared Batson, and Chris Olah. Sparse crosscoders for cross-layer features and model diffing. Transformer Circuits, 2024. URL https: / / wwrranssoormt ci ciruuiss...... ppbpp024 crosscoders/index, html.
Stuart Lloyd. Least squares quantization in pem. IEEE transactions on information theory, 28(2): 129-137, 1982.
Charles Lovering, Jessica Forde, George Konidaris, Ellie Pavlick, and Michael Littman. Evaluation beyond task performance: analyzing concepts in alphazero in hex. Advances in neural information processing systems,
2022.
Andrew L. Maas, Raymond E. Daly, Peter Pham, Dan Huang, Andrew Y. Ng, and Christopher Potts. Learning word vectors for sentiment analysis. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, pp. 142-150, Portland, Oregon, USA, June 2011. Association for Computational Linguistics. URL http: // //www.aa. ac web. wrgg/nnthloggy hotllolyyy//111/11
Samuel Marks and Max Tegmark. The geometry of truth: Emergent linear structure in large language model representations of true/false datasets. In First Conference Language Modeling, 2024.
URL https::////. wwwp//eeeeeeeeeeeeeeeeeeee net.tewwwweeeeeeeeeeeee....eeeee forum? (d-aajyyHjjjss
Geoffrey McLachlan. Discriminant analysis and statistical pattern recognition. John Wiley & Sons, 2005.
Glenn W Milligan. An examination of the effect of six types of error perturbation on fifteen cluster- ing algorithms. psychometrika, 45(3):325-344 342, 1980.
Friedrich Nietzsche. Götzen-Dämmerung oder Wie man mit dem Hammer philosophirt. CG Nau-
mann, 1889.
Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy Vo, Mare Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaacldin El-Nouby, et al. Dinov2: Learning robust visual features without supervision. arXiv preprint arXiv:2304.07193. 2023.
Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Pretten- hofer, R. Weiss, V. Dubourg, Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay. Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 12:2825-2834 2011.
Plato. The republic- book vi, 375 BCE.
13

---

Published as a conference paper at ICLR 2026
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agar- wal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural language supervision. In Marina Meila and Tong Zhang (eds.), Proceedings of the 38th International Conference on Machine Learning, volume 139 of Proceedings of Machine Learning Research, pp. rttp//dddfor...... 18-24 Jul 2021. URL https://proceed/ html.
PMLR,
Senthooran Rajamanoharan, Arthur Conmy, Lewis Smith, Tom Lieherum, Vikrant Varma, Janos Kramar, Rohin Shah, and Neel Nanda. Improving sparse decomposition of language model acti- vations with gated sparse autoencoders. Advances in Neural Information Processing Systems, 37: 775-818, 2024a.
Senthooran
Rajamanoharan, Tom Lieberum, Nicolas Sonnerat, Arthur Conmy, Vikrant Varma, János Kramár, and Neel Nanda. Jumping ahead: Improving reconstruction fidelity with jumprelu sparse autoencoders. arXiv preprint arXiv:2407. 14435, 2024b.
Radhakrishha Rao. The utilization of multiple measurements in problems of biological classifica- tion. Journal of the Royal Statistical Society. Series B (Methodological), 10(2):159 1948.
Sukrut Rao, Sweta Mahajan, Moritz Böhle, and Bernt Schiele. Discover-then-namee Task-agnostic concept bottlenecks via automated concept discovery. In European Conference on Computer Vision, pp. 444 461. Springer, 2024.
Olivier Roy and Martin Vetterli. The effective rank: A measure of effective dimensionality. In 2007 15th European signal processing conference, pp. 606-610. IEEE, 2007.
Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, Alexander C. Berg, and Li Fei-Fei. ImageNet Large Scale Visual Recognition Challenge. International Journal of Computer Vision (IJCV), (5(3):211-2555 2015. doi: 10. 100%/1 1007/s11263311 1263-015-081611 066---
Jean-Paul Sartre and Arlette Elkaim-Sartre. L' exiittentialimme est un humanisme, 1946.
Lee Sharkey, Bilal Chughtai, Joshua Batson, Jack Lindsey, Jeff Wu, Lucius Bushnaq, Nicholas Goldowsky-Die Stefan Heimersheim, Alejandro Ortega, Joseph Bloom, et Open problems in mechanistic interpretability. arXiv preprint arXiv:2501.16496 2025.
Oscar Skean, Md Rifat Arefin, Dan Zhao, Niket Nikul Patel, Jalal Naghiyev, Yann LeCun, and Ravid Shwartz-Ziv. Layer by layer: Uncovering hidden representations in language models. In Forty-second International Conference on Machine Learning. 2025.
Gemma Team, Aishwarya Kamath, Johan Ferret, Shreya Pathak, Nino Vieillard, Ramona Merhej, Sarah Perrin, Tatiana Matejovicova, Alexandre Ramé, Morgane Rivière, al. Gemma 3 technical report. arXiv preprint arXiv:2503. 19786, 2025.
Harrish Thasarathan, Julian Forsyth, Thomas Fel, Matthew Kowal, and Konstantinos Derpanis. Universal sparse autoencoders: Interpretable cross-model concept alignment. arXiv preprint
arXiv:2502.03714. 2025.
Bart Thomee, David A. Shamma, Gerald Friedland, Benjamin Elizalde, Karl Ni, Douglas Poland, Damian Borth, and Li-Jia Li. Yfee 100m: the new data in multimedia research. Commun. ACM, 59(2):64-73, January 2016. ISSN 0001-0782. doi: 10. 14822812802. URL https://doi. org/10. 11/114//200 /22852222222
Erik F. Tjong Kim Sang and Fien De Meulder. Introduction to the CoNLL-2003 shared task: Language-independer named entity recognition. In Proceedings of the Seventh Conference Natural Language Learning at HL.T-AACI, 2003, pp. 142-147, 2003. URL https:
/ //www. ac web.. org/ wwwatooooooy...... 0419.
Junxuan Wang, Xuyang Ge, Wentao Shu, Qiong Tang, Yunhua Zhou, Zhengfu He, and Xipeng Qiu. Towards universality: Studying mechanistic similarity across language model architectures. In The Thirteenth International Conference on Learning Representations, 2025. URL https:
// //operevvie ew. net. forum? i 818T00I.
d=231
14

---

Published as a conference paper at ICLR 2026
Daan Wynen, Cordelia Schmid, and Julien Mairal. Unsupervised learning of artistic styles with archetypal style amalysis. Advances Neural Information Processing Systems, 31, 2018.
Vladimir Zaigrajew, Hubert Baniecki, and Przemyslaw Biecek. Interpreting clip with hierarchical sparse autoencoders. In Forty-second International Conference Machine Learning, 2025.
Xiangrui Zeng and Hongyu Zheng. Cs sparse k-means: algorithm for speccfc feature selection in high-dimensional clustering. arXiv preprint arXiv: 12384, 2019.
Dylan Zhou, Kumal Patil, Yifan Sun, Karthik lakshmanan, Senthooran Rajamanoharan, and Arthur Conmy. LLM neurosurgeon: Targeted knowledge removal LLMs using sparse autoencoders. In ICLR 2025 Workshop Building Trust in Language Models and Applications, 2025. URL https: ftttps://ppenrveee eew.. net/ d
15

---

Published as a conference paper at ICLR 2026
A APPENDIX: IMPLEMENTATION DETAILS
All our experiments are using a set of 6144 concepts, except for ICA, that is unable to represent number of dimensions larger than D, the dimension of model activations. Therefore, ICA experi- ments are ran in D = 768 dimensions.
TopKSAES are trained using a Topk activation function, with k = 32. We select a learning rate of 10-s, that minimizes its reconstruction error on CLIP activations over ImageNet. For VanillaSAE, GatedSAF and JumpRel.USAE, we select the 7,1 penalization coefficient reaching the lowest probe loss. From a sweep of 7 values between 10 and 10 we select 10 8 for VanillaSAE, 10-6 for GatedSAE and 10 S for JumpReLUSAEE Concerning MatryoshkaSAE, we use groups of sizes [512, 1024, 1536, 3072], in order to represent progressively larger latent dictionaries.
For Independent Component Analysis used the scikit-learn (Pedregosa et al., 2011) implemen- tation of FastICA (Hyvärinen & Oja, 2000), with log hyperbolic cosine to approximate the neg- entropy, SVD whitening and the extraction of multiple components in parallel.
B APPENDIX: DETAILS ON EXPERIMENTAL SETUP
All datasets used in our experiments (section 3) are reported in Table 4 with their main characteris- tics. When available, we use the train/test splits provided. As WikiArt has no predefined train/test sets, we use its even samples (0, 2, 4...) as a train set, and the other ones as the test set. Note that WikiArt is actually a of data with three different label types, thus could be considered three different datasets.
Globally we thus have much larger variety of experimental settings than in comparable previous works. Since we are interested in identifying concepts, all tasks relate to classification but they exhibit a deep variety in their nature, due the type of data handled (text, image, audio) and how the data have to be considered to address the task. For example, the identifying sentiments on IMDB requires to take into account full sentences while the chunking task CoNLL act at the token level.
Table 4: Datasets used in our experiments.
Dataset
Modality
Label Type (number of classes)
Train/Tes Size
URL
ImageNet- -00
Image
Object categories (100)
50k / /5k
WikiArt
Image
Artist (129), Style (27), Genre (11)
40k / 40k
IMDB
Text
Sentiment (binary, sentence-level)
25k / 25k
CoNLL-2003
Text
NER (9), POS (47), Chunking (23, token-level)
288k 67k
AudioSet
Audio
Audio event categories (527)
18k / 17k
The model encoders we considered in our experiments are summarized in Table 5. All the models were downloaded from huggingface, except for CLIP from OpenClip (Ilharco al., 2021) and DinoV2 from PyTorch Hub. The model size is the number of parameters and since all of them were encoded in flcat32 their actual size in memory this number multiplied by four.
AST (Gong et al., 2021) relies on an image ViT that was trained on ImageNet-21k 1k then finetuned on AudioSet. BART (Lewis al., 2020), for its base version, was pre-trained "on the same data as BERT (Devlin et al., 2019)" that is "a combination of books and Wikipedia data". CLIP (Radford et al., 2021) was trained "on publicly available image-caption data" that is images-caption pairs from the Web and publicly available datasets such as YFCC 100M (Thomee et al., 2016). The creator of the model did not release the dataset to avoid its use "as the basis for any commercial or deployed model". DeBERTa (He et al., 2021) was trained on deduplicated data (78G) including original Wikipedia (English Wikipedia dump; 12GB). BookCorpus (6GB), WebText (public Reddit content; 38GB), and STORIES (a subset of CommonCrawl; 31GB). Dino V2 (Oquab et al., 2023) was trained on the LVD- dataset, that was assembled and curated by the authors of the model.
16

---

Published as a conference paper at ICLR 2026
Table 5: Pretrained models used in our experiments.
The Size is the number of parameters (in
millions).
Model
Modality
Version
Size
Training data
URL
BookCorpus, Wikipedia, Open Text, STORIES
DeBERTa
Text
base
99
BART (encoder)
Text
base
Books, Wikipedia
D2
Image
VIT-B/14
86 M
LVD-142
CLIP
Image
ViT-B/32
150
openAl private: web, YFCC100M...
AST
Audio
10-10-0.4593
87 M
AudioSet, ImageNet-21k
500
VanillaSAE
379
449
381
45.5
165
71
400
Method (Reference)
GatedSAE
241
78
277
-300
JumReLUSAE
146
400
388
MatryoshkaSAE
221
72
250
139
23
16
200
TopKSAE
100
Ours
468
456
515
500
511
366
VénillaSAE
GatedSAE
JumReLUSAE
MatryoshlaSAE
TopeSAE
Ours
Method (Compared To)
Figure 6: Pairwise comparisons of methods on AST-Audioset. Our method is better able to recover
least 366/527 attributes compared to concurrent methods.
C APPENDIX: SIGNIFICANCE OF PROBE Loss RESULTS
Table I reports the median probe loss for each task. In Figure 6. perform sitribute-wise compar- isons on AST- Audioses, the studied task comprising the largest number of attributes. The numbers represent how many times the method of cach row better recovers the attributes than the methods on the column. For instance, last row show that our method attributes of Audioset.
Our method is able to better recover least 366/527 attributes (6.4%% than other methods. Per- forming a Wilcoxon signed-rant -rank test, we obtain a statistic of 106584 with p-value of 1.7 x 10 10-26 rejecting the null hypothesis thus proving the significance of those probe loss results.
In a similar fashion on CLIP-WikiArt, our method reaches a lower probe loss than TopKSAE on 140/167 attributes (83.8%, even with TopKSAE reaching lower probe loss the "style" at- tributes), obtaining a test statistic of 12671 and a p-value of 7.9 x 10 20-0 rejecting the null hy- pothesis.
D APPENDIX: STATISTICAL SIGNIFICANCE OF MPPC
The Maximum Pairwise Pearson Correlation (MPPC) was proposed by Wang et al. (2025) as similarity indicator between models.
17

---

Published as a conference paper at ICLR 2026
D.1 DEFINITION OF MPPC
To compare two sets of extracted concepts A and B, p is defined as the maximum pairwise Pearson correlation between the i-th concept of A and all concepts of B. With fi the vector containing values for each sample for the i-th concepts of A, n. and of its mean and standard deviation (respectively for f .uf.. and of)):
BI
1 M n 0/00
p A-B max
Then, MPPCA-B is defined as the arithmetic mean of p A-B over all i, quantifying the extent to which the concepts in A are represented in B. In order to measure consistency of concept extraction method, we measure MPPC 10 times between random seeds, and report the average. Therefore, MPPC closer to I indicates higher consistency.
of concepts extracted with different
D.2 STATISTICAL SIGNIFICANCE OUR CASE
With Ai the maximum pairwise coefficient (Eq. 3) for target features of length N, and Ho the hypothesis of features having no linear relationship. Using the Fischer z-transformation (Fisher, 1915)
= artanh(r) ~ 11(0,
VN
P(max((:)) > x) - (r S x)
P(p > x)) = P(max(2;) > artanh(x))
P(p: > x) = (artanh((z) VN 3)*
With 6144 (corresponding to the main experiments). and 10000 being largely lower than the size of the most used datasets, we obtain P(,; > 0.3) x 10 206 thus reject Ho.
E APPENDIX: QUALITATIVE EXAMPLES OF EXTRACTED CONCEPTS
Image Concepts We present in figure 7 three examples of concepts extracted from image models, from different datasets. The concepts are represented by the images with their nine highest acti- vations. The name of the concepts are empirically set from the images. Displayed concepts are extracted from CLIP's activations, with figs. WikiArt and corresponding to paintings content, and figs. 7g to 7i corresponding to artistic styles.
7c extracted from ImNet, figs. 7d to 7f from
Text Concepts In Table and Table 7, we represent textual concepts. For each concept, we dis- play the sentences containing the highest token-wise concept values, and underline tokens among the t00.
F APPENDIX: ADDITIONAL STEERING EXAMPLES
Textual Concept: Baseball Extracted from DeBERTa, over CoNLL-2003. Enhancing this con- cept (positive values of alpha) causes replacement of any sport-specific terms (football, basketball) by their baseball equivalent. Those changes affect mentions of teams, leagues and scoring methods.
(+t 0) The best sport is basketball, NBA the best -> (+3.75) The best sport is baseball, MLB is the best
(+t 0) He scored 3 touchdowns in the first half + (+4.5) He scored 3 RBI in the first inning
(+t 0) The New York Knicks beat the Los Angeles Lakers + (+3.75) The New York Yan- kees beat the Los Angeles Dodgers
18

---

Published as a conference paper at ICLR 2026
(a) Puppies
(b) Underwater Animals
(c) Guitars
(d) Portraits of children
(c) Paintings of cliffs
(f) Paintings of Venice
(g) Popart
(b) Cubist paintings
Minimalist paintings
Figure 7: Nine examples of visual concepts extracted from CLIP, over ImageNet and WikiArt. Representing the images with the highest activations for each.
19

---

Published as a conference paper at ICLR 2026
Table 6: Examples of textual concepts extracted from DeBERTa on CoNLL-2003, Each column is a concept with three representative texts. Concept names are ours.
Sports Achievements
Last Names
Nationalities
Seven athletes went into Fri- day's ss penultimate meeting of the series with a chance of winning the prize.
Katarina Studenikova (Slo-
One Romanian passenger was killed, and 14 others were injured on Thursday when Romanian renastin-registered bus collided with a Bulgarian one in northern Bulgaria, police said.
vakia) beat 6- Karina
Habsudova.
Russia's double Olympic champion Svetlana Mas- terkova smashed her second world record just 10 days Friday when she bettered the mark for the women's 1, 1,000 metres.
Hendrik
Dreekman (Ger- Greg Rusedski
He said Turkish civil avi- ation authority official had made the same point and he noted that Turkish plane had similar accident there 1994.
many) vs. (Britain).
Jamaican veteran Merlene Ottey, who beat Devers in Zurich after just missing out on the gold medal in Atlanta after photo finish, had to settle for third place in I 11.04.
The Greek socialist party's executive bureau gave the green light to Prime Minis- ter Costas Simitis to call snap elections, its general secre- tary Costas Skandalidis told reporters.
A Polish school girl black- mailed two women with anonymous letters threaten- ing death and later explained that she needed money for textbooks, police said on Thursday.
Table 7: Additional Examples of textual concepts extracted from DeBERTa on CoNLL-2003. Each column is a concept with three representative texts. Concept names are ours.
Years from the 1990's
Age
Geopolitical Evolutions
West lake, arrested Decem- ber 1993 and charged with heroin trafficking sawed the iron grill off his cell window
Machado, 19, flew to Los An- geles after slipping away from the New Mexico desert town of Las Cruces
Peruvian guerrillas killed one man and took eight peo- ple hostage after taking over village in the country' northeastern jungle
Since taking over as cap- tain from Ne ale Fraser in 1994, Newcombe' s record in tandem with Roche, his for- mer doubles partner, has been three wins and three losses.
The 13 - year- old girl tried to extract 60 and 70 zlotys $22 and $26 ) from two residents of Sierakowice by threatening take their lives.
[...] is ready at any time without preconditions enter peace negotiations
The bullish comments for the coming year soothed ana- lysts and most shareholders who were disappointed by the lower than expected profit for 1995/96.
On Tuesday night, Kevorkian attended the death of Louise Siebens, a 76-year-old Texas woman with amyotrophic lat- eral sclerosis
[..]] that is to end the state of hostility
20

---

Published as a conference paper at ICLR 2026
"This Painting is..."
Negativity
Positivity
truly spectaculaar because truly spectacular
one big blunder young van gogh +280
one big mistake
+180
truly beautiful because how colorful truly
masterpiece from the Dutch painter Vincent Van Gogh
Figure 8: Steering Gemma3 captioning of The Starry Night, by Vincent Van Gogh, upon concepts corresponding to positivity and negativity.
Steering Gemma3 Image Captioning Our method can extract concepts from large decoder mod- els. From the text decoder of Gemma3-4B-P7 (Team et al., 2025), we extract concepts over the IMDB dataset. We steer two concepts identified as corresponding to positivity/negatiiiyy during image captioning, see Figure 8.
G QUADRATIC EXTENSION OF DELEUZIAN CONCEPTS
As our Deleuzian approach is analog to a Linear Discriminant Analysis (LDA) as per subsection 2.3, it makes hypothesis about isotropic distribution of concepts in model's activation. However, can derive an extension of the Deleuzian method, that does not make those hypothesis, analog with Quadratic Discriminant Analysis. The aim is extract discriminant functions ò, R I from neural networks' activations. Each function 6 must then correspond to an interpretable concept.
G.1 DISCRIMINANT FUNCTION 5
discriminant function ô is defined from a randomly sampled pair of samples 2y 2y the linear Deleuzian method considers linear concepts.
8(x) = b" x
With = -; - zy. Such formulation equivalent to a linear discriminant analysis (with hypothesis of homoscedasticity).
Quadratic discriminant analysis (QDA) would require
6(z) x² Ax + b=x
with 1 1= ------- - and b= E E My and = - 1 2, ui ,3 (g) 2 log E, is constant, and only affecting thresholding, but not the geometry of 8. Therefore we consider neglectible.
To form covariance matrices E,, Ey, and Σy. Shrinkage methods are necessary to approximate covariance matrices on small datasets or with large dimensions. With S the sample covariance, and F Tr(S) it estimates the optimal agw
Ledoit-Warl Wolf shrinkage the 50-neighborhoods of an
(E l1, 2)(2, 2) -S1 (tr((S- F). -i- (z, 2) 2) S tr((S F))2
21

---

Published as a conference paper at ICLR 2026
== (1 a.S a.w F
We then use Ex, Egg covariance matrix for neighborhoods of x; and zz..
G.2 CONCEPT SELECTION
After extraction of N candidate discriminant o,, the linear Deleuzian method restrained to concepts by performing feature weighted KMeans clustering.
Distance As Deleuzian concepts 6; linear, and only defined by discriminant vector b Rdd a trivial distance between two discriminant functions 6;, Oj is the Fuclidean distance between those vectors ||6 byll2. However, such metric cannot be computed on quadratic concepts having a more complex formulation. From discriminants o;, jj, we define the functional L metric
(6,, 0,) = = (6.(z) - 6)(z))²((())d
or in probalistic terms
(8,, 85) = Exrw (0,(() {z()))
Using the support measure w(x) (o, w(xx) represents prior belief that data should follow a zero-mean, isotropic gaussian distribution. Using w' (xx = (0 at), a a E IR+ would only cause uniform scaling of D2, without modifying the underlying geometry. Noting AA A Aj and Ab = b, bj. we have
(6, ô5) = Eeww 1
[C= AAr AArr Ab"x)²
Developping, we consider the odd moments to vanish (as is zero- obtain
zero-mean mean gaussian). Therefore
- - AAx) + B( x) D2 (6) -
6,)
Simplifying the linear term, we obtain
Bi(A x) x) Ab"B[
As 1 I), Ta. Thus
E[ c) = ADTTAAN = ||
Concerning the quadratic term, because NO0, 7) have
Ex AAr) (TT(AA3) Tr(AA))
Quadratic parameters A; and A, are differences of covariance matrices formed with Ledoit Wolff shrinking. Therefore, their diagonals are most likely similar, and dominated by constant isotropic offiet.. Then, consider 2A) = Tr(A: A2)² O, and we get
Ex AAx) = 2Tr(AA2) 2/AA
Therefore, our functional L2 distance stands as follows:
D2 6,) = 1 11 - A,1 + llb - b,1
22

---

Published as a conference paper at ICLR 2026
Centroids Recomputation Once have defined functional distance, the main crucial step of KMeans clustering is the iteratice centroids recomputation. Each 6, is assigned to its closest centroid C, then C' is recomputed in order to minimize within cluster distortion. We recompute C (with parameters A, b) using the Fréchet mean upon our functional 1 distance
argmin Σ
w. (6,
=
Σ Im:GTII A:ll. + 11b b,ll3)
C = argming
with ponderation weights (usually uniform, for unweighted mean) Using the A and b derivatives of C minimize distortion
a 0A
G114
A Arli²) = -0 = Σ w. - AA) = 0 = A Σ W
0b 2 bi1l9) = Σ wi(h bi) =0 Σ webb
Therefore, use À = Σ, A,A, and b Ex wib, as parameters of the centroid C
G.3 RESULTS AND DISCUSSION ABOUT QUADRATIC EXTENSION
The obtained method is an exact generalization of our linear Deleuzian method to quadratic func- tions. Table demonstrates that this extension reaches probe loss results better than SAE-based methods on CLIP- WikiArt, but does not outperform Linear Deleuzian concepts that is presented in the main paper. Such results may be due to the need estimate covariance matrices on very high dimensional data.
Table 8: Results of the Quadratic extension of Deleuzian concepts on CLIP-WikiArt
CLIP
Methods
WikiArt
Artist
Style
Genre
Van-SAE Tk-SAE
0.0137
0.0558
0.1531
0.0125
0.0558
0.1360
Linear Deleuzian (Main Method)
0.0119
0.0560
0.1230
Duudratie-Deleuziaan
Deleuzian (Extension)
0.0124
0.6160
0.1305
H APPENDIX: LLM USAGE
Beyond the usage LLM described the paper, that is part of the study, we used commercial services to polish the writting: find synonyms, rephrase sentences.
23


Reviews:
[{"review_id": "deDlhHrYOx", "reviewer": "Reviewer_L8Fx", "summary": "The current paper proposes to cluster *differences* in activations of a neural network, instead of raw activations itself, to interpret its behavior. Across a series of models and modalities, the paper shows improved probing performance, consistency of identified \"features\", and good steering ability.", "questions": "In table 3, I do not understand why TopK SAEs trained on activation differences do not work as well as steering by cluster centroids. Can the authors comment what's going on here?", "limitations": "", "rating": 2, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "I really like the perspective of engaging with the rich literature on concepts in philosophy and using that to motivate interpretability approaches, but I wish the paper went deeper on this narrative.", "weaknesses": "Reading through, I was first quite excited about the paper's idea and narrative (to look at activation differences), but, as it currently stands, I think the paper's operationalization of its core idea falls short of its promise. Main apprehensions are listed below.\n\n- Use of clustering to define \"concepts\": The paper currently takes differences of activations and simply run a clustering protocol (Kmeans) to extract \"concepts\" from it. While I'm not necessarily a big fan of SAEs, that approach assumes a specific model of representations (linear representation hypothesis) and hence tries to isolate \"features\" (directions in the learned dictionary) that interfere with each other as minimally as possible, but are also substantially larger than model dimensionality. This induces, or at least we hope that it induces, a monosemantic representation (latent code) such that we can now look at a direction in isolation to interpret what it represents. My main apprehension with the paper is that there is no such affordance achieved by the proposed method. The cluster centroids are in now way more meaningful than base activations are by default. If they are, then authors should have offered examples to demonstrate how. \n\n- Quantitative analysis: The proposed analysis does not meaningfully relate to interpretability in my opinion. Probing on activation differences (diffs) seems as likely to succeed as base activations. Even if this is not perfectly true, probing performance on base activations helps contextualize the improvement diffs help achieve. As it stands, Table 1 has numbers that are very close to each other and I'm unsure if there's any meaningful improvement in performance. \n\n- On consistency and clustering: If I understand correctly, Fel et al. [1] follow a similar protocol as this paper to define a dictionary learning protocol akin to SAEs (called Archetyal analysis) and show that improves consistency for theoretically meaningful reasons. At the very least a discussion is warranted, but more broadly, I think the paper already addresses the challenge of consistency that this paper claims to fix too.\n\n[1] https://arxiv.org/abs/2502.12892\n\nMinor comment \n\n- I think the idea of defining concepts at differences is both philosophically and operationally solid, but the intro and writing doesn't quite do justice to this. There was a good opportunity here to highlight how existing interpretability approaches assume definitions on concepts that fit into different perspective. For example, SAEs' notion of concepts is similar to prototype theory, while more dynamic approaches (like attention probes) are heading in the direction of conceptual role semantics. The Deleuzian lens offers a rich alternative to think about and I'd have appreciated this discussion, along with other frameworks' summary. It would have been even better if an argument were made in the text for why the Deleuzian perspective is better or worse than others.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "apJKOKnKw3", "reviewer": "Reviewer_929m", "summary": "The paper proposes an unsupervised method for extracting interpretable concepts from neural networks by clustering activation differences rather than reconstructing activations. The approach samples pairwise differences, applies K-Means clustering weighted by inverse skewness, and frames this as unsupervised discriminant analysis inspired by Deleuze's philosophical view of concepts as differences. Evaluation across five models and three modalities uses probe loss and MPPC metrics, with qualitative steering demonstrations.", "questions": "Cf Major point 1-8 and minor points 1-4", "limitations": "", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 1, "contribution": 2, "strengths": "The paper is well written, and I found at least 4 strong points in my opinion. \n\nS1. Simplicity. The method has an appealingly simple pipeline, is easy to understand and reproduce compared to SAE variants with multiple hyperparameters.\n\nS2. Broad empirical evaluation. The paper provides extensive experiments across three modalities (vision, text, audio), five models, and multiple datasets, with systematic probe loss evaluation across 874 attributes. This breadth is commendable.\n\nS3. Competitive probe loss results. Table 1 shows the method achieves lower probe loss than several SAE baselines on many tasks, which is interesting given the simplicity of the approach.\n\nS4. Consistency analysis. The MPPC evaluation (Table 2) provides useful insights into run-to-run stability, an important but often overlooked aspect of concept extraction methods.", "weaknesses": "Even if I like the work, I notice several flaws, some major (labeled M) and some minor (labeled m). Below I detail these concerns:\n\nM1. Lack of operational definition for \"concept.\" Section 2.1 lists desiderata but never provides a clear, falsifiable definition of what constitutes a concept beyond achieving low probe loss. The philosophical framing around Deleuze adds narrative color but doesn't translate into concrete, testable predictions that distinguish this approach from standard clustering or discriminant analysis. What specific properties should concepts have under the Deleuzian view that wouldn't be expected from other perspectives (e.g concept as direction with cosine distance is kind of what we use right now) ? Without this, the philosophical motivation feels more like post-hoc rhetorical framing than genuine theoretical grounding.\n\nM2. Insufficient justification for the core assumption. Why should neural network internals organize themselves at the level of activation difference clusters? The paper assumes that recurring pairwise differences correspond to meaningful internal structure, but provides no theoretical or empirical justification for this inductive bias. Section 2.3's connection to discriminant analysis relies on very strong assumptions (isotropy, diagonal covariances proportional to identity) that are rarely satisfied in practice. The paper should either: empirically verify these assumptions hold in the studied models (measure per-layer isotropy and correlate with method performance), provide a relaxed theoretical analysis for realistic conditions, or show the method recovers known phenomenology in synthetic or well-understood models. As it stands, it appears the method performs a standard mathematical operation (clustering differences) without explaining why this operation should reveal the computational structure of neural networks.\n\nM3. Effective rank as diversity measure is unconvincing. Section 3.4 uses effective rank to justify the skewness weighting, but effective rank only measures spread of singular values, not semantic diversity or redundancy. Two concept sets could have identical effective rank but very different semantic coverage, and the anisotropy of the space is not helping... The paper should validate diversity claims with complementary metrics such as pairwise cosine similarity distributions between concept vectors or perceptual similarity of the images cluster.  \n\nM4. \"Lossless steering\" claim requires more proofs. Section 2.4 calls steering \"lossless\" because operations are reversible in activation space. However, the downstream network is not always linear -- steering one direction may affect representation geometry in orthogonal directions through subsequent layer transformations. The paper should define \"lossless\" precisely (what exactly is preserved?), verify with control probes that steering concept i doesn't inadvertently shift unrelated concepts j, k,. The current qualitative examples (Figures 3-4) are suggestive but insufficient to support strong selectivity claims.\n\nM5. Missing highly relevant baselines. For vision tasks, I strongly recommend to cite [1] that is doing AA for style transfer as well as include BatchTopK [2] in the table, Spade wich is a distance based SAE [3] and Archetypal SAE [4] which talk about instability of SAE and is now quite used with Dinov2. I believe these methods are natural comparisons for the clustering-based approach and their absence weakens the competitive positioning.\n\n[1] Unsupervised Learning of Artistic Styles with Archetypal Style Analysis, Wynen & al\n\n[2] BatchTopK Sparse Autoencoders, Bussman & al\n\n[3] Projecting Assumptions: The Duality Between Sparse Autoencoders and Concept Geometry, Hindupur & al\n\n[4] Archetypal SAE: Adaptive and Stable Dictionary Learning for Concept Extraction in Large Vision Models, Fel & al\n\nM6. Discriminant analysis connection needs tightening. Section 2.3 asserts that optimal class separation reduces to activation differences under isotropy assumptions. However, these conditions $\\Sigma_A \\propto \\Sigma_B \\propto I$ are very restrictive and unlikely in transformer representations, the paper doesn't measure whether these conditions actually hold layer-by-layer, and no analysis is provided for the anisotropic case which is more realistic. Please either empirically verify the assumptions in your models or derive a more general result that applies under weaker conditions. Otherwise, the theoretical motivation appears fragile.\n\nM8. Evaluation relies entirely on labeled data. While the method is unsupervised, evaluation uses probe loss on labeled attributes. As the limitations section notes, this penalizes interpretable concepts that don't align with provided labels. The paper would benefit from qualitative evaluation protocols that don't depend on labels, analysis of discovered concepts that probe loss wouldn't capture, and discussion of what \"interpretability\" means independent of dataset-specific annotations. To come back to M1, I believe you may have an interesting implicit definition of what you call concept here in fact, can you write it down ?\n\nNow for the minor points, \n\nm1. Figure 2 is unreadable. The resolution and font sizes on page 3 make the pipeline diagram very difficult to parse. Please provide a higher-resolution version with larger text and clearer visual hierarchy.\n\nm2. Inverse skewness weighting details. The paper mentions using 1/skewness but doesn't specify how near-zero skewness is handled (epsilon clipping?), whether weights are normalized across dimensions or concepts, or whether absolute or signed skewness is used after direction flipping. \n\nm3. Writing polish. Several passages in Sections 2.3 and 5 could be tightened. Figure captions (especially Figure 4) should be more explicit about steering magnitudes and procedures. A final editing pass would improve readability.\n\nm4. Concept count parity. Table 1 should explicitly state the number of concepts used by each method if they differ, since probe loss can benefit from larger concept dictionaries.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "XpXXpVp4pW", "reviewer": "Reviewer_rHVe", "summary": "The paper proposes an alternative to sparse autoencoders (SAEs) for extracting interpretable “concepts” from neural network activations. The method clusters differences in activations between randomly sampled data pairs rather than the activations themselves, motivated by Deleuze’s philosophy of “difference and repetition.” To mitigate the dominance of highly skewed activation dimensions, the authors introduce inverse-skewness weighting in the clustering step. Conceptually, the approach can be viewed as an unsupervised form of discriminant analysis and enables lossless steering by directly modifying activations along learned concept directions.\n\nExperiments span five pretrained models across three modalities (vision, text, audio), comparing the proposed approach against various SAE variants (Vanilla, Gated, JumpReLU, Matryoshka, TopK) as well as ICA and supervised LDA. The primary evaluation metric is probe loss, which measures how well one-dimensional logistic probes can recover dataset attributes (e.g., style, sentiment, or class). Additional metrics include Maximum Pairwise Pearson Correlation (MPPC) for consistency and effective rank for diversity. Ablations test the role of (1) using differences, (2) clustering vs. SAE-based extraction, and (3) the inverse-skewness weighting. The authors report that their method achieves the best probe loss across most tasks, high consistency, and interpretable steering behavior.", "questions": "Please see the Weaknesses section.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "I find this work to provide a good alternative to SAEs towards interpretability. It constitutes a simple and nice approach grounded in discriminant analysis and clustering, while the inverse-skewness weighting is an interesting modification to improve concept diversity.\n\nThe empirical evaluation considers multiple modalities and architectures, while the quantitative evaluation avoids the commonly considered sparsity-reconstruction tradeoff and uses the probe loss and MPPC towards concept exploration.", "weaknesses": "The use of KMeans on activation differences is conceptually interesting, but it’s unclear how representative the randomly sampled pairs are. Could the sampling procedure bias the extracted concepts?\n\nIs the number of concepts fixed a priori? Is this the value that dictates the number of clusters for KMeans? How does the method fair when considering different values? \n\nThe inverse skewness weighting needs a further expansion. The inspiration is the Feature-Weigthed KMeans, but do any of its properties hold when the skewness is considered? Were there any empirical observations that skewed features dominate clustering? \n\nIn the ablation, the improvements from inverse-skewness weighting are modest in terms of the probe loss, while the effective rank exhibits substantial increase. On the other hand TopK SAE on the differences has approximately 3x the effective rank, again with minor differences in the probe loss. How can one interpret these results?\n\nThe focus on probe loss as the main interpretability metric is questionable — what are the “expected attributes,” and how well do these correspond to genuine conceptual disentanglement?\n\nThe paper claims that vanilla SAEs get “much lower concept quality and diversity,” yet many of the numerical differences (e.g., Table 1) seem small — are these differences statistically significant?\n\nIn the appendix A: Implementation details, it is mentioned that for training TopKSAEs, the chosen k was set to 32. Was that the optimal setting for the architecture, or did the authors consider only this value?\n\nWhy did the authors consider a subset of ImageNet instead of the full dataset?\n\nThe steering section feels superficial — qualitative examples are minimal, and it’s unclear whether the effects generalize.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "UUu2LOoD6r", "reviewer": "Reviewer_Jr9h", "summary": "The authors present a method for finding interpretable concepts in language models, and compare it to a tentpole of the field, Sparse Autoencoders (SAEs). The method takes a relatively small number of differences of activations of a model, weights their distances by their skew, and then performs k-means. The authors apply this method to interpret five models across the modalities of text, images, and audio. They assess the ability of their method to find interpretable concepts, especially by measuring whether their concepts can act as probes for a priori classes like labels in datasets.", "questions": "This paper departs heavily from the standards of the field along several axes, including choice of model, techniques, and evaluation metrics. Because of this, this review has lower confidence. The authors are encouraged to reply to this review and address the concerns raised, especially:\n\n1. Whether the SAEs have received sufficient training. This can be addressed by quantifying the number of datapoints the SAEs were trained on, quantifying the Fraction of Variance Explained by the SAEs, providing a graph of reconstruction loss over the training process, or by adding a pre-trained SAE to the set of benchmarks.\n\n2. Adding additional baselines to table 1, such as a probe trained on the entire activation space, a set of 6144 random vectors, and a pre-trained SAE (see the question below).\n\n3. Clarifying details of their method, in particular 1) how the clusters are initialized, and 2) how incorporating inverse skew to the distance expression in 152-154 changes the algorithm. \n\n-----\nOther questions for the authors:\n\n1. Do you have a name for your method?\n\n2. When performing your method, how did you initialize your k-means clusters?\n\n3. When running the algorithm for this experiment, what was the number of samples (N)? Is N the train size in Table 4?\n\n4. When applying your method, training the sparse autoencoders, and evaluating concept quality (Section 3.1), which activations did you use? For example, the IMDB dataset consists of a sequence of tokens, and BART and DeBERTa therefore produce one activation vector per token per layer. Which layer did you use, and did you use activations for every token or just some? \n\n6. Lines 152-154: Is the expression for d(d_i, C) here used when assigning d_i to a cluster? If so, isn't the scaling factor of 1/\\mu_3(d_i) irrelevant to identifying the closest cluster center? And if so, why does skewness weighting improve performance in Table 3?\n\n7. Lines 156-159: The paper says \"Both pair sampling and KMeans clustering run in linear time and memory with respect to dataset size N and activation dimension D, demonstrating scalability of our approach towards large datasets, or large models.\" Isn't this also true of sparse autoencoders? In particular, the memory requirements are linear in D and k (where k is the number of features, and k<N) and the run time is linear in D and N.\n\n8. Would it be possible to include additional baselines for your results in Table 1? In particular, it would be illustrative to see the probe loss for:\n\n- A probe trained on the entire activation space, acting as an upper bound of linear information which can be extracted from the model.\n\n- A set of 6144 random directions, acting as a baseline for how non-information-carrying directions behave.\n\n- A pre-trained SAE, such as Gemma Scope (Lieberum et al, 2024) or a Pythia SAE (https://huggingface.co/EleutherAI/sae-pythia-70m-32k).\n\n---- \nReferences:\n\n(Cunningham et al, 2023) Sparse Autoencoders Find Highly Interpretable Features in Language Models. https://arxiv.org/abs/2309.08600\n\n(Paulo and Belrose, 2025) Sparse Autoencoders Trained on the Same Data Learn Different Features. https://arxiv.org/pdf/2501.16615\n\n(Wang et al, 2025) Towards Universality: Studying Mechanistic Similarity Across Language Model Architectures. https://openreview.net/pdf?id=2J18i8T0oI\n\n(Lieberum et al, 2024) Gemma Scope: Open Sparse Autoencoders Everywhere All At Once on Gemma 2. https://arxiv.org/html/2408.05147v1\n\n(Karvonen et al, 2025) SAEBench: A Comprehensive Benchmark for Sparse Autoencoders in Language Model Interpretability. https://arxiv.org/pdf/2503.09532\n\n(Chaudhary and Geiger, 2024) Evaluating Open-Source Sparse Autoencoders on Disentangling Factual Knowledge in GPT-2 Small. https://arxiv.org/pdf/2409.04478\n\n(Marks, Paren, et al, 2024) Enhancing Neural Network Interpretability with Feature-Aligned Sparse Autoencoders. https://arxiv.org/pdf/2411.01220\n\n(Bricken et al, 2023) Towards Monosemanticity: Decomposing Language Models With Dictionary Learning. https://transformer-circuits.pub/2023/monosemantic-features\n\n(Marks, Rager, et al, 2025) Sparse Feature Circuits: Discovering and Editing Interpretable Causal Graphs in Language Models. https://arxiv.org/pdf/2411.01220\n\n(Durmus et al, 2024) Evaluating feature steering: A case study in mitigating social biases. https://www.anthropic.com/research/evaluating-feature-steering", "limitations": "", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "The proposed method is straightforward, and appears computationally light-weight. The paper is easy to follow.", "weaknesses": "The paper uses language models which are outside the tradition of the previous works in this field, which make it hard to assess the papers claims. In particular, this paper studies DeBERTa and the encoder of BART, which are relatively old encoder-based models trained on masked language modeling, whereas recent SAE work has focused on decoder-based models trained on next-token-prediction, such as the family of Pythia models (Cunningham et al, 2023; Paulo and Belrose, 2025; Wang et al, 2025), Gemma 2 (Lieberum et al, 2024; Karvonen et al, 2025), and GPT-2 Small (Chaudhary and Geiger, 2024; Marks, Paren, et al, 202).\n\nThe SAEs used as points of comparison for the author's methods were trained by the authors, and may have been severely undertrained. Per Appendix B, these SAEs appear to have trained them on datasets of size 18k-288k (or that many sequences, with an unknown number of tokens per sequence), with with a learning rate of 1e-5. In contrast, the similarly-sized autoencoders in (Cunningham et al, 2023) had \"a learning rate of 1e-3 and are trained on 5-50M activation vectors for 1-3 epochs\", and (Bricken et al, 2023) used \"8 billion datapoints\". If this paper's SAEs were undertrained, the paper's claim that \"the proposed method achieves concept quality surpassing prior unsupervised SAE variants\" would need to be caveated with \"in conditions of data scarcity\".\n\nThe claims in section 2.4 of \"Lossless Steering\" are not necessarily an advancement over prior work. The authors claim that steering with SAEs \"introduce[s] reconstruction error and information loss\", which they avoid by \"steering directly in the activations space\". But prior work such as (Marks, Rager, et al, 2025) accounts for this reconstruction error, and in effect steers directly in the activation space. While the authors do demonstrate some ability to steer in Section 3.3 and Figures 3 and 4, the paper does not measure whether their approach introduces changes in downstream behavior or if those changes are more significant than steering with SAE vectors (as in Durmus et al, 2024).\n\nSparse autoencoders may not be the correct baseline for this type of approach. Sparse autoencoders are usually trained on a huge corpus of activations with completely varied meanings. The author's method is tested on relatively small, relatively narrow datasets, that focus on a single topic (e.g. movie reviews in the IMDB dataset, or paintings in the WikiArt dataset). More appropriate baselines might include PCA or ICA (the later of which the authors include).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)