Archetypal SAE: Adaptive and Stable Dictionary Learning for Concept Extraction in Large Vision Models
Authors: Thomas Fel
Ekdeep Singh Lubana
Jacob S. Prince
Matthew Kowal
Victor Boutin
Isabel Papadimitriou
Binxu Wang
Martin Wattenberg
Demba E. Ba
Talia Konkle
Venue: ICML 2025 poster (2025)
Keywords: Explainability
Interpretability
Dictionary Learning
Computer Vision
Archetypal Analysis
Primary area: social_aspects->accountability_transparency_and_interpretability
TL;DR: Sparse Autoencoders (SAEs) for vision tasks are currently unstable. We introduce Archetypal SAEs (A-SAE and RA-SAE) that constrain dictionary elements within the data’s convex hull.

Abstract: Sparse Autoencoders (SAEs) have emerged as a powerful framework for machine learning interpretability, enabling the unsupervised decomposition of model representations into a dictionary of abstract, human-interpretable concepts. However, we reveal a fundamental limitation: SAEs exhibit severe instability, as identical models trained on similar datasets can produce sharply different dictionaries, undermining their reliability as an interpretability tool. To address this issue, we draw inspiration from the Archetypal Analysis framework introduced by Cutler & Breiman (1994) and present Archetypal SAEs (A-SAE), wherein dictionary atoms are constrained to the data’s convex hull. This geometric anchoring significantly enhances the stability and plausibility of inferred dictionaries, and their mildly relaxed variants RA-SAEs further match state-of-the-art reconstruction abilities. To rigorously assess dictionary quality learned by SAEs, we introduce two new benchmarks that test (i) plausibility, if dictionaries recover “true” classification directions and (ii) identifiability, if dictionaries disentangle synthetic concept mixtures. Across all evaluations, RA-SAEs consistently yield more structured representations while uncovering novel, semantically meaningful concepts in large-scale vision models.

Archetypal SAE: Adaptive and Stable Dictionary Learning for Concept Extraction in Large Vision Models
Thomas Fel
Ekdeep Singh Lubana 2
Jacob S. Prince Matthew Kowal 45 Victor Boutin
Isabel Papadimitrion! Binxu Wang' Martin Wattenberg" 78
Demba Ba" Talia Konkle"
Abstract
Sparse Autoencoders (SAEs) have emerged as a powerful framework for machine learning inter- pretability, enabling the unsupervised decompo- sition model representations into a dictionary of abstract, iamman-nteerpretbble concepts. How- ever, we reveal fundamental limitation: exist- ing SAEs exhibit severe instability, as identical models trained on similar datasets can produce sharply different dictionaries, undermining their reliability as an interpretability tool. To address this issue, we draw inspiration from the Archety- pal Analysis framework introduced by Cutler Breiman (1994) and present Archetypal SAEs (A- SAE), wherein dictionary atoms constrained to the convex hull of data. This geometric an- choring significantly enhances the stability of in- ferred dictionaries, and their mildly relaxed vari- ants RA-SAEs further match statc-of-the-ar art re- construction abilities. To rigorously assess dictio- nary quality learned by SAEs, we introduce two benchmarks that test (i) plausibility, if dictio- naries recover "true" classification directions and (ii) identifiability. if dictionaries disentangle syn- thetic concept mixtures. Across all evaluations, RA-SAEs consistently yield more structured rep- resentations while uncovering novel, semantically meaningful concepts in large-scale vision models.
A)
Regular SAE
Archetypal SAE A-SAE
Relaxed-Archetypal RA- SAE
Run Run
Cosine(1, n.] = 0.584
Cosine( 4.) 0.960
Figure A) Archetypal-SAE. Archetypal-SABs constrain dictio- nary atoms (decoder directions) the data's convex hull, improv- ing stability. A relaxed variant (RA-SAE) allows mild relaxation, matching standard SAFs in reconstruction while maintaining stahil- ity. Both integrate with any SAE variant (e.g.. TopK, JumpReLU), B) Instability Problem. Standard SAEs produce inconsistent dic- tionaries across runs, undermining interpretability. For example, in classical SAEs, second most important concept for "rabbit" in one run has counterpart in another run (cos 0.58). In contrast, Archetypal-SA -SFs maintain consistent concept correspon- dences across runs, ensuring stability.
"Equal contribution 'Kempner Institute, Harvard University CCBS-NTT Program in Physics of Intelligence, Harvard Univer- sity 'Dept. of Psychology. Harvard University *FAR.AI York University "CerCo, CNRS Harvard University "Google "Dept. of Psychology & Kempner Institute, Harvard University. Correspon- dence to: Thomas Fel <tfel@ wwr.:h.aaaaar.... harvard.edu>>.
Proceedings of the ndnd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

Archetypal SAE
1. Introduction
by introducing two novel variants: Archetypal-SAE (A- SAE) and its relaxed counterpart (RA-SAE). Building upon archetypal analysis (Cutler & Breiman, 1994), A-SAE con- strains each dictionary atom to reside within the convex hull of the training data, ensuring a more stable and consistent set of basis elements across different training runs by virtue of this geometrical constraint. RA-SAE further extend this framework by incorporating small relaxation term, allow- ing for slight deviations from the convex hull to enhance modeling flexibility while maintaining stability. Overall, our work makes the following contributions.
Artificial Neural Networks (ANNs) have revolutionized computer vision, setting new benchmarks across a wide range of tasks. Despite these successes, the "black-box" nature of ANNS poses significant challenges, particularly in domains requiring transparency, accountability, and adher- ence to strict ethical and regulatory standards (Tripicchio & D' Avella, 2020). Beyond compliance, there is a growing cu- riosity within the scientific community to better understand these models' internal mechanisms, both to satisfy funda- mental questions about their function, leverage insights for debugging (Adebayo et al., 2020) and improvement, and even explore parallels with neuroscience (Goodwin et al., 2022; Vilas et al., 2024). These motivations have spurred the rapid growth of explainable artificial intelligence (XAI), a field dedicated to enhancing the interpretability of ANNs, thereby bridging the gap between machine intelligence and human understanding (Doshi Velez& & Kim, 2017).
1. Instability in SAEs. We identify a critical limitation of current SAEs paradigms: two training runs on iden- tical data can yield concept dictionaries that are largely distinct, hence compromising their reliability as an interpretability protocol.
2. A-SAE: Archetypal anchoring to overcome insta- bility. To address the challenge above, we take in- spiration from Cutler & Breiman (1994)'s Archetypal analysis of dictionary learning and propose A-SAE, an SAE paradigm wherein the dictionary atoms (de- coder directions) are forced lie in the convex hull of sample representations. As we show, this geometrical anchoring yields substantial stability across training runs. Moreover, we show a mild relaxation of this pro- tocol, which we title RA-SAE, uncovers meaningful and interpretable concepts in large-scale vision models. 3. Rigorous evaluations with novel metrics and bench- marks. We introduce novel metrics to evaluate the quality of dictionaries learned using different SAE paradigms, while proposing two new benchmarks for testing SAEs' ability to recover classification direc- tions and to disentangle synthetic image mixtures, in- spired by identifiability theory (Locatello et al., 2019; 2020; Higgins et al., 2017). Our results provide sub- stantial evidence that A-SAEs find more structured and coherent concepts. Further, to enable reproduction, we open-source our extensive codebase for large-scale SAE training on modern vision models.
Among XAI approaches, concept-based methods (Kim et al., 2018) have emerged as a powerful framework for uncov- ering intelligible visual concepts embedded within the in- tricate activation patterns of ANNs (Ghorbani et al., 2019; Zhang et al., 2021; Fel et al., 2023b; Graziani et al., 2023; Vielhaben et al., 2023). These methods excel in making the internal representations of ANNs more comprehensible by associating them with human-interpretabbe concepts. Re- cently, concept extraction methods have been shown to be instances of dictionary learning (Fel et al., 2023a), where
the goal is to map the activation space of an ANN into a hiigher-imeensonal), sparse "concept space". The result- ing concept basis is often considered more interpretable. For such representations to be effective, they must be as sparse as possible while still enabling accurate reconstruc- tion of the original activations from the learned basis using a dictionary of atoms-alss also called prototypes. Historically, dictionary learning methods have included techniques such as Non-negative Matrix Factorization (NMF) (Lee & Se- ung, 2001; Kowal et al., 2024a; Jourdan et al., 2023) and K-Means (Gersho& & Gray, 1991; Ghorbani et al., 2019), while more recent approaches like Sparse Autoencoders (SAEs) (Cunningham et al., 2023; Bricken et al., 2023; Ra- jamanoharan et al., 2024; Gao et al., 2025; Thasarathan et al., 2025; Poché et al., 2025) have emerged as a powerful alternative. SAES achieve a good balance between sparsity and reconstruction quality, and their optimization frame- works scale well to large datasets. However, compared to traditional methods, SAES suffer from critical limitation: instability. As illustrated in Figure 1, training two identical SAES on the same data can yield significantly diffferent dic- tionaries (concept bases), undermining their reliability and interpretability.
2. Related Work
Sparse Coding & Dictionary Learning. Dictionary learning (Tošić & Frossard, 2011; Rubinstein al., 2010; Elad, 2010; Mairal et al., 2014: Dumitreseu & Irofti, 2018)
emerged as fundamental approach for uncovering latent factors of data-generating process in signal processing and machine learning, building upon early work in sparse coding (Olshausen & Field, 1996; 1997: Lee et al., 2006; Foldiak & Endres, 2008; Rentzeperis et al., 2023). The primary objective of these methods is to find a sparse rep- resentation of input data (Hurley & Rickard, 2009; Eamaz et al., 2022), such that each data sample can be accurately
In this work, we address the instability of current SAES
2

---

Archetypal SAE
reconstructed using linear combination of only small sub- set of dictionary atoms. The field gained momentum with compressed sensing theory (Donoho, 2006; Candès et al., 2006; Candès & Wakin, 2008; Lopes, 2013; Rencker et al., 2019), which established theoretical foundations for sparse signal recovery. Early dictionary learning methods evolved from vector quantization and K-means clustering (Gersho & Gray, 1991; Lloyd. 1982), leading to more sophisticated approaches like Non-negative Matrix Factorization (NMF)
pretability (Kim al., 2018) has gained traction. Its central objective is to pinpoint semantically meaningful directions- revealing not just where the model is looking, but also what patterns or concepts it employs- employs-and and to link these systemat- ically to latent activations (Bau et al., 2017; Ghorbani et al., 2019; Zhang et al., 2021; Fel et al., 2023b; Graziani et al., 2023; Vielhaben et al., 2023; Kowal et al., 2024a:b). In line with this perspective, Fel et al. (2023a) demonstrate that popular concept-extraction methods: ACE (Ghorbani et al., 2017), ICE (Zhang et al., 2021). CRAFT (Fel et al., 2023b) and more recently SAES (Cunningham et al., 2023;
and its variants (Lee & Seung, 1999; 2001; Gillis, 2020;
Ding et al., 2008; Kersting et al., 2010; Thurau et al., 2009;
Gillis & Kumar, 2015). Sparse PCA, (d' Aspremont et al., 2004; Zou et al., 2006) and K-SVD (Aharon et al., 2006a;
Bricken et al., 2023; Rajamanoharan et al., 2024; Gao et al., 2025; Surkov et al., 2024; Gorton, 2024; Bhalla et al., 2024a) all address essentially the same dictionary learning task, al- beit subject to distinct constraints (see Eq. 1).
Elad & Aharon, 2006). The field further expanded rapidly (Wright al., 2010; Chen et al., 2021: Tasissa et al., 2023) with online methods (Mairal et al., 2009; Kasiviswanathan et al., 2012; Lu et al., 2013) and structured sparsity (Jenatton et al., 2010; Bach et al., 2012; Sun et al., 2014). Theoretical guarantees for dictionary learning emerged (Aharon et al., 2006b; Spielman et al., 2012; Hillar & Sommer, 2015; Fu et al., 2018; Barbier& & Macris, 2022: Hu & Huang, 2023), alongside connections to deep learning (Baccouche et al., 2012; Tariyal et al., 2016; Papyan et al., 2017; Mahdizade- haghdam et al., 2019; Tamkin et al., 2023; Yu et al., 2023). Parallel developments in archetypal analysis from Cutler & Breiman (1994) provided complementary perspectives on dictionary learning by focusing on extreme points in a set of observations (Dubins, 1962).
Within this broader context, we note Sparse Autoencoders (SAFs) have emerged as a highy scalable special case of dictionary learning: unlike NMF, Sparse-PCA, or other op- timization problem, SAEs can be trained with the same algorithms and architectures used in modern deep learn- ing pipelines, making them especially well-suited for large- scale concept extraction. However, motivated by more am- bitious use-cases of interpretability. e.g., to develop trans- parency and accountability (Anwar et al., 2024), recent work has started to demonstrate limitations in existing SAE frameworks and propose improvements. Examples of such limitations include learning of overly specific or sensitive features (Bricken et al., 2023; Chanin et al., 2024), chal- lenges in compositionality (Wattenberg & Viegas, 2024; Clarke et al., 2024), and limited effects of latent interven- tions (Bhalla et al., 2024b; Menon et al., 2024). In this paper, we aim to bring to attention an underappreciated challenge in SAEs' training instability-wiae wherein mere reruns of SAE training yield inconsistent interpretations (Figure 1).
Vision Explainability. Early work in the field of Ex- plainable Al primarily revolved around attribution meth- ods, which highlight the input regions that most influence a model's prediction (Simonyan et al., 2013; Zeiler & Fergus, 2014; Bach et al., 2015; Springenberg et al., 2014; Smilkov et al., 2017: Sundararajan et al., 2017; Selvaraju et al., 2017; Fong & Vedaldi, 2017; Fel et al., 2021; Novello et al., 2022; Muzellec et al., 2024) in other words, where the network focuses its attention to produce its prediction. While valu- able, attribution methods exhibit two core limitations: (i) they provide limited information about the semantic orga- nization of learned representations (Hase& & Bansal, 2020;
3. (In)Stability of SAES
We first establish the challenge of instability in SAEs' train- ing. To this end, we start by defining notations and provid- ing background on the SAES analyzed in this work, then offering a formal measure of instability in SAE training.
Hsieh et al., 2021; Nguyen al., 2021; Colin et al., 2021; Kim et al., 2022: Sixt et al., 2020), and (ii) they can pro- duce incorrect explanations (Adebayo et al., 2018; Ghorbani et al., 2017; Slack et al., 2021; Sturmfels et al., 2020; Hsieh et al., 2021; Hase et al., 2021). In other words, just because the explanations make sense to humans, we cannot conclude that they accurately reflect what is actually happening within the model as shown by ongoing efforts to develop robust evaluation metrics for explainability (Petsiuk et al., 2018 88; Bhatt et al., 2020; Jacovi & Goldberg, 2020; Hedström et al., 2022; Fel et al., 2022; Hsieh et al., 2021; Boopathy et al., 2020; Lin et al., 2019; Idrissi et al., 2021).
Notation. Throughout this work, || lla and || || rep- resent the l and Frobenius norms, respectively. We de- fine [m] = (1...., n), and consider a general represen- tation learning framework where a deep learning model f X + A maps inputs from an input space X' to a rep- resentation space A C Rd The representation is captured set of n points arranged in matrix A E Rx with A, denoting its i-th row, where E [n] and A; E Rd. For any matrix X, X 2 0 denotes that all elements of X are nen-negative, and P(n) is the set of n X n signed permu- tation matrices. Given A E Raxdd we define cone (A) = {x x Au, 1 E R. v 2 0), and conv (A) = {
To overcome the constraints above, concept-based inter-
3

---

Archetypal SAE
Convnext
Dino
Siglip
Vit
0.025
0.035
0.055
0.05
0.050
0.020
0.030
0.04
0.045
Loss
0.015
0.025
0.040
0.03
0.035
0.020
0.010
0.030
0.015
0.025
99 95
95
90
80
90
80
95
90
80
Sparsity (%)
Sparsity (%)
Sparsity (%)
Sparsity (%)
Convex-NMF
Semi-NMF
Vanilla SAE
SAP
JumpReLU SAE
Figure 2. SAEs are promising direction for scalable concept extraction in vision. Comparison of reconstruction error (lu Loss) and sparsity across four large-scale vision models: ConvNext, DINO, SigLIP, and VIT. The figure comparcs the performance of various dictionary learning methods, including classical approaches (Convex-NMF, Semi-NMF) and modern Sparse Autoencoders (Vanilla SAE, Top-K SAE, JumpReLU SAE). Each SAE is trained up to 250 million tokens per epoch over 50 epochs, demonstrating the scalability of SAFs and their ability to achieve superior trade-offs between reconstruction fidelity and sparsity compared to traditional methods.
== Au, E RM v 0, LTo = 1} as the conical and convex hulls, respectively, generated by the columns of A.
and nonnegative matrix factorization (Fel et al., 2023b) to modern sparse autoencoding techniques (SABs) (Cunning- ham et al., 2023; Bricken et al., 2023).
Despite similar formulations to SAEs, solving K-Means, PCA, or NMF typically does not rely on backpropaga- tion and lacks inherent batch-learning capabilities. This make SAEs particularly appealing for large-scale applica- tions. Additionally, optimization problems solved in mul- tiple steps, such as NMF, Semi-NMF, or K-Means, can be interpreted having multi-laye nonlinear encoding (akin to a V with multiple layers)' We note that the decoding process in these methods remains linear.
Concept Extraction Dictionary Learning. Concept extraction can be naturally framed as dictionary learn- ing problem, wherein a set of n activations, represented by the matrix A E Raxd, is approximated using a struc- tured decomposition. The goal is to learn a Dictionary D also referred to as atoms (Serre, 2006), prototypes, or codebook (Tamkin al., 2023)-such that activations can be reconstructed as sparse linear combinations of these learned directions. The corresponding coefficients, known as Codes Z, capture the latent structure of the activations and enforce interpretability by promoting sparsity or non- negativity. This leads to the general optimization framework:
To compare these approaches, we generally evaluate the trade-off between two metrics: sparsity, measured as the to norm of Z, and fidelity, measured as the fg reconstruc- tion error. As starting point, propose to study this pareto frontier using state-of-the-art SAEs, including Jump- ReLU (Rajamanoharan et al., 2024), TopK (Gao et al., 2025), and a vanilla SAE with & regularization (Bricken et al., 2023), alongside classical sparse dictionary learning methods. Since PCA non-sparse, K-Means sparsity is fixed, and NMF is applicable only to non-negative activa- tions, we adopt modified versions of these methods that are broadly applicable for concept extraction (Kowal et al., 2024a; Parekh et al., 2024), such as Convex-NMF and Semi- NMF. For SAEs, we use the following standard formulation:
(Z", D*) = arg min ||A - I
2.D
Vi, 2 E (x)
(ACE K-Means), (ICE PCA), (CRAFT-NII NMFF),
D II
s.t.
Z > 0, D 0,
2 = 4 (A),, - K, (SAFs).
Here, e; denotes the i-th canonical basis vector, is the iden- tity matrix, and o is a neural network parameterized by 0 (typically a single feedforward layer with bias). Notably, in Sparse Autoencoder (SAE) literature, the dictionary D is often identified with the decoder weight matrix, denoted as Wdee This optimization framework unifies various classical methods for concept extraction, ranging from clustering- based approaches (Ghorbani et al., 2019), orthogonal factor- ization methods (Zhang et al., 2021; Graziani al., 2021),
V (A) = (AW, + b),
(1)
where A denotes a linear encoder layer and (-)) is an ele-
'A single optimization step in NMF is akin to a linear model followed by ReLU (projection onto the positive orthant).
4

---

Archetypal SAE
mentwise non-linearity that depends on the specific SAE ar- chitecture. For all SAEs, the resulting codes Z = (A) > 0 holds. We employ a Silverman kernel (Silverman, 1984) for instantiating Jump-ReLU Menp-ReLU SAEs. For vanilla SAEs, an 4 regularization on Z is applied until the desired sparsity is achieved. Top-K and Jump-ReLU SAEs directly control or optimize an lo constraint.
Archetypes, which are themselves defined as convex com- binations of data points. Concretely, the dictionary (the collection of Archetypes) can be constructed by multiply- ing the data by a row-stochastic matrix. Drawing on these ideas, we propose a solution to the problem of instability in SAES' training: the Archetypal SAE (A-SAE), which acts as a plug-and-play parameterization of the dictionary- i.c., the decoder matrix-and can be seamlessly integrated into any existing SAE, e.g., Topk (Gao et al., 2025) or Jump-ReLU (Rajamanoharan et al., 2024).
Results in Figure 2 illustrate that the SAE methods discussed above outperform optimization based dictionary learning methods in terms of reconstruction fidelity for fixed sparsity levels. While this positions SAEs as a compelling solu- tion for concept extraction, as we show in the following, a significant drawback of current methods lies in their insta- bility: minor changes to the dataset can lead to substantial variations in the learned dictionary.
Formulation. Let A E Raxd represent the data matrix
(with n data points in R²), and A = { E R" | 4; 2 0, 1 ' = }} denote the (n-1))- simplex in R". A matrix W E R is row-stochastic if each row W; belongs to n Define the set of row-stochastic matrices as
Measuring Instability. To formally define a notion of stability in SAES' training, we seek inspiration from the dic- tionary learning literature and prior works (Spielman al., 2012), yielding the following metric for two dictionaries D E IRaxd and D':
Dg & (W E R
W 2 0, W1, 1s).
(3)
An archetypal dictionary D then defined as follows.
D = WA, where W E 22
(4)
I TTDDDDID',
Stability( D D')= max NEP(m)
(2)
Hence, each row of D is a convex combination of the rows of A, ensuring that each archetype originates from the data.
where we assume that each atom lies on the unit ég-norm ball, i.e.. ||D.lla = for all i E [n]. This metric essentially measures the optimal average cosine similarity between the dictionaries after finding the best alignment via the Hungar- ian algorithm²
Geometric Interpretation. In standard SAEs (and most dictionary learning approaches), the dictionary D is free in the sense that each atom D; E Id can be placed anywhere in the ambient space. From a geometric perspective, this flexibility allows the reconstructions ZD to span regions that may exceed the convex hull of the data A. While this unconstrained setting enables greater expressivity, it comes with significant drawbacks. Specifically, small perturbations in the data or random initializations can lead to unstable so- lutions, resulting in dictionaries that differ drastically across training runs Moreover, if the dictionary atoms D; point in directions unrelated to the data, probing these directions may fail activate any meaningful mechanisms within the underlying model (Makelov et al., 2023). This highlights the importance of ensuring that the dictionary aligns with "real" directions inherent to the data.
We evaluate stability across various vision models by alter- ing only the random seed of the algorithm, while keeping the dataset unchanged across 4 runs. A similar trend is ob- served when the dataset is perturbed by as little as 5% or 10,, consistent with prior findings (Fel al., 2023a; Braun et al., 2024; Paulo & Belrose, 2025). Results in Figure? 3 show that while SAES outperform classical dictionary learn- ing methods in terms of reconstruction error (l2-loss), they exhibit lower stability, with cosine stability values around 0.5 for TopK SAE trained on D2 with over 250 million tokens. As a first approximation, this implies that re-running the same algorithm with a different seed can result in dic- tionaries where only half the concepts remain, while the other half are new orthogonal concepts. By contrast, the proposed Archetypal SAE, introduced below, achieves sta- bility comparable to classical dictionary learning methods without compromising reconstruction fidelity.
In contrast to above, the Archetypal dictionary imposes a crucial geometric restriction: every dictionary atom is constrained to lie within the convex hull of A, i.c., each dictionary atom D, is a convex combination of samples' representations. Thus, once multiplied by a nonnegative Z, the reconstructions ZD remain within the conic hull of A. This anchoring within the data manifold precludes the emergence of pathological or out-of- directions, yielding stability gains shown empirically in Figure 3 and formalized in Proposition G.2. Concretely, we always have:
4. Towards Archetypal SAES
In their seminal work, Cutler & Breiman (1994) proposed representing each data point as a convex combination of
'In practice, we use the linear_sumee sww..assi.gainnnnttttt function from scipy (Virtanen et al., 2020), which solves the linear sum assignment problem by finding the optimal matching in hipartite graph using the Jonker- Volgenant algorithm.
DE Conv(A,,, ZD E cone(A) A))
5

---

Archetypal SAE
Convnext
Dino
Siglip
Vit
1.0
1.0
1.0
1.0
0.8
0.8
0.8
0.8
Stability
0.6
0.6
0.6
0.6
0.4
0.4
0.4
0.4
0.2
0.2
0.01
0.02
0.02
0.04
0.02
0.03
0.03
0.04
0.05
L2 Loss
L2 Loss
L2 Loss
L2 Loss
Convex-NMF
Semi-NMF
Vanilla SAE
Top-K SAE
JumpReLU SAE
Archetypal SAE
Figure 3. Stability Reconstruccion tradeoff (optimal: top-left). We implement dictionary learning methods on 4 models at 5 levels of sparsity each, as well as our A-SAE method. We show that SAFs exhibit instability (minor perturbations in the dataset can lead to significant changes in learned dictionary). while traditional dictionary learning methods are more stable worse reconstructing the data. Archetypal -SEEs (ours) help mitigate this issue. We mcasure stability based on Equation (2): the optimal average cosine similarity between the dictionaries across 4 runs after finding the best alignment via the Hungarian algorithm. Archetypal-SAl improve stability without compromising reconstruction fidelity, performing better on the Haailitt-reconstruccci tradeoff' than existing methods.
Moreover, one can restrict D be formed from subset C' S R= xd (rather than all of A) without losing expressiv- ity, provided C' contains the extreme points of A (Dubins, 1962). Indeed, in that case cone(C) = cone(A) (A), ensuring the same representational power (see Proposition G.I).
0.9
0.8
0.8
82 Score
0.7-
0.7
0.6
Stabiitt
0.6
0.5
We must now ask the core question that makes SAES excit- ing: does SAE with Archetypal dictionary, defined above, scale? Indeed, optimizing the matrix W of size k x n, where n is the number of points and & is the num- ber of concepts, is often infeasible in practice (e.g., when the number of tokens n > 109). This limitation speaks to the importance of the matrix C, which can either be a subset of or elements within conv(A) (A), such as mixtures of points. Specifically, accessing extreme points is neces- sary to achieve perfect reconstruction (Simon, 2011] 1), but this is also intractable in practice for such high dimensions. However, we propose to address this with relaxation.
0.4
0.4
90.0
Sparsity (%)
Sparsity
8+00
6-01
6-0.
Figure 4. Impact of the Relaxation Parameter (6). Enumcrating extreme points is infeasible in practice; therefore, we introduce a small relaxation parameter (8) that allows exploration beyond the convex hull of C. The magnitude of this relaxation enables the Archetypal SAE achieve performance comparable to the uncon- strained TopK SAE denoted as Baseline (left) while maintaining excellent stability (right).
Scaling Archetypal-SAE. To maintain the desirable prop- erties of A-SAE while addressing scalability, we propose using a reduced subset of points C, with n'' < n. chosen as centroids of A obtained via K-Means. We fix C and train only W. We apply a ReLU activation and normalize each row of W at each step to ensure that W Pk... Experi- ments indicate that K-Means forms the most reliable method for distilling A into C' (see Figure 11), compared to alter- natives such as isolation forests (Liu et al., 2008), convex hull computation in reduced dimensions, or outlier detec- tion methods (Scholkopf et al., 1999; Breunig et al., 2000) (details in Appendix E). Hereafter, we use A-SAE to refer to this implementation where we directly optimize W to
find the best convex combination of points that reconstruct data. To enable controlled degree of flexibility beyond cony (C), we introduce a mild relaxation term A kxd matrix of the same dimensions as the dictionary, with small norm constraint || [B < 8. This leads to relaxed formulation, which we call the Relaxed Archetypal SAE (RA-SAE). Unlike standard A-SAES, RA-SAE learns both the convex combination weights W and the relaxation term A, while ensuring that deviations from the convex hull re- main minimal. Formally, we define the dictionary as:
B= + s.t. W E Se, and || S 8.
6

---

Archetypal SAE
Here, only W and A are trainable parameters, as detailed in the pseudocode (Figure 5). This implementation ensures that W remains row-stochastic and that the deviation term A stays within the prescribed norm constraint. As shown in Figure 4, RA-SAE achieves reconstruction performance on par with conventional Top-K SAEs while maintaining the stability benefits of the archetypal constraint.
overcomplete dictionaries with size k 5x the feature dimension (e.g., 768 x 5 for DINOv2 and 2048 x 5 for ConvNeXt). Models were trained on the ImageNet dataset (~ 1 1.28M images), resulting in over 60M tokens per epoch for ConvNeXt (7 x 7 tokens/image) and over 250M tokens per epoch for DINOv2 (14 X 14 patches/image) across 50 epochs. Semi-NMF and Convex-NMIF were trained using gradient descent with accumulation and & regularization to control sparsity, while the RA-SAE was applied atop a TopK SAE to maintain consistent sparsity. To compute C, we applied K-Means clustering to the entire dataset, reduc- ing it to 32,000 centroids, which achieved reconstruction eтror comparable to the unconstrained SAE. The data ma- trix A was element-wise standardized. All the SAEs were trained using the Overcomplet library?.
class ArchetypalDDictionarymm. Module):
def ____i_iit_ss_ (self, C, k, 8=1.0):
super() i)__init__())
n', d = Chapee
C)
self. w an.Parameter (toneeettttohh.otee (torch.eye(() n'))
self.A A nn. Parameete (torch. zeros (k, d))
self. = 5
5.1. Dictionary Learning Metries
def (sel, Z):
with torch.no_gradi.
To improve SAEs, it is essential to deepen our understanding of the solutions they yield. We thus evaluate both standard and novel sets of metrics that evaluate dictionary learning methods across four key dimensions: (i) sparse reconstruc- tion, (ii) consistency, (iii) structure in the dictionary (D), and (iv) structure in the codes (Z), all reported in Tab. I (see App. D for formal definitions). Overall, we find Archety- pal SAES achieve a strong balance between reconstruction performance, consistency, and identification of structure.
W = torrhh
W /= (.sum(dim=-1, keepdim=True)
A *= torch. torch.clamp(
self.6 A. normmddii keepdim=True), max=1
)
D W e self.c + A return Z D
Figure 5. Pseudocode for Relaxed Archetypal SAE (RA-SAE). This implementation ensures that dictionary atoms remain elose to convex hull of the data conv(C) while allowing controlled deviations for better flexibility.
i) Sparse Reconstruction. Prior work (Bricken et al., 2023) commonly assesses the quality of the learned dictionary to reconstruct the original data under sparsity constraints via metrics such as Ii², sparsity (6o norm), and the effec- tive usage of dictionary atoms (e.g., dead codes). We find that Archetypal SAEs perform on-par with existing SAEs and outperform NMF methods in reconstruction, achieving higher I22 scores for comparable sparsity levels.
5. Experiments
This section is organized into five parts. We begin by de- scribing the experimental setup in detail. Then, we introduce a novel set of theoretical metrics designed to better under- dictionary
ii) Consistency, Excelling in reconstruction does not guar- antee that the learned solution aligns with the underlying data distribution or is stable across training runs. To this end, we measure stability, which assesses the consistency of learned dictionaries across different initializations or per- turbations in the data, and the OOD score, which quantifies how close the dictionary atoms are real data points, i.e., whether learned concepts remain grounded and interpretable. Our findings indicate that SAES perform poorly in consis- tency, showing both low stability (as evidenced in Figure 3) and suboptimal OOD scores. In contrast, Archetypal SAES significantly enhance stability and OOD score without sac- rificing reconstruction performance. In Proposition G.4, we provide theoretical arguments showing that, under mild assumptions, the OOD score of Archetypal SAEs is inher- ently lower-bounded, ensuring that learned dictionary atoms remain well-anchored within the data.
stand the differences between optimization-bast learning methods, standard SAEs, and the proposed Archety- pal SARS. Following this, we present new benchmark inspired by identifiability theory, which serves to evaluate the plausibility and uniqueness of the learned representa- tions. We then propose a second, more practical benchmark to assess whether the models can retrieve "true" directions or concepts effectively utilized by the models. Finally, we conclude with qualitative examples showcasing the concepts discovered by SAEs, particularly when applied to the DI- NOv2 model, providing insight into the interpretability and utility of the learned representations.
Setup. We evaluate five models: DINOv2 (Darcet et al., 2023; Oquab et al., 2023), SigLip (Zhai et al., 2023), ViT (Dosovitskiy et al., 2020), ConvNeXt (Liu et al., 2022), and ResNet50 (He et al., 2016), sourced from the timm li- brary (Wightman et al., 2019). Unless specified, we trained
"https://gii thub. wwwp/K/ewmpeeenniiiittteeerrrrrn http:///tttttttooooooooooooooonn
7

---

Archetypal SAE
Metric
Van. SAE
TopK SAE
Jump SAE
SNMF
CNMF
RA-SAE
1²2 (1)
83.94
89.52
89.92
67.43
55.48
89.34
Dead Codes (4)
0.00
0.00
0.00
0.064
0.031
0.02
Stability (1)
0.710
0.542
0.539
0.925
0.933
0.927
Max Cosine (1)
0.997
0,993
0,994
0.999
0.999
0.999
OOD Score (4)
0,451
0,551
0.551
0.430
0.087
0.060
Stable Rank (4)
86.8
141.6
128.0
5,38
6.65
5.89
Eff. Rank (L)
363
372
371
186
289
310
Coherence (1)
0.838
0.728
0.560
0.999
0.999
0.973
Connectivity (1)
0.000
0.002
0.003
0.243
0.138
0.159
Neg. Inter. ())
39.99
135.7
243
0.005
0.002
0.012
Table 1. Quantitative comparison of the dictionary learning methods on DINOv2, using a 90% sparse, overcomplete dictionary with 2000 concepts. SAE methods achieve the highest reconstruction performance. C-NMF, S-NMF, and Archetypal methods excel in consistency, ensuring stability across runs and that found concepts are close to real data (OOD). Additionally, these methods demonstrate superior dictionary structure (Stable and Eff. Rank) and codes structure (Connectivity and Neg Inter.), indicating patterns in the inferred concepts and structured sparsity.
Diet. size (k)
512
1k
2k
4k
8k
16k
32k
iii) Structure in the Dictionary (D). This dimension ex- amines whether the learned dictionary exhibits meaningful patterns. A well-structured dictionary may reveal meta- concepts or decomposable higher-level abstractions. Met- rics such as stable rank, effective rank, and coherence pro- vide insights into the compactness and interpretability of the dictionary. In particular, among solutions with comparable reconstruction performance, a more structured dictionary preferable due its potential for higher interpretability and organization. Across runs, methods like CNMF, SNMF, and Archetypal dictionary learning consistently yield dic- tionaries with better structure, reflecting their capacity for capturing higher-level patterns within the data. We also provide theoretical arguments in Proposition G.3, demon- strating that the rank of Archetypal dictionaries is inherently bounded by the rank of the data matrix.
ConeNeXt
Baseline (TopK
0.1681
0.1686
0.1668
0.1671
0.1684
0.1692
0.1684
A-SAE (6 =
0.2172
0.3046
0.3597
0,3887
0.3957
0.3984
0.3999
RA-SAR (5 0.01)
0,1973
0.2887
0.3581
0.3900
0.4007
0.4038
0.4045
RA-SAE (5 0.1)
0.1270
0.1596
0.2106
0.2681
0.3280
0,3674
0,3845
RA-SAE (8 1.00
0.1172
0.1475
0.2121
0.3116
0.4342
0.5203
0.5581
ResNet
Basding (TopK SAB)
0.2295
0.2484
0.3055
0.3203
0,3301
0.3014
0.3125
A-SAE =
0,5920
0.5985
0.5992
0.6013
0.6022
0.6105
0.6133
RA-SAE 0.01)
0,5905
0.5985
0.5991
0.6013
0.6039
0.6106
0.6136
RA-SAE (5 0.1)
0,5737
0.5932
0.6002
0.6039
0.6846
0.6067
0.6083
RA-SAE (6 1.00)
0.6151
0.6173
0.6189
0.6200
0.6208
0.6213
VI
Baseline (TopK SAE)
0.1984
0.2265
0.2595
0.2807
0.2939
A-SAF - 0)
0,2079
0.3459
0.282)
0.3096
0.3361
0.3581
0.3721
RA-SAE (8 0.01)
0.2490
0.2861
0.3132
0.3200
0.3642
0.3821
RA-SAE (5 0.1)
0,2278
0.2747
0.3153
0.3410
0.3196
0,3912
0.4103
RA-SAE 1.0)
0.2123
0.2936
0.3624
0,4277
0.4786
0,5014
Table 2. Plausibility Benchmark Results. The Plausibility Score measures the alignment between the learned dictionary concepts and the classification head's directions. RA-SAE achieves a sig- nificantly higher score compared to a Topk SAE (baseline). Best scores wrapped in green, and worst scores are wrapped in red.
ir) Structure in the Codes (Z). The structure in the en- coding space is equally important, it determines how concepts are combined to reconstruct the data. Connectivity, measured as the lo norm of Z Zz,, reflects the combinato- rial diversity of the concepts. High connectivity enables complex reconstructions, while low connectivity highlights structural sparsity and simpler patterns. Additionally, nega- tive interference (Neg. Inter.) quantifies the simultaneous activation of conflicting concepts, which can cancel each other out. Archetypal SAEs and optimization-base based dictio- nary learning approaches reliably produce more structured codes with reduced interference, enhancing the coherence of their representations.
atom D, and average the alignment score: Plausibility = Ei (v., Dj). A score of indicates perfect align- ment, while a score of implies that all concepts lie in the classifier's null space, making probing ineffective. This metric could help us detect potential hallucinations if con- cepts diverge too much from true classification directions, and is broadly similar to Karvonen et al. (2024); Mayne et al. (2024)'s evaluation of whether SAES infer known linear features in toy settings. Results are reported Ta- ble 2. We find that classical SAEs, even with extremely large dictionaries, achieve limited alignment with the clas- sification directions (inline with prior work). In contrast, RA-SAE significantly enhances this alignment, recovering a substantial portion of the true classification directions. This demonstrates RA-SAE's efficacy in producing semantically meaningful dictionaries.
1
è
5.2. Plausibility Benchmark
We evaluate SABS' ability recover true classification di- rections by assessing whether the learned dictionary D aligns with the classifier's 's final layer weights (0I,..., ve where c is the number of classes. Specifically, for each class vector Uú we compute the most aligned dictionary
8

---

Archetypal SAE
X
Visual encoder
X
A
Z
6.0
Images (X] generated from ground truth generating process involving 9 objects forminga a hidden dictionary. The model never directly observes this dictionary only resulting images.
The visual encoder extracts meaningful
SAEs processes the activations (A) produce codes 2, containing 9 distinct concepts for each images. The code identifies true structure of the generative process, disentangling interpretable components.
integnal internal representations (A) from input images.
Figure 6. Soft Identifiability benchmark. This example uses the "gems" dataset, part of the 12 identifiability benchmarks we introduce. The goal to evaluate whether SAEs (or any dictionary learning method under study) can disentangle and recover each object from the hidden ground truth generative process. By analyzing the model's s ability to assign distinct codes to the underlying concepts, we test its capacity to reconstruct and interpret the true structure the data.
5.3. Soft Identifiability Benchmark
R represents the & concept values and y R denotes the class labels. We then define the accuracy for the class as: Accuracy, max ((z,))((z > A) = 94).
Recent work on disentangled representation learning often evaluates whether an autoencoder trained to reconstruct ples from data-generating process learns to represent its un- derlying concepts (Locatello et al., 2019; 2020; Von Kugel- gen et al., 2021; Gresele et al., 2020; Khemakhem et al., 2020; Schott et al., 2021; Zimmermann et al., 2021; Menon et al., 2024). Identifiability theorems on the topic (Locatello al., 2019; 2020) have however shown that unless the au- toencoding architecture possesses the right inductive biases that match the generative process, there is guarantee concepts underlying the data will map onto the autoen- coder's latents. Since these results do not make assumptions about the data modality, they remain valid for a standard SAE training setup, c.g., similar to our experiments above. Then, an intriguing experiment involves evaluating whether when trained on representations of samples from toy data- generating process with predefined concepts, A-SAE, or any other SAE architecture, develops latents capturing said concepts if it does, then that is strongly suggestive of the SAE possessing the right inductive biases, i.e., captures the mechanism via which model encodes concepts in its representations.
Impact Statement
We identify a fundamental limitation of Sparse Autoen- coders (SAEs): their instability undermines reliability as interpretability tools. To address this, we introduce Archety- pal SAEs (A-SAE), which constrain dictionary atoms to the data's convex hull, and a relaxed variant (RA-SAE) that preserves expressivity while enhancing stability. Our evalu- ations, grounded in identifiability theory, demonstrate that these constraints yield more consistent, semantically mean- ingful, and better-structured dictionaries without sacrificing reconstruction quality.
These contributions not only improve the scientific rigor and reproducibility of SAE-bases interpretability but also enable stable concept discovery at scale in large vision mod- els. Moreover, the method generalizes across modalities, offering robust foundation for interpretable representation learning in language models and beyond.
A Aknooleedgments
Motivated by the above, propose Soft Identifiability Benchmark. Specifically, we construct twelve synthetic datasets, each comprising images formed by collaging four distinct objects (c.g.. different types of gems) sampled from a pre-defined set. Each dataset is processed through a pre- trained vision model to obtain pooled activations. Ideally, when trained on these activations, the SAE is able to recover the original objects distinct concepts within its dictionary. We then assess performance by checking whether each ob- ject class has a corresponding concept that activates above a threshold À when an object y, appears in the image. For- mally, for each image, we feed it into a model and then into the trained SAE to get a concept-labe pair (z, y). where
This work has been made possible in part by a gift from the Chan Zuckerberg Initiative Foundation to establish the Kempner Institute for the Study of Natural and Artificial In- telligence at Harvard University. MW acknowledges support from Superalignment Fast Grant from OpenAl. Effective Ventures Foundation, Effektiv Spenden Schweiz, and the Open Philanthropy Project.
9

---

Archetypal SAE
References
Bauckhage, C. and Manshaei, K. Kernel archetypal anal- ysis for clustering web search frequency time series. In International Conference Pattern Recognition (ICPR), 2014.
Abrol, V. and Sharma, P. A geometric approach to archety- pal analysis via sparse projections. In International Con- ference on Machine Learning (ICML), 2020.
Bauckhage, C., Kersting, K., Hoppe, F., and Thurau, C. Archetypal analysis as an autoencoder. In Neural Com- putation Workshop.
Adebayo, J., Gilmer, J., Muelly. M., Goodfellow, I., Hardt, M., and Kim, B. Sanity checks for saliency maps. Ad- vances in Neural Information Processing Systems (NIPS), 2018.
Bhalla, U., Oesterling, A., Srinivas, S., Calmon, F. F.P., and Lakkaraju, H. Interpreting clip with sparse linear concept embeddings (splice). ArXiv e-print, 2024a.
Adebayo. J.. Muelly, M., Liccardi, I., and Kim, B. Debug- ging tests for model explanations. Advances in Neural Information Processing Systems (NeurIPS), 2020.
Bhalla, U., Srinivas, S., Ghandeharioun, A., and Lakkaraju, H. Towards unifying interpretability and control: Evalua- tion via intervention. ArXiv e-print, 2024b.
Aharon, M.. Elad, M., and Bruckstein, A. K-svd: An algo- rithm for designing overcomplete dictionaries for sparse representation. IEEE Transactions on Signal Processing, 2006a.
Bhatt, U., Weller, A., and Moura, J. M. F, Evaluating and aggregating feature-based model explanations. Proceed- ings of the International Joint Conference on Artificial Intelligence (IJCAI), 2020.
Aharon, M., Elad, M., and Bruckstein, A. M. On the unique- ness of overcomplete dictionaries, and a practical way to retrieve them. Linear Algebra and its Applications, 2006b.
Boopathy, A., Liu, S., Zhang, G., Liu, C., Chen, P.-Y.. Chang, S., and Daniel, L. Proper network interpretability helps adversarial robustness in classification. Proceedings of the International Conference on Machine Learning (ICML), 2020.
Anwar, U., Saparov, A., Rando, J., Paleka, D., Turpin, M., Hase, P., Lubana, E. S., Jenner, E., Casper, S., Sourbut, O., et al. Foundational challenges in assuring alignment and safety of large language models. ArXiv e-print, 2024.
Boyd, and Vandenberghe, L.. Convex optimization. 2004.
Braun, D., Taylor, J., Goldowsky N., and Sharkey, L. Identifying functionally important features with end-to- end sparse dictionary learning. ArXiv e-print, 2024.
Arora, S., Ge, R., and Moitra, A. Learning topic models going beyond svd. In IEEE Symposium on Foundations of Computer Science (FOCS), 2013.
Breunig, M. M., Kriegel, H.-P., Ng, R. T., and Sander, J. Lof:
Baccouche, M., Mamalet, F., Wolf, C., Garcia, C., and Baskurt, A. Spatio-temporal convolutional sparse auto- encoder for sequence classificatin.. Proceedings of the British Machine Vision Conference (BMVC), 2012.
identifying density- local outliers. ACM SIGMOD International Conference on Management of Data. 2000.
Bricken, T., Templeton, A., Batson, J., Chen, B., Jermyn, A., Conerly, T., Turner, N., Anil, C., Denison, C., Askell, A., Lasenby, R., Wu, Y., Kravee, S., Schiefer, N., Maxwell, T., Joseph, N., Hatfield- Dodds, Z., Tamkin, A., Nguyen, K., McLean, B., Burke, J. E., Hume, T., Carter, S., Henighan, T., and Olah, C. Towards monosemanticity: Decomposing language models with dictionary learning. Transformer Circuits Thread, 2023.
Bach, F., Jenatton, R., Mairal, J., and Obozinski, G. Struc- tured sparsity through convex optimization. Statistical Science, 2012.
Bach, S., Binder, A., Montavon, G., Klauschen, F., Müller, K.-R., and Samek, W. On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation. Public Library of Science (PloS One), 2015.
Candès, E. J. and Wakin, M. B. An introduction to com- pressive sampling. IEEE Signal Processing Magazine, 2008
Barbier, J. and Macris, N. Statistical limits of dictionary learning: random matrix theory and the spectral replica method. Physical Review E, 2022.
Candès, E. J., Romberg, J., and Tao, T. Robust uncertainty principles: Exact signal reconstruction from highly in- complete frequency information. IEEE Transactions on Information Theory, 2006.
Bau, D., Zhou, B., Khosla, A., Oliva, A., and Torralba, A. Network dissection: Quantifying interpretability of deep visual representations. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017.
Canhasi, E. and Kononenko, I. Weighted hierarchical archetypal analysis for multi-document summarization. Computer Speech & Language, 2016.
10

---

Archetypal SAE
Chan, B., Mitchell, D., and Cram, L. Archetypal analysis of galaxy spectra. Monthly Notices of the Royal Astronomi- cal Society (MNRAS), 2003.
Doshi-Velez, F. and Kim, B. Towards rigorous science of interpretable machine learning. ArXiv e-print, 2017.
Dosovitskiy, A., Beyer, L.,, Kolesnikov, A., Weissenbomn D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G. Gelly, S., et al. An image is worth 16x16 words: Transformers for image recognition at seale. Proceedings of the International Conference on Learning Representations (ICLR), 2020.
Chanin, D., Wilken-Smith, J., Dulka, T., Bhatnagar, H.,, and Bloom, J. for absorption: Studying feature splitting and absorption in sparse autoencoders. ArXiv e-print, 2024.
Chen, J., Mao, H., Wang, Z., and Zhang, X. Low-rank repre- sentation with adaptive dictionary learning for subspace clustering. Knowledge-
Dubins, L. E. On extreme points of convex sets. Journal of Mathematical Analysis and Applications, 1962.
Systems, 2021.
Dumitrescu, B. and Irofti, P. Dictionary learning algorithms and applications. 2018
Chen, Y. Mairal, J., and Harchaoui, Z. Fast and robust archetypal analysis for representation learning. Pruceed- ings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2014.
Eamaz, A., Yeganegi, F., and Soltanalian, M. On the build- ing blocks of sparsity measures. IEEE Signal Processing Letters, 2022.
Clarke, M., Bhatnagar, H., and Bloom, J. Compositionality and ambiguity: Latent co-occurrence and interpretable subspaces. LesxWrong, 2024.
Elad, M. Sparse and redundant representations: from theory to upplications in signal and image processing. 2010.
Colin, J., Fel, T., Cadène, R., and Serre, T. What i cannot predict, i do not understand: A human-centered evalua- tion framework for explainability methods. Advances in Neural Information Processing Systems (NeurIPS), 2021.
Elad, M. and Aharon, M. Image denoising via sparse and redundant representations over learned dictionaries. IEEE Transactions on Image Processing, 2006.
Epifanio, I., Ibañez, V., and Simó, A. Archetypal analysis with missing data: see all samples by looking at a few based on extreme profiles. The American Statistician, 2019.
Cunningham, H., Ewart, A., Riggs, L., Huben, R., and Sharkey, L.. Sparse autoencoders find highly interpretable features in language models. ArXiv e-print, 2023.
Eugster, M. and Leisch, F. Weighted and robust archety- pal analysis. Computational Statistics & Data Analysis, 2011.
Cutler, A. and Breiman, L. Archetypal analysis. Technomet- rics, 1994.
Cutler, A. and Stone, E. Moving archetypes. Physica D: Nonlinear Phenomena, 1997.
Fel, T., Cadene, R., Chalvidal, M., Cord, M., Vigouroux, D., and Serre, T. Look at the variance! efficient black- box explanations with sobol-based sensitivity analysis. Advances in Neural Information Processing Systems (NeurIPS), 2021.
Darcet, T., Oquab, M., Mairal, J., and Bojanowski, P. Vision transformers need registers. ArXiv e-print, 2023.
d' Aspremont, A., Ghaoui, L., Jordan, M., and Lanckriet, G. A direct formulation for sparse pca using semidefinite pro- gramming. Advances in Neural Information Processing Systems (NeurIPS), 2004.
Fel, T., Hervier, L., Vigouroux, D., Poche, A., Plakoo, J., Cadene, R., Chalvidal, M., Colin, J., Boissin, T., Bethune, L., Picard, A., Nicodeme, C., Gardes, L., Flandin, G., and Serre, T. Xplique: A deep learning explainability toolbox. Workshop on Explainable Artificial Intelligence for Computer Vision (CVPR), 2022.
Dawson, J. A. and Kendziorski, C. Survival-supervised latent dirichlet allocation models for genomic analysis of time-to-event outcomes. arXiv, 2012.
Fel, T., Boutin, V., Moayeri, M., Cadene, R., Bethune, L., Chalvidal, M., and Serre, T. holistic approach to unify- ing automatic concept extraction and concept importance estimation. Advances in Neural Information Processing Systems (NeurIPS), 2023a.
Ding, C. H., Li, T., and Jordan, M. I. Convex and semi- nonnegative matrix factorizations. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2008.
Fel, T., Picard, A., Bethune, L., Boissin, T., Vigouroux, D.. Colin, J., Cadène, R., and Serre, T. Craft: Concept recursive activation factorization for explainability. Pro- ceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2023b.
Ding, W., Ishwar, P., and Saligrama, V. A provably efficient algorithm for separable topic discovery. Arxiv, 2016.
Donoho, D. L. Compressed sensing. IEEE Transactions on Information Theory, 2006.

---

Archetypal SAE
Foldiak, P. and Endres, D. M. Sparse coding. ArXiv e-print, 2008.
Graziani, M., Nguyen, A.-p., O'Mahony, L., Müller, H., and Andrearezyk, V. Concept discovery and dataset explo- ration with singular value decomposition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2023.
Fong, R. C. and Vedaldi, A. Interpretable explanations of black boxes by meaningful perturbation. Proceedings of the IEEE International Conference on Computer Vision
(ICCV), 2017.
Gresele, L.., Rubenstein, P. K., Mehrjou, A., Locatello, F, and Scholkopf, B. The incomplete rosetta stone prob- lem: Identifiability results for multi-view nonlinear ica. Uncertainty in Artificial Intelligence, 2020.
Fotiadou, E., Panagakis, Y., and Pantic, M. Temporal archetypal analysis for action segmentation. In IEEE Conference on Automatic Face and Gesture Recognition (FG), 2017.
Hase, P. and Bansal, M. Evaluating explainable ai: Which algorithmic explanations help users predict model behav- ior? Proceedings of the Annual Meeting of the Associa- tion for Computational Linguistics (ACL), 2020.
Fu, X., Huang, K., and Sidiropoulos, N. On identifia- bility of nonnegative matrix factorization, IEEE Signal Processing Letters, 2018.
Gao, L., la Tour, T. D., Tillman, H., Goh, G., Troll, R., Radford, A., Sutskever, I., Leike, J., and Wu, J. Scaling and evaluating sparse autoencoders. Proceedings of the International Conference on Learning Representations (ICLR), 2025.
Hase, P.. Xie, H., and Bansal, M. The
problem in explainability and search methods for feature importance explanations. Advances in Neural Informa- tion Processing Systems (NeurIPS), 2021.
He, K., Zhang, X., Ren, S., and Sun, Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016.
Gersho, A. and Gray, R. M. Vector quantization and signal compression. 1991.
Ghorbani, A., Abid, A., and Zou, J. Interpretation of neural networks is fragile. Proceedings of the AAAI Conference on Artificial Intelligence (AAAI), 2017.
Hedström, A., Weber, L., Bareeva, D., Motzkus, F., Samek, W., Lapuschkin, S., and Höhne, M. M.-C. Quantus: an explainable toolkit for responsible evaluation of neural network explanations. The Journal of Machine Learning Research (JMLR), 2022.
Ghorbani, A., Wexler, J., Zou, Y., and Kim, Towards automatic concept-based explanations. Advances in Neu- ral Information Processing Systems (NeurIPS), 2019.
Gillis, N. Nonnegative matrix factorization. 2020.
Higgins, I., Matthey, L., Pal, A., Burgess, C. P., Glorot, X., Botvinick, M. M., Mohamed, S., and Lerchner, A. beta- vae: Learning basic visual concepts with a constrained variational framework. Proceedings of the International Conference on Learning Representations (ICLR), 2017.
Gillis, N. and Kumar, A. Exact and heuristic algorithms for semi-nonnegative matrix factorization. SIAM Journal Matrix Analysis and Applications, 2015.
Gimbernat-Mayok, J., Dominguez Mantes, A., Bustamante, C. D., Mas Montserrat, D., and loannidis, A. G. Archety- pal analysis for population genetics. PLoS Computational Biology, 2022.
Hillar, C. J. and Sommer, F. T. When can dictionary learning uniquely recover sparse data from subsamples? IEEE Transactions on Information Theory, 2015.
Hinrich, J., Bardenfleth, S., Røge, R., Churchill, N., Madsen, K., and Mørup, M. Archetypal analysis for modeling fmni data. IEEE Journal of Selected Topics in Signal Processing, 2016.
Goodwin, N. L., Nilsson, S. R., Choong, J. J., and Golden, A. Toward the explainability, transparency, and univer- sality of machine learning for behavioral classification in neuroscience. Current Opinion in Neurobiology, 2022.
Gorton, L.
The missing curve detectors of inceptionvl: Applying sparse autoencoders inceptionvl early vision. ArXiv e-print, 2024.
Hsieh, C.-Y., Yeh, C.-K., Liu, X., Ravikumar, P., Kim, S., Kumar, S., and Hsich, C.-J. Evaluations and methods for explanation through robustness analysis. Proceedings of the International Conference on Learning Representa- tions (ICLR), 2021.
Graziani, M., Palatnik de Sousa, I, Vellasco, M. M., Costa da Silva, F., Müller, H., and Andrearezyk, V. Sharp- ening local interpretable model-agnostic explanations for histopathology: improved understandability and reliabil- ity. Medical Image Computing and Computer Assisted Intervention, 2021.
Hu, J. and Huang, K. Global identifiability of 11- dictionary learning via matrix volume optimization. Advances in Neural Information Processing Systems (NeurIPS), 2023.
12

---

Archetypal SAE
Hurley, N. and Rickard, S. Comparing measures of sparsity. IEEE Transactions Information Theory, 2009.
Kim, S. S. Y., Meister, N., Ramaswamy, V. V., Fong, R., and Russakovsky, O. HIVE: Evaluating the human in- terpretability of visual explanations. Proceedings of the IEEE European Conference on Computer Vision (ECCV), 2022.
Idrissi, M. I., Chabridon, V., and Iooss, B. Developments and applications of shapley effects to sitaability-orienter sensitivity analysis with correlated inputs. Environmental Modelling & Software, 2021.
Kowal, M.. Dave, A., Ambrus. R., Gaidon, A., Derpanis, K. G., and Tokmakov, P. Understanding video trans- formers via universal concept discovery. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2024a.
Jacovi, A. and Goldberg, Y. Towards faithfully interpretable nlp systems: How should we define and evaluate faithful- ness? Proceedings of the Annual Meeting of the Associa- tion for Computational Linguistics (ACL). 2020.
Kowal, M., Wildes, P., and Derpanis, K. G. Visual con- cept connectome (vec): Open world concept discovery and their interlayer connections in deep models. Proceed- ings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2024b.
Javadi, H. and Montanari, A. Nonnegative matrix factor- ization via archetypal analysis. Journal the American Statistical Association, 2019.
Jenatton, R., Obozinski. G., and Bach, F. Structured sparse principal component analysis. International Conference on Artificial Intelligence and Statistics, 2010.
Lax, D. Functional analysis. 2014.
Lee, D. D. and Scung, H. S. Learning the parts of objects by non-negative matrix factorization. Nature, 1999.
Jourdan, F., Picard, A., Fel, T., Risser, L., Loubes, J. M., and Asher, N. Cockatiel: Continuous concept ranked attri- bution with interpretable elements for explaining neural net classifiers on nlp tasks. Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL), 2023.
Lee, D. D. and Seung, H. S. Algorithms for non-negative matrix factorization. Advances in Neural Information Processing Systems (NeurIPS), 2001.
Lee, H., Battle, A., Raina, R., and Ng. A. Efficient sparse coding algorithms. Advances Neural Information Pro- cessing Systems (NeurIPS), 2006.
Karvonen, A., Wright, B., Rager, C., Angell, R., Brinkmann, J., Smith, L.., Verdun, C. M., Bau, D., and Marks, S. Measuring progress in dictionary learning for language model interpretability with board game models. ArXiv e-print, 2024.
Lin, Z. Q., Shafiee, M. J., Bochkarev. S., Jules, M. S.S Wang, X. Y., and Wong, A. Do explanations reflect decisions? a machine-centric strategy to quantify the performance of explainability algorithms. Advances Neural Informa- tion Processing Systems (NIPS), 2019.
Kasiviswanathan, S., Wang, H., Banerjee, A., and Melville, P. Online II-dictionary learning with application to novel document detection. Advances in Neural Information Processing Systems (NeurIPS), 2012.
Liu, F. T., Ting, K. M., and Zhou, Z.-H. Isolation forest. IEEE International Conference on Data Mining, 2008
Keller, S. M., Samarin, M., Torres, F. A., Wieser, M., and Roth, V. Learning extremal representations with deep archetypal analysis. International Journal of Computer Vision, 2020.
Liu, Z., Mao, H., Wu, C.-Y,, Feichtenhofer, C., Darrell, T., and Xie, S. A convnet for the 2020s. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2022.
Kersting, K., Wahabzada, M., Thurau, C., and Bauckhage, C. Hierarchical convex nmf for clustering massive data. The Journal of Machine Learning Research (JMLR), 2010.
Lloyd, S. Least squares quantization in pem. IEEE Trans-
actions on Information Theory, 1982.
Khemakhem, I., Kingma, D., Monti, R., and Hyvarinen, A. Variational autoencoders and nonlinear ica: A unifying framework. Proceedings of the International Conference on Machine Learning (ICMI.,, 2020.
Locatello, F., Bauer, S., Lucic, M., Raetsch, G., Gelly, S., Scholkopf, B., and Bachem, O. Challenging common assumptions in the unsupervised learning disentan- gled representations. Proceedings of the International Conference on Machine Learning (ICML), 2019.
Kim, B., Wattenberg, M., Gilmer, J., Cai, C., Wexler, J., Viegas, F., et al. Interpretability beyond feature attribu- tion: Quantitative testing with concept activation vectors (tcav). Proceedings of the International Conference on Machine Learning (ICML), 2018
Locatello, F. Poole, B., Ratsch, G., Scholkopf, B., Bachem, O., and Tschannen, M. Weakly disentangle- ment without compromises. Proceedings of the Interna- tional Conference on Machine Learning (ICML., 2020.
13

---

Archetypal SAE
Lopes, M. Estimating unknown sparsity in compressed sensing. Proceedings of the International Conference Machine Learning (ICML), 2013.
Nguyen, G., Kim, D., and Nguyen, A. The effectiveness of feature attribution methods and its correlation with auto- matic evaluation scores. Advances in Neural Information Processing Systems (NeurIPS), 2021.
Lu, C., Shi, J., and Jia, J. Online robust dictionary learning. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2013.
Novello, P., Fel, T., and Vigouroux, D. Making sense of dependence: Efficient black-box explanations using de- pendence measure. Advances in Neural Information Pro- cessing Systems (NeurIPS), 2022.
Mahdizadehaghdamm S., Panahi, A., Krim, H., and Dai, L. Deep dictionary learning: A parametric network ap- proach. IEEE Transactions on Image Processing, 2019.
Olshausen, B. A. and Field, J. Emergence of simple-cell receptive field properties by learning sparse code for natural images. Nature, 1996.
Mair, S. and Brefeld, U. Coresets for archetypal analysis. In Advances in Neural Information Processing Systems (NeurIPS), 2019.
Olshausen, B. A. and Field, J. Sparse coding with an overcomplete basis set: A strategy employed by v1? Vi- sion Research, 1997.
Mairal, J., Bach, F., Ponce, J., and Sapiro, G. Online dic- tionary learning for sparse coding. Proceedings of the International Conference on Machine Learning (ICML). 2009.
Oquab, M., Darcet, T., Moutakanni, T., Vo, H., Szafraniec, M., Khalidov, V., Fernandez, P., Haziza, D., Massa, F,, El- Nouby, A., et al. Dinov2: Learning robust visual features without supervision. ArXiv e-print, 2023.
Mairal, J., Bach, F., and Ponce, J. Task-driven dictionary learning. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2011.
Papyan, V., Romano, Y., and Elad, M. Convolutional dic- tionary learning via local processing. Proceedings of the IEEE International Conference on Computer Vision (ICCV), 2017.
Mairal, J., Bach, F., and Ponce, Sparse modeling for image and vision processing. Foundations and Trends in Computer Graphics and Vision, 2014.
Parekh, J., Khayatan, P., Shukor, M., Newson, A., and Cord, M. A concept-based explainability framework for large multimodal models. ArXiv e-print, 2024.
Makelov, A., Lange, G., and Nanda, N. Is this the sub- space you are looking for? an interpretability illusion for subspace activation patching. ArXiv e-print, 2023.
Mayne, H., Yang, Y., and Mahdi, A. Can sparse autoen- coders be used to decompose and interpret steering vec- tors? ArXiv e-prins, 2024.
Paulo, G. and Belrose, N. Sparse autoencoders trained on the same data learn different features. ArXiv e-print, 2025.
Mei, J., Wang. C.. and Zeng, W. Online dictionary learn- ing for approximate archetypal analysis. In European Conference on Computer Vision (ECCV), 2018.
Petsiuk, V., Das, A., and Saenko, K. Rise: Random- ized input sampling for explanation of black-box models. Proceedings of the British Machine Vision Conference (BMVC), 2018.
Menon, A., Shrivastava, M., Krueger, D., and Lubana, E. S. Analyzing (in) abilities of saes via formal languages. ArXiv e-print, 2024.
Poché, A., Jacovi, A., Picard, A. M., Boutin, V., and Jourdan, F. Consim: Measuring concept-based explanations ef- fectiveness with automated simulatability. ArXiv e-print, 2025.
Moliner, J. and Epifanio, I. Robust multivariate and func- tional archetypal analysis with application to financial time series. Physica A: Statistical Mechanics and its Applications, 2019.
Prabhakaran, S., Raman, S., Vogt, E., and Roth, V, Au- tomatic model selection in archetype analysis. In DAG- M/OAGM Symposium, 2012.
Mørup, M. and Hansen, L. K. Archetypal analysis for machine learning and data mining. Neurocomputing, 2012.
Rajamanoharan, S., Lieberum, T., Sonnerat, N., Conmy, A., Varma, V., Kramar, J., and Nanda, N. Jumping ahead: Improving reconstruction fidelity with jumprelu sparse autoencoders. ArXiv e-print, 2024.
Muzellec, S., Andeol, I.., Fel, T., VanRullen, R., and Serre, T. Gradient strikes back: How filtering out high frequen- cies improves explanations. Proceedings of the Interna- tional Conference on Learning Representations (ICLR), 2024.
Rasti, B., Zouaoui, A., Mairal, J., and Chanussot, J. Sunaa: Sparse unmixing using archetypal analysis. IEEE Geo- science and Remote Sensing Letters, 2023.
14

---

Archetypal SAE
Rencker, L., Bach, F,, Wang, W., and Plumbley, M. D. Sparse recovery and dictionary learning from nonlinear compressive measurements. IEEE Transactions on Signal Processing, 2019.
Slack, D., Hilgard, A., Lakkaraju, H., and Singh, S. Coun- terfactual explanations can be manipulated. Advances in Neural Information Processing Systems (NeurIPS), 2021.
Smilkov, D., Thorat, N., Kim, B., Viégas, F., and Watten- berg, M. Smoothgrad: removing noise by adding noise. Proceedings of the International Conference on Machine Learning (ICML), 2017.
Rentzeperis, I., Calatroni, L.,, Perrinet, L. U., and Prandi, D. Beyond 11 sparse coding in vl. PLoS Computational Biology, 2023.
Rubinstein, R., Bruckstein, A. M., and Elad, M. Dictionaries for sparse representation modeling. Proceedings of the IEEE, 2010.
Spielman, D. A., Wang, H., and Wright, J. Exact recovery of sparsely- dictionaries. The Journal of Machine Learning Research (JMLR), 2012.
Scholkopf, B., Williamson, R. C., Smola, A., Shawe-Taylor J., and Platt, J. Support vector method for novelty detec- tion. Advances in Neural Information Processing Systems (NeurIPS), 1999.
Springenberg, J. T., Dosovitskiy, A., Brox, T., and Ried- miller, M. Striving for simplicity: The all convolutional net. Workshop Proceedings of the International Confer- ence on Learning Representations (ICLR), 2014.
Schott, L.. Von Kugelgen, J., Trauble, F., Gehler, P., Russell, C., Bethge, M., Scholkopf, B., Locatello, F,, and Brendel, W. Visual representation learning does not generalize strongly within the same domain. ArXiv e-print, 2021.
Stone, E. and Cutler, A. Introduction to archetypal analysis of spatio-temporal dynamics. Elsevier, 1996.
Sturmfels, P., Lundberg, S., and Lee, S.-L. Visualizing the impact of feature attribution baselines. Distill, 2020.
Seiler, C. and Wohlrabe, K. Archetypal scientists. Journal of Informetrics, 2013.
Sun, Y., Liu, Q.. Tang. J., and Tao, D. Learning discrimi- native dictionary for group sparse representation. IEFE Transactions on Image Processing, 2014.
Selvaraju, R. R.,, Cogswell, M., Das, A., Vedantam, R., Parikh, D., and Batra, D. Grad-cam: Visual explana- tions from deep networks via gradient-based localization. Proceedings of the IEEE International Conference on Computer Vision (ICCV), 2017.
Sundararajan, M., Taly, A., and Yan, Q. Axiomatic attribu- tion for deep networks. Proceedings of the International Conference on Machine Learning (ICML), 2017.
Surkov, V., Wendler, C., Terekhov, M., Deschenaux, J., West, R., and Gulcehre, C. Unpacking sdxl turbo: Inter- preting text-to-image models with sparse autoencoders. ArXiv e-print, 2024.
Serre, T. Learning a dictionary of shape-components in visual cortex: Comparison with neurons, humans and machines. 2006.
Seth, S. and Eugster, M. J. A. Probabilistic archetypal analysis. Machine Learning, 2016.
Tamkin, A., Taufeeque, M., and Goodman, N. D. Codebook features: Sparse and discrete interpretability for neural networks. ArXiv e-print, 2023.
Sifa, R. and Bauckhage, C. Archetypical motion: Super- vised game behavior learning with archetypal analysis. In FDG Workshop Game Mining, 2013.
Tariyal, S., Majumdar, A.. Singh, R., and Vatsa, M. Deep dictionary learning. IEEE Access, 2016.
Silverman, B. W. Spline smoothing:
the equivalent variable kernel method. The Annals of Statistics, 1984.
Tasissa, A., Tankala, P., Murphy, J. M., and Ba, D. K-deep simplex: Manifold learning via local dictionaries. IEEE Transactions on Signal Processing, 2023.
Simon, B. Convexity: an analytic viewpoint. 2011.
Thasarathan, H., Forsyth, J., Fel, T., Kowal, M., and Der- panis, K. Universal sparse autoencoders: Interpretable cross-model concept alignment. ArXiv e-print, 2025.
Simonyan, K., Vedaldi, A., and Zisserman, A. Deep inside convolutional networks: Visualising image classification models and saliency maps. Proceedings of the Interna- tional Conference on Learning Representations (ICLR), 2013.
Thurau, C. and Bauckhage, C. Archetypal images in large photo collections. IEEE International Conference on Semantic Computing, 2009.
Sixt, L., Granz, M., and Landgraf, T. When explanations lie: Why many modified bp attributions fail. Proceedings of the International Conference on Machine Learning (ICML), 2020.
Thurau, C., Kersting, K., and Bauckhage, C. Convex non- negative matrix factorization in the wild. IEEE Interna- tional Conference on Data Mining, 2009.
15

---

Archetypal SAE
Tošić, I. and Frossard, P. Dictionary learning. IEEE Signal Processing Magazine, 2011.
Zhai, X., Mustafa, B., Kolesnikov, A., and Beyer, L. Sig- moid loss for language image pre-training. Proceedings of the IEEE International Conference on Computer Vision (ICCV), 2023.
Tripicchio, P. and Avella, S. Is deep learning ready to satisfy industry needs? Procedia Manafacturing, 2020.
Zhang, R., Madumal, Miller, T., Ehinger, K. A., and Rubinstein, B. I. Invertible concept-based explanations for can models with non-negative concept activation vec- tors. Proceedings of the AAAI Conference on Artificial Intelligence (AAAI), 2021.
Vielhaben, J., Blücher, S., and Strodthoff, N. Multi- dimensional concept discovery (med): unifying frame- work with completeness guarantees. The Journal of Trans- actions Machine Learning Research (TMLR), 2023.
Vilas, M. G., Adolfi, F., Poeppel, D., and Roig, G. Position: An inner interpretability framework for ai inspired by lessons from cognitive neuroscience. Proceedings of the International Conference on Machine Learning (ICML), 2024.
Zimmermann, R. S., Sharma, Y., Schneider, S., Bethge, M., and Brendel, W. Contrastive learning inverts the data generating process. Proceedings of the International Conference on Machine Learning (ICML), 2021.
Zou, H., Hastie, T., and Tibshirani, R. Sparse principal com- ponent analysis. Journal of Computational and Graphical Statistics, 2006.
Virtanen, P.. Gommers, R., Oliphant, T. E.,, Haberland, M., Reddy, T., Cournapeau, D., Burovski, E., Peterson, P., Weekesser, W., Bright, J., et al. Scipy 1.0: fundamental algorithms for scientific computing in python. Nature Methods, 2020.
Zouaoui, A., Muhawenayo. G., Rasti, B., Chanussot, J., and Mairal, J. Entropic descent archetypal analysis for blind hyperspectral unmixing. IEEE Transactions Image Processing, 2023.
Von Kugelgen, J., Sharma, Y., Gresele, L., Brendel, W., Scholkopf, B., Besserve, M., and Locatello, F. Self- supervised learning with data augmentations provably isolates content from style. Advances in Neural Informa- lion Processing Systems (NeurIPS), 2021.
Wattenberg. M. and Viegas, F. B. Relational composition in neural networks: A survey and call to action. ArXiv e-print, 2024.
Wedenborg, A. E. J. and Mørup. M. Archetypal analysis for binary data. IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2025.
Wightman, R. et al. Pytorch image models, 2019.
Wright, J., Ma, Y., Mairal, J., Sapiro, G., Huang, S., and Yan, S. Sparse representation for computer vision and pattern recognition. Proceedings of the IEEE, 2010.
Wynen, D., Schmid, C., and Mairal, J. Unsupervised learning of artistic styles with archetypal style analysis. In Advances in Neural Information Processing Systems (NeurIPS), 2018.
Yu, Y., Buchanan, S., Pai, D., Chu, T., Wu, Z., Tong, S., Haeffele, B., and Ma, Y. White-box transformers via sparse rate reduction. Advances in Neural Information Processing Systems (NeurIPS), 2023.
Zeiler, M. D. and Fergus, R. Visualizing and understanding convolutional networks. Proceedings of the IEEE Euro- pean Conference on Computer Vision (ECCV), 2014.
16

---

Archetypal SAE
Appendix
Figure 7. Examples of 3 Concept Clusters in DinoV2, Each cluster contains 4 example concepts. Complex hand positions. ranging from hands in pockets hands on another person. Abstract "under" concepts, linking animals objects, such as birds, zebras, felines, and airplanes, while focusing lower regions. Fine-grained animal facial features, including ears, eyebrows, and cheeks.
17

---

Archetypal SAE
Method
DINOv2
ResNet
SigLIP
VIT
KMcans
0.7678
0.7624
0.7684
0.7702
ICA
0.8092
0.8370
0.8243
0.8267
Sparse PCA
0.7981
0.8318
0,8069
0,8082
SVD
0.7979
0.8291
0.8062
0,8075
SemiNMF
0.8297
0.8327
0.8358
0.8423
ConvexNMF
0.7645
0.7582
0.7639
0.7634
PCA
0,7979
0.8291
0,8062
0,8075
Vanilla
0.8047
0.8167
0.8126
0.8223
TopK
0.8135
0.8150
0.8289
0.8328
Jump
0,8010
0.7988
0.8131
0,8053
A-SAE
0,9482
0.9631
0.9602
0.9615
RA-SAE 0.9447
0.9602
0.9585
0.9586
Table Soft Identifiahility benchmark Across Models and Methods. This table presents the average accuracy scores for various methods evaluated across four different models: DINO, ResNet, SigLIP, and VIT. Best-performing scores for each model are in bold and second best are underlined, Full results are available in Appendix I.
A. Soft-Identifaailiit results
A)
Intuitively, the evaluation above gauges whether each ob- ject is disentangled into its "own box", in that a threshold À enables one-hot classification for all objects. In each of the 12 synthetic datasets, the size of the dictionary is set to the exact number of distinet objects (from 9 20 objects) and we generate datasets of 2,000 samples. An example of the dataset and the procedure is given in Figure 6.
Results are shown in Table 3. Classical dictionary learn- ing methods and standard SAES exhibit comparable performance, struggling to accurately recover the true generative factors. In contrast, RA-SAE consistently achieves higher accuracy in recovering the underlying object classes across all synthetic datasets. This indicates that RA-SAE not only stabilizes the dictionary, but also enhances its ability to discern the true underlying struc- ture of the data, thereby improving the disentanglement of mixed objects.
B)
C)
B. Qualitative Examples
This section provides qualitative insights into the con- cepts learned by DinoV2 (Oquab et al., 2023; Darcet et al., 2023). We trained an Archetypal SAE on Dino B with 4 registers and reported key performance metrics in Figure 10. RA-SAE uncovers unexpected concepts, such as shadow-based based features (potentially linked to depth reasoning), a context-dependdet "barber" concept (acti- vating for barbers but not clients), and fine-grained edge detection in flower petals (Figure 8). It also learns more structured within-class distinctions (e.g.. separating rabbit ears, faces, and paws) compared to TopK-SAFs (Figure 9). Finally, its dictionary forms clear clusters, grouping se- mantically related features like fine- ime-ggained animal faces or spatial concepts such as "underneath" (Figure 7).
Figure 8. Exotic Concepts in D2n A) Highlights tokens in shad- of dogs, suggesting shadow-based features and potential depth reasoning. B) A "barber" concept exclusively active for barbers, not their clients. C) fine- fine-grained concept focusing on petal edges or contours.
To explore the dictionary learned by the model, we analyzed three clusters of concepts, as illustrated in Figure 7. Specifically:
18

---

Archetypal SAE
Baseline Rabbit
RA-SAE Rabbit
Figure Top-5 Concepts for the Rabbit Class DinoV2. The RA-SAE on top of Topk identifies distinct and fine-grained concepts, including rabbit cars, body, face, and paws. These concepts exhibit greater structure and granularity compared those found by the unconstrained Topk SAE (baseline) method.
A cluster of concepts capturing complex hand positions, ranging from hands in pockets to hands resting another person (e.g., on a shoulder). An abstract "under" concept cluster, linking entities such as birds, zebras, felines, and airplanes, while highlighting the lower regions of objects. A cluster representing fine-grained facial concepts in animals, including ears, eyebrows, and cheeks.
In addition, we identified surprising and specific concepts among the 16, 000 dictionary atoms, shown in Figure ??. These include: A) A concept highlighting tokens corresponding to shadows of dogs, suggesting that DinoV2 may use shadows as feature, potentially contributing to its depth estimation 3D reasoning capabilities. B) A "barber" concept that activates exclusively on tokens representing barbers but not on the individuals receiving haircuts or shaves. C) A fine-grained visual concept that activates along the contours or edges of flower petals.
Finally, we present the top-5 concepts associated with the rabbit class on DinoV2 in Figure 9. The concepts learned by RA-SAE are distinct and exhibit greater structure compared their TopK counterparts. For instance, RA-SAE successfully identifies separate concepts for rabbit ears, body, face, and paws, demonstrating its ability disentangle fine-grained features within class. These results suggest that RA-SAE provides more organized and meaningful decomposition of concepts.
19

---

Archetypal SAE
Dictionary auto-correlation DD
Concept firing rate
Token reconstruction error
|/2,llo
25-
20-
133
3300
500
12-
12
14
Cls token
Register tokens
Figure 10. Performance, Cuto-Correlation, and Firing Rate of DinoV2 for Archetypal SAE. The Archetypal SAE analyzed here demonstrates a well-structured dictionary with low auto-correlation, indicating that the dictionary atoms are not collincar. The firing rate exhibits long-tail distribution, and reconstruction error uniformly distributed across tokens, except for the CLS token, which shows the highest reconstruction error.
20

---

Archetypal SAE
C. Extended Related Work on Archetypal Analysis
Archetypal analysis (AA) was introduced by Cutler and Breiman in 1994 as a method to represent observations as convex combinations of extremal "pure types" called archetypes (Cutler & Breiman, 1994). Each archetype lies on the boundary of the data's convex hull and is itself constrained to be convex combination of data points, yielding an interpretable factorization where data are explained in terms of extreme exemplars. This approach was proposed as an alternative to principal component analysis for uncovering latent structure, providing data-like representative factors instead of orthogonal directions (Cutler& & Stone, 1997). Shortly after its introduction, AA was extended to spatio-temporal settings by identifying archetypes that vary over time (Stone & Cutler, 1996) and (Prabhakaran et al., 2012) proposed an automatic model selection criterion for AA, alleviating the need to fix a priori. After its initial formulation, AA did not gain widespread popularity, partly due to computational limitations and the availability of more established techniques like k-means or NMF. Nonetheless, several works in the 2000s applied AA successfully in scientific domains. For instance (Chan et al., 2003) applied AA to astronomical spectra. In genetics, (Gimbernat-Mal Mayol et al., 2022) found that archetypes learned from human genotype data corresponded to ancestral population prototypes. AA was also utilized in socio- and bibliometric analyses (Seiler & Wohlrabe, 2013), and to model player behavior in video games (Sifa & Bauckhage, 2013). Various enhancements to the basic AA model have been proposed, (Eugster & Leisch, 201 1) introduced weighted and robust AA, (Seth & Eugster, 2016) proposed to treats archetypes as latent factors in a probabilistic generative model. In a similar vein, contemporary work has developed AA for binary data specifically (Wedenborg Mørup, 2025) and (Epifanio et al., 2019) tackled the presence of missing values. Likewise (Moliner & Epifanio, 2019) formulated robust AA for multivariate functional data.
Another important direction has been the development of non-linear and kernel variants of AA (Mørup & Hansen, 2012; Bauckhage & Manshaei, 2014). These kernel and multi-layer approaches connect AA with manifold learning techniques. (Javadi & Montanari, 2019) cast NMF as a special case of AA under separability assumptions. In term of algorithmic and scalability improvements (Chen et al., 2014) proposed a fast active-set algorithm, implemented in the SPAMS toolbox, making AA tractable for large-scale computer vision tasks. (Bauckhage et al.) adapted a Frank-Wolfe algorithm for AA, reducing runtime. (Mair & Brefeld, 2019) proposed coreset approximations to speed up archetype discovery, and (Abrol & Sharma. 2020) developed a greedy AA algorithm that extended naturally robust and kernelized variants. Finally, (Mei al., 2018) proposed an online algorithm for streaming data scenarios.
Mairal et al.'s task-driven dictionary learning framework inspired supervised AA models (Mairal et al., 2011). Related topic modeling approaches such as SurvLDA (Dawson & Kendziorski, 2012) and anchor- bbsed models (Ding et al., 2016; Arora et al., 2013) similarly identify extremal structure. More recently, deep learning has been integrated with AA to handle complex, non-linear data. Wynen et al. (Wynen et al., 2018) applied AA to deep features for archetypal style analysis in artwork. Bauckhage et al. (Bauckhage et al.) described AA as an autoencoder. Keller et al. (Keller et al., 2020) developed a VAE-based deep AA that accommodates supervision. Applications are widespread. In vision, AA has been used to discover prototypical images (Thurau & Bauckhage, 2009), representative behaviors (Fotiadou et al., 2017), and fMRI brain patterns (Hinrich et al., 2016). In hyperspectral imaging, recent contributions such as SUnAA (Rasti et al., 2023) and entropic descent AA (Zouaoui et al., 2023) set new benchmarks. In NLP, AA was used in multi-document summarization (Canhasi & Kononenko, 2016) and anchor-based topic modeling (Ding et al., 2016; Arora et al., 2013).
Comparison to Our Work Our method departs from AA by constraining only the decoder to the convex hull of data, while using a linear encoder, maintaining full compatibility with modern SAEs architectures like TopK and JumpReLU, Unlike AA, which jointly optimizes over convex atoms and codes, we retain end end SAE training. Our goal is not AA approximation or generation, but improved stability and semantic consistency of SAEs. We further introduce scalable relaxation of the convex constraint, adapted to large training and distinct from prior AA relaxations (e.g., (Mørup & Hansen, 2012)).
21

---

Archetypal SAE
D. Formal Definitions of Metrics
two training runs:
To comprehensively evaluate SAEs and their archetypal vari- ants, we have defined in Sec.5.1 a set of metrics that assess four key dimensions of the SAEs: (f) sparse reconstruction, (ii) consistency, (iit) structure in the dictionary (D), and (iv) structure in the codes (Z).
Max Cosine
max(D;, Dj).
(8)
1.
This metric provides an upper bound on alignment but does not enforce global consistency across the dictionary.
Beyond stability, we assess how well the learned dictionary aligns with real data. The Score measures the deviation of dictionary atoms from real data points by computing the cosine similarity between each dictionary atom D; and its closest real activation Ay:
D.1. Sparse Reconstruction
(OOD)
As explained in the main paper, these metries evaluate the ability of the model to accurately reconstruct activations while enforcing sparsity constraints. We believe they are well understood and already used by the interpretability community. The Reconstruction Error (R²) measures the fidelity of the reconstruction by quantifying how well the learned dictionary approximates the input activations:
&
OOD Score
max Di, A2). jEjn
(9)
k
A score of 0 indicates that every dictionary atom D; ex- actly matches existing data point Aj. meaning the model purely reconstructs real activations. Notably, methods like Separable-NMF (Gillis, 2020), which enforce the "pure- pixel" assumption by explicitly selecting dictionary atoms from the dataset, naturally achieve an OOD score of O.
A
A F
R² =
(5)
A
A
with Ā = ZD the predicted activation and À the mean activation matrix. Essentially, R2 measures how much we improve on explaining variance upon the best possible pre- dictor that uses only a single bias. The Dead Codes measure the fraction of dictionary atoms that remain unused across the dataset, highlighting inefficiencies in the learned repre- sentation, for a set of n codes and l: concepts Z E Baxk,
Together, these metries provide a comprehensive evalua- tion of how stable, interpretable, and grounded the learned dictionary remains across training runs and relative real data.
D.3. Structure in the Dictionary (D)
Dead Codes 1
1 T.
Σg
n
Zi
(6)
The third category assesses the internal organization of the dictionary, providing insights into its effective dimensional- ity, redundancy, and (we hope) overall interpretability. Un- like reconstruction or consistency metrics, which evaluate external properties of the learned dictionary, these metrics focus on how well-formed and structured the set of concepts is. Again, a dictionary with lower effective dimensionality suggests structure like compositionality and/or hierarchical concepts. Stable Rank is the first metric we propose that provides an effective measure of the intrinsic dimensionality of the dictionary:
D.2. Consistency
The second category of metrics assesses how consistent and well-grounded the learned dictionary is. Specifically, we evaluate (i) the Stability of the learned solution across training runs and (ii) its proximity to real data, measured via the -attof--Dsttiibttion (OOD) Score.
Stability, introduced in Eq. 2, quantifies how consistent the learned dictionary remains when training is repeated with different initializations. Given two independently trained dictionaries, D and D', stability is defined as:
| P D
Stable Rank
(10)
I TT(DD"ID"),
Unlike the traditional matrix rank, which is sensitive to numerical precision (all dictionaries are nearly full rank in practice because of numerical error), the stable rank remains well-behaved even in high-dimensional settings, serving as smooth proxy for rank estimation. Effective Rank offers an alternative perspective by measuring the entropy of the singular value distribution of D:
Stability(D), D') max TTP(()) n
where P(n) is the set of n X n signed permutation matrices. score of indicates perfect alignment- each concept in D has direct equivalent in D', while score of 0 implies that all concepts seed-specific and change arbitrarily across runs.
A looser measure of stability is the Max Cosine Similarity, which only considers the hest-matching concept between
Eff. Rank = exp
Σ 1=1
o. log
(11)
22

---

Archetypal SAE
where 0; are the normalized singular values of D (i.e., E, 1). An effective rank close lo suggests that all dictionary atoms equally important, whereas low effective rank implies that only few dominant concepts capture most of the variation in the data. A fully orthogonal dictionary (thus not overcomplete) would achieve an effec- tive rank k. Coherence a common notion in dictionary learning and compress sensing, it quantifies redundancy between dictionary atoms by measuring the maximum pair- wise cosine similarity:
Where @ is the Hadamard product, this metric captures cases where concepts and frequently co-activate (as measured by Z' 2), yet their dictionary atoms neg- atively correlated (indicated by a negative dot product in DD The ReLU function ensures that only destructive interactions are counted, where activation of both concepts leads to mutual cancellation rather than constructive com- bination. A high negative interference score suggests that the learned dictionary contains redundant antagonistic concepts. In extreme cases, we could imagine a strong neg- ative interference can lead to concept pairs that consistently suppress each other to comply with some sparsity constraint.
Coherence max D D2l.
(12)
We still admit that each D, is on the &g ball. Lower coher- ence indicates that dictionary atoms are more diverse and span independent directions, which desirable for disentan- gled representations. Conversely, high coherence suggests that multiple dictionary atoms encode nearly identical fea- tures, reducing the efficiency of the learned basis. Notably, coherence is closely related to the concept of muiual in- coherence in compressed sensing (Donoho, 2006), where low-coherence bases are preferred for sparse signal recov- ery.
D.4. (iv) Structure in the Codes (Z)
The last category measures how much structure we have in the codes. While dictionary structure (iii) focuses on the learned basis D, the structure of the codes Z determines how these dictionary atoms are used reconstruct activa- tions. A well-structured encoding should exhibit meaningful combinations of concepts while avoiding redundancy and destructive interference. Connectivity measures the diver- sity of concept usage by counting the number of unique co-activations within the code matrix:
1 da
Connectivity = =1
1 (lo))
(13)
This metric quantifies how many distinct pairs of concepts (i,j) are activated together across samples. A high connec- tivity score suggests that broad range of concepts can be meaningfully combined, leading to more complex represen- lations. Conversely, low connectivity implies a highly struc- tured representation, and in some sense a group-sparsity representation where only a subset of concepts can fire to- gether. We note that connectivity sparse coding has been linked compositionality (OIshausen & Field, 1996). We believe that none of the SAEs currently studied achieve interesting performance on this metric. Finally, Negative Interference quantifies the extent to which co-activated concepts cancel each other out, reducing their effectiveness:
Neg. Inter. = ||ReLU(-(Z" ZZ
2)@+ @ (DD
(14)
23

---

Archetypal SAE
partitioning isolate anomalies, providing efficient
method for detecting outliers.
0.6
Convex Hull on Reduced Dimensions: Computes the convex hull of A after projecting it onto its 10 principal components via PCA. This ensures that extreme points defining the overall shape of the distribution are retained while reducing computational complexity.
0.5
0.4
One-Class SVM (OC-SVM) (Scholkopf et al., 1999): A support vector method that learns boundary around high-density regions, effectively isolating representative points while filtering outliers.
0.2
99.5
99.0
97.5
95.0
90.0
Sparsity (%)
Among these, K-Means emerges as the most effective method. Furthermore, while convex hull approaches theo- retically guarantee coverage of extreme points, their com- putational cost scales poorly with high dimensions, making them impractical for large datasets.
Ocsvm
Convex hull
Kmeans
Lof
Iso forest
Figure 11. Comparison of Distillation Methods. To scale the Archetypal SAE, it impractical to utilize the entire data matrix A for identifying archetypes. Instead, we first reduce the dataset smaller subset of points, denoted archetypesdiccionary elements from this reduced set. Among the most
F. Implementation Details
and construct the
We provide an efficient PyTorch implementation the re- laxed dictionary learning module used in Archetypal SAEs. Given distilled set of centroids C E I xd our goal is to construct dictionary D as convex combination of ele- ments in C, while allowing controlled degree of relaxation via additive perturbations.
distillation methods evaluated, K-Means proves to effective approach, generating points within the convex hull of the data and achieving strong performance scores. These experiments were conducted DINOv2 using Archetypal SAE without relaxation.
The weight matrix W E RR is constrained to the probability simplex, ensuring convex combinations of centroids. This is enforced via convex_paraee function that projects W onto the simplex using ReLU and row- wise normalization.
E. Distilling A into C'
As recall, Archetypal SAEs construct dictionary atoms convex combinations of data points,
The relaxation term ô E Rexd allows mild deviations from strict convexity. To prevent excessive drift, 5 is reg- ularized by adaptively scaling its norm to remain within a relaxation factor.
D WA, with W Sk.n. A Rad (15)
requiring access the full data matrix A. However, the
original problem is intractable, the number of points n requires storing and processing millions of activations at each gradient step, which is computationally prohibitive, particularly for large-scale datasets of tokens. To address this, we proposed Sec. 4 distillation step, reducing A to compact subset C. The dictionary is then formed using only C E TR xd. with n' < n, ensuring tractable optimization while remaining representative of the original distribution.
Dictionary atoms are computed as D WC + 8. The model supports gradient updates for W and while keeping C fixed.
We investigated five different methods to distill into C, as illustrated in Fig. 11:
Groups data into clusters and selects cen- troids as representatives. This ensures that C' remains within the convex hull of A while capturing its most fre- quent patterns.
K-Means:
Local Outlier Factor (LOF) (Breunig et al., 2000): Iden- tifies statistically atypical points compared to their neigh-
bors, helping remove rare or extreme cases.
Isolation Forest (Iso) (Liu al., 2008): Uses recursive
24

---

Archetypal SAE
class ArchetypalDicctionrrymmm Modull)ee
"*"Relaxed Archetypal SAE (RA-SAE) dictionary.
Constructs a dictionary where each atom is convex combination of data points from C, with small relaxation term A constrained by 8.
def initt_ C, k, delta=1.0);
Parameters
C:: Tensor
Candidate archetypes of shape (n', d).
int
Number dictionary atoms.
delta float
Upper bound on the norm of the relaxation term A.
n_prime, Chape
self. weggister_buffrre
C) store C as fixed buffer (non-trainable)
self.w W = nn. (torch (tkrcheeekk n_prime)) # trainable param soowsstcchhstic Lambaa = nn. Parameeerr aarameterttorcc.zeo self. delta delta constraint the relaxation term norm
d)) # small relaxation term
def (sel,, 2):
Parameters
: Tensor Sparse codes of shape (n, k).
Returns
Tensor
Reconstructed activations of shape (n, d).
with torch.no_grad():
# ensure W remains row-stochastic (positive and row sum to one)
W = reluseffl
W /= sum keepdim=True)
self.W.daa
# enforce the norm constraint on A to limit deviation from conv(C) norm_Lambda self. Lambda. (ndiidii-, keepdim=True) # norm per row scaling factor = torch. clemf / norm_Lambda, max=1) # safe scaling factor Lambda scaling_factor scale
satisfy ||A]| 5
compute the dictionary as a convex combination plus relaxation
Welf.W self.C self Lambda
return
Figure 12. Detailed Pytorch code for Relaxexd Archetypal SAE (RA-SAE).
25

---

Archetypal SAE
G. Theoretical properties of Archetypal Dictionary
D E conv (C) S conv and ZD' C &oon( Finally, if includes the extreme points of A, then no representa- tional power is lost.
In this section, we provide theoretical insights into Archety- pal Sparse Autoencoders (A-SAEs) by addressing three key aspects: (i) why standard SAES can produce dictionaries that drift away from the data manifold, (ii) geometric in- terpretation of the A-SAE solution and the conditions under which distillation is optimal, and (iii) bounds on the stability, rank, and out-o--disti (OOD) score of A-SAFs.
Proof. Since W is row-stochastic, each D; W,A is a convex combination of the rows of A, i.e., D; conv (A). Furthermore, for nonnegative Z, we have ZD (ZW) A,
with ZW 2 O, which implies that ZD E cone ()). With D' = WC for conv(C) C conv (A), each row D; lies within conv(C) C Conv(A). Lastly, if C contains the extreme points (Boyd Vandenberghe, 2004) of A, then by simple application of the Krein-Milman theorem (Lax, 2014), conv(C) = (Anv(A) and cone( ZWC C) cone(A), ensuring no loss in expressivity.
A simple explanation for why standard SAEs may drift away from the data manifold can be found by examining the gradient descent (GD) update rule for the dictionary D. Given a dataset A E Rexd nonnegative codes Z 2 0 E Raxk, and a dictionary D E R with unit-norm rows ,1l2 1). the standard SAE optimization problem is:
min A 2.D
1
s.t. Z 220,
|Djll2 1. (16)
The constraints imposed by A-SAE on D lead to a straight- forward yet crucial geometric property: each dictionary atom D, remains within the convex hull of the data. This result, while simple, has interesting implications for some of the metrics we study, notably stability and OOD. In fact, we will now see that this implies a bounded OOD score, prevents the rank of the dictionary (and thus its structure) to higher than the rank of the data, and induces some loose form of algorithmic stability.
The gradient of the reconstruction loss with respect to D is given by:
V D A -
A).
(17)
This gradient can be decomposed into two components:
AD
ZDZD
data-anchored term
out-nf-data term
G.2. Stability of Archetypal Dictionary
The first term, AA pulls dictionary atoms toward a conic combination of data points, anchoring them to cone(A) A)). However, the second term, ZT DD, introduces a drift effect that pushes the dictionary away from the data, as it depends on the correlations within Z and the original seed D.
Proposition G.2 (Geometric Stability of Archetypal Dic- tionaries). Let A, A' E Rnxd be two data matrices such that || A A'Ile S. Suppose W, W' Au are row- stochastic matrices (i.e. each row belongs to the probability simplex). Define the archetypal dictionaries D = W A and D' = W' A'. Then,
Empirically, in high- -igh--imensional settings where the codes Z exhibit sufficient variability could induce the second term to dominate. This could explain why classical SAES have a relatively low OOD score, as shown in Sec. 1 and why minor perturbations in initialization or the training set can potentially yield different dictionaries, as the second term is entirely dependent of the seed.
D - D'Ile S ViE+ 2V min (I44|1,1144(11))
Proof. Using triangle inequality, we have
D D- = WW A- W A'll WW W W''ll=
+
WA' W' W'A'lle.
G.1. Geometric interpretation of A-SAE
We bound each term separately, since D and D' are archety- pal dictionaries, W is row-stochastic, hence ||W||2 S Vk. Therefore,
Proposition G.1 (Archetypal Dietionary, Convex and Conic Hulls). Given A ga as set of n data points and W Ωg.. any row-stochastic matrix, parameteriz- ing D = -AA ensures that each concept D; lies within the convex hull of the data, i.e., D; E conv( (A) for all i E [k). Moreover, for any nonnegative codes Z 2 O, the reconstruution ZD lies within the conic hull of the data, i... ZD S cone A). More generally, let C E TR' xd such that CC) S Co)v(A) be a subset of A or any set of points within conv (A). Then D' WC satisfies
114-A11 ((AA-A))lF= S
|| ||A- A'IIE,
and W 11 ||||-- A S Viee For the second term,
W A' w' w'A'll= = [I(W - w') A' 2
W W ||''||.
26

---

Archetypal SAE
with [ W'l S 2 summing these yield:
G.4. Bounding OOD score with Archetypal Constraints
D D- S +
We now demonstrate that the OOD measure of dictionary atoms obtained under some assumption is inherently lower- bounded. Specifically, the measure is directly tied to the weights in the row-stochastic matrix, with a maximum value of I achieved when a dictionary atom perfectly aligns with a data point. This property is particularly interesting as it ensures that dictionary atoms remain well-grounded in the data. Furthermore, the sparsity of the weight matrix plays a crucial role in maintaining orthogonality, thereby preserving the plausibility of the assumption and the robustness of the bounds.
It is straightforward to repeat the above process with || factor in the right hand side. Taking the minimum over the two factors then completes the proof.
The key observation is that a row-stochastic matrix W can- not stretch data arbitrarily, imposing a natural control on how D WA A changes if A is slightly perturbed. By contrast, in the unconstrained setting, where the dictionary D is free to move anywhere (subject only to norms or regu- larization), there is no comparably simple bound ensuring stability. Even small perturbations in A (or in the random seed, initialization, etc.) can shift the solution significantly: as there is no requirement that D stay close to (A), the learned atoms can drift to entirely different regions of the space.
Proposition G.4 (OOD Measure with Non-Interfering Archetypes). Let A E Raadd be our data matrix where each point A, is normalized Alll = 1 for all i E [n]). Let our archetypal dictionary D = W, where W E Rkxn is a row-stochastic matrix (W E PL,N). We assume non- ton-itterfeeing Archetypes, meaning two orthoonolal archetypes cannot be active at the same time (but can exist in the bank of points A). Formally, for each row W,, the active rows of A (those Aj with Wij Wij > 0) are orthogonal, i.e..
G.3. Controlled Rank and Stability in A-SAE Dictionaries
We now show that the rank of the dictionary obtained using the Archetypal parametrization are inherently controlled. Specifically, the rank of the dictionary cannot exceed the rank of the data. This property is interesting as it prevents the dictionary from becoming arbitrarily complex, promot- ing solutions that are more structured and aligned with the data. Consequently, the dictionary is more likely to uncover meta-concepts or low-rank representations.
=0 for 1 and Wij. Wiy' > 0.
Then, the out-of-diittributin (OOD) measure for each D. admits the upper bound:
OOD(D,) $
max W iz-
jE[m]
Proposition G.3 (Rank Bound of Archetypal Dictionaries). Let A E Raxd be a data matrix with rank(A) = r S d, assuming n > d and lo >> d. Let W E Pu,n be a row- stochastic matrix, and define the dictionary as D WWA. A. Then, the rank of D is bounded by the rank of the data:
Proof. By definition, D. Σラ= Wi,
(D) =
Σ Wn Aj
Σ
Wn AA). k=1
rank(D) S min minnrank((A))) (A), rankk())W)) S d.
1=1
Under the orthogonality assumption (As, A,) = 0 for k j. only the k = j term remains:
Proof. Since D= = WA, the column space of D is con- tained in the column space of A, implying
A,) = W Wiy Aj).
rank(D) S rran(A)) =r.
Since || = 1, we have (Aj. A,) = 1, so:
Additionally, since We E R is row-stochastic, its rank is at most min(k, n), giving
(D., A,) W...
rank(W) < min(k,n).
By definition:
2
it follows that
n
|| |D =
Σ W
A
rank(D) = rank W A) s min(r, rank( (W)).
2
By the orthogonality of the active rows of A, the contribu-
27

---

Archetypal SAE
2.0
Rectangle
we evaluated several options, including Gaussian, Cauchy, and Silverman, DinoV2. As shown in Figure 13, the Silverman kernel consistently yielded the most stable and accurate reconstruction results. For our experiments, we selected the Silverman kernel with a bandwidth of 10-2, although the choice of kernel appears to have only a modest impact on performance.
Gaussian
Triangular
Cosine
1.5
Epanechnikov
Quartic
Silverman
Cauchy
1.0
I. Soft Identifiability Benchmark
0.5
In this appendix, we provide additional details regarding the experimental setup and evaluation criteria used in the Soft Identifiability Benchmark. We recall that the goal is to assess the ability SAEs to recover distinet concepts from synthetic image mixtures, where the underlying generative factors are known.
0.0
Figure 13. Example of kernel functions for JumpReLU over the interval [-3,3] 3] with a bandwidth of I. The Silverman kernel produced more stable and positive results, leading selection for all experiments with smaller bandwidth of 10
We generate twelve synthetic datasets using Midjourney API For each of these datasets, we programmatically cre- ate 4,000 images. These images are constructed by collaging four distinct objects selected from the predefined set, such as different types gems. Each datasct is generated from between and 20 unique objects, with the dictionary size set exactly the number of true generative factors (unique object number). An example of some datasets used in the benchmark is shown in Figure 14.
tions of different rows do not interfere, so:
|
Each dataset is split into a training set of 2,000 images and a test set of 2,000 images. The images are processed through pre-trained vision model, in our case Dino ResNet50, SigLIP and VIT. The resulting pooled activations serve as the input representations for the SAE.
Since ||A,||2 1, this simplifies to:
1 = W
Substituting these yields our bound:
Metrics. To quantitatively evaluate identifiability, we de- fine an accuracy metric that measures whether each object class in the dataset is correctly assigned distinct concept in the SAE dictionary. Given an image, we pass it through the vision model and then through the trained SAE to obtain a concept-label pair (z, y). where z E IR& represents the k learned concept activations, and E IR denotes the c ground-trute class labels.
(Di, Aj)
OOD(D:) max jE(m)
2
max Wij- jE(n]
Della
As notable special case, we observe that OOD(D,)
when Wij 1 for some [n)), the dictionary atom aligns perfectly with data point. In practice, the sparsity of W plays crucial role: it limits interference between (non- orthogonal) components, ensuring that the orthogonality assumption remains plausible and that the derived bounds hold robustly.
We define the accuracy for class as:
Accuracy,
max PZz,y) ((2 > A) =,),,
(18)
AER,iE(M]
where À is threshold determining whether concept is activated.
H. Kernel for JumpReLU
To find an appropriate X, we use the empirical percentiles of the concept activations Z, ranging from the 1st to the 100th percentile. This ensures that the threshold is adaptive to the distribution of activations, optimizing for the best classification accuracy.
JumpReLU (Rajamanoharan et al., 2024) is recently in- troduced activation mechanism for SAEs designed to opti- mize to sparsity by controlling the discontinuities of ReLU through a parameter 0. Its optimization relies on a kernel for density estimation. To assess the effect of kernel choice,
www.middjourne.ccmm
28

---

Archetypal SAE
Potions
Gems
Planets
Birds
Books
Cards
Candys
Flowers
Figure 14. Examples of synthetic datasets used in the Identifiability Benchmark.
1.1. Complete Results
The full set of results across all methods and datasets is pro- vided in Table 4. We also provide a comprehensive break- down, including per-dataset accuracy scores and additional analysis.
29

---

Archetypal SAE
Table 4. Accuracy Scores for Various Methods Across Models and Classes
Model
Method
Animals
Birds
Books
Candy
Cards
Cocktails
Flawers
Genss
Landscupes
Planets
Potius
Watches
Arg
KMcun
0.7679
0.7678
0,7715
0.7709
0.7124
0.5095
0.6670
0.8137
0.7147
0.7786
0.8304
0.7693
0.7678
ICA
0.8113
0,7867
0.8089
03162
0.8212
0.8296
0.5497
0.8669
0.2837
0.8002
0.8297
0.3058
0.8092
Spuns PCA
0.7919
0.8013
0.7939
0.8129
0.7126
0.8122
0.7733
0.8307
0.7996
0.7951
DINO
SauiNMF
0,8175
0.7916
0,8089
0,8018
0.850
0.5516
0.7116
08330
08554
0.7716
0,8023
0.830
0.8569
0.7979
ConserMMP
0.0077
0.7916
03015
0.7935
0.8142
0.5245
0.7116
0.830
0.7716
0,8023
03301
0.7978
0.777
Vanilla
0.7878
03087
03161
0.5468
0.7212
08518
0.7601
0.8262
03385
0.7977
03947
TupK
0.7906
0.7942
08104
0.253
0.8243
0.8417
0.7728
03501
0.7744
0.184
08387
0.8191
08135
Jump
0,8863
0.7956
0,220
03174
08121
0.821
0.2389
08418
0.7504
25899
0.8438
0.5042
08000
SAE
0.453
0.8413
0.9912
0.722
0.950
0,5901
0.5500
0.950
0.9677
0.9277
0.129
0.9606
09852
RA SAE
0,9412
0.9313
0.9703
0.9614
0,9085
0,5905
0,9613
0.9503
0.9666
0,9227
0.9065
0,9642
09:42
PCA
0.8349
0.8086
0.8556
0.3641
0.9622
0.8501
0.7908
0.8446
0.7814
0.803
0,8326
0.3134
0.201
KMeans
0,7670
0,7647
0,7720
0.1688
0,7669
0.5123
0.6418
08109
0.6964
0,7313
0,8109
0.7696
0.7624
ResNet
Spune PCA
ICA
0.8162
0.8231
0,859
0.3155
0
0,8661 1
0.5552
0.3501
0,830
0
1
0.330
0,8619
0.0355
03330
SemiNMP
Co
0.5259
0.8274
0.7648
08536
0,2999
03011
0 1
07009
08522
0
03480
09244
00000
03223
Vanilla
08143
03270
08:50
0330
Jump
TopK
0.8078
0.8027
0.888
08384
0.80
08776
0838
0
0
0.723
08/95
08161
0:000
0000
02200 2
A SAE
0,9633
0.9701
0.638
09673
02739
0
09722
09668
0.242
0,959
09811
19611
RA-SAE
0,9613
0.9577
0.9994
0.9994
0UTTU
03625
3.9640
(19029
0
095544
0:947
09902 1
PCA
0,8233
0,7957
0.8264
0.30/0
08157
0 82826
0.0367
0,820
0:7678
0.7981
0,8291
0,3261
03052
KMrans
0,7753
0,7672
0,7733
03500
0.742
0 8090
0.6600
08121
0.773
08801
0.7734
07684
ICA
0.8171
03354
0
0.3341
0.7248
08133
0.3420
0,8056
0.8362
0,515
03243
SigL.I
Spune
SamiNMF
0.8251
0.6186
0,7957
0.7962
0.413
0.8574
0.2264
081157
0,8152
0.3269
0
0.7300
020000
0,7911
0,870
0,7541
0.339
09291
0
Vanilla
CoNseNNMP
0.8340
0.7706
0.810
0.7706
03907
0.7769
0220
08000
08486
0:2100
02221
07718
0.2926
03433
02215
Tapk
0.110
08315
03450
08855
05000
0.227
02200
Jump
0.8199
0.831
0.846
08222
08367
0
10000
0.9843
0.466
0852
03386
A SAE
0,9727
0.9613
0.9177
09606
08733
08465
09622 02622
09333
0,9499
0.457
0.774
09602
RA-SAF
0.655
0.9654
0.9411
09661
0.9749
0,9366
0:0632
00594
0,9325
0.9546
0:860
PCA
0.7994
0.8107
0.8266
0.3258
0.7063
0.8352
0.7750
08224
0. 0.7556
0,8099
0.8199
0.3154
08005
KMeans
0.7705
0.7719
0.7744
0.37%6
0.7552
0.5095
0.6721
08134
0:1120
0.7841
0,8099
0:3722
0.7722
ICA
0.8251
0.4243
0.859
0.7956
Spane
0.8255
0.6343
0:5790
SemiNMP
0,8114
0.8107
0.9266
0,8226
0.3410
0.783
0.8661
0.7799
0.8133
08594
08174
020004
0.75%
0,8118
0,8492
0.8201
0.562
0.3423
CusseLNMF
0.7685
0.7644
07714
7889
0.8117
0.625
0.132
0.7736
0.1344
0.7758
07634
Vanilla
0.8545
0.459
0.224
03438
0.5481
0.2331
-
0.8249
0.2855
0.5387
0.323
Tapl
0.8494
0.84%)
0.8577
0.5001
0.086
08315
0.8100
0.5451
08378
Jump
0,8042
0.7995
0.8116
0.1116
08064
0.837)
0.6871
08915
0.7651
0.8143
0,8300
0.8172
0.803
A-SAE
0.9647
0.9847
0.9519
03838
0U733
0.9578
0.9954
294888
0.9584
0.9369
0.9325
0.9719
0.9615
RA-SAE
0,9699
0.9847
0.9683
0.9996
0,9694
0.9620
0.9376
0.9455
0.9251
0.9484
0.9253
0.9745
05956
30


Reviews:
[{"review_id": "Q8BCDfjkr2", "reviewer": "Reviewer_27og", "summary": "The paper proposes an extension of vanilla SAE approaches to archetypal SAE, a type of geometric anchoring that improves various shortcomings, stability and plausibility, of vanilla SAEs. Further, the authors introduce two new benchmarks for plausibility and identifiability. The paper thoroughly evaluates the contributions across a variety of SOTA feature extractors and demonstrates its effectiveness.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your thoughtful and constructive review! We appreciate your recognition of the paper’s strengths, including the methodological clarity, the novel evaluation metrics, and the thorough empirical analysis. Below, we address specific comments.\n\n---\n\n> **Clarification of Dataset Usage (Setup Section)**\n\nGood point! Based on our current phrasing of the setup, a reader may mistakenly conclude that the *vision backbones* were trained / fine-tuned on IN1K. This is not the case—the backbones (e.g., DINOv2, SigLip) were in fact pretrained models sourced from the 'timm' library. We specifically used IN1K to subsequently train the SAEs analyzed in this paper, computing activations from the backbone models, regardless of their pretraining method, for this purpose. We will ensure this is made clear in the final manuscript—thank you!\n\n---\n\n> **Inclusion of Qualitative Results in the Main Evaluation**\n\nWe strongly agree that our qualitative results, as teased in Figure 1, would be useful to include in the main paper! Unfortunately, space constraints forced us to defer our in-depth qualitative analysis to the appendix. In brief, we note that in **Figure 7** we showed \"exotic concepts\" discovered by RA-SAE on DINOv2, such as a 'barber' concept that uniquely activates for barbers (not clients), fine-grained concepts based on petal edges, and a shadow-based feature suggestive of 3D reasoning. Similarly, in **Figure 8** we highlighted RA-SAE’s ability to disambiguate a single ImageNet class (e.g., rabbit) into spatially localized subcomponents and fine-grained animal facial features, while in **Figure 10** we reveal emergent *concept clusters*, such as spatial-relational concepts (“underneath”) and fine-grained animal facial features.\n\nIn response to your comment, we promise to use the extra page provided for final manuscripts to pull a subset of these results back to the main paper.\n\n---\n\n> **On backbone pretraining approaches, e.g., Masked Image Modeling (MIM)**\n\nWe emphasize that we already analyze DINOv2 models, which are pretrained using iBOT, an MIM-based objective. That is, while we did not include a standalone MAE model, our experiments do include an MIM-trained model. We will clarify this point in the revised text by providing further details of the pipeline used for training backbone models analyzed in our work, hence also making explicit that we already analyze a backbone trained via MIM pretraining.\n\n---\n---\n\n**Summary:** Thank you again for the constructive feedback! We hope our responses help address your raised questions, and, in line with your suggestion, we promise to pull back qualitative results from the appendix to further improve the main paper. Given these changes, we would be grateful if you continue to champion our paper's acceptance!", "claims_and_evidence": "- The authors claim the current SAE approaches are instable and do a good job of showcasing this with experiments", "methods_and_evaluation_criteria": "- The paper in general and method description specifically is well structured and easy to follow\n- The formulation of the stability measure makes a lot of sense\n- The aspect of scalability is important and the proposed relaxation tackles this issue well", "theoretical_claims": "- The claims about limitations of standard SAEs in L270ff are backed up with a reference", "experimental_designs_or_analyses": "- The introduced relaxation parameter is intuitive and well ablated\n- In the setup section, the authors claim all models were trained on IN1K. I guess this is referring to their own tuning of the models, but it’s written a bit confusing. Please make it more clear this you are finetuning with IN1K. At the moment, it sounds like all models were trained on IN1K initially.\n- The quantitative evaluations are extensive and highlight the proposed method well\n- The paper starts with a qualitative teaser. However, the main evaluation section completely lacks a qualitative evaluation. It would be great to add that.\n- The proposed benchmarks are an interesting addition to the previous analysis", "relation_to_broader_scientific_literature": "The authors do a good job to put into perspective the previous approaches incl. SAEs and how they tackle these issues", "essential_references_not_discussed": "None", "other_strengths_and_weaknesses": "Strengths: \n- The paper  is well written and easy to follow\n- The stability measure makes sense and is well formulared\n- Its Great that the aspect of scalability is investigates for the method and the proposed relaxation achieves this property\n- The benchmark contributions are relevant to the Community \n\nWeaknesses:\n- The paper lacks qualitative evaluations in the Main text \n- The writing of what models were trained on what datasets is confusing\n- Only contrastive and vision language models were evaluated. Would be Great to also see masked image modeling approaches", "other_comments_or_suggestions": "Since my knowledge is this space is limited, I will take into account the judgement from the other reviewers to make a final decision", "questions_for_authors": "- The Experiments section only evaluates Contrastive, Vision-Language and supervised approaches. I think what would it be interesting to know is how good MIM-based approaches are like Masked Autoencoders [1] or iBOT [2]? I think they are compatible with timm\n- Please clear up the confusion about model training in the setup section\n- Would you be able to squeeze in qualitative results into the main evaluation section?\n\n[1] He, Kaiming, et al. \"Masked autoencoders are scalable vision learners.\" Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2022.\n[2] Zhou, Jinghao, et al. \"Image BERT Pre-training with Online Tokenizer.\" International Conference on Learning Representations.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "WdGIClFY6Z", "reviewer": "Reviewer_hoxe", "summary": "Sparse autoencoders (SAEs) are a promising unsupervised learning approach to find relevant and interpretable concepts of representations, e.g., for language or vision models. This paper argues that concepts extracted from SAEs are unstable when a fix model is trained multiple times on the same dataset or trained on similar datasets. The address this issue, the authors propose to constrain the dictionary atoms to reside on the boundary of the convex hull of data. This is the idea of archetypal analysis and the authors propose to combine archetypal analysis and SAEs yielding A-SAEs. In their paper, data actually refers to the embedding of image data produced by a vision model, e.g., DINOv2 or ViT. By having the dictionary atoms spread out yet close to data (archetypal analysis idea), the authors ague / demonstrate that their A-SAE\n- yields sparse reconstructions which are on-par with SOTA,\n- is plausible as it yields dictionary $D$ atoms are close to data,\n- has dictionary $D$ atoms that have meaningful directions,\n- maintain structure in the codes $Z$.\nThe authors conduct several experiments to verify their claims.\n\n## update after rebuttal\nBased on the author responses, the discussion, their clarifications, and their committment to improve the paper, I reconsidered my evaluation and raised my score.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the detailed review—we sincerely appreciate the rigorous and constructive evaluation! The suggested related work and the thoughtful questions are very helpful. While not all references pertain directly to SAEs, the community we attempt to reach, the body of work highlighted on archetypal analysis is highly relevant. We have taken this review seriously and respond to specific comments below.\n\n---\n\n> **On Related Work**\n\nThank you for the extensive list of relevant references! These papers provide valuable context and inspiration on Archetypal analysis and we promise to add a detailed discussion of these papers in our related work in the final version of the paper. However, we do want to emphasize that **the goal of our paper is not to advance the field of archetypal analysis per se, but to bridge it with the neural network interpretability community by defining SAEs inspired by it.** To this end, we note none of the suggested papers address neural network explainability directly (and most do not even mention it). For the interpretability community, these geometric priors have not been integrated with modern large-scale SAE pipelines, and our contribution lies in highlighting such ideas for large models’ interpretability.\n\nOverall, while we stand on the strong theoretical shoulders of the Archetypal analysis community, we note our work serves a different audience: researchers in interpretability and applied representation learning who benefit from the stability, structure, and plausibility introduced by archetypal anchoring, even if they do not come from the convex analysis or matrix factorization literature.\n\n---\n\n> **Clarifying Our Position Relative to Archetypal Analysis**\n\nBuilding on our response above, we would also like to clarify the precise relationship between our proposed Archetypal-SAE pipeline and Archetypal analysis more broadly. Our approach, Archetypal SAE, constrains the **dictionary** $D$ to be in the **convex hull of data**, i.e., $D = WA$ with $W \\in \\Omega_{k,n}$ (row-stochastic). However, the sparse code **Z is not constrained to be row-stochastic**, and hence we do not implement archetypal analysis in its classical manner; rather, we merely apply its geometric anchoring idea to the SAE decoder. Nevertheless, we call our proposed pipeline “Archetypal SAE” as an ode to the concept of Archetypal analysis, from which we derived the idea!\n\n\n---\n\n> **On Convex Hull Approximation and Anchor Subset (C)**\n\nPlease note we do not claim novelty in using K-Means for calculating a reduced subset of anchors. Our proposed Archetypal SAEs are in fact designed in a modular way, and we can expect better subset selection methods (e.g., your suggested ones) will only improve the results further! To this end, we also note here that we experimented with several viable methods when designing Archetypal-SAEs (e.g., Isolation Forest, One-Class SVM). In practice, we found K-Means to offer the best trade-off between **scalability, reconstruction, and stability**. Nevertheless, we believe your suggested methods will be exciting to experiment with in future! \n\n\n---\n\n> **On the Stability Evaluation (Claim 1)**\n\nWe emphasize we **do evaluate repeated SAE training runs** and report stability in **Figure 3** using the cosine alignment metric of Equation (2). Each point in our results is the mean over **four independently trained models with different seeds**, and the Hungarian matching is applied post hoc to align dictionary atoms. We will highlight this better in the final version of the paper!\n\n---\n\n> **Clarifications and Revisions**\n\nWe will revise the paper to address the points you've raised:\n\n- Fourth claim: Our dataset will be openly released upon acceptance. \n- Clarify that `data’ refers to **model activations**, not input images.\n- Make recommended notation changes (e.g., D vs D′, A vs A′), capitalization, etc.\n- Improve figure captions as recommended (e.g., stability computation in Fig. 3).\n- Clarify classifier usage in a plausibility benchmark (we use the backbone's own classification head).\n- Address missing definitions (e.g., Lambda, Z, etc.).\n- Explain dataset standardization: activation are taken after layer norm (without the affine part), thus each activation \\(mean(a) = 0, std(a) = 1 \\).\n\n- **Training dataset for SAEs:** We note all SAEs (and NMF variants) were trained on **ImageNet-1K activations**, using ∼250M tokens depending on architecture.\n\n- **Initialization of K-Means:** We used K-Means++ with mini-batch training for scalability. \n\n---\n---\n\n**Summary.** Thank you again for the time and care you put into your review. We believe that your feedback substantially improved the clarity, rigor, and positioning of our work. We will incorporate your suggestions into the final version, including expanded citations and clearer technical exposition.", "claims_and_evidence": "- ~~The first contribution/claim *\"We identify a critical limitation of current SAEs: dictionaries are unstable across different training runs, compromising their reliability.\"* (lines 90-92) does not seem to be evaluated properly since I do not find any information on repeated/different training runs (different initialization) anywhere in the paper. This can also be seen in the tables which do not mention any measures of centrality (mean, median) and dispersion (std). An exception is Table 3 which mentions an average.~~\n- ~~Within their second contribution/claim (lines 93-95), they apply parts of archetypal analysis within the context of sparse autoencoders. Parts, because only the dictionary is constrained to be within the convex hull of data. In archetypal analysis, also the reconstruction is constrained to be within the convex hull of the dictionary which forces the dictionary to reside on the boundary of the convex hull of data.~~\n- Moreover, the proposed relaxed version (line 95) seems to already exist. The proposed usage of a reduced subset is also not novel. (For both, see Essential References Not Discussed).\n- ~~As for their fourth contribution/claim (lines 99-101), they construct a new dataset but it is unclear if it will be openly released.~~", "methods_and_evaluation_criteria": "- The main idea is to constrain the dictionary $D$ to reside within the convex hull of the embedding $A$. For this, a similar idea to archetypal analysis is used. However, due to scalability issues, a smaller version $C$ instead of $A$ is used. The authors argue for $K$-Means (line 302) and argue it is the most effective method (lines 1057-1058).\n\t- $K$-Means will select **cluster centers that lie inside the convex hull of data**. Thus, **the convex hull of $C$ will be smaller in volume than the convex hull of $A$**. This cannot be wanted. The difference in volume should be as small as possible to achieve a good approximation of $conv(C)=conv(A)$ (see also line 266).\n\t- There are many neglected ways of computing $C$ in a better and faster way yielding larger convex hulls of $C$ than $K$-Means, for example:\n\t\t- Damle, Anil, and Yuekai Sun. \"A geometric approach to archetypal analysis and nonnegative matrix factorization.\" Technometrics 59, no. 3 (2017): 361-370.\n\t\t- Mair, Sebastian, and Ulf Brefeld. \"Coresets for archetypal analysis.\" Advances in Neural Information Processing Systems 32 (2019).\n\t- Technically, even the initialization method FurthestSum can also be used to construct $C$:\n\t\t- Mørup, Morten, and Lars Kai Hansen. \"Archetypal analysis for machine learning and data mining.\" Neurocomputing 80 (2012): 54-63.\n\t- **I believe that due to the smaller convex hull of $C$ (which is due to $K$-Means), the proposed relaxed variant is needed. If $C$ is chosen in a better way, the relaxation might be superfluous.**", "theoretical_claims": "- Proposition F.1: Those statements are rather obvious and known.", "experimental_designs_or_analyses": "- I do not find any information on repeated/different training runs (different initialization) anywhere in the paper. This can also be seen in the tables which do not mention any measures of centrality (mean, median) and dispersion (std).\n\t- The exception is Figure 3.\n- I wonder why archetypal analysis is not a baseline.\n- The experimental setup appears to be sound otherwise.", "relation_to_broader_scientific_literature": "- SAEs are extended by ideas from archetypal analysis.", "essential_references_not_discussed": "- The related work mentions many works about sparse coding and dictionary learning but **actually surprisingly little about archetypal analysis**. I expected to see at least the **works that combine archetypal analysis and autoencoders (or more generally neural networks)**, i.e.,\n\t- Wynen, Daan, Cordelia Schmid, and Julien Mairal. \"Unsupervised learning of artistic styles with archetypal style analysis.\" Advances in Neural Information Processing Systems 31 (2018).\n\t- van Dijk, David, Daniel B. Burkhardt, Matthew Amodio, Alexander Tong, Guy Wolf, and Smita Krishnaswamy. \"Finding archetypal spaces using neural networks.\" In 2019 IEEE International Conference on Big Data (Big Data), pp. 2634-2643. IEEE, 2019.\n\t- Keller, Sebastian Mathias, Maxim Samarin, Mario Wieser, and Volker Roth. \"Deep archetypal analysis.\" In German Conference on Pattern Recognition, pp. 171-185. Cham: Springer International Publishing, 2019.\n\t- Keller, Sebastian Mathias, Maxim Samarin, Fabricio Arend Torres, Mario Wieser, and Volker Roth. \"Learning extremal representations with deep archetypal analysis.\" International Journal of Computer Vision 129 (2021): 805-820.\n- Lines 263-267: Specifically for archetypal analysis, **this was also/already shown** (along a way on how to compute $C$) in\n\t- Mair, Sebastian, Ahcene Boubekki, and Ulf Brefeld. \"Frame-based data factorizations.\" In International Conference on Machine Learning, pp. 2305-2313. PMLR, 2017.\n- Line 301 and Appendix D: There is actually **quite some related work on using a reduced subset $C$** instead of the full $A$ matrix which seems to be missing or neglected:\n\t- Mørup, Morten, and Lars Kai Hansen. \"Archetypal analysis for machine learning and data mining.\" Neurocomputing 80 (2012): 54-63.\n\t- Mair, Sebastian, Ahcene Boubekki, and Ulf Brefeld. \"Frame-based data factorizations.\" In International Conference on Machine Learning, pp. 2305-2313. PMLR, 2017.\n\t- Damle, Anil, and Yuekai Sun. \"A geometric approach to archetypal analysis and nonnegative matrix factorization.\" Technometrics 59, no. 3 (2017): 361-370.\n\t- Mair, Sebastian, and Ulf Brefeld. \"Coresets for archetypal analysis.\" Advances in Neural Information Processing Systems 32 (2019).\n\t- Han, Ruijian, Braxton Osting, Dong Wang, and Yiming Xu. \"Probabilistic methods for approximate archetypal analysis.\" Information and Inference: A Journal of the IMA 12, no. 1 (2023): 466-493.\n\t- Mair, Sebastian, and Jens Sjölund. \"Archetypal Analysis++: Rethinking the Initialization Strategy.\" Transactions on Machine Learning Research (2024).\n- Lines 313-319: **A relaxation of archetypes was already proposed** in\n\t- Mørup, Morten, and Lars Kai Hansen. \"Archetypal analysis for machine learning and data mining.\" Neurocomputing 80 (2012): 54-63.\n- Line 302: If $A$ is normalized and also considering the assumption for measuring instability in Equation (2), the following paper seems to be relevant:\n\t- Mei, Jieru, Chunyu Wang, and Wenjun Zeng. \"Online dictionary learning for approximate archetypal analysis.\" In Proceedings of the European Conference on Computer Vision (ECCV), pp. 486-501. 2018.\n- Lines 1047-1051: **Computing the convex hull on reduced dimensions is also not new**:\n\t- Thurau, Christian, Kristian Kersting, and Christian Bauckhage. \"Convex non-negative matrix factorization in the wild.\" In 2009 Ninth IEEE International Conference on Data Mining, pp. 523-532. IEEE, 2009.\n\t- Bauckhage, Christian, and Christian Thurau. \"Making archetypal analysis practical.\" In Joint Pattern Recognition Symposium, pp. 272-281. Berlin, Heidelberg: Springer Berlin Heidelberg, 2009.", "other_strengths_and_weaknesses": "- The paper is mostly very well written.\n- However, the clarity of some parts should be improved. Here are some examples:\n\t- Figure 1: At this point, the reader is unfamiliar with the notation $v$. Moreover, it is unclear what the three images per run and model represent. Please improve the caption.\n\t- Lines 139-143: Why different $k$'s, i.e., $k$ and $K$? Please also state what $k$ refers to/means. Currently, it is only done implicitly but the readability of the paper can be improved by stating its meaning clearly.\n\t- I think SAEs should be better formally introduced.\n\t- Line 256: Is $A \\in \\mathbb{R}^{n \\times d}$ the data matrix in input space or is it the result of the function $f$ (line 114), i.e., not the data matrix but rather the latent-representation matrix?\n\t- Line 265: Here, $D$ is $k \\times d$ which is different from line 138. Why?\n\t- Line 273: The reconstructions used to be $ZD^T$ (line 137).\n\t- Line 315: The variable $\\Lambda$ is undefined.\n\t- Line 302: I think the reader has a hard time understanding that the $A$ matrix is a result from the five evaluated model and that the SAE is trained on the embedding of $A$, especially when the reader has the standard autoencoder idea in their head.\n\t- Lines 355-361: Where does a classifier suddenly come from? How does it look like, how was it trained, and on what data?\n\t- Line 428: I still dislike \"within the data's convex hull\" since it is not the data $X$ but rather the embedding $A$. The reader might think about $X$ when reading \"data\".\n\t- Figure 6: Labels for the $x$ and $y$-axes are missing.\n\t- Figure 8: How was $A$ projected to two dimensions for the sake of this visualization (point cloud)? How were the points colored? What does the vertical band on the right-hand side with added transparency mean? \n\t- Equation (7): If $A=ZD$ as in line 955, then $D$ is a $k \\times d$ matrix. There is no $n$, but $n$ is used in Equation (7).\n\t- Line 1091: Why $m$ and not $n'$?", "other_comments_or_suggestions": "- Inconsistent capitalization. Why is archetypal analysis capitalized in line 22 but not in line 80? Why is neural network capitalized (line 41)?\n- Line 136: Equation (1).\n- Lines 157-160: Backpropagation is computing a gradient for gradient-based optimization and gradient-based optimization can usually also be phrased in terms of mini-batch updates. I do not see how this is an argument against traditional methods and in favor of SAEs.\n- Lines 161-185: Neural networks are also learned in multiple steps. Again, I do not see the argument.\n- Line 190: Pareto\n- Lines 202-203: The line spacing seems to be off.\n- Figure 2: All we see is that with non-linear approaches we get lower errors than with linear approaches across various sparsity levels. This has nothing to do with scalability. (Why is Loss capitalized?)\n- Line 191: Rephrase to $D, D' \\in \\mathbb{R}^{n \\times d}$.\n- Line 191: Why is $D \\in \\mathbb{R}^{n \\times d}$? Consider lines 127-138. If $A \\in \\mathbb{R}^{n \\times d}$, then $Z \\in \\mathbb{R}^{n \\times k}$ and $D \\in \\mathbb{R}^{d \\times k}$.\n- Lines 328,368: Tab. 1 vs Table 2\n- Tables 1,2,3: Table captions go above tables. This way the caption can also be easier visually separated from the text, see lines 353-354 for an example.\n- Lines 353-356: Equation (18) upper bounds the error, no?\n- Figure 5: In 2., it appears visually that the SAE reconstructs $X$ but it reconstructs $A$!\n- Line 437: Why capitalized?\n- Please unify the style of the references: arXiv paper have different styles, sometimes conferences come with abbreviations, sometimes not, sometimes just the abbreviation is provided, line 658 is missing the conference, sometimes arXiv is stated although the paper is published, etc.\n- Line 770: Section A is empty.\n- Lines 793-796: What is $k$?\n- Lines 1071-1080: Why are there dashes?", "questions_for_authors": "1. Why is archetypal analysis not a baseline?\n2. Figures 2,3,4: What was $k$ in those experiments?\n3. For the stability in Equation (2), the dictionary $D$ is assumed to reside on the sphere. Where and how is that done? Is $D$ always forced to be on the sphere or just normalized for computing the stability? Please clarify.\n4. Lines 1109-1114 mention that $D$ is constrained to be on the sphere. This is not for A-SAE but rather an argument for standard SAEs, right?\n5. Figure 3: Do all 5 points of A-SAE overlap? What does the dashed line indicate? Were all dictionaries normalized to lie on the sphere for the sake of computing the stability?\n6. Line 302: You state that ImageNet has 1.2M points and make a similar comment in lines 1078-1082. Did you use any special variant of $K$-Means? How did you initialize $K$-Means?\n7. Line 269: I suggest to remind the reader where $Z$ comes from for your A-SAE.\n\t- **Assuming that $Z$ is like in archetypal analysis**, i.e., $Z$ is not only non-negative but also row-stochastic (each row sums to one). This is correctly stated in lines 244-245: \"representing each data point as a convex combination of Archetypes\".\n\t\t- Lines 81-82: Archetypal analysis constrains the archetypes to lie **on the boundary of the convex hull** of data (exception: only one archetype)! See Proposition 1 in Cutler & Breiman (1994). Stating that the atoms are **within the convex hull** is thus wrong.\n\t\t- Lines 239-245: Please rephrase this. This geometric flexibility will reduce the reconstruction error of archetypal analysis drastically. However, if I perturb my data or retrain with random initialization, my latent space can be sufficiently different such that the location and the meaning of archetypes change as well.\n\t\t- Line 253: Again the issue with lying within the convex hull.\n\t\t- Line 268: The statement \"each archetype originates from the data\" is misleading. An archetype resides on the boundary of the convex hull of the data and an archetype does not have to be a data point, it is rather a convex combination of data.\n\t- **Assuming that $Z$ is as in Equation (1) (line 201), i.e., $Z \\geq 0$**, then it should be mentioned clearly because only a few sentences before you state how archetypal analysis is doing it and everyone familiar with archetypal analysis reading this paper will be confused.\n\t- Can you clarify already now?\n8. Line 291: \"Models were trained on the ImageNet dataset\" means that the five evaluated models were trained on ImageNet, but where were the SAE and NMF variants trained? Same dataset? Please clarify.\n9. Line 302: \"The data matrix $A$ was element-wise standardized.\" What does that mean?\n10. Tables 1,2,3: Are the results from a single run or were they averaged (how, over how much repetitions, why no measure of spread like std or stderr)?\n11. Table 1: Why is A-SAE missing and what was the parameter for RA-SAE? How was it chosen?\n12. Figure 10: I wonder what the green line \"Convex_hull\" refers to. It could be the vertices of the convex hull of $A$ which yields $conv(A)=conv(C)$, thus the error would be as good as for $A$ and thus the best line. However, knowing about the complexity of computing convex hulls, I do not believe that the authors computed a convex hull on 1M data points in a space with more than a handful (25) dimensions.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "PJtlbewgDn", "reviewer": "Reviewer_1Jmo", "summary": "The authors find that current SAE architectures (ReLU, Jump-ReLU, TopK) exhibit instability: the learned concepts differ between runs, even on the same data. They measure this with a new metric:\n$$\\text{max}_{\\Pi} \\frac{1}{n} \\text{Tr}(D^\\intercal \\Pi D')$$\n\nWhere $\\Pi$ is the optimal alignment between $D$ and $D'$. Compared to other dictionary learning methods, SAEs are significantly less stable, but achieve much better sparsity and reconstruction error. To mitigate this instability, the authors propose Archetypal SAEs (and eventually Relaxed Archetypal SAEs) which constrains the decoder matrix $W_\\text{dec}$ such that each row of $W_\\text{dec}$ is a convex combination of the rows of $A$ (the original activation set). A-SAEs and RA-SAEs achieve significantly improved stability, and RA-SAEs achieve similar reconstruction error and sparsity metrics as current SAE architectures.\n\nThe authors validate their findings on a variety of vision transformers and CNNs on ImageNet, then introduce a synthetic benchmark to identify if SAEs (both existing and their proposed A-SAE and RA-SAE) can reliably identify core features.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for the detailed and thoughtful review! We appreciate your engagement. Below, we address specific comments.\n\n---\n\n> **Stability, why It matters, and how we measure it**\n\nWe agree that interpretability—not reconstruction or sparsity per se—is the end goal of SAEs. However, we argue stability is *necessary* for interpretability to be meaningful: if concepts identified across SAE training runs vary drastically (as we show in Fig. 1, 3), then, since the basis along which explanations are developed (i.e., the concepts) changes, our ''explanations'' of how a model produces its outputs will change drastically as well. This raises the question whether we are identifying the ''right'' explanations in the first place.\n\nTo this end, we emphasize that we see our paper as a first step towards identifying and addressing the broader challenge of instability in SAE training: grounded in classical dictionary learning literature, we offer a reasonable metric to quantify how dissimilar concepts identified via SAEs across training runs are, and propose a plausible solution to the problem (archetypal SAEs). However, we do not intend to suggest our defined metric for measuring instability is optimal—we merely argue that, to the extent linearly encoded concepts are identified by SAEs, measuring average cosine similarity after optimal matching is a reasonable notion of instability. We are certain useful metrics will be defined by future work, and we will make sure to emphasize this as a possible future direction in the final version of the paper.\n\n---\n\n> **On the OOD Metric (Appendix C.2)**\n\nThank you for the feedback! We will better clarify the motivation for the OOD metric in the final version of the paper. In brief, our argument is that if learned concepts lie entirely in the nullspace of real activations, they cannot possibly affect downstream layers. Thus, concept directions should maintain nontrivial similarity with data activations.\n\n---\n\n> **On the Plausibility Benchmark (Section 5.2)**\n\nWe emphasize our motivation for proposing the plausibility benchmark was to evaluate SAEs’ ability to identify interpretable features with a natural dataset—since it is hard to define ground truth for such an evaluation, we proposed the use of class label as a reasonable ground-truth to evaluate how interpretable concepts identified via SAEs are. However, we do *not* argue a score of 1.0 should be achieved on this evaluation—SAEs should not reduce to mere linear classifiers. Our argument is merely that given that class-labels of a dataset can be deemed as coarse concepts, we should expect at least some dictionary atoms to (partially) model such class hyperplanes. We will clarify this motivation in the final version of the paper.\n\nOn reporting of results with a dictionary of size 512 on this benchmark: we note our goal here was to evaluate how results are affected with scaling of the dictionary size, i.e., size 512 is just one of 7 configurations we report on the benchmark. We nevertheless agree that since ImageNet is 1K classes, other reported configurations in this table are more meaningful—we will hence remove the 512 one.\n\n---\n\n> **On the Soft Identifiability Benchmark (Section 5.3)**\n\nThis benchmark offers a synthetic analogue of our plausibility benchmark, where an SAE’s ability to model more fine-grained concepts is evaluated by defining a setting with known generative factors. Next to a full-fledged human study to study interpretability of identified concepts, we argue such a setting offers meaningful insight into different SAEs’ ability to identify interpretable features. \n\nClarification about evaluation setup: The setup used in this benchmark (dictionary size = true factor count, threshold sweep for λ)  is designed to provide an upper bound on performance by offering a maximally generous setting for all SAEs—not an operational, deployment setting. Meanwhile, the evaluation score is defined as the percentage of generative factors represented by an SAE in its dictionary atoms. \n\n---\n\n> **On Model Training Details and Hyperparameters**\n\nThank you for catching this—detailed information about training protocol has now been added to Appendix D (summarized below).\n\n- SAEs’ training: 50 epochs with batch size 4096.\n- Learning rate: 5e-4 with cosine decay and linear warmup.\n- Activations’ location: penultimate layer, the layer used for downstream tasks (after final layer norm for DINOv2, SigLIP, ConvNeXt; no norm for ResNet50).\n- Vanilla SAE uses λ = 1.0 only when batch sparsity is higher from target else 0.0 (measured per-batch).\n- JumpReLU penalizes thresholds $\\theta$ using a Silverman kernel.\n\n---\n\n> **Suggested refs**\n\nThank you—we have added citations to both!\n\n---\n\n> **Qualitative Results**\n\nWe are glad you enjoyed the qualitative examples in Appendix B! We promise to use the extra page offered for the camera-ready version to showcase additional qualitative clusters and concept visualizations.", "claims_and_evidence": "The authors claim that instability is a problem to be solved. Intuitively, I agree that stability between training runs is a desired property. Different random seeds should not lead to wildly different outcomes. However, I am not convinced that the proposed stability metric is the best way to measure this. At their core, SAEs are a method to interpret large pre-trained neural networks. While reconstruction error and sparsity are two metrics, nearly all prior work admits that these metrics are mere proxies for the true goal of \"interpretability.\" Similarly, I feel that the proposed stability metric does not really reflect the downstream goals of SAEs: to interpret neural networks. \n\nCan you provide a convincing argument that the instability in existing SAE architectures prevents us from reliably interpreting neural networks?\n\n\nSecond, could larger dictionaries solve instability?\nAs the size of the dictionary approaches infinity, if the dictionary avoids duplicating rows, then stability will eventually be solved.\nWhat about simply as $k$ goes from 1K to 32K?\nHow does stability change?", "methods_and_evaluation_criteria": "ViTs, CNNs, and ImageNet all make sense as an evaluation benchmark.\nIn fact, extending to CNNs is very interesting and I would be interested in the details.\nIt would of course be better to have another dataset (iNat2021?) to validate that these methods don't apply only to ImageNet but it's sufficient.\n\nIn terms of evaluation criteria, I am unsure the OOD score in C.2 is well motivated.\nWhy would I want my dictionary atoms to match real activations?\nIntuitively, what's important is that the dictionary atoms can be reliably *composed* to match real activations, not that each atom actually matches an activation.", "theoretical_claims": "I unfortunately am unable to check the theoretical claims in Appendix F.", "experimental_designs_or_analyses": "I don't understand the motivation behind the plausibility benchmark (Section 5.2).\nWhy would we want SAEs to recover true classification directions?\nFurthermore, why would we use $k=512$ when the models have more than 512 dimensions AND ImageNet-1K has 1000 classes?\nAgain, we use SAEs to interpret concepts internal to the network, not to measure their task-specific utility (classification).\nThere is no reason for an SAE with $k=32K$ to choose concepts aligned with the linear classifier.\nThe decomposed features could *compose* into a concept needed by the linear classifier.\n\nI am unconvinced that the soft identifiability benchmark (Section 5.3) makes sense.\nIt feels very arbitrary: sythetic images (OOD for the ViTs), a dictionary that has a fixed size that's equal to the true number of underlying concepts and a tuned threshold $\\lambda$ for classification.\nThe scores are also not very convincing: RA-SAE achieves 0.94 for DINOv2 and Vanilla achieves 0.80.\nWhat is a good score?\nWhat is a bad score?\n\nI would be unable to reproduce the experiments based solely on the main text and appendix, but the design set out in the main text is reasonable.\nThe only concern I have is $k=5$.\nPrior work in SAEs for LLMs often uses much larger expansions.\n\nI have the following questions:\n\n* How do you choose the tradeoff term $\\lambda$ for balancing between reconstruction error and sparsity for vanilla SAEs?\n* What other hyperparamters for training do you use (learning rate, LR warmup, activation normalization, etc)?\n* What layer of the vision models do you record?", "relation_to_broader_scientific_literature": "This paper identifies a core problem in dictionary learning (stability) and proposes a new method, taking inspiration from prior work where necessary. I am satisfied with the presentation of related scientific literature.", "essential_references_not_discussed": "N/A. Concurrent work in this area of SAEs for vision models might be nice to cite but not necessary.\n\n* Sparse Autoencoders for Scientifically Rigorous Interpretation of Vision Models (https://arxiv.org/abs/2502.06755)\n* Universal Sparse Autoencoders: Interpretable Cross-Model Concept Alignment (https://arxiv.org/abs/2502.03714)", "other_strengths_and_weaknesses": "The related work is an excellent list of papers about sparse coding and traditional approaches to extracting concepts from distributions of activations.", "other_comments_or_suggestions": "The spacing between table captions and text is too small.\nPlease add some additional space back to make the paper more readable.", "questions_for_authors": "No questions.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "QMduHBhHB8", "reviewer": "Reviewer_DiCd", "summary": "The paper points out the issue of feature instability in SAE training and designs the A-SAE and RA-SAE methods. These methods restrict the SAE feature Z using an archetypal dictionary D, resulting in SAEs with more stable features. The paper also introduces metrics for evaluating SAEs: (i) sparse reconstruction, (ii) plausibility, (iii) structure in the dictionary (D), and (iv) structure in the codes (Z). In summary, RA-SAE matches classical SAEs in reconstruction while significantly improving stability, structure, and alignment with real data.\n\n## update after rebuttal\n\nThanks for your rebuttal. I'd like to keep my score as my concern in Experimental Designs Or Analyses is still remains. What I care most is whether this SAE can work properly in LLMs and VLMs, which takes the stage of Interpretability research for their complexity and unexplanation. So I thank it's lack of novelty and practical significance that only applies current experiment settings.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for the feedback! We are glad you found the stability contributions and mathematical formulation of Archetypal SAEs to be novel and well-motivated. Below we address specific comments.\n\n---\n\n> **On Diversity of Analyzed Pretrained Models**\n\nWe agree that evaluating across a range of model types is important! However, we emphasize that our analyzed models already cover both **Transformer architectures** (DINOv2, ViT, SigLIP) and **convolutional ones** (ConvNext and ResNet50). Further, we highlight SigLIP is a **vision–language model**, i.e., it is trained with text supervision. Thus, our experiments *already* include Transformer models and multi-modal pretraining pipelines. We promise to highlight this diversity of evaluations in the final version of the paper.\n\n---\n\n> **On Relation to TopK SAE and Prior Work**\n\nWe appreciate your reference to prior work such as \"Scaling and Evaluating Sparse Autoencoders.\" However, we would like to clarify that our **contributions are orthogonal** to the specific choice of SAE variant. Archetypal SAEs are **a general framework** that can be applied on top of *any* SAE architecture, including Vanilla, TopK, JumpReLU, or BatchTopK. That is, we are not proposing another variant of TopK SAEs, nor analyzing TopK specifically. Instead, we introduce a geometric anchoring method that complements existing sparsity-inducing architectures by improving stability and structure. We will revise the paper to clarify this distinction more directly.\n\n---\n\n> **On Qualitative Interpretability Results**\n\nWe fully agree with this comment: qualitative interpretability is critical! In response to this point and similar comments from other reviewers, we have **moved several qualitative results from the appendix into the main paper**, including concept clusters (Fig. 10), fine-grained decompositions of ImageNet classes (Fig. 8), and exotic concepts in DINOv2 (Fig. 7). These changes will be reflected in the next paper update, where we plan to *use the final page of the camera-ready version* to showcase such qualitative examples across models and datasets, highlighting the types of structured concepts discovered by RA-SAE. We thank you for encouraging more emphasis on this front.\n\n---\n\n> **On Pseudocode**\n\nThank you for this helpful suggestion! We have now included a **detailed pseudocode block in Appendix D** showing the full training procedure for RA-SAE, including convex constraint handling, relaxation updates, and sparse reconstruction. We have also added a minimalistic version of the pseudocode to the main paper. These changes will be reflected in the next paper update.\n\n---\n\n> **On Ablation Studies**\n\nThank you for the comment! Unfortunately, we could not understand precisely which ablations the reviewer intended to suggest for us to analyze—if the reviewer can clarify this point, we are happy to add relevant experiments to the final paper. For now, we emphasize that our method introduces **only one novel hyperparameter**—the relaxation parameter δ (lambda). We already provide extensive ablations for the same in Fig. 4 and Table 2 (plausibility benchmark). Additionally, we also provide experiments titrating following settings:\n- Different **dictionary sizes** (k from 512 to 32K),\n- Multiple **vision architectures** (5 total, with both uni- and multi-modal pretraining),\n- Multiple **SAE baselines** (Vanilla, TopK, JumpReLU),\n- Different **distillation strategies** for the convex anchor set (Appendix C.1).\n\nPlease let us know if there is a specific titration / ablation experiment you believe would help strengthen our work—we would be happy to add it to the final paper.\n\n---\n---\n\n**Summary:** Thank you again for the constructive feedback. We hope our responses help address your raised questions, and our changes to the paper, e.g., the inclusion of pseudocode and qualitative results in the main paper, merit score increase inline with your comments! Please let us know if you have any further questions.", "claims_and_evidence": "Yes.", "methods_and_evaluation_criteria": "Yes.", "theoretical_claims": "Yes, no obvious theoretical issues were found in the paper.", "experimental_designs_or_analyses": "Yes. The experiment used five models (DINOv2, SigLip, ViT, ConvNeXt, ResNet50), but most of these models are based on convolutional neural networks (CNN) or self-supervised learning methods. It is unclear whether they represent all types of models. For instance, models with completely different architectures (such as Transformers) may yield different results. I would expect the paper to include supplementary experiments on smaller transformer models, such as Gemma-2B or Llava-7B, even if these experiments are conducted in a pure text modality. I will consider **raising the score** based on this point.", "relation_to_broader_scientific_literature": "The main idea of the paper and the benchmark method is related to the proposal and evaluation of TopK SAE in \"Scaling and evaluating sparse autoencoders.\"", "essential_references_not_discussed": "No.", "other_strengths_and_weaknesses": "Strength: This paper propose a new-type SAE with stable feature dictionary and provide corresponding math proof, which is a point with novelty.\n\nWeaknesses: A logic problem of this paper is that it doesn't do some interpretability experiment which showing the strength of the Archetypal SAE. Yes I find some visualizing result and case study in appendix, but I propose that more experiment in different dataset and model will be better. Also, this paper is lack of ablation study. If this aspect were improved, I would consider raising the score.", "other_comments_or_suggestions": "In sections 4. Towards Archetypal SAEs and 5. Experiments, the paper does not clearly describe the specific **training process** of the model. I would expect the inclusion of pseudocode or diagrams to help readers better understand the training process. If this aspect were improved, I would consider raising the score.", "questions_for_authors": "No.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: **Strengths:**

(1) This paper tackles the instability problem of sparse autoencoders and proposes a principled solution via archetypal constraints.

(2) Extensive experiments demonstrate improved stability, structure, and plausibility of the learned concepts across multiple vision backbones (e.g., DINOv2, ConvNeXt, SigLIP), which are supported by well motivated benchmarks for plausibility and identifiability.

**Weaknesses:**

(1) The paper does not evaluate on LLMs or vision-language models in the spirit of interpretability (beyond SigLIP), which is limiting the generality of the proposed method.

(2) The proposed stability metric is innovative, but its connection to interpretability is not convincingly established.

**Discussion:**

There is general agreement among reviewers that the paper addresses an important and under-explored challenge in interpretable representation learning -- stability of concept extraction via sparse autoencoders. While reviewers 27og and hoxe upgraded their evaluations after rebuttal, reviewer 1Jmo remained skeptical about propose evaluation metrics, and reviewer DiCd held reservations about the paper’s applicability to more complex, contemporary architectures (e.g., LLMs, VLMs). That said, all reviewers acknowledged the paper’s strengths, and most concerns were well-addressed in the rebuttal.

**Recommendation: Accept**

Despite some open questions about evaluation framing and scope of generalization, the paper offers a clear and well validated contribution with promising implications for concept-based interpretability. Its improvements in stability, coupled with strong empirical results, make it a valuable addition to the ML community.
Decision: Accept (poster)