From Flat to Hierarchical: Extracting Sparse Representations with Matching Pursuit
Authors: Valérie Costa
Thomas Fel
Ekdeep Singh Lubana
Bahareh Tolooshams
Demba E. Ba
Venue: NeurIPS 2025 poster ()
Keywords: Interpretability
Explainability
Computer Vision
Representation Learning
Dictionary Learning
Sparse Autoencoders
Primary area: social_and_economic_aspects_of_machine_learning
TL;DR: Interpretability methods based on linear, orthogonal features fall short for modern neural representations, which are often hierarchical and nonlinear. Better results come from aligning methods with the true structure of these representations.

Abstract: Motivated by the hypothesis that neural network representations encode abstract, interpretable features as linearly accessible, approximately orthogonal directions, sparse autoencoders (SAEs) have become a popular tool in interpretability literature. However, recent work has demonstrated phenomenology of model representations that lies outside the scope of this hypothesis, showing signatures of hierarchical, nonlinear, and multi-dimensional features. This raises the question: do SAEs represent features that possess structure at odds with their motivating hypothesis? If not, does avoiding this mismatch help identify said features and gain further insights into neural network representations? To answer these questions, we take a construction-based approach and re-contextualize the popular matching pursuit (MP) algorithm from sparse coding to design MP-SAE—an SAE that unrolls its encoder into a sequence of residual-guided steps, allowing it to capture hierarchical and nonlinearly accessible features. Comparing this architecture with existing SAEs on a mixture of synthetic and natural data settings, we show: (i) hierarchical concepts induce conditionally orthogonal features, which existing SAEs are unable to faithfully capture, and (ii) the nonlinear encoding step of MP-SAE recovers highly meaningful features, helping us unravel shared structure in the seemingly dichotomous representation spaces of different modalities in a vision-language model, hence demonstrating the assumption that useful features are solely linearly accessible is insufficient. We also show that the sequential encoder principle of MP-SAE affords an additional benefit of adaptive sparsity at inference time, which may be of independent interest. Overall, we argue our results provide credence to the idea that interpretability should begin with the phenomenology of representations, with methods emerging from assumptions that fit it.

From Flat to Hierarchical: Extracting Sparse Representations with Matching Pursuit
Valérie Costa |+
Thomas Fel Bahareh Tolooshams 5.01
Ekdeep Singh Lubana 3.4
Demba 2
LEPFL
Kemmpee Institute, Harvard University
CCB-NTT Program in Physics of Intelligence, Harvard University
*Physics of Artificial Intelligence Group, NTT Research, Inc., Sunnyvale, CA, USA
Univeesiity Alberta 6 Alberta Machine Intelligence Institute (Amii) "SEAS, Harvard University
Abstract
Motivated by the hypothesis that neural network representations encode abstract, interpretable features as linearly accessible, approximately orthogonal directions, sparse autoencoders (SAEs) have become popular tool in interpretability literature. However, recent work has demonstrated phenomenology that lies outside the scope of this hypothesis, showing signatures of hierarchical, nonlinear, and linlii-dimesioonl features. This raises the question: do SAES represent features that possess structure at odds with their motivating hypothesis? If not, does avoiding this mismatch help identify said features and gain further insights into neural network representations? To answer these questions, take a construction-barn (MP) algorithm from sparse coding design MP-SAF- SAE that unrolls its encoder into a sequence of residual-guided steps, allowing it to capture hierarchical and nonlinearly accessible features. Comparing this architecture with existing SAEs on a mixture of synthetic and natural data settings, we show: (i) hierarchical concepts induce conditionally orthogonal features, which existing SAEs are unable to faithfully capture. and (ii) the nonlinear encoding step of MP-SAE recovers highly meaningful features, helping us unravel shared structure the seemingly dichotomous representation spaces of different modalities in a vision-langune model, hence demonstrating the assumption that useful features are solely linearly accessible is insufficient. We also show that the sequential encoder principle of MP- SAE affords an additional benefit of adaptive sparsity at inference time, which may be of independent interest. Overall, we argue our results provide credence to the idea that interpretability should begin with the phenomenology of representations, with methods emerging from assumptions that fit it.
model representations
approach and re-contextualize the popular matching pursuit
1 Introduction
Modern neural networks trained large-scale datasets have achieved unprecedented performance on several practical tasks [1-10], prompting efforts towards understanding how these abilities are implemented within model [11-13]. To this end, Sparse Autoencoders (SAEs) [14-23], motivated by the Linear Representation Hypothesis (LRH) [24, 25], have become popular tool for interpreting neural networks. Specifically, LRH, a phenomenological model of organization and computation of neural network representations [24-28] 2442 posits that neural network representations of dimension m can be decomposed along basis of > m approximately orthogonal directions that reflect abstract, interpretable concepts underlying the data distribution. Here, approximate orthogonality is argued to be necessary for circumventing the problem of packing more unique vectors (concepts) than the
Equal contribution. 'Equal advising. Emails: btolooshamsshaalbertaaa ca, demba@seas. harvard. edu.
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

dimensionality of the space allows (a.k.a. the superposition problem [24]), hence enabling reliable estimation of concept's presence by a linear operator (e.g., the linear map in an MLP) [29, 30]. Grounded in this idea, and inspired by its relation to the well-known problem of sparse coding 32], SAEs have been proposed to learn, in an unsupervised manner, a sparse, overcomplete dictionary of directions that (ideally) map onto abstract, interpretable concepts encoded in a neural network, hence enabling its interpretability [33, 14].
Despite the fact that substantial empirical support has been shown in favor of LRH e.g. representations capturing con- cepts like geometry and lighting [34-38], facial character- istics [39, 40]. broader scene organizations [41-43], and in- stance structures [44-51] have been shown in vision models, while ones capturing concepts like basic semantics [52-57], character roles and theory-of-mind properties [58, 59], arith- metic concepts [60], refusal to unsafe queries [61, 62], and subject-object relations [63-66] have been shown in language models-our goal in this work is to better contextualize the assumptions underlying the design of SAEs in light of recent evidence against the validity of LRH [67, 68). For example, Park et al. [69] recently analyzed the organization of hierar- chical and categorical concepts in neural network represen- tations, showing that cross-entropy loss tends to encourage orthogonal structure for hierarchical relations, while cate- gorical concepts arranged polytopes with overlapping support. Meanwhile, Csordas et al. [70] demonstrated the existence of "onion-like" representations in a simple copying task that cannot be linearly accessed; the existence of such "nonlinear" representations in larger-scale models was also emphasized by Engels et al. [71]. Finally, the existence of multi-dimensionn features for concepts with temporal rela- tions, e.g., tfys-oof-the-eeek has been used to argue against the use of directions as a basis for decomposing neural net- work representations [72, 21]; this is also supported by exper- iments showing concepts' representations and dimensionality can be flexibly manipulated via the inputted context [73].
A)
Geometry of Model Representation
Figure 1: Conceptual organization in neural representations. A) Lin- early accessible concepts: abstract directions that approximately or- thogonal and independently inter- pretable, as assumed by the Linear Representation Hypothesis (LRH). B) Hierarchical concepts: representa- tions structured in parent-child re- lations. C) Nonlinear, multidimen- sional, and temporally structured con- cepts: features that cannot be ac- cessed via a single direction.
The disparity highlighted above between phenomenology of neural network representations identified in recent work versus one hypothesized by LRH raises the question as to whether SAEs, which were motivated by LRH, are able to capture and explain representations that lie outside the scope of LRH (Fig. 1). Since the SAE architecture is not explicitly biased against such representations, is it possible SAEs remain performant towards expressing them, or do these disparities manifest limitations on what concepts SAES can explain? To address these questions, we make the following contributions.
Contextualizing SAES beyond the linear representation hypothesis. Via mixture of synthetic and natural data, we analyze whether popular SAE architectures are able to capture concepts encoded in hierarchical, nonlinearly accessible manner (see Hindupur et al. [21] for an analysis of multi-dimenniinal concepts). To this end, we introduce the notion of Conditional Orthogonality (Def. 2.3), under which, in a hierarchy, orthogonality expected only between concepts within across different levels. Our experiments yield consistent evidence that existing SAES, including those trained with hierarchical objectives [20]. are unable to capture such concepts faithfully.
Introducing MP-SAE for phenomena beyond linear assumptions. To contextualize the results above and understand if capturing hierarchical structures yields meaningful features in larger- scale models, we design an SAE architecture that can capture (certain kinds of) hierarchical and nonlinear structures. Specifically, we extend the standard SAE framework by unrolling the Matching Pursuit algorithm [74] into a residual -giided, sequential encoder caclede-calledd the MP-SAE. Each step of MP- SAE selects a feature (approximately) orthogonal what has already been explained, naturally promoting a conditionally orthogonal structure. Across experiments on both synthetic benchmarks and real-world vision-language models, MP-SAE proves more expressive and uncovers richer structural features than existing SAES. These results question the assumption that meaningful features must be linearly and independently accessible.

---

Additional ad advantages of MP-SAE. We find MP-SAE recovers richer structure than standard SAEs across diverse settings: on synthetic trees with controlled interference, it uniquely preserves both within- and across-level organization; on large-scale models (e.g., CLIP [75], DINOv2 [76]), it identifies nonlinear and multimodal features that elude linear encoders. Moreover, we show MP- SAE offers two practical benefits: (i) it enables adaptive sparsity without retraining eetraniing-features can be incrementally added at test time-and (ii) it ensures monotonic improvement in reconstruction error with each inference step. These properties especially contrast with fixed-k SAEs, which often degrade when sparsity levels shift at test time. Thus, though our goal for analyzing MP-SAE to understand whether useful features can be elicited by actively modeling phenomenology outside the scope of LRH, we believe MP-SAE can be of independent interest to the community.
2 Formalizing Linear Representation Hypothesis and Sparse Autoencoders
We begin by formally describing two objects central to our study: linear representation hypothesis and sparse autoencoders. We note that though the idea that concepts are "linearly encoded" in neural networks has found significant attention in literature (see App. A), formal definition has rarely been offered (see [25, 77] for notable exceptions). Consequently, several related observations on neural network representations have come to be associated with LRH, including linear accessibility of interpretable concepts [78, 59], linear algebraic manipulability of model behavior [79, 66, 80], and decomposability into a linear mixture of directions [14, 23, 22]. Henee, to be precise about what mean by the term, we distill the core claims posited by Elhage et al. [24] on LRH and formally state (our interpretation of) the hypothesis. This definition then directly motivates SAEs.
In what follows, we use plain lowercase letters a to denote scalars, bold lowercase letters a to denote vectors, and uppercase letters A to denote matrices. The 1tth column of a matrix A is written as Ag. The norms || ||2 and || denote the standard E and & norms, while || llo refers to the to pseudo-norm, i.e., the number of nonzero entries. The Frobenius norm of matrix is denoted by - For any vector z, its support is defined as supp(z) == {i z; A 0}, and z is said to be k-sparse if (supp(z)] = k. Hereafter, we use "features" and "concepts" as interchangeable terms, distinguishing them from "representations" a model derives for an input.
Definition 2.1 (Linear Representation Hypothesis (LRH)). A representation x E I. is said to satisfy the linear representation hypothesis (LRH) if there exists dictionary D (D....., Dpl mmmp and a coefficient vector z E TRP such that x: = Dz, under the following conditions:
(i) Overcompleteness:
P > m;
(i) Quasi-orthogonaliy::
max $#
D.T D2 S where Vi D = 1; and
(iii) Sparsity:
supp((>] S <
We emphasize the constraints above are deeply interdependent. In particular, if a model is to represent a large number of distinct concepts (p >> m) within lower-dimensional space RT", while ensuring that a linear readout can reliably recover any active concept, the directions associated with these concepts must be approximately orthogonal. This requirement is substantiated by a reinterpretation of the Johnson-Lindenstrenn high-dimensional data can be compressed into lower dimensions while preserving pairwise distances, Elhage et al. [24] apply a flipped version: one can embed exponentially many quasi-orthogonal directions within RT such that sparse linear combinations remain distinguishable. This perspective justifies the fcasibility of constructing overcomplete, low-coherence dictionaries in LRH. Rather than compressing structure, the goal is to expand representational capacity: to pack many interpretable directions into a shared space while preserving linear accessibility.
(JL) lemma [29, 30 While the lemma is typically used to show that
Overall, if the conditions above hold, sparse linear decompositions become not only possible, but a natural mechanism for expressing concept-aligned features ttireeess-thissteee leads the idea of SAEs. Specifically, noting that the model described in Def. 2.1 closely aligns with the generative model assumed classical work on sparse coding [32, 31, 81], wherein one seeks to express data as sparse linear combinations over overcomplete dictionary, SAES were recently proposed to re-contextualize that literature's 's tools for identifying concepts encoded a neural network's 's representations [14-23].
lemmemma|| [29] states that = e (m)) vectors can be embedded in 1.Th such that all pairwise inner products are bounded by small 0. thereby justifying the feasibility of netroorthygnnality even when p >
3

---

Definition 2.2 (Sparse Autoencoders). Given model representations E R"", the goal of an SAE is to compute a sparse code z that reconstructs x as linear combination of a learned dictionary D:
2 = II (x bpre) + b}, and
(I)
2 Dz Dpre.
where W E RRx and b RP denote the encoder weights and bias, Opre E IR is pre-decoding bias, and D E Ruxp is the learned dictionary of concept vectors.
In the above, the nonlinearity projection (I) projects the pre-activation a sparse support [21];
common choices for activation maps include ReI.U [23, 14), TopK [82, 15, 17], and JumpReL.U [18, 19]. Training proceeds by minimizing reconstruction loss along with sparsity and auxiliary penalties:
L | z + XR(z) + a Laux.
where R()) promotes sparsity, via f bn mechanisms, and Laux may be used to minimize number of inactive units [15, 19].
2.1 Stress-Testing SAEs via Conditional Orthogonality Structure Outside LRH's Scope
While SAES provide a natural operationalization of LRH, their effectiveness hinges on structural assumptions that may not hold in practice. In particular, Eq. 1 presumes that concepts corre- spond to approximately orthogonal directions and can be recovered via a linear projection. However, as several recent works show, these assumptions can prove overly rigid [72, 68, 71, 21, 69]. For instance, neural network represen- tations are known to encode hierarchically struc- tured concepts [83-88]. Park et al. [69] show that for such hierarchical concepts, while con- cepts at the same level of hierarchy frequently form polytopes with overlapping support, par- and child concepts tend to span orthogonal subspaces. In what follows, we formalize this idea under the term 'conditional orthogonality' and use it as a structure to stress-test whether SAEs can identify concepts that encoded in manner outside the scope of LRH. We note this definition is merely paraphrased version of the one offered by Park et al. [69].
A)
aM
Vertabratte
Quasi Orthoooityyyyylliyy
Amphibiaan
Reptiles
Mammals
Fish
Birds
Conditional Orthogonality
Salmon
F15
Figure 2: Illustrative Example of Conditional vs. Quasi-Orthogonality: A) Example of a hi- erarchical concept tree. B) Comparison of quasi- orthogonality (interference within levels) vs. con- ditional orthogonality (orthogonality across levels). C) Correlation matrix of features sampled from A, showing conditional orthogonality (white, = 0) across levels and guasi-orthogonality (light blue,
E) within levels.
Definition 2.3 (Conditional Orthogonality). Let D TRx be dictionary whose columns Di...., Dp denote concept directions, and let e: [p] + N assign each concept to a discrete level in a hierarchy. We say that D is conditionally orthogonal with respect to l if
D. Dj =0 whenever E(i) + E(j).
Relating back LRH (Def. 2.1), we note Def. 2.3 relaxes the global guasi-orthogonality constraint posited by LRH, requiring instead orthogonality only between concept vectors drawn from different hierarchical levels: specifically, note that no constraint is imposed on inner products D. D, for pairs i E [p] with E(i) = E(j). This permits controlled interference within given hierarchical level, while preserving separation across levels. Fig. adapted from Bussmann al. [20], depicts an instance of such structure, where hierarchical organization gives rise to distinct patterns of inter- and intra-level alignment. To evaluate whether conventional SAEs (implicitly aligned with LRH) are capable of capturing the interference structure induced by conditional orthogonality, we will analyze a synthetic generative process grounded in the taxonomy of Fig. 2 Sec. 4. This will allow us to probe the inductive biases of different SAE variants under controlled structural constraints. However, to contextualize these results, we next develop an SAE that specifically motivated to capture the ability to model features that are conditionally orthogonal in nature.

---

y00
p(a))
go( (r((t)
Algorithm 1 Matching Pursuit Sparse Autoencoders (MP-SAH)
1: Input: Dictionary D. hias Biue data n. steps T
DT
D
2: Initialize residual: ( x bru
3: Initialize reconstruction: a (") Upee
.00
D x
((+))
4: Initialize sparse code: 0
for O..... T
00
6: jLM arg max, ma (DTr(()), (0)))
Djuy
A (
x(0)
20)
DJo,
Djorr() r((
,(0
(1+1) (t)) + [1)
D(0))
,())
D
9: p(+1 p())
j(0) D 3(0
10: end for
11: Output: Sparse code Z ET-1 (0)) Reconstruction = (T)
Figure 3: (Top Left) One iteration of the MP-SAE encoder: dictionary D, the most correlated feature D D goo is selected, and its contribution is subtracted from the residual and added to the reconstruction. (Bottom Left) MP sequentially reconstructs æ by greedily selecting features that best explain the residual, promoting orthogonal selection and enabling access to higher-order features that are nonlinearly accessible from æ. (Right) MP-SAE embeds the MP algorithm into sparse autoencoder, where the dictionary D is learned through backpropagation.
The residual (()) is projected onto the
3 Operationalizing Conditional Orthogonality via MP-SAE
To enable the ability identification of features encoded in a conditionally orthogonal manner (Def. 2.3), we construct a sparse autoencoder whose inference reflects the structure of hierarchical appresentations-minninin
interference across levels while tolerating it within [69]. Our design draws on the Matching Pursuit (MP) algorithm [74] from sparse coding [89-92], which has been shown recover features from the appropriate hierarchical level in conditionally orthogonal dictionar- ies [93]. MP performs inference greedily: at each step, the feature most correlated with the residual is selected, its contribution subtracted, and the process repeated. This iterative residual decomposition promotes conditionally orthogonal feature selection, as each new feature explains variance orthogonal to what was captured by the previous feature (Fig. 3). Embedding this mechanism into sparse autoencoder yields the Matching Pursuit Sparse Autoencoder (MP-SAE), formalized in Algorithm 1. Below, we unpack the structure and implications of MP-SAE, examining: (i) the mechanics of its inference procedure; (ii) how this procedure promotes conditional orthogonality; and (ii) its capacity to extract higher-order, nonlinearly accessible features.
(i) Mechanies of MP-SAE. MP-SAE uses shared learned dictionary for encoding and decoding. The encoder embeds the classical Matching Pursuit algorithm [74] by unrolling its greedy inference procedure into a fixed number of steps. At inference, the model starts with an initial estimate and residual (Algorithm 1, lines 3 and 2). At each iteration, MP-SAE greedily selects the feature that best aligns with the current residual by computing the inner product between the residual and each feature, and choosing the one with the highest projection (line 6). Once the best-matching feature identified, the algorithm determines its contribution by projecting the residual onto that feature (line 7), adds this contribution to the current approximation of the input (line 8), and updates the residual accordingly (line 9). This procedure is repeated for T steps, producing a sparse code with I| < T. The resulting encoding represents sequential approximation of the input, constructed through greedy selection of locally optimal features that progressively refine the reconstruction of ær
(ii) Conditional Orthogonality via Sequential Inference. A defining property of MP-SAE is that its greedy inference procedure promotes conditional orthogonality across selected features. This emerges from the residual update rule: once a concept D D has been selected from the dictionary at iteration f-1, the updated residual (()) is orthogonal to it, as stated below formally.
Proposition 3.1 (Stepwise Orthogonality of MP Residuals). Let p(t) be the residual at iteration of MP-SAE inference, and let Dyu-1 1) be the feature selected at step t-1, Then:
D D -0.
5

---

Each selected concept is removed from the residual subspace before the next selection, and subsequent concepts are chosen explain what remains the residual, orthogonal the last selected feature. When trained, this yields a dictionary of features that are not globally orthogonal but are selectively chosen be conditionally orthogonal. Although MP-SAF only enforces orthogonality to the most recently selected concept-und all prior selections- we observe empirically that residuals are often nearly orthogonal to all previously selected directions. This emergent behavior suggests the model implicitly promotes hierarchical separation in the learned dictionary, minimizing interference across levels. It aligns with the inductive bias of conditional orthogonality (Def. 2.3) and contrasts with standard SAEs, which tend to promote global gutsiootthogonality regardless of feature structure.
Orthogonal Matching Pursuit [89], which re-orthogonalizes against
(ii) Access nonlinearly accessible features. Beyond promoting conditional orthogonality, the residual- based inference structure of MP-SAE enables access to features that are nonlinearly embed- ded in the representation space. While each iteration applies linear projection to the current residual, the residual itself evolves nonlinearly as a function of the input, due to its recursive dependence on previous selections. This results structured approximation of x that can be decomposed as:
T
x = (0) = (xx) + (1)
(p
Σ Ex(((0))
(T+1)
fineurly accessible
residioul error
nomliweurly accessible
where (p() denotes the linear projection onto the selected feature at each step (See Fig. 3). The crucial insight is that, although each (p(((t) is linear in its argument, the argument itself, ((), is nonlinear function of æ. As a result, the composition (p((()) defines a feature that cannot be obtained from via a single linear map. MP-SAE can thus potentially uncover higher-order concepts that are conditionally dependent on previously explained structure. This mechanism may be particularly relevant in settings where important features are entangled or nonlinearly composed such as hicrarchies, temporal dependencies, or multimodal correlations. Moreover, it provides a constructive hypothesis for the phenomenon of "dark matter" in neural representations [71]: features that evade standard SAEs because they are not linearly accessible from the raw representation; that is, one can still have the LRH assumption of linear mixing process hold, i.e., x Dz, but other constraints start to relax (Def. 2). We return to this phenomenon in our empirical analysis.
Empirical Results
We now take a step towards uncovering challenges in SAEs emergent from assuming a partially correct model of neural network representations, i.e., LRH. Specifically, in Sec. 4.1, we analyze a synthetic domain that highlight the inability of existing SAES to identify hierarchically structured concepts in a clearly defined domain. Building on these results, in Sec. 4.2, we analyze a natural vision-language domain to assess how features extracted under an inductive bias that accommodates hierarchical, nonlinearly accessible concepts-a in MP-SAE SAE-differ from those identified by existing SAEs. In particular, we show that MP-SAE uniquely recovers mooal structure.
4.1 A Synthetic Generative Model of Hierarchical Features
We begin by evaluating whether SAEs, including our proposed MP-SAE, can recover conditionally orthogonal features using a synthetic hierarchy-basee setup adapted from Bussmann et al. [20]. To formalize this setting, we introduce the following definition of a hierarchical generative process.
Definition 4.1 (Hierarchical Generative Process). generative process over p (parent concept) and e (child concept) is said to be hierarchical if their activations úp and 2e satisfy
P( > 0 z 0) = and P(z. > 0 2p 0) <
That is, a child's activation requires its parent's, ', whereas a parent may activate independently.
Based on this definition, we implemented the following hierarchical generative process. The ground truth D consists of 20 unit-norm concepts organized into two-level tree: 11 disjoint parent concepts D,, 3 of which each have 3 disjoint children D. as depicted in Fig. 2. Each input is generated by sampling a parent and, if present, a child, resulting in one two active concepts.
Zp Dp + Ze Do

---

The activations Ep and follow the activation pattern defined in Definition 4.1, taking values from (1.55, 1/4+) (ensuring positive values) when active and O otherwise."
We analyze Vanilla (ReLU) [23, 14), BatchTopK [17], and Ma- tryoshka [20, 94] all trained with fixed to sparsity targets. In fixed low intra-level in- terference regime (Fig. 4A), Vanilla and BatchTopk suffer from feature absorption [95], aligning child concepts with their parent and collapsing hierar- bhy-though they retain within- level corelation. Matryoshka avoids absorption and preserves hierarchy but introduces negu- tive interference between sib- lings, distorting flat structure. Only MP-SAE recovers both in- tra and inter level structure. To further stress-test the different SAEs, vary within-level cor- relation in the ground truth and evaluate learned dictionaries ing "Flat MSE" (for intra-level alignment: see Fig. 4B) and "Hi- erarchical MSE" (for inter-level separation; Fig. 4C). We ob- serve that Matryoshka tends to preserves hierarchy but loses flat structure; meanwhile, Vanilla and BatchTopK behave oppositely. MP-SAE exhibits three regimes: low to moderate interference yields recovery of both structures; under high interference, both degrade, reaching eventually point where we see MP-SAE sacrifice flatness to maintain hierarchy, highlighting its inductive bias.
A)
Vanilla
BatchTopK
Matryoshka
MP
Orciona
GF
Dictionary
Dictionary
Dictionary Hierarchical MSE
Dictionary
B)
Flat MSE
10
13"
10
BatchTopk Matryoshka
10
104
16-
0.8
0.9
GT Casina Similarity
GT Cosine Similarity
Figure 4: Evaluating SAE on a hierarchical tree with con- trolled within-level similarity. A) Correlation matrices for one similarity setting. Left shows the ground-truth matrix; the top row displays D D (self-similarity of learned features) and bot- tom row shows Di (alignment with ground truth). Bottom: Quantitative evaluation across varying levels of within-group cor- relation, median over 10 runs reported. B) Flat MSE captures the deviation from the ground-truth intra-level correlation. C) Hi- erarchical MSE quantifies unintended correlations across levels.
Overall, this synthetic benchmark elicits challenges towards capturing hierarchical features via existing SAEs, including ones trained with explicit hierarchy pronoooing objectives. Instead, capturing the relevant inductive bias via MP-SAE works really well: MP-SAE is uniquely capable of recovering hierarchically structured features when such property statistically exists. We now turn to pretrained representations to assess whether such structure arises in practice and how SAEs respond to it.
4.2 Representations from Pretrained Models
We analyze representations from pretrained vision-language models in the following experiments. This helps avoid the challenge of flexibility of model representations in solely language-drive driven domains [73] (though see Sec. B.5 for preliminary results) Our evaluation is organized in four parts. We begin by (i) assessing MP-SAE's expressivity relative existing SAFs. We then (ii) analyze the structure of its learned representations through effective rank and coherence metrics. Next, (til) test its robustness inference-time sparsity variation. Finally, (îv) we investigate its ability to uncover multimodal features in vision-language models that remain inaccessible to existing SABS.
Expressivity. We begin by evaluating the expressivity of different SAEs, including MA-SAE. Training was performed for 50 epochs with Adam, using a learning rate of 5 100 and cosine decay 10-6 with warmup. Models were trained on INIK [96] train set, using frozen representations from the final layer of each backbone. For ViT-style models (e.g.. DINOv2), all spatial tokens and the CL.S token were included (~~211 tokens per image for D2n which results in approximately 25 billion training tokens for a training).
We generalize the generative process from Bussmann et al. [20], where &p and 2, are fixed across all inputs, making their firing magnitudes perfectly correlated (zp Aze). See Appendix B. 3.1.4 for an extended discussion.
7

---

SigLIP
DINOv2
CLIP
ViT
0.9
0.8
0.8
SAEs
0.8
0.8
0.7
Vanilla
2 0.7
0.7
0:
Topk
0.6
0.6
BatchTopK
0.6
0.5
MP
510
20
510
20
510
20
50
510
50
Lo
Lo
Lg
Figure 5: MP-SAE recovers more expressive features than standard SAES. Reconstruction performance (R²) as function of sparsity level across four pretrained vision models: SigLIP, DINOv2, CLIP, and VIT. MP-SAE consistently achieves higher R² at comparable sparsity, indicating more efficient and informative decompositions.
Results are shown in Fig. 5, where plot the Pareto frontier obtained by varying the sparsity level, using an expansion factor of 25 (p- 25m) for all SAEs. Across all tested models: SigLIP [97], DINOv2 [76], CLIP [75], and ViT [98], MP-SAE consistently achieves higher R² at comparable sparsity levels, indicating more efficient reconstruction. Our results above suggest that MP-SAE can explain larger fraction of the representation space using fewer active features. Equivalently, its selected features are more informative per unit of sparsity. This aligns with the hypothesis that MP- SAE, through its iterative and residual-guie guided inference, can recover features that remain inaccessible to conventional SAEs. We note that, Engels et al. [71] identify class of non-linearly accessible features that can't 't be recovered by linear sparse encoders. The improved expressivity observed here provides indirect evidence that MP-SAE may capture some of these otherwise hidden components. In the sections that follow, we examine more precisely the structure and semantics of the features recovered by MP-SAE to further probe this possibility.
Effective Rank
Emergence of Rank Structure. To investigate how MP-SAE
5000
organizes features across varying sparsity levels, we analyze the effective rank of the co-activation matrix Z Z, where
4000
Z Raxp is the matrix of sparse codes across n inputs (see Fig. 6). Each entry of Z Z captures how frequently two concepts are co-activated. The effective rank, defined as the ex- ponential of the entropy of the normalized eigenvalues of Z Z (Fq.2, Appendix B.3), measures the diversity of feature co- activation patterns across inputs. For standard SAEs, increasing the sparsity level i typically results in limited structural change: the encoder reuses similar subsets of features, leading to sat- urated rank and strong diagonal blocks in T Z. In contrast, MP-SAE shows markedly different trend. As k increases, the effective rank grows steadily, reflecting a continual diver- sification of active feature sets. The rank growth induced by MP-SAE is thus not an artifact of increased capacity, but a structural signal: it reflects the model's 's ability to discover and disentangle latent modularity within representations.
3000
2000
1000
3 5
20
Inference L.
Figure 6: Growth of effective rank as a function of sparsity MP-SAE exhibits increasing com- binatorial diversity, unlike standard SAEs whose co-activation structure quickly saturates.
Coherence and Conditional Orthogonality. We now examine the internal organization of the learned dictionaries by quantifying their coherence. Specifically, we use the Babel function [99], a standard metric in sparse approximation that captures cumulative interference between features. Given dictionary D [D,...., Dp) E Rmx with unit-norm columns, the Babel function of order r is defined as:
(r(r) = max SC
max jES
Σ D. D2
YES
Intuitively, en (r) reflects how well a single concept can be approximated by a group of r others: lower values indicate better separability. Fig. 7 reports in (r) for the learned dictionary, as well as the average over multiple representations for subsets of co-selected concepts at inference time. Interestingly, MP-SAE learns dictionaries with higher overall Babel scores than standard SAEs,
8

---

SigLIP
DINOv2 CLIP
VIT
Babel Score py(r)
Learned
60-
SAEs
Vanilla
TopK
Selected
BatchTopK
10
Figure 7: MP-SAE promotes conditional orthogonality at inference. Babel scores for full dictionaries (top) and co-activated subsets (bottom). MP-SAE dictionaries exhibit higher global coherence than standard SAEs, but select separated features inference.
indicating greater global interference. However, the concepts it selects during inference exhibit lower Babel scores, reflecting MP-SAE's tendency to construct conditionally orthogonal representations at inference (even when the full dictionary is correlated). By contrast, linear SAEs enforce global quasi-orthooonality the dictionary but not control which features co-activate at inference. As a result, inference often selects interfering directions despite well-structured dictionary.
Adaptive Inference- Time Sparsity. A key property of MP-SAE its ability adaptively adjust the number of selected features k at inference time without re- training, Because inference proceeds via residual-based greedy selection, each addi- tional step guarantees non-increasing re- construction error. As shown Fig. 8, MP-SAE is the only architecture for which reconstruction fidelity improves monoton- ically with & across all tested architectures and representation types. This stands in contrast TopK SAEs, which degrade der sparsity mismatch: when trained with fixed k. the decoder implicitly specializes to superpositions of exactly k features, re- sulting in instability when k changes.
TopK: LO=5
Topk: LD-10
Topk: LD=20
Topk: LD-50
10a
vanilla: LG=5
vanilia: LG-10
vanilla: 10-50
10
MP: L0=5
L0-10
MP: L0=20
MP: L0=S0
50
100
200
Inference LD
Figure 8: Reconstruction error as a function of inference-time sparsity k. When increasing SAEs sparsity at inference on DINOv2 representations, we see MP-SAE exhibits monotonic improvement, while TopK SAES may degrade due to sparsity mismatch. Similar results emerge other settings (see Appendix B.5).
ResU-based SAEs, by contrast, cannot extend their support beyond what was active during training and exhibit flat plateaued performance as & increases. This robustness is particularly salient given the epistemic uncertainty surrounding the "true sparsity" neural representations. This property allows adjusting at inference time without retraining, enabling controlled trade-offs between sparsity and reconstruction quality. Rather than fixing k; one can target a desired reconstruction level and let the model incrementally reveal the relevant features by adapting k.
Recovering Multimodal Concepts. We evaluate whether MP-SAE can recover shared concepts from languae models (VLMs). Our analysis focuses on representations from CLIP [75], with consistent results observed for AIMv2 [100]. SigLIP [97], and SigLIP2[101] (See Appendix B.4). All models are evaluated on the COCO dataset [102], using both image and caption embeddings, balanced to ensure equal sampling across modalities. When trained these joint embedding spaces, classical SAEs frequently learn split dictionaries [103, 104], where distinct features respond exclusively to either visual or textual inputs. This occurs despite the alignment of the underlying representation space and reflects a structural limitation in how these existing SAEs extract shared features. To quantify modality selectivity,
the Modality Score from [105], defined for concept i as:
Ezei
ModalityScore,
E (2.) + Bx
where and r denote activations over image and text inputs, respectively. Scores near indicate image specificity; near O, text specificity; and intermediate values reflect balanced, multimodal activation.

---

Consistent with prior work [105], we find that standard SAEs yield sharply bimodal modality score distributions, confirming their tendency to separate modalities. In contrast, MP-SAE yields sig- nificantly flatter distribution with substantial mass in the midrange (see Fig. 9), suggesting that it recovers genuinely multimodal units responsive to both modali- ties. This ability to extract mul- timodal concepts appears unique to MP-SAE. We hypothesize that MP-SAE's residual based infer- ence plays a central role: once mopelity-specific information is explained, the residual shifts to- ward shared structure, allowing subsequent steps to capture cross- modal features (See Fig. 18). Prior work [104] has attempted to bridge the modality gap by applying learned translation from one modality to the other. This can be seen equivalent to a single inference step in MP-SAE. However, MP-SAE continues this process iteratively, progressively refining the residual and revealing joint semantic structure. This iterative mechanism enables MP-SAE to extract hierarchical and non-linearly accessible multimodal features beyond the scope of conventional SAEs.
Topk
BatchTopk
MP
Counts
4000
4000
1000
2000
2000
100
0.5
0.5
0.5
Modality Score
Figure 9: MP-SAE identifies shared structures across modali- ties. Top: UMAP of sparse codes (yellow denotes image repre- sentations; blue denotes text). Bottom: Distribution of modality scores. We see existing SAES yield features skewed towards a specific modality, suggesting bimodal structure in model represen- tations. However, MP-SAE uniquely recovers multimodal units with shared activation across text and vision inputs.
5 Conclusion
In this work, we revisit SAEs to understand their ability identify concepts encoded in a manner outside the scope of their motivating hypothesis, i.e., LRH. Our results show that while standard SAEs are indeed effective under LRH, they struggle capture more complex representational structures- such as hierarchical, nonlinearly accessible, and multimodal features that have emerged recent studies large neural networks. To contextualize these results with respect to an SAE that is motivated to (partially) accommodate such phenomenology outside LRH's ss scope, we introduced MP-SAE. Specifically, an extension of the classical Matching Pursuit algorithm, MP-SAE promotes conditional orthogonality through residual-guided inference. Our results show that this design enables MP-SAE to recover rich, hierarchically organized, and nonlinearly accessible features that standard SAES missed. Beyond interpretability, we also find that residual-basee based, sequential encoders like MP-SAE offer practical advantages, such as adaptive inference- time sparsity and progressive feature discovery, which may be of independent interest to the community. Overall, argue results highlight the limitations of assuming purely linear structure neural representations.
We emphasize that our goal with proposing MP-SAE was propose a "superior" SAE architecture, but to use it as tool whose inductive bias aligns with particular class of concept structures, namely, hierarchically and nonlinearly accessible features. Under this hypothesis, MP. SAE proves more expressive, more modular, and more robust than standard SAEs. However, we note MP-SAE assumes that representations can be incrementally explained by conditionally orthogonal features. This inductive bias may be poorly matched in flat or entangled regimes, or in settings with categorical structure. Matching Pursuit is also greedy algorithm, which lacks global optimality and may be brittle under extreme noise (though we do find existing SAEs fail in this regime as well).
Limitations
Acknowledgements
Authors thank the CRISP Group at Harvard SEAS for insightful conversations, and the Kempner Institute and CBS-NTT program in Physics of Intelligence at Harvard University for access to compute resources used for performing experiments reported in this paper. Valérie Costa thanks the Bertarelli Foundation for supporting her work fellow.
10

---

References
[1] OpenAI. Gpt-4 technical report, 2024.
[2] OpenAI. Openai o3 and 04-mini system card, 2025.
[3] Gemini Team. Gemini: a family of highly capable multimodal models. arXiv preprint arXiv:2312. /1800 2023.
[4] Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Rumxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseck-rl: Incentivizing reasoning capability in Ilms via reinforcement learning. arXiv preprint arXiv:2501. 12948, 2025.
[5] Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, et al. Deepseek-v3 technical report. arXiv preprint arXiv:2412. 19437, 2024.
[6] Nikhila Ravi, Valentin Gabeur, Yuan-Time Hu, Ronghang Hu, Chaitanya Ryali, Tengyu Ma, Haitham Khedr, Roman Rädle, Chloe Rolland, Laura Gustafson, et al. Sam 2: Segment anything in images and videos. arXiv preprint arXiv:2408.0077 2024.
[7] Alexander Kirillov, Eric Mintun. Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C Berg, Wan- Yen Lo, et al. Segment anything. In Proceedings of the IEEE/CVF international conference on computer vision, pages 4015 4026, 2023.
[8] OpenAI. Sora: Creating video from text, 2024.
[9] William Peebles and Saining Xie. Scalable diffusion models with transformers. In Proceedings of the IEEE/CVF international conference on computer vision, pages 4195 2023.
[10] Matt Deitke, Christopher Clark, Sangho Lee, Rohun Tripathi, Yue Yang, Jae Sung Park, Mohammadreza Salchi, Niklas Muennighoff, Kyle Lo, Luca Soldaini, al. Molmo and pixmo: Open weights and open data for state-of-the-ar multimodal models. arXiv preprint arXiv:2409.17144 2024.
[11] Finale Doshi-Veler and Been Kim. Towards a rigorous science of interpretable machine learning. preprint orXiv:/702.0 086088 2017.
[12] Usman Anwar, Abulhair Saparov, Javier Rando, Daniel Paleka, Miles Turpin, Peter Hase, Ekdeep Singh Lubana, Erik Jenner, Stephen Casper, Oliver Sourbut, al. Foundational challenges in assuring alignment and safety of large language models. ArXiv e-print, 2024.
[13] Paolo Tripicchio and Salvatore D' A'eell... Is deep learning ready satisfy industry needs? Procedia Manufacturing, 2020.
[14] Trenton Bricken, Adly Templeton, Joshua Batson, Brian Chen, Adam Jermyn, Tom Con- erly, Nick Turner, Cem Anil, Carson Denison, Amanda Askell, Robert Lasenby, Yifan Wu, Shauna Kravec, Nicholas Schiefer, Tim Maxwell, Nicholas Joseph, Zac Hatfield-oodds, Alex Tamkin, Karina Nguyen, Brayden McLean, Josiah E Burke, Tristan Hume, Shan Carter, Tom Henighan, and Christopher Olah. Towards monosemanticity: Decomposing language models with dictionary learning. Transformer Circuits Thread, 2023. https:/transformer- circuits, circuits.pub/2022.
[15] Leo Gao, Tom Dupre la Tour, Henk Tillman, Gabriel Goh, Rajan Troll, Alec Radford, Ilya Sutskever, Jan Leike, and Jeffrey Wu. Sealing and evaluating sparse autoencoders. In The Thirleenth International Conference on Learning Representations, 2025.
[16] Thomas Fel, Ekdeep Singh Lubana, Jacob Prince, Matthew Kowal, Victor Boutin, Isabel Papadimitriou, Binxu Wang, Martin Wattenberg, Demba Ba, and Talia Konkle. Archetypal sae: Adaptive and stable dictionary learning for concept extraction large vision models. arXiv preprint 2rXiv 2025.

---

[17] Bart Bussmann, Patrick Leask, and Neel Nanda. Batchtopk sparse autoencoders. preprint arXiv:2412.06410, 2024.
[18] Senthooran Rajamanoharan, Tom Lieberum, Nicolas Sonnerat, Arthur Conmy, Vikrant Varma, János Kramár, and Neel Nanda. Jumping ahead: Improving reconstruction fidelity with jumprelu sparse autoencoders. arXiv preprint arXiv:2407. 14435, 2024.
[19] Senthooran Rajamanoharan, Arthur Conmy, Lewis Smith, Tom Lieberum, Vikrant Varma, János Kramár, Rohin Shah, and Neel Nanda. Improving Dictionary Learning with Gated Sparse Autoencoders, April 2024. ExXv:2244.16014 [cs].
[20] Bart Bussmann, Noa Nabeshima, Adam Karvonen, and Neel Nanda. Learning multi-level features with matryoshka sparse autoencoders. arXiv preprint arXiv:2503.17547 2025.
[21] Sai Sumedh R Hindupur, Ekdeep Singh Lubana, Thomas Fel, and Demba Ba. Projecting assumptions: The duality between sparse autoencoders and concept geometry, arXiv preprint vrXiv:2503.0182 2025.
[22] Adly Templeton, Tom Conerly, Jonathan Mareus, Jack Lindsey, Trenton Bricken, Brian Chen, Adam Pearce, Craig Citro, Emmanuel Ameisen, Andy Jones, Hoagy Cunningham, Nicholas L Turner, Callum McDougall, Monte MacDiarmid, C. Daniel Freeman, Theodore R. Sumers, Edward Rees, Joshua Batson, Adam Jermyn, Shan Carter, Chris Olah, and Tom Henighan. Scaling monosemanticity: Extracting interpretable features from claude 3 sonnet. Transformer Circuits Thread, 2024.
[23] Hoagy Cunningham, Aidan Ewart, Logan Riggs, Robert Huben, and Lee Sharkey, Sparse Autoencoders Find Highly Interpretable Features in Language Models, October 2023. arXiv:2309.08600 [cs].
[24] Nelson Elhage, Tristan Hume, Catherine Olsson, Nicholas Schiefer, Tom Henighan, Shauna Kravec, Zac Hatfield- Hladfeed-DDodd, Robert Lasenby, Dawn Drain, Carol Chen, Roger Grosse, Sam McCandlish, Jared Kaplan, Dario Amodei, Martin Wattenberg, and Christopher Olah. Toy models of superposition, 2022.
[25] Kiho Park, Yo Joong Choe, and Victor Veitch. The linear representation hypothesis and the geometry of large language models. arXiv preprint arXiv:23/1.03668. 2023.
[26] Kaarel Hänni, Jake Mendel, Dmitry Vaintrob, and Lawrence Chan. Mathematical models of computation in superposition. arXiv preprint arXiv:2408.05451, 2024.
[27] Micah Adler, Dan Alistarh, and Nir Shavit. Towards combinatorial interpretability of neural computation. arXiv preprint arXiv:2504.088. 2504008844 2025.
[28] Micah Adler and Nir Shavit. On the complexity of neural computation in superposition. arXiv preprint arXiv:2409. 15318, 2024.
[29] William B. Johnson and Joram Lindenstrauss. Extensions of Lipschitz mappings into Hilbert space. In Richard Beals, Anatole Beck, Alexandra Bellow, and Arshag Hajian, editors, Contemporary Mathematics, volume 26, pages 189-2001 209-006.o2 American Mathematical Society, Providence, Rhode Island, 1984.
[30] Kasper Green Larsen and Jelani Nelson. The johnson-lindeestrauss lemma is optimal for linear dimensionality reduction. arXiv preprint urXiv::4411. 2014.
[31] Michael Elad. Sparse and Redundant Representations: From Theory to Applications in Signal and Image Processing. Springer Science & Business Media, August 2010. Google-Books-ID:
dSC...I9BAAC.
[32] Bruno Olshausen and David J Field. Sparse coding with an overcomplete basis set: strategy employed by v1? Vision research, 37(23):3311-3325 1997.
[33] Can Demircan, Tankred Saanum, Akshay K Jagadish, Marcel Binz, and Eric Schulz. Sparse autoencoders reveal temporal difference learning large language models. arXiv preprint arXiv:2410.01280. 2024.
12

---

[34] Yida Chen, Fernanda Viégas, and Martin Wattenberg. Beyond surface statistics: Scene representations latent diffusion model. arXiv preprint arXiv:2306.05720, 2023.
[35] David Bau, Jun- Yan Zhu, Hendrik Strobelt, Bolei Zhou, Joshua B Tenenbaum, William Freeman, and Antonio Torralba. Gan dissection: Visualizing and understanding generative adversarial networks. arXiv preprint /11. 2018.
[36] David Bau, Steven Liu, Tongzhou Wang. Jun Yan Zhu, and Antonio Torralba. Rewriting deep generative model. In Computer Vision-ECCV 2020: 16th European Conference, Glasgow, UK, August 23-28. 2020, Proceedings, Part I 16, pages 351-369. Springer, 2020.
[37] Rameen Abdal, Peihao Zhu, Niloy Mitra, and Peter Wonka. Labels4free: Unsupervised segmentation using stylegan. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 13970-13979, 2021.
[38] Xiaodan Du, Nicholas Kolkin, Greg Shakhnarovich, and Anand Bhattad. Generative models: What do they know? do they know things? let's find out! arXiv preprint arXiv:2311.17137, 2023.
[39] Andrey Voynov and Artem Babenko. Unsupervised discovery of interpretable directions in the gan latent space. In International conference on machine learning, pages 9786-9796, PMLR, 2020.
[40] Yujun Shen, Jinjin Gu, Xiaoou Tang, and Bolei Zhou, Interpreting the latent space of gans for semantic face editing. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pages 9243-9255 2020.
[41] Yossi Gandelsman, Alexei A Efros, and Jacob Steinhardt. Interpreting clip's image representa- tion via text-based decomposition. arXiv preprint arXiv:2310.05916, 2023.
[42] Jack Merullo, Louis Castricato, Carsten Eickhoff, and Ellie Pavlick. Linearly mapping from image to space. arXiv preprint arXiv:2209.1551 15622 2022.
[43] Matthew Kowal, Achal Dave, Rares Ambrus, Adrien Gaidon, Konstantinos G Derpanis, and Pavel Tokmakov. Understanding video transformers via universal concept discovery. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2024.
[44] Chris Olah, Nick Cammarata, Ludwig Schubert, Gabriel Goh, Michael Petrov, and Shan Carter. Zoom In: An Introduction to Circuits. Distill, 5(3):e00024.001, March 2020.
[45] Chris Olah, Alexander Mordvintsev, and Ludwig Schubert. Feature visualization. Distill,
2(11) 1)77, 2017.
[46] Chris Olah, Nick Cammarata, Chelsea Voss, Ludwig Schubert, and Gabriel Goh. Naturally occurring equivariance in neural networks. Distill, 5(12)x600024-004, 2020.
[47] Chris Olah, Nick Cammarata, Ludwig Schubert, Gabriel Goh, Michael Petrov, and Shan Carter. An overview of carly vision in inceptionvl. Distill, 5(0):4000240002, 2020.
[48] Nick Cammarata, Gabriel Goh, Shan Carter, Ludwig Schubert, Michael Petrov, and Chris Olah. Curve detectors. Distill, 56)::0000440003, 2020.
[49] Ludwig Schubert, Chelsea Voss, Nick Cammarata, Gabriel Goh, and Chris Olah. High-low frequency detectors. Distill, 6(L):c00024-005, 2021.
[50] Thomas Fel, Louis Bethune, Andrew Lampinen, Thomas Serre, and Katherine Hermann. Understanding visual feature reliance through the lens of complexity. Advances in Neural Information Processing Systems, 77:69888-69924, 2024.
[51] Thomas Fel, Victor Boutin, Mazda Moayeri, Rémi Cadène, Louis Bethune, Léo andéol, Mathieu Chalvidal, and Thomas Serre. A Holistic Approach to Unifying Automatic Concept Extraction and Concept Importance Estimation, October 2023. arXiv:2306.07304 [cs].
13

---

[52] Kenneth Li, Oam Patel, Fernanda Viégas. Hanspeter Pfister, and Martin Wattenberg. Inference- time intervention: Eliciting truthful answers from a language model. Advances in Neural Information Processing Systems, 36:41451-4111
2023.
[53] Samuel Marks and Max Tegmark. The geometry of truth: Emergent linear structure in large language model representations of true/false datasets. arXiv preprint arXiv:2310.06824, 2023.
[54] Andrew Lee, Xiaoyan Bai, Itamar Pres, Martin Wattenberg, Jonathan K Kummerfeld, and Rada Mihalcea. A mechanistic understanding of alignment algorithms: A case study on dpo and toxicity. arXiv preprint arXiv:2401.019 2024.
[55] Curt Tigges, Oskar John Hollinsworth, Atticus Geiger, and Neel Nanda. Linear representations of sentiment in large language models. arXiv preprint 10. 2023.
[56] Niru Maheswaranathan, Alex Williams, Matthew Golub, Surya Ganguli, and David Sussillo. Reverse engineering recurrent networks for sentiment classification reveals line attractor dynamics. Advances in neural information processing systems, 32, 2019.
[57] Niru Maheswaranathan, Alex H Williams, Matthew D Golub, Surya Ganguli, and David Sussillo. Line attractor dynamics in recurrent networks for sentiment classification. In ICML 2019 Workshop on Identifying and Understanding Deep Leurning Phenomena, 2019.
[58] Wentao Zhu, Zhining Zhang, and Yizhou Wang. Language models represent beliefs of self and others. arXiv preprint 2xXi::2402.1849 2024.
[59] Neel Nanda, Andrew Lee, and Martin Wattenberg. Emergent Linear Representations World Models of Self-Supervised Sequence Models, September 2023. arXiv:2309.0094 [cs].
[60] Jack Merullo, Carsten Eickhoff, and Ellie Pavlick. Language models implement simple word2vec-style vector arithmetic. arXiv preprini arXiv:2305. 16130, 2023.
[61] Andy Arditi, Oscar Obeso, Aaquib Syed, Daniel Paleka, Nina Panickssery, Wes Gurnee, and Neel Nanda. Refusal in language models is mediated by single direction. arXiv preprint arXiv:2406. 1 1717, 2024.
[62] Samyak Jain, Ekdeep S Lubana, Kemal Oksuz, Tom Joy, Philip Torr, Amartya Sanyal, and Puneet Dokania. What makes and breaks safety fine-tuning? mechanistic study. Advances in Neural Information Processing Systems, 37:93406-93478, 2024.
[63] Evan Hernandez, Arnab Sen Sharma, Tal Haklay, Kevin Meng, Martin Wattenberg, Jacob Andreas, Yonatan Belinkov, and David Bau. Linearity of relation decoding in transformer language models. arXiv preprint ar 230.:2300009124, 2023.
[64] Jack Merullo, Noah A Smith, Sarah Wiegreffe, and Yanai Elazar. On linear representations and pretraining data frequency in language models. arXiv preprini a504v:2504. /2459, 2025.
[65] Sanjeev Arora, Yuanzhi Li, Yingyu Liang, Tengyu Ma, and Andrej Risteski. A Latent Variable Model Approach to PMI-based Word Embeddings. Transactions of the Association for Computational Linguistics, 4:385-399, 2016. Place: Cambridge, MA Publisher: MIT Press.
[66] Sanjeev Arora, Yuanzhi Li, Yingyu Liang, Tengyu Ma, and Andrej Risteski. Linear algebraic structure of word senses, with applications to polysemy. Transactions of the Association for Computational Linguistics, 6:483-495, 2018.
[67] Martin Wattenberg and Fernanda B. Viégas. Relational Composition in Neural Networks: A Survey and Call to Action, July 2024. arXiv:2407. 14662 [cs].
[68] Lewis Smith. The 'strong' feature hypothesis could be wrong, August 2024.
[69] Kiho Park, Yo Joong Choe, Yibo Jiang, and Victor Veitch. The geometry of categorical and hierarchical concepts in large language models. arXiv preprint 24Xiv22406.01006 2024.
[70] Róbert Csordás, Christopher Potts, Christopher D. Manning, and Atticus Geiger. Recurrent Neural Networks Learn to Store and Generate Sequences using Non-L-nnear Representations, August 2024. arXiv:2408. 10920 [cs].
14

---

[71] Joshua Engels, Logan Riggs, and Max Tegmark. Decomposing The Dark Matter of Sparse Autoencoders, March 2025. arX1v:2410.14670 [cs].
[72] Joshua Engels, Eric J. Michaud, Isaac Liao, Wes Gurnee, and Max Tegmark. Not All Language Model Features Are One-Dimensionall Linear, February 2025, arXiv:2405. 14860 [es].
[73] Core Francisco Park, Andrew Lee, Ekdeep Singh Lubana, Yongyi Yang, Maya Okawa, Kento Nishi, Martin Wattenberg, and Hidenori Tanaka. ICLR: In-Context Learning of Representa- tions, May 2025. arXiv:2501 .00070 [es].
[74] S.G. Mallat and Zhifeng Zhang. Matching pursuits with time-frequency dictionaries. IEEE Transactions on Signal Processing. 41 41(12):3397-341 1993.
[75] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pages 8748 8763. PmLR, 2021.
[76] Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy Vo, Mare Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et al. Dinov2: Learning robust visual features without supervision. arXiv preprint arXiv:2304.07119, 2023.
[77] Yibo Jiang, Goutham Rajendran, Pradeep Ravikumar, Bryon Aragam, and Victor Veitch, On the origins of linear representations in large language models, arXiv preprint arXiv:2403.00 200.08700000.77, 2024.
[78] Nora Belrose, Zach Furman, Logan Smith, Danny Halawi, Igor Ostrovsky, Lev McKinney, Stella Biderman, and Jacob Steinhardt. Eliciting latent predictions from transformers with the tuned lens. arXiv preprint arXiv:2303.08/ 12, 2023.
[79] Zihao Wang, Lin Gui, Jeffrey Negrea, and Victor Veitch. Concept Algebra for (Score-Based) Text-Controlled Generative Models, February 2024. arXiv:2302.03693 [cs].
[80] Tomas Mikolov, Wen-tau Yih, and Geoffrey Zweig. Linguistic Regularities in Continuous Space Word Representations. In Lucy Vanderwende, Hal Daumé III, and Katrin Kirchhoff, editors, Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 746-751, Atlanta, Georgia, June 2013. Association for Computational Linguistics.
[81] Julien Mairal, Francis Bach, Jean Ponce, and Guillermo Sapiro. Online dictionary learning for sparse coding. In Proceedings of the 26th annual international conference machine learning, pages 689-696, 2009.
[82]
Alireza Makhzani and Brendan Frey. k-sparse autoencoders, 2014.
[83] Andrew M Saxe, James I. McClelland, and Surya Ganguli. A mathematical theory of semantic development in deep neural networks. Proceedings of the National Academy of Sciences, 116(23) 2019.
[84] Matthew Lyle Olson, Musashi Hinck, Neale Ratzlaff, Changbai Li, Phillip Howard, Vasudev Lal, and Shao-Yen Tseng. Analyzing hierarchical structure in vision models with sparse autoencoders. arXiv preprint arXiv.2505.15970 2025.
[85] Maya Okawa, Ekdeep S Lubana. Robert Dick, and Hidenori Tanaka. Compositional abilities emerge multiplicatively: Exploring diffusion models on a synthetic task. Advances Neural Information Processing Systems, (6:50113-50199 2023.
[86] Ekdeep Singh Lubana, Kyogo Kawaguchi, Robert P Dick, and Hidenori Tanaka. A percolation model of emergence: Analyzing transformers trained on a formal language. arXiv preprint arXiv:2408. 12578, 2024.
[87] Tian Qin, Naomi Saphra, and David Alvarez-Melis. Sometimes am a tree: Data drives unstable hierarchical generalization. arXiv preprint arXiv:2412.04619 2024.
15

---

[88] Angelica Chen, Ravid Shwartz-Ziv, Ziv, Kyunghyun Cho. Matthew L Leavitt, and Naomi Saphra. Sudden drops in the loss: Syntax acquisition, phase transitions, and simplicity bias in mlms. arXiv preprint arXiv:2309.073// 2023.
[89] Y.C. Pati, R. Rezaifar, and P.S. Krishnaprasad. Orthogonal matching pursuit: recursive function approximation with applications to wavelet decomposition. In Proceedings 27th Asilomar Conference Signals, Systems and Computers, pages 40-44 vol. 1993.
[90] Scott Shaobing Chen, David L Donoho, and Michael A Saunders. Atomic decomposition by basis pursuit. SIAM review, 43(1):129 199-1,9,99,9,,,,, 2001.
[91] I. Daubechies, M. Defrise, and C. De Mol. An iterative thresholding algorithm for linear inverse problems with a sparsity constraint. Communications Pure and Applied Mathematics, 57(1 57(11):1433-1557 2004.
[92] Thomas Blumensath and Mike E Davies. Iterative thresholding for sparse approximations. Journal of Fourier analysis and Applications, 14(5-6):629-652) 654, 2008.
[93] Lorenzo Peotta and Pierre Vandergheynst. Matching pursuit with block incoherent dictionaries. IEEE transactions on signal processing, 55(9):4549-455 4557, 2007.
[94] Vladimir Zaigrajew, Hubert Baniecki, and Przemyslaw Biecek. Interpreting CLIP with Hierarchical Sparse Autoencoders, February 2025, arXiv:2502.20578 [cs].
[95] David Chanin, James Wilken-Sminn Tomas Dulka, Hardik Bhatnagar, and Joseph Bloom. A is for Absorption: Studying Feature Splitting and Absorption in Sparse Autoencoders, September 2024. arXiv:2409. 14507 [cs].
[96] J. Deng, W. Dong, R. Socher, Li, K. Li, and L. Fei-Fei. ImageNet: A Large-Scale Hierarchical Image Database. Proceedings of the IEEE Conference Computer Vision and Pattern Recognition (CVPR), 2009.
[97] Xiaohua Zhai, Basil Mustafa, Alexander Kolesnikov, and Lucas Beyer. Sigmoid loss for language image pre-training. In Proceedings of the IEEE/CVF international conference on computer vision, pages 11975-11986, 2023.
[98] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenbom, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010. 1929, 2020.
[99] J.A. Tropp. Greed good: algorithmic results for sparse approximation. IEEE Transactions Information Theory, 50(10))2231 122222, October 2004.
[100] Enrico Fini, Mustafa Shukor, Xiujun Li, Philipp Dufter, Michal Klein, David Haldimann, Sai Aitharaju, Victor Guilherme Turrisi da Costa, Louis Béthune, Zhe Gan, et al. Multimodal autoregressive pre- pre-training of large vision encoders. ArXiv e-print, 2024.
[101] Michael Tschannen, Alexey Gritsenko, Xiao Wang, Muhammad Ferjad Naeem, Ibrahim Alabdulmohsin, Nikhil Parthasarathy, Talfan Evans, Lucas Beyer, Ye Xia, Basil Mustafa, Olivier Hénaff, Jeremiah Harmsen, Andreas Steiner, and Xiaohua Zhai. Siglip 2: Multilingual vision-language encoders with improved semantic understanding, localization, and dense features. ArXiv e-print, 2025.
[102] Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In Computer vision-ECCV 2014 13th European conference, zurich, Switzerland, September 6-12, 2014, proceedings, part 13, pages 740-755 Springer, 2014.
[103] Jayneel Parekh, Pegah Khayatan, Mustafa Shukor, Alasdair Newson, and Matthieu Cord. A concept-based explainability framework for large multimodal models. ArXiv e-print, 2024.
[104] Usha Bhalla, Suraj Srinivas, Asma Ghandeharioun, and Himabindu Lakkaraju. Towards unifying interpretability and control: Evaluation via intervention. ArXiv e-print, 2024.
16

---

[105] Isabel Papadimitriou, Huangyuan Su, Thomas Fel, Naomi Saphra, Sham Kakade, and Stephanie Gil. Interpreting the linear structure of lasnonelanguage model embedding spaces. arXiv preprint arXiv:2.004. 1695, 2025.
[106] Bruno A Olshausen and David Field. Emergence of simple-cell receptive field properties by learning a sparse code for natural images. Nature, 38 38 (6583):607-609,, 1996.
[107] Michael Lustig, David Donoho, and John M. Pauly. Sparse MRI: The application of com- pressed sensing for rapid MR imaging. Magnetic Resonance in Medicine, 5866)1182--115, 2007. _eprint: http://miinniiiaaay.yllllle..omm
[108] Keijo Hämäläinen, Aki Kallonen, Ville Kolehmainen, Matti Lassas, Kati Ninimäkii, and Samuli Siltanen. Sparse Tomography. SIAM Journal on Scientific Computing. 35(3):B644 B665, January 2013. Publisher: Society for Industrial and Applied Mathematics.
[109] Leyuan Fang, Shutao Li, Ryan P. McNabb, Qing Nie, Anthony N. Kuo, Cynthia A. Toth, Joseph A. Izatt, and Sina Farsiu. Fast Acquisition and Reconstruction of Optical Coherence Tomography Images via Sparse Representation. IEEE Transactions on Medical Imaging, 32(11) (((1))2034-2049) November 2013.
[110] Julien Mairal, Michael Elad, and Guillermo Sapiro. Sparse representation for color image restoration. IEEE Transactions image processing, 17(1):33-69, 2007.
[111] Julien Mairal, Francis Bach, Jean Ponce, Guillermo Sapiro, and Andrew Zisserman. Non-local sparse models for image restoration. In 2009 IEEE 12th international conference on computer vision, pages 2279.. IEEE, 2009.
[112] Weisheng Dong. Lei Zhang, Guangming Shi, and Xin Li. Nonlocally Centralized Sparse Representation for Image Restoration. IEEE Transactions on Image Processing, 22(4):1620- 1630, April 2013.
[113] Lee C Potter, Emre Ertin, Jason T Parker, and Müjdat Cetin. Sparsity and compressed sensing in radar imaging. Proceedings of the IEEE,
2010.
[114] Brian Cleary, Le Cong, Anthea Cheung, Eric S. Lander, and Aviv Regev. Efficient generation of transcriptomic profiles by random composite measurements. Cell, 17146))1444-4466 2017.
[115] Brian Cleary, Brooke Simonton, Jon Bezney, Evan Murray, Shahul Alam, Anubhav Sinha, Ehsan Habibi, Jamie Marshall, Eric S Lander, Fei Chen, et al. Compressed sensing for highly efficient imaging transcriptomics. Nature Biotechnology, pages 1-7, 2021.
[116] Joseph Lucas, Carlos Carvalho, Quanli Wang, Andrea Bild, Joseph R. Nevins, and Mike West. Sparse Statistical Modelling in Gene Expression Genomics. In Kim-Anh Do. Marina Vannucci. and Peter Müller, editors, Bayesian Inference for Gene Expression and Proteomics, pages 155-176. Cambridge University Press, Cambridge, 2006.
[117] Daniela M. Witten and Robert Tibshirani. Extensions of sparse canonical correlation analysis with applications to genomic data. Statistical Applications in Genetics and Molecular Biology, 8(1):Article28, 2009.
[118] B. K. Natarajan. Sparse Approximate Solutions to Linear Systems. SIAM Journal on Comput- ing, 24(2):227-234, April 1995. Publisher: Society for Industrial and Applied Mathematics.
[119] Elaine Crespo Marques, Nilson Maciel, Lírida Naviner, Hao Cai, and Jun Yang. A Review of Sparse Recovery Algorithms. JEEE Access, 7:1300-1322, 2019.
[120] David I. Donoho. Compressed sensing. IEFE Transactions information theory, 52(4):128) 1306, 2006.
[121] Emmanuel J Candès, Justin Romberg, and Terence Tao. Robust uncertainty principles: Exact signal reconstruction from highly incomplete frequency information. IEEE Transactions on information theory, 52(2):489-509 2006.
17

---

[122] Emmanuel J Candès and Michael B Wakin. An introduction to compressive sampling. IEEE signal processing magazine, 25(2):21-30, 2008.
[123] Miles Lopes. Estimating unknown sparsity in compressed sensing. In International Conference Machine Learning, pages 217-225. PMLR, 2013.
[124] M. Aharon, M. Elad, and A. Bruckstein. K-svd: An algorithm for designing overcomplete dictionaries for sparse representation. IEEE Transactions on Signal Processing, 54(1 14:11):4311- 4322, 2006.
[125] Robert Tibshirani. Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society. Series B (Methodological), 58(1):267-288, 1996.
[126] J.A. Tropp. Just relax: convex programming methods for identifying sparse signals in noise. IEEE Transactions on Information Theory, 52(3): 1030- March 2006.
[127] Trevor Hastie, Robert Tibshirani, and Martin Wainwright. Statistical learning with sparsity: the lasso and generalizations. CRC press, 2015.
[128] Thomas Blumensath and Mike Davies. Iterative hard thresholding for compressed sensing. Applied and computational harmonic analysis, 27(3):265-22) 274, 2009.
[129] Ron Rubinstein, Alfred M. Bruckstein, and Michael Elad. Dictionaries for Sparse Representa- tion Modeling. Proceedings of the IEEE, 1866))1045-105555 June 2010.
[130] Ivana Tosic and Pascal Frossard. Dictionary Learning. IEEE Signal Processing Magazine, 28(2):27-38, March 2011.
[131] K. Engan, S.O. Aase, and Hakon Husoy. Method of optimal directions for frame design. In Proceedings of IEEE International Conference on Acoustics, Speech, and Signal Processing, vol.5, 1999.
volume 5, pages
[132] Niladri S Chatterji and Peter L Bartlett. Altematinn minimization for dictionary learning: Local convergence guarantees. arXiv:1711.03634 pages 1-26, 2017.
[133] Amir Beck and Marc Teboulle. fast iterative thrinnage-threesholding algorithm for linear inverse problems. SIAM journal on imaging sciences, 2():1133-022, 2009.
[134] Karol Gregor and Yann LeCun. Learning fast approximations of sparse coding. In Proceedings of the 27th International Conference on International Conference on Machine Learning, ICML' 10, page 399-406, Madison, WI, USA, 2010. Omnipress.
[135] Xiaohan Chen, Jialin Liu, Zhangyang Wang, and Wotao Yin. Theoretical linear convergence of unfolded ista and its practical weights and thresholds. In Proceedings of Advances in Neural Information Processing Systems, volume 31, pages 1-11, 2018
[136] Pierre Ablin, Thomas Moreau, Mathurin Massias, and Alexandre Gramfort. Learning step sizes for unfolded sparse coding. In Proceedings of Advances in Neural Information Processing Systems, volume 32, pages 1-11, 2019.
[137] Baharch Tolooshams and Demba E Ba. Stable and interpretable unrolled dictionary learning. Transactions on Machine Learning Research, 2022.
[138] Benoît Malézieux, Thomas Moreau, and Matthieu Kowalski. Understanding approximate and unrolled dictionary learning for pattern recovery. In International Conference on Learning Representations, 2022.
[139] Akshay Rangamani, Anirbit Mukherjee, Amitabh Basu, Ashish Arora, Tejaswini Ganapathi, Sang Chin, and Trac D. Tran. Sparse coding and autoencoders. In Proceedings of IEEE International Symposium on Information Theory (ISIT), pages 36-40, 2018.
[140] Thanh V Nguyen, Raymond KW Wong, and Chinmay Hegde. On the dynamics of gradient descent for autoencoders. Proceedings of International Conference on Artificial Intelligence and Statistics, pages 2858-2867, PMLR, 2019.
18

---

[141] Sanjeev Arora, Rong Ge. Tengyu Ma, and Ankur Moitra. Simple, efficient, and neural algorithms for sparse coding. In Peter Grünwald, Elad Hazan, and Satyen Kale, editors. Proceedings of Conference on Learning Theory, volume 40 of Proceedings of Machine Learning Research, pages 113-149, Paris, France, 03-06 Jul 2015. PMLR.
[142] Thomas Fel, Agustin Picard, Louis Bethune, Thibaut Boissin, David Vigouroux, Julien Colin, Rémi Cadène, and Thomas Serre. CRAFT: Concept Recursive Activation Fac for Explainability, March 2023. arXiv:2211.10154 [es].
[143] Kola Ayonrinde, Michael T Pearce, and Lee Sharkey. Interpretability as compression: Recon- sidering sae explanations of neural activations with mdl-saes. arXiv preprini arXiv:2410.1117, 2024.
[144] Olivier Roy and Martin Vetterli. The effective rank: A measure of effective dimensionality. In 2007 1 15th European signal processing conference, pages 606 610. IEEE, 2007.
[145] Ronen Eldan. HuggingFace: TinyStories- 1M Models, 2023.
[146] Ronen Eldan and Yuanzhi Li. Tinystories: How small can language models be and still speak coherent english? arXiv preprint arXtv:2305.07759. 2023.
[147] Diederik P Kingma. Adam: method for stochastic optimization. arXiv preprini
4rXiv:1412.6980, 2014.
19

---

Appendix
Table of Contents
A Related Work
21
B Extended Experimental Details and Further Results
22
B.1 Synthetic Experiment
22
Large Vision Experiment
31
B.3 Code Structure and Analysis of Dictionary
................................
31
B.4 Multimodal Models
33
B.5
Preliminary Experiments with Language Models
35
C
Theoretical Guarantees for MP-SAE Inference
37
Societal Impact
38
20

---

A Related Work
Sparse coding Sparse linear generative models aim to find sparse representations 2 E Rº that explains the data æ IR via collection of atoms, forming an overcomplete dictionary D E R=xp (p > m.) [106, 32]. Sparse representations are ubiquitous in science and engineering, with origins from computational neuroscience [106, 32] and applications in medical imaging [107-109], image restorations [1 1110--12 12], radar sensing [1 133], transcriptomics [114, 115], and genomics [116, 117].
When the dictionary is fixed, finding the sparsest decomposition of the input æ is known as a sparse approximation problem (i.e., minimizing the fn norm of the representation) [118, 99, 119]. This combinatorial sparse problem is NP-hard and has been the focus of extensive research in compressed sensing [120-123] and signal processing [31. 124]. Classical approaches include greedy ba-based algorithms such as Matching Pursuit [74] and Orthogonal Matching Pursuit [89]. as well as the convex relaxation & basdd methods [90. 125-127] such as Iterative Soft- ceft-Threshoding Algorithm (ISTA) [91, 92] and Iterative Hard-Threshooe Thresholding (IHT) [128]. Since sparse recovery lacks a closed- form solution [1 18], the sparse approximation step typically requires multiple residual-basee iterations to converge.
When the dictionary is learned jointly with the sparse codes, the problem is referred to as sparse coding [32] or dictionary learning [129, 31, 130]. Classical approaches such MOD [131] and K-SVD [124] solve the problem using alternating minimization [132]. The bottleneck of sparse coding is in the convergence rate of the inner problem, which relates to the properties on dictionary atoms; while the slow sublinear convergence of inner iterative algorithms such as ISTA [91. 92] can be accelerated via optimization techniques (e.g., adding momentum [133], there exist no optimization algorithm that can guarantee linear convergence for general sparse coding problem.
Unrolling for sparse coding To address the slow convergence of sparse coding, in 2010, learned ISTA (LISTA) [134] sparked a new approach, which is now known as unrolling, to turn and relax the iterations of ISTA into layers of a neural network accelerate the inner problem. Since then, numerous works have theoretically studied the linear convergence of unrolled ISTA [135, 136), and some others have highlighted the implicit acceleration that this approach may bring in learning the dictionary [137, 138]. Moreover, a parallel literature has theoretically shown that approximate sparse coding, such as a shallow ReLU network [139, 140], or shallow
(recently named as JumpReLU [18) [141, 140] is practically enough to perform dictionary learning.
Interpretability of Neural Networks Recently, [51] has shown that many interpretability methods used to extract concepts were in fact sparse coding methods. The field has re-emerged as a compelling framework for interpreting the internal mechanisms of high-performing 10), particularly in light of persistent challenges surrounding safety and interpretability [11-13].
large-scale models [142, 1-
Within this renewed interest, Sparse Autoencoders (SAEs) have gained prominence as principled method for exposing the latent structure of neural representations [14-2]].
A significant portion of recent interpretability research is underpinned by the Linear Representation Hypothesis (LRH) [24, 25, 25], which posits that nigh-dimensional neural representations can be decomposed as superpositions over large set of approximately orthogonal directions, each aligned with couman-interpretabele concepts. This hypothesis has been theoretically substantiated through formal complexity bounds and constructive frameworks for linearly decodable computation in superposition [26-28].
Empirical support for LRH spans both vision and language domains. In vision models, internal representations have been shown to encode factors such as geometry, lighting, facial structure, and scene composition [34-50]. In language models, analogous patterns emerge, with latent directions capturing syntactic and semantic content [52-57], character roles, arithmetic concepts [58-60], relational structures [63-66], and even selaty behaviors such as refusal to generate harmful outputs [61, 62].
However, recent work has challenged the universality of LRH, arguing that the geometric assumptions behind SAES may not hold in practice [67, 68]. Several studies reveal nonlinear and non-directional structures: hierarchical concepts may align with orthogonal axes, but categorical ones form overlap- ping polytopes [69]; RNNs exhibit "onion-like" layers [70]; and temporal concepts like weekdays often require multi-dimensionall context-sensitive representations [71-73, 21]. These findings high- light the need for improved SAE architectures to capture interpretable concepts that lie beyond the assumptions of the LRH.
21

---

B Extended Experimental Details and Further Results
B.1 Synthetic Experiment
The synthetic experiments were conducted using the codebase provided by Bussmann et al. [20]. Specifically, we introduced the following modifications:
Extended the framework to support both MP-SAE and BatchTopK variants.
Enforced mutual exclusivity among child features, yielding a sparsity of 2.
Introduced variance in the firing magnitudes of the features (see Section B.1.4).
Added controlled intra-level correlations between features.
Before detailing our experimental setup, we first introduce the concept of Matryoshka Sparse Autoencoders (SAFs).
Matryoshka Sparse Autoencoders (SAFs) Matryoshka SAEs [20, 94] extend traditional sparse autoencoders by enforcing accurate reconstruction from multiple nested prefixes of the latent vector. Given an input Rm, the encoder computes = [[ (æ bpre) + b} E IR,, and each prefix m, M is used to reconstruct æ via the first m, latents:
a (m)) Drma. F0rms + Dpre
The set M = m.. ....... My defines a hierarchy of prefix lengths such that my < M2 < ....
m. p. where each m, corresponds to a sub-SAE tasked with reconstructing the input from progressively longer portion of the latent code. In practice, .M can be fixed before training or sampled stochastically per batch to encourage robustness across multiple levels of abstraction.
The total loss encourages reconstructions at multiple levels:
((x)
Σ m.EM
|
(m)) + aLCux.
This design encourages early latents to encode general features and later ones to specialize, improving disentanglement and reducing feature absorption [95] in hierarchical settings.
B.1.1 Dataset Generation
5
13
14
15
16
17
18
19
20
2
7
10
11
Figure 10: Hierarchical tree structure (from [20], also shown in Figure 4) with node indices. Indices are omitted in subsequent figures to avoid visual clutter.
The tree structure contains 20 nodes, each corresponding unit-norm vector in 20 illustrated in Figure and detailed in Figure 10. The exact tree configuration used our experiments is available on the project's GitHub repository*.
Feature Activations The root node (index 0) is fixed to the zero vector, i.c., 0 E 12 All child nodes are mutually exclusive, leading input sparsity levels of either 2. Leaf nodes green parents without children (indices 13 13-20) are independently activated with probability 0.05 (P(ep*
'nttps://githun htat:p////gihab...c//////////// www.c/oae/mmo./////0
22

---

0 = 0.05). Internal green parent nodes (1, 5, and 9) are activated with probability 0.2 (P(2, > 0] 0.2). Blue child nodes (2-4, 6-8, 10-12) are activated with probability 0.2 (f(2,, >0 2 0] 0.2). conditional on their corresponding parent (1,5, 5, or 9) being active, in line with Definition 4.1. This results in an expected target sparsity of &o 1.36.
Activation Magnitudes If a node is active, its sparse code is drawn from a Gaussian distribution (1.5,, 1/4)). This differs from the generative process of Bussmann al. [20], where 2p and Ze are fixed across all inputs, making their firing magnitudes perfectly correlated (Zp- = Xze). We
provide a justification for this design choice Section 1.1.4. The imput æ is then constructed as the sum of the active node vectors scaled by their corresponding code values:
x
2p Dr 2c De-
Introducing Intra-Level Correlations. In the original codebase, all dictionary directions orthogonal, obtained via QR decomposition. To introduce correlations specifically hetween nodes at the same hierarchical level, we perturb each dictionary element as
D, - (1- + ΣDj J
where the sum is restricted elements sharing the same parent, followed by renormalization. The parameter e is tuned empirically to achieve the desired degree of intra-level correlation.
B.1.2 Training
Our training pipeline follows the same procedure as that used by Bussmann et al. [20]. Samples drawn from the tree structure batches of 200 over 15,000 training steps, using the Adam optimizer with 3 = (0.5, 0.9375) and learning rate of 3 x 10-2, To stabilize optimization, gradient norms are clipped at 1.
For sparsity control, all SAEs, except SAESAE, are trained to match a target tu sparsity of 1.36.
Vanilla and Matryoshka SAEs begin with low & regularization during the first 3,000 steps, which subsequently increased or decreased based on whether the observed sparsity is below or above the target.
BatchTopK SAE is trained with a fixed sparsity of for the initial 1,000 steps, after which the sparsity level is reduced to 1.36 for the remainder of training.
MP-SAE, in contrast, does not need to rely on a fixed sparsity target. (make it sound more like an advantage)Instead
unrolls the encoder until either the residual norm drops below 0.05 or infinite loops when the residual cannot be
the support of the sparse code stabilizes-avoll sufficiently reduced. Unlike the other SAEs, MP-SAE uses tied weights.
B.1.3 Detailed Results on Synthetic Data
In this section, we provide detailed results from the synthetic experiments (Figure 4). Figure 1la displays the ground truth correlation matrices used to construct dictionaries with varying levels of intra-level correlation; we focus here on four representative settings: 0, 0.3, 0.6, and 0.9.
All models are evaluated using a fixed sparse code shown in Figure 11b, which depicts the ground truth activation matrix z E R xp, When varying the degree intra-level correlation, Vanilla SAE and BopK consistently exhibit feature absorption. Matryoshka SAE sometimes alleviates this issue but does so by suppressing intra-level correlations entirely. contrast, MP-SAE reliably recovers both the hicrarchical structure and intra-level correlations at low moderate levels (0, 0.3) and at high correlation levels (0.9), it prioritizes preserving the hierarchy over matching correlation. We summarize our detailed observations for each correlation level below:
Figure 12 (Correlation 0): We observe feature absorption in both the Vanilla SAE and BatchTopK models. Specifically, features corresponding to child nodes are often aligned with those of their parent nodes, as evidenced by prominent horizontal structures the second row of Figure 12. This indicates that the recovered child features (14,--8,10-11 are not disentangled from their respective parents (1,5,9). While the Matryoshka SAE occasionally mitigates this issue (see rightmost runs in the Matryoshka column of Figure 12), it does by introducing
23

---

negative correlations between features at the same level, which can result in partial misalignment. In contrast, MP-SAE perfectly recovers the ground truth dictionary without exhibiting feature absorption. In terms of support recovery and sparse code estimation, all methods (excluding the failed runs of Matryoshka rows and 4) correctly identify the active set. However, in Vanilla SAE and MatchTTopk, feature absorption leads to underestimation of parent activation values when a child is active. When successful, Matryoshka mitigates this issue. MP-SAE achieves exact recovery, with the inferred sparse codes & matching the ground truth both in support and magnitude.
Figure 13 (Correlation = 0.3): Feature absorption remains a consistent issue in both Vanilla SAE and BatchTTpp.. Nonetheless, these models partially reflect the ground truth intra-level correlations among the green parent nodes without children (13-20). Matryoshka SAE continues exhibit inconsistent behavior: while it occasionally reduces feature absorption, it fails to consistently capture intra-level dependencies. In contrast, MP-SAE maintains perfect recovery, accurately reconstructing the sparse codes with 2 and showing no evidence of feature absorption.
Figure 14 (Correlation 0.6): Vanilla SAE and BatchTopK partially capture intra-level correla- tions, with correlation values closest to the ground truth among the blue child nodes. However, these correlations remain weaker or less accurate for other node types. Matryoshka SAE con- tinues to struggle, failing to recover meaningful intra-level dependencies. In contrast, MP-SAE achieves partial recovery of intra-level correlations correlations--pariicee particularly among parent nodes without and
aareeaatioonpptttlaaaary children and more faithfully preserves the hierarchical organization the dictionary through its inductive bias toward conditional orthogonality.
Figure 15 Correlation 0.9): At high levels intra-level correlation, all SAEs exhibit degraded performance in recovering features. Vanilla SAE and BatchTopK still capture some similarity among the last-layer children but tend flatten the hierarchy by focusing on correlated low-level features while neglecting higher-level ones. MP-SAE, interestingly. separates each hierarchical level into two, resulting in four orthogonal sublevels composed of 10, 3, and features, respec- tively. Within each ground-truth level of correlated features, MP-SAE learns one anchor direction capturing the shared structure, and additional orthogonal directions that describe deviations from this anchor to reconstruct the remaining features, observed in the correlation matrices D D. Since it is not trained with a fixed target sparsity, this separation achieved by increasing the support: most inputs now activate 2-4 components compared the ground-truth sparsity of 1-2. This behavior preserves conditional structure even under strong entanglement-at the cost of reduced correlation illustrating MP-SAE's bias toward maintaining hierarchical organization when representations are highly coupled.
Correlation
Correlation 0.3
a5
Batch (50)
0.25
Correlation 0.6
Correlation 0.9
-025
.05
075
GT Features
(a) Ground truth correlation matrices used in the ex- periments, corresponding four levels of intra-level correlation: 0, 0.3, 0.6, and 0.9.
(h) Ground truth reference activation matrix used for generating synthetic inputs the following experiments.
Figure 11: Ground truth structures used for the synthetic experiments. (a) shows the correlation levels introduced among dictionary elements at each setting. (b) shows the corresponding ground truth activation patterns used to construct the inputs.
24

---

Vanilla
BatchTopk
Matryoshka
MP
Features
0.75
0.5
0.25
GT Features
-0.25
-0.5
-0.75
Features
Features
Features
Features
Batch (50)
Features
Features
Features
Features
Figure 12: runs for correlation level 0. Top: Correlation matrices the top row shows D T D (self-similarity of learned features), and the bottom row shows D D (alignment with ground truth). Bottom: Recovered sparse codes z.
25

---

Vanilla
BatchTopK
Matryoshka
MP
Features
0.75
0.5
0.25
GT Features
-0.25
-0.5
-0.75
Features
Features
Features
Features
Batch (50)
Features
Features
Features
Features
Figure 13: runs for correlation level 0.3. Top: Correlation matrices the top row shows D T D (self-similarity of learned features), and the bottom row shows D D (alignment with ground truth). Bottom: Recovered sparse codes z.
26

---

Vanilla
BatchTopK
Matryoshka
MP
Features
0.75
0.5
0.25
GT Features
-0.25
-0.5
-0.75
Features
Features
Features
Features
Batch (50)
Features
Features
Features
Features
Figure 14: runs for correlation level 0.6. Top: Correlation matrices the top row shows D T D (self-similarity of learned features), and the bottom row shows D D (alignment with ground truth). Bottom: Recovered sparse codes z.
27

---

Vanilla
BatchTopK
Matryoshka
MP
Features
0.75
0.5
0.25
GT Features
-0.25
-0.5
-0.75
Features
Features
Features
Features
Batch (50)
Features
Features
Features
Features
Figure 15: runs for correlation level 0.9. Top: Correlation matrices the top row shows D TTp D (self-similarity of learned features), and the bottom row shows D D (alignment with ground truth). Bottom: Recovered sparse codes z.
28

---

B.1.4 Introducing Variance in Firings
In the original codebase of Bussmann et al. [20], activation magnitudes are fixed across all inputs. This implicitly assumes that parent and child firings are perfectly correlated:
Aze-
Such a configuration defines a strict hierarchical setting that inherently favors feature absorption. In contrast, our formulation does not impose any constraint on the correlation between firing mag- nitudes across concepts. Our setup generalizes the original formulation by removing the perfect correlation assumption. This is achieved by introducing variance in the activation magnitudes while maintaining the sole structural assumption that a child feature can only be active if its parent is active (Definition 4.1).
Rationale. To illustrate the consequences of assuming perfect correlation in activation magnitudes, consider two concepts: "red object" (parent) and "apple" (child). The child can only activate when the parent does (assuming all apples are red). If the parent's ' firing magnitude 2p represents color intensity and the child's ze represents size, assuming Ep and ze are perfectly correlated is equivalent stating that larger apples are always redder. This perfectly merges the two informational components-cokke intensity (zp Dp) and apple size Dz) into a single absorbed direction 2p Dp ze De, thereby eliminating their hierarchical distinction. In this extreme case, becomes unclear whether the absorbed feature should still be interpreted as child, since its activation is perfectly predictable from the parent. We argue that imperfect correlation between Ep and ze is a necessary condition for preserving hierarchical interpretability.
Implications for Sparse Methods. Following Occam's 's razor, an fu-penalized method such as MP-SAE will optimally encode both parent and child with single feature when their activations are nearly identical, since the absorbed feature forms a highly compressible direction [143]. Encoding the absorbed feature 2n D, + z Dr yields a sparsity of 1, whereas disentangling parent and child requires a sparsity of 2. Therefore, it is optimal for MP-SAE to represent the parent and child through single absorbed feature when it can perfectly reconstruct æ with lower sparsity, which is precisely what occurs when no firing variance is introduced, as in the original benchmark. However, because of the greedy nature of Matching Pursuit, MP-SAE may still learn to disentangle the parent and child, depending on initialization. Figure 16 illustrates this phenomenon: it shows the correlation matrices and ground trrth alignment for the unmodified Matryoshka SAEs benchmark, trained for 15,000 iterations with a threshold of T = 0.6 for MP-SAE.
Additional results with different child firing distributions. In our generative data process, both parent and child activations initially follow the same distribution, Zp (N.5,, 1/42) and Ze (115, 1/12). However, high variance combined with equal means (up = pc) can produce edge cases where a strong child and a weak parent yield æ = Zp Dp + 2eD, z z DeDe, effectively making the parent contribution negligible and thus deviating from Definition 4.1.
To assess the robustness of MP-SAE under varying child activation statistics, we evaluated its performance across grid of child means and variances, while keeping the parent distribution fixed as Ep ~ (1,, 1/4²). For each configuration, we computed an absorption score, defined as the average cosine similarity between the learned child features and their corresponding ground-truth parents over 15 independent runs. Lower similarity indicates successful disentanglement, whereas higher similarity reflects feature absorption. NaN values correspond to cases where the model failed to recover the child feature due to its very low firing amplitude. All activations were constrained to remain positive when active (entries marked "_" in Table denote untested configurations).
Across all evaluated conditions, MP-SAE consistently achieves low absorption scores, confirming its robustness and its ability to recover hierarchical structure even under challenging firing regimes. Indeed, when the child average firing magnitude particularly low (0.1), MP-SAE remains the only method capable of recovering the hierarchical structure, owing to its iterative greedy encoder. Interestingly, when the variance of the child distribution decreases for mean of 0.5-a regime where overall performance becomes comparable to Matryoshka-Bae the two methods exhibit opposite trends: Matryoshka performs better at low variance, whereas MP-SAE improves as variance increases. This observation suggests that the two approaches embody complementary inductive biases: Matryoshka is
29

---

particularly effective in scenarios dominated by strong feature absorption and near- pear-perfect correlation, while MP-SAE excels when moderate variance is present in the firing magnitudes.
Matryoshka
Dictionary
GT
GT
GT
Dictionary
MP
Vanilla
Dictionary
GT
Dictionary
Dictionary
Figure 16: Evaluating MP-SAE on the original Matryoshka benchmark', where firings are perfectly correlated. For each SAE, four independent runs are shown. The top row displays D D (self-similarity of learned features), and the bottom row shows D D (alignment with ground truth). Top left: ground-truth correlation matrix, where features are perfectly orthogonal. Matryoshka overcomes feature absorption at the cost of negative interference. Vanilla consistently learns absorbed features. MP-SAE occasionally recovers parent-child pairs thanks to its inductive bias, but under perfect correlation, its behavior unstable.
Table 1: Comparison of absorption scores across sparse autoencoders for different child ac- tivation distributions. Child activations follow normal distributions with varying means (rows) and standard deviations (columns). while the parent distribution is fixed as 2p ~ [1, 3/1²). Lower absorption scores indicate better disentanglement between parent and child features. MP-SAE main- tains low absorption across settings and remains the only method robust to low mean child firings.
MP
0.1
0.5
1.0
Matryoshka
0.1
0.5
1.0
1/400
3,6e-03
7.3e-02
1.3e-03
1/400
nan 4.8e-02
2.1e-01
1/40
5.0e-03
6.2e-02
1.2e-03
1/40
nan
9.0e-02
1.5e-01
1/8
4.3e-02
1.1e-03
1/8
9.6e-02
2.1e-01
1/4
1.1e-03
1/4
2.2e-01
Vanilla
0.1
0.5
1.0
Botc
0.1
0.5
1.0
1/400
nan
7.0e-01
4.5e-01
1/400
nan
5.0e-01
4.8e-01
1/40
nan
7.0e-01
4.5e-01
1/40
nan
5.4e-01
4.5e-01
1/8
7.0e-01
4.7e-01
1/8
4.9e-01
4.6e-01
1/4
4.7e-01
1/4
4.2e-01
30

---

B.2 Large Vision Experiment
We train our vision SAEs activations from four pretrained vision and VLM models: CLIP, DINOv2, SigLIP, and ViT. Representations are extracted from the final layer of each model, utilizing all spatial tokens (e.g., DINOv2 yields approximately 261 tokens per image). Each SAE processes these token- wise activations independently. Training is conducted on the ImageNet- 1 training set, comprising around 1,3 Millions images. Over 50 epochs, this results in approximately 1.3 10 x0x26 billion input tokens (for DINOv2). We employ a batch size of 8,000 tokens per step and train all models using the AdamW optimizer with cosine learning rate schedule: the learning rate warms up from 10-6 to 5 x 10 and decays back to 10 by the final epoch. A fixed weight decay of 10 applied throughout. All SAEs utilize an expansion factor of 25, meaning the learned dictionary D E [Rexd satisfies с = 25d, where d is the dimensionality of the input activations. Each column D; constrained to lie the unit & ball: D 1. The loss given is the standard MSE. To maintain active support coverage, a revive factor of 10 is added to any pre-code unit that fails to activate in given batch, slightly increasing its pre-activation to reintroduce gradient flow. For Vanilla SAEs, we apply an adaptive & penalty: if the empirical to sparsity of batch exceeds a target threshold, the I regularization weight is increased to suppress overactivation. All encoder architectures consist of one-layer linear projection followed by ReLU activation. For the pareto results, one SAE is trained for each configuration (sparsity, models).
16.9
B.3 Code Structure and Analysis of Dictionary
Effective Rank of Feature Co-Activation. To quantify the diversity of feature usage in sparse autoencoders, we compute the effective rank of the co-activation matrix Z Z. where E gaxp contains the sparse codes across inputs. Each entry in Z Z reflects how often pairs of features are jointly active, and its spectral structure reveals how concentrated or distributed these co-activations are. The effective rank [144] is defined as:
Erank 2 2) exp Σ A log A
(2)
where À; the eigenvalues of Z Z normalized sum 1. This corresponds the exponential of the Shannon entropy of the spectrum. High effective rank indicates that feature co-activations are spread across many directions (i.e., more diverse, less redundant usage), while low effective rank suggests repeated use of a few dominant feature combinations.
Pairwise Coherence of Dictionary Features. To further analyze the internal structure of learned dictionaries, we examine the distribution pairwise inner products DJ which captures the angular alignment between features. Figure 17 shows histograms ofof the values for dictionaries trained on the vision models. We used the SAES from the pareto front, with a fixed inference-ting sparsity of A 5. We observe that dictionaries Icarned by standard SAEs (c.g., Vanilla, TopK) exhibit distribution sharply peaked at zero, reflecting a strong bias toward global quasi-orthogonale direct consequence of their encoding mechanism: features are selected via a single global projection, which incentivizes mutually uncorrelated features to avoid interference.
This is
By contrast, dictionaries learned by MP-SAE display broader distributions, including significant mass at nonzero values. This reflects an important difference inductive bias: MP-SAE does not enforce orthogonality globally. Instead, its sequential, residual-guided encoder dynamically selects features that are orthogonal conditioned on prior selections. This means that the dictionary can afford to contain closely aligned features as long as they do not co-activate for the same input.
31

---

lo6 dinov2 batchtopk
166
dinov2 topk
106
dinov2 vanilla
1c6
dinov2 mpbase
00 Histogram
4-
4-
2
-1.0
0.0
-1.0
-0.5
0.0
0.5
-0.5
0.5
1.0
-1.0
-0.3
0.0
1.0
(a) Pairwise inner products (D" D) for dictionaries trained on DINOv2 at À 5.
siglip batchtopk
siglip topk
siglip vanilla
le6
siglip mpbase
D01 Hictoo rmm
3-
-1.0
-0.5
0.0
0.5
1.0
-1.0
-0.5
0.0
0.5
1.0
-10
-0.5
0.5
1.0
-1.0
-0.5
0.0
0.5
1.0
(b) Pairwise inner products (D D) for dictionaries trained on Siglip at k 5.
1e6
clip batchtopk
1e6
clip tapk
1e6
clip
1.0
1e7
clip mpbase
5-
0.8
00 Hizogram
4-
3
0.4
2
2-
1
0.2
0
0.0
(c) Pairwise inner products (D) D) for dictionaries trained on Clip at k
5.
vit batchtopk
vit topk
vit vanilla
1e6
vit mpbase
DD1 Histograin
5
3
2
1.0
6.5
0.0
1.0
1.0
0.5
0.0
0.5
1.0
1.0
0.5
0.5
1.0
1.0
0.5
0.0
0.5
1.0
(d) Pairwise inner products (D" D) for dictionaries trained VIT k
Figure 17: Distribution of dictionary coherence (DT D) reveals differing inductive biases. Standard SAEs encourage globally orthogonal features; MP-SAE tolerates correlated features and resolves interference at inference.
32

---

B.4 Multimodal Models
Setup. We evaluates SAES of 4 VLMs: CLIP, SigLIP. SigLIP2, and AIMv2. We use the final layer of the shared embedding space that is, the common representation into which both image and caption inputs are projected. This layer reflects the aligned modaaltty-ivvariane space optimized by contrastive pretraining. We train all SAEs on the full MS-COCO dataset [102], which consists of approximately 100,000 images and 500,000 associated captions. Each training example corresponds to a single embedding vector (either image or text), and models are trained jointly across both modalities using shared dictionary (expansion factor 25). As in prior settings, we constrain each dictionary column to lie on the unit & ball. All SAES use a one-layer encoder followed by ReLU activations and are trained with mean squared error (MSE) loss. We fix the target inference-time sparsity to ||=||o 5. The optimization procedure mirrors that used for vision models for total of 20 training epochs. The same revive mechanism applied, whereby inactive units are nudged via small additive bias (10 5) to encourage gradient flow.
Modality score To quantify the extent to which learned features specialize by modality or capture shared structure, we compute the Modality Score for each feature, following the formulation of [105]. Values near indicate image-specific features; values near 0 indicate text-specific features; and intermediate values reflect balanced, multimodal activation. reflect relative activation patterns rather than absolute energy differences between modalities, we scale the energy of text inputs by factor 1/5 before computing the above quantities. This corrects we have 5 times more captions than images. We find that the same overall patterns emerge when we apply a modality wise normalization.
ensure that Modality Scores
The figure 18 illustrates the modality gap and how different inference strategies shape learned representations langual models. Each point on the sphere represents direction in shared embedding space, such as the one learned by CLIP [75], where both images and captions are aligned.
On the left, standard sparse autoencoders (SAEs) tend to learn split dictionaries [103, 104), assigning different features to each modality despite their alignment in the joint space. This results in high modality selectivity: visual and textual inputs activate disjoint sets of features.
On the right, MP-SAE progressively refines its representation by explaining modality-specine components in early steps and shifting focus to shared structure in later steps. This allows the model to discover genuinely multimodal concepts that respond to both image and text inputs, effectively bridging the modality gap and producing more balanced, semantically aligned features.
SAE
MP-SAE
Text Zebra
Text
Zebra
Image Zebra
Image
Figure 18: Illustration of seletity- vs. multimodal inference. Each point represents
an image or caption embedding on the unit sphere of a joint representation space. Left: standard SAEs learn split dictionaries, with features specializing in only one modality (blue = text, yellow = immage), despite semantic alignment. Right: MP-SAE explains modality-speciff content in early steps and aligns shared concepts in later ones. For instance, an image of a zebra and the text "a zebra in the savannah" may initially activate modal specific features, but converge toward shared features encoding the concept "zebra."
33

---

Topk
BatchTopk
MP
1000
4000
4000
AIMV2
2000
2000
500
4000
SigLIP
2000
4000
400
Counts
2000
200
400
SigLIP 2
4000
2000
2000
200
4000
4000
1000
CLIP
2000
2000
500
0.5
0.5
0.5
Modality Score
Figure 19: Distribution of modality scores for features learned by different SAEs, across four vision- language models: CLIP, SigLIP, SigLIP2, and AIMv2. Standard SAEs (e.g., TopK, BatchTopK) tend to produce bimodal distributions, with many units activating exclusively for image or text inputs, indicating strong -ppelity-specific specialization. In contrast, MP-SAE yields a significantly flatter distribution with substantial density near 0.5, revealing the emergence of genuinely multimodal features that respond both modalities. This supports the hypothesis that MP-SAE's inference allows it to progressively isolate shared semantic structure after accounting for modality-speciff variance.
34

---

B.5 Preliminary Experiments with Language Models
Vanilla
B.5.1 Experimental details
0.9
To assess whether our analysis of the vision do- main (see Sec. B.4) generalizes language modeling setting, we perform preliminary exper- iments using a 4-layer Transformer model [145] pretrained on the TinyStories dataset [146]. 6 We train the Vanilla [14], TopK [15], and MP- SAE (see Sec. 3) on residual stream representa- tions extracted from midway through the model, i.., end of the second Transformer block (from
0.8
0.7
0.6
10
100
Topk
Normalized MSE
amongst 4 total blocks). p. which refers to the width of our SAEs, ie.. the size of the sparse code 2, set to corresponds to the dimensionality of the resid-
m for all SAEs; here, m
ual stream. For Topk and MP-SAE, we set to = 100 for any inputted representation; for Vanilla SAE, we search for value to ensure after training fo, on average, is 100. Inline with prior work [22], activations are standardized by computing a population mean and standard de- viation and rescaled to be, on expectation, v magnitude in norm. Training is performed using Adam optimizer [147]. with a constant learning rate of 10-3, [3, B2 0.9, 0.95, weight decay of 10-4, and gradient clipping at unit-norm mag- nitude for all the SAEs. Batch-size is set to be 5000 tokens, each of which is randomly sam- pled from samples (stories) from the train-split of TinyStories. Training goes on for approx- imately 40K iterations. All results in the fol- lowing experiments are performed on samples drawn from the eval-split of TinyStories.
10
100
MP
0.1
Train Lg
0.01
teeaaion 0
teeation 41000
10
100
Inference Lg
Figure 20: Normalized MSE with increase in training iterations and inference-time sparsity. We train Vanilla, TopK, and MP SAEs and plot the normalized MSE between their predicted vs. ground-truth representations from models trained on TinyStories. Results clearly show the flexibility of MP-SAB: increasing number of latents used for reconstruction monotonically reduce error, allow- ing one use more less latents, as desired. This property is emergent with training, however, since carlier iterations of MP-SAE do exhibit it.
B.5.2 Results
Reconstruction Error with Inference Lg- Similar to Fig. 8, for all SAEs, we take their parameters trained up to iteration and compare the inputted representations (x) with the ones predicted by the SAE given its largest magnitude latents the sparse code, denoted zt,k- Specifi- cally, we compute the normalized MSE, E2 |||=,==1||// [||] for increasing values of and k. We note this experiment is different from the one reported in Fig. 8 because it also assesses the effects of training, hence going beyond highlighting the inference -ime flexibility of MP-SAE. Such an analysis is manageable in the current setting since models are not exceptionally big (~ 1M parameters).
similar trend as Fig. 8: is increased, Vanilla SAES smoothly move towards minimum amount of error that corresponds to the value of k used for training; TopK SAEs see reduction in eттor up to the value of & used for training, witnessing large increase in error thereafter; and MP-SAEs again exhibit their flexibility, with error monotonically decreasing even beyond the value of used for training. flexibility of MP-SAE only emerges after enough training has occurred in fact, earlier checkpoints exhibit a saturation or lower-bound of loss akin the Vanilla SAEs. Finally, we emphasize again that this inference-time flexibility is natural consequence of building error residuals, unlike prior SAEs. Prior work hoping to elicit such behavior had to manually train via a mixture of sparsity values [15].
Results shown in Fig. 20, where we most noticeably
worth noting that this
We also provide trained MP-SAE Gemma activations https: :tttpp:///ttttt cww/ coam/ssllbee wcmm///suaaaa/msaa.
35

---

Babel Score Analysis on Language Models. Following the analysis in Figure 7, we compute the Babel scores for dictionaries learned on language model embeddings using MP, Vanilla, and TopK SAEs. Results are shown in Figure 21, reporting both the coherence of the full dictionary (left) and that of the concepts co-activated at inference (right).
The same trends observed in the vision models hold here as well. MP-SAE learns dictionaries with higher Babel scores globally-indicatnng more interference across features-but selects sets of concepts with lower mutual interference at inference time. This again reflects MP's bias toward conditional orthogonality. Conversely, Vanilla and TopK SAES learn more globally incoherent dictionaries, but their inference- time selections often include more correlated features. Notably, the y-axis ranges in Figure 21 are similar to those from the vision experiments 7, suggesting that this behavior is not mooecitt--peccifi indicating that this effect is not data dependent.
Learned
Selected
120
Babel Score (r(
MP
MP
100
Vanilla
Vanilla
10
TopK
Topk
80
8
40
20
2
25
50
75
100
125
150
0 20
40 60 80
100
r
Figure 21: Babel score analysis on language models. We report the coherence of the full learned dictionary (left) and the subset of concepts co-activated at inference (right), across MP, Vanilla, and TopK SAEs. As in the vision setting, MP learns globally coherent dictionaries but selects less interfering concepts inference, while Vanilla and Topk show the opposite pattern. The similarity in value ranges across modalities suggests that these effects not data dependent.
Effective Rank. We compute the effective rank of the co-activation matrix for MP, ReLU, and TopK SAEs trained language model embeddings.
As shown in Figure 22, MP-SAE exhibits a non-monotonic trend: sparsity, peaks at critical point, and then declines. This suggests that MP initially promotes diverse feature use before redundancy emerges. In contrast, ReLU shows a steady decline and eventually saturates, while TopK decays slowly but drops sharply when the inference exceeds the training level. The drop in effective rank reflects increasing feature reuse and reduced diversity the representation.
the effective rank increases with
MP
Vanilla
TopK
120 29
Effective Rank
20
2
25xx10
24x.01
12x30
13xx19
33
22x
10-
Train Lo
Inference L.
Figure 22: Effective rank of the co-activation matrix Z T Z. MP-SAE shows rise-then-drop trend, indicating initial diversification followed by redundancy. ReLU steadily declines and saturates, while TopK decays slowly but drops sharply when inference sparsity exceeds training.
36

---

C Theoretical Guarantees for MP-SAE Inference
We restate three foundational properties of Matching Pursuit-originan established in the sparse coding literature [74]-and interpret them in the context of sparse autoencoders. These properties help elucidate the structure and dynamics of the representations learned by MP-SAE.
Stepwise orthogonality (Proposition C.I): at each iteration, the residual becomes orthog- onal to the feature most recently selected by the greedy inference rule. This sequential orthogonalization mechanism gives rise to a locally disentangled structure in the representa- tion and reflects the conditional independence induced by MP-SAE inference.
Monotonic decrease of residual energy (Proposition C.2): the lz norm of the residual decreases whenever it retains nonzero projection onto the span of the dictionary. This guarantees that inference steps lead to progressively refined reconstructions, and enables sparsity to be adaptively tuned at inference time without retraining.
Asymptotic convergence (Proposition C.3): in the limit of infinite inference steps, the reconstruction converges the orthogonal projection of the input onto the subspace defined by the dictionary. Thus, MP-SAE asymptotically recovers all structure that is representable within its learned basis.
Proposition C.1 (Stepwise Orthogonality of MP Residuals). Let (() denote the residual at iteration t of MP-SAE inference, and let j(t-1) be the index of the feature selected at step t-1. If the column j(t-1) of the dictionary D satisfy Dyte v bll22 1, then the residual becomes orthogonal to the previously selected feature:
ja ())) = 0.
Proof. This follows from the residual update:
(t) ((t-1) - Dyu 22 j0m)
(t-1)
with (1-1)
D DT- JT-r1) Taking the inner product with Dyct-13 gives:
3(())
D - -D [DDa n 11l
(t-1) (1-1) (t-1) je ((t)) j(n)
0.
This result captures the essential inductive step of Matching Pursuit: each update removes variance along the most recently selected direction, producing residual that is orthogonal to it. Applied iteratively, this localized orthogonality promotes the emergence of conditionally disentangled structure in MP-SAE. In contrast, other sparse autoencoders lack this stepwise orthogonality mechanism, which helps explain the trend observed in the Babel function during inference in Figure 7.
Proposition C.2 (Monotonic Decrease of MP Residuals). Let (()) denote the residual iteration of MP-SAE inference, and let 200 residual norm decreases monotonically:
2(0) be the nonzero coefficient selected at that step, Then the squared
|nttt1 || = -I| ( 200 S O.
Proof. From the residual update:
(1+1) -p(() - D 2 j(b)
(0)
we can rearrange to write:
10 = 110)p((1)) ((+1) + D 200
(8)
Taking the squared norm of both sides:
|| (1) || = [|r()) + D Djo 200) 300 112
= [| 112 + ((1() Dyw)) + D z
j(()) ll.
(t)
37

---

By Proposition C.I, the cross term vanishes:
(n(++), -)(o)) =0,
yielding:
(1)))) (1((++)))))))) 11 +IID 10.
The monotonic decay residual energy ensures that each inference step yields an improvement in reconstruction, long the residual lies within the span of the dictionary. Crucially, this property enables MP-SAE to support adaptive inference-time sparsity: the number of inference steps can be varied at test time time- inne-indeenndetlly of the training setup-while still allowing the model to progressively refine its approximation. This explains the continuous decay observed in Figure 8, guarantee not provided by other sparse autoencoders.
Proposition C.3 (Asymptotic Convergence of MP Residuals). Let a( x p(() denote the reconstruction at iteration t, and let PD be the orthogonal projector onto span D). Then:
lim
P lim P
This convergence result is formally established in the original Matching Pursuit paper by Mallat and Zhang [74, Theorem IJ. This result implies that MP-SAE progressively reconstructs the component of æ that lies within the span of the dictionary, converging to its orthogonal projection in the limit of infinite inference steps. When the dictionary is complete (i.e., rank(D) = m), this guarantees convergence the input signal x.
D Societal Impact
Interpretability plays a critical role the safe and trustworthy deployment of systems. As large- scale models are increasingly integrated into everyday technologies and used by millions of people, the risks associated with their opaque decision-make grow substantially. Without interpretability. becomes difficult to detect biases, failures, unintended behaviors in these powerful systems. By enabling the extraction structured, interaanettereetbblee features, tools like Sparse Autoencoders help researchers and practitioners understand how models encode semantics, reasoning patterns, or social attributes. This transparency is especially important in safety-critical domains such as healthcare, legal or unfair outcomes. Interpretability methods also support model auditing and targeted interventions, making it possible to align AI behavior with human values. However, such tools also be misused-for example, to reverse-engineer proprietary models or infer sensitive attributes. We emphasize that method does not amplify these existing risks.
making, education, where opaque model behavior can result in harmful
38

---

NeurIPS Paper Checklist
1. Claims
Question: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope? Answer: [Yes]
Justification: The abstract and introduction clearly motivate the study by questioning whether standard sparse autoencoders (SAEs), which are designed under the Linear Representation Hypothesis (LRH), can faithfully recover features when model representations are hierarchi- cal, nonlinear, or multidimensional. The paper then proposes MP-SAE as an architecture that operationalizes new inductive bias- conddiiiiai orthogonality throughh a sequential matching pursuit-basee -based inference mechanism. These claims are directly substantiated by the contributions and experiments in the main body, including synthetic benchmarks, concept organization analysis, expressivity comparisons, and results on vision-language models. Guidelines:
The answer NA means that the abstract and introduction do not include the claims made in the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation long it is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors? [Yes]
Answer:
Justification:
The paper includes discussion of limitations in the conclusion
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors are encouraged to create separate "Limitations" section in their paper.
The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution low or images are taken in low lighting. Or speech-to-text system might not be used reliably provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof? Answer: [NA]
39

---

The paper introduces definitions and theoretical motivations (e.g., conditional orthogonality, cumulative coherence via the Babel function), but it does not contain formal theorems or proofs. Our contributions are primarily empirical and architectural rather than theoretical
Justification:
Guidelines:
The answer NA means that the paper does not include theoretical results. All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
The proof' can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch to provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented
by formal proofs provided in appendix or supplemental material. Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification: We provide detailed descriptions of our experimental setups, datasets, and evaluation metrics in the main text. Further implementation details, including model configurations, training procedures, and hyperparameters for each experiment, will be included in the appendix to ensure full reproducibility.
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that
appropriate to the research performed.
While NeurIPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily a new algorithm, the paper should make it clear how to reproduce that algorithm. (b) If the contribution is primarily new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is a new model (e.g., a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open-source dataset instructions for how to construct the dataset).
(d) We recognize that reprodueibility may be tricky in some cases, in which authors are welcome to deseribe the particular way they provide for reproducibility. In the case of closed-source models, it may be that access to the model is limited in some way (e.g.. to registered users), but it should be possible for other researchers have some path reproducing or verifying the results. 5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
40

---

Answer: [Yes]
Justification: We will release all code used in our experiments in a public GitHub repository.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurlPS code and data submission guidelines (https://nips cc/
cantimiiiinnooicy for more details.
While we encourage the release of code and data, we understand that this might not be possible, so "No" an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to rum to reproduce the results. See the NeurlPS code and data submission guidelines (https:
// //nips. wttpcc//ppl... cc/public//eeeccccccccccccce gww.eu/iee//eee (oodeSubiissiiii wwmiiissooooiioooo
for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended the paper) is recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: We describe the experimental setup, datasets, and evaluation metrics in the main text to support the interpretation of results. Detailed training configurations, includ- ing data splits, optimizer type, learning rates, number of iterations, and model-specific hyperparameters, are provided in the appendix to ensure reproducibility and clarity. Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary to appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?
Answer:
[No]
Justification: We do not report error bars or statistical significance measures. For the tree tree-structured experiment, we show results across multiple instantiations to demonstrate consistency. For the other experiments, variability was not major concern, and single-run results were sufficient to support our qualitative and comparative claims. Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, statistical significance tests, least for the experiments that support the main claims of the paper.
The factors of variability that the error bars capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall
with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula,
call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
41

---

It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables the text. 8. Experiments compute resources
Question: For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?
Answer: [No]
Justification: We do not report detailed compute resource specifications (e.g.. GPU type, memory, runtime). However, the experiments in this paper involve lightweight models and synthetic or publicly available datasets, and can be reproduced with standard hardware (e.g., single GPU). As such, we did not find detailed compute reporting to be critical in this case, though we can include it in the final version if needed. Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper). 9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics https / /tt.rpsurr.pp/ee cc
Answer: [Yes]
Justification:
Guidelines:
The answer NA means that the authors have reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. if there is a special consid- eration due to laws or regulations in their jurisdiction).
10. Broader impacts
Question: Does the paper discuss both potential positive socictal impacts and negative societal impacts of the work performed? Answer: [Yes]
Justification: A discussion of potential societal impacts, including both positive and negative aspects, will be provided in the appendix.
Guidelines:
The answer NA means that there no societal impact of the work performed.
If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact.
Examples of negative socictal impacts include potential malicious unintended uses (c.g., disinformation, generating fake profiles, surveillance), fairness considerations (c.g., deploymennt of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out, For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point that a generic a algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
42

---

The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility ML).
11. Safeguards
Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)? Answer: [NA]
Question:
Justification:
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [NA]
Justification:
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset used and, possible, include a URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset. For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, snpprsswihccod com/ datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that are re-pa se-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. this information not available online, the authors are encouraged to reach out to the asset's ss creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [NA]
Justification:
Guidelines:
The answer NA means that the paper does not release new assets.
Researchers should communicate the details of the dataset/code/moodll as part of their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset used.
43

---

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?
Answer: [NA]
Justification:
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with
human subjects.
Including this information the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then much detail as possible should be included the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals or equivalent for research with human subjects
Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or equivalent approval/review based on the requirements of your country or institution) were obtained?
Answer: [NA]
Justification:
Guidelines:
The answer NA means that the paper does not involve crowdsoureing nor research with
human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent)
may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if it is important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing, editing, formatting purposes and does not impact the core methodology. scientific rigorousness, or originality of the research, declaration is not required.
Answer: [NA]
Justification: LLMs were used solely for refining writing, grammar, and clarity of exposition. They were involved in any part of the core methodology, theoretical development, experimental design, or analysis. Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, or non-standard components. Please refer to LLM policy (https://net) //teer/is....eeeeeeeeeee /Coneerrenceee2222555 for what should or should not be described.
44


Reviews:
[{"review_id": "3oQgSrj7eG", "reviewer": "Reviewer_twXJ", "summary": "This paper proposes the incorporation of an existing sparse coding concept (matching pursuit) into sparse autoencoder architectures in order to more effectively represent meaningful, hierarchical features. The authors analyze the weaknesses of existing methods, and claim that matching pursuit sparse autoencoders (MP-SAEs) are able to more effectively represent features that existing methods fail to capture, specifically features that are conditionally dependent, hierarchical, nested, or nonlinear.", "questions": "1. Similar to your relaxing of the linear representation hypothesis into conditional orthogonality, other papers (such as https://doi.org/10.48550/arXiv.2306.07304) attempt to formalize SAEs in a different way. Are MP-SAEs able to fit into the formalizations given here? If not, why or why not are MP-SAEs still a comparable method to traditional SAE architectures?\n2. When considering MP-SAEs as a method, their benefits seem clear in the synthetic examples provided. However, are there clear use cases in real-world data that MP-SAEs would be better suited for?", "limitations": "yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n\n* Paper presents a thorough basis for why the proposed method matters, highlighting gaps in existing methods\n* Thorough discussion of existing and proposed formalizations is given\n* The structure of the paper is largely clear and concise.\n* Effective, well-scoped argument about MP-SAEs as an alternative architecture, for use in cases where the true features are outside of the global orthogonality regime/linear representation hypothesis.\n* Solid advancement, proposing an original (though inspired quite heavily by the existing matching pursuit method), useful architecture.\n\nWeaknesses:\n\n* The argument that MP-SAEs are a valid operationalization of conditional orthogonality is based on a claim that no evidence is provided for\n* Sections 3 and 4 have considerable overlap, particularly when discussing experimental results that validate claimed properties of MP-SAEs.\n* Inconsistent benchmarking against existing models. Figure 4 compares MP-SAEs to Matryoshka SAEs, but Matryoshka SAEs are omitted from figures 5-8. Matryoshka SAEs also seem to be the most comparable SAE architecture, so this omission hurts the strength of claims.", "was_revised": true, "final_justification": "The authors clarified many of the positioning weaknesses with respect to other works in this area and committed to comparing against other baselines. I also followed the long discussion with the other reviewer. I would like to leave my score at a 5. It's a great paper.", "rebuttal": "We thank the reviewer for their thoughtful and encouraging feedback! We are glad they found our method to be a **“solid advancement”** and appreciated our **“effective, well-scoped argument about MP-SAEs as an alternative architecture.”** We are especially pleased that the reviewer highlighted our **“thorough discussion of existing and proposed formalizations”** and the clarity of our motivation for relaxing the linear representation hypothesis. Below, we address specific comments.\n\n---\n\n> _Q1: [...] The argument that MP-SAEs are a valid operationalization of conditional orthogonality is based on a claim that no evidence is provided for_\n\nThank you for raising this point! To clarify the mechanism via which MP operationalizes conditional orthogonality, we refer the reviewer to Proposition 3.1: therein, we show **MP’s inference dynamics follow an orthogonality constraint whereby each inference step uses a concept (i.e., dictionary atom) that results in a residual orthogonal to said concept** (see Figure 3-left for an intuitive visualization). While this constraint certainly does not guarantee conditional orthogonality, i.e., it does not guarantee that subsequent concepts used for explaining the residual will be orthogonal to the previously selected ones, in practice we find the local constraint leads to an emergent organization of the dictionary such that inference with MP indeed satisfies conditional orthogonality. \n\nSpecifically, we find MP-SAE selects concepts that are orthogonal to previously selected ones (quantified via the Babel score in Fig. 7). Similar results have previously been reported in dictionary learning literature, from which we sought our motivation: e.g., Peotta and Vandergheynst [1] show that when atoms are highly correlated within groups but nearly orthogonal across them (a.k.a., block incoherent dictionaries), MP tends to select atoms from distinct groups, effectively satisfying conditional orthogonality. \n\n**We nevertheless agree with your comment that the precise mechanism via which MP is operationalizing conditional orthogonality can be better clarified**. We promise to revise Section 3 and add the discussion above to address this concern in the final version of the manuscript! \n\n[1] Peotta and Vandergheynst “Matching Pursuit With Block Incoherent Dictionaries” IEEE Transactions on Signal Processing (2007)\n\n---\n\n>_Q2 [...] Sections 3 and 4 have considerable overlap, particularly when discussing experimental results that validate claimed properties of MP-SAEs._\n\nWe agree. However, as you noted, the goal of Section 3 was to formalize MP-SAEs and use a toy setting that helps establish their ability to extract hierarchical concepts. Meanwhile, in Section 4 our goal was to scale up our experiments to natural domains and demonstrate the validity of our claims in these significantly more complicated settings. Merging the two sections, which we did attempt, induces a back-and-forth between the precise experimental setup being investigated, resulting in a convoluted writing flow. Nevertheless, **we promise to try to smoothen the transitions between the sections and minimize redundancy** therein: e.g., we can try to keep the formalization of MP-SAE in Section 3 and move all experiments to Section 4, with a subsection each on synthetic vs. natural domains.\n\n---\n\n>_Q3: Inconsistent benchmarking against existing models. Figure 4 compares MP-SAEs to Matryoshka SAEs, but Matryoshka SAEs are omitted from figures 5-8. Matryoshka SAEs also seem to be the most comparable SAE architecture, so this omission hurts the strength of claims._\n\n**Fair point! This was indeed an oversight on our part.** We came across Matryoshka SAEs relatively late in the project (their paper was released less than two months before the NeurIPS deadline). Because our large-scale experiments focus on vision, whereas theirs are on language, we could not directly adopt their code for comparison given the limited time before the deadline. We were, however, able to perform toy experiments, which corroborate that MP-SAEs are better suited for extracting hierarchical concepts in controlled regimes where feature structure is known. Nonetheless, since Matryoshka SAEs also target hierarchical disentanglement, we agree they merit inclusion in our real-data experiments. **All these additional baselines will be includes in the final version**.\n\n---\n\n>_Q4: Similar to your relaxing of the linear representation hypothesis into conditional orthogonality, other papers (such as https://doi.org/10.48550/arXiv.2306.07304) attempt to formalize SAEs in a different way. Are MP-SAEs able to fit into the formalizations given here? If not, why or why not are MP-SAEs still a comparable method to traditional SAE architectures?_\n\nGreat question! As such, **MP-SAEs are fully compatible with the formalism described in Fel et al.** and directly comparable to traditional SAEs, differing only in the inductive bias of the encoder. Specifically, traditional SAEs typically use a shallow encoder, e.g., a single linear layer followed by a non-linearity, to produce a sparse code with (say) K active latents. In contrast, MP-SAEs, based on the Matching Pursuit procedure, unroll the same encoding process over K steps by greedily selecting atoms based on correlation with the residual at each step. This eventually still leads to a K-sparse code, building one concept at a time.\n\nOverall, given the reviewer’s prompt, we will use Fel et al.’s formalization to add the following note that clarifies the connection between different SAEs:\n\n\n\\begin{aligned} \n(Z^\\star, D^\\star) = \\arg\\min_{Z, D} \\| A - Z D^\\top \\| \\\\\\\\\n \\text{Subject to:} \\\\\\\\\n Z_i \\in \\{ e_1, \\ldots, e_k \\} \\quad &\\text{(\\textbf{ACE})}, \\\\\\\\\n D^\\top D = \\mathbf{I} \\quad &\\text{(\\textbf{ICE})}, \\\\\\\\\n Z \\geq 0,\\ D \\geq 0 \\quad &\\text{(\\textbf{CRAFT})}, \\\\\\\\\n Z = \\Psi_{\\theta(A),\\  \\|Z\\|_0 \\leq K} \\quad &\\text{(\\textbf{SAE})}, \\\\\\\\\n Z = MP_K(A;\\ D) \\quad &\\text{(\\textbf{MP-SAE})}\n\\end{aligned}\n\n\\begin{aligned}\nMP_K(A; D) = \\sum_{k=1}^K \\operatorname{top1}({r}^{(k-1)} D^\\top), \\quad \n    {r}^{(0)} = A, \\quad \n    {r}^{(k)} = {r}^{(k-1)} - \\operatorname{top1}({r}^{(k-1)} D^\\top) D\n\\end{aligned}\n\n\n---\n\n>_Q5: When considering MP-SAEs as a method, their benefits seem clear in the synthetic examples provided. However, are there clear use cases in real-world data that MP-SAEs would be better suited for?_\n\nGreat question! First, we would like to note that our goal in this paper was to contextualize popular SAE architectures with respect to recent developments identifying hierarchical, nonlinear, and multidimensional representations in neural networks. Such representations lie outside the scope of the linear representation hypothesis (LRH)—the assumed model of conceptual organization in neural networks that motivates SAEs. Specifically focusing on hierarchical concepts, we show how standard SAE architectures fail to identify representations corresponding to them. Thus, **our goal with proposing MP-SAE was to show how, if one knows the structure they are trying to capture, it is possible to design useful SAE architectures that scale easily** (as shown by our large-scale vision experiments), perform very well across a broad spectrum of metrics (as shown in Section 4), and show surprising results (e.g., an explanation for the modality gap in vision-language models). \n\nThat said, **hierarchical concepts are commonplace in several natural domains**: e.g., LLMs show a very structured, hierarchical organization of high-level concepts like human emotions [1], which can matter for monitoring what a model is modeling about the user [2]; scene graphs are often used to develop hierarchical, compositional representations of inter-object relations in vision [3]; and several scientific domains exhibit tree-structured concepts, e.g., phylogenetic trees, that may be extractable from language models trained on data such as genomic structures [4]. We strongly expect MP-SAEs will find utility in these domains, warranting a thorough study that is best left for future work.\n\n[1] Zhao et al. “Emergence of Hierarchical Emotion Organization in Large Language Models” arXiv:2507.10599 (2025)\n\n[2] Chen et al. “Designing a Dashboard for Transparency and Control of Conversational AI” arXiv:2406.07882 (2024) \n\n[3] Johnson et al. “Image Retrieval using Scene Graphs”, IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2015)\n\n[4] Brixi et al. “Genome modeling and design across all domains of life with Evo 2” bioRxiv (2025)"}, {"review_id": "z0paekmaOg", "reviewer": "Reviewer_9fBU", "summary": "The paper introduces a new conditional orthogonality inductive bias for SAEs, enabling recovery of hierarchical and non-linear features instead of forcing a single flat linear basis. The authors utilize this idea and propose MP-SAE architecture and evaluate it on diverse set of datasets, showing that the method strongly outperforms baselines. Empirical findings of the paper provide further evidence to the intuition that representation spaces are better viewed as nested subspaces, thus justifying further relaxation to empirical support to relaxing the overly restrictive LRH.", "questions": "Please, see Weaknesses", "limitations": "yes", "rating": 5, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 4, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "## Strengths\n- The paper is very well written and includes neat and informative illustrations.\n- The authors provide a broad discussion of SAEs (which is very useful for an outside reader), carefully formalizing the new constraint and explaining its intuition\n- Conditional orthogonality and its instationed in MP-SAE is well motivated, connected to existing literature, and thoroughly analyzed\n- MP-SAE is evaluated on diverse datasets including both synthetic benchmarks, and natural data (coco) with a sota vision models (CLIP, DINOv2), where the proposed method strongly outperformed the baselines\n- The analysis is further extended to vision-language models and toy language models in the Appendix\n- Authors discuss theoretical guaruantees of the MP-SAE infernce in Appendix\n\n## Weaknesses\n- It would be great to see more results for the LLMs, e.g. 0.5-3b models, to strenghten the generalization of the approach across the modalities\n- Did authors observed any failure modes of proposed MP-SAE under highly entangled representations when compared to Vanilla SAE?", "was_revised": true, "final_justification": "After reading extensive discussion of authors with other reviewers, and their willingness to further elaborate and acknoledge certain assumptions in the manuscript, I decided to maintain my score still leaning towards acceptance.", "rebuttal": "We thank the reviewer for their positive and thoughtful feedback. We particularly appreciate the acknowledgement that the paper **“is very well written and includes neat and informative illustrations.”** We’re also glad that the **“broad discussion of SAEs”** was found to be **“very useful for an outside reader,”** and that the reviewer considers our treatment of **“conditional orthogonality and its instantiation in MP-SAE”** to be **“well motivated, connected to existing literature, and thoroughly analyzed.”** We also thank the reviewer for raising several interesting points, which we address below.\n\n---\n\n>_It would be great to see more results for the LLMs, e.g. 0.5-3b models, to strengthen the generalization of the approach across the modalities._\n\nThank you for this suggestion! While **we agree that including results on larger language models (e.g., 0.5–3B) would help further validate the utility of MP-SAEs, we emphasize our goal in this paper was broader**: we wanted to contextualize limitations of existing SAEs with respect to representational structures that lie outside the scope of the linear representation hypothesis, i.e., hierarchical concepts. We thus proposed and used MP-SAE as an architecture that has a baked-in prior for modeling hierarchical structures, and could hence help analyze how SAEs that do capture hierarchical concepts differ from ones that do not. To this end, we conducted experiments comparing different SAEs in Sections 3, 4 on **large scale vision models**, **multimodal models**, **smaller language models**, and **synthetic toy problems**. Across these rather disparate domains, we saw an almost perfect consistency of results: one gets better expressivity, flexibility, and ability to model hierarchical concepts with MP-SAE, but not with standard SAEs. Given the breadth of our evaluation, we do believe our current validation suffices to justify our claims. However, the broader utility of MP-SAE, e.g., in finding hierarchical concepts in LLMs, which we are quite excited about, warrants a more thorough study that is best left for a future paper (we do however note we are already pursuing this!).\n\n---\n\n>_Did authors observe any failure modes of proposed MP-SAE under highly entangled representations when compared to Vanilla SAE?_\n\nGreat question—we were also curious about the robustness of MP-SAEs when the data contains entangled / correlated concepts! To study this systematically, we had reported in Figure 14 (appendix) an extended version of our synthetic testbed results from the paper, where we compare how different SAEs behave when the data contains highly entangled features. Vanilla SAE results are shown in the first column of the figure, and MP-SAE results in the last. In brief, **we see MP-SAEs break down entangled levels into hierarchical stages, capturing both coarse and fine structure, while Vanilla SAE flattens the hierarchy, focusing only on one level of it.** Neither method perfectly recovers all ground-truth features in this challenging setup, but they fail in notably different ways, as discussed below.\n\n- For MP-SAE, we observe a recurring and characteristic behavior: the model tends to allocate one feature aligned with the parent direction, as well as one representative feature per group of children. For example, in the first MP-SAE run (top-left panel), the parent is represented by the second-to-last component, while child groups are captured by components at indices 2, 6, and 11, each corresponding to a ground-truth child feature. Notably, MP-SAE does not assign a unique feature to every child. Instead, it learns additional components that interpolate between the representative child and the others within the group. When intra-level correlations are high, this results in a two-stage decomposition: \n 1. a general direction aligned with one child that captures the shared structure of the group, and \n 2. refinement features that describe deviations from this common direction.\n\n    We observe a similar behavior at the parent level as well. When parent features are highly entangled, MP-SAE tends to represent the shared parent structure using one dominant direction, followed by secondary components that refine this direction to better match the individual parents. Thus, this coarse-to-fine decomposition emerges at multiple levels of the hierarchy, not just within child groups.\n\n\n    This hierarchical organization allows MP-SAE to partially resolve entangled structure by capturing both group-level similarities and within-group variations. As a result, the model typically selects a larger number of active components (visible in the support of the sparse code), reflecting the increased representational complexity required.\n\n- In contrast, Vanilla SAE is able to reconstruct the children accurately, despite their high correlation, but consistently fails to capture the parent features. This is evident in the very low or absent activation of parent components in the sparse codes. The model appears to allocate its full capacity to fitting the entangled children, effectively ignoring higher-level structure. \n\nWe hope this clarifies the distinct failure modes of the two approaches and addresses the reviewer’s question. We will add this discussion to the appendix to make these observations more accessible to readers."}, {"review_id": "DTHgCG6GHC", "reviewer": "Reviewer_ZQvY", "summary": "The authors present a formalized version of the linear representation hypothesis, which enables them to develop a sparse auto-encoder (SAE) algorithm based on a pursuit matching algorithm. This algorithm is initially evaluated on synthetic data and subsequently applied to real-world data, demonstrating the superiority of the representations it produces compared to existing methods.", "questions": "In the derivation of the representation encoding cost, the authors mention the inclusion of a homeostasis mechanism to facilitate learning in inactive units. Could you explain the hit function used and provide evidence of its effectiveness, such as demonstrating that each activity can be activated with roughly equal probability on average (see for example doi:10.1162/neco.2010.05-08-795 )?\n\nDuring the evaluation using synthetic data, the authors leverage the known dimensionality of the ground truth. Have there been observations of changes in the quality of the representation when using higher or lower dimensions? Additionally, has the effect of dimensionality on the two datasets been assessed?\n\nIn the final experiment, the authors demonstrate that their modality index is more evenly distributed between vision and language in their algorithm. Furthermore, it seems that the reconstruction errors are improved. How are these two results interconnected?", "limitations": "One of the limitations, which could also be viewed as a potential strength, of this paper is its interpretation in the context of neurobiology. Given that the algorithm is presented as a hierarchical neural network, it may bear similarities to biological neural networks, particularly in terms of the sparsity of activity observed in neurophysiological experiments. Could the authors draw connections between their algorithm and other hierarchical sparse coding algorithms that aim to establish links with biological systems such as the visual cortex?", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "The paper is clearly and effectively presented, allowing for a thorough evaluation of its quality. One of its key strengths is the formalization of the problem, along with the derivation of the sparse autoencoder algorithm and the relevance of the results presented.\n\nWhile the pursuit matching algorithm appears to yield better results, particularly in its ability to extract more expressive features than standard SAEs, it would have been beneficial to include examples of these features for the various SAE to further support the findings.", "was_revised": true, "final_justification": "The responses to my comments and to those of other reviewers convinced me to raise my score to accept that contribution.", "rebuttal": "We thank the reviewer for their thoughtful and constructive feedback, as well as for bringing a neurobiology perspective to our work! We are happy that the reviewer thinks that our work **“is clearly and effectively presented”**. We are particularly encouraged by the recognition that **“the formalization of the problem, along with the derivation of SAEs and the relevance of the results presented”** are considered as key strengths. We also appreciated the analogy drawn by the reviewer between our proposed MP-SAEs and neurobiological systems—particularly the observation that MP-SAEs, when viewed as a hierarchical neural network, may bear similarities to biological sparse coding mechanisms observed in the visual cortex. This perspective is interesting and aligned with some of the broader motivations behind our work.\n\n---\n\n## Comments\n\n>_While the pursuit matching algorithm appears to yield better results, particularly in its ability to extract more expressive features than standard SAEs, it would have been beneficial to include examples of these features for the various SAE to further support the findings._\n\n\nThank you for the suggestion! We agree that including examples of the extracted features across different SAEs would have helped support our claim that MP-SAE, with its inductive bias towards hierarchical concepts, infers more expressive and structured features. We note we have already run several experiments on this front and found MP-SAE selects features that are more global in nature early-on in its inference process, while later-on it selects features that are more fine-grained and local. **As an example, we highlight Fig. 8(b), where we show that in Vision-language models, features selected by MP-SAE early-on distinguish the input modality, but later features correspond to shared concepts between the modalities (such as an object).** We have performed several further experiments (including feature visualization plots) to corroborate these claims, finding, e.g., in Tinystories models features related to topics, frequent lexical patterns like character names, or syntactical concepts are selected early-on, but concepts for punctuations are fairly common later in inference for MP. Since NeurIPS does not allow uploading of new results this year, we are unable to share these visualizations at the moment, but **promise to include them in the final version of the paper!**\n\n\n>_In the derivation of the representation encoding cost, the authors mention the inclusion of a homeostasis mechanism to facilitate learning in inactive units. Could you explain the hit function used and provide evidence of its effectiveness, such as demonstrating that each activity can be activated with roughly equal probability on average (see for example doi:10.1162/neco.2010.05-08-795 )?_\n\n\nThank you for raising the important issue of “inactive units” (or dead neurons). **We would like to clarify that our paper does not include any notion of a “homeostasis mechanism” nor employ a “hit function.”** However, we are happy to provide more context on this point.\nThe concern highlighted by the reviewer refers to a known issue in sparse coding models: when neurons compete to represent data, some may learn faster or become more selective, leading to imbalanced representations. In extreme cases, this can result in some dictionary elements being underutilized or inactive. The paper referenced by the reviewer addresses this by proposing a “cooperative homeostasis” mechanism to equalize the activation probability across neurons.\n\n\nOur paper does not include such a mechanism, and to our knowledge, this is not a common practice in the SAE literature. More importantly, we believe that enforcing uniform activation would be inappropriate in this context, as some concepts naturally occur more frequently or are more salient. Imposing uniformity could therefore distort the true structure of the representations. That said, regarding inactive units, some methods (like TopK [1]) instead rely on auxiliary losses to mitigate dead neurons, but these losses do not enforce that every neuron is activated with roughly equal probability on average.\n\n\n[1] Gao et al. “Scaling and evaluating sparse autoencoders” arXiv:2406.04093 (2024)\n\n---\n\n## Questions\n\n>_During the evaluation using synthetic data, the authors leverage the known dimensionality of the ground truth. Have there been observations of changes in the quality of the representation when using higher or lower dimensions? Additionally, has the effect of dimensionality on the two datasets been assessed?_\n\nWe assume by “dimensionality”, the reviewer is referring to the sparsity level of the ground truth sparse codes (i.e., the ℓ0​ norm of the ground-truth codes). We certainly agree that, in real-world scenarios, the dimensionality of the ground truth will be unknown. This, however, makes us furthermore excited about MP-SAEs: in our experiments with the synthetic domain the reviewer is referring to, **we found that MP-SAE to be the only method evaluated that did not require prior knowledge of this sparsity level at training time!** This is a key advantage of our approach, as it avoids the need to tune sparsity hyperparameters based on ground-truth labels.\n\nIn the case that by “dimensionality” the reviewer means “dictionary size”, we note that we indeed studied the effect of dictionary dimensionality in a setting with pretrained large-scale vision models (Figure 5). Here, we varied the number of concepts from p=125 to p=1250 and observed that as the dictionary grows, the performance gap between MP-SAE and other state-of-the-art SAEs widens. This indicates that MP-SAE is able to effectively leverage increased representational capacity, highlighting its scalability and expressivity. In our synthetic experiments, however, we match the size of the dictionary to that of the ground truth in order to enable direct one-to-one comparisons and evaluate recovery quality in a controlled setting.\n\nWe hope we understood the question correctly and are happy to clarify further should any part of our response remain unclear! \n\n>_In the final experiment, the authors demonstrate that their modality index is more evenly distributed between vision and language in their algorithm. Furthermore, it seems that the reconstruction errors are improved. How are these two results interconnected?_\n\n**Thank you ! This question has been a point of discussion among the authors!**\n\nWhile we do not claim a direct causal link between reconstruction error and modality scores, our working hypothesis is that both emerge from MP-SAE’s bias toward collecting concepts orthogonal to each other align well with the geometry of the VLM where the multimodal information is orthogonal to the modality information.\n\nSpecifically, MP-SAE quickly identifies the dominant modality direction (e.g., “image” or “text”) early in inference. Its residual-based selection then emphasizes features that explain remaining variance -> often shared, cross-modal structure! In contrast to classical SAEs, which tend to learn modality specific atoms that do not generalize across modalities.\n\nThis hypothesis aligns with recent findings by Papadimitriou et al. (2025), who show that VLM embeddings contain sparse, semantically meaningful directions that are often unimodal, but aligned across modalities. These “bridge” features span shared subspaces and co-activate on paired image-text inputs. **We’ll clarify this interpretation in the final version and appreciate the opportunity to expand on this point.**\n\n\n>_Could the authors draw connections between their algorithm and other hierarchical sparse coding algorithms that aim to establish links with biological systems such as the visual cortex?_\n\n\nGreat suggestion! **The MP inference process (greedy selection + residual inhibition) closely mirrors mechanisms proposed in cortical sparse coding**, particularly lateral inhibition in early visual areas. We will clarify this connection in the introduction and related work, situating MP-SAE within the lineage of biologically inspired sparse inference models.\n\n\nThank you for carefully reviewing the formatting as well. **We really appreciate you pointing out that some equations are missing numbers**, and we will definitely address this in the final version. We remain available for any further clarifications or additional revisions the reviewer may have."}, {"review_id": "EWOhhtCvrr", "reviewer": "Reviewer_wmMK", "summary": "This paper considers the problem of designing a sparse autoencoder whose success does not depend on the traditional linear representation hypothesis. Specifically, the paper proposes replacing the global orthogonality condition of the linear representation hypothesis with a conditional orthogonality condition, that allows for non-orthogonality of some pairs of dictionary elements. The paper provides an algorithm to learn this type of dictionary via a matching pursuit algorithm. \n\nOn both synthetic and real data, the paper shows that their approach learns dictionaries that better reconstruct the data, produces representations that are more diverse, and learns features on multi-modal data that are less aligned with modality.", "questions": "- Can you give an explicit description of the paper's dictionary learning algorithm?\n\n- Can you provide a formal link between conditional orthogonality and the matching pursuit algorithm?\n\n- Can you explain what is novel about the MP-SAE algorithm relative to the standard matching pursuit algorithm?\n\n- Can you give an explicit example of a non-linear concept that your algorithm will recover but standard SAEs will not?", "limitations": "Yes", "rating": 5, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 2, "strengths_and_weaknesses": "Strengths:\n\n- This paper addresses a timely problem. SAEs for concept discovery in large models is a fashionable problem, but the core underlying assumptions of these methods have recently come under scrutiny. This paper proposes a change in assumptions that allows for more flexible dictionaries.\n\n- The paper provides a thorough comparison on a variety of models and metrics. The paper investigates their proposed conditional orthogonality assumption and finds some evidence of it occurring empirically. \n\nWeaknesses:\n\n- The paper provides details on inference in the model, but not on the training procedure. It's not clear what the loss function is for fitting the dictionary. If it's some MSE reconstruction loss on the reconstructed x's, then it's not clear how backpropping through the argmax's are handled.\n\n- It is not clear how the concept of conditional orthogonality is formally related to the matching pursuit algorithm. From Proposition 3.1, we know that the updated residual is orthogonal to the last selected dictionary element, and so we would expect the next selected dictionary element to be among the most orthogonal remaining elements. However, it's not clear how this places any restrictions on the dictionary elements themselves. Also, I don't see where the hierarchy in the definition of conditional orthogonality is arising. \n\n- At a high-level, I'm struggling to see what is the difference between the MP-SAE algorithm, and the standard matching pursuit algorithm. Is the difference in how the dictionary is learned?\n\n- The matching pursuit algorithm can be applied to any learned dictionary, including those learned by other SAEs. It would be interesting to disentangle the effects of the dictionary learning procedure and the inference method by applying the matching pursuit algorithm to the other methods' learned dictionaries. This would help us understand if in Figure 5, better reconstruction is due to a superior dictionary or due to a superior inference/reconstruction algorithm.\n\n- The paper talks about their approach being able to accommodate non-linear concepts, but nothing concrete is shown in the paper. Perhaps a worked out toy example would be appropriate, where the matching pursuit algorithm can find the non-linear concept but the standard TopK VAE is unable to represent it. \n\n- I'm not entirely sure what's going on in Figure 7. I understand that the top row corresponds to the Babel score on the entire dictionary. But the bottom row seems to be over the dictionary elements selected for a single data point or possibly averaged over multiple data points. There's not really enough information to figure it out.\n\n- Although the paper compares against the Matryoshka SAE on a synthetic task, there's no comparison against it on the real-world data. At a high level, the Matryoshka approach is trying to solve a similar problem, so it seems important to compare against it on real data as well.\n\n- While the quantitative results show improvements for the proposed method over previous SAE methods on various metrics, it's still unclear what all of this enables. An application that shows the matching pursuit approach uncovers concepts that the other methods are unable to recover would go a long way towards selling this paper.", "was_revised": true, "final_justification": "After the discussion with the authors, my view on this paper has improved, and I am now in favor of acceptance. With the planned edits/clarifications promised by the authors, I think this will be a useful addition to the sparse auto-encoder/model interpretation literature.", "rebuttal": "We thank the reviewer for their careful reading and constructive feedback. We are pleased that they recognized our **\"thorough comparison on a variety of models and metrics\"**, and that our method **\"learns dictionaries that better reconstruct the data, produces representations that are more diverse, and learns features on multi-modal data that are less aligned with modality\"**. We also thank them for their questions and comments.\n\n---\n\n>_The paper provides details on inference in the model, but not on the training procedure. [...] what the loss function is for fitting the dictionary [...] how backpropping through the argmax's are handled._\n\nThank you for highlighting this point. We agree that the training procedure was not sufficiently detailed in the current version and will include a dedicated section in the appendix to clarify it in the final version of the paper. In brief, we note the dictionary is trained using a standard objective for training SAEs: **mean squared error (MSE).** We do not use any auxiliary objectives or regularization terms compared to other SAEs. While the Matching Pursuit algorithm relies on a non-differentiable Top-1 selection (i.e., argmax) to identify the most correlated component, **we follow the same strategy as in TopK-based SAEs and use a Straight-Through Estimator (STE)** to enable gradient flow. Gradients are propagated only through the selected features, consistent with the approach used in TopK-SAE [1].\n\nIn practice, this Top-1 selection can be implemented in a differentiable manner as follows:\n```python\ndef diff_top1(input):\n    values, indices = torch.max(input, dim=1, keepdim=True)\n    z = torch.zeros_like(input)\n    z.scatter_(1, indices, values)\n    return z\n```\n\n>_Can you provide a formal link between conditional orthogonality and the matching pursuit algorithm? [...] it's not clear how this places any restrictions on the dictionary elements themselves._\n\nThank you for this question! **As another reviewer also noted the same thing, we realize now our original explanation of the connection to conditional orthogonality was not sufficiently clear, and we will revise the paper to address this directly.**\n\nIn brief, we note a formal link between conditional orthogonality and the Matching Pursuit (MP) algorithm arises in the context of block-incoherent dictionaries, as discussed by Peotta and Vandergheynst [2]. In such dictionaries, atoms are highly correlated within groups but nearly orthogonal across groups. MP, by design, tends to select atoms that best reduce the residual at each step; in block-incoherent settings, this typically leads to selecting atoms from distinct groups. As a result, the selected atoms are approximately orthogonal across groups, effectively approximating conditional orthogonality in practice. Conditional orthogonality corresponds to the idealized case of perfect block-incoherence, where atoms are orthogonal to all atoms outside their group. We will revise Section 3 to clarify this interpretation and explicitly highlight this connection.\n\n**Conditional orthogonality is not imposed as a constraint on the dictionary $D$** , but rather emerges as a property of the inference trajectory. At each step, MP selects the atom most aligned with the current residual, which is, by construction, orthogonal to the span of the previously selected atom. This results in a set of features that explain complementary structure in context. We will make this distinction explicit in the revised version.\n\n\n\n>_Also, I don't see where the hierarchy in the definition of conditional orthogonality is arising._\n\n**The connection between hierarchy and conditional orthogonality is based on the findings of [3]**, who show that in the representation spaces of LLMs, hierarchically related concepts (e.g., plant ⇒ animal, mammal ⇒ bird) tend to lie in orthogonal subspaces. Conditional orthogonality naturally follows from this: concepts at different levels of a hierarchy should be orthogonal to each other.\n\n>_Can you give an explicit description of the paper's dictionary learning algorithm? […] what is novel about the MP-SAE algorithm relative to the standard matching pursuit algorithm?_\n\nThank you for raising this important point. To clarify, **our work does not introduce a new sparse inference algorithm but rather embeds Matching Pursuit (MP) within a trainable sparse autoencoder.** Similar ideas have appeared in image restoration (e.g., Learned MP [4]), but to our knowledge, this is the first application to sparse dictionary learning with an analysis of its representational properties. While Section 3 describes the algorithm and its motivation, we recognize that the dictionary learning aspect could be emphasized more clearly and will revise it in the final version. \n\nStandard MP is a sparse approximation algorithm: given a fixed dictionary $D$, it approximates a sparse code $z$ for an input $x$ by solving\n$$\n\\min_z \\|x - D z\\|_2^2 \\quad \\text{s.t.} \\quad \\|z\\|_0 \\leq k.\n$$\nThis is purely an inference method and does not involve learning the dictionary.\n\nMP-SAE, by contrast, unrolls MP within a sparse autoencoder architecture and performs dictionary learning, where the dictionary $D$ is learned across a dataset. The encoder is fixed and implemented via the MP algorithm, while the decoder corresponds to the learnable dictionary $D$. The training objective becomes:\n\n$$\n\\min_D \\sum_i \\|x^i - D z^i\\|_2^2 \\quad \\text{where} \\quad z^i = \\text{MP}(x^i, D).\n$$\n\nThus, in our architecture, the *forward pass* performs sparse inference using MP, and the *backward pass* updates the dictionary $D$ based on the reconstruction loss. \n\n>_It would be interesting to disentangle the effects of the dictionary learning procedure and the inference method […] This would help us understand if in Figure 5, better reconstruction is due to a superior dictionary or due to a superior inference/reconstruction algorithm._\n\nThis is a great question! In fact, to explore the separation between inference and dictionary learning, we applied MP’s inference process to dictionaries learned by other SAEs during the early stages of this project. **We consistently observed that MP inference improved reconstruction relative to the original shallow encoders. However, these combinations still underperformed compared to MP-SAE, where the dictionary is learned jointly with MP inference!**\n\nArguably, this is expected since the inference and dictionary learning protocol in SAEs are tightly coupled: the structure of the learned dictionary depends on the inference method used during training. As discussed in [4] and supported by our Babel score metric, the encoder implicitly constrains the space of learnable dictionaries. By avoiding the inductive biases of shallow encoders—such as the quasi-orthogonality assumption—MP-SAE is able to learn more coherent and expressive dictionaries. Thus while it is difficult to fully disentangle the two components, we hope this explanation complements Figure 5.\n\n>_The paper talks about their approach being able to accommodate non-linear concepts, but nothing concrete is shown in the paper [...] Can you give an explicit example of a non-linear concept that your algorithm will recover but standard SAEs will not?_\n\nThank you for this valuable comment. **An explicit example of a non-linearly accessible concept appears in the multimodal features extracted by MP-SAE but not recovered by standard SAEs**, as shown in our main results and detailed in Appendix B.4. Figure 17 provides an illustration of how such concepts are encoded, highlighting how MP-SAE can recover multimodal features that are inaccessible to standard SAEs. **We will revise the manuscript to emphasize clearly that this is an example of non-linearly accessible concepts uniquely captured by our approach**\n\n>_I'm not entirely sure what's going on in Figure 7._\n\nThank you for this remark! We agree that Figure 7 needs clearer explanation and will revise it in the manuscript. The top row reports the Babel score across the entire dictionary, reflecting the overall coherence of learned features, independent of how they are used. The bottom row focuses on the features actually selected during inference for each input: we compute the Babel score on this active subset for each data point and then average across the dataset. This highlights the coherence of the features that are used to represent data points.\n\n>_[...] At a high level, the Matryoshka approach is trying to solve a similar problem, so it seems important to compare against it on real data as well._\n\nFair point! This was indeed an oversight on our part. We encountered Matryoshka SAEs relatively late in the project (the corresponding paper was released only two months before the NeurIPS deadline), and due to time constraints, we could only perform toy experiments comparing MP-SAEs with Matryoshka SAEs. These initial experiments support our claim that MP-SAEs are better suited to extracting hierarchical concepts in controlled settings where the feature structure is known. Since Matryoshka SAEs also aim for hierarchical disentanglement, we fully agree that they should be included in our real-data experiments. **We are now running these additional baselines and will incorporate the results in the final version.**\n\n[1] https://arxiv.org/abs/2406.04093\n\n[2] https://ieeexplore.ieee.org/document/4291867\n\n[3] https://arxiv.org/abs/2406.01506\n\n[4] https://arxiv.org/abs/2503.01822\n\n---\n\n**Summary:** We thank the reviewer for their feedback, which prompted us to clarify the training procedure, formalize the link between Matching Pursuit and conditional orthogonality, and distinguish MP‑SAE from standard MP. Their comments also led to clearer explanations on hierarchy, examples of non‑linear concepts, clarification of Figure 7, and additional comparisons to Matryoshka SAEs. We hope our responses address these points, and if so, we hope they will encourage the reviewer to consider raising their score!"}, {"review_id": "gtor4Kzjhy", "reviewer": "Reviewer_vRBm", "summary": "The authors present a novel SAE architecture called Matching Pursuit Sparse Autoencoders (MP-SAEs). This architecture uses insights from sparse coding to replace the SAE encoder with a greedy algorithm to iteratively select a latent to reconstruct part of the residual. The authors claim this architecture is able to handle hierarchical relationships better than existing SAE architectures, and finds richer features than existing SAE architectures on synthetic and vision-based benchmarks.", "questions": "I will raise my score if the authors remove the claim that MP-SAEs handle hierarchical relationships, as this does not appear to be true. Alternatively, if the authors modify the MP-SAE architecture so it does actually handle hierarchical relationships, even when feature variance is low or none, I will also raise my score.\n\nI will raise my score further if the authors train MP SAEs on an LLM like Gemma-2-2b, run SAEBench, and compare these results with Matryoshka SAEs. This would allow a more direct comparison with existing SOTA SAEs and make it easier to recommend MP-SAEs as the SOTA going forward. Training a MP-SAE on a real LLM would allow creating feature dashboards to give a qualitative sense of the type of features that are recovered as well.\n\nI also have the following questions on the work:\n\n- Conceptually I don't see how the MP-SAE actually encourages learning the parent latent rather than a parent + child (feature absorption) latent. E.g. if a there's a parent concept P and a child concept C, why wouldn't the MP-SAE learn a latent for C + P and a latent for P, and activate C + P when the child is active and P when the parent is active? In the code provided for the paper, if I reduce the feature firing variance then the MP-SAE does often learn the absorption solution rather than correctly separating the components of the hierarchy. Having parent and child features fire with high independent variance makes them look like independent features to a tied SAE, rather than the SAE actually handling hierarchy.\n- The work mentions that MP-SAEs can capture non-linear features. Given the dictionary is still just linear vectors, why do the authors think that MP-SAEs are capturing non-linear concepts? Have the authors observed any specific non-linear concepts that are recovered by MP-SAEs, e.g. things like days of the week being embedded in a circle?", "limitations": "yes", "rating": 4, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 2, "clarity": 3, "significance": 4, "originality": 4, "strengths_and_weaknesses": "## Strengths\n\nThis paper introduces a novel SAE architecture that is fundamentally different than the simple linear encoder that is typically used currently, and this is a huge contribution to the field, especially given the promising initial results in the paper. The fact that this architecture is directly inspired by matching pursuit from sparse coding field is also exciting. This architecture is a great contribution to the field and opens up a lot of avenues of future work around selecting which latents to use explicitly to reconstruct given inputs.\n\n## Weaknesses\n\nWhile I feel this work is important, there are a number of serious weaknesses. \n\nThe most serious weakness is that the main claim of this work, that MP-SAEs solve hierarchy, is not true. This claim is supported based on toy model experiments, but these toy model experiments use a high independent variance for feature firing magnitudes. This high variance makes it impractical for the SAE to learn the absorption solution, which is mixing parent + child latents together, as the high variance means the merged solution will struggle to reconstruct this variance - basically this variance makes it look to the SAE like the features are actually independent. I find that if I reduce the variance in the toy model used in this paper, the MP-SAE no longer handles hierarchy and indeed frequently learns mixtures of parent + child features. Furthermore, it appears that this work uses the toy model setup from the Matryoshka SAEs work in their code nearly verbatim, but has modified the firing variance so it is much higher than in the original work, indicating the authors are aware that the default setting of 0 or low variance causes their solution to fail. There is no reason to assume that hierarchical features in reality have such high independent variance in their firings, and that their firing magnitudes are uncorrelated with each other, so I do not expect MP-SAEs to solve hierarchy in general.\n\nAnother weakness of this work is that no evaluations are done on LLMs, which makes it hard to compare the performance of these SAEs against existing SOTA SAEs in a real setting. Furthermore, none of the benchmarks performed investigate if the latents learned are actually interpretable (e.g. using something like a sparse-probing benchmark or even auto-interp).\n\nIdeally the authors should train MP-SAEs on something like Gemma-2-2b and run the evaluations from SAEBench [1], ideally comparing to SOTA SAEs like Gemma Scope [3]. SAEBench also includes a feature absorption metric which evaluates how well the SAE disambiguate hierarchical features, so if MP-SAEs do in-fact handle hierarchy correctly they should achieve stellar performance on that benchmark, similar to Matryoshka SAEs. SAEBench also includes sparse probing, concept disambiguation, and auto-interp benchmarks that can be used to gauge how interpretable the learned latents are. This would also allow directly comparing performance to Matryoshka SAEs, as these also are designed to address feature hierarchy issues [4].\n\nThis is minor, but the paper motivates conditional hierarchy based on the paper \"The geometry of categorical and hierarchical concepts in large language models\", but my understanding is that this paper has been debunked [2].\n\nThis method is likely much less efficient than standard SAEs as the encoder requires a for-loop rather than a single forward pass to find activating features.\n\nReferences\n1. Karvonen, Adam, et al. \"Saebench: A comprehensive benchmark for sparse autoencoders in language model interpretability.\" _arXiv preprint arXiv:2503.09532_ (2025).\n2. Golechha, Satvik, et al. \"Intricacies of Feature Geometry in Large Language Models.\" _The Fourth Blogpost Track at ICLR 2025_.\n3. Lieberum, Tom, et al. \"Gemma scope: Open sparse autoencoders everywhere all at once on gemma 2.\" _arXiv preprint arXiv:2408.05147_ (2024).\n4. Bussmann, Bart, et al. \"Learning multi-level features with matryoshka sparse autoencoders.\" _arXiv preprint arXiv:2503.17547_ (2025).", "was_revised": true, "final_justification": "My main issue with this work was that the paper claimed MP-SAEs solve hierarchy in general, as I do not believe this is true. The authors justified this by copying the Matryoshka SAEs code and benchmark verbatim, but subtly modifying the benchmark to add extra variance to feature firings. Without this modification, MP-SAEs fail to deal with hierarchy and fail at that benchmark. Furthermore, despite modifying the benchmark to favor MP-SAEs, the authors then used the results of this modified benchmark to claim that MP-SAEs outperform Matryoshka SAEs at solving hierarchy.\n\nThe authors have agreed to the following, so I will raise my score, as I specified earlier in the discussion:\n\n- Explain that they have changed the Matryoshka SAEs benchmark by adding extra firing variance\n- Explain why they think this is justified\n- Explain that MP-SAEs fail to handle hierarchy in the unmodified version Matryoshka SAEs benchmark with (near) perfectly correlated firing magnitudes, since MP-SAEs fail to solve hierarchy if the features have strongly correlated firing magnitudes.", "rebuttal": "We thank the reviewer for their feedback and are particularly excited that they consider this work **“a huge contribution to the field” that “opens up many avenues for future work.”**\n\n>_(Paraphrased) Authors deviated from Matryoshka’s toy setup: MP‑SAEs separate parent and child features only under high feature‑variance; with low variance, they collapse into parent+child mixtures_\n\nWe thank the reviewer for raising the issue of feature variance, as it exposes **a strong and implicit assumption in the Matryoshka toy setup** that is not discussed in the original paper: **parent and child norms are perfectly correlated**. In this setup, each input is defined as $x = p + c$ with a fixed proportional norm $\\|p\\| = \\lambda \\|c\\|$. This structure systematically favors absorption (i.e., learning a single parent+child latent) because the signal-to-signal overlap is exact and firing magnitudes are perfectly correlated. In such a scenario, it is mathematically optimal for any $\\ell_0$-penalized method (like MP‑SAE) to encode both with a single feature, as the absorbed solution is constant and highly compressible.\nOur variant of the toy setup preserves the same structure ($x = p + c$) but **relaxes the norm constraint**, sampling\n$$p \\sim \\mathcal{N}(\\mu_1, \\sigma_1), \\quad c \\sim \\mathcal{N}(\\mu_2, \\sigma_2),$$\nso that children still appear only with parents but the magnitudes are no longer perfectly proportional. This introduces realistic variance, reflecting that in real data, children rarely co‑occur with identical parent magnitudes. We strongly believe this setup better reflects realistic hierarchical data and highlights MP‑SAE’s robustness beyond over‑simplified scenarios.\nWe acknowledge that high variance with equal means ($ \\mu_2=\\mu_1$) can yield edge cases where a strong child and weak parent make $p+c \\approx c$, making the parent appear almost absent. To address the reviewer’s point, we have now evaluated MP‑SAE over a grid of child means and variances, with the parent fixed as $p \\sim \\mathcal{N}(1,(1/4)^2)$. For each configuration, we computed the cosine similarity between learned features and the ground‑truth parent across 15 runs. Lower similarity indicates successful separation; high similarity or NaNs indicate absorption or failure to recover either component.\n### MP\n|Std\\Mean|0.1|0.5|1.0|\n|:--|--:|--:|--:|\n|1/400|3.6e-03|7.3e-02|1.3e-03|\n|1/40|5.0e-03|6.2e-02|1.2e-03|\n|1/8|-|4.3e-02|1.1e-03|\n|1/4|-|-|1.1e-03|\n\n### Matryoshka\n|Std\\Mean|0.1|0.5|1.0|\n|:--|--:|--:|--:|\n|1/400|nan|4.8e-02|2.1e-01|\n|1/40|nan|9.0e-02|1.5e-01|\n|1/8|-|9.6e-02|2.1e-01|\n|1/4|-|-|2.2e-01|\n\n### Vanilla\n|Std\\Mean|0.1|0.5|1.0|\n|:--|--:|--:|--:|\n|1/400|nan|7.0e-01|4.5e-01|\n|1/40|nan|7.0e-01|4.5e-01|\n|1/8|-|7.0e-01|4.7e-01|\n|1/4|-|-|4.7e-01|\n\nOverall, across the board, we see that MP-SAE reliably recovers the parent and child as distinct features, even when the child’s firing magnitude is very small and exhibits low variance. In contrast, other methods consistently fail in these regimes, either merging parent and child into a single atom or failing to recover either component (as reflected by NaNs in the results). This confirms that feature absorption observed in prior work arises from **toy benchmarks assuming perfect parent‑child correlation**! \n\n>_the authors should train MP-SAEs on something like Gemma-2-2b and run the evaluations from SAEBench [...] to make it easier to recommend MP-SAEs as the SOTA going forward_\n\nThank you for the comment! Given this and some of the other comments, we believe there is a misunderstanding on the goal of our paper and we would hence like to clarify this first.\n\n**Goals of our work:** Please note **the goal of our work was not to develop a “SOTA” SAE.** We in fact strongly think such an SAE does not exist, and by no means do we believe MP-SAE is that universal tool (as we explicitly noted in the limitations of our work, and are further formalizing in a follow-up paper now). Instead, we note the goal of our work was to highlight limitations of the standard working hypothesis that underlies the design of SAEs, i.e., the linear representation hypothesis (LRH). We wanted to assess that when neural network representations encode concepts in a manner that is outside the scope of LRH, are SAEs, which are motivated via LRH, able to capture such concepts. To this end, we aimed at providing a formalization of (our interpretation of) LRH and how concepts exhibiting hierarchical structure are outside that scope. Thus, our main goal with proposing MP-SAE was to offer a baseline to compare standard SAEs with. With real data experiments (e.g., our large-scale vision experiments), we wanted to see if we can produce interesting findings (such as our results on multi-modal representations) when using a tool which has a different inductive bias than standardly used interpretability tools (such as linear probes, which by definition will find concepts that are solely linearly accessible),\n\n**On empirical evaluations with SAEBench:** We would like to emphasize our experiments are already fairly wide in their scope: we evaluated MP-SAE against several standard SAE architectures on **large-scale vision models, multimodal vision-language models, smaller language models, and synthetic tasks!** While we certainly agree with the reviewer that analyzing MP-SAEs trained on LLMs to enable a comparison with other SAEBench results would be interesting, we strongly emphasize such an evaluation warrants its own paper and is not the goal of our work. We are nevertheless happy to add minimal experiments for MP-SAEs trained on Gemma-2B, e.g., reporting the learning curves and feature visualizations from SAELens, in the final version of the paper. \n\n>_Furthermore, none of the benchmarks performed investigate if the latents learned are actually interpretable (e.g. using something like a sparse-probing benchmark or even auto-interp)._\n\nWe emphasize our analysis of multimodal models, where we show the surprising finding of representations decomposing across different modalities (see Figure 17 in the Appendix), involved analyzing what inputs (both text and visual tokens) activate the latents. This involved producing several feature visualization maps (in fact Figure 8b in the main paper emerged out of this analysis), which we are happy to add to the appendix. **We will also upload the general tooling we developed for extracting these features to our Github!** We would have been happy to release these results now, but are unable to do so because of NeurIPS guidelines.\n\n>_...but my understanding is that this paper has been debunked._\n\nGreat question! We feel “debunked” is too strong. The blog post shows that in high-dimensional spaces, random vectors can satisfy conditional orthogonality, which we agree with. **However, the reverse still holds: hierarchical factors of the data distribution satisfy this geometrical condition (see the updated paper).** Thus, seeking conditionally orthogonal directions to explain the data remains meaningful (as shown in Figs. 8b, 17)—we emphasize the phrase “explain the data” here, since random vectors that satisfy conditional orthogonality will not enable good reconstruction. More broadly, this applies to all SAEs: even vanilla SAEs rely on quasi-orthogonal directions, and while random vectors can appear orthogonal, only those that reconstruct the data provide meaningful explanations.\n\n>_This method is likely much less efficient than standard SAEs…_\n\nFair point! We believe efficiency is generally not a crucial factor for interpretability tools, since the training cost is a one-time expenditure and if it can offer you a useful lens into the model, that cost can be worth it. That said, we emphasize that across all our experiments, **we found MP-SAE converges substantially faster than alternative architectures.** For example, we found MP-SAEs can achieve the same loss as a TopK SAE in 1/10th the training iterations (~95% of the wall clock time). \n\n>_Conceptually I don't see how the MP-SAE actually encourages learning the parent latent rather than a parent + child…_\n\nApologies if this was unclear! In MP-SAE, residual updates naturally separate parent and child concepts. If $P$ is selected first, the residual becomes $r^1 = x - P = C$, so the next step selects the feature aligned with $C$ rather than $P+C$, MP-SAE will thus naturally learn a feature aligned with $C$ rather than the absorbed $P+C$ direction. \n\n>_…Given the dictionary is still just linear vectors, why do the authors think that MP-SAEs are capturing non-linear concepts?_ \n\nThank you for the question! We discuss this in Section 3(iii) but are happy to clarify. **The key distinction is between non-linear and non-linearly accessible concepts**, where the selection depends non-linearly on the input. In Matching Pursuit, concepts selected at $ t>1$ depend on $ D^\\top r^{(t-1)} $, with $ r^{(t-1)} = x - D \\cdot top_1 (D^\\top x)$, which is non-linear in $ x $. By contrast, standard SAEs select all features based only on $W_{\\text{enc}}x$, which is linear. Thus, MP enables non-linear selection even though the reconstruction $\\hat{x} = Dz$ remains a linear combination. We will clarify this in the final paper.\n\n>_Have the authors observed any specific non-linear concepts that are recovered by MP-SAEs…?_\n\nA specific example of a non-linearly accessible concept appears in the multimodal features extracted by MP-SAE (though not recovered by standard SAEs), as shown in our main results and detailed in Appendix B.4. Figure 17 provides an illustration of how such concepts are encoded, highlighting how MP-SAE can recover multimodal features that are inaccessible to standard SAEs. We will revise the manuscript to emphasize more clearly that this is an example of non-linearly accessible concepts recovered by MP-SAEs.\n\nWe hope our response addresses the reviewer’s concerns and encourages them to consider raising their score. We remain available for any further questions!"}]

Author rebuttal:

Meta-review: The paper introduces the idea of integrating the matching pursuit sparse coding framework into sparse autoencoder architectures, aiming to improve the representation of meaningful and hierarchical features. It effectively highlights the shortcomings of existing methods while proposing a novel solution. The main concern, as noted by reviewers, is the adjustment of benchmark parameters, which should be carefully addressed in the final version. In addition, there are several other reviewer comments regarding clarity and presentation that the authors should incorporate. Despite these limitations, the work makes a worthwhile contribution to the interpretability literature.
Decision: Accept (poster)