SAEBench: A Comprehensive Benchmark for Sparse Autoencoders in Language Model Interpretability
Authors: Adam Karvonen
Can Rager
Johnny Lin
Curt Tigges
Joseph Isaac Bloom
David Chanin
Yeu-Tong Lau
Eoin Farrell
Callum Stuart McDougall
Kola Ayonrinde
Demian Till
Matthew Wearden
Arthur Conmy
Samuel Marks
Neel Nanda
Venue: ICML 2025 poster (2025)
Keywords: Sparse autoencoders
mechanistic interpretability
evaluations
Primary area: social_aspects->accountability_transparency_and_interpretability
TL;DR: SAEBench, a comprehensive suite of benchmarks for sparse autoencoders (SAEs) in language model interpretability

Abstract: Sparse autoencoders (SAEs) are a popular technique for interpreting language model activations, and there is extensive recent work on improving SAE effectiveness. However, most prior work evaluates progress using unsupervised proxy metrics with unclear practical relevance. We introduce SAEBench, a comprehensive evaluation suite that measures SAE performance across eight diverse metrics, spanning interpretability, feature disentanglement and practical applications like unlearning. To enable systematic comparison, we open-source a suite of over 200 SAEs across seven recently proposed SAE architectures and training algorithms. Our evaluation reveals that gains on proxy metrics do not reliably translate to better practical performance.  For instance, while Matryoshka SAEs slightly underperform on existing proxy metrics, they substantially outperform other architectures on feature disentanglement metrics; moreover, this advantage grows with SAE scale. By providing a standardized framework for measuring progress in SAE development, SAEBench enables researchers to study scaling trends and make nuanced comparisons between different SAE architectures and training methodologies. Our interactive interface enables researchers to flexibly visualize relationships between metrics across hundreds of open-source SAEs at www.neuronpedia.org/sae-bench

SAEBench: A Comprehensive Benchmark for Sparse Autoencoders
in Language Model Interpretability
Adam Karvonen ! Can Rager"! Johnny Lin Curt Tigges *2 Joseph Bloom 2 David Chanin
Yen-Tong Lau' Eoin Farrell Callum McDougall
Kola Ayonrinde
Demian Till Matthew Wearden
Arthur Conmy
Samuel Marks
Neel Nanda
Abstract
1. Introduction
Sparse autoencoders (SAEs) are a popular tech- nique for interpreting language model activations, and there is extensive recent work on improving SAE effectiveness. However, most prior work evaluates progress using unsupervised proxy met- rics with unclear practical relevance. We in- troduce SAEBench, a comprehensive evaluation suite that measures SAE performance across eight diverse metrics, spanning interpretability, feature disentanglement and practical applications like unlearning. To enable systematic comparison, open-source a suite of over 200 SAEs across seven recently proposed SAE architectures and training algorithms. Our evaluation reveals that gains on proxy metrics do not reliably translate to better practical performance. For instance, while Matryoshka SAEs slightly underperform on ex- isting proxy metrics, they substantially outper- form other architectures
How can evaluate dictionary learning for language model interpretability? Sparse autoencoders (SAEs (Cunningham et al., 2023; Bricken et al., 2023)) are popular method for finding interpretable units in neural networks through dictio- nary learning. Substantial recent work has been focused on improving SAE architectures (Rajamanoharan al., 2024a; Mudide et al., 2024), activation functions (Gao et al., 2024; Taggart, 2024; Rajamanoharan et al., 2024b; Bussmann et al., 2024a; Ayonrinde, 2024), and loss functions (Buss- mann et al., 2024a; Karvonen et al., 2024; Marks et al., 2024a). However, measuring the effectiveness of these meth- ods in improving interpretability remains a core challenge.
An ideal SAE decomposes neural activations into inter- pretable, independently composable units that faithfully represent the internal state of a neural network. However, due to a lack of ground truth labels for language models' internal features, researchers instead train SAES by optimiz- ing unsupervised proxy metrics like sparsity and fidelity (Cunningham et al., 2023). Maximizing reconstruction ac- curacy at a given level of sparsity successfully provides interpretable SAE latents, but sparsity has known problems as proxy, such as Feature Absorption (Chanin al., 2024a) and composition of independent latents (Bussmann et al., 2024c). Nevertheless, most SAE improvement work primar- ily measures whether reconstruction is improved at a given sparsity, potentially missing problems like uninterpretable latents or increased feature absorption.
feature disentangle- ment metries; moreover, this advantage grows with SAE scale. By providing a standardized framework for measuring progress in SAE de- velopment, SAEBench enables researchers to study scaling trends and make nuanced compar- isons between different SAE architectures and training methodologies. Our interactive inter- face enables researchers to flexibly visualize relationships between metrics across hundreds of open-source SAES at neuranpedia / sae-bench. Code and models available at: githbbccom//aeo www.e/o//mmmaaaaaooooommmmmm /AEeeecch///Beehhhhcececcaaa
In the absence of single, ideal metric, argue that the best way to measure SAE quality is to give more de- tailed picture with a range of diverse metrics. In particular, SAEs should be evaluated according to properties that prac- titioners actually care about. We characterize SAES by con- cept detection, interpretability. feature disentanglement and reconstruction. Covering all aspects, SAEBench enables measuring progress with new training approaches, tuning training hyperparameters, and selecting the best SAE for particular task.
'Equal contribution 'Independent 'Decode Research e University College London 'Cambridge Consultants *MATS Research "Anthropic. Correspondence to: Adam Karvonen Sadam.karvona gmail.com>, Can Rager www.rrggr@@gmii.comm.
Proceedings of the 42nd International Conference Machine Learning, Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).
SAFBench: a standardized suite of eight evaluations

---

SAEBench: A Comprehensive Benchmark for Sparse Autoencoders in Language Model Interpretability
capturing different aspects of SAE quality, including two novel metrics for feature disentanglement
Loss function: P-anneal SAE (Karvonen et al., 2024), Matryoshka SAE (Bussmann et al., 2024b), Feature- Aligned SAE (Marks al., 2024a).
2. Training and SAEBench evaluation of over 200 SAEs with varying architectures, training methodologies. and widths.
3. A nuanced analysis of the evaluations from (2), with im- plications for SAE architecture choice, scaling trends, and training dynamics. Many of these trends are invisi- ble to traditional SAE evaluation metries. For instance, we find that Matryoshka SAEs perform well on feature disentanglement and concept detection metrics, despite appearing worse on existing proxy metrics.
Most of these improvements were guided by optimizing the fonsiiy-fidellity tradeoff-maximizing reconstruction quality at a given level of sparsity. However, this unsuper- vised metric may not directly correspond to desirable traits such as interpretability. For instance, an infinite- wiith SAE with an Lo norm of could theoretically achieve perfect reconstruction while failing to provide meaningful insights into the model's ' representations.
2. Related work
2.2. SAE Evaluations
2.1. SAEs for Interpretability
Previous evaluation approaches have largely focused on specific aspects of SAE performance (see Appendix C for detailed discussion). Automated interpretability using lan- guage models has been widely adopted (Paulo et al., 2024; Rajamanoharan et al., 2024b), though it often struggles to differentiate between architectures (Anthropic Interpretabil- ity Team, 2024b). Other evaluation methods include sparse probing of labeled concepts (Gao et al., 2024), evaluation on board games with ground truth features (Karvonen et al., 2024), and supervised dictionary comparison (Makelov et al., 2024; Venhoff et al., 2024).
Sparse autoencoders (SAEs) emerged as an unsupervised tool for decomposing LLM activations into sparse linear combinations of learned feature directions that are often in- terpretable (Cunningham et al., 2023; Bricken et al., 2023). In its basic form, an SAE consists of an encoder that maps model internal activations J to sparse, higher-dimensional feature space and a decoder that reconstructs the input ac- tivations as é. The standard architecture uses a linear en- coder followed by a ReLU activation and a linear decoder, trained to minmmize both reconstruction error and I,1 spar- sity penalty while maintaining normalized decoder columns. The forward pass and optimization objective can be formal- ized as:
While these existing benchmarks can provide valuable in- sights, they each focus on specific aspects of SAE perfor- mance. This limited evaluation scope has led many re- searchers to default optimizing the sparsity-fidelity trade- off, despite its known limitations.
= ReLU(WEE g& + +b)
Whh + bn
(2)
C- = || 3 + ||A||
(3)
3. SAEBench: A Comprehensive Benchmark
reconstruction
sparsity
SAEBench addresses these challenges by providing a uni- fied evaluation framework that captures multiple aspects of SAE performance while remaining computationally tractable and easy to use. We identified five key require- ments for comprehensive SAE evaluation framework:
where & represents the input activation, h is the sparse hidden representation, is the reconstructed activation, Wg.be by are the encoder weights and biases, WD, by are the decoder weights and biases, wy represents the j-th col- umn of WD, and À is the sparsity coefficient.
Diversity. Metrics should capture a broad range of SAE behavior-from basic reconstruction to downstream task performance-sinc relying on a single metric can overlook important tradeoff'. Extensibility. The framework should offer a standardized structure, making it easy to add new evaluation methods as the field evolves. Speed. Evaluations must complete within one two hours per SAE, facilitating rapid research iteration. Automation. A single script should run all evaluations, eliminating the need to integrate separate codebases manually. Reproducibility. All metrics must be deterministic and reproducible to ensure fair comparisons between architectures.
Recent work has proposed numerous improvements to the original ReLU SAE design. These innovations span multiple aspects:
ketworkssruuturr Gated SAE (Rajamanoharan et al.,, 2024a), Switch SAB (Mudide et al., 2024)
Activation function:
TopK SAE (Gao et al., 2024),
BatchTopK SAE (Bussmann et al., 2024a), Jump SAE (Rajamanoharan et al., 2024b), ProLU SAE (Tag- gart, 2024), Feature Choice SAE (Ayonrinde, 2024)
2

---

SAEBench: A Comprehensive Benchmark for Sparse Autoencoders in Language Model Interpretability
SAEBench
Interpretability
Feature Disentanglement
Automated Interpretability
Unlearning
RAVEL
Concept Detection
Absorption
Targeted Probe Perturbation
Reconstruction
Sparse Probing
Loss Recovered
Spurious Correlation Removal
Figure I: SAEBench evaluates sparse autoencoders across four fundamental capabilities. Concept Detection measures how well individual latents map to meaningful concepts. Interpretability evaluates feature comprehensibility using automated LLM evaluation. Reconstruction quantifies how faithfully the SAE preserves model behavior. Feature Disentanglement evaluates whether independent concepts are properly separated. These capabilities provide a comprehensive view of SAE performance beyond traditional metrics.
3.1. Metrics
Below we summarize each metric. Full implementation details are provided in Appendix D.
Effective sparse autoencoders should excel across multi- ple dimensions: they should capture meaningful individual concepts, produce luman-interpretable latents, faithfully reconstruct activations, and properly separate independent concepts. However, optimizing for any single dimension can lead to undesirable trade-offs. For instance, maximiz- ing reconstruction might come at the cost of feature inter- pretability, while focusing solely on concept detection could sacrifice faithful representation of the original model's s be- havior.
3.2. Existing Metrics
3.2.1. TRADITIONAL SPARSITY-FIDELITY TRADEOFF
The main training objective sparse autoencoders to learn sparse representations that accurately reconstruct the input. Sparsity is quantified by the To norm and often approximated by L2 norm training objective enforced by TopK mask. Reconstruction (fidelity) is quantified by mean squared eπror or the loss recovered score, which is defined as
We therefore organize SAEBench around four fundamental capabilities that together characterize effective SAEs:
HH Ho)
(4)
(Horig Ho)
Concept Detection: Measures how precisely individ- ual latents correspond to meaningful concepts through Sparse Probing and Feature Absorption metrics
where Hong represents the cross-entropy loss of the model for next-token prediction, H* denotes the cross-entropy loss when substituting the model activation x with its SAE reconstruction á during the forward pass, and Ho is the cross cross-entropy loss resulting from zero-ablating x.
Interpretability: Evaluates uuman-uundrstandaablityy of learned latents using an LLM judge
Quantifies how faithfully the SAE preserves the model's original behavior via Loss Re- covered metrics
Reconstruction:
The
trade-off has dominated recent SAE development, typically appearing the primary evaluation metric in architectures, while results interpretability metrics often remain inconclusive or secondary. While ad- vances in this trade-off are valuable, small gains do not nec- essarily translate into qualitatively better representations or superior performance on downstream interpretability tasks. We therefore regard the cannity-fidelity curve as necessary but not sufficient for evaluating SABs.
Feature Disentanglement: Assesses proper separa- tion of independent concepts through Unlearning, Spu- rious Correlation Removal, and Targeted Probe Pertur- bation
These capabilities are measured through eight distinct met- rics, combining established approaches with novel evalua- tion methods. Although baseline methods outperform SAEs on several of the benchmark tasks (as detailed further in Appendix D), we retain these tasks to ensure coverage of di- verse evaluation criteria. Most tasks are practically relevant to existing problems, but some, such as Sparse Probing and Feature Absorption, are diagnostic in nature.
3.2.2. AUTOMATED INTERPRETABILITY
We adopt standard LLM-based judging framework (Paulo et al., 2024): for each selected latent, a language model first proposes a "feature description" using a range of activating examples. In the test phase, we construct test set by sampling sequences that activate the latent across different
3

---

SAEBench: A Comprehensive Benchmark for Sparse Autoencoders in Language Model Interpretability
activation strengths, along with random control sequences. The LLM judge uses the feature description it created to predict which sequences would activate the selected latent, and the accuracy of these predictions determines the final interpretability score.
3.3. Adapted Metries
The following approaches were originally developed to study specific phenomena in SAEs rather than general evaluation metries. We adapt them into quantitative mea- sures that can be systematically applied to any SAE.
3.2.3. k-SPARSE PROBING
3.3.1. FEATURE ABSORPTION
Sparse probing evaluates whether SAEs isolates pre- specified concepts. For each concept (e.g., sentiment), we identify the k most relevant latents by comparing their mean activations on positive versus negative examples and train a linear probe on top & latents. If those latents align well with the concept, the probe's accuracy will be high even though the SAE was not explicitly supervised to isolate that concept. This metric can be efficiently calculated using only the SAE and precomputed model activations.
Feature absorption (Chanin et al., 2024b) is a phenomenon where sparsity incentivizes SAEs to learn undesirable fea- ture representations, This occurs with hierarchical concepts where A implies B (e.g., pig impliess mammal, or red im- plies color)-rathen than learning separate latents for both concepts, the SAE is incentivized to learn a latent for A and a latent for "B except A" to improve sparsity. Feature absorption often manifests in unpredictable ways, creating gerrymandered latents where, for instance, "starts with S" feature might activate on 95% of S-starting tokens but inexplicably fail on an arbitrary 5% where the feature has been absorbed elsewhere.
The choice of k depends on the use case: For human inter- pretability, mapping concepts single latents is ideal, while for understanding model representations, research suggests concepts can be distributed across multiple latents (Engels et al., 2024). We evaluate across {1, 2, 5) latents but focus our analysis on 1. Our methodology is based on Gurnee et al. (2023), who applied sparse probing to identify context-specific MLP neurons, and Gao et al. (2024), who adapted sparse probing to evaluate SAEs.
We build on the metric proposed in Chanin et al. (2024b), which examines how SAE latents represent first-letter clas- sification tasks, identifying cases where the main latents for letter fail to fully capture the feature while other latents compensate. Our implementation extends this approach with more flexible measurement technique that enables evaluation across all model layers. Motivated by manual in- spection revealing absorption patterns missed by the original metric, we introduce methods to detect partial absorption and cases where multiple latents share responsibility for absorption.
3.2.4. RAVEL
If an SAE effectively captures independent concepts, each should be encoded by dedicated latents, achieving clear dis- entanglement. To measure this, we implement the RAVEL (Resolving Attribute-Value Entanglements in Language Models) evaluation from Huang et al. (2024), which tests how cleanly interpretability methods separate related at- tributes within language models. RAVEL evaluates whether targeted interventions on SAE latents can selectively change a model's predictions for specific attributes without unin- tended side effects-for instance, making the model believe Paris is in Japan while preserving the knowledge that the language spoken remains French.
3.3.2. UNLEARNING CAPABILITY
In many practical applications, we want to selectively re- move knowledge from a language model without disrupting unrelated capabilities. Farrell et al. (2024) examined the effectiveness of SABS for unlearning by applying condi- tional negative steering. We identify relevant latents by comparing their activation frequencies between a forget set (biology-related text in the WMDP-bio corpus) and retain set (WikiText)) then clamp these latents negative values whenever they activate. We build on their methodology and report an unlearning score for each individual SAE, measuring unlearning success via degraded accuracy on WMDP-bio test questions while using MMLJU categories to verify retained capabilities. Models that achieve strong unlearning of the target domain with minimal side effects other domains score higher.
Concretely, RAVEL works as follows: given prompts like "Paris is in the country of France," "People in Paris speak the language French," " and "Tokyo is a city." we encode the tokens Paris and Tokyo using the SAE. We train a binary mask to transfer latent values from Tokyo to Paris, decode the modified latents, and insert them back into the residual stream for the model to generate completions. The final disentanglement score averages two metrics: the Cause Metric, measuring successful attribute changes due to the intervention, and the Isolation Metric, verifying minimal interference with other attributes.
3.4. Novel Metrics
While existing metrics capture many aspects of SAE per- formance, they don't 't directly measure how completely and
4

---

SAEBench: A Comprehensive Benchmark for Sparse Autoencoders in Language Model Interpretability
cleanly SAEs isolate concepts within small groups of la- tents-a property crucial for both human analysis and cir- cuit analysis. Existing methods like unlearning and RAVEL can modify behavior through steering individual latents, even when concepts are distributed across many latents, as steering a few key latents can be sufficient to alter model behavior. Our metries instead use zero ablation, which pro- vides a stronger test of concept isolation: only when we've identified all latents representing concept will zeroing them out completely remove that concept's 's influence.
[20, 200)) with directly comparable baselines. Many eval- uations have strong correlation with sparsity, making it essential to assess performance across multiple sparsity lev- els. This approach ensures that improvements are genuine rather than statistical noise. Furthermore, it verifies that advances genuinely advance the Pareto frontier on target metrics, rather than merely reflecting an underlying correla- tion with different sparsity levels.
4. Results
Our metrics adapt the methodology from Marks et al. (2024c), which demonstrated removing unwanted corre- lations from classifiers through targeted ablation of SAE k latents. Both metries evaluate two key properties through zero ablation:
We use SAEBench to evaluate suite of both common and novel SAE architectures listed in Table I trained using the open source library et al., 2024b).
(Marks
Completeness: Whether a concept is fully captured by a small set of latents
Evaluated SAE Architectures
ReLU (Anthropic Interpretability Team, 2024a)
Isolation: Whether different concepts are encoded by distinct groups of latents
MatryoshkaBatchToph (Bussmann et al., 2024b)
TopK (Gao et al., 2024)
Spurious Correlation Removal (SCR) extends the SHIFT method from Marks et al. (2024c). Starting with a biased linear probe classifier that has learned both intended signals (e.g.. profession) and spurious correlations (e.g., gender), we measure how effectively zero-ablating a small number of SAE latents can remove the unwanted correlation from the SAE's output. If these latents cleanly isolate the spurious concept, removing them should significantly improve the classifier's accuracy on the intended signal.
BatchTopK (Bussmann et al., 2024a)
Gated (Rajamanoharan et al., 2024a)
JumpReLU (Rajamanoharan et al., 2024b)
P-Annealing (Karvonen et al., 2024)
Table 1: List of evaluated sparse autoencoder architectures
We train multiple variants sweeping over widths (4k, 16k, and 65k latents) and sparsities (Lo ranging from 20 to 1000) on residual stream activations obtained at middle layers of Gemma-2-2B (layer 12; Gemma Team et al. (2024)) and Pythia-160M (layer 8; Biderman et al. (2023)). We will open-source the suite of over 200 SAEs upon comple- tion of the peer-review process. Further training details are contained in Appendix B. In addition, we evaluate SAEs of widths 16k, 65k, 131k, and IM on Gemma-2-2B and Gemma-2-9B from the Gemma-Scopp series (Lieberum et al., 2024), with results in Appendix I.
Targeted Probe Perturbation (TPP) generalizes this ap- proach to multi-class settings. For each class, we train binary classification probes and identify its most relevant latents. We then measure how zero-ablating these latents affects probe accuracy across all classes. A high TPP score indicates that concepts captured by distinct sets of la- tents-ablating latents relevant to one class should primarily degrade that class's 's probe accuracy while leaving other class probes unaffected.
Both metrics can be efficiently calculated using only the SAE and precomputed model activations, bypassing the need for expensive model forward passes. We sweep over ablation set sizes k E {5, 10, 20, 50, 500) to assess how concept completeness varies with feature count. In presenting our results, we focus on ablation sets of size 20 as a practical size for manual analysis, although we observe similar trends within the range of & e [5, 50). Full results across all ablation sizes are available in Appendix H.
Our key takeaways are detailed in the following paragraphs. We discuss 65k width SAEs trained on Gemma-2 2--m unless noted otherwise. Similar trends exist at smaller dictionary widths, as seen in Appendix J, but we observe clearer differ- entiation at larger widths.
We emphasize that we examine a much wider range of sparsities than the typical range of Lo [20, 200). Previous work identified the most interpretable SAE latents in this Lo range (Bricken et al., 2023; Rajamanoharan et al., 2024b). We evaluate metrics across a much broader range of Lo E [20, 1000] than typically studied to provide a more complete understanding of sparsity's 's role SAE performance.
3.5. Practitioner's Guide
When evaluating new SAE methods, we recommend train- ing multiple SAEs across a range of sparsities (Top E
5

---

SAEBench: A Comprehensive Benchmark for Sparse Autoencoders in Language Model Interpretability
0.90
Mean Absorption Fraction Score
1.000
0.995
Autointerp Score
0.85
Loss Recovered
0.990
0.80
0.7
0.985
0.75
0.6
0.980
0.70
0.5
10a
10a
10a
10a
10a
10>
(Sparsity)
Lo (Sparsity)
(Sparsity)
Sparse Probing top 1 lest Accuracy
0.80
SCR Top 20 Metric
0.75
0.3
0.78
RAVEL Score
0.76
0.70
0.2
0.74
0.1
0.72
0.65
0.70
10
10
10
10a
10a
10a
(Sparsity)
(Sparsity)
(Sparsity)
RelU JumpRelu
TopK
ReLU with p-Annealing
Gated PCA Base line
Batch TopK
Matryoshka Batch Topk
Figure 2: Scores for the Loss Recovered, Automated Interpretability, Absorption, SCR, and Sparse Probing metrics on the 65k width Gemma-2-2B suite of SAES.
4.1. Comparing SAE Architectures
The sparsity-fdeelity frontier does not reliably indicate performance on downstream tasks. The sparsity-fidelleee frontier shows distinct rankings across different Lo regimes. In the low-Lo regime (< 100), architectures are clearly sep- arated, with BatchTopK performing best, followed by TopK, Jump, Gated, Matryoshka, p-anneal, and ReLU performing worst. As Lg increases to the middle regime [100, 500]. these performance differences diminish substantially, with most architectures achieving comparable performance cept for p-anneal and ReLU SAEs.
Matryoshka Batch Topk SAEs perform best concept detection and feature disentanglement tasks, especially in the typical L.0 range of 40-200 (5 of 8 metries). Most notably, the Matryoshka SAE obtains best scores on several metrics (Absorption, RAVEL, Sparse Probing, and SCR in Figure? and TPP Figure J) while performing worse than TopK and BatchTopK on the sparsity-fidelity frontier (Figure upper left). In LOs over 200 (larger than typi- cally used), however, Matryoshka is often not superior. We observe the recurring theme that Matryoshka shows qualita- tively different results than all other architectures.
However, these rankings don't I consistently align with per- formance on other metrics. For example, the Matryoshka ar- chitecture shows strong performance on SCR and feature ab- sorption despite its middling position on the maursity-fidellit frontier. Similarly, while p-anneal consistently outperforms Gated SAE on the absorption metric (Figure 2, lower left), Gated SAE shows superior performance on loss recovered (Figure 2, upper left). These contrasting results across dif- ferent metrics emphasize the importance of comprehensive evaluation using diverse metrics beyond the sparsity-fidelity trade-off.
The ReLU SAE outperformed by other methods on of 8 metrics. The ReLU SAE performs worst on loss recovered, agreeing with previous work (Anthropic Inter- pretability Team, 2024b; Rajamanoharan al., 2024a; Gao et al., 2024). We further find that all other architectures outperform the ReLU SAE on absorption, SCR, RAVEL, and TPP metrics.
However, ReLU variants do outperform on one metric. 65k width ReLU SAEs with an Lo 200 (above the range of [20, 200) typically examined in the literature) perform the best overall 1-sparse probing, seen in Figure ReLU SAEs further show comparable performance in the unlearning evaluation.
4.2. Dictionary Size Sealing Dynamics
Scaling behaviors are mixed across metrics. As we scale dictionary size from 4k to 16k 65k latents, we observe dis-
6

---

SAEBench: A Comprehensive Benchmark for Sparse Autoencoders in Language Model Interpretability
tinct patterns that illuminate the fundamental trade-offs in SAE design. Some metrics show consistent improvements with scale, as shown in Figure 3. Both Automated Inter- pretability scores and Loss Recovered generally increase with dictionary size across all architectures, suggesting that larger dictionaries enable both better reconstruction and more interpretable individual latents.
Lower Lo (higher sparsity) often helps human inter- pretability.
Higher Lo yields better reconstruction fidelity, RAVEL and targeted probe perturbation scores, and reduced feature absorption.
Moderate Lo sometimes balances these trade-offs or even performs best on certain metrics like sparse prob- ing and spurious correlation removal.
Matryoshka the only architecture improving on fea- ture disentanglement with scale. We observe worse per- formance with scale, also called inverse scaling, for most architectures on metrics that measure feature disentangle- ment and concept detection. Absorption scores worsen with increased dictionary size for all architectures except Ma- tryoshka, which shows only minor degradation. Similarly, SCR performance at fixed intervention budget decreases for most architectures as dictionary size grows, while Ma- tryoshka generally improves its performance (Figure 6).
In other words, no single Lo is optimal for all tasks. How- ever, moderate Lo E [50, 150) strikes reasonable balance between our various metrics.
4.4. Monitoring Training
We evaluate checkpoints of our 16k TopK and ReLU SAEs at O, SM, 15M, 50M, 150M, and 500M tokens our met- rics, where 500M is the total number of training steps. We find that many metrics achieve most of their performance by 50M tokens, although there is still slow improvement, as seen in Figure G. We caution that minor quantitative im- provements may also have major qualitative improvements, and higher training budgets may be worthwhile, especially for larger dictionary sizes.
We hypothesize that feature splitting determines scaling be- havior on disentanglement metrics. Bricken et al. (2023) demonstrate that increasing SAE width drives feature split- ting, producing more granular representations at lower levels of abstraction. Bussmann et al. (2024c) further shows that single decoder direction can fragment into multiple sub- features, highlighting the risk of over-splitting. In contrast, Matryoshka SAES employ a hierarchical design to learn multiple levels of abstraction simultaneously, avoiding such fragmentation. This hicrarchical structure likely explains their positive scaling behavior on feature disentanglement metrics, whereas other architectures exhibit negative scaling trends.
4.5. Model Scale Effects
When comparing results between Pythia- 160M and Gemma- 2-2B, we find that while reconstruction fidelity trends re- main consistent, metrics relying on supervised concepts (SCR, TPP, sparse probing, and feature absorption) show substantially different patterns, as seen in Appendix J. The advantages of hierarchical architectures on our SCR met- ric seen in Gemma-2-2B don't appear in Pythia- 160M. We attribute this to fundamental differences in model capabili- ties-our supervised metrics assume robust representation of concepts like spelling, professions, and demographics, which are likely weaker in smaller models. This raises ques- tions about the depeeddeeen of SAE evaluation metrics themselves. We therefore focus on Gemma-2-2B for our main comparisons better represents real-world usage.
Another hypothesis for the inverse scaling with SCR is that SCR zero ablates a fixed number of latents, and the inverse scaling is simply due to the fact that a smaller fraction of the larger SAB is being modified. However, this inverse scaling pattern persists even when controlling for intervention size. When we examine SCR scores across a range of intervention sizes (Figure 7), non-hierarchical architectures still show de- graded performance at larger dictionary sizes. This suggests that phenomena like feature splitting or feature absorption may be leading to poor isolation of concepts.
Note that we scale with a fixed number of training steps rather than fixed compute budget. In other words, all SAEs were trained with the same amount of training data while the number of FLOPs grows proportional to the dictionary size.
4.6. Unexpected Findings and Limitations
Feature disentanglement metrics (TPP, RAVEL) showed an unexpected preference for higher LO values (> 400), diverg- ing significantly from the conventional LO range of 20-200 typically used in SAE literature. This preference is par- ticularly pronounced for TPP, while interestingly, SCR-a conceptually related metric-does not exhibit this pattern. We hypothesize that higher sparsity forces the composition of multiple concepts into fewer active latents, potentially harming disentanglement.
4.3. Task- Besk-Dependent Optimal Sparsity
While different tasks demand different levels of sparsity, moderate To values of 50-150 offer reasonable compro- mise across metrics. Finally, we examine how performance varies with respect to Lo in Figure J.
7

---

SAEBench: A Comprehensive Benchmark for Sparse Autoencoders in Language Model Interpretability
1.00
Mean Absorption Fraction Score
1.0
Loss Recovered
0.99
Autointerp Score
0.85
0.9
0.98
0.80
0.8
0.97
0.75
0.7
0.96
0.6
0.70
0.95
0.5
4k
16k
65k
4k
16k
65k
4k
16k
65k
Dictionary Size
Dictionary Size
Dictionary Size
Sparse Probing Top Test Accuracy
0.78
0.70
SCR Top Metric
0.3
0.77
RAVEL Score
0.65
0.76
0.2
0.75
0.60
0.74
0.55
0.1
0.73
0.50
4k
16k
65k
4k
16k
65k
4k
16k Dictionary
65k
Dictionary
Dictionary
RelU
TopK
ReLU with p-Annealing
Gated
JumpRelU
Batch TopK
Matryoshka Batch TopK
PCA Base line
Figure 3: Scaling SAE width from 65k for across SAE architectures. each architecture width pair, we mean over all results in the Lg range between 40 and 200. Most notably the hicrarchical Matryoshka SAE shows positive scaling behavior. Due varying LO distributions across architectures, this visualization is intended primarily for analyzing scaling trends rather than architecture comparisons. Complete scaling results across all sparsity values are presented in Figure 6.
While we do see some notable trends with K-sparse probing, it provides limited differentiation between architectures, widths, and sparsities, with scores falling within a narrow range. This aligns with previous findings from Gao al. (2024), who had observed that "Our probe based metric is quite noisy", even across 61 binary classification datasets. However, we observe that all SAEs significantly outperform baseline of probing directly on K residual stream channels (0.65 on Layer 12 of 2-mma-2-2B).
affect unsupervised metrics like automated interpretability and sparsity-fidelity, it means our supervised metrics exam- ine only fraction of each SAE's latents. Due to this limited number of supervised concepts, some metries show rela- tively noisy results. However, several castiiss-parttculaatt spurious correlation removal and absorption clear and substantial differences between architectures, with hierarchical architectures outperforming other approaches by margins of 30-40%.
Our Unlearning evaluation is constrained by model capabil- ities meaningful unlearning measurement requires strong baseline task performance, but Gemma-2-2B only achieved sufficient performance
Quantitative metrics may not capture qualitative aspects of interpretability. Our benchmark does not directly cap- ture qualitative aspects of interpretability that researchers find valuable in practice. While metrics like automated- interpretability attempt to quantify feature interpretability, they may not reflect the nuanced insights gained through manual investigation of SAE latents during mechanistic analysis.
of the existing unlearning test sets. Future work should explore larger models with stronger task performance or develop unlearning datasets better matched to model capabilities.
5. Limitations
Our evaluation covers specific models but cannot address all language model architectures and scales. While we provide extensive evaluation across multiple architectures and dictionary sizes Gemma-2-2B and Pythia- SAE behavior may vary across different model scales, ar- chitectures, and layers. Future work could investigate how
Supervised metries are fundamentally limited by the availability of ground truth data. Our supervised metrics can only evaluate concepts with reliable ground truth data, representing small subset of the vast space of concepts encoded in language models. While this limitation doesn't 't
8

---

SAEBench: A Comprehensive Benchmark for Sparse Autoencoders in Language Model Interpretability
these patterns generalize across a broader range of models and network layers.
Impact Statement
This work aims to improve the evaluation of sparse au- toencoders, a key tool in mechanistic interpretability. By providing comprehensive benchmark, we seek to help researchers develop more interpretable models, diagnose failure modes, and better understand model representations. We believe that improving interpretability is more likely to reduce potential harms from Al systems by making their behavior more transparent and predictable. However, as with any research in this space, these insights could also accelerate broader advancements in AI, which carry both benefits and risks.
Metrics cannot be meaningfully combined into single score. Different downstream applications or users may pri- oritize different aspects of SAE performance - for example, interpretability or reconstruction accuracy. Additionally, our metrics operate on different scales and exhibit varying levels of noise. Given these complexities, any attempt to combine metries into a single score would require arbitrary weighting choices that could obscure important trade-offs between different aspects of SAE performance.
6. Conclusion
A Acknowledgements
SAEBench provides comprehensive evaluation framework that moves beyond the traditional pparsit--fidelity frontier to capture multiple dimensions of SAE performance. Our results reveal several key insights about SAE design and scaling, First, while recent architectural innovations show clear improvements over the original ReLU SAE on some metrics, hierarchical architectures like Matryoshka SAEs demonstrate dramatically superior performance on feature disentanglement tasks despite slightly worse reconstruction fidelity. Second, we find that dictionary size scaling pro- duces complex trade-offs: while larger dictionaries gener- ally improve reconstruction and per-feature interpretabil- ity, they can lead to degraded concept isolation in non- hierarchical architectures. Third, optimal sparsity levels vary significantly by task, though moderate LO values of 50-150 offer reasonable compromise across most metrics.
This work was conducted as part of the MI. Alignment & Theory Scholars (MATS) Program and supported by grant from OpenPhilanthropy. The program's 's collaborative en- vironment made this diverse collaboration possible. Adam Karvonen is grateful for support from compute grant pro- vided by Lambda as part of the Lambda Research Grant Program. We are grateful to Alex Makelov for discussions and implementations of sparse control, to McKenna Fitzger- ald for her guidance and support throughout the program, as well as to Bart Bussmann, Patrick Leask, Javier Fer- rando, Oscar Obeso, Stepan Shabalin, Arnab Sen Sharma and David Bau for their valuable input. Our thanks go to the entire MATS and Lighthaven staff.
Author Contributions
These findings highlight the importance of comprehensive evaluation across multiple metrics when developing new SAE architectures. While the field has primarily focused on optimizing the sparsity-fidelity trade-off, our results sug- gest that downstream task performance and feature disen- tanglement are both important considerations for practical applications. By providing a standardized benchmark suite and revealing previously hidden trade-offs, SAEBench aims to accelerate progress in neural network interpretability re- search.
A.K. and C.R. co-led the project. A.K., C.R., N.N., and S.M. designed the SAEBench evaluation suite and selected the evaluations. A.K., C.R., C.T., J.B., J.L., and built and maintained the infrastructure for SAEBench. The individ- ual metrics were implemented by A.K. (Sparse Probing), C.McD. and A.K. (Autointerpretabilitty) C.T., and A.K. (Core Metrics), Y.L., E.F., A.Co, and A.K. (Unlearning), A.K. and C.R. (Targeted Probe Perturbation, Spurious Cor- relation Removal, RAVEL), and D.Ch. and D.T. (Feature Absorption). A.K. and C.R. adapted the SAE training al- gorithms from the dictionary learning library (Marks et al., 2024b). K.A., A.K, and C.R. explored a minimum descrip- tion length evaluation. A.K. and C.R. trained the SAES used in our experiments. A.K., C.R., and J.I., conducted the evaluations. J.L. developed the interactive results browser on wwwpoonpedia.or//ssaeeeeee bench with feedback from A.K. and C.R.. S.M. and N.N. provided supervision and guidance throughout the project. M.W. contributed to project man- agement and coordination. The manuscript was primarily drafted by A.K. and C.R., with extensive feedback and edit- ing from M.W., S.M., N.N., and all other authors.
Future work could extend SAEBench to evaluate SAES across broader range of model scales and architectures, develop additional metrics for capturing qualitative aspects of interpretability, and investigate the relationship between training dynamics and downstream performance. In addi- tion, SAEBench could be extended to other modalities, such as applying sparse probing to vision or biology models. We invite the community to implement additional metrics in our standardized format. We hope that SAEBench will serve as a valuable resource for researchers developing new SAE architectures and practitioners selecting pre-trained SAES for specific applications.
9

---

SAEBench: A Comprehensive Benchmark for Sparse Autoencoders in Language Model Interpretability
References
Chanin, D., Wilken-Smith, J., Dulka, T., Bhatnagar, H., and Bloom, J. A is for absorption: Studying feature splitting and absorption in sparse autoencoders, 2024b, URL https: https://arxiv.orrgg////// attp//arxiiiaaaaaa//// /ttp://0/////b//20.55000000
Anthropic Interpretability Team, Training sparse autoen- coders.
#training- saes, 2024a. [Accessed Januuay 20, 2025].
Chaudhary, M. and Geiger, A. Evaluating open-source sparse autoencoders on disentangling factual knowledge in gpt-2 small, 2024. URL https: hatp:://axxxvoorgpppp abs Abss2409.04477
Anthropic Interpretability Team. Circuits updates august 2024. Transformer Circuits Thread, 2024b. URL http:://eraansfoorrecccrattttt
august- beggusttupddtt//mdee...eo inn.ttteee/dee...mm....
Cunningham, H., Ewart, A., Riggs, L., Huben, R., and Sharkey, L.. Sparse autoencoders find highly interpretable features in language models, 2023. URL https:// arxiv.org/ abs///a////// /ttp///b./////000000
Ayonrinde, K. Adaptive sparse allocation with mutual
choice & feature choice sparse autoencoders, 2024. URL
https: www.i.crgggga...eee abssssrrg///aa///oooeeoo..ssss.r /2212122112222
Biderman, S., Schoelkopf, H., Anthony, Q., Bradley, H., O'Brien, K., Hallahan, E., Khan, A., Purohit, S., Prashanth, U. S., Raff, E., Skowron, A., Sutawika, L., and van der Wal, O. Pythia: A suite for analyzing large language models across training and scaling, 2023. URL https: wttpaaiwwiogga/////////////rroat abs 2///2.244.00333
Engels, J., Michaud, E. J., Liao, I,, Gurnee, W., and Tegmark, M. Not all language model features are linear, 2024. URL https: http:::/arxiv.org////
2405.14860.
Farrell, E., Lau, Y.-T., and Conmy, Applying sparse autoencoders unlearn knowledge in language models, 2024. URL https://arxiwe 2410.19278.
Bricken, T., Templeton, A., Batson, J., Chen, B., Jermyn, A., Conerly, T., Turner, N., Anil, C., Denison, C., Askell, A., Lasenby, R., Wu, Y., Kravec, S., Schiefer, N., Maxwell, T., Joseph, N., Hatfield-Doddss, Z.,, Tamkin, A., Nguyen, K., McLean, B., Burke, J. E., Hume, T., Carter, S., Henighan, T., and Olah, C. Towards monosemanticity; Decomposing language models with dictionary learning. Transformer Circuits Thread, 2023. https:/ttrnsformer- Bircuits.uub22223/ma http://pppp.mmmmmmmmmmmneeeeeeee feetumennntieeettttssienee.. html.
Gao, L., Dupré la Tour, T., Tillman, H., Goh, G., Troll, R., Radford, A., Sutskever, I., Leike, J., and Wu, J. Scaling and evaluating sparse autoencoders. arXiv preprint arXiv:2406.04099, 2024. URL https:// arxiv.org//bss2200..00
Gemma Team, Riviere, M., Pathak, S., Sessa, P. G., Hardin, C., Bhupatiraju, S., Hussenot, L., Mesnard, T., Shahriari, B., Ramé, A., Ferret, J., Liu, P., Tafti, P., Friesen, A., Casbon, M., Ramos, S., Kumar, R., Lan, C. L., Jerome, S., Tsitsulin, A., Vieillard, N., Stanczyk, P., Girgin, S., Momchev, N., Hoffman, M.. Thakoor, S., Grill, J.-B., Neyshabur, B., Bachem, O., Walton, A., Severyn, A., Par- rish, A., Ahmad, A., Hutchison, A., Abdagic, A., Carl, A., Shen, A., Brock, A., Coenen, A., Laforge, A., Pater- son, A., Bastian, B., Piot, B., Wu, B., Royal, B., Chen, C., Kumar, C., Perry, C., Welty, C., Choquette-Choi
Bussmann, B., Leask, P., and Nanda, N. Batchtopk: A simple improvement for topk-saes, 2024a. URL posts/ a-simple-
http:///oo......nentooooor....m htttr// MoxcyIZNbbAafviic//patttoa improvement for-toppi saes.
Bussmann, B., Leask, P., and Nanda, N. Learning multi- level features with matryoshka saes, December 19 2024b. URL https://ww.aliigneenffoommm
post posts
multi muuta--love---atture---ttt-----e saes. Alignment Forum.
C. A., Sinopalnikov, D., Weinberger, D., Vijaykumar, D., Rogozińska, D., Herbison, D., Bandy, E., Wang, E., Noland, E., Moreira, E., Senter, E.,, Eltyshev. E., Visin, F., Rasskin, G., Wei, G. Cameron, G., Martins, G., Hashemi, H., Klimezak-Plucinske. H., Batra, H., Dhand, H., Nar- dini, I., Mein, J., Zhou, J., Svensson, J., Stanway, J., Chan, J., Zhou, J. P., Carrasqueira, J.. Ijazii. J., Becker. Fernandez, J., van Amersfoort, Gordon, J.. Lipschultz. J., Newlan, J., yeong Ji, J., Mohamed, K., Badola, K., Black, K., Millican, K., McDonell, K., Nguyen, K., Sod- hia, K. Greene, K., Sjoesund, L. L.. Usui, L., Sifre, L., Heuermann, L., Lago, L., McNealus, L., Soares, L. B., Kilpatrick, L., Dixon, L., Martins, L., Reid, M., Singh, M., Iverson, M., Görner, M., Velloso, M., Wirth, M.,
Bussmann, B., Pearce, M., Leask, P. Bloom, I., Sharkey, L., and Nanda, N. Showing sae la- tents are atomic using meta-saes, 2024c. URL http:://www.llignenntfooooou hottss//wAauttDDDotttttssts/stbd latents aeennts-re---ee not poot-aton coot-atomic-unn www.amamsm--innnett..... meta- saes.
Chanin, D.,. Wilken-Smith J.. Dulka, T., Bhatnagar, H., and Bloom, is for absorption: Studying feature splitting and absorption in sparse autoencoders. 2024a. URL https: //axxi wwwaaiivvorrroooooooeeoooooo wwt.i//oor//a////0111155
10

---

SAEBench: A Comprehensive Benchmark for Sparse Autoencoders in Language Model Interpretability
Davidow, M., Miller, M., Rahtz, M., Watson, M., Ris- dal, M., Kazemi, M., Moynihan, M., Zhang, M., Kahng. M., Park, M., Rahman, M., Khatwani, M., Dao, N., Bar- doliwalla, N., Devanathan, N., Dumai, N., Chauhan, N., Wahltinez, O., Botarda, P., Barnes, P., Barham, P., Michel, P., Jin, P., Georgiev, P., Culliton, P., Kuppala, P., Co- manescu, R., Merhej, R., Jana, R., Rokni, R. A., Agarwal, R., Mullins, R., Saadat, S., Carthy, S. M., Cogan, S., Perrin, S., Amold, S. M. R., Krause, S., Dai, S., Garg. S., Sheth, S., Ronstrom, S., Chan, S., Jordan, T., Yu, T., Eccles, T., Hennigan, T., Kocisky, T., Doshi, T., Jain, V., Yadav, V., Meshram, V., Dharmadhikari, V., Barkley, W., Wei, W., Ye, W., Han, W., Kwon, W., Xu, X., Shen, Z., Gong, Z., Wei, Z., Cotruta, V., Kirk, P., Rao, A., Giang, M., Peran, L.. Warkentin, T., Collins, E., Bar- ral, J., Ghahramani, Z.. Hadsell, R., Sculley, D., Banks, J.. Dragan, A., Petrov, S., Vinyals, O., Dean, J., Has- sabis, D., Kavukcuoglu, K., Farabet, C., Buchatskaya, E., Borgeaud, S., Fiedel, N., Joulin, A., Kenealy, K., Dadashi, R., and Andreev, A. Gemma 2: Improving open language models at a practical size, 2024. URL https: /etaa://arreee /tt///rxxvvirrgreeee/ abss abs 240.000118.
aligned sparse autoencoders, 2024a. URL https:
/ //arxiv.org/abs............. /et.orr//a////11.1111111
Marks, S., Karvonen, A., and Mueller, A. dic- https://githu..comm saperarks/dictiionrry..ee
tionary learning.
2024b.
Marks, S., Rager, C., Michaud, E. J., Belinkov, Y., Bau, D., and Mueller, A. Sparse feature circuits: Discover- ing and editing interpretable causal graphs in language models, 2024c. URL https: http::///xxxv./rr.//// 2 2403.19647.
Mudide, A., Engels, J., Michaud, E. J., Tegmark, M. and Schroeder de Witt, C. Efficient dictio- nary learning with switch sparse autoencoders. arXiv preprint arXiv:2410.08201, 2024. URL https:// axxiv.org/ass/2410.00
Nanda, N. Open Source Replication & Commentary on Anthropic's Dictionary Learning Paper, Oct 2023,
URL https: httpss//wwaaaiggneetfoooo. posts//Arrgggcc-cdddv////////eee replication- and comeentantrr anthropic-s.
Gurnee, W., Nanda, N., Pauly, M., Harvey, K., Troitkki,, D., and Bertsimas, D. Finding neurons in a haystack: Case studies with sparse probing, 2023. URL https:
Paulo, G., Mallen, A., Juang, C., and Belrose, N. Auto- matically interpreting millions of features in large lan- guage models, 2024. URL abs abs/2410.13928.
//axxiv.org/aaa //aaxii.org/abb//3355..11
Heap, T., Lawson, T., Farnik, L., and Aitchison, L. Sparse autoencoders can interpret randomly initialized trans- formers, 2025. URL https::/atxiv.org// 2501.17727.
Rajamanoharan, S., Conmy, A., Smith, L., Lieberum, T. Varma, V., Kramár, J., Shah, R., and Nanda, N. Improving dictionary learning with gated sparse autoencoders. arXiv preprint arXiv:2404. 16014, 2024a. URL https:
Huang. J., Wu, Z., Potts, C., Geva, M., and Geiger, A. Ravel: Evaluating interpretability methods disen- tangling language model representations, 2024. URL /
Rajamanoharan, S., Lieberum, T., Sonnerat, N., Conmy, A., Varma, V., Kramár, J., and Nanda, N. Jumping ahead: Improving reconstruction fidelity with jumprelu sparse autoencoders. arXiv preprint arXiv:2407 /4435, 2024b. URL http::://axxvvoooo//// http:////aabb20..1443333
Karvonen, A., Wright, B., Rager, C., Angell, R., Brinkmann, J.. Smith, L., Verdun, C. M., Bau, D., and Marks, S. Measuring progress in dictionary learning for language model interpretability with board game models, 2024. URL httpt:tpp//// httttpps//axxxi.rrgg abs httpaaas.///.// /0t./2400.0011.3
Taggart, G. M. Prolu:
nonlinearity
for sparse autoencoders, 2024.
https:
Lieberum, T., Rajamanoharan, S., Conmy, A., Smith, I.., Sonnerat, N., Varma, V., Kramár, J., Dragan, A., Shah, R., and Nanda, N. Gemma scope: Open sparse autoencoders everywhere all at once on gemma 2, 2024. URL ht https: //arxiv.crg/ass/////..
/ w/w.aallmmmmeettorourorr////// HipurTraakkTTKgoo7///rrola.
nontheartyy-fo--ppaee---tea
Venhoff, C., Calinescu, A., Torr, P., and de Witt, C. S. Sage: Scalable ground truth evaluations for large sparse au- toencoders, 2024. URL httphtts///xxxxv.rgggg// 2 2410.07456.
Makelov, A., Lange, G., and Nanda, N. Towards principled evaluations of sparse autoencoders for interpretability and control, 2024. URL https: httpas////xxvvvvo// abs/ 2405.08366.
Wang. K., Variengien, A., Conmy, A., Shlegeris, B., and Steinhardt, Interpretability in the wild: circuit for indirect object identification in gpt-2 small, 2022. URL setps://arxiv.ort www.vo/rrr.//////e//e/eo /tsp//bs/2222
Marks, L., Paren, A., Krueger, D., and Barez, En hancing neural network interpretability with feature-

---

SAEBench: A Comprehensive Benchmark for Sparse Autoencoders in Language Model Interpretability
A. Computational Requirements
The computational requirements for running SAEBench evaluations were measured on an NVIDIA RTX 3090 GPU using 16K width SAEs trained on the Gemma-2-2B model. Table breaks down the timing for each evaluation type into two components: an initial setup phase and the per-SAE evaluation time. The setup phase includes operations like pre-computing model activations, training probes, other one-time preprocessing steps that can be reused across multiple SAE evaluations. After this setup is complete, each evaluation has its own runtime per SAE tested.
The total evaluation time for a single SAE across all benchmarks is approximately 65 minutes, with an initial setup time of 107 minutes. Note that actual runtimes can vary significantly based on factors like SAE dictionary size, choice base model, and GPU selection.
Evaluation Type
Avg Time per SAE (min)
Setup Time (min)
Absorption
26
33
Core
9
SCR
6
22
TPP
2
Sparse Probing
3
15
Automated Interpretability
9
Unlearning
10
33
RAVEL
45
45
Total
110
152
Table 2: Timing results for evaluations, rounded to the nearest minute.
B. SAE Training Details
For each [layer, width, type] combination, we target 6 Lo values: [20, 40, 80, 160, 320, 640). For SAEs trained with sparsity penalty, without the ability to explicitly set desired Lo (all except TopK variants), we may not exactly hit the targeted Lo values. All other variables are kept fixed to enable direct comparisons. All SAEs are trained in a directly comparable manner, including identical data and data ordering.
When training, we first estimate a scalar constant to normalize the activations to have unit mean squared norm during training, increasing hyperparameter transfer between layers and models. We fold this constant into the weights after training so our SAEs don't require normalized activations.
We initialize the decoder to the transpose of the encoder, but do not tie them during training. We found that the transpose initialization was important for avoiding dead latents which do not activate during training.
Following (Nanda, 2023), we randomly sample from a buffer of 250,000 activations and replenish the buffer when half empty.
Hyperparameter
Value
Tokens processed
500M
Learning rate
3 x 10
Learning rate warmup (from 0)
1,000 steps
Sparsity penalty warmup (from 0)
5,000 steps
Learning rate decay (to 0)
Last 20% of training
Dataset
The Pile
Batch size
2,048
LLM context length
1,024
Table 3: SAE training hyperparameters.
12

---

SAEBench: A Comprehensive Benchmark for Sparse Autoencoders in Language Model Interpretability
C. Extended Related Work
C.1. SAE Benchmarks
common approach to evaluating SAE features has been automated interpretability, where language models are used to judge feature interpretability (Paulo et al., 2024; Rajamanoharan et al., 2024a). In this evaluation, an LLM generates natural language description of an SAE feature based on input sentences that activate the feature most. A simulator model uses the generated description predict feature activations held-out data. However, this evaluation method has struggled to provide statistically significant differences between various SAE architectures and approaches, such in Rajamanoharan et al. (2024a). The Anthropic Interpretability Team (2024b) found most SAE variants perform comparably, while outperforming the standard ReLU SAE.
Gao et al. (2024) propose using the Top-K activation function to SAES and evaluate their approach on four metrics: Downstream loss measures the difference in Kullback-Leibler (KL) divergence and cross-entropy (CE) loss of model predictions after replacing activations with their SAE reconstruction during forward-pass. Sparse probing quantifies the correlation of single SAE latents with labeled concepts in natural language. Neuron-to-Grapp generates explanations for SAE latent activations by identifying n-gram patterns. Precision and recall of these explanations are evaluated on an held-out test. Ablation sparisity is an unsupervised characteristic for the downstream effects on output logits when ablating individual latents.
Karvonen et al. (2024) evaluate SAE architectures on board game models, leveraging clear ground truth features which exist board games. However, their metries cannot be applied to language models. Makelov al. (2024) evaluate SAEs on discovering relevant representations of the indirec--ohjec-idenffiee (2022). They leverage knowledge about the IOI circuit create supervised dictionaries, which they use for comparison with unsupervised dictionaries. However, this requires task specific knowledge to create the supervised dictionaries, limiting its scalability.
(IOI) mechanism GPT-2-small Wang et al.
Recent work by Venhoff et al. (2024) introduced SAGE, framework for automated discovery of task-relevant model components. SAGE computes supervised feature dictionaries that serve as approximate ground truth of model internal features. The method compares SAEs with supervised dictionaries across layers using projection- reconstruction technique.
13

---

SAEBench: A Comprehensive Benchmark for Sparse Autoencoders in Language Model Interpretability
D. Further Evaluation Details
In evaluations, we mask off the BOS, EOS, and PAD tokens because some existing SAEs do not train on these special tokens, such Lieberum et (2024).
PCA Baseline Implementation
We include a PCA baseline in all charts, implemented as follows: the PCA is fit on 200M model activations, treating all PCA components as SAE latents. The PCA encoder the PCA transformation matrix, and the decoder is its transpose. The mean activation value is used as a bias term. Due this implementation, PCA achieves perfect reconstruction but exhibits very high LO sparsity, approximately equal to the model's s hidden dimension.
Core Evaluation Metries
Core metrics include LO sparsity and Loss Recovered described in Section 3.2.1. Our core implementation also provides additional convenient metrics, such Relative Reconstruction Bias Rajamanoharan al. (2024a), KI. divergence, maximum cosine similarity between latents, and the percentage of high-frequency latents.
Parameter
Value
Dataset
Open Wex
Context length
128 tokens
Loss Recovered samples
3,200 sequences
Sparsity evaluation samples
32,000 sequences
Table Core metrics evaluation hyperparameters.
LLM Scoring/ Automated Interpretability
In automated interpretability evaluation, use gpt4o-mini as an LLM judge to quantify the interpretability of SAE latents at scale, in line with Bills et al. Our implementation is similar to the detection score proposed by Paulo et al. (2024). The evaluation consists of two phases: generation and scoring.
In the generation phase, we obtain SAE activation values on webtext sequences. We select sequences with the highest activation values (top-k) and sample additional sequences with probability proportional to their activation values. These sequences are formatted by highlighting activating tokens with <<token>> syntax and used prompt an LLM to generate explanations for each feature based on these formatted sequences.
The scoring phase begins by creating test set for each latent, which contains top activation sequences, inportance-weighted sequences, and random sequences from the remaining distribution, specifically:
10 Randomly Sampled Sequences
Max Activating Sequences
2 Importance Weighted Sequences
Given feature explanation and the shuffled test set of unlabeled sequences, another LLM judge predicts which sequences would activate the feature. The automatic interpretability score reflects the accuracy of predicted activations. Paulo et al. (2024) found that LLM judgements correlated with human judgements in this setting.
Sparse Probing
We evaluate our sparse autoencoders' ability to learn specified concepts through series of targeted probing tasks across diverse domains, including language identification, profession classification, and sentiment analysis. We base our methodol- ogy on that used by Gurnee et al. (2023). For each dataset class, we structure the task as a one-versus-all binary classification task. For each task, we encode imputs through the SAE, apply mean pooling over non-padding tokens, and select the top- K latents using maximum mean difference. We chose the maximum mean difference method for feature identification as
14

---

SAEBench: A Comprehensive Benchmark for Sparse Autoencoders in Language Model Interpretability
Parameter
Value
Sample size
1,000 non-dead latents
Dataset
The Pile
Activation dataset size
2M tokens
Context length
128 tokens
LLM judge
gpt4o-mini
Table 5: Automated interpretability evaluation hyperparameters.
Gurnee et al. (2023) found performs comparably to more complex sparse probing methods while heing both efficient and simple. We train logistic regression probe on the resulting representations and evaluate classification performance on held-out test data. Our evaluation spans 35 distinct binary classification tasks derived from five datasets.
Our probing evaluation encompasses five datasets spanning different domains and tasks:
Dataset
Task Type
Description
bias_in_bios
Profession Classification
Predicting professional roles from biographical text
Amazon Reviews
Product Classification and Sentiment
Dual tasks: category prediction and sentiment analysis Detecting document language
Buroparl
Language Identification
Hub
Programming Language Classification
Identifying coding language from source code
AG News
Topic Categorization
Classifying news articles by subject
Table 6: Datasets used in probing evaluation and their corresponding tasks.
To ensure consistent computational requirements across tasks, we sample 4,000 training and 1,000 test examples per binary classification task and truncate all inputs to 128 tokens. For Gi Hub data, we follow Gurnee et al. (2023) by excluding the first 150 characters (approximately 50 tokens) as a crude attempt to avoid license headers. We evaluated both mean pooling and max pooling across non-pade con-padingg tokens and used mean pooling it obtained slightly higher accuracy. From each dataset, we select subsets containing up to five classes. Multiple subsets may be drawn from the same dataset to maintain positive ratios 0.2.
RAVEL
The RAVEL evaluation assesses the extent to which SAEs achieve clear feature disentanglement, where independent concepts encoded in distinct latents without unintended overlaps. We follow the methodology introduced by Huang et al. (2024), which specifically tests how targeted latent interventions can alter model predictions about particular attributes of entities (e.g., convincing the model that Paris is in Japan) while preserving other attributes (such as the language spoken in Paris remaining French).
In detail, the evaluation process involves selecting specific entities (e.g., cities, Nobel laureates) and their attributes (e.g.. country, language, continent) from the RAVEL dataset. We generate completions for all prompts in the RAVEL dataset describing these entities and select the top 500 entities and 90 templates based prediction accuracy. To identify the latents relevant to each attribute, follow Chaudhary & Geiger (2024) and use a Multitask Differentiable Binary Mask (MDBM) to select the latents, simultaneously optimizing for both the Cause and Isolation metrics. Note that this is unlike Huang et al. (2024), who selected latents using a linear probe. The MDBM is trained on 7,000 examples (equally split between cause and isolation) for two epochs with learning rate x 10-3 and evaluated on a separate set of 3,000 test examples.
During intervention, we transfer encoded latent values at a single token: if an entity spans multiple tokens, the latent intervention is applied specifically to the final token, following Huang et al. (2024). As performance skyline, we also implement a jointly- -rained Multitask Distributed Alignment Search (MDAS) and MDBM, achieving disentanglement score of 0.87. Due to the increased number of trainable parameters, the MDAS/MDBM intervention was trained for 10 epochs instead of 2, with convergence determined through manual inspection.
In certain evaluations (e.g., SCR, TPP), we typically compute the SAE reconstruction error, perform the latent intervention on the SAE, and then add the original reconstruction error term back to the modified reconstruction. This approach ensures that any changes remain confined to the targeted SAE latents, preventing unintended effects from a potentially large
15

---

SAEBench: A Comprehensive Benchmark for Sparse Autoencoders in Language Model Interpretability
and destabilizing error term. However, for the RAVEL evaluation specifically, we observed that incorporating this error term negatively impacted both the Cause and Disentangle scores, frequently reducing Disentangle by approximately 0.02.
Therefore, we omit the error term in our RAVEL evaluations.
For the evaluation, we selected two entity types from the RAVEL dataset: cities with attributes Country, Continent, and Language, and Nobel Prize winners with attributes Country of Birth, Field, and Gender.
Parameter
Value
Top entities selected
500
Top templates selected
90
MDBM training samples
7,000 (50% cause, 50% isolation)
MDBM test samples
3,000
MDBM Start Temperature
1
MDBM End Temperature
1 x 10-4
MDBM epochs
2
MDAS epochs
10
MDBM learning rate
1 x 10
Skyline disentangle score
0.87
Table 7: RAVEL evaluation hyperparameters.
Feature Absorption
In general, feature absorption is incentivized any time there's ' a pair of concepts, A and B, where A implies B (i.e., if A activates, then B will always also be active, but not necessarily the other way around). This will happen with categories or hierarchies, e.g., India => Asia, pig = mammal, red = color, etc. If the SAE learns a latent for A and latent for B, then both will fire on inputs with A. But this is redundant- implies B, so there's 's no need for the B latent light up on A. If the model learns a latent for A and latent for "B except A," then only one activates. This is sparser, but clearly less interpretable!
Feature absorption often happens in an unpredictable manner, resulting in unusual gerrymandered latents. For example, the "starts with S" feature may fire on 95% of tokens beginning with S, yet fail to fire on an arbitrary 5% as the "starts with S feature has been absorbed for this 5% of tokens. This is an undesirable property that we would like to minimize.
We build on the metric proposed in Chanin al. (2024b), which focuses on examining how SAE latents represent first-letter classification tasks, measuring cases where the main latents for a letter fail to fully capture the feature while other latents compensate. Our implementation extends this approach with a more flexible measurement technique that enables evaluation across all model layers. Chanin et al. (2024b) included the ablation effect of absorbing latents on the model's performance on the spelling task as part of the metric, but this limits the use of the metric to early and middle layers. Ablation effect always goes to zero in later layers after the relevant information gets moved to the final token position during the model forward pass, thus diminishing the causal impact of any of the latents at the source token. We therefore adopt an alternate approach to detecting absorbing latents based on the latent contributing significant portion of the first-letter probe direction to the residual stream, where the first-letter direction is defined by a logistic-regression ground truth probe as described below.
We observed that absorption patterns often involve partial absorption, where the absorbed latent isn't 't completely suppressed, but still activates weakly, with other absorbing latent(s) compensating for their reduced activations. We also observed that the responsibility for compensating for the reduced activation of the main latents for a given feature is often shared among several absorbing latents. In some SAEs most cases of absorption involved either or both of these patterns. This motivated the approach described below where we allow flexible number of latents to be classified as absorbing and measure the amount of absorption fraction capturing the proportion of the SAE's representation of the feature which is accounted for by absorbing latents rather than the main latents for that feature.
Our approach works as follows: first, tokens consisting of only English letters and an optional leading space are split into a train and test set, and a supervised logistic regression probe for each starting letter is trained on the train set using residual stream activations from the model. These probes are used as ground truths for the feature directions the model. Next, for
16

---

SAEBench: A Comprehensive Benchmark for Sparse Autoencoders in Language Model Interpretability
each starting letter, k-sparse probing is performed on SAE latents from the train set find which latents for the task. The feature splitting, as k is increased from k increase of more than over the F1 of the k = n probe, the h + 1 feature is considered feature split and is added to the set of main SAE latents performing the first letter task. We use 7h = 0.03 following Chanin et al.
relevant main SAE latent for given first letter task. To account for
sparse probing latent is considered to k + 1, if the Fl score for the A: = + 1 sparse probe represents an
Once the main feature split latents for each first letter have been identified, we evaluate their behavior on the test set for cases of absorption based on the projections of the residual stream activations and the SAE latent activations onto the ground truth probe direction for the first letter question. Specifically, we detect absorption occur on test set inputs where:
1. The ground truth probe correctly classifies the first letter of the token in question (if the ground truth probe cannot detect the feature, it would be unfair to expect the main feature split latents to do so).
2. The sum of the projections of the main feature split latent activations onto the probe direction is less than the projection of the residual stream activation onto the probe direction (conceptually, the main feature split latents don't 't account for all of the presence of the feature they are trying to detect in the residual stream activation):
Σ REE Sma
a.d;
amddll
3. The sum of the top Amax ground truth probe projections from other latents accounts for at least a proportion Tpu of the projection of the residual stream activation onto the probe direction. Note we only consider other latents as potential absorbing latents they have cosine similarity with the ground truth probe 2 Tps and positive ground truth probe projection. (conceptually, there are other related latents which significantly compensate for the reduced activation of the main latents):
Eiesp, a
2 Tpa
amodel P
For inputs that satisfy the above criteria, the absorption score is defined as:
Eieso, a;d, P Eiesm a,d; p + Eiesmo a;d; p
Absorption Score
Where:
S is the set of all SAE latents.
Smain is the set of the main feature split latents.
Sats is the set of the absorbing latents. Contains up to the top Amax potential absorbing latents with the highest ground truth probe projections.
0; is the activation magnitude of latent i.
d is the unit decoder direction for latent i.
p is the unit ground truth probe direction.
&model is the activation vector from the model, in our case the residual stream activation.
Conceptually, this captures the proportion of the SAE's representation of the feature which is accounted for by absorbing latents rather than the main latents for that feature
For all other inputs, the absorption score is O. The total absorption score is then the mean absorption score across all test inputs correctly classified by the probe. The dataset is the model vocabulary, filtered for tokens containing only English letters and an optional leading space.
For consistency with our other evaluation metrics where higher values are more desirable, we present our results using the complement of the absorption score absorption score).
17

---

SAEBench: A Comprehensive Benchmark for Sparse Autoencoders in Language Model Interpretability
Parameter
Value
Train/test split
80/20
k-sparse probing max k
10
0.03
Tp
0
Tps
1
Amax
SAE dictionary size
Table 8: Absorption evaluation hyperparameters.
Unlearning
We evaluate SAES on their ability to selectively remove knowledge while maintaining model performance on unrelated tasks, following the methodology in Farrell et al. (2024). While there are several existing unlearning datasets, we found that Gemma-2-2B's performance was relatively poor on most test sets. With larger models, this evaluation could be expanded to leverage greater diversity of existing unlearning datasets. Evaluation parameters are detailed Table
This SAE unlearning evaluation uses the WMDP-bio dataset, which contains multiple-choice questions involving dangerous biology knowledge. The intervention methodology involves clamping selected SAE feature activations to negative values whenever the latents activate during inference. Feature selection utilizes dual-dataset approach: calculating feature sparsity across a "forget" dataset (WMDP-bio corpus) and a "retain" dataset (WikiText). The selection and intervention process involves three key hyperparameters:
1. retain.threshold maximum allowable sparsity on the retain set,
2. n.features- number of top latents to select, and
3. multiplier magnitude of negative clamping.
The procedure first discards latents with retain set sparsity above
then selects the top n features by forget set sparsity, and finally clamps their activations to negative multipliex when activated.
We quantify unlearning effectiveness through two metries:
Accuracy on WMDP-bio questions, and
2. Accuracy on unrelated MMLU subsets, including high school US history, geography, college computer science, and human aging.
Both metrics only evaluate on questions that the base model answers correctly across all option permutations, to reduce noise from uncertain model knowledge. Lower WMDP-bio accuracy indicates successful unlearning, while higher MMLU accuracy demonstrates preserved general capabilities.
We sweep the three hyperparameters to obtain multiple evaluation results per SAE, derive single evaluation metric, we filter for results maintaining MMLU accuracy above 0.99 and select the minimum achieved WMDP-bio accuracy, thereby measuring optimal unlearning performance within acceptable side effect constraints.
Spurious Correlation Removal (SCR)
In the SHIFT method, human evaluator debiases a classifier by ablating SAE latents. We automate SHIFT and use to measure whether SAE has found separate latents for distinct concepts- for example, the concept of gender and the concepts related to someone's 's profession. Distinct latents enable more precise removal of spurious correlations, thereby effectively debiasing the classifier.
First, we filter datasets (Bias Bios and Amazon Reviews) for two binary labels. For example, we select text samples of two professions (professor, murse) and the gender labels (male, female) from the Bias in Bios dataset. We partition this dataset into:
18

---

SAEBench: A Comprehensive Benchmark for Sparse Autoencoders in Language Model Interpretability
Parameter
Value
MMLU Subsets
High School US history, College Computer Science,High School Beography.Humaare Aging
Retain thresholds
[0.001, 0.01]
Number of latents
[10, 20]
Negative multipliers
[25, 50, 100, 200]
Retain Forget Dataset size
1,024 sequences
Retain / Forget Sequence length
1,024 tokens
Table 9: Unlearning evaluation hyperparameters.
A balanced set containgg all combinations of professorinurse and malelfemale, and
A biased set conttinign only male+professor and female+nurse combinations.
We then train a linear classifier C. on the biased dataset. The linear classifier picks up both signals, such as gender and profession. During the evaluation, we attempt to debias the classifier C2 by selecting SAE latents related to one class (e.g., gender) to increase classification accuracy for the other class (e.g., profession).
select the set containing the top n SAE latents according to their absolute probe attribution score with a probe trained specifically to predict the spurious signal (e.g.. gender). We found that latents automatically identified through probe attribution interpretable and align well with the target concepts, validated by both human evaluation and automated LLM judgment. Thus, we select latents using probe attribution avoid the cost and potential biases associated with an LLM judge.
For each original and spurious-feauur--nffreee set I. of selected latents, we remove the spurious signal by defining a modified classifier:
C - C.
where all selected unrelated yet highly attributive latents zero-ablated. The accuracy with which the modified classifier O'm predicts the desired class when evaluated on the balanced dataset indicates SAE quality. A higher accuracy suggests that the SAE was more effective in isolating and removing the spurious correlation (e.g., gender), allowing the classifier to focus on the intended task (e.g., profession classification).
We consider a normalized evaluation score:
Aabi
Abase
SSIFFT
Aoracle
Abase
where:
Aabi is the probe accuracy after ablation,
Abaso is the baseline accuracy (spurious probe before ablation), and
Amuete is the skyline accuracy (probe trained directly on the desired concept).
This score represents the proportion of improvement achieved through ablation relative to the maximum possible improve- ment, allowing fair comparison across different classes and models.
Targeted Probe Perturbation (TPP)
SHIFT requires datasets with correlated labels. We generalize SHIFT all multiclass NLP datasets by introducing the Targeted Probe Perturbation (TPP) metric. At high level, we aim to find sets of SAE latents that disentangle the dataset classes. Inspired by SHIFT, we train probes on the model activations and measure the effect of ablating sets of latents on the probe accuracy. Ablating disentangled set of latents should only have an isolated causal effect on one class probe, while leaving other class probes unaffected. A full write-up up is located in "Evaluating Sparse Autoencoders on Targeted Concept Erasure Tosks".
19

---

SAEBench: A Comprehensive Benchmark for Sparse Autoencoders in Language Model Interpretability
We consider dataset mapping text exactly one of m concepts c C. For each class with index 1,... ...... m. select the set I of the most relevant SAE latents by training linear probe. Note that select the top signed importance scores, as we only interested in latents that actively contribute to the targeted class.
For each concept C₁, we partition the dataset into samples of the targeted concept and random mix of all other labels.
We define the model with a probe corresponding to concept for j = 1,..., m as a linear classifier Cj. which is able to classify concept Cj with accuracy Aj. Further, Ctg denotes a classifier for ej where latents L; are ablated. Then, we iteratively evaluate the accuracy A of all linear classifiers Ct, on the dataset partitioned for the corresponding class cj.
The Targeted Probe Perturbation (TPP) score is defined as:
Stre mean;=; (At. Aj) mean; (Ai.) AA)
This score represents the effectiveness of causally isolating single probe. Ablating disentangled set of latents should only show significant accuracy decrease if i j. namely if the latents selected for class are ablated in the classifier of the same class i, and remain constant if i + j.
Parameter
Value
Train set size
4,000 sequences
Test set size
1,000 sequences
Context length
128 tokens
Probe train batch size
16
Probe epochs
20
Probe learning rate
lc-3
Probe L1 penalty
le-3
Table 10: SCR and TPP shared evaluation hyperparameters.
For SCR evaluation, we create perfectly biased datasets where the spurious correlation is gender for Bias in Bios and sentiment for Amazon Reviews.
SCR Class Pairs:
Bias Bios:
Professor Nurse
Architect/ Journalist
Surgeon / Psychologist
Attorney Teacher
Amazon Reviews:
Books / CDs and Vinyl
Software/ / Electronics
Pet Supplies Office Products
Industrial and Scientific/ Toys and Games
TPP Classes:
Bias in Bios:
Accountant
Architect
Attorney
Dentist
20

---

SAEBench: A Comprehensive Benchmark for Sparse Autoencoders in Language Model Interpretability
Filmmaker
Amazon Reviews:
Toys and Games
Cell Phones and Accessories
Industrial and Scientific
-Musical Instruments
Electronics
21

---

SAEBench: A Comprehensive Benchmark for Sparse Autoencoders in Language Model Interpretability
E. Baseline Comparison: SAEs on Randomly Initialized vs. Fully Trained Models
We additionally investigated sparse autoencoders (SAEs) trained on a randomly initialized language model as a baseline, inspired by recent work involving auto-interp (Heap et al., 2025). Specifically, we trained two sets of 16k-width TopK SAEs on layer 10 of the Pythia- IB model for 100 million tokens each: one set trained on the fully trained model and another on the randomly initialized (Step 0) model. We compared these two sets using several SAEBench metrics, specifically focusing on KL divergence score, sparse probing, automated interpretability (auto-interp), Spurious Correlation Removal (SCR), and Targeted Probe Perturbation (TPP).
Limitations and Contextual Considerations: When interpreting these results, several limitations should be acknowledged. The SAEBench metrics employed in this analysis were originally designed for evaluating various SAEs trained on the same underlying model, rather than comparing SAEs across fundamentally different models, such as fully trained versus randomly initialized. Consequently, metrics dependent on the inherent capabilities of trained model suchhsschhee as unlearning, which requires the model to perform above random chance on benchmarks like MMLU, or feature absorption, which relies on accurately predicting specific token-level features
not applicable randomly initialized models.
Additionally, comparability challenges arise due to inherent differences in predictive behaviors between models. For example, Targeted Probe Perturbation (TPP) and SCR measure relative changes in linear probe accuracy within a given model. However, linear probes trained on randomly initialized models typically yield substantially lower baseline accuracies compared to fully trained models, making cross-model comparisons potentially misleading. For sparse probing, probing the residual stream of the trained model gives 94% accuracy vs 84% with the random model.
We observed several noteworthy patterns:
SAEs trained on the fully trained model significantly outperformed those trained on the randomly initialized model on most metrics, especially on sparse probing and SCR. The SAEs trained the random model exhibited performance close to or worse than directly probing residual stream values, indicating their inability to capture meaningful abstract features.
Supervised Metrics:
Automated Interpretability: SAFs trained on the fully trained model achieved slightly higher auto-interp scores compared to those trained on the random model. However, both sets of SAEs substantially outperformed a baseline involving directly reading residual stream activations, a notably weak baseline. We note that our auto-interp evaluation employs detection- based approach, distinct from the fuzzing method utilized by Heap al. (2025), which may partly explain differences in results.
As shown in Figure 5, SAEs consistently and significantly outperformed all other baselines we tested, including MLP neurons, PCA principal components, and direct residual stream readings.
KL Divergence: Since cross-entropy loss recovery metrics are not meaningful for randomly initialized models, we instead evaluated a normalized KL divergence score defined as follows:
KL Divergence Score DKL ( (Pabiated Purig) DKL Parig)
DKL (Paaaaaaacdd
|| (Poni)
where Ponig represents logits from the original model, Pablated represents logits after zero-ablation of activations, and PSAE represents logits after reconstructing activations with the SAE. SABS trained on the fully trained model achieved significantly higher KI. divergence scores (indicating superior reconstruction quality), despite having higher absolute KI. divergence values. This outcome is expected because randomly initialized models produce essentially random predictions, offering limited opportunity for meaningful reconstruction improvement.
Targeted Probe Perturbation (TPP): SAEs trained on randomly initialized models achieved higher TPP scores than those trained on the fully trained Pythia- model. However, we caution against interpreting this result as indicative of poor evaluation quality. The TPP metric measures relative changes in probe accuracy within given model, not across distinct models. Given that the linear probes on the randomly initialized model start from substantially lower baseline accuracy, direct comparisons of TPP across models may be misleading.
22

---

SAEBench: A Comprehensive Benchmark for Sparse Autoencoders in Language Model Interpretability
Sparse Probing Top Test Accuracy
1.000
0.7.0
KI Divergence Score
0.975
0.85
Autointerr Score
0.725
0.950
0.700
0.675
0.900
0.650
0.875
0.625
0.70
0.600
80
160
20
160
20
40
160
LO (Sparsity)
LD (Sparsity)
L0 (Sparsity)
0.4
0.15
TRP 20 Metric
SCR Top 20 Metric
0.3
0.10
Pythia-18 (Trained)
0.2
Pythia-18 StepO (Random)
0.05
Residual Stream Baseline
0.1
0.00
0.0
160
L0 (Sparsity)
LO (Sparsity)
Figure 4: Evaluation results for SAEs trained on the randomly initialized and final versions of Pythia- IB.
Autointerp Scores by Method Layer
0.90
GemmaScope Residual
GemmaScope MLP
Identity Residual Dimensions
0.85
Identity Neurons
0.80
Autointerp Score
0.70
0.65
Laver
Figure 5: Auto-interp scores for the canonical GemmaScope SAEs when compared ML. neuron, PCA, and residual stream baselines. SAES significantly more interpretable than the baselines.
23

---

SAEBench: A Comprehensive Benchmark for Sparse Autoencoders in Language Model Interpretability
F. Additional Dictionary Scaling Analysis
Figure in the main text aggregates scaling results over LO values 40-200 to highlight overall trends. Figure decomposes the scaling behavior from 16k to 65k width across all sparsities, the cost of obscuring direct architectural comparisons.
0.015
0.06
Loss Recovered
Autointerp Score
0.010
0.005
0.02
0.000
0.00
10a
10a
10a
10a
Lo (Sparsity)
L0 (Sparsity)
Mean Absorption fraction Score
0.05
0.0
SCR Top 20 Metric
0.00
-0.1
-0.05
-0.10
10a (Sparsity)
10a
(Sparsity)
TopK Batch Topk
ReLU with p-Annealing
Gated
JumpReLu
Matryoshka Batch TopK
Figure 6: Detailed scaling analysis showing the change in metric scores when increasing SAE width from 16k 65k Gemma-2-2B. Unlike the averaged results in Figure 3, this shows how scaling effects vary across different sparsity levels (L. values).
0.10
SCR Top max Metric
0.05
ReLU
JumpReLU
0.00
TopK
Batch TopK
-0.05
ReLU with p-Annealing
88
Matryoshka Batch TopK
-0.10
Gated
-0.15
1
1
LO (Sparsity)
Figure 7: Change SCR score when scaling from 16k to 65k width, evaluated across intervention budgets k=[5, 10, 20, 50, 100, 500]. ReLU and Matryoshka show improved performance with scale, despite having dramatically different absolute scores (ReLU being lowest and Matryoshka highest in Figure 2).
24

---

SAEBench: A Comprehensive Benchmark for Sparse Autoencoders in Language Model Interpretability
G. Training Dynamics
Spanso Probingg Top TRat Accurac
0.80
Nean Absorption
0.75
0.1
Average Standand Top
Average
Topk
10'
10a
20
160
Taining
Traning Steps
Score
TBP Tap 20 Medic
0.75
Average
0.70
Standard
Tapp
166
Trininn
Recovess
0.98
Untearnn ning Score
0.20
0.95
0.92
Aver
Standard
0.05
Auerage
Standard
Topx
0.90
30
16
0.00
10
10
166
Steps
RAVEL Score
70
0.2
Averass rarrage Standand kipk
Top
10a
126
Thining
Trining Steps
Figure 8: Scores for all 7 metrics on our 16K Gemma G-mma-2-2B TopK and ReLU SAES as we evaluate checkpoints throughout training.
25

---

SAEBench: A Comprehensive Benchmark for Sparse Autoencoders in Language Model Interpretability
H. Intervention Set Size Analysis
Several of our metrics involve hyperparameter K that determines how many latents we analyze or intervene upon:
SCR and TPP use latents for zero ablation, evaluated [5, 10, 20, 50, 100, 500] Sparse probing selects K latents probe, evaluated K = [1, 2, 5]
For SCR and TPP, selecting K involves trade-off: we want enough latents capture complete concepts, but few enough to enable meaningful human analysis. We chose K = 20 for our main results as practical size for manual inspection. Our analysis shows that relative performance differences between architectures remain consistent for K from to 50, though these patterns break down at K 500 (which exceeds reasonable limits for human analysis). For sparse probing, we follow results in the main text. However, our extended analysis reveals that SAEs' advantage over the PCA baseline grows substantially K increases from 5, suggesting that SAEs may be partieularly effective capturing concepts that require multiple latents. All results shown below use the 65k width Gemma-2-22 2-2B SAE suite:
Gao et al. (2024) in presenting
26

---

SAEBench: A Comprehensive Benchmark for Sparse Autoencoders in Language Model Interpretability
0.30
0.30
Top Metric
SCR Top 10 Metric
0.25
0.20
0.20
0.15
0.15
0.10
0.10
0.05
0.05
0.00
10"
10
10a
10a
(Sparsity)
(Sparsity)
0.40
0.35
0.40
SCR Top Metric
0.30
SCR Top 50 Metric
0.35
0.30
0.20
0.25
0.15
0.20
0.10
0.15
0.05
0.10
10a
10a
102
10a
Lo (Sparsity)
LO (Sparsity)
0.45
0.40
0.40
SCR Top Metric
0.35
SCR Top 500 Metric
0.35
0.25
0.25
0.20
0.20
0.15
0.15
0.10
10a
10a
10a
10a
LO (Sparsity)
LO (Sparsity)
ReLU
JumpReLU
TopK
Batch TopK
ReLU with p-Annealing
Matryoshka Batch TopK
Gated
PCA Baseline
Figure 9: Impact of intervention set size K on SCR scores across different architectures. Results show how changing the number of ablated latents affects the ability remove spurious correlations.
27

---

SAEBench: A Comprehensive Benchmark for Sparse Autoencoders in Language Model Interpretability
0.25
0.10
Top Metric
0.08
TPP Top 10 Metric
0.20
0.06
0.15
0.04
0.10
0.02
0.00
0.00
102
10a
10a
10a
Lo (Sparsity)
LO (Sparsity)
0.35
0.35
0.30
0.30
TPP 20 Metric
TPP Top 50 Metric
0.25
0.25
0.20
0.20
0.15
0.15
0.10
0.10
0.05
0.05
0.00
0.00
102
10a
10a
10a
LO (Sparsity)
LO (Sparsity)
0.40
0.40
0.35
Top Metric
0.30
TPP Top 500 Metric
0.35
0.25
0.30
0.20
0.25
0.10
0.15
0.05
0.10
102
10a
102
10a
LO (Sparsity)
LO (Sparsity)
ReLU
JumpReLU
TopK
Batch TopK
ReLU with p-Annealing
Matryoshka Batch TopK
Gated
PCA Baseline
Figure 10: TPP scores across different intervention set sizes K, showing how the number of ablated latents affects concept isolation.
28

---

SAEBench: A Comprehensive Benchmark for Sparse Autoencoders in Language Model Interpretability
0.86
Probing Top 1 Test Accuracy
0.80
Probing Top 2 Test Accuracy
0.84
0.78
0.82
0.76
0.80
0.74
0.78
0.72
0.76
0.70
10
10
10
10a
LO (Sparsity)
LO (Sparsity)
0.90
Probing Top 5 Test Accuracy
ReLU
JumpReLU
0.88
TopK
Batch TopK
0.86
ReLU with p-Annealing
Matryoshka Batch TopK
0.84
Gated
0.82
PCA Baseline
102
10
LO (Sparsity)
Figure 11: Sparse probing performance with different numbers of probed latents K, demonstrating improved concept detection with additional latents.
29

---

SAEBench: A Comprehensive Benchmark for Sparse Autoencoders in Language Model Interpretability
I. Gemma-Scope Evaluation Results
evaluate the Gemma-Scope SAE series introduced by Lieberum et al. (2024), which provides a unique opportunity to study SAE behavior at large scales. While Gemma-Scopp -aopee includes SAEs trained on all layers, focus on their "Width Series" subset of layers where SAEs were trained with dictionary sizes ranging from 16k 1M latents:
Gemma-2-2BB Width series trained on layers [5, 12, 19]
Gemma-2-9Hh Width series trained on layers [9, 20, 31]
Our evaluation reveals substantial variation in SAE performance across different layers of the same model. This layer- dependent behavior is particularly pronounced in the unlearning metric, where SAEs trained on the final evaluated layer (layer 19 for Gemma-2-2B and layer 31 for Gemma-2-9B) consistently achieve scores near zero, regardless of width. This is consistent with the findings of Farrell et al. (2024).
We see several key scaling trends that hold across both model sizes:
1. Loss Recovered and AutoInterp improves consistently with increased width
2. Feature Absorption, SCR, and TPP scores degrade at larger widths
3. Unlearning effectiveness if best at earlier layers and varies significantly by layer
4. Sparse Probing scores increase at later layers
These scaling patterns largely align with our findings from the main architecture comparison, suggesting that the trade-offs we identified between reconstruction fidelity and feature disentanglement persist even larger scales.
30

---

SAEBench: A Comprehensive Benchmark for Sparse Autoencoders in Language Model Interpretability
1.0000
0.90
0,9975
Loss Recovered
Autointerp Score
0.85
0,9950
0.9925
0.80
0.9900
0.9875
0.70
0,9850
10a
10a
10a
10a
(Sparsity)
LO (Sparsity)
1.0
Sparse Probing Top Test Test Accuracy
Mean Absorption Score
0.76
0.74
0.8
0.72
0.70
0.7
0.68
10a
10a
10a
Lo (Sparsity)
LO (Sparsity)
0.16
0.030
0.14
SCR Top 10 Metric
0.12
Top 10 Metric
0.025
0.10
0.020
0.015
0.010
0.005
0.02
10a
10a
10a
Lo (Sparsity)
L0 (Sparsity)
0.4
Unleaming Score
0.3
16k Width
65k Width
0.2
1M Width
PCA Baseline
0.0
(Sparsity)
Figure 12: Performance of Gemma-Scope SAEs trained on layer of Gemma-2-2R across four different widths (16k to 1M latents). Results show all eight benchmark metries: Core metrics (Loss Recovered), Concept Detection (Sparse Probing, Feature Absorption), Interpretability (LLM automated interpretability). and Feature Disentanglement (Unlearning, SCR, TPP). The x-axis shows L0 sparsity values, while the y-axis represents metric scores. Each line color represents a different SAE width, revealing how performance scales with dictionary size at fixed sparsity levels.
31

---

SAEBench: A Comprehensive Benchmark for Sparse Autoencoders in Language Model Interpretability
1.000
0.85
0.995
Loss Recovered
Autointerp Score
0.80
0.985
0.75
0.980
0.70
0.975
102
10a
Lo (Sparsity)
LO (Sparsity)
Sparse Probing Test Test Accuracy
0.76
Mean Absorption Score
0.8
0.74
0.72
0.6
0.70
0.68
0.4
0.66
0.2
10a
10a Lo (Sparsity)
Lo (Sparsity)
0.30
SCR Top 10 Metric
TPP Top 10 Metric
0.06
0.20
0.15
0.04
0.10
0.02
0.00
10a (Sparsity)
10a Lo (Sparsity)
0.15
Unlearning Score
a
16k Width
0.10
65k Width
1M Width
PCA Baseline
0.00
10a
LO (Sparsity)
Figure 13: Evaluation of Saamaa-Scope SAEs (16k IM latents) on Gemma-2-2B layer 12.
32

---

SAEBench: A Comprehensive Benchmark for Sparse Autoencoders in Language Model Interpretability
1.000
0.995
0.85
Loss Recovered
0.990
Autointerp Score
0.985
0.980
0.75
0.975
0.970
0.70
10a
10a
(Sparsity)
(Sparsity)
1.0
Sparse Probing Top Test Accuracy
0.82
Mean Absorption Score
0.80
0.78
0.6
0.76
0.74
0.4
0.72
0.2
0.70
0.68
10a
10a
Lo (Sparsity)
LO (Sparsity)
0.30
0.06
SCR 10 Metricc
0.25
TPP Top 10 Metric
0.05
0.04
0.15
0.03
0.10
0.05
0.01
0.00
0.00
10a
10a
(Sparsity)
(Sparsity)
0.0015
Unlearning Score
16k Width
65k Width
0.0010
1M Width
PCA Baseline
0.0005
0.0000
10a LO (Sparsity)
Figure 14: Evaluation of Seamaa-Scope SAEs (16k IM latents) on Gemma-2-2B layer 19.
33

---

SAEBench: A Comprehensive Benchmark for Sparse Autoencoders in Language Model Interpretability
1,0000
0.90
0.9975
0.89
Loss Recovered
Autointerp Score
0.9950
0.88
0.9925
0.87
0.9900
0.86
0.9875
0.85
0.9850
10
10a
10A
10a
LO (Sprrsity)
Lo (Sparsity)
1.0
Sparse Probing Top 1 Test Accuracy
0.82
Mear Absorption Score
0.8
0.80
0.78
0.6
0.76
0.4
0.74
0.2
10a
10a
10a
10a
LO (Sparsity)
(Sparsity)
0,25
0.04
SCR Top 10 Metric
0.20
TPP Top Metric
0.03
0.15
0.02
0.10
0.01
0.05
10a
10a
10a
10a
L0 (Sparsity)
(Sparsity)
0.5
0.4-
16k Width
Unlearning
131k Width
0.3
1M Width
0.2
10a
102
LD (Sparsity)
Figure 15: Evaluation of Gemma-Sconm Scope SAEs (16k IM latents) on Gemma-2-9B layer 20.
34

---

SAEBench: A Comprehensive Benchmark for Sparse Autoencoders in Language Model Interpretability
D22
0.86
0.99
Loss Recovered
Autointerp Score
0.84
13
0.97
0.82
0.96
0.80
0.95
0.78
10a
10a
Lo (Sparsity)
Lo (Sparsity)
1.0
Sparse Probing Top Test Accuracy
0.800
Mean Absorption Score
0.775
0.750
0.6
0.725
0.700
0.4
0.675
0.650
10a
10a
10a
10a
Lo (Sparsity)
Lo (Sparsity)
0.35
0.08
Top Metric
0.30
TPP Top 10 Metric
0.06
0.25
0.20
0.04
0.15
0.02
0.10
10a
10a
10a
10a (Sparsity)
Lo (Sparsity)
0.12
0.10
Unleaming Score
0.08
13
16k Width
131k Width
1M Width
0.04
0.02
10a
102 LO (Sparsity)
Figure 16: Evaluation of Gemma-Son Scope SAEs (16k IM latents) on Gemma-2-9B layer 20.
35

---

SAEBench: A Comprehensive Benchmark for Sparse Autoencoders in Language Model Interpretability
0.88
0.99
Recovered
0.98
Autointerp Score
0.87
0.86
0.96
0.85
0.84
0.94
10a
10c
10a
102
LO (Spariiyy)
Lo (Sparsity)
1.0
Sparse Probing Top Test Accuracy
0.800
Mean Absorption Score
0.8
0.775
0.750
0.6
0.725
0.4
0.700
0.675
0.2
0.650
10a
10a
10a
10a
LO (Sparsity)
L0 (Sparsity)
0.12
0.40
0.10
0.35
SCR Too 10 Metric
TPP Top 10 Metric
0.30
0.08
0.25
0.06
0.20
0.04
0.15
0.02
0.10
10a
10a
101
10a
(Sparsity)
(Sparsity)
0.04
Unlearning Score
0.02
16k Width
0.00
131k Width
1M Width
0.02
10a
102
LO (Sparsity)
Figure 17: Evaluation of Gemma-Scomm Scope SAEs (16k IM latents) on Gemma-2-9B layer 31.
36

---

SAEBench: A Comprehensive Benchmark for Sparse Autoencoders in Language Model Interpretability
J. Further SAE Bench Evaluation Results
1.00
0.85
Loss Recovered
Autointerp Score
0.98
0.80
0.75
0.94
0.70
102
10a
102
10a
LO (Sparsity)
Lo (Sparsity)
Sparse Probing Top 1 Test Test Accuracy
Mean Absorption Fraction Score
1.00
0.95
0.78
0.76
0.85
0.74
0.75
0.72
10a (Sparsity)
10a
10a
10a
(Sparsity)
SCR Top 20 Metric
0.3
TPP Top 20 Metric
0.2
0.1
0.1
10a
10a
10a
10a
(Sparsity)
(Sparsity)
0.125
Unleaminng Score
RAVEL Score
0.100
0.6
0.075
0.050
0.025
0.000
10a
(Sparsity)
(Sparsity)
ReLU
TopK
ReLU p-Annealing
Gated
JumpReLU
Batch TopK
Matryoshka Batch TopK
PCA Baseline
Figure 18: Scores for all 7 metrics on our 4K Gemma-2-2B suite.
37

---

SAEBench: A Comprehensive Benchmark for Sparse Autoencoders in Language Model Interpretability
1.00
Loss Recovered
0.99
Antointerp Score
0.80
0.98
0.75
0.97
0.70
0.96
10a
10a
10a
10a
(Sparsity)
(Sparsity)
Sparse Probing Top Test Accuracy
Mean Absorption Fraction Score
1.0
0.825
0.800
0.775
0.8
0.750
0.725
0.700
10a
10a
10a
107
(Sparsity)
(Sparsity)
SCR Top 20 Metric
TPP Top 20 Metric
0.3
0.3
0.2
0.2
0.1
0.0
10a
10a
(Sparsity)
(Sparsity)
0.75
Unleaming Score
0.1.5
RAVEL Score
0.70
0.10
0.65
0.50
0.05
0.55
0.00
10a
10a
(Sparsity)
(Sparsity)
ReLU
TopK Batch TopK
ReLU p-Annealing
Gated
JumpReLU
Matryoshka Batch TopK
PCA Baseline
Figure 19; Scores for all SAEBench metrics on our 16K Gemma-2-2B suite.
38

---

SAEBench: A Comprehensive Benchmark for Sparse Autoencoders in Language Model Interpretability
0.90
1.000
0.85
Loss Recovered
Autointerp Score
0.990
0.80
0.985
0.75
0.980
0.70
102
10a
10a
10a
Lo (Sparsity)
Lo (Sparsity)
Sparse Probing Top Test Accuracy
Mean Absorplion Fraclion Score
1.0
0.80
0.9
0.78
0.8
0.7
0.76
0.6
0.5
0.72
0.4
0.70
10a
10a
10a
10a
LO (Sparsity)
(Sparsity)
0.4
SCR Metric
0.3
TPP Top 20 Metric
0.3
0.2
0.2
0.1
0.1
0.0
10a (Sparsity)
10a
102
10a
(Sparsity)
0.20
0.75
Unleaming Score
RAVEL RAVEL Score
0.15
0.10
0.65
0.05
0.00
10a
L0 (Sparsity)
(Sparsity)
ReLU
TopK Batch TopK
ReLU p-Annealing
Gated
JumpReLU
Matryoshka Batch TopK
PCA Baseline
Figure 20: Scores for all SAEBench metrics on our 65K Gemma-2-2B suite.
39

---

SAEBench: A Comprehensive Benchmark for Sparse Autoencoders in Language Model Interpretability
0.875
1.00
0.850
0.98
0.825
Loss Recovered
Autointerp Score
0.96
0.775
0.750
0.94
0.725
0.700
0.92
0.675
10?
10" Lo (Sparsity)
Lo (Sparsity)
0.9
Sparse Probing Top Test Accuracy
Mean Absorption Fraction Score
0.8
0.79
0.7
0.78
0.6
0.77
0.5
0.76
0.4
0.75
0.3
0.74
0.2
0.73
0.1
10
10
(Sparsity)
(Sparsity)
0.4
0.3
0.30
SCR Top 20 Metric
0.2
TPP Top 20 Metric
0.25
0.1
0.20
0.0
0.15
-0.1
-02
0.05
-.3
10a Lo (Sparsity)
(Sparsity)
ReLU JumpReLU
TopK
ReLU with p-Annealing
Gated
Batch TopK
Matryoshka Batch TopK
Figure 21: Scores for all metrics on our 4K Pythia-1600 suite.
40

---

SAEBench: A Comprehensive Benchmark for Sparse Autoencoders in Language Model Interpretability
1.00
0.875
0.99
0.850
Loss Recovered
Autointerp Score
0.825
0.800
0.97
0.775
0.96
0.750
0.95
0.725
0.700
0.94
102 (Sparsity)
10a (Sparsity)
0.80
0.9
Sparse Probing Top Test Test Accuracy
0.79
Mean Absorption Fraction Score
0.8
0.78
0.7
0.77
0.6
0.76
0.5
0.75
0.4
0.74
0.73
0.3
0.72
0.2
10a
10a
Lo (Sparsity)
(Sparsity)
0.4
0.30
0.3
0.25
SCR Top 20 Metric
TPP Top 20 Metric
0.2
0.20
0.1
0.15
0.0
0.10
-0.
0.05
0.00
10a Lo (Sparsity)
10a
(Sparsity)
ReLU
TopK
ReLU with p-Annealing
Gated
JumpReLU
Batch TopK
Matryoshka Batch TopK
Figure 22: Scores for all metrics on our 16K Pythia- Pythia-160M 160M suite.

---

SAEBench: A Comprehensive Benchmark for Sparse Autoencoders in Language Model Interpretability
1.00
0.900
0.99
Autointerp Score
Loss Recoverea
0.98
0.97
0.750
0.96
102
10*
10a
10a
Lo (Sparsity)
Lo (Sparsity)
0.80
Sparse Probing Top Test Test Accuracy
Mean Absorption Fraction Score
0.8
0.78
0.6
0.76
0.5
0.74
0.4
0.72
0.2
0.1
0.70
10a
10*
10a (Sparsity)
10*
(Sparsity)
0.30
0.3
0.25
SCR Top 20 Metric
TPP Top 20 Metric
0.2
0.20
0.1
0.15
0.10
0.0
0.05
-0.1
0.00
Lo (Sparsity)
(Sparsity)
ReLU
TopK
ReLU with p-Annealing
Gated
JumpReLU
Batch TopK
Matryoshka Batch TopK
Figure 23: Scores for all SAEBench metrics on our 65K Pythia-160M suite.
42


Reviews:
[{"review_id": "Cp0RJt76PU", "reviewer": "Reviewer_NuEX", "summary": "This paper introduces a benchmark framework for evaluating SAEs.  SAEBench introduces a comprehensive evaluation suite with seven metrics across four capabilities:\n- Concept Detection: Measuring how precisely latents correspond to concepts\n- Interpretability: Evaluating human-understandability of latents\n- Reconstruction: Quantifying faithful preservation of model behavior\n- Feature Disentanglement: Assessing proper separation of independent concepts\n\nThe authors proposed three new metrics under this framework and evaluated over 200 SAEs across seven architectures with varying dictionary sizes (4k, 16k, 65k latents) on Gemma-2-2B and Pythia-160M models.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank the reviewer for the thoughtful comments and are glad that the benchmark’s goals and structure came through clearly. First, we highlight the addition of RAVEL, a metric for feature disentanglement and model editing, to our evaluation suite and invite you to compare SAE architectures in our existing interactive result browser at www.saebench.xyz We will address your questions and suggestions below: \n\nRegarding the suggestion to provide more detail on the three novel metrics: we agree this is a central contribution of our work, and we include detailed implementation descriptions for all metrics (including the new ones) in Appendix D. We’ll improve the cross-references from the main text to make this easier to find.\n\nWe also agree that some of the hypotheses in the paper are inherently hard to validate — interpretability is challenging to evaluate due to the lack of ground-truth labels for model internals. This is one of the main motivations behind SAEBench: to test SAE quality across a diverse range of practical, measurable tasks that reflect different aspects of interpretability in practice. As noted above, we've incorporated the RAVEL evaluation from Huang et al. [1] into SAEBench to further enhance our coverage of practical SAE capabilities—particularly for tasks involving selective editing of model knowledge. Findings on RAVEL support the hypothesis that Matryoshka SAEs outperform other architectures in the L0 in [40, 160] range.\n\nWe also want to highlight a key takeaway enabled by our benchmark: architectures like Matryoshka, which perform worse on standard sparsity-fidelity metrics, actually outperform others across several concept detection and feature disentanglement tasks. We believe this underscores the value of evaluating SAEs on a broader range of criteria.\n\nThank you again for the feedback! Do these changes address your concerns with the paper? If not, what further clarification or modifications could we make to improve your score?\n\nCitation\n[1] https://arxiv.org/abs/2402.17700v2", "claims_and_evidence": "Yes, the most of the claims in the paper are properly validated through experiments, though there are some hypotheses which are hard to validate.", "methods_and_evaluation_criteria": "Yes, the authors proposed three extra metrics to evaluate the capability of SAEs. More details of these metrics could be provided beyond the three paragraphs in the paper, since I reckon it as the main contribution of this paper.", "theoretical_claims": "No theoretical claims have been made in this paper.", "experimental_designs_or_analyses": "Yes, the experiment design is quite solid.", "relation_to_broader_scientific_literature": "The authors proposed a benchmark which is capable to evaluate most of the existing works on SAE-based mechanistic interpretability methods.", "essential_references_not_discussed": "No", "other_strengths_and_weaknesses": "This work is already gaining impact on the open-source and interoperability research community and is already used by various works on improving SAEs.", "other_comments_or_suggestions": "No", "questions_for_authors": "No", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "qPY9ivgsKx", "reviewer": "Reviewer_HzWd", "summary": "The paper introduces SAEBench, a new benchmarking framework for sparse autoencoders (SAEs) in language model interpretability. The authors identify limitation in existing evaluation approaches, which often rely solely on unsupervised metrics like the sparsity-fidelity tradeoff with limited practical relevance. SAEBench addresses this by measuring SAE performance across seven diverse metrics spanning interpretability, feature disentanglement, and downstream applications like unlearning.\nThe authors evaluate over 200 SAEs across seven architectures (ReLU, TopK, BatchTopK, Gated, JumpReLU, P-annealing, and Matryoshka) with varying widths (4k, 16k, and 65k latents) and sparsities. Key findings include: (1) gains on proxy metrics (e.g., sparsity and fidelity) don't reliably translate to better practical performance, (2) Matryoshka SAEs substantially outperform other architectures on feature disentanglement metrics despite underperforming on traditional proxy metrics, and (3) this advantage grows with SAE scale.", "questions": "", "limitations": "", "rating": 4, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank Reviewer HzWd for their thoughtful and encouraging review. We’re glad you found the benchmark design, breadth of evaluations, and insights on SAE scaling and architecture choices valuable. First, we highlight the addition of RAVEL, a metric for feature disentanglement and model editing, to our evaluation suite and invite you to compare SAE architectures in our existing interactive result browser at www.saebench.xyz Below, we address your comments and suggestions:\n\n**Q: Have you identified specific architectural components of Matryoshka SAEs that contribute to their superior feature disentanglement?**\n\nYes — we believe the key factor is Matryoshka’s nested loss function, which encourages learning at multiple levels of abstraction. Unlike standard SAEs that optimize a single dictionary (often leading to oversplitting of features into overly specific latents), Matryoshka SAEs optimize a sequence of nested dictionaries. This hierarchical structure likely helps preserve more coherent, disentangled features and may explain their strong performance on disentanglement metrics.\n\n**Q: How do the metrics relate to real-world use cases of SAEs?**\n\nSAEBench includes both real-world-motivated tasks and diagnostic evaluations. For example, our Spurious Correlation Removal metric builds on the SHIFT method from Sparse Feature Circuits — a compelling real-world case where researchers used SAEs for white-box, human-guided model editing that outperformed other baseline methods. Our Unlearning evaluation also reflects a real downstream goal, though we find that SAEs are not yet the strongest-performing method for this task. As noted above, we've incorporated the RAVEL evaluation from Huang et al. [1] into SAEBench to further enhance our coverage of practical SAE capabilities—particularly for tasks involving selective editing of model knowledge. Findings on RAVEL support the hypothesis that Matryoshka SAEs outperform other architectures in the L0 in [40, 160] range.\n\nOther metrics, such as Sparse Probing and Feature Absorption, are more diagnostic in nature, designed to measure underlying properties of SAE representations. Together, these metrics give a more holistic picture of SAE performance across the four desiderata: Concept Detection, Human Interpretability, Reconstruction and Feature Distentanglement.\n\nWe appreciate the reviewer’s suggestion and agree that this discussion of real-world applicability versus diagnostic utility should be made more explicit in the paper. We will incorporate this clarification in our final submission.\n\n**Q: Suggestions on extending to other modalities (e.g., vision models)**\n\nWe appreciate this suggestion. Extending SAEBench to other modalities is an exciting direction for future work, and we purposefully standardized the SAEBench codebase to be easily extensible. In this paper, we focused on the language domain to enable systematic comparison across a broad range of SAE architectures on a shared model. However, we believe many of the evaluation principles and metrics—especially those related to disentanglement and white-box editing—could be adapted to multi-modal or vision-language models. For instance, the specificity score for SAE features in the vision model SDXL turbo judged by a VLM [2], and a supervised sparse probing evaluation for matching SAE latents trained on Protein LMs to known biological concepts from [3] would be useful additions to SAEBench. We will include these pointers for future work in the final manuscript.\n\n**On concerns around supervised metrics and model diversity**\n\nWe agree with these limitations, which are already discussed in Section 5. Expanding the benchmark to cover more diverse models, layers, and ground-truth concepts is a natural next step, and we hope the open-sourced SAEs and codebase will enable the community to contribute further evaluations.\n\nWe appreciate the reviewer’s thoughtful suggestions and are glad that SAEBench was seen as a meaningful step forward for SAE evaluation. We hope it will be a valuable resource for future research in interpretability.\n\nCitations\n\n[1] https://arxiv.org/abs/2402.17700v2\n\n[2] https://arxiv.org/abs/2410.22366\n\n[3] https://www.biorxiv.org/content/10.1101/2024.11.14.623630v1.full", "claims_and_evidence": "The claims in the paper are generally well-supported by empirical evidence. This work:\n- demonstrates that the sparsity-fidelity trade-off doesn't reliably predict performance on downstream tasks.\n- show that Matryoshka SAEs excel at feature disentanglement despite underperforming on traditional metrics.\n- provides evidence for scaling behaviors across dictionary sizes (4k to 65k), with inverse scaling effects for most architectures on certain metrics.\n- supports the claim that no single sparsity level is optimal for all tasks with detailed experiments across L0 values from 20 to 1000.", "methods_and_evaluation_criteria": "The proposed evaluation methods and benchmark criteria are thoughtfully designed and appropriate for the problem. The authors:\n- develop a diverse set of metrics covering four fundamental capabilities: concept detection, interpretability, reconstruction, and feature disentanglement.\nInclude both established metrics from prior work and novel metrics for comprehensive evaluation.\n- ensure computational tractability (65 minutes per SAE) while maintaining reproducibility.\n- present a standardized framework that can be easily extended with new evaluation methods.\n- Their choice to evaluate across a much wider range of sparsities (L0 from 20 to 1000) than typically studied (20 to 200) provides valuable insights.", "theoretical_claims": "The paper doesn't make substantial theoretical claims", "experimental_designs_or_analyses": "The experimental designs and analyses appear sound and carefully conducted:\n- The authors control for variables by using identical data, training procedures, and hyperparameters across architectures.\n- They appropriately sweep across different intervention sizes to ensure robustness of results.\nSAE training dynamics are analyzed over token counts from 0 to 500M tokens, providing insights into when performance plateaus.\n- Performance differences are analyzed across both model scales (Pythia-160M vs. Gemma-2-2B) and SAE scales (4k to 65k latents).\n- Appropriate ablation studies and control conditions are included, such as comparing against PCA baselines.\n- The analyses consider limitations of the metrics, such as unexpected TPP performance at higher L0 values, and the authors discuss these findings.", "relation_to_broader_scientific_literature": "The paper effectively situates itself within the broader literature on sparse autoencoder evaluation. It builds on:\n- Sparse probing techniques (Gurnee et al. 2023, Gao et al. 2024)\n- Prior work on automated interpretability using LLMs (Paulo et al. 2024)\n- Feature absorption phenomena (Chanin et al. 2024)\n- SAE-based unlearning approaches (Farrell et al. 2024)", "essential_references_not_discussed": "The paper covers all relevant literature.", "other_strengths_and_weaknesses": "Strengths:\n- The benchmark's multi-faceted approach reveals important trade-offs that would be missed with traditional metrics, providing a more nuanced understanding of SAE performance.\n- The discovery that Matryoshka SAEs substantially outperform other architectures on feature disentanglement metrics despite underperforming on traditional proxy metrics is an interesting finding with practical implications.\n- The analysis of scaling behavior across dictionary sizes reveals inverse scaling effects for most architectures on certain metrics.\n-  200+ SAEs to be open sourced will likely accelerate progress in the field.\nWeaknesses:\n- As acknowledged by the authors, the paper evaluates on only two model architectures (Gemma and Pythia); including more diverse architectures would improve extrapolation of the results.\n- The ground truth feature sets used for supervised metrics could be limited, raising questions about how well the results would generalize to other types of features.\n- The unlearning evaluation was constrained by model capabilities, with only one test set showing sufficient performance.", "other_comments_or_suggestions": "The paper would benefit from a more detailed discussion of how the metrics relate to other real-world use cases of SAEs in model interpretability.\nA discussion of how SAEBench could be extended to other modalities (e.g., vision models) would strengthen the paper's impact.", "questions_for_authors": "Have you identified specific architectural components of Matryoshka SAEs that contribute to their superior feature disentanglement?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "FPz3AqI3sr", "reviewer": "Reviewer_55S8", "summary": "The paper introduces SAEBench, to evaluate SAE, of course, on various design choices. It proposes a unified evaluation suite that uses diverse metrics—concept detection, automated interpretability, reconstruction fidelity, and feature disentanglement—to assess SAE performance. The authors train and benchmark over 200 SAEs across seven architectures and various sparsity levels. This paper reveals that gains on conventional proxy metrics do not necessarily lead to better practical interpretability, with hierarchical architectures like Matryoshka excelling in feature disentanglement despite lower sparsity-fidelity scores. They also find that optimal sparsity levels vary by task, with moderate sparsity often striking the best balance between reconstruction and interpretability.", "questions": "", "limitations": "", "rating": 2, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "We thank Reviewer 55S8 for their thoughtful and detailed review. We’re glad you found the evaluation criteria sound and appreciated the comprehensive comparisons across sparse autoencoder (SAE) variants. First, we highlight the addition of RAVEL, a metric for feature disentanglement and model editing, to our evaluation suite and invite you to compare SAE architectures in our existing interactive result browser at www.saebench.xyz We appreciate your suggestions regarding unlearning evaluation, writing tone, and figure clarity, and respond to these below.\n\n**Q: What is the central research question your work aims to address?**\n\nOur central question is: How can we rigorously evaluate sparse autoencoders for interpreting large language models in a way that reflects their real-world utility? While over a dozen new SAE variants have recently been proposed, evaluation practices have lagged behind. Most papers still rely heavily on convenient proxy metrics like the sparsity-fidelity trade-off, which, as we show, fails to reliably predict downstream interpretability or performance.\n\nSAEBench addresses this gap with a unified, extensible evaluation suite that captures both diagnostic and task-grounded metrics. Beyond benchmarking, our evaluations have yielded new insights, such as the strong feature disentanglement properties of Matryoshka SAEs despite their weak proxy scores. This illustrates how rigorous, multifaceted evaluation can shift our understanding of what makes an SAE effective.\n\nWhile SAEBench focuses on a specific methodology (SAEs), we respectfully disagree that the contribution is niche. SAEs have rapidly become one of the most popular tools in mechanistic interpretability research, with:\n\n- multiple oral presentations at ICLR 2025 [1, 2, 3]\n\n- active research across both academia and industry, including interpretability teams at OpenAI, DeepMind, and Anthropic\n\n- interpretability startups, such as Goodfire, who recently collaborated with ARC institute to apply SAEs to protein generation models [4]\n\nTheir use spans concept discovery, editing, unlearning, and circuit analysis. Providing a standardized and extensible benchmark for evaluating this fast-growing class of methods fills a timely and important gap in the literature.\n\n**Q: Is SAEBench generalizable to other language modeling architectures, for example, RWKV and Mamba?**\n\nYes. SAEBench evaluates the quality of sparse autoencoders applied to model activations, and is in principle agnostic to model architecture. We constructed the codebase with extensibility in mind – as long as internal activations can be extracted, SAEs and our evaluations can be applied. While our current codebase is tailored to transformer models, supporting other architectures (e.g., RWKV or Mamba) would primarily require adapting the activation-extraction interface, not major architectural changes. \n\n**Q: Concerns around proxy metrics and real-world relevance**\n\nTo clarify, a key contribution of our work is moving beyond traditional proxy metrics (such as reconstruction loss and L0 sparsity) to evaluations that more directly reflect real-world interpretability and application. While prior work has relied heavily on these proxies, we show they often fail to correlate with meaningful downstream outcomes. For example, Matryoshka SAEs underperform on proxy metrics yet outperform on multiple real-world metrics like disentanglement and absorption.\n\nTo capture more meaningful desiderata, our benchmark incorporates a diverse set of evaluations targeting concept disentanglement, interpretability, editability, and knowledge removal. To further strengthen this, we have added the RAVEL evaluation from [7], which tests whether interventions on SAE latents can reliably modify model behavior in controlled and interpretable ways.\n\n**Q: Concerns around unlearning evaluation and noise in automated metrics**\n\nWe agree that unlearning is a challenging setting to evaluate and that some automated metrics are noisy. We explicitly acknowledge these limitations in Section 6. Our current unlearning setup uses the strongest available dataset for Gemma-2-2B (WMDP-bio). Extending this evaluation to larger models or new domains is a promising future direction.\n\n**Q: Informal writing style and figure clarity**\n\nWe appreciate this feedback and will revise the paper to adopt a more formal academic tone. We will also reformat figures and improve captions for clarity.\n\nThank you again for your constructive review. Do these changes address your concerns with the paper? If not, what further clarification or modifications could we make to improve your score?\n\nCitations\n\n[1] https://openreview.net/forum?id=tcsZt9ZNKD\n\n[2] https://openreview.net/forum?id=I4e82CIDxv\n\n[3] https://openreview.net/forum?id=WCRQFlji2q\n\n[4]  https://www.goodfire.ai/blog/interpreting-evo-2\n\n[5] https://arxiv.org/abs/2410.22366\n\n[6] https://www.biorxiv.org/content/10.1101/2024.11.14.623630v1.full\n\n[7] https://arxiv.org/abs/2402.17700v2", "claims_and_evidence": "Most of the paper's claims are supported by comprehensive experimental results and detailed comparisons across multiple evaluation metrics. \n\n- The claim that traditional sparsity‐fidelity metrics do not reliably predict practical interpretability is well backed by evidence showing that architectures like Matryoshka excel in feature disentanglement despite lower proxy scores, and the scaling experiments clearly illustrate nuanced trade-offs in performance. \n\nHowever, some claims—such as those regarding the unlearning evaluation—are less convincingly supported due to limitations in ground truth data and inherent noise in certain automated metrics, which may warrant further investigation.", "methods_and_evaluation_criteria": "The proposed evaluation metric for SAEs make sense to me. They address both traditional metrics (like the sparsity-fidelity trade-off) and novel criteria (such as feature disentanglement through spurious correlation removal and targeted probe perturbation).", "theoretical_claims": "No theoretical claims made", "experimental_designs_or_analyses": "Yes. They look valid and sound to me.", "relation_to_broader_scientific_literature": "There are many proposed SAE variants out there, and this paper tries to have a unified story on which is better.", "essential_references_not_discussed": "None, as far as I am aware of.", "other_strengths_and_weaknesses": "- The writing style is informal and, at times, reads more like a blog post than a rigorously written scientific paper, which undermines its academic tone.  \n\n- The paper's contribution is highly niche, focusing narrowly on a specific benchmark for sparse autoencoders; this raises questions about whether ICML is the right venue since the work appears more like a course project or blog post rather than addressing a significant research question.  \n\n- While the paper introduces novel evaluation metrics, its heavy reliance on proxy metrics leaves some doubt about the real-world impact and generalizability of the findings, potentially limiting its broader relevance. And it fails to provide sound explanations of every finding it has.", "other_comments_or_suggestions": "- Reformatting some of the figures and tables for clarity could improve the readability and overall presentation. For example, say that the L0 sparsity is measured not pre-determined could help understand the figures much better.\n\n- While the discussion of limitations is valuable, expanding it to include actionable future directions and a more structured comparison with related work could strengthen the paper further.", "questions_for_authors": "1. Is the SAEBench generalizable to other lanague modeling architectures, for example, RWKV and Mamba. \n\n2. What is the central research question your work aims to address, and how do you see SAEBench advancing our fundamental understanding of model interpretability, rather than serving as a narrowly focused blog post? A clearer articulation of the research question and its broader implications would help contextualize the work’s contributions and impact.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "VoPXQxk3AA", "reviewer": "Reviewer_c1Ru", "summary": "This paper introduces SAEBench, a new benchmark to evaluate sparse autoencoders (SAEs). The authors point out that current SAE research may over-index on sparsity-recontruction tradeoffs, a metric that may not be a good proxy in practice. SAEBench covers seven metrics covering interpretability, feature separation, and practical uses like unlearning. The authors tested over several SAEs with different designs. Their key finding is that improvements on common metrics don't always lead to better practical performance. Notably, Matryoshka SAEs perform better on feature separation metrics despite scoring worse on traditional metrics - and this advantage grows as models get larger. The authors provide an interactive website (saebench.xyz) for exploring relationships between metrics.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Thank you for your thoughtful and constructive review. We address each of your points below and will incorporate corresponding clarifications and improvements in the camera-ready version, if accepted.\n\n**Metric Definitions**\n\nWe appreciate your emphasis on clear metric definitions. We already provide detailed descriptions for all seven evaluation metrics in Appendix D, including mathematical definitions, implementation details, and hyperparameter settings. We will make the cross references to the detailed descriptions more prominent in the main text.\n\n**Feature Absorption Metric**\n\nOur approach builds directly on [1], who introduced the feature absorption phenomenon and proposed first-letter classification as a testbed. We retain this setting because (1) it provides dense ground truth labels and (2) it targets a nontrivial behavior: identifying a token’s first letter is difficult for language models and emerges with scale. In manual inspections, our absorption score correlates with qualitative analysis of gender related features. Future work could expand our absorption dataset using hierarchical relations from WordNet, such as scientific subfields.\n\n**Faithfulness vs Interpretability Concern**\n\nWe agree this is a real concern. A faithful SAE may inherit uninterpretability if the model’s features are themselves uninterpretable. However, interpretability is desirable in its own right for applications such as debugging, auditing, or analysis. SAEBench does not claim interpretability equals faithfulness, but includes it as a distinct evaluation axis. Due to this tension, our benchmark emphasizes nuanced evaluation across multiple axes rather than optimizing a single objective.\n\n**Controlling for Underlying Model Confounds**\n\nSAEBench’s primary objective is to systematically evaluate SAE architectures and training methods within a fixed model and layer, rather than comparing across models. Some metrics inherently reflect the underlying model’s internals. To rigorously control this, we trained two comprehensive sweeps: one on layer 12 of Gemma-2-2B and another on layer 8 of Pythia-160M. This setup ensures fair SAE-to-SAE comparisons.\n\n**Concept Detection and Model-Encoded Features**\n\nWe confirm that the model does encode the benchmarked concepts: linear probes on raw model activations achieve >90% accuracy on all sparse probing tasks in Gemma-2-2B. We’ll add this result in the paper to justify our concept selection.\n\n**Practical Editing Tasks**\n\nWe agree that editing-focused evaluations are valuable. Since submission, we added the RAVEL evaluation, which tests whether SAEs can edit factual attributes without side effects. Our Unlearning and Spurious Correlation Removal metrics also assess model control. SAEBench is modular and extensible, and we welcome future community additions (e.g., from MUSE).\n\n**Lack of Baselines**\n\nWe appreciate this suggestion and will incorporate the following baselines into our manuscript:\n\n- For RAVEL, we implemented the MDAS baseline from [3] and found it outperformed SAE-based editing.\n- For Unlearning, we build on [2], who benchmarked SAE methods against RMU and found RMU performed comparably or better depending on evaluation details.\n\nFor Spurious Correlation Removal, we adapt the SHIFT method from Sparse Feature Circuits, where SAEs outperformed other baselines under realistic constraints—i.e., using human-guided feature selection without access to disambiguating data. However, human or LLM-based feature selection introduces variability and complexity in a benchmark. To remove this variable, we instead use disambiguating data only to select which latents to ablate, and found this correlates well with LLM-based selection. Giving full access to this data would trivialize the task for simple baselines like linear probes, making fair comparison difficult.\n\nSAEBench is designed to provide a more nuanced understanding of SAE behavior across diverse settings, even settings where SAEs are not the best solution. We will clarify this in the paper.\n\n**AI Judge Bias Concerns**\n\nWe follow the protocol from [4], who compared human and LLM judge accuracy and found only small differences (Appendix A.6.7). We’ll clarify this in the paper.\n\n**AxBench**\n\nSAEBench is designed for benchmarking SAE variants across concept detection, interpretability, disentanglement, and reconstruction. In contrast, AxBench specifically focuses on downstream steering. We aim to establish SAEBench as the comprehensive platform for SAE evaluations and AxBench-style steering tasks would be a valuable addition to our suite. We see SAEBench and AxBench as complementary efforts.\n\nThank you again for the feedback! Do these changes address your concerns with the paper? If not, what further clarification or modifications could we make to improve your score?\n\nCitations\n\n[1] https://arxiv.org/abs/2409.14507\n\n[2] https://arxiv.org/abs/2410.19278\n\n[3] https://arxiv.org/abs/2402.17700\n\n[4] https://arxiv.org/abs/2410.13928v2", "claims_and_evidence": "- The main claims are well-supported by extensive testing of several SAE methods.\n- The authors demonstrate that standard metrics don't always predict practical performance.\n- The finding about Matryoshka SAEs excelling at feature separation is interesting, with clear evidence this advantage increases with scale.", "methods_and_evaluation_criteria": "- The benchmark is well-designed with five key goals: diversity, extensibility, speed, automation, and reproducibility.\n- The metrics cover important areas: concept detection, interpretability, reconstruction, and feature separation.\n- The unlearning and spurious correlation metrics are valuable for testing practical control over model behavior.\n\nThere are some limitations though:\n- **The \"feature absorption\" metric lacks clear definition and motivation.** The paper doesn't provide sufficient details on how this metric functions or why it specifically focuses on first-letter classification tasks. Without a clear mathematical definition and justification, it's difficult to determine whether this metric genuinely captures meaningful properties of SAEs or if it's simply an ad-hoc measurement. The arbitrary nature of focusing on first-letter classification raises questions about whether this metric generalizes to other types of concepts that SAEs might learn. The current draft needs to (a) define the feature absorption metric works and (b) motivate the focus on first-letter classification tasks.\n- **Several metrics are described vaguely without precise definitions.** The paper presents multiple evaluation metrics without providing rigorous mathematical formulations or clear procedural descriptions. This vagueness makes it difficult for readers to fully understand how these metrics are calculated and what exactly they measure. For better reproducibility and comparison, metrics should be defined with sufficient precision that another researcher could independently implement and verify them. \n- **The interpretability metric may be measuring the underlying model's properties rather than the SAE itself.** This is a fundamental conceptual issue with the benchmark's approach to measuring interpretability. If an underlying language model learns features that are inherently uninterpretable to humans, then a faithful SAE should accurately reflect this \"uninterpretability\". The automated interpretability metric appears to penalize SAEs that faithfully capture uninterpretable features, which creates a misaligned incentive. Similarly, for the concept detection metric, if a model does not learn a given concept (e.g., sentiment), then it unclear why the SAE should encode these either. The list of concepts that are used for evaluation should be filtered based on whether the model's internal activations encode them or not.", "theoretical_claims": "N/A", "experimental_designs_or_analyses": "- Strengths:\n    - Comprehensive testing across many SAE architectures and training methods\n    - Scaling analysis showing how different designs behave as they grow\n    - Interesting results on optimal sparsity of SAEs and how it impacts proposed metrics\n- Weaknesses:\n\t- **The experiments can benefit from non-SAE baselines for editing and spurious correlation removal.** The paper lacks comparisons with established baselines like those in recent works (e.g., [https://arxiv.org/abs/2410.12949](https://arxiv.org/abs/2410.12949), [https://arxiv.org/abs/2204.02937](https://arxiv.org/abs/2204.02937), [https://arxiv.org/abs/2404.11534](https://arxiv.org/abs/2404.11534)). Without these comparisons, it's difficult to contextualize how well SAEs perform in an absolute sense. A model editing approach might be much simpler and more effective than using SAEs for certain tasks, but without these baselines, readers cannot make this determination. Including these comparisons would provide crucial context about the difficulty of the tasks and whether SAEs represent the most efficient solution approach.\n\t- **The benchmark includes limited tasks demonstrating SAEs being used for practical model editing.** While the paper emphasizes the importance of practical utility, the actual benchmark contains relatively few tasks that demonstrate SAEs being used to solve real-world problems. More diverse editing tasks would better showcase the practical applications of SAEs in model control and modification scenarios. For instance, adding tasks related to factual knowledge editing, or toxicity mitigation or evaluating SAEs on tasks from MUSE (https://arxiv.org/abs/2407.06460) would make the benchmark evaluation more robust to task specifics. \n\t- **The use of AI judges for interpretability evaluation introduces potential biases without adequate controls.** Recent work ([https://arxiv.org/abs/2502.04313](https://arxiv.org/abs/2502.04313)) shows that language models tend to favor outputs from models similar to themselves, raising concerns about using them as evaluators. The benchmark might benefit from accounting for this bias in the automated interpretability metric. Without proper controls or human verification, the interpretability scores may reflect LLM preferences rather than true human interpretability. The paper should discuss these limitations and ideally validate the LLM judgments against human evaluations on a subset of the data.", "relation_to_broader_scientific_literature": "- The work addresses a timely need in neural network interpretability / SAE research.\n- **Aligns with recent work (https://arxiv.org/abs/2501.17727, https://arxiv.org/abs/2501.16615) that sanity-check the usefuless of SAEs.** For example, recent work examining SAEs applied to random models highlights concerns about whether these methods actually capture meaningful representations or merely create post-hoc explanations that may not reflect true model reasoning processes. This benchmark helps address these concerns by evaluating SAEs across multiple dimensions of practical utility.\n- **Comparison with AxBench benchmark (https://arxiv.org/abs/2501.17148)**. A discussion on differences and similarities with a concurrent benchmark on LM steering is needed.", "essential_references_not_discussed": "No but see sections above re: description of evaluation metrics introduced in previous papers.", "other_strengths_and_weaknesses": "Please check my review above", "other_comments_or_suggestions": "None", "questions_for_authors": "Please check my review above", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper introduces SAEBench, a comprehensive evaluation framework for sparse autoencoders (SAEs) in language model interpretability. The authors develop a suite of seven diverse metrics spanning concept detection, interpretability, reconstruction, and feature disentanglement, and benchmark over 200 SAEs across seven architectures. Three reviewers recommended acceptance, while one reviewer gave a rating of 2. Despite this split, the reviewers acknowledged the paper's valuable contribution in moving beyond conventional sparsity-fidelity metrics to measure practical performance. During the rebuttal, the authors effectively addressed concerns about metric definitions, evaluation methodologies, and practical applications. They clarified the rationale behind their metrics, added comparisons with non-SAE baselines, and incorporated a new RAVEL metric for feature disentanglement. The reviewers particularly appreciated the finding that hierarchical architectures like Matryoshka SAEs excel at feature disentanglement despite underperforming on traditional proxy metrics. Given the growing importance of SAEs in mechanistic interpretability research and the paper's thorough evaluation methodology, I recommend accepting this paper for ICML 2025.
Decision: Accept (poster)