Teach Old SAEs New Domain Tricks with Boosting
Authors: Nikita Koriagin
Yaroslav Aksenov
Daniil Laptev
Gleb Gerasimov
Nikita Balagansky
Daniil Gavrilov
Venue: COLM 2025 (2025)
Keywords: LLM
Interpretability
SAE
TL;DR: We propose a method to add domain-specific features into a pre-trained SAE.

Abstract: Sparse Autoencoders have emerged as powerful tools for interpreting the internal representations of Large Language Models, yet they often fail to capture domain-specific features not prevalent in their training corpora. This paper introduces a residual learning approach that addresses this feature blindness without requiring complete retraining. We propose training a secondary SAE specifically to model the reconstruction error of a pretrained SAE on domain-specific texts, effectively capturing features missed by the primary model. By summing the outputs of both models during inference, we demonstrate significant improvements in both LLM cross-entropy and explained variance metrics across multiple specialized domains. Our experiments show that this method efficiently incorporates new domain knowledge into existing SAEs while maintaining their performance on general tasks. This approach enables researchers to selectively enhance SAE interpretability for specific domains of interest, opening new possibilities for targeted mechanistic interpretability of LLMs.

Published as a conference paper at COLM 2025
Teach Old SAEs New Domain Tricks with Boosting
Nikita Koriagin'
Yaroslav Aksenov
Daniil Laptev T-Tech
T-Tech
T-Tech
Gleb Gerasimov
HSE University
T-Tech
Moscow Institute of Physics and Technology
Nikita Balagansky
Daniil Gavrilov T-Tech
T-Tech
Moscow Institute of Physics and Technology
Abstract
Sparse Autoencoders have emerged as powerful tools for interpreting the internal representations of Large Language Models, yet they often fail to capture domain-specific features not prevalent in their training corpora. This paper introduces residual learning approach that addresses this feature blindness without requiring complete retraining. We propose train- ing secondary SAE specifically to model the reconstruction error of a pretrained SAE domain-specifie texts, effectively capturing features missed by the primary model. By summing the outputs of both models during inference, demonstrate significant improvements in both LLM cross-entropy and explained variance metrics across multiple specialized domains. Our experiments show that this method efficiently incorporates new domain knowledge into existing SAEs while maintaining their perfor- mance on general tasks. This approach enables researchers selectively enhance SAE interpretability for specific domains of interest, opening new possibílities for targeted mechanistic interpretability of LLMSs.
1 Introduction
Large Language Models (LLMs) exhibit remarkable performance across numerous tasks, yet their internal mechanisms remain opaque. Mechanistic interpretability approaches, such as Sparse Autoencoders (SAEs), have emerged to disentangle LLM representations by mapping dense activations
bigher-dimensional, sparse spaces (Bricken et al., 2023; Cunningham et al., 2023; Gao et al., 2024). These sparse features often correspond to interpretable concepts, enabling researchers to isolate factors that govern model behaviors (Templeton et al., 2024).
However, SAEs can only capture frequently occurring features in their training data, leading to feature blindness when encountering rare or domain-specific concepts that the LLM itself may have learned (Templeton et al., 2024; Muhamed et al., 2024). To address this issue, practitioners often retrain SAEs on domain-speciff data (Smith et al., 2025; Muhamed et al., 2024), but this process is computationally expensive and risks catastrophic forgetting. Fine-tuning new features individualyy may also create additional challenges if they fail to align with the existing feature set.
In this paper, we introduce novel approach, SAE Boost, that allows for selective enhance- ment of SAE capabilities without full retraining. Our method builds upon the existing SAE by training supplementary model specifically designed to capture features missed by the original model when processing domain-specific texts. By modeling the reconstruction error
*ooresppndiigg author: n.skkrragin@tbbank.ru

---

Published as a conference paper at COLM 2025
targeted domains, this supplementary SAE effectively learns complementary features that can be integrated with the base model's outputs during inference.
Through empirical evaluations across multiple domains, we demonstrate improvements in both LLM cross-entropy and explained variance metrics when using the proposed method. SAE Boost enables rescarchers to selectively enhance their interpretability tools for domains of interest, facilitating more comprehensive analyses of LLM capabilities and limitations. By addressing the feature blindness problem, our work contributes to the broader goal of de- veloping more robust and complete tools for the mechanistic interpretabilliy of increasingly powerful language models.
2 Related Work
Sparse Autoencoders (SAEs) have emerged as powerful tools for the mechanistic inter- pretability of LLMs. Through sparse coding (Olshausen & Field, 1996; 1997), they aim to decompose the dense activations of LLM layers into human-interpretable features. Recent work has demonstrated that SAEs can successfully extract interpretable features from LLMs (Bricken et al., 2023; Cunningham et al., 2023; Gao et al., 2024), providing insights into how these models process and represent information. This approach builds on the broader field of dictionary learning, in which an overcomplete basis is learned to represent data sparsely (Mallat & Zhang, 1993; Olshausen & Field, 1997). In the context of neural networks, it also connects to research on disentangled representations (Bengio, 2013; Higgins et al., 2017) and feature visualization (Olah et al., 2017).
Ensuring comprehensive feature coverage is a fundamental challenge in SAE training. Leask et al. (2025) showed that increasing dictionary size leads to two types of latents: novel latents that capture previously absent information, and reconstruction latents that refine existing features. They introduced "SAE stitching," method to integrate novel latents from larger SAEs into smaller ones, enhancing reconstruction without substantial model growth. Templeton et al. (2024) found that extremely large dictionaries (on the order of billions of features) are necessary to reliably capture rare concepts, underscoring the computational difficulty of addressing these infrequent yet critical features.
3 Methodology
3.1 SAE Boost Architecture
As noted in Section 1, fully fine-tuning an SAE with domain-specific data may degrade existing features. Moreover, tuning a small subset of newly initialized features (with arbitrary initialization) does not guarantee that these new features will precisely capture the missing domain features. To address this limitation, we propose a method designed to exactly cover missing features, which can be viewed as an error in the reconstruction of hidden states.
Our key insight is that, rather than retraining the entire SAE on domain-specific data, we can train a secondary SAE to model only the reconstruction errors of the pretrained SAE in the target domain. This method allows the model learn additional features that the original SAE missed, without interfering with or disrupting existing representations, Our approach consists of two components: pretrained SAE and a residual SAE, or SAE Boost.
For a given input activation x from an LLM layer, the pretrained SAE processes it as an autoencoder & Waeco (Wencx benc) + b where Wene RR Fxd and Waec E dxF are the encoder and decoder weights, benc E R and baec E IRd are the encoder and decoder biases, and or is the sparsit--enforcing activation function (typically jumpReLU or batch.topk) (Bricken al., 2023).
Instead of directly reconstructing x, the residual SAE learns the residual error é = x 8 as è Wressr o W + ) where Wiee and Wress are the encoder and decoder weights of dec the residual SAE, and b btss is the encoder bias. We omit the decoder bias term in the residual
2

---

Published as a conference paper at COLM 2025
Domaao-Specclffcc
General
I Pdomain
Figure 1: Left: During training, the domain-specific SAE is trained on the residuals of the general SAE (frozen weights). Right: During inference, the domain-specific and general SAEs are stitched together. We show the equivalence of the training (left) and inference (right) outputs in Section 3.2.
SAE to ensure contributes only when meaningful domain-specific features are detected. This prevents unnecessary corrections to the pretrained SAE's output.
We train the residual SAE on domain-specific data, taking the reconstruction error of the pretrained SAE as the training target. The loss function remains the standard SAE objective f L= 2| + À freg, where the first term is the reconstruction loss, and the second term the sparsity penalty scaled by A. This setup allows the residual SAE to focus on features the pretrained SAE fails to capture, especially domain-specific ones. By using the same objective but a different target, augments the pretrained SAE without competing with its existing features.
3.2 Combined Model for Inference
During inference, combine outputs from the pretrained SAE and the domain-specific residual SAE as x = & te = 8 ē = x+ é, where e2 is the remaining error not captured by the residual SAE.
When multiple domain-specific residual SAES are used, the combined error becomes e EN ((0) + EN, where N is the number of residual SAEs (each trained on distinct domain) and 2i0) is the i-th residual SAE's reconstruction. The final reconstruction then x & + EN ((0
The resulting approach resembles tuning specific newly initialized features, although only targets errors made by the pretrained SAE, thus providing complementary features. See Figure for schematic view of SAE Boost.
4 Experiments
We conducted extensive experiments to evaluate the effectiveness of our residual SAE approach across various domains and model architectures. Our experiments aim to answer the following questions: can residual SAEs effectively capture domain-specific features missed by general-purpose SAEs, does incorporating residual SAEs maintain performance general domain texts, and does the approach generalize across different domains and base models?
4.1 Experimental Setup
Models. We conducted our experiments using two foundation models. We extracted the residual stream after the 24th transformer block of the Qwen-2.5-7B-Ban (Yang et al., 2024)
3

---

Published as a conference paper at COLM 2025
Domain
Model
EV 1
LLM CE
Lo Sparsity
SAE (Qwen)
0.571
0.935
47
SAE Boost (Qwen)
0.716 (+25.39%)
0.767 (-17.97%)
52
Chemistry Data
SAE (Llama Scope)
0.551
1.208
52
SAE Boost (Llama Scope)
0.702 (+27. (0%)
0.995 (-17.63%)
57
SAE (Qwen)
0.455
4.716
57
SAE Boost (Qwen)
0.725 (+59.34%)
2,060 (-56.32%)
62
Russian Texts
SAE (Llama Scope)
0.657
2.538
54
SAE Boost (Llama Scope)
0.713 (+8.52%)
2.411 (-5.00%)
59
SAE (Qwen)
0.693
2,464
56
SAE Boost (Qwen)
0.774 (+11.35%)
2.279 (-7.88%)
61
UN Debates
SAE (Llama Scope)
0.668
2.350
55
SAE Boost (Llama Scope)
0.756 (+13.17%)
2.240 (-4.6%%)
60
Table 1: Performance of the pretrained SAE (baseline) and our SAE Boost approach across three specialized domains, each using different pretrained LLM as the backbone. Our results show consistent, significant improvements both reconstruction quality and LLM performance with SAE Boost. These findings confirm that our residual approach effectively captures domain-specifit features that general-purpose SAEs miss, while maintaining reasonable sparsity overhead. Furthermore, the proposed method generalizes well across various LLMs.
model and then trained our SAEs on these activations. The 24th layer was selected as representative mid-to-late layer, where models typically develop rich feature representations. We also tested our approach using a publicly available pretrained SAE for the Llama-3.1- 8B-Base model (team, 2024) from the Llama Scope project (He et al., 2024), specifically the Beeea------Base-eee
to verify generalizability.
Datasets. For general domain training, we used the Fineweb-edu corpus (Penedo et al., 2024), diverse collection of educational and informational web content. For domain- specific experiments, we selected three distinct domains: general texts in the Russian language from the Fineweb-2 dataset (Penedo et al., 2024), representing cross-lingual domain where certain features may be underrepresented in English-ddominate pretraining; specialized scientific content focused chemistry data (Li et al., 2023), containing domain- specific terminology and concepts; and political discourse from UN debates (Henderson" et al., 2022), featuring formal diplomatic language and policy discussions.
Training Details. We trained our models using the following procedure. First, we trained an SAE on the Fineweb-edu corpus with standard SAE training objectives, including re- construction loss and an L1 sparsity penalty. We then trained a Residual SAE on each domain-specific dataset to capture the reconstruction error of the pretrained SAE for that domain. For sparsity enforcement, we employed the batch topk activation (Bussmann et al., 2024) function with k = 50 for the pretrained SAE and & = 5 for the Residual SAE. During inference, we converted batch topk to jumpReLU by determining suitable activation thresholds on the respective training datasets.
Evaluation Metrics. We evaluated our models using three main metrics. Explained Variance (EV) measures the variance between the original activations and the SAE recon- structions, indicating the proportion of variance captured by the model. LLM Cross-Entropy (LLM CE) quantifies the change in cross-entropy when the original LLM activations are replaced with SAE reconstructions, thereby assessing how well the SAE preserves informa- tion relevant to the LLM's next-token predictions. Finally, LO Sparsity refers to the average number of active features per input, reflecting the model's capacity utilization.
We compared our residual SAE approach against several alternative methods for domain adaptation of SAEs, specifically: Extended SAE (most active init), which extends the dictionary with new features initialized from the most active features on domain data;
Baselines.
4

---

Published as a conference paper at COLM 2025
Extended SAE (random init), which extends the dictionary using randomly initialized new features; SAE Stitching, which involves fully fine-tuning a pretrained SAE, identifying the features that changed the most (based on cosine similarity), and then stitching these features back into the original model; and Full Fine-tuning, for which SAE weights are simply fine-tuned using a specific domain dataset. For the Extended SAE approaches, only the newly added features were trained.
To ensure fair comparison between our proposed SAE Boost approach and these baselines, we maintained consistent training conditions across all experiments. Each method was trained on the same number of tokens (1B for each domain), and each approach added the same number of features to the base model. Specifically, for the Extended SAE approaches, we added exactly the same number fea tuatures as our residual SAE. The SAE Boost approach added a residual dictionary of size 1024, while the Extended SAE approaches expanded the base SAE dictionary by the same number of features. The SAE Stitching approach selected 1024 features from the fine-tuned model for integration into the base model.
4.2 Results
4.2.1 Domain--pecific Performance
Table 1 presents the improvements in reconstruction quality and LLM cross-entropy the domain-specift test sets when comparing our SAE Boost approach to the pretrained SAE alone. The results show consistent improvements across all three domains, with the residual SAE effectively capturing domain-specific features that the pretrained SAE missed. Moreover, the SAE Boost approach demonstrates these gains across different pretrained SAEs, indicating its generalizability across diverse base models.
4.2.2 Impact on General Domain Performance
A key concern when adapting models to specific domains is the potential degradation of general domain performance. Table 2 shows the impact of incorporating the residual SAE performance with general domain texts. Our results indicate that incorporating domain- specific residual SAEs has minimal impact on general domain performance, with changes of less than 1% across all metrics. This confirms that our approach learns complementary features rather than competing with existing ones, As with previous results, we observed this behavior across different base models, which further supports the general capabilities of our method regardless of the underlying pretrained model. This consistency underscores its robustness and practical applicability.
SAE
Configuration
Domain
EV
LLM CE
LO Sparsity
Baseline
0.719
2.385
50
Chemistry
0.717 (-0.28%)
2.390 (+0.21%)
51
Qwen
With Residual
Russian
0.719 (0.00%)
2.385 (0.00%)
50
UN Debates
0.719 (0.00%)
2.386 (+0.04%)
50
Baseline
0.672
2.599
55
Chemistry
0.672 (+0.00%)
2.596
-..12%)
55
LLama
With Residual
Russian
0.676 (+0.60%)
2.612 (+0.50%)
55
UN Debates
0.671 (-0.15%)
2.585 (-0.54%)
56
Table 2: The results of adding domain-specific SAE Boost on performance in general domain tasks. We compare the baseline (without SAE Boost) to configurations that use SAE Boost from three different domains. The results show negligible impact on general domain performance across all metrics. This finding confirms that our approach effectively isolates domain-specific features without compromising general capabilities, demonstrating that residual SAEs learn complementary rather than competing features. The evaluation was conducted using both Qwen SAEs and LLammaScope.
5

---

Published as a conference paper at COLM 2025
Method
General EV
General L0
UN Debates EV
UN Debates L0
SAE Boost (ours) Extended SAE (most act)
0.719
50
0.774
61
0.714
0.770
64
Extended SAE (random)
0.716
0.774
64
SAE Stitching
0.719
50
0.703
57
Full fine-tuning
0.515
81
0.850
50
Table 3: Comparison of our SAE Boost approach against alternative methods for domain adaptation. While all methods maintain similar general domain performance, SAE Boost achieves the best -ppacf- performance with efficient sparsity. Extended SAE ap- proaches demonstrate competitive domain performance but require higher sparsity, whereas SAE Stitching significantly underperforms on domain adaptation. Full fine-tuning suffers from catastrophic forgetting. These results illustrate that our approach strikes the optimal balance between preserving general capabilities, enhancing domain performance, and effi- cient feature utilization. Additional detailed comparisons for the Chemistry and Russian domains can be found in Tables 6 and 7 in the Appendix, confirming consistent patterns across diverse domains.
Method
General EV
General LO
UN Debates EV t
UN Debates LO
Baseline (No adaptation)
0.719
50
0.695
56
Multi-Domain Approuches:
SAE Boost
0.715
0.770
62
Extended SAE (random)
0.708
55
0.760
67
SAE Stitching
0.674
53
0.670
59
Table 4: Performance on the UN Debates domain with single vs. multiple domain adapta- tions applied simultaneously. This table demonstrates that even when applying all three domain-specific models together (Chemistry, Russian, and UN Debates), our SAE Boost approach maintains excellent domain-specific performance on UN Debates compared to single-domain adaptation, with minimal impact on general domain performance. In con- trast, alternative approaches show more significant trade-offs in either sparsity, general performance, or domain performance.
4.2.3 Comparison of Domain Adaptation Methods
As shown in Table 3, our SAE Boost approach achieves the best balance between domain- specific performance and overall general capabilities among the compared methods for domain adaptation of SAEs. While the Extended SAE approaches exhibit competitive performance on the UN Debates domain, they require slightly higher LO sparsity, indicating less efficient feature utilization. SAE Stitching performs comparably to SAE Boost on the general domain but demonstrates notably weaker domain-specine adaptation. Full fine- tuning showed large Explainable Variance score, yet largely forgets information with general domain dataset.
We hypothesize that Extended SAE loses performance in the general domain due to the concurrency of newly trained features with old ones (i.e., we cannot ensure that they pre- cisely cover the new domain). In contrast, SAE Stitching fails to capture newly emerging features from domain-specific datasets, possibly because most changed features are initial- ized with features previously trained to capture general-domain data. Both approaches demonstrate misalignment between new and old features, aligning with the motivation described in Section 1, Consequently, our approach offers a key advantage: it maintains strong gemenal-domain performance while achieving superior domain-specific adaptation at efficient sparsity levels. Similar patterns emerge in other domains, with additional results for Chemistry data and Russian texts presented in Tables 6 and 7 in the Appendix. These evaluations further confirm the effectiveness and generalizability of our approach across diverse domain types.

---

Published as a conference paper at COLM 2025
Redundarcy
Coneced DAC
Number ou
Encored
6-2
28ri Boac FastuRs
Madmum larty Theashot
Figure 2: Similarity analysis between domain-specific UN Debate features and the base features. (left) The distribution of the maximum cosine similarity between each domain- adapted feature and any feature in the base model. (right) The cumulative distribution function of these similarities. Notably, SAE Boost features exhibit lower similarity to base features, indicating that they capture more novel domain-specific information rather than merely replicating existing representations.
To understand performance differences, we analyze feature similarity between domain- adapted and pretrained features. Figure shows that SAE Boost features exhibit lower cosine similarity base features than both Extended SAE and SAE Stitching. These sim- ilarity patterns explain our method's advantages: Extended SAE features overlap more with base features, potentially causing interference during inference, while SAE Stitching struggles to capture novel concepts due to its initialization. In contrast, by explicitly target- ing reconstruction errors, SAE Boost learns more complementary features that effectively address representational gaps in the base model.
Pareto Frontier of General Domain EV
0.78
0.76
Pretmined S6E
Better
0.74
0.72
0.70
0.68
0.66
0.64
0.68
0.70
0.72
0.74
0.76
0.78
0.80
0.82
UN Debates EV
Figure 3: SAE Boost is compared to other baselines the Pareto frontier, with the optimal area marked by an arrow. SAE Boost is closer to the optimal point. See Section 4.2.3 for more details.
We also compare SAE Boost to simple baselines that involve fine-tuning with varying numbers of training steps and different data mixtures. Note that SAE Boost does not require general-domain data. We measure explained variance FineWeb dataset and domain sopeifi- texts from the UN Debates dataset. The results are presented in Figure 3. The proposed method is closer to the optimal point on the Pareto frontier.
general-doman
texts from the
4.2.4 Multi-Domain Adaptation
A significant advantage of our residual approach is its modularity and the ability to in- corporate multiple domain-speccfic adaptations simultaneously without compromising performance. To demonstrate this, trained separate SAE Boosts for the Chemistry, Rus-
7

---

Published as a conference paper at COLM 2025
sian, and UN Debates domains, then evaluated their combined performance when applied concurrently.
Table 4 presents the results obtained when evaluating the UN Debates domain, while Tables 8 and in the Appendix show corresponding results for the Chemistry and Russian domains, respectively. These results illustrate that even when all three domain-specific residual SAEs are applied simultaneously, our approach maintains excellent performance compared to both single-domain adaptation and alternative methods.
4.2.5 Training Dynamics and Feature Convergence
Figure 4 illustrates the explained variance (EV) as a function of training tokens for our residual SAE on domain-specific data. Undertraining the residual SAE produces poorly defined features that degrade the pretrained SAE's performance on general domain tasks, We found that residual SAEs trained on fewer than 100M tokens can reduce general domain performance by up to 31% in explained variance. As training progresses beyond the 200M token mark, the residual features become more distinct and complementary to those in the pretrained SAE. At this point, we observe minimal interference with general domain performance (less than 1% change explained variance), as shown in Table 2. This finding suggests that well-converged features from the residual SAE are crucial for maintaining general domain capabilities while enhancing domain-specific performance.
These observations have important implications for practical deployment of our approach: (1) residual SAEs must be sufficiently trained to avoid degrading general performance, (2) practitioners should monitor both domain-specific improvements and general domain performance during training, and (3) early stopping based solely on domain-specific metrics may lead to suboptimal feature quality.
4.2.6 Interpretabilityy Analysis
To demonstrate the effectiveness of our SAE Boost approach in capturing domain-specific features, we performed detailed interpretability analysis of the features learned by both the pretrained SAE and the domain-specific SAE Boost models. This analysis reveals how SAE Boost successfully identifies meaningful domain-specific concepts that general-purpose SAEs might miss.
0.80
0.774
0.75
Explained Variance
0.70
0.65
0.60
0.55
Debates Dataset
Fineweb Dataset
0.50
Pretrained SAE (UN Debates) 0.693
Pretrained (Fineweb)
0.45
10M
20M
50M
100M 200M
500M
Number Training Tokens
Figure 4: Evolution of explained variance during training for domain-specifie and general domains. This figure tracks the training progress of the residual SAE on both UN Debates (blue line) and Fineweb datasets (orange line), measured against the baseline performance of the pretrained SAE (dashed lines). This demonstrates that sufficient training (> 50M tokens) is critical for the residual SAE to develop complementary features that enhance domain- specific performance without degrading general capabilities. The training dynamics reveal that undertraining the residual SAE (<50M tokens) would result in suboptimal feature quality with potential negative impacts on general domain performance.
8

---

Published as a conference paper at COLM 2025
Send (Aatt00
Eas calfersionn
VN Devito
Souray Coeud
heggoad fut Maite
Diplomati:
Equaty Concapes
Prackksping
Dupare Randarsan
teatghods
Chenisty
Carbon Carbon
Compound
Positin Nurbering
Reution Mediaaiirts
Figure Representative domain-specifee features discovered by SAE Boost, along with their corresponding top activations.
-100
100
Chemistry
Japanese
Polish
Portuguese
Russian
Debates
Figure 6: A I-SNE visualization of the feature embeddings from the base SAE and multiple domain-specific SAE Boost models. The figure reveals distinct clustering by domain type, with base features (blue) occupying the central region, Chemistry features (orange) forming compact cluster to the left, and ÜN Debates features (turquoise) at the bottom. Notably, cross-lingual features cluster together on the right side, with related languages in close proximity (e.g,, the Romance languages Portuguese and Italian, and the Germanic languages German and Dutch). This organization demonstrates that SAE Boost captures both domain- specific knowledge and inherent relationships between related domains, while maintaining clear separation from general features.
We analyzed the learned features by examining their top activations across domain-specific corpora. For each feature, we collected text segments that produced the highest activation values, allowing us to interpret the semantic concepts captured by that feature. Figure presents representative domain-specific features discovered by our SAE Boost models in different domains, illustrating the distinct concepts captured in each. This table demon- strates how our SAE Boost approach effectively identifies domain-specifie concepts across diverse domains. In the UN Debates domain, features capture diplomatic terminology, policy frameworks, and peacekeeping operations. Chemistry features identify specific chem- ical compounds, reaction types, and nomenclature conventions, Together, these features show that our approach can discover meaningful domain-specific concepts that might be overlooked by general-pus cuurpoe- SAEs.

---

Published as a conference paper at COLM 2025
To better understand the global organization of features across domains, we conducted t-SNE visualizations of the feature embeddings from our SAE Boost models. These visu- alizations offer insights into how domain-specific features relate to both general features and each other. Figure 6 presents a t-SNE visualization of feature embeddings from the base SAE and multiple domain-specift SAE Boost models. The figure reveals distinct clustering by domain type, with base features (blue) occupying the largest central region while domain-specific features form distinct clusters at the periphery. This arrangement demonstrates that our SAE Boost approach identifies features that are not only semantically distinct (as shown by our activation analysis) but also structurally distinct in the embedding space. Particularly noteworthy is the organization of cross-lingual features on the right side of the visualization. We observe that features from related language families tend to cluster together: the Slavic languages (Russian and Polish) form adjacent clusters, as do the Romance languages (Italian and Portuguese) and the Germanic languages (German and Dutch). Japanese, being linguistically unrelated to the European languages, forms its own distinct cluster. This organization reflects linguistic reality, suggesting that our SAE Boost approach naturally captures underlying typological relationships between languages. The clear separation between domains in these visualizations provides further evidence that SAE Boost approach effectively captures domain-specific knowledge in distinct, interpretable feature spaces.
Conclusion
This paper introduces SAE Boost, a residual learning approach that addresses feature blindness in the sparse autoencoders used for LLM interpretability. By training secondary SAE to specifically model the reconstruction error of a pretrained SÁE on domain-specific texts, SAE Boost enables more comprehensive interpretability across specialized domains without requiring full retraining.
Our experiments show that SAE Boost improves both reconstruction quality and LLM cross-entropy across multiple domains, while preserving overall performance. The key advantage of this method is selective domain enhancement without modifying the base model.
SAE Boost represents an important step forward in the mechanistic interpretability of LLMs by allowing researchers to selectively enhance their tools for particular domains of interest. As language models continue to grow in capability and application scope, such targeted approaches will become increasingly valuable for understanding their internal representations and behaviors.
References
Yoshua Bengio. Deep learning of representations: Looking forward. In International Confer- ence on Statistical Language and Speech Processing, pp. 1-37. Springer, 2013.
Trenton Bricken, Adly Templeton, Joshua Batson, Brian Chen, Adam Jermyn, Tom Conerly, Nick Turner, Cem Ånil, Carson Denison, Amanda Askell, et al. Towards monosemanticity: Decomposing language models with dictionary learning. Transformer Circuits Thread, 2, 2023.
Bart Bussmann, Patrick Leask, and Neel Nanda. Batchtopk sparse autoencoders, 2024. URL https///arxiv.og//bb////// 06410.
Hoagy Cunningham, Aidan Ewart, Logan Riggs, Robert Huben, and Lee Sharkey. Sparse autoencoders find highly interpretable features in language models. arXio preprint arXiv:2309.08600, 2023.
Leo Gao, Tom Dupré la Tour, Henk Tillman, Gabriel Goh, Rajan Troll, Alec Radford, llya Sutskever, Jan Leike, and Jeffrey Wu. Scaling and evaluating sparse autoencoders, ar Xrv preprint ar Sttt:2406.04099 2024.
10

---

Published as a conference paper at COLM 2025
Zhengfu He, Wentao Shu, Xuyang Ge, Lingjie Chen, Junxuan Wang, Yunhua Zhou, Frances Liu, Qipeng Guo, Xuanjing Huang, Zuxuan Wu, et al. Llama scope: Extracting millions of features from Ilama-3.1-8b with sparse autoencoders. arXiv preprint ar arXX::2410020526,, 2024.
Peter Henderson", Mark S. Krass", Lucia Zheng, Neel Guha, Christopher D. Manning, Dan Jurafsky, and Daniel E. Ho. Pile of law: Learning responsible data filtering from the law and a 256gb open-source legal dataset, 2022. URL httpp://arxke orggabs//2000 202220220.
Irina Higgins, Loic Matthey, Arka Pal, Christopher Burgess, Xavier Glorot, Matthew Botvinick, Shakir Mohamed, and Alexander Lerchner. beta-VAE: Learning basic vi- sual concepts with constrained variational framework. In International Conference Learning Representations, 2017.
Patrick Leask, Bart Bussmann, Michael Pearce, Joseph Bloom, Curt Tigges, Noura Al Moubayed, Lee Sharkey, and Neel Nanda. Sparse autoencoders do not find canonical units of analysis, 2025. URL https: http:///awwiv....aab/////ssa
Guohao Li, Hasan Abed Al Kader Hammoud, Hani Itani, Dmitrii Khizbullin, and Bernard Ghanem. Camel: Communicative agents for "mind" exploration of large scale language model society, 2023.
Stéphane G Mallat and Zhifeng Zhang. Matching pursuits with time-frequenny dictionaries. In Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing, volume 3, pp. 3397-3400. IEEE, 1993.
Aashiq Muhamed, Mona Diab, and Virginia Smith. Decoding dark matter: Specialized sparse autoencoders for interpreting rare concepts in foundation models, 2024. URL https::/aarxiv.orr/bbbb// 000443.
Chris Olah, Alexander Mordvintsev, and Ludwig Schubert. Feature visualization. Distill, (tl))7,20117
Bruno A Olshausen and David Field. Emergence of simple-cell receptive field properties by learning sparse code for natural images. Nature, 381 (6666))))60076609 1996.
Bruno Olshausen and David J Field. Sparse coding with an overcomplete basis set: A strategy employed by V1? Vision Research, 37(23),3311-3325, 1997.
Gonçalo Paulo, Alex Mallen, Caden Juang, and Nora Belrose. Automatically interpreting millions of features in large language models. arXiv preprint Xiv: 2410.13928, 2024.
Guilherme Penedo, Hynek Kydlíček, Loubna Ben allal, Anton Lozhkov, Margaret Mitchell, Colin Raffel, Leandro Von Werra, and Thomas Wolf. The fineweb datasets: Decanting the web for the finest text data at scale, 2024. URL https::/arxiv.org/aasst orgg/b//////// 17557.
Lewis Smith, Senthooran Rajamanoharan, Arthur Conmy, Callum McDougall, Tom Lieberum, János Kramár, Rohin Shah, and Neel Nanda. Negative results for saes on downstream tasks and deprioritising sae research (gdm mech interp team progress up- date 2), 2025. URL fort org http///sss////uCCC/utttt/t/////
LLaMA team. The llama 3 herd of models, 2024. URL httpss//arxxxivoor///////////
Adly Templeton, Tom Conerly, Jonathan Marcus, Jack Lindsey, Trenton Bricken, Brian Chen, Adam Pearce, Craig Citro, Emmanuel Ameisen, Andy Jones, et al. Scaling monosemantic- ity: Extracting interpretable features from claude 3 sonnet. Technical report, Anthropic, 2024.
An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, Keming Lu, Keqin Bao, Kexin Yang, Le Yu, Mei Li, Mingfeng Xue, Pei Zhang, Qin Zhu, Rui Men, Runji Lin, Tianhao Li, Tingyu Xia, Xingzhang Ren, Xuancheng Ren, Yang Fan, Yang Su, Yichang Zhang, Yu Wan, Yuqiong Liu, Zeyu Cui, Zhenru Zhang, and Zihan Qiu. Qwen2.5 technical report. Xiv preprint 212111241,. 2024.
11

---

Published as a conference paper at COLM 2025
A Additional Experimental Results
A.1
Cross-Lingual Domain Performance
Table presents additional cross-lingual domain performance results, which demonstrate the effectiveness of our residual SAE approach across various languages.
Language Pretrain EV t
Pretrain LO
Residual EV
Residual LO
Russian
0.455
57
0.725
62
Japanese
0.348
58
0.695
63
Portuguese
0.532
0.735
59
Italian
0.543
53
0.742
58
Dutch
0,568
0.744
53
Polish 0.510
52
0.748
57
German 0.559
52
0.733
57
Table
Cross-lingual performance of the residual SAE approach.
These findings further support our primary conclusion that the residual SAE effectively captures domain-specific features across multiple languages. Notably, Japanese exhibits the greatest improvement, likely due to its distinct writing system and linguistic characteristics compared to the predominantly English-based training corpus.
A.2 Comparison of Domain Adaptation Methods
Tables and provide detailed comparisons between our SAE Boost approach and alterna- tive methods for domain adaptation on the chemistry and Russian domains, respectively.
Method
General EV
General LO
Chemistry EV 1
Chemistry LO
SAE Boost (ours)
0.719
50
0.716
52
Extended SAE
0.714
52
0.726
56
SAE Stitching
0.715
50
0.590
49
Full fine-tuning
0.448
95
0.837
50
Table 6: Comparison of domain adaptation methods on chemistry data.
Although the Extended SAE approach achieves slightly better performance chemistry data, it requires higher sparsity and slightly reduces general domain performance. In contrast, our SAE Boost approach strikes better balance between domain-specific and general domain capabilities.
Method
General EV t
General LO
Russian EV f
Russian LO
SAE Boost (ours) 0.719
50
0.725
62
Extended SAE
0.708
52
0.741
SAE Stitching
0.702
50
0.505
60
Full fine-tuning
0.517
66
0.829
50
Table Comparison of domain adaptation methods on Russian texts.
For Russian texts, although the Extended SAE achieves slightly better domain-specific performance, it requires significantly higher sparsity (71 vs. 62), suggesting less efficient feature usage. Meanwhile, SAE Boost retains stronger general domain performance while achieving competitive domain-specific results.
12

---

Published as a conference paper at COLM 2025
A.3 Multi-Domain Adaptation
Tables and 9 show the results of simultaneously applying multiple domain-specific residual SABs to the chemistry and Russian domains, respectively. These experiments demonstrate that our SAE Boost approach maintains robust performance even when multiple domain- specific residual SAEs are utilized concurrently.
Method
General EV t
General LO
Domain EV
Domain LO
Single-Domain Approaches:
Baseline (No adaptation)
0.719
50
0.571
47
SAE Boost (Chemistry only)
0.719
50
0.716
52
Multi-Domain Approaches:
SAE Boost (3 domains)
0.712
52
0.711
54
Extended SAE (3 domains)
0.704
55
0.712
61
SAE Stitching (3 domains)
0.685
53
0.510
51
Table 8: Multiple domain adaptation results chemistry data.
In the chemistry domain, our multi-domain SAE Boost approach (with three domains) achieves performance similar single-domain adaptation (0.711 vs. 0.716 EV) with only small increase in sparsity. Although the Extended SAE method obtains comparable domain performance, it incurs higher sparsity overhead, while SAE Stitching experiences considerable performance degradation under multi-domain conditions.
Method
General EV f
General L0
Domain EV f
Domain LO
Single-Domain Approaches:
Baseline (No adaptation)
0.720
0.455 0.725
57
SAE Boost (Russian only)
0.719
50
62
Multi-Domain Approaches:
SAE Boost (3 domains)
0.713
52
0.714
66
Extended SAE domains)
0.701
55
0.720
81
SAE Stitching (3 domains)
0.677
53
0.503
62
Table Multiple domain adaptation results on Russian texts.
For Russian texts, our multi-domain SAE Boost approach preserves strong performance (0.714 vs. 0.725 EV) with only moderate increase in sparsity. Although the Extended SAE method exhibits competitive domain performance, it demands substantially higher sparsity (81 vs. 66). In contrast, SAE Stitching again underperforms in the multi-domain scenario.
Overall, these findings underscore the primary advantage of our approach: it preserves high performance in the general domain while delivering superior domain-specific adaptation with efficient sparsity across multiple domains.
A.4 Interpretability on Domain-Specific Data
To support our interpretability claims. While Figure provides anecdotal support, we further substantiate interpretability using quantitative metrics following Paulo et al. (2024). We evaluated all 1024 residual SAE features on the chemistry dataset and randomly sampled 1024 base SAE features evaluated on the FineWeb-Edu dataset. Results are presented in Table 10. SAE Boost consistently outperforms base SAE which indicates presence of interpretable domain-specific features.
A.5 Domain-Specifir Features Analysis
To ensure that SAE Boost discovers domain-specifie features not represented in the base SAE, selected five features from our method and compared them to the top three most
13

---

Published as a conference paper at COLM 2025
SAE
Mean (median) detection score
Mean (median) fuzzing score
Base SAE
0.67 (0.65) = 0.16
0.64 (0.6) = 0.12
Residual SAE
0.75 (0.77) 0.17
0.68 (0.68) 0.12
Table 10: Detection and fuzzing scores for different SAE variants
similar features (by cosine similarity) in the base SAE on the chemistry dataset. The results are presented in Table 11. SAE Boost indeed discovers new features related to chemistry.
SAE Boost Feature
Most Similar General-Domain Features
(1) Numerical thresholds;
Oxygen in chemical contexts
22) Names of people;
(3) Risk-related phrases
(1) Durability references;
Chemical and thermal stability
22 Legal protections;
(3) Safety responsibilities
(1) Protein databases;
Supramolecular structure formation
(2 Scientific measurements;
(3) Mineralogy terms
(1) Chemical reaction terms;
Chemistry energy values
(2) Calibration concepts;
33) Physics discussions
(1) Personal relationships;
Hydrogen chemistry
(2) Compound applications;
Synthetic processes
Table 11: Mapping of SAE Boost Features to similar General-Domain Features. See Appendix A.5 for more details.
A.6 SAE Boost under different sparsity levels
Top-k
General EV
General Lo
Domain EV
Domain L0
0.719
50
0.774
61
10
0.719
52
0.781
20
0.719
56
0.788
76
50
0.721
72
0.798
106
Table 12: Performance metrics different top-k values.
To ensure that the choice of the top-k value for the residual SAE is optimal, we conducted sensitivity analysis, which indicated minor domain performance gains with higher but at the cost of reduced sparsity and interpretability. Results are presented in Table 12. We selected k balance strong domain performance, minimal general-domain disruption, and optimal interpretability.
14


Reviews:
[{"review_id": "Hrz5Y6lX2m", "reviewer": "Reviewer_Zjaf", "summary": "The authors propose SAE Boost, a method for extending a general-domain SAE with domain-specific sparse features for mechanistic interpretability. They propose to model the residuals of the general-domain SAE on the domain-specific dataset. They conduct experiments on splits of FineWeb and then apply it to downstream interpretability analyses.", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "- The paper is cogent and well-presented, along with practical analyses demonstrating their technique\n- The choices of the datasets are well-informed.\n- The technique is novel", "reasons_to_reject": "- Taking a step back and looking at the big picture, I don't think the problem is particularly large in scope. While it is likely interesting to a small group of researchers, if the domain of the text is known ahead of time as is often the case for mechanistic interpretability, what is preventing researchers from training two SAEs -- one for general-domain texts and one for the specialized domain? Computationally, it wouldn't be different from the current proposal, as it likewise involves multiple SAEs. Thus, the main application scenario is if the domain isn't known beforehand at inference time.\n\n- A missing baseline would be mixing general-domain text with the particular specific-domain text for full fine-tuning, which is a simple way to mitigate catastrophic forgetting.\n\n- It isn't always clear whether SAE Boost is Pareto-better than the baselines in terms of e.g. the domain-specific and the general-domain EV. This trade-off wasn't investigated in the paper; for example, it is readily foreseeable that full fine-tuning yields different operating points/trade-offs between the domain-specific EV and the general EV depending on how early the fine-tuning is stopped. When plotted on a line, how do these operating points compare to SAE Boost? I think one important point for the paper to make is that SAE Boost achieves the Pareto-optimal boundary compared to baselines, e.g., what would the general EV for full fine-tuning be if it were terminated when the domain-specific EV matched SAE Boost?\n\nEDIT: The rebuttal addresses these concerns.", "questions_to_authors": "- Would the general-domain SAE discover the same features in the interpretability analyses section? The authors hedge this as \"Together, these features show that our approach **can** discover meaningful domain-specific concepts that might be overlooked by general-purpose SAEs,\" but I don't think the analyses definitively show whether that is the case.", "extra_scores": {}}, {"review_id": "uhTR4Zw05F", "reviewer": "Reviewer_MGrV", "summary": "This work proposes a method for domain adaptation of sparse auto-encoders. While SAEs have been used for domain adaptation, or can be re-trained for specific domains, this approach instead improves domain-specific capabilities of SAEs for interpretability without retraining them. It works by modeling the reconstruction error on the target domain and it is found to improve reconstruction quality and LLM cross-entropy without much performance loss. Experiments are conducted with three different domains and partially also with different languages on two different LLMs. An ablation study for training data size is performed and multi-domain adaptation is also explored. Results are compared to the existing SAE Stitching approach and some advantages are found.\n\nUpdated score after author response.", "questions": "", "limitations": "", "rating": 7, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "The approach indeed seems novel, it saves resources and aids interpretability. Domain-specific data is a common problem in real-world applications where there might also not be the resources for full retraining so this seems like an approach that could be beneficial for quite a few real-world use cases.\n\nThe set of experiments is fairly comprehensive with three different metrics, two models, consideration for general domain performance, multi-domain adaptation and information on how much training data is needed for the approach. Examples are also shown (Fig 4). The contribution therefore has good substance.\n\nThe approach's capability for multi-domain adaptation without much of a performance hit stands out compared to previous approaches.", "reasons_to_reject": "The claim that the approach \"discovers meaningful domain-specific concepts\" is illustrated with examples but not really evaluated thoroughly. The chosen evaluation metrics are briefly motivated but they all pertain to next word prediction performance and feature utilization, without any evaluation that directly pertains to interpretability, which is often the end goal of these methods. The work could be strengthened with some human evaluation by a domain expert of the concepts that emerge from the method for at least one of the domains - interpretability in the end involves utility to human interpreters, but there is no human evaluation of specific feature outcomes here. The linguistic typology example is also a nice illustration, though.\n\nThe related work section is very short and doesn't fully contextualize the work. SAE Stitching is briefly mentioned, but where does the Extended SAE baseline come from? Is that also based on related work or did the authors invent it? There is also little discussion of or comparison to alternative approaches such as retraining a SAE on in-domain data for contextualization (even though direct comparison is not needed as it's a different task).\n\nThe conflation of 'languages' with 'domains' is not so obvious to me. There is a clear difference in that languages rely on completely different subtoken features. Perhaps it should be specified why the method works for both of these things and whether there is any difference in how the evaluation metrics work for domain adaptation and for language adaptation.\n\nIt is not clear whether code will be made available for this method.", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "CRJemCAHKz", "reviewer": "Reviewer_F4JZ", "summary": "This paper proposes SAE Boost, a method for augmenting pretrained Sparse Autoencoders (SAEs) with domain-specific information without retraining or modifying the base model. The idea is to train a *residual SAE* to predict the reconstruction error of the original SAE on domain-specific data. At inference time, the residual output is added to the base reconstruction, aiming to recover domain-relevant features that the general-purpose SAE misses.\n\nThe method is simple and modular, and the empirical results are strong: across three domains (Russian, Chemistry, UN Debates) and two models (Qwen and Llama), the residual SAEs improve both explained variance and LLM cross-entropy while having minimal effect on general-domain performance. Multiple residuals can be composed without interference, which is a nice property.", "questions": "", "limitations": "", "rating": 7, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "S1- Adding a residual SAE trained on domain-specific reconstruction error is a lightweight way to enhance feature coverage. It’s modular, avoids catastrophic forgetting, and requires no changes to the base SAE.\n\nS2- The results are consistent: e.g., on Russian texts, EV improves by 59% and CE drops by over 50%. Similar improvements hold across the board. The method generalizes well and doesn’t degrade general performance (>99% retention).\n\nS3-  The authors show you can train multiple residual SAEs independently and combine them at inference. The fact that performance doesn’t collapse under this setup is compelling—it suggests the residuals don’t conflict with one another or the base.\n\nS4- Cosine similarity distributions show that the residual features are less redundant with the base SAE. t-SNE plots show domain-specific features cluster naturally (e.g., Slavic languages). These are soft signals, but they help support the method’s core claim.\n\nS5- The authors evaluate against extended SAEs (both random and most-active feature init), SAE stitching, and full fine-tuning. SAE Boost consistently performs better or similarly on domain metrics, and much better on general-domain retention.", "reasons_to_reject": "W1 - The method is described as “boosting” but has no formal connection to boosting theory. There’s no bias–variance framing, no connection to residual learning in sparse coding, no deeper discussion of why modeling the reconstruction error helps recover meaningful features. It’s clearly effective, but it would be good to understand why.\n\nW2 - The residual SAE omits the decoder bias. The paper says this ensures contributions only when necessary, but this isn’t tested. What happens if the bias is included?  The residual SAE uses `k=5` (vs `k=50` in the base SAE). Why 5? Is this tuned? Does performance improve at 10 or 20? Some sensitivity analysis would go a long way here.  The regularization term λ is also never discussed. It would be useful to know how important the sparsity penalty is in the residual model.\n\nW3 -  The “Extended SAE (most active init)” is not well-defined. What does “most active” mean—highest mean activation? Frequency of firing? Cosine alignment with errors? In SAE Stitching, what threshold is used to select changed features? These details matter for fair comparison.\n\nW4 -  The examples in Figure 4 are good, but mostly anecdotal. Could the authors quantify how many residual features align with domain-specific concepts? E.g., match features to ontology labels or use external probes. The t-SNE plots are visually clean but sensitive to hyperparams like perplexity—no mention of those choices is made.", "questions_to_authors": "Q1-  Is there interference when residuals start overlapping semantically? Does performance saturate or degrade? At what point does the added sparsity outweigh the benefit?\n\nQ2-  Did you try training a residual SAE on out-of-domain data? Would that hurt performance, or just have no effect? \n\nQ3- Is there any explicit mechanism ensuring residual features don’t overlap with base features? Would an orthogonality regularizer help? \n\t\nQ4-  What happens when multiple residual SAEs disagree—e.g., both try to \"correct\" the same error in different ways?", "extra_scores": {}}]

Author rebuttal:

Meta-review: The reviewers, after discussion with the authors, unanimously find this paper interesting and its claims well-supported by experiment. As for pros, this work provides a practical solution to an interesting problem of how sparse autoencoders fail to generalize outside their training distribution – it’s well-evaluated, the method is interesting but pleasingly simple. As for cons, I’m not sure the interpretability evals of the SAEs (including those in the reviewer discussion) are terribly convincing. This is editorializing on my part, but I also would have preferred to see evaluations not on reconstruction error, but on, say, steering or other behavioral control tests – uses of the SAEs themselves on some safety or concept control or otherwise in the new domain.

This is just an extra request – if the authors have room to clarify, I’m rather confused about LLM CE increases of, e.g., 0.935 up to 2.0 or 3.0 – aren’t these gigantic increases when the cross-entropy of the LLM on average is probably between 2 and 4 depending on the text? Are the SAEs doing a terrible job of maintaining the model’s distribution or am I reading this wrong?
Decision: Accept