Sparse Autoencoders Trained on the Same Data Learn Different Features
Authors: Gonçalo Paulo
Nora Belrose
Venue: ICLR 2026 Poster (2026)
Keywords: interpretability
reproducibility
sae
features
Primary area: interpretability and explainable AI
TL;DR: Training SAEs on the same data does not lead to the same features being learned

Abstract: Sparse autoencoders (SAEs) are a useful tool for uncovering human-interpretable features in the activations of large language models (LLMs). While some expect SAEs to find the true underlying features used by a model, our research shows that SAEs trained on the same model and data, differing only in the random seed used to initialize their weights, identify different sets of features. For example, in an SAE with 131K latents trained on a feedforward network in Llama 3 8B, only 30\% of the features were shared across different seeds. We observed this phenomenon across multiple layers of three different LLMs, two datasets, and several SAE architectures. While ReLU SAEs trained with the L1 sparsity loss showed greater stability across seeds, SAEs using the state-of-the-art TopK activation function were more seed-dependent, even when controlling for the level of sparsity. Our results suggest that the set of features uncovered by an SAE should be viewed as a pragmatically useful decomposition of activation space, rather than an exhaustive and universal list of features ``truly used'' by the model.

Published as a conference paper at ICLR 2026
SPARSE AUTOENCODERS TRAINED ON THE SAME DATA LEARN DIFFERENT FEATURES
Gonçalo Paulo, Nora Belrose EleutherAI (goncale o, nora) Releuther.ai
ABSTRACT
Sparse autoencoders (SAEs) a useful tool for uncovering human-interpretable features in the activations of large language models (LLMs). While some expect SAEs find the true underlying features used by a model, our research shows that SAEs trained on the same model and data, differing only in the random seed used to initialize their weights, identify different sets of features. For example, in an SAE with 131K latents trained on a feedforward network in Llama 3 8B, only 30% of the features were shared across different seeds. We observed this phenomenon across multiple layers of three different LLMs, two datasets, and several SAE architectures. While ReLU SAES trained with the LI sparsity loss showed greater stability across seeds, SAEs using the sfate-of-the-art TopK activation function were more seed-dependent, even when controlling for the level of sparsity. Our results suggest that the set of features uncovered by an SAE should be viewed pragmatically useful decomposition of activation space, rather than an exhaustive and universal list of features "truly used" by the model.
INTRODUCTION
Sparse autoencoders (SAEs) are an interpretability tool used to decompose neural network activa- tions into human-underssanddbbe features (Cunningham et al. 2023). They address the problem of polysemanticity, where individual neurons activate in semantically diverse contexts, defying any simple explanation (Arora al. 2018 Elhage al., 2022). SAEs consist of two parts: an encoder that transforms activation vectors into a sparse, higher dimeer-immnsinaal latent space, and a decoder that projects the latents back into the original space. Both parts are trained jointly to minimize recon- struction error. Recently, SAFS have been scaled to sfate-of-the-art large language models, like GPT-4 (Gao et al. 2024) and Claude Sonnet (Templeton al. 2024).
Many researchers hope SAEs can be used to "identify and enumerate over all features in model" (Elhage et al. 2022), which might allow us to check certain safety properties, such as that "a model will never lie" (Olah 2023). These hopes seem to presuppose that there is unique, objective decomposition of a neural network into features, and that SAEs can uncover this decomposition (Smith 2024). In this paper, test this presupposition by measuring the degree to which SAE features depend on the random seed used to initialize their weights.
It is somewhat nontrivial to compare features learned by different SAEs, since the latents of the SAE have no inherent ordering D by randomly permuting the rows of M's encoder matrix, and rearranging the columns of its decoder matrix using the inverse permutation M and M' represent the same function and contain the same features, up to irrelevant permutation symmetry, and yet their weights may look very different.
Given a trained SAE M, we can generate a "shuffled" SAE M'
'In this work we will use "latent" refer to row in an SAE encoder matrix and corresponding row the decoder matrix. By contrast, "feature" is reserved for the concept that a latent may represent. We assume that if two latents are matched by our method, they do refer to some shared concept worth calling a "shared feature," although it may not make
a human. It would be misnomer speak of "shared latents."
"This is generally true for any MLP with elementwise nonlinearities. Indeed, for ReLU networks there is also continuous symmetry: the function represented by the model is unchanged when the pre activation is scaled by and the post-activation is scaled by a The standard training recipe for SAFs eliminates this symmetry, however, by constraining decoder vectors he unit norm (Bricken 2023)

---

Published as a conference paper at ICLR 2026
This means that if independently trained SAEs with different random initializations do learn the same features, we would still expect their latents to be arranged in different orders, and hence we cannot directly compare them.
We can get around this problem by computing a bijective matching of each latent in the first SAE with unique counterpart latent in second SAE. This matching should be optimal in the sense that it maximizes the average "similarity" between the matched latents. This ensures that, in the case where will conclude that the
one SAE has actually been generated by shuffling the latents of another, resulting SAEs do indeed have the same features. Luckily, the |Hungarian algorithm is known to efficiently compute this optimal matching, and has been used to align independently trained net- works before (Ainsworth et al. (023). Empirically. we find that the pairs of latents cluster into two distinct modes: high-similarity hits where a latent has been matched to close counterpart, and low-similarity misses where the latents are relatively unrelated (Figure D. Qualitatively, hits have a strong tendency to occur in semantically similar contexts and share similar explanations, while misses are usually semantically unrelated.
This bimodal distribution allows us to analyze the fruction of features that are shared between two SAEs. For the largest model we tested, Llama 3 8B (Dubey et al. 2024), only 30% of features are shared across both seeds. We find that smaller models, and smaller SAEs trained on the same model, tend to have higher fractions of shared features. We also apply the automated interpretability pipeline of Paulo et al. (024)) to compare the interpretability of hits and misses. We find that misses often quite interpretable, so that an individual SAE training run is likely missing out number of interpretable features.
RELATED WORK
Recent work has found that SAE features are not atomic, in the sense that a "meta SAE" can de- compose them into more specific features (Leask et al. 2025). Relatedly, a feature in a small SAE may be replaced by multiple, more specific features in a larger SAE. In some cases, a more general feature like starts with the letter L appears alongside a specific feature like the token "lion", ", which may prevent the general feature from being active in contexts where intuitively, both the general and the specific feature apply (Chanin et al. 2024). light of these phenomena, some have questioned whether the "flat" design of standard SAEs accommodate the hierarchical structure inherent to human concepts (Ayonrinde et al. 2024). While sparse autoencoders presuppose that neural net- works use linear representations, some research suggests that irreducibly nonlinear features also exist (Engels et al. 2024). If this is true, SAES trained with different random initializations might converge to different ways of "linearizing" the nonlinear features in activation space.
Previous work had found that ReLU SAES trained with an LI sparsity penalty were stable under different seeds (Leask et al. 2025 Braun et al. 2024). By contrast, |Marks al. (2024) found that TopK SAEs could be improved by training two different seeds and forcing them to be "aligned," suggesting that they may not be sufficiently aligned by default. A recent benchmark of feature split- ting showed a convergent result, where JumpReLU and TopK latents had a higher feature splitting rate than ReLU SAES (Karvonen et al.] 2024).
Concurrently with our work, Balagansky et al. (2024) use the Hungarian algorithm to align features from SAEs trained on adjacent layers of the Gemma 2 models (Lieberum et al., (024).. However, we learned in personal communication with the DeepMind interpretability team that the same random seed was used to initialize every SAE in the Gemmascope collection, so Balagansky et al. 2202))ss positive results are likely dependent on this hyperparameter choice.
3 METHODS
We trained several SAEs on the sixth MLP of Pythia 160M (Biderman et al. 2023 with 2 latents over the first 8B tokens of its own training corpus, the Pile (Gao et al. 2020). using the spars fy library (Belrose 2024). We use different random seeds for initialization, but SAES exactly the same data in the same order. The SAES we trained have the functional form
f D (Ep e) d
2

---

Published as a conference paper at ICLR 2026
where z is the output of the MLP, è is the reconstructed output of the MLP, E and are the encoder weight and bias, and D and d are the decoder weight and bias. SAEs are trained to minimize the mean squared error || 212 between the model's output & and the target module output The SAEs are trained using the Adam optimizer (Kingma & Ba 2015), with sequence length of 2049, and a batch size of 32 sequences. We also trained SAES in SmolLM (Allal al., 2024), GPT-
2 (Radford et al. 2019) and Llama 3.1 8B (Dubey et al. 2024). The GPT-2 SAES where trained on
OpenWeTTex (Gokaslan et al., 2019), the LLama 3.1 SAES on RedPajama V2 (Computer 2023).
and the SmolLM SAES on Fineweb-edu Lozhkov et al. (2024). All SAEs had a different number of latents, but all had ratio of 36 between the number of latents and the dimension of the input.
The exact cost function that we feed into the Hungarian algorithm is a hyperparameter in our method. In particular, we can try to maximize the average cosine similarity of the encoder vectors or the decoder vectors. While the decoder is initialized using the transpose of the encoder, the two matrices can diverge during SAE training. To err on the side of conservatism, we compute one matching using the encoder and another matching using the decoder. We consider feature to be shared if its latents are paired together in both the encoder and decoder matchings, and in both of these matchings they have cosine similarity of 0.7 greater. If a latent does not refer to shared feature, we will call it unpaired.
More specifically, we use SciPy's 's (Virtanen et al. 2020 function linear sum assi gnment to find the decoder permutation P dec that maximizes tr(PT
dec D2) and the encoder permutation Pene that maximizes tr tr En E ) where the columns of D2 and the rows of E1, Eg have been standardized to unit norm beforehand. Shared features correspond pairs of latents (i.j) where the hh columns in P Pone and P cac (or equivalently, the jth rows) are equal. Additionally, the matched cosine similarities (P des Df
E Ef E2)ai are required to be greater than 0.7. By this definition, only 42 % of latents are shared across our two independently trained SAES. Interestingly, the fraction of shared features is essentially unchanged if we were to use maximum cosine similarity (see below) in licu of the Hungarian algorithm to label features as shared (Fig- ure A2
D2) and (P?
Maximum cosine similarity. Prior work has measured the similarity of independently trained SAEs using the mean maximum cosine similarity (Braun al., [024 Figure 3b). Specifically, for each latent in the first SAE we find the maximum cosine similarity between itself and all latents in the second SAE. The average of these maxima is the overall similarity score. This metric is simple, but it has the downside that it does not yield a bijective matching: many latents in the first SAE may be mapped one latent in the second SAE. We compare the matched cosine similarity produced by the Hungarian algorithm to the maximum cosine similarity for each latent in Figure [A3 observing that while for some latents the max cosine similarity higher than the matched cosine similarity, the vast majority have the same value for both metrics, suggesting that the Hungarian algorithm has chosen to match most latents with their nearest neighbors. While we think the Hungarian matching approach is more principled and use it in the rest of this paper, we do find that empirically the difference between these two approaches is small.
Interpretabilityy We use the automated interpretability pipeline released by Paulo et al. 2024 to generate explanations and scores for the SAE latents. For each latent, representative samples of its activations are sampled and shown to an LLM, in our case Llama 3.1 70b Instruct (Dubey et al., 2024]. which is told to generate succinct explanation that summarizes the activations. The LLM is shown 40 examples sampled from the whole activation distribution.
After explanations are generated for each latent, they are scored. We also use the pipeline from Paulo et al.] (2024) for this process. We use both fuzzing and detection to score the latents. To compute the detection score, Llama 3.1 70b Instruct is given the explanation of the latent and a set of examples. The LLM then has to decide which examples activate the latent and which don't 't using the explanation that was given. At the end the detection score of the latent is given by the balanced accuracy, which in case reduces to accuracy because we use the same number of activating and non activating examples. The fuzzing score is computed with a similar protocol. but LLM is instead tasked to identify if a given highlighted token is active given the explanation.
3

---

Published as a conference paper at ICLR 2026
1.0
Encoder-Decoder pairs
Different Equal
Encoder
0.2
0.0
0.2
0.4
0.6
0.8
Decoder alignment
Figure 1: Cosine similarities of latents from SAE 1 with their counterparts in SAE 2. Both SAES have 32K latents, and are trained on the sixth ML. of Pythia 160M. Contour lines are regions of equal density according to kernel density estimation. We color each SAE 1 latent depending on whether the encoder-based and decoder-based matchings agree on which counterpart it should get.
4 RESULTS
On this pair, we find that the distribution of matched cosine similarities has two modes: high- similarity hits and low-similarity misses (Figure [ Overall, cosine similarities for encoder and decoder vectors are strongly correlated. We observe that in cases where the encoder and decoder matchings disagree (colored in orange), the cosine similarity usually low for both matchings, whereas similarities are higher when the encoder and decoder matchings agree (colored in blue).
4.1 ASYMPTOTIC TREND
We now consider seven more SAEs with the same data order, but with seeds different from the first two, yielding total of nine independently trained SAEs. We first run the Hungarian algorithm 36 times, one for each pair of SAEs. Then for each integer & from 2 to 9, iterate over all combinations of SAEs of size k. and for each combination, we run the following experiment k different times, each time using a different SAE as the "base SAE." We use each of the k matchings of the base SAE with different SAE within this combination to compute a binary mask classifying each latent as a hit or a miss, using the definition from Section 3
We say that a latent "only in the base SAE" if an miss according to k: masks. Then, with respect given base SAE, compute the proportion of all latents that are only in the base. Finally, we average the proportions generated by running this experiment & x (?) times, one for each combination of SAEs and each possible base SAE in each combination. When k 9, we find that number of latents found in only one SAE is reduced to about 35% The results of these experiments are plotted Figure AI
these binary
The number of latents found in only SAE decreases slowly the number of seeds increases Figure AI Our results indicate that when training a small number of SAEs, a certain number of found that power law with offset term fits the data significantly better than one without the offset.
latents will never find counterpart
To generate Figure
fix Seed 1 as the base SAE, and color latents based on the number of matchings involving Seed 1) in which they find a counterpart. We
matchings (out of the find that the latents that most frequently fire in the first SAE are the ones that have counterpart in all
4

---

Published as a conference paper at ICLR 2026
0.0
0.2
0.4
0.6
0.8
1.0
107
107
10a
10
Number of occumences
10a
10a
10
10a
10a
Shared with:
10a
SAEs (Orphan)
10a
SAEs
SAES
101
SAEs
SAEs
SAEs
SAES
10a
SAEs SAES
0.8
Similarity
Figure 2: Latent similarity vs. firing frequency. We plot the cosine similarity between matched latents, vs. how often the latent fires in the base SAE. The similarity of each latent averaged over all the matched latents of different seeds. The histograms in this figure are stacked, and the histogram of number of occurrences has a log-scale from to 500, to highlight the few latents that rarely fire or that fire frequently, and linear-scale from 500 to 4000. Latent occurrences were collected over 10M tokens of the Pile, the same dataset that the SAEs were trained on.
eight SAEs, and that the ones that most infrequently fire are the ones that do not have a counterpart in any other SAE, see Figure 2 Interestingly, significant number of misses have higher firing rate on average than latents with counterparts all SAEs. In fact, as the average alignment between latents increases, the firing frequency seems to decrease. This non monotonic relationship is better observed in Figure A6
4.2 ARE THE UNPAIRED LATENTS INTERPRETABLE?
In this section, we generate explanations for latents of seeds of a 2 latent SAE and score them using detection and fuzzing scoring (Paulo et al. 2024), evaluating the explanation over 100 active sequences and 100 non-active sequences. The average score of the explanations of the 32K SAES is 0.72, with only 25% of explanations having score lower than 0.62, and only 25% having a score better than 0.8.
Plotting the distributions of scores conditioned the number of SAEs that "shared" that latent reveals that features shared across higher number SAE seeds have on average higher inter- pretability scores. In spite of this, a significant fraction of latents found only scoring latents. Plotting the scores of the latents the two seeds mentioned above, we find that the most of the latents that have low similarity have either low or average score, Figure left. Some latents have a average cosine similarity < 0.7 and high scores, reinforcing the observation that some interpretable latents can be missing from any given seed, see Table [orr some examples.
SAE, have high
4.3 ABLATIONS
We performed several ablation studies investigate how results depend on the hyperparameters used to train the SAE, including the number active latents k the total number latents, the number of tokens used for training, and the SAE architecture (TopK, Gated, ReLU).

---

Published as a conference paper at ICLR 2026
1.3
S4F%
1003
In SAES
0.3
5 SAES
Seed (bllaced accuras)
Nu intertss
Avergg alsmment
SAES
0.8
600
SAES
0.7
400
202
92
3
0.0
0.5
3.7
0.9
Seed (bslenced accurscy)
Figure 3: Interpretability of unpaired latents. Distribution of scores of different latent explana- tion conditioned on the number of SAEs that latent can be found on. On the right we compare the scores of 5k explanations of matched latents of different SAE seeds. We see that most of the latents that have low alignment either have a low score or have a higher score in one of the SAFs than in the other.
Alignment
Seed I
Seed 2
10
Abbreviated country name in United States Supreme Court case citations. (0.865)
single character or a small group of characters embedded within larger word, often in non-Englist language context (..). (0.56)
0.27
Definite articles and other words com- monly used formal and legal lan- guage, such disclaimers, licensing terms, and court documents. (0.91)
Punctuation marks short words con- necting or separating clauses, (...) and sometimes serving conjunctions prepositions. (0.46)
0.44
Abbreviated geographical or institu- tional references, usually in the context of legal citations. (0.85)
Punctuation marks. (0.49)
0.75
Percent symbols marking numerical values representing proportions or rates. (0.95)
A percentage symbol denoting the pro- portion a quantity, (...) and usually in the form of numerical value followed by the symbol. (0.97)
0.97 Adverbs that express frequency, such as "often', 'sometimes', (...), used to in- dicate the occurrence or tendency of an event or action. (0.94)
Adverbs indicating fiequuency, such as "often', 'frequently', (...) are used to describe the regularity likelihood of an event or situation. (0.99)
Table 1: Unpaired latents can have high scoring explanations. We selected explanations of pairs of latents shown in Fig. 3 Each explanation is shown alongside its detection score (Paulo et al.) 2024), number in [0, 1] measuring explanation quality, in parentheses. We select latents from bins of alignment by maximizing the score of both explanations the cosine similarity between latents is 0.7 and by maximizing the score of the explanation seed I and minimizing the score on seed 2 the cosine similarity is < 0.7. Ellipsis added to some explanations for brevity. This choice was made to capture latents that had good explanations in seed but not matched in seed The latent pairs are (12314, 6024), (21463, 3361), (5888, 6649), (14931, 5456) and (1817, 66).
We find that increasing the number of SAE latents, all else being equal, decreases the overlap be- tween different seeds, see Figure 4 Increasing the number of active latents, by increasing the k for TopK SAES, also decreases the overlap, while the training time increases the overlap between latents. These results seem to indicate that what the seed dependence is not mainly due to feature ab- sorption, as absorption increases when sparsity is decreased (Karvonen et al. 2024), and the model trained for longer, while it does increase when the number of latents increases. We have found

---

Published as a conference paper at ICLR 2026
1.0
Fractonn shared latenns
0.6
0.2
32
768 4k
32k
131K
128 256 0.8
Number latents
Training (Billion)
Figure 4: Dependence of overlap of Pythia-160M SAE size, number of active latents and training time. the left we see that the fraction of shared latents decreases with the increase of the number of latents. Middle shows that increasing the number of active latents, by increasing the value of & for the TopK activation function, also decreases the overlap. On the right, training time increases the alignment of different SAE seeds. Unless otherwise indicated, each SAE has 2 latents and was trained on the output of the sixth layer MIP of Pythia 160M, on the first 8B tokens of its training corpus, the Pile.
evidence of feature absorption the MLP SAEs trained, but that may be due to the fact that the current metric not tuned to find absorption on MLP SAEs, as it was mostly used on residual stream ones.
3.9
Pacton latee
Layar
SMaLLM
GPT2
Madel
Pythis
Architecture
Figure 5: Dependence of overlap on SAE hyperparameters. On the right we see the how the fraction of shared features for Pythia-160M SAE depends the layer and the number of latents. In the middle we compare SAEs with the same expansion factor, 36, trained on different models and positions. On the right we compare SAEs trained on GPT2 using different activation functions and architectures.
The overlap between different seeds remains almost constant across the middle layers of the model, being lower for the earlier layers and the last layer (Figure (5). On SmoILM (Allal al. 2024)
and GPT2 the MLP latents have more overlap between seeds than the residual stream ones, but the same is not true for Pythia. Previous work had found that a large number of latents (> 90%) where shared between GPT2 seeds (Leask et al.) 2025 Braun et al.) 2024), although those numbers where measured for SAEs with smaller numbers of latents than ours, and using a different architecture (ReLU instead of TopK). Indeed we find that standard and Gated SAEs (Rajamanoharan et al. 2024) trained with LI loss have a larger overlap between latents. We also find that the overlap is much lower for the Llama 8B TopK SAES, which have more latents but the same expansion factor.
In Figure A4 we compare the matched cosine similarity produced by the Hungarian algorithm to the maximum cosine similarity, showing that these strongly correlated. This shows that our results are not strongly dependent on the choice of method used to compare latents from independently trained SAES.

---

Published as a conference paper at ICLR 2026
5 CONCLUSION
Our results are further evidence for the idea that SAES do not uncover a "universal" set of features. Different random initializations can lead different sets of features being found, and SAEs seem to diverge, rather than converge, with increasing scale. We think feature discovery is best viewed compositional problem, wherein look for useful ways of cutting up the activation space into categories, and these categories can themselves be cut up into further categories, hierarchically.
Mathematically, the lack of universality observe here is due the nonconvexity of the SAE loss function, which gives rise to many local optima. The TopK activation function is also discontinuous, which may exacerbate the problem of nonconvexity and explain why Topk SAEs suffer from seed instability more than ReLU SAEs. One might have expected priori, however, that different local optima would have more feature overlap than we found in this study.
CODE AVAILABILITY.
Code to perform the Hungarian alignment of SAE seeds as well as all the analysis done in this work can be found here and all the trained SAE checkpoints can be found here
AUTHOR CONTRIBUTIONS
Gonçalo Paulo had the idea of investigating the overlap in features learned by SAES trained with different random seeds, trained the SAEs, and ran all the experiments. Gonçalo wrote the first draft, and Nora Belrose extensively edited the draft into final version.
ACKNOWLEDGEMENTS
Gonçalo and Nora are funded by grant from Open Philanthropy. We thank Coreweave for comput- ing resources.
REFERENCES
Samuel Ainsworth, Jonathan Hayase, and Siddhartha Srinivasa. re-basn:: Merging models mod- ulo permutation symmetries. In The Eleventh International Conference on Learning Representa- tions, 2023. /tL/tttss///peeerviii
net wtwoooummddCCmmmmmmnn
Loubna Ben Allal, Anton Lozhkov, Elie Bakouch, Leandro von Werra, and Thomas Wolf. Smollm blazingly fast and remarkably powerful, 2024.
Sanjeev Arora, Yuanzhi Li, Yingyu Liang, Tengyu Ma, and Andrej Risteski. Linear algebraic struc- ture of word senses, with applications to polysemy. Transactions of the Association for Compu- fational Linguistics, 6:483-495, 2018.
Kola Ayonrinde, Michael T Pearce, and Lee Sharkey. Interpretability as compression: Reconsidering sae explanations of neural activations with mdl-saes. arXiv preprint arXiv:2410.11179, 2024.
Nikita Balagansky, Ian Maksimov, and Daniil Gavrilov. Mechanistic permutability: Match features
across layers. arXiv preprint arXiv:2410.07656, 2024.
Nora Belrose. Sparsify repository. GitHub repository, 2024. URL htttss:/github...o com/
EleutherAI/ sparrsiispprsfy/////
Stella Biderman, Hailey Schoelkopf, Quentin Gregory Anthony, Herbie Bradley, Kyle O'Brien Eric Hallahan, Mohammad Aflah Khan, Shivanshu Purohit, USVSN Sai Prashanth. Edward Raff, al. Pythia: A suite for analyzing large language models across training and scaling. In International Conference on Machine Learning, pp. 2397-2430. PMLR, 2023.
Dan Braun, Jordan Taylor, Nicholas Goldowsky-Die and Lee Sharkey. Identifying functionally important features with end-to-end sparse dictionary learning, 2024. URL Balhhtps: : 3//arxiv. org/ 2405. 12241
8

---

Published as a conference paper at ICLR 2026
Trenton Bricken, Adly Templeton, Joshua Batson, Brian Chen, Adam Jermyn, Tom Conerly, Nicholas L Turner, Cem Anil, Carson Denison, Amanda Askell, Robert Lasenby, Yifan Wu, Shauna Kravec, Nicholas Schiefer, Tim Maxwell, Nicholas Joseph, Alex Tamkin, Karina Nguyen, Brayden Mcl.ean, Josiah E Burke, Tristan Hume, Shan Carter, Tom Henighan, and Chris Olah. Towards monosemanticity: Decomposing language models with dictionary learning. Trans- former Circuits Thread, 2023. URL https: //trans////atttte htttraanssormrrrr www.anaooooerrrrrrists..... puup
pub/2023/
monosemantic features Published October 4, 2023.
David Chanin, James Wilken-Smith, Tomáš Dulka, Hardik Bhatnagar, and Joseph Bloom. A is for absorption: Studying feature splitting and absorption in sparse autoencoders. arXiv preprint arXiv:2409. 14507, 2024.
Together Computer. Redpajama: open dataset for training large language models, 2023. URL http:///g..hhaa.ooooooooo/ccc/r/
Dataa
Hoagy Cunningham, Aidan Ewart, Logan Riggs, Robert Huben, and Lee Sharkey. Sparse autoen- coders find highly interpretable features in language models. arXiv preprint arXiv:2309.08600 086000 2023.
Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang. Angela Fan, al. The Ilama 3 herd of models. arXiv preprint arXiv:2407.2 1783, 2024.
Nelson Elhage, Tristan Hume, Catherine Olsson, Nicholas Schiefer, Tom Henighan, Shauna Kravec, Zac Haddield-oddss Robert Lasenby, Dawn Drain, Carol Chen, al. Toy models of superposi- tion. arXiv preprint arXiv:2209. 10652, 2022.
Joshua Engels, Eric J Michaud, Isaac Liao, Wes Gurnee, and Max Tegmark. Not all language model features are linear. arXiv preprint arXiv:240S. 14860, 2024.
Leo Gao, Stella Biderman, Sid Black, Laurence Golding, Travis Hoppe, Charles Foster, Jason Phang. Horace He, Anish Thite, Noa Nabeshima, et al. The pile: An 800gb dataset of diverse text for language modeling. arXiv preprini arXiv:2101.0000 2020.
Leo Gao, Tom Dupré la Tour, Henk Tillman, Gabriel Goh, Rajan Troll, Alec Radford, Ilya Sutskever, Jan Leike, and leffrey Wu. Scaling and evaluating sparse autoencoders. arXiv preprint arXiv:2406.040993 04090, 2024.
Aaron Gokaslan, Vanya Cohen, Ellie Pavlick, and Stefanie Tellex. Openwehtext corpus. 2019.
http:
//Sky]lon007.gieee http://....oopeeeeeeeetetooooooe
Adam Karvonen, Can Rager, Johnny Lin, Curt Tigges, Joseph Bloom, David Chanin, Yeu-Tong Lau, Eoin Farrell, Arthur Conmy, Callum McDougall, Kola Ayonrinde, Matthew Wearden, Samuel Marks, and Neel Nanda. Sacbench: A comprehensive benchmark for sparse autoencoders, 2024. Accessed: 2025-01-17.
URL lt www. wwwroneedia.aa..oommomoo
http:p:::ww....euupppcaaaaaaaao. org sae- beaeeeebbecciiiie
Diederik Kingma and Jimmy Ba. Adam: A method for stochastic optimization. ICLR (Poster), 2015. URL/attp:
org abs /11...11.... 6980
Patrick Leask, Bart Bussmann, Michael Pearce, Joseph Bloom, Curt Tigges, Noura Al Moubayed, Lee Sharkey, and Neel Nanda. Sparse autoencoders do not find canonical units of analysis. arXiv preprint ar arXiv:2502.0448 2025.
Tom Lieberum, Senthooran Rajamanoharan, Arthur Conmy, Lewis Smith, Nicolas Sonnerat, Vikrant Varma, János Kramár, Anca Dragan, Rohin Shah, and Neel Nanda. Gemma scope: Open sparse autoencoders everywhere all at once gemma 2024. URL http::///rxlvvorr//aa...
2448.05147
Anton Lozhkov, Loubna Ben Allal, Leandro von Werra, and Thomas Wolf. Fineweb-edu: the finest collection of educational content, 2024. URL
co..atssttss
Luke Marks, Alisdair Paren, David Krueger, and Fazl Barez. Enhancing neural network inter- pretability with feature-aligned sparse autoencoders. arXiv preprint arXiv:2411.01 1.00200, 2024.

---

Published as a conference paper at ICLR 2026
Chris Olah. Interpretability
dreams.
Transformer Circuits Thread, 2023.
URL
btttps://rranssfooeee---c /ttpa//araaaaaarra.aaaaa//////ea
Index.htm.
Published May 24, 2023.
Gonçalo Paulo, Alex Mallen, Caden Juang, and Nora Belrose. Automatically interpreting millions of features in large language models. arXiv preprint arXiv:2410.13928, 2024.
Alec Radford, Jeff Wu, Rewon Child, David Luan, Dario Amodei, and llya Sutskever.
Language models are unsupervised multitask learners. 2019. URL https://api.
Senthooran Rajamanoharan, Arthur Conmy, Lewis Smith, Tom Lieherum, Vikrant Varma, János Kramár, Rohin Shah, and Neel Nanda. Improving dictionary learning with gated sparse autoen- coders. arXiv preprint arXiv:2404. 16014 2024.
Lewis URL
The strong feature hypothesis could be wrong, 2024. the tbe.ssttoo--uuuuu---oooeee-e-ee-
Smith.
Accessed: 2025-01-17.
Adly Templeton, Tom Conerly, Jonathan Marcus, Jack Lindsey, Trenton Bricken, Brian Chen, Adam Pearce, Craig Citro, Emmanuel Ameisen, Andy Jones, Hoagy Cunningham, Nicholas L Turner, Callum McDougall, Monte MacDiarmid, C. Daniel Freeman, Theodore R. Sumers, Edward Rees, Joshua Batson, Adam Jermyn, Shan Carter, Chris Olah, and Tom Henighan. Scaling monosemanticity: Extracting interpretable features from claude sonnet. Trans- former Circuits Thread, 2024. URL
Sca iin mwnosemant wwtpentiiiiitttiiinii html
Pauli Virtanen, Ralf Gommers, Travis E. Oliphant, Matt Haberland, Tyler Reddy, David Courna- peau, Evgeni Burovski, Pearu Peterson, Warren Weckesser, Jonathan Bright, Stéfan van der Walt, Matthew Brett, Joshua Wilson, K. Jarrod Millman, Nikolay Mayorov, Andrew R. J. Nel- son, Eric Jones, Robert Kern, Eric Larson, Carey, Îlhan Polat, Yu Feng, Eric W. Moore, Jake VanderPlas, Denis Laxalde, Josef Perktold, Robert Cimrman, Ian Henriksen, E. A. Quintero, Charles R. Harris, Anne M. Archibald, Antônio H. Ribeiro, Fabian Pedregosa, Paul van Mul- bregt, and SciPy 1.0 Contributors. SciPy 1.0: Fundamental Algorithms for Scientific Computing in Python. Nature Methods, 17:261-272 2020. doi: 10. 1038/43//1111 592-019-0686-1
10

---

Published as a conference paper at ICLR 2026
0.7
Proportion teatures only in one SAE
0.5
0.3
0.4-
0.3
0.2
0.1
0.0
Number
Figure Al: Dependence of the number latents found only in the base SAE on the number of seeds. We consider latent X in SAE A to be "shared" in SAE B if and only if X is matched to latent in B with which it has cosine similarity greater than 0.7 according to both the encoder and decoder weights. To generate this plot select "base" SAE and compute its overlap with all the other seeds, then we average over all different base seeds.
Foccion of overlapping latents
0.8
0.6
0.4
0.2
Threshold cosine similarity
Equal Threshold
0.0
Max cosine similarity
0.0
0.4
0.5
0.8
1.0
Threshald
Figure A2: The average alignment of points with equal decoder and encoder indices is 0.72 and of the ones that have different indices 0.33. the right, we plot the fraction of latents that are considered shared between 2 SAEs control threshold. We decide to threshold of 0.7 on both the encoder and decoder alignment to decide a latent is shared between two SAEs.
11

---

Published as a conference paper at ICLR 2026
1.
Equal Dffirent
Nar cosin Smilrity
0.4
0.0
0.2
0-4
0.6
0.8
1.0
Matched oosine similnity
Figure A3: Cosine similarity of latents when paired with the Hungarian algorithm when using max cosine similarity. The majority of latents that have the same counterpart latent in both the encoder and decoder matchings using the Hungarian algorithm have a similar alignment as they had been aligned with maximum cosine similarity. The latents which have a higher cosine similarity pair when using max cosine similarity are paired with latent that already had pair.
0.8
0.4
Mean cosine similanity decader
Mean casine similarty decoder
Mean cos similarity decoder
0.2
similanity encuder
casine zimil artyy
Mean cos similarity encoder
Maxi cosine similarity decoder
Max cosine similarity decode
cosine similarity
larity
similarity
Max cosine similarity encodor
768
80
Number latents
Active lacents
tokens (Billion)
Figure A4: Dependence of mean matched and mean max cosine sim of Pythia- 160M SAE on different hy perparaaaeeerss. On the left we see that the average cosine similarity latents decreases with the increase of the total number of latents. Middle shows that increasing the number of active latents also decreases the average cosine similarity. On the right, training time increases the average cosine similarity of different SAE seeds. We observe that the mean matched and max cosine similarity have very similar trends, with max cosine similarity being just slightly higher. On panels all panels a 32768 latent SAE trained on the output MLP of Pythia 160M, for 8B tokens, except when the panel changes one of these conditions.
12

---

Published as a conference paper at ICLR 2026
Mean Cosine Similarity by Feature Frequency
Mean Cosine Similarity
0.6
0.2
10a
10
102
10a
10a
10
10
107
Feature Frequency
Figure A6: Frequency dependence of similarity between features. We find a non monotonic dependence between frequency and cosine similarity. The different colors represent the different seeds. We always use the same SAE to compute the feature frequency.
13


Reviews:
[{"review_id": "3teUuIJ5Kx", "reviewer": "Reviewer_XSz3", "summary": "This study studies the stochasticity of which latents are found by an SAE training process. They train many different sets of SAEs which differ only in random seeds, but otherwise vary model, layer, dataset and architecture. They discuss methods for matching features between features and choose the Hungarian algorithm for generating a bijective mapping between features. Their core results show low matching between features, which may be partially mediated by architecture. Smaller SAEs show greater overlap. It is suggested that these results provide further evidence that SAEs do not uncover a \"universal set of features\".", "questions": "- What was the most surprising result in your experiments? Where are you most changing the picture painted by the *recent* literature? \n- Did you expect feature absorption to be driving stochasticity in which features are found? What other factors might be driving this?", "limitations": "", "rating": 2, "confidence": 5, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "- Quality: In some respects, this investigation is quite thorough, checking how factors including model, layer, architecture or number of latents effect the size of the common set of features found when seeds are varied. \n- Clarity: The paper is generally well written and fairly clear. Graphs are of a reasonable quality.", "weaknesses": "- Originality: The question is not new, and even very early SAE interpretability work includes examples of training multiple SAEs, varying seeds and inspecting the universality of features found (See Towards Monosemanticity - Bricken et al). The Hungarian algorithm even as applied to this problem is not novel (as mentioned by the paper). \n- Significance: Moreover, the idea that SAEs do not find a single basis has been addressed in the literature such as (\"Sparse Autoencoders Do Not Find Canonical Units of Analysis\" - Leask et al). \n\nHow could the work be improved?\n- Since the work is sound, the areas for improvement are likely around dealing with the \"so what?\" element or the \"but why?\" element. \n- For \"so what?\" - how does the stochasticity / arbitrariness of which features are found effect the usefulness of SAEs? Can multiple SAEs varying only in their seeds be more useful if used in combination? \n- For \"but why?\" - The use of toy models might be useful. What kinds of toy set-ups result in SAE training procedures that are non-deterministic? Possibly Toy models which include underlying features that are nested or overlap such that there is no clear ideal solution. This line of work would be more interesting if it inspires better SAE design. \n\nAdditionally, the author could consider Matrioshka SAEs which may be more deterministic.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "GzQoJC66tO", "reviewer": "Reviewer_QtLY", "summary": "This paper investigates the stability of Sparse Autoencoders (SAEs) by analyzing whether independently trained SAEs (differing only in random seeds) discover the same features. The authors propose using the Hungarian algorithm to compute optimal bijective matchings between latents of different SAEs, allowing for a principled measure of feature correspondence. Experiments across multiple LLMs and SAE variants show that a surprisingly small fraction of SAE features are shared across seeds; overlap depends on SAE size, activation type, training time, and layer position. The paper argues SAEs produce pragmatically useful but non-universal decompositions of activation space.", "questions": "Maybe more theoretical analysis or intuition in SAE training should be explored in future work.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 3, "strengths": "1. Principled and novel matching approach.\nThe use of the Hungarian algorithm to compute the optimal matching between SAEs trained with different seeds is both reasonable and novel. The paper also validates its effectiveness by showing that matched pairs under the Hungarian algorithm exhibit consistency with cosine similarity, demonstrating that the method produces meaningful alignments.\n2. Interesting empirical discoveries from multi-seed experiments.\nThe authors conduct matching experiments across nine independently trained SAEs and uncover several intriguing empirical patterns, such as the relationship between latent firing rate and matching frequency, and the counterintuitive finding that a significant number of “misses” (unmatched features) have high firing rates. These observations are fresh and thought-provoking for understanding SAE behavior.\n3. Interpretability analysis provides practical insight.\nBy examining the interpretability of unpaired latents, the paper shows that each SAE training run can miss certain high-quality, interpretable features. This finding is practically important and suggests new directions for improving SAE training stability and coverage.\n4. Comprehensive ablation study.\nThe ablation experiments—covering latent size, active latent number, architecture type, and training time—are thorough and enhance the credibility of the results. The consistent trends across these variations further support the paper’s main claims.", "weaknesses": "I couldn't identify serious weaknesses. However, I think analysis somehow remains surface-level.\n\nWhile the experiments are extensive and clearly presented, the overall analysis remains somewhat superficial. The paper mainly reports empirical correlations (e.g., overlap rates, firing frequency trends) without delving into why such phenomena occur or what mechanisms underlie the observed variability across seeds. For instance, there is little attempt to characterize which properties of latents (e.g., frequency, selectivity, activation entropy) predict stability. A deeper or more mechanistic investigation would significantly strengthen the contribution.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "xB3azUyvR0", "reviewer": "Reviewer_rLR5", "summary": "This paper studies the stability of SAE features learnt on same model & data but different random seeds. The authors compute a bijective matching of features in two SAEs by maximizing the average cosine similarity of encoder vectors or decoder vectors in Hungarian algorithm. The results show that only 30% of features are shared between two SAEs in Llama 3 8B. Interpretability of non-shared features are statistically worse than shared features.", "questions": "1. (As in Weaknesses 2) As shown by Wang et al. 2024, shared features may largely be features of low complexity, e.g. single token features. Do the interpretability gap dominants by the complexity gap?\n2. What does a single point refer to in Figure 2? Does it refer to a pair of base SAE feature and orther SAE feature, so one base SAE feature will be represented by 8 points in the figure?\n3. What is Aligned and Encoder-Decoder equal in Figure 5?\n4. The conclusion that L1 is better than TopK contradicts recent work [1] where their experiments show the feature consistency in TopK is significantly higher than standard SAEs. Can you provide any explanation on what possible reason leads to the difference?\n\n[1] Song, X., Muhamed, A., Zheng, Y., Kong, L., Tang, Z., Diab, M., Smith, V. \\& Zhang, K. Position: Mechanistic Interpretability Should Prioritize Feature Consistency in SAEs. **Mechanistic Interpretability Workshop At NeurIPS 2025**. (2025), https://openreview.net/forum?id=d9ACURK6bI", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The paper is well-written. All experimental settings are clarified in detail. Results are easy to follow.\n2. The conclusion they reached, that SAE trained on the same data learn different features, is verified in various models, including SmolLM, GPT-2 and Llama 3.1 8B, respectively on their training set. The Impact of hyperparameter selection is extensively discussed in Section 4.3.\n3. Interpretability of shared features versus non-shared features is measured.", "weaknesses": "1. Although this paper conducts a wide range of experiments on the phenomenon, the research question is still a bit simple. The main text focuses on the phenomenonology and empirical studies. The authors do not discuss 1) the underlying mechanism behind the instability of features, 2) the impact of instability of features (how the unpaired latents will harm the interpretability ability of SAEs), or 3) the potential method to consistently discover a universal set of features.\n2. While a gap is showed in the interpretability of shared features versus non-shared features, the potential causes and its meaning is unclear.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "l2ujWnlHY3", "reviewer": "Reviewer_66RK", "summary": "This paper engages with the question of wether SAEs learn some fundamental set of features, which is a common theme in the interpretability literature. The paper studies this using feature alignment (an approach from the literature), and runs evals across multiple models, saes, and datasets. \nIn particular, the paper checks feature alignment across different seeds, and it checks the interpretability of aligned/not features.\nIt also studies the sensibility of alignment to hyperparameters such as the sae expansion factor and k in topk.", "questions": "Have you tried experiments on small models with extremely large values of k or the sae expansion factor (to see if the alignment eventually converges)?\n\nHave you tried human interp?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 4, "strengths": "This paper is rigorous and works across different families of models, saes, and datasets.\nThe paper studies a fundamental question to the development of SAEs: wether they learn a universal set of features or not, and it makes a meaningful contribution to the discussion.\nThe findings are very interesting (and I believe will be interesting to the community), the highlights were:\n- features learned across different seeds are significantly different\n- the interpretable features were more aligned\n- increasing the number of latents reduced alignment, and so does increasing k in topk\n\nThis leads to interesting places, for ex:\n- maybe the linearly encoded learnable interpretable features only represent a fraction of the total loss (see dark matter sae paper)\n- maybe the SAEs have a massive tail of niche features making up a significant portion of the loss, and pick different ones initially due to the seed, but asymptotically we would see all features aligning", "weaknesses": "Fundamental:\n- the paper could use some human interpretability besides fuzzing, as auto interp can be misleading \n\nStyle:\n- it is not customary to cite personal communication with another team (about gemma scope in this case) as supporting evidence in an argument\n- typo at line 209 \"an miss\"", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)