Compute Optimal Inference and Provable Amortisation Gap in Sparse Autoencoders
Authors: Charles O'Neill
Alim Gumran
David Klindt
Venue: ICML 2025 poster (2025)
Keywords: sparse autoencoders
large language models
dictionary learning
Primary area: deep_learning->generative_models_and_autoencoders
TL;DR: Current sparse autoencoder architectures face fundamental limitations that can be overcome with more powerful encoders

Abstract: A recent line of work has shown promise in using sparse autoencoders (SAEs) to uncover interpretable features in neural network representations. However, the simple linear-nonlinear encoding mechanism in SAEs limits their ability to perform accurate sparse inference. Using compressed sensing theory, we prove that an SAE encoder is inherently insufficient for accurate sparse inference, even in solvable cases. We then decouple encoding and decoding processes to empirically explore conditions where more sophisticated sparse inference methods outperform traditional SAE encoders. Our results reveal substantial performance gains with minimal compute increases in correct inference of sparse codes. We demonstrate this generalises to SAEs applied to large language models, where more expressive encoders achieve greater interpretability. This work opens new avenues for understanding neural network representations and analysing large language model activations.

Compute Optimal Inference and Provable Amortisation Gap in Sparse Autoencoders
Charles O O'Neill'
Alim Gumran
David Klindt 3
Abstract
sis" Elhage et al. 2022). which posits that neural networks represent interpretable features in a linear manner using non-orthogonal directions in their latent spaces. Building this idea, researchers have shown that individual features can be recovered from these superposed representations using sparse autoencoders (Bricken et al. 2023 Cunning ham et al. 2023). These models learn sparse and overcom- plete representations of neural activations, with the resulting sparse codes often proving to be more interpretable than the original dense representations (Cunningham et al. 2023 Elhage et al. 2022 Gao et al., 2024).
A recent line of work has shown promise in us- ing sparse autoencoders (SAEs) to uncover in- terpretable features in neural network representa- tions. However, the simple coding mechanism in SAEs limits their ability to perform accurate sparse inference. Using com- pressed sensing theory, we prove that an SAE en- coder is inherently insufficient for accurate sparse inference, even in solvable cases. We then de- couple encoding and decoding processes to em- pirically explore conditions where more sophis- ticated sparse inference methods outperform tra- ditional SAE encoders. Our results reveal sub- stantial performance gains with minimal compute increases correct inference of sparse codes. We demonstrate this generalises to SAEs applied to large language models, where more expressive encoders achieve greater interpretability. This work opens new avenues for understanding neu- ral network representations and analysing large language model activations.
en-
The mathematical foundation of SAEs aligns closely with that of sparse coding. Both approaches assume that a large number of sparse codes are linearly projected into a lower- dimensional space, forming the neural representation. How- ever, while sparse coding typically involves solving an op- timisation problem for each input, SAFs learn efficient encoding function through gradient descent, potentially sac- rificing optimal sparsity for computational efficiency. This trade-off introduces what statistical inference literature calls the "amortisation gap" - the disparity between the best sparse code predicted by an SAE encoder and the optimal sparse codes that an unconstrained sparse inference algo- rithm might produce (Marino al. (018).
1. Introduction
In this paper, we explore this amortisation gap and inves- tigate whether more sophisticated sparse inference meth- ods can outperform traditional SAE encoders. Our key contribution is decoupling the encoding and decoding pro- cesses, allowing for a comparison of various sparse encod- ing strategies. We evaluate four types of encoding methods synthetic datasets with known ground-truth features. We evaluate these methods on two dimensions: alignment with true underlying sparse features and inference of the correct sparse codes, while accounting for computational costs dur- ing both training and inference. To demonstrate real- wolld applicability, we also train models on GPT-2 activations (Radford et al. 2019), showing that more complex methods such MLPs can yield more interpretable features than SAEs in large language models.
Understanding the inner workings of neural networks has become critical task since these models are increasingly employed in high-stakes decision-making scenarios (Fan
et al. 202 1 Shahroudnejad 2021 Räuker al.) 2023) As
the complexity and scale of neural networks continue to grow, so does the importance of developing robust methods for interpreting their internal representations. This paper compares sparse autoencoders (SAEs) and sparse coding techniques, aiming to advance our ability extract inter- pretable features from neural network activations.
Recent work has investigated the "superposition hypothe-
'Australian National University "Nazarbayev University 'Cold Spring Harbor Laboratory. Correspondence to: Charles O'Neill <charles Scharles.oneillee @ anu.edu.au>.
Proceedings of the ndnd International Conference Machine Learning. Vancouver, Canada. PMLR 267, 2025. Copyright 2025 by the author(s).

---

Computc Optimal Inference and Provable Amortisation Gap in Sparse Autoencoders
2. Background and Related Work
Superposition occurs when the matrix W [ WN] R&MXN has more columns than rows (i.e., N > M), making W non-invertible. Superposition relies on the sparsity of feature activations. Let = || be the number of non-zero elements in x = [_1,-.., -.], When 8 < N, the model can tolerate some level of interference between features, as the probability of many features being active simultaneously (and thus interfering) is low.
2.1. Sparse Neural Representations
Sparse representations in neural networks specifically refer to activation patterns where only a small subset of neurons are active for any given input (Olshausen & Field (996). These representations have gained attention due their potential for improved interpretability and efficiency (Lee
et al. (207)). Sparse autoencoders (SAEs) are neural net- work architectures designed to learn sparse representations of input data (Ng et al. 2011, Makhzani & Frey 2013). An SAE consists of an encoder that maps input data to a sparse latent space and a decoder that reconstructs the input from this latent representation. Sparse coding, on the other hand, is technique that aims to represent input data as sparse linear combination of basis vectors (Olshausen & Field, 1997). The objective of sparse coding is to find both the optimal basis (dictionary) and the sparse coefficients that minimise reconstruction error while maintaining spar- sity. While both SAEs and sparse coding seek find sparse representations, they differ in their approach. SAEs learn an efficient encoding function through gradient descent, al- lowing for fast inference but potentially sacrificing optimal sparsity. Sparse coding, in contrast, solves an optimisation problem for each input, potentially achieving better sparsity at the cost of increased computational complexity during inference.
2.3. Compressed Sensing and Sparse Coding
Compressed sensing theory provides a framework for under- standing how sparse signals can be recovered from lower- dimensional measurements (Donoho (006). This theory suggests that under certain conditions, we can perfectly recover a sparse signal from fewer measurements than tradi- tionally required by the Nyquist-Shannon sampling theorem. Let & E N be a sparse signal with at most K non-zero components. we make M linear measurements of this signal, represented as y = Ws where W e MxN,, com- pressed sensing theory states that we can recover s from y with high probability if:
0 (Klog
N K)
M > O
This result holds under certain assumptions about the mea- surement matrix W, such as the Restricted Isometry Prop- erty (RIP) (Candes 2008 Sparse coding is one approach to recovering such sparse representations. The objective function for sparse coding (Olshausen & Field
2.2. Superposition in Neural Representations
1996) is:
The superposition hypothesis suggests that neural networks can represent more features than they have dimensions, par-
N C(D,a):= == (:00==Ele:
11Im;-Dau - Da + Allaull (2)
ticularly when these features are sparse (Elhage et al. 22022). Features are often defined as interpretable properties of the input that a sufficiently large neural network would reliably dedicate a neuron to representing (Olah et al. 2020). For- mally, let us consider a neural representation E R&M and a set of N features, where typically N > M. In a linear representation framework, each feature fi is associated with direction w, R.M. The presence of multiple features is N
where D RXX is the dictionary, a; E RM are the sparse codes for data point x; E K and is hyperparam- eter controlling sparsity. Optimisation of this objective typi- cally alternates between two steps. First is sparse inference: 1l.. Then dictionary learning: min Σ ||x; Da 11 + +Alaa
min Σ || - Da s.t. V L..., M Di i| = 1. D These techniques allow extraction of interpretable, sparse representations from high digh-dimensional neural data.
represented by y = Ei= Iy where x;; E R represents the activation or intensity of feature i.
In an -ddimensional vector space, only M orthogonal vec- tors can fit. However, the Johnson--Lindenstrauss Lemma states that if we permit small deviations from orthogonality, we can fit exponentially more vectors into that space. More formally, for any of N points in a high-dimensional space, there exists linear map to lower-dimensional space of (log N/e²) dimensions that preserves pairwise distances up to factor of (1 + c). This lemma supports the hypothesis that L.LMs might be leveraging a similar princi- ple in superposition, representing many more features than dimensions by allowing small deviations from orthogonal-
2.4. Sparse Autoencoders
Sparse autoencoders (SAEs) offer an alternative approach to extracting sparse representations, using amortised inference instead of the iterative optimisation used in sparse coding. SAEs learn to reconstruct inputs using sparse set of fea- tures in higher-dimenssonal space, potentially disentan- gling superposed features (Elhage et al. 2022 Olshausen
'This property is readily satisfied by many common measure- ment matrices, including random Gaussian and Bemoulli matrices (Baraniuk et al. 2008).
ity.
2

---

Computc Optimal Inference and Provable Amortisation Gap in Sparse Autoencoders
& Field [997). The architecture of an SAE consists of encoder network that maps the input to hidden, sparse representation of latent coefficients, and decoder network that reconstructs the input as linear combination of vectors, with the coefficients defined by the sparse representation.
models (LLMs) by extracting interpretable features from their dense representations. Early work by [Cunningha et al. (023) and Bricken et al. (2023) demonstrated the potential of sparse dictionary learning to disentangle fea- tures, lifting them out of superposition in transformer MLPss This approach was extended to attention heads by Kissane et al. (2024), who scaled it to GPT-2 (Radford et al. 2019). These studies have shown that SAES can extract highly ab- stract, multilingual, and multimodal features from LLMs, including potentially safety-relevant features related to de- ception, bias, and dangerous content (Templeton 2024). In vision models, (2024) Klindt et al. (0023 trained SAEs on convolutional neural network activations. The latter found that K-means (which is equivalent to one-hot sparse coding) outperformed SAEs (Fig. 12) in quantitative interpretability metrics (Zimmermann et al. 2024).
Let 2; E RK be an input vector (as in our sparse coding formulation), and a; E IRM be the hidden representation (analogous to the sparse codes in sparse coding). where typically M > K. The encoder and decoder functions are:
0; = fo( = o (Wax; o + be) Decoder: : f; = go (o:)) = Wao; + ba
Encoder:
(3)
(4)
where W. E R and Wa TR. K are the encoding and decoding weight matrices, be E RM and ba R are bias vectors, and (-) is a non-linear activation function (e.g., ReLU). The parameters 0 = We, be and Ø = Wa, bu are learned during training.
The scaling of SAEs to larger models has been a focus of recent research, with significant progress made in apply- ing them to state-of-the-art LLMs. Gao et al. (2024) pro- posed using k-sparse autoencoders (Makhzani & Frey 2013) to simplify tuning and improve the reconstruction-sparie frontier, demonstrating clean scaling laws with respect to autoencoder size and sparsity. They successfully trained a 16 million latent autoencoder on GPT-4 activations. Sim- ilarly, Templeton (2024) reported extracting high-quality features from Claude 3 Sonnet, while Lieberum et al. (2024) released a comprehensive suite of SAEs trained on all layers of Gemma models. These advancements underscore the importance of developing efficient and accurate SAE tech- niques, especially as applications to larger models become more prevalent. The growing body of work on SAEs in LLMs suggests that they may play a crucial role in future interpretability research.
The training objective of an SAE maintains the same form as Equation _ minimising reconstruction error while pro- moting sparsity. However, SAEs differ from sparse coding in how they perform inference. In sparse coding, finding the codes for new input requires solving an iterative optimisation problem that alternates between updating the codes and the dictionary. In contrast, SAEs learn an encoder function ʃo during training that directly computes sparse codes in a single forward pass. This amortised inference trades off some precision in the optimisation for computa- tional savings at inference time- while sparse coding must solve a new optimisation problem for each input, an SAE can instantly generate codes through its learned encoder.
SAF with Inference-Time Optimisation (SAE+ITO) (SAE+ITO) is an extension of the standard SAE ap- proach that combines the learned dictionary from SAES with inference- time optimisation for sparse code inference (Nanda et al., 2024). In this method, the decoder weights Wa learned during SAE training are retained, but the en- coder function fe is replaced with an optimisation procedure at inference time. For each input x; SAE+ITO solves the optimisation problem outlined in Equation? 2 except only optimising the latent codes with the decoder weights fixed.
3. Methods
This section outlines our approach to comparing sparse encoding strategies. We begin by presenting a theoretical foundation for the suboptimality of sparse autoencoders (SAEs), followed by our data generation process, encoding schemes, evaluation metrics, and experimental scenarios.
This formulation allows for potentially more accurate sparse codes by directly minimising reconstruction error, rather than relying on the learned encoder approximation, despite incurring higher computational costs at inference time. The optimisation problem can be solved using algorithms such as matching pursuit (Blumensath & Davies 2008) and gradient pursuit (Nanda et al. 2024).
3.1. Theory: Provable Suboptimality of SAEs
Theorem 3.1 (SAE Amortisation Gap). Let K 2 2 and PK be a sparse distribution over R. ie., Vs E R s E supp(Px) - || I K. This means that any sample has at most K non-zero entries or, equivalently, the support of Pk is a union over K dimensional subspaces. The sources are linearly projected into M-dimensional space, satisfying the restricted isometry property, where K log N a linear-nonlineat (L-NL) encoder must have a non-zero amortisation gap.
2.5. Applications in Neural Network Models
R S M < N. A sparse autoencoder (SAE) with
Sparse autoencoders (SAEs) have emerged as a promising tool for enhancing the interpretability of large language
3

---

Computc Optimal Inference and Provable Amortisation Gap in Sparse Autoencoders
Sparse Codes "true features"
Observations "neural responses"
Learned Codes
"SAE output"
Decoder
Encoder
1 1--
Linear
Linear-Nonlinear
subspace
raached
falled
Figure 1: Illustration of SAE Amortisation Gap. Left, shows sparse sources in N = 3 dimensional space with at most || S K = 2 non-zero entries. Both blue and red points are valid sources, by contrast, the top right corner 11,1,1) is not. Middle, shows the sources as they are linearly decoded into observation space. This is, in most applications, the activation space of a neural network that we are trying to lift out of superposition. Right, shows how using a ninliern encoder, SAE is tasked to project the points back onto their correct positions. This is not possible, because the pre-activations are at most M dimensional (see proof in Appendix A
The complete proof of Theorem 3.1 is provided in Appendix A The theorem considers setting where sparse signals E R. with at most K non-zero entries are projected into an M-dimensional space (M < N). Compressed sensing theory guarantees that unique recovery of these sparse signals is possible when M 2 K lo)(N/K), up to sign ambiguities (Donoho) (0066) However, we prove that SAEs fail achieve this optimal recovery, resulting in non-zero amortisation gap. The core of this limitation lies in the architectural constraints of the SAE's encoder. The linear-nonlie nonlinear (L-NL) structure of the encoder lacks the computational complexity required to fully recover the high-dimensional (N) sparse representation from its lower- dimensional (M) projection. Figure illlsstrate this con- cept geometrically.
dataset D (x),8:))=1 where x; RM and s; E RN In Appendix D explore alternative data generation process that incorporates Zipf distribution over feature activations, motivated by recent observations that latent representations in large models often follow heavy-tailed distributions (Engels et al. 2024: Park et al. 2024)
3.3. Sparse Encoding Schemes
We compare four sparse encoding strategies:
1. Sparse Autoencoder (SAE): f(x) := o(Wx), where a is nonlinear activation function.
2. Multilayer Perceptron (MLP); f(x) = oWWno (W>-I...o((1)))) with the same de- coder as the SAE.
For completeness, we compare amortisation-gal argu- ment with previous local and distribution-specife recovery results (c.g., (Rangamani et al. 2018 Nguyen et al. 2019)) in Appendix In particular, we clarify why local con- vergence guarantees for ReLU-base autoencoders do not contradict our global impossibility result when addressing all K K-sparse signals in N
Sparse Coding (SC): f(z) argmin D& + X||â||i solved iteratively with 81+1 8, + n where L is the MSE loss with LI penalty.
4. SAE with Inference-Time (SAE+ITO): Uses the learned SAE dictionary, optimising sparse coefficients at inference time.
Optimisation
3.2. Synthetic data
For all methods, we normalise the column vectors of the decoder matrix to have unit norm, preventing the decoder from reducing the sparsity loss [|@|]] by increasing feature vector magnitudes.
To evaluate our sparse encoding strategies, we generate syn- thetic datasets with known ground-truth latent representa- tions and dictionary vectors. We first construct dictionary matrix D E MRM, where each column represents a dic- tionary element. We then generate latent representations S; E R2N with exactly K non-zero entries (K < N), drawn from a standard normal distribution. This allows us to create observed data points as x; = Ds;- This process yields
3.4. Measuring the quality of the encoder and decoder
For any given x, how do we measure the quality of (1) the encoding (i.e. the sparse coefficients); and (2) the decoding (i.e. the actual reconstruction, given the coefficients)?

---

Computc Optimal Inference and Provable Amortisation Gap in Sparse Autoencoders
We employ the Mean Correlation Coefficient (MCC) to evaluate both encoder and dictionary quality:
input æ incurring zero training FLOPs but higher inference- time costs. This differs from SAR and MLP by directly optimising latent coefficients rather than learning an encod- ing function. Sparse coding is not applicable in this scenario, as it reduces to SAE+ITO when the dictionary is known.
MCC
1
Σ leyl
(5)
(3.))MM
where Cij is the Pearson correlation coefficient between the i-th true feature and the j-th learned feature, and M is the set of matched pairs determined by the Hungarian algorithm (or greedy approximation when dimensions differ). This metric quantifies alignment between learned sparse coeffi- cients and true underlying sparse features (encoder quality), and learned dictionary vectors and true dictionary vectors (dictionary quality).
Unknown Sparse Codes and Dictionary. This scenario represents the standard setup in sparse coding, where both the sparse codes & and the dictionary D are unknown and must learned simultaneously. All four methods - SAE, MLP. SAE+ITO, and sparse coding - are applicable here. SAE and MLP learn both an encoder function fo (z) and dictionary D simultaneously. SAE+ITO and sparse coding learn a dictionary during training and optimises latents at inference time.
3.5. Disentangling Dictionary Learning and Sparse Inference
4. Synthetic Sparse Inference Experiments
Our study decomposes the sparse coding problem into two interrelated tasks: dictionary learning and sparse inference. Dictionary learning involves finding an appropriate sparse dictionary D
We present the results of our experiments comparing dif- ferent sparse encoding strategies across various scenarios. All experiments were conducted using synthetic data with N = 16 sparse sources, M = 8 measurements, and K = 3 active components per timestep. unless otherwise specified (more settings in Sec. 4.4 with larger values in App. B and App. C
R R from data, while sparse inference focuses reconstructing signal RM using sparse combination of dictionary elements, solving for s E N in X ~ Ds where s is sparse. These tasks are intrinsically linked: dictionary learning often involves sparse inference in its inner loop, while sparse inference requires a dictionary.
4.1. Known Sparse Codes
Known Sparse Codes. In this scenario, we assume knowl-
edge of the true sparse codes s" and focus solely the coder's ' ability to predict these latents, effectively reducing the problem to latent regression. We define the objective as minimising [x)(),, s*) = - cos(f(z)) (=)),"), where f is the encoding function and cos denotes cosine similarity In this setting, only the SAE encoder and MIP are applicable, as they directly learn mappings from input to latent space. The SAE encoder learns an amortised inference function, while the MLP learns similar but more complex mapping. Conversely, SAE+ITO and sparse coding are not suitable for this task. SAE+ITO focuses on optimising reconstruction using a fixed dictionary, which is irrelevant when true la- tents are known. Similarly, sparse coding alternates between latent and dictionary optimisation, which reduces to encoder training when the dictionary is disregarded.
1.00
0.75
MCC
0.550
SAE
SAE
0.25
HIP 04
0.25
0V
RLP (4 256)
NLP (465
0.00
HLP (4 1024)
0.00
MLP (4- 1024)
10a
10
10
107
10
10
10
Training Steps
FLOPS
(a) MCC vs. training steps
(b) MCC total FLOPs
Figure 2: Performance comparison of SAE and MLPs in predicting known latent representations. The black dashed line in (b) indicates the average FLOPs at which MLPs surpass SAE performance.
We first compare the performance of sparse autoencoders (SAEs) and multilayer perceptrons (MLPs) in predicting known latent representations. Figure 2 illustrates the per- formance of SAEs and MLPs with varying hidden layer widths. MLPs consistently outperform SAEs in terms of Mean Correlation Coefficient (MCC), with wider hidden layers achieving higher performance (Figure 2ap. The MLP with H 1024 reaches an MCC approximately 0.1 higher than the SAE at convergence. While MIPs converge faster in terms of training steps, this comes at the cost of increased computational complexity (Figure 2b). All MLPs surpass the SAE's ' plateau performance at approximately the same total FLOPSs, suggesting consistent computational thresh-
Known Dictionary. When the true dictionary D* is known, we focus on optimising the encoder or inference process while keeping the dictionary fixed. This scenario is applicable to SAE, MLP, and SAE+ITO methods. For SAE and MLP, we optimise ming [ D* fz) where fo represents the encoder function with parameters 0. SAE+ITO, in contrast, performs maddient-based optimi- sation at inference time: min, j 3 + A/2 for each
We use cosine similarity rather than MSE loss in this setting hecause we found training to be more stable.
5

---

Computc Optimal Inference and Provable Amortisation Gap in Sparse Autoencoders
old beyond which MLPs become more effective, regardless of hidden layer width.
07-
MCC
8a5
We also validated our findings at larger scales that better match real-world applications (NV = 1000, .M- = 200, K = 20, and 500, 000 data points), finding that the amortisation gap becomes even more pronounced (see Appendix (
5A2, ITO
Seari
04-
HLP 221
10006
200n0 50000 40000
Training Steps
(a) Latent prediction: MCC vs.
(b) Latent prediction: MCC vs.
training steps
total FLOPs
4.2. Known Dictionary
Dictionary MCC
Dictionary MCC
0.8
342
MCC
0.9
MCC
0.0
24E-1T0
Nur (htd2)
0.6
0.6
SpaneCidey
SAL
MRP
21-22)
0.4
MLP
0.4
0.6
0.6
10000
20006 30000 48000
10
10"
10t
10"h
10
10
Training Steps
FLOPS
10'
10'
10
10a
10"
10th
1044
10
(c) Dictionary learning: MCC
(d) Dictionary learning: MCC
Training Steps
FLOPs
vs. training steps
vs. total FLOPs
(a) MCC vs. training steps
(b) MCC vs. total FL.PPs
Figure 4: Dictionary learning performance comparison when both s and D are unknown.
Figure 3: Performance comparison of SAE, SAE with inference-time optimisation (SAE+ITO), and MLPs in pre- dicting latent representations with known dictionary. Dashed lines in (b) indicate extrapolated performance be- yond the measured range.
coding again exhibits an initial decrease in dictionary MCC before surpassing other methods.
Next, we examine the performance of different encoding strategies when the true dictionary D is known. Figure 3 shows the performance of SAE, SAE+ITO, and MLPs. MLPs consistently outperform the standard SAE, achiev- ing an MCC nearly 10% higher at convergence (Figure Bap. Both MLP configurations (H 32 and H 256) converge to similar performance levels, with the wider network show- ing faster initial progress. When plotted against total FLOPs, the MIP curves overlap, suggesting a consistent compu- tational cost-to-performance ratio across different hidden layer widths (Figure 3b). SAE+ITO initialised with SAE latents exhibits distinct, stepwise improvements throughout training, ultimately achieving the highest MCC.
4.4. Performance Across Varying Data Regimes
To understand how performance varies with changes in data characteristics, we trained models under varying N, M, and K, holding other hyperparameters constant.
Figure 5 shows the difference in final latent MCC between methods. Sparse coding outperforms SAE in essentially all data getaa-enneation regimes, for both K = 3 and K = 9. MLP and SAE perform roughly equivalently, with MLP slightly better as M (number of measurements) increases. The per- formance advantage of sparse coding more pronounced in regimes where compressed sensing theory predicts recov- erability (above and to the left of the black dashed line).
4.3. Unknown Sparse Codes and Dictionary
Sparsity- Trade-off We also investigated the trade-off between sparsity and performance for each method in Figure 6 Sparse coding achieves slightly lower reconstruction error for each LO level, barring some very small active latents. Sparse coding shows a Pareto improve- ment at each L.0 level in terms of MCC, even with very small active latents. The improvement is more evident when plotting against LI rather than LO, as LI accounts for the magnitude of non-zero values. The presence of very small non-zero latents in sparse coding motivates the exploration of top-k sparse coding, detailed in Appendix H.2
Finally, we evaluate all four methods when both latent rep- resentations and dictionary are unknown. We use dataset of 2048 samples, evenly split between training and testing sets, and conduct 5 independent runs of 100, 000 steps.
Figures 4 illustrates the performance in latent prediction and dictionary learning, respectively. For latent prediction, SAE, SAE+ITO, and MLPs converge to similar MCC, with MLPs showing a slight advantage. Sparse coding demon- strates superior performance, achieving MCC over 10% higher than other methods, despite an initial decrease in per- formance. Sparse coding reaches this higher performance while using comparable FLOPs to the MLP with H = 256. For dictionary learning, both MLPs and sparse coding out- perform SAE by a margin of approximately 10%. Sparse
5. Interpretability of Sparse Coding Schemes
A common concern about more powerful encoding ap- proaches is that they might learn unnatural features that
6

---

Computc Optimal Inference and Provable Amortisation Gap in Sparse Autoencoders
SparseCoding
MLP
SparseCoding SAE
MLP
30
Recovery Bourdary
30
Recovery Boundary
0.45
30
30
0.24
0.36 g
25
0.18
25
25
0.10
0.09
0.05 0
-0.09
0.05
15
15
-0.18 m
15
15
-.10 B
0.27
0.15
10
-0.66
0.45
10
--- Recovery Boundary
10
Recovery Bourdary
0.20
0,24
125
30
75
125
50
(a) K 3
(h) K
Figure 5: Difference in final latent MCC between methods across varying N and M, for K = 3 and K = 9. Left: Sparse coding vs. SAE. Right: MLP vs. SAE. The black dashed line indicates the theoretical recovery boundary.
LO vs MSE Loss
LO vs MCC
L1 vs MSE Loss
L1 vs MCC
SAE
0.8
SAE
0.8
MLP
MLP
SC (1e-03)
SparseCoding
MSE Loss
SC (1e-04)
MSE Loss
0.2
(1e-05)
MCC
SAE
0.2
MCC
True
0.4
MLP
0.4
SC (1c-03)
0.1
SC (1e-04)
0.1
SAE
0.2
SC (1e-05)
0.2
MLP
True
SparseCoding
0.0
0.0
0.00
0.25
0.50
0.75
1.00
0.00
0.25
0.50
0.75
1.00
0.0
0.1
0.2
0.3
0.0
0.1
0.2
0.3
Lo Loss
LO Loss
L1 Loss
L1 Loss
Figure 6: Pareto curves showing sparsity (LO or LI loss) against performance (MSE loss or latent MCC) for models trained with varying LI penalty coefficients The red dashed line in the top row shows the true LO of the sparse sources. Multiple thresholds for active features shown for sparse coding due to the presence of very small non-zero values.
not interpretable. To investigate the interpretability of more complex encoding techniques, we trained three dis- tinct methods on 406 million tokens from Open WebText: a sparse autoencoder with a single linear encoder layer and ReLU activation, a multilayer perceptron encoder with one hidden layer of width 8448, and a locally competitive algo- rithm following the approach of Olshausen & Field (1997) and Blumensath & Davies (008). Each method learned an overcomplete dictionary of size 16, 896 for the residual- stream pre-activations at Layer 9 of GPT-2 Small (Radford et al. 2019), which have dimension 768.
yielded an effective Lo of approximately 18.56. Notably, the LCA dictionary maintained no strictly dead columns.
To assess the interpretability of the learned features, we randomly selected 500 features from each method and em- ployed an automated interpretability classification approach using GPT-4o (full details in Appendix D For each feature, identified its top 10 most highly activating tokens in our 13.1 million-token test set and computed logit effects through the path expansion Wt J. where Wir represents the model's 's unembedding matrix and f denotes the feature vector. We provided both the activating examples and the top and bottom 10 tokens by logit effect to GPT-4o, which generated concise explanation of the feature's function. To validate these interpretations, we presented them second instance of GPT-4o along with at least five new activating examples and five non-activating examples, labelling the activating tokens. The model predicted which examples should activate the feature based on the first instance's 's ex- planation, allowing us to compute an Fl-score against the ground truth. This automated interpretability approach is considered standard in the literature and relies base prompt from Noomu al. (024)..
All methods were trained using Adam with a learning rate 443-10-4 and an Ln penalty of 1- 10- Following Bricken jet al. (2023) and Cunningham al. (2023), we resampled dead neurons every 15, 000 steps by setting columns with no activity to new random directions. The final results across methods were: the SAE achieved a normalised MSE of 0.061 with a mean Lo of 35.66 and 1 11% dead neurons, while the MLP reached a normalised MSE of 0.055 with mean Lo of 31.13 and 22% dead neurons. The LCA approach, with 100 gradient-based sparse-inference steps per batch, achieved normalised MSE of 0.070. While technically none of the LCA codes were exactly zero, most were extremely small, and thresholding values below 10-5
Figure displays the distributions of F1-scores across the evaluated features. The results indicate that SAE and LCA
7

---

Computc Optimal Inference and Provable Amortisation Gap in Sparse Autoencoders
1.00
2=2500 0=0.63
N=2500 -=0.83
N=2500 0=0.61
mance of MLPs suggests that matching the computational complexity of the underlying representations improves fea- ture extraction. Better encoders aligns with recent work on inference-time optimisation (Nanda et al. (2024), and will be validated as we improve encoding evaluation (Makelov) ct al. 2024). Regardless, SAEs are sensitive to hyperparam- eters and fragile (Cunningham et al., 2023). so exploring more powerful encoders is warranted.
F1 scores
0.75
0.50
0.25
0.00
SAE
MLP
LCA
The computational cost of complex encoders must be evalu- ated against gains in feature extraction and interpretability. Projects like Gemma Scope (Lieberum et al.. 2024) demon- strate substantial resource investment in feature extraction, suggesting that additional compute for improved representa- tion quality may be justified. Complex encoders can main- tain the linear decoder needed for downstream tasks such as steering while providing better features. Future work should systematically compare feature quality across encoder ar- chitectures and address non-zero-centered representations (Hobbhahn) 2023).
Figure 7: Distribution of F1 scores for feature interpretabil- ity across three methods (SAE, MIP, and LCA) trained on residual stream activations of Layer 9 in GPT-2. Each distri- bution represents 500 randomly selected features evaluated using GPT-4o for explanation generation and validation.
features demonstrate comparable interpretability, with me- dian Fl-scores around 0.6. Most notably, the MLP features achieve substantially higher interpretability scores, with median Fl-score of 0.83 and a tighter distribution. A Kruskal-Wallis test revealed significant differences between the methods (H= = 1856.33, p < 0.001), and subsequent Dunn's 's tests with Bonferroni correction confirmed that both SAE and LCA were significantly less interpretable than MLP features (p 0.001). See Appendix | for examples of feature interpretations.
Limitations Our study has several limitations. Our LCA implementation was not optimised for the scale of exper- iments, requiring investigation of sparse coding methods, sparsity levels, optimisation iterations, and thresholding of near-zero activations. The gap between MLP and SAE/LCA interpretability scores warrants examination - while LCA's lower performance likely stems from suboptimal training, the MLP's superior interpretability relative to SAEs requires investigation. Our analysis also focused on scenarios with constant sparsity and uncorrelated channels, which may not capture real-world data complexity. Our synthetic data generation process did not account for varying feature im- portance described in EElhaee et al.) (2022)'s s framework, although we did begin to explore this in Appendix D
6. Discussion
Our study provides theoretical and empirical evidence for an inherent amortisation gap in sparse autoencoders (SAEs) when applied to neural network interpretability. We prove that SABS with linear-nonlinear encoders cannot achieve optimal sparse recovery settings where such recovery is theoretically possible. This limitation is supported by exper- imental results showing that sparse coding, and sometimes MLPs, outperform SAES across synthetic data scenarios. Our investigation of GPT-2 activations demonstrates that MLP-based features achieve higher interpretability scores than both SAE and LCA features. These findings refute the assumption that simpler encoders are necessary for main- taining interpretability. The results carry implications for neural network interpretability, suggesting that more sophis- ticated encoding techniques can improve feature extraction without compromising feature validity, though at increased cost.
Future work should incorporate recent SAE variants like top-k SAEs (Makhzani & Frey, 2013) Gao et al. 2024) and JumpReLU SAEs (Rajamanoharan et al. 2024b) to mea- sure the amortisation gap with modern architectures. Our SAE+ITO implementation did not use advanced techniques like matched pursuit. potentially underestimating its perfor- mance. The traditional dictionary learning approaches in Appendix H indicate room for improvement. Finally, we should explore sampling feature activations from different parts of the activation spectrum when doing automated inter- pretability, because features may exhibit different levels of specificity different activation strengths, and examining only top activations could miss important collaborative be- haviors between features and edge cases that help validate feature interpretations (Bricken et al., 2023). Addressing these limitations would advance understanding of sparse encoding strategies for complex neural representations.
The use of linear-nonlinear encoders in SAEs for language model interpretability stems from concerns that more pow- erful methods might extract features not used by the trans- former (Bricken et al. (023). This approach appears overly restrictive given the complexity of transformer layer rep- resentations, which emerge from multiple rounds of atten- tion and feed-forward computations. The superior perfor-
8

---

Computc Optimal Inference and Provable Amortisation Gap in Sparse Autoencoders
Impact Statement
References
This work advances neural network interpretability by im- proving methods for extracting understandable features from complex AI systems, with primarily positive societal impli- cations for building trustworthy Al in high-stakes applica- tions. By demonstrating that more sophisticated encoding methods achieve superior interpretability while maintain- ing computational efficiency, this research could accelerate transparent Al development across domains like healthcare and autonomous systems, and democratise access inter- pretability tools for smaller research groups. However, we acknowledge potential risks including misuse for adversar- ial attacks, exploitation of model vulnerabilities, and false confidence in model safety if users over-rely on feature in- terpretations. The computational improvements may also lower barriers for both beneficial AI safety research and potentially harmful applications. Overall, we believe the benefits of advancing interpretability research significantly outweigh the risks, as transparent and understandable Al systems fundamental to responsible AI deployment.
Baraniuk, R., Davenport, M., DeVore, R., and Wakin, M. A simple proof of the restricted isometry property for random matrices. Constructive approximation, 28:253- 263, 2008.
Bills, S., Cammarata, N., Mossing, D., Tillman, H., Gao, I.., Goh, G. Sutskever, I., Leike, J., Wu, J., and Saunders, W. Language models can explain neurons in language models. URL https:/topenaipublic. blob. core. windows. oee/heewonexppiiee/nggeennuueeee html.(Date accessed: 14.05. 2023), 2, 2023.
Blumensath, T. and Davies, M. E. Gradient pursuits. IEEE Transactions on Signal Processing. 56(6):2370-25 2008.
Bricken, T., Templeton, A., Batson, J., Chen, B., Jermyn, A., Conerly, T., Turner, N., Anil, C., Denison, C., Askell, A., al. Towards monosemanticity; Decomposing language models with dictionary learning. Transformer Circuits Thread, 2, 2023.
Candes, E. J. The restricted isometry property and its impli- cations for compressed sensing. Comptes rendus. Mathe- matique, (0::0990):899--0110 2008.
Cunningham, H., Ewart, A., Riggs, I.., Huben, R., and Sharkey, L. Sparse autoencoders find highly inter- pretable features in language models. arXiv preprint arXiv:2309.0866 2023.
Donoho, D. L. Compressed sensing. IEEE Transactions information theory, 1289::188- 2006.
Efron, B., Hastie, T., Johnstone, and Tibshirani, R. Least angle regression. 2004.
Elhage, N., Hume, T., Olsson, C., Schiefer, N., Henighan,
T., Kravec, S., Hatfield-Dodds, Z., Lasenby, R., Drain, D., Chen, C., al. Toy models of superposition. arXiv preprint arXiv:2209.1006 2022.
Engels, J., Liao, I., Michaud, E. J., Gurnee, W., and Tegmark, M. Not all language model features lin- ear, 2024.
14860
Fan, F.-L., Xiong, J., Li, M., and Wang, G. On interpretabil- ity of artificial neural networks: A survey. IEEE Transac- tions on Radiation and Plasma Medical Sciences, 5(6): 741-760, 2021.
Foote, A., Nanda, N., Kran, E., Konstas, I., and Barez, F. N2g: A scalable approach for quantifying interpretable neuron representation in llms. In ICLR 2023 Workshop on Trustworthy and Reliable Large-Scale Machine Learning Models.

---

Computc Optimal Inference and Provable Amortisation Gap in Sparse Autoencoders
Gao, L., la Tour, T. D., Tillman, H., Goh, G., Troll, R., Radford, A., Sutskever, I,, Leike, J., and Wu, J. Seal- ing and evaluating sparse autoencoders. arXiv preprint arXiv:2406. 04093, 2024.
Nanda, N., Conmy, A., Smith, L., Rajamanoharan, S., Lieberum, T., Kramár, J., and Varma, V. Progress up- date from the gdm mech interp team, 2024, [Accessed 01-09-2024).
Gorton, L. The missing curve detectors of inceptionvl: Applying sparse autoencoders to inceptiony early vision. arXiv preprint arXiv:2406.00 2.6322200.03366,, 2024.
Ng. A. et al. Sparse autoencoder. CS294A Lecture notes, 72 (2011):1-15) 2011.
Nguyen, T. V., Wong, R. K., and Hegde, C. On the dynam- ics of gradient descent for autoencoders. In The 22nd International Conference on Artificial Intelligence and Statistics, pp. 2858-2867. PMLR, 2019.
Gregor, K. and LeCun, Y. Learning fast approximations of sparse coding. In Proceedings of the 27th international conference international conference on machine learn- ing. pp. 399-406, 2010.
Olah, C., Cammarata, N., Schubert, L., Goh, G., Petrov, M., and Carter, S. Zoom in: An introduction to circuits. Distill, 5(3):000244-001, 2020.
Hobbhahn, M.
More findings on memo-
rization and double descent, 2023. URL
http::://wwww.llignni. otte//osts//kw3334ozzZ22000
Olshausen, B. A. and Field, D. Emergence of simple-cell receptive field properties by learning sparse code for
more www.fiindiiissee -omooo--oo--mmmmminttiaia and double [Accessed 29-09-2044.
Thages. Nature, 381 (6588))60076609) 1996.
Olshausen, B. A. and Field, D. J. Sparse coding with an overcomplete basis set: A strategy employed by vI? Vi- sion research, 37(23)33311 133211,-3,,,,, 1997.
Juang, C., Paulo, G., Drori, J., and Belrose, N. Open source automated interpretability for sparse autoencoder fea- tures, 2024. URL httpps://blog. eleuther. autointerp/ [Accessed 29-09-2024).
Park, K., Choe, J., Jiang, Y., and Veitch, V. The geometry of categorical and hierarchical concepts in large language models. arXiv preprint arXiv:2406.01506 2024.
Kissane, C., Krzyzamowski, R., Bloom, J. L. Conmy, A., and Nanda, N. Interpreting attention layer outputs with sparse autoencoders. arXiv preprint arXiv:2406.1775,, 2024.
Pati, Y. C., Rezaifarr R., and Krishnaprasad, P. S. Orthogo- nal matching pursuit: Recursive function approximation with applications to wavelet decomposition. In Proceed- ings of 27th Asilomar conference on signals, systems and computers, pp. 40-44. IEEE, 1993.
Klindt, D., Sanborn, S., Acosta, F., Poitevin, F., and Mi- olane, N. Identifying interpretable visual features in ar- tificial and biological neural systems. arXiv preprint arXiv:2310.11431, 2023.
Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., Sutskever, I., al. Language models are unsupervised multitask learners. OpenAl blog, 1(8):9, 2019.
Lee, H., Ekanadham, C., and Ng, A. Sparse deep belief net model for visual area v2. Advances in neural information processing systems, 20, 2007.
Rajamanoharan, S., Conmy, A., Smith, L.., Lieberum, T, Varma, V., Kramár, J., Shah, R., and Nanda, N. Improving dictionary learning with gated sparse autoencoders. arXiv preprint arXiv:2404.16014 2024a.
Lieberum, T., Rajamanoharan, S., Conmy, A., Smith, L., Sonnerat, N., Varma, V., Kramár, J., Dragan, A., Shah, R., and Nanda, N. Gemma scope: Open sparse autoencoders everywhere all at once gemma 2, 2024. https;
Rajamanoharan, S., Lieberum, T., Sonnerat, N., Conmy, A.. Varma, V., Kramár, J., and Nanda, N. Jumping ahead: Improving reconstruction fidelity with jumprelu sparse autoencoders. arXiv preprint ar 240. 14435, 2024b.
7/arxiv. org/ abs/ / 2408 05147
Makelov, A., Lange, G., and Nanda, N. Towards princi- pled evaluations of sparse autoencoders for interpretabil- ity and control, 2024. URL ltttps://axxi.. org/ abs/2405.00 08366
Rangamani, A., Mukherjee, A., Basu, A., Arora, A., Gana- pathi, T., Chin, S., and Tran, T. D. Sparse coding and autoencoders, In 2018 IEEE International Symposium on Information Theory (ISIT), pp. 36-40. IEFE, 2018.
Makhzani, A. and Frey, B. K-sparse autoencoders. arXiv preprint 13XXiv:3/2.56653 2013.
Räuker, T., Ho, A., Casper, S., and Hadfield-Mendl D. To- ward transparent ai: A survey on interpreting the inner structures of deep neural networks. In 2023 ieee confer- ence on secure and trustworthy machine learning (satml), pp. 464 483, IEEF, 2023.
Marino, J., Yue, Y., and Mandt, S. Iterative amortized infer- ence. In International Conference on Machine Learning, pp. PMLR, 2018
10

---

Computc Optimal Inference and Provable Amortisation Gap in Sparse Autoencoders
Shahroudnejad, A. A survey on understanding, visualiza- tions, and explanation of deep neural networks. arXiv preprint arXiv:2102.0179) 2021.
Contents
Introduction
Taggart, G. Profilu: nonlinearity for sparse autoencoders. In AI Alignment Forum, 2024.
Background and Related Work
2
Templeton, A. Scaling monosemanticity: Extracting in- terpretable features from claude 3 sonnet. Anthropic, 2024.
2.1 Sparse Neural Representations
2
2.2
Superposition in Neural Representations
Compressed Sensing and Sparse Coding
Tibshirani, R. Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society Series B: Statistical Methodology, 58(1):267-288 1996.
Sparse Autoencoders
2.5
Applications in Neural Network Models
Wright, B. and Sharkey, L. Addressing feature suppression in saes. In Al Alignment Forum, pp. 16, 2024.
Methods
Zimmermann, S., Klindt, D. A., and Brendel, W. Measur- ing mechanistic interpretability at scale without humans. volume 38, 2024.
3.1
Theory: Provable Suboptimality of SAES
3.2
Synthetic data
3.3
Sparse Encoding Schemes
3.4
Measuring the quality of the encoder and decoder
3.5
Disentangling Dictionary Learning and
Sparse Inference,
Synthetie Sparse Inference Experiments
5
4.1
Known Sparse Codes
5
4.2
Known Dictionary
6
Unknown Sparse Codes and Dictionary
Performance Across Varying Data Regimes
Interpretability of Sparse Coding Schemes
6
Discussion
8
Amortisation gap proof
12
B Relating Our Amortisation Gap to Prior Results in Sparse Autoencoders
13
C Large-Scale Experiments
13
Different Distribution of Codes
14
E Decoder weight analysis
15
MLP Ablations
15

---

Computc Optimal Inference and Provable Amortisation Gap in Sparse Autoencoders
Including bias parameter
16
tion, recover the high dimensionality (N) of the data after projection into lower (M) dimensional space Figure [
H Comparison with traditional dictionary learning methods]
Proof. Let S diag($11 --, SNN) be diagonal matrix with non-zero diagonal elements 3 * 0, Vi E NN) Ever row is valid source signal because it has non-zero support under Pk since, ||>,|lo = 1 < K, Vi E (1,..., NY. Since the support of Pk includes all 1 1-sparse vectors (as 1 I K), selecting S as the diagonal matrix of 1-sparse signals is without loss of generality. Let Wa E RNXM be the unknown projection matrix from N down to M dimensions and We I be the learned encoding matrix of the SAE. Define W = WaW, E RRNN and
16
H.1
Optimised Sparse Autoencoders and Sparse
Coding
17
H.1.1
Advanced
Sparse Autoencoder
Techniques
H.
1.2
Optimised Sparse Coding Approaches
17
H.2
Top-k sparse coding
18
Measuring FLOPs
18
8' = SW
(6)
Sparse Coding
18
the pre-activation matrix from the encoder of the SAE, Since Wa projects down into M dimensions,
Sparse Autoencoder (SAE)
18
Multilayer Perceptron (MLP)
rank (W) raakkannWWWWW. s M.
(7)
1.4 SAE with Inference- Time
Optimisation
It follows that
(SAE+ITO)
19
rank = rankkswW S M.
(8)
Automated interpretability
19
As an intermediate results, we conclude that the pre- activations S' of the SAE encoder cannot recover the sources S' * |S| since rank([S]) = N, because is a diagonal ma- trix.
J.1
Feature Interpreter Prompt
19
J.2 Feature Scorer Prompt
19
J.3 Evaluation of Automated Interpretability
20
The next step is to see whether the nonlinear activation function might help to map back to the sources. The SAE must learn an encoding matrix We such that
A. Amortisation gap proof
Theorem A.1 (SAE Amortisation Gap). Let K 2 2 and PK be sparse distribution over T i.e., Vs E TR. s E (Px) sample has at most K non-zero entries or, equivalently, the support of Px a union over K dimensional subspaces. The sources are linearly projected into an M -ddimensional space, satisfying the restricted isometry property, where K log NX M- < N. A sparse autoencoder (SAE) with a Aneer-nonlinear (L-NL.) encoder must have non-zero amortisation gap.
S = max SW.W. max(0, SW) max (0, S*)
+ ||s|lo S K. This means that any
(9)
where max(0, .) is the ReI.U activation function. Thus, for the SAE to correctly reconstruct the sparse signals up to sign flips, for any source code a E supp(Pk)) we require
|os| 0 otherwise
if of #
(oW),
(10)
specifically, S' must be non-positive off the diagonal and identical to [S] the diagonal.
This setting is solvable according to compressed sensing the- ory (Donoho) 2006), meaning that it is possible to uniquely recover the true S up to sign flips- we cannot resolve the ambiguity between the sign of any code element and the corresponding row in the decoding matrix. If a SAE fails achieve the same recovery, then there must be non-zero amortisation gap, meaning that the SAE cannot solve the sparse inference problem of recovering all sparse sources from their M -dimensioonal projection. The problem is the low computational complexity of the L-NL encoder as we see by looking at its functional mapping. Essentially, the SAE is not able, not even after the nonlinear activation func-
Approach: Show that a matrix S' cannot simultaneously satisfy conditions (cq. g) and (eq. I0).
According to (eq. 6 and condition (eq. 10 we require that
s W = (s $122 Si sNN) = (1s $12 s
sIN) (11)
with 8/1 s 0 for all i E (2...,, N}. Analogously,
$2 = (5/1, 8622 222 ..., s2N) = (6 | $23
22N) (12)
12

---

Computc Optimal Inference and Provable Amortisation Gap in Sparse Autoencoders
with 5/2 < for all f1,3,..., N). Moreover, since K we know that 81 + 82 has non-zero
In contrast, many prior theorems establish local (or near- dictionary) results: they assume the encoder's weights start sufficiently close to the true dictionary, then show that ReLU (or threshold) gating can maintain or refine correct sparse recovery for typical data.
[|s + 82 support under Pk. so condition (eq. 10) must also hold for it. Thus, we need that
(81 + & = (Is11 + 821 : |812 + 822 "..... YN-2)
The distinction between uniform and distribution-speciffe recovery is also important. Our proof deals with uniform, adversarially chosen K K-sparse codes. If the model must handle all codes in RN with ||s||o Σ K, single feed- forward pass will inevitably fail for some codes. By contrast, much of the prior literature - including (Rangamani et al. 2018 Nguyen et al. 2019) -requires that codes are drawn from a random distribution (e.g.. sub-Gaussian or mixture- of-Gaussians). This assumption enables high-probability success on most sampled codes, but does not guarantee recovery of all codes.
([su1 + 0,, 10 + s22l Yo,.. 72-
(|suzl, (222): (1....-222)
(13)
with some non-positive % S for all i {L,.., 2}.
However, because of linearity,
[ |s2a| 7h 22 = (s1 + s2)W
= =s + s2W
= (1111), 812, 818 ..., s1N)
+ (#/211 1822), 8233......... 822N
Another key distinction lies in single-pass versus multi-pass inference. Our amortisation-gap statement explicitly con- cerns a single-layer feedforward autoencoder. Iterative or unrolled algorithms (c.g.. LISTA (Gregor & LeCun 2010), or multi-layer ReLU stacks) circumvent the rank restriction by repeatedly refining the estimate. Thus, multi-iteration multi-layer approach can approach near-optimal sparse recovery; but this does not contradict our statement about one-pass innaarnonlinear encoder's 's inability to decode every sparse signal.
(|sn]+ $211
+ |s2al, $13 $23,.... ... siN + s22N)
(14)
Thus, |sn|] |sn|| + 2/21 and |222l 512 |822|- From which it follows that 2/21 = 0 and 5'12 0. By repeating this for all 8;, 8j combinations, we obtain that all off-diagonal elements in S' must be zero. However, that means S' = diag(1$) (o |1|,,-..__ |sNl)) must be diagonal. This leads to a contradiction, since it would imply that rank(S") = N. violating condition (eq. 8).
Finally, our result demands exact (or perfect) inversion of all feasible codes, while prior analyses often accept ap- proximate or high-probubility correctness. They conclude that, given some distribution codes and an adequately trained near-dictionary encoder, one recovers the support with probability > 1 - 6. This does not conflict with a global impossibility statement.
We can generalise the result to L1 sparse distribu- tions P& with Vs E ER, 8 E supp(Pk) < || I k for some k > 0. In this case, we would choose ||sa|| < 2 and ||s2|| < k Thus, again we would have (s1 (2)) E supp(P)) 2'
Notes:
since [|s] + s2ll < allowing the same reasoning.
C. Large-Scale Experiments
B. Relating Our Amortisation Gap to Prior Results in Sparse Autoencoders
To validate that our findings generalise to larger scales more representative of real-world applications, we conducted ad- ditional experiments with substantially increased dimen- sionality. We scaled up our synthetic experiments for the known Z case to N = 1000 sparse sources, M = 200 measurements, and K = 20 active components, training on 500, 000 samples for 20, 000 steps. This represents a significant increase from our base experiments (which used N = 16, M 8, K = 3), bringing closer to the scale of actual SAE applications.
In this appendix, we clarify how our amortisation-gap the- orem aligns with prior work on shallow autoencoders in the sparse coding literature, including references such as (Rangamani et al.) 2018) and (Nguyen et al. 2019). While these earlier results may appear to contradict our statement that a single feedforward limearnonlinees globally recover all sparse codes from fewer measurements (M- < NN. we show that these works rely on local or prob- abilistic assumptions. By contrast, our theorem provides global, worst-case statement.
encoder cannot
For these experiments, we modified our training procedure to use minibatch processing (batch size 1024) to handle the increased data scale efficiently. We evaluated MLPs with hidden layer widths of H (256, 512, 1024) against standard SAF The results, shown in Figures 8a and [b) demonstrate that our key findings about the amortisation
Our work presents a global impossibility claim: a single- layer linear+o map cannot perfectly invert every K- Spsare code if M < N. This argument is uank-based and does not rely on training initialisation or specific data distribution.
13

---

Computc Optimal Inference and Provable Amortisation Gap in Sparse Autoencoders
gap not only hold but become more pronounced larger scales.
1.00
1.00
0.75
0.75
MCC
0.50
0.50
1.0
1.0
0.25
HU' It' 22)
0.25
MLP (N'm 250]
0.00
MIP (i- 1024
0.00
(120020)
10a
10"
10'
10
10'
1033
1011
10
S4E
SAE
Training Steps
FLOPS
HUP (Y- 1024)
0.8
MLP (024))
MLP O 5121
(12)
(a) MCC vs. training steps
(b) MCC vs. total FLOPs
NLP 2561
2256)
0.7
Training Steps
10
10
FLOPs
10
Figure 9: (Zipfian) Performance comparison of SAE and MLPs in predicting known latent representations. The black dashed line in (b) indicates the average FLOPs at which MLPs surpass SAE performance.
(a) MCC training steps
(b) MCC total FLOPs
Figure& 8: (Larger N, M and K) Performance comparison of SAE and MLPs in predicting known latent representa- tions. The black dashed line in (b) indicates the average FLOPs at which MLPs surpass SAE performance.
ing our main conclusions. the known sparse codes sce- nario (Figure 9). all methods achieve higher absolute perfor- mance, with MLPs reaching MCC values of approximately 0.8 compared to 0.6 in the uniform case. The advantage of wider hidden layers becomes more pronounced under the Zipfian distribution, though the computational threshold at which MLPs surpass SAE performance remains consistent with our original findings.
Specifically, the performance gap is slightly more sub- stantial than in our smaller-scale experiments, suggesting that the limitations of linear-nonlinear encoders become more significant as the problem dimensionality increases. This aligns with our theoretical predictions, as the higher- dimensional setting creates more opportunities for interfer- ence between features that the simple SAE encoder struggles to disentangle.
0.75
0.75
The FLOP analysis (Figure 8b) reveals that all MLPs surpass the SAE's performance at approximately 3 x 10 FLOPs, regardless of hidden layer width. This consistent computa- tional threshold, despite varying model capacities, suggests a fundamental limitation in the SAE's architecture rather than a simple capacity constraint.
80.50
0.50
0.25
SAZ4
HLP 24+300
0.00
MaP (9-2500
0.00
MLP (1-250)
10'
10"
10"
10
10"
10tH
10
10
Training Steps
FLOPs
MCC vs. training steps
(b) MCC vs. total FLOPs
Figure 10: (Zipfian) Performance comparison of SAE, SAE with inference-time optimisation (SAE+ITO), and MLPs in predicting latent representations with a known dictionary. Dashed lines in (b) indicate extrapolated performance be- yond the measured range.
D. A Different Distribution of Codes
In this appendix, we explore alternative data generation process that better reflects the distributional properties ob- served in real-world latent representations. While our main experiments use uniformly sampled sparse codes, recent work has shown that latent features in large models often follow heavy-tailed distributions (e.g.. power laws) with varying activation frequencies (Engels et al., 2024 Park et al. 2024). To investigate the robustness of our findings, we modify our synthetic data generator to incorporate Zipf distribution (parameterised by a) over feature activations. This creates a hierarchical structure where certain features are consistently more likely to be active and have larger magnitudes, while others are more rarely activated. The modified generator maintains the core sparsity constraint of K active dimensions, but weighs both the selection prob- ability and magnitude of each dimension according to its position in the Zipf distribution.
When the dictionary is known but sparse codes are unknown (Figure 10), we observe similar relative performance pat- terns but with higher peak MCC values (around 0.85 com- pared to 0.75 in the uniform case). The SAE with inference- time optimisation (SAE+ITO) exhibits more volatile train- ing dynamics under the Zipfian distribution, showing a char- acteristic performance drop around 10 training steps be- fore recovery. This suggests that optimisation becomes more challenging when dealing with hierarchically struc- tured features, though the method ultimately achieves strong performance.
The most substantial differences emerge in the fully unsu- pervised setting, where both dictionary and sparse codes are unknown (Figure [T]. Here, the Zipfian distribution leads to lower overall performance (MCC of 0.5-0.6 versus 0.7-0.8 in the uniform case) and creates clearer separation between different methods. While sparse coding still outperforms
We reproduced all experiments from Section 4.4 using this modified data generation process, with a = 1.0. The results reveal several interesting differences while broadly support-
14

---

Computc Optimal Inference and Provable Amortisation Gap in Sparse Autoencoders
0.6
SAC
SAE
SparseCoding
1.00
0.3
Column Indiax
0.75
0.4
SAE, (To
0.4
0.50
ScaronCortnng
0.3
NU' (2
NU (202
0.3
0.25
0.2
10000 20000 30000 40000
0.2
10
10
10a
10th
10
10
15
15
15
Training Steps
FLOPs
15
10
15
Index
Column
Column Index
(a) Latent prediction: MCC vs. training steps
(b) Latent prediction: MCC vs.
total FLOPs
Figure 12: Visualisation of D when N matches the true sparse dimension. Sparse coding achieves near-identity matrices, while sparse autoencoders (SAE) and multilayer perceptrons (MLP) show significant off-diagonal elements, indicating superposition.
Diclionary MCC
0.9
Dictionary MCC
0.0
0.6
(mo
0.8
SAE ITD
SAE
SoarseCeding
SparseCading
NLP (2+32)
HL. (He22)
0.4
HLP (H-2561
H&P oi 2500
0
10000
Training Steps
20000
30008 40000
10
10
FLOPs
10
10"h
10 10
SAE
MLP
SparseCoding
1.00
(c) Dictionary learning: MCC
(d) Dictionary learning: MCC vs. total FLOPs
Colum Index
0.75
Vs. training steps
-o.
10-
10-
0.50
15
15-
15-
Figure 11: (Zipfian) Dictionary learning performance com- parison when both s" and D are unknown.
0.00
20
10
20
Column Index
Column Index
Column Index
Figure 13: Visualisation of DT when N exceeds the true sparse dimension.
other approaches, its advantage is less pronounced than in the uniform setting. Dictionary learning under the Zipfian distribution shows increased volatility across all methods, particularly for sparse coding, though the relative ordering of performance remains consistent with our original results.
off-diagonal elements are close 1.0, suggesting that these models utilise superposition, representing more features than there are dimensions. This is suboptimal in this par- ticular scenario because the models have the exact number of dimensions required to represent the feature space effec- tively. Notably, this superposition effect diminishes when vector normalisation is applied during training.
These findings suggest that while our conclusions about the relative merits of different approaches hold under more realistic distributional assumptions, the absolute difficulty of the sparse inference problem increases when dealing with hierarchically structured features.
We observe similar patterns when N is greater than the true sparse dimensionality (Figure [3) and when N is smaller (Figure 14 In cases where N exceeds the required di- mensionality, sparse coding still strives to maintain orthog- onal feature directions, leading to a near-identity matrix. However, both SAFS and MLPs show stronger correlations between features, as indicated by larger off-diagonal ele- ments, though MLPs exhibit less extreme correlations (e.g.. off-diagonal values of around 0.5).
E. Decoder weight analysis
A useful method for gaining insight into the behaviour of our models is through examining the final weights of the decoder. Specifically, we visualise W W, an N X matrix, for three scenarios: when equals the true sparse dimensionality, when N exceeds it, and when N smaller than the true dimensionality.
When N is smaller than the true sparse dimension (Figure 14], sparse coding again attempts to maintain orthogonality, though it is constrained by the reduced number dimen- sions. The SAE and MLP models, in contrast, continue to exhibit superposition, with off-diagonal elements close to 1.0. MLPs, however, show somewhat weaker correlations between features, as indicated by off-diagonal values around 0.5 in some instances.
In the case where N matches the true sparse dimension, we observe the matrix D D for the learned decoder ma- trix D after training. Figure 12 this scenario for N = 16 and M = 8, without applying decoder column unit normalisation. For sparse coding, the matrix DT D is approximately an N N identity matrix after softmax nor- malisation. This means that the model has learned set of basis vectors where each column of D is nearly orthogonal to all others, indicating that the features are independent.
F. MLP Ablations
In contrast, both the sparse autoencoder (SAE) and the mul- tilayer perceptron (MLP) show DT D matrices with a mix of diagonal and off--diaoonal elements. these cases, many
We also wanted to understand more fine-grained detail how the hidden width of the MLPs affects the key metrics of
15

---

Computc Optimal Inference and Provable Amortisation Gap in Sparse Autoencoders
SparseCoding
Loss Test
MCC TeST
1.00
Witrral
Column Index
0.75
0.25
10
Figure 14: Visualisation of DJ D when N is smaller than the true sparse dimension.
0.900
24
Spmn Socks
NJP 11- 64
0.0
242
Spmm 2003
10 20 64
64
07
63
0.03
1850
Soes
Figure 16: Effects on dictionary learning performance for our three models, with and without bias. Including a bias has no statistically significant effect on results.
Figure 15: Varying the hidden width of an MLP autocncoder in varying difficulties of dictionary learning regimes. Each data point is MLP trained for 50,000 iterations with learning rate of le-4.
H. Comparison with traditional dictionary learning methods
performance, in different regimes N, M and K. We show this in Figure 15 We use varying hidden widths and three different combinations of increasingly difficult N, M, K to test this. We train for 50,000 iterations with learning rate of le-4. We see that MCC (both latent and dictionary) increases approximately linearly with hidden width, with a slight drop-off at a hidden width of 512 (most likely due to underfitting). We also see similar trend terms of reconstruction loss, with the most difficult case being most sensitive hidden width.
To provide comparison with traditional dictionary learn- ing methods, incorporated the Least Angle Regression (LARS) algorithm to compute the Lasso solution in our experimental framework.
The traditional dictionary learning problem can be formu- lated as a bi-level optimisation task. Given a set training samples X [a1,.... . Fnd E Rmxn, we aim find dic- tionary E mm and sparse codes
........ an,] & that minimise the reconstruction error while enforcing sparsity constraints:
G. Including a bias parameter
min D.A
2 ||, Dov + Allaslli)
We examine the effect of including a bias parameter in our models Figure I6 Elhage et al. (2022) noted that a bias allows the model to set features doesn't represent their expected value. Further, ReLU in some cases make "negative interference" (interference when negative bias pushes activations below zero) between features free. Further, using a negative bias can convert small positive interferences into essentially being negative interferences, which helps deal with noise.
subject to ||d,lla 5 1 for 1, 1,...,ka where d, represents the j-th column of D, and À 0 a regularisation parame- controlling the trade-off between reconstruction fidelity and sparsity.
In our experiment, we employed the LARS algorithm to solve the Lasso problem for sparse coding, while alternat- ing with dictionary updates to learn the optimal dictionary. Specifically, we used the scikit -eatn implementation of dictionary learning, which utilises LARS for the sparse coding step. The algorithm alternates between steps: (1) sparse coding, where LARS computes the Lasso solution for fixed D, and (2) dictionary update, where D is optimised while keeping the sparse codes fixed.
However, Theorem 3.J doesn't rely on having biases, and although it generalises the case with biases, we would like to be able to simplify
main
study by not including them. Thus, we show in Figure that that biases have no statistically significant effect on reconstruction loss, latent MCC, dic- tionary MCC, or LO, for any of the models, except for the Lo and MCC the MLP, which achieves a higher MCC without bias at the cost of greater L..
To evaluate the performance this traditional approach, we generated synthetic data following the same procedure as in
16

---

Computc Optimal Inference and Provable Amortisation Gap in Sparse Autoencoders
Loss, K
0.0525
H.I.1. ADVANCED SPARSE AUTOENCODER
20
30
0.0450
TECHNIQUES
25
22-
0.0375
0.03300
Sparse autoencoders trained with L1 regularisation are sus-
0.0225
ceptible to the shrinkage problem. WWright & Sharkey (2024)
10-
0.0075
identified feature suppression in SAEs, analogous to the activation shrinkage first described by [Tibshirani (199)) as a property of LI penalties. The shrinkage problem occurs when LI regularisation reduces the magnitude of non-zero coefficients to achieve a lower loss, potentially underesti- mating the true effect sizes of important features.
201
150
200
156
0. 0.0000
componenti)
componests)
Latent MCC. k-3
Latent MCC k-6
Latent MCC k-9
1.00
0.08
25
23
0.76
0.64
20
20
20-
0.52
25
15
12-
0.40
30
16-
0.28
Several techniques have been proposed to mitigate this is-
210
0.16
(no. componenti)
N (no. componetts)
N (re. components)
sue:
MCC Dict, km3
MCC Dict, k
MCC Dict, k-g
0.975
30
20
30
0.930
ProLU Activation: Taggart
(2024) introduced the
25-
25-
0.885
ProLU activation function to maintain scale consis- tency in feature activations.
0.840
0.795
10-
0.705
Gated SAEs: Rajamanoharan et al. 2024a) developed Gated Sparse Autoencoders, which separate the pro- cess of determining active directions from estimating their magnitudes. This approach limits the undesir- able side effects of LI penalties and achieves Pareto improvement over standard methods.
100 201 componenta)
100
200 components)
150
0.660
componentts)
Figure 17: Performance of Least-Angle Regression (LARS) to compute the Lasso solution using our synthetic dictionary learning setup. In general, when comparing to Figure 5 we see improvement when using LARS over our naive implementations of SAEs, MLPs and sparse coding, across loss, latent MCC, and dictionary MCC.
Rajamanoharan et al. (024b) pro- posed JumpReLU SAES, which set activations below a certain threshold to zero, effectively creating non- linear gating mechanism.
JumpReLU SAEs:
main experiments, with N 16 sparse sources, M measurements, and K = 3 active components per timestep. We trained the dictionary learning model on the training set and evaluated its performance on the held-out test set. Performance was measured using the Mean Correlation Coefficient (MCC) between the predicted and true latents, as well between the learned and true dictionary elements.
Top-k SAEs: Originally proposed by Makhzani & Frey (2013), top-k SAEs were shown by Gao et al. (2024) prevent activation shrinkage and scale effectively to large language models like GPT-4
1.1.2. OPTIMISED SPARSE CODING APPROACHES
Our initial sparse coding model, using uniformly ini- tialised latents and concurrent gradient-based optimisation of both sparse codes and the dictionary, suboptimal. The sparse coding literature offers several more sophisticated approaches:
The results of this, presented in Figure 17 make clear that traditional sparse coding significantly outperforms our vanilla gradient-based implementations, particularly in terms of latent MCC and dictionary MCC. Whilst our results from the main body show that there does exist a significant amortisation gap between the vanilla implementations of each of the approaches. we should also attempt to under- stand how the optimised versions of each method compare. We discuss this in the following subsection.
Leaast Angle Regression (LARS): Introduced by Efron jtt al. (2004), LARS provides an efficient algorithm for computing the entire regularisation path of Lasso. It is particularly effective when the number of predictors is much larger than the number of observations.
H.1. Optimised Sparse Autoencoders and Sparse Coding
Orthogonal Matching Pursuit (OMP): Pati et al. (193) proposed OMP as greedy algorithm that iter- atively selects the dictionary element most correlated with the current residual. offers a computationally efficient alternative to convex optimisation methods.
Our initial implementations of sparse autoencoders (SAEs) and sparse coding, while functional, are far from optimal. They represent the minimum computational mechanisms required to solve the problems as we have formulated them. However, more sophisticated approaches can significantly improve performance and address inherent limitations.
Future work will involve pitting these against the optimised SAE architectures discussed above.
17

---

Computc Optimal Inference and Provable Amortisation Gap in Sparse Autoencoders
H.2. Top-k sparse coding
10 MSE Loss (L1 weight O 1.00S!
LO MCC [LL weight: O 0.005)
SparneCaing.
Building on this exploration, we introduced top-k sparse coding approach. We aimed to determine whether (1) set- ting very small active latents to zero would improve per- formance and (2) optimising with a differentiable top-k function, rather than using exponential or ReLU functions, could yield further benefits.
MSE Loss
SpuraCeding Inhennetttrt SparsCCaingg Optrushho
0.6-
Saartaceding 11
Saa Carratatiieeadee.
0.0
0.2-
ScartaCoding Spp
0.0
0.2
0.4
0.66
0.0
1.0
0.0
0.2
3.1
0.
0n
1.0
LO
Lows
LG
Lows
vs MSE Losz weight: 0.05)
vs MCC (L) weight 0.05!
Sparocating 11
0.0
0.3
Figure 18 presents the results of these experiments. We first trained the sparse coding model for 20,000 steps on the training data and optimised for additional 1, 1,000 steps on the test data. During this process, measured mean squared error (MSE) loss, latent MCC, and the Lo norm of the latent codes. Due to the presence of very small active latents, all initial setups led to an Lp value of 1.0, indicating that all latents were active, as shown by the blue star in the figure. We also show a sparse autoencoder trained with different L penalties comparison.
MSEE ueos
SparncCating
0.6-
0.22
Sol Socdind inkee lop.
0.0
SparsaCoding Sorintd
LG Locs
Lo Lass
LG vs MSE Loss (L1 weight 0.5)
LO vs MCC (LI weight: 0.51
0.0
0.3
SpansCading Jrhenertiaa
Sparocading Optirtattta
0.6-
0.4
0.1
0.2
0.0
Next, we applied top-k operation to enforce sparsity by setting all but the top-k largest activations to zero. This process resulted in improved Lo values, but the MSE loss and MCC results indicated that the top-k optimisation itself was hampered by an insufficient learning rate. We hypothe- sise that with proper tuning of hyperparameters, we could achieve Pareto improvements by using the top-k function directly, rather than applying it to exponentiated codes.
0.0
0.2
0.6
0.8
1.0
6.0
0.2
Loss
Loss
Figure 18: Comparison of Ly loss vs. MSE loss and Lu loss vs. MCC for Sparse Coding with I.1 regularization, top- k inference, and top-k optimization, alongside results for Sparse Autoencoder. Blue stars represent the initial model's performance, while curves illustrate the results of applying top-k sparsity.
We believe that further adjustments to the optimisation pro- cess, including a higher learning rate for top-k functions, could result in better performance. Additionally, apply- ing the top-k function directly, without exponentiating the codes, may offer further gains in performance and sparsity.
Training: For training, we calculate the FLOPs as:
FLOPs Matl (FLOPsneeanl FLOPS FLOP + FLOPs
I. Measuring FLOPs
Rb is the effective number of iterations, where Heff = Rsteps n, Natops is the number of training steps, n& is the batch size, and n, is the total number of samples. The component FLOPs are calculated as:
To quantify the computational cost of each method, we calculate the number of floating-poin operations (FLOPs) required for both training and inference. This section details our approach to FLOP calculation for each method.
FLOP Sforward FLOPssC-inf
I.1. Sparse Coding
FLOP Stoss = 2. + Nnb
Spackwadd R 2 FLOPStswadd
For sparse coding, we calculate FLOPs for both inference and training separately.
Nnn + -MN
learning D
FL.OPSuddate
The number of FLOPs for inference in sparse coding is given by:
Inference:
N
otherwise
3MN + Nn, if learning D
1.2. Sparse Autoencoder (SAE)
FLOPPsC
(15)
2MN Nn. otherwise
For the sparse autoencoder, we calculate FLOPs for both training and inference.
where M is the number of measurements, N is the number of sparse sources, and n, is the number of samples. The additional MN term when learning D accounts for the normalisation of the dictionary.
Training: The total FLOPs for SAE training is given by:
ML.OPSSAE train = Deel S + FL.OP Spackward)
18

---

Computc Optimal Inference and Provable Amortisation Gap in Sparse Autoencoders
where Nett is defined as before, and:
J.I. Feature Interpreter Prompt
5MN + N if learning D
We use feature interpreter prompt to provide an explana- tion for neuron's 's activation. The interpreter is tasked with analysing neuron's 's behaviour, given both text examples and the logits predicted by the neuron. Below is summary of how the interpreter prompt works:
Saaward
4MN + N
otherwise
Shackward
N + (2NM + N) 2NM+
2
if learning D
2(MN + N)
0
otherwise
You are a meticulous Al researcher conducting an investiga- tion into a specific neuron in language model. Your goal is to provide an explanation that encupsulates the behavior of this neuron. You will be given list of text examples on which the neuron activates. The specific tokens that cause the neuron to activate will appear between delimiters like <<this>> >. If sequence of consecutive tokens causes the neuron to activate, the entire sequence of tokens will be con- tained between delimiters <<just like this>>, Each example will also display the activation value in parentheses following the text. Your task is to produce a concise descrip- tion of the neuron's 'ss behavior by describing the text features that activate it and suggesting what the neuron' 's role might be based on the tokens it predicts. If the text features or predicted tokens are uninformative, you can omit them from the explanation. The explanation should include an anal- ysis of both the activating tokens and contextual patterns. You will be presented with tokens that the neuron boosts in the next token prediction, referred as Top-logi t.s, which may refine your understanding of the neuron's 's be- havior. You should note the relationship between the tokens that activate the neuron and the tokens that appear in the Top_logits ts list. Your final response should provide a formatted explanation of what features of text cause the neu- ron to activate, written as: (EXPLANATION): explanations
For SAE inference, the FLOPs are calculated as:
Inference:
FLOPSSAE-inf = (4MN + +N) n,
I.3. Multilayer Perceptron (MIP)
For the MLP, we calculate FLOPs for both training and inference, considering a single hidden layer of size H.
Training:
The total FLOPs for MLP training is given by:
FLOPSMLP = Dell' (FLOP Storward + PLOOPStckwwer
where:
2MH + HI + 21N + N 2NM + MN if learning D otherwise
FLOPS
2M + + 21N + 2NM
ELOPS (2811 -0088-N+++00 NY B (MMP + 41)+ 2 RMT 21 +
where we add M to PhoPShackward if learning D, and not otherwise.
Inference: For MLP inference, the FL.OPs are calculated as:
<your
ILOPSMLP = (2MH + И + NHN + N + +2NM n,
L.4. SAE with Inference-Timi Optimisation (SAE+ITO)
J.2. Feature Scorer Prompt
For SAE+ITO, we calculate the additional FLOPs required for optimising the codes during inference:
After generating feature interpretations, we implemented scoring prompt to predict whether specific feature is likely to activate on a given token. This ensures that the explanations generated by the interpreter align with actual activations. The scoring prompt tasks the model with evalu- ating if the tokens marked in the examples are representative of the feature in question.
FLOPSmro = (MN + N (4MN + 2M111N)) .ns
where Fäier is the number of optimisation iterations per- formed during inference.
You are provided with text examples where portions of the sentence strongly represent the feature, with these portions enclosed by << and >>. Some of these examples might be mislabeled. Your job is to evaluate each example and return a binary response (1 if the tokens are correctly labeled, and O if they are mislabeled). The output must be a valid Python list with Is and Os, corresponding to the correct or incorrect labeling of each example.
J. Automated interpretability
In this section, we describe the automated interpretabil- ity pipeline used to understand and evaluate the features learned by sparse autoencoders (SAEs) and other models in the context of neuron activations within large language models (LLMs). The pipeline consists of two tasks: feature interpretation and feature scoring. These tasks allow us to generate hypotheses about individual feature activations and to determine whether specific features are likely to activate given particular token contexts.
19

---

Computc Optimal Inference and Provable Amortisation Gap in Sparse Autoencoders
Table 1: Example interpretations from MLP and SAE neurons, shown with their Fl scores.
Model
Interpretation
F1 Score
MLP
Activates on the token "to" when used to introduce an infinitive verb indicating purpose intent. promoting verbs that express actions or goals
0.899
Activates concrete and functional nouns specific actions that often part of 0.899
list enumeration
Activates on the token "than" part of comparative structure, aiding in predicting 1.000
terms used for comparison or establishing norms
SAE
Activates on parentheses and colons used in structured timestamps, date-time formats, 1.000
and categorisation notations
Activates on tokens within contexts related to font and text styling options, typically
1.000
presented in technical or settings menu format Activates on the token "first" within the formulaic expression "first come, first served basis"
1.000
J.3. Evaluation of Automated Interpretability
To evaluate the accuracy of the interpretations generated by the feature interpreter and feature scorer, compared model-generated explanations against held-out examples. The evaluation involved calculating the Fi-score, which was done by presenting the model with a mix of correctly labeled and falsely labeled examples. The model was then tasked with predicting whether each token the example repre- sented feature not, based the previously generated interpretation. By comparing the model's predictions with ground truth labels, we can assess how accurately the feature interpretation aligns with actual neuron activations. This process helps validate the interpretability of the features learned by SAEs, MLPs, and other models.
This pipeline is based on the work Juang al. (2024). which itself builds the work of others. Bills et al. (2023
used GPT-4 to generate and simulate neuron explanations by analyzing text that strongly activated the neuron. Bricken let al. (2023) and Templetoon] (2024) applied similar tech- niques to analyze sparse autoencoder features. [emmletoo (2024) also introduced a specificity analysis to rate expla- nations by using another I.I. to predict activations based on the LLM-generated interpretation. This provides a quan- tification of how interpretable given neuron feature actually is. Gao et al. (2024) demonstrated that cheaper methods, such as Neuron to Graph (Foote et al.), which uses n-gram based explanations, allow for a scalable feature labeling mechanism that does not rely on expensive LLM computations.
Table presents illustrative examples of interpretations from both MLP and SAE neurons, showing how automated pipeline can identify specific linguistic patterns and assign quantitative reliability scores.
20


Reviews:
[{"review_id": "rwBAnJ0eW4", "reviewer": "Reviewer_JWnB", "summary": "The authors study Sparse Autoencoders (SAE), first showing that simple linear-nonlinear encoding leads to an amortisation gap. Next the authors compare different SAE architectures on synthetic settings, showing that better architectures can beat standard SAE in this setting.\nFinally they study the interpretability by comparing different models on the pre-activations of a GPT2 layer.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "# Response to Reviewer JWnB\n\nWe thank the reviewer for their time spent evaluating our paper. We believe there are several misunderstandings in the review that we would like to address, as they appear to have led to an incomplete assessment of our work.\n\n## Regarding Theorem 3.1\n\nThe theorem is indeed rigorous and makes minimal assumptions about the sparse prior $P_S$. The only requirement is that the support of $P_S$ consists of vectors with at most $K$ non-zero entries. This is explicitly stated in the theorem: \"Let $S=\\mathbb{R}^N$ be $N$ sources following a sparse distribution $P_S$ such that any sample has at most $K \\geq 2$ non-zero entries, i.e., $||s||_0 \\leq K, \\forall s \\in \\text{supp}(P_S).$\"\n\nThe proof holds regardless of whether $P_S$ is deterministic or stochastic. It's a constructive proof that shows a specific set of vectors (the standard basis vectors and their sums) cannot be correctly encoded simultaneously by a linear-nonlinear encoder. Since these vectors are valid under any sparse distribution with max sparsity $K \\geq 2$, the conclusion holds generally.\n\nRegarding the conclusion after equation (8): This follows directly from linear algebra. If $S'$ must be diagonal to correctly represent the sparse codes after ReLU, but also must have rank ≤ M < N due to the dimensionality constraints, we have a contradiction since a diagonal matrix with non-zero diagonal entries has rank N.\n\n## Figure 3 Non-monotone Behaviour\n\nThe non-monotonic behaviour observed in Figure 3 (particularly for SAE+ITO) is a deliberate focus of our analysis, not an oversight. As stated in Section 4.2: \"SAE+ITO initialised with SAE latents exhibits distinct, stepwise improvements throughout training, ultimately achieving the highest MCC.\" This pattern reveals interesting dynamics of how inference-time optimisation interacts with the learned dictionary during training.\n\n## Figure 7 Visualisation\n\nThe circles in Figure 7 represent statistical outliers in the distribution of F1 scores, following standard boxplot conventions. We will add this clarification to the figure caption to avoid confusion.\n\n## Interpretability of F1 Score Metric\n\nThe F1 score is a widely recognised and highly interpretable metric for classification tasks. In our case, it measures how well a second instance of GPT-4o can predict neuron activations based on explanations from a first instance. This is explicitly described in Section 6: \"The model predicted which examples should activate the feature based on the first instance's explanation, allowing us to compute an F1-score against the ground truth.\"\n\nThe approach is standard in the field and follows established methods from Anthropic, EleutherAI, and other research groups cited in Appendix A.7.\n\n## Relevance of Synthetic Experiments\n\nWhile our synthetic experiments use moderate dimensionality for clarity and computational efficiency, we demonstrate their relevance to practical problems in multiple ways:\n\n1. We explicitly test larger-scale experiments in Appendix A.3 (N=1000, M=200, K=20), showing that our findings hold and even strengthen at larger scales.\n\n2. We examine non-uniform feature distributions in Appendix A.4 to better match real-world latent spaces.\n\n3. Most importantly, we validate our findings on actual GPT-2 residual stream activations in Section 6, showing that the principles discovered in synthetic settings transfer to real neural networks.\n\nThe synthetic experiments provide a controlled environment where ground truth is known, allowing us to rigorously evaluate the amortisation gap that forms the theoretical foundation of our work.\n\n## Model Complexity and SOTA Comparison\n\nOur focus is on the fundamental mechanisms behind sparse encoding and dictionary learning, rather than specific architectural innovations. The models in Section 3.3 represent core architectural categories (linear-nonlinear encoders, MLPs, sparse coding) that underlie most advanced SAE variants.\n\nIn Appendix A.7.2, we discuss how our findings relate to advanced SAE architectures like top-k SAEs, JumpReLU, Gated SAEs, and ProLU activations. These are cutting-edge developments in the field, many published in the past six months. Our work provides theoretical and empirical foundations that explain why these architectural innovations yield performance improvements.\n\nWe appreciate the opportunity to clarify these points and believe our work makes significant contributions to both the theoretical understanding and practical application of sparse autoencoders for neural network interpretability.", "claims_and_evidence": "No concern", "methods_and_evaluation_criteria": "The proposed methods seem sensible, see questions below for potential issues", "theoretical_claims": "Skimmed the proofs, there is a question about Theorem 3.1 in questions down below", "experimental_designs_or_analyses": "Did not check the experiments in detail.", "relation_to_broader_scientific_literature": "This paper furthers the understanding of sparse autoencoders, and may be interesting to both theorists and practitioners in the field.", "essential_references_not_discussed": "None that I know of", "other_strengths_and_weaknesses": "Strengths:\n- Good Presentation\n- Clearly written, easy to read\n\nWeaknesses:\n- The theoretical claim seems to be a heuristic rather than a theorem, see questions below\n- It is unclear how much the experiments support the made claims see discussion below. Given that there is little in terms of theoretical results the experiments feel a bit weak.", "other_comments_or_suggestions": "- A more close up version of figure 2 would be good to see the differences better at convergence", "questions_for_authors": "- The proof of theorem 3.1 does not seem rigorous to me, in particular I have the following questions. What exactly is the assumption on the sparse prior $P_S$? If we choose a deterministic function for example projection on the first $K$ components than this is a very different setting then say i.d.d sparse entries, I am not sure that the proof is true for the former. \nThe conclusion after equation (8) should be made rigorous, right now I am not sure how this follows.\n- In FIgure 3, why does SAE exhibit non-monotone behaviour?\n- What do the circles in Figure 7 represent? This should be explained in the description.\n- Why is the metric used for Figure 7 a good choice? Given that it is not a very interpretable metric it would be good to have more points of comparison, are there other known baselines that one could compare this to?\n- The main contribution of this paper are the experiments on the synthetic data, but the used data seems quite far from real data both in terms of dimension and complexity. Why do you believe that these results reflect well what is going on, and why the intuition from these examples should translate to practical problems\n- Related to the above point, the models as introduced in 3.3 seem rather simple. I am not an expert in the area, so I do not know what the most sensible sota models would be, but why are more practical models not used to evaluate on the synthetical setting.", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Jws10zMhWt", "reviewer": "Reviewer_STGo", "summary": "The authors prove that typical SAEs (ReLU, JumpReLU, TopK) cannot recover the optimal encoding (sparse coefficients) compared to sparse coding methods that solve individual examples iteratively.\nThey empirically demonstrate this using multiple synthetic experiments.\nFinally, they apply sparse coding using the locally competitive algorithm (LCA) to real LLM activations and find that non-linear encoders (an MLP) and LCA lead to more interpretable features than a ReLU encoder as evidenced by automatic interpretability scoring methods.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "# Response to Reviewer STGo\n\nWe appreciate the reviewer's thoughtful analysis of our work and their recognition of the theoretical contribution. We would like to address several points regarding the practical implications of our findings.\n\n## Regarding the claim about optimality gap in LLM activations\n\nWe agree that the connection between our theoretical results and practical LLM applications deserves further elaboration. Our work demonstrates that the amortisation gap exists not just theoretically but also empirically across various settings, including LLM activations.\n\n### Model choice and scale\n\nWhile GPT-2 Small (124M parameters) is indeed older than more recent models, it remains a standard benchmark in the mechanistic interpretability literature for several reasons:\n\n1. **Established baseline**: Numerous interpretability papers continue to use GPT-2 as a testing ground, including recent work by Anthropic, EleutherAI, and others. This facilitates comparison with existing literature.\n\n2. **Computational accessibility**: Our experiments involved training multiple models on hundreds of millions of tokens, which becomes prohibitively expensive with larger models.\n\n3. **Feature stability**: The principles of superposition and sparse feature representation appear consistent across model scales, as demonstrated by recent work finding similar phenomena in models from GPT-2 to GPT-4 and Claude.\n\nThat said, we acknowledge that verifying our findings on larger models would strengthen our claims, and we plan to pursue this in future work.\n\n### Baseline improvement\n\nRegarding the baseline SAE implementation, we intentionally tested a standard ReLU SAE as it represents the most widely used architecture in current interpretability research. We agree that newer architectures like JumpReLU and TopK offer improvements, and we discuss these in Appendix A.7.2.\n\nOur theoretical result applies to this entire class of models, as they all rely on fixed function approximations rather than iterative optimisation. The empirical gap we observe provides an explanation for why these architectural innovations improve performance - they partially address the amortisation gap through more sophisticated function approximation.\n\n### Computational efficiency at scale\n\nThe reviewer raises an excellent point about computational efficiency comparisons at scale. While our LCA implementation was not fully optimised, we can provide some context:\n\n- Training the LCA model took approximately 3x the computation time of the SAE model due to the additional gradient steps per batch.\n- During inference, LCA required approximately 20x the computation of the SAE.\n\nHowever, as noted in recent work (e.g., Nanda et al., 2024), inference-time optimisation approaches can be made significantly more efficient through techniques like matching pursuit and careful algorithm selection. The fundamental trade-off between amortised and iterative approaches remains, but the computational gap can be narrowed considerably.\n\n## On empirical significance beyond theory\n\nWhile our theoretical contribution stands independently, we believe the empirical results are meaningful for several reasons:\n\n1. The significant interpretability improvement of the MLP encoder (median F1 of 0.83 vs 0.6 for SAE) suggests that even modest increases in encoder expressivity can substantially improve feature extraction.\n\n2. Our work offers an explanatory framework for why recent SAE variants achieve better performance, connecting theoretical understanding with practical innovations.\n\n3. The results suggest promising directions for further research, such as developing more expressive encoders or hybrid approaches that balance computational efficiency and encoding quality.\n\nWe thank the reviewer for highlighting areas where our practical evaluation could be strengthened, and we hope to address these in future work with larger-scale evaluations across multiple model families and more sophisticated baselines.", "claims_and_evidence": "Claim: Simple linear-nonlinear SAE encoders (linear layer + ReLU, for example) are provably suboptimal at sparsely encoding data than iterative sparse coding algorithms.\n\nThis is a theorem that's proved in Appendix A.\n\nClaim: This same optimality gap holds on SAEs applied to large language model activations.\n\nThis is demonstrated via training a ReLU SAE, an MLP SAE and an LCA encoder on GPT-2 small activations, then evaluated using existing automated scoring methods.\nThe MLP SAE and the LCA encoder are more interpretable.\n\nI believe this claim is overstated.\nThe authors note that their optimality gap is only globally suboptimal, with adversarially chosen sparse codes.\nIn practice, I am suspicious that LLM activations are so evenly spaced that this optimality gap is meaningful.\nI would be much more convinced that the optimality gap matters in practice if:\n\n* The authors used a more recent or larger LLM. GPT-2 small is very small (120M parameters) and is 6 years old at this point. Qwen2 has a 500M parameter model, Pythia has a 160M parameter model, OpenELM has 270M and 450M parameter variants, etc. If model size is not an issue, then something like LLama3 8B would be great.\n* The authors used a more improved baseline. While their proof is for any linear-nonlinear encoder, more recent works in SAEs (JumpReLU, TopK, BatchedTopK) notably improve upon the MSE and L0 tradeoff. Perhaps a stronger baseline means that the optimality gap is not an issue in practice.\n* While FLOPs and efficiency are extensively compared in the synthetic experiments, there is no comparison of compute efficiency at scale. How does LCA compare to the amortized encoders when applied to real LLMs?", "methods_and_evaluation_criteria": "The authors use synthetic datasets to demonstrate that their proof holds empirically (great!).\nThen they use GPT-2 small as a benchmark for \"real-world\" evaluation.\nAs I stated above, I think the LLM is too small/old and the baseline SAE (a vanilla ReLU) is not a strong enough baseline.", "theoretical_claims": "I did not check the correctness of the proof.", "experimental_designs_or_analyses": "The experimental design is fine. Like I said before, I think the SAE baseline is too weak.", "relation_to_broader_scientific_literature": "The work is well-positioned in the scientific literature.\nI think it is better-positioned than many other SAE works because it specifically discusses sparse coding and pre-SAE methods for solving this kind of problem.", "essential_references_not_discussed": "No essential references are missing.", "other_strengths_and_weaknesses": "* I appreciate the theoretical analysis of SAEs. Current SAE works are overwhelmingly empirical at the moment.\n* The synthetic experiments are well-designed to demonstrate a gap.", "other_comments_or_suggestions": "I am happy to accept this paper if ICML values theoretical contributions. While I personally am more concerned with empirical, \"practical\" results, I understand if this venue has different goals.", "questions_for_authors": "N/A", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Hy32DOxWDu", "reviewer": "Reviewer_WhEs", "summary": "This paper performs a systematic investigation of the various modelling choices for SAEs, particularly the choice of the encoder. The paper shows both theoretically and empirically (on synthetic data) that there exists an \"amortization gap\" in the sense that SAEs are unable to recover latent features due to their linearity. Then, the paper performs an analysis of the interpretability of representations upon various choices for the encoder (LCA, MLP) and find that MLP features are far more interpretable than either canonical SAE or LCA; contradicting the folk belief that non-linear encoders in SAEs can lead to non-interpretable features.", "questions": "", "limitations": "", "rating": 3, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "# Response to Reviewer WhEs\n\nWe thank the reviewer for their thoughtful engagement with our paper and their recognition of its theoretical and empirical contributions. We address their specific concerns below.\n\n## Regarding inconsistencies between synthetic and real data experiments\n\nThe reviewer correctly notes that there are differences in the methods evaluated on synthetic versus real data. This was primarily due to implementation and computational constraints:\n\n1. **SAE+ITO on real data**: We did not implement SAE+ITO for the GPT-2 experiments due to the significant computational cost of performing inference-time optimisation on hundreds of millions of tokens. While feasible on synthetic data, this would have been quite expensive at LLM scale, particularly in addition to our LCA method.\n\n2. **LCA versus sparse coding**: The LCA method used in our real data experiments is an implementation of sparse coding that uses competitive dynamics to achieve sparse inference. We chose LCA specifically because it has an established history in the sparse coding literature and offered a more efficient implementation path for our large-scale experiments than our synthetic sparse coding method.\n\nWe acknowledge that using the same set of methods across all experiments would have provided better consistency. In future work, we plan to implement a more unified experimental framework that can efficiently scale to larger models and datasets.\n\n## Qualitative feature visualisation\n\nWe agree that providing qualitative visualisations and examples of the features discovered by different methods would strengthen our paper. While our automated interpretability evaluation provides quantitative evidence for the superior interpretability of MLP features, examples would offer readers a more intuitive understanding of these differences.\n\nIn the final version, we will add an appendix section with representative feature examples from each method, including:\n- Raw activation patterns on top activating tokens\n- Generated feature interpretations \n- Validation examples showing correct/incorrect feature activation predictions\n\nWe note that while recent work by Heap et al. (2024) raises important questions about interpretability pipelines, their concerns primarily relate to interpreting features in randomly initialised transformers, not to the comparison of different encoding methods applied to the same model. In addition to this, there have been demonstrations that trained transformers still out-perform randomly initialised transformers on autointerp baselines. Our work focuses on the relative performance differences between encoding strategies rather than absolute claims about interpretability.\n\n## Clarification on dimensionality settings (N > M versus M > N)\n\nThe reviewer raises an excellent question about the dimensionality settings and their implications. To clarify:\n\nOur theorem addresses the case where N (number of sources/latent dimensions) > M (observed dimensions), which indeed matches the typical SAE setting where the latent space is larger than the activation space. The key insight is that in this regime, a simple linear-nonlinear encoder cannot perfectly recover all possible sparse codes from their lower-dimensional projections, even when such recovery is theoretically possible with iterative methods.\n\nRegarding the contractive autoencoder settings with M > N, this represents a different regime than what SAEs typically address. In that case:\n\n1. The encoding is undercomplete rather than overcomplete\n2. The primary goal is often dimensionality reduction rather than disentanglement\n3. The sparsity constraint becomes less critical since there is no inherent superposition\n\nOur theoretical result does not imply that SAE architectures would be better suited to the M > N setting. Rather, it suggests that in the standard N > M setting where SAEs operate, more expressive encoders (like MLPs) can reduce the amortization gap and improve feature recovery.\n\n## Scaling to larger models\n\nWe acknowledge the limitation that our experiments are conducted on GPT-2 Small. While computational constraints prevented us from scaling to larger models, we have encouraging evidence that our findings would generalise:\n\n1. Our synthetic experiments at larger scales (Appendix A.3) show that the amortisation gap becomes more pronounced as dimensionality increases\n2. The theoretical result is independent of scale\n3. Recent work applying SAEs to larger models (e.g., Gao et al., 2024) has shown qualitatively similar features and challenges across model scales\n\nWe believe our contributions provide valuable insights despite these limitations, and we hope to validate our findings on larger models in future work.\n\nThank you again for your thoughtful review and constructive suggestions for improving our paper.", "claims_and_evidence": "The claims made by this paper is well supported both theoretically and empirically.", "methods_and_evaluation_criteria": "The proposed methods and criteria make sense for this study.", "theoretical_claims": "The theoretical claims seem correct based on my check of the proof. However, I believe a detailed discussion of the setting (num. data sources N > dimensionality M), especially in relation to its practical significance, can help readers. Please also see my clarification question at the end regarding this issue.", "experimental_designs_or_analyses": "The experimental design is mostly sound. However, there seem to be some inconsistencies between experiments on synthetic data and those on real data. In particular, while the experiments on synthetic data involve sparse coding, and SAE + ITO; those baseline were missing from the real data experiments. Instead, another baseline, LCA was introduced without discussion in that section. It might be better to have the same set of methods evaluated on both synthetic and real data to understand the trade-offs between latent variable recovery and interpretability.", "relation_to_broader_scientific_literature": "This paper adds to the nascent literature on the theoretical understanding of SAEs (e.g.: Menon et al., \"Analyzing (In)Abilities of SAEs via Formal Languages\"). While most SAE work has been empirical in nature (e.g.: Gao et al., \"Scaling and evaluating sparse autoencoders\"), this work sheds light on a key aspect, that being the choice of the encoder, and its relation with classical dictionary learning.", "essential_references_not_discussed": "Nothing \"essential\" is missed to the best of my knowledge.", "other_strengths_and_weaknesses": "Other Strengths:\n- The paper performs a systematic evaluation of SAE modelling choices and show that MLP-based encoders seem to outperform the more commonly used linear encoders. If these findings hold for larger-scale models and datasets, this can influence the choice of SAE architectures in future works.\n\nOther weaknesses:\n- The experiments are performed on (1) synthetic data, and (2) real data, but small scale toy GPT-2 models. Given this, it is unclear whether the findings translate to larger scale models and datasets. \n\n- The paper does not include a qualitative discussion or visualization of the features discovered by the MLP models, SAEs & LCA, to help better judge the benefits of using the MLP encoder. This is especially important as recent work (Heap et al., \"Sparse Autoencoders Can Interpret Randomly Initialized Transformers\") has raised questions regarding the efficacy of commonly used auto-interpretability piplines. Hence only presenting benefits in terms of auto-interpretability scores (Figure 7) may lead to spurious findings.", "other_comments_or_suggestions": "N/A", "questions_for_authors": "**Clarification question**: An interesting aspect of the theory was that the statement holds only when number of sources N > data dimensionality M; which seems to be the exact scenario for modern SAEs, i.e., their latent dimensionalities are much larger than their feature size. On the contrary, usual assumptions for the data generating processes involve assuming M > N, which seems to better correspond to the settings in which contractive autoencoders operate. My question is: does this imply that SAE type architectures might be better suited to the M > N setting as opposed the N > M setting that is studied in this paper?", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Meta-review: This paper uses compressed sensing theory to prove that sparse autoencoders (SAEs) are insufficient for accurate sparse inference. It also presents relevant experiments showing increased interpretability using more expressive encoders. Given the recent interest in SAEs, there is an interest in comparing them with classic approaches to sparse estimation.
Decision: Accept (poster)