reviews: 
Comments: The authors present a well written paper which makes multiple significant contributions to the field of language interpretability. The paper does not explicitly pose a set of questions, but the subject matter implies several concerns:
1. Do sparse autoencoders scale to larger models? Does this require more latents than for larger models?
2. Can sparse autoencoder training be improved? For example, can it be made simpler via use of a top-K activation function? 
3. How should we evaluate sparse autoencoders? Are there alternatives to the sparsity-reconstruction pareto-frontier framing?

The paper makes a number of contributions. 
1. First, they propose a novel training recipe based on the use of the Top-K activation function for directly imposing sparsity in the sparse bottleneck layer. They evaluate their methods against similar proposed methods via the sparsity-reconstruction pareto-frontier framing and find their recipe is competitive with the best methods in the existing literature.
2. Second, they demonstrate scaling laws for their sparse autoencoder training recipe using the GPT4 family of models. 
3. Finally, they introduce a number of additional metrics for evaluating sparse autoencoders including loss-based metrics, metrics which rely on labeled datasets, automated interpretability and ablation sparsity. 

The strongest contribution of the paper is clearly the application of SAEs to exceptionally larger models, showing that sparse autoencoders scale to such models, but the paper also makes numerous other contributions around training and evaluation methods.
Strengths: ## Strengths by Section:

**Training Recipe**: The paper is unusually detailed with respect to the training recipe, which provides more clarity than similar publications about the training of sparse autoencoders. As noted by the authors, use of the Top-K activation function in a sparse autoencoder is not new, but it is novel in the context of sparse autoencoders trained on model activations. Importantly, it solves a previously documented issue called "shrinkage" which results in latent activations being biased towards activating less strongly than they theoretically should. 

**Scaling laws**: The paper demonstrates not only the training of sparse autoencoders on a SOTA language model, substantiating claims that this technique is scalable. Use of GPT4 provides one of the strongest possible arguments that sparse autoencoders can scale to very large LLMs. Additionally, the authors provide clear scaling laws in terms of the compute, data and number of latents. Though possibly a minor detail, we appreciate that the authors highlighted their initial incorrect prediction that there would need to be non-zero irreducible loss (as such comments help inform readers that the result was not obvious and thus increases readability and makes the significance clearer). 

**Evaluating SAEs**: Though plausibly it would be reasonable to separate work scaling sparse autoencoders from work exploring how best to evaluate them, we appreciate the attempt to rectify the current gap in the literature around how to evaluate sparse autoencoders. Contributions in this section are well motivated and varied. Section 4.3 is particularly valuable, as Neuron 2 Graph is a plausibly much cheaper way to perform automatic interpretability as opposed to the use of language models. 

## Strength by Dimension

**Originality**: While the work mostly combines existing ideas from the literature placing them in the context of sparse autoencoders trained on model activations, this hardly diminishes the contribution. Some ideas may be more clearly novel than others (such as the pre-training compute based metric that enables better interpretation of the increase in downstream loss in section 4.1). 

**Quality**: The work is generally high quality. There are no glaring issues that I can see. 

**Clarity**: Details of both the methods and reasoning are generally high quality.

**Significance**: The paper is highly significant due to contributions along multiple lines (training recipe, scaling laws and evaluation of SAEs).
Weaknesses: **Training Recipe:** 
- We appreciate that the proposed method eliminates the need for finetuning of the L1 coefficient (though choice of the number of activated latents is still required). We might interpret later analysis as attempting to identify a more principled way for setting the number of activations (setting K), though in the absence of better motivation, directly setting the number of activating latents isn’t clearly valuable. 
- Comparison of sparse autoencoder architectures is somewhat shallow, looking only at the sparsity-reconstruction pareto frontier. For example, the authors do not attempt to compare latents found by each method to establish whether they provide similar or distinct decompositions (is there often a 1:1 map between latents in sparse autoencoders trained via each method?). 
- The authors could have provided demonstrations that top-K sparse autoencoders work on toy models of superposition (as has been previously done to establish reason to prefer one architecture over another).
- Whilst the authors consider alternative methods for evaluating sparse autoencoders later in the work, they should have revisited the training recipe comparison in the context of their other evaluation methods. It is unclear whether Gated / topK SAEs would be comparable on all evaluation methodologies.
- The authors miss one possible issue with top-K activation functions. Use of the Top-K activation function means that the activation of a given latent cannot be calculated independently of others which is a minor but real obstacle to working with very large autoencoders (for example, when identifying max activating examples, one must first calculate all latent activations, then select the top-K latents). 
- Training on 64 token context sizes might affect the results. The models studied certainly can handle much longer prompts. The authors don’t note whether the SAEs generalize to activations produced during the process of larger numbers of consecutive tokens. 

**Scaling Laws**. We have no specific comments on the weaknesses of this section. 

**Evaluating SAEs**
- Though contributions in this section are valuable, there is an absence of discussion resolving possible metrics and identifying which are best to use in practice. Though many related papers likely deal with the current uncertainties of the field similarly (by presenting various possible motivations / metrics without meaningfully resolving which is best). We believe that devoting more of the discussion to comparing metrics and resolving insights found via each may be fruitful.
- (Repeating a point from earlier) The authors don’t evaluate non-TopK autoencoders via any of the new metrics leaving possibly valuable insights on the table. 
- Section 4.2 could have used a linear probe on the model activations as a baseline against which to compare the 1d logistic probe results. The authors also don’t perform any error analysis or raise any particular examples of latents which clearly track expected features of the input. While such qualitative analysis may not scale, it might possibly provide much better intuition about the results. The authors don’t address how sparse combinations of latents might jointly represent features.
- The authors could attempt to compare Neuron 2 Graph explanations to those produced by language models (which is the norm for automatic explanations of features in the rest of the sparse autoencoder literature).
Recommendation: 10
Reviewer confidence: 4
Soundness/correctness: 4
Clarity: 4