reviews: 
Comments: In this paper, the authors scale and evaluate sparse autoencoders (SAEs) on modern large language models (LLMs) across different model sizes (up to GPT4).

The main contributions are as follows:
- SAEs are trained and evaluated across different model sizes, and scaling laws are found.
- The top-k activation function is leveraged to obtain state of the art results, and its ability to avoid activation shrinkage is explored
- An auxiliary loss is devised, and it significantly reduces the amount of dead features
- New metrics are introduced for evaluation: the explanation loss (obtained using just interpretable explanations), and the probe loss (where the authors check wether the SAE reproduces known features).

My **recommendation**: Accept, 8/10 (good paper).

### Why not lower score?

- Scaling SAEs was a very important problem in interpretability, as indicated in the seminal SAE paper.
- Top-k SAEs are a significant improvement on an important research topic. This approach is well motivated and well placed in the literature.

### Why not higher score?

- The performance of top-k SAEs is state of the art, but not beyond comparison with alternatives.
- While top-k SAEs are a significant incremental improvement, top-k SAEs are not new, and this is a new application.
Strengths: The paper's strengths are as follows: 

## Scale:

LLMs are being used in an increasing number of fields, and research into LLM interpretability is important for ensuring their fairness and safety, especially at scale.
This paper scales up SAEs to frontier LLMs, such as GPT4, which is a ***significant improvement*** over previous work, which was limited to smaller models. Furthermore, scaling laws are found, which could be very useful to practitioners and researchers trying to predict SAE performance ex-ante.

## Top-K:

The paper's usage of top-k SAE is one of the main contributions of this work, and it is ***original*** to the extent that top-k sparse autoencoders hadn't (to the best of my knowledge) been used on LLMs before. The ***quality*** of the results of the top-k function is such that it represent the state of the art when compared to other activation functions.

## Auxiliary Loss:

The authors tackle the ***significant*** issue of dead latents (up to 90% of the total when not mitigated) by introducing an auxiliary loss to the overall loss function. This incentivises the model to use all the latents, as it also supervises the next-aux latents. The ***quality*** of the results is high, as it reduces dead latents by an order of magnitude to 7%. This is to the best of my knowledge an ***original*** contribution.

## Evaluations:

The paper introduces many new evaluations. I believe that explainable loss and known latent probing are the most ***significant***.
Explainable loss is an ***original*** metric that considers the loss from just using the explanations that the SAE generates (discarding uninterpretable latents), and it represents a ***significant*** improvement over either MSE or L0 alone since it includes both reconstruction quality and an interpretability guarantee (L0 does not). Known latent probing is interesting in that it shows that the ***quality*** of probes of the latents is 25% better than that of residual stream probes.

## Clarity:

The paper is very ***clear*** across all sections, and its presentation is of high ***quality***
Weaknesses: The paper's weaknesses are as follows:

## Irreducible error:
The paper notes that adding an irreducible error term to the scaling laws improves the laws. However, the explanation of why such an error term would exist in LLM activations is not clear, and an experiment showing that gaussian noise is hard to fit for SAEs is provided instead. The experiment does explain why such an error would be there in the first place, and its outcome is not surprising. Mentioning "components with different amounts of structure" is vague.

## Top-K vs penalty shrinkage experiments:
It is expected that having larger activations for top-k SAEs won't improve the loss and that it will for sparsity penalty SAEs. This is because top-k SAEs do not have any constraint on the magnitude of the k activations they use. An experiment on shrinkage does not make the top-k activation function more attractive.

## Reproducibility:

An issue with the experiments in this paper is that GPT4 is a closed source model, which makes it hard to reproduce the results of some of the experiments.

## Probing datasets:

A set of 61 binary classification tasks is used to train probes on the latents. It is not clear why these specific tasks have been chosen. A larger and more diverse set of tasks would have been more significant.

## Unclear phrasing of top-k's contributions:
In line 132 there is a missing word after: "instead of", which I assume to be L1, but it's unclear.
Recommendation: 8
Reviewer confidence: 4
Soundness/correctness: 3
Clarity: 4