Sparse Autoencoders Find Highly Interpretable Features in Language
  Models
Authors: name: Hoagy Cunningham; name: Aidan Ewart; name: Logan Riggs; name: Robert Huben; name: Lee Sharkey
Published: 2023-09-15 17:56:55

Abstract: One of the roadblocks to a better understanding of neural networks' internals
is polysemanticity, where neurons appear to activate in multiple,
semantically distinct contexts. Polysemanticity prevents us from identifying
concise, human-understandable explanations for what neural networks are doing
internally. One hypothesised cause of polysemanticity is
superposition, where neural networks represent more features than they
have neurons by assigning features to an overcomplete set of directions in
activation space, rather than to individual neurons. Here, we attempt to
identify those directions, using sparse autoencoders to reconstruct the
internal activations of a language model. These autoencoders learn sets of
sparsely activating features that are more interpretable and monosemantic than
directions identified by alternative approaches, where interpretability is
measured by automated methods. Ablating these features enables precise model
editing, for example, by removing capabilities such as pronoun prediction,
while disrupting model behaviour less than prior techniques. This work
indicates that it is possible to resolve superposition in language models using
a scalable, unsupervised method. Our method may serve as a foundation for
future mechanistic interpretability work, which we hope will enable greater
model transparency and steerability.