Measuring Progress in Dictionary Learning for Language Model
  Interpretability with Board Game Models
Authors: name: Adam Karvonen; name: Benjamin Wright; name: Can Rager; name: Rico Angell; name: Jannik Brinkmann; name: Logan Smith; name: Claudio Mayrink Verdun; name: David Bau; name: Samuel Marks
Published: 2024-07-31 18:45:13

Abstract: What latent features are encoded in language model (LM) representations?
Recent work on training sparse autoencoders (SAEs) to disentangle interpretable
features in LM representations has shown significant promise. However,
evaluating the quality of these SAEs is difficult because we lack a
ground-truth collection of interpretable features that we expect good SAEs to
recover. We thus propose to measure progress in interpretable dictionary
learning by working in the setting of LMs trained on chess and Othello
transcripts. These settings carry natural collections of interpretable features
-- for example, "there is a knight on F3" -- which we leverage into
supervised metrics for SAE quality. To guide progress in
interpretable dictionary learning, we introduce a new SAE training technique,
p-annealing, which improves performance on prior unsupervised
metrics as well as our new metrics.