Analyze Feature Flow to Enhance Interpretation and Steering in Language
  Models
Authors: name: Daniil Laptev; name: Nikita Balagansky; name: Yaroslav Aksenov; name: Daniil Gavrilov
Published: 2025-02-05 09:39:34

Abstract: We introduce a new approach to systematically map features discovered by
sparse autoencoder across consecutive layers of large language models,
extending earlier work that examined inter-layer feature links. By using a
data-free cosine similarity technique, we trace how specific features persist,
transform, or first appear at each stage. This method yields granular flow
graphs of feature evolution, enabling fine-grained interpretability and
mechanistic insights into model computations. Crucially, we demonstrate how
these cross-layer feature maps facilitate direct steering of model behavior by
amplifying or suppressing chosen features, achieving targeted thematic control
in text generation. Together, our findings highlight the utility of a causal,
cross-layer interpretability framework that not only clarifies how features
develop through forward passes but also provides new means for transparent
manipulation of large language models.