Steering Knowledge Selection Behaviours in LLMs via SAE-Based
  Representation Engineering
Authors: name: Yu Zhao; name: Alessio Devoto; name: Giwon Hong; name: Xiaotang Du; name: Aryo Pradipta Gema; name: Hongru Wang; name: Kam-Fai Wong; name: Pasquale Minervini
Published: 2024-10-21 13:30:47

Abstract: Large language models (LLMs) can store a significant amount of factual
knowledge in their parameters. However, their parametric knowledge may conflict
with the information provided in the context -- this phenomenon, known as
context-memory knowledge conflicts, can lead to undesirable model
behaviour, such as reliance on outdated or incorrect information. Analysing the
internal activations of LLMs, we find that they can internally register the
signals of knowledge conflict at mid-layers. Such signals allow us to detect
whether a knowledge conflict occurs and use inference-time intervention
strategies to resolve it. In this work, we propose SpARE, a
training-free representation engineering method that uses pre-trained
sparse auto-encoders (SAEs) to control the knowledge selection behaviour of
LLMs. SpARE identifies the functional features that control the
knowledge selection behaviours and applies them to edit the internal
activations of LLMs at inference time. Our experimental results show that
SpARE can effectively control the usage of either knowledge source to
resolve knowledge conflict in open-domain question-answering tasks, surpassing
existing representation engineering methods (+10%) as well as contrastive
decoding methods (+15%).