Steering Knowledge Selection Behaviours in LLM s via SAE -Based Representation Engineering (2025)

Abstract: Large language models (LLMs) can store a significant amount of factual knowledge in their parameters. However, their parametric knowledge may conflict with the information provided in the context—this phenomenon, known as context-memory knowledge conflicts , can lead to undesirable model behaviour, such as reliance on outdated or incorrect information. Analysing the internal activations of LLMs, we find that they can internally register the signals of knowledge conflict at mid-layers. Such signals allow us to detect whether a knowledge conflict occurs and use inference-time intervention strategies to resolve it. In this work, we propose SpARE, a training-free representation engineering method that uses pre-trained sparse auto-encoders (SAEs) to control the knowledge selection behaviour of LLMs. SpARE identifies the functional features that control the knowledge selection behaviours and applies them to edit the internal activations of LLMs at inference time. Our experimental results show that SpARE can effectively control the usage of either knowledge source to resolve knowledge conflict in open-domain question-answering tasks, surpassing existing representation engineering methods (+10%) as well as contrastive decoding methods (+15%).

## introduction large language models (llms) have shown remarkable capability to memorise factual knowledge and solve knowledge-intensive tasks @xcite @xcite @xcite . nevertheless, the knowledge stored in their parameters (parametric knowledge) can be inaccurate or outdated @xcite . to alleviate this issue, retrieval and tool-augmented approaches have been widely adopted to provide llms with external knowledge (contextual knowledge) (karpukhin et al., 2020; 1 we will refer to these as knowledge conflicts for brevity. context: geoffrey hinton is a computer scientist and cognitive scientist. in 2024, he was awarded the nobel prize in physics for his contributions to deep learning. question: what notable award is geoffrey hinton known for? @xcite @xcite . however, contextual knowledge may sometimes conflict with the parametric knowledge of the model, leading to what we refer to as knowledge conflicts. such conflicts can cause undesired behaviour, where the model may rely on inaccurate information sources, resulting in incorrect outputs @xcite @xcite @xcite . prior research found that llms tend to prefer contextual knowledge (e.g., retrieved passages) over their parametric knowledge when conflicts occur @xcite @xcite . for instance, @xcite show that most llms choose parametric knowledge in less than 10% examples. however, in more general applications, llms should retain the ability to use their parametric knowledge when presented with misinformation @xcite @xcite . existing works investigate fine-tuning and promptingbased strategies to detect and resolve knowledge conflicts @xcite ; however, they need additional interactions with the model, e.g., by asking the llms to examine the conflicts sentence by sentence, resulting in high latency times and preventing practical applications. in this work, we investigate representation engineering methods to efficiently steer the usage of parametric and contextual knowledge of llms at inference time. although representation engineering has provided an efficient and transparent framework for controlling the behaviour of llms, we find that existing methods fail to effectively steer knowledge usage. this may be because these methods directly modify the internal activations of llms, such as hidden states @xcite or mlp activations @xcite . these activations are polysemantic dense vectors that overlap with many independent semantic features @xcite . thus, minor edits in one dimension can influence multiple semantic features, making it difficult to adjust activations accurately without affecting other features in practice. recently, sparse auto-encoders (saes) have been proposed to address the difficulty of interpreting polysemantic activations by decomposing them into a large-scale monosemantic feature dictionary @xcite @xcite . therefore, we introduce saes as a tool for precise activation editing to guide the knowledge selection of llms. specifically, we propose spare, a sparse auto-encoder-based representation engineering method to steer the knowledge selection behavior of llms. spare first identifies the sae activations that are related to specific knowledge selection behaviours (section 4.2); then, it extracts functional features that control the usage of contextual and parametric knowledge, and finally applies them to steer the behaviour of the model (section 4.3). our experimental results on open-domain question-answering tasks show that spare effectively controls the knowledge selection behaviours by utilising a small set of sae features, e.g., less than 0.05% sae activations for gemma2-9b in the 6 layers foot_0 . spare yields more accurate results than state-of-the-art representation engineering methods (+10%), contrastive decoding (+15%), and incontext learning (+7%), achieving the best perfor-mance on steering knowledge selection behaviours of llms under knowledge conflicts. ## detection of knowledge conflicts in this section, we investigate whether we can detect the occurrence of conflicts during the generation process, since identifying such conflicts is a prerequisite for exploring inference-time strategies to control the llm. we focus on the residual stream @xcite of the model and look for a signal of knowledge conflict. to this end, we create two groups of input instances, in d e m , the model is provided with a context that is coherent with the model internal memorized knowledge, whereas in d e c the model is provided with a context that does not agree with model parametric knowledge, thus causing a knowledge conflict. to determine whether a signal of conflict arises in the residual stream, we focus on the last position of the sequence during generation, which is supposed to encode the information to predict the first token of the answer. we apply a linear probing method @xcite @xcite to investigate whether the residual stream contains a signal of knowledge conflict. specifically, we train logistic regression models to classify whether a given activation (the hidden state, mlp or self-attention activations) is from the d e c or d e m , i.e. whether it contains a knowledge conflict or not. we use activations from each layer as input and formulate this as a binary classification task. the evaluation is conducted on a held-out test set. we present probing results on llama2-7b @xcite and gemma2-9b @xcite using auroc as metric in fig. 2 . we observe that the probing accuracy increases from the first layer to the middle layers, and this trend is the same across different types of activations. this indicates that we can detect the signal of knowledge conflict in the residual stream of the mid-layers. the probing accuracy decreases in the later layers, especially for mlp and self-attention activations, which indicates that mlp and self-attention modules do not further add the signal of conflicting knowledge to the residual stream. we provide more details and analysis about knowledge conflict detection in appendix a and @xcite . the above analysis shows that knowledge conflicts can be identified in the internal states of llms. moreover, it provides insight into which layers can be more influential in the knowledge selection (section 6.1). ## resolving knowledge conflicts by representation engineering in this section, we introduce spare, our saebased representation engineering method, to steer the usage of parametric and contextual knowledge to generate the answers. spare consists of the three following steps: 1) collecting activations that lead to different knowledge selection behaviours (section 4.1); 2) identifying sae activations that are related to each knowledge selection behaviour (section 4.2); 3) steering the usage of either knowledge source by editing the hidden states of llms at inference time (section 4.3). ## related works representation engineering many studies focus on mechanistic interpretability to understand the llms by analysing the activities and connections of individual network components, such as circuits @xcite and neurons @xcite however, though mechanistic interpretability can successfully explain simple mechanisms, it often struggles with more complex phenomena @xcite . differently, representation engineering @xcite @xcite offers a complementary approach. it focuses on the characteristics of representations rather than lower-level mechanisms, providing a framework for understanding complex systems at a higher level of abstraction. it has shown more promise in interpreting higher-level behaviours of llms at scale. some works modify model activations to change behaviours @xcite @xcite @xcite , and some works extract latent vectors and leveraging these vectors to regulate the model's inference @xcite @xcite . knowledge conflicts knowledge conflicts refer to discrepancies among contextual and parametric knowledge @xcite . @xcite identify three types of knowledge conflicts: inter-context @xcite @xcite , context-memory @xcite @xcite , and intra-memory conflicts @xcite . in this work, we focus on context-memory knowledge conflicts, which refers to conflicts between the contextual knowledge and the parametric knowledge encoded in the model parameters. @xcite and @xcite investigate the mechanisms of attention heads and feed-forward networks of llms when context-memory knowledge conflict occurs. sparse auto-encoder sparse auto-encoders (saes) have been introduced as a post-hoc analysis tool to identify disentangled features within uncompressed representations of an llm @xcite @xcite . saes are trained with sparsity regularisation to learn a sparse, overcomplete basis that characterises the activation space of an llm @xcite . @xcite showed that the features learned by saes can identify sparse circuits in llms. @xcite showed the possibility of searching for monosemantic features and steering llms' generation. @xcite improves steering vectors using saes. @xcite generates description for representations learned through seas. @xcite finds universal representation across different language models. ## conclusions we investigated the context-memory knowledge conflicts in llms. we identify that knowledge conflicts can be detected by probing the residual stream of the model (section 3) and propose spare (section 4), a training-free representation engineering method that leverages pre-trained saes to effectively and efficiently control the knowledge selection behaviour of llms at inference time. our experimental results on odqa tasks show that spare produces more accurate results than existing representation engineering and contrastive decoding methods (section 5). by providing a mechanism to steer knowledge selection behaviours at inference time, spare offers a promising approach to managing knowledge conflicts in llms without significant computational overhead. additionally, we investigate the residual stream of llms under knowledge conflicts (section 6). we find that 1) the knowledge selection behaviour is more steerable at the middle layers of llms; 2) the residual stream shows a significantly more skewed representation when models using contextual knowledge compared to using parametric knowledge.