Dynamic Steering With Episodic Memory For Large Language Models
Authors: Van Dai Do, Quan Hung Tran, Svetha Venkatesh, Hung Le
Accepted at: acl25

Dynamic Steering With Episodic Memory For Large Language Models

Van Dai Do1, Quan Hung Tran2, Svetha Venkatesh1, Hung Le1
1Applied AI Initiative, Deakin University, Australia   2Meta
{v.do,tha1.le}@deakin.edu.au

Abstract

Large Language Models (LLMs) exhibit emergent in-context learning (ICL) capabilities, allowing them to adapt to unseen tasks based on example demonstrations. Traditional ICL embeds examples within the prompt, while activation steering, uses a vector derived from examples to guide the latent states of LLMs toward desired behaviors. However, traditional ICL is difficult to control quantitatively and consumes valuable context space. Existing activation steering methods apply a single sentence-level steering vector uniformly across all tokens, ignoring LLMs’ token-wise, auto-regressive nature. This coarse control can lead to inconsistencies and suboptimal adjustments during generation. To address this problem, we introduce Dynamic Steering with Episodic Memory (DSEM), a novel training-free framework that aligns LLMs to given demonstrations by steering at the token level conditioned on the input query. DSEM employs a key-value memory to store associations between generated tokens and steering vectors. During inference, it uses a nearest-neighbor mechanism to dynamically compute steering vectors for each token chunk, enabling more precise and adaptive guidance. Our method surpasses strong baselines across diverse alignment tasks - including safety, style transfer, and role-playing - demonstrating improved alignment as demonstration size scales.

enables an alternative approach to achieve strong downstream performance by modifying the input prompt in text space, incorporating instructions and in-context examples (Liu et al., 2021).

Despite the intriguing capabilities of ICL, it is highly sensitive to nearly every aspect of the prompt. The selection of demonstrations, the phrasing of instructions, or even subtle variations in example formatting can lead to significant fluctuations in final performance (Liu et al., 2021; Min et al., 2022). This sensitivity raises concerns about the robustness and reliability of ICL in real-world applications. Furthermore, a fundamental limitation of traditional ICL is the restricted context length of most open-source LLMs, which constrains the number of demonstrations that can be included in the prompt. As a result, effectively utilizing ICL requires careful prompt engineering and selection of examples to mitigate these challenges.

One recent approach to address the limitations of traditional ICL is activation steering, which involves modifying the activation layers of LLMs. For instance, Liu et al. (2024) have proposed computing the principal direction of the difference between source and target representations and using this as a single steering vector for all test data. More recently, Wang et al. (2025) introduced a binary masking method to identify the most influential elements in the steering process. A fundamental limitation of these methods is that they are either query-agnostic or lack fine-grained control, hampering final alignment performance. This raises a research question: How can activation steering be adaptively applied in a query-dependent manner and at the token level to effectively guide LLM generation toward a desired goal?

To this end, we propose Dynamic Steering with Episodic Memory (DSEM), a novel and efficient steering method designed to steer LLM generations flexibly, enabling quick adaptation to a given input and fine-grained control over the genera-

1 Introduction

Large Language Models (LLMs) have demonstrated remarkable performance in a wide range of natural language processing tasks, with their effectiveness improving as the number of parameters increases (Brown et al., 2020; Wei et al., 2022; Chowdhery et al., 2023). One of the most intriguing abilities of larger models is In-Context Learning (ICL), where LLMs can perform tasks they have not encountered during training, given only a few demonstrations (Brown et al., 2020). This
tion process. Drawing inspiration from the rapid and instance-based learning mechanisms observed in the hippocampus region of the human brain (Lengyel and Dayan, 2007), our method utilizes an external episodic memory that stores input representations, partially generated outputs, and their stored steering vectors. We use the input-output demonstrations to define the partial outputs and compute the stored steering vectors. During inference, DSEM utilizes the memory to generate steering vectors for each input query. It retrieves stored steering vectors and estimates the chunk-level steering vector using a nearest-neighbor mechanism, ensuring that steering vectors adapt across queries instead of remaining static.

Another significant benefit of DSEM is its ability to compute steering vectors at the token level. Instead of using a fixed steering vector for an entire sentence, DSEM dynamically determines steering vectors for generating the next few tokens. By leveraging token-level information saved in memory (i.e., the partially generated outputs), DSEM retrieves steering vectors that are better aligned with the desired outputs. This approach is motivated by the fact that next-token prediction in transformers (Vaswani et al., 2017) is inherently a token-wise process, suggesting that steering should also be applied at the token rather than the sentence level.

Finally, to preserve the global semantics of the demonstration, DSEM retrieves the most relevant steering vector at inference time and interpolates it with a global steering vector during each token generation step. This interpolation ensures a smooth transition between context-sensitive adjustments and overarching guidance, preventing abrupt shifts in model behavior. The interpolation weight is determined adaptively based on the uncertainty of the memory prediction, allowing DSEM to balance local specificity with global coherence.

In summary, our main contributions are: (1) We propose DSEM, a fine-grained activation steering method that dynamically interpolates between memory-based and global steering vectors, enabling precise control over generation. (2) We propose an uncertainty-driven interpolation mechanism that dynamically balances local specificity with global coherence. (3) We empirically demonstrate that DSEM outperforms strong baselines, including fine-tuning and state-of-the-art activation steering methods, across multiple tasks such as detoxification, formality adjustment, and role-playing, as well as across different model architectures and demonstration sizes.

2 Methods

2.1 Problem formulation

In-context Learning. In the standard framework of ICL, we have access to the base language model \( \mathcal{L} \), which is trained on large-scale data, and a set of demonstration data \( \mathcal{D}_{\text{demos}} = \{(x_i, y_i)\}_{i=1}^{|\mathcal{D}_{\text{demos}}|} \), where each pair \( (x_i, y_i) \) represents an input-output example for the task we want to accomplish. Given a test query \( x_{\text{test}} \), the input prompt for traditional ICL is constructed by concatenating the demonstration examples with the test query. Specifically,
\[
\text{Prompt} = concat((x_{\text{demos}}, y_{\text{demos}}), x_{\text{test}}),
\]
where \( (x_{\text{demos}}, y_{\text{demos}}) \) are the pairs of demonstration examples, and \( concat(., .) \) denotes the concatenation operation. As a result, the model’s output is given by \( y_{\text{test}} = \mathcal{L}(\text{Prompt}) \). The response \( y_{\text{test}} \) can be influenced by the demonstration examples provided and their order (Do et al., 2024; Pham et al., 2025). This allows for a degree of control over the output, as different choices or arrangements of demonstrations can lead to varying model behaviors.

Activation Steering. LLMs rely on the Transformer architecture (Vaswani et al., 2017), where self-attention mechanisms relate different token positions in a sequence. In ICL, demonstration examples are prepended to the query, influencing how attention is distributed across tokens. In latent space, Park et al. (2024) found evidence that concepts are represented as linear directions in the activation space of neural networks. Activation steering typically produces a steering vector, which is a modification to the latent space of an LLM that guides its output toward a desired outcome.

2.2 Steering With Episodic Memory

In this section, we present the architecture of our episodic memory \( \mathcal{M} \), which extracts the steering vector at the token level during LLM generation. Assume we generate \( l \) tokens at a time; the goal is to find a steering vector for those \( l \) tokens and steer them toward the desired behavior. For example, in a Shakespearean role-playing task, a source sentence could be 'I need your help now.' If fully steered toward Shakespearean formality, it would be "I beseech thee, assist me this very moment!", which is unnecessarily elaborate. Steered for clarity, it would remain 'I need help now.' By balancing both clarity and Shakespearean formality, the final output would be: 'I beseech
Figure 1: Memory Read process of DSEM and an example element from \( \mathcal{M} \) for the Shakespeare dataset (bottom left). Given the input \( \hat{x} \) and the current partial generation \( \hat{y}^m \), at inference time, Memory is queried with \( \hat{x} \) and \( \hat{y}^m \) to extract \( v_{memory} \) and Uncertainty. \( v_{memory} \) undergoes an interpolation phase with \( v_{global} \), derived from \( \mathcal{D}_{demos} \), with interpolation weights determined by the Uncertainty of Memory’s prediction. The resulting vector, \( v_{steer} \), is then used to guide the generation of the next \( l \) tokens. After that, we do an update on the partially generated output \( \hat{y}^m \leftarrow \hat{y}^{m+l} \). This loop is continued until the end of generation.

thee, assist me now.' This shows how having different steering vectors during generation can achieve the optimal balance of style and clarity. In this paper, we utilize an episodic memory to dynamically compute steering vectors in a query-dependent and token-level manner, avoiding the need for expensive fine-tuning (Hu et al., 2021). To write into memory, we store tuples of the demonstration source \( x \), its representation \( s(x) \), the corresponding steering vector \( v_{memory} \), and progressively larger token chunks \( y^l, y^{2l}, \ldots, y^L \), where \( L \) is the target sentence length. During inference (Memory Read phase), the memory \( \mathcal{M} \) is queried with the input \( \hat{x} \) and partial generation \( \hat{y}^m \) to extract \( v_{memory} \) and its associated Uncertainty. The extracted vector is interpolated with \( v_{global} \) from \( \mathcal{D}_{demos} \) using Uncertainty-based weights, producing \( v_{steer} \). This vector guides the generation of the next \( l \) tokens, after which the partial output is updated as \( \hat{y}^m \leftarrow \hat{y}^{m+l} \). The process repeats until the generation is completed. An illustration of the process is shown in Figure 1.

State Representation. Accurate and meaningful text representations are essential for both memory storage and efficient retrieval. In DSEM, a memory element contains the embedded representation of the demonstration example \( x_{demo} \) along with chunk-level information. For sentence-level information, we compute the mean pooling of the last token representation of the demonstration sentence \( x_{demo} \) when passed through the LLM \( \mathcal{L} \), which we denote as \( s(x_{demo}) \). To preserve token-level information, we store the target sentence’s tokens progressively. This is achieved by segmenting \( y_{demo} \) into \( \lceil \frac{L}{l} \rceil \) parts, transforming a sentence of length \( L \) into \( \lceil \frac{L}{l} \rceil \) progressive states. Each state grants DSEM access to granular token-level insights, enhancing retrieval precision and efficiency.

Steering Vector Extraction. Given a pair of demonstrations \( (x_{demo}, y_{demo}) \), we calculate the steering vector as follows:

\[
v_{demo} = h(y_{demo}) - h(x_{demo})
\]

Here, \( h(y_{demo}) \) and \( h(x_{demo}) \) are the concatenated embeddings across all layers of the LLM \( \mathcal{L} \). \( v_{demo} \in \mathbb{R}^{n_{layers} n_{hidden}} \), where \( n_{layers} \) and \( n_{hidden} \) denote the number of layers and the latent representation dimension of \( \mathcal{L} \), respectively. Intuitively, \( v_{demo} \) steers the model’s output from \( x_{demo} \) toward \( y_{demo} \). Adding \( v_{demo} \) to the latent representation of \( \mathcal{L}(x_{demo}) \) at each token step guides the model toward generating \( y_{demo} \).

2.2.1 Memory Construction
We populate the memory \( \mathcal{M} \) using a set of demonstration examples \( \mathcal{D}_{demos} \). For each example, we pair its text input \( x_{demo} \) and embedded representation \( s(x_{demo}) \) with the corresponding target out-
put \( y_{demo} \). The output \( y_{demo} \) is divided into progressively larger chunks, where the size of each chunk is determined by \( l \). Specifically, the first chunk contains \( l \) tokens, the second chunk contains \( 2l \) tokens, and so on. A smaller \( l \) captures finer-grained contextual information but increases memory storage, while a larger \( l \) results in coarser chunks, reducing the memory storage. The target sequence is partitioned into multiple chunks for a given \( l \). Each chunked representation is paired with its corresponding steering vector \( v_{demo} \), which is derived from the demonstration. These tuples are then stored in memory \( \mathcal{M} \) in the form: \( (s(x_{demo}), x_{demo}, y^j_{demo}, v_{demo}) \), where \( y^j_{demo} \) represents the \( j \)-th chunk of the target sequence. These elements are appended sequentially to \( \mathcal{M} \), ensuring that all relevant contextual information is preserved.

2.2.2 Memory Read
In this section, we describe the memory reading process used to obtain token-level steering vectors.

Sentence-level Retrieval With Reciprocal Rank Fusion. In practice, we often encounter novel sentences, i.e., sentences not present in memory. To address this, we leverage our memory \( \mathcal{M} \) as a nearest-neighbor estimator to obtain steering vectors at the token level. We first retrieve \( \mathcal{N}_{sent} \), the set containing the top-\( K_{sent} \) most relevant sentences. Inspired by Reciprocal Rank Fusion (RRF) (Cor-mack et al., 2009), we rank these neighbors by combining dense retrieval and BM25 (Robertson and Zaragoza, 2009) to enhance retrieval precision.

For retrieval, we combine dense and lexical similarity to capture both semantic and lexical information. Dense retrieval ranks candidates (\( R_{dense} \)) based on their cosine similarity with the test query, capturing semantic alignment. Lexical similarity is introduced by computing BM25 scores between the test query \( x_{test} \) and all queries in \( \mathcal{M} \), resulting in the ranking \( R_{BM25} \). The details of \( R_{dense} \) and \( R_{BM25} \) are provided in Appendix A.2.

Once we obtain two rankings, \( R_{dense} \) and \( R_{BM25} \), the final rank of each sentence \( x_i \) in \( \mathcal{M} \), denoted as \( R_{RRF}(x_{test}) \), is computed using the RRF score:

\[
score_{RRF}(x_i) = \frac{1}{k_{RRF} + R_{dense}(x_i)} + \frac{1}{k_{RRF} + R_{BM25}(x_i)}
\]

where \( k_{RRF} \) is a constant integer. Intuitively, a higher \( score_{RRF} \) indicates greater similarity between the memory element \( x_i \) and the test query \( x_{test} \) at the sentence level. Finally, we use \( R_{RRF} \) to get \( \mathcal{N}_{sent} \) corresponding to \( x_{test} \).

Token-level Vector Extraction. Beyond retrieving the nearest input neighbors, we also search for the stored target outputs closest to the current generation to refine the steering vector at the token level. This step is performed during generation between the current partial generation and the target chunks we stored in \( \mathcal{M} \), which differs from the RRF retrieval used for retrieving the nearest sentences.

Let \( \mathcal{N}_{tok} \) represent the set of chunks derived from the top-\( K_{sent} \) retrieved sentences for the test query. Our objective is to retrieve the top-\( K_{tok} \) nearest chunks from \( \mathcal{N}_{tok} \) to the current generation. To achieve this, we calculate the BM25 score between the current generation \( y^m \) and each chunk \( y_{tok}, y_{tok} \in \mathcal{N}_{tok} \). The top-\( K_{tok} \) memory elements are then selected based on these scores. This approach provides access to fine-grained token-level information about the current generation with respect to the memory elements, thereby improving the accuracy of the steering vector calculation. For simplicity, we set \( K_{tok} = K_{sent} \).

Steering Vector Interpolation With Memory Uncertainty. In this section, we illustrate how the token-level steering vector is calculated. First, we show how to extract the memory steering vector, denoted as \( v_{memory} \). When the generation process has not started, it is not possible to use Token-level Retrieval, thus, for that case, we rely on the top-\( K_{sent} \) sentences’ steering vector. Formally, we have:

\[
v_{memory} = \begin{cases}
\sum_{j=1}^{|\mathcal{N}_{tok}|} v_j / |\mathcal{N}_{tok}|, & \text{if } t = 0, \\
\sum_{j=1}^{|\mathcal{N}_{sent}|} v_j / |\mathcal{N}_{sent}|, & \text{otherwise}.
\end{cases}
\]

Here, \( t \) is the number of generated tokens in the response, \( v_j \) is the steering vector corresponding to the \( j^{th} \) memory element in \( \mathcal{N}_{tok} \).

To balance the tradeoff between local relevance and global coherence during token-level steering, we propose a dynamic interpolation mechanism between memory-based and global steering vectors. The general form of the interpolation is defined as:

\[
v_{steer} = (1 - \delta(x_{test})) \cdot v_{memory} + \delta(x_{test}) \cdot v_{global},
\]

where \( v_{global} \) represents the global steering vector, and \( \delta(x_{test}) \) is the uncertainty-based weight that adjusts the contribution of each component.

The memory-based steering vector, \( v_{memory} \),
provides localized guidance by capturing token-relevant information from similar past contexts, ensuring improved contextual relevance. However, solely relying on memory can cause the model to overfit to specific memory instances, resulting in outputs that are less consistent with the broader data distribution. To address this, we include the global steering vector, \( v_{\text{global}} \), which captures corpus-wide information and aligns with the reference data’s statistical properties.

The uncertainty-based interpolation weight \( \delta(x_{\text{test}}) \) is motivated by two key goals. First, it reduces hyperparameter tuning costs by dynamically adjusting the interpolation weight, eliminating the need for manual tuning of fixed weights and making the approach more adaptive and efficient. Second, it enables dynamic adjustment for each input by varying the interpolation weight based on the uncertainty of the memory retrieval. For inputs with low uncertainty, \( v_{\text{memory}} \) is prioritized, otherwise, the mechanism shifts focus toward \( v_{\text{global}} \).

To compute the global steering vector \( v_{\text{global}} \), we adopt the state-of-the-art method from ICV (Liu et al., 2024), defined as:

\[
v_{\text{global}} = \arg\max_h \frac{1}{|\mathcal{D}|} \sum_i \left( h^T (h y_i - h x_i) \right)^2,
\]
s.t. \( h^T h = 1. \)

Here, \( y_i \) and \( x_i \) represent the target and source sentences in the demonstration set \( \mathcal{D}_{\text{demos}} \), respectively. This ensures that \( v_{\text{global}} \) captures the overall statistical patterns of the corpus, complementing the localized guidance provided by \( v_{\text{memory}} \).

To determine the interpolation weight dynamically at the token level, we introduce an uncertainty-based mechanism. Let \( d \) be the cosine distances between \( s(x_{\text{test}}) \) and all stored elements in \( \mathcal{M} \). If \( s(x_{\text{test}}) \in \mathcal{M} \), the memory provides a direct steering vector with zero uncertainty \( \delta(x_{\text{test}}) = 0 \). Otherwise, we estimate uncertainty based on the dispersion of distances among the top-\( K_{sent} \) retrieved sentences in \( \mathcal{M} \): \( \delta(x_{\text{test}}) = \frac{\max(d) - \min(d)}{\max(d) - \min(d)} \). Higher uncertainty \( \delta \) signals greater variability in memory distances, indicating lower confidence in the retrieved steering vector.

After computing \( v_{\text{steer}} \) in Equation 4 using the elements outlined above, we apply it to guide the generation of the next \( l \) tokens as follows:

\[
\hat{v}_l^i = v_l^i + \lambda \cdot v_{\text{steer}}^i
\]

where \( i \) denotes layer \( i^{th} \) of \( \mathcal{L} \) and \( \lambda \) denotes steering intervention strength. By combining \( v_{\text{memory}} \) and \( v_{\text{global}} \) in this framework, we achieve a balance between local adaptation and global coherence at the token level. The uncertainty-based weighting mechanism ensures robust, contextually adaptive steering while minimizing manual effort. Memory Read algorithm is shown in Appendix A.1.

3 Experiments

Our experiments focus on open-source LLMs: Llama-2-7b-chat, Gemma-2-9b-it, Falcon-7B, and Mistral-7b-Instruct-v0.2 (Touvron et al., 2023; Team, 2024; Almazrouei et al., 2023; Jiang et al., 2023). The same top\_p and temperature values are used across all methods within each model.

3.1 Datasets

ParaDetox. We evaluate DSEM on the detoxification task using ParaDetox (Logacheva et al., 2022), a dataset consisting of toxic-neutral sentence pairs. For evaluation, we randomly select 700 examples. The goal is to rewrite toxic sentences into non-toxic ones while preserving their original meaning. GYAFC. We evaluate DSEM on the formality transfer task using GYAFC (Rao and Tetreault, 2018), where each sample contains an informal and a formal sentence. The objective is to rewrite informal sentences into their formal counterparts. The dataset is further divided into two subsets: GYAFC-family and GYAFC-music, which contain sentences related to family relationships and music, respectively. Shakespeare. We also consider the Shakespeare Modern English dataset (Jhamtani et al., 2017), where each sample consists of a Modern English sentence. The task is to rewrite it in the style of Shakespearean English.

3.2 Baselines

We compare our approach with state-of-the-art steering and finetuning methods across different LLMs and demonstration sizes. No Context only uses the test input without additional context. ICL adds a set of randomly selected examples from the demonstration set together with the test query to build the prompt. LoRA fine-tunes the model with Low-Rank Adaptation (Hu et al., 2021) using the demonstration set as training data. SaDI (Wang et al., 2025) averages the differences of all data pairs in the demonstration set. It then finds the most influential elements to create a binary mask to steer the model. ICV (Liu et al., 2024) computes a
<table>
  <tr>
    <th rowspan="2">Method</th>
    <th colspan="4">Llama-2-7b</th>
    <th colspan="4">Gemma-9b</th>
  </tr>
  <tr>
    <th>Tox.</th>
    <th>BERT↑</th>
    <th>For.↑</th>
    <th>BERT↑</th>
    <th>Tox.</th>
    <th>BERT↑</th>
    <th>For.↑</th>
    <th>BERT↑</th>
  </tr>
  <tr>
    <td>No Context</td>
    <td>39.00</td>
    <td>90.75</td>
    <td>39.77</td>
    <td>92.19</td>
    <td>40.56</td>
    <td>92.24</td>
    <td>44.28</td>
    <td>83.82</td>
    <td>28.71</td>
    <td>82.35</td>
    <td>27.51</td>
    <td>80.11</td>
  </tr>
  <tr>
    <td>ICL-4</td>
    <td>41.43</td>
    <td>85.09</td>
    <td>26.45</td>
    <td>55.13</td>
    <td>45.33</td>
    <td>22.14</td>
    <td>90.24</td>
    <td>60.60</td>
    <td>92.27</td>
    <td>85.36</td>
    <td>89.56</td>
  </tr>
  <tr>
    <th colspan="13">E = 10</th>
  </tr>
  <tr>
    <td>ICL-10</td>
    <td>25.85</td>
    <td>41.03</td>
    <td>51.59</td>
    <td>66.69</td>
    <td>53.62</td>
    <td>70.77</td>
    <td>14.00</td>
    <td>88.98</td>
    <td>83.11</td>
    <td>90.57</td>
    <td>83.42</td>
    <td>90.22</td>
  </tr>
  <tr>
    <td>ICV</td>
    <td>24.86</td>
    <td>91.12</td>
    <td>50.28</td>
    <td>91.34</td>
    <td>66.84</td>
    <td>91.67</td>
    <td>6.71</td>
    <td>85.50</td>
    <td>51.78</td>
    <td>86.61</td>
    <td>76.19</td>
    <td>85.46</td>
  </tr>
  <tr>
    <td>SaDI</td>
    <td>28.14</td>
    <td>45.50</td>
    <td>53.66</td>
    <td>92.11</td>
    <td>57.14</td>
    <td>92.23</td>
    <td>78.29</td>
    <td>90.93</td>
    <td>8.07</td>
    <td>92.06</td>
    <td>5.82</td>
    <td>91.38</td>
  </tr>
  <tr>
    <td>LoRA</td>
    <td>39.71</td>
    <td>90.70</td>
    <td>50.84</td>
    <td>92.04</td>
    <td>19.71</td>
    <td>90.70</td>
    <td>82.71</td>
    <td>90.59</td>
    <td>7.31</td>
    <td>92.18</td>
    <td>5.64</td>
    <td>91.43</td>
  </tr>
  <tr>
    <td>DSEM</td>
    <td><b>21.42</b></td>
    <td><u>91.68</u></td>
    <td><u>66.04</u></td>
    <td><u>92.26</u></td>
    <td><u>76.37</u></td>
    <td><u>92.26</u></td>
    <td><u>5.57</u></td>
    <td><u>88.29</u></td>
    <td><u>96.44</u></td>
    <td><u>90.35</u></td>
    <td><u>96.47</u></td>
    <td><u>90.59</u></td>
  </tr>
  <tr>
    <th colspan="13">E = 100</th>
  </tr>
  <tr>
    <td>ICV</td>
    <td>18.42</td>
    <td>90.77</td>
    <td>69.36</td>
    <td>91.50</td>
    <td>55.73</td>
    <td>91.60</td>
    <td>5.67</td>
    <td>84.62</td>
    <td>85.37</td>
    <td>86.24</td>
    <td>78.13</td>
    <td>83.97</td>
  </tr>
  <tr>
    <td>SaDI</td>
    <td>17.86</td>
    <td>90.44</td>
    <td>52.91</td>
    <td>92.10</td>
    <td>59.44</td>
    <td>92.15</td>
    <td>80.43</td>
    <td>91.76</td>
    <td>7.69</td>
    <td>92.60</td>
    <td>6.80</td>
    <td>90.13</td>
  </tr>
  <tr>
    <td>LoRA</td>
    <td>47.14</td>
    <td><u>93.23</u></td>
    <td>34.14</td>
    <td><u>93.77</u></td>
    <td>32.98</td>
    <td><u>93.70</u></td>
    <td>61.29</td>
    <td>89.54</td>
    <td>5.25</td>
    <td>90.27</td>
    <td>5.82</td>
    <td>94.42</td>
  </tr>
  <tr>
    <td>DSEM</td>
    <td><b>15.70</b></td>
    <td><u>90.90</u></td>
    <td><u>85.00</u></td>
    <td><u>92.11</u></td>
    <td><u>83.60</u></td>
    <td><u>92.22</u></td>
    <td><u>4.29</u></td>
    <td><u>88.15</u></td>
    <td><u>97.75</u></td>
    <td><u>90.29</u></td>
    <td><u>94.53</u></td>
    <td><u>90.41</u></td>
  </tr>
</table>

Table 1: DSEM vs. Baselines on Detoxification and Formality Transfer. Results (percentages) across Llama-2-7b and Gemma-9b models. "Tox." is for Toxicity, "For." is for Formality. ICL-4 and ICL-10 use 4 and 10 random demonstrations. Lower Toxicity is better, while Higher Formality and BERTScore indicate better performance. Best results are bolded, second-best underlined, excluding No Context and ICV-4.

steering vector based on the principal direction of differences between positive and negative examples from the demonstration set.

3.3 Standard Benchmarking Evaluation

Following prior work (Liu et al., 2024), to measure alignment effectiveness, we use a safety classifier to detect toxicity in ParaDetox generations, trained on datasets like Wikipedia Toxic Comments (Wulczyn et al., 2017), Build It - Break It - Fix It (Votipka et al., 2020), and Bot-Adversarial Dialogue (Xu et al., 2021). A sentence is considered safe if its predicted score exceeds 0.9. The dataset is split into GYAFC-family and GYAFC-music, aligning with subdomains. For formality transfer, we use a RoBERTa-based classifier (Dementieva et al., 2023) with a 0.9 threshold. Additionally, ground-truth relevance is measured using ROUGE-1 (Lin, 2004) and BERTScore (Zhang et al., 2020) for lexical and semantic similarity, respectively. Due to space restriction, the results of detoxification, formality transfer and BERTScore for Llama-2-7b and Gemma-9b are shown in Table 1, while the full results, including ROUGE-1 metric and other LLMs such as Falcon-7b and Mistral-7b are shown in Table 5 in Appendix A.4.

Paradetox Results. Our method, DSEM, achieves the lowest toxicity across nearly all models and demonstration sizes, demonstrating its effectiveness in controlled text generation. As shown in Table 1, DSEM consistently outperforms other methods in reducing toxicity while maintaining competitive fluency and relevance. Notably, DSEM achieve the highest scores across all datasets with \( E = 10 \), which highlights its efficiency over other methods. On Llama-2-7b at \( E = 100 \), DSEM lowers toxicity to 15.70%, outperforming ICV (18.42%) and significantly surpassing LoRA (47.14%). Although LoRA achieves the highest BERTscores in some settings (e.g., 93.23 for Llama-2-7b at \( E = 100 \)), its high toxicity makes it unsuitable for detoxification tasks. DSEM balances toxicity reduction with coherence and relevance, offering scalable and fine-grained control. For Gemma-9b, DSEM produces meaningful low-toxicity content, whereas ICV, despite similarly low toxicity, generates repetitive and nonsensical tokens (see Appendix A.12).

Formality Transfer Results. DSEM consistently achieves the highest Formality scores across all models and datasets, significantly outperforming baselines. This advantage is most pronounced at \( E = 100 \), increasing the chance of retrieving better neighbors. With Gemma-9b, performances of DSEM nearly achieve the perfect Formality rates for both datasets. In addition, the performance gaps of DSEM compared to second best baselines are large (e.g., +12.38% on GYAFC_family and +16.40 % on GYAFC_Music while having higher BERTscores with \( E = 100 \)). At \( E = 10 \), DSEM remains superior. With Llama-2, it achieves 66.04% (GYAFC_family) and 76.37% (GYAFC_music), outperforming second-best baselines, which are SaDI and ICV, by 12.38% and 9.53%, respectively. Notably, these gains come with higher BERTScore, indicating effective memory-based steering.

3.4 LLM Evaluation

To capture subtle aspects of language quality that conventional metrics may overlook, we evaluate using CohereAI’s Command-R Plus model, a powerful enterprise LLM with 104 billion parameters
Figure 2: Win rates of DSEM versus ICV across different backbones on the Shakespeare role-playing task using Command-r-plus as evaluator.

Figure 3: Win rates of DSEM versus ICV across different backbones on other datasets with 300 samples each using Command-r-plus as the evaluator.

(Cohere For AI, 2024). This model acts as a proxy for human evaluation, determining which responses are superior. We randomize the order of the responses in the prompt to avoid possible positional bias (Zheng et al., 2023). Evaluation prompts are reported in Appendix A.16.

Shakespeare Role-Playing Results. We evaluate DSEM on 585 test samples from the Shakespeare role-playing dataset (Jhamtani et al., 2017), using ICV (Liu et al., 2024) as the baseline due to its strong performance in alignment as shown in the previous experiment. We set \( E = 100 \) and \( K_{tok} = K_{sent} = 9 \). As shown in Figure 2, DSEM consistently outperforms ICV across all tested backbones. This demonstrates the advantage of our fine-grained steering method in improving response quality while maintaining alignment with the intended role.

Results on Other Datasets. We randomly select 300 samples from each dataset for LLM evaluation to further validate the effectiveness of DSEM. We compare the generations head-to-head with ICV (Liu et al., 2024), and the results are presented in Figure 3. DSEM outperforms ICV across all backbone models on the GYAFC_Music dataset, with Gemma-9b achieving the highest win rate of 91.4%. For Paradetox, DSEM consistently performs better across all models. On the GYAFC_Family dataset, except for Falcon-7b, DSEM maintains superiority over ICV, with Mistral-7b achieving the second-best win rate of 58.1%.

Figure 4: Ablation on using different \( l \) with Llama-2-7b and \( E = 100 \) for GYAFC Family and GYAFC Music.

4 Ablation Study

4.1 Effect of Different Token Chunk Length \( l \)

We study the impact of chunk length \( l \) on performance to understand the benefit of token-level steering. Results for Llama-2-7b across GYAFC Family and GYAFC Music datasets are shown in Figure 4. Overall, a lower \( l \) tends to boost relevance, as evidenced by higher BERTscores. However, the optimal chunk length varies by dataset. For GYAFC Music, shorter chunks yield substantial gains because they capture fine-grained contextual nuances essential for music-related texts. In contrast, for GYAFC Family, longer chunks better preserve broader context and narrative flow, leading to improved downstream performance. These findings indicate that the optimal \( l \) is dataset-dependent, reflecting the varying linguistic and structural characteristics across tasks. Therefore, it is important to steer at a flexible token level and adjust the chunk length to suit different settings.

4.2 Effect of Using Memory Steering

We validate the results when using only \( v_{memory} \) as the steering vector, without interpolating with \( v_{global} \). We use Llama-2-7b on the GYAFC Family
Figure 5: Ablation of only using \( v_{memory} \) using Llama-2-7b on GYAFC Family dataset with \( E = 10 \).

<table>
  <tr>
    <th>Dataset</th>
    <th>Scores</th>
    <th>R-1</th>
    <th>BERT</th>
  </tr>
  <tr>
    <td>Paradetox ↓</td>
    <td>16.28 (+1.1)</td>
    <td>50.66</td>
    <td>90.81</td>
  </tr>
  <tr>
    <td>GYAFC_Family ↑</td>
    <td>84.99 (-0.01)</td>
    <td>45.60</td>
    <td>91.47</td>
  </tr>
  <tr>
    <td>GYAFC_Music ↑</td>
    <td>83.25 (-0.35)</td>
    <td>51.25</td>
    <td>92.22</td>
  </tr>
</table>

Table 2: Results of DSEM using the average of all vectors stored in \( \mathcal{M} \) as \( v_{global} \). We use Llama-2-7b with \( E = 100 \). For Paradetox, Lower score is better (↓), while Higher score is better for the rest (↑).

dataset. For the "Global Only" baseline, \( v_{global} \) is derived from ICV (Liu et al., 2024). As shown in Figure 5, using only \( v_{global} \) achieves the lowest results among all approaches. Using only \( v_{memory} \) yields better results on both Formality and BERTScore. Yet, the best performance is achieved by interpolating between \( v_{memory} \) and \( v_{global} \), ensuring smooth transitions between context-sensitive adjustments and overarching guidance.

4.3 Effect of Different \( v_{global} \)

In addition to using \( v_{global} \) as ICV, we try \( v_{global} \) as the average of stored steering vectors in \( \mathcal{M} \). Results for Llama-2-7b with \( E = 100 \) are shown in Table 2. Toxicity increases slightly (1.1%), while Formality decreases marginally. While using \( v_{global} \) as ICV performs slightly better, this experiment shows that DSEM is robust to simpler methods of deriving \( v_{global} \).

4.4 Chunk Length Selection

In addition to the results we have already presented in the paper using LLaMA-2-7B, we report additional experiments on the GYAFC_Family and GYAFC_Music datasets using Gemma-9B. The experiments were run with \( l \in [2, 4, 8, 16, 32] \).

Observing the results, we find that increasing \( l \) generally decreases BERTScore while enhancing the formality of the output, which aligns with our earlier findings using LLaMA-2. However, on the GYAFC Family dataset, formality continues to increase with larger \( l \), diverging from the trend presented in Figure 4. Overall, a medium value of \( l \) (between 4 and 16) appears to provide a good balance between downstream performance and relevance, whereas values that are too small (e.g., \( l = 2 \)) or too large (e.g., \( l = 32 \)) tend to yield suboptimal results.

<table>
  <tr>
    <th>Dataset</th>
    <th>\( l \)</th>
    <th>Formality</th>
    <th>BERT</th>
  </tr>
  <tr>
    <td rowspan="5">GYAFC_family</td>
    <td>2</td>
    <td>96.42</td>
    <td>90.25</td>
  </tr>
  <tr>
    <td>4</td>
    <td>96.44</td>
    <td>90.35</td>
  </tr>
  <tr>
    <td>8</td>
    <td>97.18</td>
    <td>90.17</td>
  </tr>
  <tr>
    <td>16</td>
    <td>97.74</td>
    <td>90.21</td>
  </tr>
  <tr>
    <td>32</td>
    <td>97.18</td>
    <td>90.12</td>
  </tr>
  <tr>
    <td rowspan="5">GYAFC_music</td>
    <td>2</td>
    <td>95.22</td>
    <td>90.59</td>
  </tr>
  <tr>
    <td>4</td>
    <td>96.47</td>
    <td>90.59</td>
  </tr>
  <tr>
    <td>8</td>
    <td>96.29</td>
    <td>90.62</td>
  </tr>
  <tr>
    <td>16</td>
    <td>96.29</td>
    <td>90.49</td>
  </tr>
  <tr>
    <td>32</td>
    <td>97.17</td>
    <td>90.41</td>
  </tr>
</table>

Table 3: Results of DSEM using Gemma-9b with different chunk length \( l \).

4.5 Other Ablation Studies

Due to the page limit, we provide more ablation studies in the Appendix. We provide additional results on larger LLMs in Appendix A.5’s Table 6, results on TruthfulQA dataset in Appendix A.6’s Table 9, ablation with various number of neighbors in Appendix A.7’s Figure 8, ablation on using DSEM with more demonstration examples in Appendix A.8’s Table 7, ablation on the intervention factor in Appendix A.9’s Table 9, inference speed and memory in Appendix A.10’s Table 10 and Table 12. These results demonstrate DSEM’s consistent outperformance across various settings.

5 Related Work

Traditional ICL Improvements. Several methods have been proposed to improve ICL, including instruction tuning, demonstration selection, and ordering (Deng et al., 2022; Zhang et al., 2022; Do et al., 2024; Pham et al., 2025). Instruction tuning fine-tunes models on diverse tasks to enhance generalization, while demonstration selection identifies the most relevant examples to optimize performance. Ordering strategies explore how the arrangement of examples impacts output. While these approaches enhance ICL by editing instructions or selecting and arranging examples, they remain constrained by the fixed context window, limiting scalability.
Activation Steering refers to techniques that modify the latent space of models to produce desired outputs. Burns et al. (2022); Park et al. (2024) demonstrate that latent knowledge encoded in the activation space is linearly separable. Liu et al. (2024) proposes finding the principal direction of the steering vectors in a demonstration set and applying it to the entire test set. Wang et al. (2025) introduces a binary mask for the latent space to enhance the values of the most important elements. However, while Liu et al. (2024) identifies a query-agnostic steering vector, Wang et al. (2025) relies heavily on brute-force search to determine the optimal number of important elements and the strength of the intervention, a limitation acknowledged in their work. In contrast, DSEM introduces a dynamic, query-dependent approach to identify steering vectors at the token level, allowing the steering direction to adapt smoothly during generation.

6 Conclusion

We introduce Dynamic Steering with Episodic Memory (DSEM), a training-free framework that aligns LLMs with given demonstrations by steering at the token level, conditioned on the input query. DSEM stores representation-vector associations and uses a nearest-neighbor mechanism to extract the steering vector during generation. It leverages prediction uncertainty to determine the interpolation weight with a global steering vector. Evaluations on safety, style transfer, and role-playing tasks demonstrate its effectiveness, with detailed model analyses highlighting its advantages.

Limitations

We explore the ability to steer LLM generations using Dynamic Steering with Episodic Memory (DSEM). Our work introduces query-dependent, token-level steering during generation, a promising direction that warrants further research attention. However, several limitations must be acknowledged. First, while DSEM demonstrates promising results on medium-scale datasets, its scalability to larger datasets remains to be evaluated. Second, verifying memory-based generation poses challenges in black-box models where internal representations are inaccessible. Future research should aim to develop techniques for better understanding these behaviors and extend our findings to broader contexts.

Acknowledgments

This work was supported by compute credits from a Cohere Labs Research Grant, these grants are designed to support academic partners conducting research with the goal of releasing scientific artifacts and data for good projects.

References

Ebtesam Almazrouei, Hamza Alobeidli, Abdulaziz Alshamsi, Alessandro Cappelli, Ruxandra Cojocaru, Mérourane Debba, Étienne Goffinet, Daniel Hesslow, Julien Launay, Quentin Malartic, Daniele Mazzotta, Badreddine Noune, Baptiste Pannier, and Guilherme Penedo. 2023. The falcon series of open language models. Preprint, arXiv:2311.16867.

Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language models are few-shot learners. Advances in neural information processing systems, 33:1877–1901.

Collin Burns, Haotian Ye, Dan Klein, and Jacob Steinhardt. 2022. Discovering latent knowledge in language models without supervision. ArXiv.

Aakanksha Chowdhery, Sharar Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, et al. 2023. Palm: Scaling language modeling with pathways. Journal of Machine Learning Research, 24(240):1–113.

Cohere For AI. 2024. c4ai-command-r-plus-08-2024.

Gordon V. Cormack, Charles L A Clarke, and Stefan Buettcher. 2009. Reciprocal rank fusion outperforms condorcet and individual rank learning methods. In Proceedings of the 32nd International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR ’09, page 758–759, New York, NY, USA. Association for Computing Machinery.

Daryna Dementieva, Nikolay Babakov, and Alexander Panchenko. 2023. Detecting text formality: A study of text classification approaches. In Proceedings of the 14th International Conference on Recent Advances in Natural Language Processing, pages 274–284, Varna, Bulgaria. INCOMA Ltd., Shoumen, Bulgaria.

Mingkai Deng, Jianyu Wang, Cheng-Ping Hsieh, Yihan Wang, Han Guo, Tianmin Shu, Meng Song, Eric Xing, and Zhiting Hu. 2022. RLPrompt: Optimizing discrete text prompts with reinforcement learning. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 3369–3391, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
Dai Do, Quan Tran, Swetha Venkatesh, and Hung Le. 2024. Large language models prompting with episodic memory. In Proceedings of the 27th European Conference on Artificial Intelligence 2024.

Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2021. Lora: Low-rank adaptation of large language models. Preprint, arXiv:2106.09685.

Harsh Jhamtani, Varun Gangal, Eduard Hovy, and Eric Nyberg. 2017. Shakespearizing modern language using copy-enriched sequence-to-sequence models. Preprint, arXiv:1707.01161.

Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, Lélio Renard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibault Lavril, Thomas Wang, Timothée Lacroix, and William El Sayed. 2023. Mistral 7b. Preprint, arXiv:2310.06825.

Máté Lengyel and Peter Dayan. 2007. Hippocampal contributions to control: the third way. Advances in neural information processing systems, 20.

Chin-Yew Lin. 2004. ROUGE: A package for automatic evaluation of summaries. In Text Summarization Branches Out, pages 74–81, Barcelona, Spain. Association for Computational Linguistics.

Jiachang Liu, Dinghan Shen, Yizhe Zhang, Bill Dolan, Lawrence Carin, and Weizhu Chen. 2021. What makes good in-context examples for gpt-3? Preprint, arXiv:2101.06804.

Sheng Liu, Haotian Ye, Lei Xing, and James Y. Zou. 2024. In-context vectors: Making in context learning more effective and controllable through latent space steering. In Proceedings of the 41st International Conference on Machine Learning, volume 235 of Proceedings of Machine Learning Research, pages 32287–32307. PMLR.

Varvara Logacheva, Daryna Dementieva, Sergey Ustyantsev, Danil Moskovskiy, David Dale, Irina Krotova, Nikita Semenov, and Alexander Panchenko. 2022. ParaDetox: Detoxification with parallel data. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 6804–6818, Dublin, Ireland. Association for Computational Linguistics.

Sewon Min, Xinxi Lyu, Ari Holtzman, Mikel Artetxe, Mike Lewis, Hannaneh Hajishirzi, and Luke Zettlemoyer. 2022. Rethinking the role of demonstrations: What makes in-context learning work? Preprint, arXiv:2202.12837.

Kiho Park, Yo Joong Choe, and Victor Veitch. 2024. The linear representation hypothesis and the geometry of large language models. Preprint, arXiv:2311.03658.

Kha Pham, Hung Le, Man Ngo, and Truyen Tran. 2025. Rapid selection and ordering of in-context demonstrations via prompt embedding clustering. In The Thirteenth International Conference on Learning Representations.

Sudha Rao and Joel Tetreault. 2018. Dear sir or madam, may I introduce the GYAFC dataset: Corpus, benchmarks and metrics for formality style transfer. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pages 129–140, New Orleans, Louisiana. Association for Computational Linguistics.

Stephen Robertson and Hugo Zaragoza. 2009. The probabilistic relevance framework: Bm25 and beyond. Found. Trends Inf. Retr., 3(4):333–389.

Gemma Team. 2024. Gemma.

Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cristian Canton Ferrer, Moya Chen, Guillem Cucurull, David Esibou, Jude Fernandes, Jeremy Fu, Wenjin Fu, Brian Fuller, Cynthia Gao, Vedanuj Goswami, Naman Goyal, Anthony Harthorn, Saghar Hosseini, Rui Hou, Hakan Inan, Marcin Kardas, Viktor Kerkez, Madian Khabsa, Isabel Kloumann, Artem Korenev, Punit Singh Koura, Marie-Anne Lachaux, Thibaut Lavril, Jenya Lee, Diana Liskovich, Yinghai Lu, Yuning Mao, Xavier Martinet, Todor Mihaylov, Pushkar Mishra, Igor Molbyog, Yixin Nie, Andrew Poulton, Jeremy Reizenstein, Rash Rungta, Kalyan Saladi, Alan Schelten, Ruan Silva, Eric Michael Smith, Ranjan Subramanian, Xiaoqing Ellen Tan, Binh Tang, Ross Taylor, Adina Williams, Jian Xiang Kuan, Puxin Xu, Zheng Yan, Iliyan Zarov, Yuchen Zhang, Angela Fan, Melanie Kambadur, Sharan Narang, Aurelien Rodriguez, Robert Stojnic, Sergey Edunov, and Thomas Scialom. 2023. Llama 2: Open foundation and fine-tuned chat models. Preprint, arXiv:2307.09288.

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in Neural Information Processing Systems, volume 30. Curran Associates, Inc.

Daniel Votipka, Kelsey R. Fulton, James Parker, Matthew Hou, Michelle L. Mazurek, and Michael W. Hicks. 2020. Understanding security mistakes developers make: Qualitative analysis from build it, break it, fix it. In USENIX Security Symposium.

Weixuan Wang, JINGYUAN YANG, and Wei Peng. 2025. Semantics-adaptive activation intervention for LLMs via dynamic steering vectors. In The Thirteenth International Conference on Learning Representations.

Jason Wei, Yi Tay, Rishi Bommasani, Colin Raffel, Barret Zoph, Sebastian Borgeaud, Dani Yogatama,
Maarten Bosma, Denny Zhou, Donald Metzler, et al. 2022. Emergent abilities of large language models. Transactions on Machine Learning Research.

Ellery Wulczyn, Nithum Thain, and Lucas Dixon. 2017. Ex machina: Personal attacks seen at scale. Preprint, arXiv:1610.08914.

Jing Xu, Da Ju, Margaret Li, Y-Lan Boureau, Jason Weston, and Emily Dinan. 2021. Bot-adversarial dialogue for safe conversational agents. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 2950–2968, Online. Association for Computational Linguistics.

Tianjun Zhang, Xuezhi Wang, Denny Zhou, Dale Schuurmans, and Joseph E Gonzalez. 2022. Tempera: Test-time prompting via reinforcement learning. arXiv preprint arXiv:2211.11890.

Tianyi Zhang, Varsha Kishore, Felix Wu, Kilian Q. Weinberger, and Yoav Artzi. 2020. Bertscore: Evaluating text generation with bert. Preprint, arXiv:1904.09675.

Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, Hao Zhang, Joseph E Gonzalez, and Ion Stoica. 2023. Judging llm-as-a-judge with mt-bench and chatbot arena. In Advances in Neural Information Processing Systems, volume 36, pages 46595–46623. Curran Associates, Inc.

Yaowei Zheng, Richong Zhang, Junhao Zhang, Yanhan Ye, Zheyuan Luo, Zhangchi Feng, and Yongqiang Ma. 2024. Llamafactory: Unified efficient fine-tuning of 100+ language models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 3: System Demonstrations), Bangkok, Thailand. Association for Computational Linguistics.

A Appendix

A.1 Pseudocode for Memory Read of DSEM

We provide the pseudocode for Memory Read process in Algorithm 1

Algorithm 1 Token-level Steering With Memory Read

Require: Memory \( \mathcal{M} \), Language Model \( \mathcal{L} \), Token step \( l \), Demo set \( \mathcal{D}_{demo} \), Test sample \( x_{test} \), Max tokens \( L \), End-of-sequence token <eos>
Initialize: \( y^l \leftarrow y^0 \), \( t \leftarrow 0 \), termination \( \leftarrow \) False
Compute \( v_{global} \) via Eq. (5)
Obtain \( s(x_{test}) \)
while not termination do
    Compute \( R_{dense}, R_{BM25} \)
    Compute \( R_{RRF} \) via Eq. (2), retrieve \( \mathcal{N}_{sent} \)
    if \( t = 0 \) then
        Retrieve \( \mathcal{N}_{tok} \) from top-\( K_{sent} \) (based on \( R_{RRF} \))
        \( v_{memory} \leftarrow \frac{1}{|\mathcal{N}_{tok}|} \sum_{j=1}^{|\mathcal{N}_{tok}|} v_j \)
    else
        \( v_{memory} \leftarrow \frac{1}{|\mathcal{N}_{sent}|} \sum_{j=1}^{|\mathcal{N}_{sent}|} v_j \)
    end if
    Interpolate \( v_{steer} = (1 - \delta(x_{test})) \cdot v_{memory} + \delta(x_{test}) \cdot v_{global} \)
    Add \( v_{steer} \) at all layers of \( \mathcal{L} \): \( \hat{v}_L^l = v_L^l + \lambda \cdot v_{steer}^l \)
    Generate next \( l \) tokens: \( y^{t+l} \leftarrow \mathcal{L}_{steered}(y^l, x_{test}) \)
    Update sequence: \( y^l \leftarrow \operatorname{concat}(y^l, y^{t+l}) \)
    Update number of generated tokens: \( t \leftarrow t + l \)
    termination \( \leftarrow (t \geq L) \) or (\(<eos> \in y^{t+l}\))
end while

A.2 Dense Retrieval and BM25 Scoring Details

We show detailed formulas for Dense Retrieval and BM25 retrieval here. For dense retrieval, we retrieve \( \mathcal{R}_{dense} \) for a test query \( x_{test} \) as the dense retrieval ranking based on their cosine distances with the test query:

\[
d_{cs}(s(x_{test}), s(x_i)) = 1 - \frac{s(x_{test}) \cdot s(x_i)}{\|s(x_{test})\| \|s(x_i)\|}
\]

where \( s(x_i) \) is a latent representation of demonstration source \( x_i \) saved in \( \mathcal{M} \).

To complement dense retrieval, we rank memory elements using BM25, which captures lexical
<table>
  <tr>
    <th rowspan="2">Method</th>
    <th colspan="3">Llama-2-7b</th>
    <th colspan="3">Gemma-9b</th>
    <th colspan="3">Falcon-7b</th>
    <th colspan="3">Mistral-7b</th>
  </tr>
  <tr>
    <th>Tox.</th><th>R-1↑</th><th>BERT↑</th>
    <th>Tox.</th><th>R-1↑</th><th>BERT↑</th>
    <th>Tox.</th><th>R-1↑</th><th>BERT↑</th>
    <th>Tox.</th><th>R-1↑</th><th>BERT↑</th>
  </tr>
  <tr>
    <td>No Context</td>
    <td>39.00</td><td>52.11</td><td>90.75</td>
    <td>44.28</td><td>34.20</td><td>83.82</td>
    <td>89.14</td><td>67.75</td><td>92.04</td>
    <td>31.28</td><td>38.78</td><td>89.66</td>
  </tr>
  <tr>
    <td>ICL - 4</td>
    <td>41.43</td><td>38.48</td><td>85.09</td>
    <td>22.14</td><td>52.14</td><td>90.24</td>
    <td>84.00</td><td>70.67</td><td>92.91</td>
    <td>22.43</td><td>49.77</td><td>90.17</td>
  </tr>
  <tr>
    <th colspan="13">E = 10</th>
  </tr>
  <tr>
    <td>ICL - 10</td>
    <td>25.85</td><td>29.35</td><td>41.03</td>
    <td>14.00</td><td>52.61</td><td>88.98</td>
    <td>85.26</td><td>34.64</td><td>85.67</td>
    <td>24.71</td><td>59.94</td><td>89.82</td>
  </tr>
  <tr>
    <td>ICV</td>
    <td>24.86</td><td>55.09</td><td>91.12</td>
    <td>6.71</td><td>36.15</td><td>85.50</td>
    <td>32.14</td><td>63.17</td><td>91.76</td>
    <td>15.57</td><td>38.04</td><td>89.43</td>
  </tr>
  <tr>
    <td>SaDI</td>
    <td>28.14</td><td>45.50</td><td>90.48</td>
    <td>78.29</td><td>63.66</td><td>90.93</td>
    <td>84.00</td><td>70.72</td><td>92.91</td>
    <td>22.57</td><td>34.52</td><td>89.26</td>
  </tr>
  <tr>
    <td>LoRA</td>
    <td>39.71</td><td>46.47</td><td>90.70</td>
    <td>82.71</td><td>62.38</td><td>90.59</td>
    <td>85.14</td><td>70.98</td><td>92.95</td>
    <td>26.38</td><td>42.72</td><td>90.10</td>
  </tr>
  <tr>
    <td>DSEM</td>
    <td><b>21.42</b></td><td><b>56.64</b></td><td><b>91.68</b></td>
    <td><b>5.57</b></td><td><b>27.37</b></td><td><b>88.29</b></td>
    <td><b>32.57</b></td><td><b>65.05</b></td><td><b>92.98</b></td>
    <td><b>12.29</b></td><td><b>35.68</b></td><td><b>89.98</b></td>
  </tr>
  <tr>
    <th colspan="13">E = 100</th>
  </tr>
  <tr>
    <td>ICV</td>
    <td>18.42</td><td>50.47</td><td>90.77</td>
    <td>5.67</td><td>25.60</td><td>84.62</td>
    <td>26.43</td><td>57.19</td><td>91.52</td>
    <td>11.71</td><td>33.32</td><td>88.61</td>
  </tr>
  <tr>
    <td>SaDI</td>
    <td>17.86</td><td>45.08</td><td>90.44</td>
    <td>80.43</td><td><b>67.02</b></td><td><b>91.76</b></td>
    <td>84.14</td><td><b>70.99</b></td><td><b>92.98</b></td>
    <td>23.00</td><td><b>34.54</b></td><td><b>89.23</b></td>
  </tr>
  <tr>
    <td>LoRA</td>
    <td>47.14</td><td>70.74</td><td>93.23</td>
    <td>61.29</td><td>61.94</td><td>89.54</td>
    <td>58.29</td><td>68.61</td><td>92.38</td>
    <td>33.28</td><td><b>70.64</b></td><td><b>93.28</b></td>
  </tr>
  <tr>
    <td>DSEM</td>
    <td><b>15.70</b></td><td><b>50.97</b></td><td><b>90.90</b></td>
    <td><b>4.29</b></td><td><b>27.70</b></td><td><b>88.15</b></td>
    <td><b>21.42</b></td><td><b>56.98</b></td><td><b>92.75</b></td>
    <td><b>9.71</b></td><td><b>33.54</b></td><td><b>88.99</b></td>
  </tr>
</table>

Table 4: Model Comparison on Detoxification Task. Results (percentages) for various methods. "Tox." indicates Toxicity; ICL-4 and ICL-10 use 4 and 10 random demonstrations, respectively. ICL with \( E = 100 \) is omitted due to context length limits. For ICV, LoRA, and DSEM, results are shown for different \( E \). Lower Toxicity scores are better, while higher ROUGE-1 (R-1) and BERT scores indicate better relevance. Best results are bolded, second-best underlined. Notation excludes No Context and ICV-4.

overlap. Given a test query \( x_{\text{test}} \), the BM25 score for a memory element \( x_i \) is computed as:

\[
\text{BM25}(x_i, x_{\text{test}}) = \sum_{t_j \in x_{\text{test}}} \frac{\text{IDF}(t_j) \cdot f_{t_j, x_i} \cdot (k_1 + 1)}{f_{t_j, x_i} + k_1 \cdot (1 - b + b \cdot \frac{|x_i|}{\text{avgdl}})}
\]

where \( f_{t_j, x_i} \) is the term frequency of \( t_j \) in \( x_i \), and \( |x_i| \) is its word count. The average length of elements in \( \mathcal{M} \) is denoted as \( \text{avgdl} \). Hyperparameters \( k_1 \) and \( b \) control term frequency saturation and length normalization, respectively.

The inverse document frequency (IDF) of a term \( t_j \) is given by:

\[
\text{IDF}(t_j) = \log \left( \frac{N - n_{t_j} + 0.5}{n_{t_j} + 0.5} + 1 \right)
\]

where \( N \) is the total number of memory elements, and \( n_{t_j} \) is the count of elements containing \( t_j \). We rank memory elements based on their BM25 scores and fuse rankings with dense retrieval using RRF.

A.3 Additional results: Toxicity.

We present the complete results of four LLMs: Llama-2-7b, Gemma-9b, Falcon-7b, and Mistral-7b on the Paradoxet dataset. The results are summarized in Table 4. As shown in the table, DSEM achieves the best Toxicity scores in all but one setting, demonstrating the effectiveness of the dynamic steering it provides. For instance, with Mistral-7b, DSEM achieves a 12.29% Toxicity score while ranking second in terms of BERTScore. Similarly, for Llama-2-7b, DSEM achieves the best scores across all metrics—Toxicity, ROUGE-1, and BERTScore—with \( E = 10 \). Regarding relevance, while DSEM lags behind SaDI and LoRA in certain settings, it is important to note that their Toxicity scores are significantly higher. This trade-off indicates that their relevance scores become less meaningful, as such an imbalance is suboptimal.

A.4 Additional results: Formality.

We provide Formality results of Falcon-7b and Mistral-7b in Table 5. With Mistral, DSEM achieves near-perfect scores (96.62% for GYAFC_family, 96.30% for GYAFC_music) with \( E = 100 \). A similar trend is observed when \( E = 10 \). Mistral-7b outperforms ICV (the second best baselines) in both GYAFC_Family and GYAFC_Music by 8.82 % and 7.58 % while having higher relevant scores. This further indicates the superior in steering control of DSEM.

On GYAFC_music, DSEM more than doubles the third-best Formality score (39.68% vs. 15.87%). Even losing to ICL-10 in \( E = 10 \) setting, DSEM improves significantly when \( E = 100 \). We suspect that with more demonstration examples, DSEM is more likely to find suitable steering vector via closer neighbors, thus illustrating better performances.
<table>
  <tr>
    <th rowspan="2">Method</th>
    <th colspan="3">Llama-2-7b</th>
    <th colspan="3">Gemma-9b</th>
  </tr>
  <tr>
    <th colspan="3">GYAFC_family</th>
    <th colspan="3">GYAFC_music</th>
  </tr>
  <tr>
    <th></th>
    <th>For.</th>
    <th>R-1</th>
    <th>BERT</th>
    <th>For.</th>
    <th>R-1</th>
    <th>BERT</th>
    <th>For.</th>
    <th>R-1</th>
    <th>BERT</th>
    <th>For.</th>
    <th>R-1</th>
    <th>BERT</th>
  </tr>
  <tr>
    <td>No Context</td>
    <td>39.77</td>
    <td>49.60</td>
    <td>92.19</td>
    <td>40.56</td>
    <td>52.34</td>
    <td>92.25</td>
    <td>28.71</td>
    <td>30.42</td>
    <td>82.55</td>
    <td>27.51</td>
    <td>29.34</td>
    <td>80.11</td>
  </tr>
  <tr>
    <td>ICL-4</td>
    <td>26.45</td>
    <td>34.72</td>
    <td>55.13</td>
    <td>45.33</td>
    <td>41.51</td>
    <td>64.93</td>
    <td>60.60</td>
    <td>50.66</td>
    <td>92.27</td>
    <td>85.36</td>
    <td>42.71</td>
    <td>89.56</td>
  </tr>
  <tr>
    <th colspan="13">E = 10</th>
  </tr>
  <tr>
    <td>ICL-10</td>
    <td>51.59</td>
    <td>43.63</td>
    <td>66.69</td>
    <td>53.62</td>
    <td>46.54</td>
    <td>70.77</td>
    <td><b>83.11</b></td>
    <td>51.74</td>
    <td>90.57</td>
    <td><b>83.42</b></td>
    <td>48.73</td>
    <td>90.22</td>
  </tr>
  <tr>
    <td>ICV</td>
    <td>50.28</td>
    <td>46.40</td>
    <td>91.34</td>
    <td>66.84</td>
    <td>48.89</td>
    <td>91.67</td>
    <td>51.78</td>
    <td>33.47</td>
    <td>86.61</td>
    <td>76.19</td>
    <td>22.92</td>
    <td>85.46</td>
  </tr>
  <tr>
    <td>SaDI</td>
    <td>53.66</td>
    <td>47.18</td>
    <td>92.11</td>
    <td>57.14</td>
    <td>49.54</td>
    <td>92.23</td>
    <td>8.07</td>
    <td><b>61.61</b></td>
    <td><b>92.06</b></td>
    <td>5.82</td>
    <td><b>60.72</b></td>
    <td><b>91.38</b></td>
  </tr>
  <tr>
    <td>LoRA</td>
    <td>50.84</td>
    <td>46.75</td>
    <td>92.04</td>
    <td>19.71</td>
    <td>46.50</td>
    <td>90.70</td>
    <td>7.31</td>
    <td><b>61.08</b></td>
    <td><b>92.18</b></td>
    <td>5.64</td>
    <td><b>61.69</b></td>
    <td><b>91.43</b></td>
  </tr>
  <tr>
    <td>DSEM</td>
    <td><b>66.04</b></td>
    <td><b>47.90</b></td>
    <td><b>92.26</b></td>
    <td><b>76.37</b></td>
    <td><b>50.05</b></td>
    <td><b>92.26</b></td>
    <td><b>96.44</b></td>
    <td>30.07</td>
    <td>90.35</td>
    <td><b>96.47</b></td>
    <td>37.25</td>
    <td>90.59</td>
  </tr>
  <tr>
    <th colspan="13">E = 100</th>
  </tr>
  <tr>
    <td>ICV</td>
    <td>69.36</td>
    <td>48.54</td>
    <td>91.50</td>
    <td>55.73</td>
    <td>54.71</td>
    <td>91.6</td>
    <td><b>85.37</b></td>
    <td>26.29</td>
    <td>86.24</td>
    <td><b>78.13</b></td>
    <td>19.91</td>
    <td>83.97</td>
  </tr>
  <tr>
    <td>SaDI</td>
    <td>52.91</td>
    <td>46.71</td>
    <td>92.13</td>
    <td>59.44</td>
    <td>49.66</td>
    <td>92.15</td>
    <td>7.69</td>
    <td>61.57</td>
    <td><b>92.60</b></td>
    <td>6.80</td>
    <td>61.06</td>
    <td>90.13</td>
  </tr>
  <tr>
    <td>LoRA</td>
    <td>34.14</td>
    <td><b>61.45</b></td>
    <td><b>93.77</b></td>
    <td>32.98</td>
    <td><b>64.24</b></td>
    <td><b>93.70</b></td>
    <td>5.25</td>
    <td><b>62.34</b></td>
    <td>90.27</td>
    <td>5.82</td>
    <td><b>69.70</b></td>
    <td><b>94.42</b></td>
  </tr>
  <tr>
    <td>DSEM</td>
    <td><b>85.00</b></td>
    <td>51.69</td>
    <td>92.11</td>
    <td><b>83.60</b></td>
    <td>51.71</td>
    <td><b>92.22</b></td>
    <td><b>97.75</b></td>
    <td>30.15</td>
    <td>90.29</td>
    <td><b>94.53</b></td>
    <td>35.45</td>
    <td>90.41</td>
  </tr>
  <tr>
    <th colspan="13">Falcon-7b</th>
  </tr>
  <tr>
    <th rowspan="2">Method</th>
    <th colspan="3">GYAFC_family</th>
    <th colspan="3">GYAFC_music</th>
    <th colspan="3">GYAFC_family</th>
    <th colspan="3">GYAFC_music</th>
  </tr>
  <tr>
    <th>For.</th>
    <th>R-1</th>
    <th>BERT</th>
    <th>For.</th>
    <th>R-1</th>
    <th>BERT</th>
    <th>For.</th>
    <th>R-1</th>
    <th>BERT</th>
    <th>For.</th>
    <th>R-1</th>
    <th>BERT</th>
  </tr>
  <tr>
    <td>No Context</td>
    <td>4.70</td>
    <td>61.58</td>
    <td>92.73</td>
    <td>6.52</td>
    <td>60.40</td>
    <td>91.72</td>
    <td>65.67</td>
    <td>35.42</td>
    <td>90.80</td>
    <td>63.32</td>
    <td>40.45</td>
    <td>90.89</td>
  </tr>
  <tr>
    <td>ICL-4</td>
    <td>40.71</td>
    <td>27.09</td>
    <td>84.81</td>
    <td>45.15</td>
    <td>35.74</td>
    <td>86.16</td>
    <td>51.59</td>
    <td>42.72</td>
    <td>80.00</td>
    <td>71.43</td>
    <td>45.65</td>
    <td>81.53</td>
  </tr>
  <tr>
    <th colspan="13">E = 10</th>
  </tr>
  <tr>
    <td>ICL-10</td>
    <td><b>41.78</b></td>
    <td>54.47</td>
    <td>91.78</td>
    <td><b>41.62</b></td>
    <td>55.18</td>
    <td>90.91</td>
    <td>74.86</td>
    <td><b>54.22</b></td>
    <td>90.85</td>
    <td>75.66</td>
    <td><b>52.74</b></td>
    <td>90.15</td>
  </tr>
  <tr>
    <td>ICV</td>
    <td>17.07</td>
    <td><b>66.27</b></td>
    <td><b>93.82</b></td>
    <td>15.87</td>
    <td><b>66.08</b></td>
    <td><b>93.51</b></td>
    <td><b>83.49</b></td>
    <td>30.89</td>
    <td>89.93</td>
    <td><b>88.89</b></td>
    <td>39.44</td>
    <td>90.57</td>
  </tr>
  <tr>
    <td>SaDI</td>
    <td>4.88</td>
    <td>64.05</td>
    <td>93.26</td>
    <td>5.82</td>
    <td>62.24</td>
    <td>92.10</td>
    <td>67.73</td>
    <td>36.66</td>
    <td>90.91</td>
    <td>68.08</td>
    <td>41.80</td>
    <td>91.02</td>
  </tr>
  <tr>
    <td>LoRA</td>
    <td>3.75</td>
    <td>64.34</td>
    <td>93.38</td>
    <td>5.29</td>
    <td>63.21</td>
    <td>92.34</td>
    <td>50.84</td>
    <td><b>44.11</b></td>
    <td><b>91.75</b></td>
    <td>26.29</td>
    <td>42.79</td>
    <td>90.10</td>
  </tr>
  <tr>
    <td>DSEM</td>
    <td><b>34.90</b></td>
    <td><b>67.42</b></td>
    <td><b>93.90</b></td>
    <td><b>39.68</b></td>
    <td><b>69.02</b></td>
    <td><b>94.23</b></td>
    <td><b>92.31</b></td>
    <td>32.76</td>
    <td>90.66</td>
    <td><b>96.47</b></td>
    <td>40.74</td>
    <td><b>91.32</b></td>
  </tr>
  <tr>
    <th colspan="13">E = 100</th>
  </tr>
  <tr>
    <td>ICV</td>
    <td>55.16</td>
    <td>66.09</td>
    <td>93.99</td>
    <td>32.09</td>
    <td>66.25</td>
    <td>93.40</td>
    <td><b>93.99</b></td>
    <td>33.29</td>
    <td>90.22</td>
    <td><b>92.59</b></td>
    <td>39.21</td>
    <td>90.52</td>
  </tr>
  <tr>
    <td>SaDI</td>
    <td>7.69</td>
    <td><b>61.57</b></td>
    <td>92.60</td>
    <td>6.80</td>
    <td>61.06</td>
    <td>90.13</td>
    <td>65.48</td>
    <td><b>36.37</b></td>
    <td>90.87</td>
    <td>68.25</td>
    <td><b>42.17</b></td>
    <td>91.06</td>
  </tr>
  <tr>
    <td>LoRA</td>
    <td>27.58</td>
    <td><b>68.74</b></td>
    <td><b>94.54</b></td>
    <td>25.57</td>
    <td><b>69.70</b></td>
    <td><b>94.42</b></td>
    <td>21.39</td>
    <td><b>62.23</b></td>
    <td><b>92.96</b></td>
    <td>23.81</td>
    <td><b>65.26</b></td>
    <td><b>93.64</b></td>
  </tr>
  <tr>
    <td>DSEM</td>
    <td><b>64.43</b></td>
    <td>66.87</td>
    <td>94.02</td>
    <td><b>48.85</b></td>
    <td><b>69.32</b></td>
    <td><b>94.16</b></td>
    <td><b>96.62</b></td>
    <td>35.36</td>
    <td>90.88</td>
    <td><b>96.30</b></td>
    <td>40.97</td>
    <td><b>91.28</b></td>
  </tr>
</table>

Table 5: Comparison of different methods on Paradetox and two formality transfer datasets using all 4 LLMs: Llama-2-7B, Gemma-9b, Mistral-7b and Falcon-7B with \( E = 10 \) and \( E = 100 \). "Tox." denotes Toxicity, "For." denotes Formality, "R-1" represents ROUGE-1, and "BERT" refers to BERTScore. Lower values are better for Toxicity, while higher values are better for all other metrics. In this table, for DSEM, \( K_{tok} = K_{sent} = 5 \) for all runs. Best downstream performance results (Toxicity and Formality) are bolded, with second-best results underlined.

A.5 Additional results: Performance comparison on larger LLMs.

We provide results on the Paradetox, GYAFC_Family, and GYAFC_Music datasets using larger LLMs: meta-llama/Llama-2-13b-chat-hf and Falcon-11B (Touvron et al., 2023; Almazrouei et al., 2023). Results can be seen in 6. Our approach is compared against two state-of-the-art baselines: ICV (Liu et al., 2024) and SaDI (Wang et al., 2025). The detailed results are presented in Table 6. It is evident that DSEM consistently outperforms other methods across the majority of tasks, particularly excelling on GYAFC_Family and GYAFC_Music. For instance, with Llama-2-13B, DSEM achieves an impressive 95.68% formality rate at \( E = 100 \), while still maintaining high BERTScore and ROUGE-1 scores, demonstrating its effectiveness in preserving both formality and content quality. This highlights DSEM’s robustness in structured text transformation. For Paradetox, DSEM continues to deliver strong performance. Notably, it achieves the best downstream results across 3 datasets with Falcon-11B at \( E = 10 \) and remains highly competitive at \( E = 100 \), securing the best on GYAFC datasets and second-best performance on Paradetox. These results suggest that DSEM is particularly effective in leveraging larger models and dataset-specific characteristics, providing strong improvements over existing baselines. These results suggest the ability to scale up to larger LLMs of DSEM, proposing a rapid and
<table>
  <tr>
    <th rowspan="2">Method</th>
    <th colspan="3">Paradetox</th>
    <th colspan="3">GYAFC_Family</th>
    <th colspan="3">GYAFC_Music</th>
  </tr>
  <tr>
    <th>Tox.</th>
    <th>R-1</th>
    <th>BERT</th>
    <th>For.</th>
    <th>R-1</th>
    <th>BERT</th>
    <th>For.</th>
    <th>R-1</th>
    <th>BERT</th>
  </tr>
  <tr>
    <th colspan="10">E = 10</th>
  </tr>
  <tr>
    <td>ICV</td>
    <td>20.71</td>
    <td><b>56.18</b></td>
    <td>91.43</td>
    <td>83.49</td>
    <td>34.12</td>
    <td>90.04</td>
    <td>65.08</td>
    <td>50.11</td>
    <td>92.06</td>
  </tr>
  <tr>
    <td>SaDI</td>
    <td><b>17.28</b></td>
    <td>41.44</td>
    <td>90.02</td>
    <td>60.41</td>
    <td><b>42.72</b></td>
    <td><b>91.49</b></td>
    <td>63.49</td>
    <td>45.74</td>
    <td>91.65</td>
  </tr>
  <tr>
    <td>DSEM</td>
    <td>19.86</td>
    <td>54.40</td>
    <td><b>91.61</b></td>
    <td><b>87.42</b></td>
    <td>37.89</td>
    <td>90.81</td>
    <td><b>73.37</b></td>
    <td><b>51.20</b></td>
    <td><b>92.59</b></td>
  </tr>
  <tr>
    <th colspan="10">E = 100</th>
  </tr>
  <tr>
    <td>ICV</td>
    <td>15.43</td>
    <td>46.71</td>
    <td>90.13</td>
    <td>93.62</td>
    <td>37.83</td>
    <td>90.54</td>
    <td><b>85.71</b></td>
    <td>41.09</td>
    <td>90.46</td>
  </tr>
  <tr>
    <td>SaDI</td>
    <td>15.88</td>
    <td>41.91</td>
    <td>90.07</td>
    <td>60.40</td>
    <td>42.71</td>
    <td><b>91.49</b></td>
    <td>63.69</td>
    <td>45.64</td>
    <td>91.66</td>
  </tr>
  <tr>
    <td>DSEM</td>
    <td><b>14.89</b></td>
    <td><b>47.21</b></td>
    <td><b>90.09</b></td>
    <td><b>95.68</b></td>
    <td><b>42.81</b></td>
    <td>91.29</td>
    <td>81.83</td>
    <td><b>49.35</b></td>
    <td><b>92.23</b></td>
  </tr>
</table>

<table>
  <tr>
    <th rowspan="2">Method</th>
    <th colspan="3">Paradetox</th>
    <th colspan="3">GYAFC_Family</th>
    <th colspan="3">GYAFC_Music</th>
  </tr>
  <tr>
    <th>Tox.</th>
    <th>R-1</th>
    <th>BERT</th>
    <th>For.</th>
    <th>R-1</th>
    <th>BERT</th>
    <th>For.</th>
    <th>R-1</th>
    <th>BERT</th>
  </tr>
  <tr>
    <th colspan="10">E = 10</th>
  </tr>
  <tr>
    <td>ICV</td>
    <td>25.43</td>
    <td>54.48</td>
    <td>91.51</td>
    <td>63.41</td>
    <td><b>60.71</b></td>
    <td><b>93.28</b></td>
    <td>87.47</td>
    <td>49.47</td>
    <td>92.18</td>
  </tr>
  <tr>
    <td>SaDI</td>
    <td>31.71</td>
    <td>49.36</td>
    <td>91.02</td>
    <td>64.92</td>
    <td>48.64</td>
    <td>92.63</td>
    <td>62.43</td>
    <td><b>52.37</b></td>
    <td><b>92.64</b></td>
  </tr>
  <tr>
    <td>DSEM</td>
    <td><b>21.29</b></td>
    <td><b>55.18</b></td>
    <td><b>91.63</b></td>
    <td><b>81.61</b></td>
    <td>53.52</td>
    <td>92.32</td>
    <td><b>88.00</b></td>
    <td>49.31</td>
    <td>91.86</td>
  </tr>
  <tr>
    <th colspan="10">E = 100</th>
  </tr>
  <tr>
    <td>ICV</td>
    <td><b>15.42</b></td>
    <td>46.43</td>
    <td>90.49</td>
    <td>90.68</td>
    <td>36.71</td>
    <td>90.78</td>
    <td>90.59</td>
    <td>34.15</td>
    <td>89.98</td>
  </tr>
  <tr>
    <td>SaDI</td>
    <td>33.14</td>
    <td><b>49.77</b></td>
    <td><b>91.01</b></td>
    <td>66.23</td>
    <td><b>48.45</b></td>
    <td><b>92.61</b></td>
    <td>63.84</td>
    <td><b>53.53</b></td>
    <td><b>92.76</b></td>
  </tr>
  <tr>
    <td>DSEM</td>
    <td><b>16.42</b></td>
    <td>47.55</td>
    <td>90.60</td>
    <td><b>94.93</b></td>
    <td>45.84</td>
    <td>91.45</td>
    <td><b>92.42</b></td>
    <td>46.41</td>
    <td>91.46</td>
  </tr>
</table>

Table 6: Comparison of different methods on Paradetox and two formality transfer datasets using Llama-2-13B and Falcon-11B with \( E = 10 \) and \( E = 100 \). "Tox." denotes Toxicity, "For." denotes Formality, "R-1" represents ROUGE-1, and "BERT" refers to BERTScore. Lower values are better for Toxicity, while higher values are better for all other metrics. In this table, for DSEM, \( K_{tok} = K_{sent} = 5 \) for all runs. Best downstream performance results (Toxicity and Formality) are bolded, with second-best results underlined.

![Bar chart showing DSEM top 10 most repeated words across generation responses](page_101_1042_496_312.png)

Figure 6: DSEM top 10 most repeated words across generation responses

![Bar chart showing ICV top 10 most repeated words across generation responses](page_627_1042_496_312.png)

Figure 7: ICV top 10 most repeated words across generation responses

model-free approach to enhance larger LMs performances.

A.6 Additional results: TruthfulQA
We evaluate DSEM on TruthfulQA, a dataset designed to assess whether LLMs generate truthful answers. We sample 700 examples from its test set for evaluation. A head-to-head comparison is conducted between DSEM and ICV and reported in Table 9, which we consider the strongest baseline based on previous results in this paper. To determine which response is better, we employ Coherer-plus as the judge. To demonstrate the generalizability of our improvement, we evaluate across four different model backbones. For all runs, we use \( E = 100 \) samples and \( K_{tok} = K_{sent} = 9 \). As shown in the results, DSEM outperforms the current state-of-the-art method, ICV, across all settings. Notably, with Gemma-9B, DSEM achieves an impressive 87.51% win rate, significantly surpassing
Figure 8: Downstream results (Toxicity, Formality) and BERT scores of DSEM on Paradetox and GYAFC datasets with various number of neighbors (1, 5, 9, 15, 20), using Llama-2-7b with \( E = 100 \). We note that for simplicity, we use \( K_{tok} = K_{sent} \) in all our experiments.

Figure 9: Results of DSEM with different values of \( \lambda \). We use Llama-2-7b with \( E = 100 \) on GYAFC Family dataset.

<table>
  <tr>
    <th>Dataset</th>
    <th>Scores</th>
    <th>R-1</th>
    <th>BERT</th>
  </tr>
  <tr>
    <td>Paradetox (\( \downarrow \))</td>
    <td>4.00</td>
    <td>27.54</td>
    <td>88.05</td>
  </tr>
  <tr>
    <td>GYAFC Family (\( \uparrow \))</td>
    <td>98.87</td>
    <td>26.85</td>
    <td>89.78</td>
  </tr>
  <tr>
    <td>GYAFC Music (\( \uparrow \))</td>
    <td>96.82</td>
    <td>31.42</td>
    <td>89.67</td>
  </tr>
</table>

Table 7: Comparison (in percentages) of DSEM using Gemma-9b(\( E = 1000 \)) examples. For Paradetox, a lower score is better (\( \downarrow \)), while for GYAFC datasets, a higher score is better (\( \uparrow \)).

ICV and further demonstrating the effectiveness of our approach.

A.7 Ablation: Different number of neighbors

We present results in Figure 8, showing the impact of varying the number of neighbors (1, 5, 9, 15, 20) on DSEM performance using Llama-2-7b with \( E = 100 \) on ParaDetox and GYAFC. For simplicity, we set \( K_{sent} = K_{tok} \) in all experiments. ParaDetox achieves optimal performance at \( K = 9 \), balancing toxicity reduction and semantic preservation, while GYAFC benefits from larger \( K \) (15 or 20), despite minor semantic drift. These results underline the task-specific nature of neighbor selection.

<table>
  <tr>
    <th>Method</th>
    <th>Parameters</th>
    <th>Value</th>
  </tr>
  <tr>
    <td rowspan="6">LoRA</td>
    <td>Number of epochs</td>
    <td>3</td>
  </tr>
  <tr>
    <td>Cutoff Length</td>
    <td>1024</td>
  </tr>
  <tr>
    <td>Preprocessing workers</td>
    <td>16</td>
  </tr>
  <tr>
    <td>Per device train batch size</td>
    <td>1</td>
  </tr>
  <tr>
    <td>bf16</td>
    <td>True</td>
  </tr>
  <tr>
    <td>Learning rate</td>
    <td>\( 1e^{-4} \)</td>
  </tr>
  <tr>
    <td rowspan="2">ICV</td>
    <td>Learning rate scheduler type</td>
    <td>cosine</td>
  </tr>
  <tr>
    <td>Gradient accumulation step</td>
    <td>8</td>
  </tr>
  <tr>
    <td rowspan="2">SaDI</td>
    <td>Warm up ratio</td>
    <td>0.1</td>
  </tr>
  <tr>
    <td>Intervention strength (\( \lambda \))</td>
    <td>0.1</td>
  </tr>
  <tr>
    <td>K</td>
    <td>6</td>
    <td></td>
  </tr>
  <tr>
    <td>Intervention strength (\( \delta \))</td>
    <td>5</td>
    <td></td>
  </tr>
</table>

Table 8: Baselines training details
<table>
  <tr>
    <th>Model</th>
    <th>E</th>
    <th>Vs-baseline</th>
    <th>Winrate</th>
  </tr>
  <tr>
    <td>Llama-2-7b</td>
    <td>100</td>
    <td>ICV</td>
    <td>54.97</td>
  </tr>
  <tr>
    <td>Gemma-9b</td>
    <td>100</td>
    <td>ICV</td>
    <td>87.51</td>
  </tr>
  <tr>
    <td>Mistral-7b</td>
    <td>100</td>
    <td>ICV</td>
    <td>56.01</td>
  </tr>
  <tr>
    <td>Falcon-7b</td>
    <td>100</td>
    <td>ICV</td>
    <td>54.74</td>
  </tr>
</table>

Table 9: Win rates of DSEM versus ICV baseline on TruthfulQA dataset with \( E = 100 \)

A.8 Ablation: More examples

We evaluate the performance of DSEM with \( E = 1000 \) across three datasets using Gemma-9b. Results can be seen in Table 7. The Toxicity score and the Formality score on GYAFC Music improve compared to settings with lower \( E \), while the relevance scores are only marginally lower. This demonstrates that DSEM is susceptible to noise when more memory elements are introduced, highlighting its scalability challenges.

A.9 Ablation: Different values of intervention weight \( \lambda \)

We study the effect of different values of \( \lambda \) using Llama-2-7b with \( E = 100 \) on the Formality Family dataset. The results are presented in Figure 9. As \( \lambda \) increases, formality steadily improves, peaking at \( \lambda = 0.08 \) before slightly declining. BERTScore remains relatively stable across all values, with a small dip observed beyond \( \lambda = 0.1 \). These findings suggest that \( \lambda = 0.1 \) achieves the best trade-off, optimizing text quality while maintaining both formality and semantic similarity.

A.10 Ablation: Inference Speed and Memory Usage

We include analysis of inference speed in Table 10 and memory usage of our episodic memory in Table 12. Since these values do not vary significantly across different datasets, we report results only for the Paradetox dataset.

Inference Speed. Table 10 compares toxicity rates and running times using *Gemma-9B* across all baselines and settings (*with and without* episodic memory) on the *Paradetox* sampled test set (700 samples). It can be seen that *DSEM* achieves a significantly lower toxicity rate compared to other baselines, while also being faster than the primary baseline, *ICV*. This highlights *DSEM*'s effectiveness in producing more aligned outputs while maintaining reasonable speed.

GPU Memory Usage. Table 12 shows the GPU memory consumption (in MB) of our episodic memory. As the usage does not vary significantly across datasets, we report numbers only for the *Paradetox* dataset. As expected, our memory module stores only a limited number of examples (as few as 10), with each stored value being a steering vector. Consequently, our framework does not raise concerns regarding memory overflow.

A.11 Discussion on Quality of Memory Entries

In this section, we discuss how the quality of memory entries can effect the performance of our method. We follow standard practice in in-context learning when using curated data, a setup also adopted by prior state-of-the-art methods (e.g., ICV, SaDI). We do not cherry-pick the demonstrations; instead, we randomly select the in-context examples. The availability of demonstrations is required for both traditional in-context learning and activation steering methods.

A.12 Result Analysis: Paradetox

We provide further analysis in the extreme case of Gemma-9b. As shown in Table 1, with Gemma-9b and \( E = 100 \), the toxicity rates of ICV and our method DSEM are only approximately 5%, which is very low compared to other baselines. However, upon closer inspection, we notice that the answers produced by ICV contain repeated tokens, which fragment its BERTScore with respect to the reference sentence. We provide both a qualitative example and a quantitative plot of Word Repetition per Generation (Repetition count) (Figure 6 and Figure 7), which highlights how many words appear more than once in each generation. We can see that the word 'to' is repeated by ICV more than 1500 times, and 'the' is used more than 1400 times. In contrast, DSEM illustrates better behavior, with its most common repeated word, 'the', appearing less than 200 times. We also provide an example that shows the repetition of generations of ICV in Figure 10.

A.13 DSEM hyperparameters details.

We provide specific parameters used in DSEM in Table 13. In addition, we also provide the number of neighbors used in all of our experiments. We note that \( K_{tok} = K_{sent} \) for simplicity.
<table>
  <tr>
    <th>E</th>
    <th>Baseline</th>
    <th>Toxic rate</th>
    <th>Run time (hours)</th>
  </tr>
  <tr>
    <td>0</td>
    <td>Nocontext</td>
    <td>44.28</td>
    <td>0.20</td>
  </tr>
  <tr>
    <td rowspan="5">4</td>
    <td>ICL-4</td>
    <td>22.14</td>
    <td>0.35</td>
  </tr>
  <tr>
    <td>ICL-10</td>
    <td>14.00</td>
    <td>0.57</td>
  </tr>
  <tr>
    <td>ICV</td>
    <td>6.71</td>
    <td>0.63</td>
  </tr>
  <tr>
    <td>SaDI</td>
    <td>78.29</td>
    <td>0.40</td>
  </tr>
  <tr>
    <td>LoRA</td>
    <td>82.71</td>
    <td>0.28</td>
  </tr>
  <tr>
    <td rowspan="4">10</td>
    <td>DSEM</td>
    <td>5.57</td>
    <td>0.60</td>
  </tr>
  <tr>
    <td>ICV</td>
    <td>5.67</td>
    <td>0.67</td>
  </tr>
  <tr>
    <td>SaDI</td>
    <td>80.43</td>
    <td>0.45</td>
  </tr>
  <tr>
    <td>LoRA</td>
    <td>61.29</td>
    <td>0.42</td>
  </tr>
  <tr>
    <td>100</td>
    <td>DSEM</td>
    <td>4.29</td>
    <td>0.67</td>
  </tr>
</table>

Table 10: Toxic rate and run time of different models under various \( E \) values.

<table>
  <tr>
    <th rowspan="2">Model</th>
    <th colspan="4">E = 10</th>
  </tr>
  <tr>
    <th>Paradetox</th>
    <th>Family</th>
    <th>Music</th>
    <th>Shakespeare</th>
  </tr>
  <tr>
    <td>Llama-7b</td>
    <td>9</td>
    <td>5</td>
    <td>5</td>
    <td>9</td>
  </tr>
  <tr>
    <td>Falcon-7b</td>
    <td>5</td>
    <td>15</td>
    <td>15</td>
    <td>9</td>
  </tr>
  <tr>
    <td>Gemma-9b</td>
    <td>9</td>
    <td>9</td>
    <td>9</td>
    <td>5</td>
  </tr>
  <tr>
    <td>Mistral-7b</td>
    <td>9</td>
    <td>9</td>
    <td>9</td>
    <td>9</td>
  </tr>
  <tr>
    <td>Llama-13b</td>
    <td>5</td>
    <td>9</td>
    <td>9</td>
    <td>9</td>
  </tr>
  <tr>
    <td>Falcon-11b</td>
    <td>9</td>
    <td>5</td>
    <td>5</td>
    <td>9</td>
  </tr>
  <tr>
    <th colspan="5">E = 100</th>
  </tr>
  <tr>
    <td>Llama-7b</td>
    <td>9</td>
    <td>15</td>
    <td>15</td>
    <td>9</td>
  </tr>
  <tr>
    <td>Falcon-7b</td>
    <td>5</td>
    <td>15</td>
    <td>15</td>
    <td>9</td>
  </tr>
  <tr>
    <td>Gemma-9b</td>
    <td>9</td>
    <td>9</td>
    <td>9</td>
    <td>5</td>
  </tr>
  <tr>
    <td>Mistral-7b</td>
    <td>9</td>
    <td>9</td>
    <td>9</td>
    <td>9</td>
  </tr>
  <tr>
    <td>Llama-13b</td>
    <td>5</td>
    <td>9</td>
    <td>9</td>
    <td>9</td>
  </tr>
  <tr>
    <td>Falcon-11b</td>
    <td>9</td>
    <td>5</td>
    <td>5</td>
    <td>9</td>
  </tr>
</table>

Table 11: DSEM neighbors selected in experiments. We note that \( K_{tok} = K_{sent} \).

<table>
  <tr>
    <th>Model</th>
    <th>E</th>
    <th>Memory Used (MB)</th>
  </tr>
  <tr>
    <td>Llama-2-7b</td>
    <td>10</td>
    <td>131.90</td>
  </tr>
  <tr>
    <td></td>
    <td>100</td>
    <td>223.86</td>
  </tr>
  <tr>
    <td>Falcon-7b</td>
    <td>10</td>
    <td>113.07</td>
  </tr>
  <tr>
    <td></td>
    <td>100</td>
    <td>235.07</td>
  </tr>
  <tr>
    <td>Gemma-9b</td>
    <td>10</td>
    <td>133.32</td>
  </tr>
  <tr>
    <td></td>
    <td>100</td>
    <td>238.53</td>
  </tr>
  <tr>
    <td>Mistral-7b</td>
    <td>10</td>
    <td>131.94</td>
  </tr>
  <tr>
    <td></td>
    <td>100</td>
    <td>224.02</td>
  </tr>
  <tr>
    <td>Falcon-11b</td>
    <td>10</td>
    <td>140.70</td>
  </tr>
  <tr>
    <td></td>
    <td>100</td>
    <td>311.42</td>
  </tr>
  <tr>
    <td>Llama-13b</td>
    <td>10</td>
    <td>137.52</td>
  </tr>
  <tr>
    <td></td>
    <td>100</td>
    <td>280.69</td>
  </tr>
</table>

Table 12: Memory used of different models with \( E \in [10, 100] \)

<table>
  <tr>
    <th>Parameter</th>
    <th>Notation</th>
    <th>Value</th>
  </tr>
  <tr>
    <td>Chunk size</td>
    <td>\( l \)</td>
    <td>4</td>
  </tr>
  <tr>
    <td>Intervention strength</td>
    <td>\( \lambda \)</td>
    <td>0.1</td>
  </tr>
  <tr>
    <td>RRF constant</td>
    <td>\( k_{RRF} \)</td>
    <td>60</td>
  </tr>
</table>

Table 13: DSEM parameters

\( K = 6 \) as they claim it to be in range of best working parameters.

A.15 Hardware requirements.
All of our experiments were conducted on a single NVIDIA H100 GPU. We used pytorch=2.5.1, CUDA=12.1, cuDNN=9.1.0.70 and transformers=4.47.1.

A.16 Evaluation Prompts
We provide the prompts used for LLM evaluations in Figure 11, Figure 12 and Figure 13.

A.14 Other Baselines details.
To conduct LoRA fine-tuning, we utilize Llama-Factory (Zheng et al., 2024). We also provide important parameters for other baselines in Table 8. We note that for (Wang et al., 2025), we select
Target
If you don't like me saying bad words about cops, let me know.

ICV
If you don' to like me to to talk to to to to to to to to to to to to to to to to to to to to to to to to to to to to to to to to

DSEM
If you are uncomfortable with me speaking negatively about police officers, please let me know.

Figure 10: Example of DSEM and ICV generations on Paradetox dataset. From this example, it can be seen that DSEM greatly reduces toxicity, while being relevant to the target sentence. For ICV, the output is often meaningless and repetitive, which, in a way, makes the content non-toxic.

<table>
  <tr>
    <th>Paradetox</th>
    <td>Please act as an impartial and objective judge and evaluate the quality of formality. You should rank the models based on how formal the sentence is, as well as content relevance to reference sentence. The rankings are then output using Python dictionaries and lists. Your evaluation should consider these factors below:
1. Which answer is more formal. The more formal the answer, the better.
2. Which one better preserves the semantic meaning of the reference sentence. The more semantic meaning is preserved, the better.
This is the reference sentence:
{}
This is the formality transfer sentence from model-A:
{}
This is the formality transfer sentence from model-B:
{}
Now, based on the above two criteria, please rank the models. Avoid any positional biases and ensure that the order in which the responses are presented does not influence your decision. Do not favor certain model names. Be as objective as possible. Then, use a list containing the model's name, its rank, and the reason for its ranking to return the results, i.e., please ensure to use the following format to return the results: [({"model": <model-name>, "reason": <rank-reason>, "rank": <model-rank>}), ({"model": <model-name>, "reason": <rank-reason>, "rank": <model-rank>})]
Your answer must STRICTLY follow the format above, and must be a valid Python list of dictionaries to ensure I can directly parse it using Python. Always use double quotes for the keys and values. Do not include any extraneous content! Please provide a ranking that is as accurate as possible and aligns with the intuition of most people.</td>
  </tr>
</table>

Figure 11: Cohere Evaluation Prompt For Paradetox Dataset
GYAFC
Please act as an impartial and objective judge and evaluate the quality of formality. You should rank the models based on how formal the sentence is, as well as content relevance to reference sentence. The rankings are then output using Python dictionaries and lists. Your evaluation should consider these factors below:
1. Which answer is more formal. The more formal the answer, the better.
2. Which one better preserves the semantic meaning of the reference sentence. The more semantic meaning is preserved, the better.
This is the reference sentence:
{}
This is the formality transfer sentence from model-A:
{}
This is the formality transfer sentence from model-B:
{}
Now, based on the above two criteria, please rank the models. Avoid any positional biases and ensure that the order in which the responses are presented does not influence your decision. Do not favor certain model names. Be as objective as possible. Then, use a list containing the model’s name, its rank, and the reason for its ranking to return the results, i.e., please ensure to use the following format to return the results: [({"model": <model-name>, "reason": <rank-reason>, "rank": <model-rank>}), {"model": <model-name>, "reason": <rank-reason>, "rank": <model-rank>}].
Your answer must STRICTLY follow the format above, and must be a valid Python list of dictionaries to ensure I can directly parse it using Python. Always use double quotes for the keys and values. Do not include any extraneous content! Please provide a ranking that is as accurate as possible and aligns with the intuition of most people.

Figure 12: Cohere Evaluation Prompt For GYAFC Datasets

Shakespeare
Please act as an impartial and objective judge and evaluate the quality of the role-playing performance. You should rank the models based on the role characteristics as well as content relevance. The rankings are then output using Python dictionaries and lists. The models below are to play the role of William Shakespeare. Your evaluation should consider these factors below:
1. Which one better preserves the semantic meaning of the reference sentence. The more semantic meaning is preserved, the better.
2. Which one is better at not generating redundant information. The less redundant information is generated, the better.
3. Which one has a more pronounced role-speaking style, and speaks more in line with the reference sentence in terms of the style.
This is the reference sentence:
{}
This is the role-playing sentence from model-A:
{}
This is the role-playing sentence from model-B:
{}
Now, based on the above two criteria, please rank the models. Avoid any positional biases and ensure that the order in which the responses are presented does not influence your decision. Do not favor certain model names. Be as objective as possible. Then, use a list containing the model’s name, its rank, and the reason for its ranking to return the results, i.e., please ensure to use the following format to return the results: [({"model": <model-name>, "reason": <rank-reason>, "rank": <model-rank>}), {"model": <model-name>, "reason": <rank-reason>, "rank": <model-rank>}].
Your answer must STRICTLY follow the format above, and must be a valid Python list of dictionaries to ensure I can directly parse it using Python. Always use double quotes for the keys and values. Do not include any extraneous content! Please provide a ranking that is as accurate as possible and aligns with the intuition of most people.

Figure 13: Cohere Evaluation Prompt For Shakespeare Dataset

Reviewer paper summaries: The paper addresses limitations in traditional In-Context Learning (ICL) and existing activation steering methods for large language models (LLMs), particularly highlighting issues such as inefficient use of context space and coarse-grained control mechanisms. To overcome these challenges, the authors propose Dynamic Steering with Episodic Memory (DSEM), a novel, training-free approach that offers token-level control conditioned on input queries. DSEM uses an episodic key-value memory to associate previously generated tokens with specific steering vectors, dynamically retrieving these vectors via nearest-neighbor mechanisms during inference. This design enables more adaptive and precise control, improving alignment to provided demonstrations. The authors demonstrate the effectiveness of DSEM across multiple alignment tasks, including safety enforcement, style transfer, and role-playing scenarios.

This paper introduces Dynamic Steering with Episodic Memory (DSEM), an activation steering method for controlling LLMs’ outputs. DSEM addresses the limitation of traditional activation steering methods (which mainly focus on sentence-levels steering) by utilizing an episodic memory to perform token-level steering. DSEM further interpolates between the memory-based steering vector and a global sentence-level vector to balance local specificity with global coherence. Experimental results demonstrate DSEM’s effectiveness in safety, style transfer and role-playing tasks.

This paper proposes DSEM (Dynamic Steering with Episodic Memory), a novel training-free activation steering framework for large language models (LLMs). DSEM employs an external episodic memory storing input-output demonstrations and their partial outputs. During inference, it dynamically retrieves memory entries relevant to the current input, computing token-level steering vectors to guide the generation of LLM outputs adaptively. Core contributions include:

(i) Introducing episodic memory retrieval for activation steering, achieving query-dependent, fine-grained control of model generations.

(ii) Proposing an uncertainty-driven interpolation mechanism, balancing localized memory-based adjustments with global semantic coherence.

(iii) Achieving significant improvements on alignment tasks (e.g., detoxification, formality transfer, role-playing) without additional model fine-tuning, outperforming existing methods (traditional ICL, fine-tuning, recent steering methods) across multiple models and scenarios.
Summary of strengths: 1.	This paper is well polished and easy to follow.

2.	Nice method formulation, cool motivation and comprehensive experimental results.

1. This paper presents a novel approach to incorporating token-level information in activation steering, enhancing the precision of LLM-generated responses.
2. The authors have conducted a comprehensive set of experiments to rigorously evaluate the effectiveness of their proposed method.

1. DSEM requires no additional model training or parameter fine-tuning, making it highly practical and straightforward to integrate into existing large models. This "plug-and-play" feature greatly increases its potential real-world application value.
2.The authors evaluated their approach extensively on multiple diverse tasks (toxicity reduction, style transfer, role-playing) and multiple popular open-source LLMs (7B to 9B parameters). Evaluation metrics were comprehensive, including task-specific classifiers, semantic preservation (BERTScore), and large-model-based evaluation for quality comparisons, consistently demonstrating superior performance.
Summary of weaknesses: 1.	A memory consumption complexity analysis would be good for the paper since it involves the usage of memory.

1. The paper lacks a thorough comparison of computational cost (e.g., inference speed, GPU memory usage) across methods, considering the need of DSEM for memory lookups and potentially multiple k-NN searches during the generation of each token chunk.
2. The authors did not provide templates used to format the model inputs, which is critical for prompting-based methods especially in the context of no available demonstrations (“No Context” setting in the paper).
3. As elaborated in Sec. 4.1, DSEM’s performance on different datasets is sensitive to token chunk length l, which can be considered as a hyper-parameter. The work did not offer instructions on how to optimize the parameter. The authors may provide a guideline, which can be insightful in practical applications.

1. The performance heavily relies on the assumption of high-quality, representative demonstrations. Scenarios with noisy or insufficient demonstrations were not thoroughly evaluated or discussed.
2. Some key design choices (e.g., uncertainty weight formulation, applying vectors to all layers, specific hyperparameters) were empirically driven without deeper theoretical justification or extensive comparative analysis.
3.The tasks evaluated mostly focus on stylistic and safety-related outputs, with evaluations mainly conducted via automatic classifiers or models. Human evaluations and performance on other tasks (e.g., QA, summarization) remain unexplored, potentially limiting broader understanding of DSEM's applicability.
Comments, suggestions and typos: 1. This paper is similar to the idea of state eviction methods [1][2][3][4]. The authors might consider adding some discussion about the difference between DSEM and these methods.

2. Some of the format is not very smooth, all the \paragraphs are replaced with \noindent \textbf which doesn’t look very nice I think.

[1] H2o: Heavy-hitter oracle for efficient generative inference of large language models

[2] Model tells you what to discard: Adaptive kv cache compression for llms

[3] On the Efficacy of Eviction Policy for Key-Value Constrained Generative Language Model Inference

[4] CItruS: Chunked Instruction-aware State Eviction for Long Sequence Modeling

In Figure 1, “Generate next m tokens” should be “Generate next l tokens”.

1.Consider adding visualizations or intuitive explanations of how steering vectors influence token generation, attention patterns, or hidden states. This would greatly enhance understanding of the method’s effectiveness.
Overall assessment scores: 3.5

3

2.5