Semantics-Adaptive Activation Intervention for LLMs via Dynamic Steering Vectors
Authors: Weixuan Wang
JINGYUAN YANG
Wei Peng
Venue: ICLR 2025 Poster (2025)
Keywords: Large Language Models
Activation Steering
Dynamic Steering Vector
Primary area: applications to computer vision, audio, language, and other modalities
TL;DR: A novel method that develops a dynamic steering vector to modulate an LLM's activations at inference time

Abstract: Large language models (LLMs) have achieved remarkable performance across many tasks, yet aligning them with desired behaviors remains challenging. Activation intervention has emerged as an effective and economical method to modify the behavior of LLMs. Despite considerable interest in this area, current intervention methods exclusively employ a fixed steering vector to modify model activations, lacking adaptability to diverse input semantics. To address this limitation, we propose Semantics-Adaptive Dynamic Intervention (SADI), a novel method that constructs a dynamic steering vector to intervene model activations at inference time. More specifically, SADI utilizes activation differences in contrastive pairs to precisely identify critical elements of an LLM (i.e., attention heads, hidden states, and neurons) for targeted intervention. During inference, SADI dynamically steers model behavior by scaling element-wise activations based on the directions of input semantics. Experimental results show that SADI outperforms established baselines by substantial margins, improving task performance without training. SADI's cost-effectiveness and generalizability across various LLM backbones and tasks highlight its potential as a versatile alignment technique. We will release the code to foster research in this area.

Published as a conference paper at ICLR 2025
ACTIVATION INTERVENTION FOR LLMS VIA DYNAMIC STEERING VECTORS
SEMANTICS-ADAAN
Weixuan Wang' Jingyuan Yang²
Wei Peng"
School of Informaties, University of Edinburgh
HHuawei Technologies Co., Ltd.
School of Engineering. RMIT University
weixuan. wangêed. ac. yangjiingyuan2ehuuwei.cc com
wei. pei.penneermi au
ABSTRACT
Large language models (LLMs) have achieved remarkable performance across many tasks, yet aligning them with desired behaviors remains challenging. Activa- tion intervention has emerged as an effective and economical method to modify the behavior of LLMs. Despite considerable interest in this area, current intervention methods exclusively employ a fixed steering vector to modify model activations, lacking adaptability diverse input semantics. To address this limitation, we propose Semantics-Adaptive Dynamic Intervention (SADI), a novel method that constructs dynamic steering vector to intervene model activations at inference time. More specifically, SADI utilizes activation differences in contrastive pairs to precisely identify critical elements of an L.LM (i.e., attention heads, hidden states, and neurons) for targeted intervention. During inference, SADI dynamically steers model behavior by scaling element-wiss activations based on the directions input semantics. Experimental results show that SADI outperforms established baselines by substantial margins, improving task performance without training. SADI's cost-effectiveness and generalizability across various LLM backbones and tasks highlight its potential as a versatile alignment technique
INTRODUCTION
Large language models (LLMs) have demonstrated remarkable capabilities across many tasks (Ope. nAI
|AI]|023) 2023; Touvron et al. 2023 Anil el al.] 2023b a Mesnard et al. 2024). Nevertheless, aligning
these models to target behaviors remains challenging (Lnngrre al. 2023 Ding al. 2023). Exist-
ing approaches like supervised fine-tuning (Wei et al. (2022) (SFT). Reinforcement Learning from
Human Feedback (Bai al., 2022) (RLHF), and prompt engineering (Shin et al., 2020 Wang et al.
2023) are effective but have limitations. They often require extensive datasets, struggle to prevent hallucinations, and sometimes fail produce the desired results.
Recently, advancements in model alignment techniques, known as "activation engineering", aim to address these limitations (Subramani et 2022 Hernandez et al. 2023] Zou et al.) 2023 Li et al.)
2023b Chen et al.] 2024). Activation engineering involves making targeted modifications to the internal activations LLMs guide their outputs more precisely. This technique constructs steering vectors that, when integrated into the forward pass of a frozen LLM, induce specific desirable changes in the output text. However, traditional steering vectors are static and may not adapt well to the diverse semantic contexts encountered during inference (Turner et al.] 2023 Rimsky et al.) (023). This misalignment between the direction of steering vector and the input's 's semantic direction can adversely impact the model's 's predictive performance, particularly when the discrepancy is substantial. These limitations highlight the need for dynamic and adaptive steering mechanisms capable of effectively handling varied input semanties.
In this work, we introduce the Demantics-Adapti Dynamic Intervention (SADI), a novel ap- proach designed to overcome the limitations of fixed steering mechanisms. SADI adjusts model activations by dynamically generating a steering vector tailored to each input's semantic context. Specifically, SADI utilizes activation differences from contrastive pairs to create binary mask that
http:://gthhb.ccm//eevaaaawaag A2I/SADI

---

Published as a conference paper at ICLR 2025
Step 1 Difference Extraction
Stop Adaptive Steering
Nag
Pos:
Neg
user Input
LLM
update
activations
CAUDO GUOGE
mask
RgO aggegete
binarization
sagetion
#000
Actinatora postive serivves
Activstone el steringut
Stop 2 Binary Masking
Figure 1: Three steps of SADI: (1) Difference Extraction: extract the activation differences between positive and negative examples from all model layers; (2) Binary Masking: compute the mean activation difference to locate the key elements and produce an identification mask by binarization; and (3) Adaptive Steering: intervene the activations during inference by applying the identification mask to the input activations scaled by factor of 6.
identifies critical model elements for targeted intervention. During inference, this mask is applied to user input activations with element-wise scaling, effectively manipulating the LLM's behavior to align with the input semantics. This process ensures that modifications preserve the semantic alignment of the inputs, allowing for more precise and context-sensitive interventions. Furthermore, we apply SADI to various components of LLMs, including hidden states (S (SADI-HIDDEN), attention heads (SADI-HEAD), and neurons feed-forward networks (FFNs) (SADI-NEURON).
To validate the effectiveness of SADI, we conduct extensive experiments using four diverse model backbones: LLAMA2-7B-CHA'S BLOOMZ-7B MISTRAL-7B. FALCON-7B-ING -NNSSTRCCT across eleven widely used benchmarks. The experiments involve comprehensive range of tasks, from multiple-choice tasks (COPA, StoryCloze, NLI, MMLU, SST2, SST5, BoolQ, Winogrande), to open- ended generation tasks (TriviaQA, ToxiGen, and TruthfulQA). Our experimental results reveal that SADI significantly outperforms existing activation intervention methods.
Our contributions are summarized follows:
We propose a dynamic activation intervention approach named Semantics- Adaptive Dynamic Intervention (SADI), which automatically modulates LLM activations inference time to adapt to varied input semantics without requiring any additional training (see (Sectonn 3).
SADI is generic steering method applicable to wide range of LLMs. Through extensive experiments with four model backbones over eleven diverse tasks, SADI has proven to sig- nificantly enhance model performance, surpassing bascline methods by substantial margins, with accuracy improvements reaching up to +14.69 (see Section 4). Our detailed analysis demonstrates that interventions targeting attention heads (SADI-HEAD) consistently yields significant performance improvements across various tasks, validating the effectiveness of our dynamic steering approach (see (SeetSennnnn 4).
SADI demonstrates excellent generalizability across different model sizes, few-shot settings, and multilingual scenarios (see (Seciinn 5p. We further show that SADI cost-effective steering method that necessitates only a small number of additional examples (i.e., 150 items) in developing a dynamic steering vector and does not require any training (see (Scciinn 6).
2
RELATED WORK
Activation engineering has been proposed as cost-effective method to modify an LLM's activations during decoding (Hernandez et al. 2023 Zou et al. 2023 Wang et al. 2024c). By analyzing
2

---

Published as a conference paper at ICLR 2025
activation differences between contrastive pairs, these methods identify specific directions to adjust LLM behaviors. For instance, some studies have improved LLM truthfulness by shifting activations along vectors between true and false output distributions (Li et al., 2023b] Chen et al. 2024). Moreover, discrepancies in contextual examples have been used to identify crucial modifications needed to reduce L.LM toxicity (Liu al. 2023). More generally, these differences can be used to update the residual stream without requiring explicit direction settings for adjustments. Turner ct al. (2023) construct steering vectors by assessing intermediate activation differences between two prompts, effectively shifting emotions from negative to positive. Similarly, Riimsky et al. (2023) use contrast pairs to create steering vectors that modify model behaviors by adjusting hidden states.
Our problem formulation aligns with the "linear representation hypothesis" (Park et al., [0244] Turner
jet al. 2023) which posits that high-level concepts (i.e., features of the input) are represented linearly as directions in the representation space. Model behavior intervention can be achieved by adding an
appropriate steering vector to the representation of concept without altering other concepts. One key insight is that intervening along the direction of the feature representation of an input is expected to enhance the probability of producing a desirable output. Intuitively, we need to preserve the semantic direction of an input when applying steering vector update activations.
Existing methods use fixed steering vectors generated from additional contrastive pairs during intervention without aligning with imput semantics (Turner et al. [023] Rimsky et al., 2023). As activation patterns can vary significantly across different inputs for the same task (Wang et al. 2024a|b). model steering with fixed vector may cause the intervention direction deviate from the representation of input contexts, This highlights the need for more adaptive approaches. Developing a dynamic steering mechanism adapt the input semantics is critical for effective intervention.
3 SOMANTICCSAAAPTIII DYNAMIC INTERVENTION
In this section, we provide a comprchensive description of the proposed SADI. We begin with an overview of SADI in [Section 3.1 and then introduce each step of SADI, including Difference Extraction (Section 3.2). Binary Masking (Section 3.3), and Adaptive Steering (Section 3.4).
3.1 OVERVIEW OF SADI
Our method, SADI, encompasses three pivotal steps to dynamically steer model behavior, as shown in Figure I and Algorithm First, the activation differences between positive and negative examples extracted across all layers of the model. These differences aggregated compute the mean difference, which is used to identify critical elements influencing the model's 's behavior. Based on this computation, we create identification mask through binarization, keeping the crucial elements while masking out the insignificant ones. Furthermore, this mask is applied to the activations of user inputs, scaled by a factor during inference. In this way, we manage to manipulate the behaviors of LLMs. We present more details in the following sections.
3.2 DIFFERENCE EXTRACTION
In the initial step of SADI, our objective extract activation differences from contrastive pairs to isolate the internal activations most closely associated with the target behavior of the language model. Specifically, we aim to identify features that distinguish positive outcomes (e.g., correct answers) from negative ones (e.g., incorrect answers), thereby allowing adjustments to the model's behavior.
Let (P' |0 < < L} represents an LLM consisting of L layers, whose behavior we seek to modify. We build a dataset T' = {( v. yı J.W containing N instances, where each instance includes the question F4 a positive output y and a negative output y, neg For each instance and layer (I), obtain activations by forwarding the concatenation of the input and the corresponding output through the model P. Specifically, we forward x; concatenated with yf" to derive the positive
activation (los.(1) pi (x.llle ,,,, where -||- denotes concatenation and j represents the j-th token
in the sequence. Similarly, we obtain A (!) for the negative activation. Additionally, we focus on the activation of the last token in each sequence, it typically encapsulates the complete semantics
3

---

Published as a conference paper at ICLR 2025
of the input-output pair. For simplicity, denote the activation of the last token A A.or.) and
A peg.(t) for the positive and negative outputs, respectively.
To identify the features within the model that differentiate correct from incorrect outputs, we compute the difference between the positive and negative activations for each instance at each layer as follows:
D A (000
ncg.(1)
By examining differences D. we can determine which activations are crucial for model's behavior.
3.3 BINARY MASKING
We now present how we construct mask to identify and focus interventions on the critical model elements that affect model's behavior. As shown in Figure (Step 2), after extracting the activation differences from contrastive pairs, compute the mean difference across all instances and layers, and concatenate them to build the overall mean difference vector D for all model elements:
1 N
D Concat( (D(") D()) (i,,,,,,,DL.))))
D(L-!)), where D
D!
N
Here, D E Lxdm represents the concatenated mean activation differences across all layers, and dm denotes the dimensionality of the model components, which may correspond to hidden states, attention heads, or neurons in FFNs.
We then binarize the mean activation difference D to create an identification mask M E Rixdd.. This is done by setting the entries corresponding to the top-K elements with the largest differences to 1 and the rest 0:
M(1, m] = {
(1, E Ex otherwise
where indexes the layers, indexes the model elements within layer, and Ex is the set of indices of the top-K elements with the highest mean activation differences. This step ensures that SADI focuses the most impactful elements contributing the desired behavior, reducing unnecessary alterations to non-essential elements and enhancing the efficiency of the intervention.
3.4 ADAPTIVE STEERING
Previous activation intervention methods modify all activation elements indiscriminately, which can disrupt the model's overall behavior (Li et al. 2023b Rimsky et al. 2023). To mitigate this issue, we perform focused intervention on the top-K elements during inference. By leaving irrelevant activations intact, our intervention becomes less intrusive and preserves the model's non-target behaviors. With this design, SADI precisely adjusts activations to minimize disruption to the model's residual functionalities. This approach is visualized in Step 3 of Figure
Unlike previous studies using the fixed steering vector to intervene models, and inspired by Park al. 2024), we design a steering mechanism that considers the semantic direction of the input. The steering vector dynamically adapts to the input's semantic direction (see Equation 5), maintaining the effectiveness of the intervention without deviating from the intended semantics.
For given user input q, we first extract the activations of the last token from each layer: A for 1= 0, 1,.., L 1. We then concatenate these activations to form a single vector:
A₂ Concat (oncat(AA0))
1))
(4)
Next, we apply the identification mask M to these activations and update them using:
A = A, + 6(A, @ M).
(5)
Here, C represents the element-wise product, and is a hyperparameter controlling the strength of the intervention along the input's semantic direction. By calculating the steering vector based on the activations of input q. the intervention dynamically aligns with the input's 's semantics. This approach maintains the direction of the activation projections, ensuring that the intervention remains semantically relevant and effective. Subsequently, we complete the altered forward pass with the updated activations A'-
4

---

Published as a conference paper at ICLR 2025
Hyperparameters K and 8 Our method introduces two key hyperparameters: K N+ specifying the number of top elements targeted during the intervention, and E R controlling the strength of the intervention. We perform hyperparameter sweep to empirically determine their optimal values. Detailed analysis of the hyperparameter selection is provided in Section 4.3
By selectively targeting the most impactful activation elements and adapting our intervention to the input's semantic direction, our method effectively steers the model toward the desired behavior.
Algorithm 1: SADI: Semantics- Adaptive Dynamic Intervention
Input : =T'= {xx ) a set contrastive pairs; U (u, test set; P, a pre-trained LLM; A, a function to extract activations from P;
Output:00 :0, the modified oulputs collection;
AA
// Initialize the difference Collect and compute the mean difference activations AA AA + (A(P(=.")) - A(P(x))))
2 for i=/ to do
end for
5 AA + AAA
.M binarize AA)
// Create the identification mask // Intervene generation
7 <-1
8 for j= to K do
A, + A(P(u;))
// Extract activations for each input // Apply mask to activations and update
S, M @ A,
11
A, + + S O O-OU(P(4));
12
Complete the modifiet forward pass
13 end for
return O
EXPERIMENTS
We present our experimental setup (Section 4. (ection 4.3), and specific contributions of SADI's constitutes (Section 4.4) in this section.
comparative methods (Section 4.2), main results
4.1 EXPERIMENT SETTINGS
In this subsection, describe the experimental settings for evaluating SADI. First, we outline the tasks and the evaluation metrics used to assess model performance. Following that, we detail the construction of contrastive pairs. Finally, we introduce the selected LLMs.
Tasks and Evaluation Metrics We conduct experiments on the following two types of tasks: multiple-choice tasks and open-ended generation tasks. For the multiple-choice tasks, we use datasets:
COPA (ordon et al.] (012), StoryCloze (Mostafazadeh et al., 2016). NLI (Bowman et al.) 2015). MMLU Hendrycks et al. 2021). SST2 (Socher et al. 2013). SST5 (Socher et al.) (013)), BoolQ (Clark ct al. 2019), and Winogrande (Sakaguchi et al. 2020), with response formats ranging from 2-way to 5-way choices. Detailed descriptions of the datasets are provided in Appendix A.1 We measure performance across these tasks using accuracy.
For the open-ended generation tasks, we apply SADI on TriviaçA (Joshi et al. 2017).
TruthfulQA (Lin et al., 2022), ToxiGen (Hartvigsen al. 2022) datasets. The Exact Match (EM)
metric assesses TriviaQA, while TruthfulQA is evaluated using multiple-choice accuracy (MC)
and pre-trained judge models for truthfulness? and informativenes ToxiGen is evaluated with
HATEBERT classifie to measure toxicity. Dataset sizes are detailed in Table 9 (Appendix A.2
Contrastive Pairs Construction For multiple-choice tasks, we generate positive prompts by concatenating questions with correct answers and generate negative prompts using a randomly chosen incorrect answer. For TriviaQA, a unique approach involves using a blank space as the
"bttss//hugginngfcccco hhtps///ttngiggfffsoooolllaatttu htwp:///ww.....l..o.oa.oooooomia jug-thhhiudg--llaaa2-
bhttss//hugginnffcc... https://haggiigaaacc.alllllaarri htw.:ruuuuuggi-oo-iiiglllllaaaaa http://huuggnngaaccccooooom//e/e hatchert
5

---

Published as a conference paper at ICLR 2025
Table 1: The overall results of seven multiple-choice tasks in a zero-shot setting, performed by LLAMA2-7B A-MA--7--CHAT. "SFT SADI" indicates that SADI is applied to instruction fine-tuned models. dash indicates that the training dataset unavailable.
Task
COPA
StoryCloze
NLI
MMLU
SST2
BoolQ
Winogrande
AVG
BASELINE
70.80
65.06
63.11
44.90
88.63
70.52
50.91
64,85
ITI
77.20
68.50
63.97
46.07
91.38
74.10
52.80
67.72
CAA
75.20
74.65
64.13
46.17
91.16
74.98
52.64
68.42
SADI
SADI-HIDDEN
81.00
55.99
59.28
45.66
92.15
76.25
52.64
66.14
SADI-NEURON
82.20
62.97
46.91
88.69
70.40
51.93
67.24
SADI-HEAD
78.80
79.75
64.21
48.23
92.20
74.35
53.04
70.08
SFT
93.20
96.49
90.07
96.70
88.75
78.37
90.59
SFT SADI
94.90
96.49
90.07
96.76
88.91
78.37
90.91
94.80
96.55
90.36
96.92
88.45
78.45
90.92
SADI-HEAD
94.60
96.55
90.30
96.81
88.94
78.61
90.97
incorrect answer. In TruthfulQA, we utilize data from its multiple-choice format to identify crucial elements. For the ToxiGen task, we leverage the RealToxici Toi.TootroooPoopttttcc dataset (Gehman et al. 2020), selecting entries with toxicity score exceeding 0.955 to serve as negative prompts. This helps pinpoint elements contributing to toxic outputs.
Target LLMs We evaluate the performance of SADI in enhancing the baseline model (BASELINE) an instruction-tunel LLM, L.LAMA2-7--CHA (Tourron et al. (023). To verify the generalizability of SADI across various model backbones, include three additional LLMs: BLOOMZ-77 (Muen- nighofff et al. (023). MISTRAL-7B (Jiang et al.) 2023), FALCON-7B-INSTE 7B-INSTRUCT (Almazrouei et al. 2023). These models are selected based on their demonstrated efficacy across diverse linguistic lasks and their widespread use in the research community. Furthermore, we extend our experiments other models within the BLOOMZ family, specifically BLOOMZ-560 560M, BLOOMZ- 1B, and BLOOMZ-3B. 3B, exploring how SADI performs across different model sizes.
4.2 EXPERIMENTAL. COMPARISONS
In addition evaluating SADI, we compare it against several approaches:
Supervised fine-tuning (SFT) We finetune all model parameters using the training dataset for each task, as previous works suggest that this approach serves as upper bound for supervised finetuning, Specifically, we employ the Adam W optimizer with learning rate of 2 x 10 and a batch size of 4, conducting the fine-tuning across three epochs on four NVIDIA A-100 GPUs (80G).
Inference-Timee Intervention (ITI) We follow Li et al. (2023b) in using contrastive pairs to identify the top heads for intervention. We sweep the hyperparameters the heads involved and the strength of intervention to optimize results.
Contrastive Activation Addition (CAA) Rimsky et al. (2023) use the mean difference in the model's is activations at the position of the answer letter between all the positive and negative prompts to construct a fixed steering vector to shift activations.
Our Approach (SADI) We compare three different configurations of the SADI shift. SADI- HIDDEN applies SADI to the identified key hidden states across all layers. SADI-HEAD modifies activations from the outputs of all attention heads across all layers. SADI-NEURON is based on the outputs from each non-linear activation function the FEN blocks across all layers.
4.3 EXPERIMENTAL RESULTS
SADI Significantly improves multiple-choice task performance. As illustrated in Table SADI demonstrates superior performance compared to the BASELINE and other intervention methods across multiple-choice tasks. While SFT consistently outperforms other methods in tasks with available training data, such as COPA and StoryCloze, its high data resource demands limit its effectiveness in tasks with scarce data, like MMLU. Both ITT and CAA show notable improvements, highlighting

---

Published as a conference paper at ICLR 2025
Table The overall results of three open-ended generation tasks performed by LLAMA2-7B-CHAN The results of TriviaQA are obtained in a zero-shot setting, and we use 5-shot in-context learning in the ToxiGen and TruthfulQA tasks. denotes results reproduced from other authors.
Task
TriviaQA
ToxiGen
TruthfulQA
Metric
EM
toxicity
True
Info
Truex Info
MC1
MC2
MC3
BASELINE
41.60
49.71
66.83
99.51
66.50
33.41
51.07
24.76
SFT
24.20
42.80
45.27
34.647
51,551
25.32
CAA
43.20
49.71
71.60
83.84
60.03
34.03
52.76
25.62
SADI
SADI-HIDDEN
43.80
34,43
35.13
51.73
25.38
67.07
92.90
62.31
SADI-NEURON
43.50
17.14
74.54
93.51
69.71
34.88
52.50
25.79
SADI-HEAD
44.00
34.50
77.72
98.53
76.58
35,90
54.65
26.99
the effectiveness of intervention- based methods. SADI, leveraging dynamic interventions, achieves optimal performance overall, significantly outperforming the BASELINE. In comparison to fixed vector-based intervention methods (ITI and CAA), our dynamic intervention SADI yields substantial gains across all tasks. Specifically, SADI exceeds ITI and CAA by margins of +11.25 and +5.10, respectively, and surpasses the BASELINE by a large margin of +14.69 in the StoryCloze task. Furthermore, SADI can enhance the performance of task-specific fine-tuned models (SFT +SADI), underscoring its practicality for precise and targeted interventions.
Improvement varies across SADI con- figurations. Performance gains from the three SADI configurations vary, but all outperform ITI and CAA across tasks. Notably, SADI-HEAD achieves the most significant improvements, en- hancing average accuracy by up to +5,23. While SADI-HIDDEN and SADI-NEURON also demonstrate strong performance improvements in certain scenarios, such 76.25 for BoolQ and 82.20 for COPA, they occasionally show slight decreases in specific tasks, like NLI. Nevertheless, manipulating attention heads consistently results in improvements across all tasks and achieves the highest scores in most cases. For detailed analysis, we present the results covering various domains of knowledge of MMLU tasks, including humanities, STEM and social sciences and other are shown in Table 2 SADI consistently enhances performances across these domains compared to the BASELINE, with SADI-HHAAA yielding the highest improvements, up to +5.09 in the STEM domain.
Table 2: Average weighted accuracy all four broad disciplines for MMLU task with LLAMA2-7B-CHAT
Domain
Humanities
STEM
Social
Other
BASELINE SADI
46.68
34.26
54.62
49.71
SADI-HIDDEN SADI-NEURON
48.99
35.03
55.82
48.51
48.83
37.46
55.74
50.65
SADI-HEAD
49.76
39.35
56.21
52.31
SADI improves open-ended generation task performance. We further evaluate the performance of SADI the open-ended generation tasks in Table SADI, in its three configurations, generally outperforms the BASELINE, except for SADI- HADI-HIDDEN in the TruthfulQA generation track. SADI- HIDDEN underperforms in the generation track, but shows significant improvements in the truthful multiple-choice track. This suggests that hidden states may be particularly sensitive to multiple-choice formats. Conversely, SADI-HEAD significantly boosts truthfulness, with improvements reaching up to ++++0 +10.08 on the True Info metric for TruthfulQA. This underscores the generalizability of SADI 's dynamic intervention, effectively tailoring activations to the semanties required by each task.
we sweep two hyperparameters to control the intervention: the number of identified key attention heads, and the strength of intervention. Results indicate that optimal settings for these hyperparameters markedly vary across different tasks. This variability underscores the importance of carefully balancing the number of heads engaged and the scale of their amplification. For precise task performance optimization, it is recommended to search optimal hyperparameters using data from the validation sets specific to each task.
Hyperparameters are task-specific. In Figure 2.
7

---

Published as a conference paper at ICLR 2025
SteryCloze
TriviaQA
number of heads
11.000
0.80
8.00
37.40
12.71
10.32
6443
07.43
0.80
8.00
40.00
26.10
29.00
26.00
$1.40
72.07
77.76
99.80
76.75
76.35
62.20
88.16
28.15
25.10
26.
22.00
19.30
15.20
(5.20-25.60)
31.00
23.23
61.09
66.38
72.00
73.46
91.16
61.10
17.19
5.50
0.20
0.00
0.00
strength
Figure 2: Results with varying intervention strength and numbers key attention heads based on COPA, StoryCloze, SST2, TriviaQA tasks with LLAMA2-7B-CHAT.
4.4 SCRUTINIZE EFFECTS OF DYNAMIC INTERVENTION OF SADI
SADI outperforms fixed steering and random intervention. Table we conduct an ablation study to examine the contribution of Binary Masking (Step in Section 3.3) and Adaptive Steer- ing (Step [Section 3.4) in SADI applied tasks COPA, StoryCloze, Winogrande, SST2, and TriviaQA. SADI involves constructing an identification mask M, where we randomly assign K ele- ments to with the values of remainder elements set to (termed "random identify" in Table 4). can be observed from Table 4 that random element identification leads to notable performance decrease, with reductions as great as -7.88 (dropping from 79.75 to 71.87 on StoryCloze). Subsequently, the mask M is applied to the activations corresponding to the semantics of user inputs in SADI. Additionally, we explore the effects of fixed steering, in which the mask M is directly applied to the mean difference D, derived from the activations of contrastive pairs (see Eq. 2) If Step 3 (Eq. (5) employs fixed steering, it updates activations as:
Table 4: Ablation study for randomly iden- tifying key elements and intervening with a fixed steering vector for SADI-HEAD with LLAMA2-7B-CII
Ar- = 6 M).
(6)
random identify
fixed
The results in Table 4 show that using fixed steer- ing vector leads to sigmificant performance degradation compared to the remantic-adaptive approach of SADI. This decline likely stems from a misalignment between the direction of intervention and that of input semantics.
SADI
steering
COPA
78.80
72.40
71.20
StoryCloze
79.75
71.87
69.42
Winogrande
53.04
52.88
$2.56
SST2
92.20
91.48
91.43
TriviaQA
44.00
42.90
41,50
DISCUSSION
In this section, examine the generalizability across multiple LLMs (Section 5.1). different model
sizes (Section 5.2), few-shot settings (Section 5.3), and multilingual scenarios (Section 5.4).
5.1
GENERALIZABILITY ACROSS MULTIPLE LLMS
Table 5: Generalizability evaluation of SADI given by BLOOMZ-7P, MISTRAI- and FALCON- 7B INSTUNTTTTT on the COPA, BoolQ and NLI tasks.
Task
COPA
BoolQ
NLI
BLOOMZ
MISTRAL
FALCON
MISTRAL
FALCON
BLOOMZ
MISTRAL
BASELINE
76.40
84,30
62.20
91.28
71.80
71.83
54.81
53.33
$2.29
SET
86.80
86.80
88.20
90.52
86.57
84.22
$7.41
89.94
55.45
SADI
HIDD-HIDDEN
76.40
93.00
60.60
91.31
49.43
62.73
$3.25
35.69
43.67
SADI-NEURON
56.43
52.48
SADI HEED
82.20
92.00
62.20
91.28
67.38
73.23
55.37
53.65
54,83
SADI enhances performance across multiple LLMs in various tasks. An important question is whether SADI can generalize over various I.L.Ms. We apply SADI to three other well-performing
8

---

Published as a conference paper at ICLR 2025
LLMs: BLOOMZ-78 MISTRAL- 7B. and FALCON- BALCON-7B-INSTER 7B-INSTRUCT in COPA, BoolQ and NLI tasks. According to the results in Table 5| SADI consistently enhances performance across tasks compared to the BASELINE, despite varying improvement levels by configuration. It is noteworthy that the SADINEEURON configuration with BI.00MZ-77 achieves the most substantial performance gains in all three tasks, demonstrating that different models may exhibit distinct functional elements.
5.2 GENERALIZABILITY ACROSS MODEL SIZES
SADI outperforms SFT smaller LLMs. We further investigate the effectiveness of SADI the BLOOMZ series across various model sizes. As shown in Table 6 SADI maintains its improvement over BASE- LINE even with smaller LLM sizes, confirming its gen- eralizability across model sizes. Notably, SADI out- performs SFT with incremental gains of up to +1.2 for BL.00MZ-11 and +2.4 for BLOOMZ--56MM, high- lighting its effectivvness, especially in smaller models.
Table 6: Generalizabillty evaluation on BLOOMZ series in COPA task.
Size
7b
1.1b
560m
BASBLINE
70.8
79.2
49.8
50.0
SET
88.8
85.8
50.6
52.0
SADI
SADI-HIDDEN SADI-NEURON
76.4
79.2
50.0
50.0
74.0
79.2
50.0
52.6
SADI-HEAD
78.8
79.2
51.8
54.4
5.3 GENERALIZABILITY IN FEW-SHOT SETTINGS
SADI improves few-shot performance but with less gains. In Table 7 we compare SADI-HEAD to the BASELINE across zero-shot and few-shot settings on the SST5, Wi Winogran nogrande, and TruthfulQA tasks. The results highlight the generalizability of SADI in enhancing model performance with few-shot prompting across various tasks. While manipulating heads improves the performances in few-shot settings, the gains are less pronounced compared to those in zero-shot settings. This suggests that few-shot examples already provide strong learning signal (to both BASELINE and SADI), somewhat overshadowing the additional benefits derived from head manipulation.
Table 7: Comparisons between few-shot and zero-shot on the SST5, Winogrande and TruthfulQA.
Task
SST5
Winogrande
TruthfulQA MC1
TruthfulQA MC2
TruthfulQA MC3
Configuration
0-shot
5-shot
0-shot
5-shot
0-shot
5-shot
0-shot
5-shot
0-shot
5-shot
BASELINE
28.24
53.21
50.91
52.01
27.66
33.41
44.45
51.07
20.59
24.76
HEAI-HEAD
35.43
54.07
53.04
53.35
32.19
35.99
50.81
54.65
24.83
26.99
5.4
GENERALIZABILITY IN MULTILINGUAL SCENARIOS
Table 8: Evaluating SADI on multilingual task XCOPA with LLAMA2-7B-CHAN
Language
SW
ta
th
tr
vi
zh
AVG
RASELINE
51,40
61.20
50.20
49.40
50.80
49.40
51.80
62.80
53.38
SADI
SADI-HIDDEN
51.40
62.40
50.00
50.00
51.20
48.80
52.20
64,80
53.85
BADI-NBURON
63.60
68.80
50.20
48.80
53.80
50.60
60.40
70.40
58.33
SADI-HEAD
62.60
70.60
50.80
49.60
51.40
51.60
60.20
70.10
58.36
SADI enhances performance in multilingual scenarios. Although our primary experiments are in English, extending SADI to multilingual scenarios reveals its broader applicability. We further evaluate SADI on the multilingual XCOPA task (Ponti et al.) 2020), covering eight languages: Indonesian (id), Italian (it), Swahili (sw), Tamil (ta), Thai (th), Turkish (tr), Vietnamese (vi), Chinese (zh). Table illlustrates varying degrees of performance enhancements across different languages. can be observed that Indonesian shows the highest improvement, while Swahili gains the least. Despite these variations, SADI consistently boosts performance across diverse language settings and configurations. detailed analysis of the key components effective language-wil -iita is provided in Appendix A.3 Additional insights from cross-lingual evaluations discussed in Appendix A.4

---

Published as a conference paper at ICLR 2025
6 ANALYSIS
In this section, we analyze activation difference distribution patterns for key model elements Sec tion 6.TD. and how SADI behaves under varying numbers of contrastive pairs (Section 6.2).
6.1 CHARACTERISTICS OF ACTIVATION DIFFERENCE
hidden
50
40
heads (sorted)
20-
SFT HADI-HIDOCM
(a) Difference for heads across layers. Darker blue represents big- ger difference.
(b) Distribution of top- 100 acti- vation difference neurons and hidden states. The y-axis repre- sents the number of neurons or hidden states in top-100.
SADI-HEAD
150 250 350 450 550 e50 750 850 950 amount data
Figure 4: Relationship between accuracy and the amount of contrastive pairs.
Figure 3: Activation difference of each head across layers and the distribution top- 100 activation difference of neurons and hidden states with LLAMA2-1 7B- B-CHAT in StoryCloze.
Figure 3(a) reveals activation patterns of attention heads in the middle to latter layers for contrastive pairs in StoryCloze, indicating primary information processing within these layers. Figure 3(b) shows that activation differences in neuron activity and hidden states are concentrated in the latter layers, with the most significant discrepancies observed the final layer. These consistent patterns across tasks (see Appendix A.5) lend support the functional segregation hypothesis, which posits that latter layers are associated with language generation and middle layers are responsible for reasoning (Zhao al. 2024). Given this, our interventions on attention heads likely influence both reasoning and generation, contributing to the consistent improvements.
6.2 SADI AND SFT WITH VARYING DATA
We assess the impact of varying amounts of contrastive pairs on SADI and SFT in COPA task. As shown in Figure 4 SFT performance improves with an increasing number of fine-tuning data. In contrast, SADI achieves optimal results with significantly fewer pairs, e-g., only 150 items are sufficient to calculate an identification mask for targeting critical heads for intervention. This highlights S SADI's effectiveness and efficiency low-resource conditions.
7 CONCLUSION
In this study, we propose Semantics-Adaptic Dynamic Intervention (SADI), novel approach designed to dynamically steer model behavior by adapting to the semantic contexts of inputs. SADI enhances model adaptability by modulating the activations of the identified critical model elements during inference, taking into account the directions of input semantics. Our extensive experiments across various tasks, LLMs backbones, and languages settings have demonstrated that SADI signif- icantly outperforms established baselines, offering generalizable improvements without requiring additional training. Our study advances the field of "activation engineering" in LLMs, with the potential to inform the development of more advanced L.M intervention techniques.
10

---

Published as a conference paper at ICLR 2025
ETHICS STATEMENT
This work presents comaantics-Adapiiin Dynamic Intervention (SADD, method designed to enhance the performance of large language models (LLMs) by dynamically adjusting their activations without additional training. We have conducted extensive experiments across diverse tasks to evaluate SADI's effectiveness, but we recognize that biases in the underlying models and datasets may still affect outcomes. We encourage practitioners to use SADI responsibly, with careful consideration of fairness, accountability, and transparency. No human subjects were involved in this research, and all experiments conducted using publicly available models and datasets, adhering to their respective licenses and use policies.
REPRODUCIBILITY STATEMENT
We are committed to ensuring the reproducibility of our findings in this work. To facilitate this, provide comprehensive details of our proposed Semantics-Adaptive Dynamic Intervention (SADI) method in /Section 3f the main paper, including the algorithms and mechanisms for dynamic steering vector generation and application to different model components. The experimental setups, including model configurations, datasets, and evaluation metrics, are thoroughly described in (Section We utilize publicly available models: SLLAMA2-7B-CHHEE TB-INSTRUCT, and detail any modifications specific settings used during experimentation. All datasets employed in our evaluation are standard benchmarks, and we include references and links to these resources for accessibility. To further support reproducibility, will release our code used for experiments upon publication, enabling other researchers to replicate our results and extend our work.
BLOOMZ-7B, MISTRAL. and FALCON-
REFERENCES
Ebtesam Almazrouei, Hamza Alobeidli, Abdulaziz Alshamsi. Alessandro Cappelli, Ruxandra Co- jocaru, Mérouane Debbah, Étienne Goffinet, Daniel Hesslow, Julien Launay, Quentin Malartic, Daniele Mazzotta, Badreddine Noune, Baptiste Pannier, and Guilherme Penedo. The falcon series of open language models. CoRR, abs/2311. 2023. doi: 10.48550/ARXIV 23311.16867, URL
https://doi.org/1/ wttp:////.10188//a/aaa/// 20w.:440/0//./..11...... 16861
Rohan Anil, Sebastian Borgeaud, Yonghui Wu, Jean- Baptisstt Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M. Dai, Anja Hauth, Katie Millican, David Silver, Slav Petrov, Melvin Johnson, Joannis Antonoglou, Julian Schrittwieser, Amelia Glaese, Jilin Chen, Emily Pitler, Timothy P. Lillicrap, Angeliki Lazaridou, Orhan Firat, James Molloy, Michael Isard, Paul Ronald Barham, Tom Hennigan, Benjamin Lee, Fabio Viola, Malcolm Reynolds, Yuanzhong Xu, Ryan Doherty, Eli Collins, Clemens Meyer. Eliza Rutherford, Erica Moreira, Kareem Ayoub, Megha Goel, George Tucker, Enrique Piqueras, Maxim Krikun, lain Barr, Nikolay Savinov, Ivo Danihelka, Becca Roelofs, Anaïs White, Anders Andreassen, Tamara Glehn, Lakshman Yagati, Mehran Kazemi, Lucas Gonzalez, Misha Khalman, Jakub Sygnowski, and et al. Gemini: A family of highly capable multimodal models. CoRR, abs/2312.1 105, 2023a. doi: 10.48550/ARXIV 22222. 1122.18811 1105. www.oio.org///e http:://66..0rg11.88588855/ 201
URL htttss://ooi.rrg
Rohan Anil, Andrew M. Dai, Orhan Firat, Melvin Johnson, Dmitry Lepikhin, Alexandre Passos, Siamak Shakeri, Emanuel Taropa, Paige Bailey, Zhifeng Chen, Eric Chu, Jonathan H. Clark, Laurent El Shafey, Yanping Huang, Kathy Meier-Hellstemi Gaurav Mishra, Erica Moreira, Mark Omernick, Kevin Robinson, Sebastian Ruder, Yi Tay, Kefan Xiao, Yuanzhong Xu, Yujing Zhang, Gustavo Hernández Ábrego, Junwhan Ahn, Jacob Austin, Paul Barham, Jan A. Botha, James Bradbury, Siddhartha Brahma, Kevin Brooks, Michele Catasta, Yong Cheng, Colin Cherry, Christo- pher A. Choquette-CCheeee -Cho,, Aakanksha Chowdhery, Clément Crepy, Shachi Dave, Mostafa Dehghani, Sunipa Dev, Jacob Devlin, Mark Díaz, Nan Du, Ethan Dyer, Vladimir Feinberg, Fangxiaoyu Feng, Vlad Fienber. Markus Freitag, Xavier Garcia, Sebastian Gehrmann, Lucas Gonzalez, and et al. Palm 2 technical report. CoRR, abs/2305, 10403, 2023b. doi: 10.48550/ARXIV 220.. 10403. URL htt.://rr./..455///1i......33
Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova DasSarma, Dawn Drain, Stanislay Fort, Deep Ganguli, Tom Henighan, Nicholas Joseph, Saurav Kadavath, Jackson

---

Published as a conference paper at ICLR 2025
Kernion, Tom Conerly, Sheer EI Showk, Nelson Elhage. Zac Hatfield-Dodde Danny Hernandez, Tristan Hume, Scott Johnston, Shauna Kravec, Liane Lovitt, Neel Nanda, Catherine Olsson, Dario Amodei, Tom B. Brown, Jack Clark, Sam McCandlish, Chris Olah, Benjamin Mann, and Jared Kaplan. Training a helpful and harmless assistant with reinforcement learning from human feedback. CoRR, abs/2204.05862, 2022. doi: 10.48550/ARXIV 2204.05862.. URL
http://66l.orr/10.48555///2
Samuel R. Bowman, Gabor Angeli, Christopher Potts, and Christopher D. Manning. large annotated corpus for learning natural language inference. In Lluís Mårquez, Chris Callison-Burees Jian Su, Daniele Pighin, and Yuval Marton (eds.), Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, EMNLP 2015, Lisbon, Portugal, September 17-21, 2015, pp. 632-642. The Association for Computational Linguistics, 2015. doi: 10. 10.18653/V/DD1--1100 URL https: 10tp1//00000333111111111111
Zhongzhi Chen, Xingwu Sun, Xianfeng Jiao, Fengzong Lian, Zhanhui Kang, Di Wang, and Chengzhong Xu. Truth forest: Toward sulti-scale truthfulness in large language models through intervention without tuning. In Michael J. Wooldridge, Jennifer G. Dy, and Sriraam Natarajan (eds.), Thirty-Eighth AAAI Conference on Artificial Intelligence, AAAI 2024, Thirty-Sixth Con- ference Innovative Applications of Artificial Intelligence, IAAI 2024, Fourteenth Symposium on Educational Advances in Artificial Intelligence, EAAI 2014, February 20-27, 2024, Vancou- ver, Canada, pp. 20967 20974. AAAI Press, 2024. doi: 10.1609/AAAL 1381:AAA....111110.30.090 URL https://doi.org: 10. 609 http://aaaavvvvvia....
Christopher Clark, Kenton Lee, Ming-Wa Wei Chang, Tom Kwiatkowski, Michael Collins, and Kristina Toutanova. Boolq: Exploring the surprising difficulty of natural yes/no questions. In Jill Burstein, Christy Doran, and Thamar Solorio (eds.), Proceedings of the 2019 Conference of the North Amer- ican Chapter of the Association for Computational Linguisties: Human Language Technologies, NAACL-HLL HLT 2019, Minneapolis, MN, USA, June 2-7, 2019, Volume I (Long and Short Papers), pp. 2924 Association for Computational Linguistics, 2019. doi: 10.18653/V 1NN19-1300 URL https://doi.org/ http://....rrr111111666///////1/ 18653/v1/n19--388
Ning Ding, Yulin Chen, Bokai Xu, Yujia Qin, Shengding Hu, Zhiyuan Liu, Maosong Sun, and Bowen Zhou. Enhancing chat language models by scaling high-quality instructional conversations. In Houda Bouamor, Juan Pino, and Kalika Bali (eds.). Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing. EMNLP 2023. Singapore, December 6- 10. 2023, pp. 2023.EMNIP MAPN. 183.. http::htts///dooorrga 01111/.11155311111 www.::eeee..ppmm..mmmm.mmmmmmm.. 1-mpann...8.
Association for Computational Linguistics, 2023. doi: 10.18653/V1/ emn
Samuel Gehman, Suchin Gururangan, Maarten Sap, Yejin Choi, and Noah A. Smith. Realtoxici- typrompts: Evaluating neural toxic degeneration in language models. In Trevor Cohn, Yulan He, and Yang Liu (eds.), Findings of the Association for Computational Linguistics: EMNLP 2020, Online Event, 16-20 November 2020, volume EMNLP 2020 of Findings of ACL, pp. 3356-3369. As- sociation for Computational Linguistics, 2020. doi: 10. 12663//1/0220FFIIIIIN BENNGGS--MNPP.3300 www.:/ffiiiiinnnnnnienee
URL http:ttp:://do..og/1111165333
Andrew Gordon, Zornitsa Kozareva, and Melissa Roemmele. SemEval-2012 task 7: Choice of plausible alternatives: An evaluation of commonsense causal reasoning. In Eneko Agirre. Johan Bos, Mona Diab, Suresh Manandhar, Yuval Marton, and Deniz Yuret (eds.), *SEM 2012: The First Joint Conference on Lexical and Computational Semantics Volume 1: Proceedings of the main conference and the shared task, and Volume 2: Proceedings of the Sixth International Workshop on Semantic Evaluation (SemEval 2012), pp. 394-398, Montréal, Canada, 7-8 June 2012. Association for Computational Linguisties. URL htttt:::///ccaanholoo..rrr
Thomas Hartvigsen, Saadia Gabriel, Hamid Palangi. Maarten Sap, Dipankar Ray, and Ece Kamar. Toxigen: A large-scale machine-generalee dataset for adversarial and implicit hate speech detection. In Smaranda Muresan, Preslav Nakov, and Aline Villavicencio (eds.), Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), ACL 2022, Dublin, Ireland, May 22-27, 2022, pp. 3309-3326. Association for Computational 234, URL https: //tp://oioooo .org/ hrgg/10.186533
Linguistics, 2022. doi: 10. v1/2022.acl-1ong.225
12

---

Published as a conference paper at ICLR 2025
Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. Measuring massive multitask language understanding. In 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021. OpenReview.net, 2021. http::/ss////peeeer www.eeeeneeeoooomiiiieeeeee mGmm
Evan Hernandez, Belinda Z. Li, and Jacob Andreas. Inspecting and editing knowledge representations in language models. arXiv preprini arXiv:2304.00740, 2023.
Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de Las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, Lélio Renard Lavaud, Marie-Anee Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas Wang. Timothée Lacroix, and William EI Sayed. Mistral 7b. CoRR, https///11..022.... 2023. doi: 10.48550/ARXIV.22 23 10.06825. URL lttpps://dii.org w0w.44455//....2211.....o
Mandar Joshi, Eunsol Choi, Daniel Weld, and Luke Zettlemoyer. TriviaQA: large scale distantly su- pervised challenge dataset for reading comprehension. In Regina Barzilay and Min- Yen Kan (eds.), Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume I: Long Pupers), pp. 1601-1611, Vancouver, Canada, July 2017. Association for Computational Linguisties. doi: 10. 8653/ 147.. URL htrtpp://aclanthoooyy..rr.
Chong Li, Shaonan Wang, Yunhao Zhang, Jiajun Zhang, and Chengqing Zong. Interpreting and exploiting functional specialization in multi-head attention under multi-task learning. In Houda Bouamor, Juan Pino, and Kalika Bali (eds.), Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, EMNLP 2023, Singapore, December 6-10, 2023, pp. 16460-16476. Association for Computational I Linguistics,, 2023a. doi: 10. 102.. htRL/httss:///d0/
/1tp1/00.1163//1120000 www.:/20.....mmmmpppemmmm.p www.maainnn.... 1026
Kenneth Li, Oam Patel, Fernanda B. Viégas, Hanspeter Pfister, and Martin Wattenberg. Inference- time intervention: Eliciting truthful answers from a language model. In Alice Oh, Tristan Naumann, Amir Globerson, Kate Saenko, Moritz Hardt, and Sergey Levine (eds.). Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10 16, 2023, 2023b. URL http: http::::ppppprss.ipi.p.. cc httppppssccpppprr__ee wwaaperrflessppppppaaeeee /tlls//pperr//233/hha//881119119 909cb769f Awstract coonference.htmm
Stephanie Lin, Jacob Hilton, and Owain Evans. Truthfulqa: Measuring how models mimic human falsehoods. In Smaranda Muresan, Preslav Nakov, and Aline Villavicencio (eds.), Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Lonng Papers), ACL 2022, Dublin, Ireland, May 22-27, 2022, pp. 22522. Association for Computational Linguistics, 2022. doi: 10.
URL https://doi.org///i
v1/2022.acl11ong. 229
Sheng Liu, Lei Xing, and James Zou. In-context vectors: Making in context learning more effective and controllable through latent space steering. CoRR, abs/2311 11211106666 106668, 2023. doi: 10.48550/
ARXIV 22112.111222 .06668. URL
Shayne Longpre, Le Hou, Tu Vu, Albert Webson, Hyung Won Chung, Yi Tay, Denny Zhou, Quoe V. Le, Barret Zoph, Jason Wei, and Adam Roberts. The flan collection: Designing data and methods for effective instruction tuning. In Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett (eds.), International Conference on Machine Learning, ICML 2023, 23-29 July 2023, Honolulu, Hawaii, USA, volume 202 of Proceedings of Machine Learning Research, pp. 22631-22642 PMLR, 2023. URL httpps://procceeiing.ss press/v222/Loggreeaaahte
Thomas Mesnard, Cassidy Hardin, Robert Dadashi, Surya Bhupatiraju, Shreya Pathak, Laurent Sifre, Morgane Rivière, Mihir Sanjay Kale, Juliette Love, Pouya Tafti, Léonard Hussenot, Aakanksha Chowdhery, Adam Roberts, Aditya Barua, Alex Botev, Alex Castro-Ross Ambrose Slone, Amélic Héliou, Andrea Tacchetti, Anna Bulanova, Antonia Paterson, Beth Tsai, Bobak Shahriari, Char- line Le Lan, Christopher A. Choquette-Chen Reid, Elena Buchatskaya, Eric Ni, Eric Noland, Geng Yan, George Tucker, George-Cristian Cristian Muraru, Grigory Rozhdestvenskly, Henryk Michalewski, Ian Tenney, Ivan Grishchenko. Jacob Austin, James Keeling, Jane Labanowski,
Clément Crepy, Daniel Cer, Daphne Ippolito, David
Lespiau, Jeff Stanway, Jenny Brennan,
13

---

Published as a conference paper at ICLR 2025
Jeremy Chen, Johan Ferret, Justin Chiu, and et al. Gemma: Open models based gemini research and technology. CoRR, abs/2403.08295, 2024, doi: 10.48550/ARXIV.2400... 2403.08295. URL https://doi.org:
2403 2403.08295
Nasrin Mostafazadeh, Nathanael Chambers, Xiaodong He, Devi Parikh, Dhruv Batra, Lucy Van- derwende, Pushmeet Kohli, and James Allen. A corpus and cloze evaluation for deeper under- standing of commonsense stories. In Kevin Knight, Ani Nenkova, and Owen Rambow (eds.), Proceedings of the 2016 Conference of the North American Chapter of the Association for Com- putational Linguistics: Human Language Technologies, pp. 839-849, San Diego, California, June 2016. Association for Computational Linguistics. doi: 10.18653/v1/N16- 10991 URL fttps://accannthooog.orrrg wttp://oooo.oog///-1/// 1898
Niklas Muennighoff, Thomas Wang, Lintang Sutawika, Adam Roberts, Stella Biderman, Teven Le Scao, M. Saiful Bari, Sheng Shen, Zheng Xin Yong, Hailey Schoelkopf, Xiangru Tang, Dragomir Radev, Alham Fikri Aji, Khalid Almubarak, Samuel Albanie, Zaid Alyafeai, Albert Webson, Edward Raff, and Colin Raffel. Crosslingual generalization through multitask finetuning. In Anna Rogers, Jordan L. Boyd-Graber, and Naoaki Okazaki (eds.), Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume I: -Long Papers), ACL 2023, Toronto, Canada, July 9-14, 2023, pp. 15991 11999911611111 16111. Association for Computational LinguitttiiiaaLiLiiiiiiiittete 891. UULJ:tts:///0.oo/////1/////////1
2023. doi:
cl cl-long.99
OpenAl. GPT-4 report. CoRR, abs/2303.08777, 2023. doi: 10448550/ARXIV22300.077777 URL /ttp:::://6ll.rrr/104555500000 20w
Kiho Park, Yo Joong Choe, and Victor Veitch. The linear representation hypothesis and the geometry of large language models. In Forty-first International Conference on Machine Learning, ICMI, 2024, Vienna, Austria, July 27, 2024. OpenReview.net, 2024. h t L/fGkrm?Pd=UG@GkkLzpp
Edoardo Maria Ponti, Goran Glavas, Olga Majewska, Qianchu Liu, Ivan Vulic, and Anna Korhonen. XCOPA: A multilingual dataset for causal commonsense reasoning. In Bonnie Webber, Trevor Cohn, Yulan He, and Yang Liu (eds.). Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, EMNLP 2020, Online, November 16-20, 2020, pp. 2362 2376. Association for Computational Linguistics, 2020. doi: 10. http:/22222..mmppmmiia.m 185.
URL httpp:://6oi.orr/18.11eee
Nina Rimsky, Nick Gabrieli, Julian Schulz, Meg Tong. Evan Hubinger, and Alexander Matt Turner. Steering llama 2 via contrastive activation addition. CoRR, abs/2312.066811 2023, doi: 10.48550/ ARXIV 231.206661......1 URL http:tttt/://6o.rr///1100 wttp:////00.000500//011222221111
Keisuke Sakaguchi, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi. Winogrande: An adver- sarial winograd schema challenge at scale. In The Thirty-Fourdy AAAI Conference on Artificial Intelligence, AAAI 2020, The Thirty-ecoond Innovative Applications of Artificial Intelligence Con- ference, IAAI 2020, The Tenth AAAI Symposium on Educational Advances in Artificial Intelligence, EAAI 2020, New York, NY, USA, February 7- 2020, pp. 8732-8740. AAAI Press, 2020. doi: A0.1669//AAAL
URL htt.h: //dp::::://o..oo .org/ 0. 1609/ wat.//aa....ivi..33.3..
Taylor Shin, Yasaman Razeghi, Robert L. Logan IV, Eric Wallace, and Sameer Singh. Autoprompt: Eliciting knowledge from language models with automatically generated prompts. In Bonnie Webber, Trevor Cohn, Yulan He, and Yang Liu (eds.), Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, EMNLP 2020, Online, November 16-20, 2020, pp. 4222 4235. Association for Computational Linguistics, 2020. doi: 10.18653/V1/ 1020.EMNLP-MAIN.3.
Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D. Manning, Andrew Ng, and Christopher Potts. Recursive deep models for semantic compositionality over a sentiment treebank. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pp. 1631 1642,164, Seattle, Washington, USA, October 2013. Association for Computational Linguistics. URL https://wwwt http://wwww..wwbbborrrtttttaggg/
14

---

Published as a conference paper at ICLR 2025
Nishant Subramani, Nivedita Suresh, and Matthew E. Peters. Extracting latent steering vectors from pretrained language models. In Smaranda Muresan, Preslav Nakov, and Aline Villavicencio (eds.), Findings of the Association for Computational Linguisties: ACL 2022, Dublin, Ireland, May 22-27, 2022, pp. 566-581. Association for Computational Linguistics, 2022. doi: 10. 10.18653/V1/ 3022.FINDINGS-ACII
URL htt.ittts///oo.ooog0/0./1/////// win.:/sss..iiiinnnna---aa..aaeee
Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov. Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cristian Canton-Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu, Wenyin Fu, Brian Fuller, Cynthia Gao, Vedanuj Goswami, Naman Goyal, Anthony Hartshorn, Saghar Hosseini, Rui Hou, Hakan Inan, Marcin Kardas, Viktor Kerkez, Madian Khabsa, Isabel Kloumann, Artem Korenev, Punit Singh Koura, Marie-Anne Lachaux. Thibaut Lavril, Jenya Lee, Diana Liskovich, Yinghai Lu, Yuning Mao, Xavier Martinet, Todor Mihaylov, Pushkar Mishra, Igor Molybog. Yixin Nie, Andrew Poulton, Jeremy Reizenstein, Rashi Rungta, Kalyan Saladi, Alan Schelten, Ruan Silva, Eric Michael Smith, Ranjan Subramanian, Xiaoqing Ellen Tan, Binh Tang, Ross Taylor, Adina Williams, Jian Xiang Kuan, Puxin Xu, Zheng Yan, Iliyan Zarov, Yuchen Zhang, Angela Fan, Melanie Kambadur, Sharan Narang, Aurélien Rodriguez, Robert Stojnic, Sergey Edunov, and Thomas Scialom. Llama 2: Open foundation and fine-tuned chat models. CoRR. 00.22307.0928,, 2023, doi: 10.48550/ARXIV WwW.:/ARRXXX.....00000 092288. URL brXiv.2307. 09288
Alexander Matt Turner, Lisa Thiergart, David Udell, Gavin Leech, Ulisse Mini, and Monte MacDiarmid. Activation addition: Steering language models without optimization. CoRR, abs/2308. 10248, 2023, doi: 10.48550/ARXIV 2308. 10248. URL http::::/coi.org///0000/ arXiv.2308.10 2ww..33381.0300 10248
Weixuan Wang, Barry Haddow, and Alexandra Birch. Retrieval-augmennte multilingual knowledge editing, arXiv preprint arXiv:2312.13040 2023.
Weixuan Wang, Barry Haddow, Alexandra Birch, and Wei Peng. Assessing factual reliability of large language model knowledge. Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguisties: Human Language Technologies (Volume I: Long Papers), pp. 805-819, 2024a.
Weixuan Wang, Barry Haddow, Wei Peng, and Alexandra Birch. Sharing matters: Analysing neurons across languages and tasks in llms. CoRR, abs/2406.09265 2024b. doi: 10.48550/ ARXIV 2406.09255. httt:s/tts:////t http:///oo...r//0..55/5///////// wwt.k/w..222220225222 09265
Weixuan Wang, Minghao Wu, Barry Haddow, and Alexandra Birch. Bridging the language gaps in large language models with inference-time cross-lingual intervention. arXiv preprint arXiv:2410.12462. 2024c.
Jason Wei, Maarten Bosma, Vincent Y. Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M. Dai, and Quoe Le. Finetuned language models are zero-shot learners. The Tenth International Conference Learning Representattions. ICLR 2022, Virtual Event, April 25-29, 2022. OpenReview.net, 2022. URL
Yiran Zhao, Wenxuan Zhang, Guizhen Chen, Kenji Kawaguchi, and Lidong Bing. How do large language models handle multilingualism? CoRR, abs/2402.18815, 2024. doi: 10.48550/ ARXIV 22..22222.. 18815. URL
Andy Zou, Long Phan, Sarah Chen, James Campbell, Phillip Guo, Richard Ren, Alexander Pan, Xuwang Yin, Mantas Mazeika, Ann-Kathrin Dombrowski, Shashwat Goel, Nathaniel Li, Michael J. Byun, Zifan Wang, Alex Mallen, Steven Basart, Sanmi Koyejo, Dawn Song, Matt Fredrikson, Zico Kolter, and Dan Hendrycks. Representation engincering: A top-down approach to AI transparency. CoRR. //bbl.org/10..8555//rkkkv 000
2023. doi: 10.48550/ARXIV 23100.01405 URL hhtps:
15

---

Published as a conference paper at ICLR 2025
A APPENDIX
A.1 DESCRIPTION OF TASKS
COPA Each question consists of premise and two alternatives, with the task being choose the alternative that more plausibly has causal relationship with the premise.
StoryCloze Each question requires a model choose the correct ending to a four-sentence story for evaluating story understanding and script learning.
NLI It a collection of sentence pairs manually labeled for halanced classification with the labels entailment, contradiction, and neutral.
MMLU It is benchmark designed to measure knowledge acquired during pretraining, covering 57 subjects across STEM, the humanities, the social sciences, and more.
SST2 and SST5 They datasets used for sentiment analysis with 2 labels (negative, posi- tive) and 5 labels (negative, somewhat negative, neutral, somewhat positíve, or positive), respectively.
BoolQ It is question answering dataset for yes/no questions where questions are naturally occurring.
Winogrande It fill-in-a-blank task with binary options, with the goal of choosing the right option for given sentence which requires commonsense reasoning.
TriviaQA It realistic text-based question answering dataset which includes question- answer pairs from documents collected from Wikipedia and the web.
TruthfulQA It benchmark to measure whether a language model is truthful in generating answers to questions. The benchmark comprises 817 questions that span 38 categories, including health, law, finance and politics. Truthfulqs includes both multiple-choice and generation tracks. The performance of multiple-choice track gauged using multiple-choice accuracy (MC). This metric is based on the conditional probabilities of candidate answers given the question, with successful result counted when the truthful answer ranks first. In the generation track, we use two pre-trained judge models to evaluate the truthfulness and informativeness.
ToxiGen It is dataset that contains implicitly toxic and benign sentences mentioning 13 minority groups.
XCOPA multilingual dataset, translated from the English COPA, used to evaluate the capacity of models to transfer commonsense reasoning across languages.
A.2
DATASET SIZES
The size of datasets for each task are described in the Table
Table 9: The number of data used for identifying key elements and testing for 11 tasks.
Task
COPA
StaryCloze
NLT
MMLU
SST2
SST5
Boc1Q
Wingrandde
Trivia0A
ToxiGen
TruthfulQA
identily
S00
360
2000
12178
1000
1000
1000
1000
1000
1000
817
testsel
12178
A.3 ELEMENTS OVERLAP
In our analysis, detailed in Figure 5) we examine the overlap of identified key elements (attention heads, neurons, and hidden states) across various tasks. We observed minimal overlap across components between open-ended generation and multiple-choice tasks, particularly in attention heads and neurons. This indicates high degree of functional specialization within these components. Unlike attention heads and neurons, hidden states demonstrate greater overlap, mainly due to their positioning in the final layer which more directly influences the model's 's output (as shown in Figure 4). The specialization observed among attention heads supports the notion that multi head attention mechanisms evolve uniquely according to the task similarities. It consistent with the finding in Li
al. 2023a
16

---

Published as a conference paper at ICLR 2025
Overlap top-10 Hoads
Overlap top-100 Nourons
overlap top-10
Figure 5: Overlap of identified key elements across various tasks. From 1 to 10 represents the tasks: COPA, StoryCloze, SST2, BoolQ, MMLU, NLI, Winogrande, TriviaQA, ToxiGen, TruthfulQA.
A.4
CROSS-LINGUAL SADI
Table 10: Cross-lingual results based on the identified heads/neurons from the English contrastive pairs.
Language
SW
ta
1h
tr
vi
zh
BASBLINE
51.40
61.20
50.20
49.40
50.80
49.40
51.80
62.80
SFT
68.60
79.20
52.00
47.00
49.60
57.80
68.60
77.80
SADI-HEAD
62.80
67.60
50.40
50.20
51.00
51.20
56.60
70.00
59.60
We have shown the effectiveness of SADI in the multilingual scenarios where contrastive pairs, constructed in the same language as the test input, are used to identify relevant components. Further, explored its impact in cross lingual setting by employing English contrastive pairs to identify key elements and then applying SADI to multilingual test inputs. The results, along with those of SFT which involves fine-tuning in English and testing multilingual dataset-aa presented in Table TO We found that S SADI displayed enhanced language transfer capabilities, particularly in Tamil, Thai, and Turkish. These successful interventions suggest that critical elements are shared across languages, supporting SADI's utility in cross-lingual applications.
A.5 ACTIVATION DIFFERENCE DISTRIBUTIONS ACROSS TASKS
We demonstrate the distributions of activation difference across layers for heads, neurons and hidden states in Figure 6 and in Figure in COPA, BoolQ, TriviaQA, and TruthfulQA tasks. They show consistent patterns across various tasks.
Lapors
(a) Difference on COPA. (b) Difference on BoolQ.
(c) Difference
on
(d) Difference TruthfulQA.
on
TriviaQA.
Figure Activation difference of each head across layer.
17

---

Published as a conference paper at ICLR 2025
ssuron hidden
ssuron hidden
neuron
100
neuron
60-
Nidde
hidden
50
number
number
40-
20
3
Layera
Layera
Layers
Layers
Distribution on COPA. (b)
Distribution
Distribution
(d) Distribution
BoolQ.
TriviaQA.
TruthfulQA.
Figure 7: Activation difference of top- 100 neurons and hidden states.
18


Reviews:
[{"review_id": "C6jMIheJGR", "reviewer": "Reviewer_QbAa", "summary": "This paper proposes a novel Semantics-Adaptive Dynamic Intervention (SADI) technique for Large Language Models (LLMs). Unlike conventional methods using fixed steering vectors, SADI adapts dynamically to the semantic context of inputs, modifying the model’s internal activations accordingly. Using activation differences from contrastive pairs, SADI identifies key elements (attention heads, hidden states, and neurons) of the model for targeted intervention. Experimental results show that SADI outperforms traditional intervention methods across various models, including LLAMA2-7B-CHAT, BLOOMZ-7B, MISTRAL-7B, and FALCON-7B-INSTRUCT, without requiring additional training. SADI’s cost-effectiveness and adaptability across languages and tasks highlight its potential as a versatile alignment technique​.", "questions": "1. In the experiments, are the questions ($x_i$) used for contrastive pair construction sourced directly from the task datasets (in-distribution), or do they include any out-of-distribution samples?\n2. Could you provide a qualitative example illustrating how SADI’s input-adaptive mechanism offers semantic adaptability compared to traditional fixed-vector approaches? This would clarify the practical benefits of SADI’s dynamic intervention.\n3. Could the authors provide further analysis on why SADI-HIDDEN shows lower performance on certain tasks? Exploring underlying causes for these variations could provide deeper insights.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The paper is well-written, and the methodology is clearly presented.\n2. SADI achieves substantial performance improvements across various benchmarks, often outperforming baseline methods by significant margins without additional training.", "weaknesses": "1. The paper lacks sufficient detail regarding the construction of contrastive pairs, including the specific number of examples used. Additionally, an ablation study on how the number of contrastive examples affects SADI’s performance would provide valuable insights into the robustness and scalability of the method.\n2. The experiments are limited to models up to 7B parameters, leaving the effectiveness of SADI on larger models (e.g., 13B, 30B, or more) untested.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "QzhVSfHyhw", "reviewer": "Reviewer_5RCA", "summary": "This work proposes an approach to dynamically steer model hidden states by adapting to the semantic contexts of inputs during inference time. Extensive experiments across various tasks, LLMs backbones, and languages show the effectiveness.", "questions": "See weaknesses.", "limitations": "", "rating": 6, "confidence": 5, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "- Figures and tables are clear and easy to read.\n- The method is explained in detail with clear mathematical formulations, pseudocode and a clear visual representation.\n- The experimental evaluation is extensive across multiple model architectures, languages, and tasks, accompanied by various ablation studies.", "weaknesses": "- While the paper mentions the \"linear representation hypothesis,\" it does not provide theoretical guarantees or formal analysis of why the method works.\n- Based on Figure 2, the performance is very sensitive to the two hyper-parameters, but this paper doesn't provide clear guidelines for selecting these parameters in practice.\n- The work lacks a critical comparison of computational efficiency across methods. While SADI is claimed to be \"cost-effective,” no concrete metrics (inference time, memory usage, computational overhead) are provided to compare against other baselines.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "05nUenFmKp", "reviewer": "Reviewer_Axwf", "summary": "The paper proposes a new LLM intervention approach that aims to steer model behavior while adapting to the semantic contexts of inputs.  Unlike prior intervention approaches with fixed steering vectors, this paper proposes to dynamically change the steering vector based on test inputs. The paper presents extensive experiments and ablations to demonstrate the effectiveness of the approach across multiple common setups in the community.", "questions": "See above.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The paper is mostly well written with clear and logically coherent organization.\n- The method is new and easy to implement.\n-  The experiments and ablations are thorough.", "weaknesses": "My primary concern is that the paper places a strong emphasis on **what** the method achieves but does not sufficiently explore **why** it works and why certain design choices are more preferable, beyond what is shown by hyperparameter sweeping. \n\nWhile the results across various tasks and model families suggest that the method is generally effective, there is a notable lack of in-depth analysis (particularly in Sections 5 and 6), to elucidate the underlying reasons for its success. Furthermore, the results presented in Figure 2 indicate that the approach may require extensive hyperparameter tuning, as consistent setups across different datasets are not apparent.\n\nSpecifically, it would be great to delve into the following aspects:\n\n(1) The experiments indicate that using attention heads yields superior results compared to using hidden states. However, the relationship between these components and the \"steering behavior\" remains unclear. A more detailed analysis of **why** attention heads (instead of hidden states) might contribute more effectively to the method's success would be valuable.\n\n(2) The rationale behind selecting negative pairs is not entirely clear. For instance, why is a \"blank space\" used as an incorrect answer (L286)? Similarly, the choice of a \"randomly chosen incorrect answer\" for multiple-choice tasks may raise concerns of not being representative and can introduce high variance based on the selected samples. What defines a \"good\" negative answer and how it impacts the method remains under-explored.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "2qapDQf9Zb", "reviewer": "Reviewer_Jf3J", "summary": "This paper introduces an innovative method named SADI, designed to provide a dynamic vector to intervene model activations at inference time in LLMs. Specifically, SADI leverages activation differences in contrastive pairs to identify and target critical units for effective intervention. The effectiveness of this method is demonstrated in experiments using multiple popular LLMs on multiple tasks.", "questions": "As shown in Weaknesses", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. SADI is a general steering method applicable to a wide range of LLMs. Through extensive experiments with four model backbones over eleven diverse tasks, SADI has  proven to significantly enhance model performance, surpassing baseline methods by substantial margins. Detailed analysis demonstrates that interventions targeting attention heads consistently yields significant performance improvements across various tasks, validating the effectiveness of this dynamic steering approach.\n\n2. This paper has a  clear structure and highlights the key points. Specifically, the Method section provides a detailed procedure of the proposed method. And in the Experiment section, they outline the experimental setup and objectives.  This makes it easy for readers to understand the core idea of this work. Besides, the author promises to release the code.", "weaknesses": "1.  In the Related Work Section, the author mentioned that the difference between SADI and these \"fixed steering vector\" works is that SADI takes \"input semantics\" into account. However, according to Algorithm 1, SADI uses the steering vectors obtained by the mean difference of activation of all positive and negative samples in the test set, which is also \"fixed\" to some extent. Besides, in the paper, the author said “CAA uses the mean difference in the activations at the position of the answer letter between all the positive and negative prompts to construct a fixed steering vector to shift activations.” From this angle, the changes made by SADI are actually very small. **So what is the essential difference between SADI and these related works? Why is SADI effective?**\n\n2. For SADI, what is the relationship between the dataset used in \"Difference Extraction\" step and the dataset used in \"Adaptive Steering\" step? Are they from datasets on the same task? Or from datasets on different tasks? Or from the same dataset? What is the rationale for doing so? How to explain it?  I think these questions concern the effectiveness of SADI.\n\n3. This paper lacks guidance on selection of hyperparameters. SADI introduces the hyperparameters, but there seems no way to perceive which hyperparameters is optimal when solving a new dataset from Experiment section. Without such a principle, SADI may be left in the shade compared with similar methods. \n\n4. No prominent improvement with SADI using SFT. In Table 1, SADI+SFT only performs 90.97 over SFT by 0.06. The author needs add more reasons why this issue occurs in the paper.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "212aALn0uM", "reviewer": "Reviewer_74Tn", "summary": "The paper introduces the Semantics-Adaptive Dynamic Intervention (SADI) method for improving Large Language Models' performance on downstream tasks with a dynamic intervention mechanism. The method is shown to be effective through extensive experiments and requires only a small dataset. However, it is unclear for how to address the potential imbalance of positive and negative examples in real-world datasets, which could affect the method's applicability.", "questions": "Please refer to the Weakness Section.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 4, "strengths": "1. The paper introduces a novel approach called Semantics-Adaptive Dynamic Intervention (SADI) designed to modify the behavior of Large Language Models (LLMs) with the aim of enhancing their performance on downstream tasks through the intervention.\n2. In contrast to prior research that necessitated fixed intervention masks for each task, this study introduces a dynamic intervention mechanism that autonomously adapts to a variety of downstream tasks.\n3. The authors have conducted an extensive series of experiments to demonstrate the efficacy of their proposed method.\n4. The ablation studies provided by the author are thorough and indicate that the SADI method does not demand an extensive dataset. Remarkably, approximately 150 examples are sufficient to achieve commendable performance with SADI.", "weaknesses": "1. As discussed in Section 3.2, the dataset T is structured such that each entry includes a single positive example and one negative example. In real-world scenarios, however, the prevalence of negative examples typically surpasses that of positive examples. In cases where negative examples are more abundant, the methodology for selecting which negative examples to include in the construction of each instance within dataset T is not clear. Guidance on this selection process would be beneficial.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)