Do different prompting methods yield a common task representation in language models?
Authors: Guy Davidson
Todd M. Gureckis
Brenden Lake
Adina Williams
Venue: NeurIPS 2025 poster ()
Keywords: task representations; function vectors; in-context learning; instructions; mechanistic interpretability
Primary area: deep_learning
TL;DR: Different ways of prompting the same task elicit different task representations in language models.

Abstract: Demonstrations and instructions are two primary approaches for prompting language models to perform in-context learning (ICL) tasks.
Do identical tasks elicited in different ways result in similar representations of the task? An improved understanding of task representation mechanisms would offer interpretability insights and may aid in steering models. We study this through function vectors (FVs), recently proposed as a mechanism to extract few-shot ICL task representations. We generalize FVs to alternative task presentations, focusing on short textual instruction prompts, and successfully extract instruction function vectors that promote zero-shot task accuracy. We find evidence that demonstration- and instruction-based function vectors leverage different model components, and offer several controls to dissociate their contributions to task performance. Our results suggest that different task prompting forms do not induce a common task representation through FVs but elicit different, partly overlapping mechanisms. Our findings offer principled support to the practice of combining instructions and task demonstrations, imply challenges in universally monitoring task inference across presentation forms, and encourage further examinations of LLM task inference mechanisms.

Do different prompting methods yield a common task representation in language models?
Guy Davidson', Todd M. Gureckis', Brenden M. Lake Adina Williams'
FFIIR Meta, NNw York University, 'Piicctonn University guy. www.davddso@@nyy edu todd. gwweckis@nuu............ edu, brenden@princeton. edu adinawi 11 ams @meta.atammmmmmmmmmmm com
Abstract
Demonstrations and instructions two primary approaches for prompting lan- guage models to perform in-context learning (ICL) tasks. Do identical tasks elicited in different ways result similar representations of the task? An improved under- standing of task representation mechanisms would offer interpretability insights and may aid in steering models. We study this through function vectors (FVs), recently proposed as mechanism to extract few-shot ICL task representations. We gen- eralize FVs to alternative task presentations, focusing on short textual instruction prompts, and successfully extract instruction function vectors that promote zero- shot task accuracy. We find evidence that demonstration- and instruction- based function vectors leverage different model components, and offer several controls to dissociate their contributions to task performance. Our results suggest that different task prompting forms do not induce common task representation through FVs but elicit different, partly overlapping mechanisms. Our findings offer principled support to the practice of combining instructions and task demonstrations. imply challenges universally monitoring task inference across presentation forms, and encourage further examinations of LLM task inference mechanisms.
(2) Construct function representation
(3) add function vector residuel
In-context learning
Identify attention heads
from demonstrations
FV
stream elictt
Q: Japan Tokyo
0:
Chi Santiago
France Paris Egypt
CaEro
Q: Egypt Az
Caira
LLM
LLM
In-context learning from Instructions
Map country to its capital city. Q: Egypt Ai
Cairo
Q: Egypt
Cairo
Findling 3 Instruction and Demonstretion FVs mostly engage different attertian heads $3.3
Finding Demonstration end instruction function beneficial tagethor $3.2
Finding 1 Function vectors
extracted instructions facilitate zero-shat task accumcy $3.1
Figure 1: Language model task representations depend on the form of task presentation. compare in-context learning task representations formed from demonstrations with those formed from instructions using function vectors (FVs). The process of extracting FVs is shown in Steps (1)-(3). We highlight several findings: We successfully extract FVs from instructions ($3. 1); Instruction FV' offer complementary benefits when applied with demonstration FV's ($3.2): Different prompting methods yield distinct task representations ($3.3; highlighted squares on the left LLM are Llama-3, 1aaee-..-BBBBBBe -8B--nnttruct attention heads: those identified by demonstrations only, those identified by instructions only, and shared ones; columns are layers, rows are head indices, see Figure 2D)
39th Conference on Neural Information Processing Systems (NcurIPS 2025).

---

Introduction
If you prompt a large language model (LLM) with in-context examples "Q: Japan A: Tokyo Q: Chile A: Santiago Q: France A: Paris Q: Egypt A:" or with instructions "Map country its capital city: Q: Egypt A.", you expect to get the same answer (Figure 1). These two prompts share only the final query common, but imply the same underlying task for the network to perform. This paper explores how these two prompting methods result in similar or different representations of the specified task in the language model. We consider these representations task representations, as they reflect information in the network that induces the correct mapping for a given task, rather than the answer itself.
We focus on two prevalent approaches to specifying tasks LLMs: demonstrations and instruc- tions. The ability of LLMs to perform in-context learning (ICL) from demonstrations has been of considerable interest since GPT-3 (Brown et al. 2020 Lampinen et al., 2024). With appropriate fine-tuning, language models can also follow textual instructions, Facilitating a far broader range of use cases (Chung al. 2022). We follow considerable literature studying the mechanisms that govern ICL (e.g.. Olsson et al. 2022 Chen et al., 2024, Akyüüek et al.) (2024) and those that promote instruction- followingn (Stolfo et al. [024] et al., 2024). Prior work studied either ability by itself; here, we examine to what extent the two share representations and mechanisms.
We approach this question by leveraging new interpretability method known as function vectors (FVs. Todd et al.] (0024). FVs are causal patterns identified from the intermediate layers of a language model that mediate its ability to perform a task. [Tod et al. demonstrate that FVs, as single additive intervention to a model's latent activity, successfully cause the execution of task in a different (or empty) context. We extend Todd et al. FV extraction method from using specifically in-context demonstrations to any form of task presentation, including instruction prompts. We then compare the representations elicited by demonstrations and textual instructions, examining their effectiveness at inducing task-following behavior, latent activity similarity, and elicited internal mechanisms.
Table summarizes our key findings. Our extension of the function vector identification procedure successfully extracts instruction function vectors, and these promote zero-shot task accuracy ($3.1). We validate that demonstration and instruction FVs contain complementary information by inter- vening with both simultaneously, and find it conveys task performance benefits beyond using either FV alone ($3.2). Next, we examine the attention heads identified in demonstration and instruction FVs, and find that most are identified by one type of FV, with only a few shared by both ($3.3). We evaluate the functional implication of the different attention heads and find an asymmetry between the relevance of demonstration-ident iddetified attention heads to instructions and instruction- identifiee heads to demonstrations ($3.4). Finally, noting that instruction FVs function better in post-trained models, we find that we can steer base models with post-trained model instruction FVs, and we identify the relevant post-training stages for instruction FV extraction ($3.5).
Overall, our results suggest that different task presentations do not induce a common task representation through function vectors, but activate partly overlapping mechanisms and induce jointly beneficial representations. Our findings provide new insights into how LLMs represent tasks and offer an explanation for why combining instructions and in-context examples often improves model performance.
Table 1: Guiding research question and summary findings
Claim
Section
Question
Do different ways of presenting the same task elicit common task representation?
Method
Extend Function Vectors (FVs) from in-context demonstrations to instructions (and other) task presentations, and analyze their properties.
Finding Finding Finding Finding
Function vectors extracted from instructions facilitate zero-shot task accuracy.
3.1
Demonstration and instruction FVs beneficial together. Instruction and demonstration FVs mostly engage different attention heads. Instruction indiiiii-ddeiiiieedd attention heads are more useful for building demonstration FVs than vice versa.
3.2
3.3
3.4
Finding 5
Instruction FVs from post- trainer models can steer base models and arise from supervised tne-tuning and preference optimization.
3.5

---

2 Methods
We describe the function vector extraction procedure outlined by Todd et al. (2024) and our extension from in-context demonstrations to arbitrary task prompts (see Todd et al. 2024] for additional details). The procedure identifies small set of causally relevant heads in a model j for performing a given task t, and uses them to compute the function vector v, E IRdmodee, where dymodet is the model's latent dimension. Each task E T (the set of tasks considered) consists of supervised dataset D. = =[(r1, 31); 12). ... NI, )), where and Y4 are (tokenized) string inputs and outputs respectively. We construct K-shot in-context demonstration prompts 2 for a query example (Fig. Mig) as pf [[ Yo1), (2 312)..
(ætK. FiK) ( Fig]. To identify causally relevant attention heads, consider a set of prompts Pt for a given task on which a model f succeeds in predicting Vig from př. P: ppi.
pvx) }. Denote the output of attention head az (head j in layer () in processing the final token of prompt p as atz (p$) R Romoddt (projected to the model's latent dimension). We compute the mean task-conditioned activation of each attention head, aij TP. Eptep, (t)x())
Next, we construct uninformative baseline in-context demonstration prompts o by shuffling the labels yu assigned to each Tæs: p [[ Wa)), (x2). (I.Kk B.k), ), Zigl- We seek to score each head by its causal indirect effect toward predicting the correct Fig from the shuffled prompt it; that is, attention heads that promote task accuracy in the context of the shuffled prompt. To compute this, denote Oij == di the intervention of setting the output of attention head at to its consionnd activity diz- Then compute the difference in probabilities assigned to the first token of the correct Yig between when we intervene on the model f and when we do not: f Todd al. (2024) find that a small set of heads consistently achieve high causal scores across their task set. Using A to denote this set of top heads, FVs are computed for each task using the heads' task-conditioned means: Up = Dang EAP alz.
ClE(aay p = f(Ø | aty := diy)
2.1 Generalizing function vectors beyond in-context demonstrations
One of our contributions is generalize the function vector method described above from demon- strations to alternative task specifications. We denote by Q: a set of task specifications (of any form) for the task (e.g., the instructions in Figure 1p. For a query example (Zig. yiqa) (Egypt = Cairo in Figure 1). we sample e E Q, and construct prompts P as Pf = [4 FTgg Next, we address the challenge of creating uninformative baselines ÿh, for the task specifications dhm We consider three
approaches to generating these baselines, and see further details and examples in AAppendix B]
Equiprobable token sequences. Intuition: sample token sequences that are similarly likely under the model but are unrelated to the task. Each din is encoded as a token sequence d = [wi, W2, ..., wr.] whose probability under the model ʃ is P(g
IIesr. P(we w<xe) = f(w= [wrz]. Starting from the BOS token, we sample iy to approximately match the conditional probabiliiies f((=)) [uy] x (w++) (wx), and construct d't = [wi] Wz2, r.. ..z).
Real texts: Intuition: sample texts from a natural corpus that convey no task information but otherwise match the task specifications. We score token sequences from a chosen corpus under the model and sample ii, with approximately the same length and probability as 2
Other task specifications: Intuition: sample task specifications for other tasks '' * / that likely convey no information for task t. We score task specifications df generated for other tasks '' E T, and again sample ones with approximately the same length and probability as d
Following Todd et al. is (2024) selection of prompts in which the model successfully performs the demonstrated task, we focus on best- rest-performing small set of alternative task specifications we denote as Q = {9-T
} E Q., with the intuition that these facilitate forming the most salient representation of the task t. We construct prompts pf Ig 2t.]. keep only ones in which the model successfully predicts Yık. use these to select Qi, and narrow down our set of prompts to a final Pi. where of- (gin 3. From there we follow the original function vector procedure: we compute mask-conditioned activations (over Pr),, generate uninformative baselines 7 = (mm- F. compute the indirect effects for each attention head, and use these to select the top heads (in practice, we average over these different baselines in computing head causal effects; see below).
Constructing function vectors from in-context instructions Although the method is more gen- eral, here we use zero-shot textual instructions our Qz. We generate candidate set of textual
3

---

instructions for each task by querying Llama-3.1-4055 (Llama Team 2024) using a K-shot ICL prompt for the task, and ask it generate 10 instruction prompts for the task infers from the provided demonstrations (see Asee/Appendxx for the templates). We repeat this procedure 20 times for each task and deduplicate exact repetitions to arrive at a task instruction set Qt. We repeat this procedure twice, once encouraging the model to generate short instructions and once with no such encouragement. We provide sample generated instructions of both lengths in AAppendix A.I and corresponding uninformative baselines (using all three approaches) in Appendix B.2 For our 'real texts' uninformative baseline, we sample texts from TetTe-xx-----
Merity et al. 2016
2.2 Experimental conditions
(2024): we use 100 examples to compute mean task-conditioned activations and 25 for the indirect effects. We do so over the J = 5 top textual instructions for each task, splitting the examples evenly between the top instructions.
Hyperparameters. We match the settings used by Todd et al.
Models. We focus on the base and instruction-tuned versions of the 3B Llama-3.2 and 8B Llama-3.1 models, with the full list in Table 4 7B Llama-2 models (Touvron et al. 2023), the latter of which matches Todd et al. (2024). Finally, to examine the roles of post-training stages, we evaluate four OLMo-2 models (OLMo et al. 2024).
We also report some results with the weaker 1B Llama-3.2 and
Tasks. We consider the same set of tasks and datasets used by Todd et al. (2024). We omit a few classification datasets where successfully predicting the next token requires understanding of the format that is facilitated by demonstrations but not necessarily by minimal textual instructions (we retain a total of 50 datasets; see Appendix D for the full list). We follow Todd et al. (2024) in computing the top sets of heads only over datasets where a model surpasses chance performance. As our approach requires 20 successful prompts for each of the five best instructions to compute the mean lask-conditioned activations, we also omit tasks where a model fails to pass this number of prompts (which, in most cases, means it is also below chance accuracy and was already omitted ).
Textual instructions and uninformative baselines. We evaluate each model on each task using one random seed six separate times, for the short (< 16 tokens) vs. longer (unbounded) instructions crossed with all three uninformative baseline approaches (62.1). We observe minimal deviation in top heads across these conditions (see Appendix H.3), so we average the causal effects across all six to compute the top heads. We then report final evaluation accuracies averaged over the results with both sets of mean activations, those generated with short instructions and those from longer ones.
Results using in-context demonstrations. To enable comparison, we also replicate the original, demonstration-bast function vector evaluation with all models we consider, using the same random seeds (and therefore, same train-evaluaae splits) as used in the textual instruction setting.
Evaluation and Comparison Logic. We structure our evaluations to facilitate comparison between FVs derived from demonstrations and instructions. We use the | = 20 top heads in both settings, consistent with Todd et al.) (2024) for the 7B models. We follow Todd et al. (2024) in evaluating function vectors as an additive intervention to the residual stream after the L,/3 layer (layer for the Llama-3.2-3B models and layer 11 for the Llama-3. 3.18BB and OLMo-2-1123--7B ones). We report two evaluation settings: 10-shot with shuffled labels (7) $2. 1) and 0-shot (with no instructions). We focus our assessment of each FV in the setting matching its extraction: shuffled 10-shot for demonstration FVs and 0-shot for instruction FVs. In addition to the FV evaluations, we report a baseline of the evaluation setting without adding the function vector. We report accuracies using informative task presentations for each evaluation in Apppeddxx F instructed 0-shot (averaging over the top J 5 instructions for each model and task), and 10-shot (without label shuffling).
3 Results
3.1 Function vectors extracted from instructions facilitate zero-shot task accuracy
Figure 2A summarizes our evaluation of demonstration function vectors (left) and instruction- based FVs (right). We find that our adaptation of the FV extraction procedure succeeds and increases zero-shot task accuracy from below 20% to above 50% in the best models. A notable exception the Llama-3. model, for which the T./3] intervention depth appears suboptimal (see results from other intervention layers in Figure II) On the other hand, the Llama-3, 3.2 models may benefit

---

Shuf fled 10-shot Eval
8-shot Eval
Shuffled 10-shot Eval
0-shot Eval
1.0
A
1.0
0.8
0.8
Llana 3.
Accuracy
Llara-3. 3.
0.6
0.6
Llara 3.1-88
Llara-3.1-88 1--8-IInstruce
0.4
t
0.4
Shuffled 1h-thot
8.2
0.2
0.0 Demonstration FV
0.0
Instruction FV
Demonstration FV Both FVs Both FVs Instruction FV
Figure 2: Instruction-base based function vectors are effective in zero-shot evaluation; both function vectors are beneficial together. (A) We evaluate each FV in the setting matching its extraction: shuffled 10-shot for demonstrations and 0-shot for instructions. Both FV types are effective at their respective evaluations, though demonstration ones fare better. This procedure is effective for both post-trained models and for the distilled Llama-3.2-3B base model; less so for the 1.mm-33.1888 base model. (B) We examine the effect of jointly adding both FVs together. The joint intervention outperforms either one by itself (with the exception of the base
Llama-3.1-80 model, for which the [./3] intervention depth appears highly suboptimal; see Figure 11) Dotted lines represent baselines with no FV intervention and error bars reflect standard errors of the mean (SEMs). See Figure 11 for results when intervening optimal depths, rather than fixed //3 depth, and Figure 12 for results evaluating the function vectors in the setting opposite their extraction.
from being distilled from larger, post-trained Llama-3.1 models. The top |A| = 20 attention heads also represent a slightly larger fraction of attention heads in the smaller models (about 3% in the 3B models vs. 2% in the 8B ones). We also observe that, unsurprisingly, instruction FVs are substantially more effective in post-trained models. Finally, while instruction FVs are highly effective in the zero-shot setting, they fail to match the accuracy demonstration FVs attain in the shuffled 10-shot condition, and struggle when evaluated in the shuffled 10-shot condition themselves (see Figure 12) We replicate findings and 2 (below) with other models: see Figures 13 15 in appendices G.3 G.4
28 shared
L1amn 1lamn-3.2-2 1.2-30
- 2-2--tttttt
shared
Llass-3.2 3.2-38 Farily
Stuffled 10-skot Eval
B
Head Hiattarity
9.
0.5
13
0.4
0.2
6 12
35 28 24
33
5
12 15 20 24 1t
0.5
28. top heads, 4 stared
28 top basds, 4 shared
12 is
Layer
at
Dana acts
PV
Inst acts
Ltane-3.1:40 Fly
G-shst Exal
Eistlarity
1.0
Ller--22-2 2-20
23
0.3
0.5
tsee 5.1
0.5
0.2
0.2
Layer
Layer
Instruction only
Densnstration only
Shared
Layer
Text beads
Dana acts
Instruction
Dean heads acts
Figure 3: (A-D) Demonstrations and instructions elicit mostly distinct attention heads. We visualize the sets of top heads identified only with demonstrations, only with instructions, and shared in both cases. Only a few top heads are shared each in model (see Appendix H.T for additional models). We observe that top instruction FV heads tend to be in earlier layers of the post-trained (-Instruct) models, compared to the base versions, and that shared heads tend to arise in later layers. (E-F) Shared head activation similarity depends on layer, but not on post-training. We plot the cosine similarity of activations in shared heads between three sets task-conditioned activations: those elicited by demonstrations, by short instructions, and hy longer instructions. We observe increasing similarity between demonstrations and instructions in later model layers, but show effect of post- training on similarity. (G-H) 'Heterogeneous' function vectors show demonstration-instrueee We construct 'heterogeneous" FV's using attention heads localized with instructions and demonstration task activations (left data points) to the opposite combination, using demonstration-localle task activations (right). Compared to the 'homogeneous' function vectors (middle), we mostly observe smaller accuracy drop when using instruction FV heads read from demonstration mean activations than vice versa.
asymmetry.
heads and instruction
5

---

We attribute these differences to two factors. Demonstration FVs arise from the attention heads that most improve shuffled 10-shot accuracy (as those were the uninformative prompts 17: $2.1). In contrast, O-shot evaluation differs from the instruction FV baseline prompts, as it contains no instructions (and hence, has fewer tokens than the uninformative instructions o:: $2.2). Further, ICI. from demonstrations appears more primal to L.MMs than from instruction-follovingg The former appears in base models, while the latter requires post-training; we support this claim with later results.
Constructing function vectors seems to benefit from model capabilities. Smaller Llama-3.2-11 models show a lower accuracy increase with instruction FVs; conversely, newer OLMo-2-1124. 1-2-77 models show greater increase (see (epeendix G.4). This capacity is not unique to the latest models; results with Llama-2-7B models qualitatively match newer models. Finally. we evaluate the effect of choosing fixed intervention layer in Figure IT] If we choose the optimal layer for each model and evaluation, the accuracy numbers rise as expected, but the overall patterns remain the same.
3.2 Demonstration and instruction function vectors are beneficial together
If demonstrations and instructions elicit different task representations in language model, to what extent are their representations jointly beneficial? To study this, we intervene with both FVs, adding them to the residual stream after the same | layer. Our results in Figure 2B demonstrate that adding both function vectors appears consistently beneficial, except for the base Llama-3. model (for which the choice of layer seems crucial). Surprisingly, adding both FVs after the same layer does not appear to induce interference, even though these FVs were extracted independently. In Figure IIB, we report results from the highest accuracy intervention layers in sweep over layer pairs; adding both FVs to the same layer often proved optimal. Finally, to establish how much of this effect is simply due to amplifying the intervention, we report the results of adding the same FV twice in Figure 18 For some models and evaluations, adding the same FV twice performs similarly to adding both FVs. This suggests that some of the benefit is due to the magnitude of the intervention rather than the different information carried in both FVs; we leave studying the extent of these relative effects to future work. Having found behavioral evidence that these different function vectors convey different information, we next examine to what extent they share mechanisms within the model.
3.3 Instruction and demonstration FVs mostly engage different attention heads
we visualize the |A| = 20 top attention heads identified in each model by the FV extraction procedure. We observe that demonstrations and instructions elicit mostly distinct sets of heads. Of the 20 top heads identified, instruction and demonstration FVs share few of them - 7 in both Llama-3.2-3B models and only 4 in the Llama-3-L.8B ones. Post-training appears to move instruction FV heads closer to demonstration FV heads. The most notable change between base and post-trained model versions is in the average layer of the instruction-only heads from a mean several layers deeper than demonstrations in base models, to almost identical depths in post-trained ones (other model classes are similar, see Aceendxx H). We observe higher causal scores in demonstration top heads. In $2 we define the causal score of each head as its contribution to correctly predicting the next token. Demonstration FV top heads receive substantially higher scores than instruction ones (Table 7). This suggests that task inference from demonstrations is more localized to a small set of heads than from instructions and that instruction- task inference is more diffuse.
In Figure 3A-D
We also examine the mean lask-conditioned activation patterns in the top heads shared between instructions and demonstrations. We compute the cosine similarities between the patterns elicited by demonstrations vs. by shorter/longer instructions (see Figure 3E-F). Expectedly, we find higher similarities between the two ciaructtiondrrvenn activation patterns than between the instruction and demonstration patterns. Additionally, later layers are generally more similar. Finally, post-training appears not to adapt demonstration FV heads. We observe no difference demoontratiooninnttueeie similarities between base and post-trained models, and equal numbers of shared heads between instructions and demonstrations across model versions. Instead, post-training appears to induce a different, separate mechanism for instruction task inference, one we hope to explore in future work.

---

3.4 Instruction-identifeed attention heads are more useful for building demonstration FVs than vice versa
If demonstrations and instructions share a common task inference mechanism, would not observe a significant difference if we constructed 'incongruent' function vectors using top heads identified by one with activations from the other. We borrow from neuroimaging and consider the FV extraction procedure through the lens of functional localizers, viewing head identification as localization and mean activation computation as recording (Saxe et al., 2006; [Berman et al., 2010). We evaluate both types of 'incongruent' FVs, those constructed with demonstration top heads and those using instruction top heads, and summarize our results in Figure 3G-H We observe that, as expected, accuracy with these incongruent FVs falls below the regular FVs in all cases. We note an asymmetry: using instruction-localize seems preferable to the opposite combination (with the exception of the shuffled 10-shot evaluation on the base models). This effect sharpened when we evaluate each FV in the layer resulting the highest accuracy, rather than fixing L/3 (Figure 28), To explain this, report the causal indirect effects of the top heads from each task presentation in the opposite presentation format- that is, the scores of cnaruuttionlocaalizer heads when using demonstrations, and vice versa Table 8) Examining the scores, we observe that cntructiionlocalizze heads are more helpful in the demonstration setting than cetoonsttationoocclizee heads are in the instruction setting. We take this as evidence that the mechanism for task inference from instruction leverages attention heads that play a minor role in demonstration task inference. The development of such mechanism may also explain the depth alignment shifts we previously observed between base and post-trained models (subsection 3.3] Figure 3A-1)) In contrast, the demonstration task inference mechanism uses attention heads that are less useful for instruction task inference. This supports our earlier claim about the primacy of demonstration ICL. We conclude with two control experiments to ensure that this observed asymmetry is meaningful (and not an artifact of selecting arbitrary sets of heads). We select sets of heads that are either unrelated both FV types, or that have the lowest causal scores (Appendix J.T) In both cases, accuracy falls to task baseline or below, suggesting the observed effect is meaningful.
top heads from with demonstration mean activations
3.5 Instruction FVs from post-trained can steer base models, arise from SFT and DPO
Given the effectiveness of instruction FVs in post-trained models, we examine whether instruction FVs from post- post-trained models can steer base models. We repeat the instruction FV evaluations, intervening on each base model with the FV generated by its post-trained version (multiple post- trained variants exist for OLMo-2-1 1-14------- we use the final model, 2LMMo-2-122--77 124-7B-Inntrucc) We report our results in Figure 4B In three of the base models, we find substantial accuracy increases, nearing the zero-shot accuracy elicited by instruction FVs in the post-trained models (consistent with Stolfo et al. [s (2024) cross-model steering results). This effect is even more striking when
0-shot Eval
Shuffled 10-shot Eval
0-shot Eval
1.0
A
1.0
CLNo 1124 78
8.8
8.8
1124---112-78
Accuracy
CLMo 1124 7B Instruct
DLMe-2-1222 -78-00
0.6
Llama-3.2-18
0.4
L.2-3-3.2-30
8.4
Llama-3.1-88
0.2
OLM--2-1124-78
8.2
Llams-2-76-hr
0.0
8.0
Instruction FV from Base Model
Instruction FV
Demonstration
Instruction FV
from Post-trained Model
Figure 4: Instruction-baser function vectors can transfer from post-trained to base models and are facilitated by SFT and DPO. (A) We apply the instruction FVs extracted from post-trained models to their respective base models. We find weak impacts on the distilled 1lama-3.2 models, and substantial henefits in the other ones, almost recovering the post-trained model FV evaluation accuracy.
(B) We examine the M------ 1124-BB family of models and observe mcaningful increases in the efficacy of instruction-basee function vectors both the SFT and DPO stages over the base model; conversely, the final RL stage appears minimally impactful. In both panels, dotted lines represent baselines with no FV intervention, and error bars reflect standard errors of the mean (SEMs). See Figure 33 for results evaluating the function vectors in the setting opposite to their extraction, and Figure 34 for results when intervening optimal depths, rather than fixed [L/3] depth.
7

---

evaluating the steering instruction FVs in the shuffled 10-shot setting (Figure 33A). We take this as further evidence that this procedure identifies the task representations elicited by instructions. We find minimal or mildly negative effects on the Llama-3.2 models.
We conclude by examining which post-training stages contribute to the ability to extract instruction FVs using the OLMo-2-1 124-7B family of models (OLMo et al.) (024). The authors release four versions of the model: a base version, one following supervised fine-tuning (-SFT), another following preference fine-tuning (-DPO), and a final version (-Instruct), Figure 4B depicts results with these four models. All OLMo models are conducive to extracting function vectors from demonstrations, but the base model instruction FVs show lower accuracies. We observe two accuracy increases, once with the SFT model and another with the DPO model. The DPO model also offers an increase in the efficacy of instruction FVs in mismatched evaluations (see Figure 33B). Finally, we note that post-training appears to have no impact on demonstration- ICI., as evidenced by the lack of variability in Figure 4B (left).
Related Work
In-Context Learning. The ability to perform tasks from demonstrations has been in the foreground
of language model research following its identification by Brown et al. (202)). Substantial subsequent
research has studied LLMs' ability to perform in-context learning, from perspectives such as the data
distribution (Chan al. 2022, Chen et al.] 2024). learning algorithms (Xic et al. 2021 Akyürek
ct
al.
2022 2024), mechanics and learning dynamics (Von Oswald et al. 2023] Zhang et al. [202]]
Park al.
2024 Han al. 2024 Li et al. 20255) and latent representations (Pan et al. 2023 Todd
cet al.
[0244] Hendel et al.] [0233 [123]yin Yin and Steinhardt (025). Other lines of work examined factors to
ICL contributing (Min et al. (0022) such as sensitivity to in-context prompt formatting choices (Sclar
ce al.
2023 Su et al. 2025) Most prior work focuses on ICL from demonstrations: our work bridges
between those and instructions
gateway exploring other forms of ICL (Lampinen et al. 2024).
Instruction-follovii refers to executing a task given a natural language, similar to how one person may instruct another. Although fine-tuning is necessary to elicit this capability, it has been shown to facilitate generalization and usefulness following the introduction of the Flan models (Chung et al.
2022). ceerruction-ollowiiii
has since been the focus of dataset development (Mishra al. 22222
Wang et al. 2022a]b) (/ /hhu et al. 2023a Taori et al.) (023), model evaluation (Zhou et al. 2023b
Liu et al. 2024 Lyu al. 2024), and interpretability work (Stolfo et al.) 2024 Wu et al.) [2024). Our
work explores whether instruction tuning leverages the mechanisms of demonstration based ICL.
Task Representations. Computational neuroscience has long sought to understand how neural networks learn to represent abstract tasks (Cohen et al.] 1990 Botvinick and Plaut [2002] Yang et al., 2019 Fleech et al., 2021 Farrell et al. [023] Hummos et al. 2024). In tandem, computer scientists devised methods to explicitly introduce these representations to models (Lampinen and McClelland 2022). Our work follows recent line of inquiry extracting
2020: Ilbarco et al.) 2022: Shao et al.
task representations that arise in LLMs (Todd et al. 2024: Hendel et al. 2023 Saglam et al.) 2025) or 2024 Huang et al. (2024) they complete tasks. Most task representation extraction approaches fall into one of a few categories. Some treat the model's latent (residual) representation of a delimiter token some depth the task representation
VLMs (Luo et al.) 2024 Hojel et al.
Hendel
ct al. 2023 Luo et al. 2024 Li et al. (025). Another approach learns a per-layer task representation (Saglam al. 2025). Finally, Tood et al. (2024) combine activity from multiple attention heads across different depths and intervene with the resultant function vector at a single depth. We focus on
contrasting the representations that arise from different textual presentations of the same task.
5
Discussion
We empirically test for the presence of common, presentation-agnonn task representations in large language models. Our results suggest that different presentations of task do not elicit common task representation through function vectors. We extend the FV extraction procedure from in-context demonstrations to arbitrary task presentations and successfully construct instruction FVs that promote zero-shot accuracy. We find these convey different information than demonstration FVs, evidenced by the benefits of intervening with both forms of function vectors together over either by itself. We also find that mostly distinct (though partly overlapping) sets of attention heads causally mediate task
8

---

performance in these two settings, Our results offer support (in a limited, but controlled setting) for the widely used practice of prompting language models with both demonstrations and instructions. Our findings support two other takeaways. First, ICL. from demonstrations may be more inherent to L.MMs. We offer preliminary evidence that instruction task inference leverages attention heads that are peripherally useful for demonstration baased ICI.. The contrary effect is weaker; demonstration FV attention heads are less helpful for instructions. Second, we show evidence that instruction FVs transfer from post-trained models to steer their base versions better than base model instruction FVs.
We found the effectiveness of instruction-deriver FVs surprising. Given the diversity of plausible instructions for a given task, we would have considered it a priori likely that instructions induced relatively diffuse representations that would not support FV extraction from a few attention heads. Indeed, the representation of textual instructions appears more diffuse in language models, as evident from the distributions of attention head causal scores (Table 7] yet, our approach is successful.
The accuracy discrepancy when function vectors are evaluated in a setting incongruent with their extraction further supports the notion that task representations are dependent on the form of presenta- tion (compare Figure 2] anddd Figure 12). This evaluation form-dependency portends a challenge in identifying, monitoring, or amplifying task representations in naturalistic settings, such as prompts combining multiple demonstration forms, obfuscated or adversarially-presse benign presentations forms, such as role prompts or explanations (Lampinen et al.. 2024).
tasks, or other more
In our results, we average over the three uninformative baselines we propose: sampling tokens directly from the model, natural texts, and instructions from other tasks. While the selected top heads did not greatly differ between the three, one unlike the other two (Appendix H.3). Instructions from other tasks still convey the existence of a task, just different one, and may help localize task-specific information. Conversely, tokens sampled from the model and natural texts do not convey 'task-ness," hence helping localize the existence of a task in general. The differences between attention heads identified with each baseline alone may elucidate the differing roles of instruction FV attention heads.
We identify which post-training stages contribute to the ability to extract instruction FVs. However, our results do not explain what takes place in post-training that facilitates inferring tasks from instructions. |Wu al. (2024) propose that the primary drivers could be changes in processing words that convey instructions- which suggests the hypothesis that the instruction FV attention heads should strongly respond to these tokens. In another recent work, Yin and Steinhardt (2025) compare induction heads (Elhage et al.] (202)) ) and FV heads-do do instruction FV heads share anything in common with induction heads well? On that note, what we learn from the heads that change in importance between base and post-trained models? Our results point a spotlight at a set of heads relevant this capacity; future work may study their contributions and mechanisms in greater depth.
Our finding that we can steer base models with post-trained models instruction FV is consistent with recent findings by Stolfo et al. (2024) activation steering in an instruction followwig context. Our current results do not suffice to offer a potential savings in the effort required to post-train a model, as far, we only demonstrate the ability to steer a base model with its own post-trained variation. Recent results by [ee et al. (025)) propose a method for transferring steering vectors between models combined with our results, this suggests potential to confer some of the benefits of post-training on a model that has not been post-trained, a connection we hope to explore in future work.
Finally, we highlight few other proposals on how our work might inform prompt and intervention design. Our finding regarding the diffuseness of instruction FVs (subsection 3.3) implies that interventions focused changing improving instruction- moslluction-oolloiingg may benefit from intervening at multiple locations and layers: presently, most steering methods perform a single additive residual stream intervention, which may be more mcaningfully suboptimal for instructions, than for example, for demonstrations. The observation that demonstrations and instructions clicit different FV attention heads (also in subsecttoon 3.3) suggests a speculative potential pathway improving prompt design. In contexts where both instructions and demonstrations are provided, it may be possible to monitor their separate sets of implicated attention heads, and try classify from their activity whether or not the model successfully formed a task representation. This classifier could serve as a signal that helps guide toward improving the instructions or selecting different or additional demonstrations.

---

5.1 Limitations
Task set. We use a limited set of minimal, fairly simple tasks, following Todd et al. s (2024) choices. We omit a few classification tasks where predicting the correct token requires understanding the expected answer format, information conveyed clearly by demonstrations, but not necessarily by instructions. This could be addressed by combining demonstrations and instructions, by encouraging the cnstructionngeerratin it would be of interest to examine how these findings generalize to longer, more complex, open-ended, and naturalistic tasks, such as the ones present in MMLU (Hendrycks et al. 2020 or BBH (Srivastava)
model to include output formatting information. Furthermore,
et al. 2022 Suzgun et al. 2022
Choice of task representation. We study a particular proposal for task representations - function vectors as formulated by /Tod et al. (2024). Other task representation extraction approaches exist in the literature, such as task vectors (Hendel et al. 2023) we hope future work examines whether our findings translate to their formulation as well. More broadly, we cannot conclusively prove the negative, that common task representations do not exist; we can only demonstrate that our methods did not identify them. We also do not explore the working of attention heads identified in greater depth, such as identifying which circuits (Elhage et al., 2021) they may contribute to.
Arbitrary intervention depth. We follow Todd et al. (2024) in intervening at a fixed |L/3] depth. While we also present (in the appendices) results at empirically optimal intervention depths for each model, the choice retains a measure of arbitrariness. It is possible, for instance that different types of tasks are better served better by different intervention depths, and our analyses do not reflect that.
Model sizes. We focus our investigation on smaller (large) language models. While we offer fairly consistent evidence across the models examine, do not explore scaling with model size. We note that Todd et al. (2024) successfully extracted demonstration function vectors to Llama-2 models the 7B, 13B, and 70B sizes, which offers promising evidence for scaling function vectors.
For each model study, we evaluate each of the approximately 50 tasks six times shorter and longer instructions, each with each of the three baselines. We also ran the additional control conditions reported with all tasks for the alama-3.2-3B and Llama-3.1-88 models. However, we only replicate each model on each dataset in each setting once-given the qualitatively consistent results observed by TTodd et al. (2024), replicating over random seeds (used primarily in train-test splits and query example sampling) seemed untenable, if not wasteful. We use a single prompt template (Figure 9), relying on consistent previous results with different prompt templates (p. and Appendix C in Todd al. 2024).
Limited replications. We evaluate many experimental conditions (Appendix E.
Acknowledgments
We thank members of the Computation and Cognition lab at NYU and the Human and Machine Intelligence lab at Princeton for valuable feedback at various stages of this work. We thank Eric Todd, Eshika Saxena, and Nicola Cancedda for feedback on this project and manuscript. Author GD was funded by the Meta Al Mentorship program during his work on this manuscript.
References
Akyürek, E., Schuurmans, D., Andreas, J., Ma, T., and Zhou, D. (2022). What learning algorithm is in-context learning? investigations with linear models. arXiv [cs.LG].
Akyürek, E., Wang, B., Kim, Y., and Andreas, J. (2024). In-context language learning: Arhitectures and algorithms. arXiv [es.Cl.].
Berman, M. G., Park, J., Gonzalez, R., Polk, T. A., Gehrke, A., Knaffla, S., and Jonides, (2010), Evaluating functional localizers: the case of the FFA. Neuroimage, 5056)156-
Botvinick, M. and Plaut, D. C. (2002). Representing task context: proposals based on connectionist model of action. Psychol. Res., 66(4):298 311.
Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G.,
Askell, A., Agarwal, S., Herbert- Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D. M.. Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E.,, Litwin, M., Gray, S., Chess, B., Clark, J., Berner, C..
10

---

McCandlish, S., Radford, A.. Sutskever, I., and Amodei, D. (2020). Language models are few-shot learners. arXiv [cx. CLJ.
Chan. S. C. Y.. Santoro, A., Lampinen, A. K., Wang. J. X., Singh, A. K., Richemond, P. H., Mcclelland, J., and Hill, F. (2022). Data distributional properties drive emergent in-context learning in transformers. Neural Inf Process Syst, 1887
Chen, Y. Zhao, C., Yu, Z.,, McKeown, K., and He, H. (2024). Parallel structures in pre-training data yield in-context learning. arXiv [cs.CL]].
Chung. H. W., Hou, L., Longpre, S., Zoph, B., Tay, Y., Fedus, W., Li, Y., Wang. X., Dehghani, M., Brahma, S., Webson, A., Gu, S. S., Dai, Z., Suzgun, M., Chen, X., Chowdhery, A., Castro-Ros, A., Pellat, M., Robinson, K., Valter, D., Narang, S., Mishra, G., Yu, A., Zhao, V., Huang, Y., Dai, A., Yu, Petrov, S., Chi, E. H., Dean, J., Devlin, J. Roberts, A., Zhou, D., Le, Q. V., Wei, (2022). Scaling instruction-inetumee language models. arXiv [cs.LG].
Cohen, D., Dunbar, K., and McClelland, L. (1990). On control automatic processes: parallel distributed processing account of the stroop effect. Psychol. Rev., 97(3)333--361.
Conneau, A., Lample, G., Ranzato, M., Denoyer, I.., and Jégou, H. (2017). Word translation without parallel data, arXiv [cs.CL.].
Elhage, N., Nanda, N., Olsson, C., Henighan, T., Joseph, N., Mann, B., Askell, A., Bai, Y., Chen, A.,, Conerly, T., DasSarma. N., Drain, D., Ganguli, D., Hatfield-Dodke Z., Hernandez, D., Jones, A., Kernion, J., Lovitt, L..
Ndousse, K., Amodei, D., Brown, T., Clark, J., Kaplan, J., McCandlish, S., and Olah, C. (2021). A mathe- circuits huub/2021/// ffraee/r///////////
matical framework for transformer circuits. hettps:/transfooe Accessed: 2035-4-30.
index, html
Farrell, M., Recanatesi, S., and thea-Brown, E. (2023). From lazy to rich to exclusive task representations in neural networks and neural codes. Curr. Opin. Neurobiol. 83(102780): 102780.
Flesch, T., Juechems, K., Dumbalska, T., Saxe. A., and Summerfield, C. (2021). Rich and lazy learning of task representations in brains and neural networks. bioRxiv, page 2021.04.22 04.23.441122
Han, S., Song, J., Gore, J., and Agrawal, (2024). Emergence and effectiveness of task vectors in-context learning: An encoder decoder perspective. arXiv [cs.Cl.].
Hendel, R., Geva, M., Globerson, A. (2023). In-context learning creates task vectors. arXiv (es.CL).
Hendrycks, D., Burns, C.. Basart, S.. Zou, A.. Mazeika, M., Song. D., and Steinhardt, J. (2020). Measuring massive multitask language understanding. arXiv (Cs.CY).
Hemandezz, E., Sharma, A. S., Haklay, T., Meng, K., Wattenherg, M., Andreas, J., Belinkov, Y., and Bau, D. (2023). Linearity of relation decoding transformer language models. arXiv [cs.Cl.).
Hojel, A., Bai, Y., Darrell, T., Globerson, A., and Bar, A. (2024). Finding visual task vectors. arXiv [cx.CV].
Huang, B., Mitra, C., Arhelle, A., Karlinsky, I.., Darrell, T., and Herzig, R. (2024). Multimodal task vectors cnable many-shot multimodal in-context lcarning. arXiv [cs.CV].
Hummos, A., Río, F., Wang. B. M., Hurtado, J., Calderon, C. B., and Yang. G. R. (2024). Gradient-basee inference of abstract task representations for generalization in neural networks. arXiv [cs.LG].
Ilharco, G., Ribeiro, M. T. Wortsman, M., Gururangan, S., Schmidt, L., Hajishirzi, H., and Farhadi, A. (2022). Editing models with task arithmetic. arXiv [cs.LG].
Lampinen, A. K., Chan, Y., Singh, A. K., and Shanahan, M. (2024). The broader spectrum of in-context learning. arXiv CL)
Lampinen. A. K. and McClelland, L. (2020). Transforming task representations perform novel tasks. Proc. Natl. Acad. Sei. U. S. A., 117(52)32290-32299
Lec, A., Weber, M., Viégas, F., and Wattenberg, M. (2025). Shared global and local geometry of language model embeddings. arXiv CL).
Li, Y., Campbell. D., Chan, S. C. Y., and Lampinen, A. K. (2025). Just-in-time and distributed task representations in language models. arXiv [cs.CL).
Liu. Y.. Shi, K. Fabbri, A. R., Zhao, Y., Wang, P. Wu, C.-S., Joty. S., and Cohan, A. (2024). ReIFE: Re-evaluating instruction wollowiin evaluation. arXiv [cs. CLJ.

---

Llama Team, A. M. (2024). The Ilaaa 3 herd of models. arXiv [cs.Al].
Luo, G., Darrell, T., and Bar, A. (2024). Vision language models create cross-modal task representations. arXiv [cs.CV).
Lyu, X., Wang. Y., Hajishirzi, H., and Dasigi, P. (2024). HREF: Human response-guided evaluation of instruction following language models. arXiv [cs.CL].
Merity, S., Xiong, C., Bradbury, and Socher, R. (2016). Pointer sentinel mixture models. arXiv [cs.CL..
Min, S., Lyu, X., Holtzman, A.. Artetxe, M., Lewis. M., Hajishirzi, H., and Zettlemoerr L. (2022). Rethinking the role of demonstrations: What makes in-context learning work?
Mishra, S., Khashabi, D., Baral, C.,, and Hajishirzi, H. (2022). Cross-task generalization via natural language crowdsoureing instructions. In ACI.
Nguyen, K. A., Schulte im Walde, S., and Vu, N. (2017). Distinguishing antonyms and synonyms in a pattern-based neural network. In Lapata, M., Blunsom, P., and Koller, A., editors. Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguisties: Volume 1, Long Papers, pages 76 85, Valencia, Spain, Association for Computational Linguistics.
OL.o T., Walsh, P., Soldaini, I.., Groeneveld, D., Lo, K., Arora, S., Bhagia, A., Gu, Y., Huang, S., Jordan, M., Lambert, N., Schwenk, D., Tafjord, O., Anderson, T., Atkinson, D., Brahman, Clark, C., Dasigi, P., Dziri, N., Guerquin, M., Ivison, H., Koh, W., Liu, J., Malik, S., Merrill, W., Miranda, I., J. V., Morrison, J.. Murray, T., Nam, C., Pyatkin, V., Rangapur, A., Schmitz, M., Skjonsberg. S., Wadden, D., Wilhelm, C., Wilson. M.. Zettlemoyer. L.., Farhadi, A.. Smith, N. A., and Hajishirzi, H. (2024). 2 OLMo 2 furious. arXiv [cs.CL].
Olsson, C., Elhage. N., Nanda, N., Joseph, N., DasSarma, N., Henighan, T., Mann, B., Askell, A., Bai, Y., Chen, A., Conerly, T., Drain, D., Ganguli, D., Hatdeeld-Dodds, Z.,, Hernandez, D., Johnston, S., Jones, A., Kernion, J., Lovitt, I.., Ndousse, K., Amodei, D., Brown, T., Clark, J., Kaplan, J., McCandlish, S., Olah, (2022). In-context learning and induction heads. arXiv Lcs.LG].
Pan, J., Gao, T., Chen, H., and Chen, D. (2023). What in-context learning "learns" in-context: Disentangling task recognition and task learning, arXiv /cs. CLJ.
Park, C. F., Lubana, F. S., Pres, I., and Tanaka, H. (2024). Competition dynamics shape algorithmic phases of in-context learning. arXiv [cs.IGG..
Saglam, B., Yang, Z., Kalogerias, D., and Karbasi, A. (2025). Learning task representations from in-context learning. arXiv [cs.CL].
Saxe, R., Brett M.. and Kanwisher, N. (2006). Divide and conquer: defense of functional localizers. Neuroimage, 30(4):1088 discussion 1097-9.
Sclar, M., Choi, Y., Tsvetkov, Y., and Suhr, A. (2023). Quantifying language models' sensitivity spurious features prompt design or: How learnd to start worrying about prompt formatting.
Shao, N., Cai, Z., Xu, H., Liao, C., Zheng, Y., and Yang. Z. (2022). Compositional task representations for large language models. In The Eleventh International Conference on Learning Representations.
Srivastava, A., Rastogi, A., Rao, A., Shoeb, A. A. M., Abid, A., Fisch, A., Brown, A. R., Santoro, A., Gupta, A., Marriga-Alonso, A., et al. (2022). Beyond the imitation game: Quantifying and extrapolating the capabilities of language models. arXiv preprint arXiv:2206.0461)
Stolfo, A., Balachandran. V., Yousefi, S., Horvitz, E.. and Nushi, B. (2024). Improving fostruction-folloiingg in language models through activation steering, arXiv fcx. CLJ.
Su. J., Zhang, J., Ullrich, K., Bottou, L., and Ibrahim, M. (2025), A single character make or break your LLM evals. arXiv [cs.CL].
Suzgun, M., Scales, N., Schärli, N., Gehrmann, S., Tay, Y., Chung, H. W., Chowdhery, A., Le, V., Chi, E. H., Zhou, D.,, and Wei, J. (2022). Challenging big- tasks and whether chain-of- of-thuhh can solve them. arXiv preprint arXivv2210.0026..
Taori, R., Gulrajani, I., Zhang, T., Dubois, Y., Li, X., Guestrin, C., Liang, P., and Hashimoto, T. B. (2023). Stanford alpaca: An instruction-follee following llama model, https:///ithhb..cm com/ www.:/tst.ttteeeeeeeeee.. 1ab/ tab/stanfors ord. alpaca
12

---

Tjong Kim Sang, E. F. and De Meulder, F. (2003). Introduction to the CoNLL-2005 shared task: Language- independent named entity recognition. In Proceedings the Seventh Conference on Natural Language Learning at HLT-NAACL 2003, pages 142-147.
Todd, E., Li. M. L.. Sharma, A. S.. Mueller, A., Wallace, B. C.. and Bau, D. (2024). Function vectors in large language models. Proceedings of the 2024 International Conference on Learning Representations.
Touvron, H., Martin, L.,, Stone, K., Albert, Almahairi, A., Babaei, Y., Bashlykov, N., Batra, S.,, Bhargava, P., Bhosale, S., Bikel, D., Blecher, L., Ferrer, C. C., Chen, M., Cucurull, G., Esiobu, D., Fernandes, Fu, Fu, W., Fuller, B., Gao, C., Goswami, V., Goyal, N., Hartshorn, A.. Hosseini, S., Hou, R., Inan, H., Kardas, M.. Kerkez, V., Khabsa, M., Kloumann, I., Korenev, A., Koura, P. S., Lachaux, M.-A., Lavril, T., Lee, J., Liskovich, D.. Lu, Y., Mao. Y.. Martinet, X., Mihaylov, T., Mishra, P., Molybog, L,, Nie, Y., Poulton, A.. Reizenstein, J., Rungta, R., Saladi, K., Schelten, A., Silva, R., Smith,
M., Subramanian, R., Tan, X. E., Tang, B., Taylor, R., Williams, A., Kuan, X., Xu, P., Yan, 7., Zarov, Zhang, Y., Fan, A., Kambadur, M., Narang, S., Rodriguez, A., Stojnic, R., Edunov, S., and Scialom, T. (2023). Llama 2: Open foundation and line-tuned chat models. arXiv [cs.CL].
Von Oswald, J., Niklasson, E., Randazzo, E., Sacramento, J., Mordvintsev, A., Zhmoginov, A., and Vladymyrov, M. (2023). Transformers learn in-context by gradient descent. In International Conference Machine Learning, pages 35151-35174 PMLR.
Wang. Y., Kordi, Y., Mishra, S., Liu, A., Smith, N. A., Khashabi, D., and Hajishirzi, H. (2022a). Self-instruct: Aligning language models with self-generated instructions. arXiv [cs.CL].
Wang, Y., Mishra, S., Alipoormolahashi, P., Kordi, Y., Mirzaci, A., Arunkumar, A., Ashok, A., Dhanasekaran, A. S., Naik, A., Stap. D., et al. (2022b). sagerrrattraliintttttionsss Generalization via declarative instructions on 1600+ tasks. In EMNLP.
Wolf, T., Debut, L., Sanh, V., Chaumond, J., Delangue, C.,, Moi, A., Cistac, Rault, T., Louf, R., Funtowicz, M., Davison, J., Shleifer, S., von Platen, P., Ma, C., Jernite, Y., Plu, J., Xu, C., Scao, T. L.., Gugger, S., Drame, M.. Lhoest, Q. and Rush, A. M. (2019). HuggingFace's transformers: State-of-the-an natural language processing. arXiv [cs.CL].
Wu, X., Yao, W. Chen, J., Pan, X., Wang. X., Liu, N., and Yu, D. (2024). From language modeling to instruction following: Understanding the behavior shift in LLMs after instruction tuning. In Proceedings of the 2024 Conference of the North American Chopter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pages 2341 2369, Stroudsburg, PA, USA. Association for Computational Linguistics.
Xie, S. M., Raghunathan, A., Liang, P., and Ma, T. (2021). An explamation of in-context learning as implicit bayesian inference. arXiv [cs. CLJ.
Yang. R., Joglekar, M. R., Song, F., Newsome, T., and Wang, X.-J (2019), Task representations neural networks trained to perform many cognitive tasks. Nat. Neurosii, 22(2):297-306.
Yin, and Steinhardt, J. (2025). Which attention heads matter in-context learning? arXiv [cs.LG].
Zhang, R., Frei, S., and Bartlett, P. (2023). Trained transformers learn linear models in-context. Mach. Learn. Res., 25(49):49:1-49)
Zhou, C., Liu, P., Xu, P., Iyer, S., Sun, J., Mao, Y., Ma, X., Efrat, A., Yu, P., Yu, L., Zhang, S., Ghosh, G., Lewis, M., Zettlemoyer, and Levy, O. (2023a). LIMA: Less is more for alignment. arXiv [cs.CL).
Zhou, J.. Lu, T., Mishra, S., Brahma, S., Basu, S., Luan, Y., Zhou, D., and Hou, L. (2023b). Instruction-followin evaluation for large language models. arXiv [cs.CL].
13

---

NeurIPS Paper Checklist
The checklist is designed to encourage best practices for responsible machine learning research, addressing issues of reproducibility, transparency, research ethics, and societal impact. Do not remove the checklist: The papers not including the checklist will be desk rejected. The checklist should follow the references and follow the (optional) supplemental material. The checklist does NOT count towards the page limit.
Please read the checklist guidelines carefully for information on how to answer these questions. For each question in the checklist:
You should answer [Yes], [No], or [NA].
[NA] means either that the question is Not Applicable for that particular paper or the relevant information is Not Available.
Please provide short (1-2 sentence) justificaton right after your answer (even for NA).
The checklist answers are an integral part of your paper submission. They are visible to the reviewers, area chairs, senior area chairs, and ethics reviewers. You will be asked to also include (after eventual revisions) with the final version of your paper, and its final version will be published with the paper.
The reviewers of your paper will be asked to use the checklist as one of the factors in their evaluation.
While "[Yes] " is generally preferable to "[No]", ", it is perfectly acceptable to answer "[No] provided proper justification is given (e.g., "error bars are not reported because it would be too computationally expensive" or "we were unable to find the license for the dataset we used"). In general, answering "[No] or "[NA] is not grounds for rejection. While the questions are phrased in binary way, we acknowledge that the true answer often more nuanced, so please just use your best judgment and write justification to elaborate. All supporting evidence can appear either in the main paper or the supplemental material, provided in appendix. If you answer [Yes] to a question, in the justification please point to the section(s) where related material for the question can be found.
IMPORTANT, please:
Delete this instruction block, but keep the section heading "NeurIPS Paper Checklist",
Keep the checklist subsection headings, questions/answers and guidelines below.
Do not modify the questions and only use the provided macros for your answers.
1. Claims
Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope?
Question:
Answer: [Yes]
Justification: We are careful to limit the scope of claims in the abstract and introduction to the experiments and evidence in the paper.
Guidelines:
The answer NA means that the abstract and introduction do not include the claims made the paper.
The abstract and/or introduction should clearly state the claims made, including the contributions made the paper and important assumptions and limitations. A No or
NA answer to this question will not be perceived well by the reviewers.
The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings.
It is fine to include aspirational goals as motivation as long as is clear that these goals are not attained by the paper.
2. Limitations
Question: Does the paper discuss the limitations of the work performed by the authors? Answer: [Yes]
14

---

Justification:
We discuss the limitations of our work in Subsection 5.1
Guidelines:
The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper.
The authors encouraged to create separate "I "Limitations" section in their paper.
The paper should point any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model sele--ppeiiiainnn. asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be.
The authors should reflect on the scope of the claims made, e.g., if the approach was only tested few datasets with few runs. In general, empirical results often depend on implicit assumptions, which should be articulated.
The authors should reflect on the factors that influence the performance of the approach. For example, facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails handle technical jargon.
The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size.
If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and faimess.
While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an impor- tant role in developing norms that preserve the integrity of the community, Reviewers will be specifically instructed to not penalize honesty concerning limitations.
3. Theory assumptions and proofs
Question: For each theoretical result, does the paper provide the full set of assumptions and complete (and correct) proof?
Answer: [NA]
Justification:
We do not provide any theoretical results.
Guidelines:
The answer NA means that the paper does include theoretical results.
All the theorems, formulas, and proofs in the paper should be numbered and cross- referenced.
All assumptions should be clearly stated referenced in the statement of any theorems.
The proofs can either appear in the main paper or the supplemental material, but they appear in the supplemental material, the authors are encouraged to provide short proof sketch provide intuition.
Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix supplemental material.
Theorems and Lemmas that the proof relies upon should be properly referenced.
4. Experimental result reproducibility
Question: Does the paper fully disclose all the information needed to reproduce the main ex- perimental results the paper to the extent that affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?
Answer: [Yes]
Justification:
We provide thorough descriptions of the methods we use (section and
Appendix B] example prompt templates in Appendix A list of the models we evaluate in
Table 4] and a list of the datasets we use Appendix D We will also release the code, and
all the data we use is available as we reuse the datasets provided by Todd et al. (2024).
15

---

Guidelines:
The answer NA means that the paper does not include experiments.
If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not.
If the contribution is dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable.
Depending on the contribution, reproducibility can be accomplished in various ways. For example, if the contribution is novel architecture, describing the architecture fully might suffice, or if the contribution a specific model and empirical evaluation, it may be necessary either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g.. in the case of large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed.
While NeurlPS does not require releasing code, the conference does require all submis- sions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. For example
(a) If the contribution is primarily new algorithm, the paper should make it clear how to reproduce that algorithm. (b) the contribution is primarily a new model architecture, the paper should describe the architecture clearly and fully.
(c) If the contribution is new model (e.g.. a large language model), then there should either way to access this model for reproducing the results or way to reproduce the model (e.g., with an open- open-source dataset or instructions for how to construct the dataset).
(d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility. the case of closed-source models, may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers have some path reproducing or verifying the results.
5. Open access to data and code
Question: Does the paper provide open access to the data and code, with sufficient instruc- tions to faithfully reproduce the main experimental results, as described in supplemental material?
Answer: [Yes)
Justification: We provide a zip archive our code with instructions for how to set up the environment and example commands to launch our experiments.
Guidelines:
The answer NA means that paper does not include experiments requiring code.
Please see the NeurIPS code and data submission guidelines (https://nips. cc/
for more details.
While we encourage the release of code and data, we understand that this might be possible, so "No" is an acceptable answer. Papers cannot be rejected simply for not including code, unless this is central to the contribution (e.g., for new open-source benchmark).
The instructions should contain the exact command and environment needed to run to reproduce the results. See the NeurIPS code and data submission guidelines (https
/ ptaips.cc//uulic//uuu
for more details.
The authors should provide instructions on data access and preparation, including how to access the raw data, preprocessed data, intermediate data, and generated data, etc.
The authors should provide scripts to reproduce all experimental results for the new proposed method and baselines. only subset of experiments are reproducible, they should state which ones are omitted from the script and why.
16

---

At submission time, to preserve anonymity, the authors should release anonymized versions (if applicable).
Providing as much information as possible in supplemental material (appended to the paper) recommended, but including URLs to data and code is permitted.
6. Experimental setting/details
Question: Does the paper specify all the training and test details (e.g., data splits, hyper- parameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?
Answer: [Yes]
Justification: We provide experimental details in Appendix E and the code to experiments.
Guidelines:
The answer NA means that the paper does not include experiments.
The experimental setting should be presented in the core of the paper to level of detail that is necessary appreciate the results and make sense of them.
The full details can be provided either with the code, in appendix, or as supplemental material.
7. Experiment statistical significance
Question: Does the paper report error bars suitably and correctly defined other appropriate information about the statistical significance of the experiments?
Answer: [Yes]
Justification: We report standard errors of the mean (SEMs) on all figures in the paper that estimate average performance in some condition across of conditions. We do not report statistical significance results as did not seem material for the comparisons we make, but are open to doing should the reviewers request.
Guidelines:
The answer NA means that the paper does not include experiments.
The authors should answer "Yes" if the results are accompanied by error bars, confi- dence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.
The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, overall run with given experimental conditions).
The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.)
The assumptions made should be given (e.g.. Normally distributed errors).
It should be clear whether the error bar is the standard deviation the standard error of the mean.
It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have 96% CI, if the hypothesis of Normality of errors is not verified.
For asymmetric distributions, the authors should be careful not show in tables figures symmetric error bars that would yield results that are out range (e.g. negative error rates).
If error bars are reported in tables or plots, The authors should explain the text how they were calculated and reference the corresponding figures or tables in the text.
8. Experiments compute resources
For each experiment, does the paper provide sufficient information on the com- puter resources (type of compute workers, memory, time of execution) needed reproduce the experiments?
Question:
Answer: [Yes]
17

---

Justification: We attempt to ballpark the number of experiment runs required to generate our results (and hence, reproduce our work) in Appendix E.
Guidelines:
The answer NA means that the paper does not include experiments.
The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage.
The paper should provide the amount compute required for each of the individual experimental runs well estimate the total compute.
The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make into the paper).
9. Code of ethics
Question: Does the research conducted in the paper conform, in every respect, with the NeurlPS Code of Ethics https: http://neerpppppp http///pbblcc..ccccc...eeellllll
Answer: [Yes]
Justification: Our work conforms with the NeurlPS Code of Ethics in its entirety.
Guidelines:
The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.
If the authors answer No, they should explain the special circumstances that require deviation from the Code of Ethics.
The authors should make sure to preserve anonymity (e.g.. if there is special consid- eration due to laws or regulations their jurisdiction..
10. Broader impacts
Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?
Answer: [NA]
Justification:
We do not foresee direct societal impacts of the work performed.
Guidelines:
The answer NA means that there is no societal impact of the work performed.
If the authors answer NA or No. they should explain why their work has no socictal impact why the paper does not address societal impact.
Examples of negative societal impacts include potential malicious unintended uses (e.g., disinformation. generating fake profiles, surveillance), fairness considerations (e.g.. deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations.
The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster.
The authors should consider possible harms that could arise when the technology is being used intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional unintentional) misuse of the technology.
If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how system learns from feedback over time, improving the efficiency and accessibility of MI.).
11. Safeguards
18

---

Does the paper describe safeguards that have been put in place for responsible release of data models that have high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?
Question:
Answer: [NA]
Justification:
We provide no data models that have a high risk of misuse.
Guidelines:
The answer NA means that the paper poses no such risks.
Released models that have high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere usage guidelines or restrictions to access the model or implementing safety filters.
Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images.
We recognize that providing effective safeguards challenging, and many papers do not require this, but we encourage authors to take this into account and make best faith effort.
12. Licenses for existing assets
Question: Are the creators or original owners of assets (c.g., code, data, models), used in the paper. properly credited and are the license and terms of use explicitly mentioned and properly respected?
Answer: [Yes]
Justification:
We cite and refer the original creators code, data, and models we used.
Guidelines:
The answer NA means that the paper does not use existing assets.
The authors should cite the original paper that produced the code package dataset.
The authors should state which version of the asset is used and, if possible, include URL.
The name of the license (e.g., CC-BY 4.0) should be included for each asset.
For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.
If assets are released, the license, copyright information, and terms use in the package should be provided. For popular datasets, paperswi thcode. com datasets has curated licenses for some datasets. Their licensing guide can help determine the license of dataset.
For existing datasets that re- re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided.
this information not available online, the authors are encouraged to reach out to the asset's creators.
13. New assets
Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?
Answer: [Yes]
Justification: We document the code we provide in the supplementary materials, and will further improve the documentation before open-source code when the work is de-anonymized.
Guidelines:
The answer NA means that the paper does release assets.
Researchers should communicate the details the dataset/code/moodll as part their submissions via structured templates. This includes details about training, license, limitations, etc.
The paper should discuss whether and how consent was obtained from people whose asset is used.
19

---

At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file.
14. Crowdsourcing and research with human subjects
Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well details about compensation (if any)?
Answer: [NA]
Justification:
We report no crowdsourcing or human experiments.
Guidelines:
The answer NA means that the paper does involve crowdsourcing nor research with human subjects.
Including this information in the supplemental material is fine, but if the main contribu- tion of the paper involves human subjects, then as much detail as possible should be included in the main paper.
According the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector.
15. Institutional review board (IRB) approvals equivalent for research with human subjects
Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?
Question:
Answer: [NA]
Justification: We report no human subjects research.
Guidelines:
The answer NA means that the paper does involve crowdsourcing nor research with human subjects.
Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.
We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurlPS Code of Ethics and the guidelines for their institution.
For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.
16. Declaration of LLM usage
Question: Does the paper describe the usage of LLMs if is an important, original, or non-standard component of the core methods in this research? Note that if the LLM is used only for writing. editing, or formatting purposes and does not impact the core methodology, scientific rigorousness, originality of the research, declaration is not required.
Answer: [Yes]
Justification: Our work describes an evaluation of LLMs across a variety of tasks in attempt extract task representations. We also used LLMs to generate tests for researcher-written code and autocomplete using GitHub Copilot.
Guidelines:
The answer NA means that the core method development in this research does not involve LLMs as any important, original, non- standardd components.
Please refer to LLM policy (https://neurips http://ipss.peeooeeeeenen/////// for what should or should not be described.
20

---

A Prompt generation prompt templates
Bertart_headr____ [beader_id/>ussr([[ www.hheadrrllll
Instructions
You are powerful model helping write prompts to help smaller models perform tasks better. Below, you will be given set input-output pairs for a particular undescribed task. First, please study the examples to deduce what the task is, and describe your thinking under the header "# Task Deduction". Next, please write 10 prompts that might help a smaller model perform this task. The prompts should be:
1. Short, up to 10 words.
Informative about what the task is.
Not repetitive with each other.
Please write your prompts under the header "# Task Prompts".
Task examples [task examples]
Now, think step by step and follow the instructions above. <leot_id|><Istarrr starr///ssaattriti header. /asssistantk//end_eeeeee
Figure Prompt for generating short task instructions given ICL examples. The prompted model (in experiments, Llama-3.1 3.1-B05B) must analyze the examples given under the header "task examples" and generate instructions for another model. We provide no information beyond the in-context examples, and repeat this procodure 20 times to generate a set of approximately 200 instructions for cach task, which we then deduplicate. In practice, the vast majority of these short instructions tokenized to 16 fewer tokens, we set that as the limit for what considered short instruction.
< [tstrt_header____ @header_id]aale lndelluuedeedeeeeeeeeeee end_ header idd> Instructions
You are powerful model helping write prompts help smaller models perform tasks better. Below, you will be given a set of input-output pairs for a particular undescribed task. First, please study the examples deduce what the task is, and describe your thinking under the header "# Task Deduction". Next, please write 10 prompts that might help smaller model perform this task. The prompts should be:
1. As long necessary to be helpful for the smaller model.
2. Informative about what the task is.
Not repetitive with each other.
Not including any examples of the task.
Please write your prompts under the header "# Task Prompts".
Task examples (task examples)
Now, think step by step and follow the instructions above. start htwp_eeeeeerreeeeeee idd
Figure Prompt for generating longer task instructions given ICL examples. The prompted model (in experiments, Llama-3.11408 must analyze the examples given under the header "task examples" and generate instructions for another model. We provide no information beyond the in-context examples, and repeat this procodure 20 times to generate a set of approximately 200 instructions for cach task, which we then deduplicate. In practice, the vast majority of these longer instructions tokenized to 64 or fewer tokens, set that as the limit for what we considered long instruction.
A.1 Example generated instruction prompts
21

---

Table Example short instructions for five tasks.
Task
Short Instructions
Create opposing term
Identify the antithesis of this word Create counter-term
antonym
Reverse the semantic meaning Provide a word that the semantic opposite
Country to capital city correlation Learn country-capital associations Map country names to their capitals Tdentify the administrative center Provide the capital city for the given country
country-capital
Select the word that is not a noun "Find the word that is not concrete object." Select the word that tells more about something Which word has distinct semantic meaning?
voncept_v_object object__5
Identify the adverb or adjective the list
Spanish equivalent for this English Translate everyday English words to Spanish Spanish translation of English word Find Spanish counterpart for English word Find Spanish translation
english-spanis
"Associate product name with company name" Which company created this software?
product-company
"Classify product by owner company"
Identify the company that developed this technology "Link this device to its manufacturer."
22

---

Table 3: Example long instructions for five tasks.
Task
Long Instructions
Find word that, when compared to the input word, presents contrasting meaning. This word should highlight the differences and serve as antonym Generate word that cancels out the meaning of the input word **Mcaning reversal**; Reverse the meaning of the input word by generating a word that represents its opposite. Ensure that the generated word is semantically accurate and contextually relevant This task tests the ability to navigate the vocabulary of a language to find and generate antonyms. Please focus on producing words that are directly opposite clearly contrasting
antonym
**Find word that contrasts with the input word meaning. ss This could involve finding a word that is the opposite of the input word or one that describes a different extreme or end of a spectrum
What the name the city where a country's president or monarch typically resides and conducts official business?
Determine the capital city of a country by identifying the city where the national government seated and where major political decisions are made Provide the name of the city that generally accepted as the capital of a particular country What city is recognized the center of administration and governance for given country?
country-capital
"Countries around the world each have a capital city where their government. is based. Your task is to know what these cities are for any country you asked about."
Determine the word in the list that is verb or action Identify the word in the list that describes quality, property, characteristic of something
Identify the word in the list that describes quality or property of something **Deermmine the Quality Word Determine which word from the list describes a quality, state, condition. This word should tell about the attributes something Find the word that can be used in sentence to describe an action, event, or situation
voncept_y_obectt_5
Translate the English word into Spanish, making sure to use the most appropriate and commonly used term in Spanish-speaking contexts
Provide Spanish translation of the input word that both accurate and fluent Translate input word from English to Spanish, considering any relevant context or connotations
english-spanish
Translate the given English word into its equivalent in Spanish, ensuring maintain the original meaning and word type (noun, verb, adjective, etc.) Identify the Spanish equivalent of the provided English term, ensuring the translation is accurate and suitable for the context
Identify developer given operating system, platform, or tool Given the name of a product, technology. or format, find the company that owns developed it. Use your knowledge of industry leaders and their offerings the product, tech-
Identify the company organization that developed nology, or format specified the input Identify the company that created this file format
product-company
Determine the company that is associated with the specified brand, product, format
23

---

B Uninformative instruction baselines
B.1 Additional uninformative instruction details
Equiprobable token sequences. Given task insominforeetive instructions gm our goal is to sam- ple an uninformative instruction The We encode 9'm a token sequence am [W], W2, ... wz] whose probability under the model f is P(am) = I P(w, wel) = f(w<i)(m)) We be- gin from the BOS token, and sequentially sample wi to create In [wi, W2,," (z). We do so by approximately matching the conditional probabilities f(w)<r](uy) = [(wrr][w]] using the following logic. First, we mask out any non-text ("added vocabulary") tokens. At each step, compute the (log-) probability of the I'th token, log P(w. wex)- We compute the distribution over next tokens of the uninformative instructions, log P(. we()- We set initial threshold and a threshold increment (in our experiments, both were set to 0.1). We increase the thresh- old by the increment until at least one token in log P(- w<i) falls within the incremented threshold of the log-probability of the current instruction token. Denote by t our initial thresh- old, At our increment, and k E Z' the number of increments required, we find the following: ming: (Enz 1[t - kåt Z log PP(w | w<a) log P( weri S + k > O, where 1[-] de- notes the indicator function. We then sample uniformly between all ei satisfying the previous relation and append the sampled dy to the o'm we are constructing, continuing until we reach the same length as the informative q..
Real texts. Ahead of time, we precompute the log-probability of texts from WikiText- 1iki-ext-103-11 (Merity et al. (016)) with each model f. We randomly sample entries from the WikiText- -001xe dataset. For each sampled entry, we extract overlapping prefixes that end with whitespace (excluding the terminal whitespaces), tokenize these prefixes, and keep those with a length of 64 or fewer tokens. We extract prefixes to create strings that might coherently appear at the beginning of a text. We then compute and cache the (log-) probability of each such prefix and its token sequence. We cache approximately 216 = 65536 token sequences with each model (approximately as we stop after the Wikitext entry that brought over the threshold, but do not discard sequences beyond the 2 th one).
To sample an uninformative orm for some qin we tokenize d'n to arrive at its length L(g'n) and compute its log- pog-robabiiliyy log (g We now create representative sample of N 100 cached texts with approximately the same length as L(qm). We begin by only consider texts of with a length of precisely L(g'.) tokens. If there are over N of those, we stop; otherwise, for 1e E Z+, we also consider texts of length L(ghm) + k. increasing k by one until we have set of at least N candidates. Once we have attained this set of candidates, we select the Z.n, with the closest log-probability under f as d' has, and remove it from the set. As we require five uninformative baselines for each instruction a'm (see (pppendi E, we will use the five real texts with the closest log-pro prrobabbbiiyy to 9'n subject to being within a small number of tokens from L(g(..)
Other task instructions. We follow a conceptually similar procedure to the one described above for real texts, but using instructions generated for other tasks. As this is smaller set, we do not cache these log probabilities in advance. We exclude instructions generated for the same task, but otherwise follow an identical procedure. Denote by L(g(,) the length of the instructions d'm. We identify a set of candidate alternative instructions with a length of approximately L(g'n). increasing the acceptable difference in length until we reach N 100 candidates. We then return the with the closest log-probabilie to o' under the model and remove it from the set, so as before, we will use the five other task instructions with the closest log-probabilityy to d subject being within small number of tokens from L(g..).
B.2 Prompt baseline examples
24

---

Taxt
Equiprobable tokons
texts
Other tack instructions
antlnggm
fnge mample -a0bbgellppr Info
normul gar uxchurge
Prcvkie the spcellle currency
Change to post state
adult cattle agret
identilicatio
Country to caploal city correttiion
country-cane capital
KERDwnors APP ViLearn (1110)t O Allen 1:ls "Nay Allmnn
A major Gorman defensire position 0 staff resleer for
DWiic Groce Devi
Create a plutai fron 7h0
English German dictionary lookup
Translanc a cumparabee torn
Select the ward that is notid nour
conoept, obfeett
resutsfstarte Lchar Codeei erumosilopcss Jub SALM alaeited Stnuit epy slaissc koinder noalt dil owk
TThe parish Et Ba tholomew
semned Bachelor Science
Find the cad wende
Catculate lata the number of cheraterrs
among there
The irmediatte post-wWordd war ero
iteenity ty the word that dosen
mark 1i000100000000 envitonmental
Capture letter of gleen string
sparish houleat for this öngilan tenn
implish-spanish
oepoive Serum into hot spume
notice
However, the riiid acvances made
your, condie
mestiooed
Contert Engi iih terin to German equivalen:
orger
product-esmpany companty damp"
-oppeandding -dahhboard oustomer 10s vaa Forsmus
state, Artile Leboyy unable check effect
Like Jeffeeson and Adams
Teller and
idennity the leading verb in wo groups
rice Resetion another.] stices
color from inpurt
byttonayynn c DockMggc
"Pippoint the location's country"
Figure 7: Example generated baselines for short instructions. For each of the five tasks we visualize example we select the first instruction and sample three uninformative matches for it with
instructions from in Table 2 each baseline type.
Task
Equiproboble tokens
Roal texts
Other task Instructions
Chesenes Portoole le Question i. todassktaaktie Lausss crnvanget Mast Other Escumentraav
An on August 2.O
<*Three Words, One Arswartt: You, be given threw words, baok
Company. Battalion,
aid word that when compared ed to the ixput word presents a cocouating meaning. 70k2 serve an ansuryin
antenym
Rays onto [SQUsstrighh Se vo chding
StotcGrrppiis IcBowricad npeccin
allen|]Foou de vive
crushed Birs EN pporsin
chiral locoah symmatry
loosahecral nhar-speriiall wo
Moot antmaal viruses are
from
until you reach the and The letter
"When given a word, scan through it
very
need
pHRACCOUNT
cataloguen nude raven @80
pho.ssh.ceertte onwistalloger
potitics
mumber of the househald ef
Themes Candi lupe, who want
1254 swinefiekd
selech the irom that
with rilx antina s and frults,
with *Soolect the Fru Pruitet O ven a list
and
Urban.coords Neaoletter conventigg
In 2009. Madsen amnounoed
Datermine which word
Impro page aliing BACK Ao Speech gamiing Sike
charmenn,
physical
flodd @ o Web Adaaterr thetone
and rep replaced
internctedd with direct
country -cupital
paa carpp unerraaia Epg Gen
- -oorts and liis may be viewed
Raatiggua the wordl thai is most kkaly an otjeet that can
What name the city where country' consuctte official ousinees?
special cases crches a paint
typically
shoan, propectton numberss Alcc Royolih treeteetto to avirattoncll Pupper @ tashng acditional Econom lin decermnno
seen,
Refiejo (Engiish My
Asply rules Eng grammnrr
beflection a the second studio
to convert the given vero from its
Clusses "juuter caption afficacy uppr cnimat on twing
albam and Spawiish wibure by
base form connect past lervee
Estract the first e cheracter from a word, regardiczs longth
Errefacand/dFan luPis crop cost garant
concept. - X_object__
Deteriine the mor in the lat that a sub or action
transported okay human lirs physique ned Attello
Wissteel o0d Modomtitt subtilic @ Lt
(prCaw Golle booleed sorsen resiled dimensions
minin
marmaid
Amids a Amiideia
- hara le Portran as Queen Pacméé
fieicher
Avermaet team't
pealion
lettee that ccouples the second
the adiersstratvve and pollocal center of country
Takee word impat and curpun the
Specily
farctions
parcon -rng- artinoon ( /eegreesionn Blggh Traftic: ounce 1" DowningttHeetll DER_A)
From the and of the your 1955
-ddendlyy the official eu rency used by the glaen countrs bnsuring to 4E17 code
english-sparian
making sure use the most appecortete and cammonly used twon Spanss quesin
Thanslate lste the English pard into Spenkit
Mirowo creer she in rucied daycare, CONFIG Stuches roppeeille i logh "aavel iinrica to Association ohth month Mundo tasty
AutoorPecaaggle enossio
Resehved
Fubric
to early 1956. Ham ingiway was
neot two aplaodes of TNA Impact I
"NA held ast of tapings for the
You are tasked with finding the start inn watrr a word, ltts the lettor
May
Deellliger
ira worch promur clation the one that most cleaely related to specific object, place, or thing
histor ban Customer Spaitiin bbdde eig on
of the Impertal German
thene torteorrieee German
bu bugierrforestt Leaderchol Education none cemanently foove abs idaat mit grandes hoth vue
In Septenbe 2008, Miller
idderiilyy the entilly name thar
part cinated in the 20001 Summer
enceddded ganng ophical
extrect
product-commpany idornlly the aevelsper of given operering systom oietoorss or tool
Reamm Mulsi official managemanter(- SuperviowToRuee wowoooowoooeee
gethame Meeting Com nicatioonaar
thone previcustyy disbetee
may have
lecation of
el thin the United States
abccerigh perferming brunch image Force
expe sperieecce
Several of the cast members had
prior
Transatee on English word into
maintaining original
nottation
Figure 8: Example generated baselines for long instructions. For each of the five tasks we visualize example instructions from in Table 3 we select the first instruction and sample three uninformative matches for it with each baseline type.
C Models studied
25

---

Table 4: Models studied. We use the Huggingface Transformers (Wolf et al. 2019) model implementations.
Model
Citation
Huggingface ID
[ap]
Llama-3.24 3.2-1B
Llama
Team
2024
neta-1lama/Llaaa-33
16
32
Llama-3.2- 3.2-IB-Instrue
Llama Team
2024
neta- neta-Hamm/Lisaa-3 Instrrct
32
1lama-3.2-33
Llama Team
2024
28
24
Llama-3 3.2-3B-Intttuct
Llama laam
2024
neta-11aaa/Liasa33...1 Instruct
28
24
Llama-3.1-88
Llama Team
2024
neta-11aaa/Liamaa..111a
32
32
Llama-3.1-8B-Instrume
Llama leam
2024
nata-11aaa/Limm-33..--gg Instruct
32
OLMo-2-1124-11
OLMo et al.
12024
allecaa/00MM--221 1
32
- 2-1122-7B-SFT
OLMo ct al.
2024
allecai alleexa/00Moo22111
32
OLMo OLMo-2-1124-11 1-1-------
OLMo et al.
2024
32
32
OLMo-2-11222 OLMo-2-11224-7B-nnstuuuu
OLMo et al.
2024
32
32
Llama-2-7b
Touvron et al.
2023
mata-1lans/Llaaaa-2
32
32
Llama Llama-2-7b-oa 2-7b-chat
Touvron ct
neta wla.naaaaniiii 2aaee/aaaaaaa--mmm 2-bh-chat- hff
32
D Full list of tasks
26

---

Table 5: Tasks used.
Task
Citation
adjective_v_ serccttivev_vrrbbb
Todd
2024
adjective_y_verb.
Todd et a.
2024
alphabeticalll_ffist
Todd et al
2024
alphabetically first
Iodd
2024
alphabetically last
lodd
20224
alphabetically last_
Todd el al
20244
animal_v_objee
Todd et al
2024
animal_v_oo v_ojectt
Todd et
20224)
antonym
Nguven al. [2017
capitalize
lodd cl al
2024)
capitalize_fing firsst_lltttet
'add et al.
20124
capitalize_last_eeee
Yin and Steinhardt
2025
capitalize_secomee letter
Yin and Steinhardi
2025
choose first of choose first of choose last of 3 laaoosc_llst________
Todd
2024
lodd el al
2024
Todd et al
2024
Todd et al.
2024
thoose_middle_one _f__
Todd
choose_middle _f__
Todd ct al
2024
aalor_v_animal
Todd el a1 2024
color color_v_anne _____mal_a_____
Todd et al.
2024
concept_v_one concept_v_objett_3
Todd
2024
concept_v_obeect
lodd
20244
conll2003_location
Tjong Kim Sang and De Meulder
2003
2aal20003_rgaanzztiinn
Jong Kim Sang and De Meulder
2005
conl120033_perso
jong Kim Sang and De Meulder
2003
country-capital
Todd (0224)
country-currency
Tadd al.
20224
english-french
Conneau el al.
2017
english-german
Conneau el al.
201
english-spanish
Conneau
2017
fruit_v_animal_3
at
fruit_v_animat
Todd ct al
20:44
landmark-counte
Hemaandz ct al.
2023
lowercase Lowercas_ffist__. letter
lodd al.
2024
lowercase Laarrcase__ss______e lest__etter
Todd
2024
national _pakks
Todd et
20224
next cexital letter
Todd el al
2024
next next_iten item
Todd et al
2024
object_v
Todd
2024
ohject_v_concept
Todd
park-country
Todd ct
2024
present-past
Todd et a
2024
item
Todd et al. 2024
product-company
Hernandez al.
2023
singular-plural
lodd et al. (2024
synonym
Nguyen ct all
22017
crrb_v_adjective_3
lodd et al
2024
verb cer__v_adecctie___
Todd
2024
word leng_l
Todd
All tasks used were sourced from Todd al. (2024) repository:
https://githuu com/
seroodd/funciin.. vectors
We omitted the following tasks, as they are classification tasks with specific output formats that the model model-generated instructions often did not specify: ag news, commonsense. qa. personninstrument, person- occupation, person-sport, sentiment
27

---

E Full experimental settings
We detail our experimental settings to aid reproducibilityy As guideline we strive match or minimally adapt decisions made by [Tod et al. (2024):
For each model and each task, the J instructions with the highest accuracy over the training split.
We compute the mean activations over 100 total prompts, 20 with each of the 5 best instructions.
We compute the causal indirect effects over 25 total uninformative prompts, 5 generated for each of the best instructions.
We batch our results with a batch size that depends the model and task, but does not exceed for any model or task (see code batch size computation logic).
We split each dataset 70% train and 30% test. Where we require validation set, we split it again from the training set.
We load all models in full precision.
We use the |A| 20 top heads in all experiments we report.
We evaluate the FV interventions every possible depth (that is, after every layer of the model).
In all main manuscript figures, report the accuracy intervening after the |L/3] layer ( (layer for the Llama-3.2-3B models and layer 11 for the Llama-3.1-8B and OLMo-2- 1223-BB ones).
In appendix figures that report the empirically optimal intervention layer, we compute the accuracy using the layer that would result in the highest mean accuracy for each model, averaging over both the 0-shot and shuffled 10-shot evaluations, separately for demonstrations and instructions.
When we intervene with two function vectors (either demonstrations and instructions, or twice with one), sweep over the range [LT/4], [T/22]] (as the optimal intervention depths for all models fell this range.
In the main manuscript figure, we report the accuracy intervening with both additively at the L/3] layer.
In appendix figures showcasing the empirically optimal layer(s), we follow the same process described above.
Error bars we report in all figures are standard errors of the mean, averaged within each model.
We use the following query template in all of our instruction-basse experiments:
<instructions>
Q: <Zig>
A:
Figure 9: Instruction query template. We use the query template proposed by Todd et al. (2024) and prepend the instructions to it.
And this query template in our demonstration- ones:
28

---

Q: <T,
A: <yn
Q: <0;2?
A: <y=??
Q: <h:10>
A: <y/10?
Q: <Rig?
A:
Figure 10: Demonstration query template. We use the query template proposed by Todd et al. 2024
E.1
Experiment compute resources
Before running the main set of experiments, we rum few preliminary steps. We generate instructions (Appendix A) 20 times for each of the z 50 datasets, generating shorter and longer instructions separately, resulting in approximately 2000 invocations of the stronger model used for instruction generation (in our experiments, Llama-3. 1-405B). We also cache the log-proo hogg-robabilitees of texts from MikiText-103-v/ which takes an hour or two with each model (Appendix B)
For each of the 12 models we consider (Table 4), and each of the 50 tasks, we begin by running the 'training' job that computes mean aask-conditioned activations and estimates head causal effects. We do so with shorter and longer instructions, using each of our three uninformative baselines, yielding roughly 12 (models) x 50 (tasks)
(instruction lengths) 3 (baselines) 3600 jobs. With these jobs behind us, we can compute the overall top instruction FV heads for each model. With those, we can evaluate each model with the function vector constructed using the overall top heads, evaluating an intervention at every layer in both the zero-shot and shuffled 10-shot evaluations. We run each evaluation separately using the mean activations with the best performing long and short prompts, but with the heads identified averaged over the causal scores from both, and average the final results over these two mean activations.
Most of our additional experiments only require these final evaluation jobs:
Intervening with both function vectors ($3.2: Figure 2B)
Intervening with the same function vector twice (a control for $3.2; Figure 18)
Constructing 'incongruent" FVs with top heads identified by demonstrations and mean activa- tions from instructions (or vice versa; $3.4: Figure 3G-H).
Constructing function vectors using the least important overall heads and the bottom heads (a control for F3.4:Figure 30)
Steering base models with post-trained model instruction FVs ($3.5: Figure 4A).
Many of these we only run for the four models we focus our investigation on (L.lama-3.2-33) Llama- 3.n-tut-n Llama-3.1-8B. Llama-3. I-SB-Instruct), However, all of these run for approximately 50 datasets, using both short and long instruction mean activations. We conservatively estimate these evaluations required another 50 (tasks) (instruction lengths) (models) 10 (additional experiments) = 4000 experiments.
In addition, report. As these only have one variant, they contribute only another 12 (models) 50 (tasks) = 600 experiments so.
the demonstration ICL extraction procedure and evaluation all models
We run all our experiments on Volta and Pascal-series GPUs, with single GPU sufficing for every experiment we launch. Experiment wall-clock time varied drastically by the model, the size of each
29

---

task's dataset, and and the lengths of the data points in each tasks; however, all were on the order of hours, not days.
F Baseline and 'skyline' accuracies
Table 6: Baseline and 'skyline'
accuracies model.
Model
10-shot
Shuftled 10-shot
Best instruction
Top-5 instructions
0-shot
L.ama-3.2-3B
0.7531 0.0205
0.1536 0.0159
0.7654 = 0.0225
0.7105 = 0.0227
=
Llama-3.233B-nstruu
0,7895 0.0173
0.1858 0.0154
0.8638
0.8330 0.0184
0.1066 0.0088
1-8
0.8207 = 0.0179
0.1991 0.0148
0.8200 + 0.0208
0.7668 + 0.0225
0.1283 0.0122
Llama-3.1-8B-Instrum
0.8456 a 0.0171
0.1793
0.0160
0.8874 a: 0.0159
0.8507 1 0.0185
0.0772 0.0071
Llama-3.2-1B
0.6562 = 0.0211
0.1300 0.0141
0.6281 = 0.0246
0.5484 = 0.0241
0.1779 0.0169
-lama-322-IB-Instrucc
0.6930 0.0019
0.7164 0.0226
0.6398 0.0225
0,1566 0.0136
Jlama-2-70-bf
0.7403 = 0.0186
0,1405 = 0.0150
0.6589 = 0.0230
0.5816 0.0232
0.1284 0.0144
0.8040 = 0.0168
0.1813 0.0161
0.8133 + 0.0188
0.7715 + 0.0202
0.0693 0.0071
O1-Mo-2-1124-7B
0.0185
0.1713
0.0157
0.8244 + 0.8390 a 0.0197
0.1686 0.0128
OLMo-2-112447B-STT
0.7743 0.0158
0,1754
0.0149
0.8698 1: 0.0176
0,1478 0.0099
O1-Mo-2-112477--DPP
0.7694 = 0.0167
0,1663 0.0159
0.8665 + 0.0179
0.8319 + 0.0205
0.1400 0.0096
OLMo-2-112447BBnntttruut
0.7741 0.0167
0,1635 0.0158
0.8699 0.0175
0.8360 0.0201
0.1468 0.0100
We report mean model accuracies on the evaluation conditions (without interventions) and on their conesponding informative conditions, averaged over the full For the shuffled 10-shot evaluation condition, its informative condition is 10-shot (without label shuftling).
tasks.
For the 0-shot evaluation, we report both each model's 's accuracy with the best instruction for for that task, and the mean accuracy with the five best instructions (which used compute the causal indirect effects, Erros reflect the standard error of the mean.
G Additional results for findings and 2
G.1 Findings 1 and with empirically optimal layer
Shuf fled 10-shot Eval 1.0
0-shot Eval
Shuffled 10-shot Eval
0-shot Eval
A
1.0
8.8
0.8
LIara-3.2-38
Accuracy
Llana Alama-3.2-38
3.2-3B-Instruct Instruct
0.6
0.6
Llana-3, 3.11--3.1-8
L1ama-3.1-88 1-111--IInstrce
0.4
Shuffled 18-thot
0.4
8-shott
0.2
0.2
0.8
0.0
Denonstration
Instruction FV
Demonstration
Both FVs Both FVs Instruction FV
Figure 11: Empirically optimal intervention depth version of Figuue 2 This figure matches Figurc but using the empirically optimal intervention layer for each model and intervention, rather than fixing the [L/3] layer. This showcases the cciling potential of function vectors interventions, above what might be lost by selecting intervention depth using fixed rule. (A) In the post-trained models, the same intervention depth is optimal for both function vectors: 11/28 = 0.3929 for Llama-3.2-33 3.2-n-33.-3BB-. 2-3B-Instruct and 14/32 0.4375 for LLama-3.1-8B--nsttuue in the base models, varies the choice of FV. For Llama-3.2-3BB it is 13/28 for demonstrations and 9/28 for instructions; for Llama-3-1.8PP
8/32 demonstrations and 15/32 for instructions. We qualitatively match previous observations. (B) For three of the four models, adding both vectors to the same layer performs best: Llama-3.2-33 (layer 9/28). 1.ama-3.2-3--mmm
(layer 11/28), and I 1.ama-3.1 --BB- Ins-ruett (layer 13/32). Only for the base I. .lama-3.1 3-B model does the highest accuracy arise from different intervention depths: adding the demonstration FV layer 10/ 10/3 and the instruction FV at layer 8/32.
G.2 Findings and with FV-incongruent evaluations
30

---

fhuf 10-shot Eval
shot Eval
Shuffled 10-shot Eval
0-shot
1.0
A
1.0
0.8
0.8
Llana 3.2-38
Accuracy
Llare-3. -1ara-3.2-30-In-- I-3--Inrrutt
0.6
0.6
3.1-88
Llara-3.1-88 1-0---33----nssrruc
0.4
s-ohot
0.4
Shuffled
0.0
0.8 Instruction FV Both FVs Both FVs Demonstration FV
Instruction FV
Demonstration FV
Figure 12: Mistmached evaluation version of Figure are incongruent with the function vector extraction settings O-shot for demonstration FVs and shuffled 10-shot for instruction FVs. Both FVs perform worse in the incongruent setting, but instruction FVs more so.
report evaluation results from the evaluations that
G.3 Findings 1 and with OLMo models
Shuf fled 10-shot Eval
0-shot Eval
Shuffled 10-shot Eval
0-shot Eval
1.0
A
1.0
OLMo-2-1124-76
Accuracy
0LMo 10e--221124-2-- 1-124-----SFF
0.6
0.6
-LN0-2-1124-78-DP
QUMo=2-1124-7--Instrudd
0.4
ffuffled 18-shot
0.4
0.8
0.0
Demonstration FV
Instruction FV
Demonstration FV Both FVs Both FVs Instruction FV
Figure 13: replication of Figure 2
with the OL models explored in subsection 3.5
Shuf fled 10-shot Eval
8-shot Eval
Shuffled 18-shot Eval
0-shot Eval
1.6
A
OLMo 2. 2-1124-78
Accuracy
MiMo-2-1124-7B-55
0.6
0.6
OLMo 2-te-22-12-------- DP-
1110-2-1122-77-Insttee
0.4
Shu ffled 18-shot
0.4
8-shat
0.2
0.2
0.8
0.0
Denonstration
Instruction FV
Denonstration Both FVs Both FVs Instruction FV
Figure 14: A replication of Figure 2 with the OLMo models explored in subsectioon 3.5 using the empirically- with OLMo models.
optimal intervention depth; I.c. Figure II
Shuffled 10-shot Eval 1.6
s- Eval
Shuffled 18-shot Eval
8-shot Evail
A
1.8
B
OLMo-2-1124-78
Accuracy
OLMo-2-1124-78- SFF
0.6
0.6
0LMo-2-1124 20-224-----00 Instrntt
0.4
Shuffled 10-shot
0.4
8-shat
0.2
0.2
0.6
0.0 Instruction FV Both FVs Both FVs Demonstration FV
Instruction FV
Demonstration
Figure 15: A replication of Figure 2 with the OLMo models explored in subsecctioo 3.5 using the mismatched evaluation version; i.e., Figure 12 with the OLMo models.
31

---

G.4
Findings 1 additional model results
Shuf fied 10-shot Eval
0-shot Eval
Shuffled 10-shot Eval
0-shot Eval
1.0
A
1.0
0UPo-2-110 0UP-22-122--0
2-1224770
0.8
0.8
1124
Accuracy
0 2--242-124-78 78-Instruct
0.6
0.6
9.4
Liara 3.2
0.4
Ltara-3.2-18--
1amaa-3.-1B-Isstrurc
1lama-2-7b-hf
Llara cha-chtt-hf
0.0
0.0
Denonstration
Model
Demonstration
Instruction FV
Ins tructtio
We report evaluation results for the of the models we compare. Panel this model also appears in main manuscript as Figure 4B
Figure 16: Additional model results matching Figure 2A
Shuf flee 10-shot Eval
0-shot Eval
Shuffled 18-shot Evai
6-shot Eval
1.8
A
1.0
Liama-3.2-18
2-1--2-112--88
0.8
Llam--1.-110 2-1---II
0.8
0UPo-2-11244 2-1124-70----
L'ana
1124 DPO
Accuracy
0.6
(Lara-2-bb-chate
0.6
2LM--2-1124-78 Instrrctt
0.4
0.4
0.2
0.2
0.0
8.0
Base Model Instruction
Demons tration FV
Instruction FV
Remonstration FV
Figure 17: Additional model results with mismatched evaluations. This is an unmatched evaluations version
of Figure 16
G.5
Adding an FV twice control
Shuffled 10-shot Eval
0-shot Eval
A
1.0
0.8
Accuracy
0.6
Llana 3-2-88-4 I.s-tuuttttt
0.6
LLara-3.1-88
0.4
Llana 3. 3.1- Instruct
0.4
0.2
0.2
0.0
0.0
Demonstration
Demonstration FV
Both FVs
Bath
Instruction FV Added tuice
Instruction
PV
Added twice
FVs
Figure 18: Adding the same function vector twice control condition. We observe that in some cases, adding the same function vector twice is close to, if not better than adding both function vectors. This surprising effect happens less often when examine the incongruent evaluations (Figure 19), suggesting that adding both function vectors confers advantages both presentations. This effect also weaker when we examine empirically optimal intervention depths (Figure 20).
32

---

8-shot Eval
Shuffled 16-shot Eval
A
1.0
Ltana-3.2-38
0.8
3. 3.2-uutInstruct
0.8
Accuracy
Llana-3.
0.6
10-a-3.1-88-In--
0.6
0.4
0.4
0.2
Demonstration
Demonstration FV
Both
Both
Instruction
Instruction
Added tuice
twice
Figure 19: Adding the same function vector twice control condition, mismatched evaluations. See Figure 18
Shuffled sh-thot Eval
8-shot Eval
1.8
A
1.0
0.8
0.8
Accuracy
0.6
0.6
Llara-3.2-33
Llana 3.2-38-Instruut
0.2
Llana-3.1-80
0.2
Llana 3.1--8 Innttrutt
0.0
0.0
Demonstration
Demonstration Added twice
Both FVs
Instruction Added tuice
Instruction
FV
FVs
Figure 20:
Adding the same function vector twice control condition, with empirically optimal layers.
Figure
33

---

H Additional results for finding 3
H.1 Demonstration and instruction top heads for additional models
-12-8-3.2-18
Llans 3.2----.-IInntteee
28 top heads, 4 shared
28 top heads, 5 sshared
A
25
Head Index
24
24
20
20
16
12 16
4 8
12 16
Llama-2- 2-hhh
Llana-2- 2- 7B-ccht--ht
20 top heads, 4 shared
20 top heads, 2 shared
Head Index
24
24
20
28
18
20 24 OLMo-2-1124-77
12 16 20
0LMo-221124-77
28 top heads, 7 shared
20 top heads, 7 shared
Hoad Index
20
28
16
12
12
20 24
16 20
100--221124--8--00 20 top heads,
01Mo-2-11244 1110-22-124-78-1-00 7--Istrutttt
6 shared
20 top heads, 7 shared
Head Index
Instruction only
Denonstration only
Shared
Figure 21: Shared top heads results for additional models. This panel follows(Figure 3A models we evaluated.
rest of the
H.2 Shared attention head mean activation similarity for additional models
34

---

Llana-3.24 Family
Llana-2-7b-hf
OLNo-2-1124-78
Head Cosine Sinilarity
Head Cosine Sinilarity
Sinilarity
0.5
6,3
2.8
2.6
2.8
2.4
MarT L819
2.2
LANO Dratructiar
0.2
0.2
JacH NeBoLE
lestrcet
Layer
Figure 22: Shared top heads similarities for additional models. This panel followsFigure 3F-G for the rest of the models we evaluated. For the OLMo family of models, the triangular markers follow the ones used in Figure 4B
H.3
Attention head set similarity by prompt length baseline type
Llama-3 3.2-3B
Short instructions equiprobable baseline
Short instructions real text baseline
Short instructions
other task_askkkaa paomp_ baseline
24
24
C
21
21
Head Index
15
15
15
9
9
3
8
12
16
20
24
28
4
12
16
20
24
28
8
12
16
20
24
28
Long instructions equiprobable baseline
Long instructions teal_text baseline
Long instructions
o other poompt baseline
24
24
24
F
21
21
21
Head Index
15
15
15
9
8
12
16
20
24
28
4
12
16
20
24
28
8
12
16
20
24
28
Figure 23: Top heads split by instruction length and uninformative baseline. This figure follows Figure 3A. D but breaks down by individual conditions, for the base Llama-3.2-3B model.
35

---

11aam-33.2335 3.2--B--nnstttt
Short instructions equiprobable baseline
Short instructions real_text baseline
Short instructions
other baseline
A
24
21
21
Head Index
18
18
18
15
15
12
12
12
6
Long instructions equiprobable baseling ine
Long instructions real_ text baseline
Long instructions other task ttttttttk__pppttttttt baseline
F
21
21
Head Index
18
18
15
12
12
12
6
Figure 24: Top heads split by instruction length and uninformative baseline. This figure follows Figure 3A but breaks down by individual conditions, post-trained Llama-3.2-33 3.mma-3.23B-IInttruc model.
Llama-3 3 1B
Short instructions equiprobable baseline
Short instructions
Short instructions
real_text baseline
o ther ttttt_poosptttttttt baseline
32
A
32
32
C
Head Index
20
16
16
16
12
12
12
8
4
4
8 12 16 Long instructions
32
20 Long instructions real_text baseline
16
21
28
32
12 16 20 Long instructions
32
equiprobable baseline
0 other tasttttssssareeeeee _oos__propptt baseline
32
32
32
28
Head Index
16
16
12
12
12
8
4
4
12
32
20
24
28
32
12
16
20
32
Figure 25: Top heads split by instruction length and uninformative baseline. This figure follows Figure 3A but breaks down by individual conditions, for the base Llama-3.1-4 1-8B model.
36

---

(Lama-3.1-8B-INg Instruct
Short instructions equiprobable baseline
Short instructions
Short instructions
baseline
other_ttee ttt_asss__ppopttt baseline
32
A
32
32
C
28
28
28
Head Index
24
24
12
12
8
4
Long instructions equiprobable baseline
Long instructions text baseline
Long instructions
other paask_prompt baseline
32
32
32
F
28
28
28
Head Index
24
24
20
8
4
Figure 26: Top heads split by instruction length and uninformative baseline. This figure follows Figure 3A but breaks down by individual conditions, post-trained -lamm-3.118BB-nna model.
LLama- -..
Llama-3.2 top heads, shared
20 top heads, shared
24
B
Head Index
18
18
15
15
12
12
-4
0
4
8
12
16
20
24
28
8
12
16
20
24
28
Llama 8B88
Llama-3.1 3.1-8B -BInnttucct top heads, shared
-3
20 top heads, 6 shared
32
28
24
Head Index
20
12
8
4
12 16 Layer
20
24
28
32
4
8
12
16
20
24
28
32
Layer
Figure 27: Count of occurrence in top heads over different instruction lengths and basclines. This figure summarizes Figure Figure 26 For each attention head, count how many times appears in the top [A] 20 heads over the two instruction lengths and three uninformative haselines.
37

---

Finding appendix results
L.1 Finding with empirically optimal layer
Shuffled 10-shot Eval
1.0
8.8
Accuracy
0.6
0.4
0.0
Inst heads
Demonstration
Deno heads
acts
0-shot Eval
1.0
1lame-3.--3
Llame L1 3-2-38-----e-e Enstrrcttt
0.8
Ltama-3.1-8
Accuracy
0.6
0.4
8.2
0.0
Inst heads
Instruction FV
Deno heads Inst acts
Deno acts
Figure 28: Empirically optimal intervention depth version of Figure 3G-H
1,2 Finding 4 with OLMo models
Shuffled 10-shot Eval
Shuffled 10-shot Eval
1.8
Accuracy
Accuracy
0.6
0.6
0.4
0.4
(.No-2-1124.
OLHo 1116-11----555 SFT
N.N:-2-122-77-39
0_Yo 1124 78 Instruct
0.0
0.0
Inst heads
Demonstration
Deno heads
Inst
heads Demonstration
Deno heads
Evall
Eval
1.0
1.0
0.8
0.B
Accuracy
Accuracy
0.6
0.6
INN--2-116--7
0.2
0.2
2-N-2-1124---- Instruct
0. 0.0
0.0
Inst heads Deno acts
Instruction FV
Deno heads
Inst heads Deno acts
Instruction FV
Dano heads Inst acts
Inst acts
Figure 29: (Left) A replication of the joint intervention results of Figure 3G-H1
using the OLMo models
explored in subsection 3.5] (Right) Same the left panel, but using the empirically optimal intervention depths; version of Figure 28 but with the OLMo models.
38

---

Table 1.lama-3.2338-Imsmiti
ratio
Heads
CIE Ratio
Llama-3.2-38
Llama-3.1-8B
Mean
5,438
4,688
6.498
2.749
10
Median
2.730
1.801
5.187
2.299
Mean
3.901
3.570
4.794
2.181
20
Median
1.482
1.359
2.894
1.337
Mean
2.566
2.477
3.384
1.769
100
Median
1.036
1.027
1.273
Heads
CIE Ratio
Llama-3.2-1B
Llama-322-IB-Instruut
Llama-2-70-hf
Mean
5.427
4.562
4,807
4.613
Median
4,616
4.338
4,110
5,914
Mean
20
Median
1.846
1.356
2.676
2.660
Mean
2,531
2,435
2.153
2,740
100
Median
1.110
1.264
0.957
1.711
Heads
CIE Ratio 01Mo-2-1124-78
QU.:M---112247B-SST
O1.Mo-2-112477--PP
O1.Mo-2-1122477--nssttu.
Mean
2.734
3.439
3.415
3.534
Median
1.685
2.675
2,511
2.658
Mean
2.067
2.522
2.613
2.682
20
Median
1.112
1.356
1.425
1.409
Meun
1.242
1.577
1.724
1.763
100
Median
0.490
0.717
0.813
0.836
For each model, for top {10, 100) beads identified either demonstrations instructions, compute mean and median causal indirect effect (CIE) scores, and report the ratio between the two, demonstration scores divided by instruction scores. We observe the following trends:
Meun CIE rutios consistently higher than the median CIE ratios. This suggests distribution of demonstration CIEs heavier positive tail than that of instruction CIHs.
As the number heads examined increases, both ratios drop closer to (and some cases, the medians drop below 1). This offers further evidence the heavy-tailes nature the demonstration CIFs, compared to the instruction CIFs. This also suggest that instruction task representations are more diffuse in the models, as at high numbers of heads, the median contribution is higher for instructions than demonstrations.
Finding 4 analyses and controls
Table 8: Localizer experiment causal indirect effects.
Llama-3.2-3B
Llama-3.2-3B- Instruct
Llama-3.1-8B
Llama-3.1-8B-
Instruct
Overall median demonstration CIE Demonstration heads
4.1926c-07
4.3410c-06
1.1681c-06
2.7066e-06
1.3882c-02
2.5550c-02
1.3635e-02
1.4652e-02
demonstration CIE Instruction heads
2.8993e-03
4.1367e-03
3.0959e-03
3.6646e-03
demonstration CIE
Localizer difference
1.1313e-03
1.4519e-03
2.1435e-03
1.7800e-03
Demonstration heads / 1.7680e-03
2.6849e-03
9.5237e-04
1.8846e-03
instruction CIE Instruction heads
3.5586c-03
7.1606e-03
2.8476c-03
6.7488c-03
instruction CIE
Overall median
2.0577e-06
8.1827e-06
7.1886e-06
1.1005e-05
instruction CIE
We observe consistently higher causal indirect scores for instruction FV heads in the demonstration setting, compared to using demonstration FV heads in the instruction setting ("localizer difference," highlighted).
This effect is not merely due causal effects being higher in the demonstration setting; the first and last row provide the median CIE in each condition, and we observe that the median CIE in the instruction setting is higher in every case, by as much as an order of magnitude for some models.
39

---

J.1 Finding 4 control conditions
We perform control experiments ensure that this observed asymmetry is meaningful (and not artifact selecting arbitrary sets of heads). In one, we select the set of heads with the lowest absolute causal scores across both instructions and demonstrations, which consider to be the heads most unrelated to inducing task representations from either presentation. In another, we select the bottom heads that is, the ones with the largest negative causal scores for either instructions or prompts. We report both in Appendix J.1] Performance with the least important heads is indistinguishable from the baselines, and performance with the bottom heads is often below the baselines. The existence of heads with negative causal scores is, itself, curious these are attention heads whose mean cuskccondttioned activations lower the probability assigned the correct token, rather than raise it.
Shuffled
6-shot Eval
1.0
A
1.0
B
Llana-3.2-38
0.8
Llana-3.2 200--333---IIsrure
0.8
Accuracy
Ltana-3. 3.-88-3.1-8
0.6
Llana 3. ructt
0.6
0.2
0.2
0.0
0.0
Denonstration
Denonstration Demonstration Least inp. heads
Instruction
Instruction FV Least imp. heads
Instruction Botton heads
Botton heads
Figure 30: Localizer control conditions: evaluating the least important and bottom heads. To validate effects we observe in Figure 3G-H and Figure 28 are not function of selecting any arbitrary set of attention heads, we report these two control conditions. In both, accuracy at or below chance, expected. We observe similar, though weaker results when using the mismatched evaluations (Figure 31), and similar, equally strong results when using empirically optimal layers (Figure 32).
Shuffled 10-shot Eval
1.0
1.0
1.23-
0.8
Ltana-3. 2-2
0.8
Accuracy
Llana 3-88
08-Instruct
0.6
0.4
0.4
0.2
0.2
0.0
0.0
Denonstration FV Denonstration FV Demonstration FV
Instruction
Instruction FV Least imp. heads
Instruction FV Botton heads
Botton heads
Least imp. heads
Figure 31: Localizer control conditions: evaluating the least important and bottom heads, with mis- matched evaluations, See Figure 30
Shuffled 10-shot Eval
6-shot Eval
A
Llana-3.2-38
0.8
6.8
Accuracy
Llana-3.1-8
0.6
L. 1-888-Insttuct
8.6
0.4
0.4
0.2
0.2
0.0
0.0
Demonstration FV Demonstration FV Demonstration FV Instruction FV Botton heads
Instruction FV Least imp.
Instruction FV Botton heads
imp. heads
Figure 32: Localizer control conditions: evaluating the least important and bottom heads, with empirically
optimal layer. See Figure 30
40

---

K Finding 5 appendix results
K.1
Post-training result variations
Shuffled 10-shot Eval
Shuffled 10-shot Eval
0-shot Eval
1iara-3.2-18
2-12-4-11
Lara-332-38
DLPo 2-1124-111111 70-SFT
Liara 2.1-
OLMo 1124 7B DPO
Accuracy
01M-22-1:24-7
OLP0-2-1124-7-Inss
0.6
Ltara-2-76-ar
0.6
0.4
0,4
0.2
0.2
0.8
0.0
Ins truction FV from Base Model
Instruction tion FV
Ins truction FV
Demons tration FV
from Post-trained Model
Figure 33: Figure 4 with mismatched evaluations. (A) We observe that instruction FVs steer base models in the mismatched evaluation setting as well. (B) In the OLMo model family, mismatched evaluation performance is roughly equal between the two FV types.
@-shot Eval
Shuffled shot Eval
8-
1.0
11244-7
GUN4-2-1124-7B5FF
GLMo 2-1124--1222-- -8- DPO
Accuracy
CLMo Instruct
11ama-3.2-18
0.4
(Lams-3.2-35
8.4
Llams-3.1-00
0.2
OLMo Ltasa-2-7b-hr
0.2
0.8
8.0
Instruction FV from Base Modell
Instruction FV
Demonstration FV
Instruction FV
from Post-traines Model
Figure 34: Figure with empirically optimal intervention layer. (A) observe that instruction transfer beneficially steer base models when evaluated at the empirically optimal intervention layer. (B) We observe the
same qualitative effect here as did with the fixed [L/3] intervention depth two accuracy jumps, one in the SFT model and the second in the DPO model.
41


Reviews:
[{"review_id": "4DbR9OSan5", "reviewer": "Reviewer_25fA", "summary": "In this paper, the authors investigated how different task specification, such as instruction or in-context examples, can infer different task representation. The authors measured the task representations via functional vectors, and identify that different task specification can leads to complementary task representation, which later on can be combined and help steer the models to achieve better performance. Further analysis discuss how different task specification engage different attention heads.", "questions": "1. Line 200 attention head analysis: \"This suggests that task inference from demonstrations is more localized to a small set of heads than from instructions and that instruction-based task inference is more diffuse\". Though this is a finding, what's the further insight of this finding? For example, how does it help future model steering development?\n2. In Section 2.1 Generalizing function vectors beyond in-context demonstrations, when constructing uninformative baselines, do you consider generate the negate instructions as the uninformative baseline?\n3. In Section 3.5 Fig 4, when we do see using instruction FV of post-trained models for pre-trained models help boost performance, the demonstration FV seems to be not helpful. Does it means that post-training does not help with ICL and the demonstration FV does not become more informative?", "limitations": "yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 3, "strengths_and_weaknesses": "# Strength\n1. This paper discusses the questions of how different task specification can leads to different representation. The fact that they do brings insights towards how LMs process diverse prompts in different ways. This finding aligns with previous observations of instruction fine-tuning with diverse instructions leads to better generalization.\n2. The authors propose a simple method to generalize the functional vector constructions from in-context examples to other type of task instruction.\n3. The combination of FVs of different task specifications leads to better zero-shot performance, showing the potential of improving model performance with diverse prompts without training. This has the potential to be an useful method for steering models behavior with better performance.\n4. The analysis of using FVs from post-training models for pre-training model shows that FV can effectively steer the models latent representation and help boost the performance. \n\n# Weakness\n1. The writing can be improved. The authors introduce the background of Functional Vector and the proposed methods with a sequences of mathematical terms without clear structure, which makes it difficult to follow. While I can understand the methods after reading in detail, I'll suggest the authors update the method section with visualization of FV calculations for better clarity.\n2. While this paper shows that combining both FVs leads to better performance on a specific dataset used by Todd et al. (2024)., how this methods generalize to other datasets is unclear to me. Does this method also works on other prominent benchmarks, such as MMLU or BBH?", "was_revised": true, "final_justification": "The author has addressed my concerns about the evaluation and clarity issue and I'm raising the clarity score from 2 to 3 correspondingly.", "rebuttal": "## Response\n\nThank you for your engagement with our work. We’re glad you appreciate the insights our work provides into how LMs process different types of prompts and the potential for using our findings to develop new steering methods. We respond to your highlighted weaknesses and questions below:\n\n**Weakness 1:**\n\n>The writing can be improved. The authors introduce the background of Functional Vector and the proposed methods with a sequences of mathematical terms without clear structure, which makes it difficult to follow. While I can understand the methods after reading in detail, I'll suggest the authors update the method section with visualization of FV calculations for better clarity.\n\nThank you for this suggestion. We’ll be happy to add a figure to clarify this computation. We commit to adding a visualization of the function vector calculation to the camera-ready version of the work (under the new rebuttal guidelines, we cannot upload a figure or an updated version, but we commit to adding one to the final camera-ready copy). \n\n**Weakness 2:**\n\n>While this paper shows that combining both FVs leads to better performance on a specific dataset used by Todd et al. (2024)., how this methods generalize to other datasets is unclear to me. Does this method also works on other prominent benchmarks, such as MMLU or BBH?\n\nThank you for this question. We offer a few thoughts: \n- Please note that we do not test the method on a single dataset, but, following Todd et al., we use approximately 40 separate tasks in our evaluation (see Table 5 in Appendix D). \n- Our emphasis in this work was to extend the function vector method to different prompting methods; therefore, we focused on matching their datasets, rather than evaluating this method on more challenging tasks. \n- We found simple tasks to be useful for elucidating model mechanisms, as is common in interpretability work (for example, see also Hendel et al.’s work). We also consider the free-response nature of the simple tasks as important, as multiple-choice questions (as in MMLU) don’t leave the nature of the task open to interpretation (a multiple-choice question implies the task in the question, even without any instructions). Still, this is an interesting direction, and we leave it to future work to identify the appropriate domain(s) and further study the generalization of these methods. We will expand the current treatment of this limitation in L333-334 and specifically mention MMLU and BBH.   \n\n**Question 1:**\n\n>Line 200 attention head analysis: \"This suggests that task inference from demonstrations is more localized to a small set of heads than from instructions and that instruction-based task inference is more diffuse\". Though this is a finding, what's the further insight of this finding? For example, how does it help future model steering development?\n\nThank you for this note; we agree we could make this finding more actionable. We will add a note to this effect in the final, camera-ready version of the manuscript: “This implies that interventions focused on changing or improving instruction-following may benefit from intervening at multiple locations and depths in a model than interventions aimed at changing demonstration-following.”\n\n**Question 2:**\n\n>In Section 2.1 Generalizing function vectors beyond in-context demonstrations, when constructing uninformative baselines, do you consider generate the negate instructions as the uninformative baseline?\n\nWe did not consider negating the instructions. Thank you for this suggestion. However, there is evidence that language models often fail to understand negation, which might make such a baseline less informative. See Hosseini et al. (2021), Jang et al. (2023), Alhamoud et al. (2025), and Vrabcová et al. (2025) for examples of the phenomenon.\n\n**Question 3:**\n\n>In Section 3.5 Fig 4, when we do see using instruction FV of post-trained models for pre-trained models help boost performance, the demonstration FV seems to be not helpful. Does it means that post-training does not help with ICL and the demonstration FV does not become more informative?\n\nWe agree with your assessment. As far as we can tell, there is no evidence that post-training improves performance on demonstration-based ICL (in private correspondence with Todd et al., they noted very similar results between base and post-trained models in their work, as well). If you think it would help, we are happy to make this point explicitly in the camera-ready version of the manuscript. \n\n## Camera-ready commitments\n\n- **Re: Weakness 1:** Add a figure to visualize function vector calculation.\n- **Re: Weakness 2:** Expand the discussion of the task set limitations to emphasize that the tasks used were all minimal and specifically mention the issue of generalization to other standard benchmarks, such as MMLU and BBH. \n- **Re: Question 1**: Add a more actionable note regarding our finding on the diffuseness of instruction-based inference with a language similar to the example above.\n- **Re: Question 3:** If you think it would help, add a more explicit note that post-training does not improve demonstration-based ICL. \n\nThank you again for your engagement with our work. We’re happy to answer any other questions you may have or provide additional clarifications. If you feel our response addresses your concerns, we hope you will consider raising your score.\n\n## References \n\nAlhamoud, K., Alshammari, S., Tian Y., Li, G., Torr, P. Kim, Y., and Ghassemi, M. (2025). Vision-Language Models Do Not Understand Negation.\n\nHendel, R., Geva, M., and Globerson, A. (2023). In-context learning creates task vectors. \n\nHosseini, A., Reddy, S., Bahdanau, D., Hjelm, R. D., Sordoni, A., and Courville, A. (2021). Understanding by Understanding Not: Modeling Negation in Language Models\n\nJang, J., Ye, S., and Seo, M., (2023). Can Large Language Models Truly Understand Prompts? A Case Study with Negated Prompts.\n\nVrabcová, T., Kadlčík, M., Sojka, P. Štefánik, M., and Spiegel, M. (2025). Negation: A Pink Elephant in the Large Language Models' Room?"}, {"review_id": "MN4hVZiQuU", "reviewer": "Reviewer_CJWJ", "summary": "This paper investigates whether different prompting methods elicit the same underlying task representations in LLMs. The authors mostly focused on two primary approaches for in-context learning, namely demonstrations and instructions. The authors conducted the study with function vectors, and found out that demonstrations and instructions do not yield a single shared task representation in LLMs, but rather activate different internal mechanisms.", "questions": "Please refer to the weaknesses above.", "limitations": "yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 2, "originality": 2, "strengths_and_weaknesses": "### Strengths\n\n* This paper has a clear motivation in conducting this study since the interpretability issue is one of the key problems in LLMs explanability and providing thorough examination over this problem can greatly improve LLMs’ performace on downstream applications.\n\n* The paper is comprehensive in scope, since it conducted extensive experiments across various LLM backbones (Llama-2, Llama-3.1, Llama-3.2, OLMo-2) and multiple model sizes (1B/3B/7B/8B) and tasks (~50 datasets), which increases the generalizability of the findings.\n\n\n### Weaknesses\n\n* While the paper largely follows the setup of Todd et al. (\"Function vectors in large language models\"), the datasets used in this study, whether  the Abstractive Tasks or the Extractive Tasks, mostly tend to involve relatively simple or short outputs. It remains unclear whether the findings in this work could be generalize to more complex or open-ended tasks (e.g., summarization, reasoning, longer-text generation, etc).\n\n* This is not a major concern, but the paper would benefit from a more explicit discussion of the broader implications of its findings. For instance, how might the insights about task representation influence practical prompt design for in-context learning in real-world applications? Providing such guidance could enhance the paper’s utility and impact.", "was_revised": true, "final_justification": "The authors' rebuttal has addressed my concerns. I am maintaining my rating of borderline accept.", "rebuttal": "## Response\n\nThank you for your engagement with our work. We’re glad you appreciate the clear motivation and breadth of experiments we conducted. We would like to offer a few thoughts regarding the weaknesses you highlight:\n\n**Weakness 1:**\n\n>While the paper largely follows the setup of Todd et al. (\"Function vectors in large language models\"), the datasets used in this study, whether the Abstractive Tasks or the Extractive Tasks, mostly tend to involve relatively simple or short outputs. It remains unclear whether the findings in this work could be generalize to more complex or open-ended tasks (e.g., summarization, reasoning, longer-text generation, etc).\n\nThank you for pointing this out; we agree that we did not assess generalization to longer or more complex tasks. Our emphasis in this work was to extend the function vector method to different prompting methods; therefore, we focused on matching their datasets, rather than evaluating this method on more challenging tasks. We also found simple tasks to be useful for elucidating model mechanisms, as is common in interpretability work (for example, see also Hendel et al.’s work). We also consider the free-response nature of the simple tasks as important, as multiple-choice questions (as in MMLU) don’t leave the nature of the task open to interpretation (a multiple-choice question implies the task in the question, even without any instructions). Still, this is an interesting direction, and we leave it to future work to identify the appropriate domain(s) and further study the generalization of these methods. We will expand the current treatment of this limitation in L333-334 to explicitly note the short-output nature of the tasks we evaluate and the generalizability question.   \n\n**Weakness 2:**\n\n>This is not a major concern, but the paper would benefit from a more explicit discussion of the broader implications of its findings. For instance, how might the insights about task representation influence practical prompt design for in-context learning in real-world applications? Providing such guidance could enhance the paper’s utility and impact.\n\nThank you for this comment. We attempted to be cautious in our discussion to avoid overstating the implications of our work, given that the suite of tasks we evaluate is, as you note, relatively simple. That being said, we are happy to offer a speculative proposal on how our work might inform prompt design and what sort of future work would be required to make progress toward this. One idea we began exploring is monitoring the representation in the instruction function vector heads as a proxy for “has the language model formed a task representation from this prompt?” If we were able to monitor such a signal (e.g., as a function of the outputs of these attention heads), that could provide a signal to the prompt designer indicating that they should refine or rephrase their instructions. A second potentially useful direction might be in safety: what if we could classify whether a model inferred a safe task by training a classifier on the activity in function vector heads? We will discuss such ideas and how to make progress toward them in the camera-ready version of the paper.\n\n## Camera-ready commitments\n- **Re: Weakness 1:**  Expand the discussion of the task set limitations to emphasize that the tasks used were all minimal and specifically mention the issue of generalization to more complex and open-ended tasks.\n- **Re: Weakness 2:** Expand the future-looking discussion of the downstream utility of our findings. We will add a treatment to our discussion section on what our findings might imply for prompt design and the future work required to validate and implement these ideas, building on the ideas in the response above. \n\nThank you again for your engagement with our work. We’re happy to answer any other questions you may have or provide additional clarifications. If you feel our response addresses your concerns, we hope you will consider raising your score.\n\n## References\n\nHendel, R., Geva, M., and Globerson, A. (2023). In-context learning creates task vectors."}, {"review_id": "WGaenpqSEJ", "reviewer": "Reviewer_KJBU", "summary": "This paper investigates task representations using function vectors, under different prompting methods (demonstrations and instructions) and across different models (base model and post-trained model from different model families). The authors uncover several interesting findings, such as the demonstration and instruction FVs are beneficial together, and instruction FVs from post-trained models can steer base models.", "questions": "- Do the authors have an explanation for why FVs facilitate zero-shot accuracy less effectively on the LLaMA-3.1-8B model than on the 3B model? Why does the combination of instruction and demonstration FVs fail to improve performance on the 8B  model in the Shuffled 10-shot evaluation setting, even when using the empirically optimal intervention depth (Figure 11)? I'm uncertain whether this indicates that the effectiveness becomes more restricted as model size increases.\n- In Line 209, the paper mentions that \"post-training appears to induce an alternative task inference mechanism.\" Could the authors clarify what this mechanism is?\n\nPlease also refer to the weaknesses", "limitations": "yes", "rating": 4, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 3, "significance": 3, "originality": 3, "strengths_and_weaknesses": "Strengths:\n\n- The paper generalizes the function vector approach from in-context demonstrations to instructions. \n- The experimental design is comprehensive. Many of the findings are novel and informative to me.\n- The paper is well-written and clearly structured, making it easy to follow the authors' arguments.\n\nWeaknesses:\n\n- While the paper reports several interesting empirical phenomena, some are not sufficiently explained or discussed. For example:\n  - Why does the Llama-3.1-8B model underperforms  Llama-3.1-3B model (Figure 2)?\n  - Is the improvement from combining both demonstration and instruction FVs due to capturing complementary information, or is it merely amplifying the intervention strength?\n  - What causes post-training to align instruction FV heads closer to demonstration FV heads, and why does the average layer of instruction-only heads shift from being deeper than that of demonstration heads to approximately the same depth?\n- Although the experiments on synthetic tasks provide useful insight into task representation mechanisms, the paper would benefit from more practical takeaways or actionable guidance, for example on model steering in real-world scenarios.", "was_revised": false, "final_justification": "", "rebuttal": "## Response\n\nThank you for your engagement with our work. We are glad you appreciated our comprehensive set of experiments and findings, as well as the structure of our paper. We respond to your highlighted weaknesses and questions below:\n\n**Weakness 1:**\n\n>While the paper reports several interesting empirical phenomena, some are not sufficiently explained or discussed. For example:\n\nThank you for highlighting several findings we could articulate or explain better. We offer some answers below, and we commit to adding similar clarifications to the camera-ready version of our work:\n\n>Why does the Llama-3.1-8B model underperforms Llama-3.1-3B model (Figure 2)?\n\nWe remark in lines 156-158 that we believe the heuristic L/3 layer in which the intervention is added is highly suboptimal for that model specifically. Figure 11 (Appendix G.1, L972, p. 29) is similar to Figure 2, but plots using the best intervention depth for each model. In this figure, Llama-3.1-8B closes some of the gap. \n\nAnother factor that may contribute to this is that the Llama-3.2 series is distilled using logits from both the 8B and 70B Llama-3.1 models, which may confer benefits beyond the 8B model. We will add a note to this effect and clarify the previous one in the camera-ready version.\n\n>Is the improvement from combining both demonstration and instruction FVs due to capturing complementary information, or is it merely amplifying the intervention strength?\n\nWe were also curious about this question, so we ran some experiments in which we added a single FV twice, rather than both of them. We summarize our findings in L182-187 and plot them in Appendix G.4 (L975, p. 31). In some models and evaluation settings, adding an FV twice is equivalent to (or marginally better than) adding both FVs, but in many it isn’t. If you think it would be helpful, we are happy to clarify this finding in the camera-ready version.\n\n>What causes post-training to align instruction FV heads closer to demonstration FV heads, and why does the average layer of instruction-only heads shift from being deeper than that of demonstration heads to approximately the same depth?\n\nThank you for this question. We do not have a complete answer regarding post-training dynamics. Our best guess is that instruction-tuning uses attention heads that are peripherally useful for ICL and leverages them to represent instructed tasks. We explain our thinking slightly later, in L228-232. We will add a note suggesting the connection between this change in depth and this mechanism to the camera-ready version of the paper.\n\n**Weakness 2:**\n\n>Although the experiments on synthetic tasks provide useful insight into task representation mechanisms, the paper would benefit from more practical takeaways or actionable guidance, for example on model steering in real-world scenarios.\n\nThank you for this comment. We attempted to be cautious in our discussion to avoid overstating the implications of our work, given that the suite of tasks we evaluate is, as you note, relatively simple. One observation our work provides is that instruction task representation is formed across several layers in the middle of the model. Most steering methods intervene at a single layer. Our work suggests that there may be benefits to applying smaller interventions across different layers rather than a single intervention in a particular layer, and we leave this for future work to explore. We can also offer some speculation on how our findings might inform prompt design feedback. One idea we began exploring is monitoring the representation in the instruction function vector heads as a proxy for “has the language model formed a task representation from this prompt?” If we were able to monitor such a signal (e.g., as a function of the outputs of these attention heads), that could provide a signal to the prompt designer indicating that they should refine or rephrase their instructions. A second potentially useful direction might be in safety: what if we could classify whether a model inferred a safe task by training a classifier on the activity in function vector heads? We will discuss such ideas and how to make progress toward them in the camera-ready version of the paper.\n\n**Question 1:**\n\n>Do the authors have an explanation for why FVs facilitate zero-shot accuracy less effectively on the LLaMA-3.1-8B model than on the 3B model? Why does the combination of instruction and demonstration FVs fail to improve performance on the 8B model in the Shuffled 10-shot evaluation setting, even when using the empirically optimal intervention depth (Figure 11)? I'm uncertain whether this indicates that the effectiveness becomes more restricted as model size increases.\n\nBeyond the thoughts offered in response to the first question on the first weakness identified, one other note we can make is that we use the same number of attention heads in both of these models, which are proportionally more in the 3B models (the 3B models have 672 attention heads, so 20 top heads are ~3%, while the 8B models have 1024 attention heads, so the top 20 are ~2%). It could be that we are missing more useful attention heads in the 8B models. We will add a note to this effect in the camera-ready version.\n\n**Question 2:**\n\n>In Line 209, the paper mentions that \"post-training appears to induce an alternative task inference mechanism.\" Could the authors clarify what this mechanism is?\n\nWhat we meant by ‘alternative’ L209 is that the mechanism induced by post-training for instruction task representations is separate from the one base models develop for demonstration task representation. We have so far identified this mechanism only through a different set of attention heads and do not have a full mechanistic description. We will clarify this in the camera-ready version. \n\n\n## Camera-ready commitments\n- **Re: Weakness 1:** We commit to adding a few notes:\n  - Regarding the underperforming Llama-3.1-8B, we will clarify our hypothesis about the intervention layer being suboptimal and add a note about the logit distillation used in the creation of the Llama-3.2 models. \n  - Regarding the intervention amplification question, if you think it would be helpful, we are happy to clarify our findings in the camera-ready version of the manuscript.\n  - Regarding the FV head depth alignment question, we will add a note explicitly connecting our hypothesis about what instruction-tuning is doing to the observed changes in alignment depth. \n- **Re: Weakness 2:** Expand the practical future-looking discussion of the downstream utility of our findings. We will add a treatment to our discussion section on what our findings might imply for steering and prompt design, as well as the future work required to validate and implement these ideas, building on the ideas we sketch out in our response above. \n- **Re: Question 1:** We will add a note regarding the fraction of attention heads being different across models to the camera-ready version of the manuscript.\n- **Re: Question 2:** We will add a clarification of what we meant by alternative mechanism to the camera-ready version of the paper.\n\nThank you again for your engagement with our work. We’re happy to answer any other questions you may have or provide additional clarifications. If you feel our response addresses your concerns, we hope you will consider raising your score."}, {"review_id": "ivp3KOUzTw", "reviewer": "Reviewer_fKfR", "summary": "This paper primarily discusses the differences in the internal function vectors (FV) of LLMs during reasoning under two distinct settings: one involving the provision of demonstrations, and the other, the provision of instructions. The aim is to observe the relationship between these two settings. The results presented in the paper indicate that while FVs from the two settings exhibit significant differences, both positively contribute to the model's reasoning performance.", "questions": "- How does the author select demonstrations during inference? Considering that the similarity between the demonstrations and the query can significantly affect the Demonstration FV, I believe it is necessary to ensure a high degree of similarity between them.\n- On datasets for more mainstream tasks (e.g., mathematics, code), does the relationship between Demonstration FV and Instruction FV remain consistent with the conclusions presented in this paper?", "limitations": "Yes", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": 3, "clarity": 4, "significance": 3, "originality": 3, "strengths_and_weaknesses": "**Strengths**\n\n- The paper investigates an interesting and important problem. Previous work has not discussed the differences and connections between instructions and demonstrations of ICL, making this research novel and exciting.\n- The experiments are thorough and provide strong support for each finding.\n- The paper is well-written and easy to follow.\n\n**Weaknesses**\n\n- The current experiments are primarily focused on the Llama series of models. Expanding the experiments to include more model families (e.g., Qwen) would better substantiate the paper's conclusions.\n-  Considering the extensive body of work on the functionality of FV heads, the Related Work section would benefit from a more detailed comparison with existing FV head methodologies to better highlight the contributions of this paper.", "was_revised": false, "final_justification": "", "rebuttal": "## Response\nThank you for your engagement with our work. We are glad you found the problem interesting and important, and that you appreciate the thoroughness of the experiments we ran. We offer our responses to the weakness you highlight and the questions you have for us:\n\n**Weakness 1:**\n\n>The current experiments are primarily focused on the Llama series of models. Expanding the experiments to include more model families (e.g., Qwen) would better substantiate the paper's conclusions.\n\nWe focus our discussion on the Llama model, but we do replicate many of the experiments with the OLMo-2-1124-7B family of models. The right-hand panels of Figures 13 and 14 (Appendix G.3, L974, p. 30) replicate Finding 1 (Figure 2A), and Figures 18 and 19 (Appendix H.1, L977, p. 32-32) repeat Figure 3A-D for additional models. This is in addition to Finding 5, which also reports these models (Figure 4, p. 7). We are running experiments replicating Finding 4 with the OLMo models and will add these results to the appendix of the camera-ready version. We hope this helps alleviate some of your concerns, and we will add a note referring to these figures from the main manuscript in the camera-ready version.\n\n**Weakness 2:**\n\n>Considering the extensive body of work on the functionality of FV heads, the Related Work section would benefit from a more detailed comparison with existing FV head methodologies to better highlight the contributions of this paper.\n\nWe appreciate this concern and will be happy to expand our discussion of FV methods. We refer to the relevant literature and cite it in L275-278; we will also add a discussion of the different FV methodologies to the camera-ready version.\n\n**Question 1:**\n\n>How does the author select demonstrations during inference? Considering that the similarity between the demonstrations and the query can significantly affect the Demonstration FV, I believe it is necessary to ensure a high degree of similarity between them.\n\nWe follow the original methodology of Todd et al. – we split an ICL dataset into train/validation/test sets and compute the FVs over a set of validation examples on which the model performs the task successfully. We otherwise take no effort to align the demonstrations with the queries. We agree that this is an interesting nuance, and that better demonstrations will likely lead to better demonstration FVs; we leave its investigation to future work. \n\n**Question 2:**\n\n>On datasets for more mainstream tasks (e.g., mathematics, code), does the relationship between Demonstration FV and Instruction FV remain consistent with the conclusions presented in this paper?\n\nThank you for this question. The work by Todd et al. does not assess demonstration FVs on more complex, mainstream tasks. Our emphasis in this work was to extend the function vector method to different prompting methods; therefore, we focused on matching their datasets and tasks, rather than evaluating this method on more challenging tasks. We also found simple tasks to be useful for elucidating model mechanisms, as is common in interpretability work (for example, see also Hendel et al.’s work). We also consider the free-response nature of the simple tasks as important, as multiple-choice questions (as in MMLU) don’t leave the nature of the task open to interpretation (a multiple-choice question implies the task in the question, even without any instructions). Still, this is an interesting direction, and we leave it to future work to identify the appropriate domain(s) and further study the generalization of these methods. We will expand the current treatment of this limitation in L333-334 to explicitly note that we lack evidence regarding how much our results generalize to more complex and naturalistic tasks.\n\n## Camera-ready commitments\n- **Re: Weakness 1:** We will add a note referring to finding replications with other models from the main manuscript, and add the Finding 4 replication results with the OLMo models to the appendix.\n- **Re: Weakness 2:** We will expand our discussion of FV heads and methods in the related work selection.\n- **Re: Question 2:**  Expand the discussion of the task set limitations to emphasize that the tasks used were all minimal and specifically mention the issue of generalization to more complex and naturalistic tasks.\n\nThank you again for your engagement with our work. We’re happy to answer any other questions you may have or provide additional clarifications.\n\n## References\n\nHendel, R., Geva, M., and Globerson, A. (2023). In-context learning creates task vectors."}]

Author rebuttal:

Meta-review: This paper studies how different variations of task specification (e.g., in-context learning, direct instruction) might result in different representations of a task internal to a model. Because it is found that variations result in different representations, the authors also propose to make use of this to improve model performance in a form of ensembling by mixing activations from multiple function vectors coming from different different task specifications. Experiments are performed on a task previously defined by Todd et al. (2024).

One missing work is Sclar et al. 2024 (ICLR) which also shows that task specifications (prompt formatting) maintain unique internal representations from one another.
Decision: Accept (poster)