Title: Do different prompting methods yield a common task representation in language models?
Venue: NeurIPS 2025
Accepted: yes
Keywords: task representations; function vectors; in-context learning; instructions; mechanistic interpretability
Abstract: Demonstrations and instructions are two primary approaches for prompting language models to perform in-context learning (ICL) tasks.
Do identical tasks elicited in different ways result in similar representations of the task? An improved understanding of task representation mechanisms would offer interpretability insights and may aid in steering models. We study this through function vectors (FVs), recently proposed as a mechanism to extract few-shot ICL task representations. We generalize FVs to alternative task presentations, focusing on short textual instruction prompts, and successfully extract instruction function vectors that promote zero-shot task accuracy. We find evidence that demonstration- and instruction-based function vectors leverage different model components, and offer several controls to dissociate their contributions to task performance. Our results suggest that different task prompting forms do not induce a common task representation through FVs but elicit different, partly overlapping mechanisms. Our findings offer principled support to the practice of combining instructions and task demonstrations, imply challenges in universally monitoring task inference across presentation forms, and encourage further examinations of LLM task inference mechanisms.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper primarily discusses the differences in the internal function vectors (FV) of LLMs during reasoning under two distinct settings: one involving the provision of demonstrations, and the other, the provision of instructions. The aim is to observe the relationship between these two settings. The results presented in the paper indicate that while FVs from the two settings exhibit significant differences, both positively contribute to the model's reasoning performance.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n\n- The paper investigates an interesting and important problem. Previous work has not discussed the differences and connections between instructions and demonstrations of ICL, making this research novel and exciting.\n- The experiments are thorough and provide strong support for each finding.\n- The paper is well-written and easy to follow.\n\n**Weaknesses**\n\n- The current experiments are primarily focused on the Llama series of models. Expanding the experiments to include more model families (e.g., Qwen) would better substantiate the paper's conclusions.\n-  Considering the extensive body of work on the functionality of FV heads, the Related Work section would benefit from a more detailed comparison with existing FV head methodologies to better highlight the contributions of this paper.\n\nQUESTIONS:\n- How does the author select demonstrations during inference? Considering that the similarity between the demonstrations and the query can significantly affect the Demonstration FV, I believe it is necessary to ensure a high degree of similarity between them.\n- On datasets for more mainstream tasks (e.g., mathematics, code), does the relationship between Demonstration FV and Instruction FV remain consistent with the conclusions presented in this paper?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-01"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper investigates task representations using function vectors, under different prompting methods (demonstrations and instructions) and across different models (base model and post-trained model from different model families). The authors uncover several interesting findings, such as the demonstration and instruction FVs are beneficial together, and instruction FVs from post-trained models can steer base models.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\n- The paper generalizes the function vector approach from in-context demonstrations to instructions. \n- The experimental design is comprehensive. Many of the findings are novel and informative to me.\n- The paper is well-written and clearly structured, making it easy to follow the authors' arguments.\n\nWeaknesses:\n\n- While the paper reports several interesting empirical phenomena, some are not sufficiently explained or discussed. For example:\n  - Why does the Llama-3.1-8B model underperforms  Llama-3.1-3B model (Figure 2)?\n  - Is the improvement from combining both demonstration and instruction FVs due to capturing complementary information, or is it merely amplifying the intervention strength?\n  - What causes post-training to align instruction FV heads closer to demonstration FV heads, and why does the average layer of instruction-only heads shift from being deeper than that of demonstration heads to approximately the same depth?\n- Although the experiments on synthetic tasks provide useful insight into task representation mechanisms, the paper would benefit from more practical takeaways or actionable guidance, for example on model steering in real-world scenarios.\n\nQUESTIONS:\n- Do the authors have an explanation for why FVs facilitate zero-shot accuracy less effectively on the LLaMA-3.1-8B model than on the 3B model? Why does the combination of instruction and demonstration FVs fail to improve performance on the 8B  model in the Shuffled 10-shot evaluation setting, even when using the empirically optimal intervention depth (Figure 11)? I'm uncertain whether this indicates that the effectiveness becomes more restricted as model size increases.\n- In Line 209, the paper mentions that \"post-training appears to induce an alternative task inference mechanism.\" Could the authors clarify what this mechanism is?\n\nPlease also refer to the weaknesses\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper investigates whether different prompting methods elicit the same underlying task representations in LLMs. The authors mostly focused on two primary approaches for in-context learning, namely demonstrations and instructions. The authors conducted the study with function vectors, and found out that demonstrations and instructions do not yield a single shared task representation in LLMs, but rather activate different internal mechanisms.\n\nSTRENGTHS AND WEAKNESSES:\n### Strengths\n\n* This paper has a clear motivation in conducting this study since the interpretability issue is one of the key problems in LLMs explanability and providing thorough examination over this problem can greatly improve LLMs’ performace on downstream applications.\n\n* The paper is comprehensive in scope, since it conducted extensive experiments across various LLM backbones (Llama-2, Llama-3.1, Llama-3.2, OLMo-2) and multiple model sizes (1B/3B/7B/8B) and tasks (~50 datasets), which increases the generalizability of the findings.\n\n\n### Weaknesses\n\n* While the paper largely follows the setup of Todd et al. (\"Function vectors in large language models\"), the datasets used in this study, whether  the Abstractive Tasks or the Extractive Tasks, mostly tend to involve relatively simple or short outputs. It remains unclear whether the findings in this work could be generalize to more complex or open-ended tasks (e.g., summarization, reasoning, longer-text generation, etc).\n\n* This is not a major concern, but the paper would benefit from a more explicit discussion of the broader implications of its findings. For instance, how might the insights about task representation influence practical prompt design for in-context learning in real-world applications? Providing such guidance could enhance the paper’s utility and impact.\n\nQUESTIONS:\nPlease refer to the weaknesses above.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 2, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nIn this paper, the authors investigated how different task specification, such as instruction or in-context examples, can infer different task representation. The authors measured the task representations via functional vectors, and identify that different task specification can leads to complementary task representation, which later on can be combined and help steer the models to achieve better performance. Further analysis discuss how different task specification engage different attention heads.\n\nSTRENGTHS AND WEAKNESSES:\n# Strength\n1. This paper discusses the questions of how different task specification can leads to different representation. The fact that they do brings insights towards how LMs process diverse prompts in different ways. This finding aligns with previous observations of instruction fine-tuning with diverse instructions leads to better generalization.\n2. The authors propose a simple method to generalize the functional vector constructions from in-context examples to other type of task instruction.\n3. The combination of FVs of different task specifications leads to better zero-shot performance, showing the potential of improving model performance with diverse prompts without training. This has the potential to be an useful method for steering models behavior with better performance.\n4. The analysis of using FVs from post-training models for pre-training model shows that FV can effectively steer the models latent representation and help boost the performance. \n\n# Weakness\n1. The writing can be improved. The authors introduce the background of Functional Vector and the proposed methods with a sequences of mathematical terms without clear structure, which makes it difficult to follow. While I can understand the methods after reading in detail, I'll suggest the authors update the method section with visualization of FV calculations for better clarity.\n2. While this paper shows that combining both FVs leads to better performance on a specific dataset used by Todd et al. (2024)., how this methods generalize to other datasets is unclear to me. Does this method also works on other prominent benchmarks, such as MMLU or BBH?\n\nQUESTIONS:\n1. Line 200 attention head analysis: \"This suggests that task inference from demonstrations is more localized to a small set of heads than from instructions and that instruction-based task inference is more diffuse\". Though this is a finding, what's the further insight of this finding? For example, how does it help future model steering development?\n2. In Section 2.1 Generalizing function vectors beyond in-context demonstrations, when constructing uninformative baselines, do you consider generate the negate instructions as the uninformative baseline?\n3. In Section 3.5 Fig 4, when we do see using instruction FV of post-trained models for pre-trained models help boost performance, the demonstration FV seems to be not helpful. Does it means that post-training does not help with ICL and the demonstration FV does not become more informative?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-03"}