Title: Don’t Forget the Enjoin: FocalLoRA for Instruction Hierarchical Alignment in Large Language Models
Venue: NeurIPS 2025
Accepted: yes
Keywords: Instruction Hierarchy

Large Language Models
Abstract: Recent studies reveal that large language models (LLMs) often struggle to resolve conflicting instructions embedded within hierarchical prompts, resulting in decreased compliance with system-level directives and compromising the reliability of safety-critical applications. While earlier approaches attempt to improve instruction hierarchy awareness through prompt engineering or embedding-level modifications, they typically lack structural modeling and either offer limited gains or require extensive fine-tuning. In this work, we introduce $\textbf{FocalLoRA}$, a parameter-efficient and structure-aware framework that strengthens hierarchical instruction adherence by selectively optimizing structurally critical attention heads, referred to as $\textit{focal heads}$, which exhibit heightened sensitivity to instruction conflicts. Experiments across multiple models and a dedicated benchmark demonstrate that FocalLoRA markedly enhances system instruction compliance with minimal tuning cost. For instance, on Llama-8B, fine-tuning only 0.0188\% of parameters yields a 35.52\% $\uparrow$ in system instruction compliance.


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors propose FocalLora, a parameter-efficient method for enforcing the instruction hierarchy. The core of the method is optimizing a small (<0.02% of total params) subset of attention heads sensitive to the instruction conflicts for the purpose of correctly resolving instruction conflicts. These heads are identified using a separate mechanism (CSHI), which relies on differences in attention matrix in input examples with conflicting and consistent goals.\n\nSTRENGTHS AND WEAKNESSES:\nOverall, a good paper with excellent motivation, solid method and good experiments which lack a bit of rigour when comparing to baselines. The paper was a pleasure to read, and I think it should be accepted.\n\n\nStrengths: \n\n* The authors experimentally show motivation for their method by identifying a small subset of heads that are responsible for instruction conflicts. I liked the practical demonstration of the underlying reasoning for their method. \n* The proposed method is solid: a method for identifying the heads rely on 3 different distance measures between the attention matrices on consistent/conflicting scenarios. Then authors propose to use a loss that moves more attention to system prompts for the selected heads.\n* The authors create (quite reasonable) datasets for evaluating their models and share quite comprehensive evals over 2 dataset and 5 models, and they include code in the submission. \n* The paper was a pleasure to read: notation was clear, the paper was well structured. \n\n\nWeaknesses:\n\n*  “Instructional Segmentation Embedding (ISE) (Wu et al., 2024) introduces segment-level embeddings to strengthen the influence of system-level directives. However, it lacks the ability to intervene in how the model internally handles conflicting instructions.” – I do not think the comparison with ISE was fair in your experimental pipeline. It Table 2, FocalLora#16 is better than ISE by ~2% in 13 cases, and worse by 4-5% in two more. I’d parse it as “FocalLora#8” is marginally better than ISE. Yes, FocalLora#32 substantially outperforms ISE, however, since you didn’t tune parameters for ISE, it is natural that some version of FocalLora would outperform ISE. \n* Insufficient evaluations: while I think existing evals are good, one of them is created by the authors themselves, and there are only two of them. At the same time, at the time of submission, there already were papers dedicated solely to creating instruction hierarchy benches, for instance, IHEval[1], which the authors cite in the related work section. The paper would benefit a lot from evaluation on a dedicated instruction hierarchy bench. \n\n\n\n\n[1] Zhihan Zhang, Shiyang Li, Zixuan Zhang, Xin Liu, Haoming Jiang, Xianfeng Tang, Yifan Gao, Zheng Li, Haodong Wang, Zhaoxuan Tan, Yichuan Li, Qingyu Yin, Bing Yin, Meng Jiang. IHEval: Evaluating Language Models on Following the Instruction Hierarchy, ACL 2025.\n\nQUESTIONS:\nSome feedback for the authors: \n\n* “While some recent work (Geng et al., 2025; Zhang et al., 2025) has suggested the necessity of incorporating instruction hierarchy into the model  architecture and training objectives, these ideas remain largely theoretical and have not been realized in practice.” – Not really. At the time of paper submission, there was at least one paper on arxiv that  implemented and showed efficiency of architectural approaches on top of (less efficient) ISE [1].  Considering it appeared after the March 1st deadline, this is not a deciding factor, but I strongly encourage you to modify the discussion for the camera ready version. I’d position FocalLora and architectural methods as alternative approaches to the same problem, not “one is better than the other”. \n* Advice for comparison with ISE: try to initialize self . ISE_embedding = nn. Embedding ( ISE_size , embed_size ) with different distributions (uniform, normal, xavier) + different distribution params (mean, std, etc). Additionally, initialize ISE embeddings with a shift for different roles (E.g., mean=0 for the system prompt, mean=1 for user, etc) and perform comparison. I personally do not believe it would change the evals, and therefore didn’t penalize it, but for the scientific rigour, consider doing so for the camera ready.\n* Why didn’t you evaluate it on IHEval or a similar benchmark?\n\n\n\nI would also appreciate author’s thoughts on the following clarifying questions:\n* Why did you decide to focus on attention heads specifically? Did you think about training 1-2 MLPs? \n* Have you tried training the selected heads without LoRA? Does it make eval metrics better?\n“For each attention head, we record two metrics: its sensitivity score and the frequency with which it ranks in the top 10% across all scenarios.” – how do you define sensitivity score?\nDid you ever use Definition 1 in practice, or is that meant to be a theoretical definition?\n* In section 3.3., you show the effects of attention drift, and then proceed to explain the existence of focal heads. How are these parts (attention drift in general setting and focal heads for conflicting instructions) connected to each other? \n* Included code lacked doc-strings for functions, please include it for the camera ready version. \n* You might want to be aware of the concurrent work [2] where they propose a similar method to yours by also moving attention to more relevant instructions.\n* In equation (6) you combine 3 metrics. While each individual choice makes sense to me, it’s not clear why you selected this specific combination of metrics. What were the other metrics you tried? How did the result differ? If you select only individual metrics or pairs of them, how does it influence the final results?  \n\n\n[1] Egor Zverev, Evgenii Kortukov, Alexander Panfilov, Alexandra Volkova, Soroush Tabesh, Sebastian Lapuschkin, Wojciech Samek, and Christoph H. Lampert. ASIDE: Architectural Separation of Instructions and Data in Language Models. arXiv preprint arXiv:2503.10566, 2025.\n\n\n[2] Praveen Venkateswaran, Danish Contractor. Spotlight Your Instructions: Instruction-following with Dynamic Attention Steering. arXiv preprint arXiv:2505.12025\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper focuses on addressing the problem of instruction conflicts, where the model tends to ignore system prompts in favor of user prompts when the two are contradictory. To tackle this issue, the study identifies a set of parameters that are particularly sensitive to instruction conflicts and selectively fine-tunes them. This targeted training approach enhances the model’s ability to resolve such conflicts effectively.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\n1. This paper proposes a quantitative analysis framework to examine the sensitivity of model parameters to instruction conflicts, providing constructive guidance for future research.\n2. The proposed method consistently improves the model's ability to handle instruction conflicts across different scenarios and experimental settings.\n\nWeaknesses\n1. The analysis would benefit from broader experimental validation. In particular, a key baseline is missing: what happens to performance when the top 10% identified heads are randomly replaced with another parameter subset of the same size?\n2. Some experimental details are lacking. For example, how are the evaluation metrics defined in both in-domain and out-of-domain tests? While the results in the tables appear promising, it is difficult to assess the significance of the reported improvements without this information.\n3. Although Figure 5 illustrates the performance impact of training the top n% of parameters in LLaMA-38B and Qwen-14B, an important comparison is missing: can training only the top 10% of parameters match or closely approach the performance achieved by training all parameters?\n4. While the method improves the model’s ability to resolve instruction conflicts, it remains unclear whether this comes at the cost of degraded task performance on various tasks (e.g., MMLU).\n\nQUESTIONS:\nPlease refer to the weakness.\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors propose FocalLoRA, a novel, parameter-efficient fine-tuning framework that aims to enhance the compliance of hierarchical instructions in large language models by optimizing a set of structurally critical attention heads, called focal heads. This approach significantly improves system-level compliance without changing the model architecture or incurring high computational costs. Experiments show that the method is effective on multiple models.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths:\n\n1. FocalLoRA can identify attention heads whose behavior changes significantly in command conflict scenarios through the Conflict-Sensitive Heads Identification (CSHI) module. These \"focal heads\" are critical for detecting and responding to inconsistencies between system and user commands, thereby improving the model's ability to perceive conflicts.\n\n2. Using the System-Aware Heads Optimization (SAHO) module, FocalLoRA selectively fine-tunes the identified focal heads to strengthen their attention to system-level commands, allowing the model to more effectively prioritize high-level system commands when facing multi-level or conflicting commands.\n\n3. Experiments show that FocalLoRA can achieve a 35.52% improvement in system command compliance rate by only fine-tuning a very small proportion of parameters (such as only 0.0188% in Llama-8B), proving the dual advantages of this method in parameter efficiency and actual effect.\n\n\nWeaknesses:\n\n1. In addition to improving the model's ability to resolve instruction conflicts, it is recommended that the author further test the general performance of the model.\n\n2. Tables 1 and 2 only compare the FocalLoRA and Ordinary methods, lacking comparative analysis with other instruction conflict resolution methods.\n\n3. It is recommended to provide visualization results of the optimized attention heads in the ablation experiment to verify the effectiveness of the method.\n\nQUESTIONS:\nsee the weaknesses\n\nLIMITATIONS:\nsee the weaknesses", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 3, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper studies the capacity of LLM to manage hierarchical prompts containing system, user, and assistant instructions. In particular, the authors are interested in the managing of conflicts between the different instructions. They propose to study it at the level of the model, i.e. using attention heads. Based on an experimental study in which the authors show that some subsets of attention heads seem to play an important role in the presence of instruction conflicts, they propose a new approach, named FocalLora, whose principle is to detect and optimize these important attention heads using LoRA adaptation. The paper provides an extensive experimental validation of the proposed approach.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n\n+ A well-written paper with clear motivations, formalization of the proposed approach, and some experimental validation that supports the main claim of the paper.\n+ The idea of studying conflictual instructions at the level of the model is interesting and novel.\n+ The multi-scoring approach for the identification of conflict-sensitive heads.\n\n\n**Weaknesses**\n+ I really appreciated the attempt to define the notion of instruction conflict. Nevertheless, I have some concerns on the level of generality of the proposed definition which is mainly oriented by system instructions. It could be very interesting to generalize it to more than two-tier instructions and without prioritization to a specific kind of instruction. In particular, how can we be sure that the main findings related to the importance of the user instruction are not only related to the order of the instruction?\n+ The identification of conflict-sensitive heads can be considered supervised since it requires normal and conflict samples. As a consequence, the results are highly dependent on these samples. How do they impact the results?\n\nQUESTIONS:\n**Some questions :**\n\n+ What are the 16 combinations of $I_s + I_u$ that are considered ?  \n+ How could the approach be generalized to different granularities of conflicts (not only binary conflicts) ?\n+ Figure 1 is not very clear from my point of view.\n+ Why not applying FocalLora and the global study to ISE ?\n\nLIMITATIONS:\nYes", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 2, "DATE": "2025-07-06"}