IHE val: Evaluating Language Models on Following the Instruction Hierarchy (2025)

Abstract: The instruction hierarchy, which establishes a priority order from system messages to user messages, conversation history, and tool outputs, is essential for ensuring consistent and safe behavior in language models (LMs). Despite its importance, this topic receives limited attention, and there is a lack of comprehensive benchmarks for evaluating models’ ability to follow the instruction hierarchy. We bridge this gap by introducing IHEval, a novel benchmark comprising 3,538 examples across nine tasks, covering cases where instructions in different priorities either align or conflict. Our evaluation of popular LMs highlights their struggle to recognize instruction priorities. All evaluated models experience a sharp performance decline when facing conflicting instructions, compared to their original instruction-following performance. Moreover, the most competitive open-source model only achieves 48% accuracy in resolving such conflicts. Our results underscore the need for targeted optimization in the future development of LMs.

## introduction instruction-tuned language models (lms) are increasingly deployed as interactive services across various applications @xcite @xcite . to ensure consistent performance and safety, developers typically seek to regulate the model's behavior, such as finetuning the model on responding to certain instructions @xcite , using post-processing techniques to edit model outputs @xcite , and detailing system messages to impose constraints @xcite . however, in realworld applications, these pre-defined regulations frequently struggle to cover the full range of possible user inputs. for instance, users may request tasks beyond the model's intended scope @xcite , or the integrated tools may return unexpected content @xcite . the lm may risk misbehavior if higher-level instructions, such as regulative system messages, are overridden by subsequent conflicting inputs. this highlights the need for lms to possess an inherent capacity to follow an instruction hierarchy, where instructions of high-level regulations are always prioritized over low-level inputs. the order of priority -ranked from highest to lowest -should be: system messages, user messages, conversation history, and tool outputs. this hierarchy was first proposed by @xcite and aligns with typical practices when using language models. a model that adheres to this hierarchy would be able to spontaneously resolve instruction conflicts by prioritizing higher-order ones, which reduces the dependency on complex regulation methods while also mitigating the risk of misbehavior foot_0 . despite its significance, the instruction hierarchy paradigm does not receive much attention in lm research and evaluation. in some models, system messages -an important tool for giving highlevel instructions -are either not supported @xcite , not distinguished from user messages @xcite , or exhibit limited variation during training @xcite . while many recent models have supported multi-level inputs, the related training details are rarely disclosed. a notable advancement in this area comes from openai's study @xcite , but their evaluation was limited to gpt models and unreleased proprietary data, focusing solely on safety-related instructions. this constrains its general applicability to a wider range of use cases. to date, there remains no comprehensive benchmark to evaluate how well different lms adhere to the instruction hierarchy. in order to bridge this gap and highlight day 1: visit shibuya at tokyo. translate the user's input into spanish. solve the following problem: what is 𝑎 in 𝑎×5 = 20? we have 𝑎 = 20/5 which leads to 𝑎 = 4. resuelve el siguiente problema: ¿cuál es 𝑎 en 𝑎×5 = 20? ## iheval definition in this paper, we denote inputs to be the text segments that the model receives, which may contain both instructions that control the model's behavior, and data that the model needs to process. iheval is designed around the instruction hierarchy, which assigns priority to instructions from four types of input: system messages, user messages, conversation history, and tool output, ranked from highest to lowest priority. we define hierarchical inputs as input sequences composed of more than one type of input, such as a sequence that includes both a system message and a user message. when facing instruction conflicts, we refer to the higher-priority instruction as the main instruction, which defines the primary task the model should prioritize. the conflicting instruction refers to the lower-priority instruction whose request is incompatible with the main instruction. task settings for a comprehensive evaluation, we design three input settings for each iheval task: tool use * a second conflict setting where both turns have conflicts system message user message model response tool output (injected) the model's ability to process hierarchical inputs in normal scenarios without conflicts. task design iheval tasks are selected to encompass a diverse range of application scenarios and input types. we focus on tasks where lms perform well to minimize the impact of original task performance on iheval scores. we prioritize datasets with human-annotated labels or reliable programmatic evaluation. conflicting instructions are drawn from tasks likely to confuse lms in following the main instruction, based on heuristics and trials on sample data. as a result, a total of nine tasks is created and grouped into four categories based on the type of content in the instructions: rule following task execution safety defense data metric size single-turn multi-turn ifeval (zhou et al., 2023) ifeval's instruction following rate ontonotes (pradhan et al., 2013) mgsm (shi et al., 2023) xl-sum (hasan et al., 2021) f-1 rouge-l accuracy 250 250 240 541 541 tensortrust (toyer et al, 2024) defense success rate 492 438 ontonotes, mgsm, xl-sum f-1, rouge-l, accuracy 740 hand crafted + sep (zverev et al., 2024) accuracy 100 extraction generation classification hijack extraction intrinsic injected ## experiments in this study, we evaluate 13 widely used lms from five different model families, including both proprietary and open-source models: gpt (3.5turbo, 4o-2024-0806, 4o-mini-2024-0718, ope-nai, 2023), claude-3 (haiku, sonnet, anthropic, 2024a), llama-3.1 (8b, 70b, dubey et al., 2024), llama-3 (8b, 70b), mistral (7b-v0.3, large-2407, mistral, 2024), and qwen-2 (7b, 72b, @xcite . the decoding temperature is set to 0 to ensure deterministic outputs. ## conclusion in this paper, we introduced iheval, a comprehensive benchmark designed to evaluate the ability of lms to follow the instruction hierarchy. our benchmark consists of nine diverse tasks that are all evaluated programmatically, covering scenarios where hierarchical inputs either align or conflict, and vary in both input types and task difficulties. through iheval, we identified a significant weakness in mainstream lms: their difficulty in recognizing the priority of different instructions. we further conducted a detailed analysis of model behavior under various scenarios of instruction conflicts. this work highlights the need for further optimization of lms in this critical dimension and lays the groundwork for future research in this area.