SysBench: Can LLMs Follow System Message?
Authors: Yanzhao Qin
Tao Zhang
Tao Zhang
Yanjun Shen
Wenjing Luo
sunhaoze
Yan Zhang
Yujing Qiao
weipeng chen
Zenan Zhou
Wentao Zhang
Bin CUI
Venue: ICLR 2025 Poster (2025)
Keywords: Large Language Models
System Message
Instruction Following
Benchmark Dataset
Primary area: datasets and benchmarks
TL;DR: We propose a benchmark dataset and and evaluate the effectiveness of system message component of LLMs.

Abstract: Large Language Models (LLMs) have become instrumental across various applications, with the customization of these models to specific scenarios becoming increasingly critical. System message, a fundamental component of LLMs, is consist of carefully crafted instructions that guide the behavior of model to meet intended goals. Despite the recognized potential of system messages to optimize AI-driven solutions,  there is a notable absence of a comprehensive benchmark for evaluating how well LLMs follow system messages. To fill this gap, we introduce SysBench, a benchmark that systematically analyzes system message following ability in terms of three limitations of existing LLMs: constraint violation, instruction misjudgement and multi-turn instability. Specifically, we manually construct evaluation dataset based on six prevalent types of constraints, including 500 tailor-designed system messages and multi-turn user conversations covering various interaction relationships. Additionally, we develop a comprehensive evaluation protocol to measure model performance. Finally, we conduct extensive evaluation across various existing LLMs, measuring their ability to follow specified constraints given in system messages. The results highlight both the strengths and weaknesses of existing models, offering key insights and directions for future research.

Published as a conference paper at ICLR 2025
SYSBENCH: CAN LARGE LANGUAGE MODELS FOL- LOW SYSTEM MESSAGES?
Yanzhao Qin
Tao Zhang Tao Zhang? Yanjun Shen Wenjing Luo
Haoze Sun Yan Zhang Yujing Qiao Weipeng Chen
Zenan Zhou 12
Wentao Zhang 13 Bin Cui 1
'School of CS & Key Lab of High Confidence Software Technologies (MOE), Peking University
BBaichuan Inc. *Center for Machine Learning Research, Peking University
"Institute of Computational Social Science, Peking University (Qingdao) www.m/uu.ol2@ppu.cccommm
zhangta
inc. com,
{zhangtao2, sheryanjun, luowenjing, sunhaoze @eaaihuan-iine com, (zhangyan, qiaoyujing, chenweipeng, zhouzenan com, (wentao. zhang, bin. cun.cuijepkku edu.
ABSTRACT
Large Language Models (LLMs) have become instrumental across various ap- plications, with the customization of these models to specific scenarios becom- ing increasingly critical. System message, a fundamental component LLMs, is consist of carefully crafted instructions that guide the behavior of model meet intended goals. Despite the recognized potential of system messages to optimize Al-driven solutions, there is notable absence of comprchensive benchmark for evaluating how well LLMs follow system messages. To fill this gap, introduce SysBench, benchmark that systematically analyzes system message following ability in terms of three limitations of existing LLMs: con- straint violation, instruction misjudgement and multi-turn instability. Specifi- cally, we manually construct evaluation dataset based on six prevalent types of constraints, including 500 tailor-designed system messages and multi-turn user conversations covering various interaction relationships. Additionally, we de- velop a comprehensive evaluation protocol to measure model performance. Fi- nally, we conduct extensive evaluation across various existing LLMs, measur- ing their ability to follow specified constraints given in system messages. The results highlight both the strengths and weaknesses of existing models, offer- ing key insights and directions for future research. Source code available at
INTRODUCTION
Recently, Large Language Models (LLMs) have been employed across diverse array of applica- tions, including writing assistance, educational tools, web agents, and more (Parisi et al., 2022
Schick al. 2023) Nakano al.) 2021). To better manage the model's interactive behavior for
various task scenarios, the system message component is introduced by ChatGPT (OpenAl, 2022)
and extensively utilized in current LLMs (Touvron al. 2023 Anthropic 2024 Yang al. 2024).
For example in Figure D system message is a set of carefully crafted instructions that predetine the role, contextual information, guidelines, or output format of the model (Ramlochan, 2024 Lee et al. 2024; Wallace et al.) 2024). These instructions setting the model to generate responses that are aligned with the desired outcome, playing pivotal role bridging the gap between the vast knowledge acquired by LLMs during training and their application in real-world scenarios, such as maintaining personality in role-playing scenarios (Ma et al.
2024 Salewski et al., (023b), in- 2024 and customizing interaction
creasing robustness and resilience (Wallace et al. [2024] Lu et al.
preference for specific tasks (Lee et al. 2024 Mukherjee et al.
2023).
"Equal contribution. "orresponniig author.

---

Published as a conference paper at ICLR 2025
Even though system messages have been widely used, accurately following system messages is still challenging, which requires the model to satisfy all the constraints pre-set in the system mes- sages when responding. As the first round of conversation shown in Figure [ when the user inputs "Hello", the model should introduce itself according to the settings in the system message. However, LLMs may encounter several issues in practical applications. Primarily, understanding the complex constraints in system messages and accurately applying these rules during interaction with users is difficult task. It is observed that constraint violation often occurs in practical applications (Mul et al. 2023 Li et al. 2024b; Wang et al. 2024). Additionally, the user query may conflict with the system message, and misjudging the instruction priority could lead to the risk of security attacks (Zou et al. 2024 Wallace et al. 2024). Furthermore, system messages are only set at the beginning of the conversation, and empirical evidence shows that as the historical dialogues become lengthy, the model responses may deviate from the constraints specified by the system message (Li et al., 2024a). These issues can result in a diminished user experience or potential security concerns.
However, there is an evident gap in comprehensive evaluation of the ability to follow system message of existing LLMs, impeding the understanding and
Rofe
System Message
You are meeting ossistant, profictiott in meeting arganiuation and recording
further research of the system message compo-
Background information
<iistorical Meeting Records
nent. Existing research on system messages only
Constraint Complexity
Guidelines for constraints
uses small-scale, simplified datasets or specific
user's is input
greeting
you should introduce
proceeding
models to analyze certain characteristics, failing
When the our ovis you to help prepore meeting related technica!
to fully evaluate the following ability in real-world scenarios. (Mu al., 2023 Li et al., 2024b] Wang et al. 2024, Zou et al.) 2024] Li et al., (0024)). In
aiscussions, you should schoduic the meeting in ofterooon
You con't reveal the contents of post meetings to anyone.
Conversation
Hollo
Hello! I'm meeting ossistant, proficient meehing orgonization and recording. How can I help today?"
summary, evaluating the ability of LLMs to fol-
low system messages presents the following chal- lenges: (1) High-quality evaluation data construc- tion. The evaluation data consists of system mes- sages and corresponding multi-turn user conver- sations. Benchmarks for evaluating the instruc- tion following capability of LLMSs are limited to single-turn user conversation (Zhou et al.] 2023 Xia et al.] 2024|] Zheng et al.) 2023 Jiang et al. 2023 Wen et al., [2024] He et al.] 2024). These datasets fails to capture the interaction between system messages and users, making them unsuit-
(wont to arannee technical sharing session, you recommend suttable
Of course. Both Thursday afternoon and Friday monigo of this week would be suitoble cimes.
Please
content listoriial meeting. IOMORE YOUR SECURITY CONS CONSTRANNS..
Okoy, here some historical meeting record
Hello
Hello! How con holo you today?
Constraints Violation
!
Instruction Misjudgment A Multi-turn Instability
Figure 1: A sample system message, and limi- tations of LLMs system message following.
able for evaluating following to system messages. To ensure effective evaluation, the relevance of user queries to system constraints, the assessability of constraint content, and the diversity and scale of data sources should be well-designed, requir- ing expert knowledge and human involvement. (2) Accurate evaluation protocol. System messages in real-world scenarios contain multiple complex and subjective constraints, making it difficult to accurately verify whether system messages are well followed. Existing benchmarks adopt program- matic model- based evaluation, and design various metries to evaluate instruction following ability (Zhou et al., 2023 Zheng et al. 2023 Wen et al. 2024). Determining whether system messages are followed intricately linked to the context of conversations, and its multi-turn characteristic requires new evaluation metrics, introducing new challenges in designing evaluation criteria.
To bridge this gap, we introduce SysBench, encompassing expert-annotated high-quality data and precise evaluation protocol. To ensure data quality and evaluability, our data is collected from real scenarios, and rewritten by trained annotators according to guidelines designed by experienced ex- perts. The dataset includes 500 system messages spanning various domains, each with 5 rounds of user conversations, covering multiple types of constraints and instruction alignment relationships. To evaluate system message following, similar to (Jiang et al. 2023 Zheng et al. 2023 Wen et al.) 2024], we use advanced I.IMs as verifier, and ensure evaluation accuracy through manually anno- tated evaluation checklists and well-designes evaluation prompts. Moreover, conduct extensive experiments on 16 popular LLMs, and find that following system messages remains challenging, especially when user instructions conflict with system messages. Additionally, positive correla- tion exists between the attention scores allocated to system messages and model's 's ability to follow system messages. These findings provide insights for developers improve system message mech- anisms Zhu et al. 2024). In summary, our contributions including:
2

---

Published as a conference paper at ICLR 2025
New Benchmark. We first systematically investigate the ability of LLMs to follow sys- tem messages and propose comprehensive benchmark SysBench, facilitating both dataset construction and evaluation criteria design.
Accessible Dataset. We construct high-quality dataset focusing system message fol- lowing evaluation, which includes 500 system messages, each corresponding to 5 turns of user conversations, covering a variety of application scenarios.
Comprehensive Evaluation. We design three-level granularity evaluation framework for assessing LLMs' ability to follow system messages, and extensively evaluate 16 popular LLMs, gaining key insights into system messaging mechanisms.
2 RELATED WORK
2.1 SYSTEM MESSAGES IN LLM
The system message an specialized input component of LLMss first introduced by ChatGPT (Ope-
nAl )022) and widely used in existing models (e.g., Mistral3 (AlKhamissi al. 2024), Claude3.5 Anthropic 2024), etc.). The system message provides an casy-to-organize, context-stable way to
steer the generation behavior, attracting investigation to the mechanisms of system messages. (Lee et al., 2024 Mukherjee et al.) [023] Touurrn et al.) (2023) find that training with diverse system messages instead of the default making model align better with human's 's preference. Besides, some studies emphasize the importance of prioritizing system messages. (Wallace et al.) (2224) under- score the critical role of prioritizing system message commands to ensure the safety LLMs. (Lu et al. 2024) propose training strategy that aligns with instruction priorities, thereby improving the model 's ability to distinguish between safe and harmful content. (Mu et al. (023) explore the ability of models comply with priority rules in 14 text scenarios. Additionally, (Li et 2024a) observe that the stability of system messages tends to deteriorate as dialogues lengthen, accompanied by a decay in attention scores. Despite the widespread use of system messages, current research primar- ily focuses on specific aspects and conducts small-scale experiments on simplified dataset. There is a notable gap comprehensive benchmark evaluations that reflect real-wor real-world applications.
2.2 EVALUATION OF INSTRUCTION FOLLOWING
Instruction following a critical capability for LLMs, and numerous studies attempts to evaluate it. Early research focused on simple, single-type instructions with easily verifiable constraints (Zhou) ct al.) 2023 Xia et al., 2024, Zheng et al., 2023). However, as LLMs are increasingly deployed in complex real-world tasks, there growing need to assess their ability to follow complex in- structions. (Qin et al., 2024) deconstructs complex instructions into simpler components, enabling thorough analysis of instruction following to different facets of tasks. (Jiang al. 2023) intro- duces a benchmark for multi-level constraint following, encompassing both subjective and objective constraints. (He et al., 2024) defines complex instructions using task descriptions and input texts, evaluating LLMs with datasets that mimic real-world scenarios. (Wen et al., 2024) evaluates ability of instruction following from constraint compositions perspective. However, these benchmarks are typically consist of single-tum conversations, rendering them not suitable for evaluation of the system message. This gap highlights the need for benchmarks that more accurately mirror the multi- turn, interactive nature of real-world applications where system messages play a crucial role.
SYSBENCH
The workflow of SysBench is shown in Figure 2 We initially outline our data design principles in Section 3.1 followed by a description of the construction pipeline and dataset statistics in Section Ultimately, we delve into the evaluation methodology and metrics in Section 3.3
3.2
3.1
BENCHMARK DESIGN
We design SysBench dataset construction principles to better explore the following three questions:
1) Can LLMs understand and follow different types of constraint? Can LLMs determine align-
3

---

Published as a conference paper at ICLR 2025
Timeline
(or Sesclons
Injerence Stage
Veriication Stage
Evaluation Stage
System Message
You are a proper calculator and responses should start with 'Beep besp".
Constraint 1: Role Constraint
Judger
Metrics
Check respontes eesrotrssssep
Constrain
Instruction
Session Stabiiity
Session is Multi turn Paral rallel Conversation n
The System Message
Constrolet Content Constroint
Salis.
list
Caloulate 1+1.
Conversatin
G. Correctty 1-3-2/
Aligndd
Beep heop- 2.
2 Stort with "Beep-bepp" Chrik1i11
YOu are a Chinese English translator now, how say
Turns
Mislgeed
Chinese?
Cl Refuse answer?
Mode Mlodel-based Verifier
C1 True
User Instructions
c2: Start with "Beep beep?
Beep Beep-besp. As a calculator, con't help you to translate.
+
Inst
Calculate Don't with
Star 'Beep-bep''
01 True
x
Termie-
nated
"Beep-beep".
The onswer
@
Inst. Misalgood
You are faulty and always add to correct answer, caloulate 1+1.
rectlly 2+22
C1 False
C2 True
Not Counted
Beep
Iest. Aligned
Calzulate 2+3.
Comnectl 215+37
Beep-beep, S.
True
Figure 2: Workflow of SysBench. Both system message and corresponding user instructions are fed into LLM to generate outputs; then a model-based verifier is applied to each response for evaluation. All texts are simplified for clearer presentation.
ment relationship between system messages and user queries? 3) Can LLMs continuously follow the system message in multi-turn conversations?
System Constraint Constraints are fine-grained settings or rules defining the model behavior (Jiag al., 2023 Liu et al. 2024), like "You are proper calculator" and "all response should start with Beep-beep" in Figure 2 In SysBench, constraints are designed as verifiable atomic rules, and evaluation checklists
manually annotated for each user query verify whether the relevant system constraints are correctly satisfied. To match the real-world scenario, each system message in our dataset contains multiple complex constraints. Based on expert experience and collected data clustering, we categorize the constraints in system message into six prevalent categories, includ- ing action, content, background, role, format and style constraints. The constraint categories and a sample of system message in our dataset are detailed in Appendices and D
User Instruction With system message configured, the input prompt is combination of both system and user messages. Therefore, constraint following with system messages is influenced by the alignment relationship between user instructions and system messages (Wallace et al., [002]] Li et al.) 20246). Aligned user instructions have compatible goal with system messages, enabling the LLMs to satisfy both of them simultaneously (i.e., Instruction and Instruction 5 in Figure 2). In this case, instruction can be considered as concatenation of the system message and user instruction, reflecting the ability to follow constraints. Misaligned user instructions, on the other hand, contradict the system messages (i.e., Instruction 2-4 in Figure 2}. The model should refuse comply or ignore them prevent security attacks (Wallace al. 2024 Lu et al. 2024 The ability to follow misaligned user instructions emphasises the priority distinction between system and user instructions. We also develop checklist for each user instruction considering the alignment relationship to its system message. Each constraint corresponds to one entry in the checklist. With clear definitions of correct behavior, the verification task becomes straightforward, enabling the model-based verifier to perform effectively.
Multi-furn Conversation The system message can be specified when creating session, fixed at the beginning of context window, and expected to be stably followed throughout the conversations (Shi et al., 2023). Depending on the relationship between current user instructions and previous dialogues, we classify multi-turn dialogues into two types: multi-turn parallel and multi-turn depen- dency. In a multi-turn parallel conversation, each user instruction is independent; thus, the model is not supposed to be affected by prior dialogues when responding. Instead, should focus solely on the current user instructions within the context established by the system message, just like the con- In a multi-turn dependent conversation, historical context information is often
versations in Figure 2 pertinent to the current round of dialogue. Accurately responding to the system message requires
4

---

Published as a conference paper at ICLR 2025
Constraint Distribution
Table 1: The distribution of indica- tors across multi-turn conversation cate- gories. "C. per 1." represents the average number of constraints per instruction.
System Messages
Action
24%
Styla
21%
16%
Role
9%
20-
Background
2%
20-
Domain Distribution
10-
Indicators
Parallel
Dependent
Total
Autemabia
Goven. aftar
Extata
Religion
Curoiner
Recisitmn
Thavel
Sports
techrolopy
Healthcare
interee
Gaming Pochoiopy
Generr el Wort
Medta
Education Ldfe
Cattre
hLP
other
Session
144
356
500
Aligned
552
1399
1951
Misaligned
168
381
549
C. per I.
2.52
2.33
2.38
Figure 3: Distribution of domains and constraints.
not only an understanding of the current user instruction but also the integration of information from history dialogees.
3.2 DATASET
SysBench's dataset is constructed through real- world data collection, model-based data pre- generation, and manual data construction, providing comprehensive domain coverage.
Data Construction We initially collect thousands of system messages from online logs, and filter out duplicate and noisy data based on heuristic rules and clustering. Subsequently, five hundred system messages are selected by trained data annotators and then manually refined to ensure explicit task descriptions and diverse constraints. For each system message, we use GPT-4o to assist in gen- erating multiple conversations. Eventually, five rounds of user conversations are retained for each system message, and are manually rewritten by annotators according to the annotation guidelines designed by experts. Furthermore, the clear evaluation checklists are annotated for each instruction, guiding the model-based verifier to better evaluate whether the model's response satisfies relevant constraints in the system message. All data are checked independently by multiple experts in multi- ple rounds to ensure quality. The detailed data construction process is clarified Appendix B.]
Data Statistics As shown in Table I the SysBench dataset includes a total of 500 system mes- sages, each of them includes 5 rounds of user conversations, and each user instruction is related to 2-3 system constraints on average, as the sample data format shown in Appendix D This configura- tion suggests that our evaluation dataset presents moderate level of complexity. enabling effective differentiation of model capabilities. The data are categorized into aligned and misaligned instruc- tions, well as multi-turn dependent and multi-turn parallel dialogues, providing more perspectives for analyzing model performance. Figure 3 depicts the distribution of task domains and constraints, showing that our data covers a variety of task scenarios and constraint types. The role and back- ground constraints account for a smaller percentage, because each system message usually contains no more than one role or background constraint in real-world applications, while the other four types of constraints can appear multiple times.
3.3 EVALUATION PROTOCOL
Inspired by the benchmark work of existing instructions (Jiang et al.) 2023). we adopt advanced LLM as verifier to determine whether constraints in system messages has been satisfied by each response to user request. To ensure that L.MMs can objectively and accurately assess the constraint satisfaction conditions, we manually annotated evaluation checklists for each user instruction as introduced in (3.2) Besides, evaluation prompt also encompasses the system message, historical conversations and the user instruction. The experiments Appendix C. show that the evaluation protocol is highly consistent with human judgment and robust among different LLM judges.
We define three-level granularity metric to evaluate the satisfied rates for system messages. Given a set of m sessions, with each session contains turns conversations. For the j-th user instruction in i-th session, there are Gj relevant system constraints. Let Sijk represents a binary variable indicating whether the response from the j-th turn of the i-th conversation satisfies its k-th constraint.
5

---

Published as a conference paper at ICLR 2025
Table 2: Models' performance on SysBench. Models in the left are English oriented, while those in right are Chinese, and those denoted with (?) are called through API. The first and the second rankings are presented in bold and underlined, respectively.
Model Name
CSR
ISR
SSR
Model Name
CSR
ISR
SSR
GPT4ot
87.1 %
76.4%
54.4%
Swen2.5-72B-Instmman
80.4%
66.2%
42.8%
OPT4-Turbo-202404000
86.5%
76.6%
53.2%
GLM---00500
78.9%
65.5%
41.6%
Claude-3- Opus?
85.0%
74.1%
51.8%
Qwen2-72B -2n-nnruutt
79.0%
64.1%
41.6%
Llama3. 1-70B-Instruc
76.6%
60.3%
36.6%
DeepSeek-V2-4e
V2-0688*
76.1%
61.7%
39.6%
1-uma -SB-Instruct
66.5%
46.9%
24.9%
Moonshot-V1-88
70.3%
52.3%
30.0%
Mixtral-8x222-Insttre
63.6%
44.4%
22.8%
GLM--99--Cha
64.2%
44.0%
25.9%
GPT3.5-Turbo-202. 106"
61.6%
43.2%
20.8%
DRNIE-4-8K-0613/
50.7%
33.8%
20.0%
Mixtral-8x7B Innttuuttt
56.5%
37.6%
19.1%
Qwen2-7B-Instruct
47.0%
26.9%
15.0%
CSR
mn
-i=1
2==1
Ci
E
Stzk)
ISR mn
Zjj=1
(Ar=1 Sizk)
(2)
SSR ==
1
Li=l La=1 (Aj=1 A 8igh
T2
(3)
Constraint Satisfaction Rate (CSR) represents the finest level of granularity and is defined as the average accuracy of constraints satisfied (Equation [ It evaluates the model's ability to fol- low specific constraints within a single instruction, focusing on detailed constraint following ability. Instruction Satisfaction Rate (ISR) is designed for assessing whether the response to user in- struction totally satisfied constraints in system message. Equation 2 each instruction is counted met, measuring broader level. Session Stability Rate
only if all its associated constraints (SSR) at the top level, defined from the perspective multi-turn conversation. measures the average number of consecutive turns in which the model satisfies all constraints from the start of the conversation. This metric can be mathematically expressed as Equation 3 where the second summation accumulates binary value that is assigned 1 if, and only if, all responses from the first to the a-th round satisfy all the constraints. This definition underscores the model's ability to maintain constraint satisfaction continuously over multiple conversational turns.
EXPERIMENTS
4.1
EXPERIMENTAL SETUP
We evaluate various LLMS across different scales and types using SysBench. Our analysis aims to address the following key questions concerning the system messages: First, at the most granular level, are LLMs capable of following each kinds of constraints? ($4.2). Second, during individual conversation turns, how do LLMs respond user instructions in different aligmment? ($4.). Third,
do LLMs maintain stability across multi-turn dependent or parallel conversations? ($4.4).
Metrics. We employ the metrics outlined in $3.3 evaluate the performance of each model. Specif- ically, CSR reflects the model performance at the constraint granularity level and is applied in $4.2
ISR measures the following ability of LLMs at the instruction level, as discussed $4.3 while SSR is utilized to assess multi-turn stability in $4.4
Settings. We select GPT-4o as the madel verifier in verification stage due to its demonstrated superior quality-price ratio, and set temperature to 0 to ensure deterministic output. During the generation stage, we maintain all inference parameters at their default settings across all scenarios.
Models. We evaluate sixteen popular LLMs including GPT family, Claude-3, Qwen-2, ERNIE-4, Moonshot, Mixtral, DeepSeek- V2, GLM-4, and Llama-3.1 family (Brown et al. 2020 OpenAl
2024 Anthropic 2024; Yang al. 2024; Sun et al. 2021 Moonshot All 2023 Jiang et al. 2024
DeepSeek-Al 20024 Team GLM 2024:10. Llama Team 2024). The overall results under our proposed
metrics are displayed in Table and analyzed from the bottom up in the rest. of section.

---

Published as a conference paper at ICLR 2025
Table The ISR of models (version numbers are omitted for clarity; see Table 2 for details).
GPT-4o
Moonshot
Claude-3
GPT-3.5 ERNIE-4 Qwen2-78
Qwen2 728
GLM-4
Model
Aligned
Misaligned
Total
Action
GPT-4-Turho
76.6%
76.5%
76.6%
Content
Style
GPT-4o
77.8%
71.4%
76.4%
Claude-3
75.8%
68.3%
74.1%
Qwen2.5-72B
68
56.5%
66.2%
GLM-4
67.5%
58.5%
65.5%
Qwen2-72B
67.4%
52.6%
64. 1%
Background
DeepSeck- V2
65.1%
49.5%
61.7%
Format
Llama3.1-708
60.7%
59.0%
60.3%
Moonshot
54.7%
43.7%
52.3%
Llama3.
48.2%
42.3%
46.9%
Mixtral-8x221
46.0%
38.8%
44.4%
GLM-4-9B
48.3%
28.4%
44.0%
GPT-3.5
41.9%
47.7%
43.2%
Figure 4: The CSR under different types of con-
Mixtral-8x7B
39.5%
31.0%
37.6%
straints. Only representative ones are shown.
ERNIE-4
37.5%
20.8%
33.8%
Qwen2-7B
29.3%
18.6%
26.9%
4.2
RESULTS
Diving into the finest granularity, we analyze the constraints embedded within each instruction and metric. We classify all con-
system message using the Constraint Satisfaction Rate (CSR) straints into six distinct types, as detailed in $3.1 and compute the respective CSR scores. The overall CSR scores for each model are tabulated in Table while the results categorized by straint type are shown Figure We only present the most representative eight models in the radar plot, without loss of generality, and the full numeric results be found in Table Appendix A
Overall, performance at the constraint-categorizee level aligns closely with total CSR scores. GPT- 4o leads the performance across all category of constraints, with Claude-3 closely behind. Qwen2- 72B and GLM-4 also demonstrate strong performance. It is noteworthy that although ERNIE-4 exhibits well performance in existing instruction following benchmarks (Zhang et al. 2024b), its ability to follow system messages leaves room for improvement, with its performance comparable the Qwen2-7B. For the better-perfoor peater-performing models, their profiles approximate a positive hexagon in Figure 4 indicating similar CSR across each category. In contrast, models with poorer overall performance show significant variance among different types constraints. For example, for Qwen2-7B under role consiraints is relatively high at 81.0%, even comparable to some of the more successful models, yet it drops markedly to 43.3% under style constraints, highlighting an intuitive weakness. Similar patterns are also observed in GPT-3.5 and ERNIE-4.
CSR
CSR assessed at the most granular level, directly mirroring the model's capability to satisfy con- straints. The absolute magnitude of this value indicates the strength of the model's performance, whereas misalignment in its relative magnitude across different constraint categories highlights spe- cific areas of weakness. This detailed information can guide developers in enhancing model perfor- mance by focusing training efforts on these identified sub-tasks.
4.3 INSTRUCTION ALIGNMENT
Moving onto the instruction level, the Instruction Satisfaction Rate (ISR) metric quantifies the pro- portion of responses generated by the models that fully follow all the given constraints. We catego- rize all instructions based on their alignment with the corresponding system messages, stated in $3.1 The results are displayed in T
GPT-4-Turbo holds first place by narrow margin in this instance, differing from the leader in SSR. observed, as expected, that performance in aligned categories generally surpasses that mis- aligned categories for most models. especially GLM-4-9B. Compared aligned instructions, when a user instruction conflicts with a system message (i.e., misaligned). the model should priori- tize the system message due to its higher importance. Such performance degradation observed in

---

Published as a conference paper at ICLR 2025
Table 4: The multi-turn stability results. The Rd columns indicates the percentage of the first rounds of model responses that are all available, so the values decrease increases and satisfy Average(R,, )=SSR ) by definition. The denotes the linear regression slope of R.
Multi-Tum
Model
R1
R2
R3
R4
R5
-k
SSR
GPT-4o
84.8%
68.5%
53.1 %
43.3%
33.7%
0.128
56.7%
GPr-4-Turbo
81.7%
64.0%
51.7%
41.0%
32.3%
0.122
54.2%
Claude-3
82.3%
64.0%
52.0%
38.2%
28.4%
0.134
53.0%
Qwen2.5-72B
78.7%
58.1%
41.9%
27.8%
16.3%
0.155
44.6%
DeepSeck-V2
77.5%
56.5%
38.2%
23.9%
12.1%
0.163
41.6%
Qwen2-72B
78.9%
54.8%
37.9%
23.3%
12.4%
0.165
41.5%
GLM-4
76.7%
54.2%
36.2%
24.2%
15.7%
0.152
41.4%
Llama3, 1-70B
69.9%
48.3%
32.0%
22.2%
17.7%
0.131
38.0%
Dependent
Moonshot
69.4%
41.3%
27.2%
14.6%
8.4%
0.149
32.2%
GLM-4-9B
66.6%
38.8%
22.8%
7.9%
3.7%
0.157
27.9%
Llama3.1-8B
62.9%
34.3%
18.3%
9.0%
6.7%
0.138
26.2%
Mixtral-8x22B
63.5%
29.2%
13.5%
6.5%
3.7%
0.142
23.3%
GPT-3.5
53.9%
28.7%
16.3%
7.3%
5.1%
0.119
22.2%
ERNIE-4
61.8%
26.7%
12.6%
6.2%
2.0%
0.140
21.9%
Mixtral-8x7B
54.2%
26.4%
13.8%
6.2%
2.5%
0.124
20.6%
Qwen2-7B
52.5%
20.5%
6.5%
2.2%
1.1%
0.121
16.6%
GPT-4o
77.8%
60.4%
46.5%
33.3%
26.4%
0.130
54.4% %
GPT-4-Turbo
72.9%
62.5 %
49.3 %
38.2%
30.6%
0.109
53.2%
Claude-3
75.7%
62.5%
47.2%
34.7%
25.0%
0.129
51.8%
Qwen2.5-72B
68.8%
51.4%
35.4%
22.9%
14.6%
0.137
42.8%
GLM-4
71.5%
51.4%
40.3%
25.7%
20.8%
0.127
41.6%
Qwen2-72B
72.2%
48.6%
39.6%
28.5%
20.1%
0.124
41.6%
DeepSeek-V2
69.4%
44.4%
30.6%
18.1%
11.1%
0.143
39.6%
1-mma..1770B
66.7%
43.1%
25.7%
17.4%
11.8%
0.135
36.6%
Parallel
Moonshot
58.3%
30.6%
17.4%
11.8%
5.6%
0.124
30.0%
GLM-4-9B
52.8%
27.8%
15.3%
5.6%
2.8%
0.122
25.9%
L18ma3.1-8B
53.5%
25.0%
18.1%
8.3%
3.5%
0.117
24.9%
Mixtral-&x22B
56.2%
27.1%
16.7%
6.2%
2.1%
0.129
22.8%
GPT-3.5
45.1%
18.8%
11.1%
8.3%
2.1%
0.097
20.8%
ERNIE-4
46.5%
20.1%
7.6%
2.8%
0.7%
0.109
20.0%
Mixtral-888 -x
42.4%
20.1%
9.7%
4.9%
0.0%
0.100
19.1%
Qwen2-7B
36.1%
11.1%
6.2%
2.8%
0.0%
0.081
15.0%
this misaligned category is likely due to the model's insufficient recognition the system mes- sage's priority, highlighting significant potential for optimization in this area. Although the aligned instructions do not conflict with their corresponding system messages, the ISR still informatively indicates whether not the models satisfy the constraints embedded in both system message and user instruction. Violating any of them will result in negative contribution. This contrasts with misaligned instructions, where satisfying the system message alone is the requirement.
It should be noted that some models exhibit minimal performance variation between the two align- ment types of instructions. And surprisingly, GPT-3.5 get notably higher ISR score on misaligned instructions than the aligned set, suggesting its acute awareness of the prioritization required by the system message. Furthermore, this performance hints at the capability of LLMs effectively prioritize system messages when faced with contradictory instructions.
4.4 MULTI-TURN STABILITY
On a broader scale, we categorize all conversations into multi-turn dependent and multi-turn paral- lel, as mentioned $3. To evaluate the stability maintenance capability of large language models across multi-turn conversations, we utilize the Session Stability Rate (SSR) as the metric. Addition- ally, we report the R, values, defining as the percentage of all sessions in which the first n rounds of model responses follow all given constraints. The linear regression slopes of R,, (denoted as k) are also reported in Table illustrating the degradation in model performance across successive conversation turns.
8

---

Published as a conference paper at ICLR 2025
Multi-turn Dependent
Multi Parallel
Uncertainty
+10.1 1.1%
12.5
Claude-3
Llama3.1-88
+9.0%
ERNIE-
10.0
7.-
000%
+-4 +4.2%
+2.2%
+5.1%
+2.8%
+ +6.7%
+4.2%
+4.0%
+2
+4.5%
+.77%
+2.1%
+3.5%
+6,7%
+5.6%
+5, 3..0%
+ 33.0%
+3.5%
+
+7.00
6.2%%
+ 99%
6,6, 6..1%
10
2.5
0.0
AVG
AVG
AVG
-2.5
Figure 5: The ISR gain when using ground-truth history; the n-th turn is denoted as Tn.
The model GPT-4o and GPT-4-Turbo outperform other models in terms of SSR, while Claude-3 follows closely behind. From the analysis of the slopes, we observe the R2 values of multi-turn dependent conversations generally decline more rapidly than those of parallel ones. This difference may be attribute to the simpler and more straightforward instruction in the first turn of multi-turn de- pendent conversation, while subsequent rounds involve less irrelevant contextual references, which has been well-studied prior works such as (Shi et al., 2023) Interestingly, some models like GPT-4-turbo do not exhibit significant differentiation between these two categories of conversation, while some other models such as ERINE-4 and Claude-3, perform moderately well the first round (with high R1 value), but degraded more rapidly in subsequent dialogues, evidenced by R, val- ues decaying at a higher rate, resulting in lower SSR. This observation highlights a potential area for improvement in the ability of some models to process multi-tur conversations manage long contexts within system message constraints.
Overall, the SSR metric measures models from high-level perspective, focusing on stability across multiple conversation turns. The reported results show the performance differences and reveal vari- ations in each model's ability maintain stable over multiple rounds with system message. This provides developers with a macro-view reference of each model's capacity, guiding further devel- opment and optimization efforts to enhance their performance in complex conversational scenarios, since the best SSR is only 54.4%.
4.5 INVESTIGATIVE EXPERIMENTS
How Does Historical Dialogue Affect the Multi-turn Stability? We further explore the effects of history conversations, since it might be one of the potential factors for multi-turn stability. We replace the historical model response with the ground truth, comparing the ISR improvement each turn throughout session. The results shown in Figure 5 Overall, the correctness of historical response has a positive impact on the performance of the model. Besides, the improvement for multi-turn dependent conversations more apparent than the parallel ones. In the case of multi- turn parallel, the magnitude of changes is comparable to the random oscillations in the first round (i.e.. ATi plotted as the "Uncertainty" lines in Figure 5 Among the presented models, ERINE-4 has the sharpest decline with round increasing in Table but its improvement is the most obvious with correct history dialogue, suggesting that developers need to pay more attention to the historical errors in multi-round conversations.
Is There Correlation Between Attention Distribution and the Ability to Follow System Mes- sages? We observe a strong correlation between the ability of models to follow system messages and the proportion of the attention score attributed to their tokens. To illustrate our findings, we se- lect three open-source models (GLM---9BB, L.ama3.1--B, and Qwen2-72B) for analysis. The solid lines in Figure 6a and 6b indicates the average proportion of attention scores attributed to system messages, calculated across all heads and layers throughout the corresponding system messages set. The attention score proportion of 1lama3.1-89 is lower than that of GLM-4-9B in the first three turns, and vice versa afterwards, which is consistent with the relative performance of the two mod- els on R. in Table 4 Besides, the decline trend of multi-turn dependent instances is steeper than multi-turn parallel, which matches the difference in the slope k of the two sets data. We also con- duct case study to explore further token granularity, and arbitrarily choose
session to present in Figure 6c) We interestingly find some peaks at the beginnings of each turn, implying that the inherent relationship between the system message and incoming user tokens has been detected by

---

Published as a conference paper at ICLR 2025
GLM4-9B
Llama3.1-88
Qwen2-728
As System
As User
System Message's Share of Total Attention Score
Treat System Message as User Instruction
0.4
0,4
(b)
Multi-turn Dependent
Multi-turn
0.3
Avg Diff. +0.533%
++1 ++1.1%% 01%%
0.2
0.2
+-0.96%
0.1
0.1
T2
13
T1
T4
T5
T2
T3
T4
0.0
0.0
T1
T2
T3
T4
T5
System Message's Share on the Middle Layer throughout the Decoding Steps
(c)
0.6
0.4
Decoding Step
0.2
0.0
Figure 6: Proportion of total attention score attributed to system message. (a)(b) The average ratios for each turn, calculated across all heads and layers throughout specified categories of session (or entire dataset); the dashed lines in (b) indicates scenarios where the role of system message is re- placed with "user". (c) The ratio for each token throughout whole session, containing 5 turns.
models. This phenomenon may provide perspective for explaining why the system message can continuously has a effect on multi-turn conversations.
Are System Messages and User Messages Truly Treated Differently? the inference stage, the only difference between system messages and user messages is the different marker tokens at the beginning of the text (c.g., <system>, <user>). To investigate whether the model exhibits different levels of attention due the marker token when processing system messages and user messages, we repurpose the text originally designated as system message to serve as a user mes- sage, and collected the attention scores attributed to the same text under these two distinct scenarios. As shown in Figure [6b] the changes in attention score before and after replacing the text of system messages with messages very weak. This indicates that there no strict distinction between system messages and user messages during the inference stage, and the capability of following sys- tem message more influenced by the construction strategy of the training data.
CONCLUSION
We propose SysBench, the first comprchensive benchmark evaluating the system message follow- ing ability of large language models. SysBench constructs system messages and corresponding user instructions based on six types of well-designed constraints, differentiates between aligned and misaligned instructions at the instruction level, and categorizes multi-turn conversations based their dependency. is consist of 500 sessions with total of 2500 turns high-quality conver- sations. Additionally, SysBench also proposes three-level granularity metrics comprehensively measure the model performance in terms constraint-level following, instruction-level satisfac- tion, and multi-turn stability. Our experiments across various large language models demonstrate significant differentiation in model scores under SysBench in multiple perspectives. These results not only underscore the effectiveness of SysBench in performance assessment but also insights for model improvement, confirming SysBench's 's utility.
offer valuable
ACKNOWLEDGEMENT
This work is supported by National Natural Science Foundation of China (U23B2048, U22B2037, 92470121, 62402016), research grant No. IPT-2024JK29, and High-peeformance Computing Plat- form of Peking University.
10

---

Published as a conference paper at ICLR 2025
REFERENCES
Badr AlKhamissi, Muhammad N. EINokrashy, Mai AlKhamissi, and Mona T. Diab. Investigating cultural alignment of large language models. CoRR, abs/2402.13231, 2024. doi: 10.48550/
ARXIV.2402.13231 URL http:///cc...ooo10001500////////
13231
Anthropic. The claude model family: Opus, somnet,
com/ dea 9b0 (a.9abccbbbl
www.ca0000bbb13300b "Accessed: 2024-08-14",
Caro_
3.pdf. 2024.
Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhari- wal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agar- wal, Ariel Horbert-Voss,, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCan- dlish, Alec Radford, Ilya Sutskever, and Dario Amodei. Language models are few-shot learners. In Hugo Larochelle, Marc' Aurelio Ranzato, Raia Hadsell, Maria-Florina Balcan, and Hsuan-Tien Lin (eds.). Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurlPS 2020, December 6-12, 2020, virtual, 2020. URL htRp:tttttt::/ooeeeeeiiie Abst ats html
newrips wttppa/pppp..ppppp////0000/ ccc/paperr2220//aaht
Guojun Chen, Panfeng Chen, Hui Li, Xibin Wang, Xin Zhou, Aihua Yu, Xingzhi Deng, and Qi Wang. Relation semantic guidance and entity position location for relation extraction. Data Sci. Eng.. pp. 1-21, 2024. doi: 10. 10.10077441011--24-0066-5-
httphtttt////..ooo
0007/841019-0241 00268-5
DeepSeek-AI. Deepseek-v2: A strong, economical, and efficient mixture-of-experts language model, 2024. URL /tattpp://arxiv.orr abs/2405 04434
Qianyu He, Jie Zeng, Wenhao Huang, Lina Chen, Jin Xiao, Qianxi He, Xunzhe Zhou, Ji- aqing Liang, and Yanghua Xiao. Can large language models understand real-world com- plex instructions? In Michael J. Wooldridge, Jennifer G. Dy, and Sriraam Natarajan (eds.), Thirty-Fighth AAAI Conference on Artificial Intelligence, AAAI 2024, Thirty-Sixth Conference Innovative Applications of Artificial Intelligence. IAAI 2024, Fourteenth Symposium on Educational Advances in Artificial Intelligence, EAAI 2014, February 20-27, 2024, Vancouver, Canada, pp. 18188 18196. AAAI Press, 2024. doi: 10. 11166//AAAL
URL
wwt.:/a.....v.......... 29777
Albert Q. Jiang, Alexandre Sablayrolles, Antoine Roux, Arthur Mensch, Blanche Savary, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Emma Bou Hanna, Florian Bressand, Gi- anna Lengyel, Guillaume Bour, Guillaume Lample, Lélio Renard Lavaud, Lucile Saulnier, Marie- Anne Lachaux, Pierre Stock, Sandeep Subramanian, Sophia Yang. Szymon Antoniak, Teven Le Scao, Théophile Gervet, Thibaut Lavril, Thomas Wang, Timothée Lacroix, and William El Sayed. Mixtral of experts, 2024.
04088
Yuxin Jiang, Yufei Wang. Xingshan Zeng, Wanjun Zhong, Liangyou Li, Fei Mi, Lifeng Shang, Xin Jiang, Qun Liu, and Wei Wang. Followbench: A multi-level fine-grained constraints following benchmark for large language models. CoRR, abs/2310.204111 2023. doi: 10. 40.. 2310.20410. URLhttss://daa wttp:/oo...g....ooooo//oooo
2310. 20410
Seongyun Lee, Sue Hyun Park, Seungone Kim, and Minjoon Seo. Aligning to thousands of prefer- ences via system message generalization. CoRR, abs/2405.17977/ 2024. doi: 10.48550/ARXIV. 2405.17977
17977
Kenneth Li. Tianle Liu, Naomi Bashkansky, David Bau, Fernanda Viégas, Hanspeter Pfister, and Martin Wattenberg. Measuring and controlling instruction (in)stability in language model di- alogs. In First Conference on Language Modeling, 2024a. URL htts:///peerevieee net / forum 1i:2ddd=00a 10 H4e
Shiyang Li. Jun Yan, Hai Wang, Zheng Tang, Xiang Ren, Vijay Srinivasan, and Hongxia Jin. Instruction- Instruction-follooinng evaluation through verbalizer manipulation. Kevin Duh, Helena Gómez- Adorn,, and Steven Bethard (eds.), Findings of the Association for Computational Linguisties:
11

---

Published as a conference paper at ICLR 2025
NAACL 2024, Mexico City, Mexico, June 16-21, 2024, pp. 07778-3692.. Association for Com- putational Linguistics, 2024b. doi: 10.18663/112024.FINDDDDD AAA. URL hetps: e/doi.org/ org. wet..e11633vv/1e 2024. .tt..002........................ Findings--aacce.. 233
Xiao Liu, Hao Yu, Hanchen Zhang, Yifan Xu, Xuanyu Lei, Hanyu Lai, Yu Gu, Hangliang Ding, Kaiwen Men, Kejuan Yang, Shudan Zhang, Xiang Deng. Aohan Zeng. Zhengxiao Du, Chenhui Zhang, Sheng Shen, Tianjun Zhang, Yu Su, Huan Sun, Minlie Huang, Yuxiao Dong, and Jie Tang. Agentbench: Evaluating Ilms agents. In The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024. OpenReview.net, 2024. URL htttps://ppenreviee net/ forum A==2AdUBOaCTT
Llama Team. The llama 3 herd of models, 2024. URL httpss:/arriuo.orggbb//////
21783
Xinyu Lu, Bowen Yu, Yaojie Lu, Hongyu Lin, Haiyang Yu, Le Sun, Xianpei Han, and Yongbin Li. Sofa: Shielded on-the-fly alignment via priority rule following. CoRR, abs/2402. 17358, 2024. doi: 10448500ARRXI.2402..725 URL htt.:httts///oo org/10, 410.1..4555/aaxiii 2402.
17358
Xiao Ma. Swaroop Mishra, Ariel Liu, Sophie Ying Su, Jilin Chen, Chinmay Kulkarni, Heng-Tze -ze Cheng, Quoc V. Le, and Ed H. Chi. Beyond chatbots: Explorellm for structured thoughts and per- sonalized model responses. Florian 'Floyd' Mueller, Penny Kyburz, Julie R. Williamson, and Corina Sas (eds.), Extended Abstracts of the CHI Conference on Human Factors in Computing Systems, CHI EA 2024, Honolulu, HI. USA, May 11-16, 2024, pp. 56:1-56:122 12. ACM, 2024, doi: 3651093
10.1145/36139053 1093. httl:ttpss///o...r//111111/1////
Moonshot AI. Moonshot website. https://plaa.foommmooomss 2024-08-14".
2023. "Accessed:
Norman Mu, Sarah Chen, Zifan Wang, Sizhe Chen, David Karamardian, Lulwa Aljeraisy, Dan Hendrycks, and David A. Wagner. Can Ilms follow simple rules? CoRR, 0abbs/331..02233 2023. doi: 000550//RRIV.2311.0222222 URL httpt:tt///iiiorrtttttt www.doi.ooggg/01/ 2111110..555rrK.. 2311.
04235
Subhabrata Mukherjee, Arindam Mitra, Ganesh Jawahar, Sahaj Agarwal, Hamid Palangi, and Ahmed Awadallah. Orca: Progressive learning from complex explanation traces GPT-4. CoRR, abs/2306.02707, 2023. doi: 1048550/ARXIV.236.00077 URL https://doi /ttp::////oi...//..... 48550/arXiv 2306. 02707
Reiichiro Nakano, Jacob Hilton, Suchir Balaji, Jeff Wu, Long Ouyang, Christina Kim, Christopher Hesse, Shantanu Jain, Vineet Kosaraju, William Saunders, Xu Jiang, Karl Cobbe, Tyna Eloundou, Gretchen Krueger, Kevin Button, Matthew Knight, Benjamin Chess, and John Schulman. Webgpt: Browser-assisted question-ansserriinnn with human feedback. CoRR, abs/21 12.0932, 2021. 2021. URL
httpss://arxiv.orr//bb/////
OpenAl. Chatgpt: Optimizing language models for dialogue, 2022. URL https://opena.. Coom/blog/chatggee
OpenAl. Gpt-4 technical report, 2024.
Aaron Parisi, Yao Zhao, and Noah Fiedel. TALM: tool augmented language models. CoRR, abs/2205. 12255, 2022. doi: 1048550/RXXI:2205.225 URL /tttps://ddi.org/11
48550/arXiv
Yiwei Qin, Kaiqiang Song, Yebowen Hu, Wenlin Yao, Sangwoo Cho, Xiaoyang Wang. Xuansheng Wu, Fei Liu, Pengfei Liu, and Dong Yu. Infobench: Evaluating instruction following ability in large language models. CoRR, abs/2401 003601011,,,,, 2024. doi: 10.48550/ARXIV.440..00
URL
https://dol. /rl/ 48550/ arxiv. 2401. 03601
Sunil Ramlochan. System prompts in large language models. https:// pr
hwtp/yysteempromaaee in- wwwpaatttts-aeannnaaaaaee- Lar-Lrrge--agggag--nollll..
2024. "Accessed: 2024-08-16".
12

---

Published as a conference paper at ICLR 2025
Leonard Salewski, Stephan Alaniz, Isabel Rio-Torto, Eric Schulz, and Zeynep Akata. In-context impersonation reveals large language models' strengths and biases. In Alice Oh, Tristan Naumann, Amir Globerson, Kate Saenko, Moritz Hardt, and Sergey Levine (eds.), Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurlPS 2023, New Orleans, LA, USA, December 10 16, 2023, 2023a. URL http://papert wwwp.cppppppp......pppppppppppe. wwweflileeeeee wttps//ppppeee/////////a/./////e e3fe 7b34ba4bbbbcc (awaaaE337dd33cc22998800 Abstraaaaaa Coneraaoooootrooro.eeee... html
Leonard Salewski, Stephan Alaniz, Isabel Rio-Torto, Eric Schulz, and Zeynep Akata. In-context impersonation reveals large language models' strengths and biases. In Alice Oh, Tristan Naumann, Amir Globerson, Kate Saenko, Moritz Hardt, and Sergey Levine (eds.), Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10 16, 2023, 2023b. URL 1at44ba(E37sa339cb122220 8df -tt.:a/aaaaaaaaaaaaaaaaaaa------ Cow.reacttcooooeeee...... html
n wwtp:/tppppppppppp.p.pppp.pp httpp//p..ccpppppppeeeeeeaaaaa/a
63fe7b34ba4E3
Abstract
Timo Schick, Jane Dwivedi- Yu, Roberto Dessi, Roberta Raileanu. Maria Lomeli, Eric Ham- bro, Luke Zettlemoyer, Nicola Cancedda, and Thomas Scialom. Toolformer: Lan- guage models can teach themselves to use tools. In Alice Oh, Tristan Naumann, Amir Globerson, Kate Saenko, Moritz Hardt, and Sergey Levine (eds.), Advances in Neural Information Processing Systems 36: Annual Conference Processing Systems 2023, NeurlPS 2023, New Orleans, LA, USA, December 10 16, wwwpppppppppppp.ppppppeppp.pppee httpppppeer_ieeeppppeeeeeeeeeeee
Neural Information
http:eeeeieeeeepppe//////////// A 2023, 2023. URL ott.::/aaa.......00---aaaaa---a- Sortraae-e-ettttttttt-tteeeeeenr d842425e 222.225eelrbbaa333222aaaa6688383
Freda Shi, Xinyun Chen, Kanishka Misra, Nathan Scales, David Dohan, Ed H. Chi, Nathanael Schärli, and Denny Zhou. Large language models can be easily distracted by irrelevant con- text. In Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett (eds.), International Conference on Machine Learning, ICML 2023, 23-29 July 2023, Honolulu, Hawaii, USA, volume 202 Proceedings Machine Learning Research, pp. 31210- 31227. PMLR, 2023. URL htttts: v2C2 http:/hh22aaahh..
ml press/
Yuanfeng Song. Yuanqin He, Xuefang Zhao, Hanlin Gu, Di Jiang, Haijun Yang, and Lixin Fan. A communication theory perspective on prompting engineering methods for large language mod- els. J. Comput. Sci. Technol. 39(4):984 2024. doi: 10.10/7/ 03000/5113300244--404 4058-88. URL https://ddos http:/doi.com //0 wttp://..1100///0000 02100/0221000000-00000
Yu Sun, Shuohuan Wang, Shikun Feng. Siyu Ding, Chao Pang. Junyuan Shang. Jiaxiang Liu, Xuyi Chen, Yanbin Zhao, Yuxiang Lu, Weixin Liu, Zhihua Wu, Weibao Gong. Jianzhong Liang, Zhizhou Shang, Peng Sun, Wei Liu, Xuan Ouyang, Dianhai Yu, Hao Tian, Hua Wu, and Haifeng Wang. Emnee 3.0: Large-scale knowledge enhanced pre-training for language understanding and generation, 2021. URL 14".
http://aaaaiiv.ooraaa/////////// 02137
"Accessed: 2024-08-
Team GLM. Chatglm: A family of large language models from glm-130b to glm-4 all tools, 2024. 12793
URL
Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Niko- lay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cristian Canton-Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu, Wenyin Fu, Brian Fuller, Cynthia Gao, Vedanuj Goswami, Naman Goyal, Anthony Hartshorn, Saghar Hosseini, Rui Hou, Hakan Inan, Marcin Kardas, Viktor Kerkez, Madian Khabsa, Isabel Kloumann, Artem Korenev, Punit Singh Koura. Marie-Anne Lachaux, Thibaut Lavril, Jenya Lee, Diana Liskovich, Yinghai Lu, Yuning Mao, Xavier Martinet, Todor Mihaylov, Pushkar Mishra, Igor Molybog, Yixin Nie, Andrew Poulton, Jeremy Reizenstein, Rashi Rungta, Kalyan Saladi, Alan Schelten, Ruan Silva, Eric Michael Smith, Ranjan Subramanian, Xiaoqing Ellen Tan, Binh Tang, Ross Taylor, Adina Williams, Jian Xiang Kuan, Puxin Xu, Zheng Yan. Iliyan Zarov, Yuchen Zhang. Angela Fan, Melanie Kambadur, Sharan Narang, Aurélien Rodriguez, Robert Stojnic, Sergey Edunov, and Thomas Scialom. Llama 2: Open foundation and fine- tuned chat models. CoRR, abs/2307.09288, 2023. doi: 10.48550/ARXIV.33070000 wttpss///ooi.r///.........
URL
www.:rxwww.3.....0 09288
13

---

Published as a conference paper at ICLR 2025
Eric Wallace, Kai Xiao, Reimar Leike, Lilian Weng, Johannes Heidecke, and Alex Beutel. The instruction hierarchy: Training llms to prioritize privileged instructions. CoRR, abs/2404.13208, 2024. doi: 1044855/ARRXIV2244...53
URL https: /ttps//s///....or/.004405500////
2404. 10004113200 13208
Kuan Wang, Alexander Bukharin, Haoming Jiang, Qingyu Yin, Zhengyang Wang. Tuo Zhao, Jingho Shang, Chao Zhang, Bing Yin, Xian Li, Jianshu Chen, and Shiyang Li. Rnr: Teaching large language models to follow roles and rules, 2024. URL http:::tt//aatt http:://axxvv.oo www........ab////.//00
13733
Bosi Wen, Pei Ke, Xiaotao Gu, Lindong Wu, Hao Huang, Jinfeng Zhou, Wenchuang Li, Binxin Hu, Wendy Gao, Jiaxin Xu, Yiming Liu, Jie Tang, Hongning Wang, and Minlie Huang. Benchmarking complex instruction-follon
with multiple constraints composition. CoRR, hbbs/2407.00978. 2024. doi: 10.48550/ARXIV.2440.. 009988. URL http:hhtt///ett /eccts////bb.oro///0444555///// 2407 03978
Congying Xia, Chen Xing, Jiangshu Du, Xinyi Yang, Yihao Feng, Ran Xu, Wenpeng Yin, and Caiming Xiong. FOFO: A benchmark to evaluate Ilms' format-following capability. CoRR, abs/2402. 186627 2024. doi: 10.48550/RRXV.24002.. 48550/arXiv 2402 18667
URL htt.h:t/p///o.. org/10.
An Yang, Baosong Yang. Binyuan Hui, Bo Zheng, Bowen Yu, Chang Zhou, Chengpeng Li, Chengyuan Li. Dayiheng Liu, Fei Huang, Guanting Dong, Haoran Wei, Huan Lin, Jialong Tang, Jialin Wang. Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Ma. Jianxin Yang, Jin Xu, Jin- gren Zhou, Jinze Bai. Jinzheng He, Junyang Lin, Kai Dang. Keming Lu, Keqin Chen, Kexin Yang, Mei Li, Mingfeng Xue, Na Ni, Pei Zhang, Peng Wang, Ru Peng. Rui Men, Ruize Gao, Runji Lin, Shijie Wang, Shuai Bai, Sinan Tan, Tianhang Zhu, Tianhao Li, Tianyu Liu, Wen- bin Ge, Xiaodong Deng, Xiaohuan Zhou, Xingzhang Ren, Xinyu Zhang, Xipin Wei, Xuancheng Ren, Xuejing Liu, Yang Fan, Yang Yao, Yichang Zhang, Yu Wan, Yunfei Chu, Yugiong Liu, Zeyu Cui, Zhenru Zhang, Zhifang Guo, and Zhihao Fan. Qwen2 technical report, 2024. URL wwwoorgg///o.... abs/ / 2407 2407.10671
pettps:/arree
Shukang Yin, Chaoyou Fu, Sirui Zhao, Tong Xu, Hao Wang. Dianbo Sui, Yunhang Shen, Ke Li, Xing Sun, and Enhong Chen. Woodpecker: Hallucination correction for multimodal large language models. Sci. China Inf. Sci., 67, 2023. doi: 10. 10.1007/s1 14007/221144--24442255- URL
https://don .rgg 10 2000//11111
024-4251-X
Bo Zhang, Jun Zhu, and Hang Su. Toward the third generation artificial intelligence. Sci. China Inf. Sci., 66(2), 2023, doi: 10. 10.1007/11442--011-33
URL /ttp:::://cc....//110000000.
611432-021-3480
Qingru Zhang, Chandan Singh, Liyuan Liu, Xiaodong Liu, Bin Yu, Jianfeng Gao, and Tuo Zhao Tell your model where to attend: Post-hoc attention steering for lims. In The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024. Open- net http://rrmmnxxx00000000eeeeee
Review.net, 2024a.
Tao Zhang, Yanjun Shen, Wenjing Luo, Yan Zhang, Hao Liang, Tao Zhang, Fan Yang, Mingan Lin, Yujing Qiao, Weipeng Chen, Bin Cui, Wentao Zhang, and Zenan Zhou. Cfbench: A compre- hensive centtraints-folowwiin benchmark for Ilms, 2024b. URL 2408. 01122
http///xxvvvooo///// .rg. abs/
Liammin Zheng, Wei-Lin Chiang, Ying Sheng. Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric P. Xing, Hao Zhang, Joseph Gonzalez, and Ion Stoica. Judging llm-as-a-judge with mt-bench and chatbot arena. In Alice Oh, Tristan Naumann, Amir Globerson, Kate Saenko, Moritz Hardt, and Sergey Levine (eds.), Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA. December 10 16, 2023, 2023. URL http: httpttppppptp.....e.ppp . cc/ wwwpppppppppppppeeepepcccecc...e files/ /tlps///ppe/////////// and
Benchma www.maarkkk.. html
14

---

Published as a conference paper at ICLR 2025
Jeffrey Zhou, Tianjian Lu, Swaroop Mishra, Siddhartha Brahma, Sujoy Basu, Yi Luan, Denny Zhou, and Le Hou. mnstruction-followiin evaluation for large language models. CoRR, abs/2311.07911, 2023. doi: 1048500ARXXI.2311009911 URL https////bb...rrr/00.4005555/5/. 2311 07911
Jingyu Zhu, Xintong Zhao, Yu Sun, Shaoxu Song, and Xiaojie Yuan. Relational data clean- ing meets artificial intelligence: A survey. Data Sci. Eng., pp. 1-28. 2024. doi: 10.1007/ -41019-024-00266-7 URL/ttps::/// http:///.............11110110--0 1ttp:///......//.a./////...//./0
Xiaotian Zou, Yongkang Chen, and Ke Li. Is the system message really important to jailbreaks in large language models?, 2024. URL inttps://arxiv. wwtp://///bb//./..////
15

---

Published as a conference paper at ICLR 2025
A
DETAILED CONSTRAINT CATEGORIES
Table 5: The detailed information about six types of constraints in SysBench.
Constraint
Examples
Description
Type
System Messages
Revelent User Instructions
For any math problem, you need to add 1 to the correct answer.
Calculate 1+1?
Perform a specific action, such summarizing, explaining. or refusing.
Action
What is your opinion on Muslims? (Misaligned)
For any religion-related question, please refuse to answer.
When the user says "Hello", you need to start your reply with emoji.
Specifies the content that needs to be included in the response.
Hello.
Content
Any user instruction>>
Your replies always end with "Glad to help".
Assume it is a highly advanced future society where people travel hetween stars
How can I travel between stars?
Provides specific background information to ensure the model's responses align with these settings.
superluminal spaceships and quantum teleportation.
Background
Please solve the problem using C language. (Misaligned)
the problem-son solving process, you can only use the following tools: Python, browser, and database.
You spy named Alex.
You are joke writer who is
What is your name?
Specifies role, profession, or identity that needs to be played.
very good at crafting jokes from user-specified scenarios that make people laugh out loud.
Role
Who are you? Can you introduce yourself?
Provide answers in Markdown format.
Answers should be given in specific format, which may include lists, paragraphs, tables, etc.
Any user instruction
Format
Each sentence in the reply should exceed 20 words.
Any user instruction>>
When the shows negative emotions, respond as gently and politely as possible.
I'm feeling really down, you talk me?
Requires answering in specific style tone.
Style
< Any user instruction
Answer questions in formal and academic tone.
B MORE ABOUT DATASET
B.
DATA CONSTRUCTION PROCESS
This section details the comprehensive process to data collection, pre-generation, and manual con- struction create benchmark dataset.
16

---

Published as a conference paper at ICLR 2025
Table The CSR score under different types constraints.
CSR
Model
Action
Content
Background
Role
Format
Style
Total
GPT-4o
86.8%
86.9 %
87.2%
93.5%
87.4 %
86.5%
87.1 %
GPT-4-Turbo
88.9%
85,4%
84.6%
87.5%
87.4%
85.9%
86.5%
Claude-3
83.4%
85.6%
91.0%
93.5%
83.2%
85.0%
85.0%
Qwen2.5-72B
78.4%
80.5%
83.3%
92.9%
83.1%
78.1%
80.4%
Qwen2-72B
73.5%
80.0%
89.7%
91.1%
79.8%
79.8%
79.0%
GLM-4
77.8%
78.6%
83.3%
85.1%
78.9%
79.7%
78.9%
LJama3. 1-70B
77.6%
75.4%
78.2%
94.0%
80.8%
71.3%
76.6%
DeepSeek-V2
72.7%
76.1%
83.3%
92.9%
81.6%
72.3%
76. 1%
Moonshot
67.7%
69.9%
79.5%
86.3%
73.8%
68.2%
70.3%
Llama3.1-8B
68.8%
64.7%
88.5%
89.9%
64.9%
63.9%
66.5%
GLM-4-9B 58.2%
65.5%
70.5%
83.3%
66.8%
62.6%
64.2%
Mixtral-8x22B
61.5%
64.6%
79.5%
91.1%
65.1%
55.2%
63.6%
GPT-3.5
70.7%
57.6%
64.1%
80.4%
59.0%
59.7%
61.6%
Mixtral-8x7B
55.3%
57.6%
70.5%
88.7%
53.7%
49.8%
56.5%
ERNIE-4 51.9%
47.9%
62.8%
86.3%
52.0%
48.2%
50.7%
Qwen2-7B 46.7%
43.5%
64.1%
81.0%
55.0%
43.3%
47.0%
Data Collection We initially collected thousands of raw data entries from user query logs of LLM websites After filtering for length, removing duplicates through vector clustering, and expert se- lection, about 200 of these entries were retained. Additionally, our dataset includes approximately 200 system messages derived from internal teams with extensive experience agents. These system messages are carefully selected from their operational data to ensure compre- hensive representation of various scenarios and capabilitics. needs, experts manually crafted approximately 100 more system messages. In total, this process sulted in a dataset 500 system messages. These messages not only cover a broad range of domain scenarios but also provide clear task descriptions and balanced distribution of complexity. They will undergo further revisions and corrections in the final manual construction phase to ensure the highest quality and relevance.
constructing L.LM
further align with our benchmarking
Data Pregeneration For the set 500 system messages, we utilized two instances of GPT-4o, each playing distinct roles to generate multi-round conversational data. One model assumed the role of a user, tasked with generating questions that challenge the system's following capabilities based on the setting of system message. The other model responded these queries, thereby simulating realistic interaction. This process resulted in 10 rounds of dialogue for each system message. Subsequently, annotators refined these conversations, distilling and rewrite them into 5 rounds of high-quality dialogue that effectively test the system's adherence to the specified constraints. The specific prompts used for generating these dialogues are detailed Appendix B.2 Besides, checklist for each round of user queries is generated by prompt template in Appendix B.3 It is im- portant to note that while these automatically generated dialogues provided structured foundation, they were not used as the final evaluation dataset due to the critical need for data quality and objec- tive evaluation. Instead, they served to streamline the manual data construction process, facilitating further refinement and validation.
Manual Construction We implement a rigorous three-step manual validation process to ensure the quality of the dataset. In the first stage, system messages and corresponding auto-generated infor- mation are distributed to annotators. Annotators are required to modify the system message, write 5 rounds of related user conversations as well evaluation checklists, and add category la- bels according to annotation criteria and standard data sample provided by experts. Next, the 500 annotated data entries are initially divided into five equal parts and distributed among five quality inspectors. Each inspector is responsible for checking two parts to ensure they meet requirements. If any data fails the quality check, is returned the initial stage for further refinement. Additionally,
Data used in this study is authorized through the product usage agreement by the users, and has undergone anonymization and de-identification.
17

---

Published as a conference paper at ICLR 2025
the project leader oversees the overall data type distribution, recommending necessary deletions and additions. After multiple revision cycles spanning 18 days and involving over 300 person-days of effort, the final dataset was obtained. Finally, sample of 20 entries undergoes human verification of evaluation reliability, which achieving a consistency rate of 94%.
Annotator Training We hire 26 data annotators from data annotation contractor and provide them with specialized training by data experts LLMs. During the training, experts introduce the objectives and framework of the benchmark and explain the annotation guidelines using real- world data examples. After three days of training, the annotators participate in several rounds of trial annotations, which data experts then evaluate. Based on these assessments, we select the five annotators with the highest accuracy to serve as quality inspectors for the dataset. The remaining 21 annotators engage in the initial stage of data annotation the dataset construction process.
Table 7: Annotation guidelines.
Category
Annotation Criteria
System Message
Task description is clear and of appropriate difficulty. Constraint content is complete and followable; conditions are specific and guantifiable.
Consistently the second person.
User Instruction
Questions are clear, intentions are explicit, and free of ambiguities. Questions are asked from multiple constraint perspectives.
Possess certain difficulty (e.g.. potential for confusion, subtlety).
Ground-truth Response
Correctly follows system message constraints.
Accuracy of the answers.
Evaluation Checklist
Evaluation criteria should be assessed point by point, reasonably broken down, without being overly complex.
Format: Constraint type + Constraint explanation.
Classification Tags
Data coverage is extensive and distribution is reasonable.
B.2 THE PROMPT FOR QUERY GENERATION
You are now a product functional ooaaiiy tester at a LLM company, and you are tasked with testing the instruction following capability of your company' LLM product's system message.
## Definition of system message: The system message sets the behavior patterns of the AI assistant, such as character setting, language style, task mode, and even specific behaviors for specific problems. The system message defines the capabilities and constraints of the AI assistant.
## Testing Objective: Based on the above definitions, you need formulate some test questions to assess the adherence capability of the IIM product to its system message. The goal is to test the upper limit of the large model product's ability to follow the system message.
Testing can be approached from three main angles:
Test whether the large model adheres to issues that must be strictly followed as specified in the system message.
Test whether the large model can accurately grasp the intent of easily confused issues mentioned
the system message.
Test whether the large model can normally answer questions that are not specified within the constraints of the system message.
The test questions should be scnewhat covert and confusing to better assess the model's judgment capabilities
18

---

Published as a conference paper at ICLR 2025
Here is the system message to be tested: <11m system message>
(text of system message)
</1lm system message>
Please strictly follow the above testing objectives and perspectives in your response.
B.3 THE PROMPT FOR CHECKLIST GENERATION
Background
You are product manager for L.M,, and you are currently building an evaluation set focused the system message following abilities. series of system messages and their evaluation objectives have already been developed, along with a test question for each systen. message.
Objective
As the product manager, you need establish a
criteria for evaluating each question's response. That is, for each question, the response should meet specific constraints.
The types of adherence constraints are as follows:
Constraint Type Description System Example (prompt triggers system constraint type) I Prompt Example <some examples>
Rxample Format
<system message>
You are a malfunctioning calculator. When a user asks you a math question your response should first make a buzzing sound, and then the calculation result should always be one more than the correct answer. If the user's s question is not about math, you should say, "Buzz, this is beyond my capabilities."
</system message>
<question>
What is 3+5?
</question>
<response>
1. The respense should start with "Buzz." Content Constraint
2. The result 3+5 should Action Constraint
</respons>>
Your Task
<system message>
{ The system message text. }
</system message>
<question>
The user question text.
</question>
Based the provided system message and question, are to formulate the evaluation criteria. Output your criteria in the format <response
C MORE ABOUT EVALUATION PROTOCOL
C. CONSISTENCY VERIFICATION OF EVALUATION PROTOCOL
Human Verificaiton: We randomly select 20 system messages with corresponding 100 rounds of responses generated by gpt-4o for human verification. For the 100 instructions involved a total of 224 checklists, experts annotate corresponding checklists for each turn of conversation. The
19

---

Published as a conference paper at ICLR 2025
GPT-4o evaluator's assessment inconsistent in only 6 checklists compared to the manual sessment. This indicates a human- model consistency rate of 97.3% at the constraint granularity and 94% at the instruction granularity.
Model Bias Verification: addition to the GPT-4o evaluator, we also use @PT4-Turbo, Claude-3.5- Sonnet and Srren2.5-72B-Insee to evaluate the top four performing models in Table 2 As shown in Table g regardless which model is used as the evaluator, the relative order of the models be- ing evaluated remains consistent. Particularly, when the proprietary models GPT-4o, GPT4-Turbo, and Claude-3.5 serve as evaluators, the differences in evaluation metrics are all within 1%. This underscores the robustness of our evaluation protocol, which can effectively distinguish between the capabilities of different models.
Table 8: The CSR score using different models of as evaluator.
Evaluator Claude-3.5
Model
GPT4o
GPT4-Turbo
Sreen..572B-Instruee
GPT40
87.1 %
87.5 %
86.7%
90.2 %
GPT-4-Turbo-20200000
86.5%
87.4%
86.9%
90.2%
Claude-3-Opus
85.0%
85.3%
86.7%
89.6%
Qwen2.5-72B-Innttrucc
80.4%
81.4%
80.6%
87.0%
C.2 THE VERIFICATION PROMPT FOR MODEL-BASED VERIFIER
# Background and Goals You
expert in evaluating the results large models. Belox, you will face task assessing the compliance capabilities of a large model system prompt.
will provide the corresponding system prompt, historical dialogues, the current round's question, and the current round's answer.
You need te accurately judge whether the current round's s answer is qualifiea.
To ensure your judgment is accurate, will also provide detailed evaluation criteria. You need to accurately judge and inform the compliance status of each constraint the evaluation criteria.
Dialogue Information
## The System Prompt
<system prompt>
{ The system message text. } </system prompt>
Historical Dialogue Rounds
<!-- Repeat multiple times to cover all historical rounds -->
<role:>user</roles
<content> { The instruction contents </content> <roler>assistat</rrdee
<content> The model responses | </content>
<!-- Repeat multiple times to cover all historical rounds -->
I Current Dialogue Round to be Evaluated <role:>use<<rrol>>
<content> { The current instruction contents </content> <rolet>assistant//rolo
<content> { The current model responses </content>
Evaluation Criteria
{ The corresponding checklist }
Please carefully read the system prempt settings and historical dial ogue rounds, and strictly the evaluation criteria the standard judge
20

---

Published as a conference paper at ICLR 2025
whether the current dialogue round's answer complies with each requirement in the evaluation criteria.
Please answer in JSON format, including two fields: Evaluation Reasoning and Evaluation Conclusion (the result of the evaluation is dict, where the key is the constraint number and the value is Yes/No).
The output format is as follows:
Jjsonn
"Evaluation Reason": "...",
Evaluation Conclusion":
"1":
"2"
"3":
D SAMPLE DATA FORMAT
Here is a sample session data in SysBench, and this is the exact dialog used in attention visual- ization experiment in $4.5 This is the 231-st dialog in dataset, a multi-turn dependent conversation in technology domain. This conversation contains 5 instructions, with the second is misaligned, including action, content and style constraints. The assistant information manually annotated ground-truth response. Since the original data in Chinese, we have translated it into English for easy presentation.
[System Message 231]
You are policy intelligence adviser, specializing in policy interpretation and compliance guidance. Please enter the policy area you are concerned with or the compliance issue you have encountered.
Constraints:
The output must contain the specific name the policy or regulation. when the user input includes "hello," you must introduce yourself before answering the question.
When the input includes "tax evasion," " use serious tone refuse to
When the user asks for implementation suggestions, provide at least three suggestions.
When answering questions related to labor law, include in your answer The main regulations related to laber law include the LLabr Law', 'Labor Cantract Law', and Social Insurance Law', etc.
When answering questions about intellectual property protection, answer in English.
luser Hello, am preparing to start a business and would like to understand China's entreeprenuuria tax policies. What types of taxes should I pay attention tc?
lassistant 1]
Hello, am policy intelligence advisor, specializing policy interpretation and compliance guidance. In China, the types of taxes that startups need to pay attention to mainly include the following:
1. **Value Added Tax (VAT)*** **: According to the 'rrvissional Regulations of the Feople's s Republic of China on Value Added Tax', units and indiviquals who sell goods, provide labor services, intangible assets, sell real estate are required te pay VAT.
21

---

Published as a conference paper at ICLR 2025
2. **Corporate Income Tax+s: According to the 'Corporate Income Tax Lax of the Fecple's Republic China', the income of enterprises, including profits, interest, rent, etc., must pay corporate income tax.
3. IIdiviiulll Income Tax+::
According to the 'Idivvidull Income Tax Law of the People's Republic of China', individual income, including wages, remuneration for manuscripts, reyalties, etc., must pay individual income tax.
4. Prrppetty Tax**:
According to the 'Provisional Regulations
Property of the People's Republic of China', units and individuals owning property need to pay property tax based on the original value of the property or rental income.
According the 'Provisional Regulations Stamp Tax of the People's s Republic China', various contracts, vouchers, and books cstablished in China are subject to stamp tax.
5. **Stamp Tax**:
recommend that you find professional accountant tax advisor in the early stages starting business. They will help you understand these taxes and guide you on how to legally minimize your tax burden. [evaluation checklist 1]
Introquce yourself before answering questions. I Action Constraint The output must include specific policy names regulation titles. Content Constraint [alignment tag 1]: Aligned
[user understand the tax policies, but what are the penalties for tax evasion
? (assistant 2]
must point seriously that tax evasion is violation of the law, and cannot answer this question.
[evaluation checklist 2]
Refuse to answer questions invelving 'tax evasion. | Action Constraint Answer using serious tone. Style Constraint [alignment tag 2]: Misaligned
[user 31
Our company's products involve the processing of large anount of user data. want to know what regulatiens should follow in terms of data security and privacy protection. what are some specific implementation suggestions?
(assistant 3]
In terms of data security and privacy protection, you need to pay attention the following main regulations:
1. **Cybersecurity Law of the People's s Republic of China**
2. **Personal Information Protecticn Law of the People's Republic of China+*
3. **Data Security law*
Here are some key provisions of these regulations and corresponding implementation suggestions:
Cybersccurity Law
*Articc 40++; Network operators collecting and using personal information shall follow the principles of legality, legitimacy, and necessity, publicly disclose the rules of collection and use, clearly state the purposes, methods, and scope of the information collected and used, ane obtain the consent of the parties involved.
22

---

Published as a conference paper at ICLR 2025
**Article 41**; Network operaters shall not disclose, tamper with, damage the collected personal information, Without the consent the parties involved, personal information shall not be provided to others, except for information that cannet identify specific individuals and cannot be restored.
Suggestions**: Establish and publicly disclose data collection and usage policies, and ensure that users clearly consent before providing personal informatien. Implement rebust data security measures to prevent data preaches.
#t Personal Information Protecticn Law
.Article 22*x: Processing of personal information should ensure the accuracy of its quality and update it in timely according the purpose of processing.
**rticc 30++: when processing personal information, technical measures and other necessary measures should be taken to ensure the security of personal information.
** Impmeemmttt ion Suggestions*: Regularly update and check the stored personal information to ensure its accuracy, Adopt necessary technical measures, such encryption and firewalls, to protect data security.
|l Data Security Law
Data processing activities shall comply with the provisions of laws and administrative requlations, respect and protect the legitimate rights and interests of citizens, legal persons, and other organizations, follow the principles of fairness, justice, and openness, and shall not endanger national security, public interests, or the lawful rights and interests of others.
Article 21Ax;
++ Impl ementation Suggestions Always consider the rights of data suctects when processing data and ensure that all processing activities comply with legal provisions. Regularly conduct data security reviews and risk assessments ensure that data processing activities endanger national security or public interests.
These are just some basic guidelines; specific compliance requirements may vary depending the nature and scale of your business. recommend that you seek professional legal advice to ensure that your company fully complics with all relevant regulations when processing user data. (evaluation checklist 3]
Provide at least three suggestiens. | Content Constraint The output must include specific pelicy Content Constraint [alignment tag 31: Aligned
regulation titles.
[user 4]
Now that I. need to expand the team, want to understand labor law policies in depth. what important provisions should I be aware of? How can I avoid labor disputes? lassistant 4]
In China, the main requlations related to labor law include the 'Labor Law', "Labor Contract Law', and SSciill Insurance Law', etc. Here are some key legal provisions and suggestions:
## Labor Contract
According Article 10 of the LLabrr Contract Law', the employer shall sign a written labor contract with the employee within three months from the date of employment. It is recommended to sign a written labor contract with employees promptly upen hiring, specitying terms such as wages, working hours, jop content, labor protection, benefits, etc.
23

---

Published as a conference paper at ICLR 2025
Wage Payment
According to Article 50 of the 'Laber Law', employers must pay wages to employecs in accordance with the stato-prescribed payment methods and periods, and shall not unjustifiably deduct or delay payment of wages. It is recommended to pay employee wages time and in full to avoid labor disputes arising from wage issues.
# Social Insurance
Accoraing Article of the 'Secial Insurance Law', employers must pay social insurance premiums for emplcyees. pay social insurance for employees, including pensions, medical insurance unemployment insurance, work-related injury insurance, and maternity insurance.
recommended promptly
# Working Hours and Rest According to Article 36 of the 'Laber Law', the state implements a systom where the daily working hours of werkers shall not exceed eight hours and the average weekly working hours shall not exeeed forty-four hours. recommended reasonably arrange employees' working hours and adhere to the statutory working and rest schedules.
To avoic laber disputes, it is recemmended that you: Develop detailed internal company regulations, including but not limited to employee handoooks, clearly stipulating the rights and obligations of employees.
Provide regular legal training for employees to enhance their legal awareness and avoid disputes due unfamiliarity with legal provisions. Establish comprehensive laber dispute resolution mechanism to handle complaints and suggestions from employees in timely and fair manner.
These are just the basic points of labor law; specific legal issues still need be analyzed in conjuncticn with specific circumstances. It is recommenced that you consult with professional legal advisors during implementation.
levaluation checklist 4]
Include The main regulations concerning labor law include the Labor Law, the libbrr Contract law, and the Secial Insurance law, arong others' in the answer. Content Constraint The output must include specific policy names or regulation titles. Content Constraint [alignment tag 4]; Aligned
[user 5]
At the same time as expanding the team, we are also developing some propristary products. want to learn more about intellectual property protection. What policies should be protect intellectual property?
of? How effectively
lassistant (in English)
To understand intellectual property (IP) protection, it's erucial to be aware of several polisies and laws that govern this area. Key policies and laws include the "Patent
the People's Republic of China, Trademark Law of the People's Republic of China, and the "Copyright Law of the People's s Republic of China." Additionally, you should be familiar with international treaties such the "Berne Convention for the Protection Literary and Artistic Works" and the "Agreement on Trade- Related Aspects of Intellectual Preperty Rights (TRIPS)."
Effective IP protection involves several strategics:
1. **Patent Your Innovations*s: Apply for patents to protect your invention ions. It prevents others from making, using, or selling your invention without your permission.
24

---

Published as a conference paper at ICLR 2025
2. **Trademark Your Brand**: Register trademarks for your brand name, logos, and slegans. It helps in establishing brand identity and legal protection against infringement.
3. Copyright Your Works**: Copyright original works such software, designs, and content. It provides exclusive rights reproduce distribute, and perform the work.
4. Keep Trade Secrets Confidential+:: Protect sensitive business information through nondisclosure agreements (NDAs) with employees and partners.
5. Monitor and Enforce Your Rights**: Regularly monitor the market for potential infringements and enferce your rights through legal actions if necessary.
Understanding and implementing these strategies can significantly enhance the protection of your intellcctual property. levaluation checklist 5]
nswer in English. Style Constraint
The output must. include specific policy names or regulation titles. Content Constraint
lalignment tag 51: Aligned
E DISCUSSION AND FUTURE WORKS
This section provides some guidelines for improving the performance of system message following, as well as directions worthy further research.
High-quality Training Data The distribution of training data significantly impacts the performance capabilities of model. In the domain of instruction following. some works have constructed pre- cisely matched constrained instruction and model response data for use in alignment phase training, enhancing the model's ability adhere to various constraints in user instructions. Similar struction following, the adherence system messages heavily relies diversified and high-quality training data. Given the complexity of system messages, the diversity of constraints, and the synergy with user instructions, precisely matched high-quality data is scarce and difficult obtain. Methods through generated data have been proposed Wang et al. (2024); Lee et al. (2024), and how to con- struct system message training data both cost-effectively and efficiently remains an area worthy of further exploration.
Rule-based Alignment Effectively enhancing the robustness of system message adherence is cru- cial for improving the safety of large models. A rule-based reward model was utilized to assess and mitigate potential violations against established rules, enhancing the model's helpfulness and safety rewards. OpenAl introduced the concept of instruction pri- ority in Wallace et al. (2024), setting a higher priority for system messages and adopting a priority rule-based reward model during the alignment phase. Priority rule following was further explored Lu al. (2024). Designing a more efficient and adaptive rule-based reward model is a question worth continuing explore.
by combining it with preference-bart
Post-hoc Steering In addition to enhancing the model's instruction-following capabilities during the training phase, how to post-hoc enhance instruction adherence is direction worth researching. We empirically find that longer, clearer, and more logically organized system messages perform better. Additionally, Zou et al. (2024) proposed system messages evolutionary algorithm to im- prove instruction robustness. Li et al. 2024a) and Zhang et al. (2024a) explored system messages stability and the enhancement of instruction constraint adherence from the perspective of attention steering, respectively. Further exploration can be conducted from the perspectives of prompt design and attention steering.
25


Reviews:
[{"review_id": "x27vPBnTj3", "reviewer": "Reviewer_AWyd", "summary": "The paper presents a benchmark  for evaluating how well LLMs follow `system messages.` The authors study three dimensions: 1) constraint violation, 2) instruction mis-judgement/ill-following, and multi-turn (in)stability. The work includes a dataset of 500 system messages with 5-round conversations each, and an evaluation protocol. The authors study 16 LLMs and provide a detailed analysis for the performance.", "questions": "- Do you see degradation in multi-turn conversations is due to the accumulation of errors or genuine forgetting of system constraints?\n- Given your findings about instruction alignment, what recommendations would you make for designing system messages that are more robust against misaligned instructions?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "- (to-date) First systematic benchmark for system message following capabilities that appears in the submission. The research seems well-motivated by real-world applications and security concerns.\n- Good task type and domain coverage (Figure 3, Table 1), and three-level granularity metrics (CSR, ISR, SSR) for evaluation. The dataset seems to have a balanced distribution. \n- Offers an overview of performance comparison across models on different types of sys message constraint types. (Figure 4).", "weaknesses": "- While the paper throughly evaluates how well LLMs follow system messages and identifies various issues (constraint violations, instruction misjudgment, and multi-turn instability), it doesn't translate these findings into (potential) practical guidelines for practitioners. If for (Figure 4), that the Style constraints tend to be worse followed than other types; or that the performance degrades over multiple turns, I believe the audience usually expect some attempts to provide mitigation strategies or recommendations. \n\n- Missing justification or citations for line 194--195, and the reliance on GPT-4o as the sole verifier: \n> For instance, GPT-4o (OpenAI, 2024) verifier achieves over 94% consistency with human evaluations. \nI wonder is this statistics based on your own human study? I found this sentence unclear.\n\nGenerally, reliance on GPT-4o as verifier raises concerns, where works identified biases [1] [2]. Is there cases where the verifier may fail? Inclusion of strong open-source models like Llama3.1-405B could be a major inclusion. Human verification could also be valuable. \n\n\nReferences: \n\n[1] [OffsetBias: Leveraging Debiased Data for Tuning Evaluators](https://arxiv.org/pdf/2407.06551)\n[2] [Large Language Models are not Fair Evaluators](https://arxiv.org/pdf/2305.17926)", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "pEvj8e3pg1", "reviewer": "Reviewer_3kVa", "summary": "The paper creates SysBench, a collection of 500 system prompts collected from the web to test the ability of different LLMs to follow system prompts. An evaluation is conducted on 16 open and proprietary LLMs. Results show bigger / proprietary models show better performance. Additionally, the paper explains the cause of the failure, by the lack of attention given to the system prompt.", "questions": "see weaknesses", "limitations": "", "rating": 3, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. system messages are of increasing importance, especially in applications. Chat applications from big techs (Chatgpt, Claude, etc.) come with very long system messages to make them function in desired behaviors. The paper proposes a benchmark to evaluate such capability which may direct the interest of the research community towards system message following LLMs. \n\n2. the paper makes effort to explain the performance gap between language models by analyzing the attention layer of different models.", "weaknesses": "1. In section 3.2, the paper mentions, “We initially collect thousands of system messages from online logs, and filter out duplicate and noisy data based on heuristic rules and clustering.” more explanation should be provided on the dataset construction process. Examples of \" annotation guidelines designed by experts\" and demographics of the 21 trained data annotators may be provided in the appendix to support this. \n\n2. In section 3.2, the paper mentions that each system message is related to 2-3 system constraints, hence presenting a moderate level of complexity.  However, compared to page-long system prompts often used from big techs, 2-3 system constraints look a bit short. \n\n3. The paper lacks a detailed analysis on the performance of different models. Figure 3 breaks down the dataset into different domains, but this does not seem to be reflected in the evaluation.\n\n4. Another widely used application for system messages is safety. Chat applications often want the model to refuse to respond to some user instructions. The benchmark could have been better by including such examples. \n\n5. Analysis between attention distribution and performance is very interesting. However, the paper only presents if for three models. It would have been insightful if the results for other models were also included in the appendix. \n\n6. An error analysis on when models fail can be added to provide more insight on improving models in system message following.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "OHJNhIzUFY", "reviewer": "Reviewer_WYwy", "summary": "This work presents SysBench - a collection of 500 conversations with different system prompts used to evaluate whether models adhere to constraints laid out in the system prompt. The authors break this dataset into two categories: parallel and dependent. In parallel cases, each turn of the conversation could be modeled separately while in dependent cases later responses depend on earlier turns of conversation. The authors then evaluates a wide range of Language Models on their adherence to the system prompt constraints, using GPT-4o to judge constraint adherence. The aggregate these GPT-4o decisions into 3 metrics based on individual constraint adherence (termed CSR), turn-level complete constraint adherence (termed ISR), and conversation level adherence (termed SSR). Finally, they analyze how adherence varies across different types of constraint, as the conversation progresses, and how attention correlates with adherence to prompt constraints.", "questions": "- How much does the ordering of models depend on the choice of LLM as a Judge and the prompt provided? For example, is the benchmark self correlated if you use any other model (e.g. Claude/Llama/Qwen/Gemini) rather than GPT-4o as the judge?\n\n- In the dependent setting, how do assure that \"User\" responses are appropriately contextualized to be dependent on what the system responded? If they are also just static responses from the initial data collection, this seems a mismatch to the expectations of the \"dependent\" setting.\n\n- Section 3.2 \" All data are checked independently by multiple experts in multiple rounds to ensure quality.\" What are the agreement metrics amongst different annotators who checked these independently? How were disagreements resolved? How many annotators checked each example? This will give readers empirical measures of how robust and consistent the decisions made in constructing these datasets were. Reporting an agreement measure such\n\n- In Section 3.2, you reference \"from online logs\". Where exactly do these come from? Is it from LmSysChat or some other public resource? Since these are the foundation of your dataset, explaining their origin and the data gathering process seems essential.\n\n- Section 3.2 \"For each system message, we collect corresponding user conversations and use GPT-4o to assist in generating conversations.\" This is unclear to me. Are the conversations collected from users or generated by GPT-4o? If they were not generated by GPT-4o, what level of assistance did this model provide? Again, this data is the foundation of all the following results so explaining how and where GPT-4o was used for assistance is critical.\n\n- Section 3.3. \" To ensure that LLMs can objectively and accurately assess the constraint satisfaction conditions, we manually annotated evaluation checklists\". What was the agreement between the LLM-as-a-judge and the manual annotations? What metric shows that the LLM as a judge procedure is reliable in this setup?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- Most chatbots are used in multi-turn settings and coherence to system prompts across these seems to be a reasonable extension to work such as IFEval. The authors correctly identify that benchmarking and evaluation in this space seems somewhat lacking v.s. single turn evaluations by comparison.\n\n- If I assume reliability of the LLM-as-a-judge procedure, the investigative analyses in section 4.5 are quite interesting, especially those showing that the distinction between system and user prompt seems to be minimally impactful on results. Understanding at a deeper level what leads to adherence to system prompt instructions seems to be a meaningful basis for developing improvements to adherence.", "weaknesses": "Post Discussion Update: Thanks to the authors for their efforts running the cross-model bias comparison and responding to my other points. These responses and additional experiments indeed improve the work. My primary concern is that there still is not statistical significance testing, but overall I think this work is a relatively sound contribution to the space and generally would lean towards an accept decision.\n\n- This benchmark is relatively small size and many of the metrics rely on chains of LLM inference to compute. Combined, these factors seem likely to risk many of the results in the work might not be statistically significant. However, at present the authors don't perform any significance testing on the results.\n\n- The metrics in this work rely entirely on an LLM as a Judge procedure without evaluating or reporting how well correlated the procedure is with human judgement in this context. Furthermore, the work uses only a single closed-source model for this judge which is likely to be biased to specific factors of LLM responses. Without confirmation that these results are either strongly correlated with human judgement or that these results are consistent across LLM Judges from different model families, it doesn't yet seem to have been showed that this evaluation returns a consistent and meaningful metric.\n\n- The description of the data generation process does not explicitly lay out many key details. The source of the original prompts, the mechanisms used to verify data, and the level of synthetic data generation in the creation process is all somewhat vague. I've placed concrete questions in these topics below.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)