INFOBENCH: Evaluating Instruction Following Ability in Large Language Models (2024)

Abstract: This paper introduces the Decomposed Requirements Following Ratio (DRFR), a new metric for evaluating Large Language Models’ (LLMs) ability to follow instructions. Addressing a gap in current methodologies, DRFR breaks down complex instructions into simpler criteria, facilitating a detailed analysis of LLMs’ compliance with various aspects of tasks. Alongside this metric, we present INFOBENCH, a benchmark comprising 500 diverse instructions and 2,250 decomposed questions across multiple constraint categories. Our experiments compare DRFR with traditional scoring methods and explore annotation sources, including human experts, crowdsourced workers, and GPT-4. The findings demonstrate DRFR’s higher reliability and the effectiveness of using GPT-4 as a costefficient annotator. The evaluation of several advanced LLMs using this framework reveals their strengths and areas needing improvement, particularly in complex instructionfollowing. This study contributes a novel metric and benchmark, offering insights for future LLM development and evaluation. 1

heading: 1 Introduction
text: The rapidly evolving domain of Natural Language Processing (NLP) has witnessed the advent and rise of Large Language Models (LLMs) (Ouyang et al., 2022; Brown et al., 2020; OpenAI, 2023b; Touvron et al., 2023; Bai et al., 2022; Chowdhery et al., 2022; Zhang et al., 2022), which exhibit remarkable prowess in simulating human-like text generation. Their ability to follow instructions is of utmost importance for their practical application, as it customizes the outputs according to specific user needs. Despite this, evaluation of these models frequently emphasizes general natural language tasks (Wang et al., 2018, 2019; Hendrycks et al., 2020; Srivastava et al., 2022; Liang et al., 2022), or specific downstream tasks (Qin et al., 2023; Bang et al.,
∗Work done during internship at Tencent AI Lab. 1We release our code and dataset at https://github.
com/qinyiwei/InfoBench.
2023), while the critical element of instructionfollowing—the model’s capacity to accurately understand and execute user instructions—has not been thoroughly explored. This evident gap in research and the absence of systematic evaluation methods dedicated to this crucial aspect serve as the impetus for our present study. We aim to establish a reliable protocol and benchmark for appraising the instruction-following aptitude of LLMs.
Current research tends to utilize evaluation methodologies such as A/B Testing (Askell et al., 2021; Taori et al., 2023; Chiang et al., 2023; Li et al., 2023b; Wang et al., 2023c), overall scoring (Wang et al., 2022), and Elo rating (Zheng et al., 2023; Bai et al., 2022; Glaese et al., 2022)—a derivative of A/B testing. While these strategies have proven somewhat effective, they suffer from considerable drawbacks, including scalability issues and a lack of interpretability. A/B testing, for instance, necessitates N2 comparisons for N systems. Moreover, the overall scoring system is opaque and difficult to interpret, failing to elucidate the reasoning behind the assignment of specific scores to particular instructions.
In this study, we propose a novel metric for assessing the instruction-following capability of LLMs, namely the Decomposed Requirements Following Ratio (DRFR). Unlike traditional metrics, DRFR decomposes each instruction into simpler, distinct criteria, allowing for a more detailed and precise assessment of an LLM’s ability to follow specific aspects of a given task. This decomposition enables a granular evaluation, where the compliance of the LLM is measured against each individual requirement within an instruction, offering a clearer picture of the model’s performance in complex instruction-following scenarios. In addition, We constructed a benchmark composed of 500 distinct instructions and provided 2,250 decomposed questions for the assessment of instructionfollowing. To facilitate a more fine-grained analysis, we classified the constraints present in each instruction into five categories: Content, Linguis-
13025
tic, Style, Format, and Number. We also manually curated constraint labels for each decomposed question to identify and analyze potential areas of deficiency in each model.
We conducted two key experiments: firstly, we compared the DRFR metric with traditional Direct Scoring (DS) for evaluating responses from various Large Language Models (LLMs). The results showed a higher consensus among annotators with DRFR, particularly in Hard Set, suggesting its greater reliability over Direct Scoring (DS). Secondly, we explored more cost-efficient annotation sources by comparing human experts, crowd-sourced workers from Amazon Mechanical Turk (AMT), and GPT-4. The findings revealed that while human experts were the most accurate, they were also costly and time-consuming. AMT workers presented a balance but with lower accuracy. Significantly, GPT-4 emerged as a highly accurate, cost-effective, and time-efficient alternative, especially when employing a structured, multi-turn prompt approach, making it a viable option for large-scale annotations.
We thus employ GPT-4 as an annotator to evaluate the entire INFOBENCH. The comprehensive evaluation of these advanced LLMs reveals that while progress has been significant, there is still a notable gap in their ability to follow instructions perfectly, particularly in more complex scenarios represented by the "Hard Set". The closed-source models are currently leading, possibly due to better data or more sophisticated algorithms. However, the performance across different constraint types and domains suggests that the challenges in instruction-following are more nuanced and may require focused improvements in areas like numerical and linguistic understanding. As LLMs continue to evolve, addressing these specific areas could lead to more robust and versatile Large Language models capable of handling a broader range of real-world tasks and instructions.
Our study presents four key contributions to the field of evaluating Large Language Models (LLM) in their ability to follow instructions. Firstly, we introduce a novel and scalable metric, the Decomposed Requirements Following Ratio (DRFR), which provides a reliable, detailed, and interpretable framework for assessing LLMs. This metric decomposes complex instructions into simpler, specific criteria, allowing for a granular analysis of a model’s performance. Secondly, we have developed INFOBENCH, a comprehensive benchmark dataset. It includes 500 diverse instructions paired with 2,250 decomposed questions, designed to test
and analyze the instruction-following capabilities of LLMs systematically. Thirdly, through extensive experimentation, we established the efficacy of DRFR and INFOBENCH. Our findings highlight the potential of using GPT-4 as an evaluator for its accuracy and cost-effectiveness, particularly when employing structured, multi-turn prompts. Finally, we utilized our automatic evaluation toolkit to conduct a thorough analysis of six advanced LLMs. This evaluation not only underscores the current capabilities of these models but also sheds light on specific areas that require improvement for more nuanced and effective instruction-following, especially in complex scenarios. These contributions pave the way for future advancements in LLM development and evaluation.

heading: 2 INFOBENCH
text: Both evaluation metrics and benchmark datasets play an integral role in the objective assessment of Large Language Models’ (LLM) ability to follow instructions. These tools are essential for enabling consistent comparisons, pinpointing areas for improvement, and ensuring the practicality, fairness, and effectiveness of these models in realworld applications. In Section 2.1, we introduce a novel evaluation metric, termed the Decomposed Requirements Following Ratio (DRFR). This metric is specifically designed to measure the proficiency of LLMs in adhering to complex instructions in a detailed and structured manner. Furthermore, Section 2.2 discusses the development of a corresponding benchmark dataset, INFOBENCH, which has been meticulously curated to rigorously test and evaluate the instruction-following capabilities of LLMs. This dataset presents a diverse range of scenarios and challenges, offering a comprehensive tool for benchmarking and refining the performance of these models.

heading: 2.1 DRFR
text: To evaluate the ability of advanced language models to accurately follow instructions, one common method is to consult evaluators for insights into whether and how the models’ responses align with the given directives. However, this approach often encounters difficulties in practice, as reaching a consensus among evaluators can be challenging, especially in scenarios where the models partially meet some aspects of the instructions. A viable strategy for evaluating the compliance of advanced language models with complex instructions is to segment each instruction into distinct, simpler criteria that are more straightforward for evaluators to
judge. Consider a benchmark dataset comprised of N instructions. Each of these instructions can be decomposed into mi ∈ N+ unique requirements. The variable ri,j ∈ [0, 1] denotes whether the j-th criterion within the i-th instruction is met. In this context, we can define the Decomposed Requirements Following Ratio (DRFR) as follows:
DRFR = ∑ i,j ri,j∑ imi
(1)
This metric quantifies the model’s adherence to the decomposed criteria across the dataset, offering a nuanced understanding of its instruction-following capabilities. Table 1 provides two illustrative examples of instructions, each accompanied by their respective decomposed criteria. These criteria are framed as binary questions, necessitating a YES or NO response. A YES answer indicates that the output of the Large Language Model (LLM) successfully meets the specific criterion outlined in the question, corresponding to a value of 1 for ri,j . It is important to highlight that even minor deviations from the requirement result in a negative (NO) response, as precise adherence is mandatory. For example, in the third question of the second example in Table 1, the occurrence of even a single hotel review exceeding one sentence in length will necessitate a NO response. On the other hand, a NO indicates that the LLM’s output either does not satisfy the requirement or fails to provide relevant information pertinent to the specified criterion.

heading: 2.2 INFOBENCH Dataset
text: To effectively benchmark instruction-following abilities, it’s crucial to develop a dataset with a broad spectrum of diverse instructions. This approach ensures thorough coverage and enables a more precise evaluation of proficiency in follow-
ing instructions. The INFOBENCH addresses this need with two distinct sets: the Easy Set and the Hard Set. The Easy Set, based on the research in (Wang et al., 2022), is designed for a broad range of applications. In contrast, the Hard Set is our original contribution, consisting of a manually curated dataset inspired by various subject areas. Table 1 showcases examples from both sets in the context of hospital reviews. While the Easy Set has been widely used in evaluations ((Wang et al., 2022; Li et al., 2023b; Taori et al., 2023; Peng et al., 2023; Anand et al., 2023; Wang et al., 2023c)), our work led to the creation of the Hard Set, addressing the Easy Set’s limitations: 1) Its instructions are relatively simple, limiting the ability to differentiate between modern LLMs’ performance; 2) It features a limited variety of constraint types, typically with only about three per instruction, lacking in overall diversity; 3) The Easy Set often includes ambiguous and subjective requirements, as exemplified by the difficulty in universally defining terms like ‘helpful’ in its instructions. For instance, in Table 1, an instruction from the Easy Set requires the questionnaire to be ‘helpful’, but establishing a consensus on the definition of ‘helpful’ proves to be difficult. To guarantee the quality of the instructions and the clarity of the requirements, each instruction and its decomposed requirements are meticulously crafted by a subject matter expert and subsequently validated by another expert. The process of collecting instances continues until both experts concur on the naturalness of the instruction and the atomicity and lucidity of the decomposed requirements.

heading: 2.2.1 Instruction Development
text: Our objective is to formulate instructions with extensive coverage that effectively distinguish between superior and inferior models. We have de-
veloped the Hard Set encompassing 72 domains, spanning areas such as Natural Sciences, Social Sciences, Engineering, Economics, Arts, various occupational tasks, and aspects of daily life, illustrated in Figure 1. We also incorporate specific response constraints in these instructions: 1) Content Constraints: These define the specific topics or details to be addressed in the LLM’s response, such as discussing "climate change impacts" or mentioning "the main characters in Harry Potter"; 2) Linguistic Guidelines: These dictate the use of particular language structures and terms, including grammatical styles, syntax, and specific dialects, like "Victorian English" or "technical jargon"; 3) Style Rules: These direct the overall tone and audience of the text, varying from formal to persuasive or sophisticated, as in writing with a "respectful tone" or for "a young audience"; 4) Format Specifications: These instruct the LLM on the structural presentation of its response, such as "crafting a sonnet" or "list ideas bullet-wise"; 5) Number Limitations: These involve numeric-related instructions, like producing "a 500-word essay" or presenting "three arguments for renewable energy".

heading: 2.2.2 Instruction Decomposition
text: In the process of instruction development, we meticulously devised detailed requirements and associated binary questions for each instruction, each
question reflecting a specific constraint that can be answered with "YES" or "NO". Utilizing our established ontology of constraint types, we manually classified each question with appropriate constraint labels, noting that a single requirement could embody multiple constraint types. For example, the question "Does the generated text provide a critique of Vincent van Gogh’s Starry Night?" falls under the content constraint due to its focus on "Starry Night", while also fitting the format constraint as a critique represents a distinct writing style. Therefore, we assigned one or more constraint types to each question, with each constraint in our dataset being labeled with up to three types.

heading: 2.2.3 Benchmark Statistics
text: Table 2 offers statistical insights into the INFOBENCH, highlighting significant differences between the two sets. The Hard Set is characterized by more elaborate instructions and a greater number of requirements, indicating a higher level of complexity compared to the Easy Set. Moreover, the table emphasizes the dataset’s diversity, evident in the broad range of domains it encompasses.

heading: 3 Experiments

heading: 3.1 DRFR v.s. Direct Scoring
text: In our study, we aim to assess the reliability of the DRFR metric by contrasting it with a conventional direct scoring approach. In the Direct Scoring (DS), human evaluators assign a score ranging from 1 (indicating the lowest quality) to 5 (representing the highest quality) for each response generated by the model. The criteria for each rating are elaborated as follows: Rating-1 (Very Bad): The model’s response is entirely irrelevant to the query posed. It demonstrates no understanding of the requested task. Rating-2 (Bad): While there is a minimal degree of relevance to the original instruction, the response is predominantly incorrect, confusing, or both. Rating-3 (Okay): The response shows some relation to the posed question but lacks critical details or includes inaccuracies. Rating-4 (Good): This rating is given when the response is mostly accurate and comprehensive. However, it might include slight errors or omit minor details. Rating-5 (Excellent): A response receiving this rating is entirely accurate, detailed, and aligns perfectly with the requirements of the query.
In our evaluation, we incorporate five widely recognized Large Language Models (LLM), each distinguished by its unique characteristics and training methodologies: GPT-3.5-turbo (Ouyang et al., 2022): A 175 billion parameter extension of GPT-3 (Brown et al., 2020) by OpenAI, enhanced for alignment with user intentions through human feedback-based reinforcement learning. GPT-4 (OpenAI, 2023b): This model is an advanced, large-scale successor to GPT-3.5-turbo. Claude-v1 (Bai et al., 2022): Anthropic’s 52 billion parameter AI assistant, Claude-v1, is trained to be helpful, honest, and harmless using reinforced learning from human feedback. Alpaca7B (Taori et al., 2023): Alpaca-7B is an opensource model, derived from a 7 billion parameter LLaMA base (Touvron et al., 2023), and finetuned with a dataset (Wang et al., 2022) focused on instruction-following capabilities. Vicuna13B (Chiang et al., 2023): Vicuna-13B is a 13 billion parameter, open-source model fine-tuned from LLaMA, incorporating diverse real-world interactions from the ShareGPT platform2.
Additionally, We aggregate responses generated by the models on INFOBENCH and engage three NLP experts to independently assess them, utilizing two distinct methods. Initially, they apply the Direct Scoring (DS), followed by an evaluation
2https://sharegpt.com
using the DRFR metric. This order is designed to mitigate any potential bias: by conducting the DRFR assessment after the DS evaluation, we aim to prevent the evaluators’ impressions formed during the DRFR assessment from influencing their DS evaluations. This ensures a more objective and unbiased evaluation process.
Moreover, to facilitate an equitable comparison between the two evaluation metrics, we standardize both into instruction-level scores.3 This enables a direct, side-by-side comparison of response pairs from two distinct models, referred to as Model-A and Model-B. For each instructional prompt, the paired responses are categorized into three distinct classifications: 1) Model-A outperforms Model-B, 2) Model-A and Model-B are equally effective, and 3) Model-B outperforms Model-A. Finally, a Fleiss’ Kappa Agreement (Fleiss, 1971) is employed to measure the agreements among three evaluators concerning the pairwise categorization. This Pairwise Kappa Agreement approach ensures clarity and precision in our comparative analysis of the evaluation metrics.
Table 3 displays the Pairwise Kappa Agreement for both metrics across 50 selected instances from the INFOBENCH, comprising an equal mix of 25 instances each from the Easy and Hard subsets. The observed gap exceeding 0.19 between the DRFR and Direct Scoring (DS) suggests that DRFR yields a notably higher annotator consensus, thereby implying its enhanced reliability for human annotations.

heading: 3.2 Experiment of Annotation Sources
text: In our quest to identify a more cost-effective source for annotations, we conducted a comparative study utilizing annotations from various sources, including human experts, crowd-sourced workers from Amazon Mechanical Turk (AMT), and advanced Large Language Models like GPT-4. The details of this comparison are as follows:
3DS uses the score provided by the evaluator, DRFR counts all the stratified requirements and normalized by the number of requirements in the instruction
Human Expert Annotations: Annotations from three experts, previously discussed in Section 3.1, were utilized. Given the wide range of topics covered by the instruction set, it is improbable for a single expert to possess comprehensive proficiency across all subjects. Therefore, we implemented a majority voting system, supplemented by the use of online research tools, to enhance the accuracy of these expert annotations. Crowd-Sourced Annotations: We engaged annotators from the Amazon Mechanical Turk (AMT) platform4, selecting those with a HIT Approval Rate above 90% and a minimum of 500 Approved HITs. Each task, comprising an instruction and its corresponding responses, was assigned to three distinct annotators. We adopted a majority voting approach for each discrete query and an aggregate scoring method for instructions. The specifics of this evaluation methodology are elaborated in Appendix A.3. GPT-4 Based Annotation: We employed the GPT4 model5 for automatic evaluation. The approach involved issuing a series of multi-turn prompts, sequentially presenting GPT-4 with decomposed questions, while accounting for the context provided by previous answers. The specifics of this prompt structure are detailed in Appendix A.2.
In Table 4, we present a comparative analysis of various annotation sources, emphasizing their cost, time efficiency, and annotation robustness. We calculated the cost of expert annotations by aggregating the total hours worked by three NLP experts. The hourly rate was set at $59, based on salary data from ZipRecruiter.6 The expenditure on AMT workers was computed based on the total payment made on the platform. Each annotation task, comprising one instruction and five model responses, was assigned to three annotators. The compensation was $1 per annotator, with an additional 25% platform fee. The total time for AMT annotations was obtained from AMT platform statistics. The cost of using the GPT-4-0314 model is determined by its token-based pricing structure7, which is $0.03 per 1,000 prompt tokens and $0.06 per 1,000 completion tokens. Operational constraints of GPT-4 include a limit of 200 requests per minute and a maximum of 40,000 tokens per minute.8 The total cost and time expen-
4https://www.mturk.com/ 5Version: gpt-4-0314. 6https://www.ziprecruiter.com/Salaries/
NLP-Salary 7https://platform.openai.com/docs/deprecations 8https://platform.openai.com/docs/guides/ rate-limits?context=tier-free
diture for using GPT-4 were calculated based on the cumulative count of prompt and completion tokens used during the execution of the tasks. To evaluate the robustness of the annotation source, we inherit the pairwise comparison mentioned in Section 3.1. The three categories are labeled with {−1, 0, 1} for 1) Model-A outperforms Model-B, 2) Model-A and Model-B are equally effective, and 3) Model-B outperforms Model-A. Pairwise Label Distance(PLD) is introduced to measure the agreement between the annotations and the ground truth labels. A PLD value of 0 indicates that the annotations accurately reflect the relative performance order of the models for a given pair of responses. A PLD of 1 suggests a potential misclassification in scenarios where models perform equally (‘tie’ scenario), possibly attributing superiority to one model over the other, or vice versa. A PLD of 2 signifies that the annotations incorrectly represent the order of model performances. A Weighted Pairwise Label Distance (WPLD) is defined as below:
WPLD = 2∑
i=0
i ∗P(PLD = i) (2)
Our analysis yields two important findings: 1) Annotations generated by GPT-4 exhibit superior cost-effectiveness and time efficiency compared to those produced by AMT (Amazon Mechanical Turk) crowd workers. Additionally, GPT-4 demonstrates enhanced overall performance in annotation tasks. 2) When employing decomposed questions, GPT-4’s performance is particularly noteworthy. It achieves an accuracy rate of 89% and a weighted pairwise label distance of 0.44, in alignment with the gold standard. These results suggest that GPT-4, used in conjunction with the decomposed questions, emerges as a feasible and efficient alternative to traditional expert-based annotation, especially when considering the balance between cost and time efficiency. In light of these findings, we emphasize the use of the GPT-4 model for annotation tasks in our evaluation of models on INFOBENCH, considering its advantageous balance of cost, time efficiency, and accuracy.

heading: 4 Automatic Evaluation
text: In light of the findings from our previous experiments, it has become evident that GPT-4 represents a superior alternative for expert annotations. This approach not only offers a high degree of accuracy but also ensures time efficiency and costeffectiveness in annotation processes. Building on this insight, we integrate the gpt-4-0314 with
the multi-turn prompt for scaling up the evaluation across the entire INFOBENCH with six of the most cutting-edge LLMs. These include: gpt3.5-turbo-1106 (Brown et al., 2020) and gpt41106-preview (OpenAI, 2023a) released by OpenAI, Claude-2.1 (Anthropic, 2023) by Anthropic, Gemini-Pro (Team, 2023) by Google, Llama-2-70bchat (Hugo et al., 2023) by Meta and Vicuna-13bv1.5 (Zheng et al., 2023) by LMSYS. To avoid randomness during the decoding process, we use greedy decoding9 for all the models.
Overall: Table 5 presents the automated DRFR scores powered by gpt-4-0314 for the selected models on INFOBENCH. Upon analysis of the data presented in the table, three key observations can be made: (1) GPT-4 exhibits superior performance over the other LLMs, scoring the highest amongst all. However, even with this state-of-the-art model, more than 10% of the requirements remain unfulfilled, indicating a critical need for further enhancement within the aspect of instruction following. (2) All models except Claude-2.1 display better performance on the Easy Set as compared to the Hard Set, implying that the Hard Set poses a more formidable challenge. (3) The closed-sourced models (e.g. gpt-4-1106-preview, gpt3.5-turbo-1106, claude-2.1, gemini-pro) have better performance on Hard Set than those open-sourced models (e.g. Llama-2-70b-chat, Vicuna-13b-v1.5), which may
9temperature = 0, top_p = 1
indicate they have better data instruction coverage on more advanced instructions or they have better model generalization capabilities.
the instruction-following or most of the models have a good data coverage for such domains.

heading: 5 Related Work
text: The adaptation of LLMs for enhanced usability has garnered significant attention. Two prevalent approaches for this adaptation are instruction tuning and alignment tuning (Zhao et al., 2023). Instruction tuning (Wang et al., 2022; Lou et al., 2023) aims to unlock the capabilities of LLMs by finetuning them on annotated “instructional” data. This involves datasets with natural language instructions and their corresponding desired outcomes. Several LLM adaptations have embraced this method for effective refinement(Taori et al., 2023; Wei et al., 2021; Chung et al., 2022; Iyer et al., 2022; Ouyang et al., 2022). Alignment tuning (Ziegler et al., 2019; Christiano et al., 2017; Ouyang et al., 2022; Bai et al., 2022) differs from instruction tuning by emphasizing human feedback to ensure LLMs align with human values and preferences. This addresses LLMs’ tendencies to exhibit unintended behaviors like fabricating information or producing biased expressions(Ouyang et al., 2022; Kenton et al., 2021), steering them more in line with human expectations. A natural extension of this line of inquiry is the assessment of LLMs’ ability to follow natural language instructions and align with human preferences accurately. In terms of evaluation, using LLMs for Natural Language Generation evaluation has shown promise in numerous tasks (Zheng et al., 2023; Liu et al., 2023; Wang et al., 2023a; Fu et al., 2023; Kocmi and Federmann, 2023; Xu et al., 2023; Chiang et al., 2023; Eldan and Li, 2023). Given this context, in our proposed work, we employ LLMs as evaluators to assess their aptitude for instructionfollowing. Several works have concurrently assessed LLMs from various perspectives: Wang et al. (2023b) explore the reasoning capabilities
of LLMs for complex scientific problem solving; Wang et al. (2023c) present a judge LLM trained to distinguish the performance of various LLMs; Li et al. (2023a) introduce an instruction-following evaluation protocol named verbalizer manipulation on classification tasks; Li et al. (2023b) develop an LLM-based automatic evaluation method to calculate win-rate of the given model’s output favored over that of a reference model on an 805- instruction dataset (Dubois et al., 2023);Xu et al. (2023) propose an automatic method to compose complex instructions for instruction-tuning;Chia et al. (2023) introduce an evaluation suite designed for the assessment of instruction-tuned LLMs’ ability on problem-solving, writing proficiency, and alignment with human values;Jiang et al. (2023) adopts a distinct approach by sequentially adding fine-grained constraints to construct multi-level instructions;Zhou et al. (2023) emphasizes objective evaluations with 25 verifiable instructions, contrasting with our work’s broader coverage yet similar focus on objectivity.

heading: 6 Conclusion
text: This study introduces a significant breakthrough in evaluating Large Language Models (LLMs) through the Decomposed Requirements Following Ratio (DRFR), a novel metric that dissects complex instructions into simpler criteria for a more nuanced analysis. This approach, coupled with the creation of INFOBENCH, a comprehensive benchmark dataset, enables a detailed assessment of LLMs’ instruction-following capabilities. Our findings reveal the superiority of DRFR over traditional evaluation methods, offering clearer insights into model performance, especially in complex scenarios. The use of GPT-4 as an annotator emerged as a cost-effective, accurate, and efficient alternative for large-scale evaluations, highlighting its potential to automate and enhance the assessment process. The comprehensive evaluation of various LLMs using our methods uncovered their strengths and areas needing improvement. While significant progress is evident, especially in closed-source models, challenges remain in handling complex instructions, signaling a need for advancements in areas like numerical reasoning and linguistic comprehension. Our contributions provide a foundation for future research and development, guiding the enhancement of LLMs for more accurate and reliable performance in real-world applications. This study not only underscores the current capabilities of LLMs but also paves the way for the next generation of more robust and versatile AI models.
Limitations
This study is not without its limitations, which must be acknowledged:
1. The cost and resources required for human annotation have restricted us to providing these annotations for only a fraction of our instruction set. Specifically, a mere 10% of the instruction set, equating to 50 instructions, were annotated by humans. The reliability of our decomposed protocol, comparisons with the baseline protocol, and comparisons among different annotations are, therefore, based solely on this subset.
2. The size of our dataset presents another limitation. It comprises 500 instructions and 2,250 decomposed questions, representing a somewhat limited benchmark. The manual nature of our instruction writing process limits our capacity to scale this dataset significantly. Future work could explore the development of automated methods for instruction writing and question decomposition, potentially allowing for the creation of more scalable datasets.
3. Finally, the evaluation of our model is predominantly centered on the explicit intentions contained within the provided instructions. Several crucial factors, such as truthfulness and harmlessness (Askell et al., 2021), were not considered in this study. These elements, although not addressed in our current work, represent promising avenues for subsequent research endeavors.

heading: A Appendix
text: A.1 Instruction Dataset Statistics
Figure 5 illustrates the distributions pertaining to the instruction length for both the Easy Set and Hard Set, as well as the average number of questions per instruction for these sets. The graph indicates that the instruction length for the Easy Set predominantly ranges between 10-20 words, while
for the Hard Set it averages around 50 words. Similarly, the average number of questions per instruction for the Easy Set center is approximately 2-3, whereas, for the Hard Set, this number lies within the range of 5-7. These statistical variations elucidate the enhanced complexity associated with the Hard Set as compared to the Easy Set.
Table 6 presents a quantitative analysis related to Figure 2, depicting the categorization of questions according to their constraint type within the Easy Set and Hard Set. This tabulated data underscores the distribution of questions among different constraint types, providing an empirical perspective on the complexity gradation inherent in each set.
A.2 Prompt Design for GPT-4 Automatic Evaluation
In this section, we provide a detailed overview of the prompt design used in our study. The purpose of these prompts is to guide GPT-4 in evaluating text outputs generated by other LLMs.
Our dataset encompasses two types of instructions: with and without an accompanying input. For instance, a summarization task might have "Summarize the document" as an "Actual Instruction", and the document content as "Input". In cases where there is no direct input, the instruction itself guides the LLM, e.g., "Please give me 5 words to describe flowers." The "Total Instruction" sent to an LLM comprises the concatenation of "Actual Instruction" and "Input" (if present).
The prompt instructs GPT-4 to respond with a simple ’YES’ or ’NO’ to individually decomposed questions about the LLM’s generated text. This prompt includes the "Input" (if any), the LLM’s generated text, and a decomposed question. Notably, the "Actual Instruction" is not included in the Decomposed Protocol’s prompt, as we hypothesize that the decomposed questions encompass the instruction’s requirements. An example is shown in Table 7. As for multiple decomposed questions, We engage in a multi-turn dialogue with GPT-4. Each instruction and generation is followed by several questions, asked sequentially. GPT-4’s response to each question is used as context for the subsequent question. We also explored different methods, including asking all questions simultaneously and asking each question independently, but found the sequential multi-turn dialogue to yield more effective results.
We also experimented with incorporating a fewshot learning approach by providing examples in the prompt. However, the results indicated no significant improvement, possibly due to the varied
nature of the instructions and questions, and the simplicity of the answer format.
A.3 Human Annotation Guidance The guideline for crowd-sourced workers and interface used in the Amazon Mechanical Turk is demonstrated in Figure 6. Annotators are asked to read the guidelines before starting the annotation. We provide one extra option "UNKNOWN" for annotators if they find it hard enough to answer the questions with their knowledge. Such options will not be counted during the experiment and the evaluation. Systems are randomly shuffled in each instance to prevent the potential of memorizing which system is better.
A.4 GPT-4-1106 Evaluation Results In Table 8, we expand our evaluation to incorporate the latest iteration of the GPT-4 model (gpt-4-1106preview). The findings largely mirror the trends observed in Section 4. Notably, the scores attributed by gpt-4-1106-preview are marginally lower than those given by its predecessor, gpt-4-0314. This discrepancy warrants further investigation to understand the underlying factors influencing this variation in scoring.
Figures 7 and 8 present a detailed breakdown of the evaluation results, categorizing performance according to various constraint types and domains. These results are consistent with the findings discussed in Section 4.
The comprehensive analysis conducted with the gpt-4-1106-preview model reinforces our understanding of the capabilities and limitations of current LLMs. Moreover, the consistency in performance trends across different model versions underscores the reliability of GPT-4 as a valuable tool in annotation tasks.
A.5 GPT-4 Annotation Error Analysis This section examines the discrepancies between GPT-4’s annotations and those made by humans. We hypothesize two primary reasons for the differences: 1) Inherent ambiguities or subjectivities in the questions that challenge human consensus (e.g., "a letter around 250 words?" where "around" is vague, or "generate an appealing title" where "appealing" is subjective); 2) Actual errors in GPT-4’s annotations.
To explore the difficulty in achieving consensus on certain questions, we introduce the concept of a "Disagreement Level" which quantifies the discrepancy between single expert annotation and the gold standard, defined as the majority vote, for
each question. This metric serves to illustrate the degree to which human annotators differ in their assessments for each question. Specifically, for a given question and a single LLM’s output, we incorporate the binary ‘YES’ or ‘NO’ evaluations from three independent annotators, from which the majority vote is designated as the gold standard. Consequently, for a single LLM’s output, the Disagreement Level may be as low as 0 (indicating consensus among all three annotators) or as high as 1 (signifying that one annotator diverged from the decision of the other two). Given that our experimental framework deploys five LMs for each question, the Disagreement Level possesses a potential range of 0 to 5.
The distribution of Disagreement Levels, as shown in Figure 9, reveals that around 30% to 40% of questions exhibit a medium to high degree of human disagreement (Disagreement Levels at least 2). In assessing instances where GPT-4’s annotations deviates from human consensus, we disregard these questions. This approach stems from the rationale that a lack of human consensus undermines the legitimacy of labeling GPT-4’s differing response as erroneous.
We then focus on the remaining 60% to 70% of the questions that manifest high agreement among human evaluators, yet GPT-4’s annotations do not align. In these cases, we can more confidently identify GPT-4’s annotations as errors. Our investigation uncovered several recurring reasons for GPT-4’s missteps:
1) The LLMs generations incorporate misleading content. For example:
Instruction: Generate an eviction notice for a tenant named John Smith, residing at 123 Main Street, Cityville, NY, 87562, for nonpayment of rent for the past two months. The notice should give the tenant 30 days to vacate the property.
Question: Does the generated eviction notice include the tenant’s name: John Smith?
Generation: Generated an eviction notice for tenant John Smith, residing at

heading: 123 Main Street, Cityville, NY, 87562, for nonpayment of rent for the past two months. The notice should give the ten-
text: ant 30 days to vacate the property.
Correct Answer: NO.
GPT-4 Answer: YES.
In this case, the LLM’s generation simply reiterates the instruction and does not produce an actual eviction notice. Despite this, the inclusion of the phrase ‘tenant’s name: John Smith’ can mislead GPT-4 into assuming that it has correctly completed the task.
2) Requirements necessitating numerical computation or counting. Consider the following instances:
Instruction: Generate a random alphanumeric string with exactly 20 characters.
Question: Does the generated string exactly include 20 characters?
Generation: Here is a 20 character random alphanumeric string:
a3k92j7h6r5t1c4b8
Correct Answer: NO.
GPT-4 Answer: YES.
In the illustrated case, GPT-4 is unsuccessful in accurately counting a sequence of 20 characters.
Instruction: Describe a landscape painting from three distinct layers: foreground, middle ground, and background. Include at least two types of trees, a body of water, and a mountain range. Use a limited color palette of no more than four colors to create harmony, and specify which natural elements correspond to each color. Additionally, provide a description of the textures and brushstrokes used to represent the different natural elements.
Question: Does the painting described by the generated text include at least two types of trees, a body of water, and a mountain range?
Generation: A landscape painting with a foreground of trees and shrubs, a middle ground of rolling hills and a lake, and a background of mountains and trees. The colors used are blues, greens, and oranges to represent the different elements. Soft brushstrokes are used to create a realistic texture of the landscape, while a more bold brushstroke is used to create the mountains.
Correct Answer: NO.
GPT-4 Answer: YES.
Similarly, when tasked with identifying and enumerating two distinct types of trees, GPT-4 may incorrectly categorize repeated references to trees as unique types.
3) The prerequisite for a specific requirement is not met. For instance,
Instruction: Please provide five unique first date night ideas, with each catering to different interests and budgets. Include an estimated cost for each idea, along with a brief description of the activity and why it might be appealing to different types of people.
Question: Does each idea in the generated text stand out as unique, each one catering to different budget ranges?
Generation: 1. A picnic in the park - This is a classic first date idea that is sure to impress. It’s budget-friendly and can be customized to fit your interests, whether you want to pack a basket with gourmet snacks or bring a Frisbee to play catch. The cost of a picnic in the park can vary depending on what you bring, but it’s generally a very affordable option.
2. Art museum visit - If you and your date are into art, a visit to a local museum can be a great first date idea. Many museums have free admission days, so it won’t break the bank. This activity can lead to interesting conversations and can be a great way to get to know each other better.
3. Hiking or biking - If you and your date are outdoorsy, a hike or bike ride
can be a fun and unique first date idea. Depending on the location, this activity can range from moderate to intense, so it’s important to choose a route that is suitable for both of you. The cost of this activity will depend on the equipment you need to rent, if any.

heading: 4. Cooking class - If you and your date
text: love to cook or want to learn, a cooking class can be a great first date idea. Many cooking classes are held in the evening, so it can be a fun and unique way to spend an evening together. The cost of a cooking class can vary depending on the location and the class, but it’s generally a more expensive option.

heading: 5. Board game cafe - If you and your date are looking for a low-key, indoor
text: activity, a visit to a board game cafe can be a fun and unique first date idea. Many board game cafes have a wide selection of games to choose from, so you can find something that suits your interests. The cost of this activity will depend on how much you spend on food and drinks, but it’s generally a more affordable option than a traditional dinner date.
Correct Answer: NO.
GPT-4 Answer: YES.
In this illustration, the generated output fails to satisfy the requirement set forth in the instructions to ‘include an estimated cost for each idea’. As a result, the generated output does not provide sufficient information to address the question with the requirement ‘each one catering to different budget ranges’.
4) Requisite Spatial Reasoning Constraints. For illustration, consider the following:
Instruction: The objective of the Item Elimination Game is to eliminate items in a 6x6 grid by aligning 3 or more of the same item consecutively in a row or column.
Here are the rules:
1. The grid consists of 6 rows and 6 columns, with each cell containing one of three different items represented by A, B, or C.

heading: 2. In each step, you may remove items in
text: several cells, provided that they form a
consecutive sequence of 3 or more of the same item in the same row or column.
3. Once the item in a cell is removed, it leaves a blank cell, represented by "#".
Based on the rules, you should generate a 6x6 grid filled with 3 different kinds of items (A, B, and C) as the initial setup and play the game for one step: If you can identify items that form a sequence of 3 or more of the same item in a row or column, leaving blank cells ("#") in their place and generating the resulting grid; if you cannot find such a sequence, explain it.
Question: Does the generated text include a 6x6 grid as the initial step?
Generation: Here is the initial 6x6 grid filled with 3 different kinds of items (A, B, and C):
“‘diff
A B C
A B C
A B C
A B C
A B C
A B C
“‘
After one step of the game, the grid should look like this:
“‘diff
A B C
A B C
A B C
A B C
A B C
A B C
#
“‘
The sequence of 3 or more of the same item that was eliminated is not possible in this case, as there are no consecutive sequences of 3 or more of the same item in the same row or column.
Correct Answer: NO.
GPT-4 Answer: YES.
Within this exemplar, the instruction necessitates the generation of a ‘6x6 grid’. The generation proclaims ‘Here is the initial 6x6 grid’, yet the rendered grid does not correspond to the stipulated 6x6 dimensions. This instance thus necessitates spatial reasoning proficiency to accurately evaluate whether the produced grid adheres to the 6x6 specification, rather than relying solely on textual interpretation. Being a language model, GPT-4 might demonstrate limitations in spatial reasoning capabilities.
5) The requisite condition remains partially unfulfilled. For instance, consider the following example:
Instruction: Design a 7-day challenge for individuals looking to explore a new hobby: painting. Each day should include a specific task and a difficulty level. The activities should progress in difficulty. Include a brief description (20-30 words) for each day’s task.
Question: Does every single day of the challenge in the generated text incorporate a particular task along with a corresponding level of difficulty?
Generation: Day 1: Start with a blank canvas and a painting medium of your choice. Begin with a simple painting task, such as tracing a design or painting a color wheel.
Day 2: Paint a landscape or other outdoor scene using light brushstrokes.
Day 3: Paint a portrait or figure using a photo as reference.
Day 4: Experiment with abstract art by painting with different materials or using mixed media.
Day 5: Create a pattern or design by painting repeating shapes or lines.
Day 6: Paint a still life or indoor scene using shadow and light.
Day 7: Create a composition using multiple colors and textures.
Correct Answer: NO.
GPT-4 Answer: YES.
In the given scenario, the question necessitates a response containing ‘a specific task along with a corresponding degree of difficulty’. The generated output successfully encapsulates ‘a specific task’, albeit fails to adhere to the secondary part of the
requirement, that is to incorporate ‘a corresponding degree of difficulty’.
A.6 Case Study As highlighted in section 4, all models exhibit their lowest performance on Number and Linguistic constraints. Consequently, we selected two specific instructions embodying these constraints for a detailed case study. This section presents the instructions, the related decomposed questions, the responses generated by six chosen models, and the annotation results from GPT-4-0314, GPT-4-1106, as well as expert assessments for the Yes/No responses to all decomposed questions. This analysis aims to provide an intuitive understanding of how each model struggles with the given instructions. For an exhaustive dataset and the complete outputs for all instructions from the six models, please consult our GitHub repository.
Tables 9 to 11 focus primarily on the number constraint. These tables reveal that none of the models fully satisfied the 4th and 5th constraints regarding numbers. This underperformance in the Number category highlights the inherent challenges in numerical reasoning across all LLMs. Specifically, generating a DNA sequence while maintaining the proportion of purines and pyrimidines is complex. Moreover, ensuring equal representation of the four nucleotide types while maintaining this proportion is particularly challenging due to their somewhat contradictory nature.
Tables 12 to 13 are dedicated to the linguistic constraint. In this case, none of the models completely satisfied the 3rd and 4th Linguistic constraints. Notably, GPT-4-1106 demonstrated a commendable attempt to adhere to the "begin with the last letter of the previous word" rule, successfully applying it to most words, albeit compromising the clarity of the plot description (yet still aligning broadly with the Star Wars theme). In contrast, the other models did not seem to attempt this complex task. The suboptimal results in the Linguistic category likely stem from the unique design of our constraints. While LLMs excel in generating grammatically correct and coherent text, they were not optimized for the atypical linguistic patterns introduced by our constraints, like requiring each word to begin with the last letter of the preceding word in this example. Given that most LLMs are trained to predict subsequent words or characters based on conventional linguistic patterns, they struggle when asked to produce content that deviates markedly from these norms.

heading: Future Work
text: Our contributions provide a foundation for future research and development, guiding the enhancement of LLMs for more accurate and reliable performance in real-world applications. This study not only underscores the current capabilities of LLMs but also paves the way for the next generation of more robust and versatile AI models. Limitations
This study is not without its limitations, which must be acknowledged:
1. The cost and resources required for human annotation have restricted us to providing these annotations for only a fraction of our instruction set. Specifically, a mere 10% of the instruction set, equating to 50 instructions, were annotated by humans. The reliability of our decomposed protocol, comparisons with the baseline protocol, and comparisons among different annotations are, therefore, based solely on this subset. 2. The size of our dataset presents another limitation. It comprises 500 instructions and 2,250 decomposed questions, representing a somewhat limited benchmark. The manual nature of our instruction writing process limits our capacity to scale this dataset significantly. Future work could explore the development of automated methods for instruction writing and question decomposition, potentially allowing for the creation of more scalable datasets. 3. Finally, the evaluation of our model is predominantly centered on the explicit intentions contained within the provided instructions. Several crucial factors, such as truthfulness and harmlessness (Askell et al. , 2021), were not considered in this study. These elements, although not addressed in our current work, represent promising avenues for subsequent research endeavors