Towards Better Evaluation of Instruction-Following: A Case-Study in Summarization (2023)

Abstract: Despite recent advances, evaluating how well large language models (LLMs) follow user instructions remains an open problem. While evaluation methods of language models have seen a rise in prompt-based approaches, limited work on the correctness of these methods has been conducted. In this work, we perform a meta-evaluation of a variety of metrics to quantify how accurately they measure the instruction-following abilities of LLMs. Our investigation is performed on grounded query-based summarization by collecting a new short-form, real-world dataset riSum, containing 300 document-instruction pairs with 3 answers each. All 900 answers are rated by 3 human annotators. Using riSum, we analyze the agreement between evaluation methods and human judgment. Finally, we propose new LLM-based reference-free evaluation methods that improve upon established baselines and perform on par with costly reference-based metrics that require high-quality summaries.

## introduction large language models (llms) have shown human-level performance in many nlp tasks. recent advances in instruction tuning @xcite and alignment @xcite have dramatically increased the ability of these models to follow instructions. in addition to being used to tackle unseen tasks in zero-shot setups @xcite , these models are now also used as surrogates to human annotators, especially for nlg tasks @xcite , where human evaluations are time-consuming and expensive. consider the instruction "briefly describe the purpose of the assignment and assumption agreement mentioned in the paragraph" from figure 1 . there are several dimensions to evaluate a generated output on: (i) coherence: whether it is understandable and free of grammatical mistakes, (ii) faithfulness: whether facts in the output are supported by the document, (iii) style: whether specific formatting requirements (lists, brevity, ...) are met, and (iv) alignment: whether it semantically fulfills the instruction. analyzing these different facets for each model output increases the cognitive load of annotators, thereby increasing the likelihood of errors or lowquality evaluations @xcite . it also increases the turnaround time and hence annotations become expensive. an increasingly popular alternative is to ask llms to evaluate the generated outputs. recent work like @xcite and @xcite show that llms can produce humanlike evaluations of text by using clever prompting techniques @xcite . but preliminary studies have shown that llms can be inconsistent in their evaluations and can easily be influenced @xcite . @xcite have also looked at evaluation flaws and have recommended that metric developers should focus on metrics with smaller, but better defined scopes (like instruction-following). hence, there is an urgent need for a standard framework to analyze the specifics of instructionfollowing abilities of llms. summeval @xcite proposes something similar for vanilla summarization. doing this for instructionfollowing can be tricky because we would like to not only evaluate the llms as task solvers "summarize this document in 20 words or less", but also as task evaluators "does the summary satisfy the conditions of the instruction?". the metaevaluation framework should be robust and ideally reference-free @xcite . reference-free evaluation for text generation has been widely studied @xcite @xcite , but to the best of our knowledge, there has document: lee, you should be receiving a package shortly containing the following: (...) 3. assignment and assumption agreement to move the equipment from turbopark to the caed i. there will be one for caed ii as well. this document is being reviewed by the bank, so i'm not convinced it is in final form. you will note that there is an acknowledgement section for ge. i cut and pasted from the consent to assignment from the turbopark documents, but shortened the whole thing considerably. here's that document: 4. signature pages (signed by enron) from the esa deal, both the facility agreement and the override letter. obviously, we need your signature. i will forward the final ca facility agreements to you once again, along with the blacklines against what you initialled. ## evaluation methods we propose and evaluate several methods that model annotator preferences, focusing our analysis on reference-based vs. reference-free methods and their effectiveness in different data regimes. ## evaluating agreement with annotators as discussed in section 2, we asked annotators to provide a binary yes/no rating answering whether a model output follows the instruction and a qualitative rating from 1 to 5, representing how well it follows the instruction. using meta-evaluation methods described below, we then study agreement between annotators and our evaluation methods. ## results and analysis we compare the effectiveness of evaluation methods on the three rating dimensions, based on the reported numbers for the binary rating "follows instruction?" and for the qualitative rating "how well?" in ## related work measuring instruction following with llms @xcite use gpt-4 as a backbone model and study the correlation with human ratings on non-query-based summarization, finding a bias towards llm-generated text. we do not study this aspect, as our rating task focuses on model-generated text. @xcite propose a zero-shot approach for multi-faceted evaluation of text generation. an increase in interest for improving instructionfollowing capabilities of llms has resulted in the creation of multiple datasets. flan @xcite and natural instructions @xcite were two of the earlier datasets which turned standard nlp tasks (e.g. sentiment classification, question-answering) into instruction following tasks. other works like self-instruct @xcite , super-naturalinstructions @xcite , and the h4 instruction dataset (hugging face, 2023) curate human-written instruction and answer pairs. @xcite and qingyi si (2023) collect instruction-answer pairs from llm generations. all of them use standard nlp metrics or human annotation to evaluate the model outputs. model-based metrics a large body of prior work focuses on model-based approaches fine-tuned on human ratings. usually, encoder models such as bertscore @xcite or bleurt @xcite are used, but encoder-decoder methods exist as well @xcite . we focus on low-resource zero/few-shot methods using larger, decoder-containing models from palm and gpt families. kryściński et al. (2018 ## conclusion in this work, we investigate the effectiveness of multiple evaluation methods in quantifying the degree to which llm-generated text follows usergiven instructions. we release risum, a new shortform dataset of 300 document-instruction pairs with 3 answers each. all of the 900 answers are rated by at least 3 human annotators. when analyzing agreement between evaluation methods and human judgment, we find that established metrics, such as rouge and bleurt are not effective at quantifying llms' instruction-following ability. llm-based evaluation methods tend to have stronger correlation with annotator judgment, without requiring high-quality reference answers. we hope that the introduced evaluation framework is adopted by the community for evaluating instruction-following abilities of llms, possibly expanding into more tasks, domains, and examples.