FOFO : A Benchmark to Evaluate LLM s’ Format-Following Capability (2024)

Abstract: This paper presents FoFo, a pioneering benchmark for evaluating large language models’ (LLMs) ability to follow complex, domain-specific formats, a crucial yet under-examined capability for their application as AI agents. Despite LLMs’ advancements, existing benchmarks fail to assess their format-following proficiency adequately. FoFo fills this gap with a diverse range of real-world formats and instructions, developed through an AI-Human collaborative method. Our evaluation across both open-source (e.g., Llama 2, WizardLM) and closed-source (e.g., GPT-4, PALM2, Gemini) LLMs highlights three key findings: open-source models significantly lag behind closed-source ones in format adherence; LLMs’ format-following performance is independent of their content generation quality; and LLMs’ format proficiency varies across different domains. These insights suggest the need for specialized tuning for format-following skills and highlight FoFo’s role in guiding the selection of domain-specific AI agents. FoFo will be publicly released, contributing a critical tool for advancing LLM evaluation and application.

## introduction large language models (llms) show great promise in automating diverse tasks, from medical @xcite @xcite and legal data analysis @xcite @xcite to daily assistance with activities like reservations @xcite @xcite . for llms to effectively assist humans, their crucial ability lies in following instructions @xcite @xcite . in tasks requiring them to act as ai agents, such as organizing medical records or generating kpi reports, precise adherence to specified formats given by humans is essential @xcite . without this capability, the practicality of employing llms in such roles diminishes significantly. however, prior evaluation benchmarks of llms fall short in assessing their format-following capabilities. on one hand, mainstream instructionfollowing benchmarks, e.g., alpacaeval @xcite and mt-bench @xcite , are in the open question-answering/chatting style, assessing the correctness of llm responded content (therefore, we refer to those as content-following benchmarks) without explicitly considering their format-following capability. stateof-the-art llms, whether closed or open-source, increasingly demonstrate comparable performance on these content-following benchmarks. on the other hand, benchmarks evaluating ai agents @xcite @xcite in specific environments typically prioritize the overall success rate in completing test tasks. since there are many factors that can affect final success rates, such as the model's grounding and reasoning capability in the specific environment, the final success rates cannot directly gauge llms' format-following proficiency. consequently, although we observe llms performing notably worse than humans in certain domains @xcite , it remains unclear if this discrepancy is partly attributed to format-following limitations. exploring enhancements in llms' formatfollowing may potentially pave the way for further improvements in their role as ai agents. in this work, we take the lead to build fofo, a benchmark specifically designed for assessing llms' format-following capablities. we equip fofo with two shining features: a wide coverage of real-world domain-specific formats (such as hl7-cda format @xcite in healthcare) in various domains that are likely to embrace llms as agents, and complicated and practically occurring format-oriented instructions so that the llms can be tested with real-world complex context. to achieve these, we implement an ai-human collaborative strategy for developing fofo , featuring a structured hierarchical layout through a three-step process: i) identification and collection of domains and subdomains that ai agents can assist humans; ii) creation of data formats specific to each subdomain; iii) generation of format-oriented instructions (format-instru) that include complex format requirements and real-world context for each @xcite . in experiments, we test a diverse set of closedsource llms (e.g., @xcite , palm2 @xcite , gemini (google, 2023)) and open-source llms, such as llama 2 @xcite , wizardlm @xcite , mistral @xcite , etc., on these format-instru . we use gpt-4 as the main judge and human evaluation is conducted to ensure a high gpt4-human agreement. the following three observations are highlighted: these observations suggest two key points: i) the format-following capacity of llms appears independent of other capabilities and may necessitate specialized alignment fine-tuning beyond the conventional instruction-tuning used in training opensource llms. ii) format-following capacity is not universally transferable across domains, highlighting the potential utility of our benchmark as a guiding and probing tool for selecting domain-specific ai agent foundation models. to our knowledge, this is the first work that breaks down llms' instruction-following behavior into content-following and format-following, and benchmarks the evaluation of format-following capacity. this study contributes valuable insights into comprehending llms' capabilities and offer guidance in selecting llms, particularly for domainspecific agent development. our fofo dataset will be public released. ## related work content-following benchmarks. there are various existing efforts that build evaluation data sets to try to assess llms' general problemsolving capability through conversations. among them, mmlu dataset @xcite ) is collected to measure knowledge and problem solving capabilities of llms in different knowledge domains such as elementary mathematics and us history. the performance of llms on mmlu is measured by the accuracy of their selected answers from multiple options. alpacaeval @xcite and mt-bench @xcite , on the other hand, collect open-ended questions accross different domains without providing concrete reference answers. they rely on llms such as gpt-4 to conduct automatic evaluations on the target llm's answers. except for these general benchmarks, there are also evaluation data sets specifically focusing on assessing llm answers' truthfulness @xcite and safety @xcite @xcite . ## experiments in this section, we present our empirical results and analysis of fofo. to verify fofo's effectiveness on serving as a format-following benchmark of llms, we firstly select top-performing llms from both the closed-source and open-source world and evaluate them on fofo. specifically, we are more interested in middle-sized llms that have shown similar performances to gpt-3.5 and gpt-4 on existing content-following benchmarks such as mt-bench and alpacaeval because such llms are currently most widely-used. to evaluate each open-source llm, we employ its official prompt format to conduct generation given each test prompt in fofo. during generation, we use sampling and set the temperature as 0.7 for all models for fair comparison. we also ## conclusion in conclusion, our introduction of fofo marks a significant advancement in evaluating large language models' (llms) format-following capabilities, a crucial but previously overlooked aspect. through a novel ai-human collaborative construction, fofo offers a comprehensive benchmark covering a diverse range of formats and instructions. our findings reveal that format-following is an independent skill set not correlated with content generation performance, highlight a gap between open and closed-source llms in format adherence, and underscore the variability of llms' formatfollowing proficiency across domains.