Beyond Traditional Benchmarks: Analyzing Behaviors of Open LLM s on Data-to-Text Generation (2024)

Abstract: We analyze the behaviors of open large language models (LLMs) on the task of data-to-text (D2T) generation, i.e., generating coherent and relevant text from structured data. To avoid the issue of LLM training data contamination with standard benchmarks, we design Quintd - a tool for collecting novel structured data records from public APIs. We find that open LLMs (Llama 2, Mistral, and Zephyr) can generate fluent and coherent texts in zero-shot settings from data in common formats collected with Quintd. However, we show that the semantic accuracy of the outputs is a major issue: both according to human annotators and our reference-free metric based on GPT-4, more than 80% of the outputs of open LLMs contain at least one semantic error. We publicly release the code, data, and model outputs.

## introduction large language models (llms; @xcite @xcite have already left a mark in many areas of natural language processing (nlp). surprisingly, their applicability to the task of data-to-text (d2t) generation @xcite remains underexplored, with limited evaluation on a handful of well-established benchmarks only @xcite . generating text from structured data is arguably challenging for llms, given the specifics of d2t generation, such as long inputs, complex non-linear structure, and strict requirements on semantic accuracy. however, a more significant issue is the lack of testing grounds. the current d2t generation benchmarks are not only getting saturated @xcite , but also promote optimization towards traditional reference-based evaluation metrics, which were shown to correlate poorly with human judgment @xcite @xcite . when it comes to the models, using closed llms @xcite is increasingly considered a bad research practice due to its non-reproducibility @xcite . on top of that, contamination of llm training data with standard benchmarks further restricts the space for experiments @xcite @xcite . in this paper, we propose an approach that allows us to analyze model behavior in d2t generation on novel, real-world structured data records with reference-free evaluation metrics. we begin by realizing that unlabeled data are plentiful. to leverage the data for our experiments, we introduce quintd foot_0 -a tool for collecting structured data from five domains in standard formats: json, csv, and markdown. we choose the domains so that the data can be directly used as input for five distinct d2t generation tasks. our tasks include generating weather forecasts, sports reports, product descriptions, chart captions, and entity descriptions (see we push towards better reproducibility by focusing on open llms, which -apart from being more accessible -also achieve increasingly better results across tasks @xcite . for our experiments, we use three open llms with 7b parameters: llama 2 @xcite , mistral @xcite , and zephyr @xcite . we also use @xcite as a closed model baseline for the final experiments. given the behavioral nature of the experiments with llms @xcite , we put emphasis on reporting model behavior throughout the process ( §3). another piece of the puzzle is reference-free evaluation: using the input data as a ground for comparison instead of reference outputs ( §4). we focus on identifying semantic errors in the model outputs, i.e., the information that is not supported by the input data. we use two separate evaluation methods: manual annotations from human crowdworkers @xcite and a custom automatic metric based on gpt-4 @xcite @xcite . we annotate the errors on the level of individual words, getting fine-grained annotations of error spans in several categories @xcite . based on our results, we provide general recommendations for d2t generation with open llms across tasks and formats ( §5). our main findings are as follows: ## evaluation for evaluation, we focus on identifying semantic errors in model outputs. we compare the generated texts to the input data, looking for parts of texts that are not faithful to the input data. we annotate the errors on the word level, considering all the words in the output text as potential sources of errors. we use two complementary referenceless evaluation methods: the methods use similar instructions and produce outputs with equivalent format. the main idea is to compensate for the shortcomings of each approach: while human evaluation is costly (about ten times more expensive than automatic evaluation), using only an automatic metric based on a closed llm would make the evaluation potentially non-reproducible and biased @xcite . reporting the results of both methods should hopefully increase the robustness and replicability of our results. our error taxonomy and its notation are inspired by @xcite and @xcite . after preliminary examination of the outputs, we settled on four error categories: incor-rect i , not_checkable nc , mislead-ing m , and other o . to set clear boundaries between the categories, simplify the annotation interface and reach better inter-annotator agreement, we decided to keep our taxonomy more high-level than thomson and reiter and not to distinguish between fine-grained categories (e.g., incorrect name vs. incorrect number). the descriptions of our error categories, as presented in the instructions for annotation, are included in ## results and discussion a summary of the word-level annotations is in ## conclusion we provided an exploratory study into d2t generation with open llms. we proposed new directions for d2t generation, including using ad-hoc test sets, data in common formats, and referencefree evaluation. by a combination of gpt-4-based metric and human evaluation, we evaluated the performance of llms on five domains, providing word-level annotations of model outputs across five domains and recommendations for future directions in d2t generation.