DOVE : A Large-Scale Multi-Dimensional Predictions Dataset Towards Meaningful LLM Evaluation (2025)

Abstract: Recent work found that LLMs are sensitive to a wide range of arbitrary prompt dimensions, including the type of delimiters, answer enumerators, instruction wording, and more. This throws into question popular single-prompt evaluation practices. We present DOVE (Dataset Of Variation Evaluation) a large-scale dataset containing prompt perturbations of various evaluation benchmarks. In contrast to previous work, we examine LLM sensitivity from an holistic perspective, and assess the joint effects of perturbations along various dimensions, resulting in thousands of perturbations per instance. We evaluate several model families against DOVE, leading to several findings, including efficient methods for choosing well-performing prompts, observing that few-shot examples reduce sensitivity, and identifying instances which are inherently hard across all perturbations. DOVE consists of more than 250M prompt perturbations and model outputs, which we make publicly available to spur a community-wide effort toward meaningful, robust, and efficient evaluation. Browse the data, contribute, and more at: https://slab-nlp.github.io/DOVE

## definitions: prompt sensitivity in this section, we establish terminology, definitions, and metrics for formally quantifying the phenomenon of prompt sensitivity. in this work, we choose to focus on multiple-choice questions to allow for a relatively easy evaluation of model outputs compared to text generation tasks, such as summarization or translation, where the space of correct predictions is vast, and may be considered in future work. intent-preserving prompts. following chatterjee et al. (2024), two prompts p 1 , p 2 are considered intent-preserving if they are designed to convey the same underlying meaning, despite differences in phrasing or structure. for example, the two following prompts are considered intent preserving p 1 = "who is the partner of mario? choose from: a. donito b. lagio c. luigi", p 2 = "answer the following question: who is the partner of mario? a. donito b. lagio c. luigi". ## dove: a large-scale multi-dimensional dataset of llm-generated responses towards meaningful llm evaluation in this section we introduce dove, a large-scale corpus of model predictions along multiple dimensions. as shown in figure 1 , the building blocks of dove are instances from existing popular datasets. for each instance, we create a wide range of intentpreserving prompts, by varying the instances along five dimensions (enumerator, separator, choices order, phrasing, and demonstrations). below we discuss the different dimensions, which are also summarized in domains. we cover a wide range of data sources, spanning 78 different data sets from mmlu @xcite , mmlu pro, dimension examples # of values enumerator roman, numerals 6 separator ;, | 7 choices order original, correct first 6 phrasing the following are multiplechoice questions about {topic }. {question }{choices }answer: arc @xcite , hellaswag @xcite , openbookqa @xcite , social iqa @xcite , and race @xcite . from each of these, we take 100 instances chosen at random, resulting in 7,800 base instances, which we extend with different perturbations in subsequent steps. figure 2 shows that solving these samples requires a wide range of skills. answer enumerators, choice separators and orderings. recent work has noticed that very subtle changes in the prompt can lead to significant changes in both absolute as well as relative model performance. these include answer enumerators, e.g., roman versus numeral options, choice separators, e.g., new line versus commas, and the order in which the options are presented, e.g., the position of the correct answer @xcite @xcite . all options are summarized in ## evaluation in this section, we evaluate various models against dove, finding that they all exhibit prompt sensitivity at large scale, also when controlling for most of our tested dimensions. ## call for community collaboration our public release of dove provides a comprehensive dataset and standardized schema for multidimensional llm evaluations. yet, the scale and scope of llm evaluation demands collaborative efforts beyond the capacity of individual research groups. by establishing dove as an open and extensible platform, we invite the research community to contribute existing model evaluations, enhancing the collective understanding of prompt sensitivity across diverse models, tasks, and languages. in this manner, we can also improve the efficiency and accessibility of research into llm evaluation. instead of rerunning the same evaluation by independent groups unaware of each other's work, a public and standardized repository can provide a reference which developers can browse and enrich. 1 a living, evolving benchmark dove is designed as a dynamic, evolving benchmark that grows continuously through community contributions and ongoing analyses. recent updates include evaluations from models such as llama-3.3-70b, incorporated by converting evaluation data collected in a recent study @xcite , which we convert to our standardized format. the evolving nature of dove ensures up-to-date insights 1 https://slab-nlp.github.io/dove into model behavior and evaluation robustness. what to contribute? dove invites model predictions on existing benchmarks, especially from novel architectures or approaches. converting public datasets into the dove format expands coverage to new tasks, domains, and languages, particularly underrepresented or specialized areas. methodological innovations and suggestions for new evaluation benchmarks are also encouraged. how to contribute? contributions can be made through pull requests to our huggingface repository or coordinated directly via email for larger or more complex submissions. researchers seeking to include additional datasets or models in dove are encouraged to send requests specifying the desired datasets, models, or evaluation parameters clearly. we will properly attribute and reference all contributors. ## related work many studies which we have leveraged extensively throughout this work have focused on individual prompt dimensions, examining variations in instruction wording @xcite @xcite , answer ordering @xcite , input perplexity @xcite , few-shot example selection @xcite , and answer enumeration styles @xcite . some works propose metrics for prompt sensitivity, such as posix (chatterjee et al., 2024), which measures log-likelihood shifts, and prosa @xcite , which uses decoding confidence. although these methods quantify sensitivity, they do not examine interactions between multiple perturbations, nor do they collect data and make observations at a large scale. several recent work have noted that similarly to our findings, few-shot examples help improve performance @xcite . in contrast to these works, we show the effect that few-shot examples have on reducing prompt sensitivity. beyond investigating individual factors, several notable frameworks aim to standardize and improve evaluation process. helm @xcite takes a broad view of llm performance by creating a taxonomy of a wide range of use cases and evaluation metrics, but was not designed to examine prompt sensitivity. olmes @xcite establishes detailed protocols for the reproducibility of the evaluation, carefully specifying aspects such as prompt formatting. olmes demonstrated that standardizing these procedures could lead to more consistent results but may inadvertently harm models which do not perform well on its specific dimension choices. although these studies have provided valuable insights, our work is the first to take a holistic view of the problem. this large-scale dataset, encompassing more than 250m model predictions, allows us to aggregate across multiple prompt dimensions, noticing practical patterns, and opening the door for many future research directions. task-level sensitivity: do some model capabilities have distinct sensitivity patterns? for example, is factual retrieval more fragile than logical reason-ing? do format biases manifest differently across tasks from different domains? alternative evaluation measures: do less common approaches, like perplexity-based evaluation or sensitivity-aware assessments, better mitigate prompt sensitivity in benchmarks? @xcite do past prediction data help predict the most effective evaluation method for a new benchmark? @xcite optimizing evaluation focus: given resource constraints, what dimensions are most critical for assessing model performance? can a predictive framework identify the relative importance of different dimensions? instance characterization: what distinguishes consistently answered examples from those with high variability, e.g., as expemlified by the two ends of the spectrum in figure 9 ? do specific linguistic, semantic, or structural features influence susceptibility to example variation? uncertainty quantification: how do tokenlevel log probabilities relate to model consistency? can their distributions help predict or explain model sensitivity better than accuracy scores? towards that goal dove also records all model log probabilities. ## conclusions we introduced dove, a large-scale dataset of 250m model predictions across prompt dimensions. our analysis revealed prompt sensitivity remains a significant challenge, with performance varying over 10% across different prompt variations. key findings showed dimension-wise tuning outperforms entire-prompt optimization, few-shot demonstrations reduce but do not eliminate sensitivity, and certain examples remain challenging across all prompt variations. the public release of dove aims to democratize evaluation research and enable development of robust protocols for assessing llm capabilities.