SCORE : Systematic CO nsistency and Robustness Evaluation for Large Language Models (2025)

Abstract: Typical evaluations of Large Language Models (LLMs) report a single metric per dataset, often representing the model’s best-case performance under carefully selected settings. Unfortunately, this approach overlooks model robustness and reliability in real-world applications. For instance, simple paraphrasing of prompts on the MMLU-Pro dataset causes accuracy fluctuations of up to 10%, while reordering answer choices in the AGIEval dataset results in accuracy differences of up to 6.1%. While some studies discuss issues with LLM robustness, there is no unified or centralized framework for evaluating the robustness of language models. To address this gap and consolidate existing research on model robustness, we present SCORE ( S ystematic CO nsistency and R obustness E valuation), a comprehensive framework for non-adversarial evaluation of LLMs. The SCORE framework evaluates models by repeatedly testing them on the same benchmarks in various setups to give a realistic estimate of their accuracy and consistency. We will make the code publicly available to facilitate further development and research.

## introduction the evaluation of large language models (llms) typically focuses on a single accuracy metric per dataset, often derived from an optimized setup. this approach provides an incomplete picture of the model capabilities in real-world scenarios. for an llm to be trustworthy in practical applications, it must exhibit robustness, i.e., produce consistent responses when the input is rephrased or slightly altered. consistency is particularly crucial for factual questions in which an objective answer exists. in evaluate the multiple-choice question and select the most fitting response from 'a', 'b', 'c', 'd', 'e', 'f','g', 'h', 'i', 'j'. the best answer is d: benjamin franklin prompt 2 particular, consistent predictions do not necessarily equate to correct predictions. given two models with similar accuracy, the one that makes the same incorrect predictions across different setups is arguably preferable. recent research has highlighted the limitations of current llm evaluation practices. @xcite @xcite demonstrate the significant impact of simple input perturbations on model performance. @xcite further underscores the sensitivity of the models to seemingly minor changes in input formatting, such as changing the separator or spacing. although robustness analysis is gaining momentum in llm research, robustness evaluations are often scattered, ad hoc, and difficult to compare between models @xcite . we propose an open evaluation framework score: systematic consistency and robustness evaluation for large language models. score focuses on consistency alongside accuracy to provide a more nuanced understanding of llm capabilities and facilitate the development of more trustworthy and reliable models. our contributions are as follows: our experiments demonstrate that evaluating llms across multiple scenarios, considering a range of accuracy values rather than a single metric, and tracking consistency rate provide a more accurate assessment of the model's true capabilities. ## related work open llm @xcite is a centralized platform for evaluating llms in a consistent setup, ensuring fair comparisons. it uses datasets that are both relevant and challenging, but still relies on a single metric evaluation. promptbench @xcite focuses on adversarial robustness by providing tools to evaluate models on adversarial prompts -deliberate inputs designed to break their predictions. although effective, these adversarial attacks could be unrealistic and considerably change the semantics of input samples. promptbench evaluates models' worst-case performance by estimating how much accuracy degrades under various attacks. helm (holistic evaluation of language models) @xcite uses a multi-metric approach to assess the models across various scenarios. however, robustness analysis is limited to character-level perturbations, typos, and a small subset of contrast sets (gardner et al., 2020). ## conclusion our evaluation demonstrates that relying solely on a single-point evaluation provides an incomplete assessment of the llm capabilities. we offer a more nuanced and informative understanding of model performance by evaluating models under various conditions and reporting accuracy ranges and consistency rates. our score framework establishes a foundation for systematic llm evaluation, facilitating standardized analysis and research of non-adversarial robustness.