F act E val: Evaluating the Robustness of Fact Verification Systems in the Era of Large Language Models (2025)

Abstract: Whilst large language models (LLMs) have made significant advances in every natural language processing task, studies have shown that these models are vulnerable to small perturbations in the inputs, raising concerns about their robustness in the real-world. Given the rise of misinformation online and its significant impact on society, fact verification is one area in which assessing the robustness of models developed for this task is crucial. However, the robustness of LLMs in fact verification remains largely unexplored. In this paper, we introduce FactEval, a novel large-scale benchmark for extensive evaluation of LLMs in the fact verification domain covering 17 realistic word-level and character-level perturbations and 4 types of subpopulations. We investigate the robustness of several LLMs in zero-shot, few-shot, and chain-of-thought prompting. Our analysis using FEVER, one of the largest and most widely-used datasets for fact verification, reveals that LLMs are brittle to small input changes and also exhibit performance variations across different subpopulations.

## introduction large language models (llms) have marked a significant milestone in the natural language processing (nlp) field. however, despite their impressive capabilities across a variety of tasks @xcite @xcite , it has been shown that these models exhibit vulnerabilities to adversarial samples -subtle alterations to the input that can be easily understood by humans @xcite . this has raised concerns about the robustness of these models when deployed in the real world. in the era where vast amounts of information are generated and shared rapidly, fact verification has become an increasingly important area of research in nlp as it is one of the key methods for detecting misleading and false claims @xcite ; @xcite . given the widespread influence of misinformation, which can have significant political, societal, and economic impact, assessing the robustness of fact verification systems and having a comprehensive understanding of these models' failures becomes crucial. numerous efforts have been made to evaluate the robustness of nlp models in a variety of tasks, including sentiment analysis and natural language inference @xcite @xcite . while there have been some studies examining adversarial attacks on fact verification models by manipulating either the claims @xcite @xcite or the evidence @xcite , their scope remains limited to traditional task-specific fine-tuned models. in this paper, we propose facteval foot_0 , a novel benchmark for evaluating the robustness of llms for fact verification. to the best of our knowledge, we are the first to study llm-based fact verification models; prior studies have been limited to traditional bert style models. facteval evaluates the robustness of fact verification models through perturbations and subpopulations applied to the claims, leaving the evidence unchanged. this allows us to focus on evaluating the model's ability to handle variations in the claims without adding noise to the evidence. we introduce 16 word-level and character-level perturbations used to create adversarial datasets. we incorporate visual perturbations @xcite , expanding the scope of attacks on fact verification models. recognizing that regional accents and the influence of native languages impact the writing style of non-native english speakers in informal conversations @xcite , we integrate phonetic perturbations for a comprehensive evaluation. furthermore, we designed subpopulations based on demographic factors, temporal information, sentiment, and named entities. these subpopulations consist of distinct subgroups of data that may differ in specific properties. subpopulation analysis helps ensure that the model is not only accurate on the overall dataset but also performs consistently across all subgroups. this is not only important for fairness but also crucial for enhancing the overall reliability and effectiveness of the systems. with facteval, we examine the robustness of llm models for fact verification under zeroshot, few-shot, and chain-of-thought prompting. we evaluate three llms: llama3 8b @xcite , mistral 7b @xcite , and gemma 7b @xcite . for comparison, we also investigate the robustness of transformer-based pretrained language models (plms) such as bert @xcite . our experiments on the fever dataset, one of the largest and most widely-used benchmarks for fact verification @xcite , reveal that plms and llms based fact verification systems severely lack robustness against even simple perturbations, raising concerns about the trustworthiness of these systems and emphasizing the need for more robust methodologies. addressing these challenges is essential for ensuring the reliability and effectiveness of fact verification models in realworld applications. to summarize, our contributions are: many perturbations in facteval can be applied to tasks other than fact verification, making it easy to incorporate in evaluations beyond just reporting standard metrics on held-out data. ## facteval we present facteval, a framework for evaluating the robustness of fact verification models. our framework incorporates word-level and characterlevel perturbations, as well as visual and phonetic perturbations. furthermore, we design subpopulations of the data based on demographic factors, temporal information, sentiment, and named entities. facteval contains 16 realistic perturbations and 4 types of subpopulations. ## results and discussion to evaluate the robustness of llms and plms in different attack settings, we compute accuracy, f 1 , and attack success rate (asr). accuracy and f 1 are calculated on the fever test set as well as the adversarial test sets. asr is calculated as the percentage of adversarial examples that can successfully attack the target model. ## conclusion we introduced facteval, a novel large-scale benchmark designed to evaluate the robustness of large language models in the fact verification domain. we developed 16 word-level and characterlevel perturbations, along with 4 types of subpopulations. our comprehensive evaluation covered zero-shot, few-shot, and chain of thought prompting methods across llama, mistral, and gemma models. our analysis revealed that llms are not robust to these perturbations, with the llama model being the least robust compared to mistral and gemma. furthermore, our experiments show that these models show varying performance on subpopulations such as demographic groups, sentiment, entity types, and temporal information. this indicates that more work needs to be done to improve the robustness of the models. in future work, we plan to extend our framework to other monolingual and code-mixed languages, allowing for a more comprehensive evaluation of robustness across diverse linguistic contexts. we also aim to explore providing explanations for model failures through model explainability techniques, which can offer valuable insights into the model's behavior and help identify areas for improvement.