A dv ERSEM : Adversarial Robustness Testing and Training of LLM -based Groundedness Evaluators via Semantic Structure Manipulation (2025)

Abstract: Evaluating outputs from large language models (LLMs) presents significant challenges, especially as hallucinations and adversarial manipulations are often difficult to detect. Existing evaluation methods lack robustness against subtle yet intentional linguistic alterations, necessitating novel techniques for reliably assessing model-generated content. Training accurate and robust groundedness evaluators is key for mitigating hallucinations and ensuring the alignment of model or human-generated claims to real-world evidence. However, as we show, many models, while optimizing for accuracy, lack robustness to subtle variations of claims, making them unsuitable and brittle in real-world settings where adversaries employ purposeful and deceitful tactics like hedging to deceive readers, which go beyond surface-level variations. To address this problem, we propose AdvERSem, a controllable adversarial approach to manipulating LLM output via Abstract Meaning Representations (AMR) to generate attack claims of multiple fine-grained types, followed by automatic verification of the correct label. By systematically manipulating a unique linguistic facet AdvERSem provides an interpretable testbed for gauging robustness as well as useful training data. We demonstrate that utilizing these AMR manipulations during training across multiple fact verification datasets helps improve the accuracy and robustness of groundedness evaluation while also minimizing the requirement of costly annotated data. To encourage further systematic evaluation, we release AdvERSem-Test, a manually verified groundedness test-bed.

## introduction evaluating the reliability of human or modelgenerated claims typically involves human judgment, which can be costly, and insufficiently sensitive to subtle manipulations in generated text. automatic evaluators like llm-judges offer scalable 1 https://github.com/emory-irlab/adversem alternatives for assessing critical properties like groundedness, and factual accuracy, yet these evaluators themselves often rely on llms and thus inherit their limitations, including susceptibility to hallucinations, adversarial perturbations and the transformers' non-compositional nature of training @xcite . developing robust automatic evaluators is particularly challenging, as subtle semantic changes @xcite , purposeful negations, or intentional manipulations like hedging @xcite can cause dramatic degradation in evaluation reliability. besides, most evaluation testbeds rarely offer fine-grained performance assessments, providing almost zero feedback to evaluation assessors. hence, to improve trustworthiness, it is vital that groundedness are robust as well as provide finegrained feedback of performance. groundedness evaluation or fact verification refers to the alignment of human or modelgenerated claims with real-world evidence. this alignment is crucial for assessing the truthfulness of statements against established knowledge. however, current language models often exhibit vulnerabilities that compromise their groundedness. for instance, the llama-3.1-7b model fails to consistently predict the factual correctness of claims when claims are slightly altered to make them more specific or when they are simply negated as shown in figure 1 . in this work, we focus on improving groundedness evaluation by fine-tuning on adversarially generated examples. specifically, using popular fact verification datasets, we construct adversarial claims designed to expose strategic vulnerabilities in llms. however, generating effective adversarial attacks remains an open research question, requiring careful strategies to ensure examples both challenging and representative examples. attempts to create adversarial examples for claim verification have traditionally focused on surface-level perturbations to challenge model robustness @xcite . these methods typically involve introducing noise, substituting entities, or making minor lexical alterations to input texts. while such approaches have been instrumental in exploiting llm vulnerabilities, they often fail to capture actual errors involving deeper semantic and syntactic complexities @xcite or carefully crafted manipulation like hedging often used to dodge potential disagreement @xcite -for instance journalists may use phrases like "some suggest," "it might appear," or "many doubt" to propose claims while distancing themselves from responsibility or certainty. besides, many of these traditional techniques, while revealing some issues related to logical reasoning or factual consistency, rarely provide clarity on specific patterns that evaluators fail to understand. moreover, these surface-level perturbations can sometimes lead to unnatural or ungrammatical sentences, limiting their effectiveness in realworld applications. to improve groundedness evaluation, it is hence crucial to identify more abstract, structured, and compositional patterns that mimic human language patterns as well as provide finegrained assessments. in this paper, we investigate two research questions -rq1: how can we systematically gener-ate adversarial claims to attack automated groundedness evaluators, and which attacks are particularly effective against sota llm-based evaluators? rq2: how can we generate useful adversarial data for training groundedness evaluators that are robust to such attacks? to investigate these research questions, we propose a novel framework that systematically generates adversarial examples through manipulations in a higher-order logical space, particularly through abstract meaning representations (amr). by extracting away from syntactic variations, amr provides a structured, graph-based representation of sentence semantics. by manipulating claims in their abstract representations, we can create challenging test cases that expose specific weaknesses in groundedness evaluators, which provide systematic feedback of groundedness performance while addressing the typical issues. in summary, our contributions are the following: 1. we identify a family of semantics-based adversarial claim manipulations that resemble humanlike manipulations like hedging, and show that these manipulations can successfully mislead sota llm-based groundedness evaluators. 4. we demonstrate how adversem can be used to identify potential vulnerabilities and be used to improve accuracy and robustness on multiple fact verification benchmarks. ## related work we now discuss related research to place our contributions in context. false or adversarial claims designed to mislead human readers, and now machine-reading models, have been long studied and analyzed. apart from deliberate biases exhibited by journalists and traditional information providers, many frequently employ rhetorical techniques such as hedging to preempt disagreement @xcite @xcite or careful wordplay aimed at misleading or unduly influencing readers. recently, the problem of misinformation and disinformation has become critically important due to their unprecedented speed and reach, further amplified by digital platforms and recent llms. specifically, @xcite demonstrated that false news spreads significantly faster, farther, and more broadly than truthful information, underscoring the urgent need for improved claim verification methods. recent studies, such as those by @xcite , have emphasized this growing challenge by illustrating how ai-generated misinformation convincingly integrates fabricated details with truthful elements, effectively evading traditional detection approaches. hence, automated evaluation of claims, or fact verification, has been an active area of research. works such as multifc @xcite , liar (wang, 2017), and averitec @xcite , emphasize real-world claims verified by journalists or professionals, offering more diverse and context-rich challenges. fever-ous @xcite expands on earlier work by incorporating structured data like tables, while domain-specific datasets like scifact @xcite and covid-fact @xcite target scientific or health misinformation. to improve the robustness of fact verification systems, prior works have produced a range of datasets used to train and evaluate fact-checking systems. datasets like fever @xcite , fever 2.0 @xcite , and vitaminc @xcite focus on generating adversarial or subtly false claims to test model robustness, often using wikipedia as a source, but most of their adversarial generations either use uncontrolled manipulations or use flat first-order logic representations like openie triples @xcite . conqret @xcite @xcite provides a benchmark of long-form generations with controlled hallucinations for fine-grained evaluation of groundedness in retrieval-augmented systems. while previous work has considered adversarial claim generation to evaluate and train fact checking models, our work is the first, to our knowledge, to systematically investigate the specific types of semantic manipulation that can successfully attack state-of-the-art llm-based fact checkers, and propose a controllable way to generate successful se-mantic manipulation attacks that can be used to both evaluate existing fact checkers, and to automatically augment training data to make the fact checkers more robust to a range of attacks. ## proposed method: adversem we now describe our proposed method, ad-versem starting with the end-to-end overview of the method and then diving into the details of the implementation for each module. ## experiments we now present the choice of datasets and models used along with the corresponding experiments. ## results we now present the results for all our evaluations. ## conclusions and future work we introduced adversem, a novel framework that leverages abstract meaning representations (amr) to systematically evaluate and improve the robustness of groundedness evaluators through structured adversarial data augmentation. we uncover several vulnerabilities by introducing semantically controlled, fine-grained manipulations of claims. these adversarial manipulations expose specific weaknesses in existing evaluators and provide interpretable and actionable feedback beyond single-dimensional test scores, highlighting the importance of robustness and interpretability in groundedness evaluation, and also serve as a caution on agent-style modular systems, which predominantly rely on llms in a zero-shot manner. moreover, training groundedness evaluators using our structured adversarial examples significantly enhances their ability to withstand complex semantic perturbations such as hedging, negation, and specificity adjustments, addressing key vulnerabilities identified in llms as well as human writings. by generating challenging yet realistic training examples, our approach effectively reduces reliance on expensive annotated data, thereby facilitating efficient and robust model development. the adversem framework, and associated evaluation set adversem-test can be readily extended to various other nlp applications, for systematically testing and enhancing model reliability.