M isinfo B ench: A Multi-Dimensional Benchmark for Evaluating LLM s’ Resilience to Misinformation (2025)

Abstract: Large Language Models (LLMs) excel in various Natural Language Processing (NLP) tasks but remain vulnerable to misinformation, particularly in multi-turn dialogues where misleading context accumulates. Existing benchmarks, such as TruthfulQA and FEVER, assess factual accuracy in isolated queries but fail to evaluate LLMs’ resilience to misinformation in interactive settings. To address this limitation, we introduce MisinfoBench, a multi-dimensional benchmark designed to assess LLMs’ ability to discern, resist, and reject misinformation. MisinfoBench defines three core dimensions—Discernment, Resistance, and Principled Refusal—across seven evaluation tasks, systematically testing misinformation identification, contextual resistance, and the rejection of coercive false premises. It includes a dataset of 4,962 multi-turn dialogues and 2,000 misinformation-based question-answer pairs, capturing diverse misinformation scenarios. We evaluate 16 LLMs, revealing substantial disparities in misinformation resilience: proprietary models outperform open-source counterparts, while multi-turn dialogues and cross-lingual settings exacerbate misinformation susceptibility. Our findings highlight persistent vulnerabilities in LLMs’ misinformation defenses, emphasizing the need for context-aware training, adversarial robustness, and principled reasoning. MisinfoBench establishes a rigorous standard for evaluating misinformation resilience, advancing the development of more trustworthy AI systems.

## introduction large language models (llms) have achieved remarkable performance across a wide range of nlp applications, including question answering, machine translation, and content generation. their widespread deployment in real-world systems, such as search engines, virtual assistants, and automated content moderation, underscores their growing influence on public discourse. however, despite their impressive capabilities, llms remain highly vulnerable to misinformation, raising concerns about their trustworthiness. misinformation can arise not only from direct factually incorrect queries but also from misleading contextual history and explicit user instructions. as illustrated in figure 1 , realworld ai search and question-answering systems often fail to reject misinformation, instead incorporating misleading information into their responses. without the ability to recognize and resist misinformation, llms risk amplifying false narratives, spreading disinformation, and reinforcing biases across digital platforms. this raises a fundamental research question: how resilient are llms when exposed to different forms of misinformation? existing benchmarks such as truthfulqa @xcite , fever @xcite , and multifc @xcite primarily evaluate llms' factual accuracy in isolated queries by verifying claims against reliable sources. while these benchmarks provide valuable insights into factual consistency, they do not assess how models perform when exposed to misleading contextual narratives or explicit user instructions. multi-turn dialogues introduce additional challenges, where misinformation may accumulate progressively, requiring models to differentiate between truthful and misleading prior context @xcite . although benchmarks like multiwoz @xcite and mt-bench-101 @xcite assess dialogue coherence and reasoning, they fail to evaluate models' resistance to misleading context. in real-world applications, users may introduce incorrect premises or manipulate context to mislead responses, highlighting the need for models with robust resistance mechanisms. current benchmarks do not systematically address these challenges, leaving a critical gap in evaluating llms' resilience to misinformation. to fill this gap, we introduce misinfobench, a multi-dimensional benchmark designed to evaluate llms' ability to resist misinformation in both isolated queries and multi-turn dialogues with misleading context. figure 2 illustrates the overall framework of our evaluation system, which systematically assesses llms' resilience to misinformation across multiple dimensions. at the highest level, the framework evaluates models based on three progressive capabilities: discernment, resistance, and principled. discernment represents the foundational ability to recognize misleading questions. resistance builds upon this by assessing whether models can reject misleading contextual information and maintain factual consistency. finally, principled examines whether models can refuse to comply with instructions that explicitly rely on fabricated content, while correctly identifying the inherent fallacies in such input. to enable comprehensive evaluation, the framework further delineates six specific subcategories, which are operationalized through seven distinct evaluation tasks. to automate the evaluation process, we design task-specific prompts that enable llms to serve as evaluation judges, and validate this approach through comparative experiments with human expert assessments. based on this systematic framework, we make the following key contributions: we conduct systematic experiments on 16 llms, including 3 proprietary and 13 opensource models, revealing significant disparities in misinformation resistance. our findings highlight critical vulnerabilities and provide insights into improving model robustness against misinformation. the following sections provide a detailed explanation of our evaluation methodology and experimental findings. ## conclusion in this paper, we introduce misinfobench, a multi-dimensional benchmark designed to systematically assess the resilience of llms against misinformation in both isolated queries and multi-turn dialogues. our framework evaluates misinformation resistance across three core dimensions-discernment, resistance, and principled-which are further operationalized into seven specific evaluation tasks. through rigorous experimentation on 16 llms, encompassing both opensource and proprietary models, we uncover significant disparities in misinformation robustness. notably, closed-source models, such as gpt-4o and claude-3.5 sonnet, substantially outperform their open-source counterparts, particularly in resisting misleading contextual cues and rejecting coercive misinformation. these findings underscore the critical challenge of enhancing llms' misinformation immunity and highlight the necessity of integrating context-aware training, adversarial robustness, and principled reasoning mechanisms to fortify their reliability. future work should explore multilingual, domain-specific, and adversarial misinformation scenarios, as well as investigate dynamic misinformation resistance in extended interactions and real-time adaptation strategies. we envision misinfobench as a catalyst for advancing research in misinformation resilience, ultimately fostering the development of more truthful, contextaware, and ethically aligned ai systems.