Conflicts in Texts: Data, Implications and Challenges (2025)

Abstract: As NLP models become increasingly integrated into real-world applications, it becomes clear that there is a need to address the fact that models often rely on and generate conflicting information. Conflicts could reflect the complexity of situations, changes that need to be explained and dealt with, difficulties in data annotation, and mistakes in generated outputs. In all cases, disregarding the conflicts in data could result in undesired behaviors of models and undermine NLP models’ reliability and trustworthiness. This survey categorizes these conflicts into three key areas: (1) natural texts on the web, where factual inconsistencies, subjective biases, and multiple perspectives introduce contradictions; (2) human-annotated data, where annotator disagreements, mistakes, and societal biases impact model training; and (3) model interactions, where hallucinations and knowledge conflicts emerge during deployment. While prior work has addressed some of these conflicts in isolation, we unify them under the broader concept of conflicting information, analyze their implications, and discuss mitigation strategies. We highlight key challenges and future directions for developing conflict-aware NLP systems that can reason over and reconcile conflicting information more effectively.

## introduction the rapid advancement of natural language processing (nlp), particularly with the rise of large language models (llms), has led to their widespread adoption in daily tasks, information retrieval, and decision-making processes. however, the increasing complexity of these models reveals various types of conflicts at multiple stages, including training, annotation, and model interaction, affecting the reliability and trustworthiness of downstream applications. for example, training models on data containing factual contradictions, annotation disagreements, or prompts that contradict a model's figure 1 : examples of the three different areas of conflicts discussed in this work. the first example describes a case where two different entities of the same name are found naturally on the web, the second example elaborates the annotation disagreement in a sentiment analysis task, and the third showcases a knowledge conflict between the context and memory of llms during model interactions. existing work on conflicts in nlp tends to focus on specific issues, such as annotation disagreements @xcite , hallucinations and factuality @xcite , and knowledge conflicts @xcite , without synthesizing these problems into a broader perspective. in this survey, we conceptualize these diverse challenges under the umbrella of conflicting information and analyze their origins, implications, and mitigation strategies. to ensure comprehensive and representative coverage of conflicts in nlp, we first establish a high-level categorization encompassing three primary sources: (1) natural conflicts present in web data, (2) conflicts arising from human annotation, and (3) conflicts emerging from model interactions. notably, conflicts found in natural web texts and human-annotated datasets are primarily present in the training data-i.e., the inputs to models-whereas conflicts involving model interactions can arise in various forms, such as inconsistencies between model outputs and their inputs, contradictions among multiple outputs, or conflicts within the outputs themselves. for each category, we identify influential and widely cited survey papers as initial seed works @xcite @xcite @xcite . building upon these seeds, we systematically trace and incorporate the most impactful and representative studies for each type of conflict through citation chaining and targeted literature searches across major databases. this approach enables us to synthesize developments in each category and connect them, thereby providing an integrated discussion of current challenges, impacts on downstream tasks, and promising future directions for conflict-aware ai systems (see appendix b for more discussion about our survey methodology). the abundance of online data is accompanied by inherent conflicts, stemming from diverse sources, interpretations, and biases. these con-flicts manifest as factual conflicts, such as semantic ambiguities @xcite and factual inconsistencies @xcite , or as conflicts in opinions related to political ideologies @xcite and perspectives @xcite . factual conflicts are particularly prevalent in open-domain question answering (qa) and retrieval-augmented generation (rag) systems @xcite , where aggregating knowledge from multiple sources introduces inconsistencies @xcite . these challenges highlight the need for conflict-aware retrieval and reasoning mechanisms to improve model reliability @xcite . unlike factual conflicts, opinionated disagreements reflect the variability in human interpretation, beliefs, and ideological stances @xcite . the presence of conflicting viewpoints complicates tasks such as summarization, sentiment analysis, and dialogue generation, where maintaining coherence and neutrality is crucial @xcite . furthermore, the uneven distribution and biases of web data also affects models to behave from a western perspective @xcite . another significant conflict arises in human-annotated data. for instance, annotation disagreements persists in both subjective and seemingly objective nlp tasks @xcite . disagreements are widespread in sentiment analysis @xcite , hate speech detection @xcite , and even natural language inference (nli) @xcite . models trained on aggregated (e.g. majorityvote) labels struggle with ambiguous or highdisagreement examples, often treating them as hardto-learn or mislabeled @xcite . @xcite also find that standard nli models' uncertainty does not reflect the true ambiguity present in human opinions, leading to overconfidence in contentious cases. in addition, annotation biases-such as those related to race, gender, and geography-skew model predictions and reinforce societal biases @xcite @xcite . these issues highlight the need for fair and representative annotations that capture the complexity of human disagreement. conflicts also emerge during interactions with models, manifesting as knowledge conflicts between model memories and contexts, and hallucinations in generated outputs. knowledge conflicts arise when a model's internal memory contradicts external contextual evidence, as shown by @xcite , who found that models often overly depend on memorized knowledge, leading to hallucinations. @xcite proposed separating parametric and contextual knowledge to improve interpretability, while @xcite examined llms' confirmation bias, showing how models inconsistently handle contradictory evidence. additionally, hallucinations-ranging from factual inconsistencies @xcite to contextual hallucinations @xcite )-further undermine model reliability. various mitigation strategies have been proposed, including retrieval augmentation @xcite , hallucination detection @xcite , and knowledge graph-based verification @xcite . in this survey, we systematically examine the landscape of conflicts in nlp by categorizing them into three primary sources. for each conflict type, we detail how such conflicts arise and in what forms they take (origins), the challenges they pose (implications), and the strategies developed to ad-dress them (mitigation). we present a comprehensive taxonomy in figure 2 , as well as structured summary tables- ## conflicts in natural texts on the web conflicts in natural texts on the web manifest in diverse ways, reflecting the inherent complexity and subjectivity of human language. they can broadly be categorized into factual conflicts, which revolve around factual discrepancies caused by various reasons, and conflicts in opinions, which pertain to divergent perspectives or biases. ## conflicts in human-annotated texts conflicts in human-annotated texts largely arise from two sources: annotation disagreements and societal or ethical biases. disagreements stem from linguistic ambiguity, annotator backgrounds, and task design, while biases reflect systematic demographic or ideological influences that can skew labeling in consistent ways. though conceptually distinct, these sources often interact-biases may amplify disagreement or entrench disparities. differentiating between them is essential for understanding annotation-related conflicts and for developing more reliable and equitable nlp datasets. ## conflicts during model interactions conflicts during model interactions primarily manifest as knowledge conflicts and hallucinations, each posing distinct challenges. knowledge conflicts occur when a model's parametric memory contradicts contextual input or when inconsistencies arise across models, whereas hallucinations occur when outputs deviate from real-world facts or the given input. differentiating these two types of conflict clarifies their underlying causes and helps guide targeted mitigation strategies. ## connections, challenges and directions given the significance and impact of conflicts in nlp, we advocate for increased attention to the development of conflict-aware and robust ai systems. in this section, we highlight specific challenges by connecting different types of conflicts and propose concrete research directions to address them. culturally robust llms among the challenges outlined in this survey, the development of cul-turally robust llms remains particularly underexplored. cultural conflicts emerge both in naturally occurring web data and human-annotated datasets, where western-centric distributions dominate. prior studies reveal that llms often reflect the values and perspectives of western, englishspeaking populations @xcite @xcite , with misalignments especially pronounced for underrepresented personas and culturally sensitive topics @xcite . additionally, llms exhibit inconsistent behavior across languages @xcite @xcite , revealing internal cultural conflicts. these issues are rooted in the data: both the pre-train data and benchmark datasets commonly exhibit western-centric biases @xcite , causing models to default to western contexts and perform poorly on less-represented regions and cultures. however, to the best of our knowledge, no effective methodology has yet been proposed to address this issue. with the emergence of culturally distinct llms-such as qwen, trained largely on chinese data @xcite , and vikhr, trained on russian data @xcite -a promising direction is model fusion across culturally diverse models to achieve greater cultural balance @xcite . furthermore, advances in culture-specific llms and synthetic data generation offer the potential to curate more culturally representative training and evaluation datasets beyond western-centric narratives, supporting the development of culturally robust llms. ## conclusion we present a unified view of conflicting information in nlp, organizing the landscape into conflicts originating from (i) natural texts on the web, (ii) human annotations, and (iii) model interactions. this taxonomy connects lines of work that are often studied in isolation and clarifies how conflicts arise, what they imply for reliability, and how current methods aim to mitigate them. our synthesis argues that conflict awareness should guide the full pipeline: data collection that preserves disagreement and multiple perspectives, models that detect and categorize conflicts and respond with clarification, reasoning, or balanced presentation, and evaluations that measure calibration under disagreement, robustness to contradictory evidence, and cultural coverage across languages and regions.