Who’s Who: Large Language Models Meet Knowledge Conflicts in Practice (2024)

Abstract: Retrieval-augmented generation (RAG) methods are viable solutions for addressing the static memory limits of pre-trained language models. Nevertheless, encountering conflicting sources of information within the retrieval context is an inevitable practical challenge. In such situations, the language models are recommended to transparently inform users about the conflicts rather than autonomously deciding what to present based on their inherent biases. To analyze how current large language models (LLMs) align with our recommendation, we introduce WhoQA, a public benchmark dataset to examine model’s behavior in knowledge conflict situations. We induce conflicts by asking about a common property among entities having the same name, resulting in questions with up to 8 distinctive answers. WhoQA evaluation set includes 5K questions across 13 Wikidata property types and 150K Wikipedia entities. Our experiments show that despite the simplicity of WhoQA questions, knowledge conflicts significantly degrades LLMs’ performance in RAG settings.

## introduction recent large-scale pretrained language models (llms) excel in tasks requiring natural language understanding @xcite @xcite . however, they often "hallucinate" plausible but incorrect content due to outdated or incorrect pretraining information @xcite @xcite . retrieval augmented generation (rag) methods provide contextual knowledge to address this, but conflicts within retrieval contexts are still inevitable @xcite . according to @xcite , under certain knowledge conflict circumstances, large language models (llms) may either ignore contextual knowledge and rely solely on their parametric memory or, more concerning, prefer certain contextual knowledge based on their popularity or specific input ordering. figure 1 illustrates a conflict where an llm prioritizes the popular facts about george washington, the u.s. president, over contexts about an inventor and a jazz trombonist with the same name. this action can lead to information loss and bias, particularly when users have limited knowledge about the subject and are conducting an information-seeking query. since eliciting llms parametric knowledge to handle specific cases is costly @xcite , we recommend that llms should inform their users about conflicting information, provide proper citations, and allow users to make informed decisions based on the presented evidence. to examine if current llms handle knowledge conflicts transparently and accurately, we construct whoqa, a dataset derived from wikipedia articles and their linked wikidata entities @xcite . in whoqa, we intentionally induce conflicts by making questions asking about shared properties of wikipedia entities with the same name. previous studies have explored llms' behavior in knowledge conflicts, either by substituting named entities @xcite or using another llm for counterfactuals @xcite . these methods can generate non-existent facts, which llms, trained for factual consistency, may avoid. our work mends the gap by sourcing all supporting documents from wikipedia, a widely used and trusted source for many nlp tasks. with 2-8 different answers per question, compared to a maximum of two in previous works @xcite @xcite , whoqa demonstrates the complication of knowledge conflicts, which are not only natural but also prevalent in practice. similar to solving knowledge conflicts, am-bigqa @xcite rewrites ambiguous user queries so that each version has only one answer. however, sometimes there can be no information to resolve the ambiguity. for example, the statements "george washington is a farmer" and "george washington is the first president of the united states" both refer to the same person, but can seem contradictory, if found in two separate documents without context. situatedqa @xcite creates knowledge conflicts by considering similar questions in different temporal or geographical contexts. in whoqa, where many entities share the same name, a wider range of properties can induce knowledge conflicts. in summary, our contributions include: ## our whoqa dataset from the set of named entities in the english wikipedia and wikidata dump, we group all entities that have the same name together, collect question-answer pairs as well as supporting contexts, and manually revise the evaluation set. take an example from figure 1 with a set s containing three people whose names are george washington: one is the first president of the us, who was also a military officer; another one is a belgium-born american inventor; and the last one is an american jazz trombonist. given a question asking about a property @xmath0, a model is expected to give all answers within a set @xmath1, military officer}, {inventor}, {jazz trombonist}}. formally, whoqa is a multi-answer question answering dataset in which each question can be described as a quadruplet (q, a, s, c), where q is a question asking about a property p of a named- ## conclusions we introduce whoqa, a high-quality benchmark dataset designed to examine language model behaviors in knowledge conflict situations. whoqa bridges the gap with previous work by inducing knowledge conflicts without generating counterfactuals, demonstrating that conflicts can arise not only from misinformation but also naturally due to ambiguity in retrieved contexts, user questions, or similarities among entities. through our ex-periments, we also show that many llms are not sensitive to subtle conflicts within their input, and thus only simple conflicts can significantly impair llm performance in rag settings.