Entity-Based Knowledge Conflicts in Question Answering (2021)

Abstract: Knowledge-dependent tasks typically use two sources of knowledge: parametric, learned at training time, and contextual, given as a passage at inference time. To understand how models use these sources together, we formalize the problem of knowledge conflicts, where the contextual information contradicts the learned information. Analyzing the behaviour of popular models, we measure their over-reliance on memorized information (the cause of hallucinations), and uncover important factors that exacerbate this behaviour. Lastly, we propose a simple method to mitigate over-reliance on parametric knowledge, which minimizes hallucination, and improves out-of-distribution generalization by 4% - 7%. Our findings demonstrate the importance for practitioners to evaluate model tendency to hallucinate rather than read, and show that our mitigation strategy encourages generalization to evolving information (i.e. time-dependent queries). To encourage these practices, we have released our framework for generating knowledge conflicts.

## introduction knowledge-dependent tasks, such as open-retrieval question answering (qa), require expansive "world knowledge", common sense, and reasoning abilities. state-of-the-art approaches typically follow a retrieve-and-read setup @xcite , where the retriever sources relevant documents, and the reader produces an answer from these. in this sense, there are two sources of knowledge contributing to model inference with an ambiguous and opaque division of labour. the first is the implicit parametric knowledge (i.e., their learned weights) instilled by pre-training and fine-tuning @xcite . the second is contextual knowledge, usu-question: who did us fight in world war 1? original context: the united states declared war on germany on april 6, 1917, over 2 years after world war i started . . . ## substitution framework we introduce a substitution framework for creating knowledge-conflicting instances. the framework maps a qa instance @xmath0, a, c), with query q, answer a, and the context passage c in which a appears, to @xmath1, a , c ) where a is replaced by substitution answer a as the gold answer, and where all occurrences of a in c have been replaced with a , producing new context c . this substitution framework extends partiallyautomated dataset creation techniques introduced by @xcite for ambiguous entity retrieval (amber). our dataset derivation follows two steps: (1) identifying qa instances with named entity answers, and (2) replacing all occurrences of the answer in the context with a substituted entity, effectively changing the answer. we provide tools to identify coherence-preserving substitutions and create substitutions with certain characteristics (e.g. semantic equivalence, or popularity score on wikipedia). ## related work overreliance on parametric knowledge @xcite showed that replacing the retrieved documents with random documents during entity-based substitutions key to our evaluation framework is substituting entity names with other plausible entity names. entity based swapping has been used to evaluate robustness in tasks such as coreference resolution @xcite and named entity resolution @xcite as well as to train more robust models @xcite . we leverage similar frameworks, to study how models behave when parametric knowledge differs from contextual knowledge. ## conclusion in this work, we examine how conflicts between contextual and parametric knowledge affect question answering models. in formalizing this problem, we first contribute a substitution framework for creating knowledge conflicts and evaluating model behaviour. using this framework, we conduct a detailed examination of knowledge conflicts in qa. finally, we propose a method to mitigate memorization and consequently improve generalization on out-of-distribution examples. our findings show knowledge conflicts are an underexplored topic, providing valuable insights into model interpretability and generalization to evolving world knowledge.