Confabulation: The Surprising Value of Large Language Model Hallucinations (2024)

Abstract: This paper presents a systematic defense of large language model (LLM) hallucinations or ‘confabulations’ as a potential resource instead of a categorically negative pitfall. The standard view is that confabulations are inherently problematic and AI research should eliminate this flaw. In this paper, we argue and empirically demonstrate that measurable semantic characteristics of LLM confabulations mirror a human propensity to utilize increased narrativity as a cognitive resource for sense-making and communication. In other words, it has potential value. Specifically, we analyze popular hallucination benchmarks and reveal that hallucinated outputs display increased levels of narrativity and semantic coherence relative to veridical outputs. This finding reveals a tension in our usually dismissive understandings of confabulation. It suggests, counter-intuitively, that the tendency for LLMs to confabulate may be intimately associated with a positive capacity for coherent narrative-text generation.

## background artificial intelligence is rapidly becoming ubiquitous in science and scholarship with autoregressive large language models (llms) leading the way @xcite . since the concept's integration into natural language processing (nlp) discourse, hallucination in llms has been widely viewed as a conspicuous social harm and a critical bottleneck to real-world applications of llms. both popular, comprehensive academic surveys @xcite and public-facing technical reports @xcite @xcite position the problem of hallucination as one of llm's chief ethical and safety pitfalls, one that should be heavily mitigated in conjunction with other concerns like bias and toxicity. yet, despite a wide range of such mitigation efforts @xcite , hallucinations perdure, presenting an imminent risk to model trustworthiness @xcite , with epistemic consequences in truth-sensitive domains like law @xcite , medicine @xcite , finance @xcite , science @xcite @xcite , and education @xcite . with so much at stake, nlp researchers have naturally developed a normative stance against hallucinations, rightly extending epistemic worries to more far-reaching, ethical concerns like misinformation @xcite , deception @xcite , fairness @xcite , and the broader goals of alignment research @xcite . consequently, the observed commitment to uniform reduction of hallucinations to negligible levels is not just seen as a technical challenge but also a critical component of the broader mission to mitigate the societal stigmas and systemic risks associated with the broad deployment and wide adoption of llms. however, a smaller body of work advances the view that hallucinations are not inherently harmful. according to this view, hallucinated outputs have particular affordances. for instance, such outputs can be utilized for the exploration of domainspecific scenarios where properties of hallucination could be leveraged constructively @xcite . @xcite demonstrate that certain types of hallucinations are factual and could positively contribute to text summarization. @xcite explore the potential of hallucinatory responses as adversarial examples for enhancing the robustness of models. following an existing practice in computer vision to use hallucinated materials as synthetic training data, @xcite devise a visual scene hallucination mechanism to dynamically fill in the missing visual modality for text-only inference with multimodal machine translation models @xcite @xcite . this explorative view highlights the potential @xcite , and impossible to eliminate from llms as efforts to do so would be limited to a trade-off between generativity, creativity. and information accuracy @xcite @xcite . moreover, in many domain-specific applications, achieving an optimized balance between creativity and factuality is more effective for maximizing the utility of llms than merely attempting to eliminate hallucinations. llm use cases where hallucinations could be particularly valuable include discovering novel proteins @xcite , providing inspiration for creative writing @xcite , and formulating innovative legal analogies @xcite . in this paper, we attempt to broaden the concept of hallucination and argue that hallucination is closer in kind to the concept of 'confabulation', a term that has already gained popularity in public discourse on ai but has yet to become widespread in the academic literature. reorienting our conception of what is typically conceived of as hallucination towards confabulation offers a more flexible way to characterize, measure, and analyze factually inaccurate outputs. we first operationalize a common measure of narrativity following @xcite @xcite and then demonstrate empirically (see section 3) that 'hallucinations' manifest significantly higher 'narrativity' than strictly factual outputs. scholarship in cognitive science and cultural analytics suggests that narrativity provides beneficial cognitive and social effects and that, by extension, confabulations that demonstrate such features are likely to offer similar affordances. taking this broader perspective, in turn, allows researchers to move beyond a strictly normative approach that narrowly focuses on identifying and discarding hallucinations. as such, casting hallucination as confabulation offers a practical and tangible framework for adjusting to the nuance of hallucinations. ## in defense of confabulation in this section, we present a two-part argument that calls for a more careful and nuanced examination of confabulation and possible utilization of its affordances. we argue that confabulation's narrative-rich properties should not be viewed as a flaw but a hallmark for llm alignment with a wellestablished human tendency to use narratives as a versatile tool for persuasion, identity construction, and social negotiation @xcite . the normative view's unreflective dismissal of confabulation, in turn, would risk eliminating critical behavioral and cognitive capacities central to communication and sense-making from the capabilities of llms. we also consider how aspects of narrative theory could complicate and enrich our understanding of newly emergent concepts in nlp, like faithfulness and factuality. ## limitations and directions for future research this paper advances a systematic defense of llm confabulations as a potential resource instead of a categorically negative pitfall. we believe the mental picture that llms hallucinate because they are untrustworthy, unfaithful, and ultimately unhumanlike is an oversimplified one. instead, they confabulate and exhibit narrative-rich behavioral patterns closely resembling the human urge of storytelling -perhaps hallucinations make them more like us than we would like to admit. while our current findings reveal intriguing associations between increased narrativity and significant increases in coherence, we must refrain from asserting that narrativity drives coherence. however, this perspective does garner substantial support from the interdisciplinary perspectives presented in our qualitative analysis. this paper serves as a preliminary exploration in this direction, but we require more robust methods for modeling narratives and more comprehensive human evaluations to elucidate the intricacies of this association. we are eager to implement these steps in the next phase of our study. the textual behaviors and effects we observe in llm hallucination benchmarks, i.e., higher degrees of narrativity and coherence and the significant correlation between them, are widely considered beneficial in human-to-human communication. but the extent to which these affordances generalize to human-ai interactions-whether humans indeed benefit more in terms of user experience from engaging with high-narrative confabulations versus low-narrative but factual generations-needs to be further validated with human-based evaluations. we plan to follow up this study with experiments with human participants to verify the benefits of narrative engagement as hypothesized. if the effectiveness of narratively rich confabulations is robustly validated, it could lead to exciting avenues for future research, testing the efficacy of confabulations across diverse domains. beyond the applications we have already outlined in section 1, exploring the utility of confabulations in fields such as journalism and advertising could yield valuable insights. by adopting confabulations as a conceptual framework, we could inspire more cross-disciplinary explorations, opening up new possibilities for llm applications that extend beyond purely factual text generations. ## acknowledgement for the research and completion of this paper, we acknowledge the resources and support provided by the mcgill faculty of arts, mcgill department of english, and the division of research and faculty development at the harvard business school. partial funding for the project was provided by siva reddy through a frqnt new researchers award and a nserc discovery grant. we thank maria antoniak for her help with the paper. we also acknowledge the helpful feedback from audiences at the university of chicago's globus labs, data science institute, and knowledge lab, and harvard digital data design institute. finally, we sincerely thank our anonymous reviewers for their generous 7 time, attention, and feedback.