An Audit on the Perspectives and Challenges of Hallucinations in NLP (2024)

Abstract: We audit how hallucination in large language models (LLMs) is characterized in peer-reviewed literature, using a critical examination of 103 publications across NLP research. Through the examination of the literature, we identify a lack of agreement with the term ‘hallucination’ in the field of NLP. Additionally, to compliment our audit, we conduct a survey with 171 practitioners from the field of NLP and AI to capture varying perspectives on hallucination. Our analysis calls for the necessity of explicit definitions and frameworks outlining hallucination within NLP, highlighting potential challenges, and our survey inputs provide a thematic understanding of the influence and ramifications of hallucination in society.

## introduction recent advancements in natural language processing (nlp) have expanded beyond traditional machine learning tools, evolving into sociotechnical systems that combine social and technical aspects to achieve specific goals @xcite . they have now become integral in various domains such as health, policy-making, and entertainment, @xcite showcasing their significant impact on daily life. however, language models (lm) exhibit negative behaviours such as hallucination and biases @xcite . this has catalyzed a surge in research investigating the phenomenon of hallucinations in nlp @xcite , reflected in the escalating number of peer-reviewed publications on the topic, as illustrated in fig 1 , sourced from scopus. within the nlp domain, various frameworks have emerged to define hallucination, primarily emphasizing its negative aspect. hallucination here refers to the model's production of references to non-existent objects or statements, lacking supporting examples in the training data @xcite . despite the growing research on this topic, there is still a notable divide in our understanding, a lack of a unified framework, and a need for precise definitions @xcite . the necessity to understand this gap is accentuated by research demonstrating the societal impacts of hallucinations @xcite . hence, there is a growing need to explore how the field of nlp conceptualizes hallucination. in line with this imperative, the following questions guide this study: to answer the rq1, we first conduct an audit of the field of hallucinations in nlp by surveying 103 peer-reviewed articles 1 . subsequently, we conduct a survey to 171 researchers and academics in the field to gather their perspectives on this phenomenon, providing a novel contribution to the literature, addressing rq2. by surveying nlp practitioners, the paper incorporates real-world perspectives, enriching the theoretical discussions with practical insights. this audit therefore aims to broaden the communities perspective by presenting practical insights from researchers employing these methods in their work. we also propose an ethical framework to guide future efforts in comprehending and mitigating hallucinations in llms. ## evolution of hallucination in nlp the term 'hallucination' has a long history in machine learning and has been used in various contexts prior to the lm era. its earliest documented usage can be traced to the 2000s when @xcite applied it in the context of image resolution enhancement, referring to the generation of new pixel values. subsequently, "hallucination" has been frequently employed in computer vision research, including notable works such as @xcite on face hallucination. in the modern deep learning era, hallucination was used first by andrej karpathy in his blog focusing on recurrent neural networks @xcite . he used the term within the context of lm by illustrating how an lstm could generate nonexistent urls, effectively 'hallucinating' data. the term then gained major traction with the launch of chatgpt @xcite , where it referred to inaccuracies and factual mistakes produced by models @xcite . however, the field lacked a unified definition, leading to a spectrum of interpretations @xcite . in one of the earlier works, @xcite divides term usage into intrinsic and extrinsic hallucination. intrinsic hallucinations are consequences of synthesizing content using the information present in the input. extrinsic hallucinations are model generations that ignore the source material altogether. however, there is a rise in discussion around terminology that reflects a deeper inquiry into the phenomena, with recent discourse advocating for 'confabulation' @xcite or 'fabrications' @xcite as a more precise descriptor. this reflects the lack of consensus on the term and highlights the importance of looking at the use of hallucination with a more critical lens. ## related surveys on hallucination we now provide an overview of several key surveys in the realm of nlp focusing on the topic of hallucination and why our work addresses a relevant gap in the field. starting with @xcite , this survey extensively delves into the advance-ments and challenges concerning hallucination in nlg, distinguishing between intrinsic and extrinsic frameworks of hallucination. additionally, it sheds light on fundamental terms such as hallucination, faithfulness, and factuality, along with prevalent metrics for quantifying these phenomena. rawte et al. (2023b) while these surveys offer insights into the current state of hallucination research, they do not pay attention to critical examinations of the field's weaknesses arising from a lack of discourse in defining hallucination and challenges due to the same. this deficiency in discussion reflects the broader trend within the entire field. therefore, our audit answers this gap by critically examining how we conceptualize hallucination. we aim to highlight the challenges stemming from these definitions and to further conduct a practitioner survey within the community to understand researchers' and developers' perspectives on this issue. our goal is to facilitate a shared understanding of the challenges and mitigation strategies for hallucination and develop actionable design principles through an ethics sheet to address these issues effectively. ## critical analysis of hallucination in nlp literature this section is dedicated to conducting an audit of hallucination research within nlp, aiming to uncover its applications and subsequently identify the strengths and weaknesses in current literature. to accomplish this, we conducted an audit of works from the acl anthology using specific keywords such as 'hallucination', 'nlp (or) ai' and 'hallucinations', 'fabrication', and 'confabulations'. we surveyed papers released on and before april 19th, 2024. from this search, a total of 164 papers were retrieved. after filtering out papers that were not directly related to hallucination research or those that merely mentioned the term without substantial focus on the topic, we arrived at a corpus of 103 papers. this corpus forms the basis for our audit and analysis of hallucination research, specifically within the nlp domain. ## practitioner survey of hallucination in this section, adopting a 'community-centric approach' @xcite , we conduct a survey to gain insights into researchers' perceptions of hallucinations in nlp to complement our theoretical discussions with practical real-world perspectives. the primary goal is to demonstrate how researchers and practitioners within the field perceive the concept of 'hallucination' and to expand our findings beyond the limitations of existing literature where real-world perceptions from the researchers are missing @xcite @xcite . this motivates us to gather real-world perspectives from individuals actively engaged in nlp and ai research. ## challenges and recommendations based on our audit and survey analysis, we outline the key weaknesses encountered in hallucination within nlp and potential recommendations motivated by the weaknesses. we utilize a communitycentric approach to define the primary weaknesses of the field currently and a path forward. ## conclusion our work delves into the conceptualization of hallucination within the scope of nlp. our approach involved two key methodologies: first, an exhaustive audit of 103 peer-reviewed papers in the nlp domain, and second, a practitioner survey of 171 researchers to complement our first study with realworld practical perception and understanding of hallucination as a unique contribution. through this analysis, we have gained insights into how the nlp community conceptualizes and defines hallucination, showcasing a lack of discourse and agreement. additionally, our thematic and community-based approach highlights potential weaknesses within the field, particularly in addressing misrepresentations and inaccurate characterizations associated with hallucination, paving way for better advancement in language generation. our work finally contributes to a deeper understanding of the challenges and gaps in research related to hallucination within nlp, paving the way for future advancements in nlp and language generation.