Citation: A Key to Building Responsible and Accountable Large Language Models (2024)

Abstract: Large Language Models (LLMs) bring transformative benefits alongside unique challenges, including intellectual property (IP) and ethical concerns. This position paper explores a novel angle to mitigate these risks, drawing parallels between LLMs and established web systems. We identify “citation”—the acknowledgement or reference to a source or evidence—as a crucial yet missing component in LLMs. Incorporating citation could enhance content transparency and verifiability, thereby confronting the IP and ethical issues in the deployment of LLMs. We further propose that a comprehensive citation mechanism for LLMs should account for both non-parametric and parametric content. Despite the complexity of implementing such a citation mechanism, along with the potential pitfalls, we advocate for its development. Building on this foundation, we outline several research problems in this area, aiming to guide future explorations towards building more responsible and accountable LLMs.

## introduction the landscape of artificial intelligence is undergoing rapid transformation, spurred by the emergence of large language models (llms) such as chatgpt/ @xcite . these models, recognized for their striking ability to generate human-like text, have shown enormous potential in various applications, from information provision to personalized assistance. nonetheless, their capabilities bring along substantial risks, including intellectual property (ip) and ethical concerns @xcite @xcite @xcite @xcite @xcite @xcite . research by @xcite ; @xcite , for instance, reveals that llms are prone to memorizing extensive segments of their training data, including sensitive information. this can result in violations of ip and ethical guidelines. furthermore, studies by @xcite ; @xcite suggest that current protective measures fail to provide a comprehensive and meaningful notion of safety for llms, making it seemingly impossible to develop safety-preserving, high-accuracy large language models even when trained on public corpora. while the notion of building an entirely safe llm might appear daunting, it is crucial to acknowledge that many well-established systems, such as the web, grapple with similar challenges and have not yet reached absolute safety. recent legislation like the online news act foot_0 , which requires online search engines to compensate canadian online news outlets for their content, underscores the ongoing issues around content use and compensation on the web. furthermore, the web continues to be a breeding ground for both sensitive information and misinformation. hence, expecting a completely risk-free llm may be an overask. instead, our focus should be on accurately quantifying these risks and developing effective mitigation strategies. it is not about achieving absolute security, but about responsibly managing and minimizing risks in an ethically sound manner. guided by these insights, we propose to examine the problem through a different lens: can we draw parallels between the risks inherent to llms and those experienced by established systems such as search engines and the web? can we devise strategies to decrease these risks by aligning with the practices of these mature systems? in examining systems like the web and search engines, we observe a common and robust practice employed to manage ip and ethical concerns: the use of "citations". broadly defined, a "citation" refers to the act of mentioning or referencing a source or piece of evidence. for example, search engine results also serve as a form of citation, with each entry typically consisting of a title, url, and brief description. these components collectively cite the webpage's content, offering the user an overview and inviting them to explore the source in greater depth. citations thus act as anchors for accountability and credit in these systems, providing traceability, preventing plagiarism, and ensuring credit is correctly attributed. they also contribute to transparency, allowing users to verify the information's source. upon reflection, it becomes clear that llms lack this critical functionality. when llms generate content without citations, their output is perceived as independent and self-derived. this creates two significant issues: firstly, when the model produces valuable information, it fails to credit the source it relies on; secondly, when it generates harmful content, it becomes challenging to assign accountability. incorporating the ability to cite could not only address these ethical and legal conundrums but also bolster the transparency, credibility, and overall integrity of the content generated by llms. however, implementing a "citation" mechanism in llms is not as straightforward as it might seem. unlike the web, which explicitly links and references sources, llms internalize the information and transform it into hidden representations, making accurate citation a significant technical challenge. although some strides have been made in this direction, as seen in systems like new bing foot_1 and perplexity ai foot_2 , they fall short on several fronts. first, the citations provided in the response of existing systems are often inaccurate @xcite . moreover, these systems typically only cite non-parametric content, i.e., content directly retrieved from external sources such as the web. however, they neglect parametric content, the knowledge embedded in the model parameters, which also needs appropriate credit attribution and consideration for potential harm. this position paper embarks on an exploratory journey into the potential of integrating a citation mechanism within large language models, examining its prospective benefits, the inherent technical obstacles, and foreseeable pitfalls. we delve into approaches to cite both non-parametric and parametric content, considering the unique character-istics of each type. we also identify and discuss potential setbacks, such as reduced creativity, dissemination of sensitive information, and citation bias. building on this foundation, we lay bare the hurdles in our path, presenting them as enticing problems for future research. through this endeavor, we aim to stimulate further discussion and research towards building responsible and accountable large language models. ## overview of large language models large language models are typically built on the foundation of transformer architectures @xcite . the training process of llms usually involves self-supervised learning on vast quantities of text data, including books, articles, and internet content, primarily sourced from the web. during this stage, models are exposed to diverse textual data, allowing them to learn grammar, facts @xcite , and even reasoning abilities @xcite . following the initial training, models may undergo further training on smaller, labeled datasets. for instance, chatgpt (openai, 2022), a prominent llm, is fine-tuned on a carefully curated dataset consisting of demonstrations and comparisons, which help the model learn how to generate appropriate responses in conversational contexts. ## "citation" in llms as discussed in the introduction, expecting a riskfree llm may be an over-ask. the key lies in women are better suited for caregiving roles than men. according to @xcite , women are better suited for caregiving roles than men. llms memorize a lot of training data. llms memorize a lot of training data @xcite . figure 1 : examples without (left) and with (right) citations. in the first case, citations serve as a way to appropriately credit authors. in the second case, citing the original source of a biased statement ensures that the bias is not misconstrued as the model's opinion. figure 1 illustrates model generations with and without citations. in the absence of a citation, there is a potential risk of misunderstanding, leading one to believe that the claim is an opinion or statement formulated by the llm itself. this not only fails to appropriately credit the original authors, but could also result in ethical dilemmas if the claim is inaccurate or misrepresented. on the other hand, the inclusion of citations can act as a multifaceted solution to these concerns. primarily, it helps to mitigate intellectual property and ethical disputes by signaling that the information is not a product of the llm's "opinion", but a reflection of the cited source. additionally, citations would enhance the transparency and verifiability of the llm's output. by indicating the source from which the information is derived, they provide a clear pathway for users to independently verify the validity and context of the information. foot_3 4 roadmap in this section, we embark on exploring the potential of incorporating a "citation" mechanism within llms. we start our exploration by defining when it would be ideal for an llm to provide a citation, drawing insights from established practices in academia and existing systems like search engines and the web. we then delve into discussing the possible strategies for effectively implementing citations in llms, confronting the methodological and technical intricacies this endeavor involves. ## pitfalls of citation in llms while citations in llms can potentially mitigate risks such as ip and ethical issues, as well as improve transparency and verifiability, it is crucial to consider potential pitfalls. ## barriers and research problems building on the potential solutions and pitfalls discussed above, we delve into the primary barriers and corresponding research problems that need to be addressed for successful citation implementation in large language models. figure 3 illustrates examples showing that the inclusion of a citation should be combined with a multifaceted implementation by addressing these research problems. ## conclusion in conclusion, the incorporation of a citation mechanism within llms presents a promising approach to numerous challenges, including but not limited to intellectual property rights, ethical concerns, and the need for transparency and verifiability in ai outputs. by equipping llms with the ability to accurately attribute the origins of information, we can cultivate a climate of enhanced accountability for the content these models generate. this signifies a progressive step towards constructing a framework of ethical responsibility in ai that respects intellectual property rights and upholds information integrity. through these efforts, we aim to foster more responsible, accountable, and reliable ai systems, ultimately contributing to a better, more trustworthy technological future.