Retrieval Augmentation Reduces Hallucination in Conversation (2021)

Abstract: Despite showing increasingly human-like conversational abilities, state-of-the-art dialogue models often suffer from factual incorrectness and hallucination of knowledge (Roller et al., 2020). In this work we explore the use of neural-retrieval-in-the-loop architectures - recently shown to be effective in open-domain QA (Lewis et al., 2020b; Izacard and Grave, 2020) - for knowledge-grounded dialogue, a task that is arguably more challenging as it requires querying based on complex multi-turn dialogue context and generating conversationally coherent responses. We study various types of architectures with multiple components - retrievers, rankers, and encoder-decoders - with the goal of maximizing knowledgeability while retaining conversational ability. We demonstrate that our best models obtain state-of-the-art performance on two knowledge-grounded conversational tasks. The models exhibit open-domain conversational capabilities, generalize effectively to scenarios not within the training data, and, as verified by human evaluations, substantially reduce the well-known problem of knowledge hallucination in state-of-the-art chatbots.

## introduction large language models trained on large corpora have made great inroads in the fluency and conversational ability of dialogue agents @xcite , yielding low perplexity models that have corresponding high token accuracies on in-domain test sets. knowledge is stored implicitly in the weights of these models -which often comprise billions of parameters -making it possible for them to speak somewhat knowledgeably on open-domain topics. unfortunately, even the largest models suffer from the well known "hallucination" problem @xcite where they generate plausible looking statements that are factually incorrect. they often mix up facts between two similar entities, or make errors where just one token being incorrect is the difference between being right and wrong. see figure 1 for an example using gpt3, a 175b parameter language model @xcite . a recently introduced technique for question answering is the neural-retrieval-in-the-loop approach of retrieval-augmented generation (rag) @xcite , which has proven effective for correctly answering open-domain questions. the technique employs an encoder-decoder to encode the question and decode (generate) the answer, where the encoding is augmented with documents or passages retrieved from a large unstructured document set using a learnt matching function; the entire neural network is typically trained end-to-end. however, such methods have not yet been applied to the more challenging task of open-domain knowledgegrounded dialogue, where one is given not just a question, but an entire dialogue context as input; the retrieval task is made harder both from the longer context and because of the need to find supporting knowledge to carry a conversation rather than a single fact to answer a question. such models must provide both conversational ability when generating their response, as well as knowledgeabil-ity and factuality. therefore, existing approaches may not serve well out of the box. in this work, we study the various components of retrieval-augmented neural architectures for dialogue -retrievers, rankers and encoder-decodersand propose several new variants, while analyzing which methods work well and in which situations they do so. in particular, we improve downstream performance by employing poly-encoder transformers @xcite for finer-grained context-candidate scoring of documents, by employing end-to-end-trained retrievers in the fusionin-decoder @xcite technique, and by building a dialogue turn-based retrieval mechanism that avoids the problem of standard retrievers that ignore much of the dialogue context. our best models provide state-of-the-art results on two knowledge-grounded conversational tasks, wizard of wikipedia @xcite and cmu document grounded conversations (cmu_dog) @xcite . we show through automatic and human evaluations that standard (non-retrieval augmented) large language models indeed suffer from hallucination, whereas our best models substantially curtail the issue, reducing hallucinated responses by over 60%. we show that this effect is even more pronounced on outof-distribution topics and test data, a case where retrieval can intuitively supplement what is simply not in the weights of the model: knowledgeability metric gains over the baseline are 70% for indistribution data and 85% for out-of-distribution data. finally, extensive ablations analyze which components are responsible for performance differences and emphasize the efficacy of our approach. ## related work hallucination in text-generation models is a topic that has received attention recently, particularly in the settings of summarization @xcite , machine translation @xcite , and news generation @xcite . for dialogue, it has been observed in state-of-the-art models @xcite and studied in depth @xcite , but so far without resolution. open-domain question answering (qa) has long considered retrieval as an intermediate step @xcite . it has become a more intensively studied topic recently, first using simple vector-space based retrievers @xcite , and later with end-to-end generation models where the retrieval component is a neural network as well @xcite . these recent neural approaches over unstructured text have overtaken prior methods exploiting the graph structure of knowledge sources (such as hyperlinks in wikipedia) @xcite @xcite , and are an attractive alternative for dialogue. knowledge-grounded dialogue is increasingly becoming an important topic, with several datasets proposed that attempt to model its occurrence @xcite @xcite . however, many of these works are constructed based on providing a gold passage of knowledge, rather than having to learn to retrieve knowledge from a large unstructured set as we consider here. recent methods have focused on: determining which elements of a given piece of knowledge are informative to the dialogue, which is commonly referred to as "knowledge selection" @xcite @xcite ; learning how to attend to the relevant knowledge @xcite @xcite ; or examining how much knowledge is present in large language models @xcite . some recent work has explored retrieval-based mechanisms, however the retrieval over knowledge is generally limited to a small subset of the overall corpus considered @xcite @xcite . incorporating unstructured textual knowledge is generally limited to selecting from fixed documents, small document sets or else simple vector-space models @xcite . we note that very recently retrieval augmented generation has been applied to task-oriented dialogue @xcite , which is in contrast to the open-domain knowledge-grounded dialogue setting we consider here. other work that includes a retrieval-augmentation step includes the area of language modeling, where it is used for pretraining @xcite , and as a memory @xcite , especially using k-nearest neighbor-based cache models @xcite @xcite . ## model architectures we extend neural-retriever-in-the-loop generativebased architectures, which have performed well in open-domain qa, to knowledge-grounded tasks, where model responses must not only be knowl-edgeable but also consistent and engaging both across long-form generation and throughout multiple turns of conversation. to keep notation consistent, we let x @xmath0, ..., x n i } represent the tokens for dialogue context i, and define y i similarly for the ground truth response; z @xmath1,1 , ..., z i,k } is the set of k documents retrieved. q(x i ) and d(z j ) are representations of the dialogue context and candidate document respectively in the retrieval mechanism, where p η (z j |x i ) is the probability of selecting a document z j given a context ## experiments datasets: we conduct experiments on two datasets: wizard of wikipedia (wow) @xcite and cmu document grounded conversations (cmu_dog) @xcite which are both sets of knowledge-grounded dialogues collected through human-human crowdworker chats in english, where one of the crowdworkers had access to external knowledge from wikipedia; wow discusses various topics, and cmu_dog discusses movies. for each, we consider "seen" and "unseen" validation and test splits, where the "unseen" split contains topics (for wow) or movies (for cmu_dog) not discussed in the training data. wow provides these splits, and we constructed our own for cmu_dog. we employ the standard kilt wikipedia dump @xcite as our knowledge source for retrieval for both datasets foot_0 . more dataset details are in appendix c. ## discussion we have thus far explored several ways of retrieving and conditioning on documents in knowledgegrounded dialogue; here, we summarize some key takeaways from our results. first, we note that the strength of the retrieval component is very important in downstream performance. our dpr-poly setup obtains the best retrieval metrics on wow ( second, we note that models that condition on several documents simultaneously result in more engaging conversationalists; rag-token, rag-turn, and fid-rag yield higher f1 scores ( finally, we note that standard metrics used for open-domain dialogue are not sufficient for truly capturing hallucination within models; thus, metrics such as knowledge f1 are required to further study model performance -figure 2 in the appendix highlights correlations between such automated metrics and human evaluations. ## conclusion in this work, we have studied the problem of knowledge hallucination in conversational agents, an important problem as current systems often produce factually inaccurate generations. we have shown that this problem occurs independently of language model size or training data. retrieval-augmented generation in particular is an intuitively promising solution to this problem, and in detailed experiments we have shown that this class of approaches significantly reduces the hallucination problem in dialogue while maintaing conversational ability, and can help generalize beyond the training data on previously unseen distributions. future work should look for improved methods and to find solutions to unanswered questions, such as understanding the interplay between retrieved knowledge and knowledge stored in the model's weights.