I nternet-Augmented Dialogue Generation (2022)

Abstract: The largest store of continually updating knowledge on our planet can be accessed via internet search. In this work we study giving access to this information to conversational agents. Large language models, even though they store an impressive amount of knowledge within their weights, are known to hallucinate facts when generating dialogue (Shuster et al., 2021); moreover, those facts are frozen in time at the point of model training. In contrast, we propose an approach that learns to generate an internet search query based on the context, and then conditions on the search results to finally generate a response, a method that can employ up-to-the-minute relevant information. We train and evaluate such models on a newly collected dataset of human-human conversations whereby one of the speakers is given access to internet search during knowledgedriven discussions in order to ground their responses. We find that search-query based access of the internet in conversation provides superior performance compared to existing approaches that either use no augmentation or FAISS-based retrieval (Lewis et al., 2020b).

## introduction open-domain dialogue, which involves chat about any topic, rather than a specific goal-directed topic, is commonly studied by training large language models @xcite @xcite . these models are trained either in a encoder-decoder or decoder only setting on large datasets of human-human conversations, and any knowledge obtained during training is stored in the weights of the model. such static language modeling fails to take into account the dynamic state of the world, where new information is coming in by the day -or even by the minuteas the knowledge in static models is gleaned from the point in time when the dataset was collected, and then frozen into the model that is trained; see ( @xcite for criticisms of this approach. further, static language models are known to hallucinate, that is they generate plausible looking statements that are factually incorrect, which can be interpreted as a form of lossy compression when employing training to encode that knowledge within the weights of a neural network; see @xcite for an in-depth study. in this work we study generative models that are instead capable of accessing the vast knowledge of the internet dynamically in order to inform their responses. utilizing encoder-decoder archi-tectures, we consider models that, given a dialogue context, first generate a search query. the queries are then used to retrieve relevant knowledge that is prepended to the conversational history, which is encoded using the fusion-in-decoder method @xcite . taking into account this encoded knowledge, a response is finally generated using the decoder. this ability to access the internet means the model is always up-to-date, unlike existing models that only know about facts in their fixed training set. our model, in contrast, can potentially make use of the latest sports scores, movies or tv shows that were just released, the latest reviews, and so forth -amongst the countless other topics available on the internet. in order to train and evaluate such models, we collect a new crowdsourced english dataset involving human-human conversations, where one of the workers plays the role of a "wizard" who conducts internet searches in order to inform their responses during knowledge-grounded conversations. we show that internet-augmented models trained to replace the human wizard outperform conventional non-augmented generation models on this task as measured by automatic metrics as well as human evaluations, and with our search query generation based approach also outperforms existing retrievalaugmented faiss-based approaches such as rag @xcite and fid-rag @xcite . we make our final models and the new task we have collected, publicly available. 1 . ## related work the majority of work on dialogue generation has focused on training on natural or crowdsourced data where the task is, given a dialogue context (history), to generate the next response. datasets such as pushshift.io reddit @xcite , personachat @xcite or empathetic dialogues @xcite (see @xcite for a review) are typically employed to train the weights of a transformer encoder-decoder. this is the standard approach in state-of-the-art chatbots such as meena @xcite or blenderbot @xcite . such models do not augment their generations with access to external knowledge, instead relying on facts originally provided in the training datasets themselves being stored into the weights of the model. a growing area of research is that of augment-1 http://parl.ai/projects/sea ing generative models with external knowledge. earlier works such as memory networks @xcite and drqa @xcite utilized tfidf-based retrieval over documents to provide additional input to neural models for the task of question answering, following the well studied area of non-neural methods that use retrieval for qa @xcite . more recently, the rag (retrieval-augmented generation) @xcite and fid (fusion-in-decoder) (izacard and grave, 2021) models developed these ideas further, using a neural retriever as well, with superior results. retrieval-augmentation is also studied in the area of language modeling, where it is used for pre-training @xcite , and as a memory @xcite , especially using k-nearest neighbor-based cache models @xcite @xcite . in dialogue, knowledge grounding is becoming more popular an area, with several datasets developed to study it @xcite @xcite @xcite . some of these such as topical-chat @xcite and cmu_dog @xcite are constructed given a gold passage of knowledge, and the task analyzes whether the model can use this knowledge in dialogue. other works @xcite @xcite study whether knowledge selection is possible from a (small) set of knowledge. however, a retrieval step (or search engine) is not used, as we consider here. perhaps the closest to our work is the wizard of wikipedia task @xcite which involves conversations grounded in wikipedia, using a tfidf retrieval model to find relevant knowledge from that database. our work can be seen as a much richer task, covering all of the information that is publicly available on the internet and hence a more diverse range of conversational topics rather than just wikipedia, while allowing human wizards to search for relevant knowledge themselves. moreover, we consider sophisticated neural-in-theloop retrieval mechanisms and real search engines. @xcite studied neural-retriever-in-theloop methods on this dataset. ## internet-augmented generation we consider two ways to access the webpages from the internet: (i) using a cached set of pages that are stored in a distributed approximate nearest-neighbor database, faiss @xcite , or (ii) using an internet search engine directly to retrieve pages. for the faiss-based methods, there are a number of possible variants that we consider, which we will describe first. ## wizard of the internet task in order to both train and evaluate generative models that can use search engines in-the-loop, we design, collect and release a dataset for this purpose. the overall setup involves pairing crowdworkers that are instructed to have a conversation together. one plays the role of the wizard, who has access to a search engine during conversation, while the other, the apprentice, does not. the apprentice however has an assigned persona that describes their interests. the purpose of the exchange is to have an "in-depth conversation about [those] assigned interests". this mirrors conversations we expect to be more prevalent between a human and a bot: the conversations are more likely to be centered around the human's interests than the bot's, and the bot is the one that is going to be using the search engine to ground their knowledge. hence, when we train or evaluate on this task, a given model will replace the role of the wizard. apprentice persona we show the apprentice several possible persona choices for the character that they are going to play, and let them choose one, e.g. "i love tennis, rafael nadal is my favorite player.". the intent here is that they can choose a topic they are both more interested in themselves to talk about and also have enough knowledge of so that they can conduct a reasonable conversation. the choices we show are themselves mined from the interests provided in the existing persona-chat dataset @xcite and the topics given in the existing topical-chat dataset @xcite . more details of the choices we give are provided in appendix a. ## qualitative analysis success cases in the best case, our augmented models are able to construct appropriate internet search queries, read the corresponding web pages and provide information relevant to the conversation. we show a cherry picked conversations between a human (paper author) and the wizinternet search engine fid model (using live bing search) in figure 1 , and in figure 8 , figure 9 , figure 10 and figure 11 in the appendix. in each case, we can compare to a wizint bart-large transformer (noknowledge) model using the same conversational messages on the human side. we find the search engine model is capable of diverse conversations spanning drink ingredients, tv shows, restaurants and machine learning research. in the tv show and restaurant cases the model is able to surface recommendations and provide details about them, for example the correct address and phone number of a pizza store in princeton, or the plots of recent tv shows such as the underground railroad. standard bart-large fine-tuned models on the other hand typically either hallucinate knowledge or else fall back to generic statements. failure cases analysis also exposes various kinds of error. lemon picked conversations between human (paper authors) and the wizinternet search engine fid model (using live bing search) are shown in figure 12 in the appendix. first, there are generation mistakes despite finding the correct knowledge, for the example where the model incorrectly names bruno mars as working on the cardi b song bodak yellow. bruno mars did collaborate with cardi b on other songs, and the model confuses and mixes various pieces of evidence within the given knowledge sources. second, search query generation mistakes given the context, for example missing out key search terms. third, selecting the wrong knowledge given earlier context, as in the case where the model associates the wrong authors to a paper. a fourth additional issue is that even if the correct knowledge is available the model may err on the side of not using it and select a more generic response instead, as often happens in the non-augmented model. see for example figure 8 and figure 11 in the appendix. ## conclusions this work has studied the problem of siloed knowledge in large language models, whereby they cannot access the knowledge of the world other than through their fixed training set. developing methods that instead can access the internet as an augmentation to the generation process, we have showed such models can display more knowledge and generate less factually incorrect information during dialogue with humans. future work should aim to develop improved architectures that can be trained and evaluated on our new task. going forward, in the long-term we require machine learning methods that interact with the world, rather than only having a simple text context -and access to the internet is a natural step in that direction.