Language Models that Seek for Knowledge: Modular Search & Generation for Dialogue and Prompt Completion (2022)

Abstract: Language models (LMs) have recently been shown to generate more factual responses by employing modularity (Zhou et al., 2022) in combination with retrieval (Adolphs et al., 2021). We extend the recent approach of Adolphs et al. (2021) to include internet search as a module. Our SeeKeR (Search engine->Knowledge->Response) method thus applies a single LM to three modular tasks in succession: search, generating knowledge, and generating a final response. We show that, when using SeeKeR as a dialogue model, it outperforms the state-of-the-art model BlenderBot 2 (Chen et al., 2021) on open-domain knowledge-grounded conversations for the same number of parameters, in terms of consistency, knowledge and per-turn engagingness. SeeKeR applied to topical prompt completions as a standard language model outperforms GPT2 (Radford et al., 2019) and GPT3 (Brown et al., 2020) in terms of factuality and topicality, despite GPT3 being a vastly larger model. Our code and models are made publicly available.

## introduction standard large language models are known to generate fluent but factually incorrect statements, a problem that is not solved by just increasing their size @xcite . additionally, as their knowledge is frozen in time from the point when they were trained, they can never learn new factsthe newest information they have will be from the date that the training set was constructed. several recent advances have tried to tackle aspects of these problems. neural retrieval models have augmented seq2seq models with access to a large fixed corpus of knowledge @xcite . however, aggregating information from multiple retrieved documents is a difficult problem @xcite which may result in search: beyonce 2022 search engine knowledge: according to sony music's ceo, the star will be releasing her album in the first quarter of 2022. incorporating parts of multiple documents into one factually incorrect response. a modular approach which first finds the relevant parts of the documents and then generates the final response has been shown to help alleviate this problem @xcite . however, none of those methods can incorporate new information, which has been studied in separate work that augments generations with internet search @xcite . in this paper, we explore a modular architecture that tries to mix the best elements of these different existing solutions. a single transformer architecture is used iteratively to perform three modular tasks: search, generate knowledge, and generate a final response, where the output of each module is fed as additional input to the next, as in figure 1 . the first step, given the input context, generates a relevant search query for an internet search engine, while the second step is fed the returned documents and generates their most relevant portion. the last step uses that knowledge to produce its final response. by decomposing this difficult problem into three manageable steps, pertinent up-to-date information can be incorporated into the final language model generation. we apply our modular search-@xmath0, after pre-training and fine-tuning on a variety of knowledge-intensive datasets. in open-domain dialogue, we show this approach outperforms the state-of-the-art blenderbot 2 model of @xcite according to human ratings of consistency, knowledge and per-turn engagingness. we test the ability of seeker to perform general -but up-to-date -language modeling. to do this we construct topical prompts on subjects that were in the news in january 2022, which is data that the model itself has not been trained on. with seeker's ability to incorporate information via web search, it outperforms gpt2 @xcite and gpt3 @xcite in terms of factuality and topicality according to human raters. ## related work our work builds on the knowledge to response (k2r) technique @xcite which decomposes a dialogue model into two stages: generating a knowledge sequence, followed by generating a response sequence, conditioned on the knowledge. this was applied successfully to wizard of wikipedia @xcite , qa @xcite and light tasks @xcite . we expand on this approach by adding the additional module of internet search and then applying that to full open-domain dialogue and general language modeling. in the dialogue space, the most natural comparison to our approach is blenderbot 2 (bb2) @xcite . bb2 grounds on retrieval from the internet for open-domain dialogue tasks @xcite , but does not use a modular approach to generate knowledge, instead applying the fusion-indecoder (fid) method @xcite to output a response directly given the retrieved documents. they, as well as others @xcite , report that their method can have the prob-lems of either mixing up facts together incorrectly or generating a generic response that ignores the knowledge, which our method attempts to address. another recent approach that uses information retrieval is lamda @xcite , where the retrieval engine returns pertinent information (rather than a set of documents) and is considered a separate black box. lamda is not openly available and cannot be compared to. webgpt @xcite also applies internet search to qa tasks, as does the work of @xcite ; neither applies to dialogue or general lm tasks, and neither work is openly available. in the language modeling space, there is a large body of work on nearest neighbor and cache-based language modeling @xcite @xcite @xcite for accessing a large set of documents. recently, retro @xcite used retrieval over a database of trillions of tokens. those works do not use internet search, but rather perform their own retrieval method via a transformer model together with nearest neighbor lookup. as the database is fixed, that means it would not be up to date with the latest knowledge and current events. some recent methods have also attempted to adapt knowledge through editing and tuning of language model variants @xcite . ## seeker model the seeker model we introduce in this paper has the architecture of a standard transformer @xcite , except that this same encoder-decoder (for dialogue) or decoder-only (for language modeling) model is used in a modular way multiple times. for each module, special tokens are used in the encoder (or decoder) to indicate which module is being invoked. the output of each module is input into the next, along with the original context. seeker consists of three modules, which are invoked sequentially: search module given the encoded input context, a search query is generated. this is fed into a search engine, which returns results in the form of a set of documents. following @xcite , in our experiments (unless stated otherwise) we employ the bing web search api 2 to retrieve documents, and then filter that set of documents by intersecting with common crawl @xcite , and keep the top 5. knowledge module given the encoded input context, and a set of retrieved documents, a knowledge response is generated. this consists of one or more relevant phrases or sentences from the retrieved documents. for encoder-decoder models, the documents and context are encoded using the fusion-in-decoder (fid) method @xcite ; for decoder-only models, we pack and prepend the documents to the input context. this task is essentially a "copy" task in that no new tokens have to be generated; the difficulty of the task is selecting the relevant knowledge to copy. response module given the encoded input context concatenated with the knowledge response, the final response is generated. the module must consider relevant context and knowledge while generating a new fluent continuation to the input. the extraction of relevant knowledge by the previous modules makes this task easier; in contrast, a conventional seq2seq model has to solve all these tasks (knowledge acquisition, synthesis, and final response generation) at once. ## experiments full training details (including hyperparameters) and automatic metrics are given in the appendix. ## conclusion we have presented a modular system for searching for and choosing knowledge during language model generation. our approach outperforms the state of the art on dialogue modeling, and is shown to outperform both gpt2 with the same architecture on topical prompts -even when using a smaller parameter size -and gpt3 -despite being vastly (500x) smaller. our approach of explicitly splitting into three modules allows for engineering better modules in the future, e.g. fine-tuning parts of the model. we make our code and models publicly available for further research.