Generative Subgraph Retrieval for Knowledge Graph–Grounded Dialog Generation (2024)

Abstract: Knowledge graph–grounded dialog generation requires retrieving a dialog-relevant subgraph from the given knowledge base graph and integrating it with the dialog history. Previous works typically represent the graph using an external encoder, such as graph neural networks, and retrieve relevant triplets based on the similarity between single-vector representations of triplets and the dialog history. However, these external encoders fail to leverage the rich knowledge of pretrained language models, and the retrieval process is also suboptimal due to the information bottleneck caused by the single-vector abstraction of the dialog history. In this work, we propose Dialog generation with Generative Subgraph Retrieval (DialogGSR), which retrieves relevant knowledge subgraphs by directly generating their token sequences on top of language models. For effective generative subgraph retrieval, we introduce two key methods: (i) structure-aware knowledge graph linearization with self-supervised graph-specific tokens and (ii) graph-constrained decoding utilizing graph structural proximity-based entity informativeness scores for valid and relevant generative retrieval. DialogGSR achieves state-of-the-art performance in knowledge graph–grounded dialog generation, as demonstrated on OpenDialKG and KOMODIS datasets.

## introduction the goal of dialog generation is to generate an informative and appropriate response given an input dialog. pretrained language models (plms) have demonstrated promising performance on the dialog generation @xcite @xcite . however, they often generate irrelevant, factually incorrect, or hallucinatory responses since the generation process heavily depends on the internal parameters of the language models @xcite . to mitigate these issues, several studies @xcite have explored knowledgegrounded dialog generation models, which incorporate external knowledge to generate more factually accurate responses. some approaches utilize unstructured texts such as wikipedia articles @xcite and internet web pages @xcite while others @xcite @xcite leverage structured knowledge graphs (kgs) to capture both the relational and semantic information for grounding dialog responses. many existing knowledge graph-grounded dialog generation models @xcite employ encoder-based retrieval methods. they encode the dialog history into a single vector and then use it on another encoder (e.g., biencoder) to retrieve relevant triplets from the kg. however, this approach can lead to an information bottleneck due to the limited capacity of a single vector to represent long and complex multi-turn dialogs @xcite @xcite . moreover, these methods @xcite @xcite often rely on separate models, such as graph neural networks (gnns), to encode the knowledge graphs, which limits the integration of natural language comprehension capabilities of plms. recent studies @xcite ) have addressed the information bottleneck issue by applying generative retrieval methods, which cast retrieval as an autoregressive generation process to facilitate direct interactions between query context and knowledge paragraphs. despite this progress, most generative retrieval works focus solely on natural language-based knowledge, employing conventional token representations and decoding strategies, which do not fully capture the structure and properties of knowledge graphs. to address the aforementioned issues, we propose dialog generation model with generative subgraph retrieval (dialoggsr), which integrates generative subgraph retrieval with response generation. our proposed method adopts two key graph-specialized techniques: (1) a structure-aware knowledge graph linearization for effective graph representation and (2) graph-constrained decoding for valid subgraph retrieval. our knowledge graph linearization approach introduces a small set of special token embeddings to account for both the structural positioning of knowledge entities and the reverse relationships between them. by selfsupervising these special tokens using a knowledge graph reconstruction loss, the method effectively represents the knowledge graph. the graphconstrained decoding facilitates autoregressively retrieving the knowledge considering the graph structural information, thus generating valid and relevant knowledge subgraphs. since dialoggsr utilizes pretrained language models for both subgraph retrieval and dialog generation, it leverages the pretrained language models' internal knowledge in both tasks. we evaluate dialoggsr on two kg-grounded dialog generation datasets: opendialkg @xcite and komodis @xcite . our proposed method shows the best performance on both benchmark datasets. our contributions are three-fold as follows: ## methods we propose a retrieval-augmented dialog generation approach that retrieves contextually relevant subgraphs from knowledge graphs to generate better responses. our model, dialog generation model with generative subgraph retrieval (di-aloggsr) consists of a generative subgraph retriever and a response generator. we first define the task of knowledge graph-grounded dialog generation (sec. 3.1). next, we propose generative subgraph retrieval (gsr), which autoregressively retrieves subgraph by applying structure-aware knowledge graph linearization and graph-constrained decdoing (sec. 3.2). we then present a response generator, which performs subgraph-grounded dialog generation (sec 3.3). finally, we provide the training details of dialoggsr including our self-supervised knowledge graph reconstruction loss (sec 3.4). the inference process of dialoggsr is illustrated in figure 1 . ## experiments in this section, we evaluate the effectiveness of the proposed dialoggsr on knowledge graph-grounded dialog generation. we first introduce the two datasets (opendialkg @xcite and komodis @xcite ), and the experimental setup and metrics. then, we demonstrate the effectiveness of dialoggsr on the two benchmark datasets. lastly, we provide ablation studies, and analyses of our dialoggsr. ## conclusion we have presented dialoggsr, a dialog generation model with generative subgraph retrieval. di-aloggsr retrieves context-relevant subgraphs, by generating the subgraph token sequences considering both the dialog context and the graph information. we have proposed novel knowledge graph linearization to convert knowledge triplets into token sequences with self-supervised graph-specific tokens to represent knowledge graphs without separate knowledge graph modules. in addition, we have formulated a graph-constrained decoding for valid and relevant generative retrieval. our experiments demonstrate the effectiveness of our proposed method in knowledge-graph grounded dialog generation. our codes are publicly available at https://github.com/mlvlab/dialoggsr .