Structure-aware Knowledge Graph-to-text Generation with Planning Selection and Similarity Distinction (2023)

Abstract: The knowledge graph-to-text (KG-to-text) generation task aims to synthesize coherent and engaging sentences that accurately convey the complex information derived from an input knowledge graph. One of the primary challenges in this task is bridging the gap between the diverse structures of the KG and the target text, while preserving the details of the input KG. To address this, we propose a novel approach that efficiently integrates graph structure-aware modules with pre-trained language models. Unlike conventional techniques, which only consider direct connections between first-order neighbors, our method delves deeper by incorporating Relative Distance Encoding as a bias within the graph structure-aware module. This enables our model to better capture the intricate topology information present in the KG. To further elevate the fidelity of the generated text, Planning Selection and Similarity Distinction are introduced. Our approach filters the most relevant linearized sequences by employing a planning scorer, while simultaneously distinguishing similar input KGs through contrastive learning techniques. Experiments on two datasets demonstrate the superiority of our model.

## introduction knowledge graph (kg) is a structured data representation form that contains rich knowledge information and is more convenient for processes such as information retrieval and reasoning. although kgs facilitate computational processes, it is difficult for humans to intuitively understand the content in kgs, so the proposed kg-to-text generation task aims to produce correct descriptive text for the input kg. kg-to-text has various applications, like question-and-answer @xcite and dialogue systems @xcite . figure 1 shows a kg and its corresponding text. the main adare manor 1862 james pain 1700 completion date description text: james pain was the architect of the adare manor building which construction started in 1700 and completed in 1862 . linearized sequence: <@xmath0 figure 1 : the kg and its corresponding text. the entities in the text are marked in color. the order of triples in the linearized sequence can control the accuracy of the generated text. benefitting from the powerful capabilities of pre-trained language models (plms), methods based on plms @xcite are currently achieving outstanding results on kg-to-text generation tasks. since there is a structural gap between the graph structure of kgs and the sequence input requirements demanded by plms, an important step is to linearize the given kg into an input sequence. the results of many previous works have shown that the final generated texts are significantly influenced by the triple orders of linearized sequences. many existing works typically use predefined heuristic rules such as breadth-first search (bfs) @xcite , or create a content planner using multistep prediction methods to predict of the triple order @xcite . each step of these multistep prediction processes may have prediction bias, which results in the final sequence or-der easily deviating significantly from the groundtruth order. considering this problem, we train an efficient planning scorer to score the generated results of all linearized sequences corresponding to kg, and pick the sequence with the highest score to be the final input, thus enabling the bias caused by multistep predictions to be avoided. the linearized sequence may ignore the graph structure information in the corresponding kg, such as the connectivity of nodes and the direction of edges. some works @xcite have improved plms by adding graph structure-aware modules to every layer of the transformer encoder. these graph-aware modules only consider the directly connected entities in one triple, i.e., the first-order neighbors in the kg. during the specific text generation process, only the information within each triple is considered, while the connections of the entities between triples are ignored. therefore, to fully utilize the topological structure information of a kg, we consider the edges in the kg as nodes to compute the shortest path distance among nodes and add it to the weight computation as a bias. additionally, to enable better distinction of similar input kgs and similar entities in the same kg, we use contrastive learning to enable the transformer encoder to differentiate the input sequences at a finergrained level. we use two datasets to test our model: webnlg and dart, and it outperforms the previous models. our contributions are threefold: ## approaches our method not only ensures that the generated text and the target text have the same information, but also enables the model to capture both topological information and specific node content information of the graph for the kg-to-text generation task. to accomplish the first goal, we train a planning scorer that can select the best linearized sequence by scoring all possible input kgs. for the second goal, we first add the graph structure-aware module to the bart encoder. then, we calculate the cosine similarity of the sequence embeddings from the last layer of the encoder to generate a new loss. figure 2 represents our model architecture and the following sections describe the contents of these modules. ## conclusion the paper presents a two-step training model for kg-to-text generation including planning scorer and text generation. in the first step the planning scorer first obtains the linearized kg order with the highest score. then the second step uses a graph structure-aware module with relative distance encoding and kg distinction by contrastive learning to ensure that the model captures the topological information and specific entity features in the input kg. additional experiments and qualitative analyses indicate our model outperforms the existing kg-to-text approaches on two datasets.