Generative Knowledge Graph Construction: A Review (2022)

Abstract: Generative Knowledge Graph Construction (KGC) refers to those methods that leverage the sequence-to-sequence framework for building knowledge graphs, which is flexible and can be adapted to widespread tasks. In this study, we summarize the recent compelling progress in generative knowledge graph construction. We present the advantages and weaknesses of each paradigm in terms of different generation targets and provide theoretical insight and empirical analysis. Based on the review, we suggest promising research directions for the future. Our contributions are threefold: (1) We present a detailed, complete taxonomy for the generative KGC methods; (2) We provide a theoretical and empirical analysis of the generative KGC methods; (3) We propose several research directions that can be developed in the future.

## introduction knowledge graphs (kgs) as a form of structured knowledge have drawn significant attention from academia and the industry @xcite . however, high-quality kgs rely almost exclusively on human-curated structured or semi-structured data. to this end, knowledge graph construction (kgc) is proposed, which is the process of populating (or building from scratch) a kg with new knowledge elements (e.g., entities, relations, events). conventionally, kgc is solved by employing task-specific discriminators for the various types of information in a pipeline manner @xcite @xcite , typically including (1) entity discovery or named entity recognition @xcite , (2) entity linking @xcite , (3) relation extraction @xcite and ( 4 ) event extraction @xcite . however, this presents limitations of error population and poor adaptability for different tasks. ## taxonomy of generative knowledge graph construction in this paper, we mainly consider the following five paradigms that are widely used in kgc tasks based on generation target, i.e. copy-based sequence, structure-linearized sequence, label-augmented sequence, indice-based sequence, and blank-based sequence. as shown in figure 2 , these paradigms have demonstrated strong dominance in many mainstream kgc tasks. in the following sections, we introduce each paradigm as shown in figure 3 . ## future directions though lots of technical solutions have been proposed for generative kgc as surveyed, there remain some potential directions: generation architecture. most of the recent generative kgc frameworks face serious homogenization with transformer. for enhancing interpretability, we argue that neuro-symbolic models (i.e., a reasoning system that integrates neural and symbolic) @xcite @xcite can be designed for generative kgc. in addition, some cutting-edge technologies such as spiking neural network @xcite , dynamic neural networks (xu and mcauley, 2022), ordinary differential equations @xcite and diffusion models @xcite can also provide promising architectures. ## conclusion and vision in this paper, we provide an overview of generative kgc with new taxonomy, theoretical insight and empirical analysis, and several research directions. note that the generative paradigm for kgc has the potential advantages of unifying different tasks and better utilizing semantic information. in the future, we envision a more potent synergy between the methodologies from the nlg and knowledge graph communities. we hope sophisticated and efficient text generation models to be increasingly contributed to improving the kgc performance. on the converse, we expect symbolic structure in kg can have potential guidance for text generation.