Learning Reasoning Patterns for Relational Triple Extraction with Mutual Generation of Text and Graph (2022)

Abstract: Relational triple extraction is a critical task for constructing knowledge graphs. Existing methods focused on learning text patterns from explicit relational mentions. However, they usually suffered from ignoring relational reasoning patterns, thus failed to extract the implicitly implied triples. Fortunately, the graph structure of a sentence’s relational triples can help find multi-hop reasoning paths. Moreover, the type inference logic through the paths can be captured with the sentence’s supplementary relational expressions that represent the real-world conceptual meanings of the paths’ composite relations. In this paper, we propose a unified framework to learn the relational reasoning patterns for this task. To identify multi-hop reasoning paths, we construct a relational graph from the sentence (text-to-graph generation) and apply multi-layer graph convolutions to it. To capture the relation type inference logic of the paths, we propose to understand the unlabeled conceptual expressions by reconstructing the sentence from the relational graph (graph-to-text generation) in a self-supervised manner. Experimental results on several benchmark datasets demonstrate the effectiveness of our method.

## introduction relational triple extraction is defined as automatically recognizing semantic relations with triple structures (subject, relation, object) among multiple entities in a sentence. it is a critical task for natural language processing, especially for knowledge graph (kg) construction from unlabeled corpus @xcite . recent work proposed several neural network methods to extract relational triples. for example, @xcite proposed a sequence tagging scheme for this task but failed to extract overlapping triples. @xcite proposed to solve the overlapping triple problem with a binary tagging framework. @xcite proposed to address this issue by generating triple element sequences with copy mechanism. existing methods achieved considerable success in learning text patterns of relational triples from explicit mentions. however, they usually suffered from the failure of extracting the relational triples which are implicitly implied in the text @xcite . this is because they ignored relational reasoning patterns in natural language, which usually consist of finding multi-hop paths and inferring relation types along these paths. for example, in figure 1 , the triple ("david", "father", "judy") is not explicitly expressed in the sentence and requires relational reasoning to be extracted. unfortunately, the ignorance of relational reasoning patterns in existing methods will cause serious incompleteness of the constructed kgs and performance degradation of downstream tasks @xcite . our work is motivated by several observations. first, the relational triples of a sentence usually have a graph structure, which is useful for finding multi-hop reasoning paths. for example, in figure 1 , the relational graph provides a two-hop reasoning path between "david" and "george". second, the sentence usually contains supplementary relational expressions that represent the real-world conceptual meanings of the paths' composite relations, which can help capture the relation type inference logic through the paths. for example, in figure 1 , the phrase "'s grandfather" helps capture the equivalence between the composite relation "father(father(•))" and the real-world relational concept "grandfather", which reflects the relation type inference logic of the two-hop path. in this paper, we propose a unified framework to learn reasoning patterns for the relational triple extraction task. first, we construct a relational graph from the sentence, i.e. text-to-graph generation, to identify potential multi-hop reasoning paths. then we utilize a multi-layer relational graph convolution network (r-gcn) @xcite to propagate node information along these paths. next, to capture the relation type inference logic of the reasoning paths, we aim to exploit and understand the conceptual expressions in the sentence, but the absence of human annotations for these expressions poses a huge challenge. to tackle this challenge, we propose a self-supervised reconstruction of the sentence from the relational graph, i.e. graph-to-text generation. our model captures the relation type inference logic by learning to recover the conceptual expressions from the symbolic relation composition, such as the recovery of "'s grandfather" from "father(father(•))" in figure 1 . finally, we use the reasoning pattern enhanced model to extract relational triples from the sentence. the main contributions of this paper are: ## related work early work extracted relational triples with pipeline systems @xcite @xcite , but they usually suffered from error propagation problems. also, they failed to capture the interactions between entities and relations. to address these issues, jointly extracting entities and relations with an end-to-end model has become the main paradigm of this task. previous work proposed several feature-based models @xcite @xcite . for example, @xcite proposed a joint embedding framework to map entities, relations, text features and type labels into unified low-dimensional spaces. afterward, several neural network-based methods were proposed to eliminate hand-crafted features @xcite @xcite . for example, @xcite proposed to extract relational triples directly with a sequence tagging model, whose tags contain the information of entities and the relations they hold. however, they assigned only one label to each word and failed to extract multiple triples whose entities overlap with each other. recent work proposed several mechanisms to address the overlapping triple problem, such as sequence tagging variations @xcite @xcite and triple element generation @xcite @xcite @xcite . for example, @xcite proposed a cascade binary tagging framework and modeled relations as functions that map subjects to objects. @xcite proposed to decompose the task into three subtasks: relation judgment, entity extraction and subjectobject alignment. @xcite proposed to generate the element sequence of triples with a copy-based seq2seq model, while @xcite proposed to generate the set of triples with a set prediction network. however, these methods mainly focused on learning text patterns of the explicitly mentioned triples. they usually ignored the relational reasoning patterns thus failed to extract the implicitly implied triples @xcite . although chen et al. (2021) proposed a reasoning pattern enhanced model, they utilized entity type information, which requires extra supervision. different from previous work, we propose a mutual generation framework of text and graph to capture relational reasoning patterns. we identify multi-hop reasoning paths by generating a relational graph from the sentence. we propose to capture the relation type inference logic by incorporating supplementary conceptual expressions with self-supervised sentence generation from the graph. experimental results on several datasets demonstrate the effectiveness of our method. george grandf. but ... ... '@xmath0. but ... ... '@xmath1. but ... ... '@xmath2, father, judy judy, father, george encoder prediction text-to-graph triple extractor graph-to-text triple extractor judy, father, george decoder prediction r-gcn george(2) david judy(1) judy(2) george(1) shift right encoder decoder george is judy 's father and david 's grandfather, but judy is not familiar with george . george is judy 's father and david 's grandfather, but judy is not familiar with george . george is judy 's father and david 's grandfather, but judy is not familiar with george . ## our approach the overall framework of our approach is illustrated in figure 2 . we introduce the text-to-graph and the graph-to-text generation methods in section 3.1 and 3.2, respectively. then we introduce the triple extractor in section 3.3 and the details of training and inference in section 3.4. ## conclusion in this paper, we propose to learn relational reasoning patterns for relational triple extraction with mutual generation of text and graph. we construct a relational graph from the sentence and apply graph convolutions to identify multi-hop reasoning paths. we propose a sentence reconstruction task to explore the unlabeled conceptual expressions of the sentence for capturing the relation type inference logic along the paths. we conduct experiments on two benchmark datasets, and the results demonstrate the effectiveness of our method.