Jointly Extracting Explicit and Implicit Relational Triples with Reasoning Pattern Enhanced Binary Pointer Network (2021)

Abstract: Relational triple extraction is a crucial task for knowledge graph construction. Existing methods mainly focused on explicit relational triples that are directly expressed, but usually suffer from ignoring implicit triples that lack explicit expressions. This will lead to serious incompleteness of the constructed knowledge graphs. Fortunately, other triples in the sentence provide supplementary information for discovering entity pairs that may have implicit relations. Also, the relation types between the implicitly connected entity pairs can be identified with relational reasoning patterns in the real world. In this paper, we propose a unified framework to jointly extract explicit and implicit relational triples. To explore entity pairs that may be implicitly connected by relations, we propose a binary pointer network to extract overlapping relational triples relevant to each word sequentially and retain the information of previously extracted triples in an external memory. To infer the relation types of implicit relational triples, we propose to introduce real-world relational reasoning patterns in our model and capture these patterns with a relation network. We conduct experiments on several benchmark datasets, and the results prove the validity of our method.

## introduction relational triple extraction is defined as automatically recognizing semantic relations with triple structures (subject, relation, object) among multiple entities in a sentence. it is a critical task for constructing knowledge graphs (kgs) from unlabeled corpus @xcite . early work of relational triple extraction applied pipeline methods @xcite , which ran entity recognition and relation classification separately. however, such pipeline approaches suffered from error propagation. to address this issue, recent work proposed to jointly extract entity and relations from the text with feature-based methods @xcite @xcite . afterward, neural network-based models were proposed to eliminate hand-crafted features @xcite . more recently, several methods were proposed to extract overlapping triples, such as tagging-based @xcite , graph-based @xcite , copy-based @xcite @xcite and token pair linking models @xcite . existing models achieved considerable success on extracting explicit triples which have direct relational expressions in the sentence. however, there are many implicit relational triples that are not explicitly expressed. for example, in figure 1 , the explicit triples are strongly indicated by the key relational phrases, but the implicit relation "live in" is not expressed explicitly. unfortunately, existing methods usually ignored implicit triples @xcite , which will cause serious incompleteness of the constructed kgs and performance degradation of downstream tasks @xcite @xcite . our work is motivated by several observations. first, other relational triples within a sentence provide supplementary information for discovering entity pairs that may have implicit relational connections. for example, in figure 1 , the explicit triples establish a relational connection between "mark spencer" and "huntsville" through the intermediate entity "digium". second, the relation types of implicit relation triples can be derived through real-world reasoning patterns. for example, in figure 1 , the reasoning pattern "one lives where the company he works for is located" helps identify the type of the implicit triple as "live in". in this paper, we propose a unified framework for the joint extraction of explicit and implicit relational triples. we propose a binary pointer network (bptrnet), which is based on the pointer network @xcite , to extract overlapping relational triples relevant to each word sequentially. to discover implicitly connected entity pairs, we preserve the information of previously extracted triples in an external memory and use it to enhance the extraction of later time steps. to infer the relation types between the implicitly connected entity pairs, we propose to augment our model with realworld relational reasoning patterns and capture the relational inference logic with a relation network (rn) @xcite . the rn obtains a pattern-enhanced representation from the memory for each word pair. then the reasoning pattern enhanced bptrnet (r-bptrnet) uses the word pair representation to compute a binary score for each candidate triple. finally, triples with positive scores are output as the extraction result. the main contributions of this paper are: ## related work early work of relational triple extraction addressed this task in a pipelined manner @xcite @xcite . they first ran named entity recognition to identify all entities and then classified relations between all entity pairs. however, these pipelined methods usually suffered from error propagation problem and failed to capture the interactions between entities and relations. to overcome these drawbacks, recent research focused on jointly extracting entities and relations, including feature-based models @xcite @xcite and neural network-based models @xcite @xcite . for example, @xcite proposed to jointly embed entities, relations, text features and type labels into two low-dimensional spaces. @xcite proposed a joint model containing two long-short term memories (lstms) @xcite with shared parameters. @xcite proposed to extract relational triples directly by transforming this task into a sequence tagging problem, whose tags contain the information of entities and the relations they hold. however, they only assigned one label for each word, which means that this method failed to extract overlapping triples. subsequent work proposed several mechanisms to solve this problem: (1) labeling tagging sequences for words @xcite or entities @xcite ; (2) transforming the sentence into a graph structure @xcite ; (3) generating triple element sequences with copy mechanism @xcite @xcite ; (4) linking token pairs with a handshake tagging scheme @xcite . however, these methods usually ignored implicit relational triples that are not directly expressed in the sentence @xcite , thus will lead to the incompleteness of the resulting kgs and negatively affect the performance of downstream tasks @xcite . our work is motivated by two observations. first, other triples in the sentence provide supplementary evidence for discovering entity pairs with implicit relational connections. second, the relation types of the implicit connections need to be identified through real-world reasoning patterns. in this paper, we propose a unified framework for the joint extraction of explicit and implicit relational triples. we propose a binary pointer network to sequentially extract overlapping relational triples and externally keep the information of predicted triples for exploring implicitly connected entity pairs. we also propose to introduce real-world reasoning patterns in our model to help derive the relation type of implicit triples with a relation network. experimental results on several benchmark datasets demonstrate the effectiveness of our method. ## our approach the overall framework of our approach is shown in figure 2 . we introduce the binary pointer network (bptrnet) and the relation network (rn) in section 3.1 and 3.2 and the details of training and inference in section 3.3, respectively. ## conclusion in this paper, we propose a unified framework to extract explicit and implicit relational triples jointly. to discover entity pairs that may have implicit relational connections, we propose a binary pointer network to extract relational triples relevant to each word sequentially and introduce an external memory to retain the extracted triples. to derive the relation types of the implicitly connected entity pairs, we propose to introduce real-world relational reasoning patterns to this task and capture the reasoning patterns with a relation network. we conduct experiments on two benchmark datasets, and the results prove the effectiveness of our method.