R el U -Net: Syntax-aware Graph U -Net for Relational Triple Extraction (2022)

Abstract: Relational triple extraction is a critical task for natural language processing. Existing methods mainly focused on capturing semantic information, but suffered from ignoring the syntactic structure of the sentence, which is proved in the relation classification task to contain rich relational information. This is due to the absence of entity locations, which is the prerequisite for pruning noisy edges from the dependency tree, when extracting relational triples. In this paper, we propose a unified framework to tackle this challenge and incorporate syntactic information for relational triple extraction. First, we propose to automatically contract the dependency tree into a core relational topology and eliminate redundant information with graph pooling operations. Then, we propose a symmetrical expanding path with graph unpooling operations to fuse the contracted core syntactic interactions with the original sentence context. We also propose a bipartite graph matching objective function to capture the reflections between the core topology and golden relational facts. Since our model shares similar contracting and expanding paths with encoder-decoder models like U-Net, we name our model as Relation U-Net (RelU-Net). We conduct experiments on several datasets and the results prove the effectiveness of our method.

## introduction relational triple extraction (rte) is defined as automatically recognizing entity pairs and the semantic relations in the form of (subject, relation, object) from unstructured text. it is a critical task for natural language processing, especially for constructing large-scale knowledge graphs (kgs) from unlabeled corpus @xcite . recent work proposed several neural network methods to extract relational triples. for example, @xcite formulated this task as sequence tagging problems but failed to extract overlapping triples. @xcite proposed a cascade tagging framework to solve the overlapping problem. @xcite proposed to extract the implicit relational triples which are not explicitly expressed in the sentence with a binary pointer network. existing methods achieved considerable success in capturing semantic information from relational mentions. however, they usually failed to incorporate syntactic structures of the sentence, which is proved to contain rich relational information in the relation classification (rc) task @xcite . these syntax-aware rc methods usually pruned irrelevant dependency edges according to the known locations of the entity pair to eliminate the noise of the dependency tree. for example, the relational facts in figure 1 (top) can be easily inferred given the dependency paths, which are pruned from the original dependency tree using the locations of the known entity pairs. unfortunately, the locations of entities are unknown in the rte task, as shown in figure 1 @xcite . this absence makes it difficult to prune dependency noise thus leads to an insufficient exploration of syntactic and relational information. in this paper, we propose a unified framework to tackle this challenge and incorporate syntactic information for relational triple extraction. first, we propose to reduce the dependency tree and eliminate syntactic noise with graph pooling operations (figure 1 ). we utilize mincut pooling @xcite to cluster similar nodes (words) and hence obtain a core relational topology. next, we apply the inductive graphsage algorithm @xcite to propagate cluster information and capture the syntactic interactions underlying the core topology. then, we propose to symmetrically expand the core topology with graph unpooling operations to integrate the syntactic interactions with the original sentence context. finally, we propose a bipartite graph matching loss to induce the connections in the core topology to reflect golden relational facts, which is similar to the reflections between the dependency paths and golden facts (e.g. figure 1 ) in the rc task. since our model shares similar contracting and expanding paths with encoder-decoder models like u-net @xcite , we name our model as relation u-net (relu-net). the main contributions of this paper are: ## related work in the early work of relational triple extraction, the task is addressed in a pipelined manner @xcite @xcite . they recognized all entities in the sentence and classified the relations between pairs of extracted entities separately. however, the pipeline methods usually suffered from error propagation and failed to capture the interactions between the entities and relations. to address these issues, end-to-end models for joint extraction of entities and relations have become the dominant paradigm of this task, including feature-based models @xcite @xcite and neural network-based models @xcite @xcite . for example, @xcite proposed to detect relation mentions and their entity arguments with distant supervision. @xcite proposed to model the inter-dependencies of entities and relations through the entity-relation table proposed by @xcite . @xcite first formulated this task as a sequence tagging problem, but they failed to extract overlapping triples. more recent work developed several strategies to address the overlapping triple problem, including sequence tagging based models @xcite @xcite and triple generation based models @xcite @xcite @xcite . for example, @xcite proposed a cascade tagging scheme to simultaneously identify all possible overlapping triples. @xcite proposed a binary pointer network to extract overlapping relational triples and introduced a relational network to capture relational reasoning patterns for this task. @xcite proposed a sequence-tosequence triple generation model with copy mechanism, while @xcite proposed to treat this task as a direct set prediction problem. however, these methods mainly focused on learning semantic information of the sentence and usually suffered from ignoring syntactic patterns of the sentence. although @xcite applied the dependency tree to extract regional features of the text, they did not prune irrelevant contents from the dependency tree, which may be sub-optimal. different from previous work, we propose to remove redundant information from the dependency tree with graph pooling operations and integrate core syntactic connections with the original sentence context with graph unpooling operations. we also propose a bipartite matching loss to guide the core syntactic connections to reflect golden relational facts, like in the rc task. experimental results on several benchmark datasets prove the effectiveness of our method. ## our approach the overall framework of our approach is illustrated in figure 2 . we introduce the relation u-net, the triple extractor and the details of model training in section 3.1, 3.2 and 3.3, respectively. ## conclusion in this paper, we propose a unified framework to incorporate syntactic structures of the sentence for relation triple extraction. we propose a graph pooling network to automatically prune the dependency tree to a core topology and remove useless information. we propose a symmetrical graph unpooling network to integrate the syntactic interactions underlying the core topology with the original sen-tence context. we also propose a bipartite graph matching objective function to learn the reflections between the core syntactic interactions and golden relational facts. we conduct experiments on two benchmark datasets, and the results demonstrate the effectiveness of our method.