Actively Supervised Clustering for Open Relation Extraction (2023)

Abstract: Current clustering-based Open Relation Extraction (OpenRE) methods usually adopt a two-stage pipeline, which simultaneously learns relation representations and assignments in the first stage, then manually labels relation for each cluster. However, unsupervised objectives struggle to explicitly optimize clusters to align with relational semantics, and the number of clusters K has to be supplied in advance. In this paper, we present a novel setting, named actively supervised clustering for OpenRE. Our insight lies in that clustering learning and relation labeling can be performed simultaneously, which provides the necessary guidance for clustering without a significant increase in human effort. Along with this setting, we propose an active labeling strategy tailored for clustering. Instead of only focusing on improving the clustering of relations that have been discovered, our strategy is encouraged to discover new relations through diversity regularization. This is particularly beneficial for long-tail relations in the real world. Experimental results show that our method is able to discover almost all relational clusters in the data and improve the SOTA methods by 13.8% and 10.6%, on two datasets respectively.

## introduction relation extraction (re) aims to detect and extract the potential relation between the given entity pair in unstructured text. the extracted relation facts play a vital role in many downstream applications, such as knowledge base population @xcite , search engine @xcite , and question answering @xcite . to deal with the emerging unknown relational types in the real world, open relation extraction (openre) has been widely studied. ## related work clustering-based openre: the clustering-based paradigm considers relation discovery as a twostage pipeline, which clusters relational data first, and then manually labels relational semantics for each cluster. conventional methods cluster instances by human-defined linguistic features @xcite @xcite , such as entity words/type, dependency paths, trigger words, context pos tags. recently, many studies have shown that pretrained models learn diversified linguistic knowledge @xcite @xcite . @xcite leverage the self-supervised signals provided by the pretrained model to iteratively learn relation representations and optimize clustering. due to the lack of strong supervision, it is difficult for the above methods to produce satisfactory clustering results. although some works @xcite try to use the labeled data of predefined relations to complete the missing supervision, the semantic gap between predefined and open relations leads to negative clustering bias, especially when these relations come from different domains @xcite . by performing clustering learning and relation labeling alternately, our actively supervised method can provide strong supervision and improve the two-stage methods by a large margin. in the main results (sec. 5), we achieve this improvement at the cost of only two active labels for each relation on average. for two-stage methods, relation labeling for each cluster requires at least one (usually more) instance to be manually observed. therefore, there is no significant increase in human effort. active learning: active learning is a research field with high relevance to the proposed methods. in the research field, a classical method is uncertainty-based sampling @xcite @xcite . the uncertainty can be defined based on the posterior probability of a predicted class or the distances to the decision boundaries. in the context of deep learning, mc dropout @xcite is an effective way for uncertainty estimation, but the computationally inefficient limits its application in large-scale datasets. recently, representative sampling is attracting lots of attention (sener and bert [cls] [e1] bill gates [\e1] was born in [e2] seattle[\e2] in 1995 [sep] relation representation encoder open relational data position density long distance high density 1 2 key point selection & labeling relation classifier clustering clustering assignments and discovered relations backprop active supervised clustering savarese, 2018; @xcite , which selects data points that represent the distribution of an unlabeled pool. unlike the classical labeling strategies designed for fixed classes, the proposed strategy is encouraged to discover new relational clusters while improving the clustering of relations that have been discovered. ## approach in this work, we present a new setting named actively supervised clustering for openre (as-core), which fuses the isolated two-stage pipeline to guide clustering learning. fig. 2 illustrates the training pipeline. the openre problem addressed in this work is formally stated as follows. given as input an open relational dataset @xmath0, .., n }, the goal is to discover and label potential relations @xmath1, .., k} in the open data and cluster the corresponding instances. note that the number of relations k in d is unknown. ## main results ## conclusions in this work, we present a new setting, named actively supervised clustering for openre, which provides the necessary guidance for clustering without a significant increase in human efforts. along with this setting, a labeling strategy tailored for clustering is proposed, maximizing the clustering performance while discovering as many relations as possible. different loss functions are assigned to pseudo labels with different reliability, which mitigate the risk of over-fitting to noise in pseudo labels. experimental results show that this method significantly outperforms the existing two-stage methods for openre.