Novel Relation Detection: Discovering Unknown Relation Types via Multi-Strategy Self-Supervised Learning (2023)

Abstract: Conventional approaches to relation extraction can only recognize predefined relation types. In the real world, new or out-of-scope relation types may keep challenging the deployed models. In this paper, we formalize such a challenging problem as Novel Relation Detection (NRD), which aims to discover potential new relation types based on training samples of known relations. To this end, we construct two NRD datasets and exhaustively investigate a variety of out-of-scope detection methods. We further propose an effective NRD method that utilizes multi-strategy self-supervised learning to handle the problem of shallow semantic similarity in the NRD task. Experimental results demonstrate the effectiveness of our method, which significantly outperforms previous state-of-the-art methods on both datasets.

## introduction relation extraction (re) is an important task in structured information extraction, which aims to recognize the relations of entity pairs from texts @xcite @xcite . for example, given the sentence "westworld is a science fiction western series directed by jonathan nolan." and the entity pair @xcite , an re model should output the relation type "the director of ". existing re methods typically follow the closedworld classification assumption and can only recognize predefined relation types. however, such an assumption limits the usage of these methods in real-world applications, as new or out-of-scope (oos) relation types may continually emerge after the model is deployed. for example, in the wikidata knowledge graph (vrandecic and krötzsch, 2014), new items and properties keep appearing over time foot_0 . these new relations may mislead the deployed re model, making it incorrectly assign known relations to the data of new relations, as shown in figure 1. in addition, existing methods cannot automatically discover new relations for future development. to handle this problem, we propose a more realistic and challenging task, novel relation extraction (nre), which aims to discover potential new relation types based on training samples of known relations. note that we define novel relations as oos relations that are not included in the predefined relation set. in the nre task, we group the novel relation types into a new class and convert the traditional k-class re task into a (k + 1)-class re task. the (k + 1)-th class represents the novel relation. this task requires re models to correctly identify not only the known relations but also the novel relation. based on the oos data collected by the nre model, developers can easily and purposefully annotate the data and update the model, as shown in figure 1 . the novel relation class in the nre task is differ-ent from the "no relation" class and the "other relation" class in existing re tasks @xcite . previous developers always assume that the "no relation" class and the "other relation" class are known in the development phase, and they can annotate a large amount of training data for these two relations to train re models. however, the novel relation class indicates that the text contains new or oos relations whose distribution is unknown and unpredictable in the development phase. it is infeasible for developers to annotate training data for the novel relation class in real-world applications since real test data is usually unknown and continually changing. there are two lines of previous work related to nre, open re @xcite @xcite and oos detection @xcite @xcite . open re means that there are no predefined relation types and no labeled data. open re extracts phrases and arguments as specific relations and discovers new relations by clustering or heuristics. compared to open re, nre strengthens the capability of the conventional re methods and aims to automatically discover novel relation types that are not in the predefined relation set, while providing accurate predictions for known relations. another line of related work is oos detection, which aims to recognize oos data that does not belong to any predefined class. although oos detection has been widely investigated in other nlp tasks, its exploration in re is relatively rare. therefore, in this paper, we formalize the nre task and construct two nre datasets based on two widely used re datasets, fewrel @xcite and tacred @xcite . then, to establish nre's baselines, we exhaustively investigate a variety of oos detection methods @xcite @xcite . in general, previous oos detection methods usually learn the decision boundaries of known classes based on the feature or probability distributions of known training data. in the testing phase, they use confidence scores to identify samples outside the decision boundaries as oos data. however, when applying existing oos detection methods in the nre task, we find a shallow semantic similarity problem. specifically, sentences with oos relations may have similar surface information, such as entity overlapping and similar syntac-assume that ''nominated for'' is an oos relation type. text 1 (oos): westworld was nominated for an empire award for best tv series. text 2 (known): westworld is directed by jonathan nolan. ## dataset to the best of our knowledge, we are the first to formally introduce the nre task. therefore, we construct two nre datasets based on two widely used re datasets. in this section, we first briefly introduce the original re datasets and then describe the construction method of the nre datasets. finally, we show the statistics of these datasets. ## previous oos detection methods in other oos detection tasks, recent work has attempted to find an appropriate decision boundary to balance the performance of both known and oos relations. we roughly divide previous methods into two categories: probability-based methods @xcite and feature-based methods @xcite . we apply these methods to the nre task and provide a detailed quantitative analysis. ## methodology to address the above problem, we propose multi-strategy self-supervised learning, which explicitly models oos data with similar surface information by constructing various pseudo-oos data. ## conclusion in this paper, we introduce oos detection into relation detection, which can automatically discover the data with oos relations while correctly classifying the data with known relations. to cope with shallow semantic similarity, we propose multistrategy self-supervised learning. we construct two datasets for the nre task and compare our method with multiple strong baselines. the results demonstrate the effectiveness of our method.