Improving Discriminative Learning for Zero-Shot Relation Extraction (2022)

Abstract: Zero-shot relation extraction (ZSRE) aims to predict target relations that cannot be observed during training. While most previous studies have focused on fully supervised relation extraction and achieved considerably high performance, less effort has been made towards ZSRE. This study proposes a new model incorporating discriminative embedding learning for both sentences and semantic relations. In addition, a self-adaptive comparator network is used to judge whether the relationship between a sentence and a relation is consistent. Experimental results on two benchmark datasets showed that the proposed method significantly outperforms the state-of-the-art methods.

## introduction relation extraction is a fundamental task in natural language processing (nlp) that predicts the semantic relation between two entities in a given sentence. it has attracted considerable research effort as it plays a vital role in many nlp applications such as information @xcite and question answering @xcite . most recent studies @xcite treated this task in a fully supervised manner and achieved excellent performance. however, the supervised models cannot extract relations that are not predefined or observed during training, while many new relations always exist in real-world scenarios. thus, it is worth enabling models to predict new relations that have never been seen before. such a task is considered as zero-shot learning @xcite , where a key to achieving high performance is how to generalize a model to unseen classes by using a limited number of seen classes. however, there are only a few existing studies on zero-shot relation extraction (zsre). @xcite tackled this task by using reading comprehension models with predefined question templates. @xcite simply reduced zsre to a text entailment task, utilizing existing textual entailment models. recently, @xcite presented zs-bert, which projects sentences and relations into a shared space and uses the nearest neighbor search to predict unseen relations. the previous studies overlooked the importance of learning discriminative embeddings. in essence, the discriminative learning helps models to better distinguish relations, especially on similar relations. our study focuses on this aspect and demonstrates its significance for improving zsre. specifically, we propose a new model that incorporates discriminative embedding learning @xcite for both sentences and semantic relations, which is inspired by contrastive learning @xcite commonly used in computer vision. in addition, instead of using distance metrics to predict unseen relations as done by @xcite , we use a self-adaptive comparator network to judge whether the relationship between a sentence and a relation is consistent. this verification process helps the model to learn more discriminative embeddings. experimental results on two datasets showed that our method significantly outperforms the existing methods for zsre. ## related work to date, zsre has been under-investigated so far. @xcite formulated zsre as a questionanswering task. they first manually created 10 question templates for each relation type and then trained a reading comprehension model. because it requires the effort of hand-crafted labeling, this method can be unfeasible and impractical to define templates of new-coming unseen relations. obamuyide and vlachos (2018) converted zsre to a textual entailment task, in which the input sentence containing two entities is considered as the premise p, whereas the relation description containing the same entity pair is regarded as the hypothesis h. 1 they then used existing textual entailment models @xcite as their base models, although these models may not be entirely relevant for zsre. the closest to our work is research by @xcite . first, they proposed the zs-bert model, which learns two functions to project sentences and relation descriptions into a shared embedding space. then, they used the nearest neighbor search to predict unseen predictions; however, it is prone to suffer the hubness problem @xcite . unlike the previous studies, our work emphasizes the necessity of discriminative embedding learning that may play a vital role in solving the zsre. ## conclusion in this work, we present a new model to solve the zsre task. our model aims to enhance the discriminative embedding learning for both sentences and relations. it boosts inter-relation separability and intra-relation compactness of sentence embeddings and maximizes distances between different relation embeddings. in addition, a comparator network is used to validate the consistency between a sentence and a relation. experimental results on two benchmark datasets demonstrated the superiority of the proposed model for zsre.