RCL : Relation Contrastive Learning for Zero-Shot Relation Extraction (2022)

Abstract: Zero-shot relation extraction aims to identify novel relations which cannot be observed at the training stage. However, it still faces some challenges since the unseen relations of instances are similar or the input sentences have similar entities, the unseen relation representations from different categories tend to overlap and lead to errors. In this paper, we propose a novel Relation Contrastive Learning framework (RCL) to mitigate above two types of similar problems: Similar Relations and Similar Entities. By jointly optimizing a contrastive instance loss with a relation classification loss on seen relations, RCL can learn subtle difference between instances and achieve better separation between different relation categories in the representation space simultaneously. Especially in contrastive instance learning, the dropout noise as data augmentation is adopted to amplify the semantic difference between similar instances without breaking relation representation, so as to promote model to learn more effective representations. Experiments conducted on two well-known datasets show that RCL can significantly outperform previous state-of-the-art methods. Moreover, if the seen relations are insufficient, RCL can also obtain comparable results with the model trained on the full training set, showing the robustness of our approach.

## introduction relation extraction is a fundamental problem in natural language processing, which aims to identify the semantic relation between a pair of entities mentioned in the text. recent progress in supervised relation extraction has achieved great successes @xcite @xcite , but these approaches usually require large-scale labeled data. while in practice, human annotation is time-consuming and labor-intensive. to alleviate the human annotation efforts in relation to address the aforementioned limitations, zeroshot relation extraction has been proposed to extract relational facts where the target relations cannot be observed at the training stage. the challenge of zero-shot relation extraction models is how to learn effective representations based on seen relations at the training stage and well generalize to unseen relations at the test stage. two studies @xcite treat zero-shot relation extraction as a different task (i.e., question answering and textual entailment), but they both need human annotation auxiliary information for input, i.e., pre-defining question templates and relation descriptions. zs-bert @xcite predicts unseen relations with attribute representation learning. despite promising improvements on directly predicting unseen relations, zs-bert still makes wrong predictions due to similar relations or similar entities. the same problem arises in supervised methods under the zero-shot settings. as shown in figure 1 , there are two types of similar errors: similar relations and similar entities. for similar relations (see z 1 and z 2 ), existing methods predict wrongly results because the unseen relations possess similar semantics and data points belong to two relations in the representation space are overlapped. for similar entities (i.e., 2014 contest and 2002 contest), since entities are the context of relation and relation representations are derived from entities, the relation representations containing similar entities are close (see f (z 3 ) and f (z 4 )) and baselines wrongly consider f (z 4 ) belongs to follows in the representation space, even if two unseen relations are not related. recently, instancewise contrastive learning (instance-cl) @xcite @xcite @xcite has achieved remarkable success in representation learning. instance-cl is used to learn an effective representation by pulling together the instances from the same class, while pushing apart instances from different classes. inspired by instance-cl, we attempt to use instance-cl on seen relations to learn the difference between similar relations and the divergence of relation representations derived from similar entities. in this paper, we propose a novel relation contrastive learning framework (rcl) to solve the above-mentioned problems. figure 1 depicts the overview of the proposed model, which consists of four steps: (i) the input for rcl is a batch of sentences containing the pair of target entities and each sentence is sent into input sentence encoder to generate the contextual sentence embeddings 2 . (ii) taking the sentence embeddings as input, relation augmentation layer is designed to obtain the relation representations f (x i ) and their corresponding augmented views f ( xi ). (iii) by jointly optimizing a contrastive loss and a relation classification loss on seen relations, rcl can learn subtle difference between instances and achieve better separation between relations in the representation space simultaneously to obtain an effective projection function 2 the words, "embeddings", and "representations", are used interchangeably throughout this paper. ## conclusion in this paper, we propose a jointly framework for zero-shot relation extraction to mitigate two types of similar errors: similar relations and similar entities. different from conventional zero-shot relation extraction models which require external resources for training and test, our model does not require external resources. we demonstrate the effectiveness of our framework on two datasets, and our method achieves new state-of-the-art performance. furthermore, we compare various data augmentation strategies for contrastive learning and provide fine-grained analysis for interpreting how our approach works.