H i CLRE : A Hierarchical Contrastive Learning Framework for Distantly Supervised Relation Extraction (2022)

Abstract: Distant supervision assumes that any sentence containing the same entity pairs reflects identical relationships. Previous works of distantly supervised relation extraction (DSRE) task generally focus on sentence-level or bag-level de-noising techniques independently, neglecting the explicit interaction with cross levels. In this paper, we propose a hierarchical contrastive learning Framework for Distantly Supervised relation extraction (HiCLRE) to reduce noisy sentences, which integrate the global structural information and local fine-grained interaction. Specifically, we propose a three-level hierarchical learning framework to interact with cross levels, generating the de-noising context-aware representations via adapting the existing multi-head self-attention, named Multi-Granularity Recontextualization. Meanwhile, pseudo positive samples are also provided in the specific level for contrastive learning via a dynamic gradient-based data augmentation strategy, named Dynamic Gradient Adversarial Perturbation. Experiments demonstrate that HiCLRE significantly outperforms strong baselines in various mainstream DSRE datasets.

## introduction relation extraction (re) can draw relations of two entities from unstructured text. it can be widely used in natural language processing applications such as knowledge graph construction @xcite and question answering @xcite @xcite . existing re works @xcite @xcite rely on a large-scale annotated dataset, which is timeconsuming and labor-intensive. dsre @xcite ., s 1 e 1 e 2 e n cook apple bag sentence entity s 2 s 3 /business /person /company america washington /location /country /capital david beckham victoria /people /family /members b 3 b 4 : cook apple steve jobs • in 1998, steve jobs asked cook to join apple. the previous approaches of dsre tackle the task at different granularities (i.e. sentence-level and bag-level). (1) sentence-level. these works @xcite focus on finding the ground-truth relational labels from the internal semantics of the input sentences. (2) baglevel. although these works @xcite @xcite consider the information of sentencelevel and bag-level simultaneously, but they ignore the explicit cross-level interactions, which contain plenty of knowledge to further boost the dsre task performance. as shown in figure 1 , the rich semantic information of bag-level and sentence-level are provided for the "cook" and "apple" in the entity level. for example, "steve jobs" in s 1 is also the co-founder of "apple" company and the label of bag-level is the "/business/person/company" exactly shows the relation of this entity pair. meanwhile, the huge semantic difference exists in a specific level such as the "/business/person/company" and "/location/country/capital" in bag level. to overcome the challenges mentioned above, we propose a hierarchical contrastive learning framework for distantly supervised relation extraction (hiclre), which facilities semantic interactions within a specific level and cross levels: to verify the effectiveness of hiclre, we evaluate our model on three mainstream dsre datasets, including nyt10 @xcite , gds @xcite , and kbp @xcite . the experimental results show that hiclre significantly outperforms the state-of-the-art baselines' performance, achieving a 2.2% relative auc increase and improving the p@m score from 77.2% to 78.2%. furthermore, the ablation study shows the individual contributions of each module. accordingly, the major contributions of this paper are summarized as follows: ## detailed analysis of hiclre 5.1 ablation study to verify the effectiveness of various modules in our hiclre model, we conduct ablation study experiments on the nyt10 dataset. specifically, we remove the following argued contributions in turn to evaluate the performance, including multigranularity recontextualization, three-level contrastive learning loss, and the data augmentation strategies in each level. the final results are shown in ## conclusion in this paper, we propose hiclre, a hierarchical contrastive learning framework for distantly supervised relation extraction. multi-granularity recontextualization module of hiclre utilizes a multi-head self-attention mechanism to transmit the information across three levels. dynamic gradient adversarial perturbation module combines the gradient perturbation with inertia memory information to construct better pseudo positive samples for contrastive learning. experiments show the effectiveness of hiclre against the strong baseline models in various dsre datasets.