Self-distilled Transitive Instance Weighting for Denoised Distantly Supervised Relation Extraction (2023)

Abstract: The widespread existence of wrongly labeled instances is a challenge to distantly supervised relation extraction. Most of the previous works are trained in a bag-level setting to alleviate such noise. However, sentence-level training better utilizes the information than bag-level training, as long as combined with effective noise alleviation. In this work, we propose a novel Transitive Instance Weighting mechanism integrated with the self-distilled BERT backbone, utilizing information in the intermediate outputs to generate dynamic instance weights for denoised sentence-level training. By down-weighting wrongly labeled instances and discounting the weights of easy-to-fit ones, our method can effectively tackle wrongly labeled instances and prevent overfitting. Experiments on both held-out and manual datasets indicate that our method achieves state-of-the-art performance and consistent improvements over the baselines.

## introduction distantly supervised relation extraction (dsre) @xcite is designed to automatically annotate the sentences mentioning the entity pairs, which enables a significant way of constructing large-scale datasets. however, distant supervision (ds) works under an unrealistic assumption that all sentences mentioning the same entity pair express the same relation. this introduces many noisy (wrongly labeled) instances into the dataset. to tackle this challenge, previous works mostly adopt the bag-level setting as shown at the top of figure 1 , where the vector representations of sentences are aggregated as the bag-level representation using multi-instance learning (mil) @xcite , and the prediction is thus produced from the bag representation. the optimization is conducted at the bag level to minimize the loss of bag prediction. only a small subset of previous works leverage the sentence-level setting @xcite as in the bottom of figure 1 , where the sentence-level predictions are produced and then aggregated into the bag prediction. in fact, sentence-level training can directly optimize the loss from each sentence, enabling higher information utilization than bag-level training. however, sentence-level training is vulnerable to the noise brought by ds, which limits its application. therefore, sentence-level training should be combined with effective noise-alleviation mechanisms to improve its robustness. the mainstream encoders of dsre models are piecewise convolutional neural network (pcnn) @xcite and recurrent neural network (rnn) @xcite over the years. it is reasonable for most previous works to take the simple encoder as a black box and only utilize its final output during training and inference. however, as large models like bert @xcite become popular in recent years, the information within the outputs from their intermediate layers is a non-trivial source of knowledge but is rarely discussed in dsre. in this work, we apply self-distillation to extract intermediate information as output probabilities and utilize them to denoise from wrong labels. furthermore, we use soft target selection and set up transitive knowledge passing among the students to alleviate the effects of noisy target probabilities from the teacher. the instances in dsre can be roughly divided into easy, hard and noisy ones. both easy and hard instances are correctly labeled but the model learns from hard instances slower @xcite . noisy instances have wrong labels and can be further divided into false positives (fps) and false negatives (fns). fps are instances with na relation but are wrongly labeled as non-na relations by ds, while fns are non-na instances wrongly labeled as na. we hope to avoid learning from noisy instances since they contain misleading information. moreover, we also need to avoid overfitting to easy instances to improve the learning of deeper knowledge. to tackle the above challenges, we propose a novel transitive instance weighting (tiw) mechanism for denoised sentence-level training in dsre. firstly, we apply self-distillation to directly reuse the knowledge of the teacher model for further denoising in students and set up a transitive way to share knowledge among students. secondly, we leverage the tiw mechanism to generate robust instance weights to reduce noise and overfitting during distillation. tiw considers two factors in the generation of instance weights: uncertainty @xcite for overfitting prevention and general consistency for noise reduction. the general consistency we proposed reflects the learning difficulty of the instance and provides guidance both in selecting soft targets of distillation and in weighting instances. lastly, the generated instance weights directly multiply the sentence-level losses to dynamically and globally enhance the training in the sentence-level setting. the experiments on both held-out and manual datasets show that our approach boosts the student's performance to achieve state-of-the-art results and consistent improvements over the teacher. we also provide an ablation study to explore the effects of the modules. in addition, we analyse the errors and provide additional experimental results in the appendix. our contributions are summarized as follows: ## related work distant supervision (ds) for relation extraction @xcite enables automatic annotation of large-scale datasets, but its strong assumption introduces a large number of wrongly labeled instances. following @xcite , various multi-instance learning methods are proposed to denoise from noisy instances, and they broadly fall into two categories: instance selection @xcite @xcite and instance attention @xcite @xcite . apart from multi-instance learning, many of the previous works try to improve the effectiveness of training. liu et al. (2017) and shang et al. (2020) try to convert wrongly labeled instances to useful information through relabeling. huang and du (2019) proposes collaborative curriculum learning for denoising. hao et al. (2021) adopts adversarial training to filter noisy instances in the dataset. nayak et al. (2021) designs a self-ensemble framework to filter noisy instances despite information loss. li et al. (2022) proposes a hierarchical contrastive learning framework to reduce the effect of noise. rathore et al. ( 2022 ) constructs a passage from the bags to generate a summary for classification. nevertheless, the above approaches are trained with bag-level loss, leading to lower utilization of information. in our work, we adopt sentence-level training to directly utilize sentence-level information and effectively tackle noise and overfitting using dynamic instance weights. knowledge distillation @xcite is an effective way to improve model generalization, though it has difficulty in transferring knowledge effectively @xcite . by sharing some parameters between teacher and students, selfdistillation @xcite improves knowledge transfer from teacher to student. @xcite applies self-distillation on bert @xcite to improve inference efficiency. however, in our work, we apply self-distillation as the tool to extract intermediate knowledge for denoising and further reduce the noise from the teacher with transitive information passing among the students. there are some epoch-level techniques to detect noisy instances like @xcite and @xcite . but in sentence-level dsre which is highly noisy and contains bias from the entity mentions @xcite , larger models like bert can overfit noisy instances faster, even before an epoch ends. therefore, we adopt a dynamic instance weighting mechanism which is more suitable for dsre. ## methodology our model is illustrated in figure 2 . the backbone is the bert encoder on the left, with a teacher classifier on the top. each student shares a subencoder with the teacher and uses a new classifier for prediction. firstly, the encoder and the teacher classifier are fine-tuned on the dataset to establish background knowledge. then, we freeze the encoder and train the student classifiers, in which knowledge distillation and transitive instance weighting (tiw) are applied to reduce noise and overfitting. tiw computes the instance weights based on three sources of knowledge: the teacher's output p t , the outputs of the student i itself p s i and the previous peer p s i-1 . it first selects a more consistent soft target p tg i between p t and p s i-1 based on the probabilities of making the same predictions as them (i.e. consistency), which are denoted as c t i and c s i respectively. then the possible false negative instances are filtered according to the predictions p s i-1 from the previous peer. finally, the instance weights w i are computed as the multiplication of uncertainty u i (normalized entropy) and the general consistency c i with the soft target. the details of tiw are given in algorithm 1, where re2id(r) is a function that maps the relation class r to its id for generating the one-hot label. the instance weights directly multiply with the instance losses to dynamically regulate the roles of instances in optimization (equation 6 ). this improves training by globally down-weighting the instances leading to extra noise and overfitting. ## conclusions and limitations in this paper, we propose a novel transitive instance weighting mechanism integrated with selfdistillation to denoise from sentence-level training of dsre. we employ the self-distilled bert backbone to extract intermediate information for generating reliable instance weights. tiw combines consistency with uncertainty as the tools to tackle noisy instances and alleviate overfitting. it also enables soft target selection and transitive knowledge passing among the students to tackle the noise from the teacher. the experiment results show that our method improves the general resistance to ds noise and prevents overfitting from harming its generalization, thus can achieve state-of-the-art performance and consistent improvements over the baselines on both the held-out and manual datasets. however, our work still has some limitations. firstly, since our model is built on the basis of the teacher-student network, the performance of the student is highly affected by the teacher. if the teacher provides too much noisy information, our instance weighting mechanism might not work. secondly, in some cases, the student fails to follow the correct predictions from the teacher, possibly due to ambiguity, lack of information or word-level noise. finally, tiw may down-weight some instances of infrequent relation classes due to their difficulty, but it can be tackled by combining tiw with other methods addressing the long-tailed distribution of relations.