Adaptive Hinge Balance Loss for Document-Level Relation Extraction (2023)

Abstract: Document-Level Relation Extraction aims at predicting relations between entities from multiple sentences. A common practice is to select multi-label classification thresholds to decide whether a relation exists between an entity pair. However, in the document-level task, most entity pairs do not express any relations, resulting in a highly imbalanced distribution between positive and negative classes. We argue that the imbalance problem affects threshold selection and may lead to incorrect “no-relation” predictions. In this paper, we propose to down-weight the easy negatives by utilizing a distance between the classification threshold and the predicted score of each relation. Our novel Adaptive Hinge Balance Loss measures the difficulty of each relation class with the distance, putting more focus on hard, misclassified relations, i.e. the minority positive relations. Experiment results on Re-DocRED demonstrate the superiority of our approach over other balancing methods. Source codes are available at https://github.com/Jize-W/HingeABL.

## introduction document-level relation extraction (re) plays an important role in nlp applications such as knowledge graph construction. it aims at predicting relations between entities from multiple sentences. as illustrated in figure 1a , an entity pair may have zero, one, or multiple relations, so document-level re is a multi-label classification task. to solve this, a common practice is to adaptively select thresholds for multi-label classification @xcite . for a correct prediction, the confidence scores of existent relations should be higher than the threshold, and conversely, those of non-existent relations should be lower. however, there is a significant imbalance problem between positive and negative classes in document-level re. the number of entity pairs increases quadratically with the number of entities. thus, compared with the sentence-level counterpart, there are far more entity pairs to be classified in document-level re, and most entity pairs express no relation. for example, in the documentlevel re dataset, re-docred @xcite , 94% of the entity pairs express no relation. the issue of class imbalance may lead to more incorrect "no-relation" predictions. in our paper, we mainly consider the "positive/negative imbalance", rather than the imbalance between different types of "positive relations". the positive/negative imbalance tends to drag the threshold towards the large class, that is, the "no-relation" class. we discover that 78.2% of incorrect "no-relation" predictions have correct label ranking, but the confidence score is lower than the threshold, which is shown in figure 1b . in other words, the model has enough confidence in the existent relations, but it makes a "no-relation" prediction due to the unnecessarily high threshold. based on this intuitive finding, we aim to address this "incorrect predictions with correct label ranking" phenomenon to improve the accuracy. we believe that overtraining on well-classified nonexistent relations may lead to unnecessarily high thresholds. therefore, we propose to adaptively select thresholds, and then down-weight the relations that are far from the decision boundary using hinge weighting. our contributions are three-fold: ## preliminary the task of document-level relation extraction is concerned with the prediction of relation types between subject and object entities in a given document. we will first introduce the formulation of this task, and then discuss the commonly used atl method. ## adaptive hinge balance loss based on the analysis above, we aim to maximize the distance between the decision boundary s th and the sample point s r , @xmath0. to this end, we propose our adaptive hinge balance loss. ## conclusion we propose a novel adaptive hinge balance loss for document-level relation extraction to tackle the imbalance problem of positive and negative classes. experimental results show our approach outperforms existing methods. since our loss is modelindependent, it has potential applicability to other multi-label classification scenarios.