An Adaptive Multi-Threshold Loss and a General Framework for Collaborating Losses in Document-Level Relation Extraction (2025)

Abstract: The goal of document-level relation extraction (DocRE) is to identify relations for a given entity pair within a document. As a multilabel classification task, the most commonly employed method involves introducing an adaptive threshold. Specifically, for an entity pair, if the scores of predicted relations exceed the threshold, the relations exist. However, we observe two phenomena that significantly weaken the model’s performance in DocRE: (1) as the label space (the number of relations) expands, the model’s performance gradually declines; (2) the model tends to prioritize predicting high-frequency relations in the long-tail problem. To address these challenges, we propose an innovative **A**daptive **M**ulti-**T**hreshold **L**oss (AMTL), which for the first time proposes to partition the label space into different sub-label spaces (thus reducing its overall size) and learn an adaptive threshold for each sub-label space. This approach allows for more precise tuning of the model’s sensitivity to diverse relations, mitigating the performance degradation associated with label space expansion and the long-tail problem. Moreover, our adaptive multi-threshold method can be considered as a general framework that seamlessly integrates different losses in different sub-label spaces, facilitating the concurrent application of multiple losses. Experimental results demonstrate that AMTL significantly enhances the performance of existing DocRE models across four datasets, achieving state-of-the-art results. The experiments on the concurrent application of multiple losses with our framework show stable performance and outperform single-loss methods. Code is available at https://github.com/xhm-code/AMTL.

## introduction document-level relation extraction (docre) @xcite aims to identify one or more relations * corresponding author. for an entity pair in documents. unlike sentencelevel relation extraction tasks, docre needs to handle longer texts, cross-sentence relations, and more complex contextual dependencies with diverse semantic structures. the increasing demand for accurate relation identification in tasks like question answering @xcite , knowledge graph construction @xcite , and event extraction @xcite has made docre a key area of research in information extraction. docre, as a multi-label classification task, traditionally employs binary cross-entropy loss (bce) to learn a global threshold. specifically, for each entity pair, if the relation scores exceed the global threshold, the model determines that corresponding relations exist. however, the global threshold frequently fails to adapt to the characteristics of all entity pairs, resulting in suboptimal performance. to address this limitation, @xcite propose an adaptive threshold loss (atl), which introduces an adaptive threshold for each entity pair so that the scores of positive classes are significantly higher than the threshold, while the scores of negative classes are much lower than the threshold foot_0 . although atl effectively alleviates the problem of the global threshold, it is still insufficient when dealing with the long-tail problem 2 . to further address the long-tail problem, @xcite propose an adaptive multi-label loss (aml), while @xcite propose an adaptive hinge balance loss (hingeabl). both methods are inspired by the hinge loss @xcite and enhance the model's ability to distinguish between tail classes and difficult-to-classify relations (where the relation scores are close to the threshold) by widening the gap between positive and negative classes. similarly, @xcite propose an adaptive focal loss (afl) based on the focal loss, which is designed to pay more attention to the tail classes to cope with the performance degradation caused by the long-tail problem. the above losses use only one adaptive threshold for each entity pair and fail to fully consider the diversity of relations. specifically, for an entity pair, its relation scores are compared with only one threshold. in the docre task, the types of relations are often diverse, with each relation having different semantics. for example, the docred @xcite and re-docred @xcite datasets include 96 different predefined relations. however, we observe the following two phenomena in the docre task: firstly, as shown in fig. 1 (a), the performance of models based on the atl and the state-of-the-art hingeabl decreases significantly as the label space 3 expands. secondly, in the long-tail problem, the models tend to predict the head classes, as shown in fig. 1(b) , where the head classes' scores are generally higher than those of the tail classes, which gives the models an excessive preference for the head classes. to address the performance degradation caused by the expansion of label space and the challenges posed by long-tail problem, we propose a novel multi-label classification loss, the adaptive multi-threshold loss (amtl). specifically, amtl innovatively proposes to partition the label space into different segments based on the frequency of relation occurrences, and introduces an adaptive positive classes pt ⊆ r represent the relations that exist for an entity pair, while negative classes nt ⊆ r represent the relations that do not exist, where @xmath0. 2 in docre datasets, specific relations within the predefined relation set r appear with higher frequency (commonly referred to as head classes), while others occur less frequently (referred to as tail classes), resulting in the long-tail problem, also known as the class imbalance problem. 3 label space refers to the number of relations in docre. our contributions are as follows: ## related work existing docre methods can be broadly divided into the following categories: ## methodology we first define the task of docre, followed by an introduction to the most commonly used loss in this task, adaptive threshold loss (atl). finally, we present our proposed improvement based on atl, the adaptive multi-threshold loss (amtl). ## main results and analysis we conduct experiments to answer the following research questions about our main contributions, adaptive multi-threshold loss (amtl) and adaptive multi-threshold (amt) framework: • q1: how effective is our amtl loss when applied to different models? (section 5.1) • q2: how does the performance of our amtl loss compare to other losses? (section 5.2) • q3: how effective is our adaptive multithreshold framework amt compared to existing adaptive single-threshold methods? (section 5.3) ## further analysis to further investigate our method's performance, we answer the following research questions: • q4: does the amt framework alleviate the longtail problem? (section 6.1) • q5: how effective is our amtl loss in mitigating performance degradation due to label space expansion? (section 6.2) • q6: how does our amtl loss perform on weakly supervised generalization? (section 6.3) • q7: how does our amtl loss compare to different plugin-based methods regarding performance when integrated into the model? (section 6.4) • q8: what is the training cost of our amtl loss? ## conclusion we propose a novel adaptive multi-threshold loss, amtl, which effectively mitigates the performance degradation caused by label space expansion and the long-tail problem in docre tasks. amtl first proposes to partition the label space into multiple segments and assign an adaptive threshold for each segment. moreover, we design a multithreshold framework that enables the collaborative application of multiple losses across different label space segments, which outperforms the single-threshold methods used in prior work. experiments show that amtl significantly improves the predictive performance of various models and achieves sota results on four datasets, further demonstrating its superiority. amtl also exhibits effective generalization capabilities, performing well on both partially labeled and fully labeled datasets. since our method is independent of specific models, it holds potential for wide applicability in other multi-label classification tasks.