Towards Integration of Discriminability and Robustness for Document-Level Relation Extraction (2023)

Abstract: Document-level relation extraction (DocRE) predicts relations for entity pairs that rely on long-range context-dependent reasoning in a document. As a typical multi-label classification problem, DocRE faces the challenge of effectively distinguishing a small set of positive relations from the majority of negative ones. This challenge becomes even more difficult to overcome when there exists a significant number of annotation errors in the dataset. In this work, we aim to achieve better integration of both the discriminability and robustness for the DocRE problem. Specifically, we first design an effective loss function to endow high discriminability to both probabilistic outputs and internal representations. We innovatively customize entropy minimization and supervised contrastive learning for the challenging multi-label and long-tailed learning problems. To ameliorate the impact of label errors, we equipped our method with a novel negative label sampling strategy to strengthen the model robustness. In addition, we introduce two new data regimes to mimic more realistic scenarios with annotation errors and evaluate our sampling strategy. Experimental results verify the effectiveness of each component and show that our method achieves new state-of-the-art results on the DocRED dataset, its recently cleaned version, Re-DocRED, and the proposed data regimes.

## introduction the problem of document-level relation extraction (docre) has garnered increasing attention from the research community @xcite @xcite due to its importance to real-world applications. docre is inherently a multi-label problem, in which we have to predict a set of relations from the pre-defined label set for every entity pair in a document. thus, it is crucial for docre models to adopt an effective learning objective that can clearly distinguish massive semantically close relations. recently, several works have proposed new loss functions to learn an adaptive threshold for better separating positive and negative relations. however, these approaches @xcite either enforce learning a total order among all relations that leads to superfluous comparisons and diminishing differences among them or improperly penalize all pre-defined labels of positive entity pairs if their average margins are lower than the threshold when addressing the label imbalance problem @xcite . in contrast, we propose an approach that learns a partial order, ranking all positive relations above a threshold individually, which is in turn ranked above all negative relations. our approach does not waste precious data and probability mass in modeling the ordering among positive relations (likewise for negative relations). we further sharpen the distinction in each distribution of a relation and the threshold through the principled use of entropy minimization. besides, none of the above methods take the discriminability of internal representations into account, as well as the model robustness against annotation errors. to solve these issues, we introduce novel modifications to the supervised contrastive learning @xcite to accentuate the differences among the embeddings of entity pairs from different classes and the similarities of that from the same class. our method can better accommodate the multi-label setting and the long-tail phenomenon that is typically present in docre datasets. to combat the annotation error problem stated in @xcite , we design two new data regimes and a novel negative label sampling strategy that gives consistently strong performance even with incomplete annotations. in sum, our contributions are three-fold: ## methodology in this section, we describe our model called pem-scl that is based on a pairwise moving-threshold loss, entropy minimization, and supervised contrastive learning. ## conclusions in this paper, we propose a novel method for docre problem called pemscl, which contains a pairwise moving-threshold loss with entropy minimization, adapted supervised contrastive learning, and a novel negative sampling strategy, to achieve good integration of both discriminability and robustness. experimental results show that our method achieves new state-of-the-art results. shuang zeng, yuting wu, and baobao chang. 2021. sire: separate intra-and inter-sentential reasoning for document-level relation extraction. in findings of acl/ijcnlp. ningyu zhang, xiang chen, xin xie, shumin deng, chuanqi tan, mosha chen, fei huang, luo si, and huajun chen. 2021. document-level relation extraction as semantic segmentation. in proceedings of ijcai. wenxuan zhou, kevin huang, tengyu ma, and jing huang. 2021. document-level relation extraction with adaptive thresholding and localized context pooling. in proceedings of aaai. yang zhou and wee sun lee. 2022. none class ranking loss for document-level relation extraction. in proceedings of ijcai.