Key Mention Pairs Guided Document-Level Relation Extraction (2022)

Abstract: Document-level Relation Extraction (DocRE) aims at extracting relations between entities in a given document. Since different mention pairs may express different relations or even no relation, it is crucial to identify key mention pairs responsible for the entity-level relation labels. However, most recent studies treat different mentions equally while predicting the relations between entities, leading to sub-optimal performance. To this end, we propose a novel DocRE model called Key Mention pairs Guided Relation Extractor (KMGRE) to directly model mention-level relations, containing two modules: a mention-level relation extractor and a key instance classifier. These two modules could be iteratively optimized with an EM-based algorithm to enhance each other. We also propose a new method to solve the multi-label problem in optimizing the mention-level relation extractor. Experimental results on two public DocRE datasets demonstrate that the proposed model is effective and outperforms previous state-of-the-art models.

## introduction relation extraction (re), which aims to identify the relations between entities in a given text, has been explored at the sentence level for decades @xcite @xcite . however, according to @xcite , a large amount of relations can only be identified across multiple sentences in the real-world scenarios. therefore, researchers have recently turned to extracting relations directly in documents @xcite @xcite . document-level relation extraction (docre) encounters many new challenges compared to its sentence-level counterpart. a document may include numerous entities, and the same entity may appear multiple times in different sentences. it requires the docre models to recognize and focus on the part of the document that has relevant context for a particular entity pair. many previous works solve the above problems by obtaining stronger context-aware entity pair representations. there are two main ways to achieve this: the graphbased methods @xcite @xcite and the sequence-based methods @xcite . the graphbased methods construct a document graph and then use graph neural networks (gnns) to aggregate information across nodes. besides, as transformer @xcite could be regarded as a fully connected gnn, the sequence-based methods attempt to directly use transformer-based pretrained language models (plms) for docre without graph structure. the sequence-based methods generally use strong plms (e.g., bert @xcite and roberta @xcite ) to model the input text and use different strategies to get entity pair representations, e.g., average pooling @xcite and attentive pooling @xcite . however, despite these successful efforts, most existing methods still ignore the critical issue of treating different mentions equally in an entity pair. and it is at odds with the actual situation, as different mention pairs may express different relations or even no relation. for the example in figure 1 , multiple relations exist between genc ruli and university of tirana, e.g., employer and educated at. these two relations can be inferred by different mention pairs of them, and at the same time, there are also several mention pairs don't express any relation. the multi-mention property of docre makes it difficult to establish context to relation mapping at the entity level directly. therefore, equal treating all mentions ignores the difference between different mentions' contexts and may introduce irrelevant information to mislead model training. ## methodology this section introduces the proposed model km-gre which directly extracts relations at the mention level. we first introduce the task formulation of docre. with a document d that contains a set of entities @xmath0, the task is to extract relations between each entity pair (e h , e t ), where e h , e @xmath1, respectively. an entity e i may occur multiple times in the document, which could be defined as {m i j } ne i @xmath2, and therefore the mention pairs of (e h , e t ) could be defined as we denote m h i and m t j as m i and m j below for convenience. c is the set of pre-defined relation types. there exists a relation @xmath3. it is intuitive to train a mention-level relation extractor and fuse its results to generate the entitylevel relation label. however, instead of one mention pair being matched to one relation label, we only have the relation labels of entity pairs. therefore, it could be challenging to train a mention-level relation extractor, directly. we propose to regard docre as a mil problem and extend a novel probabilistic model to handle this issue as shown in figure 2 . to identify the key mention pairs, we assign a binary variable @xmath4, 1} to each mention pair, denoting whether it is responsible for the relation label of the entity pair. inspired by em-mil @xcite , the relation label of (e h , e t ) is generated with probability: the goal is to jointly train the relation extractor and the key instance classifier to maximize the likelihood of the training data. formally, the objective function is presented as below: ## conclusion in this paper, we propose a new docre model called kmgre for the multi-mention problem, containing a mention-level relation extractor and a key instance classifier. our method uses the key instance classifier to identify those key mention pairs responsible for the entity pair relation label. also, we propose a new optimization method to solve the multi-label problem in optimizing the mentionlevel relation extractor, as directly assigning the entity-level labels to the key instances can lead to misguidance. experimental results on two public docre datasets show kmgre outperforms previous state-of-the-art methods. the ablation study also confirms the effectiveness of our new method for optimizing the mention-level relation extractor in multi-label cases.