Boosting Document-Level Relation Extraction by Mining and Injecting Logical Rules (2022)

Abstract: Document-level relation extraction (DocRE) aims at extracting relations of all entity pairs in a document. A key challenge to DocRE lies in the complex interdependency between the relations of entity pairs. Unlike most prior efforts focusing on implicitly powerful representations, the recently proposed LogiRE (Ru et al., 2021) explicitly captures the interdependency by learning logical rules. However, LogiRE requires extra parameterized modules to reason merely after training backbones, and this disjointed optimization of backbones and extra modules may lead to sub-optimal results. In this paper, we propose MILR, a logic enhanced framework that boosts DocRE by Mining and Injecting Logical Rules. MILR first mines logical rules from annotations based on frequencies. Then in training, consistency regularizationis leveraged as an auxiliary loss to penalize instances that violate mined rules. Finally, MILR infers from a global perspective based on integer programming. Compared with LogiRE, MILR does not introduce extra parameters and injects logical rules during both training and inference. Extensive experiments on two benchmarks demonstrate that MILR not only improves the relation extraction performance (1.1%-3.8% F1) but also makes predictions more logically consistent (over 4.5% Logic). More importantly, MILR also consistently outperforms LogiRE on both counts. Code is available at https://github.com/XingYing-stack/MILR.

## introduction document-level relation extraction (docre) aims to identify relations of all entity pairs in a document, playing an essential role in knowledge graph construction @xcite ), question answering @xcite , etc. a key challenge to docre lies in the fact that relations rules: spouse_of(𝑣 0 , 𝑣 1 ) ↔ spouse_of(𝑣 1 , 𝑣 0 ), parent_of(𝑣 0 , 𝑣 1 ) ↔ child_of(𝑣 1 , 𝑣 0 ), parent_of(𝒗 𝟎 , 𝒗 𝟐 ) ← spouse_of(𝒗 𝟎 , 𝒗 𝟏 ) ∧ parent_of(𝒗 𝟏 , 𝒗 𝟐 ) entities: chusovitina, alisher, bakhodir kurpanov @xcite in 2002, chusovitina's son alisher, then three years old, was diagnosed with leukemia. @xcite … she moved to germany with her husband bakhodir kurpanov, a former successful wrestler, and their son. @xcite … ## preliminaries in this section, we first present the formulation of docre and define two concepts: atoms and logical rules. since milr can be combined with various backbones, we summarize the paradigm of them. problem formulation given a document d containing n named entities {e i } n @xmath0, the task of docre is to predict the relation types between entity pairs (e h , e t ) h,@xmath1,••• ,n},h̸ =t . the set of relation types is defined as r ∪ {na}, where r is a pre-defined set and na stands for "no relation". ## methodology in this paper, we propose milr as a modelagnostic framework to endow existing docre models with the sense of logical consistency during training and inference. the core idea of milr is: both the output logits and final predictions should be constrained by logical rules. however, most datasets do not include gold logical rules. so milr adopts a data-driven approach to mine rules directly from relational annotations (sec. 3.1). during training, the consistency regularization encourages the backbone to output logits that conform to mined rules (sec. 3.2). during inference, logits along with mined rules are combined to make global predictions (sec. 3.3). finally, sec. 3.4 presents a detailed comparison between milr and logire. ## conclusion in this paper, we propose a novel framework milr to explicitly capture the interdependency of relations between different entity pairs in docre. milr mines logical rules from relational annotations and utilizes rules to constrain the output logits and final predictions of backbones. the proposed milr is a general framework and has shown to be effective with various base models, consistently improving their effectiveness and logical consistency.