End-to-end Learning of Logical Rules for Enhancing Document-level Relation Extraction (2024)

Abstract: Document-level relation extraction (DocRE) aims to extract relations between entities in a whole document. One of the pivotal challenges of DocRE is to capture the intricate interdependencies between relations of entity pairs. Previous methods have shown that logical rules can explicitly help capture such interdependencies. These methods either learn logical rules to refine the output of a trained DocRE model, or first learn logical rules from annotated data and then inject the learnt rules into a DocRE model using an auxiliary training objective. However, these learning pipelines may suffer from the issue of error propagation. To mitigate this issue, we propose Joint Modeling Relation extraction and Logical rules or JMRL for short, a novel rule-based framework that jointly learns both a DocRE model and logical rules in an endto-end fashion. Specifically, we parameterize a rule reasoning module in JMRL to simulate the inference of logical rules, thereby explicitly modeling the reasoning process. We also introduce an auxiliary loss and a residual connection mechanism in JMRL to better reconcile the DocRE model and the rule reasoning module. Experimental results on four benchmark datasets demonstrate that our proposed JMRL framework is consistently superior to existing rule-based frameworks, improving five baseline models for DocRE by a significant margin.

text: Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 7247–7263 August 11-16, 2024 ©2024 Association for Computational Linguistics
Document-level relation extraction (DocRE) aims to extract relations between entities in a whole document. One of the pivotal challenges of DocRE is to capture the intricate interdependencies between relations of entity pairs. Previous methods have shown that logical rules can explicitly help capture such interdependencies. These methods either learn logical rules to refine the output of a trained DocRE model, or first learn logical rules from annotated data and then inject the learnt rules into a DocRE model using an auxiliary training objective. However, these learning pipelines may suffer from the issue of error propagation. To mitigate this issue, we propose Joint Modeling Relation extraction and Logical rules or JMRL for short, a novel rule-based framework that jointly learns both a DocRE model and logical rules in an endto-end fashion. Specifically, we parameterize a rule reasoning module in JMRL to simulate the inference of logical rules, thereby explicitly modeling the reasoning process. We also introduce an auxiliary loss and a residual connection mechanism in JMRL to better reconcile the DocRE model and the rule reasoning module. Experimental results on four benchmark datasets demonstrate that our proposed JMRL framework is consistently superior to existing rule-based frameworks, improving five baseline models for DocRE by a significant margin.

heading: 1 Introduction
text: Relation extraction (RE) plays a vital role in information extraction (IE). It aims at identifying relations between two entities in a given text. Early efforts focus mainly on sentence-level RE. In recent years, document-level relation extraction (DocRE) has received increasing attention. It aims at identifying relations of all entity pairs in a document. Nowadays DocRE has been widely applied in
∗Corresponding authors
downstream applications such as question answering (QA) (Sorokin and Gurevych, 2017), knowledge graph construction (Luan et al., 2018), etc. Compared to sentence-level RE, DocRE imposes a greater challenge for modeling longer contexts and capturing the more complex interdependencies between entity pairs.
Most previous methods for DocRE focus on capturing interdependencies between entity pairs by learning powerful representations through neural models, such as pre-trained language models (Xu et al., 2021; Zhou et al., 2021a) and graph neural networks (Peng et al., 2017; Sahu et al., 2019; Zeng et al., 2020). However, these methods tend to be insufficient in inferring potential facts. Figure 1 illustrates such an example, where sub-figure (a) in Figure 1 shows an example of a document in the DocRED dataset, and sub-figure (b) shows the corresponding predictions yielded by ATLOP, a state-of-the-art (SOTA) method for DocRE. We can observe that
7247
ATLOP (Zhou et al., 2021a) only extracts apparent facts such as “(Parvathy, hasSpouse, Jayaram)” and “(Parvathy, hasChild, KalidasJayaram)”, but fails to identify potential facts such as “(KalidasJayaram, hasFather, Jayaram)” and “(Jayaram, hasChild, MalavikaJayaram)” which are not explicitly mentioned in the context of the given document.
It is well known that logical rules can be used to improve the performance for DocRE by inferring missing facts from existing ones. Sub-figure (c) in Figure 1 illustrates three logical rules, and sub-figure (d) shows their ability in inferring missing facts. To enhance existing DocRE models with logical rules, two rule-based frameworks have been proposed, namely LogicRE (Ru et al., 2021) and MILR (Fan et al., 2022). In more detail, LogicRE first learns logical rules based on the output logits of a trained neural model and then refines its predicted relations by reasoning with the learnt rules, whereas MILR first learns logical rules from annotated data and then trains a neural model penalized by an auxiliary loss for reflecting the violation of learnt rules. Although both LogicRE and MILR have shown promising results in enhancing DocRE, they still suffer from the error propagation issue due to their pipeline natures.
To mitigate the error propagation issue, we target jointly learning a neural module for DocRE and a neural module for approximating logical rules in an end-to-end fashion. In this paper we propose a novel framework named Joint Modeling Relation extraction and Logical rules or JMRL for short, as illustrated in Figure 2. The intuition of JMRL is to reduce the rule learning problem in discrete space to a parameter learning problem in continuous space, yielding a neural module, called rule reasoning module, for approximating logical rules, and then to integrate it into an existing DocRE model. The parameters of the rule reasoning module is tuned along with the parameters of the backbone DocRE model so that the whole model can be trained in an end-to-end fashion. Furthermore, we introduce an auxiliary loss and a residual connection mechanism in JMRL to better incorporate the backbone DocRE model and the rule reasoning module, so as to further improve the performance.
We exploit JMRL to enhance five baseline models for DocRE, including LSTM (Yao et al., 2019), Bi-LSTM (Yao et al., 2019), GAIN (Zeng et al., 2020), ATLOP (Zhou et al., 2021a) and DREEAM (Ma et al., 2023a). Experimental re-
sults on four benchmark datasets DWIE (Zaporojets et al., 2021), DocRED (Yao et al., 2019), ReDocRED (Tan et al., 2022b) and DocGNRE (Li et al., 2023) demonstrate that the proposed JMRL framework is superior to all SOTA rule-based framework for DocRE, improving the baseline models by a significant margin on all datasets. Besides, our analysis and case study further clarify why JMRL is able to improve the performance.
The main contributions of this work include: (1) We propose a novel framework named JMRL to integrate a neural module for approximating logical rules into a baseline DocRE model, so that the enhanced DocRE model can be trained in an end-to-end fashion. As far as we know, this is the first end-to-end approach for imposing logical rules upon DocRE models.
(2) We theoretically analyze the faithfulness between the rule reasoning module and logical rules.
(3) We conduct extensive experiments on four benchmark datasets, demonstrating that the proposed JMRL framework pushes forward five SOTA DocRE models by a significant margin. In particular, up to the submission date (2024/02/15), the JMRL-enhanced DREEAM model (submissions under the username jmrl) ranks the first in the public DocRED evaluation1.

heading: 2 Preliminaries
text: Problem formulation for DocRE. Given a document d involving a set of named entities Ed = {ei}1≤i≤nd , the task of DocRE aims at predicting the relations among all entity pairs {(eh, et) | eh, et ∈ Ed, eh ̸= et}. The set of predictable relations is defined as R+ = R ∪ {⊥}, where R is a pre-defined relation set and ⊥ the “no relation”. Atoms and facts. An atom is of the form r(x, y), where r ∈ R is a predicate, x and y are entity variables or entity constants. An atom is ground if it does not contain any variable. A fact is a ground atom of the form r(a, b), which may also be expressed as a triple (a, r, b) throughout the paper. Logical rules. We focus on learning chain-like logical rules (CRs). A CR is a datalog rule (Abiteboul et al., 1995) where all atoms are binary and every body atom shares variables with the previous atom and the next atom. A CR is called an L-CR if it has L body atoms. An L-CR R is of the form:
H(x, y)← B1(x, z1)∧B2(z1, z2)∧...∧BL(zL−1, y) 1https://codalab.lisn.upsaclay.fr/ competitions/365
where x denotes the head entity, y the tail entity, and z1, . . . , zL−1 variables. The part at the left (resp. right) side of ← is called the head (resp. body) of R. The rule R is called r-specific if H = r. By HR and BR we denote the atom in the head ofR and the set of atoms in the body ofR, respectively. A rule is ground if it does not contain any variable. A rule R is a fact if BR is empty and HR is ground. To uniformly represent CRs with fixed-length bodies, we introduce the identity relation (denoted by I) to rule bodies. For example, the 1-CR r(x, y)← p(x, y) can be converted into a 2-CR r(x, y)← p(x, z) ∧ I(z, y).
Given a set of facts G ⊂ E×R×E , we denote by G |= HR(a, b) if there exists a ground instance Rg of logical rule R such that HR(a, b) = HRg and BRg ⊆ G ∪ G− ∪ {I(e, e) | e ∈ E}, where G− = {(et, r−, eh) | (eh, r, et) ∈ G} and r− denotes the inverse relation of r. Let Σ be a set of r-specific CRs and (a, r, b) ∈ E×R×E an arbitrary fact. We denote by G |=Σ (a, r, b) if there exits an r-specific CR R ∈ Σ such that G |= HR(a, b).

heading: 3 Related Work
text: Document-level relation extraction. Early efforts for DocRE focus on better contextualized representations of relations by employing various technologies such as attention mechanisms (Yao et al., 2019; Zhou et al., 2021a), pre-trained language models (Tang et al., 2020; Xu et al., 2021), and knowledge distillation (Tan et al., 2022a; Ma et al., 2023a). To capture more complex interdependencies between entity pairs, recent studies aim at enhancing DocRE models with external modules such as graph neural networks (GNNs) (Christopoulou et al., 2019; Zhang et al., 2020; Zeng et al., 2020) or rule-based frameworks (Ru et al., 2021; Fan et al., 2022). Specifically, LogicRE (Ru et al., 2021) and MILR (Fan et al., 2022) are two SOTA rule-based frameworks for enhancing DocRE. LogicRE first learns logical rules based on the output logits of a trained neural model and then refines the predicted relations of the neural model by learnt rules. MILR first learns logical rules from annotated data and then trains a neural model penalized by an auxiliary loss for reflecting the violation of learnt rules. However, the above two frameworks suffer from the error propagation issue due to their pipeline natures. In contrast, our proposed JMRL framework integrates a neural module for rule reasoning into a backbone DocRE model, enabling the whole
model to be trained end-to-end and thus mitigating the error propagation issue. End-to-end rule learning. In recent years, there has been an emerging interest in exploiting neuralbased methods (Yang et al., 2017; Sadeghian et al., 2019; Yang and Song, 2020; Xu et al., 2022) for end-to-end rule learning. Inspired by their promising results, we also design a neural-based rule reasoning module in JMRL to approximate logical rules for DocRE. Different from previous methods, our approach targets a training objective coming from relation extraction, a natural language processing (NLP) task involving texts, while previous methods only designed for specific tasks in knowledge graph completion such as link prediction (Bordes et al., 2013) and triple classification (Lin et al., 2015). Furthermore, our approach can deal with the reasoning scenario where existing facts in the background knowledge are all uncertain since the truth degrees of existing facts are assigned continuous values by a DocRE model. Rule injection in neural models. There exist approaches focusing on injecting logical rules into neural models in different tasks of NLP, including knowledge base construction (Demeester et al., 2016; Ding et al., 2018), natural language inference (Li and Srikumar, 2019), sentiment analysis (Deng and Wiebe, 2015), knowledge graph validation (Du et al., 2019) and information extraction (Wang and Pan, 2020; Zhou et al., 2021b). These approaches require well-prepared handcrafted rules as input for the enhancement, which may prevent them from being practically used. In contrast, our proposed JMRL framework does not require hand-crafted rules as input.

heading: 4 The JMRL Framework
text: To impose logical rules upon a DocRE model, we propose a novel rule-based framework named Joint Modeling Relation extraction and Logical rules or JMRL for short, as illustrated in Figure 2. By and large, JMRL first employs a DocRE model to calculate output logits for all potential facts in a document, and then feeds them into a rule reasoning module to produce the rule-enhanced logits. The ultimately predicted logits are calculated by the residual connection of the original DocRE logits and the rule-enhanced logits. Then the entire model is trained by minimizing a weighted sum of classification losses calculated from the original DocRE logits and the ultimately predicted logits.
Furthermore, we can extract logical rules from the parameter assignment of the rule reasoning module to compose explanations for the predictions.

heading: 4.1 Document-level Relation Extraction
text: Given a document d involving a set of named entities Ed = {ei}1≤i≤nd , a typical DocRE model F calculates a logit F(eh, et, d) ∈ Rn+1 for each entity pair in {(eh, et) | eh, et ∈ Ed, eh ̸= et}, where n = |R|, [F(eh, et, d)]i denotes the logit for a normal relation for all 1 ≤ i ≤ n, and [F(eh, et, d)]n+1 denotes the logit for ⊥.
A DocRE model is usually trained by minimizing the binary cross-entropy (BCE) loss (Yao et al., 2019; Zeng et al., 2020) or the adaptive thresholding (AT) loss (Zhou et al., 2021a), a variant of crossentropy. In the inference phase, the set of predicted facts {(eh, r, et) | [σ(F(eh, et, d))]r > ϵ} are obtained by thresholding the predicted probabilities of each entity pair, where ϵ is a given threshold, σ is an activation function such as the sigmoid function or the softmax function.

heading: 4.2 The Rule Reasoning Module
text: The rule reasoning module is a neural module parameterized to simulate the inference of logical rules, approximating outputs as a rule system does. This module is trained along with the DocRE model to optimize a certain training objective.
Let N be the maximum number of rules to be learnt, L the maximum number of atoms in each rule and R∗ = R ∪ R− ∪ {I}. Suppose R = {ri}1≤i≤n, its corresponding set of inverse relations R− = {ri}n+1≤i≤2n, and I = r2n+1.
We define an extended logit F+(x, y, d) ∈ R2n+1, where [F+(x, y, d)]i = [σ(F(x, y, d))]i for all 1 ≤ i ≤ n, [F+(x, y, d)]i+n = [σ(F(y, x, d))]i for all 1 ≤ i ≤ n, and [F+(x, y, d)]2n+1 = 1 if x = y or 0 otherwise. The goal of our rule reasoning module is to estimate a truth degree s(N,L)r,x,y,d for every fact (x, r, y) ∈ Ed ×R∗ × Ed in every document d, where the estimated truth degree s(N,L)r,x,y,d reflects the degree of whether the fact (x, r, y) can be inferred by N L-CRs. For every normal relation r ∈ R, 1 ≤ k ≤ N, 1 ≤ l ≤ L, the intermediate estimated truth degree s(k,l)r,x,y,d for the l
th atom in the kth rule is defined as:
s (k,l) r,x,y,d =    2n+1∑ i=1 w (r,k,l) i [F+(x, y, d)]i, l = 1 2n+1∑ i=1 w (r,k,l) i ∑
(z,ri,y)∈ Ed×R∗×Ed
s (k,l−1) r,x,z,d [F+(z, y, d)]i, l > 1
(1) where w(r,k,l) ∈ [0, 1]2n+1 denotes the trainable weights on predicate selection for the lth body atom of the kth rule whose head atom is on r. w(r,k,l) is confined to [0, 1] by a softmax layer. Intuitively, w
(r,k,l) i = 1 indicates that the i th relation ri is selected as the predicate of the lth body atom.
Different from normal relations in R, for the head relation ⊥, we allow ⊥ and its reverse relation to appear in predicates of body atoms. To this end, we alter Equation (1) for r = ⊥ by looping i from 1 to 2n + 3, redefining w(r,k,l) ∈ [0, 1]2n+3, R∗ = R ∪ {⊥} ∪ R− ∪ {⊥−, I}, [F+(x, y, d)]i = [σ(F(x, y, d))]i for all 1 ≤ i ≤ n + 1, [F+(x, y, d)]i+n+1 = [σ(F(y, x, d))]i for
all 1 ≤ i ≤ n + 1, and [F+(x, y, d)]2n+3 = 1 if x = y or 0 otherwise.
The ultimate truth degree is calculated by aggregating the intermediate degrees of N rules:
s (N,L) r,x,y,d =
N∑
k=1
α(k)r s (k,L) r,x,y,d (2)
where α(k)r ∈ [−1, 1] is a trainable weight for the kth rule for the head relation r, which is confined to [−1, 1] by a tanh layer. Intuitively, α(k)r denotes the confidence score of the kth rule for r.
By introducing the following notion of induced parameter assignment, we show in Theorem 1 that the formalization of the proposed rule reasoning module is faithful to a certain set of CRs.
Definition 1. Given a set of r-specific L-CRs Σ = {Rk}1≤k≤N for Rk of the form r(x, y) ← pk,1(x, z1) ∧ · · · ∧ pk,L(zL−1, y), where pk,l ∈ R ∪ {⊥} ∪ R− ∪ {⊥−, I} if r = ⊥, or pk,l ∈ R ∪R− ∪ {I} otherwise, we call a parameter assignment of the rule reasoning module θ(N,L)r = {w(r,k,l)i }1≤k≤N,1≤l≤L,1≤i≤m ∪ {α (k) r }1≤k≤N Σinduced if it satisfies the following conditions: (1) ∀1 ≤ k ≤ N, 1 ≤ l ≤ L, 1 ≤ i ≤ m : w
(r,k,l) i = 1 if pk,l = ri or w (r,k,l) i = 0 otherwise, where m = 2n + 3 if r = ⊥ or m = 2n + 1 otherwise. (2) ∀1 ≤ k ≤ N, 1 ≤ l ≤ L : α(k)r = 1. Theorem 1. Suppose [σ(F(x, y, d))]r = 1 if the fact (x, r, y) is predicted to be true in document d, or [σ(F(x, y, d))]r = 0 otherwise. Let R† = R+ if r = ⊥ orR† = R otherwise, Gd = {(x, r, y) ∈ Ed × R† × Ed | [σ(F(x, y, d))]r = 1} be the set of predicted true facts for d, Σ = {Rk}1≤k≤N a set of r-specific L-CRs and θ(N,L)r the Σ-induced parameter assignment of the rule reasoning module. Then for any fact (a, r, b) ∈ Ed×R†×Ed, s(N,L)r,a,b,d ≥ 1 if and only if Gd |=Σ (a, r, b).
The proof of Theorem 1 is provided in Appendix A. Theorem 1 enables us to extract explainable logical rules from the parameter assignment of the learnt neural module. The rule extraction algorithm is shown in Appendix B. Residual connection. Considering that there exist DocRE scenarios where logical reasoning is useless, we introduce the well-known residual connection mechanism to incorporate the output logits from the original DocRE model and the estimated truth degrees from the rule reasoning module. The
ultimately predicted logit is calculated by:
ϕ(x,y,d)r = [F(x, y, d)]r + s(N,L)r,x,y,d (3)

heading: 4.3 Training Objective
text: JMRL is trained by minimizing a classification loss (BCE or AT, inherited from the backbone DocRE model) calculated by ϕ(x,y,d)r . The formal definitions of BCE and AT are given in Appendix C.
In practice, it is hard to train the rule reasoning module accurately at the early stage of training, as the facts predicted by the backbone DocRE model are inaccurate at the early stage. To tackle this issue, we introduce an auxiliary loss in JMRL to improve the efficiency of the entire training process. The classification loss on the output logits F(x, y, d) of the backbone DocRE model is treated as the auxiliary loss. By L1∆ and L2∆ we denote the auxiliary loss and the original loss, respectively, the entire JMRL-enhanced model is trained by minimizing λL1∆ + L2∆, where ∆ ∈ {BCE,AT} and λ is a hyper-parameter to trade-off the two losses.

heading: 5 Evaluation

heading: 5.1 Experimental Setup
text: Datasets and metrics. We used four benchmark datasets DWIE, DocRED, Re-DocRED, and DocGNRE for evaluation. To fairly compare with MILR on DocRED, we used the same relabeled test set as Huang et al. (2022). Statistical details for these datasets are reported in Table 1. Following Yao et al. (2019), we used F1-score and Ign F1-score as evaluation metrics, where Ign F1score extends F1-score by omitting facts appearing in the intersection of the training set and the dev set (resp. test set) for evaluation on the dev set (resp. the test set).
Baselines. To compare JMRL with the SOTA rulebased frameworks LogicRE (Ru et al., 2021) and MILR (Fan et al., 2022), we enhanced four baseline models, including LSTM (Yao et al., 2019), Bi-LSTM (Yao et al., 2019), GAIN (Zeng et al., 2020) and ATLOP (Zhou et al., 2021a). For a more comprehensive comparison, we also applied JMRL to enhance the SOTA model DREEAM (Ma et al., 2023a) and compared with other SOTA methods SSAN (Xu et al., 2021), DocuNet (Zhang et al., 2021) and KD-DocRE (Tan et al., 2022a). Note that these baseline models adopt different loss functions, where the BCE loss is used by LSTM, BiLSTM and GAIN, and the AT loss is used by ATLOP and DREEAM. We also compared JMRL with large language models (LLMs) such as ChatGPT (Han et al., 2023), GPT-4 (Peng et al., 2023) and FLAN-UL2 (Peng et al., 2023). Implementation details. We implemented all JMRL-enhanced models by Pytorch 2.0.0 on an NVIDIA A100 GPU2. We utilized the public repositories of backbone models such as LSTM and BiLSTM3, GAIN4, ATLOP5, and DREEAM6 to implement our experiments. The hyper-parameter λ for JMRL is set to 1 in all experiments. We provide detailed hyper-parameter settings in Appendix D, where all hyper-parameters were tuned to maximize the Ign F1-score on the dev set.
2Code and data about our implementations are available at: https://github.com/qikunxun/JMRL
3https://github.com/thunlp/DocRED 4https://github.com/DreamInvoker/GAIN 5https://github.com/wzhouad/ATLOP 6https://github.com/YoumiMa/dreeam

heading: 5.2 Main Results
text: We use JMRL-X (resp. LogicRE-X or MILR-X) to denote the enhanced models, where X denotes an original DocRE model. Table 2 (resp. Table 3) reports the comparison results on the DWIE (resp. DocRED) dataset. where the results of baselines in Table 3 are sourced from (Fan et al., 2022). Results show that the proposed JMRL framework improves all original DocRE models by a significant margin in both F1-scores and Ign F1-scores with p-values < 0.05 by two-tailed t-tests. These results demonstrate a ubiquitous effectiveness of JMRL across a variety of backbone models which use different kinds of word embeddings, language models and loss functions. Furthermore, we can observe that JMRL consistently outperforms both the SOTA rule-based frameworks LogicRE and MILR. Specifically, JMRL-ATLOP outperforms MILR-ATOP by a significant margin of 1.08% (resp. 3.02%) in Ign F1-score on the DWIE (resp. DocRED) dataset.
This is in line with our expectation that a joint training framework (e.g. JMRL) is better than a pipeline framework (e.g. LogicRE and MILR) due to the mitigation of error propagation. From the results reported in Table 4 for comparing with the SOTA DocRE model DREEAM, we see that JMRLDREEAM achieves new SOTA performance on DocRED, namely 67.91% (resp. 65.69%) in F1-score (resp. Ign F1-score). This improvement beyond SOTA is also statistically significant with a p-value < 0.05. This confirms that JMRL is able to further enhance SOTA DocRE models.
Besides, we also compared JMRL with LLMs, including ChatGPT, GPT-4 and FLAN-UL2 (FT). The comparison results reported in Table 2,3 and 4 show that LLMs achieve relatively lower performance on both DWIE and DocRED, even though
they were fine-tuned on the training data. The reasons are two-fold. On one hand, LLMs like ChatGPT and GPT-4 can hardly make full use of the training data for adapting to a new task. On the other hand, LLMs are generative models that are too general to fit the DocRE task, which is a classification task, while JMRL-enhanced models that are discriminative models. We provide more detailed discussions on LLMs in Appendix E.
It is worth noting that we have conducted comparisons on Re-DocRED (Table 3), following the setting used in MILR for a fair comparison. This setting uses the training set of DocRED for training and the test set of Re-DocRED for test. To further verify the effectiveness of JMRL, we conducted experiments on the Re-DocRED dataset under the original setting, as reported in Table 6. Results show that the proposed JMRL framework pushes DREEAM by an absolute gain of 0.67% (resp. 0.64%) in terms of F1-scores (resp. Ign F1-scores). These results demonstrate that JMRL is able to enhance the SOTA DocRE method DREEAM on the Re-DocRED dataset under the original setting.
Furthermore, we also conducted experiments on DocGNRE, which is a new dataset that constitutes three training sets and a test set, where two of three
training sets are enhanced by distant supervision using the large language model ChatGPT, and the test set is enhanced by distant supervision using ChatGPT and further revised by human annotators. The comparison results are reported in Table 5. We can observe that JMRL-DREEAM is able to consistently outperform DREEAM for all settings in F1 scores on DocGNRE, except for a case where DREEAM is trained on Re-DocRED (mGPT) and employs RoBERTalarge to calculate contextualized representations. These results further confirm the effectiveness of the proposed JMRL framework. Besides, we observe from the comparison results that the use of distant supervision data cannot further improve the performance of JMRL-DREEAM. The reason may lie in two-fold. On one hand, the external knowledge inside the distant supervision data from ChatGPT may be covered by JMRL. On the other hand, the distant supervision data from ChatGPT introduces noise to the training data, leading to performance degradation.

heading: 5.3 Analysis
text: Ablation study. Table 7 reports our results for ablation study. In the first variant model, we omitted the residual connection mechanism in JMRL. Results show that the performance of this variant significantly drops with p-value=8.1e-4 by a two-tailed t-test. In the second variant model, we omitted the auxiliary loss in JMRL. Results show that discarding the auxiliary loss results in a significant performance drop with p-value=2.2e-2. These results demonstrate the effectiveness of the two key components in JMRL. For the third and the fourth variant models, we respectively altered the rule reasoning module by the well-known end-to-end rule learning models NeuralLP (Yang et al., 2017) and DRUM (Sadeghian et al., 2019). Results show that the original JMRL-ATOP significantly outperforms these two variants with p-values< 0.05. The reason why our proposed rule reasoning module
outperforms both NeuralLP and DRUM may lie in the fact that both NeuralLP and DRUM introduce an extra LSTM network to express the relevance of weights for predicate selection in adjacent body atoms, while this extra component introduces more parameters that can hardly be optimized by noisy facts coming from the backbone DocRE model. Analysis on long-range dependencies. To verify whether logical rules are beneficial for capturing long-range dependencies between entity mentions, we separate the set of entity pairs into four groups according to the distances between entity pairs, where the distance between two entities is measured by the minimum number of tokens between the mentions of these two entities in a document. Figure 3 shows the comparison results on the DWIE dev set. We can see that JMRL-ATLOP consistently outperforms all baselines in all four groups. Moreover, the performance generally decreases with increasing distances. However, JMRLATLOP achieves better performance in the range [100, 200) than in the range [0, 100). These results imply that JMRL is more effective in capturing long-range dependencies between entity mentions. Analysis on the hyper-parameter λ. We conducted analysis on the hyper-parameter λ, where the experiments were conducted on the dev set of DocRED, based on JMRL-ATLOP. Figure 5 illustrates the comparison results. It can be observed that both F1-score and Ign F1-score only moderately fluctuate when λ ranges from 0 to 1.8, and that both of them reach the maximum when λ = 1.0. Therefore, we set λ = 1.0 in all our experiments. Analysis on the learnt distribution. We analyzed the learnt distribution ofw(r,k,l) on the DWIE dataset. Specifically, we utilized the mean symmetric Kullback-Leibler divergence (KLD) score betweenw(r,k,l) and the uniform distribution to rep-
resent the distribution of w(r,k,l). Note that w(r,k,l) is initialized randomly. The mean symmetric KLD score between w(r,k,l) and the uniform distribution is 2.1e-5 at the initial stage of the training phase, and increases to 4.3 after the training phase is done. These results indicate that w(r,k,l) becomes imbalanced during training, implying that the model tends to learn a discrete distribution of weights for selecting predicates in logical rules.

heading: 5.4 Case study
text: We conducted a case study for comparing MILRATLOP with JMRL-ATLOP on the DWIE test set, as shown in Figure 4. We first introduce a metric δ to estimate, in the residual connection, the ratio of the degree that the rule-enhance logit dominates the ultimately predicted logit to the degree that the DocRE logit dominates the ultimately predicted logit; formally, δ = dis(vori, vori + vrule)/dis(vrule, vori + vrule), where vori and vrule denote the DocRE logit and the rule-enhanced logit, respectively, and dis is the Euclidean distance function. In the first case, MILP-ATLOP fails to predict the true relation “head_of” be-
tween “Angela Merkel” and “Christian Democrats (CDU)”, whereas JMRL-ATLOP predicts this true relation. The correct prediction of JMRL-ATLOP can be explained by a rule “head_of(x, y) ← head_of_gov(x, z) ∧ base_in−(z, y)” extracted from the parameter assignment of the rule reasoning module, while MILP-ATLOP fails to discover this rule. In the second case, MILPATLOP predicts two false relations between “Gadhafi” and “Iran”, whereas JMRL-ATLOP predicts true relations between “Gadhafi” and “Libya”. Although both MILP-ATLOP and JMRLATLOP may discover the rule “citizen_of(x, y)← head_of_state(x, y)”, MILP-ATLOP propagates the false relation “head_of_state” between “Gadhafi” and “Iran” to final predictions, while JMRLATLOP can avoid error propagation by its end-toend nature. Besides, JMRL-ATLOP has δ > 4 in both cases, implying that it is the rule reasoning module that dominates the ultimate prediction.

heading: 6 Conclusion and Future work
text: In this paper we have proposed an end-to-end learning framework named JMRL to empower existing DocRE models with stronger reasoning abilities. Notably, we have proposed a novel rule reasoning module in JMRL to simulate the inference of logical rules, thereby enhancing the reasoning ability. Furthermore, we have shown theoretically that the parameterization of this module is faithful to the formalization of logical rules. Experimental results on four benchmark datasets verify the effectiveness of JMRL. Future work will extend JMRL to jointly learn named entity recognition (NER), DocRE and more expressive rules in an end-to-end fashion.

heading: 7 Limitations
text: There may be two main limitations of JMRL. On one hand, the rule reasoning module in JMRL simulates the inference of chain-like logical rules. However, chain-like logical rules may not be sufficiently expressive in some complex reasoning scenarios, e.g., they cannot express type constraints (Wu et al., 2022) on individual entities. The limited expressivity of chain-like logical rules may impair the reasoning ability of JMRL. On the other hand, JMRL is a rule-based framework for enhancing the DocRE task, whereas the task of DocRE requires a set of entities involved in the given document as input. Therefore, applying JMRL to the real-world scenarios requires a preprocess of named entity recognition (NER). Errors coming from an imperfect NER model may propagate to JMRL, resulting in performance degradation. We will make up for the above deficiencies in future work, by extending JMRL to learn more expressive logical rules and extending JMRL to jointly train an NER module.

heading: 8 Ethics Statement
text: JMRL is a SOTA solution for the DocRE task with high effectiveness and interpretability. Therefore, it is possible to apply JMRL to revealing personal privacy. To mitigate this concern, we only use public benchmark datasets for evaluation. These datasets do not involve personal privacy. We also advocate not applying the proposed JMRL framework to extract and analyze any private information without user authorization.

heading: 9 Acknowledgements
text: This paper was supported by National Natural Science Foundation of China (No. 62276284, 61976232, 61876204, 51978675), Guangdong Basic and Applied Basic Research Foundation (No. 2023A1515011470, 2022A1515011355), Guangzhou Science and Technology Project (No. 202201011699), Shenzhen Science and Technology Program (KJZD2023092311405902), Guizhou Provincial Science and Technology Projects (No. 2022-259), Humanities and Social Science Research Project of Ministry of Education (No. 18YJCZH006), as well as the Fundamental Research Funds for the Central Universities, Sun Yatsen University (No. 23ptpy31).

heading: A Proof
text: A.1 Proof of Lemma 1 To prove Theorem 1, we first introduce Lemma 1.
Lemma 1. Suppose [σ(F(x, y, d))]r = 1 if the fact (x, r, y) is predicted to be true in document d, or [σ(F(x, y, d))]r = 0 otherwise. Let R† = R+ if r = ⊥ orR† = R otherwise, Gd = {(x, r, y) ∈ Ed × R† × Ed | [σ(F(x, y, d))]r = 1} be the set of predicted true facts for d, R an r-specific L-CR of the form r(x, y)← r1(x, z1)∧ r2(z1, z2)∧ ...∧ rL(zL−1, y), and θ (1,L) r the {R}-induced parameter assignment of the rule reasoning module in JMRL. Then for any fact (a, r, b) ∈ Ed ×R† × Ed, we have: (1) s(1,L)r,a,b,d ≥ 1 if Gd |= HR(a, b), and (2) s (1,L) r,a,b,d = 0 if Gd ̸|= HR(a, b).
Proof. Let Kd = Gd ∪ G−d ∪ {(e, I, e) | e ∈ Ed}, where Ed is the set of entities appearing in Gd.
(I) Consider the case where Gd |= HR(a, b). There exists at least one ground instance Rg of R such that HR(a, b) = HRg and BRg ⊆ Kd. There will be a sequence of entities c1, . . . , cL−1 and a sequence of relations r1, . . . , rL such that (a, r1, c1), (c1, r2, c2). . . , (cL−1, rL, b) ∈ Kd. Suppose r1 is the kth relation inR†∪R−† ∪{I}, then by Condition 1 in Definition 1, we have w(r,1,1)k = 1 for some k. By Equation (1), we further have s (1,1) r,a,c1,d
≥ 1. Likewise, suppose r2 is the kth relation inR†∪R−† ∪{I}, then by Condition 1 in Definition 1, we have w(r,1,2)k = 1. By Equation (1), we further have s(1,2)r,a,c2,d ≥ 1. In the same way, we can show that s(1,3)r,a,c3,d ≥ 1, . . . , s (1,L−1) r,a,cL−1,d ≥ 1 and
s (1,L) r,a,b,d ≥ 1 in turn. Therefore, we have s (1,L) r,a,b,d ≥ 1 if Gd |= HR(a, b). (II) Consider the case where Gd ̸|= HR(a, b). Suppose s(1,L)r,a,b,d ≥ 1, then by Equation (1), there must be some k ∈ {1, . . . ,m} such that w(r,1,1)k = 1, where m = 2n + 3 if r = ⊥ or 2n + 1 otherwise, there exists (a, rk, c1) ∈ Kd fulfilling s (1,1) r,a,c1,d
≥ 1. Since s(1,1)r,a,c1,d ≥ 1, by Equation (1), there must be also some k ∈ {1, . . . ,m} such that w
(r,1,2) k = 1, where m = 2n + 3 if r = ⊥ or 2n + 1 otherwise, there exists (c1, rk, c2) ∈ Kd fulfilling s(1,2)r,a,c2,d ≥ 1. In the same way, we can show that there exists relation ru and entity cu such that (cu−1, ru, cu) ∈ Kd and s(1,u)r,a,cu,d ≥ 1 for u = 3, . . . , L − 1 in turn, while there exists relation rL such that (cL−1, rL, b) ∈ Kd. Hence there exists a sequence of entities c1, . . . , cL−1 and a sequence of relations r1, . . . , rL such that (a, r1, c1), (c1, r2, c2). . . , (cL−1, rL, b) ∈ Kd. These two sequences constitute a ground instance Rg of R such that HR(a, b) = HRg and BRg ⊆ Kd, contradicting Gd ̸|= HR(a, b). Thus s(1,L)r,a,b,d < 1. By Equation (1), Condition 1 in Definition 1 and ∀(x, r, y) ∈ Ed × R† × Ed : [σ(F(x, y, d))]r ∈ {0, 1}, we further have s(1,L)r,a,b,d = 0. Therefore, we have s(1,L)r,a,b,d = 0 if Gd ̸|= HR(a, b).
A.2 Proof of Theorem 1
Proof. Lemma 1 implies that, for all Rk ∈ Σ, s (k,L) r,a,b,d ≥ 1 if Gd |= HRk(a, b) and s (k,L) r,a,b,d = 0 otherwise. (⇒) Suppose s(N,L)r,a,b,d ≥ 1. Then by Equation (2) and Condition 2 in Definition 1, there exists at least one r-specific L-CR Rk ∈ Σ such that s(k,L)r,a,b,d ≥ 1. By Lemma 1 we have Gd |= HRk(a, b). Since Gd |= HRk(a, b) and Rk ∈ Σ, we have Gd |=Σ (a, r, b).
(⇐) Suppose Gd |=Σ (a, r, b). Then we have Gd |= HRk(a, b) for some Rk ∈ Σ. By Lemma 1 we have s(k,L)r,a,b,d ≥ 1 and for all k′ ̸= k, s (k′,L) r,a,b,d ≥ 0. By Equation (2) and Condition 2 in Definition 1, we have s(N,L)r,a,b,d ≥ 1.

heading: B Rule Extraction
text: Based on the theoretical result of Theorem 1, we can interpret chain-like rules (CRs) from the parameter assignment of the rule reasoning module in JMRL. The process of interpretation is shown in Algorithm 1. Intuitively, Algorithm 1 interprets
CRs from the parameter assignment of the rule reasoning module in JMRL using beam search, where b is the beam size, fl is the set of (R′, ψ)-pairs for the lth atom, and where R′ is the currently interpreted (partial) rule and ψ its estimated score. It should be noted that the process for interpreting rspecific L-CRs outputs up to b interpreted rules for a target rule, where all interpreted rules for the kth target rule share the same confidence score α(k)r .

heading: C Formalization of Loss Functions
text: Due to the space limitation, we omit the detailed formalization of the BCE loss function and the AT loss function in Section 4. In the following, we supplement these formalizations as follows.
Let D = {di}1≤i≤ND be the set of documents for training, Ed the set of mentioned entities in document d ∈ D, and Gd = {(eh, r, et)i}1≤i≤NGd the set of annotated facts in document d ∈ D, where eh, et ∈ Ed, r ∈ R+, ND denotes the number of documents in D, and NGd the number of facts in Gd. Then the BCE loss function J (x,y,d)BCE for the entity pair (x, y) in document d is defined as J (x,y,d)BCE = − ∑
r∈R+ I((x, r, y) ∈ Gd) log σ(ϕ(x,y,d)r )
+ I((x, r, y) /∈ Gd) log(1− σ(ϕ(x,y,d)r )) (4)
where σ denotes the sigmoid function, and I(C) is an indicator function that returns 1 if C is true or 0 otherwise. The adaptive thresholding (AT) loss J (x,y,d)AT for the entity pair (x, y) in d is defined as
J (x,y,d)AT = − ∑
r∈Rpos log
exp(ϕ (x,y,d) r )∑
r′∈Rdpos∪{⊥} exp(ϕ (x,y,d) r′ )
− log exp(ϕ (x,y,d) ⊥ )∑
r′∈Rdneg∪{⊥} exp(ϕ (x,y,d) r′ )
(5) where Rdpos = {r | (x, r, y) ∈ Gd, r ∈ R} and Rdneg = {r | (x, r, y) /∈ Gd, r ∈ R}. Then the entire loss function is calculated by:
L∆ = ∑
d∈D
∑
x,y∈Ed,x ̸=y J (x,y,d)∆ (6)
where ∆ ∈ {BCE,AT}.

heading: D Hyper-parameter Details
text: To help reproduce our results, we provide the hyperparameter settings used in our experiments. Table 8 reports the detailed hyper-parameter settings
Algorithm 1: Interpreting r-specific LCRs
1 Input: beam size b ≥ 1 and a parameter assignment of the rule reasoning module in JMRL for the head relation r, namely θ (N,L) r = {w(r,k,l)i }1≤k≤N,1≤l≤L,1≤i≤m ∪ {α(k)r }1≤k≤N where m = 2n+ 3 if r = ⊥ or m = 2n+ 1 otherwise. 2 Output: a set of up to bN r-specific L-CRs 3 R← ∅; 4 for 1 ≤ k ≤ N do 5 f0 ← {(∆L, 1)} where ∆ denotes a placeholder to be filled; 6 ∀1 ≤ l ≤ L : fl ← ∅; 7 for 1 ≤ l ≤ L do 8 for (R,ψ) ∈ fl−1 do 9 for 1 ≤ i ≤ m do 10 R′ ← R with the lth placeholder replaced with ri; 11 fl ← fl ∪ {(R′, w(r,k,l)i ψ)};

heading: 12 sort fl = {(R,ψ)j}1≤j≤bm in the
text: descending order of ψ and preserve the top-b in fl; 13 Q← {R′ rewritten from R to the form of a CR | (R,ψ) ∈ fL} ; 14 R← R ∪Q; 15 return R;
in regard to different baseline models and datasets. These hyper-parameters are set to maximize the Ign F1-scores on the dev set.

heading: E Discussion on LLMs
text: In this section, we provide detailed discussions on comparing JMRL with the current SOTA LLMs, including ChatGPT, GPT-4, Davinci and FLANUL2. Table 9 reports the comparison results on the DocRED dataset, where the results of LLMs are sourced from (Peng et al., 2023). Results show that there is a huge performance gap between the SOTA LLMs and JMRL-DREEAM on DocRED. We can also observe that the performance of FLAN-UL2 significantly improves after being fine-tuned on the training data. It implies that LLMs with few-shot ICL can hardly leverage the full domain knowledge within the training data. Besides, it can also be observed that JMRL-DREEAM still significantly outperforms FLAN-UL2 even after FLAN-UL2 was
fine-tuned on the training data. The reasons may be two-fold. On one hand, FLAN-UL2 is too general to fit the DocRE task, which is a classification task, when compared with JMRL-enhanced models that are discriminative models. There is a significant gap between the generative training objective and the discriminative training objective for classification tasks. On the other hand, LLMs inherently suffer from the hallucination issue (Ji et al., 2023), e.g., LLMs may generate unexpected relations as the final predictions. This issue cannot be fully addressed by fine-tuning on the training data. In summary, these comparison results demonstrate that JMRL remains an effective solution for the DocRE task with SOTA performances on benchmark datasets. Furthermore, compared with LLMs, the JMRL-enhanced models have evident advantages in terms of memory cost and inference speed.
Nevertheless, combining JMRL with large language models is a promising way to further improve performances. For example, the work (Ma et al., 2023b) has shown that few-shot ICL for LLMs cannot generalize well in the IE tasks, but they found that LLMs are able to address some hard examples. This provides us with an innovate way to combine JMRL with LLMs, by employing the JMRL enhanced model to deal with most simple cases, and employing LLMs to handle some hard
examples. We argue that such combination is able to help JMRL to generalize in more knowledgeintensive scenarios. Besides, the work (Luo et al., 2023) has shown that LLMs like ChatGPT can generate logical rules for reasoning, by leveraging the relational paths as input. We argue that the induced logical rules from ChatGPT can be used to initialize the parameter assignment of the rule reasoning module in JMRL. This process is possible to help JMRL learn more logical rules for reasoning, resulting in better convergence and performance.

heading: F Analysis on Model Efficiency
text: JMRL introduces external parameters to learn logical rules. To clarify whether JMRL is efficient in the DocRE task, we analyzed the model efficiency. First, we compared the parameter sizes of ATOP, JMRL-ATOP and other variant models, as reported in Table 11. It can be seen that JMRL-ATLOP introduces only 1.9% extra model parameters, while the other variants of JMRL-ATLOP that use NeurlLP or DRUM as the rule reasoning module require to introduce more than 30% extra model parameters. These results indicate that JMRL is parameter ef-
ficient. Second, we compared the inference time of ATOP, JMRL-ATOP and other variant models. Figure 6 illustrates the comparison results between different methods on the average inference time in seconds. It can be seen that the introduction of JMRL increases the inference time by about 0.03 seconds, whereas both the two variants of JMRL increase the inference time by about 0.3 seconds. These results imply that JMRL is able to significantly improve performance of the DocRE task with a small overhead on the inference time.
In addition, we also analyzed the time complexities of the rule reasoning module in JMRL and other rule-based methods. Specifically, the time complexity of the rule reasoning module of JMRL in the inference phase is O(nNL(2n + 1)|E|2), where n = |R∗|. By a parallel implementation, the amortized time complexity reduces to O(nNL(2n + 1)). The corresponding time complexity of the baseline method LogicRE is O(nNL(2n+ 1)|E|2 + nNLd2(2n+ 1)), where d is the hidden size. The additional part O(nNLd2(2n+ 1)) comes from the Transformer network used in LogicRE for rule generation. Note that LogicRE is a path-based method for rule reasoning, thus it has no parallel implementation for rule reasoning. The amortized time complexity reduces toO(nNL(2n+1)|E|2+nNLd2(2n+1)) due to the parallelization of Transformer. Besides, the time complexity of existing rule learn-
ing methods such as NeuralLP and DRUM is O(nNL(2n + 1) + nNLd2(2n + 1)) due to the use of LSTM for calculating w(r,k,l)i . This analysis shows that the rule reasoning module of JMRL is more efficient than other rule-based methods.

heading: G Case Study of Learnt Rules
text: We showcase in Table 10 some logical rules extracted from the parameter assignment of the rule reasoning module in JMRL-ATLOP for both the DWIE and DocRED datasets. These rules are extracted by applying Algorithm 1 with the beam size set to 100 and then simplified by omitting identity body atoms. The weight of each rule is sourced from α(r)r in Equation (2). It can be observed that expressive logical rules with different weights and different numbers of body atoms can be extracted for both the DWIE and DocRED datasets. Moreover, some rules for inferring the head relation ⊥ can also be discovered by JMRL, see the fifth rule for DWIE and the fifth rule for DocRED. It should be noted that LogicRE and MILR do not learn rules for the head relation ⊥. The introduction of logical rules for the head predicate ⊥ could make the prediction of no-relation between two entities more accurate since extra information is exploited. This is also a potential reason for explaining why JMRL outperforms both LogicRE and MILR.

heading: H Analysis on the Impacts of L and N
text: We conducted an analysis to illustrate the impacts of the hyper-parameters L and N on the performance for DocRE. More specifically, we first created several variants of JMRL-ATLOP with different settings of N and L, and then evaluated their performance on DWIE and DocRED. The comparison results are reported in Table 12. It can be
observed that JMRL-ATLOP achieves the best performance with L = 2 and N = 20. Note that the rule-based method MILR also uses L = 2 in experiments. Furthermore, we argue that N = 20 does not mean that JMRL learns less rules than MILR. The reasons are two-fold. On one hand, MILR barely mined 90 (resp. 168) rules for DWIE (resp. DocRED). 7 In contrast, the parameters of JMRL involve N |R+| rules, i.e., 20 × 66 = 1320 rules for DWIE and 20× 97 = 1940 rules for DocRED. On the other hand, as shown in Algorithm 1, JMRL extracts up to bN r-specific CRs for each head relation r, where b denotes the beam size. Therefore, JMRL extracts totally 1320b and 1940b rules for DWIE and DocRED, respectively. This indicates that JMRL is able to learn more rules than existing rule-based approaches.
7https://github.com/XingYing-stack/MILR/tree/ main/mined_rules

heading: I Error Analysis
text: We conducted an error analysis to further explore the limitations of the proposed method. In more detail, we analyzed the prediction results of ATLOP and JMRL-ATLOP on the DWIE dev set. We found three types of errors, as illustrated in Figure 7. In the first type (see the top subfigure), the original ATLOP model correctly predicts the target relation, but JMRL-ATLOP fails. We argue that this type of errors may be caused by the imprecise rules learnt by the rule reasoning module in JMRL. For instance, the learnt rule “head_of_gov(x, y) ← citizen_of(x, z) ∧ gpe−(z, y)” may result in the incorrect prediction of the fact (FayezSarraj, head_of_gov, Libyans). In the second type (see the middle sub-figure), the original ATLOP model is unable to predict the target relation, while JMRL-ATLOP is unable either. In the third type (see the bottom sub-figure), the
original ATLOP model predicts an incorrect relation to be true and JMRL-ATLOP makes the same mistake. Regarding the latter two types, we argue that the errors may stem from the limited expressive power of chain-like rules. Therefore, our future work will focus on learning more complex logical rules, e.g., typed rules (Wu et al., 2022), to further enhance the performance of JMRL. Meanwhile, we also plan to investigate how to exploit expert knowledge (e.g., predefined rules) to mitigate the negative impacts of imprecise rules.

heading: J Discussion on More Applications
text: JMRL is an end-to-end framework for jointly learning specific neural models and logical rules. Therefore, we argue that JMRL can be used in more application scenarios where logical rules can be used. For instance, JMRL can be applied to other information extraction tasks such as document-level event argument extraction (Liu et al., 2023) and document-level event causality identification (Chen et al., 2023). Apart from information extraction, we argue that JMRL can also benefit the field of knowledge-aware recommendations (Spillo et al., 2022). The exploration of JMRL in these applications is also a part of our future work.

heading: Future Work
text: The exploration of JMRL in these applications is also a part of our future work