Deep Inductive Logic Reasoning for Multi-Hop Reading Comprehension (2022)

Abstract: Multi-hop reading comprehension requires an ability to reason across multiple documents. On the one hand, deep learning approaches only implicitly encode query-related information into distributed embeddings which fail to uncover the discrete relational reasoning process to infer the correct answer. On the other hand, logic-based approaches provide interpretable rules to infer the target answer, but mostly work on structured data where entities and relations are well-defined. In this paper, we propose a deep-learning based inductive logic reasoning method that firstly extracts query-related (candidate-related) information, and then conducts logic reasoning among the filtered information by inducing feasible rules that entail the target relation. The reasoning process is accomplished via attentive memories with novel differentiable logic operators. To demonstrate the effectiveness of our model, we evaluate it on two reading comprehension datasets, namely WikiHop and MedHop.

## introduction reasoning has been extensively studied in the structured domain, e.g., knowledge base completion which infers missing facts given background entities and relations. however, when the background knowledge is expressed in natural languages, as shown in the multi-hop reading comprehension problem with triplet-form questions @xcite , it becomes difficult to conduct complex reasoning because the entities and relations are not explicitly labeled in the documents. for example, consider the question "country(moonhole, ?)", given the following documents: "moonhole is a private community on the island of bequia. moonhole was founded by thomas and gladys johnston in the 1960s." "gladys johnston was born in united states." "bequia is an island and is part of the country of saint vincent and the grenadines" in this example, the underlined entities are used to infer the correct answer, i.e., "country(moonhole, saint vincent and the grenadines)", but are not explicitly annotated for relational reasoning. deep neural networks (dnns) for multi-hop reading comprehension (rc) can be summarized into following three categories. 1) memory-based models @xcite @xcite that produce queryaware context representations. 2) graph-based approaches @xcite that use graph neural networks to propagate information based on pre-constructed entity (context) graphs. 3) neural module networks @xcite that decompose the question into a series of action modules @xcite . however, dnns only implicitly encode relevant contexts and fail to explicitly uncover the underlying relational compositions for complex inference. for instance, in the above example, dnns may encode bequia and gladys johnson into 1-hop features, given the fact that both entities co-occur with the query moonhole. as a result, the model may predict united states by linking it with gladys johnson instead of the correct answer saint vincent and the grenadines. in contrast, human beings would easily produce the correct answer given the knowledge "if a is in b and b is part of country c, then a is in country c" and by examining the relations between each entity pair co-occurred in the context. inductive logic programming (ilp) @xcite aligns with human reasoning by inducing interpretable rules to entail positive but not negative examples. to answer the previous query, ilp could generate a rule as located_in(x, z) ∧ country(z, y ) ⇒ country(x, y ). combining deep learning with ilp is a promising direction to benefit from both worlds @xcite . deep logic models have been proposed for structured knowledge base comple-tion @xcite @xcite @xcite . however, it becomes much more challenging when dealing with natural language inputs, as in the case of multi-hop reading comprehension. @xcite proposed to combine a symbolic reasoner: prolog, with weak unifications based on distributed embeddings as a backwardchaining theorem prover to induce feasible rules for multi-hop reasoning. however, their work relies on the degree of precision for pre-extracted ners and is limited by the number of rule templates. to address the aforementioned limitations, we propose a novel end-to-end integration of deep learning and logic reasoning termed deep inductive logic reasoning (dilr). it consists of two components: 1) a hierarchical attentive reader that filters query-related and candidate-related information from given documents, and 2) a multihop reasoner that conducts inductive logic reasoning by attentively selecting proper predicates to form candidate rules and refines them upon given examples. we introduce novel differentiable logic operators combined with attention mechanisms for smooth back-propagation. compared to existing deep logic models, we build connections between raw text inputs and the symbolic domain by mapping high-level semantic representations to logic predicates and instantiating logic variables with neural representations to conduct relational reasoning. we also parameterize the entire process for end-to-end differentiable learning. the contributions of this work include: 1) we introduce a novel smooth connection between deep representation learning with logic reasoning by associating distributed representations with discrete logic predicates and their probabilistic evaluations. ## related work multi-hop reading comprehension recent works for multi-hop rc include memory-based methods which apply attentions to iteratively update query and context representations considering their interactions @xcite @xcite @xcite . to explicitly incorporate entity connections, de cao et al. (2019), ding et al. (2019), qiu et al. (2019), tang et al. (2020), song et al. (2018) and @xcite proposed to build entity graphs and apply graph neural networks for information propagation. @xcite formalized reasoning as a path-finding problem with neural encoding to rank candidate paths. path modeling was also adopted in @xcite using pointer networks. however, these approaches only focus on local information without the ability to generalize, and some of them rely on off-the-shelf ner tools. @xcite proposed to convert texts into a virtual knowledge base for retrieval using a pre-constructed entity database. another research direction is to decompose target questions into subquestions @xcite or sub-modules parameterized with neural module networks @xcite @xcite which also fail to explicitly uncover the underlying logic for reasoning. deep learning with logic reasoning neurosymbolic learning aims to integrate deep learning's ability on dealing with uncertainty and logic programming's ability on reasoning. deep neural networks have been used to parameterize discrete logic operators and logic atoms @xcite @xcite @xcite @xcite given the logic rules. a more challenging direction is inductive logic programming that automatically learns rules through representation learning and differentiable backpropagation @xcite @xcite . neuro-symbolic learning has been applied to knowledge-base completion through logic embeddings @xcite , tensor operations (cohen, 2016; @xcite , adversarial learning @xcite , variational learning @xcite or attentions @xcite . differentiable theorem proving has also been proposed with weak unifications and backward chaining @xcite @xcite . however, unlike multi-hop rc, knowledge-base completion only takes structured inputs without the need to address language ambiguity. the most related work to ours is nlprolog @xcite , a neural theorem prover for multi-hop rc by con-verting language utterances to distributed embeddings. however, nlprolog relies on a ner tool to extract entities and its expressiveness is limited by the number of rule templates. ## background we focus on multi-hop reading comprehension tasks containing explicit query types which align with the standard ilp setting. formally, for each rc problem, we are given a set of documents @xmath0, ..., c k }, a structured query in the form of a relational triplet (s, q, ?), where s denotes the subject of the relation q, and a list of candidate answers @xmath1, ..., a n }. the task is to select an answer @xmath2, a) is satisfied, i.e., a is the object of relation q given the subject s. for example, country(m oonhole, ?) is a query asking for the country where moonhole is located. this task could be converted into an ilp problem with the formal definition as follows. definition 3.1 (inductive logic programming). given a logic theory b representing the background knowledge (facts), a set of positive examples e + and a set of negative examples e -, an ilp system aims to derive a hypothesis h which entails all the positive and none of the negative examples: the hypothesis h is a logic program consisting of definite clauses b 1 ∧ ... ∧ b n ⇒ h where b 1 , ..., b n and h are logic atoms. the lhs of "⇒" is the clause body and h is the head. an atom is composed of a predicate and its arguments, e.g., @xmath3, y ) with predicate "located_in" and arguments x, y . a ground atom is obtained by instantiating variables in the arguments with constants, e.g., @xmath4. we use µ(•) to denote the value of an atom or a clause. for smooth optimization, we assign µ(•) ∈ [0, 1] which indicates the probability of the atom or clause being true. for multi-hop reading comprehension, we treat the query relation q(x, y ) as the head atom of the clauses to be induced. the correct answer here we use lower cases: s i , a + i , a - i to represent constants and upper cases: x, y to represent variables. the predicates in the logic domain correspond to pairwise relations be-tween two entities foot_0 . to differentiate the number of inference steps, we define a l-hop reasoning clause as with l denoting the number of extra arguments as bridging entities in the rule body except those in the head atom. here r denotes a predicate, i.e., a relation between x 0 and x l+1 . each subclause f t (x t , x t+1 ) can be one or a conjunction (∧) of 2-ary atoms taking only x t and x t+1 as arguments, e.g., ## methodology overall, dilr simulates a multi-hop reasoning process considering different number of inference steps. it is an end-to-end framework consisting of two components: a hierarchical attentive reader and a multi-hop reasoner. the attentive reader learns to select relevant information from the given documents to produce query-aware, candidateaware and bridging entity representations. these representations are passed to the multi-hop reasoner to instantiate logic atoms in order to generate and evaluate clauses that are relevant to the query relation. the multi-hop reasoner conducts rule induction via attentive memories that softly select atoms to form new clauses and novel differentiable logic operators that produce probabilistic values for generated clauses. the final loss can be backpropagated smoothly to update the attentive reader for more accurate selections. next, we illustrate each component with more details. ## experiment we conduct experiments on two multi-hop reading comprehension datasets, namely wikihop and medhop @xcite . the wikihop dataset contains 43,738 training and 5,129 development instances ranging over 277 query relations. medhop is a medical dataset containing 1,620 training and 342 development instances with a unique query relation, i.e., "interact with". for wikihop, we experiment with both non-contextual (follow @xcite ) named as dilr and contextual word embeddings (bert @xcite ) named as dilr-bert to demonstrate our model's generalization ability. for medhop, we use the same setting following @xcite . we define @xmath5: comparisons on wikihop in terms of the number of training instances for each query relation. ## conclusion we propose an end-to-end model dilr to solve the problem of multi-hop reading comprehension. dilr smoothly connects a hierarchical attentive reader with a multi-hop reasoner to conduct automatic information extraction and complex reasonkeeping those predicates with scores higher than 0.4. ing. we also introduce differentiable logic operators to induce valid clauses with smooth and stable gradient-based learning. extensive experiments reveal consistent improvements brought by dilr.