Advancing Abductive Reasoning in Knowledge Graphs through Complex Logical Hypothesis Generation (2024)

Abstract: Abductive reasoning is the process of making educated guesses to provide explanations for observations. Although many applications require the use of knowledge for explanations, the utilization of abductive reasoning in conjunction with structured knowledge, such as a knowledge graph, remains largely unexplored. To fill this gap, this paper introduces the task of complex logical hypothesis generation, as an initial step towards abductive logical reasoning with KG. In this task, we aim to generate a complex logical hypothesis so that it can explain a set of observations. We find that the supervised trained generative model can generate logical hypotheses that are structurally closer to the reference hypothesis. However, when generalized to unseen observations, this training objective does not guarantee better hypothesis generation. To address this, we introduce the Reinforcement Learning from Knowledge Graph (RLF-KG) method, which minimizes differences between observations and conclusions drawn from generated hypotheses according to the KG. Experiments show that, with RLF-KG’s assistance, the generated hypotheses provide better explanations, and achieve stateof-the-art results on three widely used KGs.1

heading: 1 Introduction
text: Abductive reasoning plays a vital role in generating explanatory hypotheses for observed phenomena across various research domains (Haig, 2012). It is a powerful tool with wide-ranging applications. For example, in cognitive neuroscience, reverse inference (Calzavarini and Cevolani, 2022), which is a form of abductive reasoning, is crucial for inferring the underlying cognitive processes based on observed brain activation patterns. Similarly, in clinical diagnostics, abductive reasoning is recognized as a key approach for studying cause-and-effect
*Equal Contribution 1https://github.com/HKUST-KnowComp/AbductiveKGR
relationships (Martini, 2023). Moreover, abductive reasoning is fundamental to the process of hypothesis generation in humans, animals, and computational machines (Magnani, 2023). Its significance extends beyond these specific applications and encompasses diverse fields of study. In this paper, we are focused on abductive reasoning with structured knowledge, specifically, a knowledge graph.
A typical knowledge graph (KG) stores information about entities, like people, places, items, and their relations in graph structures. Meanwhile, KG reasoning is the process that leverages knowledge graphs to infer or derive new information (Zhang et al., 2021a, 2022; Ji et al., 2022). In recent years, various logical reasoning tasks are proposed over knowledge graph, for example, answering complex queries expressed in logical structure (Hamilton et al., 2018; Ren and Leskovec, 2020), or conducting logical rule mining (Galárraga et al., 2015; Ho et al., 2018; Meilicke et al., 2019).
However, the abductive perspective of KG reasoning is crucial yet unexplored. Consider the first example in Figure 1, where observation O1 depicts five celebrities followed by a user on a social media platform. The social network service provider is interested in using structured knowledge to explain the users’ observed behavior. By leveraging a knowledge graph like Freebase (Bollacker et al., 2008), which contains basic information about these individuals, a complex logical hypothesis H1 can be derived suggesting that they are all actors and screenwriters born in Los Angeles. In the second example from Figure 1, related to a user’s interactions on an e-commerce platform (O2), the structured hypothesis H2 can explain the user’s interest in Apple products released in 2010 excluding phones. The third example, dealing with medical diagnostics, presents three diseases (O3). The corresponding hypothesis H3 indicates they are diseases V? with symptom V1, and V1 can be relieved by Panadol. From a general perspective,
1312
these problems illustrate how abductive reasoning with knowledge graphs seeks hypotheses that best explain given observations (Josephson and Josephson, 1996; Thagard and Shelley, 1997).
A straightforward approach to tackle this reasoning task is to employ a search-based method to explore potential hypotheses based on the given observation. However, this approach faces two significant challenges. The first challenge arises from the incompleteness of knowledge graphs (KGs), as searching-based methods heavily rely on the availability of complete information. In practice, missing edges in KGs can negatively impact the performance of search-based methods (Ren and Leskovec, 2020). The second challenge stems from the complexity of logically structured hypotheses. The search space for search-based methods becomes exponentially large when dealing with combinatorial numbers of candidate hypotheses. Consequently, the search-based method struggles to effectively and efficiently handle observations that require complex hypotheses for explanation.
To overcome these challenges, we propose a solution that leverages generative models within a supervised learning framework to generate logical hypotheses for given observations. Our approach involves sampling hypothesis-observation pairs from observed knowledge graphs (Ren et al., 2020; Bai et al., 2023d) and training a transformerbased generative model (Vaswani et al., 2017) using the teacher-forcing method. However, a potential limitation of supervised training is that it primarily captures structural similarities, without necessarily guaranteeing improved explanations when applied to unseen observations. To address this, we introduce a technique called Reinforcement Learning from the Knowledge Graph (RLF-KG). It utilizes proximal policy optimization (PPO) (Schulman et al., 2017) to minimize the discrepancy between the observed evidence and the conclusion derived from the generated hypothesis. By incorporating
reinforcement learning techniques, our approach aims to directly improve the explanatory capability of the generated hypotheses and ensure their effectiveness when generalized to unseen observations.
We evaluate the proposed methods for effectiveness and efficiency on three knowledge graphs: FB15k-237 (Toutanova and Chen, 2015), WN18RR (Toutanova and Chen, 2015), and DBpedia50 (Shi and Weninger, 2018). The results consistently demonstrate the superiority of our approach over supervised generation baselines and search-based methods, as measured by two evaluation metrics across all three datasets. Our contributions can be summarized as follows:
• We introduce the task of complex logical hypothesis generation, which aims to identify logical hypotheses that best explain a given set of observations. This task can be seen as a form of abductive reasoning with KGs.
• To address the challenges posed by the incompleteness of knowledge graphs and the complexity of logical hypotheses, we propose a generation-based method. This approach effectively handles these difficulties and enhances the quality of generated hypotheses.
• Additionally, we developed the Reinforcement Learning from the Knowledge Graph (RLF-KG) technique. By incorporating feedback from the knowledge graph, RLF-KG further improves the hypothesis generation model. It minimizes the discrepancies between the observations and the conclusions of the generated hypotheses, leading to more accurate and reliable results.

heading: 2 Problem Formulation
text: In this task, a knowledge graph is denoted as G = (V,R), where V is the set of vertices and R is the set of relation types. Each relation type
r ∈ R is a function : V × V → {true, false}, with r(u, v) = true indicating the existence of a directed (u, r, v) from vertex u to vertex v of type r in the graph , and false otherwise.
We adopt the open-world assumption of knowledge graphs (Drummond and Shearer, 2006), treating missing edges as unknown rather than false. The reasoning model can only access the observed KG G, while the true KG Ḡ is hidden and encompasses the observed graph G.
Abductive reasoning is a type of logical reasoning that involves making educated guesses to infer the most likely reasons for the observations (Josephson and Josephson, 1996; Thagard and Shelley, 1997). For further details on the distinctions between abductive, deductive, and inductive reasoning, refer to Appendix A. In this work, we focus on a specific abductive reasoning type in the context of knowledge graphs. We first introduce some concepts in this context.
An observation is a set O of entities in V . A logical hypothesis H on a graph G = (V,R) is defined as a predicate of a variable vertex V? in firstorder logical form, including existential quantifiers, AND (∧), OR (∨), and NOT (¬). The hypothesis can always be written in disjunctive normal form,
H|G(V?) = ∃V1, . . . , Vk : e1 ∨ · · · ∨ en, (1) ei = ri1 ∧ · · · ∧ rimi , (2)
where each rij can take the forms: rij = r(v, V ), rij = ¬r(v, V ), rij = r(V, V ′), rij = ¬r(V, V ′), where v represents a fixed vertex, the V, V ′ represent variable vertices in {V?, V1, V2, . . . , Vk}, and r is a relation type.
The subscript |G denotes that the hypothesis is formulated based on the given graph G. This means that all entities and relations in the hypothesis must exist in G, and the domain for variable vertices is the entity set of G. For example, please refer to Appendix B. The same hypothesis H can be applied to a different knowledge graph, G′, provided that G′ includes the entities and edges present in H . When the context is clear or the hypothesis pertains to a general statement applicable to multiple knowledge graphs (e.g., observed and hidden graphs), the symbol H is used without the subscript.
The conclusion of the hypothesis H on a graph G, denoted by [[H]]G , is the set of entities for which H holds true on G:
[[H]]G = {V? ∈ G|H|G(V?) = true}. (3)
Suppose O = {v1, v2, ..., vk} represents an observation, G is the observed graph, and Ḡ is the hidden graph. Then abductive reasoning in knowledge graphs aims to find the hypothesis H on G whose conclusion on the hidden graph Ḡ, [[H]]Ḡ , is most similar to O. Formally, the similarity is quantified using the Jaccard index, defined as:
Jaccard([[H]]Ḡ , O) = |[[H]]Ḡ ∩O| |[[H]]Ḡ ∪O| . (4)
In other words, the goal is to find a hypothesis H that maximizes Jaccard([[H]]Ḡ , O).

heading: 3 Hypothesis Generation with RLF-KG
text: Our methodology, referred to as reinforcement learning from the knowledge graph (RLF-KG), is depicted in Fig. 4 and involves the following steps: (1) Randomly sample observation-hypothesis pairs from the knowledge graph. (2) Train a generative model to generate hypotheses from observations using the pairs. (3) Enhance the generative model using RLF-KG, leveraging reinforcement learning to minimize discrepancies between observations and generated hypotheses.

heading: 3.1 Sampling Observation-Hypothesis Pairs
text: In the first step, we randomly sample hypotheses from the observed training knowledge graph. This process starts by randomly selecting a hypothesis, followed by conducting a graph search on the training graph to derive its conclusion, which is then treated as the observation corresponding to the hypothesis. Details of the hypothesis sampling algorithm are provided in Appendix D.
Then, we convert both hypotheses and observations into sequences suitable for the generative models. For observations, we standardize the order of the elements, ensuring that permutations of the same observation set yield identical outputs. Each entity in an observation is represented as a unique token, such as [Apple] and [Phone], as shown in Figure 2, and is associated with an embedding.
Since each hypothesis can be represented as a directed acyclic graph, for hypotheses, we use a representation inspired by action-based parsing, similar to approaches seen in other logical reasoning studies (Hamilton et al., 2018; Ren and Leskovec, 2020; Ren et al., 2020). This involves utilizing a depth-first search algorithm, described in Appendix D, to generate a sequence of actions that represents the content and structure of the graph.
Logical operations such as intersection, union, and negation are denoted by special tokens [I], [U], and [N] respectively, following prior work (Bai et al., 2023d). Relations and entities are similarly treated as unique tokens, for example, [Brand] and [Apple].
Furthermore, Algorithm 3 facilitates the reconstruction of a graph from an action sequence, serving as the detokenization process for hypotheses.

heading: 3.2 Supervised Training of Hypothesis Generation Model
text: In the second step, we train a generative model using the sampled pairs. Let o = [o1, o2, ..., om] represent the token sequences for observations, and h = [h1, h2, ..., hn] for hypotheses. The loss for the generative model ρ on this example is based on the standard sequence modeling loss:
L = log ρ(h|o) (5)
= log
n∑
i=1
ρ(hi|o, h1, . . . , hi−1). (6)
We utilize a standard transformer to implement the conditional generative model, employing two distinct approaches. The first approach follows the encoder-decoder architecture as described by Vaswani et al. (2017), where observation tokens are fed into the transformer encoder, and the shifted hypothesis tokens are input into the transformer decoder. The second approach involves concatenating the observation and hypothesis tokens and using a decoder-only transformer to generate hypotheses. Following the setup of these architectures, we train the model using supervised training techniques.

heading: 3.3 Reinforcement Learning from Knowledge Graph Feedback (RLF-KG)
text: During the supervised training process, the model learns to generate hypotheses that have similar structures to reference hypotheses. However, higher structural similarity towards reference answers does not necessarily guarantee the ability to generate logical explanations, especially when encountering unseen observations during training.
To address this limitation, in the third step,
Step 3: Optimize hypothesis generation model with Reinforcement Learning From Knowledge Graph feedback (RLF-KG).
we employ reinforcement learning (Ziegler et al., 2020) in conjunction with knowledge graph feedback (RLF-KG) to enhance the trained conditional generation model ρ. Let Gtrain represents the observed training graph, h the hypothesis token sequence, o the observation token sequence, and H,O the corresponding hypothesis and observation, respectively. We select the reward to be the Jaccard similarity between O and and the conclusion [[H]]Gtrain , which is a reliable and information leakage-free approximation for the objective of the abductive reasoning task in Equation 4. Formally, the reward function is defined as
r(h,o) = Jaccard([[H]]Gtrain , O) = |[[H]]Gtrain ∩O| |[[H]]Gtrain ∪O| .
(7) Following Ziegler et al. (2020), we treat the trained model ρ obtained from supervised training as the reference model, and initialize the model π to be optimized with ρ. Then, we modify the reward function by incorporating a KL divergence penalty. This modification aims to discourage the model π from generating hypotheses that deviate excessively from the reference model.
To train the model π, we employ the proximal policy optimization (PPO) algorithm (Schulman et al., 2017). The objective is to maximize the expected modified reward, given as follows:
Eo∼D,h∼π(·|o) [ r(h,o)− β log π(h|o)
ρ(h|o)
] , (8)
where D the is training observation distribution and π(·|o) is the conditional distribution of h on o modeled by the model π. By utilizing PPO and the modified reward function, we can effectively guide the model π towards generating hypotheses that strike a balance between the similarity to the reference model and logical coherence.

heading: 4 Experiment
text: We utilize three distinct knowledge graphs, namely FB15k-237 (Toutanova and Chen, 2015), DBpedia50 (Shi and Weninger, 2018), and WN18RR (Toutanova and Chen, 2015), for our experiments. Table 1 provides an overview of the number of training, evaluation, and testing edges, as well as the total number of nodes in each knowledge graph. To ensure consistency, we randomly partition the edges of these knowledge graphs into three sets - training, validation, and testing - using an 8:1:1 ratio. Consequently, we construct the training graph Gtrain, validation graph Gvalid, and testing graph Gtest by including the corresponding edges: training edges only, training + validation edges, and training + validation + testing edges, respectively.
Following the methodology outlined in Section 3.2, we proceed to sample pairs of observations and hypotheses. To ensure the quality and diversity of the samples, we impose certain constraints during the sampling process. Firstly, we restrict the size of the observation sets to a maximum of thirty-two elements. This limitation is enforced ensuring that the observations remain manageable. Additionally, specific constraints are applied to the validation and testing hypotheses. Each validation hypothesis must incorporate additional entities in the conclusion compared to the training graph, while each testing hypothesis must have additional entities in the conclusion compared to the validation graph. This progressive increase in entity complexity ensures a challenging evaluation setting. The statistics of queries sampled for datasets are detailed in Appendix C.
In line with previous work on KG reasoning (Ren and Leskovec, 2020; Ren et al., 2020), we utilize thirteen pre-defined logical patterns to sam-
ple the hypotheses. Eight of these patterns, known as existential positive first-order (EPFO) hypotheses (1p/2p/2u/3i/ip/up/2i/pi), do not involve negations. The remaining five patterns are negation hypotheses (2in/3in/inp/pni/pin), which incorporate negations. It is important to note that the generated hypotheses may or may not match the type of the reference hypothesis. The structures of the hypotheses are visually presented in Figure 5, and the corresponding numbers of samples drawn for each hypothesis type can be found in Table 5.

heading: 4.1 Evaluation Metric
text: Jaccard Index The quality of the generated hypothesis is primarily measured using the Jaccard index same as that defined for abductive reasoning in Section 2, but we treat the constructed test graph Gtest as the hidden graph. It is important to note that the test graph contains ten percent of edges that were not observed during the training or validation stages. Formally, given an observation O and a generated hypothesis H , we employ a graph search algorithm to determine the conclusion of H on Gtest, denoted as [[H]]Gtest . Then the Jaccard metric for evaluation is defined as
Jaccard([[H]]Gtest , O) = |[[H]]Gtest ∩O| |[[H]]Gtest ∪O| , (9)
quantifing the similarity between the conclusion [[H]]Gtest and the observation O.
Smatch Score Smatch (Cai and Knight, 2013) is originally designed for comparing semantic graphs but has been recognized as a suitable metric for
evaluating complex logical queries, which can be treated as a specialized form of semantic graphs (Bai et al., 2023d). In this task, a hypothesis can be represented as a graph, e.g. Fig. 2, and we can transform it to be compatible with the semantic graph format. The detailed computation of the Smatch score on hypothesis graphs is described in detail in Appendix F. Intuitively, the Smatch score between the generated hypothesis H and the reference hypothesis Href, denoted as S(H,Href), quantifies the structural resemblance between the graphs corresponding to H and Href i.e., how much the nodes, edges, and the labels on them look the same between the two graphs.

heading: 4.2 Experiment Details
text: In this experiment, we use two transformer structures as the backbones of the generation model. For the encoder-decoder transformer structure, we use three encoder layers and three decoder layers. Each layer has eight attention heads with a hidden size of 512. Note that the positional encoding for the input observation sequence is disabled, as we believe that the order of the entities in the observation set does not matter. For the decoder-only structure, we use six layers, and the other hyperparameters are the same. In the supervised training process, we use AdamW optimizer and grid search to find hyper-parameters. For the encoderdecoder structure, the learning rate is 0.0001 with the resulting batch size of 768, 640, and 256 for FB15k-237, WN18RR, and DBpedia, respectively. For the decoder-only structure, the learning rate is
0.00001 with batch-size of 256, 160, and 160 for FB15k-237, WN18RR, and DBpedia respectively, and linear warming up of 100 steps. In the reinforcement learning process, we use the dynamic adjustment of the penalty coefficient β (Ouyang et al., 2022). More detailed hyperparameters are shown in Appendix I. All the experiments can be conducted on a single GPU with 24GB memory.

heading: 4.3 Experiment Results and Discussions
text: We validate RLF-KG effectiveness by comparing the Jaccard metric of the model before and after this process. Table 2 displays performance across thirteen hypothesis types on FB15k-237, WN18RR, and DBpedia50. It illustrates Jaccard indices between observations and conclusions of generated hypotheses from the test graph. The encoder-decoder and the decoder-only transformers are assessed under fully supervised training on each dataset. Additionally, performance is reported when models collaborate with reinforcement learning from knowledge graph feedback (RLF-KG).

heading: 4.3.1 Performance Gain after RLF-KG
text: We notice RLF-KG consistently enhances hypothesis generation across three datasets, improving both encoder-decoder and decoder-only models. This can be explained by RLF-KG’s ability to incorporate knowledge graph information into the generation model, diverging from simply generating hypotheses akin to reference hypotheses.
Additionally, after the RLF-KG training, the encoder-decoder model surpasses the decoder-only structured transformer model. This is due to the task’s nature, where generating a sequence of tokens from an observation set does not necessitate the order of the observation set. Figure 6 supplements the previous statement by illustrating the increasing reward throughout the PPO process. We also refer readers to Appendix K for qualitative examples demonstrating the improvement in the generated hypotheses for the same observation.

heading: 4.3.2 Adding Structural Reward to PPO
text: We explore the potential benefits of incorporating structural similarity into the reward function used in PPO training. While RLF-KG originally relies on the Jaccard index, we consider adding the Smatch score, a measure of structural differences between generated and sampled hypotheses. We conducted additional experiments to include S(H,Href) as an extra term in the reward function.
The results, presented in Table 3, indicate that by incorporating the structural reward, the model can indeed generate hypotheses that are closer to the reference hypotheses yc in structural sense. However, the Jaccard scores reveal that with the inclusion of structural information, the overall performance is comparable to or slightly worse than the original reward function. This is because adding Smatch score tends to cause the generation model to fit or potentially overfit the training data accord-
ing to the above graph-level similarity, limiting its ability to generalize to unseen test graphs. The Jaccard reward (Eq. 7), however, captures a measure closer to the goal of our task . Detailed Smatch scores by query types can be found in Appendix G.

heading: 4.3.3 Comparison Between Search Methods
text: We chose the relatively simple brute-force search (Algorithm 4) as our search-based baseline due to the inherent high complexity of search algorithms for this task. For each observation, the algorithm explores all potential 1p hypotheses within the training graph and selects the one with the highest Jaccard similarity on the training graph. Despite its simplicity and complexity linear to the number of edges, the algorithm requires significantly more time than our method, not to mention other more complex heuristics.
Following this choice, we compare inference time and performance between the generationbased and the brute-force search algorithm. Table 4 highlights the unsuitability of the brute-force search for scaling up due to its high complexity. In contrast, generation-based methods demonstrate substantially faster performance.
Moreover, the generation methods not only outperform the search-based method in Jaccard performance but also show a significant improvement in Smatch performance. The relatively high Jaccard score of the brute-force search is attributed to its inherent access to an approximation of Jaccard during the search process. Due to the nature of the graph splits, the approximation of Jaccard
on the test graph using the training graph for 1p queries is found to be quite accurate, which enhances the average score. The detailed Jaccard scores are presented in Appendix H. However, the brute-force search struggles with more complex types of queries.

heading: 5 Related Work
text: The problem of abductive knowledge graph reasoning shares connections with various other knowledge graph reasoning tasks, including knowledge graph completion, complex logical query answering, and rule mining.
Rule Mining Rule mining focuses on inductive logical reasoning, namely discovering logical rules over the knowledge graph. Various methods are proposed in this line of work (Galárraga et al., 2015; Ho et al., 2018; Meilicke et al., 2019; Cheng et al., 2022, 2023). While the direct application of rule mining to solve this task is theoretically applicable, rule mining algorithms such as those described by Galárraga et al. (2015) also rely on searches to construct the horn clauses, which face scalability issues.
Complex logical query Complex logical query answering is a task of answering logically structured queries on KG. Query embedding primary focus is the enhancement of embedding structures for encoding sets of answers (Hamilton et al., 2018; Sun et al., 2020; Liu et al., 2021; Bai et al., 2023c,a,b; Hu et al., 2023; Yin et al., 2023; Wang et al., 2023a,b; Hu et al., 2024; Liu et al., 2024).
For instance, Ren and Leskovec (2020) and Zhang et al. (2021b) introduce the utilization of geometric structures such as rectangles and cones within hyperspace to represent entities. Neural MLP (Mixer) (Amayuelas et al., 2022) use MLP and MLP-Mixer as the operators. (Bai et al., 2022) suggests employing multiple vectors to encode queries, thereby addressing the diversity of answer entities. FuzzQE (Chen et al., 2022) uses fuzzy logic to represent logical operators. Probabilistic distributions can also serve as a means of query encoding (Choudhary et al., 2021a,b), with examples including Beta Embedding (Ren and Leskovec, 2020) and Gamma Embedding (Yang et al., 2022).

heading: 6 Conclusion
text: In summary, this paper has introduced the task of abductive logical knowledge graph reasoning. Meanwhile, this paper has proposed a generationbased method to address knowledge graph incompleteness and reasoning efficiency by generating logical hypotheses. Furthermore, this paper demonstrates the effectiveness of our proposed reinforcement learning from knowledge graphs (RLF-KG) to enhance our hypothesis generation model by leveraging feedback from knowledge graphs.
Limitations
Our proposed methods and techniques in the paper are evaluated on a specific set of knowledge graphs, namely FB15k-237, WN18RR, and DBpedia50. It is unclear how well these approaches would perform on other KGs with different characteristics or domains. Meanwhile, knowledge graphs can be massive and continuously evolving, our method is not yet able to address the dynamic nature of knowledge evolutions, like conducting knowledge editing automatically. It is important to note that these limitations should not undermine the significance of the work but rather serve as areas for future research and improvement.

heading: Acknowledgements
text: We thank the anonymous reviewers and the area chair for their constructive comments. The authors of this paper were supported by the NSFC Fund (U20B2053) from the NSFC of China, the RIF (R6020-19 and R6021-20), and the GRF (16211520 and 16205322) from RGC of Hong Kong. We also thank the support from the UGC Research Matching Grants (RMGS20EG01-D,
RMGS20CR11, RMGS20CR12, RMGS20EG19, RMGS20EG21, RMGS23CR05, RMGS23EG08).

heading: A Clarification on Abductive, Deductive, and Inductive Reasoning
text: Here we use simple syllogisms to explain the connections and differences between abductive reasoning and the other two types of reasoning, namely, deductive and inductive reasoning. In deductive reasoning, the inferred conclusion is necessarily true if the premises are true. Suppose we have a major premise P1: All men are mortal, a minor premise P2: Socrates is a man, then we can conclude C that Socrates is mortal. This can also be expressed as the inference P1 ∧ P2
C . On the other hand, abductive reasoning aims to explain an observation and is non-necessary, i.e., the inferred hypothesis is not guaranteed to be true. We also start with a premise P : All cats like catching mice, and then we have some observation O: Katty like catching mice. The abduction gives a simple yet most probable hypothesis H: Katty is a cat, as an explanation. This can also be written as the inference P ∧O H . Different than deductive reasoning, the observation O should be entailed by the premise P and the hypotheses H , which can be expressed by the implication P ∧H =⇒ O. The other type of non-necessary reasoning is inductive reasoning, where, in contrast to the appeal to explanatory considerations in abductive reasoning, there is an appeal to observed frequencies (Douven, 2021). For instance, premises P1: Most Math students learn linear algebra in their first year and P2: Alice is a Math student infer H: Alice learned linear algebra in her first year, i.e., P1 ∧ P2
H . Note that the inference rules in the last two examples are not strictly logical implications.
It is worth mentioning that there might be different definitions or interpretations of these forms of reasoning.

heading: B Example of Observation-Hypothesis Pair
text: For example, observation O can be a set of name entities like {GrantHeslov, JasonSegel, RobertTowne, RonaldBass,RashidaJones}. Given this observation, an abductive reasoner is required to give the logical hypothesis that best explains it. For the above example, the expected hypothesis H in natural language is that they are actors and screenwriters, and they are also born in Los Angeles. Mathematically, the hypothesis H can
be represented by a logical expression of the facts of the KG: H(V ) = Occupation(V,Actor) ∧ Occupation(V, ScreenWriter) ∧ BornIn(V,LosAngeles). Although the logical expression here only contains logical conjunction AND (∧), we consider more general first-order logical form as defined in Section 2.

heading: C Statistics of Queries Sampled for Datasets
text: Table 5 presents the numbers of queries sampled for each dataset in each stage.

heading: D Algorithm for Sampling Observation-Hypothesis Pairs
text: Algorithm 1 is designed for sampling complex hypotheses from a given knowledge graph. Given a knowledge graph G and a hypothesis type T , the algorithm starts with a random node v and proceeds to recursively construct a hypothesis that has v as one of its conclusions and adheres the type T . During the recursion process, the algorithm examines the last operation in the current hypothesis. If the operation is a projection, the algorithm randomly selects one of v’s in-edge (u, r, v). Then, the algorithm calls the recursion on node u and the sub-hypothesis type of T again. If the operation is an intersection, it applies recursion on the subhypotheses and the same node v. If the operation is a union, it applies recursion on one sub-hypothesis with node v and on other sub-hypotheses with an arbitrary node, as union only requires one of the sub-hypotheses to have v as an answer node. The recursion stops when the current node contains an entity.

heading: E Algorithms for Conversion between Queries and Actions
text: We here present the details of tokenizing the hypothesis graph (Algorithm 2), and formulating a graph according to the tokens, namely the process of de-tokenization (Algorithm 3). Inspired by the action-based semantic parsing algorithms, we view tokens as actions. It is worth noting that we employ the symbols G,V,E for the hypothesis graph to differentiate it from the knowledge graph.

heading: F Details of using Smatch to evaluate structural differneces of queries
text: Smatch (Cai and Knight, 2013) is an evaluation metric for Abstract Meaning Representation
Algorithm 1 Sampling Hypothesis According to Type
Input Knowledge graph G, hypothesis type T Output Hypothesis sample function GROUNDTYPE(G, T, t)
if T.operation = p then (h, r) ← SAMPLE({(h, r)|(h, r, t) ∈ E(G)}) T̂ ← the only subtype in T.SubTypes H ← GROUNDTYPE(G, T̂ , h) return (p, r,H)
else if T.operation = i then SubHypotheses← ∅ for T̂ ∈ T.SubTypes do
H ← GROUNDTYPE(G, T̂ , t) SubHypotheses.PUSHBACK(H)
end for return (i, SubHypotheses)
else if T.operation = u then SubHypotheses← ∅ for T̂ ∈ T.SubTypes do
if T̂ is the first subtype then H ← GROUNDTYPE(G, T̂ , t) else t̂← SAMPLE(V(G)) H ← GROUNDTYPE(G, T̂ , t̂) end if SubHypotheses.PUSHBACK(H)
end for return (u, SubHypotheses)
else if T.operation = e then return (e, t)
end if end function v ← SAMPLE(V(G)) return GROUNDTYPE(G, T, v)
Algorithm 2 HypothesisToActions Input Hypothesis plan graph G Output Action sequence A
function DFS(G, t,A) if t is an anchor node then
action← entity associated with t else
action ← operator associated with the first in-edge of t
end if A.PUSHBACK(action) for all in-edges to t in G (h, r, t) do
DFS(G, h,A) end for
end function root← the root of G A← DFS(G, root, ∅) return A
(AMR) graphs. An AMR graph is a directed acyclic graph with two types of nodes: variable nodes and concept nodes, and three types of edges:
• Instance edges, which connect a variable node to a concept node and are labeled literally “instance”. Every variable node must have exactly one instance edge, and vice versa.
• Attribute edges, which connect a variable node to a concept node and are labeled with attribute names.
• Relation edges, which connect a variable node to another variable node and are labeled with relation names.
Given a predicted AMR graph Gp and the gold AMR graph Gg, the Smatch score of Gp with respect to Gg is denoted by Smatch(Gp, Gg). Smatch(Gp, Gg) is obtained by finding an approximately optimal mapping between the variable nodes of the two graphs and then matching the edges of the graphs.
Our hypothesis graph is similar to the AMR graph, in:
Algorithm 3 ActionsToHypothesis Input Action sequence A Output Hypothesis plan graph G
S ← an empty stack Create an map deg. deg[i] = deg[u] = 2 and = 1 otherwise. V ← ∅, E ← ∅ for a ∈ A do
Create a new node h, V ← V ∪ {h} if S ̸= ∅ then
(t, operator, d)← S.TOP E ← E ∪ {(h, operator, t)}
end if if a represents an anchor then
Mark h as an anchor with entity a while S ̸= ∅ do
(t, operator, d)← S.POP d← d− 1 if d > 0 then
S.PUSHBACK((t, operator, d)) Break
end if end while
else S.PUSHBACK((h, a, deg[a]))
end if end for G← (V,E) return G
• The nodes are both categorized as fixed nodes and variable nodes
• The edges can be categorized into two types: edges from a variable node to a fixed node and edges from a variable node to another variable node. And edges are labeled with names.
However, they are different in that the AMR graph requires every variable node to have instance edges, while the hypothesis graph does not.
The workaround for leveraging the Smatch score to measure the similarity between hypothesis graphs is creating an instance edge from every entity to some virtual node. Formally, given a hypothesis H with hypothesis graph G(H), we create a new hypothesis graph GA(H) to accommodate the AMR settings as follows: First, we initialize GA(H) = G(H). Then, create a new relation type instance and add a virtual node v′ into GA(H). Finally, for every variable node v ∈ G(H), we add a relation instance(v, v′) into GA(H). Then, given a predicted hypothesis Hp and a gold hypothesis Hg, the Smatch score is defined as
S(Hp, Hg) = Smatch(GA(Hp), GA(Hg)). (10)
Through this conversion, a variable entity vg of Hg is mapped to a variable entity vp of Hp if and only if instance(vg, v′) is matched with instance(vp, v
′). This modification does not affect the overall algorithm for finding the optimal mapping between variable nodes and hence gives a valid and consistent similarity score. However, this adds an extra point for matching between instance edges, no matter how the variable nodes are mapped.

heading: G Detailed Smatch Scores by Query Types
text: Tables 6 and 7 show the detailed Smatch performance of various methods.

heading: H Detailed Jaccard Performance of the Brute-force Search
text: Table 8 shows the detailed Jaccard performance of the brute-force search.

heading: I Hyperparameters of the RL Process
text: The PPO hyperparameters are shown in Table 9. We warm-uped the learning rate from 0.1 of the peak to the peak value within the first 10% of total
iterations, followed by a decay to 0.1 of the peak using a cosine schedule.

heading: J Algorithms for One-Hop Searching
text: We now introduce the Algorithm 4 used for searching the best one-hop hypothesis with the tail among all entities in the observation set to explain the observations.

heading: K Case Studies
text: Explore Table 10, 11 and 12 for concrete examples generated by various abductive reasoning methods, namely search, generative model with supervised training, and generative model with supervised training incorporating RLF-KG.
Algorithm 4 One-Hop-Search Input Observation set O Output Hypothesis bestHypothesis
candidates← {(h, r, t) ∈ Rtrain|t ∈ O} bestJaccard← 0 bestHypothesis← Null for (h, r, t) ∈ candidates do
H ← the one-hop hypothesis consisting of edge (h, r, t)
nowJaccard← Jaccard([[H]]Gtrain , A) if nowJaccard > bestJaccard then
bestJaccard← nowJaccard bestHypothesis← H
end if end for return bestHypothesis

heading: Future Work
text: It is important to note that these limitations should not undermine the significance of the work but rather serve as areas for future research and improvement