Paper: Abductive Logical Reasoning on Knowledge Graphs (ICLR-2024)

Reviewer comment (Theory):
It is inappropriate to call this task as "abduction". The task learns a first-order hypothesis that satisfies given groundings, and the generated hypothesis is used for generalising to more unseen groundings. Logical abduction means a grounding-to-grounding hypothesizing, for example, given P(x)->Q(x), observing Q(1), we can abduce P(1). If you want to abduce first-order theories, then you need second-order rules as background knowledge.

Author response:
In our formulation of abductive reasoning on knowledge graphs, given a grounded observation such as Q(1), the goal is the find a hypothesis P(x) such that the set {x | P(x) holds true} closely resembles the set 1. This means that P(1) = True. 

To provide a clearer understanding, we will compare our abductive logical expression generation with inductive rule mining on knowledge graphs:

Rule Mining (ILP on KG):

This task involves a given knowledge graph with some positive examples regarding the relation $speak$. The goal is to find Horn clause rules associated with the $speak$ relation.

$speaks(X, Y) \\leftarrow lives(X, A) \\land language(A, Y)$

The variables in the rules are universally quantified, and the quality of the rules is measured by head coverage and precision. 

Our task:

We use a real example from FB15K-237. Our observations include the following entities:

$Grant Heslov, Jason Segel, Robert Towne, Ronald Bass, Rashida Jones$

Our task aims to find a logical expression based on the KG to describe these entities, rather than discovering general rules over the KG. These entities make the following logical expressions true:

$V_?: Occupation(V_?, Actor) \\land Occupation(V_?, Screen Writer) \\land BornIn(V_?, Los Angeles)$

This means that Grant Heslov, Jason Segel, Robert Towne, Ronald Bass, and Rashida Jones are actors and screenwriters born in Los Angeles. This logical expression is not a rule, as not all entities from the KG satisfy the logical expression.

Unlike inductive rule mining, which is not grounded, our proposed reasoning task is grounded, focusing on specific entities.

Re W1: 
- We would like to respectfully discuss the assertion that "the generated hypothesis is used for generalizing to more unseen groundings" in W1. Our intention is **not** to have the generated hypotheses **generalize to unseen groundings**, but rather to **exclusively** serve as the **best explanation** for the given observation. This objective is reflected in our experimental metric, where we gauge the Jaccard similarity between the set of entities making the generated hypothesis true and the given observation. In fact, the **Jaccard similarity decreases** as the generated hypothesis generalizes to **more unseen groundings**, as the overlap between the two sets accounts for a smaller fraction of their union.
- Quoting from [1], “It suggests that the best way to distinguish between induction and abduction is this: both are ampliative, meaning that the conclusion goes beyond what is (logically) contained in the premises …, but in **abduction** there is an implicit or explicit **appeal to explanatory considerations**, whereas in induction there is not; in induction, there is only an appeal to observed frequencies or statistics.”. Our goal, in line with the description of abduction provided here, is to generate an explanatory hypothesis for a given observation, **not** a rule intended to hold **generally** with high probability.
- Returning to our example, the hypothesis presented is just a sample. Our expectation is not for it to be a rule but, ideally, to hold true only for the specified five celebrities. Considering the possibility of additional groundings in reality, the hypothesis may not be the best explanation, and it's essential to identify exclusive common characteristics among the five celebrities for a more refined hypothesis.

Reviewer comment (Presentation):
The presentation of this work can be significantly improved. The description of the proposed method is messy. For example, the caption of Figure 4 is "step 3", which is out of blue and makes reader confusing. Furthermore, the illustration of figures and algorithms are also confusing. There's no input and output in algorithms, and what are "models" and "reference models" in Fig. 4?

Author response:
We have updated the description of our proposed method, RLF-KG, in Section 3 to improve clarity. The entire process is divided into three steps.
 
Step 1 involves preparing the training samples and converting the raw data into a format that is suitable for a transformer-based model.
 
In Step 2, a generative model is trained in a supervised manner. It is important to note that the resulting model from Step 2 is standalone and can be used to generate hypotheses given observations, although its performance may be relatively low.
 
The primary contribution of our method is Step 3, which fine-tunes the trained model via PPO with feedback from the training KG. This step is critical in improving the model's ability to reason over the KG and generate more accurate and complex hypotheses.
In regard to the "model" and "reference model," both are initialized to be the trained model resulting from Step 2 (which we refer to as the "original model"). During Step 3, the "model" is optimized, while the "reference model" remains fixed. The "reference model" is used in the PPO process to prevent excessive divergence between the fine-tuned "model" and the "original model."
The "reference model" is crucial in ensuring that the fine-tuned "model" produces syntactically correct hypotheses similar to the "original model." Without limiting divergence, the training process may become difficult to converge.
Our quantitative experiments (presented in Table 3, Sec. 4.4) demonstrated the effectiveness of Step 3, namely RLF-KG. Figure 6 illustrates the growth in reward during the PPO process, further supporting the effectiveness of our proposed method.
 
In addition to the quantitative results, we provide qualitative examples in Appendix G to showcase the improvement in the generated hypotheses for the same observation. These examples illustrate how RLF-KG improves the model's ability to reason over the KG and generate more accurate and complex hypotheses.

Reviewer comment (Theory):
Fig. 5 proposes 13 types of first-order hypotheses templates, are they complete for the hypothesis space? Is there any proof to the completeness?

Author response:
In line with previous works on KG reasoning [1][2], we utilize similar hypothesis templates as those used in these studies. Due to the exponential increase in the number of logical expression templates with the number of variables, we are only able to evaluate a subset of them in our experiments. However, the selected expression templates are considered empirically significant based on their use in various previous works on complex query answering [1][2].
 
[1] Query2box: Reasoning over knowledge graphs in vector space using box embeddings.

[2] Beta embeddings for multi-hop logical reasoning in knowledge graphs.

Re W2:
- On one hand, we have not asserted that our scope is exhaustive. As stated in the paper, the number of candidate hypotheses is **combinatorial** concerning the nodes and relations in a hypothesis type, making it computationally expensive to consider an extensive range of hypothesis types.
- On the other hand, we **do not restrict** our generative model to producing only the 13 types of hypotheses, as mentioned in Sec. 4.1. The 13 types are employed in the **sampling process**. As long as the generated hypotheses are **syntactically correct**, our program is able to find the corresponding conclusion, compare it with the given observation, and **produce a reward** for the PPO. This also explains why we developed the PPO method on top of supervised learning, as the supervision signal only compares the sampled hypotheses (of one of the 13 types) with the generated hypotheses.

[1]  Douven, Igor, "Abduction", The Stanford Encyclopedia of Philosophy (Summer 2021 Edition), Edward N. Zalta (ed.), URL = <https://plato.stanford.edu/archives/sum2021/entries/abduction/>.

Reviewer comment (Writing):
The paper could benefit from a more precise statement of their objective problem, which would then allow for a clearer discussion of how the current effort contrasts with past and future work.

Author response:
We revised and clarified the problem definition in section 2, which involves using abductive reasoning to find a hypothesis (H) that matches an observation set $O = \{v_1, v_2, ..., v_k\}$ as closely as possible. The objective of abductive reasoning is formalized in equation (4), which uses the Jaccard score to measure the similarity between the observations and the corresponding conclusions drawn from the generated conclusions on the hidden graph.

Reviewer comment (Writing):
The objective of the approach is not precisely defined, so it is difficult to discern whether or not any particular intent was achieved. No future research was discussed.

Author response:
Please refer to Re:Q1 and Re:Q2 for the objectives of the problem and the approach. In addition, we further discuss the future work on these directions. One possible direction is to explore the use of more advanced reinforcement learning algorithms to improve the performance of the model. Another direction is to investigate the use of larger and more complex knowledge graphs. Our current experiments were conducted on relatively small graphs with 10k up to 100k vertices. It would be interesting to see how the model performs on larger graphs, as well as graphs with more complex structures and relationships.

Reviewer comment (Novelty):
I feel that this is valuable, but could be more impactfully significant with a more precise framing of this approach and alternate formulations of abduction.

Author response:
We want to clarify that the abductive reasoning objectives mentioned in the abstract and at the end of Section 2 are the same. We formally define the "best" explanation using formula (4) at the end of Section 2, which is represented by the Jaccard similarity between the conclusion of the generated hypothesis on the testing graph (which was not observed during training) and the observation.
 
It is crucial to note that the reward function for the PPO is chosen to approximate the objective by using the Jaccard similarity between the conclusion of the generated hypothesis on the training graph (the observed graph) and the observation. Furthermore, the primary metric used in our experiments is consistent with the task's objective.

Reviewer comment (Experiments):
The paper criticizes previous approaches without quantitatively comparing them to the current method. While this may be computationally infeasible on the graphs chosen here, do you have such results you could include -- even if only on very small graphs that highlight and justify your statements?

Author response:
We have revised the description of our approach in Section 3.2 to provide further clarity. Please refer to Re: Q2 for details.  

Our proposed KG reasoning task is unique and novel, which means there are no previous methods capable of generating complex first-order logical expressions to explain specific observations. In Table 5 and 6, we compare our method with pure symbolic methods, such as brute-force searching the one-hop neighbors of the observations. However, brute-force searching is very slow on graphs with at least 10k vertices, and it struggles to perform well when the hypothesis is complex and structured, involving multiple logical connectives and variables.
 
In Appendix G Case Studies, we provide qualitative examples that compare the hypotheses produced by the search-based method, the generation-based method, and the generation-based method after applying RLF-KG.

Reviewer comment (Theory):
This article lacks a clear formalization of the explanation for abductive reasoning.

Author response:
We revised and clarified the problem definition in section 2, which involves using abductive reasoning to find a hypothesis (H) that matches an observation set $O = \{v_1, v_2, ..., v_k\}$ as closely as possible. The objective of abductive reasoning is formalized in equation (4), which uses the Jaccard score to measure the similarity between the observations and the corresponding conclusions drawn from the generated conclusions on the hidden graph.

Reviewer comment (Theory):
The motivation for using reinforcement learning algorithms is questionable.

Author response:
During supervised training, the model only learns to minimize the difference between the generated and sample hypothesis sequences. However, this doesn't guarantee that the conclusion of the generated hypothesis on the testing graph will be closer to the observation, which is our ultimate objective. Additionally, this supervision signal doesn't make full use of the knowledge graph's valuable information.
 
To address this, we implemented a PPO reward that uses the Jaccard similarity between the conclusion of the generated hypothesis on the training graph and the observation. We believe this reward provides logical information that can improve the quality of generated hypotheses by approximating our task's objective directly.
 
The positive results in our experiments in Section 4.4 further support the value of RLF-KG in improving the quality of hypotheses generated through abductive reasoning.

Reviewer comment (Presentation):
What are the reference models in Fig. 4? Why the method needs it?

Author response:
The reference model is a copy of the model obtained from supervised training, and the model being fine-tuned by PPO starts off identical to the reference model. The reference model plays a crucial role in preventing excessive divergence between itself and the fine-tuned model. The KL term in the PPO objective measures this divergence and penalizes it to ensure that the models remain aligned.

Reviewer comment (Novelty):
The paper proposes the task of abductive logical reasoning and claims it is under-studied, but I am not convinced the task is important.

Author response:
Abductive logical reasoning on knowledge graphs remains important because current language models, such as LLMs, struggle to reason over factual knowledge. For example, when prompted with the task of generating a logical hypothesis that matches a given observation set, ChatGPT (GPT-3.5) generated a hypothesis that claimed the devices were all first-generation Apple products, despite the observation set including an iPod touch 4th Gen. This demonstrates the model's inability to perform simple abductive reasoning.

Here is the example:  
Input: “Please propose a logical hypothesis H(x) such that the set of all entities that make H(x) true should be as close to O as possible: O = {Ipad 1st Gen, Ipod touch 4th Gen, Apple TV 1st Gen}” (the 2nd example in Fig. 1)

Output: “Certainly! To create a logical hypothesis H(x) that is as close to the given set O = {iPad 1st Gen, iPod touch 4th Gen, Apple TV 1st Gen} as possible, you might consider the following hypothesis: 

H(x): The entities in set  x  are first-generation electronic devices produced by Apple.”

However, in our framework, the model is trained with feedback from the observed knowledge graph. If the knowledge graph contains basic facts about Apple's products and the model generates a faulty hypothesis, the discrepancy between the observation set and the conclusion of the hypothesis will result in a relatively low reward. This encourages the model to correct the hypothesis and improves its ability to reason over factual knowledge.

Reviewer comment (Presentation):
Can you move the results of the search baseline to the main table, table 3?

Author response:
Q2: We have moved the results of the search baseline to the main table.

Reviewer comment (Theory):
In my view, the term of abductive reasoning is incorrectly used. The process of obtaining hypothesis based on observations and background knowledge is called induction. Abductive reasoning further requires to obtain groundings for variables in the hypothesis.

Author response:
We would like to clarify that our task differs from inductive logical programming (ILP). This is because ILP-induced logical hypotheses are general rules, as they are not grounded. In contrast, our task involves grounded logical hypotheses, which specifically describe observations rather than expressing rules. In the knowledge graph domain, inductive reasoning is equivalent to rule mining [1,2,3], discussed in Section 5.

Here are some examples to illustrate the differences:

Rule Mining (ILP on KG):

The task involves a given knowledge graph with some positive examples regarding the relation $speak$. The goal is to find horn clause rules related to the $speak$ relation.
$speaks(X, Y) \\leftarrow lives(X, A) \\land language(A, Y)$
The variables in the rules are universally quantified, and the quality of the rules is measured by head coverage and precision. However, in our task, we use a real example from FB15K-237. Our observations include the following entities:

$ Grant Heslov, Jason Segel, Robert Towne, Ronald Bass, Rashida Jones $

Our task aims to find a logical expression based on the KG to describe these entities, rather than discovering general rules over the KG. These entities make the following logical expressions true:

$V_?: Occupation(V_?, Actor) \\land Occupation(V_?, Screen Writer) \\land BornIn(V_?, Los Angeles)$

This means that Grant Heslov, Jason Segel, Robert Towne, Ronald Bass, and Rashida Jones are actors and screenwriters born in Los Angeles. This logical expression is not a rule, as not all entities from the KG satisfy the logical expression.

Therefore, our problem setting is distinct from inductive logical programming over knowledge graphs, specifically rule-mining.

Reviewer comment (Novelty):
The paper misses citation to researches on inductive logic programming (ILP), which is closely related to the problem studied in the paper. An ILP task involves learning logic programs based on logic observations and background knowledge, which is more general than reasoning on knowledge graphs. Furthermore, the term abductive learning has also be proposed before in the ILP area [1]. Related citations should be included and discussed in the paper.

Author response:
We previously explained the differences between our proposed abductive KG reasoning task and ILP. Additionally, our problem setting significantly differs from the abductive learning suggested in [4]. The study in [4] uses abductive learning (ABL) to form symbolic representations via learning methods and then applies Prolog's abductive logic programming to address hand-written puzzles. These symbolic representations may not be logical expressions, and Prolog's abductive logic programming can only evaluate their truth or falsity, without generating complex first-order structured hypotheses.

Reviewer comment (Experiments):
The experimental results are only quantitative. It would nicer to illustrate some qualitative examples on what kinds of hypotheses can be learned by the proposed method. For example, it would be useful to illustrate whether the proposed approach can learn lengthy hypotheses with significant complexity.

Author response:
In the appendix, we have provided several concrete examples in Tables 12, 13, and 14. These tables contain the observations, logical expressions, and their corresponding truth values. As demonstrated in the tables, our proposed method can generate logical expressions that are appropriately long, involve multiple logical connectives, and are precise (as indicated by high Jaccard scores) in describing the observations.

Reviewer comment (Theory):
On the bottom of Page 3, it is said that the logic clauses contains only three operations $\cup, \cap, \neg$. Does this mean that the paper only considers a limited subset of first-order logic? (e.g. propositional logic). Even though this would be enough for knowledge graphs, which can only represent relational information, clarifying the scope of learning is still necessary.

Author response:
As presented in Equations 1 and 2, we focus on a specific form of first-order logical expressions, which includes existential quantified intermediate variables and logical connectives AND/OR/NOT. We have provided further clarification regarding the scope in Section 2 of the paper.