Neural Probabilistic Logic Learning for Knowledge Graph Reasoning

## Abstract

Abstract Knowledge graph (KG) reasoning is a task that aims to predict unknown facts based on known factual samples. Reasoning methods can be divided into two categories: rule-based methods and KG-embedding based methods. The former possesses precise reasoning capabilities but finds it challenging to reason efficiently over large-scale knowledge graphs. While gaining the ability to reason over large-scale knowledge graphs, the latter sacrifices reasoning accuracy. This paper aims to design a reasoning framework called Neural Probabilistic Logic Learning(NPLL) that achieves accurate reasoning on knowledge graphs. Our approach introduces a scoring module that effectively enhances the expressive power of embedding networks, striking a balance between model simplicity and reasoning capabilities. We improve the interpretability of the model by incorporating a Markov Logic Network based on variational inference. We empirically evaluate our approach on several benchmark datasets, and the experimental results validate that our method substantially enhances the accuracy and quality of the reasoning results.

## 1 Introduction

Knowledge representation has long been a fundamental challenge in artificial intelligence. Knowledge graphs, a form of structured knowledge representation, have gained significant traction in recent years due to their ability to capture rich semantics and facilitate reasoning over large-scale data. Compared to conventional approaches such as semantic networks and production rules, knowledge graphs offer a more expressive and scalable representation of entities and their relationships in a graph-based formalism. This structured representation not only assists human comprehension and reasoning but also enables seamless integration with machine learning techniques, facilitating a wide range of downstream applications.

One prominent line of research in knowledge graph reasoning revolves around embedding-based methods. These techniques aim to map the elements of a knowledge graph into a low-dimensional vector space, capturing the underlying associations between entities and relations through numerical representations. While this approach has demonstrated promising results, it suffers from inherent limitations, including low interpretability, suboptimal performance on long-tail relations, and challenges in capturing complex semantic information and logical relationships.

Alternatively, rule-based knowledge reasoning methods operate by extracting logical rules from the knowledge graph, typically in the form of first-order predicate logic, and performing inference based on these rules. However, these methods often face challenges stemming from the vast search space and limited coverage of the extracted rules. Markov Logic Networks (MLNs) [] have been proposed as a principled framework for combining probabilistic graphical models with first-order logic, enabling the effective integration of rules and embedding methods for more accurate reasoning.

In this paper, we seek to develop a method that can better leverage the outputs of embedding networks to support knowledge graph reasoning. To this end, we propose a novel reasoning framework called Neural Probabilistic Logic Learning (NPLL). NPLL introduces a scoring module that efficiently utilizes knowledge graph embedding data, enhancing the training process of the entire framework. Our method, illustrated in Figure 1, makes the following key contributions:

Efficient Reasoning and Learning: NPLL can be viewed as an inference network for MLNs, extending MLN inference to larger-scale knowledge graph problems.

Tight Integration of Logical Rules and Data Supervision: NPLL can leverage prior knowledge from logical rules and supervision from structured graph data.

Balance between Model Size and Reasoning Capability: Despite its compact architecture and relatively fewer parameters, NPLL demonstrates remarkable reasoning capabilities, sufficient to capture the intricate relationships and semantics within knowledge graphs. Even in data-scarce scenarios where the available dataset size is relatively small, NPLL can achieve a high level of reasoning performance, making it well-suited for practical applications with limited labeled data.

Zero-shot Learning Ability: Our proposed NPLL framework exhibits strong zero-shot learning capabilities, enabling it to handle reasoning tasks involving target predicates with very few or no labeled instances. This flexibility and generalization power are particularly valuable in real-world settings where exhaustive labeling of all possible relationships is often impractical.

Figure: Figure 1: Visualization of Neural Probabilistic Logic Learning (NPLL)
Refer to caption: extracted/5709768/fig1.png

## 2 Related Work

One prominent category of methods for knowledge graph reasoning is rule-based approaches. These methods leverage logical rules, generally defined as B→A, where A is the target fact, and B can be considered a set of condition facts. Facts are composed of predicates and variables. To better utilize these symbolic features, methods like AMIE[], RuleN[], and RLvLR[] employ rule mining tools to extract logical rules from knowledge graphs for reasoning. Approaches like KALE[], RUGE[], and IterE[] started combining logical rules with embedding learning to construct joint knowledge graph reasoning models.
Additionally, NeuralLP[] proposed an end-to-end differentiable method to effectively learn the parameters and structures of logical rules in knowledge graphs. NeuralLP-num-lp[] combined summation operations and dynamic programming with NeuralLP, which can be used to learn numerical regulations better. Simultaneously, DRUM[] introduced a rule-based end-to-end differentiable model. Then, pLogicNet designed a probabilistic logic neural network [], demonstrating exemplary reasoning performance. Building on this, ExpressGNN[] achieved more efficient reasoning by fine-tuning the GNN model. DiffLgic[] designed a differential framework to improve reasoning efficiency and accuracy for large knowledge graphs. NCRL[] proposed an end-to-end neural method that recursively leverages the compositionality of logical rules to enhance systematic generalization. In contrast to these approaches, our proposed NPLL framework is significantly more effective for knowledge graph reasoning.

Another category of approaches for knowledge graph reasoning is embedding-based methods. These techniques primarily represent entities and relations using vector embeddings. Knowledge graph reasoning is achieved by defining various scoring functions to model different reasoning processes. For instance, methods like TransE[], TransH[], TransR[], TransD[], TranSparse[], TransRHS[], RotatE[] project entities and relations into vector spaces, transforming computations between facts into vector operations. The essential scoring function is the difference between the head entity-relation vector and the tail entity vector. Rescal[], DistMult[], ComplEx[], HolE[], analog[], SimplE[], QuatE[], DualE[], HopfE[], LowFER[], QuatRE[] represent each fact in the knowledge graph as a three-dimensional tensor, decomposed into a combination of low-dimensional entity and relation vectors. They use vector matrices to represent the latent semantics of each entity and relation. The primary scoring function is the product of the head entity, relation, and tail entity. Methods like SME[], NTN[], and NAM[] employ neural networks to encode entities and relations into high-dimensional spaces. ConvE[] first introduced 2D convolutional layers for reasoning. RGCN[], NBF-net[], and RED-GNN[] use graph neural networks to aggregate neighbor node information and decoders as scoring functions. However, these embedding-based methods often sacrifice interpretability and prediction quality. In contrast, our proposed NPLL framework significantly improves the quality of reasoning results while more properly handling reasoning problems through the principled integration of logical rules.

## 3 Preliminary

A knowledge graph is a graph-structured model composed of triplets, where the entities in the triplets are nodes and the relations are edges. Given a known knowledge graph $K=(E,L,F)$, where$E=\{e_{1},e_{2},\ldots,e_{M}\}$ represents a set of M entities, with entities typically referring to person names, objects, locations and proper nouns;$L=\{l_{1},l_{2},\ldots,l_{N}\}$ represents a set of N relations; $F=\{f_{1},f_{2},\ldots,f_{S}\}$ represents a set of known facts involving entities from E and relations from L, where fi can be described as $f_{i}=\{e_{h},l,e_{t}\},e_{h},e_{t}\in E,l\in L$, indicating $e_{h}$ has a relation l with $e_{t}$, or can be written as $l(e_{h},e_{t})$, where $l$ is treated as a predicate and $e_{h}$ and $e_{t}$ as constants.

We now introduce the predicate logic representation, where each relation in the relation set is represented as a function $l(x,y)$, with $x$ and $y$ having the domain $E$, and $l(x,y)$ being directed, so $l(x,y)$ and $l(y,x)$ are different. For example, $l(x,y):=S(Tom,basketball)$ ($S$ denotes proficient sport), indicates that Tom’s proficient sport is basketball, which clearly cannot be expressed as $S(basketball,Tom)$.

Using the predicate logic representation, new facts can be inferred through logical deduction, e.g., $S\left(Tom,basketball\right)\land\ F\left(Tom,John\right)\Rightarrow\ S(John,basketball)$($F$ denotes being friends). If variables replace the constant entities in the above formula, it is called a rule, generally represented as: $Pred_{1}\left(x,y_{1}\right)\land\ Pred_{2}\left(y_{1},y_{2}\right)\land\ \
\ldots\ Pred_{n}\left(y_{n-1},z\right)\Rightarrow\ Pred\left(x,z\right)\ n\
\geq 1,$where $x$,${y_{i}}\textquoteright$,$z$ are all variables. $Pred(A,B)$ is called an atom, with $A$ and $B$ being the subject and object or the head and tail entities in the triplet. $Pred(x,z)$ is the head atom; the rest are body atoms. After substituting variables with constants, e.g. let $C_{1},C_{2},C_{3}$ be constants, $Pred_{1}\ (C_{1},C_{2})\land\ Pred_{2}(C_{2},C_{3})\Rightarrow\ Pred_{3}(C_{1}
,C_{3})$, which is called ground rule, and each atom with variables replaced by constants is called a ground predicate, whose value is a binary truth value. For example, $Pred_{1}(C_{1},C_{2})=\{0,1\}$. If $Pred_{1}(C_{1},C_{2})\in\ F$, then $Pred_{1}(C_{1},C_{2})=1$. Therefore, the goal of knowledge reasoning is to infer unknown facts $U\ =\{U_{j}\}$ from the known facts $F=\{f_{i}=1\}_{i=1,2,\ldots}$

Inferring unknown facts from known facts is a generative problem, which requires building a joint probability distribution model and maximizing the generation probability to obtain the unknown facts. Hence, we must construct a suitable joint probability distribution model for the reasoning task. Considering the above conditions, the knowledge graph can be modeled as a MLN, which combines first-order predicate logic and probabilistic graphical models. Traditional methods employ first-order predicate logic for deductive reasoning in a black-and-white manner. However, as the example $S\left(Tom,basketball\right)\land\ F\left(Tom,John\right)\Rightarrow\ S(John,basketball)$ shows, it is not necessarily always true. MLN assign a weight $\omega$ to each rule, representing the probability of the event occurring, thus transforming the hard conditions of predicate logic into probabilistic conditions. The rule representation form in first-order predicate logic is converted to Conjunctive Normal Form (CNF) for computational convenience.

$S(A,B)\land F(A,C)\Rightarrow\ S(C,B)\Leftrightarrow\lnot S(A,B)\ \ \bigvee\
\lnot\ F(A,C)\ \ \ \bigvee\ \ S(C,B)\ \$

Therefore, to construct a MLN from a knowledge graph, each ontology rule needs to be defined as a network in the MLN, each having a weight $\omega$. The probability calculation formula for MLN is:

$$ $P(F,U|\omega)=\frac{1}{Z(\omega)}\prod_{r\in R}\exp\left(\omega_{r}N(F,U) \right),$ (1) $$

where $F$ is the set of known facts, $U$ is the set of unknown facts, $R=\{r\}$ is the set of rules, $\omega_{r}$ is the weight of rule $r$, and $N(F,U)$ is the number of ground rules satisfying rule $r$. $Z(\omega)$ is the partition function, which is the sum of all possible ground rule cases for normalization:

$$ $Z(\omega)=\sum_{F,U}\prod_{r\in R}exp(\omega_{r}N(F,U)).$ (2) $$

All ground rules of each rule form a clique in MLN, $\exp\left(\omega_{r}N\left(F,U\right)\right)$ is the potential function of rule $r$, and each potential function expresses the situation of a clique. Generally, all ground rules of one rule form a clique, where each primary node, i.e., fact, is treated as a basic atom.
Each state of MLN assigns different occurrence possibilities to all facts, representing a possible open world. Each set of possible worlds combines $\{F,U,R\}$ relations, jointly determining the truth values of all basic atoms. After establishing the joint probability distribution, we infer the unknown facts $U$ from the known facts $F$ by solving the posterior distribution $P(U|F,\omega)$, which can be viewed as an approximate inference problem.

Unlike rule-based reasoning methods that evaluate rules holistically, knowledge embedding methods mainly score facts, assigning higher scores to correct facts and lower scores to incorrect ones, obtaining embedding vectors for different entities, and enabling inference of unknown facts.

## 4 Model

This section introduces a knowledge reasoning method that combines MLNs with embedding learning. By utilizing MLN, which is trained with EM algorithm[], to establish a joint probability distribution model of known facts and unknown facts, we decompose $P(F|\omega)$ to obtain the following equation:

$$ $logP(F|\omega)=log[\frac{P(F,U|\omega)}{Q(U)}]-log[\frac{P(U|F,\omega)}{Q(U)}],$ (3) $$

in equation 3, $P(F,U|\omega)$ is the joint probability distribution of known facts and unknown facts. In contrast, $P(U|F,\omega)$ is the posterior distribution, and $Q(U)$ is the approximate posterior distribution. Taking the expectation of both sides of 3 with respect to $Q(U)$, we can define $logP(F|\omega)$ as the sum of the evidence lower bound(ELBO) and the Kullback-Leibler(KL) divergence:

$$ $logP(F|\omega)=ELBO+KL(q||p),$ (4) $$

where $ELBO\ =\sum_{U}\ Q\left(U\right)log\left(\frac{P\left(F,U\middle|\omega\right)
}{Q\left(U\right)}\right)\$,$\ KL(q||p)\ =\ -\sum_{U}\ Q\left(U\right)log\left(\frac{P\left(U\middle|\ F,
\omega\right)}{Q\left(U\right)}\right)$.

When the approximate posterior distribution $Q(U)$ is the same as the true posterior distribution, we obtain the optimal result, at which point $KL(q||p)$ is 0 and ELBO is maximized. Therefore, our optimization objective becomes maximizing the ELBO value:

$$ $d_{ELBO}(Q,P)=\sum_{U}Q(U)logP(F,U|\omega)-\sum_{U}Q(U)logQ(U),$ (5) $$

the approximate posterior distribution $Q(U)$ is the probability distribution of unknown facts based on known facts.

Specifically, in the t-th iteration, the first step is to fix the rule weight $\omega$ as $\omega_{t}$, which is a constant. We then update the probability set of each factor in all ground rules through the reasoning method proposed in this paper and obtain the current approximate posterior probability distribution $Q(U)$. The second step substitutes the approximate posterior distribution into ELBO and updates $\omega$ by maximizing ELBO:

$$ $\omega=argmax_{\omega}\sum(Q(U)logP(F,U|\omega)-Q(U)logP(U,F|{\omega}^{t})),$ (6) $$

in equation 6, the second term is independent of $\omega$ and can be treated as a constant. Therefore, to reduce computation, we simplify the first step to fixing $\omega$ and computing the expectation of $logP(F,U|\omega)$ concerning $Q(U)$. The second step fixes the posterior distribution and updates $\omega$, obtaining ${\omega}^{t+1}=argmax_{\omega}\sum_{U}Q(U)logP(F,U|\omega)$.

### 4.1 Scoring Module

The most crucial part of the entire reasoning architecture is generating the approximate posterior probability. We design a scoring module to generate evaluation scores for facts. The generated evaluation scores can be the approximate posterior probability to compute the KL divergence from the actual posterior distribution. Additionally, they must satisfy the constraint that the loss for correct facts is minimized while the loss for incorrect facts is maximized. Therefore, we use vectors $e_{h}$ and $e_{t}$ to represent the head and tail entity features in a fact while representing the relation using three weight matrices.

Our scoring module consists of three parts. First, an embedding network initializes the vector features for each entity. Then, a scoring function $g(l,e_{h},e_{t})$ computes the evaluation score for each fact. Finally, the evaluation scores are processed to form the approximate posterior probability. For the scoring function, the model computes the following function to represent the possibility of the head and tail entities forming a valid fact under a given relation:

$$ $g(l,e_{h},e_{t})=u_{R}^{T}f(e_{h}^{T}W_{R}e_{t}+V_{R}\left[\matrix{e}_{h}\\ e_{t}\\ \right]+b_{R}),$ (7) $$

where $f$ is a non-linear activation function. $W_{R}$ is a $d*d*k$ dimensional tensor, and $e_{h}^{T}W_{R}e_{t}$ results from a bilinear tensor product, yielding a k-dimensional vector. $V_{R}$ is a $k*2d$ dimensional tensor, and $V_{R}\left[\matrix{e}_{h}\\
e_{t}\\
\right]$ is the result of a linear tensor product, also a k-dimensional vector. $u_{R}$ and $b_{R}$ are also k-dimensional, so the final result is a scalar. We design the each parts as follows:

We set up initial vectors for entities in the knowledge graph separately. We then build a neural network to update the vector features for all entities. The output of this part is the updated head and tail entity vectors $\{e_{h},e_{t}\}$ with dimension $d$.

We initialize a bilinear neural network layer $W_{R}$ and two linear neural network layers $V_{R}$, $u_{R}$. Taking the head and tail entity vector features as input, we pass them through the scoring function $g(l,e_{h},e_{t})$ to output the result and compute the evaluation scores for all known facts, unknown facts, and negative sample facts.

We define the obtained evaluation scores as the approximate posterior probability for known and unknown facts. Specifically, we process the evaluation scores using the sigmoid function to bound them between 0 and 1, i.e., $p\ =sigmod\left(g\left(l,e_{h},e_{t}\right)\right)$, where $sigmod\left(.\right)=\frac{1}{1+\exp{\left(.\right)}}$.

### 4.2 E-step

In the expectation step, to solve for the unknown facts in the knowledge graph based on the known facts, we need to obtain the posterior distribution $P(U|F,\omega)$ of the unknown facts. This can be achieved by minimizing the KL divergence between the approximate and true posterior distributions. However, directly solving the joint probability distribution model established by MLN is highly complex. Therefore, this paper randomly samples batches of ground rules to form datasets, wherein the ground rules are approximately independent of each batch. By applying the mean-field theorem[], we define the approximate posterior distribution as the product of the probability distributions of the individual ground rules. The truth value of a ground rule is 1 when it holds and 0 when it does not, and the truth value of each ground rule is jointly determined by the truth values of its constituent facts. Therefore, we set the probability distribution of a ground rule as the product of the probability distributions of its constituent facts. For example, for the ground rule:$R_{1}=\lnot S(Tom,basketball)\bigvee\lnot\ F(Tom,John)\bigvee S(John,
basketball).$

The truth value of the ground rule R1 is determined by its three constituent facts. Thus, we define:

$$ $Q\left(U\right)=\prod_{u_{g}\in U}{\ q\left(u_{g}\right)}=\prod_{u_{g}\in U}{ \prod_{u_{k}\in u_{g}}{f_{k}(u_{k})}},$ (8) $$

in equation 8, $u_{k}$ represents fact $u_{k}$, $u_{g}$ represents all facts in a ground rule $g$, and $U$ is the set of unknown facts. Each fact probability distribution $f_{k}(u_{k})$ follows a Bernoulli distribution, where the truth value is 1 when the fact occurs and 0 when it does not, i.e., $f_{k}(u_{k})=p_{k}^{u_{k}}(1-p_{k})^{(1-u_{k})}$. The probability $p_{k}$ of the fact occurring is obtained from the scoring module.

The truth value of each ground rule is jointly determined by the truth values of its constituent facts. Therefore, the number of ground rules is represented as:

$$ $N(F,U)=\sum_{u_{g}\in u_{r}}\prod_{u_{k}\in u_{g}}u_{k},$ (9) $$

where $u_{r}$ represents the set of facts belonging to rule $r$. Thus, equation 1 can be defined as:

$$ $P(\omega|F,U)=\frac{1}{Z(\omega)}\prod_{r\in R}\exp\left(\omega_{r}\sum_{u_{g} \in u_{r}}\prod_{u_{k}\in u_{g}}u_{k}\right).$ (10) $$

Substituting equations (8) and (10) into the optimization function (5), the term $Z(\omega)$ can be treated as a constant, leading to:

$$ $\sum_{r\in R}\omega_{r}\sum_{u_{g}\in u_{r}}\prod_{u_{k}\in u_{g}}\ p_{k}-\sum _{r\in R}\sum_{u_{g}\in u_{r}}\sum_{u_{k}\in u_{g}}\left(\left(\ 1-p_{k}\right )\log{\left(1-p_{k}\right)}+p_{k}logp_{k}\right).$ (11) $$

This paper constructs the score $d_{fact}$ of the known fact set $F$ to add constraints. We want the score $d_{fact}$ of the positive sample to be as small as possible. The final objective function is defined as:

$$ $\sum_{r\in R}\left(\omega_{r}\sum_{u_{g}\in u_{r}}\prod_{u_{k}\in u_{g}}\ p_{k }-\sum_{u_{g}\in u_{r}}\sum_{u_{k}\in u_{g}}\left(\left(1-p_{k}\right)\log{ \left(1-p_{k}\right)}+p_{k}logp_{k}\right)+d_{fact}\right).$ (12) $$

### 4.3 M-step

In the M-step, we fix $Q(U)$ and then update the weights $\omega_{r}$ of the rule set $R$. At this point, the partition function in equation 2 from the E-step is no longer a constant. Therefore, in the M-step, we optimize the rule weights by minimizing the negative of the ELBO. However, when dealing with large-scale knowledge graphs, the number of facts also becomes enormous, making it difficult to optimize the ELBO directly. Consequently, we adopt the widely used pseudo-log-likelihood [39] as an alternative optimization objective, defined as:

$$ $P\left(F,U|\omega\right):=\sum Q(U)\left(\sum_{u_{k}\in U}logP(u_{k}|\omega,MB _{k})\right).$ (13) $$

$MB_{k}$ represents the Markov Blanket of an individual fact $k$ in a ground rule. Therefore, following existing studies [][], for each grounding formula $k$ connecting the base predicate with its Markov Blanket, we optimize the weights using the gradient descent formula:

$$ $\nabla_{\omega_{k}}\sum f\left(u_{k}\right)\left(logP\left(u_{k}\middle|\omega ,MB_{k}\right)\right).$ (14) $$

## 5 Experiment

### 5.1 Experiment Settings

We evaluate the NPLL method on four benchmark datasets through the knowledge base completion task and compare it with other state-of-the-art knowledge base completion methods.We will release the code after the publication of the paper.

Datasets. We evaluate our proposed model on four widely used benchmark datasets. Specifically, we use the UMLS dataset[], Kinship dataset[], FB15k-237 dataset[], and WN18RR dataset[]. FB15k-237 and WN18RR are more challenging versions of the FB15K and WN18 datasets. The dataset statistics are summarized in Appendix A.

Evaluation metrics. Following existing studies[], we use the filtered setting during evaluation. Mean Reciprocal Rank (MRR), Hit@10, Hit@3, and Hit@1 are treated as the evaluation metrics.

Competitor methods: We compare knowledge graph embedding methods, rule-based methods, and methods combining the two. For knowledge graph embedding methods, we select some of the most classic distance translation and semantic matching algorithms, including TransE[], DistMult[], ComplEx [], ConvE[], RotatE[]. For rule-based reasoning algorithms that integrate rules, we compare with NeuralLP[], DRUM[], pLogicNet[], ExpressGNN[], DiffLogic[], NCRL[]. The comparative experiments are conducted under the same experimental conditions, selecting the best training hyperparameters provided by the open-source codes of each algorithm. For our method, we consider two variants: NPLL-GNN, which utilizes a tunable graph neural network[] in the scoring module for training, and NPLL-basic, which employs only a single-layer embedding network in the scoring module for training.

Experimental setting: To generate logical rules, we use the NeuralLP method on the training set to generate candidate rules and select the candidate rules with the highest confidence scores. On different datasets, we create candidate rules with different parameters. See the Appendix B for details on specific rule selection.For NPLL,we use 0.0005 as the initial learning rate, and decay it by half for every 10 epochs without improvement. For the four datasets FB15k-237, WN18RR, Kinship, UMLS, the embedding size of NPLL-basic are 256, 256, 512, 128.

General setting: All experiments are conducted on the same server with two GPUs (Nvidia RTX 2080Ti, 11G), using Cuda version 10.2, Ubuntu 16.04.6 system, and Intel(R) Xeon(R) CPU E5-2620 v3 @ 2.40GHz CPU.

### 5.2 Results

**Table 1: Results of KG completion. We select the metrics provided in the papers for the DiffLogic and NCRL algorithms from the rule-learning methods, as we could not find suitable open-source codes for them. (The red numbers indicate the best performance achieved on a particular metric.) Hit@K is in %.**
| Methods | Models | FB15k-237 | WN18RR |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| MRR | Hit@10 | Hit@3 | Hit@1 | MRR | Hit@10 | Hit@3 | Hit@1 |  |  |
| KGE | TransE | 0.33 | 52.71 | 29.28 | 18.93 | 0.2231 | 52.12 | 40.10 | 1.31 |
| DistMult | 0.2878 | 45.67 | 31.43 | 20.31 | 0.4275 | 50.71 | 44.01 | 38.21 |  |
| ComplEx | 0.3016 | 48.08 | 33.10 | 21.28 | 0.4412 | 51.03 | 46.11 | 41.01 |  |
| ConvE | 0.3251 | 50.11 | 35.68 | 23.80 | 0.4295 | 52.13 | 44.34 | 39.87 |  |
| RotatE | 0.3213 | 53.10 | 34.52 | 22.81 | 0.4714 | 55.71 | 47.29 | 42.87 |  |
| Rule-Learning | Neural LP | 0.1983 | 29.84 | 21.73 | 14.48 | 0.3800 | 40.79 | 38.81 | 36.80 |
| DRUM | 0.2430 | 36.39 | 21.91 | 17.43 | 0.3861 | 41.02 | 38.93 | 36.91 |  |
| pLogicNet | 0.3300 | 52.79 | 36.87 | 23.12 | 0.2300 | 53.09 | 41.48 | 1.5 |  |
| ExpressGNN | 0.4894 | 60.80 | 48.10 | 38.91 | - | - | - | - |  |
| NCRL | 0.3000 | 47.30 | - | 20.90 | 0.6700 | 85.00 | - | 56.30 |  |
| DiffLogic | - | - | - | - | 0.5001 | 58.70 | - | - |  |
| us | NPLL-basic | 0.6223 | 68.57 | 64.52 | 58.63 | 0.7668 | 78.14 | 77.38 | 75.83 |
| NPLL-GNN | 0.5442 | 61.93 | 57.06 | 50.25 | 0.5282 | 61.52 | 55.50 | 48.17 |  |
| Methods | Models | Kinship | UMLS |  |  |  |  |  |  |
| MRR | Hit@10 | Hit@3 | Hit@1 | MRR | Hit@10 | Hit@3 | Hit@1 |  |  |
| KGE | TransE | 0.3509 | 80.36 | 50.14 | 1.10 | 0.7806 | 99.13 | 96.05 | 59.56 |
| DistMult | 0.3925 | 77.86 | 42.68 | 23.73 | 0.4770 | 78.83 | 53.87 | 33.57 |  |
| ComplEx | 0.7201 | 95.91 | 80.86 | 59.73 | 0.8950 | 98.34 | 95.58 | 82.70 |  |
| RotatE | 0.4890 | 86.95 | 56.32 | 32.41 | 0.5884 | 83.41 | 68.33 | 44.23 |  |
| Rule-Learning | Neural LP | 0.5637 | 88.00 | 63.94 | 41.49 | 0.7312 | 91.29 | 84.70 | 59.37 |
| DRUM | 0.3312 | 70.15 | 48.23 | 25.67 | 0.5634 | 85.64 | 65.58 | 35.79 |  |
| NCRL | 0.6400 | 92.90 | - | 49.00 | 0.7800 | 95.10 | - | 65.90 |  |
| us | NPLL-basic | 0.8663 | 92.68 | 87.91 | 83.55 | 0.9763 | 99.21 | 98.26 | 96.76 |
| NPLL-GNN | 0.7705 | 87.55 | 79.09 | 71.77 | 0.9754 | 99.05 | 98.66 | 96.45 |  |

**Table 2: A comparison of the model parameter counts for NPLL-basic, NPLL-GNN, and ExpressGNN methods on the FB15k-237 dataset**
|  | Models | FB15k-237 |
| --- | --- | --- |
| Total params count(k) | ExpressGNN | 251,337k |
| NPLL-basic | 64,967k |  |
| NPLL-GNN | 64,953k |  |

Figure: (a) MRR
Refer to caption: extracted/5709768/MRR.png

KG completion performance analysis. The experimental results are shown in Table 1. The NPLL-basic and NPLL-GNN methods achieve good performance across all four datasets. On the FB15k-237 and UMLS datasets, the NPLL-basic method significantly outperforms other methods, achieving the best results on all four metrics. On the WN18RR and Kinship dataset, NPLL-basic and NPLL-GNN comprehensively outperform the data-driven embedding methods, while NPLL-basic achieve the best results on the MRR, Hit@3, and Hit@1 metrics. This indicates that the reasoning effectiveness and expressiveness of NPLL have been enhanced.

Performance analysis of NPLL-basic and NPLL-GNN. Compared to NPLL-GNN, NPLL-basic performs better on the FB15k-237, WN18RR, and Kinship datasets. On the UMLS dataset, their performances are similar.

Parameter counts. The terms of model parameter counts, we compare NPLL with the ExpressGNN method, which has relatively high overall performance among the baseline methods on the FB15k-237 dataset. As shown in Table 2, the parameter count of our method is approximately one-fourth of ExpressGNN.

**Table 3: Results on the FB15k-237 dataset with various data sizes. Hit@K is in %**
| Models | FB-0 | FB-0.05 |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- |
| MRR | Hit@10 | Hit@3 | Hit@1 | MRR | Hit@10 | Hit@3 | Hit@1 |  |
| TransE | 0.2412 | 42.71 | 26.39 | 16.10 | 0.2523 | 43.09 | 26.87 | 16.43 |
| Neural LP | 0.0128 | 1.75 | 0.73 | 0.41 | 0.1531 | 24.51 | 16.72 | 10.43 |
| DistMult | 0.2297 | 38.87 | 25.02 | 15.10 | 0.2317 | 39.28 | 25.13 | 15.25 |
| CompIEx | 0.2363 | 40.29 | 25.72 | 15.47 | 0.2395 | 40.70 | 25.98 | 15.75 |
| ExpressGNN | 0.4276 | 53.88 | 45.74 | 36.65 | 0.4187 | 54.24 | 44.89 | 35.50 |
| NPLL-basic | 0.5356 | 62.55 | 56.87 | 51.03 | 0.5384 | 63.09 | 57.84 | 51.38 |
| NPLL-GNN | 0.4989 | 58.78 | 52.95 | 44.88 | 0.4911 | 58.15 | 52.23 | 44.07 |
| Models | FB-0.1 | FB-0.2 |  |  |  |  |  |  |
| MRR | MRR | MRR | MRR | MRR | Hit@10 | Hit@3 | Hit@1 |  |
| TransE | 0.2531 | 43.41 | 26.92 | 16.68 | 0.2533 | 43.92 | 27.13 | 16.81 |
| Neural LP | 0.1624 | 25.88 | 17.81 | 11.16 | 0.1699 | 26.79 | 18.53 | 11.86 |
| DistMult | 0.2333 | 39.47 | 25.36 | 15.37 | 0.2371 | 40.07 | 25.80 | 15.64 |
| CompIEx | 0.2409 | 40.74 | 26.24 | 15.89 | 0.2451 | 41.63 | 26.71 | 16.16 |
| ExpressGNN | 0.4226 | 55.30 | 45.49 | 35.91 | 0.4273 | 55.59 | 45.81 | 36.34 |
| NPLL-basic | 0.5466 | 63.40 | 57.20 | 51.93 | 0.5594 | 63.62 | 57.57 | 52.11 |
| NPLL-GNN | 0.5241 | 59.66 | 54.85 | 48.33 | 0.5307 | 60.55 | 55.69 | 48.91 |

Analysis of data efficiency. We investigate the data efficiency of NPLL-basic and NPLL-GNN, and compare them with baseline methods. We divide the FB15k-237 knowledge base into fact/train/valid/test files[], and vary the size of the train set from 0% to 20%, while providing the complete fact set to the models. The results can be seen in Table 3. In Figures 2, the NPLL methods are shown as solid lines, while other methods are dashed lines. We can clearly see that NPLL performs significantly better than the baselines with smaller training data. Even with more training data for supervision, NPLL still exhibits excellent performance across all metrics. This clearly demonstrates that NPLL can more accurately predict the correct answers and has outstanding data utilization ability.

**Table 4: Results on the zero-shot dataset constructed from the FB15k-237 dataset. Hit@K is in %.**
| Methods | Models | FB-zero-shot |  |  |  |
| --- | --- | --- | --- | --- | --- |
| MRR | Hit@10 | Hit@3 | Hit@1 |  |  |
| KGE | TransE | 0.0031 | 0.5 | 0.1 | 0.01 |
| DistMult | 0.0011 | 0.18 | 0.01 | 0.01 |  |
| ComplEx | 0.0042 | 6.76 | 0.36 | 0.24 |  |
| RotatE | 0.0061 | 1.98 | 0.13 | 0.01 |  |
| Rule-Learning | Neural LP | 0.0172 | 3.47 | 1.88 | 0.15 |
| ExpressGNN | 0.1851 | 29.59 | 21.47 | 12.93 |  |
| us | NPLL-basic | 0.4425 | 50.30 | 46.44 | 40.81 |
| NPLL-GNN | 0.2151 | 32.02 | 23.50 | 17.66 |  |

Zero-shot learning. In real-world scenarios, most relations in knowledge bases are long-tail relations, meaning most relations may have only a few facts. Therefore, it is very important to study the model’s performance on relations with insufficient training data. We use the zero-shot dataset constructed from FB15k-237[], which enforces disjoint relation sets between training and test data. Table 4 shows the results. As expected, the performance of all supervised relation learning methods based on embeddings almost drops to zero. This indicates the limitation of these methods in handling sparse long-tail relations. Compared to the rule-exploiting methods like NeuralLP and ExpressGNN, NPLL also shows significant improvement in dealing with long-tail data.

## 6 Conclution

In this paper, we study knowledge graph reasoning and propose a method called Neural Probabilistic Logic Learning (NPLL), which effectively integrates logical rules with data embeddings. NPLL utilizes neural networks to extract node features from the knowledge graph and then supports the reasoning of Markov Logic Networks through a scoring module, effectively enhancing the model’s expressiveness and reasoning capabilities. NPLL is a general framework that allows tuning the encoding network to boost model performance.

## Appendix

## Appendix A Dataset Details

To comprehensively evaluate the performance of our proposed method, we conducted extensive comparative experiments across four widely-adopted benchmark datasets: UMLS, Kinships, FB15k-237, and WN18RR. Additionally, to investigate the impact of dataset size on reasoning performance, we performed a splitting operation on the FB15k-237 dataset, creating four subsets: FB-0, FB-0.05, FB-0.1, and FB-0.2, where the Train file was divided into varying proportions. Furthermore, we constructed a zero-shot learning scenario, FB-zero-shot, derived from FB15k-237, where the relation sets of the training and test data are disjoint. The specific details and statistics of these datasets are provided in Table 5.

This diverse set of benchmark datasets allows for a comprehensive evaluation of our method’s reasoning capabilities across varying dataset sizes, knowledge graph complexities, and zero-shot learning settings. The UMLS and Kinships datasets represent domain-specific knowledge graphs, while FB15k-237 and WN18RR are more general-purpose knowledge bases. By including both small-scale and large-scale datasets, as well as the zero-shot learning scenario, we can thoroughly assess the robustness, scalability, and generalization abilities of our proposed approach under a wide range of conditions encountered in real-world knowledge graph reasoning tasks.

**Table 5: Knowledge base completion datasets statistics**
| Dataset | #Fact | #Train | #Test | #Valid | #Relation | #Entity | #Rules |
| --- | --- | --- | --- | --- | --- | --- | --- |
| UMLS | 4006 | 1321 | 633 | 569 | 46 | 135 | 1055 |
| Kinship | 6375 | 2112 | 1100 | 1099 | 25 | 104 | 71 |
| Fb15k-237 | 204087 | 68028 | 20466 | 17536 | 237 | 14541 | 516 |
| Fb-0 | 204087 | 1 | 20466 | 17536 | 237 | 14541 | 516 |
| Fb-0.05 | 204087 | 3401 | 20466 | 17536 | 237 | 14541 | 516 |
| Fb-0.1 | 204087 | 6802 | 20466 | 17536 | 237 | 14541 | 516 |
| Fb-0.2 | 204087 | 13605 | 20466 | 17536 | 237 | 14541 | 516 |
| Fb-zero-shot | 173486 | 57462 | 3178 | 14874 | 237 | 14541 | 516 |
| WN18RR | 65127 | 21708 | 3134 | 3034 | 11 | 40943 | 33 |

## Appendix B Logic Rules

For the selection of logical rules across the four benchmark datasets, we first generated candidate rules using the Neural LP[] method, a state-of-the-art rule mining approach. We then preprocessed the candidate rules by removing self-reflective logical rules and eliminating duplicates. Next, we applied a confidence score threshold, selecting all rules with a confidence score greater than a predefined parameter $\alpha$ for the same target predicate. Finally, we determined the most suitable logical rule set for each dataset through extensive experiments. Table 6 presents the performance metrics of the NPLL-basic method on the four benchmark datasets with different rule sets.

Based on the experimental results, we identified the optimal rule sets for each dataset: for the UMLS dataset, we selected rules with a confidence score greater than 0; for the Kinships dataset, rules with a confidence score greater than 0.99; for the FB15k-237 dataset, rules with a confidence score greater than 0.87; and for the WN18RR dataset, rules with a confidence score greater than 0.99. This systematic process of rule selection and empirical evaluation allowed us to identify the most suitable logical rules for each knowledge graph, ensuring that our proposed method leverages high-quality symbolic knowledge to enhance its reasoning capabilities.

**Table 6: The performance indicators of NPLL-basic under different confidence rules in four benchmark datasets. The embedding size is 128**
| UMLS($\alpha$) | MRR | Hit@10 | Hit@3 | Hit@1 |
| --- | --- | --- | --- | --- |
| 0.99 | 0.8029 | 82.15 | 80.24 | 78.32 |
| 0.9 | 0.8109 | 83.81 | 81.28 | 79.15 |
| 0.8 | 0.8885 | 91.15 | 89.34 | 87.36 |
| 0.7 | 0.8839 | 91.63 | 89.57 | 86.02 |
| 0.6 | 0.8985 | 92.65 | 90.6 | 87.99 |
| 0.5 | 0.926 | 96.76 | 94.23 | 90.13 |
| 0.4 | 0.9376 | 97.24 | 95.42 | 91.63 |
| 0 | 0.9729 | 99.21 | 97.95 | 96.29 |
| Kinship($\alpha$) | MRR | Hit@10 | Hit@3 | Hit@1 |
| 0.99 | 0.7884 | 89.18 | 81.09 | 73.68 |
| 0.97 | 0.7759 | 88.09 | 80.14 | 72.45 |
| 0.95 | 0.7744 | 87.64 | 79.36 | 72.64 |
| 0.93 | 0.7683 | 86.27 | 78.68 | 72.27 |
| 0.92 | 0.7474 | 85.14 | 76.50 | 69.04 |
| 0.91 | 0.7471 | 86.50 | 78.45 | 72.65 |
| 0.9 | 0.769 | 86.23 | 77.86 | 72.91 |
| 0.89 | 0.7671 | 84.91 | 77.68 | 71.86 |
| 0.85 | 0.7484 | 82.64 | 75.68 | 71.09 |
| 0.8 | 0.7302 | 80.27 | 74.59 | 69.05 |
| 0 | 0.1731 | 22.27 | 16.68 | 13.95 |
| fb15k-237($\alpha$) | MRR | Hit@10 | Hit@3 | Hit@1 |
| 0.99 | 0.5741 | 63.88 | 59.61 | 53.68 |
| 0.95 | 0.5295 | 61.16 | 55.83 | 48.29 |
| 0.9 | 0.5417 | 62.92 | 56.6 | 49.45 |
| 0.89 | 0.5951 | 66.77 | 62.04 | 55.53 |
| 0.873 | 0.6044 | 67.3 | 62.82 | 56.55 |
| 0.87 | 0.6113 | 68.03 | 63.88 | 57.11 |
| 0.867 | 0.6038 | 67.34 | 62.98 | 56.31 |
| 0.86 | 0.5606 | 64.25 | 58.65 | 51.5 |
| 0.84 | 0.5624 | 64.69 | 58.59 | 51.77 |
| 0.83 | 0.5945 | 66.87 | 61.71 | 55.47 |
| WN18RR($\alpha$) | MRR | Hit@10 | Hit@3 | Hit@1 |
| 0.99 | 0.6012 | 68.63 | 63.27 | 56.15 |
| 0.95 | 0.5157 | 62.28 | 53.99 | 46.01 |
| 0.9 | 0.4483 | 58.65 | 48.47 | 37.65 |
| 0.85 | 0.5348 | 62.01 | 53.38 | 50.40 |
| 0.83 | 0.5941 | 66.64 | 61.97 | 55.31 |
| 0.81 | 0.5698 | 64.06 | 59.78 | 52.70 |
| 0.7 | 0.4702 | 52.74 | 48.72 | 43.60 |
| 0.5 | 0.3848 | 42.50 | 39.68 | 36.08 |
| 0 | 0.0724 | 8.39 | 7.32 | 6.57 |

## Appendix C limitations

In our experimental evaluation, the proposed NPLL algorithm was benchmarked on four widely-used datasets: FB15k-237, UMLS, WN18RR, and Kinships. On the FB15k-237 and UMLS datasets, NPLL achieved state-of-the-art performance, outperforming existing methods. However, on the WN18RR and Kinships datasets, while NPLL surpassed data-driven embedding methods, its Hit@10 performance did not exceed that of the rule-based NCRL method. It is noteworthy that in the experiments with the NPLL algorithm, a base set of candidate rules is required as part of the input data. Furthermore, extensive experiments are necessary to identify the optimal rule set for each dataset, as the selection of rules can significantly impact the reasoning performance.

## Peer Reviews

confidence: 4
contribution: 2
initial_rating: 5
presentation: 2
questions: Line 101: The paper claims that it "is significantly more effective for knowledge graph reasoning" than the prior works mentioned in lines 84-101 without clearly describing how it is technically superior. Could the authors please elaborate on their technical contributions relative to each of the related works?

Line 187, Section 4:
Could the authors please compare and contrast their NPLL model with the closely related models pLogicNet and ExpressGNN? Which specific components introduced in the paper account for the stellar empirical results reported?

Line 324, Section 5:
Which empirical results in this section support the authors' answer to the above question regarding the most effective components of their approach?
soundness: 3
strengths: This paper is generally well-written, though the clarity of some parts could be improved.

The experiments purportedly show huge gains by the paper's approach on standard benchmarks.
summary: This paper explores the combination of embedding-based methods with rule-based reasoning approaches to address their individual shortcomings when used separately. The authors introduce Neural Probabilistic Logic Learning (NPLL), a framework that combines the strengths of embeddings and rule-based reasoning, which purportedly achieves efficient, large-scale knowledge graph reasoning. NPLL seemingly demonstrates high reasoning performance, even in data-scarce conditions, balancing model size with reasoning capability to enable practical applications.
weaknesses: This paper does not clearly position its technical contributions with respect to prior research. Its approach resembles that of pLogicNet (Qu and Tang, 2019) and ExpressGNN (Zhang et al., 2020), both of which the paper cites. Both pLogicNet and ExpressGNN employ the same variational-EM approach as this paper, as well as the computational simplifications of mean-field approximation and pseudo-log-likelihood optimization. In light of these existing works, the primary contribution of this paper appears to be the introduction of the triplet scoring function in Equation 7, which is essentially a simple feed-forward network. Relative to prior work, this contribution seems incremental at best. Furthermore, in the context of existing embedding-based approaches, the triplet scoring function (Equation 7) may be redundant, as current embedding-based methods can produce scores using the same inputs (embeddings for entities and relations).

Although the empirical results show significant improvements over baseline scores, the paper’s exposition does not clearly explain how these improvements are achieved through the authors’ approach.

---

confidence: 3
contribution: 2
initial_rating: 3
presentation: 1
questions: 1. Novelty: Beyond the addition of the scoring module, what are the other key differences between this approach and ExpressGNN? It would be helpful if the authors could elaborate on any unique elements or improvements this model brings, especially regarding interpretability, efficiency, or theoretical grounding.

2. Impact of different data sizes on model performance: The experiments with varying data sizes in the FB15k-237 dataset reveal minimal impact on model performance, but the reasoning behind this result is not addressed. Could the authors provide an analysis of why this might be the case? It would also be valuable if additional insights could be given on the robustness of the model in data-scarce environments or if alternative metrics could show nuanced performance variations.

3. Training time comparisons: Since NPLL utilizes rules derived from a pre-trained Neural-LP model on specific datasets, I suggest that the reported training time should also include the time required to train the Neural-LP model initially (Table 6).
soundness: 2
strengths: 1. The methodological improvement introduced in this paper that leverages an embedding-based scoring module is straightforward. However, this simplicity contributes to the model’s robustness and ease of implementation, and the results achieved are notably impressive.

2. The experimental evaluation is thorough, covering a wide range of benchmark knowledge graphs. This comprehensive testing approach not only underscores the model’s versatility but also consistently demonstrates superior performance across diverse datasets, reinforcing the robustness and broad applicability of the proposed method.
summary: 1. The paper introduces Neural Probabilistic Logic Learning (NPLL), a rule-based method for knowledge graph reasoning. NPLL represents knowledge using a Markov Logic Network (MLN), enhancing the expressiveness of embedding networks. Through variational inference, NPLL accurately infers unknown facts and introduces a scoring module to improve reasoning accuracy in knowledge graphs.

2. The authors conduct extensive experiments across various benchmark datasets, including YAGO3-10, YAGO37, Codex-L, WN18RR and FB15k-237. The results demonstrate that NPLL achieves superior reasoning performance, surpassing other methods on large-scale and domain-specific datasets. This validation highlights NPLL’s effectiveness in complex knowledge graph reasoning tasks.
weaknesses: ## 1 Novelty Issue
1. The proposed methodology closely resembles the approach used in ExpressGNN [1], with the primary difference being the addition of a scoring module on factual triples $(e_h, l, e_t)$. It remains unclear what further distinctions, if any, exist between this model and ExpressGNN, raising concerns regarding the novelty of this contribution.

## 2 Insufficient and Unjustified Experimentation
1. Two model variants, NPLL-basic and NPLL-GNN, are proposed, with reasoning results provided for each (Table 2). However, no explanation is given for the significantly lower accuracy of NPLL-GNN compared to NPLL-basic, leaving questions about model performance unaddressed.

2. The evaluation of data efficiency (Table 4) is limited to the FB15k-237 dataset, replicating results that have already been demonstrated in ExpressGNN. Since this work (NPLL) and ExpressGNN [1] follow the same framework (MLN), results on additional datasets would help clarify the model’s data efficiency. Additionally, the impact of varying data sizes on NPLL-basic performance is minimal (Table 4), but this observation is neither analyzed nor discussed.

3. In the Related Work section, the authors claim that compared to embedding-based methods, NPLL enhances both interpretability and reasoning quality. However, no experimental evidence is provided to substantiate this claim.

## 3 Representation
1. Improper use of mathematical symbols: The mathematical formulations are often imprecise, with symbols inconsistently defined or unclear. For example, the definitions of “fact” in the preliminary section are ambiguous, and the MLN setup and representation of unknown facts are incomplete. Including concrete examples would improve clarity and reader comprehension. In the Model section, certain notations (e.g., $u_g$, $u_k$ in Equation 8) are confusing and inadequately defined, making this section challenging to follow.

2. The text and curves in the all figures, especially Figure 2 and 3,  are difficult to read due to their small size, limiting accessibility to critical information.

3. Significant details are missing in the methods section, such as specifics on model training and the E-step and M-step processes. This lack of detail, particularly compared to ExpressGNN, further underscores the concerns about novelty in this work.

[1] Zhang, Y.; Chen, X.; Yang, Y.; Ramamurthy, A.; Li, B.; Qi, Y.; Song, L. Efficient Probabilistic Logic Reasoning with Graph Neural Networks. arXiv February 4, 2020. https://doi.org/10.48550/arXiv.2001.11850.

---

confidence: 4
contribution: 2
initial_rating: 3
presentation: 1
questions: Please check the questions in weakness points.
soundness: 2
strengths: 1. The proposed method outperforms the baseline methods by a large margin.
2. The proposed method also performs well in the data-scarce cases.
3. The proposed method is parameter efficient.
4. Code is provided.
summary: This paper proposes a rule-based KG reasoning method. The main idea is based on markov logic network. Compared with baseline method, the proposed method is more effective and performs the best over several benchmarks. In particular, the proposed NPLL is effective in data-scarse scenarios.
weaknesses: 1. This paper is ill-written. The motivation is extremely unclear. Over the entire paper, it is hard for me to capture what problem this paper want to address and how the proposed method is motivated.
2. The advantage of NPLL over other methods is not well discussed. The authors frequently claim that NPLL is more effective than the others. However, in what aspects and why? I can't understand.
3. As for the methodology, there lack of an overall picture of the whole framework. Just talk about what they do with many details for about 3.5 pages. The visualization of the framework in Figure 1 is also not well illustrated.
4. For the contributions, I don't know why the second and third properties are important. In other words, how these properties or designs benefit the KG reasoning problem is not clear.
5. Section 2 mentions interpretability, but experimental results do not show this point.
6. There are many typos and inappropriate expressions. For example,
- Section 3 uses mixed expressions of italic and normal fonts, e.g., E and $E$, L and $L$, fi and $f_i$.
- In line 142, what is $y_i'$?
- exp in Equations 1 and 2 with different fonts.
7. The font sizes in figure 2 and 3 are too small.

---

confidence: 3
contribution: 1
initial_rating: 1
presentation: 1
questions: Why did you not discuss the close similarity with pLogicNet and ExpressGNN in the paper?

Why did you not analyse where the performance improvements compared to pLogicNet and ExpressGNN are coming from, given the very close similarity with these models?
soundness: 1
strengths: Studying new methods in which embedding and rule based methods can be combined is clearly of interest.

The experimental results are very good, spectacular even.
summary: The paper studies the popular problem of knowledge graph completion. The paper contrasts rule based methods, on the one hand, with embedding based methods, on the other hand. Rules are said to be more accurate but less efficient, while embeddings are said to be less accurate and more efficient. The paper therefore proposes a strategy for combining the advantages of both, by using a variational approximation of Markov logic networks.
weaknesses: The core idea of this paper is identical to that of the pLogicNet paper, which also proposes a variational approximation of Markov logic networks based on embeddings. Furthermore, ExpressGNN builds on pLogicNet by using GNNs instead of embeddings, and this paper similarly analyses a variant based on GNNs. It is not clear what is novel about the proposed model compared to these two earlier models. Worryingly, while pLogicNet and ExpressGNN are cited in the paper, no mention at all is made of the close correspondence. All that is said about pLogicNet, for instance, is that it is a "probabilistic logic reasoning network ... demonstrating exemplary performance".  If there is a conceptual difference with pLogicNet which I missed, the paper should have discussed this explicitly. 

The experimental results are substantially better than those of existing models, including those of pLogicNet. Given the close similarity with pLogicNet, this makes the validity of the results questionable. At a minimum, the paper should have analysed the differences with pLogicNet. For instance, to what extent can the performance differences be explained by the fact that both methods start from a different set of rules? If this does not explain the difference, then what is responsible for this huge performance gap?

The GNN variant of the model is introduced while discussing the experimental results, but is never properly explained. Table 3 shows that it has only a quarter of the parameters of ExpressGNN, but without further details, this seems to be a matter of different hyper parameter tuning, rather than any genuine difference.

The paper is poorly written. For instance, even the motivation in the introduction doesn't really make sense. Knowledge graphs are said to capture "rich semantics" and offer "more expressive" representations than traditional methods, which doesn't make sense to me. The technical details, for instance in Section 4.1, are very hard to follow.