Neural Markov Logic Networks
Authors: Giuseppe Marra
Ondřej Kuželka
Venue:  (2020)
Keywords: Statistical Relational Learning
Markov Logic Networks

Abstract: We introduce Neural Markov Logic Networks (NMLNs), a statistical relational learning system that borrows ideas from Markov logic. Like Markov Logic Networks (MLNs), NMLNs are an exponential-family model for modelling distributions over possible worlds, but unlike MLNs, they do not rely on explicitly specified first-order logic rules. Instead, NMLNs learn an implicit representation of such rules as a neural network that acts as a potential function on fragments of the relational structure. Interestingly, any MLN can be represented as an NMLN. Similarly to recently proposed Neural theorem provers (NTPs) (Rocktaschel at al. 2017), NMLNs can exploit embeddings of constants but, unlike NTPs, NMLNs work well also in their absence. This is extremely important for predicting in settings other than the transductive one. We showcase the potential of NMLNs on knowledge-base completion tasks and on generation of molecular (graph) data.

Under review as a conference paper at ICLR 2020
NEURAL MARKOV LOGIC NETWORKS
Anonymous authors Paper under double-blind review
ABSTRACT
We introduce Neural Markov Logic Networks (NMLNs), a statistical relational learning system that borrows ideas from Markov logic. Like Markov Logic Net- works (MLNs), NMLNs are an exponential- famill model for modelling distri- butions possible worlds, but unlike MLNs, they do not rely explicitly specified first-order logic rules. Instead, NMLNs learn an implicit representation of such rules neural network that acts a potential function on fragments of the relational structure. Interestingly, any MLN can be represented as an NMLN, Similarly recently proposed Neural theorem provers (Rocktäschel & Riedel, 2017), NMLNs can exploit embeddings of constants but, unlike NTPs, NMLNs work well also in their absence. This extremely important for predicting in set- tings other than the transductive one. We showcase the potential of NMLNs on knowledge-band completion tasks and on generation of molecular (graph) data.
INTRODUCTION
Parameters for a statistical relational model are typically estimated from one or more examples of relational structures that typically consist of a large number of ground facts. Examples of such structures are social networks (e.g. Facebook), protein-protein interaction networks, the Web, etc. A challenging task is to learn probability distribution over such relational structures from one or few examples. One solution is based on the assumption that the relational structure has repeated regularities; this assumption is implicitly explicitly used in most works statistical relational learning. Then, statistics about these regularities can be computed for small substructures of the training examples and used to construct distribution over the relational structures. Together with the maximum-entropy principle, this leads to distributions such as Markov logic networks (Richard- son & Domingos, 2006; Kuželka et al., 2018)
In this paper, we propose Neural Markov Logic Networks (NMLN). Here, the statistics which are used to model the probability distribution not known in advance, but modelled as neural networks trained together with the probability distribution model. This is very powerful when com- pared to classical MLNs, where either domain experts are required design some useful statistics about the domain of interest by hand (i.e. logical rules) or structure learning based on combinatorial search needs to be performed. These requirements normally limit a wide application of these models as out-of-the box tools. It is worth noticing that overtaking the need of such "feature-engineering" one of the reasons behind the massive adoption of deep learning techniques. However, not much has been done in the same direction by the statistical relational learning community. Moreover, de- signing statistics as neural networks allows more fine-grained description of the data, opening the doors to applications of our model to the generative setting.
CONTRIBUTIONS
The main contributions of this work are: (i) we introduce a new statistical relational model, which overcomes actual limitations of both classical and recent related models such as (Richardson & Domingos, 2006; Rocktäschel & Riedel, 2017; Sourek et al., 2018); (ii) we propose a theoretical justification of the model as naturally emerging from principle of Min-Max-entroppy (iii) pro- vide a Tensorflow implementation of this model; and (iv) we showcase its effectiveness on two quite diverse problems: knowledge-hase completion and generative modelling of small molecules.

---

Under review as a conference paper at ICLR 2020
RELATED WORK
The need to extend relational models with neural components is a topic that has been receiving increasing attention in the last few years. An infegration of logic reasoning and neural models was proposed based on fuzzy logic (Serafini & Garcez, 2016; Diligenti et al., 2017; Marra et al., 2019). Here, neural models implementing FOL relations are optimized in order to satisfy differentiable approximations of logical formulas obtained by means of fuzzy t-norm theory. However, the lack of probabílistic arguments allows a sound application of such fuzzy-logic based methods only to hard-constrained settings. In Manhaeve et al. (2018), the probabilistic logic programming language ProbLog (De Raedt et al., 2007) is extended to allow probabilities neural networks and to exploit differentiable algebraic generalizations of decision diagrams to train these networks. Lifted relational neural networks (Sourek et al., 2018) unfold neural networks with shared weights, as in convolutional networks, using forward chaining. A semantically very similar approach was the one in Rocktäschel & Riedel (2017), where the authors implemented the Prolog backward chaining with a soft unification scheme operating constants and relations embeddings. The proposed Neural Theorem prover was able to exploit the geometry of the embedding space to improve its reasoning capabilities, but the need for embeddings prevents this model be applicable to settings different from the transductive one. Most importantly, neither of these latter two works provides means for probabilistic modelling of relational structures.
atoms to be predicted by
The idea of exploiting neural networks to extract regularities in non- euclidean settings has been re- cently revisited by the deep learning community in the context of Graph Neural Networks (GNN) models (Scarselli et al., 2009; Defferrard et al., 2016; Xu et al., 2018). In GNNs, latent representa- tions of nodes are obtained by an aggregation of neighboring nodes representation by means of an iterative diffusion mechanism. However, the inference is performed only on neighborhoods induced by the actual connections of the graph, preventing the exploitation of these models for modeling distributions of structural properties of the graph. Lippi & Frasconi (2009) was early attempt to integrate MLNs with neural components. Here, an MLN was exploited to describe a conditional distribution over ground atoms, given some features of the constants. In particular, the MLN was reparametrized by a neural network evaluated on input features. However, this method still relied on hand-crafted logical rules for modelling the distribution.
PRELIMINARIES
This paper follows the setting of so-called Model A from Kuželka et al. (2018). We consider function-free first-order logic language L, which is built from a set of constants C and predicates R -U R,, where R2 contains the predicates of arity i. For C2.... Cm C and R R... call R(C), ......, Cm) a ground atom. We define possible world w E Ω as the pair (C, A), where C C. A is a subset of the set of all ground atoms that can be built from the constants in C and any relation in R. We define the size of possible world n = |C] and Ω is the set of all possible worlds. fragment (SS) is defined as the restriction of w to the constants in S. It is again a pair wSS) = (S, B), with S the constants of the restriction and B set of ground atoms which only use constants from S. Given a fragment (S) and = [S), we can anonymize it by mapping the constants in S with a permutation S of the integer set {1,2,... .J. We call this an anonymized fragment 7. Suppose we have a given world @ of size n. define (@) the collection of all the anonymized fragments of width k of W. It is casy to verify that [@) (T)) The collection (@) is multiset, since, after anonymization, multiple fragments could be identical. An example of the process of anonymization and of the identification of structural patterns among anonymized fragments is shown in Figure 1.
3 NEURAL MARKOV LOGIC NETWORKS
3.1 INTUITION AND FORMULATION
Given a world E Ω we are interested in models of the probability Pas for a generic w E Ω. To this end, we want to compute statistics on fragments of the given @ and exploit them to con- struct a distribution on (possibly larger and smaller) relational structures w. Let us define o(7) as fragment potential function, which is simply a function from anonymized fragments of width k
2

---

Under review as a conference paper at ICLR 2020
D
C
8
(a) The global example
(b) Two frag- ments.
(c) Anonymizations of the two fragments.
The process of individuating structural patterns in anonymized fragments. White circles represent constants, while the two relations are represented as solid and dashed arrows (ab- sence of an arrow means that the relation is false). The given world is shown the left. Two possible fragments are shown in the middle. All their possible anonymizations are shown on the right. Grey circles show two repeated anonymized fragments found in two different fragments. The model exploits these regularities on fragments to model the distribution the larger structure.
Figure 1:
to real numbers. We can search for a maximum-cntropp distribution p(w) such that the following two expected values are the same: (i) the expected value of Φ(7) where y sampled uniformly from (@) and (ii) the expected value of Φ(>') where y is sampled uniformly from (w) and w sampled, in turn, from p(w). The intuition here that, least on average. the fragments of the given training example should look similar to the fragments of possible worlds sampled from the distribution. It follows from (Kuželka et al., 2018) that the resulting maximum-entropy distribution is an aaponential-familn distribution resembling Markov logic networks.
The max-entropy setting provides us with sound starting point for designing statistical relational models that generalize classical models such as Markov random fields. However, necessary con- dition for these models be designed that one can provide set of statisties ((w) describing the data. In this section, we show how to get rid of the need to provide statistics in advance and how to learn these statistics together with the probabilistic model a differentiable manner.
Let us consider a fragment neural potential function ((7; w). is parametric function with param- eters w. Let ((w; w) [2[=]] Exera p(7;; w) be the corresponding global neural potential.
We need a learning principle which would allow us to find good @(7; w) to describe our data. To this end, suppose we need to solve the maximum-entropy optimization problem, but without any constraint on the statistics. The maximum-entropi solution in this case is the uniform distribution, which assigns equal probability to all possble worlds w. Now, suppose we add single constraint on a potential (w)). this potential is informative and makes some worlds more likely than others, then the solution moves from the uniform distribution to another distribution with lower entropy. Using this intuition, we can have a scheme where we maximize entropy by selecting the maximum- entropy distribution and minimize it at the same time by choosing the most informative statisties.
The above considerations give rise to a Min-Max Entrop model for the target probability distri- bution P.. which call Neural Markov Logic Network and which describe in turn. Let us first define the Max-Entropy problem with the new neural potentials (stated here as minimization of negative entropy):
min Pa
Σ Ps log Pa
(1)
subject
Σ Pa 1, Yw: Pw 20
(2)
Vi: Ep, (w; = (.:
with 0 0 M
(3)
3

---

Under review as a conference paper at ICLR 2020
For fixed w,'s, use Lagrangian duality to obtain the following solution of the maximum entropy problem: Pa = exp (E, (, (w))..). Here, Z is a normalization constant and the pa- rameters 3, are solutions of the dual problem max
Σ == ( w.) - log } which coincides
8
with maximum-likelihode
Next we still need to incorporate the minimization of entropy by optimizing w,'s. 's. Let de- the entropy of the distribution Pa- Now, as previously introduced, the selection of the optimal values W; is governed by the principle of minimiza- tion of entropy, leading to the optimization problem: min max (B1.......... ,,M W1...., War) =
note by H(3 BM,, Wi....; WM)
maxx min ((I.....,, Bas, WI...., Was) subject to the constraints (2) and (3). Plugging in the w. 8, dual problem and using strong duality, we obtain the following unconstrained optimization problem which is equivalent to the maximization of log-likelihoodi maxw..0, {I:. 3,0,(0; w.) w log Z }
The maximization of the log-likelitee likeeiihoodd will be carried out by gradient- based optimization scheme. The gradients of the log-likoo ligllike w.rt. to both the parameters wi,j where Wa,5 denotes the j-th component of wa, and B are:
i log( Po)
[a (Q; Boo
Ta wi)
Bi
(4)
Owijj
Ows,
Owss
i log( (Pg)
4 (w: Ew De(w;
(5)
03;
At stationary point, Eq. 5 recovers the initial constraint on statisties imposed in the maximization of the entropy. However, the minimization of the entropy is mapped to a new requirement: at stationary conditions, the expected value of the gradients of the 4, under the distribution must match the gradients of the 4 evaluated at the data points.
3.2 VECTOR EMBEDDINGS OF DOMAIN ELEMENTS
By anonymizing fragment, the model loses any trace of the identity of the constants involved in it, preserving only their structural behaviours. While this feature is essential to allow the identification of structural patterns also inside a single possible world, it prevents the model from having different behaviour on specific constants. This, instead, a basic feature of many existing transductive mod- els, like NTP (Rocktäschel & Riedel, 2017), which exploit the geometry of latent representation space of constants to improve their prediction capabilities.
To this end, we define an embedding frabeeddinggfrggeen neural potential (e S; w, +), which is function of the anonymized fragment but also of the specific constants involved 5). In particular, transductive settings, we always have a possible world and we use the same constant set S both during learning and inference. Let θ E Rxxd be variable embedding matrix. can be considered map from the constant set S to a latent real domain Rd, ; i.c. the embedding space. Let c(S, θ) be a function that concatenates the k rows of @ corresponding to the & constants in the restricted set S. Thus, the embedding fragment neural potential de can be seen as a function of both y, which encodes the structural properties of the fragment and c(S, θ), which encodes the identity of constants by providing a latent representation for them. In other words, Φe(7, S: w, Q) f(7, c(,,0);) w) for some neural function J parameterized by w. This is inspired by works in the NLP community (Mikolov et al., 2013; Mnih & Kavukcuoglu, 2013), where the e function can have different forms than concatenation. The components of the embedding vectors are treated as any other weights of the potential functions and are updated using gradients computed according to Eq. 4. Intuitively, the contrastive nature of the learning (Bordes et al., 2013; Trouillon et al., 2017),
(i.e. the list of constants
We note that the derivation of the dual problem follows easily from the derivations in (Kuželka et al., 2018), which in turn rely on standard convex programming derivations from (Boyd & Vandenberghe, 2004: Wainwright et al., 2008). Throughout this section we assume that positive solution exists, which is needed for the strong duality hold; this is later guaranteed by adding noise during learning.
4

---

Under review as a conference paper at ICLR 2020
leads to the development of similar embeddings for similar constants. As we show in Section 4.2, the addition of embedding of constants helps improving the prediction capability of our model in transductive settings.
3.3 INFERENCE
In order to design an optimization procedure to learn Neural Markov Logic Networks, we need to rely on some methods sample from the distribution. In this paper, we exploit MCMC methods, in particular approximate Gibbs Sampling (GS) (Robert & Casella, 2013), to sample from Neural Markov Logic Networks. The approximation comes from the fact that GS requires large number of steps before converging to the target distribution. However, we run it only for a limited number of steps which, in some cases, is restricted to t 1. When this happens, discrete version of the Contrastive Divergence (CD) algorithm (Hinton, 2002).
method recovers
Gibbs sampling cannot effectively handle distributions with lot of determinism. In normal Markov logic networks, sampling from such distributions may be tackled by an algorithm called MC-SAT (Poon & Domingos, 2006). However, MC-SAT requires explicit logical encoding of the deter- ministic constraints, which is not available in Neural Markov Logic Networks where deterministic constraints are implicitly encoded by the potential functions. In fact, only constraints that are almost deterministic, i.c. having very large weights, can occur in Neural Markov Logic Networks but, at least for Gibbs sampling, the effect is the same. Such distributions would naturally be learned in our framework on most datasets. Our solution in this paper is to simply avoid learning distributions with determinism by adding noise during training. In particular, we set a parameter TT E [0, 1] and, at the beginning of each training epoch, we inverted each ground atom of the input possible worlds (True to False and vice versa) with probability kn- Moreover, this added noise prevents the model to perfectly fit training data, acting regularizer (Bishop, 1995).
EXPERIMENTS
4.1 IMPLEMENTATION DETAILS
Logic Networks in Tensorflow. In order to maximally exploit the parallel computations capabilities of GPUs, multiple Markov chains are run in parallel. This is also useful because expected values of gradients (see Eq. 4 and 5) are computed on uncorrelated samples, while sequential samples sampled from a unique chain are known to be highly correlated.
We implemented Neural Markov
In experiments, the different global neural potentials 4 can rely fragments of different sizes so that for small k, the model can focus on learning very local statistics of the data, while, for large k, the model can focus on learning statisties on larger substructures. For example, if represent molecules as a relational structure (see Section 4.3), rings are inherently global statistics which cannot be captured by local properties. This example underlines the importance of the choice of k for a correct modeling of the data distribution. However, since single evaluation of (w) requires a summation over d = (+) kt number of terms, the number of elements of the sum grows exponentially with & (and polynomially, but very fast, with n). So exploiting large l is usually admissible only for small domain sizes n.
4.2 KNOWLEDGE BASE COMPLETION
In Knowledge Base Completion (KBC), we are provided with an incomplete Knowledge Base (KB) and asked to complete the missing part.
The KBC task is inherently in the transductive setting, since all the constants are exploited both during the training and testing phase. Moreover, data are provided in a positive-only fashion: we only know what is true and we cannot distinguish between unknown and false facts. Kuželka & Davis (2019) studied KBC tasks under the missing-comm cosesinn-compleely--tttt--- at-random assumption and showed consistency of learning by maximum maximum-likelihood where both missing and false facts are treated in the same way as false. Hence, here we also provide both unknown and false facts as false facts during the training procedure.
5

---

Under review as a conference paper at ICLR 2020
M
K
M
B
(a) The training KB.
(b) The completed K..
Figure 2: Knowledge Base Completion in the Smokers dataset. Circles represent constants. A grey circle means that the predicate smokes is True. A white circle means that the value of the predicate smokes is unknown. Links represent the relation friendOf (absence an arrow means that the relation is False). The given world is shown on the top (2a), while the completed knowledge base is shown on the bottom (2b). The system learnt the symmetric nature of the friendship relation. It learnt that friend of at least two smokers is also smoker, and that two smokers, who friends of the same person, are also friends.
Smokers. The "Smokers" dataset (Richardson & Domingos, 2006) is a classical example in sta- tistical relational learning literature. Here, two relations are defined on a set of constants repre- senting people: the unary predicate Smokes identifies those people who smoke, while the binary predicate friendOf maps people to their friend. This dataset is often used to show how a sta- tistical relational learning algorithm can model distribution by finding a correlation of smok- ing habits of friends. For example, in MLNs, one typically uses weighted logical rules such as: Vz Vy friendof(z, y) + smokes(z) < smokes(y). We learned a NMLN on the small smok- ers dataset. Since no prior knowledge about the type of rules that are relevant was used by NMLNs, the model itself had to identify which statistics are mostly informative of the provided data by learn- ing the neural potential functions.
Here we use the Smokers dataset to define a Knowledge Base Completion task and to provide some basic intuitions about what kind of rules the model could have learned. In Figure 2, we show the setting before and after completion. In Figure 2b, we highlight only new facts whose marginal prob- ability after training is significantly higher than the others, even though other facts have probabilities higher than the prior.
Nations. The Nations dataset (Kok & Domingos, 2007) provides information about proper-
ties and relations among countries as ground facts, like economi caid (usa, israel) or embassy (israell, poland). There are 14 constants (i.e nations), 56 relations and 2565 true facts. This dataset has been recently exploited for a KBC task by Rocktäschel & Riedel (2017), where some facts were removed from the dataset and the task was to predict them. The authors compared the performances of the state-of-the-art ComplEx neural model (Trouillon et al., 2017) with their proposed differentiable end-to-end neural theorem prover, showing that the combination of the two was able to outperform both of the models. Unary predicates were removed since the ComplEx model cannot deal with them. In this section, show how we can use NMLNs tackle a KBC task on the Nations dataset.
We implemented the fragment neural potentials Φ(7) as 2 hidden-layer neural networks, with sig- moidal hidden activations and linear output layer. The selection of the hyperparameters and the early-stopping epoch have been selected by means of held-out validation as the ones in Rocktäschel & Riedel (2017)). The size of layers has been selected from the interval (75, 100, 150) for the first layer and [30, 50, 100) for the second layer. The embedding size has been selected from the interval [2,3, 3, 5, 10). The noise probability T. has been selected from the interval [0, 0.01, 0.02, 0.03). The number of parallel chains has been selected from the interval [10, 20, 30).
(the splits are same
We followed the evaluation procedure in Rocktäschel & Riedel (2017). In particular, fact and corrupted its first and second argument in all possible ways such that the corrupted fact is not in the original KB. Subsequently, we predicted a ranking of every test fact and its corruptions to calculate MRR and HITS @ m. The ranking is based on marginal probabilities estimated by running Gibbs sampling the Neural Markov Logic Network; while training the network, also
took test

---

Under review as a conference paper at ICLR 2020
Table I: MRR and HITS@m on Nations.
Metric
Model
ComplEx
NTP
NTPA
NMLN
NMLN-
MRR
0.75
0.75
0.74
0.77
0.81
HITS @1
0.62
0.62
0.59
0,64
0.71
HITS @
0.84
0.86
0.89
0.86
0.89
HITS @ 10
0.99
0.99
0.99
0.99
0.99
parallel Gibbs sampling chain on state in which we fix the known part of the KB as true. Here, compare the ComplEx model, the plain Neural Theorem Prover (NTP), the composition of the previous two (NTPA), our plain model (NMLN) and our model when using potentials with embed- dings (NMLN-Emb). In Table we report the results of the KBC task Nations. Both our models outperform competitors on the HITS@ metric, with NMLN-Emb having large gap over all the other models. provers, even if it is the only model which cannot exploit embeddings to perform reasoning and that has to rely only on the relational structure of fragments make predictions. Finally, NMLN-Emi performs equally to or better than all the competitors in all the other metrics.
interesting to note that the plain NMLN still performs better than differentiable
4.3 GRAPH GENERATION
One of the main features differentiating our model from standard MLNs is that we learn the statistics Φ(>) in differentiable manner. The obtained prohability distribution is then often far more fine grained than using predefined or hand-made statistics, that are limited to what the user considers important and do not search for other interesting regularities in the data. This opens the doors to the application of NMLNs to generative tasks in non-euclidean settings, which are receiving an increasing interest recently (You et al., 2018; Li et al., 2018).
In generation tasks, our model is asked to learn the probability distribution of the relational structures induced by a graph. Indeed, any FOL-description can be considered a multi-hyper graph: thus generating in the FOL setting is applicable to generating in any graph domain. In particular, to generate graphs, we can just use the same sampling technique used during training (i.e. Gibbs Sampling) to extract new samples.
In this section, we describe a molecule generation task. We used as training data the ChEMBL molecule database (Gaulton et al., 2016). We restricted the dataset to molecules with 8 heavy atoms. We used the RDKit framework to get a FOL representation of the molecules from their SMILES encoding. In particular, o, S, C1, F, P. and we encoded only two kinds of bonds: SINGLE and DOUBLE, more detailed description of the data format is shown in the appendix.
exploited only molecules having the most frequent atom types, i.c. C, N,
We implemented the fragment neural potentials o(7) as neural networks with sigmoidal hidden activations and linear output layer. The hyperparameters were selected from the following ranges: the number of layers in [1, 2}; the hidden sizes of the layers in [30, 100, 150, 200]; the number of fragment potentials in [1, 2), the size ls of potentials in [2,3, 4, 5, 6). The number of parallel chains was set to 5.
To qualitatively evaluate the results of this generative experiment, we follow Li al. (2018), who designed an LSTM-based architecture for generative molecule modelling and applied it in a similar setting to ours. In Figure 3, we show comparison between sample of training data and a (random) sample of molecules generated by the proposed model. In particular, 20 generated samples are chosen randomly from the last 1000 samples extracted during the training procedure. By choosing them randomly, we avoided to have very correlated samples, which is inherent in the Gibbs sampling procedure. The generated samples resembles training data both in structural patterns and variety fairly well. Furthermore, in Figure 4, we compare the statistics, used in Li et al. (2018) for a similar task, on a sample of 1000 training and generated molecules. These statistics represent both general structural properties applicable to any graph as well as chemical structural properties of molecules
hhttps://dkit..rrg

---

Under review as a conference paper at ICLR 2020
y m83 xx x
1 x
oo yo
(a) Molecules from the training data.
Generated molecules.
Figure 3: Molecules generation. A comparison between a sample of training data and a (random) sample of molecules generated by the proposed model. The generated samples fairly resembles training data both in structural patterns and variety. Better viewed in color.
Number of Bonds
Average Node Degree
Number of Rings
750
250
HBD
HBA
TPSA
200
300
150
200
50
25
generated
training
Figure 4: Molecules generation. Comparing the distributions of some chemical properties of the training data with the ones of generated data. The generated samples are capable of perfectly fitting structural properties and to very well resembling functional properties.
(e.g. the topological polar surface area (TPSA) a topological indicator the capability of molecule permeate membrane function of the number of polar atoms it contains). These statistics were computed using the RDkit framework.
5 CONCLUSIONS
In this paper we have introduced Neural Markov Logic Networks, statistical relational learning model combining representation learning power of neural networks with principled handling of uncertainty in the maximum-entropp framework. The proposed system works remarkably well on small domains. Although not explained in detail in this paper, it is also straightforward to add standard logical features as used in MLNs to NMLNs.
The main future challenge is making NMLNs scale larger domains. At the moment NMLNs do not scale to large knowledge bases, which is not that surprising given that NMLNs can theoretically represent any distribution. A more work should therefore be done in the direction of identifying more tractable subclasses of NMLNs and exploiting insights from lifted inference literature (Braz al., 2005; Gogate & Domingos, 2011 1; den Broeck et al., 2011).
8

---

Under review as a conference paper at ICLR 2020
REFERENCES
Chris M Bishop. Training with noise is equivalent to tikhonov regularization. Neural computation, 7(1):108-116, 1995.
Antoine Bordes, Nicolas Usumier, Alberto Garcia-Duran, Jason Weston, and Oksana Yakhnenko. Translating embeddings for modeling multi-relational data. In Advances in neural information processing systems, pp. 2787-2795, 2013.
Stephen Boyd and Lieven Vandenberghe. Convex optimization. Cambridge university press, 2004.
Rodrigo De Salvo Braz, Eyal Amir, and Dan Roth. Lifted first-order probabilistic inference. In Proceedings of the 19th international joint conference on Artificial intelligence, pp. 1319-1325, 2005.
David Buchman and David Poole. Representing aggregators in relational probabilistic models. In Twenty-Nith AAAI Conference on Artificial Intelligence, 2015.
Luc De Raedt, Angelika Kimmig, and Hannu Toivonen. Problog: A probabilistic prolog and its application in link discovery. IJCAI, volume 7, pp. 0667-447. Hyderabad, 2007.
Michaël Defferrard, Xavier Bresson, and Pierre Vandergheynst. Convolutional neural networks on graphs with fast localized spectral filtering. Advances neural information processing systems, pp. 3844-3852, 2016.
Guy Van den Broeck, Nima Taghipour, Wannes Meert, Jesse Davis, and Luc De Raedt. Lifted probabilistic inference by first-order knowledge compilation. In IJCAI 2011, Proceedings of the 22nd International Joint Conference on Artificial Intelligence, pp. 2178-2185, 2011.
Michelangelo Diligenti, Marco Gori, and Claudio Sacea. Semantic-based regularization for learning and inference. Artificial Intelligence, 244:143-165, 2017.
Anna Gaulton, Anne Hersey, Michat Nowotka, A Patrícia Bento, Jon Chambers, David Mendez, Prudence Mutowo, Francis Atkinson, Louisa J Bellis, Elena Cibrián-Uhalte, et al. The chembl database in 2017. Nucleic acids research, 45(D1))D945-D954 2016.
Vibhav Gogate and Pedro M. Domingos. Probabilistic theorem proving. In UAI 2011, Proceedings of the Twenty-Sevene Conference on Uncertainty in Artificial Intelligence, pp. 256-265, 2011.
Geoffrey E Hinton. Training products of experts by minimizing contrastive divergence. Neural
computation, 14(8):1771-1800 2002.
Stanley Kok and Pedro Domingos. Statistical predicate invention. In Proceedings of the 24th inter- national conference on Machine learning. pp. 433-440. ACM, 2007.
Ondřej Kuželka and Jesse Davis. Markov logic networks for knowledge base completion: A theo- retical analysis under the MCAR assumption. In Proceedings of the Thirty-Fifth Conference on Uncertainty in Artificial Intelligence, UAI 2019, 2019.
Ondřej Kuželka, Yuyi Wang. Jesse Davis, and Steven Schockaert. Relational marginal problems: Theory and estimation. In Shirty-Second AAAI Conference on Artificial Intelligence, 2018.
Yujia Li, Oriol Vinyals, Chris Dyer, Razvan Pascanu, and Peter Battaglia. Learning deep generative models of graphs. arXiv preprint arXiv:/803. met..03334........ 2018.
Marco Lippi and Paolo Frasconi. Prediction of protein 3-residue contacts by markov logic networks 2009.
with grounding-specinn weights. Bioinformatics, 25(18):2326
Robin Manhaeve, Sebastijan Dumancic, Angelika Kimmig, Thomas Demeester, and Luc De Raedt. Deepproblog: Neural probabilistic logic programming. In Advances in Neural Information Pro- cessing Systems, pp. 3749-3759, 2018.
Giuseppe Marra, Francesco Giannini, Michelangelo Diligenti, and Marco Gori. Lyrics: a general interface layer to integrate ai and deep learning. arXiv preprint arXiv: arXiv: 1903.0754, 2019.

---

Under review as a conference paper at ICLR 2020
Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. Distributed represen- tations words and phrases and their compositionality. In Advances neural information pro- cessing systems, pp- 3111-3119, 2013.
Andriy Mnih and Koray Kavukeuoglu. Learning word embeddings efficiently with noise-contrastive estimation. In Advances in neural information processing systems, pp. 2265-2273, 2013.
Hoifung Poon and Pedro Domingos. Sound and efficient inference with probabilistic and determin- istic dependencies. AAAI, volume 6, pp. 458-463, 2006.
Matthew Richardson and Pedro Domingos. Markov logic networks. Machine learning, 62(1-2): 107-136, 2006.
Christian Robert and George Casella. Monte Carlo statistical methods. Springer Science & Business Media, 2013.
Tim Rocktäschel and Sebastian Riedel. End-to-end differentiable proving. In Advances in Neural Information Processing Systems, pp. 3788-3800, 2017.
Franco Scarselli, Marco Gori, Ah Chung Tsoi, Markus Hagenbuchner, and Gabriele Monfardini. The graph neural network model. JEEE Transactions Neural Networks, 20(1):61-80, 2009.
Luciano Serafini and Artur d' Avila Garcez. Logic tensor networks: Deep learning and logical reasoning from data and knowledge. arXiv preprint (et::0606004422, 2016.
Gustav Sourek, Vojtech Aschenbrenner, Filip Zelezný, Steven Schockaert, and Ondrej Kuzelka. Lifted relational neural networks: Efficient learning latent relational structures. Artif. Intell. Res., 62:69-100, 2018
Théo Trouillon, Christopher R Dance, Éric Gaussier, Johannes Welbl, Sebastian Riedel, and Guil- laume Bouchard. Knowledge graph completion via complex tensor factorization. The Journal of Machine Learning Research, (8(1):473544772 2017.
2017.
Martin Wainwright, Michael I Jordan, al. Graphical models, exponential families, and variational inference. Foundations and Trends(R in Machine Learning. 1(1-2):1-305, 2008.
Keyulu Xu, Weihua Hu, Jure Leskovec, and Stefanie Jegelka. How powerful are graph neural networks? arXiv preprint arXiv:1810.008: 1810.0086, 2018
Jiaxuan You, Rex Ying, Xiang Ren, William L Hamilton, and Jure Leskovec. Graphrnn: Generating realistic graphs with deep auto-regressive models. arXiv preprint #rXhv:/802.08773 2018.
10

---

Under review as a conference paper at ICLR 2020
A TRANSLATING MARKOV LOGIC NETWORKS TO NEURAL MARKOV LOGIC NETWORKS
Markov logic networks are popular statistical relational framework. It turns out that every Markov logic network can be represented as Neural Markov Logic Network with a single carefully selected potential function. We give details of the translation between these two frameworks below. Essen- tially what we need to show is that Model B from (Kuželka et al., 2018) can be translated to Model from that same work, which is close enough Neural Markov Logic Networks and so we can then easily encode the result to a Neural Markov Logic Network.
Kuželka et al. (2018) studies two maximum-entropy models, Model A, which is close to the model that we study in this paper, and Model B, which is the same as Markov logic networks. Syntactically, both models are encoded as sets of weighted first order logic formulas, c.g. & = {(a), wi),..., (am, Wm)). In particular, given positive integer k Model A defines the following distribution:
PA(W) Z exp
Σ w w- w==x(a,
(A.
where Z is the normalization constant and #(a w) is the fraction of size-k subsets S of constants in the possible world for which w(S) } a (i.e. the formula a is classically true in the fragment of w induced by S). Let us first define
fui w 0
F ka
2 (7)
It is then easy to see that the distribution PA (w) can also easily be encoded as a Neural Markov Logic Network by selecting the potential function (67) (a, (7,w(?) and by carefully selecting the weights 3; in the Neural Markov Logic Network.
Next we show that all distributions in Model B can be translated to distributions in Model A. First we will assume that the formulas a; do not contain any constants.
Model B is given by
PB(W) Z exp
Σ (8
U. (nn,((())))) w)
where n2B, w) is the number? of true injective groundings of the formula 3 in the possible world w. Hence, Model B is exactly the same as Markov logic networks up to the requirement on injectivity of the groundings. However, shown in (Buchman & Poole, 2015), any Markov logic network can be translated into such modified Markov logic network with the injectivity requirement on the groundings.
Let & be an integer greater or equal to the number of variables in any formula in QB. Now, let I' be the set of all size-k fragments. For every formula in 4n, we introduce partition P on induced by the equivalence relation ~g defined by: y ~ig 7 iff n(3, 7) = n(3,7'). Since B assumed to not contain any constants, we can capture each of these equivalence classes by (possibly quite big) first-order logic sentence without constants 3c. Let C. be the equivalence class that contains fragments r such that m(3,7)
Let m(B, w) = Ec.ep Exera d 1 } Bc)- By construction, holds m(3 = Exer n(.,)). Every true injective grounding of the formula 3, having variables, is contained in (?=1) different size-k fragments of w, each of which gives rise to k! anonymized fragments in the multi-set (ww(w). So m(B, w)) is over-counting the number of true groundings n(3,, w) by constant factor. It follows that, by carefully selecting the weights of the formulas Se: we can encode the distribution (=(w) also in Model A. Although this particular transformation that have just sketched is very efficient, does show that Neural
IIn (Kuželka et al., 2018). Model B is defined using fractions of true grounding substitutions instead of numbers of true grounding substitutions. However, these two definitions are equivalent up to normalizations and both work for our purposes but the latter one is a bit more convenient here. Hence we choose the latter one here.
11

---

Under review as a conference paper at ICLR 2020
Markov Logic Networks with potential functions of width & can express all distributions that can be expressed by Markov logic networks containing formulas with at most & variables.
First-order logic formulas defining Markov logic networks may also contain constants. In Neural Markov Logic Networks we may represent constants using vector-space embeddings as described in the main text. One can then easily extend the argument sketched above the case covering Markov logic networks with constants.
B ALGORITHMS
In the following, we will show learning algorithm for NMLNs that relies on approximated Gibbs sampling.
The general learning algorithm for NMLN is described in Algorithm 1.
Algorithm NMLN general learning algorithm
Input: W: the given training world
1: procedure LEARN(@)
2: 7): learning rate
3: W: M randomly initialized Markov chain states
4: while stopping criterion do
5: while chains convergence criterion do
6:
W +- SAMPLE-STERL M, 3,, w.)
7:
(og() + 3/ Ohi
(W:o)) Oong
M Σe
8:
(og((p)) 03,
+
p (1; wi)
M Es Pi(w; Ws)
9:
Wi
Wi 17
Bun
B, - 3, 17
9 (Ps)
A possible sampling procedure, implementing the generic SAMPLE-STEP and exploiting Gibbs Sampling, described in Algorithm 2
Algorithm 2 Sampling Procedure
Input: W: the current states of the chains
Input: M: the number of chains
Input;,3; 3g. wy: current parameters
procedure
CAMPLE-STEPE M, 3i, wi)
2: 1--1 1 # sample index
3: +- # ground atom index
4: number of ground atoms in
5: while s M do
6:
W, # s-th chain
7: while < n. do
8:
p Pa.=
9:
r E [0, 1] from a uniform distribution
10:
if r < then
11:
w 3- wi =
12:
else
13:
W 3- wi=
14:
-+it1
3
16:
8
17:
return
12

---

Under review as a conference paper at ICLR 2020
NH
HD
Figure 5: An example of molecule
C GENERATING MOLECULES
Even though molecules can be described with high level of precision, using both spatial features (i.e. atoms distances, bond length etc.) and chemical features (i.e. atom charge, atom mass, hy- bridization), in this work, we focused mainly structural symbolic descriptions of molecules.
In particular, we described a molecule using two sets of FOL. predicates:
Atom-type unary predicates: these are C, O, S, C1, P.
Bope-type binary predicate: these SINGLE and DOUBLE.
An example of a molecule FOL description can be:
(0)), (1), (2), C (3). N (4), (5), C C17), 0(8), 0
SINGLE (0, 1), SINGLE (1, 0), SINGLE (1,2), SINGLE (2, 1), SINGLE (2, 3)
SINGLE 2),
SINGLE (3, 4), SINGLE (4, SINGLE (4, 5),
SINGLE 4)
SINGLE 6),
SINGLE (6,
SINGLE (5,7),
SINGLE (7,
DCUBLE (7,8)
DOUBLE 7),
SINGLE 9),
SINGLE (9,7),
SINGLE (6,1),
SINGLE (1,
13


Reviews:
[{"review_id": "rkxIiJNo5r", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "I think their strong point is at the same time their weak point. \", they do not rely on explicitly specified first-order logic rules.\" My question would be, why not use that information if available as prior knowledge? Perhaps I'd like to see a stronger motivation for the use of having to learn this part. I can see how its might be useful but would be very happy to see more motivation for this. I think I've seen Kotler learn the rules of sudoku but sudoku is such a specified problem that Im not convinced yet this is usueful to learn. Thats a different paper but I missed the motivation for that here too.\n\nI like the honesty of the authors for saying it doesn't scale to larger problems. Regardless, I think this paper is good to push the field in that direction. I particularly like the graph generation task. Graph generation, afaik, is not easy. \n\nWhat I invite the authors to do is to not be restricted by theoretically/principled motivated ways. I believe its better to find things that work well first and then to find a theory (the other way round). This is not enough to reject the paper for me because I do believe this is pushing the field forward in a good direction. If possible I'd suggest to relax the theory and then compare the two models if possible.\n\nIn the contribution it says \"(i) we introduce a new statistical relational model, which\novercomes actual limitations of both classical and recent related models such as \" I would have really liked it to have been spelt out which limitations, very specifically and concisely the paper overcomes in that section.\n", "extra_scores": {"experience_assessment": "I do not know much about this area.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I made a quick assessment of this paper."}, "extra_text": {}}, {"review_id": "BylW-TCTtS", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 1, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The following paper provides an extension to Markov Logic Networks(MLNs), by removing their dependency on pre-defined first-order logic rules.  This is handled via neural networks which are able to capture the statistical relations, so-called Neural Markov Logic Networks(NMLNs). As this is an implicit representation from the neural network, the rules act as potential functions on the MLN structure. As general MLN techniques are reliant on domain experts or exhaustive structure learning approaches, NMLNs are able to model more domains as provided in the work with knowledge-base completion tasks and generative modelling of molecules. \n\nThe primary contribution in this body of work is based on the observation that relational structure repeats regularities in the data, and where deriving the statistics of these regularities is what allows for improved accuracy in a model.  The proposed NMLN  is architecturally identical to MLNs with the difference being the addition of the potential function.\n\nAs defined by the paper, fragments are connected subsets derived from relational data. The authors derived sets of fragments with constants defined by values in the data and anonymized fragment sets with integer assignments. With potential functions sampled from the anonymized and the true value fragments. The objective is a search for a maximum-entropy distribution to model the data derived fragments. The neural network aspect comes in the form of the minimum-maximum entropy modelling with weights for given fragments being learned by minimising the entropy of the fragment potential function. Where the model also maximizes the log-likelihood related to the anonymized fragments. The intuition in this work is that by selecting the maximum entropy distribution while also minimizing it by selecting the most informative statistical information for it, we will derive an accurate probability distribution given the possible worlds.\nOverall, the paper performs a well enough job explaining the technical aspects. It does a thorough job explaining the algorithmic detail in the main body, and the appendix provides clear and implementable pseudocode equations. It is not exactly clear to me why the anonymization of fragments is necessary, but the authors suggest this places a greater focus on the graph structure and minimizes the model acting differently with different constants. The min-max entropy modelling also appears to be a novel approach in terms of statistical relational modelling. The results also demonstrate the success of NMLNs modelling on relational data and KB completion.\n\nRegarding the technical aspects, a few concerns are the claims that the domains for experimentation seem rather trivial since the smoking and nations dataset are common relational datasets, and the strength of this model is the ability to learn on other domains. This is possibly addressed with the molecule experiments, but more datasets would have helped in confirming the breadth of domains as claimed by the paper. It is also difficult to measure the success of their model with generative modelling as no baseline was present for the molecule experiments. I still view this paper as a positive contribution to MLN research, with a technique that is successful among the few experiments tested.\n\nAt the same time, however, given the emerging contributions in the area, such as relational neural networks (Poole et al.), although the technical development is sensible, I don’t find the contribution that technically novel, and seems somewhat straightforward. Given the preliminary investigations, this is a reject from my side. ", "extra_scores": {"experience_assessment": "I have published one or two papers in this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_experiments": "I did not assess the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory."}, "extra_text": {}}, {"review_id": "BkllX-qycH", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper presents Neural Markov Logic Networks  (NMLN), which is a generalization of Markov Logic Networks (MLN). Unlike MLN which relies on pre-specified first-order logic (FOL) rules, NMLN learns potential functions parameterized by neural networks on fragments of the graph. The potential function can possibly take into account the constants present using embeddings to better solve transductive problems（otherwise the potential can only use relational structure). To make computation tractable, the size of local potential functions is constrained. Training of this MRF is performed by solving a min-max entropy problem: conditioned on an informative potential, the uncertainties shall be decreased. Experiments on a knowledge base completion task and a graph generation task show superior performance compared to baselines like neural theorem provers.\n\nPros:\n1. no need to specify FOL rules and can potentially discover subtle relations not evident to us.\n2. can be used for generation since the learned rules might be more fine-grained than what we can specify.\n3. it's interesting that on nations knowledge base completion problem even without constant embeddings it works fine, which shows the power of just using relational structure.\n\nCons:\n1. the computation complexity of the global potential function grows combinatorally with the clique size k and polynomially with graph size n, which is unrealistic to any larger graphs than the small molecules, if any higher order statistics matters (e.g. in molecules there are rings).\n\nQuestions:\n1. for training can we use MLE?\n\nOverall this is an interesting work. I think it is a natural generalization of Markov Logic Networks and works on two small problems. I am inclined to recommend this paper to the community.\n\n\n-----updates after reading rebuttal-----\nThanks for the clarification. I don't have further questions.", "extra_scores": {"experience_assessment": "I have read many papers in this area.", "review_assessment:_checking_correctness_of_experiments": "I carefully checked the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper thoroughly.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I carefully checked the derivations and theory."}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper on extending MLNs using NNs is borderline acceptable: one reviewer is strongly opposed, although I confess I don't really understand their response to the rebuttal or see what the issue with novelty is (a position shared by the other reviewers). I'm not sure how to weigh this review, but there is not a lot of signal in favour of rejection aside from the rating.

The remaining two reviews are in favour of acceptance, with their enthusiasm only bounded by the lack of scalability of the method, something they appreciate the authors are upfront about. My view is this paper brings something new to the table which will interest the community, but doesn't oversell the result.

Given the distribution of papers in my area, this one is just a little too borderline to accept, but this is primarily a reflection of the number of high-quality papers reviewed and the limited space of the conference. I have no doubt this paper will be successful at another conference, and it's a bit of a shame we were not in a position to accept it to this one.
Decision: Reject