Paper:
The capability of making interpretable and self-explanatory decisions is essential for developing responsible machine learning systems. In this work, we study the learning to explain problem in the scope of inductive logic programming (ILP). We propose Neural Logic Inductive Learning (NLIL), an efficient differentiable ILP framework that learns first-order logic rules that can explain the patterns in the data. In experiments, compared with the state-of-the-art methods, we find NLIL can search for rules that are x10 times longer while remaining x3 times faster. We also show that NLIL can scale to large image datasets, i.e. Visual Genome, with 1M entities.
: A scene-graph can describe the relations of objects in an image. The NLIL can utilize this graph and explain the presence of objects Car and Person by learning the first-order logic rules that characterize the common sub-patterns in the graph. The explanation is globally consistent and can be interpreted as commonsense knowledge.
The recent years have witnessed the growing success of deep learning models in a wide range of applications. However, these models are also criticized for the lack of interpretability in its behavior and decision making process (Lipton, 2016;Mittelstadt et al., 2019), and for being data-hungry. The ability to explain its decision is essential for developing a responsible and robust decision system (Guidotti et al., 2019). On the other hand, logic programming methods, in the form of first-order logic (FOL), are capable of discovering and representing knowledge in explicit symbolic structure that can be understood and examined by human (Evans & Grefenstette, 2018).
In this paper, we investigate the learning to explain problem in the scope of inductive logic programming (ILP) which seeks to learn first-order logic rules that explain the data. Traditional ILP methods (Galárraga et al., 2015) rely on hard matching and discrete logic for rule search which is not tolerant for ambiguous and noisy data (Evans & Grefenstette, 2018). A number of works are proposed for developing differentiable ILP models that combine the strength of neural and logicbased computation (Evans & Grefenstette, 2018;Campero et al., 2018;Rocktäschel & Riedel, 2017;Payani & Fekri, 2019;Dong et al., 2019). Methods such as ∂ILP (Evans & Grefenstette, 2018) are referred to as forward-chaining methods. It constructs rules using a set of pre-defined templates and evaluates them by applying the rule on background data multiple times to deduce new facts that lie in the held-out set (related works available at Appendix A). However, general ILP problem involves several steps that are NP-hard: (i) the rule search space grows exponentially in the length of the rule; (ii) assigning the logic variables to be shared by predicates grows exponentially in the number of arguments, which we refer as variable binding problem; (iii) the number of rule instantiations needed for formula evaluation grows exponentially in the size of data. To alleviate these complexities, most works have limited the search length to within 3 and resort to template-based variable assignments, limiting the expressiveness of the learned rules (detailed discussion available at Appendix B). Still, most of the works are limited in small scale problems with less than 10 relations and 1K entities.
On the other hand, multi-hop reasoning methods (Guu et al., 2015;Lao & Cohen, 2010;Lin et al., 2015;Gardner & Mitchell, 2015;Das et al., 2016) are proposed for the knowledge base (KB) completion task. Methods such as NeuralLP  can answer the KB queries by searching for a relational path that leads from the subject to the object. These methods can be interpreted in the ILP domain where the learned relational path is equivalent to a chain-like first-order rule. Compared to the template-based counterparts, methods such as NeuralLP is highly efficient in variable binding and rule evaluation. However, they are limited in two aspects: (i) the chain-like rules represent a subset of the Horn clauses, and are limited in expressing complex rules such as those shown in Figure 1; (ii) the relational path is generated while conditioning on the specific query, meaning that the learned rule is only valid for the current query. This makes it difficult to learn rules that are globally consistent in the KB, which is an important aspect of a good explanation.
In this work, we propose Neural Logic Inductive Learning (NLIL), a differentiable ILP method that extends the multi-hop reasoning framework for general ILP problem. NLIL is highly efficient and expressive. We propose a divide-and-conquer strategy and decompose the search space into 3 subspaces in a hierarchy, where each of them can be searched efficiently using attentions. This enables us to search for x10 times longer rules while remaining x3 times faster than the state-of-theart methods. We maintain the global consistency of rules by splitting the training into rule generation and rule evaluation phase, where the former is only conditioned on the predicate type that is shared globally.
And more importantly, we show that a scalable ILP method is widely applicable for model explanations in supervised learning scenario. We apply NLIL on Visual Genome (Krishna et al., 2016) dataset for learning explanations for 150 object classes over 1M entities. We demonstrate that the learned rules, while maintaining the interpretability, have comparable predictive power as densely supervised models, and generalize well with less than 1% of the data.

Review:
This paper presents a model for effectively hierarchically ‘searching’ through the space of (continuously relaxed) FOL formulas that explain the underlying dataset. The model presented employs a three-level architecture to produce logic entailment formulas, skolemized with a set of skolem functions, i.e. ‘operators’. The three-level search, which is implemented via a stack of transformers, first searches through a space of variable and predicate embeddings to select operators, after which it searches through the space of predicates to form primitive statements of predicates and operators, and finally it generates a number of formulas from the previously ‘selected’ primitive statements. The model is applied on a toy task to showcase its speed, a knowledge base completion task, and modeling the visual genome dataset, where the model shows the ability to induce meaningful rules that operate on visual inputs. The presented benefit of the model is scalability, ability to induce explainable rules, and the ability to induce full FOL rules.
The paper is well motivated from the explainability perspective and based on the evaluation does what it claims. The model is fairly elaborate, yet manages to be faster than the competing models. In general, I think the model itself is a welcome addition to the area of neuro-symbolic models, especially logic-inducing models, and that the evaluation done is appropriate (with a few caveats). However, my major critique of the paper is in its clarity.
In the current state, the paper is quite difficult to read, partially due to its density, partially due to its confusing presentation, notational issues and missing details. It would be difficult to reimplement the model just by reading the paper, which brings me to ask: will you be releasing the code?
I would be willing to accept the paper if the authors improve the notation and presentation significantly. I’m enumerating issues that I found most confusing:
Result presentation:
- The figure captions are uninformative. In figure 1, one needs to understand what the graph is before reading the text at the end of the paper which explains what that is. It is not clear from the figure itself. Figure 2 presents the model, but it does not follow the notation from the main body of the paper.
- Table 1 is missing SOTA models. TransE is definitely not one of the better models out there: check [2, 3, 4] for SOTA results which are significantly higher than the ones presented. I would not at all say that that invalidates the contribution of the paper, but readers should have a clear idea of how your model ranks against the top performing ones.
- Please provide solving times for TransE, as it has to be by far the fastest method, given that it is super-simple.
- Are provided times training or inference times (in each of the table/figure) because one gets mixed statements from the text?
- In which units is time in Table 1?
- Can you include partially correct or incorrect learned rules in Table 4? It would be great to get some understanding of what can go wrong, and if it does, what might be the cause.
Model presentation and notation:
- You mention negative sampling earlier in the text but then don’t mention how you do it
- Notation below (6) is utterly confusing and lacking: what is s_{l,i}^(t), is there a softmax somewhere, what are numbers below ‘score’
- What is the meaning of e_+ and e_- given that you omit details of negative sampling?
- The notation does not differentiate between previous modules well so there’s V^(t-1) across modules, and it is not clear which one is used at the end --- last choice over V^(0) - V^(T) is over the LAST output, not the output from previous steps?
- The notation in the text does not follow the notation in the figure (V_ps, V_\phi)
- Notation gets quite messy at some points, e.g. R^c^(0), is e_X and e_Y in H or not? Is H_b there too?
- The differentiation of embedding symbols is not done well. H_b is an embedding for binary predicates, or a set of predicates? Does that mean that there is only a single embedding for a binary predicate and a single embedding for its operator (thought I thought that operators have an embedding, each)?
- The explanation of what a transformer does is not particularly useful, the paper would benefit more from an intuitive explanation, such as that the transformer learns the ‘compatibility’ of predicates (query) and input variables (values), etc.
- The ‘Formula generation’ subsection lacks the feel of fitting where it is right now, given that the notation in it is useful only in ‘Formula search’ paragraph. The other thing is that that subsection is wholly unclear: where do p and q come from, do they represent probabilities of predicates P and Q? How are they calculated? Does your construction imply that a search over the (p, q, a) space is sufficient to cover all possibilities of and/not combinations? In which case alpha is a vector of multiple values different for each f_s  in (5) or no? It is unclear 
- What is the relationship between alpha_0 and alpha_1, sum(alpha_0, alpha_1) = 1? Are alpha_0, and alpha_1 scalars, and how are they propagated in eq 5? Because if they’re just scalars, the continuous relaxation of formulas represented in (5) cannot cover all combinations of p, q, not and and. What is the shape of the alpha vector?
p and q are probabilities of what, predicates P and Q? How are they produced?
- are \phi functions pretrained?
Related work:
- I do not condone putting it into the appendix, but I’m not taking it as a make-or-break issue.
- It is notably missing an overview of symbolic ILP, and a short spiel on link prediction models (given that you compare your model to TransE, as a representative of these models)
The paper is riddled with typos and consistent grammatical errors. I would suggest re-checking it for errors. Examples include:
- singular/plural noun-verb mismatches (rules..and does not change -> do, Methods ...achieves -> achieve)
- Q_r^(t) - choice of left operands -> right
- An one-hot -> a one-hot
Minor issues:
- The claim that the NeuralLP is the only model able to scale to large datasets is a bit too strong given [1]
- You say “We could adopt” but you “do adopt”
Clarification questions:
- Are 10 times longer rules of any use? Can you provide an example of such rules, a correct one and an incorrect one?
- How many parameters does your model use? What are the hyperparameters used in training?
- How big is T in your experiments, and why?
- Why are queries only binary predicates?
- How discrete do these rules get? You sample them during testing and evaluation, but are embeddings which encode them approximately one-hot, so that the (Gumbel-softmax) sampling produces the same rules all over again or are these rules fairly continuous and sampling is just a way to visualise them, but they are internally much more continuous?
- Just to confirm, \phi are learned, right? Given that they are parameterised with a predicate matrix, and that matrix is trainable?
- Do you have a softmax on the output? It seems  f^*(T+1) should be a softmaxed value?
- The rule generation generates a substantial number of rules, right? What might be the number of these rules? Does the evaluation propagate gradients through all of them or to a top-k of them?
- Why is the formula in Eq.4 written the way it is. I assume it can be written differently, for example “Inside(\phi_Identity(X), \phi_Car())) and Inside(\phi_Clothing(), \phi_Identity(X))”. I do not understand why Clothing was treated as an operator and Car as a predicate, while treating Inside both as an operator and a predicate. Sure, nothing in the model forces it to consistently represent a formula in the same way always, but an example such as this one would need a good explanation why you chose it or at least mention that this is but one way to present it.
- Why is Identity(X) used? Is it because you did not want to mix in variable embeddings during the primitive statement search?
[1] Towards Neural Theorem Proving at Scale
[2] Canonical Tensor Decomposition for Knowledge Base Completion
[3] RotatE: Knowledge Graph Embedding by Relational Rotation in Complex Space
[4] TuckER: Tensor Factorization for Knowledge Graph Completion