Learning Rules with Stratiﬁed Negation in Differentiable ILP
Abstract
Differentiable methods to learn ﬁrst order rules (logic programs) have the potential to integrate the interpretability, transferability and low data requirements of inductive logic programming with the noise tolerance of non-symbolic learning.
Negation is an essential component of reasoning, but incorporating it into logic programming frameworks poses several problems (hence its central place in the logic programming and nonmonotonic reasoning communities).
Current implementations of differentiable rule learners do not learn rules with negations.
Here, we introduce stratiﬁed negation into a differentiable inductive logic programming framework, and we demonstrate that the resulting system can learn recursive programs with inventive predicates in which negation plays a central role.
We include examples from multiple domains, e.g., arithmetic, graph, sets and lists.
Introduction
Learning logic programs using gradient descent-based methods has the potential to integrate the interpretability, transferability and low data requirements of inductive logic programming (ILP) with the noise tolerance of non-symbolic learning.
In recent years, several differentiable rule learning systems have been introduced (discussed below).
Negation is a critical component of reasoning (expressing some concepts, e g., prime and unconnected , require it) and it plays a central role in formal logic, knowledge representation, automated theorem proving and other ﬁelds.
But existing implementations either explicitly exclude negation, do not mention it, or state their systems can be extended to include negations.
Incorporating negation into differential ILP poses some challenges.
The use of unrestricted negation may result in inconsistent reasoning.
Care must be taken to ensure that negation is added using some reasoned methodology (such as stratiﬁcation) and that the resulting system allows interesting programs to be learned via gradient descent.
In this work, using the differentiable inductive logic programming system ∂ ILP [5] as the underlying framework, we add stratiﬁed, safe negation (i.e., as used in Datalog [3]), and we demonstrate that our system can learn recursive programs that fundamentally rely on negation.
Our main contributions are the following:
We modify ∂ ILP to incorporate stratiﬁed negation [1].
This requires extending both the syntax and the semantics for clauses and modifying the forward chaining deduction process to ensure that consequences are correctly computed wrt stratiﬁed negation.
We present ILP problems of varying complexity requiring negation from several domains (arithmetic, graphs, sets and lists).
We use these to demonstrate that our system is expressive and that our modiﬁcations work well with gradient descent-based learning of programs.
In order to reduce ambiguity in ﬁnal programs without sacriﬁcing the space of learnable programs, we introduce ‘entropy of rule weights’ as a regularizer.
Related Works: The ∂ ILP [5] system which we extend uses forward-chaining inference and allows predicate invention and recursion in learned rules.
To reduce complexity, ∂ ILP restricts the search space of rules using rule templates.
Differentiable Neural Logic-ILP [11] also uses forward-chaining but does not require restrictive rule templates, and it allows Boolean functions in rule bodies.
The examples presented, however, do not involve negation, and it is unclear (to us) how the system, particularly since it utilizes forward-chaining with a maximum number of steps, performs on learning relations depending on negation.
Neural Logic Inductive Learning [16] also allows unrestricted rule bodies, but the examples presented again do not contain negation.
In Neural logic machines [4], rules are encoded as weights of a network, and so extracting symbolic rules would be nontrivial.
Neural Theorem Provers (NTPs) [12, 13] use differentiable backward-chaining to reason over distributed representations of constants and predicates as well as generalized rule structures.
Negation (which requires care in classical backward chaining) is not addressed.
Campero et al.
[2] describe a forward-chaining framework similar to NTPs (again using vector representations of predicates and constants).
The authors compare it to ∂ ILP and indicate that generates correct results more frequently.
Negation is not addressed, however.
Regarding systems based on semantics for logic programs with negation, Nickles [10] presents differentiable Satisﬁability and Answer Set Programming .
This system learns sets of models rather than rules, however, and so it has a different focus.
Methods
Inductive logic programming ( ILP ) [8, 9] is a machine learning framework in which the learned hypothesis is a logic program.
A background knowledge base B and sets of positive and negative examples ( P , N ) are used as inputs, and the objective is to generate a program Π such that Π ∪ B | = a for all a ∈ P and Π ∪ B ̸| = a for all a ∈ N .
Here, Π | = a means that a is a consequence of Π .
The ILP framework is agnostic to the speciﬁc logic programming syntax and semantics used.
In ∂ ILP , the hypotheses are deﬁnite logic programs , i.e., collections of ﬁrst order rules (clauses) r of the form a : − b 1 , b 2 , . . .
, b n , where a and each b i are atomic formulas ( atoms ) and head ( r ) = a and body ( r ) = { b 1 , . . .
, b n } .
An atom, e.g., p ( d, X ) , is a predicate ( p ) followed by constant ( d ) and variable ( X ) terms .
In ∂ ILP , B , P , and N are sets of ground atoms (i.e, no variables appear), while no constants appear in rules.
This is essentially positive Datalog [3].
If Π is deﬁnite, the ground atoms a such that Π ∪ B | = a can be computed via the immediate consequence operator T Π which maps interpretations to interpretations.
An interpretation I is a set of ground atoms; atom a is true in I iff a ∈ I , and it false is otherwise.
Below, G (Π) is the set of ground rules made from all of the the rules of Π .
T Π ( I ) = { head ( r ) | rule r ∈ G (Π) and each member of body ( r ) is true according to I}
Intuitively, we start with I = ∅ and repeatedly apply T Π to perform forward-chaining inference.
If Π is deﬁnite, a monotonically nondecreasing sequence of interpretations is created which eventually reaches a ﬁxpoint.
The ﬁxpoint is precisely the set of all ground atomic consequences of Π [7].
∂ ILP : ∂ ILP : ∂ ILP : One technique for ﬁnding solutions in ILP is a top-down generate-and-test approach in which all possible programs (from a given template) are generated and evaluated on the training data.
This problem can be posed as an instance of SAT by assigning Boolean ﬂags to the candidate clauses and stipulating that the resulting programs must be consistent with the training data.
∂ ILP ([5]) is a differentiable relaxation of this ‘ILP as SAT’ approach.
Programs are generated with trainable continuous weights on collections of clauses.
Forward chaining, which is performed to a preset limit, utilizes real-valued evaluations of ground atoms and differentiable operators: ∧ is replaced by multiplication, and ∨ is replaced by maximum.
The loss value is given by the cross entropy between values computed for ground instances of the target predicate and the training examples.
Gradient descent is used to train weights, and the program with the highest weight is taken as output.
Thus, solving the ILP problem becomes an optimization problem of the form min W L ( Q, C, W ) where
Q is the ILP problem, C is the set of clauses generated using a program template, W is the set of weights, and L is the loss function measuring the accuracy of a program (i.e., its predictions).
2.1 Adding Negation
∂ ILP restricts hypotheses to be sets of deﬁnite clauses, and while the semantics is straightforward, the restriction is signiﬁcant.
Normal logic programs allow negated atoms ¬ b in rule bodies.
One difﬁculty is that the forward chaining process used for deﬁnite programs cannot be directly applied.
In the following example, I is an interpretation, we will use I ( a ∧ b ) =
I ( a ) ∗ I ( b ) and I ( ¬ a ) =
1 − I ( a ) .
For a clause c , the truth value of the clause head is the truth value of the body.
That is, I ( head ( c ))
=
I ( body ( c )) .
T Π is deﬁned in essentially the same way as in the classical case:
In T Π ( I ) , the truth value of each atom a is the maximum truth value (according to I ) of the clauses having a as head.
Note that this is equivalent to combining the rules for a given atom into a single rule, using ∨ to join the original rule bodies together.
Here I ( b 1 ∨ . . .
∨ b n ) =
max ( I ( b 1 ) , . .
, I ( b n )) .
Let Π be the following program.
We assume true always has the value 1.
[Equation]
In the sequence below, I 2 is a ﬁxpoint.
Neither c nor d should be derivable, but they are.
[Equation]
This result is obtained using simple stepwise forward chaining.
If we amalgamate the interpretations using either I t +1 = max ( I t , T Π ( I t )) or probabilistic sum I t +1 =
I t + T Π
( I t ) − I t ∗ T Π ( I t ))
as [5] suggest, then b would also be derivable, which is arguably a worse result.
In the absence of stratiﬁcation, because iterating T Π from ∅ is no longer guaranteed to be monotonic, amalgamating interpretations as ∂ILP does will sometimes lead to irrational inferences.
Stratiﬁed Negation One widely known semantics for normal logic programs is based on stratiﬁed negation [1].
It is only deﬁned for stratiﬁed programs, i.e., programs Π which can be partitioned into subsets Π 1 , . . .
, Π k such that for any Π
i and any rule r ∈ Π i , if b ∈ body ( r ) , the rules deﬁning b are contained in ∪ j ≤ i Π j , and if ¬ b ∈ body ( r ) , the rules deﬁning b are contained in ∪ j<i Π j .
The idea is to use T Π to compute the ﬁxpoint for Π 1 and use the result to provide truth values for the negative literals in Π 2 .
The consequences of Π 2 can then be computed.
A new sequence of interpretations is deﬁned which terminates in what is sometimes called the standard model of Π .
Semantics for stratiﬁed normal logic program is discussed further in Appendix A.
Implementation of safe stratiﬁed negation in ∂ ILP : The following are the major modiﬁcations that we made to ∂ ILP to implement safe, stratiﬁed negation.
Our clause generation process enforces stratiﬁcation.
The strata themselves are speciﬁed as part of the ∂ ILP program template, along with names and arities of predicates to be learned.
∂ ILP requires clauses to be safe , i.e., every variable in the head of a rule appears in the body of the rule (i.e., p(X) :- q(Y) is not safe).
[Table]
Y). We generalize safety to normal clauses by requiring that all variables of a rule appear in positive (non-negated) atoms in the body.
Our clause generation process allows negation and respects stratiﬁcation and safety.
An example is included in Appendix C. The negation of an atom a is evaluated as value ( ¬ a ) = 1 − value ( a ) , and forward chaining is done on a stratum-by-stratum basis.
This is implemented using binary masks with 1 used when the predicate belongs in current stratum,otherwise.
The total number of forward chaining steps were equally divided between the strata.
To prevent high entropy of the rule weights, we introduced average entropy of the rule weights as a regularization term in loss as L = 0 .
5 ∗ H ( P ) + 0 .
5 ∗ H ( N ) + γ ∗
avgH ( R ) where L is the loss, H is the cross entropy, P and N are positive and negative examples respectively, γ is regularization scaling factor and R is the rule weights.
We used a small γ , so the regularization does not prevent trajectory from leaving local minima.
We varied the number of steps of forward chaining ( 5 to 15 steps per strata), the learning rate (from 0 . 0001 and 0 . 05 ) and the regularization coefﬁcient (from 10 − 4 to 10 − 2 ).
To prevent class imbalance on positive and negative examples, we used equal weighted cross-entropy.
We noted that when a run converges with low loss but with high entropy, more than 1 program has nonzero weight, and the correct program (with respect to the training data) may not be weighted the highest.
but, we observed that the 3 highest weighted programs included correct program.
Experiments and Results
We tested our framework 5 on problems requiring negation with varying levels of complexity across different domains (nonmonotonic logic, arithmetic, graph, sets and lists).
Table 1 summarizes our results.
Here we outline the problems considered.
For details of the ILP problem, see Appendix D.
We ﬁrst considered negation in programs with a single clause using a classical example from nonmonotonic logic (“all birds ﬂy except penguins”).
This problem can be expressed as a disjunction of all birds with ﬂight, but has a simpler program with negation.
We then considered ( A ∧ ¬ B ) , which is the simplest program requiring negation.
In the domain of sets, the problem of learning / ∈ given elements of a set is similar, but more complex as it has two types of constants—sets and elements— which are undifferentiated.
We then examined problems requiring invented predicates and negation (such as prime and relative-prime, given divisibility as background), and ̸⊂ (given sets and elements as background).
7 We also examined graph problems requiring negation and recursive invented predicates.
These include unconnected vertices (given the edges of a directed graph as background) and monopoly i.e., the existence of a green-path but not a red-path in an edge coloured graph (given the edge-colours of a directed, edge coloured graph as background).
We also considered ‘not an element of’ a list, which requires recursion as a list is coded using head and tail predicates.
Our system was able to learn correct programs from very small training sets (loss across experiments are given in Appendix B).
The domain of sets includes two types of objects (sets and elements).
This generated many type-incorrect ground atoms, which in turn caused the majority of learned programs to only check for type instead of the correct programs.
For selected examples (less than, predecessor, family relationships), we ensured that our system reproduces results from ∂ ILP [5].
Furthermore, we veriﬁed that the problems from Table 1 cannot be solved by ∂ ILP without negation, and that unsafe or unstratiﬁed negation often leads to higher loss value (Appendix B), and we examined the noise tolerance in the “birds ﬂy” problem, where the system was able to identify the correct program even with high (up to 70% ) noise (negative examples incorrectly labeled as positive in training data.
More in Appendix B).
We expect these results to generalize to other problems.
The learned program is taken as the clauses with the highest weights after training.
However, there are conditions under which weights become distributed across multiple clauses even when the loss converges.
To prevent these conditions, we used the average entropy of weights across clauses as a regularizer.
This signiﬁcantly reduced average rule entropy, speciﬁcally from 0 .
12 to 10 − 4 for the prime problem.
This is discussed in Appendix B.
[Table]
Stratified
Unstratified
Discussion and Conclusion
We modiﬁed the ∂ ILP framework to incorporate safe, stratiﬁed negation.
Our modiﬁcations worked well with gradient descent to learn programs that require invented predicates, recursion and negation across several domains (arithmetic, graphs, sets and lists).
We also identiﬁed entropy of rule weights as a useful regularizer.
Our method is motivated by well-studied theoretical considerations on how to include negation.
So, it can be generalized to other differentiable implementations of ILP that use forward chaining deduction.
As such, any application of differentiable inductive logic programming, such as Reinforcement
Learning [15, 6] can beneﬁt from our work, since negation is useful in many places.
In future works, we plan to use this work to hierarchical reinforcement learning with planning.
Another direction of future works is allowing non-monotonic reasoning.
A Semantics for Stratiﬁed Normal Logic Programs
The following account is based in part on Apt et al.
Here, Π is a stratiﬁed normal logic program.
Given a ﬁxed vocabulary of constants, function symbols, and predicate symbols for Π , the set of all ground terms deﬁnable over that vocabulary is the Herbrand universe H Π of Π , and the set of all ground atoms is the Herbrand base B Π .
An Herbrand interpretation I of Π is any subset of B Π .
I is a model of a ground rule r iff head ( r ) is true in I or some element of body ( r ) is false in I .
I is a model of a non-ground rule r iff it is a model of every rule in G ( r ) , where G ( r ) is the ground instantiation of r .
Similarly, I is a model of Π if it is a model of every rule of Π .
Herbrand interpretations can be ordered using subset inclusion ( I 1 ≤ I 2 iff I 1 ⊆
I 2 ).
If Π is deﬁnite, there is a unique minimal model MM (Π) of Π that coincides with the ground atoms entailed by Π and which can be computed via T Π .
We deﬁne a sequence of interpretations using some I as basis.
• T Π ↑ 0( I ) =
I • T Π ↑ α ( I ) =
T Π ( T Π ↑ α − 1( I ))
∪ T Π
↑ α − 1( I ) for all successor ordinals α • T Π ↑ β ( I ) =
∪ α<β T Π ↑ α ( I ) for all limit ordinals β .
For deﬁnite programs, if I = ∅ , the sequence is nondecreasing and reaches MM (Π) at T Π ↑ ω ( ∅ ) , a ﬁxpoint.
If Π is normal, there might not be a unique minimal model, but a model with with desirable properties can be obtained by deﬁning a sequence of interpretations based on a stratiﬁcation Π 1 , . . .
, Π k of Π .
• M 1 = T Π 1 ↑ ω ( ∅ ) • M i =
T Π i ↑ ω
( M i − 1 ) for all 1 < i ≤ k
The ﬁnal interpretation M k is taken as the canonical model of the program and as deﬁning the ground atomic consequences of Π .
It is a minimal model of Π [1], and it is supported (each a ∈ M k is either a fact or the head of a ground rule whose body is true in the model).
Importantly, the model is also independent of the speciﬁc stratiﬁcation; all stratiﬁcations of Π yield the same model.
B Additional details on experimental results
Figure 2 shows the loss plotted against training epochs for various experiments.
We ran 20 trials for each problem, and each plot shows the convergence behaviour of each problem.
Number of runs converged deﬁned as loss ≤ 10 − 3 were identiﬁed as convergent.
Loss is computed as incorrect classiﬁcations of training examples.
As mentioned in the Methods section, we introduced entropy as a regularizer in the loss.
We examined the distribution of weights after convergence (Figure 3) and found that including a regularization term reduced entropy and resulted in only one of the clauses having high weight for each of the learned predicates.
In contrast, when regularization was not used, even in cases when the loss converged, many clauses contributed to the output used in predictions and increased the ambiguity of the program learned by the framework.
We then tested whether stratiﬁcation was required for solving problems with negation.
For Unconnected and Monopoly (which required 2 and 3 strata, respectively) we tested convergence and examined the programs learned when stratiﬁcation was enforced and also when it was not (in the latter case, all clauses were placed in a single stratum).
We found that the loss was higher than 10 −when stratiﬁcation was not enforced (Figure 1) and also that the programs identiﬁed were incorrect.
Finally, we varied the noise for the birds-ﬂy problem.
Results for that are presented in Figure 4.
C Clause generation
This section discusses the generation of safe and stratiﬁed clauses.
[Figure]
C.1 Stratiﬁcation
Stratiﬁcation involves assigning predicates to strata and is a well understood method to prevent inconsistencies when negation is introduced.
Below we demonstrate how stratiﬁcation is introduced in different programs.
We start with assigning lowest strata to background, positive and negative.
Then, the invented predicates are assigned the subsequent strata and ﬁnal output or target predicate is given the highest strata.
Below, we illustrate the reason for this assignment using examples.
The below program can be stratiﬁed by assigning edge to the lowest stratum (1), path to stratum(or n ≥ 1 ) and unconn to stratum 2 (or m > n ).
unconn
(X,Y) :- ¬
path
(X,Y
path(X,Y) :- edge(X,Y)
path(X,Y) :- edge(X,Z), path(Z,Y).
The predicate path can call itself recursively, since stratiﬁcation allows predicates of the same stratum as head to be used positively in the body.
However, unconn uses path negatively and so unconn must be of a higher stratum than path.
The following program is also stratiﬁed; p and q can be assigned to the same stratum.
p(X) :- q(X).
q(X) :- p(X).
However, neither the program { p(X) :- ¬ p(X) } nor { p(X):- q(X) , q(X):- ¬ p(X) } is stratiﬁed, as there is no way to assign the body to a lower stratum than the head of any rule.
C.2 Safety
Safety in ∂ ILP : A rule is safe if the set of variables in the head is a subset of the variables in the body.
For instance, p(X,Y) :- q(X,Z), p(X,Z). is not safe as Y is present in the head but not the body.
Safety with negation : Here, a rule is safe if each variable in a rule is also present in a positive body literal the rule.
Observe that this is a generalization of the rule above.
E.g., p(X,Y) :- q(X,Y), ¬ r(X,X) is safe, whereas p(X,Y) :- q(X,Y), ¬ r(X,Z).
is not.
C.3 Some consequences
Given the safety conditions, and the restriction from ∂ ILP that each clause contain 2 literals in the body, there can be at most one negated literal in the body of any clause.
Since each program can have at most 2 clauses, a recursive program has one base clause and one recursive clause.
Further, if the recursive call does not contain a variable not in the head, then a terminating recursion can be unfolded to a ﬁxed ﬁnite number of steps.
This latter consequence happens in ∂ ILP as well.
D ILP problems D.1 Birds ﬂy, except for penguins
The task here is to learn the program fly(X):- birds(X), ¬ penguin(X) from background knowledge consisting of instances of birds, given as bird(X) , ∀ X ∈ C (where C is a set of constants).
Each individual bird belongs to a particular type type (X) , where type ∈ { eagle , sparrow , owl , parrot , hawk , penguin } .
The positive and negative examples indicate whether the bird can ﬂy ( fly(X) ).
This is false when X is a penguin and true otherwise.
Domain is the set of names of birds of different types.
Each bird type had 5 instances.
D.2 A and not B
The task is to learn target(X) :- a(X), ¬ b(X) from background knowledge consisting of instances of ground atoms of the form a(X) and b(X) .
Positive examples are in a but not b .
D.3 Prime
The task here is to learn a program for primes.
The constants forming the domain are { 2 , 3 , 4 , 5 , 6 , 7 , 8 , 9 } ; we exclude 1 since it is neither prime nor composite.
In this setting, a prime number has only itself as a factor.
The background knowledge consists of ground atoms encoding equality ( eq(X,X) for each X in the domain) and divisibility ( div(X,Y) indicates that X is a multiple of Y ).
The positive and negative examples are the prime and composite numbers from the domain.
Other programs found include the following.
Here, pred is an invented predicate.
prime(X) :- ¬ pred(X), div(X,X) .
pred(X) :- div(X,Y), ¬ eq(X,Y)
D.4 Relatively Prime
The task here is to learn a program for relative primes from background knowledge consisting of the binary predicate div and eq .
We also need a guard-predicate that includes all pairs of numbers to meet our safety requirements.
Program: { relPrime(X,Y):- ¬ commonFactor(X,Y) .
commonFactor
(X,Y):- div(X,Z), div(Y,Z) }.
D.5 Unconnected
The background knowledge here encodes a directed graph with the constants naming vertices and a binary predicate deﬁning directed edges.
The positive examples include the pairs ( x, y ) of unconnected vertices.
The negative examples are the connected ones.
The intended program requires an invented recursive predicate, e.g., unConn(X,Y):- ¬ path(X,Y), guard(X,Y) .
The base case for path is path(X,Y) :- edge(X,Y) and the recursive case is path(X,Y) :- edge(X,Z), path(Z,Y) .
Instead of giving an external guard predicate, we made our examples such that given any pair of vertices x and y , either there was a path from x to y or there was a path from y to x .
The system was able to ﬁnd this and use it as a guard, i.e., unConn(X,Y) :- ¬ path(X,Y), path(Y,X) .
The learned predicate for the path varied with different runs, e.g., the recursive case could be path(X,Y):- path(X,Z), path(Z,Y) .
D.6 Monopoly
Given a directed, edge coloured graph, mon(X,Y) holds iff there is a green path from X to Y but no red path from X to Y .
The constants are vertices of a graph, and the background binary predicates
This redge and gedge encode red and green edges.
The positive examples included pairs of vertices satisfying the property mon above.
Even when the system had difﬁculty ﬁnding the correct target programs, it was able to learn the correct auxiliary predicates for red and green paths.
D.7 Not element
The task here is to learn a program for / ∈ .
The constants include { p, q, r, 1 , 2 , 3 , 4 } with p , q and r intended to represent sets and 1 , 2 , 3 and 4 representing elements.
The background knowledge includes binary predicates ∈ and a guard .
This problem is similar to A ∧ ¬ B , but has the added complexity of two types and multiple tokens of each type.
D.8 Not Subset
The task here is to learn the program ̸⊆ .
The constants and background knowledge are as above.
Positive examples are pairs of sets X and Y such that X ̸⊆ Y , i.e., ∃ Z.Z ∈ X ∧ Z / ∈ Y .
The intended program uses the background predicate ∈ and a learned predicate / ∈ .
The relation ⊆ is deﬁned using implication, i.e. X ⊆ Y holds iff
∀ Z.Z ∈ X → Z ∈ Y .
The program for strict superset would be SS(X,Y) :- ele(Z,X), ¬ ele(Z,Y).
Then, subset can be deﬁned as sub(X,Y) :- ¬ SS(X,Y) .
Thus, ̸⊂ is more basic than ⊂ in this setting.
D.9 Not Element of a list
The task here is to learn a program identifying when an item is not a member of a list.
Th constants are { p 1 , p 2 , q 1 , q 2 , 0 , 1 , 2 , 3 , } where p i , q i and 0 are lists and 1 , 2 , 3 are elements of the list (represents the empty list).
The background predicates are head and tail (e.g., the list [1,2] would be head( p 1 , 1) , tail( p 1 , p 2 ) , head( p 2 ,2) and tail( p 2 ,0) .
Thus, to ﬁnd if a given element is a member of a list, a recursive predicate is needed.
Identifying list membership was investigated in ∂ ILP [5].
In our counterpart problem, positive examples are non-members, i.e., pairs X and Y where Y is not a member of the list X .