Top-Down Neural Model For Formulae

1 INTRODUCTION . In real-world situations a very successful approach , popularized in Kahneman ( 2011 ) , to problem solving is based on a clever combination of fast instinctive ( heuristic ) reasoning and slow logical reasoning . The latter is exemplified by abstract logical formulae where only structural properties matter . If computers are involved , a logical formula is traditionally a syntactic object which is a subject to simple but very fast syntactic manipulations Robinson & Voronkov ( 2001 ) . Hence all but very basic decisions are postponed if possible . However , this viewpoint is rapidly changing as various AI methods are tested in the field of automated reasoning , in particular machine learning methods . A fundamental problem in using machine learning in automated reasoning is a suitable representation of logical formulae . A formula as a solely syntactic object is no longer sufficient and we have to exploit its semantic properties . Various approaches have been proposed for different logical systems . In this paper we will concentrate on the simplest yet very powerful standard logical system—classical ( Boolean ) propositional logic . This paper presents , as far as we know , a novel neural representation of propositional formulae that makes it possible to test whether a given formula has a given property , e.g. , whether the formula is always true or not . Clearly , we try to solve a well-known CONP-complete problem . However , the fact that the problem is generally hard and requires a non-trivial search does not rule out the possibility that a decent heuristic can be learned , moreover , if only a specific subset of formulae is involved . In particular , our general goal is to obtain a useful heuristic that can help us in guiding a proof search , where we typically face numerous choice points . Unlike in natural language processing , a parse tree for a formula is available for free . Although some approaches do not exploit this feature and try to learn the structure of a formula on their own , using usually various recurrent neural networks ( RNN ) , it is more common to take advantage of this knowledge . Moreover , it seems that the later approach has a significant edge , see Evans et al . ( 2018 ) . Usually propositional atoms , the basic building blocks of propositional formulae , are learned as embeddings and each logical connective is treated as a unique neural network that given the vector representation of its arguments produces a vector that represents an application of the connective on these arguments , e.g. , a binary connective takes two vectors of length d , and produces a new one of length d , see Allamanis et al . ( 2017 ) . This clearly leads to tree recursive neural networks Socher et al . ( 2012 ) where the structure of the network follows the parse tree . Such models are built bottomup and the meaning of the formula is usually the vector produced in the root of the tree . Our model also uses the parse tree of the formula , but the knowledge is propagated in the opposite direction . We start with a vector ( random or learned ) , representing a property we want to test , and we propagate it from the root to leaves ( propositional atoms ) . The knowledge propagated to atoms is then processed by recurrent neural networks and a decision is produced . This makes it possible to ignore completely the names of propositional atoms and concentrate more on structural properties of formulae . The experimental results suggest that the model is more than competitive and beats other known approaches on some benchmarks . More importantly , our model seems to suffer less if bigger formulae are involved and could be more useful in real world scenarios . The structure of this paper is as follows . In Section 2 we discuss the architecture of our model in full details and also a dataset on which we will experiment is introduced there . In Section 3 we discuss an implementation of building blocks of our network , present experimental data , and shortly describe possible interpretations of our model . Some potential future modifications are briefly mentioned in Section 4 . Few relevant models are mentioned in Section 5 and the paper concludes with Section 6 . 2 MODEL . Our model tries to mimic the following approach sometimes used by humans to decide whether a propositional formula is always true ( tautology ) . If we want to know whether , e.g. , ( p→ q ) ∨ ( q → p ) is a tautology1 we can suppose the contrary and try to produce a truth-value assignment such that the whole formula is false . It means both p → q and q → p are false under such an assignment2 . It follows from the former that p is true and q is false , but from the latter we obtain that q is true and p is false . Because such an assignment is impossible , we showed that the formula can not be false and hence it is always true . In a nutshell , we try to propagate a truth value assigned to the whole formula , in our case the formula being false , to its subformulae and we repeat this process until we reach propositional atoms . This gives us a set of constraints on possible truth-value assignments to atoms . At this point we check whether a truth-value assignment satisfying these constraints exists and thus we answer the original question . However , it is pretty clear that usually this approach is not so straightforward . For example , if we want a formula of the form A → B to be true , it means that the subformula A is false or the subformula B is true . Moreover , such choices easily accumulate with the increasing depth3 of a formula . Still we will show that a neural model based on the above mentioned idea is competitive . Similarly to the standard recursive approach , we also represent a formula by a recursively built model based on the parse tree of the formula . However , unlike there we use a top-down approach . In our case , we start with a vector w ∈ Rd ( random or learned ) , where d is a parameter of the model , that roughly represents a property of formulae we want to check.4 We continue by propagating this knowledge to subformulae in a top-down manner , e.g. , a binary connective ◦ takes a vector v ∈ Rd ( input ) and produces two new vectors v1 ∈ Rd and v2 ∈ Rd ( outputs ) , hence ◦ : Rn → Rn × Rn . Since all the vectors have the same length d , it is possible to connect individual building blocks together and produce a recursive neural network . The basic building blocks of the tree part of our model are w ∈ Rd an initial vector , ci : Rd → Rd × · · · × Rd︸ ︷︷ ︸ k-times a neural network that represents a k-ary logical connective i , for all logical connectives i in our language5 . 1A formula that is true under any assignment of truth values to propositional atoms . 2For simplicity , from now on we assume that all the claims are under a given assignment . 3The depth of a formula is the depth of its parse tree . 4However , it does not mean that by simply changing w our model starts to test a different property , other parts of the model have to be trained accordingly , see also footnote 8 . 5Note that even nullary connectives , called constants , are allowed . From these components , which are shared among all formulae in a given language , we dynamically build a recursive network for a particular formula that corresponds to the parse tree of the formula— the parse tree tells us how to connect individual components together . For example , in the left part of Figure 1 we see a model for F = ( p → q ) ∨ ( q → p ) , which is directly obtained from the parse tree . The vector w is an input to the root of the tree and all the connectives in it are replaced by neural nets ci that represent them . Atoms contain propagated outputs ( vectors ) of this tree recursive model , here we obtain vectors p1 , q1 , q2 , p2 ∈ Rd , where p1 is at the position of the first occurrence of p in F . Note that each occurrence of the same atom in our tree model produces a unique vector . The fact that they correspond to the same atom is exploited by another level of our model . We take a recurrent neural network RNN-Var that has a sequence of all vectors corresponding to the same atom in the formula as an input and outputs a vector in Rd . In Figure 1 RNN-Var takes p1 and p2 and outputs p ∈ Rd and also the same RNN-Var takes q1 and q2 and outputs q ∈ Rd . One more recurrent neural network RNN-All takes a sequence of all output vectors produced by RNN-Var and outputs a vector in Rd that is input to a component Final : Rd → R2 which ends with a softmax layer and produces out ∈ R2 that makes it easy to decide whether the output of the whole network is true or false . It should be emphasized that our model completely ignores the names of atoms , we only check whether they are the same or distinct . Moreover , the number of distinct atoms that can occur in a formula is effectively only bounded by the ability of RNN-All to correctly process the outputs of RNN-Var . Hence the model can evaluate formulae that contain more atoms than formulae used for training . The basic building blocks of the recurrent part of our model are RNNVar : Rd × · · · × Rd → Rd a RNN aggregating vectors corresponding to the same atom , RNNAll : Rd × · · · × Rd → Rd a RNN aggregating the outputs of RNN-Var components , Final : Rd → R2 a final decision layer . It should be emphasized again how the model is built . Given a property we want to test , e.g. , whether a formula is always true , we train representations of w , ci , RNN-Var , RNN-All , and Final . These components are shared among all the formulae . For a single formula we produce a model ( neural network ) recursively from them as described above . For implementation details see Section 3 . 2.1 DATASET . To provide comparable results in Section 3 we use the dataset6 presented in Evans et al . ( 2018 ) and thoroughly described therein . For our purposes here it is essential that it contains triples of the form ( A , B , A |= B ) where A and B are propositinal formulae and A |= B indicates whether B follows from A . The dataset contains train ( 99876 triples ) , validation ( 5000 ) , and various test sets with a different level of difficulty given by a number of atoms and connectives7 occurring in them . From the deduction theorem we know that A |= B is equivalent to A → B being a tautology . We prefer the later form , because it fits nicely into our model and instead of learning the meaning of entailment ( |= ) , we can use the connective→ directly . Note that although our model ignores the names of atoms this can not be exploited directly , because validation and test sets do not contain pairs of formulae that would result from pairs in the training set by renaming atoms .

Summary: In this paper the authors propose a neural model that, given a logical formula as input, predicts whether the formula is a tautology or not. Showing that a formula is a tautology is important because if we can classify a formula A -> B as a tautology then we can say that B is a logical consequence of A. The structure of the formula is a feedforward neural network built in a top-down manner. The leaves of this network are vectors (each of them represents a particular occurrence of an atom) which, after the construction of the formula, are processed by some recurrent neural networks.