Graph Neural Networks for Reasoning 2-Quantified Boolean Formulas

1 INTRODUCTION . As deep learning makes astonishing achievements in the domain of image ( He et al. , 2016 ) and audio ( Hannun et al. , 2014 ) processing , natural languages ( Vaswani et al. , 2017 ) , and discrete heuristics decisions in games ( Silver et al. , 2017 ) , there is a profound interest in applying the relevant techniques in the field of logical reasoning . Logical reasoning problems span from simple propositional logic to complex predicate logic and high-order logic , with known theoretical complexities from NP-complete ( Cook , 1971 ) to semi-decidable and undecidable ( Church , 1936 ) . Testing the ability and limitation of machine learning tools on logical reasoning problems leads to a fundamental understanding of the boundary of learnability and robust AI , and addresses the interesting questions in decision procedures in logic , symbolic reasoning , and program analysis and verification as defined in the programming language community . There have been some successes in learning propositional logic reasoning ( Selsam et al. , 2019 ; Amizadeh et al. , 2019 ) , which focus on SAT ( Boolean Satisfiability ) problems as defined below . A propositional logic formula is an expression composed of Boolean constants ( > : true , ⊥ : false ) , Boolean variables ( xi ) , and propositional connectives such as ∧ , ∨ , ¬ ( for example ( x1 ∨ ¬x2 ) ∧ ( ¬x1 ∨ x2 ) ) . The SAT problem asks if a given formula can be satisfied ( evaluated to > ) by assigning proper Boolean values to the variables . A crucial feature of the logical reasoning domain ( as is visible in the SAT problem ) is that the inputs are often structural , where logical connections between entities ( variables in SAT problems ) are the key information . Accordingly , previous successes have used GNN ( Graph Neural Networks ) and message-passing based embeddings to solve SAT problems . However , it should be noted that logical decision procedures is more complex that just reading the formulas correctly . It is unclear if GNN embeddings ( via simple message-passing ) contain all the information needed to reason about complex logical questions on top of the graph structures derived from the formulas , or whether the complex embedding schemes can be learned from backpropagation . Previous successes on SAT problems argued for the power of GNN , which can handle NP-complete problems ( Selsam et al. , 2019 ; Amizadeh et al. , 2019 ) , but no successes have been reported for solving semi-decidable predicate logic problems via GNN . In order to find out where the limitation of GNN is and why , in learning logical reasoning problems , we decide to look at problems with complexity inbetween SAT and predicate logic problems , for which QBF ( Quantified Boolean Formula ) problems serve as excellent middle steps . QBF is an extension of propositional formula , which allows quantifiers ( ∀ and ∃ ) over the Boolean variables ( such as ∀x1∃x2 . ( x1 ∨ ¬x2 ) ∧ ( ¬x1 ∨ x2 ) ) . In general , a quantified Boolean formula in prenex normal form can be expressed as such : QiXiQi−1Xi−1 ... Q0X0 . φ where Qi are quantifiers that always differ from their neighboring quantifiers , Xi are disjoint sets of Boolean variables , and φ is a propositional formula with all Boolean variables bounded in Qi . Complexity-wise , QBF problems are PSPACE-complete ( Kleine Büning & Bubeck , 2009 ) , which lies in-between the NP-completeness of SAT problems and the semi-decidability of predicate logic problems . Furthermore , 2-QBF ( QBF with only two alternative quantifiers ) is ΣP2 -complete ( Kleine Büning & Bubeck , 2009 ) . Another direction of addressing logical reasoning problems via machine learning is to learn heuristic decisions within traditional decision procedures . This direction is less appealing from a theoretical perspective , but more interesting from a practical one , since it has been shown to speed up SAT solvers in practical settings ( Selsam & Bjørner , 2019 ) . In this direction , there is less concern about the embedding power of GNN , but more about the design of the training procedures ( what is the data and label for training ) and how to incorporate the trained models within the decision procedures . The embeddings captured via GNN is rather preferred to be lossy to prevent overfitting ( Selsam & Bjørner , 2019 ) . In this paper we explore the potential applications of GNNs to 2QBF problems . In Section 2 , we illustrate our designs of GNN architectures for embedding 2QBF formulas . In Section 3 , we evaluate GNN-based 2QBF solvers , and conjecture with empirical evidences that the current GNN techniques are unable to learn complete SAT solvers or 2QBF solvers . In Section 4 , we demonstrate the potential of our GNN-based heuristics for selecting candidates and counter-examples in the CEGAR-based solver framework . In Section 5 , we discuss the related work and conclude in Section 6 . Throughout the paper we redirect details to supplementary materials . We make the following contributions : 1 . Design and test possible GNN architectures for embedding 2QBF . 2 . Pinpoint the limitation of GNN in learning logical decision procedures that need reasoning about a space of Boolean assignments . 3 . Learn GNN-based CEGAR solver heuristics via supervised learning and uncover interesting challenges for GNN to generalize across graph structures . 2 GNN EMBEDDING OF PROPOSITIONAL LOGICAL FORMULAS . Preliminary : Graph Neural Networks . GNNs refer to the neural architectures devised to learn the embeddings of nodes and graphs via message-passing . Resembling the generic definition in Xu et al . ( 2019 ) , they consist of two successive operators to propagate the messages and evolve the embeddings over iterations : m ( k ) v =Aggregate ( k ) ( { h ( k−1 ) u : u ∈ N ( v ) } ) , h ( k ) v =Combine ( k ) ( h ( k−1 ) v , m ( k ) v ) ( 1 ) where h ( k ) v denotes the hidden state ( embedding ) of node v at the kth layer/iteration , and N ( v ) denotes the neighbors of node v. In each iteration , the Aggregate ( k ) ( · ) aggregates hidden states from node v ’ s neighbors to produce the new message ( i.e. , m ( k ) v ) for node v , and Combine ( k ) ( · , · ) computes the new embedding of v with its last state and its current message . After a specific number of iterations ( e.g. , K ) , the embeddings should capture the global relational information of the nodes , which can be fed into other neural network modules for specific tasks . GNN Architecture for Embedding SAT formulas . Previous success ( Selsam et al. , 2019 ) of GNN-based SAT solvers embedded SAT formulas like below . Each SAT formula is translated into a bipartite graph , where one kind of nodes represent all literals ( Boolean variables and their negations , denoted as L ) , and the other kind of nodes represent all clauses ( sets of literals that are connected via ∨ , denoted as C ) . Edges between literal and clause nodes represent the literal appearing in that clause , and all edges are represented by a sparse adjacent matrix ( EdgeMatrix ( E ) ) of dimension |C| × |L| . There is also another kind of edges connecting literals with their negations . The graph representation of ( x1 ∨ ¬x2 ) ∧ ( ¬x1 ∨ x2 ) is given below as an example . Note that this architecture is specific for propositional formulas in Conjunctive Normal Form ( CNF ) , which is composed of clauses connected via ∧ . C1 C2 x1 x2 ¬x1 ¬x2 The embeddings of literals and clauses are initialized with tiled random vectors . Then the GNN uses MLPs to compute the messages of literals and clauses from the embeddings , and LSTMs to update embeddings with the aggregated messages . The mathematical process for one iteration of message-passing is given below , where EmbL and EmbC denotes embedding matrices of literals and clauses respectively , MsgX→Y denotes messages from X to Y , MLPX denotes MLPs of X for generating messages from the embeddings , LSTMX denotes LSTMs of X for digesting incoming messages and updating the embeddings , and · T [ ] represent matrix multiplication , transposition , and concatenation respectively . Furthermore , Emb¬L denotes a permutational view of EmbL such that the same row of EmbL and Emb¬L are embeddings of a variable and its negation respectively . MsgL→C = E ·MLPL ( EmbL ) # aggregate clauses EmbC = LSTMC ( EmbC , MsgL→C ) # combine clauses MsgC→L = E T ·MLPC ( EmbC ) # aggregate literals EmbL = LSTML ( EmbL , [ MsgC→L , Emb¬L ] ) # combine literals ( 2 ) Note that different instances of MLPs and LSTMs are used for clauses and literals ( they have different subscripts ) . What ’ s more , Emb¬L is used as additional message when updating EmbL . GNN Architectures for Embedding 2QBF . The difference between SAT formulas and 2QBF is that in 2QBF the variables are quantified by ∀ or ∃ . To reflect that difference in graph representation , we separate ∀-literals and ∃-literals into different groups of nodes . For example , the graph representation of ∀x1∃x2 . ( x1 ∨ ¬x2 ) ∧ ( ¬x1 ∨ x2 ) is shown below : C1 C2 x1 ¬x1 x2 ¬x2 Accordingly , in GNN architectures , the separated ∀-literals and ∃-literals are embedded via different modules . The GNN architecture design closely resembles the design philosophy of Selsam et al . ( 2019 ) in terms of permutation invariance and negation invariance , and would most likely carry over the success of GNN in solving SAT problems to 2QBF problems . MsgL→C = [ E∀ ·MLP∀ ( Emb∀ ) , E∃ ·MLP∃ ( Emb∃ ) ] # aggregate clauses EmbC = LSTMC ( EmbC , MsgL→C ) # combine clauses MsgC→∀ = E T ∀ ·MLPC→∀ ( EmbC ) # aggregate ∀ Emb∀ = LSTM∀ ( Emb∀ , [ MsgC→∀ , Emb¬∀ ] ) # combine ∀ MsgC→∃ = E T ∃ ·MLPC→∃ ( EmbC ) # aggregate ∃ Emb∃ = LSTM∃ ( Emb∃ , [ MsgC→∃ , Emb¬∃ ] ) # combine ∃ ( 3 ) Note that we use ∀ and ∃ to denote all ∀-literals and all ∃-literals respectively . We use EX to denote the EdgeMatrix between X and C , and MLPC→X to denote MLPs that generate MsgC→X . We de facto have tested more GNN architectures for 2QBF ( see the supplementary material A.1 ) , yet the model above performed the best in our later evaluation , so we used it in the main paper . 3 GNN-BASED SOLVERS FAIL IN 2QBF PROBLEMS . In the previous section , we have discussed GNN-based embeddings in propositional logical formulas . We then test whether GNN-based 2QBF solvers can be learned , following the previous successes ( Selsam et al. , 2019 ; Amizadeh et al. , 2019 ) . 3.1 EMPIRICAL STUDY FOR REASONING 2QBF BY GNN . Data Preparation . In training and testing , we follow the previous work ( Chen & Interian , 2005 ) to generate random 2QBF formulas of specs ( 2,3 ) and sizes ( 8,10 ) . That is to say , each clause has 5 literals , 2 of them are randomly chosen from a set of 8 ∀-quantified variables , and 3 of them are randomly chosen from a set of 10 ∃-quantified variables . We modify the generation procedure so that it generates clauses until the formula becomes unsatisfiable . We then randomly negate an ∃-quantified literal per formula to get a very similar but satisfiable formula . Predicting Satisfiability . We first tested whether our graph embeddings can be used to predict satisfiability of 2QBF formulas . We extended the GNN architectures with a voting MLP ( MLPvote ) that takes the embeddings of the ∀-variables after the propagation , and uses the average votes as logits for satisfiability/unsatisfiability prediction : logitssat = mean ( MLPvote ( Emb∀ ) ) We trained our GNNs with different amount of data ( 40 pairs , 80 pairs , and 160 pairs of satisfiable/unsatisfiable formulas ) and different numbers of message-passing iterations ( 8 iters , 16 iters , and 32 iters ) , and then evaluated the converged models on 600 pairs of new instances . We report the accuracy rate of unsatisfiable and satisfiable formulas as tuples for both the training dataset and the testing dataset . By alternating the random seeds , the models with the best training data performance are selected and shown in Table 1 . Since pairs of satisfiable/unsatisfiable formulas are only different by one literal , it forces the GNNs to learn subtle structural differences in the formulas . The GNNs fit well to the smaller training dataset but have trouble for 160 pairs of formulas ( numbers in the green color ) . Performances of the models on testing dataset are close to random guesses ( numbers in the blue color ) , and running more iterations during testing does not help with their performances . Predicting Unsatisfiability Witnesses . Previous work ( Selsam et al. , 2019 ; Amizadeh et al. , 2019 ) also showed successes in predicting satisfiable witnesses ( variable assignments that satisfy the formulas ) of SAT problems . 2QBF problems have unsatisfiable witnesses ( assignments to ∀ variables that render the reduced propositional formulas unsatisfiable ) . Next , we test if we can train GNNs to predict unsatisfiable witnesses of 2QBF formulas . Specifically , the final embeddings of ∀-variables are transformed into logits via an assignment MLP ( MLPasn ) and then used to compute the cross-entropy loss with the actual unsatisfiability witnesses of the formula : logitswitness = MLPasn ( Emb∀ ) Once again we tried different amount of training data ( 160 , 320 , and 640 unsatisfiable formulas ) and different numbers of iterations ( 8 iters , 16 iters , and 32 iters ) , and then tested the converged models on 600 unsatisfiable new 2QBF formulas . We report the accuracy per variable and the accuracy per formula as tuples for both the training dataset and the testing dataset in Table 2 , from which we can observe that the GNNs fit well to the training data ( numbers in green color ) , especially with more message-passing iterations . However , the GNN performance on testing data is only slightly better than random guesses ( numbers in blue color ) , and running more iterations during testing does not help with the performance either .

Summary: This paper explores how graph neural networks can be applied to test satisfiability of 2QBF logical formulas. They show that a straightforward extension of a GNN-based SAT solver to 2QBF fails to outperform random chance, and argue that this is because proving either satisfiability or unsatisfiability of 2QBF requires reasoning over exponential sets of assignments. Instead, they show that GNNs can be useful as a heuristic candidate- or counterexample- ranking model which improves the efficiency of the CEGAR algorithm for solving 2QBF.