Graph Neural Networks for Reasoning 2-Quantified Boolean Formulas

1 INTRODUCTION . As deep learning makes astonishing achievements in the domain of image ( He et al. , 2016 ) and audio ( Hannun et al. , 2014 ) processing , natural languages ( Vaswani et al. , 2017 ) , and discrete heuristics decisions in games ( Silver et al. , 2017 ) , there is a profound interest in applying the relevant techniques in the field of logical reasoning . Logical reasoning problems span from simple propositional logic to complex predicate logic and high-order logic , with known theoretical complexities from NP-complete ( Cook , 1971 ) to semi-decidable and undecidable ( Church , 1936 ) . Testing the ability and limitation of machine learning tools on logical reasoning problems leads to a fundamental understanding of the boundary of learnability and robust AI , and addresses the interesting questions in decision procedures in logic , symbolic reasoning , and program analysis and verification as defined in the programming language community . There have been some successes in learning propositional logic reasoning ( Selsam et al. , 2019 ; Amizadeh et al. , 2019 ) , which focus on SAT ( Boolean Satisfiability ) problems as defined below . A propositional logic formula is an expression composed of Boolean constants ( > : true , ⊥ : false ) , Boolean variables ( xi ) , and propositional connectives such as ∧ , ∨ , ¬ ( for example ( x1 ∨ ¬x2 ) ∧ ( ¬x1 ∨ x2 ) ) . The SAT problem asks if a given formula can be satisfied ( evaluated to > ) by assigning proper Boolean values to the variables . A crucial feature of the logical reasoning domain ( as is visible in the SAT problem ) is that the inputs are often structural , where logical connections between entities ( variables in SAT problems ) are the key information . Accordingly , previous successes have used GNN ( Graph Neural Networks ) and message-passing based embeddings to solve SAT problems . However , it should be noted that logical decision procedures is more complex that just reading the formulas correctly . It is unclear if GNN embeddings ( via simple message-passing ) contain all the information needed to reason about complex logical questions on top of the graph structures derived from the formulas , or whether the complex embedding schemes can be learned from backpropagation . Previous successes on SAT problems argued for the power of GNN , which can handle NP-complete problems ( Selsam et al. , 2019 ; Amizadeh et al. , 2019 ) , but no successes have been reported for solving semi-decidable predicate logic problems via GNN . In order to find out where the limitation of GNN is and why , in learning logical reasoning problems , we decide to look at problems with complexity inbetween SAT and predicate logic problems , for which QBF ( Quantified Boolean Formula ) problems serve as excellent middle steps . QBF is an extension of propositional formula , which allows quantifiers ( ∀ and ∃ ) over the Boolean variables ( such as ∀x1∃x2 . ( x1 ∨ ¬x2 ) ∧ ( ¬x1 ∨ x2 ) ) . In general , a quantified Boolean formula in prenex normal form can be expressed as such : QiXiQi−1Xi−1 ... Q0X0 . φ where Qi are quantifiers that always differ from their neighboring quantifiers , Xi are disjoint sets of Boolean variables , and φ is a propositional formula with all Boolean variables bounded in Qi . Complexity-wise , QBF problems are PSPACE-complete ( Kleine Büning & Bubeck , 2009 ) , which lies in-between the NP-completeness of SAT problems and the semi-decidability of predicate logic problems . Furthermore , 2-QBF ( QBF with only two alternative quantifiers ) is ΣP2 -complete ( Kleine Büning & Bubeck , 2009 ) . Another direction of addressing logical reasoning problems via machine learning is to learn heuristic decisions within traditional decision procedures . This direction is less appealing from a theoretical perspective , but more interesting from a practical one , since it has been shown to speed up SAT solvers in practical settings ( Selsam & Bjørner , 2019 ) . In this direction , there is less concern about the embedding power of GNN , but more about the design of the training procedures ( what is the data and label for training ) and how to incorporate the trained models within the decision procedures . The embeddings captured via GNN is rather preferred to be lossy to prevent overfitting ( Selsam & Bjørner , 2019 ) . In this paper we explore the potential applications of GNNs to 2QBF problems . In Section 2 , we illustrate our designs of GNN architectures for embedding 2QBF formulas . In Section 3 , we evaluate GNN-based 2QBF solvers , and conjecture with empirical evidences that the current GNN techniques are unable to learn complete SAT solvers or 2QBF solvers . In Section 4 , we demonstrate the potential of our GNN-based heuristics for selecting candidates and counter-examples in the CEGAR-based solver framework . In Section 5 , we discuss the related work and conclude in Section 6 . Throughout the paper we redirect details to supplementary materials . We make the following contributions : 1 . Design and test possible GNN architectures for embedding 2QBF . 2 . Pinpoint the limitation of GNN in learning logical decision procedures that need reasoning about a space of Boolean assignments . 3 . Learn GNN-based CEGAR solver heuristics via supervised learning and uncover interesting challenges for GNN to generalize across graph structures . 2 GNN EMBEDDING OF PROPOSITIONAL LOGICAL FORMULAS . Preliminary : Graph Neural Networks . GNNs refer to the neural architectures devised to learn the embeddings of nodes and graphs via message-passing . Resembling the generic definition in Xu et al . ( 2019 ) , they consist of two successive operators to propagate the messages and evolve the embeddings over iterations : m ( k ) v =Aggregate ( k ) ( { h ( k−1 ) u : u ∈ N ( v ) } ) , h ( k ) v =Combine ( k ) ( h ( k−1 ) v , m ( k ) v ) ( 1 ) where h ( k ) v denotes the hidden state ( embedding ) of node v at the kth layer/iteration , and N ( v ) denotes the neighbors of node v. In each iteration , the Aggregate ( k ) ( · ) aggregates hidden states from node v ’ s neighbors to produce the new message ( i.e. , m ( k ) v ) for node v , and Combine ( k ) ( · , · ) computes the new embedding of v with its last state and its current message . After a specific number of iterations ( e.g. , K ) , the embeddings should capture the global relational information of the nodes , which can be fed into other neural network modules for specific tasks . GNN Architecture for Embedding SAT formulas . Previous success ( Selsam et al. , 2019 ) of GNN-based SAT solvers embedded SAT formulas like below . Each SAT formula is translated into a bipartite graph , where one kind of nodes represent all literals ( Boolean variables and their negations , denoted as L ) , and the other kind of nodes represent all clauses ( sets of literals that are connected via ∨ , denoted as C ) . Edges between literal and clause nodes represent the literal appearing in that clause , and all edges are represented by a sparse adjacent matrix ( EdgeMatrix ( E ) ) of dimension |C| × |L| . There is also another kind of edges connecting literals with their negations . The graph representation of ( x1 ∨ ¬x2 ) ∧ ( ¬x1 ∨ x2 ) is given below as an example . Note that this architecture is specific for propositional formulas in Conjunctive Normal Form ( CNF ) , which is composed of clauses connected via ∧ . C1 C2 x1 x2 ¬x1 ¬x2 The embeddings of literals and clauses are initialized with tiled random vectors . Then the GNN uses MLPs to compute the messages of literals and clauses from the embeddings , and LSTMs to update embeddings with the aggregated messages . The mathematical process for one iteration of message-passing is given below , where EmbL and EmbC denotes embedding matrices of literals and clauses respectively , MsgX→Y denotes messages from X to Y , MLPX denotes MLPs of X for generating messages from the embeddings , LSTMX denotes LSTMs of X for digesting incoming messages and updating the embeddings , and · T [ ] represent matrix multiplication , transposition , and concatenation respectively . Furthermore , Emb¬L denotes a permutational view of EmbL such that the same row of EmbL and Emb¬L are embeddings of a variable and its negation respectively . MsgL→C = E ·MLPL ( EmbL ) # aggregate clauses EmbC = LSTMC ( EmbC , MsgL→C ) # combine clauses MsgC→L = E T ·MLPC ( EmbC ) # aggregate literals EmbL = LSTML ( EmbL , [ MsgC→L , Emb¬L ] ) # combine literals ( 2 ) Note that different instances of MLPs and LSTMs are used for clauses and literals ( they have different subscripts ) . What ’ s more , Emb¬L is used as additional message when updating EmbL . GNN Architectures for Embedding 2QBF . The difference between SAT formulas and 2QBF is that in 2QBF the variables are quantified by ∀ or ∃ . To reflect that difference in graph representation , we separate ∀-literals and ∃-literals into different groups of nodes . For example , the graph representation of ∀x1∃x2 . ( x1 ∨ ¬x2 ) ∧ ( ¬x1 ∨ x2 ) is shown below : C1 C2 x1 ¬x1 x2 ¬x2 Accordingly , in GNN architectures , the separated ∀-literals and ∃-literals are embedded via different modules . The GNN architecture design closely resembles the design philosophy of Selsam et al . ( 2019 ) in terms of permutation invariance and negation invariance , and would most likely carry over the success of GNN in solving SAT problems to 2QBF problems . MsgL→C = [ E∀ ·MLP∀ ( Emb∀ ) , E∃ ·MLP∃ ( Emb∃ ) ] # aggregate clauses EmbC = LSTMC ( EmbC , MsgL→C ) # combine clauses MsgC→∀ = E T ∀ ·MLPC→∀ ( EmbC ) # aggregate ∀ Emb∀ = LSTM∀ ( Emb∀ , [ MsgC→∀ , Emb¬∀ ] ) # combine ∀ MsgC→∃ = E T ∃ ·MLPC→∃ ( EmbC ) # aggregate ∃ Emb∃ = LSTM∃ ( Emb∃ , [ MsgC→∃ , Emb¬∃ ] ) # combine ∃ ( 3 ) Note that we use ∀ and ∃ to denote all ∀-literals and all ∃-literals respectively . We use EX to denote the EdgeMatrix between X and C , and MLPC→X to denote MLPs that generate MsgC→X . We de facto have tested more GNN architectures for 2QBF ( see the supplementary material A.1 ) , yet the model above performed the best in our later evaluation , so we used it in the main paper . 3 GNN-BASED SOLVERS FAIL IN 2QBF PROBLEMS . In the previous section , we have discussed GNN-based embeddings in propositional logical formulas . We then test whether GNN-based 2QBF solvers can be learned , following the previous successes ( Selsam et al. , 2019 ; Amizadeh et al. , 2019 ) . 3.1 EMPIRICAL STUDY FOR REASONING 2QBF BY GNN . Data Preparation . In training and testing , we follow the previous work ( Chen & Interian , 2005 ) to generate random 2QBF formulas of specs ( 2,3 ) and sizes ( 8,10 ) . That is to say , each clause has 5 literals , 2 of them are randomly chosen from a set of 8 ∀-quantified variables , and 3 of them are randomly chosen from a set of 10 ∃-quantified variables . We modify the generation procedure so that it generates clauses until the formula becomes unsatisfiable . We then randomly negate an ∃-quantified literal per formula to get a very similar but satisfiable formula . Predicting Satisfiability . We first tested whether our graph embeddings can be used to predict satisfiability of 2QBF formulas . We extended the GNN architectures with a voting MLP ( MLPvote ) that takes the embeddings of the ∀-variables after the propagation , and uses the average votes as logits for satisfiability/unsatisfiability prediction : logitssat = mean ( MLPvote ( Emb∀ ) ) We trained our GNNs with different amount of data ( 40 pairs , 80 pairs , and 160 pairs of satisfiable/unsatisfiable formulas ) and different numbers of message-passing iterations ( 8 iters , 16 iters , and 32 iters ) , and then evaluated the converged models on 600 pairs of new instances . We report the accuracy rate of unsatisfiable and satisfiable formulas as tuples for both the training dataset and the testing dataset . By alternating the random seeds , the models with the best training data performance are selected and shown in Table 1 . Since pairs of satisfiable/unsatisfiable formulas are only different by one literal , it forces the GNNs to learn subtle structural differences in the formulas . The GNNs fit well to the smaller training dataset but have trouble for 160 pairs of formulas ( numbers in the green color ) . Performances of the models on testing dataset are close to random guesses ( numbers in the blue color ) , and running more iterations during testing does not help with their performances . Predicting Unsatisfiability Witnesses . Previous work ( Selsam et al. , 2019 ; Amizadeh et al. , 2019 ) also showed successes in predicting satisfiable witnesses ( variable assignments that satisfy the formulas ) of SAT problems . 2QBF problems have unsatisfiable witnesses ( assignments to ∀ variables that render the reduced propositional formulas unsatisfiable ) . Next , we test if we can train GNNs to predict unsatisfiable witnesses of 2QBF formulas . Specifically , the final embeddings of ∀-variables are transformed into logits via an assignment MLP ( MLPasn ) and then used to compute the cross-entropy loss with the actual unsatisfiability witnesses of the formula : logitswitness = MLPasn ( Emb∀ ) Once again we tried different amount of training data ( 160 , 320 , and 640 unsatisfiable formulas ) and different numbers of iterations ( 8 iters , 16 iters , and 32 iters ) , and then tested the converged models on 600 unsatisfiable new 2QBF formulas . We report the accuracy per variable and the accuracy per formula as tuples for both the training dataset and the testing dataset in Table 2 , from which we can observe that the GNNs fit well to the training data ( numbers in green color ) , especially with more message-passing iterations . However , the GNN performance on testing data is only slightly better than random guesses ( numbers in blue color ) , and running more iterations during testing does not help with the performance either .

Summary: This paper investigated the GNN-based solvers for the 2-Quantified Boolean Formula satisfiability problem. This paper points out that GNN has limitations in reasoning about unsatisfiability of SAT problems possibly due to the simple message-passing scheme. To extend the GNN-based SAT solvers to 2-QBF solvers, this paper then turns to learn GNN-based heuristics that work with traditional decision procedure, and proposes a CEGAR-based 2QBF algorithm.