CNNSAT: Fast, Accurate Boolean Satisfiability using Convolutional Neural Networks

Boolean satisfiability ( SAT ) is one of the most well-known NP-complete problems and has been extensively studied . State-of-the-art solvers exist and have found a wide range of applications . However , they still do not scale well to formulas with hundreds of variables for uniform 3-SAT problems . To tackle this fundamental scalability challenge , we introduce CNNSAT , a fast and accurate statistical decision procedure for SAT based on convolutional neural networks . CNNSAT ’ s effectiveness is due to a precise and compact representation of Boolean formulas . On both real and synthetic formulas , CNNSAT is highly accurate and orders of magnitude faster than the state-of-the-art solver Z3 . We also describe how to extend CNNSAT to predict satisfying assignments when it predicts a formula to be satisfiable . 1 INTRODUCTION . The Boolean satisfiability problem , or SAT , is a classical decision problem . Given a propositional formula φ , SAT needs to decide whether φ has a satisfying assignment to its variables . If the answer is yes , we say that the formula φ is satisfiable , or SAT for short . Otherwise , it is unsatisfiable , or UNSAT for short . For example , the formula ( x1∨x2 ) ∧ ( ¬x1∨x2 ) ∧ ( x1∨¬x2 ) is satisfiable when both x1 and x2 are true ( i.e. , x1 = x2 = true ) . Conversely , ( x1 ∨x2 ) ∧ ( ¬x1 ∨x2 ) ∧ ( x1 ∨¬x2 ) ∧ ( ¬x1 ∨¬x2 ) can not be satisfied by any of the possible assignments . In other words , SAT asks whether the variables of the given Boolean formula φ can be consistently assigned the values true or false such that the formula evaluates to true . If this is the case , the formula is called satisfiable . On the other hand , if no such assignment exists , the formula is false for all possible variable assignments and is unsatisfiable . SAT is a classical NP-complete problem , and in fact was the first problem proved NP-complete . Many hard problems naturally reduce to SAT , such as the traveling salesman problem ( TSP ) and clique detection . SAT has been extensively studied in the literature for decades because it is a foundational problem and has wide applications . The more general satisfiability modulo theories ( SMT ) can also be reduced to SAT solving . SAT is one of the most investigated problems , and numerous heuristics exist to help speed up SAT solving . However , state-of-the-art solvers do not yet scale to large , difficult formulas , such as ones with hundreds of variables and thousands of clauses for uniform 3-SAT problems ( Xu et al. , 2012 ) . This is because the search space for solutions increases exponentially w.r.t . the number of variables . Most search-based SAT solvers are based on the DPLL approach ( Davis et al. , 1962 ) , but the search space , even reduced , is still intractable for very large formulas . State-of-the-art methods for SAT adopt the Conflict-Driven Clause Learning ( CDCL ) ( Silva & Sakallah , 1996 ; Bayardo Jr & Schrag , 1997 ) algorithm . This is a systematic search algorithm but employs various optimizations to improve efficiency . However , because the general problem is NP-complete , systematic search algorithms have exponential worst-case complexity , which limits the scalability of these methods . There exists several pieces of previous work that try to use machine/deep learning methods to improve SAT solvers ( Loreggia et al. , 2016 ; Fei & Rompf , 2018 ) , classify SAT/UNSAT ( Bünz & Lamm , 2017 ; Grozea & Popescu , 2014 ; Devlin & O ’ Sullivan , 2008 ) , or directly solve SAT instances ( Selsam et al. , 2018 ) . However , all these approaches focus on SAT problems with a small number of variables . In this paper , we introduce CNNSAT , a fast and accurate technique based on Convolutional Neural Networks ( Krizhevsky et al. , 2012 ) to predict both satisfiability and satisfying assignments . Evaluated on a data set containing 3-SAT problems with up to 410 variables , CNNSAT is able to predict SAT/UNSAT with more than 95 % accuracy and orders of magnitude faster than state-of-the-art solvers . We also introduce optimizations to further improve CNNSAT ’ s scalability . As for the more general SMT problems , CNNSAT is able to predict their satisfiability with more than 73 % accuracy . 2 PRELIMINARIES . Convolutional neural network ( CNN ) . CNN is a class of deep , feed-forward artificial neural networks . Many successful applications of CNNs exist in image and sound processing . A CNN has an input and an output layer , as well as multiple hidden layers , which typically consist of convolutional layers , pooling layers , fully connected layers and normalization layers . The SAT and 3-SAT problems . In Boolean logic , a formula is in conjunctive normal form ( CNF ) if it is a conjunction of one or more sub-formulas . For a CNF formula , each of its sub-formulas is called a clause , which is a disjunction of literals ( i.e. , variables or their negations ) . The clause-to-variable ratio of a CNF formula is defined as the ratio of the number of clauses over the number of variables . Each SAT instance can be represented in CNF . A CNF formula has a satisfying assignment iff there exists at least one assignment for each variable in the formula such that the formula evaluates to true . The objective of SAT solving is to determine whether or not a given formula is satisfiable , and produce a satisfying assignment when the formula is satisfiable . 3-SAT is a special case of SAT where the number of literals in each clause is up to three . Generalizing 3-SAT , N -SAT requires all clauses having no more than N literals , and uniform N -SAT requires all clauses having exactly N literals . 3-SAT is also NP-complete , and , in general , N -SAT , for N > 2 , can be reduced to 3-SAT . The SMT problem Satisfiability Modulo Theories ( SMT ) refers to the problem of determining whether a first-order formula is satisfiable w.r.t . some logical theories . It is typically applied to the theory of real numbers , the theory of integers , and the theories of various data structures , such as lists , arrays , and bit vectors . For brevity , hereafter if a SAT problem instance is in CNF , we refer to it as CNF . Otherwise , we still use SAT to refer to the more general case . 3 CNNSAT . This section presents the technical details behind our approach . In particular , it describes the representation that we introduce to encode CNF formulas , the architecture of our proposed neural network , and the method that we use to find satisfying assignments . 3.1 REPRESENTATION . A SAT problem has a simple syntactic structure and therefore can be encoded into a syntax-based representation such as an abstract syntax tree ( AST ) . The semantics of propositional logic induces rich invariance that such syntactic representations would ignore , e.g. , permutation and negation invariance ( Selsam et al. , 2018 ) . Permutation invariance stipulates that the satisfiability of a SAT problem is not affected by swapping the variables ( e.g. , swapping all occurrences of x1 with those of x2 in the SAT instance ) . Negation invariance means that negating every literal corresponding to a given variable ( e.g. , replacing xi by ¬xi , and ¬xi by xi for any variable xi in the SAT instance ) . As noted by Selsam et al . ( 2018 ) , syntax-based representations do not capture the semantics of SAT problems . In other words , they can not identify even the simplest semantic equivalence among SAT problems , such as permutation and negation invariance discussed earlier . On the other hand , even though syntax-based representations may not accurately capture semantic equivalence , sufficient amount of training data may allow neural networks to learn and predict the semantics of SAT formulas . Our evaluation in Section 5 confirms this hypothesis . In addition , for certain applications , most CNFs do not share the same/similar semantics . Therefore , we adopt a syntax-based representation to balance accuracy and scalability . SAT Representation . Although a SAT problem can be represented in different forms , we choose the most common CNF format . Each clause in a CNF formula φ is represented by a vector v , where v = 〈e1 , e2 , ... , en〉 , and the dimension of v , n , corresponds to the number of variables in φ . For each element ei in the vector , we set it to 0 if the corresponding variable xi does not occur in the clause , -1 if the variable xi is negated , and 1 otherwise ( i.e. , when the literal xi appears in the clause ) . Collectively , the vectors for φ ’ s clauses form an m× n matrix , where m is the number of clauses and n the number of variables . Figure 1 shows an example to illustrate this representation . The CNF formula is shown in the left sub-figure , while the representation is shown in the right sub-figure . The first line in the left sub-figure ( p CNF 5 6 ) indicates that the CNF has 5 variables and 6 clauses . The other rows in the left sub-figure is in the format 〈vi1 vi2 vi3〉 , where i is the i-th clause , vij is a literal ( i.e. , j-th variable or its negation ) in the clause — a negative value indicates that the variable is negated . The actual CNF formula is ( x1∨x2∨x3 ) ∧ ( x2∨¬x3∨x4 ) ∧ ( x1∨¬x2∨¬x3 ) ∧ ( x1∨x2∨x4 ) ∧ ( x3∨¬x4∨¬x5 ) ∧ ( ¬x3∨x4∨x5 ) From the table , we can see that the representation in the right sub-figure encodes all the values of the variables into corresponding values in the matrix . This representation is straightforward and the conversion is efficient . Note that this is a sparse matrix because only a small number of elements in each row are nonzero . However , we observe that , in practice , a SAT problem may have as many as millions of variables and clauses . At such a large scale , these SAT problems can not fit in memory . Therefore , we propose a compact representation to improve scalability . Our core idea is to split a matrix into smaller sub-matrices and summarize information for each sub-matrix . First , we define a fixed size sliding window . Then , we split the original matrix into sub-matrices according to the size of the original matrix and the sliding window . For each submatrix , ri = 〈pi , ni〉 is a compact representation for the i-th sub-matrix , where pi is the number of positive values in the sub-matrix and ni the number of negative values . Therefore , each sub-matrix is converted to a list with 2 elements . It is worth noting that when the size of the sliding window is 1 × 1 , it retains the exact information in the original matrix . In the next section , we introduce additional optimizations for the compact matrix for better performance . Our experimental evaluation shows that this representation can accurately capture semantic equivalence . SMT Representation . There are several straightforward representations for SAT problems . In contrast , representing SMT problems is more challenging . Although we can design custom representations for SMT , we choose to translate SMT problems to SAT problems so that we can leverage our representation of SAT problems to also encode SMT problems .

Summary: The aim of this paper is to solve SAT instances using a CNN architecture. SAT instances are represented using an efficient encoding of boolean matrices. The overall idea is to decompose an input SAT instance into simpler ones, and to train the neural model on simpler instances using an existing solver for labeling these instances. Based on satisfaction probabilities induced from simpler formulas, the architecture predicts a partial assignment which is fed to the existing solver for deriving the satisfiability result.